@lyse@lyse.isobeef.org This looks like a nice way to do it.

Another thought: if clients can’t agree on the url (for example, if we switch to this new way, but some old clients still do it the old way), that could be mitigated by computing many hashes for each twt: one for every url in the feed. So, if a feed has three URLs, every twt is associated with three hashes when it comes time to put threads together.

A client stills need to choose one url to use for the hash when composing a reply, but this might add some breathing room if there’s a period when clients are doing different things.

(From what I understand of jenny, this would be difficult to implement there since each pseudo-email can only have one msgid to match to the in-reply-to headers. I don’t know about other clients.)

@falsifian@www.falsifian.org In my opinion it was a mistake that we defined the first url field in the feed to define the URL for hashing. It should have been the last encountered one. Then, assuming append-style feeds, you could override the old URL with a new one from a certain point on:

# url = https://example.com/alias/txtxt.txt
# url = https://example.com/initial/twtxt.txt
<message 1 uses the initial URL>
<message 2 uses the initial URL, too>
# url = https://example.com/new/twtxt.txt
<message 3 uses the new URL>
# url = https://example.com/brand-new/twtxt.txt
<message 4 uses the brand new URL>

In theory, the same could be done for prepend-style feeds. They do exist, I’ve come around them. The parser would just have to calculate the hashes afterwards and not immediately.

@movq@www.uninformativ.de Another idea: just hash the feed url and time, without the message content. And don’t twt more than once per second.

Maybe you could even just use the time, and rely on @-mentions to disambiguate. Not sure how that would work out.

Though I kind of like the idea of twts being immutable. At least, it’s clear which version of a twt you’re replying to (assuming nobody is engineering hash collisions).

@prologic@twtxt.net Some criticisms and a possible alternative direction:

1. Key rotation. I’m not a security person, but my understanding is that it’s good to be able to give keys an expiry date and replace them with new ones periodically.

2. It makes maintaining a feed more complicated. Now instead of just needing to put a file on a web server (and scan the logs for user agents) I also need to do this. What brought me to twtxt was its radical simplicity.

Instead, maybe we should think about a way to allow old urls to be rotated out? Like, my metadata could somehow say that X used to be my primary URL, but going forward from date D onward my primary url is Y. (Or, if you really want to use public key cryptography, maybe something similar could be used for key rotation there.)

It’s nice that your scheme would add a way to verify the twts you download, but https is supposed to do that anyway. If you don’t trust https to do that (maybe you don’t like relying on root CAs?) then maybe your preferred solution should be reflected by your primary feed url. E.g. if you prefer the security offered by IPFS, then maybe an IPNS url would do the trick. The fact that feed locations are URLs gives some flexibility. (But then rotation is still an issue, if I understand ipns right.)

On the Subject of Feed Identities; I propose the following:

1. Generate a Private/Public ED25519 key pair
   
2. Use this key pair to sign your Twtxt feed
   
3. Use it as your feed’s identity in place of # url = as # key = ...
   

For example:

$ ssh-keygen -f prologic@twtxt.net
$ ssh-keygen -Y sign -n prologic@twtxt.net -f prologic@twtxt.net twtxt.txt

And your feed would looke like:

# nick        = prologic
# key         = SHA256:23OiSfuPC4zT0lVh1Y+XKh+KjP59brhZfxFHIYZkbZs
# sig         = twtxt.txt.sig
# prev        = j6bmlgq twtxt.txt/1
# avatar      = https://twtxt.net/user/prologic/avatar#gdoicerjkh3nynyxnxawwwkearr4qllkoevtwb3req4hojx5z43q
# description = "Problems are Solved by Method" 🇦🇺👨‍💻👨‍🦯🏹♔ 🏓⚯ 👨‍👩‍👧‍👧🛥 -- James Mills (operator of twtxt.net / creator of Yarn.social 🧶)

2024-06-14T18:22:17Z	(#nef6byq) @<bender https://twtxt.net/user/bender/twtxt.txt>  Hehe thanks! 😅 Still gotta sort out some other bugs, but that's tomorrows job 🤞
...

Twt Hash extension would change of course to use a feed’s ED25519 public key fingerprint.

@bender@twtxt.net Yes, they do 🤣 Implicitly, or threading would never work at all 😅 Nor lookups 🤣 They are used as keys. Think of them like a primary key in a database or index. I totally get where you’re coming from, but there are trade-offs with using Message/Thread Ids as opposed to Content Addressing (like we do) and I believe we would just encounter other problems by doing so.

My money is on extending the Twt Subject extension to support more (optional) advanced “subjects”; i.e: indicating you edited a Twt you already published in your feed as @falsifian@www.falsifian.org indicated 👌

Then we have a secondary (bure much rarer) problem of the “identity” of a feed in the first place. Using the URL you fetch the feed from as @lyse@lyse.isobeef.org ’s client tt seems to do or using the # url = metadata field as every other client does (according to the spec) is problematic when you decide to change where you host your feed. In fact the spec says:

Users are advised to not change the first one of their urls. If they move their feed to a new URL, they should add this new URL as a new url field.

See Choosing the Feed URL – This is one of our longest debates and challenges, and I think (_I suspect along with @xuu@txt.sour.is _) that the right way to solve this is to use public/private key(s) where you actually have a public key fingerprint as your feed’s unique identity that never changes.

@movq@www.uninformativ.de @prologic@twtxt.net Another option would be: when you edit a twt, prefix the new one with (#[old hash]) and some indication that it’s an edited version of the original tweet with that hash. E.g. if the hash used to be abcd123, the new version should start “(#abcd123) (redit)”.

What I like about this is that clients that don’t know this convention will still stick it in the same thread. And I feel it’s in the spirit of the old pre-hash (subject) convention, though that’s before my time.

I guess it may not work when the edited twt itself is a reply, and there are replies to it. Maybe that could be solved by letting twts have more than one (subject) prefix.

But the great thing about the current system is that nobody can spoof message IDs.

I don’t think twtxt hashes are long enough to prevent spoofing.

All this hash breakage made me wonder if we should try to introduce “message IDs” after all. 😅

But the great thing about the current system is that nobody can spoof message IDs. 🤔 When you think about it, message IDs in e-mails only work because (almost) everybody plays fair. Nothing stops me from using the same Message-ID header in each and every mail, that would break e-mail threading all the time.

In Yarn, twt hashes are derived from twt content and feed metadata. That is pretty elegant and I’d hate see us lose that property.

If we wanted to allow editing twts, we could do something like this:

2024-09-05T13:37:40+00:00   (~mp6ox4a) Hello world!

Here, mp6ox4a would be a “partial hash”: To get the actual hash of this twt, you’d concatenate the feed’s URL and mp6ox4a and get, say, hlnw5ha. (Pretty similar to the current system.) When people reply to this twt, they would have to do this:

2024-09-05T14:57:14+00:00	(~bpt74ka) (<a href="https://yarn.girlonthemoon.xyz/search?q=%23hlnw5ha">#hlnw5ha</a>) Yes, hello!

That second twt has a partial hash of bpt74ka and is a reply to the full hash hlnw5ha. The author of the “Hello world!” twt could then edit their twt and change it to 2024-09-05T13:37:40+00:00 (~mp6ox4a) Hello friends! or whatever. Threading wouldn’t break.

Would this be worth it? It’s certainly not backwards-compatible. 😂

The actual end-user problem is that I can’t see the thread properly when using neomutt+jenny.

@prologic@twtxt.net I believe you when you say registries as designed today do not crawl. But when I first read the spec, it conjured in my mind a search engine. Now I don’t know how things work out in practice, but just based on reading, I don’t see why it can’t be an API for a crawling search engine. (In fact I don’t see anything in the spec indicating registry servers shouldn’t crawl.)

(I also noticed that https://twtxt.readthedocs.io/en/latest/user/registry.html recommends “The registries should sync each others user list by using the users endpoint”. If I understood that right, registering with one should be enough to appear on others, even if they don’t crawl.)

Does yarnd provide an API for finding twts? Is it similar?

@prologic@twtxt.net How does yarn.social’s API fix the problem of centralization? I still need to know whose API to use.

Say I see a twt beginning (#hash) and I want to look up the start of the thread. Is the idea that if that twt is hosted by a a yarn.social pod, it is likely to know the thread start, so I should query that particular pod for the hash? But what if no yarn.social pods are involved?

The community seems small enough that a registry server should be able to keep up, and I can have a couple of others as backups. Or I could crawl the list of feeds followed by whoever emitted the twt that prompted my query.

I have successfully used registry servers a little bit, e.g. to find a feed that mentioned a tag I was interested in. Was even thinking of making my own, if I get bored of my too many other projects :-)

Serious open (for anyone) question: what makes you follow someone on twtxt? Will you just follow anyone that you come across, simply because that someone using the “decentralised, minimalist microblogging service for hackers” microblog?

@abucci@anthony.buc.ci OMFG! Dear jebus, look at the size of that! :-/ It is just a matter of time until one of those randomly falls on any of us. Just incredible!

Is it really that fucking hard to use decentralized, Self-Hosted tech? 🤔 Or do people just not know how? 😢

As a rule of thumb, I only want to own or use computing devices that are not powerful enough to run a large language model. I like things simple and small.

For following notifications I would say use webmetion refering to the the line in your twtxt.txt as per: https://darch.dk/mentions-twtxt
Or send them an email, so it would be an idea to add a # contact = mailto:me@domain.net to ones twtxt.txt

@prologic@twtxt.net They are but then again Apple’s is doing things Apple’s way 😆 Hint: punishing devs and users for using alternative stores.

@bender@twtxt.net F-Droid is a platform/app that lets you side-load/install and serve android apps without the need for Google’s play store’s blessing. I also use Aurora Store to install Play Store’s apps without having to associate my phone with Google account. 🦾 it makes me feel good about myself 🥸

my whole life, i’ve been leaving things behind. venturing far away from everything that i know. these days, i’m trying to find connections that i can still rekindle, mend, and remember. this is much much harder than what i was used to

Anyone had any intereractions with @cuaxolotl@sunshinegardens.org yet? Or are they using a client that doesn’t know how to detect clients following them properly? Hmmm 🧐

Found this in an old copyright notice from 1993:

These images are not for use with the Microsoft Windows environment. Using these patterns in a Windows environment consitutes a copyright violation.

Someone clearly didn’t like Windows.

@mckinley@twtxt.net agevault uses age, allegedly very secure (aiming to replace pgp/gpg). Comparing it with gocryptfs, from the user perspective, agevault seems simpler, though CLI exclusive. As the repository states, “Like age, it features no config options, allowing for a straightforward secure flow”. It would also run in all major OS platforms out of the box.

But agevault is also very new. Though age has been around for a while now, I don’t see an “audited” link (neither on agevault, nor age).

This tool, using age is pretty neat: https://github.com/ndavd/agevault. So simple, yet seemingly powerful!

@New_scientist@feeds.twtxt.net It’s great that US regulators have approved launching 40,000 satellites with a 5-year lifespan before we had this kind of information about what’s likely to happen when they start falling out of orbit at a rate of several per hour.

@prologic@twtxt.net My pod, which is running the same commit you are, does not return an error like that. It returns the same HTML it always has. Try it. I nuked my cache before restarting.

Edit: Oh wait, the plot thickens. I do get an error if I use curl or if I use a web browser that isn’t logged in. That’s good!

but everyone is talking about making a computer using scraps that can barely render a few lines of ACII and maybe some non-latin characters as images.

imo the only useful application would be so that I never have to get a new computer again unless mine breaks. i like being able to talk to people from around the world, so its going to have to include internet and video (y’all saw the impact tiktok had on the gaza situation, can’t deny that video is important)

tried out a demo vm setup of nixos, but i really want to level up my server game and provision the whole thing using terranix.

@prologic@twtxt.net I believe you are not seeing the problem I am describing.

Hit this URL in your web browser:

https://twtxt.net/external?nick=lovetocode999&uri=https://socialmphl.com/story19510368/doujin

That’s your pod. I assume you don’t have a user named lovetocode999 on your pod. Yet that URL returns HTTP status 200, and generates HTML, complete with a link to https://socialmphl.com/story19510368/doujin, which is not a twtxt feed (that’s where the twtxt.txt link goes if you click it). That link could be to anything, including porn, criminal stuff, etc, and it will appear to be coming from your twtxt.net domain.

What I am saying is that this is a bug. If there is no user lovetocode999 on the pod, hitting this URL should not return HTTP 200 status, and it should definitely not be generating valid HTML with links in it.

Edit: Oops, I misunderstood the purpose of this /external endpoint. Still, since the uri is not a yarn pod, let alone one with a user named lovetocode999 on it, I stand by the belief that URLs like this should be be generating valid HTML with links to unknown sites. Shouldn’t it be possible to construct a valid target URL from the nick and uri instead of using the pod’s /external endpoint?

@prologic@twtxt.net @bender@twtxt.net I partially agree with bender on this one I think. The way this person is abusing the /external endpoint on my pod seems to be to generate legitimate-looking HTML content for external sites, using a username that does not exist on my pod. One “semantically correct” thing to do would be to error out if that username does not exist on the pod. It’s not unlike having a mail server configured as an open relay at this point.

It would also be very helpful to give the pod administrator control over what’s being fetched this way. I don’t want people using my pod to redirect porn sites or whatever. If I could have something as simple as the ability to blacklist URLs that’d already help.

@lyse@lyse.isobeef.org Interesting. The yarnd --help currently says (for me):

  -R, --open-registrations            whether or not to have open user registgration

meaning it doesn’t give the default setting or warn you that you need to use -R=false and not -R false. It also leaves unclear whether --open-registrations false would work or if you need to do --open-registrations=false. It’s also unclear whether the setting change in the user interface is overridden by the command line arguments, overrides the command line arguments, is persisted across restarts.

Maybe all this is worth posting an issue for additional documentation on the git repo if there isn’t one already.

“registgration” is misspelled that way in the help by the way.

@abucci@anthony.buc.ci You can also use -R=false on the command line or leave it out entirely. When explicitly stating -R=false, there has to be an equal sign. With a space (-R false) it’s somehow parsed as -R which is equivalent to -R=true. O_o Very weird. I’d really like to see an error instead.

I still have to figure out the precedence of the settings.yaml or command line arguments. I’m probably holding it wrong, but it seems to give me different results…

@movq@www.uninformativ.de hmm, I am already using au BufNewFile,BufRead jenny-posting.eml setl completefunc=jenny#CompleteMentions fo-=t wrap, from jenny. How would I go to incorporate that there?

@abucci@anthony.buc.ci Thank you for using Lyse’s Unofficial Yarnd Help Desk: https://lyse.isobeef.org/tmp/yarnd-disable-registrations.png

A new uncensored search engine has been launched by enthusiasts, I don’t know if you’ve heard about it or not https://cdg.iounews.com The search engine uses its own database for searching. It works even for me in Windows XP

@movq@www.uninformativ.de, using the branch on topic right now, it works perfect. The only thing I found was that I had to quit neomutt, and re-open, to see the perfect thread. Other than that, I love it!

Because I saw the nick on movq (@prologic@twtxt.net, can’t mention anyone outside this pod, by the way), I looked the user up: https://tilde.pt/~marado/twtxt.txt. I wonder if the “hashes” they are using will work out of the box with jenny.

Talking about jenny, going to play with the latest now. Tata! :-)

👋 Hello @nigergibe@anthony.buc.ci, welcome to Buccipod, a Yarn.social Pod! To get started you may want to check out the pod’s Discover feed to find users to follow and interact with. To follow new users, use the ⨁ Follow button on their profile page or use the Follow form and enter a Twtxt URL. You may also find other feeds of interest via Feeds. Welcome! 🤗

👋 Hello @nigergibe@anthony.buc.ci, welcome to Buccipod, a Yarn.social Pod! To get started you may want to check out the pod’s Discover feed to find users to follow and interact with. To follow new users, use the ⨁ Follow button on their profile page or use the Follow form and enter a Twtxt URL. You may also find other feeds of interest via Feeds. Welcome! 🤗

@bender@twtxt.net What multiplexer do you use? I usually use Tmux and have my prefix mapped to C-a on my local machine and the default C-b on the remote ones so they don’t conflict if it helps.

@movq@www.uninformativ.de confirming that the issue isn’t present when using alacrity. Wow.

Correct, @bender@twtxt.net. Since the very beginning, my twtxt flow is very flawed. But it turns out to be an advantage for this sort of problem. :-) I still use the official (but patched) twtxt client by buckket to actually fetch and fill the cache. I think one of of the patches played around with the error reporting. This way, any problems with fetching or parsing feeds show up immediately. Once I think, I’ve seen enough errors, I unsubscribe.

tt is just a viewer into the cache. The read statuses are stored in a separate database file.

It also happened a few times, that I thought some feed was permanently dead and removed it from my list. But then, others mentioned it, so I resubscribed.

Recovery: 5.00 miles, 00:11:02 average pace, 00:55:11 duration
using the treadmill to slow the pace.
#running #treadmill

i think maybe they got her to add a forward number for sms and used that to activate on another device..

Seriously, why would you use nnn, vifm, ranger or foo when you have rover? Tabs, copy/move/delete, easy-to-configure file open, … an #openbsd port is required ^^ https://github.com/lecram/rover

@prologic@twtxt.net The headline is interesting and sent me down a rabbit hole understanding what the paper (https://aclanthology.org/2024.acl-long.279/) actually says.

The result is interesting, but the Neuroscience News headline greatly overstates it. If I’ve understood right, they are arguing (with strong evidence) that the simple technique of making neural nets bigger and bigger isn’t quite as magically effective as people say — if you use it on its own. In particular, they evaluate LLMs without two common enhancements, in-context learning and instruction tuning. Both of those involve using a small number of examples of the particular task to improve the model’s performance, and they turn them off because they are not part of what is called “emergence”: “an ability to solve a task which is absent in smaller models, but present in LLMs”.

They show that these restricted LLMs only outperform smaller models (i.e demonstrate emergence) on certain tasks, and then (end of Section 4.1) discuss the nature of those few tasks that showed emergence.

I’d love to hear more from someone more familiar with this stuff. (I’ve done research that touches on ML, but neural nets and especially LLMs aren’t my area at all.) In particular, how compelling is this finding that zero-shot learning (i.e. without in-context learning or instruction tuning) remains hard as model size grows.

@movq@www.uninformativ.de Variable names used with -eq in [[ ]] are automatically expanded even without $ as explained in the “ARITHMETIC EVALUATION” section of the bash man page. Interesting. Trying this on OpenBSD’s ksh, it seems “set -u” doesn’t affect that substitution.

I love shell scripts because they’re so pragmatic and often allow me to get jobs done really quickly.

But sadly they’re full of pitfalls. Pitfalls everywhere you look.

Today, a coworker – who’s highly skilled, not a newbie by any means – ran into this:

$ bash -c 'set -u; foo=bar; if [[ "$foo" -eq "bar" ]]; then echo it matches; fi'
bash: line 1: bar: unbound variable

Why’s that happening? I know the answer. Do you? 😂

Stuff like that made me stop using shell scripts at work, unless they’re just 4 or 5 lines of absolutely trivial code. It’s now Python instead, even though the code is often much longer and clunkier, but at least people will understand it more easily and not trip over it when they make a tiny change.

If some of you budding fathers want to know how I created a computer nerd to one day work for Facebook in the big USA, well you purchase a $1000 Xmas present, an enormous thick book with C++ programming, and say, you can play as many games as you like kids, but James has to create them using computer software.

SO James created once a 3D chess program with sound, took 6 months or so, really hard to beat, not based on logic moves point by point like other chess programs, this one was based on the depth of looking for patterns, set it to 5 moves ahead and you were toast every time. Nice program too, sadly gone over the years, computers suffer from bit rot. We used to try and mark rotten hard drive discs once as bad sectors, not sure how UBuntu does this these days, I see a dozen errors on the screen every time I load.

Today I would purchase for my kids AI CAD simulation software with metal 3D printer and get your child to build fancy 3D models and engines from scratch. This will make them an expert in the CAD AI industry by the time they are 14 years old. Sadly AI is here to stay and will spoil the Internet.

@falsifian@www.falsifian.org by the way, on the last Saturday of every month, we generally hold a online video call/social meet up, where we just get together and talk about stuff if, you’re interested in joining us this month.