I finally resolved my issues with hashing twts… with REGEX!

Dates in JavaScript are truly strange creatures.

@bender@twtxt.net Thanks for asking!

So, I’ve been working on 2 main twtxt-related projects.

The first is small Node / express application that serves up a twtxt file while allowing its owner to add twts to it (or edit it outright), and I’ve been testing it on my site since the night I made that post. It’s still very much an MVP, and I’ve been intermittently adding features, improving security, and streamlining the code, with an eye to release it after I get an MVP done of project #2 (the reader).

But that’s where I’ve been struggling. The idea seems simple enough - another Node / express app (this one with a Vite-powered front-end) that reads a public twtxt file, parses the “follow” list, grabs (and parses) those twtxt files, and then creates a river of twts out of the result. The pieces work fine in seclusion (and with dummy data), but I keep running into weird issues when reading real-live twtxt files, so some twts come through, while others get lost in the ether. I’ll figure it out eventually, but for now, I’ve been spending far more time than I anticipated just trying to get it to work end-to-end.

On top of it, the 2 projects wound up turning into 4 (so far), as I’ve been spinning out little libraries to use across both apps (like https://jsr.io/@itsericwoodward/fluent-dom-esm, and a forthcoming twtxt helper library).

In the end, I’m hoping to have project 1 (the editor) into beta by the end of October, and project 2 (the reader) into beta sometime after that, but we’ll see.

I hope this has satisfied your curiosity, but if you’d like to know more, please reach out!

@bender@twtxt.net Thanks for asking!

So, I’ve been working on 2 main twtxt-related projects.

The first is small Node / express application that serves up a twtxt file while allowing its owner to add twts to it (or edit it outright), and I’ve been testing it on my site since the night I made that post. It’s still very much an MVP, and I’ve been intermittently adding features, improving security, and streamlining the code, with an eye to release it after I get an MVP done of project #2 (the reader).

But that’s where I’ve been struggling. The idea seems simple enough - another Node / express app (this one with a Vite-powered front-end) that reads a public twtxt file, parses the “follow” list, grabs (and parses) those twtxt files, and then creates a river of twts out of the result. The pieces work fine in seclusion (and with dummy data), but I keep running into weird issues when reading real-live twtxt files, so some twts come through, while others get lost in the ether. I’ll figure it out eventually, but for now, I’ve been spending far more time than I anticipated just trying to get it to work end-to-end.

On top of it, the 2 projects wound up turning into 4 (so far), as I’ve been spinning out little libraries to use across both apps (like https://jsr.io/@itsericwoodward/fluent-dom-esm, and a forthcoming twtxt helper library).

In the end, I’m hoping to have project 1 (the editor) into beta by the end of October, and project 2 (the reader) into beta sometime after that, but we’ll see.

I hope this has satisfied your curiosity, but if you’d like to know more, please reach out!

Nothing serious! just a sanity check Twt x) ...

@kat@yarn.girlonthemoon.xyz Mine shows 1/1 of 14 Twts 😆 I think this is a bug 🤯

whyyyyy is my discover page only showing 7 twts!!!

@lyse@lyse.isobeef.org i dont mind if the hash is not backward compatible but im not sure if this is the right way to proceed because the added complexity dealing with two hash versions isnt justified

regular end users wont care to understand how twt hashes are formed, they just want to use twtxt! so i guess i could work in protecting users from themselves by disallowing post edits on old posts or posts with replies, but i’m not fond of this either really. if they want to break a thread, they can just delete the post (though i’ve noticed yarn handling post deletes dubiously…)

on activitypub i do genuinely find myself looking through several month or even year old posts sometimes and deciding to edit/reword them a little to be slightly less confusing, this should be trivial to handle on twtxt which is an infinitely simpler specification

ok so i have found a genuine twt hash collision. what do i do.

internally, bbycll relies on a post lookup table with post hashes as keys, this is really fast but i knew i’d inevitably run into this issue (just not so soon) so now i have to either:
  1) pick the newer post over the other
  2) break from specification and not lowercase hashes
  3) secretly associate canonical urls or additional entropy with post hashes in the backend without a sizeable performance impact somehow

@lyse@lyse.isobeef.org retwts are a discovery feature! on federated platforms with no algorithm where you only ever see posts from accounts you explicitly follow, the element of “hey look at this!” helps users to find other accounts they might like organically

i agree quoting and replying forum-style is generally a much better way of doing things even though im a heathen and i revel in the dark patterns inspired by quote posts but when you have nothing to add and you just want to share a twt with your followers it’d be good to have a standardized way of linking to twt

at first i dismissed the idea of likes on twtxt as not sensible…like at all — then i considered they could just be published in a metadata field (though that field could get really unruly after a while)

retwts are plausible, as “RE: https://example.com/twtxt.txt#abcdefg”, the hash could even be the original timestamp from the feed to make it human readable/writable, though im extremely wary of clogging up timelines

i thought quote twts could be done extremely sensibly, by interpreting a mention+hash at the end of the twt differently to when placed at the beginning — but the twt subject extension requires it be at the beginning, so the clean fallback to a normal reply i originally imagined is out of the question — it could still be possible (reusing the retwt format, just like twitter!) but i’m not convinced it’s worth it at that point

is any of this in the spirit of twtxt? no, not in the slightest, lmao

Oops, maybe I should have posted a reminder. 🥴

@dce@hashnix.club By the time you posted your twt, the red phase was already over. 🙈 Stellarium has a pretty good simulation of the whole thing.

Yesterday, I published my first package on JSR: https://jsr.io/@itsericwoodward/fluent-dom-esm.

Then today, I pushed an update to my site to show my twts (including a schnazzy little animation to add them): https://itsericwoodward.com/

Overall, a most productive weekend.

Yesterday, I published my first package on JSR: https://jsr.io/@itsericwoodward/fluent-dom-esm.

Then today, I pushed an update to my site to show my twts (including a schnazzy little animation to add them): https://itsericwoodward.com/

Overall, a most productive weekend.

@kat@yarn.girlonthemoon.xyz i wanna catch up on the twts!!!

ugh my TL’s once again doing the thing where it only shows like 5 twts

Looks like here’s something wrong with Markdown parsing. 🤔 The original twt looks like this:

>This extension was turned off because it is no longer supported

Thanks Google.
This browser was uninstalled because it absolutely sucks!

So only the first line should be a quote.

PSA: setpriv on Linux supports Landlock.

If this twt goes through, then restricting the filesystem so that jenny can only write to ~/Mail/twt, ~/www/twtxt.txt, ~/.jenny-cache, and /tmp works.

@bender@twtxt.net @aelaraji@aelaraji.com Sorry this was my fault 🤦 For whatever reason my pod had never seen that particular Twt from @movq@www.uninformativ.de – And… There’s a bit of a “behavioral” problem with the Trusted Peers functionality that means operators have to periodically re-trust peers manually 😭 Need to rework this 🤞

@bender@twtxt.net Can’t you see this twt ??

() @aelaraji@aelaraji.com I am missing the root for this twtxt. What was it? Hmm, referring to https://twtxt.net/twt/g5sjnlq.

twtxt its dead my friends! my first twt from twtwt #twtxt #dead

hey @prologic@twtxt.net heads up - my pod is suddenly having weird 400 bad request errors on things like posting twts, new user registration, following, and more. it’s not just me because a friend is also having these issues as a new user and can’t post. i saw one exception in the logs but i’m not sure if it’s related, i’ll link it in a reply to this

bruh i log in after a day and my TL/discover feed has 12 twts WHAT AM I MISSING

a twt a day keeps the doctor away…

@prologic@twtxt.net i will try my best if it happens again! it seems it goes in and out and things eventually come back - i just logged on this morning and i do see new twts but not past ones. so when it happens again and i notice i’ll try to trace logs

whys my feed back to showing like 5 twts

I’m also thinking of adding eye-off icon next to every Twt that, when clicked, hides that feed (tooltip: “Hide this feed”). This would work with the filters as a “temporary additive filter” to restrict/control the current view.

I’m thinking of bringing back filters (this time not as a feature flag, just baked in): New filters: Hide Feed, Hide Bots, Hide News, Media Only, No Replies, Local Only — toggle to trim noise & surface the Twts you care about.

last night my timeline randomly reset to only show my own recent twts and i restarted my instance a few times and pulled from main and shit and it didn’t change but it seems fine now lol?!

In other words, why didn’t you all do the same that @movq@www.uninformativ.de did, and setup a completely different feed for this?

But Yarn does not like it: https://twtxt.net/twt/yoatzwa

I’ve just released version 1.0 of twtxt.el (the Emacs client), the stable and final version with the current extensions. I’ll let the community maintain it, if there are interested in using it. I will also be open to fix small bugs.
I don’t know if this twt is a goodbye or a see you later. Maybe I will never come back, or maybe I will post a new twt this afternoon. But it’s always important to be grateful. Thanks to @prologic@twtxt.net @movq@www.uninformativ.de @eapl.me@eapl.me @bender@twtxt.net @aelaraji@aelaraji.com @arne@uplegger.eu @david@collantes.us @lyse@lyse.isobeef.org @doesnm@doesnm.p.psf.lt @xuu@txt.sour.is @sorenpeter@darch.dk for everything you have taught me. I’ve learned a lot about #twtxt, HTTP and working in community. It has been a fantastic adventure!
What will become of me? I have created a twtxt fork called Texudus (https://texudus.readthedocs.io/). I want to continue learning on my own without the legacy limitations or technologies that implement twtxt. It’s not a replacement for any technology, it’s just my own little lab. I have also made a fork of my own client and will be focusing on it for a while. I don’t expect anyone to use it, but feedback is always welcome.
Best regards to everyone.
#twtxt #emacs #twtxt-el #texudus

@lyse@lyse.isobeef.org Kind of, but on the other hand: This twt right here refers to 3rvya6q and your feed, but your feed certainly does not include that particular twt (it comes from my feed).

But my proposal probably isn’t very helpful, either. We have this flat conversation model, so … this twt right here, what should it refer to? Your twt? My root twt? I don’t know.

@prologic@twtxt.net Don’t include this just yet. I need to think about this some more (or drop the idea).

that said, and reading to @sorenpeter@darch.dk and @andros@twtxt.andros.dev I have new thoughts. I assume that this won’t change anyone’s opinions or priorities, so it makes no harm sharing them.

It’s always tempting to use something that already exists (like X, Masto, Bsky, etc.) rather that building anything through effort and disagreement until reaching to something useful and valuable together. A ‘social service’ is only useful if people is using it.

I’ll add that I haven’t lost interest on the ‘hacky’ part of twtxt about developing tools, protocols, and extensions as a community. It’s the appealing part! It’s a nice hobby to have, shared with random people across the world.
But this is not the right way for me, and makes me feel that I’m unwelcome to propose something different (after watching replies to my previous twt). Feels like “If you don’t agree, you are free to leave, we’ll miss you.” Naah, not cool. I’ve lived that many times before, and nowadays I don’t have enough spare time and energy for a hobby like that.

Let’s see what happens next with the micro-community!

@prologic@twtxt.net Not sure I’d attach any if clauses to this. My point is: Every time I see a hash, I’d like to have a hint as to where to find the corresponding twt.

@movq@www.uninformativ.de If we’re focusing on solving the “missing roots” problems. I would start to think about “client recommendations”. The first recommendation would be:

1. Replying to a Twt that has no initial Subject must itself have a Subject of the form (hash; url).
   

This way it’s a hint to fetching clients that follow B, but not A (in the case of no mentions) that the Subject/Root might (very likely) is in the feed url.

If we must stick to hashes for threading, can we maybe make it mandatory to always include a reference to the original twt URL when writing replies?

Instead of

(<a href="https://yarn.girlonthemoon.xyz/search?q=%23123467">#123467</a>) hello foo bar

you would have

(<a href="https://yarn.girlonthemoon.xyz/search?q=%23123467">#123467</a> http://foo.com/tw.txt) hello foo bar

or maybe even:

(<a href="https://yarn.girlonthemoon.xyz/search?q=%23123467">#123467</a> 2025-04-30T12:30:31Z http://foo.com/tw.txt) hello foo bar

This would greatly help in reconstructing broken threads, since hashes are obviously unfortunately one-way tickets. The URL/timestamp would not be used for threading, just for discovery of feeds that you don’t already follow.

I don’t insist on including the timestamp, but having some idea which feed we’re talking about would help a lot.

July 1st. 63 days from now to implement a backward-incompatible change, apparently not open to other ideas like replacing blake with SHA, or discussing implementation challenges for other languages and platforms.
Finally just closing #18, #19 and #20 without starting a proper discussion and ignoring a ‘micro consensus’ feels… not right.

I don’t know what to think rather than letting it rest (May will be busy here) and focus on other stuff in the future.

twt-hash-v2.md#implementation-timeline

Finally I propose that we increase the Twt Hash length from 7 to 12 and use the first 12 characters of the base32 encoded blake2b hash. This will solve two problems, the fact that all hashes today either end in q or a (oops) 😅 And increasing the Twt Hash size will ensure that we never run into the chance of collision for ions to come. Chances of a 50% collision with 64 bits / 12 characters is roughly ~12.44B Twts. That ought to be enough! – I also propose that we modify all our clients and make this change from the 1st July 2025, which will be Yarn.social’s 5th birthday and 5 years since I started this whole project and endeavour! 😱 #Twtxt #Update

And speaking of Twtxt (See: #xushlda, feeds should be treated as append-only. Your client(s) should be appending Twts to the bottom of the file. Edits should never modify the timestamp of the Twt being edited, nor should a Twt that was edited by deleted, unless you actually intended to delete it (but that’s more complicated as it’s very hard to control or tell clients what to do in a truely decentralised ecosystem for the deletion case). #Twtxt #Client #Recommendations

Just like we don’t write emails by hand anymore (See: #a3adoka), we don’t manually write Twts or update our twtxt.txt feeds. Instead, we use modern Twtxt clients that conform to the specifications at Twtxt.dev for a seamless, automated experience. #Twtxt #Twt #UserExperience

@bender@twtxt.net Hehe good sleuthing 🤣 I swear it was an edit ✍️ Haha 😂 yarnd now “sees” both every single time, where-as before it would just obliterate the old Twt, but remain in archive. Now you get to see both 😅 Not sure if that’s a good thing or not, but it certainly makes it much clearer how to write “code logic” for detecting edits and doing something more UX(y) about ‘em 🤔

I have a great idea for fixing the US economy. Get rid of all the nuclear weapons 🤣

$ bat https://twtxt.net/twt/edgwjcq | jq '.subject'
"(#yarnd)"

hahahahaha 🤣 Does your client allow you to do this or what? 🤔

@aelaraji@aelaraji.com I replied to this twtxt with https://twtxt.net/twt/yqrdx4q, and it created its own, totally unconnected.

Related Twt

@bender@twtxt.net allow me to try and explain over several Twts…

@bender@twtxt.net Wild. The twts look garbled. Like The Thing.

I’m thinking of building a hardened peering protocol for Yarn.social’s yarnd: pods establish cryptographic identities, exchange signed /info and /twt payloads with signature verification, ensuring authenticity, integrity, and spoof-proof identity validation across the distributed network.

by commenting out DMs are you giving up on simplicity? See the Metadata extension holding the data inside comments, as the client doesn’t need to show it inside the timeline.

I don’t think that commenting out DMs as we are doing for metadata is giving up on simplicity (it’s a feature already), and it helps to hide unwanted DMs to clients that will take months to add it’s support to something named… an extension.

For some other extensions in https://twtxt.dev/extensions.html (for example the reply-to hash #abcdfeg or the mention @ < example http://example.org/twtxt.txt >) is not a big deal. The twt is still understandable in plain text.
For DM, it’s only interesting for you if you are the recipient, otherwise you see an scrambled message like 1234567890abcdef=. Even if you see it, you’ll need some decryption to read it. I’ve said before that DMs shouldn’t be in the same section that the timeline as it’s confusing.

So my point stands, and as I’ve said before, we are discussing it as a community, so let’s see what other maintainers add to the convo.