@prologic@twtxt.net Of course you don’t notice it when yarnd only shows at most the last n messages of a feed. As an example, check out mckinley’s message from 2023-01-09T22:42:37Z. It has “[Scheduled][Scheduled][Scheduled]“… in it. This text in square brackets is repeated numerous times. If you search his feed for closing square bracket followed by an opening square bracket (][) you will find a bunch more of these. It goes without question he never typed that in his feed. My client saves each twt hash I’ve explicitly marked read. A few days ago, I got plenty of apparently years old, yet suddenly unread messages. Each and every single one of them containing this repeated bracketed text thing. The only conclusion is that something messed up the feed again.

@eapl.me@eapl.me Yeah, you need some kind of storage for that. But chances are that there’s already a cache in place. Ideally, the client remembers etags or last modified timestamps in order to reduce unnecessary network traffic when fetching feeds over HTTP(S).

A newsreader without read flags would be totally useless to me. But I also do not subscribe to fire hose feeds, so maybe that’s a different story with these. I don’t know.

To me, filtering read messages out and only showing new messages is the obvious solution. No need for notifications in my opinion.

There are different approaches with read flags. Personally, I like to explicitly mark messages read or unread. This way, I can think about something and easily come back later to reply. Of course, marking messages read could also happen automatically. All decent mail clients I’ve used in my life offered even more advanced features, like delayed automatic marking.

All I can say is that I’m super happy with that for years. It works absolutely great for me. The only downside is that I see heaps of new, despite years old messages when a bug causes a feed to be incorrectly updated (https://twtxt.net/twt/tnsuifa). ;-)

that’s a fair point.

Perhaps, since Twitter in 2006 never implemented read flags, every derivative microblogging system never saw that as an expected feature. This is curious because Twitter started with SMS, where on our phones we can mark messages as read or unread.
I think it all comes from the difference between reading an email (directed to you) vs. reading public posts (like a blog or a ‘wall,’ where you don’t mark posts as read). It’s not necessary to mark it as ‘read’, you just jump over it.

Reading microblogging posts in an email program is not common, I think, and I haven’t really used it, so I cannot say how it works, and whether it would be better for me or not.
However, I’ve used Thunderbird as a feed reader, and I understand the advantages when reading blog posts.

About read flags being simple, well… we just had a discussion this morning about how tracking read messages would require a lot of rethinking for clients such as timeline where no state is stored. Even considering some kind of ‘notification of unread messages or mentions’ is not expected for those minimalist client, so it’s an interesting compromise to think about.

@eapl.me@eapl.me Read flags are so simple, yet powerful in my opinion. I really don’t understand why this is not a thing in most twtxt clients. It’s completely natural in e-mail programs and feed readers, but it hasn’t made the jump over to this domain.

You write too much for my client 😂

@andros@twtxt.andros.dev How about putting the whole encrypted conversation into a sperate twtxt-file. Just like the archive feature (?). That way, the general clients don’t have to cope with the decrytption stuff and it won’t break the general public conversations.

@arne@uplegger.eu Klingt gut, Du darfst uns gern mal ein paar Bildschirmfotos vom aktuellen Stand zeigen. :-) Die erste Aufnahme sah bereits recht aufgeräumt aus.

Ich müsste auch endlich mal an meinem Client weitermachen. Aber heut nimmer.

@prologic@twtxt.net @lyse@lyse.isobeef.org First, please leave me your comments on the repository! Even if it’s just to give your opinion on what shouldn’t be included. The more variety, the better.

Second, I’m going to try to do tests with Elliptic keys and base64. Thanks for the advice @eapl@eapl.me

Finally, I’d like to give my opinion. Secure direct messages are a feature that ActivityPub and Mastodon don’t have, to give an example. By including it as an extension, we’re already taking a significant leap forward from the competition. Does it make sense to include it in a public feed? In fact, we’re already doing that. When we reply to a user, mentioning them at the beginning of the message, it’s already a direct message. The message is within a thread, perhaps breaking the conversation. Direct messages would help isolate conversations between 2 users, as well as keeping a thread cleaner and maintaining privacy. I insist, it’s optional, it doesn’t break compatibility with any client and implementing it isn’t complex. If you don’t like it, you’re free to not use it. If you don’t have a public key, no one can send you direct messages.

another one would be to allow changing public keys over time (as it may be a good practice [0]). A syntax like the following could help to know what public key you used to encrypt the message, and which private key the client should use to decrypt it:

!<nick url> <encrypted_message> <public_key_hash_7_chars>

Also I’d remove support for storing the message as hex, only allowing base64 (more compact, aiming for a minimalistic spec, etc.)

[0] https://www.brandonchecketts.com/archives/its-2023-you-should-be-using-an-ed25519-ssh-key-and-other-current-best-practices

@arne@uplegger.eu nice work with the client.
I also see you are using the Yellow CMS for your website🍋

@arne@uplegger.eu Uuhhhh, more twtxt clients, very nice! :-)

@sorenpeter@darch.dk Yes it works, thx: https://doesnm.cc/mentions.txt . I’m deleted html tags because my client do not support html rendering

@prologic@twtxt.net I say we should find a way to support mentions with only url, no nick, as per the original spec.

• For @<nick url> we already got support
  
• For @<nick> the posting client should expand it to @<nick url>, if not then the reading client should just render it as @nick with no link.
  
• For @<url> the sending client should try to expand it to @<nick url>, if not then the reading client should try to find or construct a nick base on:
  
  1. Look in twtxt.txt for a nick =
     
  2. Use (sub)domain from URL
     
  3. Use folder or file name from URL

I’m sharing new developments on the client. I now have a more stable timeline. The first version will appear in the next few weeks. #emacs #twtxt

@prologic@twtxt.net If you’ve got the feed URL in yarnd’s cache, you can easily look up a missing nick. If you can’t find it, just show the URL (or maybe just the domain name to be halfway consistent with this @nick@domain thing that yarnd invented) and be done. It’s really that simple.

When yarnds peer with each other, the odds of actually having come across that feed URL in the past are higher than with traditional clients that only have their local set of subscribed feeds. One additional improvment would be to also look at all the mentions and see if somebody used a nick for that URL and go with that.

Yeah, yarnd currently renders some really weird shit when the mention contains just a URL, but I’d call that a bug for sure.

Personally, I do not like the @nick@domain syntax at all. It looks silly to my eyes. What might have also contributed is the fact of this mentions syntax gotten screwed up so many times by yarnd in the past. But that’s a totally different topic.

Hmm, I just noticed that the feed template seems to be broken on your yarnd instance, @kat@yarn.girlonthemoon.xyz. Looking at your raw feed file (and your mates as well), line 6 reads:

# This is hosted by a Yarn.social pod yarn running yarnd ERSION@OMMIT  go1.23.4
                                                         ^^^^^^^^^^^^

Looks like the first letters of the version and commit got somehow chopped off. I’ve no idea what happened here, maybe @prologic@twtxt.net knows something. :-? I’m not familiar with the templating, I just recall @xuu@txt.sour.is reporting in IRC the other day that he’s also having great fun with his custom preamble from time to time.

That “broken” comment doesn’t hurt anything, it’s still a proper comment and hence ignored by clients. It’s just odd, that’s all.

trying to set up @movq@www.uninformativ.de’s jenny client… currently trying to find where twtxt files are stored on the server so i can set up the scp script i have for this

My client is twet which i grabbed at https://github.com/jdtron/twet

@doesnm@doesnm.p.psf.lt LOL sorry which client are you using? 🤔 You can of course have a say! There aren’t that many active/used clients at the moment, and I forget which one you’re using 🤣🤣

Lol only i use discontinued client? (with patches but i’m lost sources so they “proprietary”)

For the record; we consider the new authority on the Twtxt spec(s) going forward (has been for some years actually) to be implementers / primary maintainers of widely used clients. To date that is:

• yarnd @prologic@twtxt.net (me and others)
  
• jenny @movq@www.uninformativ.de
  
• tt @lyse@lyse.isobeef.org
  
• Timeline @darch@neotxt.dk / @eapl.me@eapl.me and others
  
• twtxt-el? – @andros@twtxt.andros.dev
  

Full list of supported and widely used clients can be found at https://twtxt.dev/clients.html – which I note a few above are actually missing from this page haha 🤣

What say you @movq@www.uninformativ.de @lyse@lyse.isobeef.org @eapl.mx@eapl.mx / @darch@neotxt.dk @andros@twtxt.andros.dev (new client author)? 🤔 Shall I PR this up?

I’m still making progress with the Emacs client. I’m proud to say that the code that is responsible for reading the feeds is almost finished, including: Twt Hash Extension, Twt Subject Extension, Multiline Extension and Metadata Extension. I’m fine-tuning some tests and will soon do the first buffer that displays the twts.

@doesnm@doesnm.p.psf.lt It is the same API that yarnc the command-line client uses.

@movq@www.uninformativ.de That’s so damn cool mate! I went through the code, but this lowlevel stuff is really not my favorite cup of tea. Having said that, it was actually really nice to see the abstractions and APIs work together and how things are getting indeed very readable in the userland programs. That’s easy to track in this extremely tiny OS implementation. Excellent work, keep on hacking!

Now, you just have to quickly add a network stack and then can write a twtxt client for it! ]:->

@andros@twtxt.andros.dev What do you mean by API? yarnd (which powers Yarn.social pods like twtxt.net) does have an API, however that API is designed for clients to interact with the pod and the user’s account and feed. e.g: there is a command-line client called yarnc and I used to maintain a mobile native app (using Flutter).

What use-case did you have in mind?

@prologic@twtxt.net Hmm, what’s this Emacs client you heard about?

@movq@www.uninformativ.de Unfortunately, there is no feed URL or nick in the User-Agent, it just consists of “xt/0.0.1”, that’s it. And this client was only active from mid-November until the end of the month.

It’ll probably remain a mystery, we’ll never know.

Once again I glimpsed at my twtxt feed access log. Now I’m wondering: is there a twtxt client named xt out there? Does anyone know? I did not find anything for “xt/0.0.1”.

I found 2 active Registries: tilde.instite and twtxt.envs.net . I think that is missing a repository or system for them to find each other. It is easy to share registry users. Your work is awesome! Maybe you are supporting twtxt with the pod and software around them. I am very busy with the Emacs client, but I like to work creating my own version of Registry using Django.

@emmanuel@wald.ovh Btw I already figured out why accessing your web server is slow:

$ host wald.ovh
wald.ovh has address 86.243.228.45
wald.ovh has address 90.19.202.229

wald.ovh has 2 IPv4 addresses, one of which is dead and doesn’t respond.. That’s why accessing your website is so slow as depending on client and browser behaviors one of two things may happen 1) a random IP is chosen and ½ the time the wrong one is picked or 2) both are tried in some random order and ½ the time its slow because the broken one is picked.

If you don’t know what 86.243.228.45 is, or it’s a dead backup server or something, I’d suggest you remove this from the domain record.

@prologic@twtxt.net No I’m not trying to standardize the domains themselves xD I was just hinting at filtering cases where nick is identical to a level of a domain; in order to show shorter format nicks within clients, i.e: @nick.domain.ltd or @nick.ltd instead of a @nick@nick.domain.ltd or @nick@nick.ltd. Just like what @sorenpeter@darch.dk already did with the nick = domain case. (unless I’m missing the point)

i always thought that i needed geodns to make a multi-region setup work at all, but it turns out all we really need is one rfc https://datatracker.ietf.org/doc/html/rfc8305 and its not even rare to see implemented on clients

What should the advantage be to nick = _compared to just not defining a nick and let the client use the domain as the handle?

What is not intuitive is that you put something in the nick field that is not to be taken literary. The special meaning of _ is only clean if you read the documentation, compared to having something in nick that makes sense in the current context of the twtxt.txt.

@doesnm@doesnm.p.psf.lt So the user should then set nick = _@domain.tld in the twtxt.txt?

It seems more intuitive and userfriendly to just use: nick = domain.tld and have then convention for clients to render the handle as @domain.tld instead of @domain.tld@domain.tld

For a feed with no nick defined (eg. https://akkartik.name/twtxt.txt) it will also be simpler and make more sense to just use the domain as the nick and render it as @domain.tld

and going back to a handle you could input in your client to look for the user/file, like @nick@domain.tls I think Webfinger is the way to go. It has enough information to know where to find that nick’s URL.

@prologic@twtxt.net does that webfinger fork made by darch work OK with yarn as it is now? (I’ve never used it, so I’m researching about it)
https://darch.dk/.well-known/webfinger/

@lyse@lyse.isobeef.org Hmmm:

Could not fetch: HTTPError('403 Client Error: Forbidden for url: https://uplegger.eu/twtxt.txt')

🤔

Thanks @bender@twtxt.net for the feedback. I fixed and expanded the article. I’m sorry for my poor interaction. Furthermore, I’m reading and writing while programming a client in Emacs.

@xuu@txt.sour.is is there anything stopping in clients from supporting this as an optional feature?

@prologic@twtxt.net What IRC client is that?

@bender@twtxt.net Well, so far, I’m using the standard web client. Haven’t found a great client yet. 🫤 Mastodon/Fediverse is also very different from twtxt, there are way more images/videos that I’d like to see – a TUI client like toot wouldn’t work for me.

Dunno, maybe I’ll make some changes in this area after christmas. Try self-hosting again or something like that …

Yes it work: 2024-12-01T19:38:35Z twtxt/1.2.3 (+https://eapl.mx/twtxt.txt; @eapl) :D

The .log is just a simple append each request. The idea with the .cvs is to have it tally up how many request there have been from each client as a way to avoid having the log file grow too big. And that you can open the .cvs as a spreadsheet and have an easy overview and filtering options.

Access to those files are closed to the public.

Maybe realize webmention in my feed? (receiving). Sending maybe will be implemened in WIP client…

@aelaraji@aelaraji.com You could use https://lyse.isobeef.org/tmp/twthash.py to generate twt hashes. I cobbled that together in order to generate test data for my client.

but for real, i really want that sword and I’m perfectly content running my daily dev stuff using an rpi. i actually used to do client work on an old rpi laptop kit, i only stopped using that baby because the battery controller let the battery go flat.

i am working on very smol deployments, where a server may use two or so replicated sqlite databases instead of a db server like postgres to seamlessly move from single to multi-node arrangements as needed. there is a clear performance limit here, but the goal is not to serve a huge number of clients. just to do as much as possible with a small number of useful components that can be upgraded to handle up to medium size workloads, without difficult data conversions or migrations. scaling beyond that point should be done via federation.

@bender@twtxt.net My made-up rule is to keep at least three full months in the main feed and when rotating, I create one feed per month.

@doesnm@doesnm.p.psf.lt There is no real recommendation I think. But if you hit half a MiB or so, it might be worth considering to rotate in order to keep the network traffic low. People with bad connectivitiy might appreciate it. I want to implement HTTP range requests in my client rewrite at some point in time (but first, it has to become kinda usable, though).

Ouiii, j’ai de nouveau internet! En fait, #sfr a changé l’armoire, a débranché tout le monde sans prévenir personne, et maintenant les autres fournisseurs doivent venir rebrancher leurs clients. C’est malhonnête… Bref, si3t.ch et puffy.cafe sont de retour

@sorenpeter@darch.dk Section 7 on emojis: Exactly that, it’s an avatar for text interfaces. The metadata name needs tweaking, but that’s a cool idea. If I implemented this in my client, I’d make the text avatar overridable by the user, though. Otherwise I’d probably only see boxes for everbody in my terminal. :-D

Thank you, @eapl.me@eapl.me! No need to apologize in the introduction, all good. :-)

Section 3: I’m a bit on the fence regarding documenting the HTTP caching headers. It’s a very general HTTP thing, so there is nothing special about them for twtxt. No need for the Twtxt Specification to actually redo it. But on the other hand, a short hint could certainly help client developers and feed authors. Maybe it’s thanks to my distro’s Ngninx maintainer, but I did not configure anything for the Last-Modified and ETag headers to be included in the response, the web server just already did it automatically.

The more that I think about it while typing this reply, the more I think your recommendation suggestion is actually really great. It will definitely beneficial for client developers. In almost all client implementation cases I’d say one has to actually do something specifically in the code to send the If-Modified-Since and/or If-None-Match request headers. There is no magic that will do it automatically, as one has to combine data from the last response with the new request.

But I also came across feeds that serve zero response headers that make caching possible at all. So, an explicit recommendation enables feed authors to check their server setups. Yeah, let’s absolutely do this! :-)

Regarding section 4 about feed discovery: Yeah, non-HTTP transport protocols are an issue as they do not have User-Agent headers. How exactly do you envision the discovery_url to work, though? I wouldn’t limit the transports to HTTP(S) in the Twtxt Specification, though. It’s up to the client to decide which protocols it wants to support.

Since I currently rely on buckket’s twtxt client to fetch the feeds, I can only follow http(s):// (and file://) feeds. But in tt2 I will certainly add some gopher:// and gemini:// at some point in time.

Some time ago, @movq@www.uninformativ.de found out that some Gopher/Gemini users prefer to just get an e-mail from people following them: https://twtxt.net/twt/dikni6q So, it might not even be something to be solved as there is no problem in the first place.

Section 5 on protocol support: You’re right, announcing the different transports in the url metadata would certainly help. :-)

Section 7 on emojis: Your idea of TUI/CLI avatars is really intriguing I have to say. Maybe I will pick this up in tt2 some day. :-)

@sorenpeter@darch.dk on 4 for gemini if your TLS client certificate contains your nick@host could that work for discovery?