the stem matching is the same as how GIT does its branch hashes. i think you can stem it down to 2 or 3 sha bytes.
if a client sees someone in a yarn using a byte longer hash it can lengthen to match since it can assume that maybe the other client has a collision that it doesnt know about.
@sorenpeter@darch.dk hmm, how does your client handles āa little editingā? I am sure threads would break just as well. š
Iām not advocating in either direction, btw. I havenāt made up my mind yet. š Just braindumping here.
The (replyto:ā¦)
proposal is definitely more in the spirit of twtxt, Iād say. Itās much simpler, anyone can use it even with the simplest tools, no need for any client code. That is certainly a great property, if you ask me, and itās things like that that brought me to twtxt in the first place.
Iād also say that in our tiny little community, message integrity simply doesnāt matter. Signed feeds donāt matter. I signed my feed for a while using GPG, someone else did the same, but in the end, nobody cares. The community is so tiny, thereās enough āimplicit trustā or whatever you want to call it.
If twtxt/Yarn was to grow bigger, then this would become a concern again. But even Mastodon allows editing, so how much of a problem can it really be? š
I do have to āadmitā, though, that hashes feel better. It feels good to know that we can clearly identify a certain twt. It feels more correct and stable.
Hm.
I suspect that the (replyto:ā¦)
proposal would work just as well in practice.
@prologic@twtxt.net I wouldnāt want my client to honour delete requests. I like my computerās memory to be better than mine, not worse, so it would bug me if I remember seeing something and my computer canāt find it.
An alternate idea for supporting (properly) Twt Edits is to denoate as such and extend the meaning of a Twt Subject (which would need to be called something better?); For example, letās say I produced the following Twt:
2024-09-18T23:08:00+10:00 Hllo World
And my feedās URI is https://example.com/twtxt.txt
. The hash for this Twt is therefore 229d24612a2
:
$ echo -n "https://example.com/twtxt.txt\n2024-09-18T23:08:00+10:00\nHllo World" | sha1sum | head -c 11
229d24612a2
You wish to correct your mistake, so you make an amendment to that Twt like so:
2024-09-18T23:10:43+10:00 (edit:#229d24612a2) Hello World
Which would then have a new Twt hash value of 026d77e03fa
:
$ echo -n "https://example.com/twtxt.txt\n2024-09-18T23:10:43+10:00\nHello World" | sha1sum | head -c 11
026d77e03fa
Clients would then take this edit:#229d24612a2
to mean, this Twt is an edit of 229d24612a2
and should be replaced in the clientās cache, or indicated as such to the user that this is the intended content.
(#hash;#originalHash)
would also work.
Maybe Iām being a bit too purist/minimalistic here. As I said before (in one of the 1372739 posts on this topic ā or maybe I didnāt even send that twt, I donāt remember š ), I never really liked hashes to begin with. They arenāt super hard to implement but they are kind of against the beauty of the original twtxt ā because you need special client support for them. Itās not something that you could write manually in your
twtxt.txt
file. With @sorenpeter@darch.dkās proposal, though, that would be possible.
Tangentially related, I was a bit disappointed to learn that the twt subject extension is now never used except with hashes. Manually-written subjects sounded so beautifully ad-hoc and organic as a way to disambiguate replies. Maybe Iāll try it some time just for fun.
@prologic@twtxt.net Yeah, that thing with (#hash;#originalHash)
would also work.
Maybe Iām being a bit too purist/minimalistic here. As I said before (in one of the 1372739 posts on this topic ā or maybe I didnāt even send that twt, I donāt remember š
), I never really liked hashes to begin with. They arenāt super hard to implement but they are kind of against the beauty of the original twtxt ā because you need special client support for them. Itās not something that you could write manually in your twtxt.txt
file. With @sorenpeter@darch.dkās proposal, though, that would be possible.
I donāt know ⦠maybe itās just me. š„“
Iām also being a bit selfish, to be honest: Implementing (#hash;#originalHash)
in jenny for editing your own feed would not be a no-brainer. (Editing is already kind of unsupported, actually.) It wouldnāt be a problem to implement it for fetching other peopleās feeds, though.
no my fault your client canāt handle a little editing ;)
(replyto:http://darch.dk/twtxt.txt,2024-09-15T12:06:27Z)
I think I like this a lot. š¤
The problem with using hashes always was that theyāre āone-directionalā: You can construct a hash from URL + timestamp + twt, but you cannot do the inverse. When I see ā, I have no idea what that could possibly refer to.
But of course something like (replyto:http://darch.dk/twtxt.txt,2024-09-15T12:06:27Z)
has all the information you need. This could simplify twt/feed discovery quite a bit, couldnāt it? š¤ That thing that I just implemented ā jenny asking some Yarn pod for some twt hash ā would not be necessary anymore. Clients could easily and automatically fetch complete threads instead of requiring the user to follow all relevant feeds.
Only using the timestamp to identify a twt also solves the edit problem.
It even is better for non-Yarn clients, because you now donāt have to read, understand, and implement a ātwt hash specificationā before you can reply to someone.
The only problem, really, is that (replyto:http://darch.dk/twtxt.txt,2024-09-15T12:06:27Z)
is so long. Clients would have to try harder to hide this. š
@aelaraji@aelaraji.com So what is it about @sorenpeter@darch.dkās feed thatās screwed with your client? (Jenny?) š¤ Kind of curious now š¤£
@falsifian@www.falsifian.org TLS wonāt help you if you change your domain name. How will people know if itās really you? Maybe thatās not the biggest problem for something with such low stakes as twtxt, but itās a reasonable concern that could be solved using signatures from an unchanging cryptographic key.
This idea is the basis of Nostr. Notes can be posted to many relays and every note is signed with your private key. It doesnāt matter where you get the note from, your client can verify its authenticity. That way, relays donāt need to be trusted.
HTTPS is supposed to do [verification] anyway.
TLS provides verification that nobody is tampering with or snooping on your connection to a server. It doesnāt, for example, verify that a file downloaded from server A is from the same entity as the one from server B.
I was confused by this response for a while, but now I think I understand what youāre getting at. You are pointing out that with signed feeds, I can verify the authenticity of a feed without accessing the original server, whereas with HTTPS I canāt verify a feed unless I download it myself from the origin server. Is that right?
I.e. if the HTTPS origin server is online and I donāt mind taking the time and bandwidth to contact it, then perhaps signed feeds offer no advantage, but if the origin server might not be online, or I want to download a big archive of lots of feeds at once without contacting each server individually, then I need signed feeds.
feed locations [being] URLs gives some flexibility
It does give flexibility, but perhaps we should have made them URIs instead for even more flexibility. Then, you could use a tag URI,
urn:uuid:*
, or a regular old URL if you wanted to. The spec seems to indicate that theurl
tag should be a working URL that clients can use to find a copy of the feed, optionally at multiple locations. Iām not very familiar with IP{F,N}S but if it ensures you own an identifier forever and that identifier points to a current copy of your feed, it could be a great way to fix it on an individual basis without breaking any specs :)
Iām also not very familiar with IPFS or IPNS.
I havenāt been following the other twts about signatures carefully. I just hope whatever you smart people come up with will be backwards-compatible so it still works if Iām too lazy to change how I publish my feed :-)
@prologic@twtxt.net wellā¦
how would that work exactly?
To my limited knowledge, Keyoxide is an open source project offering different tools for verifying oneās online persona(s). Thatās done by either A) creating an Ariande Profile using the web interface, a CLI. or B) Just using your GPG key. Either way, you add in Identity claims to your different profiles, links and whatnot, and finally advertise your profile ⦠Then there is a second set of Mobile/Web clients and CLI your correspondents can use to check your identity claims. I think of them like the front-ends of GPG Keyservers (which keyoxide leverages for verification when you opt for the GPG Key method), where you verify profiles using links, Key IDs and Fingerprintsā¦
Who maintains cox site? Is it centralized or decentralized can be relied upon?
- Maintainers? Definitely not me, but hereās their Git stuff and OpenCollective page ā¦
- Both ASP and Keyoxide Webtools can be self-hosted. I donāt see a central authority here⦠+ As mentioned on their FAQ page the whole process can be done manually, so you donāt have to relay on any one/thing if you donāt want to, the whole thing is just another tool for convenience (with a bit of eye candy).
Does that mean then that every user is required to have a cox side profile?
Nop. But it looks like a nice option to prove that Iām the same person to whom that may concern if I ever change my Twtxt URL, host/join a yarn pod or if I reach out on other platforms to someone Iāve met in her. Otherwise Iām just happy exchanging GPG keys or confirm the change IRL at a coffee shop or something. š
@sorenpeter@darch.dk There was a client that would generate a unique hash for each twt. It didnāt get wide adoption.
Interesting.. QUIC isnāt very quick over fast internet.
QUIC is expected to be a game-changer in improving web application performance. In this paper, we conduct a systematic examination of QUICās performance over high-speed networks. We find that over fast Internet, the UDP+QUIC+HTTP/3 stack suffers a data rate reduction of up to 45.2% compared to the TCP+TLS+HTTP/2 counterpart. Moreover, the performance gap between QUIC and HTTP/2 grows as the underlying bandwidth increases. We observe this issue on lightweight data transfer clients and major web browsers (Chrome, Edge, Firefox, Opera), on different hosts (desktop, mobile), and over diverse networks (wired broadband, cellular). It affects not only file transfers, but also various applications such as video streaming (up to 9.8% video bitrate reduction) and web browsing. Through rigorous packet trace analysis and kernel- and user-space profiling, we identify the root cause to be high receiver-side processing overhead, in particular, excessive data packets and QUICās user-space ACKs. We make concrete recommendations for mitigating the observed performance issues.
So this is a great thread. I have been thinking about this too.. and what if we are coming at it from the wrong direction? Identity being tied to a given URL has always been a pain point. If i get a new URL its almost as if i have a new identity because not only am I serving at a new location but all my previous communications are broken because the hashes are all wrong.
What if instead we used this idea of signatures to thread the URLs together into one identity? We keep the URL to Hash in place. Changing that now is basically a no go. But we can create a signature chain that can link identities together. So if i move to a new URL i update the chain hosted by my primary identity to include the new URL. If i have an archived feed that the old URL is now dead, we can point to where it is now hosted and use the current convention of hashing based on the first url:
The signature chain can also be used to rotate to new keys over time. Just sign in a new key or revoke an old one. The prior signatures remain valid within the scope of time the signatures were made and the keys were active.
The signature file can be hosted anywhere as long as it can be fetched by a reasonable protocol. So say we could use a webfinger that directs to the signature file? you have an identity like frank@beans.co
that will discover a feed at some URL and a signature chain at another URL. Maybe even include the most recent signing key?
From there the client can auto discover old feeds to link them together into one complete timeline. And the signatures can validate that its all correct.
I like the idea of maybe putting the chain in the feed preamble and keeping the single self contained file.. but wonder if that would cause lots of clutter? The signature chain would be something like a log with what is changing (new key, revoke, add url) and a signature of the change + the previous signature.
# chain: ADDKEY kex14zwrx68cfkg28kjdstvcw4pslazwtgyeueqlg6z7y3f85h29crjsgfmu0w
# sig: BEGIN SALTPACK SIGNED MESSAGE. ...
# chain: ADDURL https://txt.sour.is/user/xuu
# sig: BEGIN SALTPACK SIGNED MESSAGE. ...
# chain: REVKEY kex14zwrx68cfkg28kjdstvcw4pslazwtgyeueqlg6z7y3f85h29crjsgfmu0w
# sig: ...
@prologic@twtxt.net do that mean that for every new post (not replies) the client will have to generate a UUID or similar when posting and add that to to the twt?
# follow_notify = gemini://foo/bar
to your feedās metadata, so that clients who follow you can ping that URL every now and then? How would you even notice that, do you regularly read your gemini logs? š¤
@movq@www.uninformativ.de @prologic@twtxt.net Hey! I may have found a silly trick to announce my following to people hosting their feeds on the Gemini space using the requested URI
itself instead of relaying on the USER Agent
š. Iāve copied my current feed over to my (to be) Gemlog for testing. And if I do a jenny -D "gemini://gem.aelaraji.com/twtxt.txt?follower=aelaraji@https://aelaraji.com/twtxt.txt"
and this happens:
A) As a follower, I get the feed as usual.
B) As the feed owner, I get this in logs:
hostname:1965 - āgemini://gem.aelaraji.com/twtxt.txt?follower=aelaraji@https://aelaraji.com/twtxt.txtā 20 ātext/plain;lang=en-USā
You could do the same for Gopher feeds but only if you want to announce yourself by throwing in an error in their logs, then youāll need a second request to fetch the feed. jenny -D "gopher://gopher.aelaraji.com/twtxt.txt&follower=aelaraji@https:/aelaraji.com/twtxt.txt"
gave me this :
gopher.aelaraji.com:70 - [09/Sep/2024:22:08:54 +0000] āGET 0/twtxt.txt&follower=aelaraji@https:/aelaraji.com/twtxt.txt HTTP/1.0ā 404 0 āā āUnknown gopher clientā
NB: the follower=...
string wonāt appear in gopher logs after a ?
but if I replace it with a +
or a &
and it works. There will be a missing /
after the https:
. Probably a client thing.
@mckinley@twtxt.net To answer some of your questions:
Are SSH signatures standardized and are there robust software libraries that can handle them? Weāll need a library in at least Python and Go to provide verified feed support with the currently used clients.
We already have this. Ed25519 libraries exist for all major languages. Aside from using ssh-keygen -Y sign
and ssh-keygen -Y verify
, you can also use the salty
CLI itself (https://git.mills.io/prologic/salty), and Iām sure there are other command-line tools that could be used too.
If we all implemented this, every twt hash would suddenly change and every conversation thread weāve ever had would at least lose its opening post.
Yes. This would happen, so weād have to make a decision around this, either a) a cut-off point or b) some way to progressively transition.
@lyse@lyse.isobeef.org This looks like a nice way to do it.
Another thought: if clients canāt agree on the url (for example, if we switch to this new way, but some old clients still do it the old way), that could be mitigated by computing many hashes for each twt: one for every url in the feed. So, if a feed has three URLs, every twt is associated with three hashes when it comes time to put threads together.
A client stills need to choose one url to use for the hash when composing a reply, but this might add some breathing room if thereās a period when clients are doing different things.
(From what I understand of jenny, this would be difficult to implement there since each pseudo-email can only have one msgid to match to the in-reply-to headers. I donāt know about other clients.)
@bender@twtxt.net Yes, they do 𤣠Implicitly, or threading would never work at all š Nor lookups 𤣠They are used as keys. Think of them like a primary key in a database or index. I totally get where youāre coming from, but there are trade-offs with using Message/Thread Ids as opposed to Content Addressing (like we do) and I believe we would just encounter other problems by doing so.
My money is on extending the Twt Subject extension to support more (optional) advanced āsubjectsā; i.e: indicating you edited a Twt you already published in your feed as @falsifian@www.falsifian.org indicated š
Then we have a secondary (bure much rarer) problem of the āidentityā of a feed in the first place. Using the URL you fetch the feed from as @lyse@lyse.isobeef.org ās client tt
seems to do or using the # url =
metadata field as every other client does (according to the spec) is problematic when you decide to change where you host your feed. In fact the spec says:
Users are advised to not change the first one of their urls. If they move their feed to a new URL, they should add this new URL as a new url field.
See Choosing the Feed URL ā This is one of our longest debates and challenges, and I think (_I suspect along with @xuu@txt.sour.is _) that the right way to solve this is to use public/private key(s) where you actually have a public key fingerprint as your feedās unique identity that never changes.
@movq@www.uninformativ.de @prologic@twtxt.net Another option would be: when you edit a twt, prefix the new one with (#[old hash]) and some indication that itās an edited version of the original tweet with that hash. E.g. if the hash used to be abcd123, the new version should start ā(#abcd123) (redit)ā.
What I like about this is that clients that donāt know this convention will still stick it in the same thread. And I feel itās in the spirit of the old pre-hash (subject) convention, though thatās before my time.
I guess it may not work when the edited twt itself is a reply, and there are replies to it. Maybe that could be solved by letting twts have more than one (subject) prefix.
But the great thing about the current system is that nobody can spoof message IDs.
I donāt think twtxt hashes are long enough to prevent spoofing.
@bender@twtxt.net On twtxt, I follow all feeds that I can find (there are some exceptions, of course). Thereās so little going on in general, it hardly matters. š
And I just realized: Muttās layout helps a lot. Skimming over new twts is really easy and itās not a big loss if there are a couple of shitposts⢠in my ātimelineā. This is very different from Mastodon (both the default web UI and all clients Iāve tried), where the timeline is always huge. Posts take up a lot of space on screen. Makes me think twice if I want to follow someone or not. š
(I mostly only follow Hashtags on Mastodon anyway. Itās more interesting that way.)
@cuaxolotl@sunshinegardens.org Ah, thanks for reporting back! Okay, so youāre basically manually ācrawlingā feeds right now. š¤ What do you think about the idea of adding something like # follow_notify = gemini://foo/bar
to your feedās metadata, so that clients who follow you can ping that URL every now and then? How would you even notice that, do you regularly read your gemini logs? š¤
Anyone had any intereractions with @cuaxolotl@sunshinegardens.org yet? Or are they using a client that doesnāt know how to detect clients following them properly? Hmmm š§
early preview of my new web-based twtxt client https://sunshinegardens.org/static/howl/
@aelaraji@aelaraji.com Ahh I see! Interesting š§ Would you prefer that clients like yarnd
prefetch resources liks this, cache them and serve the cached copy? š¤
@bender@twtxt.net My index formatting is intact, probably because I still havenāt figured out how to set up my terminal to show RTL text correctly! š but hey, that wonāt be a problem anymore, I donāt feel like twting in Arabic. Sorry for the inconvenience.
twtxt
client by buckket to actually fetch and fill the cache. I think one of of the patches played around with the error reporting. This way, any problems with fetching or parsing feeds show up immediately. Once I think, I've seen enough errors, I unsubscribe.
@lyse@lyse.isobeef.org ah, if only you were to finally clean up that code, and make that client widely availableā¦! One can only dream, right? :-)
Correct, @bender@twtxt.net. Since the very beginning, my twtxt flow is very flawed. But it turns out to be an advantage for this sort of problem. :-) I still use the official (but patched) twtxt
client by buckket to actually fetch and fill the cache. I think one of of the patches played around with the error reporting. This way, any problems with fetching or parsing feeds show up immediately. Once I think, Iāve seen enough errors, I unsubscribe.
tt
is just a viewer into the cache. The read statuses are stored in a separate database file.
It also happened a few times, that I thought some feed was permanently dead and removed it from my list. But then, others mentioned it, so I resubscribed.
@falsifian@www.falsifian.org @bender@twtxt.net Iād certainly hate my client for automatic feed unsubscription, too.
@bender@twtxt.net Based on my experience so far, as a user, I would be upset if my client dropped someone from my follower list, i.e. stopped fetching their feed, without me asking for that to happen.
receieveFile()
)? š¤
@prologic@twtxt.net I donāt know if this is new, but Iām seeing:
Jul 25 16:01:17 buc yarnd[1921547]: time="2024-07-25T16:01:17Z" level=error msg="https://yarn.stigatle.no/user/stigatle/twtxt.txt: client.Do fail: Get \"https://yarn.stigatle.no/user/stigatle/twtxt.txt\": dial tcp 185.97.32.18:443: i/o timeout (Client.Timeout exceeded while awaiting headers)" error="Get \"https://yarn.stigatle.no/user/stigatle/twtxt.txt\": dial tcp 185.97.32.18:443: i/o timeout (Client.Timeout exceeded while awaiting headers)"
I no longer see twts from @stigatle@yarn.stigatle.no at all.
I havnt seen any emails about the outage at work. I know i have the mac crowdstrike client though. My buddy that works at a hospital says they wernt affected.
Referer
is /post
then consider that total bullshit, and ignore? š¤
@prologic@twtxt.net I was wondering if my reverse proxy could cause something but itās pretty standardā¦
server {
listen 80; server_name we.loveprivacy.club;
location / {
return 301 https://$host$request_uri;
<a href="https://yarn.girlonthemoon.xyz/search?q=%23proxy_pass">#proxy_pass</a> http://127.0.0.1:8000;
}
}
server {
listen 443 ssl http2;
server_name we.loveprivacy.club;
ssl_certificate /etc/letsencrypt/live/we.loveprivacy.club/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/we.loveprivacy.club/privkey.pem;
client_max_body_size 8M;
location / {
proxy_pass http://127.0.0.1:8000;
}
}
@aelaraji@mastodon.social @aelaraji@aelaraji.com Aw, thanks. I should install a twtxt client I guess⦠But I prefere rss, so for now, itās just a twtxt2atom script doing the job :)
Well! My 24 hrs without a GUI Web browser was quite of a nice experience.
As a matter of fact, and as long as Iām not doing any 3D work, I kind of donāt need gui applications as much as it feels like.
Even though, a couple of websites asked me to eff off because they need
JavaScript to work. Some others handed me a cold ā402 Upgrade Requiredā client
error response⦠(LOL letās not even talk about how Github repos looked
and felt like). I have managed to fix a couple of things Iāve been meaning to
for quite some time but never got, mainly to because of my browsing
habits. I tend to open a lot of tabs, read some, get distracted then
open some more and down the rabbit hole (or shall I say tabs) I go.
All in all, it was quite a nice experience.
How nice? It was an āIām dropping into a full TTY experience for another
24 hrsā kind of nice!
Although, I miss using a mouse already, but hey, I would have never
heard about gpm(8) otherwise.
Its quite nice. I have been half tempted to make a twtxt client with it
At last! my Twtxt feed is up and running and I can post to it from a remote client! Yey!
Hey @sorenpeter@darch.dk, Iām sorry to tell you, but the prev
field in your feedās headers is invalid. š
First, it doesnāt include the hash of the last twt in the archive. Second, and thatās probably more important, it forms an infinite loop: The prev
field of your main feed specifies http://darch.dk/twtxt-archive.txt and that file then again specifies http://darch.dk/twtxt-archive.txt. Some clients might choke on this, mine for example. š Iāll push a fix soon, though.
For reference, the prev
field is described here: https://dev.twtxt.net/doc/archivefeedsextension.html
My coworker started chatting over wall
this morning as we were both on the same server investigating something⦠itās the best chat client haha
Not making THREADING the default view of e-mail clients and thus teaching users that e-mail is āchaoticā (if you get a lot of mail, it becomes unusable without threading) and āneedsā full quoting all the time was one of the worst mistakes ever.
#gemini readers, I wrote a tool to download new gemfeeds entries instead of opening a client: gemini://si3t.ch/log/2024-02-28-gemfeeds-downloader.txt
Seriously, where is the suckless-style Nostr client?
>
?
@sorenpeter@darch.dk this makes sense as a quote twt that references a direct URL. If we go back to how it developed on twitter originally it was RT @nick: original text
because it contained the original text the twitter algorithm would boost that text into trending.
i like the format (#hash) @<nick url> > "Quoted text"\nThen a comment
as it preserves the human read able. and has the hash for linking to the yarn. The comment part could be optional for just boosting the twt.
The only issue i think i would have would be that that yarn could then become a mess of repeated quotes. Unless the client knows to interpret them as multiple users have reposted/boosted the thread.
The format is also how iphone does reactions to SMS messages with +number liked: original SMS
>
?
Iām also more in favor of #reposts being human readable and writable. A client might implement a bottom that posts something simple like: #repost Look at this cool stuff, because bla bla [alt](url)
This will then make it possible to also ārepostā stuff from other platforms/protocols.
The reader part of a client, can then render a preview of the link, which we talked about would be a nice (optional) feature to have in yarnd.
Un twt pour tester mon nouveau client qui doit māafficher le nombre de twt publiĆ©s
Anyone else working with Mac OS (work), Windows (client project) and Fedora (private) on the same day, almost every day?
I am back on twtxt for now. I am using twtwt client. Donāt think that it does replies so I should try jenny with mutt again.