@aelaraji@aelaraji.com @mckinley@twtxt.net rsync -avzr with an optional --progress is what I always use. Ah, I could use the shorter -P, thanks @movq@www.uninformativ.de.

@aelaraji@aelaraji.com rsync -zaXAP is what I use all the time. But that’s all – for the rest, I have to consult the manual. 😅

Image

Aggred. But reading twtxt in raw form sounds… I can’t do this

And finally the legibility of feeds when viewing them in their raw form are worsened as you go from a Twt Subject of (#abcdefg12345) to something like (https://twtxt.net/user/prologic/twtxt.txt 2024-09-22T07:51:16Z).

@sorenpeter@darch.dk Points 2 & 3 aren’t really applicable here in the discussion of the threading model really I’m afraid. WebMentions is completely orthogonal to the discussion. Further, no-one that uses Twtxt really uses WebMentions, whilst yarnd supports the use of WebMentions, it’s very rarely used in practise (if ever) – In fact I should just drop the feature entirely.

The use of WebSub OTOH is far more useful and is used by every single yarnd pod everywhere (no that there’s that many around these days) to subscribe to feed updates in ~near real-time without having the poll constantly.

Some more arguments for a local-based treading model over a content-based one:

1. The format: (#<DATE URL>) or (@<DATE URL>) both makes sense: # as prefix is for a hashtag like we allredy got with the (#twthash) and @ as prefix denotes that this is mention of a specific post in a feed, and not just the feed in general. Using either can make implementation easier, since most clients already got this kind of filtering.

2. Having something like (#<DATE URL>) will also make mentions via webmetions for twtxt easier to implement, since there is no need for looking up the #twthash. This will also make it possible to make 3th part twt-mentions services.

3. Supporting twt/webmentions will also increase discoverability as a way to know about both replies and feed mentions from feeds that you don’t follow.

Finally pubnix is alive! That’s im missing? Im only reading twtxt.net timeline because twtxt-v2.sh works slowly for displaying timeline…

@aelaraji@aelaraji.com Rsync has a ton of options and I probably still haven’t scratched the surface, but I was able to memorize the options I actually need for day-to-day work in a relatively short time. I guess I’m the opposite of you, because I don’t know any scp(1) options.

@movq@www.uninformativ.de Yes, the tools are surprisingly fast. Still, magrep takes about 20 seconds to search through my archive of 140K emails, so to speed things up I would probably combine it with an indexer like mu, mairix or notmuch.

Aunque me gusta mucho el concepto descentralizado de ‘twtxt’, este año no lo he utilizado tanto. No pude tener a mi círculo cercano, con quienes surgen las conversaciones que me gustan, y por el que se da un efecto de red significativo.

También estoy buscando un minimalismo digital, utilizando servicios que brinden alegría, valor y un uso de tiempo razonable.
Aunque es un tema controversial, ¿por qué no tener una comunidad de personas con las que sintamos que el mundo (digital al menos) es un lugar mejor?

Quizás un poco idealista el punto, aunque la intención es que el tiempo que pasamos en ‘la red’, nos ayude a crecer como personas, a disfrutar el tiempo, y a vivir esta vida digital con sentido.

Por todo esto, el poco tiempo que esté en microblogging, lo buscaré en las dos plataformas que más conversaciones significativas me generan, que por un lado es X, para todo lo profesional, y Mastodon, para lo hipster, indie, idealista, etc.

Si algo de lo que he compartido por twtxt ha sido importante para ti, o quieres que sigamos charlando, me puedes encontrar en alguna de estas otras plataformas:
https://text.eapl.mx/microblogging

@xuu@txt.sour.is I think it is more tricky than that.

https://commission.europa.eu/law/law-topic/data-protection/reform/rules-business-and-organisations/application-regulation/who-does-data-protection-law-apply_en

“A company or entity …”

Also, as I understand it, “personal or household activity” (as you called it) is rather strict: An example could be you uploading photos to a webspace behind HTTP basic auth and sending that link to a friend. So, yes, a webserver is involved and you process your friend’s data (e.g., when did he access your files), but it’s just between you and him. But if you were to publish these photos publicly on a webserver that anyone can access, then it’s a different story – even though you could say that “this is just my personal hobby, not related to any job or money”.

If you operate a public Yarn pod and if you accept registrations from other users, then I’m pretty sure the GDPR applies. 🤔 You process personal data and you don’t really know these people. It’s not a personal/private thing anymore.

@falsifian@www.falsifian.org I believe the preserve means to include the original subject hash in the start of the twt such as (#somehash)

So I whipped up a quick shell script to demonstrate what I mean by the increase in feed size on average as well as the expected increase in storage and retrieval requirements.

$ ./compare.sh
Original file size: 28145 bytes
Modified file size: 70672 bytes
Percentage increase in file size: 151.10%
...

Image

@bender@twtxt.net Ha! Maybe I should get on the Markdown train. You’re taking away my excuses.

@bender@twtxt.net

Sorry, you’re right, I should have used numbers!

I’m don’t understand what “preserve the original hash” could mean other than “make sure there’s still a twt in the feed with that hash”. Maybe the text could be clarified somehow.

I’m also not sure what you mean by markdown already being part of it. Of course people can already use Markdown, just like presumably nothing stopped people from using (twt subjects) before they were formally described. But it’s not universal; e.g. as a jenny user I just see the plain text.

@falsifian@www.falsifian.org The GDPR does not apply to the processing of data for a purely personal or household activity that is not connected to a professional or commercial activity.

@prologic@twtxt.net Do you feel the same about published vs. privately stored data?

For me there’s a distinction. I feel very strongly that I should be able to retain whatever private information I like. On the other hand, I do have some sympathy for requests not to publish or propagate (though I personally feel it’s still morally acceptable to ignore such requests).

@lyse@lyse.isobeef.org I’d suggest making the whole content-type thing a SHOULD, to accommodate people just using some hosting service they don’t have much control over. (The same situation could make detecting followers hard, but IMO “please email me if you follow me” is still legit twtxt, even if inconvenient.)

@prologic@twtxt.net Thanks for writing that up!

I hope it can remain a living document (or sequence of draft revisions) for a good long time while we figure out how this stuff works in practice.

I am not sure how I feel about all this being done at once, vs. letting conventions arise.

For example, even today I could reply to twt abc1234 with “(#abc1234) Edit: …” and I think all you humans would understand it as an edit to (#abc1234). Maybe eventually it would become a common enough convention that clients would start to support it explicitly.

Similarly we could just start using 11-digit hashes. We should iron out whether it’s sha256 or whatever but there’s no need get all the other stuff right at the same time.

I have similar thoughts about how some users could try out location-based replies in a backward-compatible way (append the replyto: stuff after the legacy (#hash) style).

However I recognize that I’m not the one implementing this stuff, and it’s less work to just have everything determined up front.

Misc comments (I haven’t read the whole thing):

• Did you mean to make hashes hexadecimal? You lose 11 bits that way compared to base32. I’d suggest gaining 11 bits with base64 instead.

• “Clients MUST preserve the original hash” — do you mean they MUST preserve the original twt?

• Thanks for phrasing the bit about deletions so neutrally.

• I don’t like the MUST in “Clients MUST follow the chain of reply-to references…”. If someone writes a client as a 40-line shell script that requires the user to piece together the threading themselves, IMO we shouldn’t declare the client non-conforming just because they didn’t get to all the bells and whistles.

• Similarly I don’t like the MUST for user agents. For one thing, you might want to fetch a feed without revealing your identty. Also, it raises the bar for a minimal implementation (I’m again thinking again of the 40-line shell script).

• For “who follows” lists: why must the long, random tokens be only valid for a limited time? Do you have a scenario in mind where they could leak?

• Why can’t feeds be served over HTTP/1.0? Again, thinking about simple software. I recently tried implementing HTTP/1.1 and it wasn’t too bad, but 1.0 would have been slightly simpler.

• Why get into the nitty-gritty about caching headers? This seems like generic advice for HTTP servers and clients.

• I’m a little sad about other protocols being not recommended.

• I don’t know how I feel about including markdown. I don’t mind too much that yarn users emit twts full of markdown, but I’m more of a plain text kind of person. Also it adds to the length. I wonder if putting a separate document would make more sense; that would also help with the length.

So I’m a location based system, how exactly do I reply to one of these two Twts from @Yarns@search.twtxt.net ? 🤔

2024-09-07T12:55:56Z	🥳 NEW FEED: @<twtxt http://edsu.github.io/twtxt/twtxt.txt>
2024-09-07T12:55:56Z	🥳 NEW FEED: @<kdy https://twtxt.kdy.ch/twtxt.txt>

Okay folks, I’ve spent all day on this today, and I think its in “good enough”™ shape to share:

Twtxt v2:

• Specification: https://docs.mills.io/uJXuisaYTRWYDrl8A2jADg?both
  
• implementation: https://gist.mills.io/prologic/afdec15443da4d7aa898f383f171ec1b
  

Image

LOl 😂 Not only have a tried to write up a full Twtxt v2 specification, I’ve also written a Bash shell script that implements the new spec 😅

Had to build a list of all feeds (that I follow) and all twts in them and there are two collisions already:

$ ./stats
Saw 58263 hashes
7fqcxaa
  https://twtxt.net/user/justamoment/twtxt.txt
  https://twtxt.net/user/prologic/twtxt.txt
ntnakqa
  https://twtxt.net/user/prologic/twtxt.txt
  https://twtxt.net/user/thecanine/twtxt.txt

Namely:

$ jenny -D https://twtxt.net/user/justamoment/twtxt.txt | grep 7fqcxaa

[7fqcxaa] [2022-12-28 04:53:30+00:00] [(#pmuqoca) @prologic@twtxt.net I checked the GitHub discussion, it became a request to join forces.

Do you plan on having them join?

Also for the name, how about:

• “progit” or “prologit” (prologic official hard fork)
  
• “git-stance” (git instance)
  
• “GitTree” (Gitea inspired, maybe to related)
  
• “Gitomata” (git automata)
  
• “Git.Source”
  
• “Forgor” (forgit is taken so I forgor) 🤣
  
• “SweetGit” (as salty chat)
  
• “Pepper Git” (other ingredients) 😉
  
• “GitHeart” (core of git with a GitHub sounding name)
  
• “GitTaka” (With music in mind)
  

Ok, enough fun… Hope this helps sprout some ideas from others if nothing is to your taste.]

$ jenny -D https://twtxt.net/user/prologic/twtxt.txt/5 | grep 7fqcxaa

[7fqcxaa] [2022-02-25 21:14:45+00:00] [(#bqq6fxq) It’s handled by blue Monday]

And:

$ jenny -D https://twtxt.net/user/thecanine/twtxt.txt | grep ntnakqa
[ntnakqa] [2022-01-23 10:24:09+00:00] [(#2wh7r4q) <a href="https://yarn.girlonthemoon.xyz/external?uri=https://twtxt.net/user/prologic/twtxt.txt">@prologic<em>@twtxt.net</em></a> I know, I was just hoping it might have also gotten fixed by that change, by some kind of backend miracles. 😂]

$ jenny -D https://twtxt.net/user/prologic/twtxt.txt/1 | grep ntnakqa
[ntnakqa] [2024-02-27 05:51:50+00:00] [(#otuupfq) <a href="https://yarn.girlonthemoon.xyz/external?uri=https://twtxt.net/user/shreyan/twtxt.txt">@shreyan<em>@twtxt.net</em></a>  Ahh 👌]

@prologic@twtxt.net Care to explain how that proves anything when someone else already got the spoofed twt with no way to tell it was? can’t an old twt just be deleted and give a similar result when grep-ed for?

Le me is worried! 😅

@bender@twtxt.net Just desktop notifications at the moment, but you could easily throw in a Ntfy server and get notified about anything you want, wherever you want. 🤣

👋 Reminder folks of the upcoming Yarn.social monthly online meetup:

I hope to see @david@collantes.us @movq@www.uninformativ.de @lyse@lyse.isobeef.org @xuu@txt.sour.is @sorenpeter@darch.dk and hopefully others too @aelaraji@aelaraji.com @falsifian@www.falsifian.org and anyone else that sees this! 🙏 We’re hopefully going to primarily discuss the future of Twtxt and the last few weeks of discussions 🤣

• Event: Yarn.social Online Meetup
  
• When: 28th September 2024 at 12:00pm UTC (midday)
  
• Where: Mills Meet : Yarn.social
  
• Cadence: 4th Saturday of every Month
  

Agenda:

• Let’s talk about the upcoming changes to the Twtxt spec(s)
  
  • See #xgghhnq
    

#Yarn.social #Meetup

@movq@www.uninformativ.de I cases of these kind of “abuse” of social trust. Then I think people should just delete their replies, unfollow the troll and leave them to shouting in the void. This is a inter-social issue, not a technical issue. Anything can be spoofed. We are not building a banking app, we are just having conversation and if trust are broken then communication breaks down. These edge-cases are all very hypothetical and not something I think we need to solve with technology.

@prologic@twtxt.net 🕵🏻 Hint: it was a twt about stolen property…

I have just made yet another convoluted twtxt notifications script! Feeling like an old dog learning new tricks! 🤣

Image

@prologic@twtxt.net “Even though there are some 😉 that have different views on this 🤣” — coyly raises the hand… LOL.

@aelaraji@aelaraji.com This is one of the reasons why yarnd has a couple of settings with some sensible/sane defaults:

I could already imagine a couple of extreme cases where, somewhere, in this peaceful world one’s exercise of freedom of speech could get them in Real trouble (if not danger) if found out, it wouldn’t necessarily have to involve something to do with Law or legal authorities. So, If someone asks, and maybe fearing fearing for… let’s just say ‘Their well being’, would it heart if a pod just purged their content if it’s serving it publicly (maybe relay the info to other pods) and call it a day? It doesn’t have to be about some law/convention somewhere … 🤷 I know! Too extreme, but I’ve seen news of people who’d gone to jail or got their lives ruined for as little as a silly joke. And it doesn’t even have to be about any of this.

There are two settings:

$ ./yarnd --help 2>&1 | grep max-cache
      --max-cache-fetchers int        set maximum numnber of fetchers to use for feed cache updates (default 10)
  -I, --max-cache-items int           maximum cache items (per feed source) of cached twts in memory (default 150)
  -C, --max-cache-ttl duration        maximum cache ttl (time-to-live) of cached twts in memory (default 336h0m0s)

So yarnd pods by default are designed to only keep Twts around publicly visible on either the anonymous Frontpage or Discover View or your Timeline or the feed’s Timeline for up to 2 weeks with a maximum of 150 items, whichever get exceeded first. Any Twts over this are considered “old” and drop off the active cache.

It’s a feature that my old man @off_grid_living@twtxt.net was very strongly in support of, as was I back in the day of yarnd’s design (nothing particularly to do with Twtxt per se) that I’ve to this day stuck by – Even though there are some 😉 that have different views on this 🤣

@movq@www.uninformativ.de @falsifian@www.falsifian.org @prologic@twtxt.net Maybe I don’t know what I’m talking about and You’ve probably already read this: Everything you need to know about the “Right to be forgotten” coming straight out of the EU’s GDPR Website itself. It outlines the specific circumstances under which the right to be forgotten applies as well as reasons that trump the one’s right to erasure …etc.

I’m no lawyer, but my uneducated guess would be that:

A) twts are already publicly available/public knowledge and such… just don’t process children’s personal data and MAYBE you’re good? Since there’s this:

… an organization’s right to process someone’s data might override their right to be forgotten. Here are the reasons cited in the GDPR that trump the right to erasure:

• The data is being used to exercise the right of freedom of expression and information.
  
• The data is being used to perform a task that is being carried out in the public interest or when exercising an organization’s official authority.
  
• The data represents important information that serves the public interest, scientific research, historical research, or statistical purposes and where erasure of the data would likely to impair or halt progress towards the achievement that was the goal of the processing.
  

B) What I love about the TWTXT sphere is it’s Human/Humane element! No deceptive algorithms, no Corpo B.S …etc. Just Humans. So maybe … If we thought about it in this way, it wouldn’t heart to be even nicer to others/offering strangers an even safer space.
I could already imagine a couple of extreme cases where, somewhere, in this peaceful world one’s exercise of freedom of speech could get them in Real trouble (if not danger) if found out, it wouldn’t necessarily have to involve something to do with Law or legal authorities. So, If someone asks, and maybe fearing fearing for… let’s just say ‘Their well being’, would it heart if a pod just purged their content if it’s serving it publicly (maybe relay the info to other pods) and call it a day? It doesn’t have to be about some law/convention somewhere … 🤷 I know! Too extreme, but I’ve seen news of people who’d gone to jail or got their lives ruined for as little as a silly joke. And it doesn’t even have to be about any of this.

P.S: Maybe make X tool check out robots.txt? Or maybe make long-term archives Opt-in? Opt-out?
P.P.S: Already Way too many MAYBE’s in a single twt! So I’ll just shut up. 😅

Bahahahaha very clever @lyse@lyse.isobeef.org I look forward to reading your report ! 🤣 However…

$ yarnc debug https://twtxt.net/user/prologic/twtxt.txt | grep -E '^pqst4ea' | tee | wc -l
0

I very quickly proved that Twt was never from me 🤣

@prologic@twtxt.net I have no specifics, only hopes. (I have seen some articles explaining the GDPR doesn’t apply to a “purely personal or household activity” but I don’t really know what that means.)

I don’t know if it’s worth giving much thought to the issue unless either you expect to get big enough for the GDPR to matter a lot (I imagine making money is a prerequisite) or someone specifically brings it up. Unless you enjoy thinking through this sort of thing, of course.

@falsifian@www.falsifian.org Do you have specifics about the GRPD law about this?

Would the GDPR would apply to a one-person client like jenny? I seriously hope not. If someone asks me to delete an email they sent me, I don’t think I have to honour that request, no matter how European they are.

I’m not sure myself now. So let’s find out whether parts of the GDPR actually apply to a truly decentralised system? 🤔

And they have arrived (well, they did around 3 hours ago, LOL). Buttery smooth, my 16 Pro (one with dark cover). It took a bit over an hour to transfer all my data.

Image

@lyse@lyse.isobeef.org yeah, tell us, @prologic@twtxt.net, what isn’t true? 🤔 You can’t just go around, “that’s not true, and that’s not true; and that, and that!” without spelling out exactly what isn’t, and why? For the love of god, why?! 😂

@falsifian@www.falsifian.org comments on the feeds as in nick, url, follow, that kind of thing? If that, then not interested at all. I envision an archive that would allow searching, and potentially browsing threads on a nice, neat interface. You will have to think, though, on other things. Like, what to do with images? Yarn allows users to upload images, but also embed it in twtxts from other sources (hotlinking, actually).

@david@collantes.us Thanks, that’s good feedback to have. I wonder to what extent this already exists in registry servers and yarn pods. I haven’t really tried digging into the past in either one.

How interested would you be in changes in metadata and other comments in the feeds? I’m thinking of just permanently saving every version of each twtxt file that gets pulled, not just the twts. It wouldn’t be hard to do (though presenting the information in a sensible way is another matter). Compression should make storage a non-issue unless someone does something weird with their feed like shuffle the comments around every time I fetch it.

@falsifian@www.falsifian.org “I was actually thinking about making an Internet Archive style twtxt archiver, letting you explore past twts” — that’s an awesome idea for a project. Something I would certainly use!

@movq@www.uninformativ.de I don’t think it has to be like that. Just make sure the new version of the twt is always appended to your current feed, and have some convention for indicating it’s an edit and which twt it supersedes. Keep the original twt as-is (or delete it if you don’t want new followers to see it); doesn’t matter if it’s archived because you aren’t changing that copy.

@prologic@twtxt.net Do you have a link to some past discussion?

Would the GDPR would apply to a one-person client like jenny? I seriously hope not. If someone asks me to delete an email they sent me, I don’t think I have to honour that request, no matter how European they are.

I am really bothered by the idea that someone could force me to delete my private, personal record of my interactions with them. Would I have to delete my journal entries about them too if they asked?

Maybe a public-facing client like yarnd needs to consider this, but that also bothers me. I was actually thinking about making an Internet Archive style twtxt archiver, letting you explore past twts, including long-dead feeds, see edit histories, deleted twts, etc.

@prologic@twtxt.net what time in UTC?

@prologic@twtxt.net cool, I will be there! Are you going to post the regular banner notice? It will serve as a reminder, at least for me.

@lyse@lyse.isobeef.org I don’t think this is true.

@david@collantes.us Well, I wouldn’t recommend using my code for your main jenny use anyway. If you want to try it out, set XDG_CONFIG_HOME and XDG_CACHE_HOME to some sandbox directories and only run my code there. If @movq@www.uninformativ.de is interested in any of this getting upstreamed, I’d be happy to try rebasing the changes, but otherwise it’s a proof of concept and fun exercise.

@falsifian@www.falsifian.org hiya from the Sunshine State (also known as a never-ending hell, LOL)!

@falsifian@www.falsifian.org this one hits hard, as jenny was just updated today. :‘-(

@david@collantes.us Hello!