7
to 12
and use the first 12
characters of the base32 encoded blake2b hash. This will solve two problems, the fact that all hashes today either end in q
or a
(oops) 😅 And increasing the Twt Hash size will ensure that we never run into the chance of collision for ions to come. Chances of a 50% collision with 64 bits / 12 characters is roughly ~12.44B Twts. That ought to be enough! -- I also propose that we modify all our clients and make this change from the 1st July 2025, which will be Yarn.social's 5th birthday and 5 years since I started this whole project and endeavour! 😱 #Twtxt #Update
July 1st. 63 days from now to implement a backward-incompatible change, apparently not open to other ideas like replacing blake with SHA, or discussing implementation challenges for other languages and platforms.
Finally just closing #18, #19 and #20 without starting a proper discussion and ignoring a ‘micro consensus’ feels… not right.
I don’t know what to think rather than letting it rest (May will be busy here) and focus on other stuff in the future.
7
to 12
and use the first 12
characters of the base32 encoded blake2b hash. This will solve two problems, the fact that all hashes today either end in q
or a
(oops) 😅 And increasing the Twt Hash size will ensure that we never run into the chance of collision for ions to come. Chances of a 50% collision with 64 bits / 12 characters is roughly ~12.44B Twts. That ought to be enough! -- I also propose that we modify all our clients and make this change from the 1st July 2025, which will be Yarn.social's 5th birthday and 5 years since I started this whole project and endeavour! 😱 #Twtxt #Update
Today I added support for Let’s Encrypt to eris via DNS-01 challenge. Updated the gcore libdns package I wrote for Caddy, Maddy and now Eris. Add support for yarn’s cache to support # type = bot
and optionally # retention = N
so that feeds like @tiktok@feeds.twtxt.net work like they did before, and… Updated some internal metrics in yarnd
to be IMO “better”, with queue depth, queue time and last processing time for feeds.
First draft of yarnd 0.16 release notes. 📝 – Probably needs some tweaking and fixing, but it’s sounding alright so far 👌 #yarnd
cacher
branch? 🤔 It is recommended you take a full backup of you pod beforehand, just in case. Keen to get this branch merged and to cut a new release finally after >2 years 🤣
@kat@yarn.girlonthemoon.xyz Yes see UPGRADE.md – I believe @xuu@txt.sour.is is now running this live after a couple of hiccups and a bug fix. So yeah if you can, that would be cool, basically looking for early beta testers (I was the alpha tester 🤣)
Proposal: Change the order of twts in the feeds
https://git.mills.io/yarnsocial/twtxt.dev/issues/26
I drop the bomb and leave! 💣 🏃➡️
#twtxt
Ha! I stand corrected, didn’t scrolled long enough. Indeed, it should be added (you will need an account on Mills’ Gitea), noted.
@bender@twtxt.net There is only one commit that I can think of that might be the cause here. Shall I revert and redeploy? 🤔
@lyse@lyse.isobeef.org I think we found a bug in the lextwt parser actually 😅
@movq@www.uninformativ.de let’s host yarnd! Or maybe wait until @prologic@twtxt.net return activitypub support which deleted in this commit
@eapl.me@eapl.me Cool!
Proposal 3 (https://git.mills.io/yarnsocial/twtxt.dev/issues/18#issuecomment-19215) has the “advantage”, that you do not have to “mention” the original author if the thread slightly diverges. It seems to be a thing here that conversations are typically very flat instead of trees. Hence, and despite being a tree hugger, I voted for 3 being my favorite one, then 2, 1 and finally 4.
All proposals still need more work to clarify the details and edge cases in my opinion before they can be implemented.
it seems to be confused with the subject right next to it.. it works better at the end of the twt string.
Yarn won’t display anything. but the parser does add it to the AST in a way that you can parse it out using twt.Attrs().Get("lang")
https://git.mills.io/yarnsocial/go-lextwt/src/branch/main/ast.go#L1270-L1272
https://git.mills.io/yarnsocial/go-types/src/branch/main/twt.go#L473-L478
Hi everyone,
I’ve drafted a Request for Comments (RFC) to improve how threads work in twtxt:
https://git.mills.io/yarnsocial/twtxt.dev/issues/18
I’d love your feedback! Please share your thoughts on anything that could be better explained, check if the proposed dates work for everyone, and I invite you to join the discussion…
TwtAttrs
https://git.mills.io/yarnsocial/go-lextwt/pulls/17
Actually it was your old feed on eapl.mx
TwtAttrs
https://git.mills.io/yarnsocial/go-lextwt/pulls/17
Actually it was your old feed on eapl.mx
@prologic@twtxt.net sure thing!
What about discussing it in https://git.mills.io/yarnsocial/twtxt.dev ?
The only con I see is that everyone would need to create an account there to participate.
@andros@twtxt.andros.dev I’ve commented on the ticket: https://git.mills.io/yarnsocial/twtxt.dev/issues/14#issuecomment-19142
In all reality, though, I don’t see that our community will come to an agreement. Some folks just don’t want to give up on the content-based addressing scheme.
@lyse@lyse.isobeef.org What do you think about this? https://git.mills.io/yarnsocial/twtxt.dev/issues/14
Edit #issuecomment-18968
@eapl.me@eapl.me @andros@twtxt.andros.dev here’s another variant as per eapl’s suggestion (#issuecomment-18968)[https://git.mills.io/yarnsocial/twtxt.dev/issues/9#issuecomment-18968]
What would you like the new twtxt logo to be?
Comments: https://git.mills.io/yarnsocial/twtxt.dev/issues/9#issuecomment-18960
i made a little twtxt feed fixer for when a feed uses other whitespace instead of tabs.
@prologic@twtxt.net we need to remove: https://git.mills.io/yarnsocial/go-lextwt/src/branch/main/ast.go#L776-L784
apparently i can’t make the edit via gitea.. i am guessing its hitting one of your firewall rules.
@<url>
. Submitting this writes @<domain url>
instead of @<nick url>
in the feed.
While I now have a somewhat working fix for it in yarnd (https://git.mills.io/yarnsocial/yarn/pulls/1232), I also have the feeling that I should fix literal formatting in lextwt as well. This also uncovered more bugs I believe: https://git.mills.io/yarnsocial/go-lextwt/pulls/28
But then there is also the question why the textarea is populated with @<url>
in the first place rather than @<nick url>
or yarnd’s own @nick@domain
/@nick
syntax. It indeed has to do something with whether I follow the mentioned feed or not.
Anyway, something to investigate for future Lyse or maybe @prologic@twtxt.net and/or @xuu@txt.sour.is. G’night!
I would like to make another proposal to the community, to discuss it calmly: https://git.mills.io/yarnsocial/twtxt.dev/issues/9 #twtxt
Amazing! My response: https://git.mills.io/yarnsocial/twtxt.dev/pulls/8#issuecomment-18500
Thanks, @andros@twtxt.andros.dev! I commented and replied here: https://git.mills.io/yarnsocial/twtxt.dev/pulls/8#issuecomment-18490
@xuu@txt.sour.is I added some logging when a “dead” peer is removed as I suspect this to be a hot candidate for all the trouble. https://git.mills.io/yarnsocial/yarn/commit/21538951f9dc71b9366db6dbb784a8078096a4c8 Does this yield anything?
code { white-space: pre }
in their CSS themes to render things as they're supposed to look like.
Well, I stand corrected, pre-wrap
even! https://git.mills.io/yarnsocial/yarn/pulls/1186
"twtxtfeevalidator/0.0.1"
UA about? I thought I could ask before throwing a 1000GB file at it 🪤 could it be the same 'xt' thing @lyse was talking about the other day?
I cobbled that together yesterday, @aelaraji@aelaraji.com. Since I was too lazy to write some tests, I simply hit your feed as I knew it contains two invalid lines right now. Sorry mate! :-( Next thing is to actually write some proper tests, improve the messages, etc.
Here’s the code: https://git.mills.io/yarnsocial/validator
Looking forward to that, @prologic@twtxt.net. :-)
Is https://git.mills.io/yarnsocial dead?
I’ve started a draft over at: https://git.mills.io/yarnsocial/twtxt.dev/src/branch/main/exts/webfinger.md
BTW, researching on that I found that @prologic was working on the following https://git.mills.io/yarnsocial/useragent#supported-user-agent-formats
I wasn’t aware of a few User-Agent Formats
.
And I’m checking that timeline
is not sending those headers. I should push a change on that soon.
@eapl.me@eapl.me Neat.
So for twt metadata the lextwt parser currently supports values in the form [key=value]
https://git.mills.io/yarnsocial/go-lextwt/src/branch/main/parser_test.go#L692-L698
Righto, @eapl.me@eapl.me, ta for the writeup. Here we go. :-)
Metadata on individual twts are too much for me. I do like the simplicity of the current spec. But I understand where you’re coming from.
Numbering twts in a feed is basically the attempt of generating message IDs. It’s an interesting idea, but I reckon it is not even needed. I’d simply use location based addressing (feed URL + ‘#’ + timestamp) instead of content addressing. If one really wanted to, one could hash the feed URL and timestamp, but the raw form would actually improve disoverability and would not even require a richer client. But the majority of twtxt users in the last poll wanted to stick with content addressing.
yarnd actually sends If-Modified-Since
request headers. Not only can I observe heaps of 304 responses for yarnds in my access log, but in Cache.FetchFeeds(…)
we can actually see If-Modified-Since
being deployed when the feed has been retrieved with a Last-Modified
response header before: https://git.mills.io/yarnsocial/yarn/src/commit/98eee5124ae425deb825fb5f8788a0773ec5bdd0/internal/cache.go#L1278
Turns out etags with If-None-Match
are only supported when yarnd serves avatars (https://git.mills.io/yarnsocial/yarn/src/commit/98eee5124ae425deb825fb5f8788a0773ec5bdd0/internal/handlers.go#L158) and media uploads (https://git.mills.io/yarnsocial/yarn/src/commit/98eee5124ae425deb825fb5f8788a0773ec5bdd0/internal/media_handlers.go#L71). However, it ignores possible etags when fetching feeds.
I don’t understand how the discovery URLs should work to replace the User-Agent
header in HTTP(S) requests. Do you mind to elaborate?
Different protocols are basically just a client thing.
I reckon it’s best to just avoid mixing several languages in one feed in the first place. Personally, I find it okay to occasionally write messages in other languages, but if that happens on a more regularly basis, I’d definitely create a different feed for other languages.
Isn’t the emoji thing “just” a client feature? So, feed do not even have to state any emojis. As a user I’d configure my client to use a certain symbol for feed ABC. Currently, I can do a similar thing in tt
where I assign colors to feeds. On the other hand, what if a user wants to control what symbol should be displayed, similar to the feed’s nick? Hmm. But still, my terminal font doesn’t even render most of emojis. So, Unicode boxes everywhere. This makes me think it should actually be a only client feature.
👋 PR to propose Feed Format Extension – Request for comment 🙏
Official yarn.social tool: git.mills.io/yarnsocial/useragent
Alright, before I go and watch Formula 1 😅, I made two PRs regarding the two “competing” ideas:
- https://git.mills.io/yarnsocial/yarn/pulls/1179 –
(replyto:…)
- https://git.mills.io/yarnsocial/yarn/pulls/1180 –
(edit:…)
and(delete:…)
As a first step, this summarizes my current understanding. Please comment! 😊
Can I get someone like maybe @xuu@txt.sour.is or @abucci@anthony.buc.ci or even @eldersnake@we.loveprivacy.club – If you have some spare time – to test this yarnd
PR that upgrades the Bitcask dependency for its internal database to v2? 🙏
VERY IMPORTANT If you do; Please Please Please backup your yarn.db
database first! 😅 Heaven knows I don’t want to be responsible for fucking up a production database here or there 🤣
@falsifian@www.falsifian.org yes, that happened around 2 years ago, on commit 5923078ea5.
Opened a couple of issues on twtxt2html. Maybe @prologic@twtxt.net will get to them after he has completed his luxurious recharging cycle. LOL.
@aelaraji@aelaraji.com I just added support for passing a custom template file via -T/--template
in case you need a custom template 👌
prologic@JamessMacStudio
Wed Sep 18 01:27:29
~/Projects/yarnsocial/twtxt2html
(main) 130
$ ./twtxt2html --help
Usage: twtxt2html [options] FILE|URL
twtxt2html converts a twtxt feed to a static HTML page
-d, --debug enable debug logging
-l, --limit int limit number ot twts (default all) (default -1)
-n, --noreldate do now show twt relative dates
-r, --reverse reverse the order of twts (oldest first)
-T, --template string path to template file
-t, --title string title of generated page (default "Twtxt Feed")
-v, --version display version information
pflag: help requested
@bender@twtxt.net It’s just a simple twtxt2html and scp … it goes like:
twtxt2html $HOME/path/to/local_twtxt_dir/twtxt.txt > $HOME/path/to/local_twtxt_dir/log.html && \
scp $HOME/path/to/local_twtxt_dir/log.html user@remotehost:/path/to/static_files_dir/
I’ve been lazy to add it to my publish_command script, now I can just copy/pasta from the twt 😅
159-196-9-199.9fc409.mel.nbn.aussiebb.net
I’m wrong! Both 404 and 410, among others, are considered dead feeds: https://git.mills.io/yarnsocial/yarn/src/branch/main/internal/cache.go#L1343 Whatever that actually means.
@stigatle@yarn.stigatle.no / @abucci@anthony.buc.ci My current working theory is that there is an asshole out there that has a feed that both your pods are fetching with a multi-GB avatar URL advertised in their feed’s preamble (metadata). I’d love for you both to review this PR, and once merged, re-roll your pods and dump your respective caches and share with me using https://gist.mills.io/
Twtxt spec enhancement proposal thread 🧵
Adding attributes to individual twts similar to adding feed attributes in the heading comments.
https://git.mills.io/yarnsocial/go-lextwt/pulls/17
The basic use case would be for multilingual feeds where there is a default language and some twts will be written a different language.
As seen in the wild: https://eapl.mx/twtxt.txt
The attributes are formatted as [key=value]
They can show up in the twt anywhere it is not enclosed by another element such as codeblock
or part of a markdown link.
@darch@neotxt.dk webmentions are dispatched from here https://git.mills.io/yarnsocial/yarn/src/branch/main/internal/post_handler.go#L160-L169
$name$
and then dispatch the hashing or checking to its specific format.
I have submitted this to be used as the hash tooling for Yarn. See it as a good example on using this in a production environment!