@movq@www.uninformativ.de Yes, exactly that. It’s awful! And it’s getting worse from my perspective. Nobody in charge is ever gonna learn anything. I figure we just fully deserve this M$ crap, every single bit. :-(

Luckily, the most important development platform still worked for me, so I could actually do something, review code, pull and push, etc. But the calls with the screenshares were nightmares. Can’t see shit on such a tiny display with today’s extreme monitor sizes people use. Looking at logs, hahahahahahaaa…

I mean bug where jenny don’t know about these id’s and tried to request from twtxt.net (prologic sent access logs)

hmmm i really should set up crowdsec and maybe a WAF like coraza or something. i don’t look at my logs as much as i should because they scare me and ignorance is bliss but i should probably cut out as much false traffic as possible especially to my biggest site (superlove)

Nouvel article pour les anxieux comme moi : Sac d’urgence. https://si3t.ch/log/2025-01-06-sac-d-urgence.txt

How in da fuq do you actually make these fucking useless AI bots go way?

proxy-1:~# jq '. | select(.request.remote_ip=="4.227.36.76")' /var/log/caddy/access/mills.io.log | jq -s '. | last' | caddy-log-formatter -
4.227.36.76 - [2025-01-05 04:05:43.971 +0000] "GET /external?aff-QNAXWV=&f=mediaonly&f=noreplies&nick=g1n&uri=https%3A%2F%2Fmy-hero-ultra-impact-codes.linegames.org HTTP/2.0" 0 0
proxy-1:~# date
Sun Jan  5 04:05:49 UTC 2025

😱

Once again I glimpsed at my twtxt feed access log. Now I’m wondering: is there a twtxt client named xt out there? Does anyone know? I did not find anything for “xt/0.0.1”.

fair lol! i should give the web app a try, i don’t think i’ll get much use out of it from my phone anyway because i suck at typing on a phone but i might as well log in!

Skew-T Log-P
⌘ Read more

@emmanuel@wald.ovh It is working! I’ve just noticed your feed link in my access.log and came by to say Hello! 👋 just give it a minute and others will notice your feed as well.

@sorenpeter@darch.dk @bender@twtxt.net @prologic@twtxt.net Right. Also, generally speaking, if you come across a new feed URL, it’s probably either via some mention in another feed or the User-Agent in your access log. Both cases typically advertise also a display name. So, you just reuse whatever you’ve seen there.

Ich habe mir einen twtxt-Log-Parser geschrieben, der mich per Mail informiert, wenn jemand “neues” auf meinen Feed zugreift. Mal gucken, wie es läuft.

Philosophie de vie, après réflexion : https://si3t.ch/log/2024-12-16-philovie.txt

Fuck me dead, what a giant piece of shit. On my Linux work laptop I have the problem that some unknown snakeoil “security” junk is dropping any IPv4 connections to ports 80 and 443. All other ports and IPv6 seem unaffected. I get an immediate “connection refused” when trying to estabslish a connection.

I had this problem four weeks ago on Friday morning the very first time at home. On Thursday evening, everything was perfectly fine. Eventually, I plugged in the LAN cable in the office and everything got automatically fixed. Nobody can explain what’s happening.

Then, last week Friday morning out of the blue, the same issue was back. So, I went to the office yesterday and it got fixed again by plugging in the network cable. This evening, I have exactly the same bloody problem again.

What the hell is going on? Does anyone have any ideas? I’m certainly not an expert, but I don’t see anything suspicious in iptables or nft rules. I also do not see anything showing up in /var/log/kern.log. Even tried to stop firewalld, flush the iptables and nft rules, but that didn’t result in any changes.

@prologic@twtxt.net Twtxt wise, it was kind of disparate at first xD with no access to logs as you may have read on the alt-feed itself. But then, @sorenpeter@darch.dk’s script came to the rescue … like, just in time 😁 Otherwise, everything else is fun as publicised, exploring and learning along the way.

@prologic@twtxt.net There’s always my log.html page if it helps… and while we’re still at it here’s the other twt #z2ymlkq for reference ;)

Ok! now that I’ve created the twtAgent.{log,csv} files manually and changed their perms so they’re writable by www-data, the twtAgent.php works the twtAgent2 doesn’t tho. + Still have no redirections, I might have to ask arround.

You were mentioned in: https://thunix.net/~skinshafi/twtxt.txt#:~:text=2024-12-02T04:56-07:00

(#6xuvs6a) @sorenpeter@darch.dk I was hoping your script could help solve our pubnix users’ Discovrability dilemma, but either I’m doing something wrong or (maybe) we have some PHP module(s) missing. I gave it a try and it returns the twtxt file content as expected when visited directly, yet there’s no logs/log files created. + I’m still not sure why my htaccess redirection doesn’t work x’) … Will keep you posted.

@sorenpeter@darch.dk I was hoping your script could help solve our pubnix users’ Discovrability dilemma, but either I’m doing something wrong or (maybe) we have some PHP module(s) missing. I gave it a try and it returns the twtxt file content as expected when visited directly, yet there’s no logs/log files created. + I’m still not sure why my htaccess redirection doesn’t work x’) … Will keep you posted.

So, I don’t think there is a way for pubnix users to check who’s pulling their Twtxt feeds. “No access logs (for obvious reasons) so, no visibility on user-agent anouncements for me and no discovrability for you (actually, me)!” xD

Yes it work: 2024-12-01T19:38:35Z twtxt/1.2.3 (+https://eapl.mx/twtxt.txt; @eapl) :D

The .log is just a simple append each request. The idea with the .cvs is to have it tally up how many request there have been from each client as a way to avoid having the log file grow too big. And that you can open the .cvs as a spreadsheet and have an easy overview and filtering options.

Access to those files are closed to the public.

did it work? I tried loading the file from /twtxt.txt, /twtAgent.php and /twtAgent2.php

Also reading the code, I see it creates two files, but I’m not sure which you are using now:
http://darch.dk/twtAgent.log <- 403 Forbidden
and
http://darch.dk/twtAgent.csv <- Not found here

I sent a few local tests, could you help me check if you see https://eapl.mx/twtxt.txt in your log file?

If you want more eyes on the agent, you can tell tell me, as I was thinking on something similar to catch those ‘following requests’.

@eapl.mx@eapl.mx Yes, the idea is to add User Agent support to #Timeline.
Right now it just adds every request to a growing log file, but I have also been working on a way to analyse it, so it only saves the time of the latest request.
I’m not sure how to make it part of timeline itself, since it requeses that you redirect/rewrite from twtAgent.php to the acctual twtxt.txt
Help with making Timeline send proper User Agents to others would be much appreciated:)

Block AI bots with relayd on #OpenBSD https://si3t.ch/log/2024-11-27-block-ai-bots-with-relayd-openbsd.txt

Nouvel article où je parle livres, aprentissage et workflow : http://si3t.ch/log/2024-11-26-resister.txt

Notes suite aux Utopiales 2040 #utopiales #utopiales2024 https://si3t.ch/log/2024-11-16-notes-utopiales-2024.txt

@bender@twtxt.net You’d be surprised how many lifetime old accounts I still have, scattered all over the internet. I just don’t have much energy to go through deleting each and every single one of them. xD and here is a bonus image for LOLs

@eapl.me@eapl.me here are my replies (somewhat similar to Lyse’s and James’)

1. Metadata in twts: Key=value is too complicated for non-hackers and hard to write by hand. So if there is a need then we should just use #NSFS or the alt-text file in markdown image syntax ![NSFW](url.to/image.jpg) if something is NSFW

2. IDs besides datetime. When you edit a twt then you should preserve the datetime if location-based addressing should have any advantages over content-based addressing. If you change the timestamp the its a new post. Just like any other blog cms.

3. Caching, Yes all good ideas, but that is more a task for the clients not the serving of the twtxt.txt files.

4. Discovery: User-agent for discovery can become better. I’m working on a wrapper script in PHP, so you don’t need to go to Apaches log-files to see who fetches your feed. But for other Gemini and gopher you need to relay on something else. That could be using my webmentions for twtxt suggestion, or simply defining an email metadata field for letting a person know you follow their feed. Interesting read about why WebMetions might be a bad idea. Twtxt being much simple that a full featured IndieWeb sites, then a lot of the concerns does not apply here. But that’s the issue with any open inbox. This is hard to solve without some form of (centralized or community) spam moderation.

5. Support more protocols besides http/s. Yes why not, if we can make clients that merge or diffident between the same feed server by multiples URLs

6. Languages: If the need is big then make a separate feed. I don’t mind seeing stuff in other langues as it is low. You got translating tool if you need to know whats going on. And again when there is a need for easier switching between posting to several feeds, then it’s about building clients with a UI that makes it easy. No something that should takes up space in the format/protocol.

7. Emojis: I’m not sure what this is about. Do you want to use emojis as avatar in CLI clients or it just about rendering emojis?

Righto, @eapl.me@eapl.me, ta for the writeup. Here we go. :-)

Metadata on individual twts are too much for me. I do like the simplicity of the current spec. But I understand where you’re coming from.

Numbering twts in a feed is basically the attempt of generating message IDs. It’s an interesting idea, but I reckon it is not even needed. I’d simply use location based addressing (feed URL + ‘#’ + timestamp) instead of content addressing. If one really wanted to, one could hash the feed URL and timestamp, but the raw form would actually improve disoverability and would not even require a richer client. But the majority of twtxt users in the last poll wanted to stick with content addressing.

yarnd actually sends If-Modified-Since request headers. Not only can I observe heaps of 304 responses for yarnds in my access log, but in Cache.FetchFeeds(…) we can actually see If-Modified-Since being deployed when the feed has been retrieved with a Last-Modified response header before: https://git.mills.io/yarnsocial/yarn/src/commit/98eee5124ae425deb825fb5f8788a0773ec5bdd0/internal/cache.go#L1278

Turns out etags with If-None-Match are only supported when yarnd serves avatars (https://git.mills.io/yarnsocial/yarn/src/commit/98eee5124ae425deb825fb5f8788a0773ec5bdd0/internal/handlers.go#L158) and media uploads (https://git.mills.io/yarnsocial/yarn/src/commit/98eee5124ae425deb825fb5f8788a0773ec5bdd0/internal/media_handlers.go#L71). However, it ignores possible etags when fetching feeds.

I don’t understand how the discovery URLs should work to replace the User-Agent header in HTTP(S) requests. Do you mind to elaborate?

Different protocols are basically just a client thing.

I reckon it’s best to just avoid mixing several languages in one feed in the first place. Personally, I find it okay to occasionally write messages in other languages, but if that happens on a more regularly basis, I’d definitely create a different feed for other languages.

Isn’t the emoji thing “just” a client feature? So, feed do not even have to state any emojis. As a user I’d configure my client to use a certain symbol for feed ABC. Currently, I can do a similar thing in tt where I assign colors to feeds. On the other hand, what if a user wants to control what symbol should be displayed, similar to the feed’s nick? Hmm. But still, my terminal font doesn’t even render most of emojis. So, Unicode boxes everywhere. This makes me think it should actually be a only client feature.

plus d’emails, si3t.ch down: https://mirror.si3t.ch/log/2024-11-04-plus-d-email-si3tch-down.txt

Been curious to see if can filter out my access.log file and output a list of my twtxt followers just in case I’ve missed someone … I came up with this awk -F '\"' '/twtxt/ {print $(NF-1)}' /var/log/user.log | grep -v 'twtxt\.net' | sort -u | awk '{print $(NF-1) $NF}' | awk '/^\(/' spaghetti monster of a command and I’m wondering if there’s a more elegant way for achieving the same thing.

@codebuzz@www.codebuzz.nl I have some shell scripts that handle some of the log formatting details, but I mostly write my mesages by hand. Lately I’ve been browsing twtxt.net since they aggregate most of the known network. I have a couple of demo aggregators sitting around, but I’m in the middle of some infra rebuilds so a lot of my services are offline rn. They’re both built on a simple social graph analysis that extracts urls for your direct follows the follows listed on each of those feeds (friend-of-a-friend replication). certain formatting operations are awkward with my setup, so I may write an app of some kind in the future. likely gemini-based, but I have a number of projects ahead of that one in the queue.

I installed GrapheneOS for the first time on Wednesday last week on a used Pixel 7a, and I’m impressed. Installation was almost seamless, and I was able to do it from another Android phone. I’ve run into very few wrinkles, even using Google’s proprietary apps with GrapheneOS’s “sandboxed” version of Google Play Services. The main problems I’ve noticed: I can’t cast, and Google Timeline doesn’t seem to work (though I imagine the intersection between people keen to use GrapheneOS and keen to have Google log their location history is pretty small).

FINALLY!! I figured my way around daemonizing #saltyd with an rc.d init script with logs and all! 🥳 still have to try a tiny modification before writing that thing… I hope it doesn’t Bork it 😅

Des favoris Nextcloud à twtxt et dmenu | https://galusik.fr/log/2021-12-12-dtwtbmk.html

Been curious about how people on Pubnix instances do manage their feed, if they have access to log? Sent in a req to join one still no res.

this log can contain ips so im place it in secret path and send link via salty

@doesnm@doesnm.p.psf.lt Do you have a sample Caddy log file you can supply? I’ll see if we can improve the tool 👌

@doesnm@doesnm.p.psf.lt Fot a sample access log? Which tool are you using?

how to parse caddy access log with useragent tool? seems it dont detect anything in json

@prologic@twtxt.net I saw those, yes. I tried using yarnc, and it would work for a simple twtxt. Now, for a more convoluted one it truly becomes a nightmare using that tool for the job. I know there are talks about changing this hash, so this might be a moot point right now, but it would be nice to have a tool that:

1. Would calculate the hash of a twtxt in a file.
   
2. Would calculate all hashes on a twtxt.txt (local and remote).
   

Again, something lovely to have after any looming changes occur.

@quark@ferengi.one Mine is a little overkill 😂 but I need to do something for practice:

#!/bin/bash
set -e
trap 'echo "!! Something went wrong...!!"' ERR

#============= Variables ==========#

# Source files
LOCAL_DIR=$HOME/twtxt

TWTXT=$LOCAL_DIR/twtxt.txt
HTML=$LOCAL_DIR/log.html
TEMPLATE=$LOCAL_DIR/template.tmpl

# Destination
REMOTE_HOST=remotHostName     # Host already setup in ~/.ssh/config

WEB_DIR="path/to/html/content"
GOPHER_DIR="path/to/phlog/content"
GEMINI_DIR="path/to/gemini-capsule/content"

DIST_DIRS=("$WEB_DIR" "$GOPHER_DIR" "$GEMINI_DIR")


#============ Functions ===========#

# Building log.html:

build_page() {
	twtxt2html -T $TEMPLATE $TWTXT > $HTML
}

# Bulk Copy files to their destinations:

copy_files() {
	for DIR in "${DIST_DIRS[@]}"; do
    # Copy both `txt` and `html` files to the Web server and only `txt`
    # to gemini and gopher server content folders
		if [ "$DIR" == "$WEB_DIR" ]; then
			scp -C "$TWTXT" "$HTML" "$REMOTE_HOST:$DIR/"
		else
			scp -C "$TWTXT" "$REMOTE_HOST:$DIR/"
		fi
	done
}

#========== Call to functions ===========$

build_page && copy_files

@aelaraji@aelaraji.com I just added support for passing a custom template file via -T/--template in case you need a custom template 👌

prologic@JamessMacStudio
Wed Sep 18 01:27:29
~/Projects/yarnsocial/twtxt2html
 (main) 130
$ ./twtxt2html --help
Usage: twtxt2html [options] FILE|URL

twtxt2html converts a twtxt feed to a static HTML page
  -d, --debug             enable debug logging
  -l, --limit int         limit number ot twts (default all) (default -1)
  -n, --noreldate         do now show twt relative dates
  -r, --reverse           reverse the order of twts (oldest first)
  -T, --template string   path to template file
  -t, --title string      title of generated page (default "Twtxt Feed")
  -v, --version           display version information
pflag: help requested

@bender@twtxt.net It’s just a simple twtxt2html and scp … it goes like:

twtxt2html $HOME/path/to/local_twtxt_dir/twtxt.txt > $HOME/path/to/local_twtxt_dir/log.html && \
scp $HOME/path/to/local_twtxt_dir/log.html user@remotehost:/path/to/static_files_dir/

I’ve been lazy to add it to my publish_command script, now I can just copy/pasta from the twt 😅

Message to the void : présentation. https://si3t.ch/log/2024-09-17-message-to-the-void.txt

Thank you @aelaraji@aelaraji.com, I’m glad you like it. I use PHP because it’s everywhere on cheap hosting and no need for the user to log into a terminal to setup it up. Timeline is not mean to be use locally. For that I think something like twtxt2html is a better fit. (and happy to see you using simple.css on you new log page;)

@prologic@twtxt.net 🤯 HOLLY! … I’m definitely adding this to my Jenny’s publish_command script!! THANK YOU! Now my website has TWO pages instead of just a boring one 😂

New: Log Page

@movq@www.uninformativ.de Damn! I’m two years late to the discussion 😅 So basically, one could just make a bash script/cron job on the side for pinging non-Http feeds from time to time and the receiving end would get it IF they check their logs.

So this is a great thread. I have been thinking about this too.. and what if we are coming at it from the wrong direction? Identity being tied to a given URL has always been a pain point. If i get a new URL its almost as if i have a new identity because not only am I serving at a new location but all my previous communications are broken because the hashes are all wrong.

What if instead we used this idea of signatures to thread the URLs together into one identity? We keep the URL to Hash in place. Changing that now is basically a no go. But we can create a signature chain that can link identities together. So if i move to a new URL i update the chain hosted by my primary identity to include the new URL. If i have an archived feed that the old URL is now dead, we can point to where it is now hosted and use the current convention of hashing based on the first url:

The signature chain can also be used to rotate to new keys over time. Just sign in a new key or revoke an old one. The prior signatures remain valid within the scope of time the signatures were made and the keys were active.

The signature file can be hosted anywhere as long as it can be fetched by a reasonable protocol. So say we could use a webfinger that directs to the signature file? you have an identity like frank@beans.co that will discover a feed at some URL and a signature chain at another URL. Maybe even include the most recent signing key?

From there the client can auto discover old feeds to link them together into one complete timeline. And the signatures can validate that its all correct.

I like the idea of maybe putting the chain in the feed preamble and keeping the single self contained file.. but wonder if that would cause lots of clutter? The signature chain would be something like a log with what is changing (new key, revoke, add url) and a signature of the change + the previous signature.

# chain: ADDKEY kex14zwrx68cfkg28kjdstvcw4pslazwtgyeueqlg6z7y3f85h29crjsgfmu0w 
# sig: BEGIN SALTPACK SIGNED MESSAGE. ... 
# chain: ADDURL https://txt.sour.is/user/xuu
# sig: BEGIN SALTPACK SIGNED MESSAGE. ...
# chain: REVKEY kex14zwrx68cfkg28kjdstvcw4pslazwtgyeueqlg6z7y3f85h29crjsgfmu0w
# sig: ...

@movq@www.uninformativ.de @prologic@twtxt.net Hey! I may have found a silly trick to announce my following to people hosting their feeds on the Gemini space using the requested URI itself instead of relaying on the USER Agent 😂. I’ve copied my current feed over to my (to be) Gemlog for testing. And if I do a jenny -D "gemini://gem.aelaraji.com/twtxt.txt?follower=aelaraji@https://aelaraji.com/twtxt.txt" and this happens:

A) As a follower, I get the feed as usual.
B) As the feed owner, I get this in logs:

hostname:1965 - “gemini://gem.aelaraji.com/twtxt.txt?follower=aelaraji@https://aelaraji.com/twtxt.txt” 20 “text/plain;lang=en-US”

You could do the same for Gopher feeds but only if you want to announce yourself by throwing in an error in their logs, then you’ll need a second request to fetch the feed. jenny -D "gopher://gopher.aelaraji.com/twtxt.txt&follower=aelaraji@https:/aelaraji.com/twtxt.txt" gave me this :

gopher.aelaraji.com:70 - [09/Sep/2024:22:08:54 +0000] “GET 0/twtxt.txt&follower=aelaraji@https:/aelaraji.com/twtxt.txt HTTP/1.0” 404 0 “” “Unknown gopher client”

NB: the follower=... string won’t appear in gopher logs after a ? but if I replace it with a + or a & and it works. There will be a missing / after the https:. Probably a client thing.