This code displays the last 10 lines of a twtxt feed without a full dowload.

FEED_URL="https://twtxt.net/user/prologic/twtxt.txt"
MAX_RANGE=$(curl -sI $FEED_URL | grep -i 'content-length' | awk '{print $2}' | tr -d '\r')
MIN_RANGE=$((MAX_RANGE - 5000))

curl -s --range "$MIN_RANGE-$MAX_RANGE" "$FEED_URL" | grep -v -e '^#' -e '^$' | head -n 10

My self-response!

@prologic@twtxt.net @david@collantes.us

#twtxt

Writing C for Curl
Comments ⌘ Read more

A special build of curl that can impersonate Chrome and Firefox
Comments ⌘ Read more

HTTPS RR in Curl
Comments ⌘ Read more

@andros@twtxt.andros.dev Can you reproduce any of this outside of your client? I can’t spot a mistake here:

$ curl -sI 'http://movq.de/v/8684c7d264/.html%2Dindex%2Dthumb%2Dgimp11%2D1.png.jpg'
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 2615
Content-Type: image/jpeg
Date: Wed, 19 Mar 2025 19:53:17 GMT
Last-Modified: Wed, 19 Mar 2025 17:34:08 GMT
Server: OpenBSD httpd

$ curl -sI 'https://movq.de/v/8684c7d264/gimp11%2D1.png'
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 131798
Content-Type: image/png
Date: Wed, 19 Mar 2025 19:53:19 GMT
Last-Modified: Wed, 19 Mar 2025 17:18:07 GMT
Server: OpenBSD httpd

$ telnet movq.de 80
Trying 185.162.249.140...
Connected to movq.de.
Escape character is '^]'.
HEAD /v/8684c7d264/.html%2Dindex%2Dthumb%2Dgimp11%2D1.png.jpg HTTP/1.1
Host: movq.de
Connection: close

HTTP/1.1 200 OK
Connection: close
Content-Length: 2615
Content-Type: image/jpeg
Date: Wed, 19 Mar 2025 19:53:31 GMT
Last-Modified: Wed, 19 Mar 2025 17:34:08 GMT
Server: OpenBSD httpd

Connection closed by foreign host.
$ 

Porting the curl command-line tool and library with Goa
For more than a decade, we have a port of the curl library for Genode available. With the use of Sculpt OS as a daily driver as well as the plan to run Goa natively on Sculpt OS by the end of the year, the itch to also port the curl command-line tool became irresistible. Of course this is a perfect territory for using Goa. In this article, I will share the process of porting the curl command-line tool and shared library … ⌘ Read more

working on my bookmarks tool, I found out that http(s)://domain.tls is not a valid resource, but http(s)://domain.tls/ is, as you can see here: https://stackoverflow.com/a/2581423

I suppose that internally the wget/curl or whatever client you are using is redirecting it?

Je recherche un outil qui me permettrait d’obtenir des snapshots de page web, un peu comme le fait archive.org. J’ai le sentiment et l’envie d’archiver pour la postérité, avant la disparition inopinée de ressources de valeur. Vous auriez des conseils? script avec #curl? Truc en #auto-hébergement?

@kat@yarn.girlonthemoon.xyz Something is broken with the TLS:

$ curl https://remix.girlonthemoon.xyz
curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error

Image

curl: (3) URL rejected: Malformed input to a URL function. Writing sender in bash was BAD idea

@kat@yarn.girlonthemoon.xyz both scripts are here under the names ‘getlyr’ and ‘now playing’ if you wanna try them out yourself, just make sure you have gum installed (also curl and jq but most people have those i think) https://git.sr.ht/~chasinglightning/dotfiles/tree/main/item/home/.local/bin

@kat@yarn.girlonthemoon.xyz i’ve really wanted to make one of those sites you can curl that’s terminal friendly but looks different on the browser like how does wttr.in do it… magic

@doesnm@doesnm.p.psf.lt I sent that mention manually for a demonstration as mentioned in the previous twt. Used the curl method.

@prologic@twtxt.net sure! I don’t know if this is what you need but, let me give it a try.

• I have Timeline installed, which has an endpoint to process #webmentions. Mine for example is https://aelaraji.com/timeline/webmention which you can find by querying https://aelaraji.com/.well-known/webfinger.
  
• If you mention someone from #Timeline itself, it takes care of querying that and sending in the mention for you.
  
• Otherwise (what I personally do) you could just:
  

curl -i -d 'source=https://twtxt.net/user/prologic/twtxt.txt#:~:text=2024-12-09T01:22:37Z' -d 'target=https://aelaraji.com/twtxt.txt' https://aelaraji.com/timeline/webmention

basically what @sorenpeter@darch.dk mentioned in his article Here.

Afterwards, the mentions are stored in their own mentions.txt feed. The one from the example above looks like this on my Timeline :

Image

Feel free to spam my endpoint if you’d like to give things a try. 👍

[P.S: personally, I don’t seem to get the mentions if I add the Text fragment part to my target]

@bender@twtxt.net highly probably, unless I learn go and implement it myself (or someone else more capable does) … but I’m so lazy I’d just copy them from twtxt.net and call it a day xD and yeah, it’s kinda rough the way things are…

• I don’t see a way to follow others, all I can do is go to the /feeds URI for a list of the server’s users/feeds.
  
• I still couldn’t figure out how to get a direct link to a user’s twtxt file, curling /feeds/usernick spits out a list of the user usernick twts, so I guess you could use that to follow them.
  
• no way to add in your # nick = usernick / # url = proto://domain.ltd/path/to/twtxt.txt …etc. Probably because that wasn’t part of the spec back then?
  

So yeah, it would make for a nice project while learning Go. :P

@prologic@twtxt.net Perfect, thanks. For my own future reference: curl -H ‘Accept: application/json’ https://twtxt.net/twt/st3wsda

@prologic@twtxt.net My pod, which is running the same commit you are, does not return an error like that. It returns the same HTML it always has. Try it. I nuked my cache before restarting.

Edit: Oh wait, the plot thickens. I do get an error if I use curl or if I use a web browser that isn’t logged in. That’s good!

@prologic@twtxt.net This does not seem to fix the problem for me, or I’ve done something wrong. I did the following:

1. Pull the latest version from git (I have commit 7ad848, same as on twtxt.net I believe).
   
2. make build and make install
   
3. Restart yarnd
   
4. Refresh cache in Poderator Settings
   

Yet I still see these bogus /external things on my pod when I hit URLs like the one I sent you recently. When I hit such a URL with curl I think it’s giving an error? But in a web browser, the (buggy) response is the same as it was before I updated.

So, this problem is not fixed for me.

installing lix fixed my busted nix install. i installed the rpm from their website and it didn’t work so i typed curl -sSf -L https://install.lix.systems/lix and was happy happy happy

Pretty cool how one can post to the Internet with a single curl command.

will have to implement some curl scanning for follows and mentions. gona be a nice chance to brush up my C-string-fu LoL

@prologic@twtxt.net

Video

#!/bin/sh

# Validate environment
if ! command -v msgbus > /dev/null; then
    printf "missing msgbus command. Use:  go install git.mills.io/prologic/msgbus/cmd/msgbus@latest"
    exit 1
fi

if ! command -v salty > /dev/null; then
    printf "missing salty command. Use:  go install go.mills.io/salty/cmd/salty@latest"
    exit 1
fi

if ! command -v salty-keygen > /dev/null; then
    printf "missing salty-keygen command. Use:  go install go.mills.io/salty/cmd/salty-keygen@latest"
    exit 1
fi

if [ -z "$SALTY_IDENTITY" ]; then
    export SALTY_IDENTITY="$HOME/.config/salty/$USER.key"
fi

get_user () {
    user=$(grep user: "$SALTY_IDENTITY" | awk '{print $3}')
    if [ -z "$user" ]; then
        user="$USER"
    fi
    echo "$user"
}

stream () {
    if [ -z "$SALTY_IDENTITY" ]; then
        echo "SALTY_IDENTITY not set"
        exit 2
    fi

    jq -r '.payload' | base64 -d | salty -i "$SALTY_IDENTITY" -d
}

lookup () {
    if [ $# -lt 1 ]; then
    printf "Usage: %s nick@domain\n" "$(basename "$0")"
    exit 1
    fi

    user="$1"
    nick="$(echo "$user" | awk -F@ '{ print $1 }')"
    domain="$(echo "$user" | awk -F@ '{ print $2 }')"

    curl -qsSL "https://$domain/.well-known/salty/${nick}.json"
}

readmsgs () {
    topic="$1"

    if [ -z "$topic" ]; then
        topic=$(get_user)
    fi

    export SALTY_IDENTITY="$HOME/.config/salty/$topic.key"
    if [ ! -f "$SALTY_IDENTITY" ]; then
        echo "identity file missing for user $topic" >&2
        exit 1
    fi

    msgbus sub "$topic" "$0"
}

sendmsg () {
    if [ $# -lt 2 ]; then
        printf "Usage: %s nick@domain.tld <message>\n" "$(basename "$0")"
        exit 0
    fi

    if [ -z "$SALTY_IDENTITY" ]; then
        echo "SALTY_IDENTITY not set"
        exit 2
    fi

    user="$1"
    message="$2"

    salty_json="$(mktemp /tmp/salty.XXXXXX)"

    lookup "$user" > "$salty_json"

    endpoint="$(jq -r '.endpoint' < "$salty_json")"
    topic="$(jq -r '.topic' < "$salty_json")"
    key="$(jq -r '.key' < "$salty_json")"

    rm "$salty_json"

    message="[$(date +%FT%TZ)] <$(get_user)> $message"

    echo "$message" \
        | salty -i "$SALTY_IDENTITY" -r "$key" \
        | msgbus -u "$endpoint" pub "$topic"
}

make_user () {
    mkdir -p "$HOME/.config/salty"

    if [ $# -lt 1 ]; then
        user=$USER
    else
        user=$1
    fi

    identity_file="$HOME/.config/salty/$user.key"

    if [ -f "$identity_file" ]; then
        printf "user key exists!"
        exit 1
    fi

    # Check for msgbus env.. probably can make it fallback to looking for a config file?
    if [ -z "$MSGBUS_URI" ]; then
        printf "missing MSGBUS_URI in environment"
        exit 1
    fi


    salty-keygen -o "$identity_file"
    echo "# user: $user" >> "$identity_file"

    pubkey=$(grep key: "$identity_file" | awk '{print $4}')

    cat <<- EOF
Create this file in your webserver well-known folder. https://hostname.tld/.well-known/salty/$user.json

{
  "endpoint": "$MSGBUS_URI",
  "topic": "$user",
  "key": "$pubkey"
}

EOF
}

# check if streaming
if [ ! -t 1 ]; then
    stream
    exit 0
fi

# Show Help
if [ $# -lt 1 ]; then
    printf "Commands: send read lookup"
    exit 0
fi


CMD=$1
shift

case $CMD in
    send)
        sendmsg "$@"
    ;;
    read)
        readmsgs "$@"
    ;;
    lookup)
        lookup "$@"
    ;;
    make-user)
        make_user "$@"
    ;;
esac