Searching yarn

Twts matching #git
Sort by: Newest, Oldest, Most Relevant
lyse
lyse.isobeef.org
 (3y ago)

I played around with parsers. This time I experimented with parser combinators for twt message text tokenization. Basically, extract mentions, subjects, URLs, media and regular text. It’s kinda nice, although my solution is not completely elegant, I have to say. Especially my communication protocol between different steps for intermediate results is really ugly. Not sure about performance, I reckon a hand-written state machine parser would be quite a bit faster. I need to write a second parser and then benchmark them.

lexer.go and newparser.go resemble the parser combinators: https://git.isobeef.org/lyse/tt2/-/commit/4d481acad0213771fe5804917576388f51c340c0 It’s far from finished yet.

The first attempt in parser.go doesn’t work as my backtracking is not accounted for, I noticed only later, that I have to do that. With twt message texts there is no real error in parsing. Just regular text as a “fallback”. So it works a bit differently than parsing a real language. No error reporting required, except maybe for debugging. My goal was to port my Python code as closely as possible. But then the runes in the string gave me a bit of a headache, so I thought I just build myself a nice reader abstraction. When I noticed the missing backtracking, I then decided to give parser combinators a try instead of improving on my look ahead reader. It only later occurred to me, that I could have just used a rune slice instead of a string. With that, porting the Python code should have been straightforward.

Yeah, all this doesn’t probably make sense, unless you look at the code. And even then, you have to learn the ropes a bit. Sorry for the noise. :-)

⤋ Read More
xuu
txt.sour.is
 (3y ago)
↳ In-reply-to » git-bug

Ah git-bug! Ive chatted with the creator when he was working on the graphql parts. Its working with git objects directly sorta like how git-repo does code reviews. Its a pretty neat idea for storing data along side the branches. I believe they don’t add a disconnected branch to avoid data getting corrupted by merging branches or something like that.

⤋ Read More
xuu
txt.sour.is
 (3y ago)
↳ In-reply-to » @movq yeah.. i rewrote it a few times because i thought there was something breaking.. but was mistaken though now i am seeing a weird cache corruption.. that seems to come and go. Media

I have found the issue with this very subtle bug.. the cache was returning a slice that would be mutated. The mutation involved appending an item and then sorting. because the returned slice is just a pointer+length the sort would modify the same memory.

          CACHE         Returned slice          
original: [A B C D]     [A B C D]
add:      [A B C D] E   [A B C D E]
sort:     [E A B C] D   [A B C D E]

fix found here:
https://git.mills.io/yarnsocial/yarn/pulls/1072

⤋ Read More
lyse
lyse.isobeef.org
 (4y ago)
↳ In-reply-to » I did a take home software engineering test for a company recently, unfortunately I was really sick (have finally recovered) at the time 😢 I was also at the same time interviewing for an SRE position (as well as Software Engineering).

@prologic@twtxt.net Error handling especially in Go is very tricky I think. Even though the idea is simple, it’s fairly hard to actually implement and use in a meaningful way in my opinion. All this error wrapping or the lack of it and checking whether some specific error occurred is a mess. errors.As(…) just doesn’t feel natural. errors.Is(…) only just. I mainly avoided it. Yesterday evening I actually researched a bit about that and found this article on errors with Go 1.13. It shed a little bit of light, but I still have a long way to go, I reckon.

We tried several things but haven’t found the holy grail. Currently, we have a mix of different styles, but nothing feels really right. And having plenty of different approaches also doesn’t help, that’s right. I agree, error messages often end up getting wrapped way too much with useless information. We haven’t found a solution yet. We just noticed that it kind of depends on the exact circumstances, sometimes the caller should add more information, sometimes it’s better if the callee already includes what it was supposed to do.

To experiment and get a feel for yesterday’s research results I tried myself on the combined log parser and how to signal three different errors. I’m not happy with it. Any feedback is highly appreciated. The idea is to let the caller check (not implemented yet) whether a specific error occurred. That means I have to define some dedicated errors upfront (ErrInvalidFormat, ErrInvalidStatusCode, ErrInvalidSentBytes) that can be used in the err == ErrInvalidFormat or probably more correct errors.Is(err, ErrInvalidFormat) check at the caller.

All three errors define separate error categories and are created using errors.New(…). But for the invalid status code and invalid sent bytes cases I want to include more detail, the actual invalid number that is. Since these errors are already predefined, I cannot add this dynamic information to them. So I would need to wrap them à la fmt.Errorf("invalid sent bytes '%s': %w", sentBytes, ErrInvalidSentBytes"). Yet, the ErrInvalidSentBytes is wrapped and can be asserted later on using errors.Is(err, ErrInvalidSentBytes), but the big problem is that the message is repeated. I don’t want that!

Having a Python and Java background, exception hierarchies are a well understood concept I’m trying to use here. While typing this long message it occurs to me that this is probably the issue here. Anyways, I thought, I just create a ParseError type, that can hold a custom message and some causing error (one of the three ErrInvalid* above). The custom message is then returned at Error() and the wrapped cause will be matched in Is(…). I then just return a ParseError{fmt.Sprintf("invalid sent bytes '%s'", sentBytes), ErrInvalidSentBytes}, but that looks super weird.

I probably need to scrap the “parent error” ParseError and make all three “suberrors” three dedicated error types implementing Error() string methods where I create a useful error messages. Then the caller probably could just errors.Is(err, InvalidSentBytesError{}). But creating an instance of the InvalidSentBytesError type only to check for such an error category just does feel wrong to me. However, it might be the way to do this. I don’t know. To be tried. Opinions, anyone? Implementing a whole new type is some effort, that I want to avoid.

Alternatively just one ParseError containing an error kind enumeration for InvalidFormat and friends could be used. Also seen that pattern before. But that would then require the much more verbose var parseError ParseError; if errors.As(err, &parseError) && parseError.Kind == InvalidSentBytes { … } or something like that. Far from elegant in my eyes.

⤋ Read More
xuu
txt.sour.is
 (4y ago)

Hi, I am playing with making an event sourcing database. Its super alpha but I thought I would share since others are talking about databases and such.

It’s super basic. Using tidwall/wal as the disk backing. The first use case I am playing with is an implementation of msgbus. I can post events to it and read them back in reverse order.

I plan to expand it to handle other event sourcing type things like aggregates and projections.

Find it here: sour-is/ev

@prologic@twtxt.net @movq@www.uninformativ.de @lyse@lyse.isobeef.org

⤋ Read More
xuu
txt.sour.is
 (4y ago)

@prologic@twtxt.net 

#!/bin/sh

# Validate environment
if ! command -v msgbus > /dev/null; then
    printf "missing msgbus command. Use:  go install git.mills.io/prologic/msgbus/cmd/msgbus@latest"
    exit 1
fi

if ! command -v salty > /dev/null; then
    printf "missing salty command. Use:  go install go.mills.io/salty/cmd/salty@latest"
    exit 1
fi

if ! command -v salty-keygen > /dev/null; then
    printf "missing salty-keygen command. Use:  go install go.mills.io/salty/cmd/salty-keygen@latest"
    exit 1
fi

if [ -z "$SALTY_IDENTITY" ]; then
    export SALTY_IDENTITY="$HOME/.config/salty/$USER.key"
fi

get_user () {
    user=$(grep user: "$SALTY_IDENTITY" | awk '{print $3}')
    if [ -z "$user" ]; then
        user="$USER"
    fi
    echo "$user"
}

stream () {
    if [ -z "$SALTY_IDENTITY" ]; then
        echo "SALTY_IDENTITY not set"
        exit 2
    fi

    jq -r '.payload' | base64 -d | salty -i "$SALTY_IDENTITY" -d
}

lookup () {
    if [ $# -lt 1 ]; then
    printf "Usage: %s nick@domain\n" "$(basename "$0")"
    exit 1
    fi

    user="$1"
    nick="$(echo "$user" | awk -F@ '{ print $1 }')"
    domain="$(echo "$user" | awk -F@ '{ print $2 }')"

    curl -qsSL "https://$domain/.well-known/salty/${nick}.json"
}

readmsgs () {
    topic="$1"

    if [ -z "$topic" ]; then
        topic=$(get_user)
    fi

    export SALTY_IDENTITY="$HOME/.config/salty/$topic.key"
    if [ ! -f "$SALTY_IDENTITY" ]; then
        echo "identity file missing for user $topic" >&2
        exit 1
    fi

    msgbus sub "$topic" "$0"
}

sendmsg () {
    if [ $# -lt 2 ]; then
        printf "Usage: %s nick@domain.tld <message>\n" "$(basename "$0")"
        exit 0
    fi

    if [ -z "$SALTY_IDENTITY" ]; then
        echo "SALTY_IDENTITY not set"
        exit 2
    fi

    user="$1"
    message="$2"

    salty_json="$(mktemp /tmp/salty.XXXXXX)"

    lookup "$user" > "$salty_json"

    endpoint="$(jq -r '.endpoint' < "$salty_json")"
    topic="$(jq -r '.topic' < "$salty_json")"
    key="$(jq -r '.key' < "$salty_json")"

    rm "$salty_json"

    message="[$(date +%FT%TZ)] <$(get_user)> $message"

    echo "$message" \
        | salty -i "$SALTY_IDENTITY" -r "$key" \
        | msgbus -u "$endpoint" pub "$topic"
}

make_user () {
    mkdir -p "$HOME/.config/salty"

    if [ $# -lt 1 ]; then
        user=$USER
    else
        user=$1
    fi

    identity_file="$HOME/.config/salty/$user.key"

    if [ -f "$identity_file" ]; then
        printf "user key exists!"
        exit 1
    fi

    # Check for msgbus env.. probably can make it fallback to looking for a config file?
    if [ -z "$MSGBUS_URI" ]; then
        printf "missing MSGBUS_URI in environment"
        exit 1
    fi


    salty-keygen -o "$identity_file"
    echo "# user: $user" >> "$identity_file"

    pubkey=$(grep key: "$identity_file" | awk '{print $4}')

    cat <<- EOF
Create this file in your webserver well-known folder. https://hostname.tld/.well-known/salty/$user.json

{
  "endpoint": "$MSGBUS_URI",
  "topic": "$user",
  "key": "$pubkey"
}

EOF
}

# check if streaming
if [ ! -t 1 ]; then
    stream
    exit 0
fi

# Show Help
if [ $# -lt 1 ]; then
    printf "Commands: send read lookup"
    exit 0
fi


CMD=$1
shift

case $CMD in
    send)
        sendmsg "$@"
    ;;
    read)
        readmsgs "$@"
    ;;
    lookup)
        lookup "$@"
    ;;
    make-user)
        make_user "$@"
    ;;
esac

⤋ Read More
prologic
twtxt.net
 (4y ago)

gogrok/gogrok: A simple, easy to use ngrok alternative (self hosted!) - gogrok - Gitea: Git with a cup of tea

Hey @eldersnake@yarn.andrewjvpowell.com I just came across this cool little project recently. Not written by me sadly 😂 But seems like it would do the trick nonetheless 🤣 – How are you going with PageKite? Is it still working okay for your Yarn pod powered by the outback of down under? 😅 LMK if you’d like me to spin this up anad you can be my first tester 🤙

⤋ Read More
prologic
twtxt.net
 (4y ago)

👋 Q&A: Let’s discuss the removal of Editing and Deleting your last Twt. This is something @fastidious@twtxt.net has raised to me on IRC and something I find quite a valid approach to this. Over time I believe the utility and value of “Editing” and “Deleting” one’s last Twt isn’t as valuable as we’d like and increased complexity and introduces all kinds of side-effects that are hard to manage correctly. I vote for the removal of this feature from yarnd, the mobile app nor API support this anyway…

⤋ Read More
xuu
txt.sour.is
 (5y ago)
↳ In-reply-to » The original twt is unavailable. It may have been edited or deleted, or is from an unknown or muted feed.

@prologic@twtxt.net @jlj@twt.nfld.uk @movq@www.uninformativ.de

 /p/tmp > git clone https://www.uninformativ.de/git/lariza.git                                                                                                    Mon May 24 23:48:18 2021
Cloning into 'lariza'...
 /p/tmp > tree lariza/                                                                                                                                    12.5s  Mon May 24 23:48:32 2021
lariza/
├── BUGS
├── CHANGES
├── LICENSE
├── Makefile
├── PATCHES
├── README
├── browser.c
├── man1
│   ├── lariza.1
│   └── lariza.usage.1
├── user-scripts
│   └── hints.js
└── we_adblock.c

2 directories, 11 files

⤋ Read More