👋 Q: How do we feel about forking the Twtxt spec into what we love and use today in Yarn.social in yarnd, tt, jenny, twtr and other clients? 🤔 Thinking about (and talking with @xuu@txt.sour.is on IRC) about the possibility of rewriting a completely new spec (no extensions). Proposed name yarn.txt or “Yarn”. Compatibility would remain with Twtxt in the sense that we wouldn’t break anything per se, but we’d divorce ourselves from Twtxt and be free to improve based on the needs of the community and not the ideals of those that don’t use, contribute in the first place or fixate on nostalgia (which doesn’t really help anyone).

I’m not super a fan of using json. I feel we could still use text as the medium. Maybe a modified version to fix any weakness.

What if instead of signing each twt individually we generated a merkle tree using the twt hashes? Then a signature of the root hash. This would ensure the full stream of twts are intact with a minimal overhead. With the added bonus of helping clients identify missing twts when syncing/gossiping.

Have two endpoints. One as the webfinger to link profile details and avatar like you posted. And the signature for the merkleroot twt. And the other a pageable stream of twts. Or individual twts/merkle branch to incrementally access twt feeds.

💡 Quick ‘n Dirty prototype Yarn.social protocol/spec:

If we were to decide to write a new spec/protocol, what would it look like?

Here’s my rough draft (back of paper napkin idea):

• Feeds are JSON file(s) fetchable by standard HTTP clients over TLS
  
• WebFinger is used at the root of a user’s domain (or multi-user) lookup. e.g: prologic@mills.io -> https://yarn.mills.io/~prologic.json
  
• Feeds contain similar metadata that we’re familiar with: Nick, Avatar, Description, etc
  
• Feed items are signed with a ED25519 private key. That is all “posts” are cryptographically signed.
  
• Feed items continue to use content-addressing, but use the full Blake2b Base64 encoded hash.
  
• Edited feed items produce an “Edited” item so that clients can easily follow Edits.
  
• Deleted feed items produced a “Deleted” item so that clients can easily delete cached items.
  

#Yarn.social #Protocol #Ideas

julien040/gut: An easy-to-use git client for Windows, macOS, and Linux

@mckinley@twtxt.net i use pass along with the android and browser-pass clients. it is very good and keeping in sync is pretty simple.

Welcome back, @quark@ferengi.one! Your web server doesn’t send back a Last-Modified header for your feed, so the official twtxt client complains not to cache it. I just fixed that, so that tt shows your feed (of course no progress has been made in the meantime). And the Date header of your server seems to be quite funny, too. ;-)

Mmm, estoy teniendo problema con el encoding UTF-8, intuyo que los clientes no tendrán mucho problema, aunque el navegador Web si lo tiene. Quizás es algo de encabezados.

(cont.)

Just to give some context on some of the components around the code structure.. I wrote this up around an earlier version of aggregate code. This generic bit simplifies things by removing the need of the Crud functions for each aggregate.

A domain object can be used as an aggregate by adding the event.AggregateRoot struct and finish implementing event.Aggregate. The AggregateRoot implements logic for adding events after they are either Raised by a command or Appended by the eventstore Load or service ApplyFn methods. It also tracks the uncommitted events that are saved using the eventstore Save method.

type User struct {
  Identity string ```json:"identity"`

  CreatedAt time.Time

  event.AggregateRoot
}

// StreamID for the aggregate when stored or loaded from ES.
func (a *User) StreamID() string {
	return "user-" + a.Identity
}
// ApplyEvent to the aggregate state.
func (a *User) ApplyEvent(lis ...event.Event) {
	for _, e := range lis {
		switch e := e.(type) {
		case *UserCreated:
			a.Identity = e.Identity
			a.CreatedAt = e.EventMeta().CreatedDate
        /* ... */
		}
	}
}

Events are applied to the aggregate. They are defined by adding the event.Meta and implementing the getter/setters for event.Event

type UserCreated struct {
	eventMeta event.Meta

	Identity string
}

func (c *UserCreated) EventMeta() (m event.Meta) {
	if c != nil {
		m = c.eventMeta
	}
	return m
}
func (c *UserCreated) SetEventMeta(m event.Meta) {
	if c != nil {
		c.eventMeta = m
	}
}

With a domain object that implements the event.Aggregate the event store client can load events and apply them using the Load(ctx, agg) method.

// GetUser populates an user from event store.
func (rw *User) GetUser(ctx context.Context, userID string) (*domain.User, error) {
	user := &domain.User{Identity: userID}

	err := rw.es.Load(ctx, user)
	if err != nil {
		if err != nil {
			if errors.Is(err, eventstore.ErrStreamNotFound) {
				return user, ErrNotFound
			}
			return user, err
		}
		return nil, err
	}
	return user, err
}

An OnX command will validate the state of the domain object can have the command performed on it. If it can be applied it raises the event using event.Raise() Otherwise it returns an error.

// OnCreate raises an UserCreated event to create the user.
// Note: The handler will check that the user does not already exsist.
func (a *User) OnCreate(identity string) error {
    event.Raise(a, &UserCreated{Identity: identity})
    return nil
}

// OnScored will attempt to score a task.
// If the task is not in a Created state it will fail.
func (a *Task) OnScored(taskID string, score int64, attributes Attributes) error {
	if a.State != TaskStateCreated {
		return fmt.Errorf("task expected created, got %s", a.State)
	}
	event.Raise(a, &TaskScored{TaskID: taskID, Attributes: attributes, Score: score})
	return nil
}

The following functions in the aggregate service can be used to perform creation and updating of aggregates. The Update function will ensure the aggregate exists, where the Create is intended for non-existent aggregates. These can probably be combined into one function.

// Create is used when the stream does not yet exist.
func (rw *User) Create(
  ctx context.Context,
  identity string,
  fn func(*domain.User) error,
) (*domain.User, error) {
	session, err := rw.GetUser(ctx, identity)
	if err != nil && !errors.Is(err, ErrNotFound) {
		return nil, err
	}

	if err = fn(session); err != nil {
		return nil, err
	}

	_, err = rw.es.Save(ctx, session)

	return session, err
}

// Update is used when the stream already exists.
func (rw *User) Update(
  ctx context.Context,
  identity string,
  fn func(*domain.User) error,
) (*domain.User, error) {
	session, err := rw.GetUser(ctx, identity)
	if err != nil {
		return nil, err
	}

	if err = fn(session); err != nil {
		return nil, err
	}

	_, err = rw.es.Save(ctx, session)
	return session, err
}

I realized my twtxt client isn’t validating what it pulls once it gets a valid response when a domain started returning js-heavy parking pages for every URL. Oops. Weekend project, I guess. 🤦🏻

For instance I normally use the same RSA key/pair on all my workstations for my ssh client, because that’s me, no-matter where I am. The only exception to this rule is I usually create a separate key for any “work” / “ company” I am a part of.

@stackeffect@twtxt.stackeffect.de
I am seeing this characters on your twts: )?â\200¨â\200¨. Which client are you using?

@stigatle@twtxt.net
A twtxt client would be nice! Or a very simple cgi script to print twts to web nicely—not a second Yarn, just something to show twts in a pretty form on the web.

@eldersnake@yarn.andrewjvpowell.com
RSS links are archaic. Clients discover them if properly linked, they do not need to be human visible.

I am noticing that Yarn doesn’t treat “outside” (that is, twts coming from a client other than Yarn) twts hashes right. Two examples:

• https://twtxt.net/twt/mwtnrna
  
• https://twtxt.net/twt/fl5dooq
  

There are many more, but those two will give you the gist. Yarn links the hash to the poster’s twtxt.txt, so conversation matching will not work.

@lyse@lyse.isobeef.org Unless you are stripping stuff on your twts, there is no much to implement. Things will be bold , italics , underlined , and so on, on a client that can render them. Since jenny uses Mutt, I can use my own regex in it to color them as I like. That’s pretty much it.

someone ought to write an bit torrent client that does the http gateway/api and BEP46 things so ipfs can be taken down a peg

Fixed another bug in my finger client: rfc1288 says lines have to end with crlf, but I was just sending lf.

Indeed! I think the first “network protocol client” I ever wrote was something that just did the PING/PONG part and passed everything else raw.

Looking at raw IRC traffic streams to debug a client issue and it’s 1997 again.

Sure. I think search, if it’s going to exist, should be the client’s responsibility. But I also value the readability of the raw twtxt file a lot more than y’all do.

i’m most interested in systems that are meant to stay small enough for folks to go build their own clients and servers easily and without big teams.

of course, there are parts of doing any networked thing that require collaboration and that’s GOOD. it just isn’t often that projects encourage many alternate clients and server to be made.

i really need to make my own client so i can stop browsing random tw.txt pods

seems like the \u2028 approach breaks some clients, maybe i will try another way https://github.com/jointwt/twtxt/pull/166

I want read-only iOS client that just does the simplest model: pull a list of feeds, make a timeline.

@xjix@xj-ix.luxe Saw your oldish note about wanting an offline/async twtxt workflow. Do you have something that works for you? My (very young!) client was designed with that in mind.

I agree clients should present things better (part of why I’m writing one!). But that should be additive. There’s a reason we’re not passing json around.

@prologic@twtxt.net Exactly, but that reduces the argument for URLs in the post. The client should figure out how to search based on the hashtag.

My silly Plan 9 rc twtxt client now has a web page: http://txtpunk.com/tw/index.html

@prologic@twtxt.netYes, I think tags should just be #foo, and let the client figure out searching if it cares.

@movq@www.uninformativ.de No argument that threading is an improvement. But I think (#hash) does that, and I think figuring out how to search should mostly be up to the client.

Hah… my silly twtxt client now has “stories” mode.☺

@prologic@twtxt.net deedum for android.

Kristall for OS X

Elaho for iOS

more here

though I can only vouch for the first two.

I have a working model for the reader portion of what I want this twtxt client to do.

@prologic@twtxt.net

Can we not have clients sign their own public keys before listing them on their Pod’s account?

Yeah.. we probably could. when they setup an account they create a master key that signs any subsequent keys. or chain of signatures like keybase does.

@prologic@twtxt.net (#gqg3gea) ha yeah. COVID makes for a timey-wimey mish-mash. Worked on some WKD and fought with my XMPP client a bit.

@prologic@twtxt.net huh.. true.. the email is md5/sha256 before storing.. if twtxt acted as provider you would store that hash and point the SRV record to the pod. .. to act as a client it would need to store the hash and the server that hosts the image.

@kas@enotty.dk [re: gopher client] If you happen to be on Windows, then Gopher Browser for Windows by Matt Owen is pretty nice, otherwise I use Lynx indeed for gopher.

@von@tilde.town having topic-specific twtxt feeds is not a silly idea. Not sure if the clients allow easy switching though.

Made my own super basic twtxt client in 3 lines of code as a bashrc function. #l33t

Added clients and articles sections and added domgoergen’s twtxt.txt to https://indieweb.org/twtxt

// todo Create a Kaios client for twtxt

@kas@enotty.dk I liked the idea that the reader’s client should enforce the length limit.