@movq@www.uninformativ.de I’m very curious…

What I like about this whole computer stuff is that you can explore how
things work. You can dig through problems and solve them. Nothing is
more satisfying than finally understanding something after you scratched
your head for some hours.

Surely you could do the same with AI? Tinker with how it works, study it, understand it, build your own and realize what it really is (without all the big tech hype)?

@prologic@twtxt.net Yeah, it’s hard to get my point across here. I tried to address that a few paragraphs down.

Yes, I can tinker with AI techniques on a general level. That’s cool but not really my area of interest.

What I certainly can’t do is learn how specific AI products work. I can’t possibly find out why Claude Code produced that particular line of code. Claude is just a magic box that does something and I have to trust it.

@movq@www.uninformativ.de I think your points are pretty clear to me, that’s fine. I’m just seeing if you can perhaps see things a different way maybe?🤔 I would challenge the assertion that you cannot understand how Claude Code generated an output; which I can demonstrate easily with a fairly trivial example by the input:

Write a program in Go that sums a list of numbers from stdin and prints the result.

Which it does so in seconds, faster than I can type. The code is correct, it compiles and does exactly what I wanted. And the code looks pretty reasonable. It handles flotas, has error handling and handles space or line separated numbers on stdin.

So going back to the understanding of how it generated this, is quite simply the most statistically relevant search space of it’s weights it has been trianed on and it has basically just produced a series of tokens, one after another that are relevant to the input, the next token and so on. It’s a trivial example I know, but it basically pattern matches it’s way through it’s vast search space just producing outputs based on context.

And every time I ask it to do the same thing, it produces basically the same result. It will sometimes not produce a go.mod, but that’s probably because doing so isn’t as statically high as writing the code to sum numbers from stdin.

@prologic@twtxt.net Ahh, I see. Okay, I’m with you there. On this high level, I can understand how the thing works.

Maybe my wording isn’t good. 🤔 Let’s take a real life example from what we do at work.

There’s this AI chatbot. It gets support requests from users, so the user says something like “I need access to a particular system”. This triggers the bot to “run” the instructions stored in a large Markdown file, like “check if the user is authorized to do this, then issue the following API requests”, and so on. This is essentially like running a little script, except it’s written in natural language (German) and there’s no “script interpreter” but just the AI.

Now, suppose that the AI doesn’t quite do what was intended. There’s some subtle bug. How do you debug this? How do you find out how the AI came to the “conclusion” to run step A instead of step B? And how do you find out how exactly you have to change your prompt so this doesn’t happen again next time?

If this was an actual script/program instead of AI, you could repeat the request and attach a debugger or throw in some printf() or whatever. How do you do that kind of thing with AI? How do you pinpoint exactly what the problem was?

(Or is this just a stupid idea? Do we have to give up that way of thinking when using AI? Is the era of debuggability over?)