The AI Engineering Nut

For the past years I’ve been spending a lot of my time thinking (and writing) about the human side of engineering. Topics like leadership, management, psychology, humanity. The trivial things.

But I cracked that nut. Done. There’s just not enough space in this post to fit the answer.

So it was time to move on to something actually challenging.

Since the opportunity arose, I decided to switch my role to become Jimdo’s first “Head of AI Engineering.” While there are definitely still management aspects to this role, only a minority of subjects to my management shall henceforth be humans.

Bow before me, ye AI subjects.

I wonder if at some point such jokes will be considered speciesism. I may be setting myself up to be canceled or worse in a few years. If you never hear from me again, call my wife.

In all seriousness, the role is all about accelerating the company’s transition to fully embrace AI in every sensible way. In the product to the customers’ benefit, but also internally. A big part is to advance engineering practice using AI tools, and ultimately to leverage AI in every part of the organization. Quite the nut.

One of the (many) intriguing aspects to explore in this space are that LLMs (Large Language Models) have a surprising amount of overlap in behavior with humans. More so than any “traditional” computer algorithm thus far.

Still, some of you will say: “LLMs ain’t people, they’re just stochastic, probabilistic token predictors; fancy auto complete. What’s the point of comparing them to humans?”

Of course, you would be right. Academically. But who wants to be academically right? You can dismissively describe many things in a similar fashion: computers are just fancy calculators. Humans are just thermoregulated biological units optimized for pattern detection. Burn! Oh no he didn’!

While all technically accurate (let’s say), not super helpful.

Ultimately, the question is not what it is, but how we use it in valuable ways. And AI — love it or hate it — has value to provide.

Working more with LLMs over the last months has clarified a couple of left-over issues I’ve had with humans.

One example: the challenge of context window management. LLMs have a hard cap in how many tokens (think: words) can fit into a conversation. This is called their context window.

Humans have similar constraints (although it’s harder to identify the exact limit): how much stuff can we productively keep in our heads? How much context do we need, what is too much, when do we start to get confused? With LLM it’s super obvious we have to strategically manage this. I would argue that with humans we do too, but do we ever really think of it that way? Thus far, I have naively argued that “more context is better.” But is it? If you keep dumping more context information on people, do they not simply get lost in TMI? Perhaps we should also more strategically curate this? 🤔

Not my problem anymore. I’m all about AI now.

You get the point, though. There’s stuff to learn here. Both on the AI side, the human side, and their overlap.

Welcome to my (new) world.

This is a head (of AI)’s up that I will be talking, writing and videotaping myself a lot more talking about AI going forward.

And obviously, you can also expect a ton more absolute banger AI jokes. None of them LLM generated, because thus far LLMs are terrible, terrible joke tellers. That’s why the world still needs me.

Still.

Anyway, that’s some context for your window (see? banger AI jokes).

I hope you’ll join me on this journey. Nevertheless, if you’re somehow AI allergic, this may be a good time to (mentally) unsubscribe. Your loss though. Because I repeat: AI jokes. I mean come on.

This is going to be awesome.

Meta comment

Since people are going to ask: none of this post was written by AI, although I will admit I did have ChatGPT generate a first version of it.

After ChatGPT’s first try, I proceeded to delete 99% (I kept “the human side of engineering” phrase at the beginning — I liked that), and crafted everything by hand. Like an animal. I then had AI critique it — devil’s advocate style — but it completely missed my signature sarcasm — so no valuable dice.

Famous last words, but I intend to keep it this way. I think (based on what people have been telling me to my face) that the reason everybody (and I mean everybody) reads my stuff is because it’s me. I am (and I quote:) cool. While little of what I’m proclaiming in these posts is technically new, what adds value is my particular way of framing things, with my signature blend of hilarious dad jokes. Try that, ChatGPT. You personality-less... LLM, you.

That said, the bland draft that ChatGPT produced did encourage me to finally write this post. I find this to be a common pattern in what AI tools have to offer: even if you don’t end up using it all, or even a little of what they produce, they are a surprisingly effective way to have the energy to just start, to do things you would otherwise not even attempt. Courage.

AI. Quite the nut. Let’s get cracking.