I was wrong about prompt engineering

Yesterday I realized that I was completely undervaluing prompt engineering.

When it first became a thing, there was a massive influx of people telling you they're going to teach you how to prompt, selling prompts, creating prompt libraries, directories, and so on. To me, initially that felt like bullshit. I was thinking that structuring thoughts properly to an LLM and using hacks to make it give a smarter reply is a waste of time — that's essentially like syntactic sugar on top of what it's going to produce anyway.

I realized that my way of thinking about this was completely wrong.

What changed my opinion is a prompt that a friend shared with me yesterday. It was essentially setting any LLM to become your personal money coach doing diagnostics. It gave very particular instructions in terms of the algorithm the LLM should follow, the frameworks it should use, and the knowledge it should encapsulate into its answers — in order to quiz me and lead me to a point where I would have meaningful insight.

I answered nine questions to a survey generated by an LLM, and I think I now have three months of content to bring into a psychotherapist session. Those were non-fluff things — deep, meaningful insights based on my personal situation, not some generic advice.

That made me realize something. Maybe prompt engineering — in the sense of making the prompt structured so the LLM produces a smarter result — is not what makes a great prompt. The value of a prompt is actually in the boundaries and limits it sets for the LLM, and the frameworks it binds the LLM to follow. The knowledge and the answer partially sit inside the prompt, and that's where the value comes from.

If you come and start talking to an LLM, even if you already know the problem you want to solve, it will follow the most generic auto-complete scenario. But that's not necessarily what you want. You might want to follow a particular framework. You might want to avoid some things. Limitations actually give the LLM a structure that makes its responses useful. It is similar to how art is sometimes born from limits — Dovlatov wrote his novels following the rule that no two words in a sentence start from the same letter. When you read him, you never notice. But his prose is extraordinary.

Also, oftentimes we don't know what we don't know. Even if a framework is useful for us, we might just not know it exists. We wouldn't ask the LLM to apply it, and nothing would happen.

So what I'm saying is — maybe there is a person who has the knowledge and knows how to help you, but they don't have time to deal with you, or you don't have access to them. If in the previous century their solution would be recording a video course and spreading it on YouTube, now they can write a very good prompt and make an LLM, in an interactive way, follow the framework that person would apply in teaching you — or bringing you to the insight you want to get to.