Your AI Coding Agent Needs More Than a Better Prompt

There's a lot of talking about how you should prompt the model in prompt engineering, which part of prompt engineering, it's what they call agent skills, which are just reusable prompts at the end of the day. But in reality, the biggest quality differentiator if it's if you give the model or the model with the agent, because nowadays models work recursively, if you give them a good thing to work against.

So usually, that's either your test or your API definition. If this API is curated and they have a good database schema, the model is really good at interpolating between these two. So you need to give them what we call the the verifier, right? Okay? Now there's different skills, I usually recommend to use a skill that it's released by the official applications. So for AWS, I use AWS skills, for React, in this case I use vercel just because they own next JS.

So whenever I'm coding with one of those, I do use the the the specific skill. And I usually prefer user invoked skills that they can control. If you use model skills, the problem is your token spend will go through the roof. Uh also, keep in mind that a lot of those skills, if they were any good, they are already added into the harness, cloud code, integrated a lot of that. But again, the biggest difference, it's always the verifier it works against.

That's why we always work on the interface and that's why you want to have uh test. Now, a quick note, a lot of people are saying that you should do TDD with agents. But the reality is that models are very good at writing code without unit test. And what I've been doing lately, it's you write the test at the end. So when you are iterating, you spend a lot less tokens and a lot less time. If you have models writing test every time they make a change and then doing it again, but you are maybe you're not sure of how things should look like in the end and you're turning around, you're just spending a lot of tokens.

So also be careful because models are not LLMs are not humans, and how we code as humans is not necessarily going to be the same thing on how you code with an LLM. And TDD, I believe it's one of those examples where people are like, oh, do TDD is going to improve your quality when you do AI. coding. I'm like, no, it it used to improve your quality as a human? Yes, but not as a necessarily as a model. End-to-end tests on the other hand are much better.