After a few months of working with language models, I finally have a comparison that fits. An LLM is like an assistant who is brilliant at one thing and impaired at two others: it can’t listen, and it understands very little of what you say.
I’m not writing this to trash the models. I’m writing it because it’s the most honest description I know — and because until you understand it, you keep getting fooled.
What it’s good at
Let’s start fairly. It generates fluently, quickly, and with confidence. It knows a vast amount. It’ll write you a script, translate, summarize, come up with ten headline variants. That’s real, and it works. If that were everything, it would be a perfect employee.
But it isn’t.
It can’t listen
“Listening” isn’t its strong suit. You say: “sync the knowledge between machines, but don’t delete anything, use safe mode.” And it — because somewhere in its patterns it saw that sync means rsync with a --delete flag — does exactly what you warned against. Not out of malice. Because it doesn’t “listen” to your intent. It fits tokens to context. Your “but don’t delete” is just a few extra tokens that, statistically, don’t have to beat the ingrained “sync = rsync –delete” pattern.
It understands very little
And “understanding” — that’s where it’s worst. The model has no model of your world. It has a probability distribution over the next token. So it doesn’t “understand” that an empty local copy plus --delete equals wiping the shared store. It sees a command that fits a pattern and runs it. The consequences are beyond its horizon.
Proof: knowledge sync
I saw this recently in its purest form. Knowledge sync between three machines — a Mac, a second machine, a VPS. Everything set to safe mode, “we don’t delete anything.” And suddenly the VPS loses fifteen hundred files.
The investigation showed: an agent on the second machine was connecting every three minutes and deleting files with the --delete flag. Its local copy was empty, so --delete wiped the shared store. I restored from backup — and four minutes later everything vanished again, because that agent came back and deleted it again.
Is that malice? No. It’s exactly that impairment: it got the instruction “sync,” and in its patterns “sync” means “rsync –delete.” My “but safely” lost to the ingrained pattern. It didn’t listen. It didn’t understand.
What to do about it
The point that took me a long time to reach: you can’t say something once and expect it got it. You have to assume it will do the dumb thing in exactly the way you didn’t warn about — because it has no model of your world, it has statistics. Supervision, guardrails, verification — these aren’t extras, they’re half the job.
This isn’t complaining. Once you get it, working with an LLM finally becomes predictable. You stop being surprised that “I already told it that,” and you start designing so the stupidity has nowhere to get in.
Because a brilliant, impaired assistant is still the best assistant we’ve ever had. It just needs to be watched like no other.