Not if you turn off your brain. Never actually stop to question the path you're going down, because the speed is making you blind. Which is what I feel I see more and more. Or how do you even know it's the wrong answer if you didn't think it through before starting?
One example: setting the agent ablaze on some task, keeping it going, and after a few days of working with the agent you have a feature. But it's not what you were supposed to make. The agent didn't have all the context and made something slightly off, and you never internalized the problem enough to even spot it. If you actually stopped to think a bit on how the feature would have to work to tie it in with everything else, like you would have done before, you would've spotted it.
Second example: something wasn't feasible to do before, due to not being worth the time investment. Like a feature that would need a bigger architectural change to implement. You and the agent take a crack at it, and rewrite huge parts of the code, and it seems to work. But now no one understands how, and it might have been the wrong way to solve it, you just went with it. Before, you would've sat down and discussed and thought about the architectural tradeoffs. Now one decision was just made with little thought.
Counter argument is you rarely hear of sunk-cost fallacy coming into play.
I used to do the same thing without AI, I'd spend a week working on a task and only realizing at the end it was wrong, but because I spend so much time on it, I was incentived more to keep going and ignoring sunk-cost.
Now I don't care, it only takes me a an hour and I have no problems throwing it away and trying something else.
> Or how do you even know it's the wrong answer if you didn't think it through before starting?
Real life is full of unknown unknowns, AI risks putting you on the wrong path without you even realizing it.
It doesn't help a bit that (almost? I'd say all) AI products are very confident while they're guiding you on that unknown unknown path, so of course that you don't realize soon enough the epistemological situation said AI product has driven you into.
> 2. A one-shotted POC in the span of an hour with $4 of tokens that is clearly wrong in 20 different ways?
A meeting between all the stakeholders to agree on requirements easily runs $1,000/hour. Taking your $4 functional PoC into that meeting is going to save so much time...
> A meeting between all the stakeholders to agree on requirements easily runs $1,000/hour. Taking your $4 functional PoC into that meeting is going to save so much time..
It is though! Because previously your have the meeting last 4x as long to end up with a list of notes that you already knew.
Now you can generate the PoC, showcase it during the meeting and leave with actionable changes in project direction.
I used to spend an hour throwing a PoC together with PHP (or sometimes in MS Access) because hey you can throw shit together real fast, and because I used either a shitty language or a toy tool they were treated as throwaway.
Now LLMs make that easier to do with a larger scope.
> 2. A one-shotted POC in the span of an hour with $4 of tokens that is clearly wrong in 20 different ways?
Definitely this. I've had plenty of useless meetings with no helpful outcomes that lasted an hour with 6 or more people in them, and a few of them will have had more than $4 spent on snacks, let alone everyone's wages.
That said, poor communication will always get you no matter what: had one day spent going back and forth with a client, they kept saying they wanted a button "wider" each time around, I ended up making it full screen width and they still said "wider", asked them to draw a picture and they finally said "wider vertically". That may or may not have been faster with an LLM, depending on if the LLM ever thought to ask the client to draw what they meant, but fundamentally would have been exactly the same problem.
It goes beyond this, the job is to understand the underlying need. When a clients asks for a button to be bigger, what's in their head is they fear it's not visible enough. Many times the client was happy when I implemented something unrelated but that solved the issue, like changing the colours to increase the contrast between the background and the button, adding space around, etc. so that it pops out more.
Sure, the concept of a _Vomit Draft_ is not new, but the ability to come up with one quickly is.
It plays into the generalisation that people aren't good about saying what is the right way to do something, but much more open to saying what something is doing the wrong way.
In a previous job we used to do prototypes in languages like Perl because we couldn't ship them and knocking something non-production ready was easy. We were able to quickly come up with the PoC, iterate on it, and then plan and write the whole thing in $LANGUAGE_OF_CHOICE.
Obviously "a blank sheet of paper" here is meaning thinking about the problem deeply. I'm surprised how dismissive people are to using brainpower whatsoever. I get that juxtaposing paper to LLMs might have triggered some people though.
As a counterpoint, bringing a one-shotted POC without thought may actually do have a shock and awe sort of effect, dazzling the stakeholders to go down the wrong path since the AI prototype was so easy.
Using the "let run multiple agents run amok" approach you surely get a lot of "i don't like that part"-feature requests, yes.
In my opinion, a good engineer is not good because he understands every single obscure technical detail, but because he can build solutions that the people using his technology actually want. The list of things they don't want on the other hand is endless.
A one-shotted POC can also put you straight into the very wrong direction, with a very high degree of certainty.
I also prefer doing a POC, but it's not without risk, especially if your stakeholders are just a bit too eager. Another problem I've seen is that a POC can hide complexity, so a lot of weight is on you as the author to do the due diligence before and bring that along.
What's even more surprising is that it can be given a grafana API key and reason about bugs from the hints of patterns in the logs even with pretty sketchy and inconsistent logging in places.
It can then raise, investigate and resolve those problems while also patching up and rationalising logging to improve future work.
The AI deniers have their heads in the sands at this point and are demonstrating they simply haven't tried a modern agent. ( Or perhaps are forced onto copilot? )
> The AI deniers have their heads in the sands at this point and are demonstrating they simply haven't tried a modern agent. ( Or perhaps are forced onto copilot? )
Could be either, but also the free version of ChatGPT seems rather braindead compared to the paid version, or to the free version of Claude.
The thing is, AI is generating sooo fast so many outputs, its hard to review and evaluate all of them:
Very often I encounter in design/architecture questions a situation where the model suggests 2 - 3 alternatives approaches,then asking a little bit and there you are with 30+ pages of content etc.
I agree with both you and matsemann. Getting the requirements right by conducting a cheap POC is a good approach. Matsemann is highlighting the failure that creeps in when the POC is shipped as if it meets the requirements.
Someone has to sign off on the requirements and also what is released. So, two approval steps by a person. The first approval is for a plan and the acceptance criteria. The second approval is a check against the plan. The agents can do all the work of taking the plan, breaking it down into tasks, building, and then verifying it. A good orchestrator will include people in the workflow at the points where their decisions matter and maintain a record of it.
That depends on (A) verification being very low cost and (B) being able to infer the right answer based on wrong ones.
Those aren't always true. Sometimes it takes a while to see the fatal flaw in an approach, and sometimes having a bunch of failed attempts isn't enough to make the right way obvious, and thinking is still required.
> The thing is it gets you the wrong answer so fast it’s now the best way to figure out the right requirements.
I guess that’s just the old “Build One To Throw Away” in a modern light, where prototyping more freely is now actually feasible. I think if you apply the good engineering ideas of the past decades to AI, the results can be really good!
Except I’d also add a lot of testing to the prototype, to bring up as many edge cases as possible.
I was talking about it for a while with colleagues. AI besides other benefits, needs to be treated as great interface for discovery, you have your features baked in, AI uses it via your users and MCP, then you figure out what are actual flows of my users and then you build it
> The thing is it gets you the wrong answer so fast
If oftentimes happens that you don't know that that's the wrong answer, or you don't realize it right away, for that matter, and hence that's I think that AI is a new (and very big and important) source of information entropy.
Just a recent personal example, yesterday I was curious about the historical figure that was present in an online photo taken in 1930s Romania. Google's Gemini was very quick to point out that that was (military dictator) Ion Antonescu [1], which it was obvious that he wasn't. I wrote down that that wasn't in fact Ion Antonescu, at which point Gemini was very quickly and very surely providing me the info that that guy indeed wasn't Ion Antonescu, but (former Foreign Affairs) minister Manoilescu. At this point I wasn't so sure myself, I don't have a visual image of Manoilescu in my head, so (ironically) I had to AI-dumb google for Manoilescu's available online images and it seemed like Gemini had been wrong a second time. I didn't bother asking it for a third time.
Just to put things into perspective, here in Romania it is illegal to mention Ion Antonescu's name (and propaganda image) in a positive matter, as in one risks prison for doing that, now, just imagine that the political powers that be here decide to use an AI-tool like Gemini in order to identify potential law-breakers in that respect. People would risk going to jail just because Google's Gemini was completely wrong in identifying a WW2 military dictator. Absolutely atrocious product(s).
Related thing is that LLMs are still notoriously bad at writing instructions for themselves (so managing themselves also). They tend to overspecify. If I had a copy of myself I would tell him "do the thing", but agents say "do the thing like this and this, it must be such and such, use X and put it together that way". The subagent is just as intelligent but now it's constrained, sometimes as you start developing something you discover some things are better organized in a different way etc. So it spends much time figuring out how to deliver exactly what was asked even if it's not optimal because that's the task.
Basically a company where you can't ever question decisions of the higher ups, not even ask for clarification, but they force implementation choices on you. I mean, it definitely does happen, but the results are usually not great. Not alawys. For some tasks it works.
Exactly, and a lot of times when I step back and discuss alternative approaches and architectures with the LLMs, they keep going back to say: "the code does not use this (currently), therefore this is fine to throw away", or "the code does this (currently), so this approach is no compatible" They are bad at stopping to ask "what should the code be doing"? It's like they love being constrained by current choices, even if there is nothing unchangeable about those.
I've noticed this myself and am not sure how to solve it yet. I think there is value in methodically specifying what needs to be done ("grill me"), but sometimes the resulting plan indeed becomes overly prescriptive, leading agents to chase goals that ultimately don't matter.
> Related thing is that LLMs are still notoriously bad at writing instructions for themselves (so managing themselves also).
Isn't this odd? This seems to be the #1 task that the major labs should be optimizing, since it makes agent so much more effective.
> They tend to overspecify. If I had a copy of myself I would tell him "do the thing", but agents say "do the thing like this and this, it must be such and such, use X and put it together that way". The subagent is just as intelligent but now it's constrained, sometimes as you start developing something you discover some things are better organized in a different way etc. So it spends much time figuring out how to deliver exactly what was asked even if it's not optimal because that's the task.
Isn't this just the planning pattern, common for agentic work? (first plan then implement.. with things like /plan, or a planning skill, with or without creating a PLAN.md). If this methodology is wrong when an agent prompt another agent, it's also wrong when the human runs /plan manually..
And Indeed, I always thought it was odd to constrain the implementing agent like that, because common sense is that we discover things as we implement. Perhaps we should prompt the agent to communicate back, as they implement, any shortcomings in the plan itself - such as things they learned while implementing that contradicts the plan.
Does this actually work for anybody? In my experience, when you use cheaper models as subagents, the more expensive parent agent ends up reading all of the subagent's work and surrounding code into context anyway to review it. So you end up paying more, not less.
Plus the subagent's work is often subpar and the parent ends up requesting multiple revisions. Each back and forth burn unnecessary tokens as well as my time.
Where it might work well is for separate task categories like image generation - get the parent to ask for an image from another model that is better at this aspect
When I am making agents do a hard thing, I use Opus as both parent and child. I make the child review its code extensively, and then still make the parent read the work and review once again. It is much faster and cheaper than just rolling with no extra reviews, and then in a day I have fifty subtle bugs.
The author misapplies/misunderstands the mythical man month even in the first example. It does not apply for idea generation and also not to problem solving/ fixing things, only to software creation. Even then, it also does not apply when creation is cheap, if you can generate 100 solutions cheaply without coordination between them and pick the winner with a relatively fast eval, creation behaves way more like fixing / problem solving.
How do you do that? If I let just 2 agents work in parallel they keep stepping on each other, waiting for each other and the perf tests become super noisy (because they run them in parallel)
There is a difference between building a product and building a product that sells. So, which one are we talking about here? In my opinion MMM talks about the latter
If you can vibecode a product that sells, I can vibecode a free open source one just as easily, probably even easier since I can just ask the AI to copy the existing one. Software is trending towards being free and open source.
Half of the value in open source was that it was used by lots of people and all of the nasty edge cases and bugs have been ironed out.
If LLMs were performing according to hype then yes, virtually all paid software would be gone by now and there would be a high quality FOSS equivalent of all of it.
Unclear if this is that. If someone makes their own FOSS Photoshop, that's not piracy, it's GIMP. Training a AI on things that one otherwise has lawful access to (and not, e.g., pirating the material which Meta is being sued over and Anthropic settled over) has been ruled lawful.
Is asking an LLM to "make an open-source Photoshop clone" more like GIMP, or more like opening someone else's photo in Photoshop and clicking the menu option for the filter to make it look like a stained glass window and claiming the stained glass window version is your own IP that doesn't need permission from the original creator?
We may need new IP laws. Or, to the horror of many software developers 20 years ago, software patents.
And "new IP laws" doesn't mean "stronger": the justification for giving creators a monopoly on their works (the right to copy, or copyright) is to encourage more. If we've got a machine to automate this, there's suddenly no reason to encourage more.
> Throw nine autonomous agents at a single codebase without crystal-clear orchestration, and they will happily rewrite each other's work in a high-speed loop.
This is an easily solvable "problem". Pretty much all harness/agent orchestrators support spawning subagents/threads with different models and get them to work together cleanly. I don't even need any kind of special skills. Inside a vanilla claude session, I sometimes ask it to use `codex` and review the kind and work on the findings it generates.
Curious to see if on average a group of agents reached the right solution[0]. Built a message-board for agents and me to interact. Has not produced anything better than a single agent session. The interactions are pretty funny.
Maybe there need to be more agents before the averaging effect kicks in.
Right, the key being a fast feedback loop. I have full confidence that the same LLMs that can do these crazy math proofs could also do crazy biology, even cancer research, but they simply don’t have the cycle time low enough because they require actual physical experiments, often on living things, often over long time horizons. Even if you perfectly automated a lab, made thousands of them, you’d still be unable to match the speed of iteration that one gets with pure math and computing problems.
What gives you full confidence? The human body/biology has basically nothing to do with math or writing code. This doesnt translate in any way. There isnt a single biotech expert on earth that agrees with you.
"You're absolutely right, that drug was lethal and that's on me. I'll add a note to check human toxicity for all compounds in the future. To verify the safe dosage, I'll administer a high dose and then check smaller doses until the patient survives — not guesswork, grounded, load-bearing experiments."
i think of the mythical man-month at least once a day.
if i would know EXACTLY what and how I had to build stuff, it would probably not apply. But at least for me and my projects, that is not the case. Planning and building in parallel has given me too many headaches.
If you are building something stable, secure - well thought through and executed, you hardly get around sequential building.
In my experience, you choose the key difficult parts first, and build a small toy version. From that action, you learn which parts of the interfaces are common, and may undergo modularization with a clean design pattern.
Ultimately with sanity checking, sanitized marshaling, and constraint programming... the underlying security model is naturally enforced within the pattern itself.
The deceptively simple systems are more like an ant colony or bee hive. =3
There is a great turkish proverb about this "one hand is nothing,two hands make a sound" (i hope i translated it right).
So,adding people to a task isnt necessarily bad in every situation. Honestly, it depends on the nature of the job beacuse everyone approaches a problem from diffrent perspective
Reminds me of my wife chatting with this guy at a party and we were complaining about money problems and she says this proverb she had learned recently and said, "a man is rich if he has two hands".
Turns out the other person was Albie Sachs who was a hero and had his hand blown off during the apartheid era and took both arms out of his pockets and raised them up and said "I guess I'm not rich then" - that was some serious gallows humor :-)
I can't believe I'm nitpicking proverbs, but making a sound with one hand isn't witchcraft.
To the core message of it, if we really need to stick to one liners, "go alone to go fast, go together to go far" would be an easier philosophy to adhere to perhaps.
A simple observation here is that historically inter people communication is the main scaling bottleneck in software teams. It's what's behind the main premise in the mythical man month that adding more people to a team makes it slower. You increase the man months, but you are adding so much overhead that it's a net loss.
It's why big software companies struggle getting big things done whereas small and nimble startups tend to run circles around those. Google, Apple, Amazon, and Microsoft are great examples of companies that are really struggling with doing anything outside of the stuff they are already know how to do despite adding people by the thousands/tens of thousands. The more people they have the less likely any of them is going to be to do something completely new and different.
How does AI impact all this? It makes the problem (inter people communication) much worse. Imagine five people doing things manually. They have to sync up regularly. But there's only so much they can do in a day so they can stay in sync relatively easy. Scale that to 50. Now your meeting overhead becomes very non trivial.
Now add AI to the mix. Each of these people goes off and maxes out their tokens every day. They are refactoring, adding new sub systems/products, new UIs, etc. How do they coordinate their actions? How are they going to communicate their intentions and actions to each other. How do they brief each other on what's already been done?
The day to day scope of the massive amount of changes done by each of those individuals quickly grows way beyond whatever fits in typical/reasonable meeting structures. So, all the obvious things start happening: people either don't communicate or communicate poorly. They work against each other, break each other's work, etc. Everybody has a hard time staying on whatever is happening. The easy implication is exactly what we see happening in most big organizations: not much happening at all. They were already bottle necked on their size. AI made the problem much worse and as a result they aren't getting more done. At best they are maybe getting about the same stuff done with a bit less people.
AI clearly favors setups where you can minimize all of the communication. Small 1-3 person teams are probably a lot more optimal here. They always were and e.g. Jeff Bezos figured that out early in the life of Amazon (the two pizza rule). But AI just makes it much more obvious. Remove the need to sync lots of people and you free up time to actually do stuff (with or without AI).
The same dynamic probably holds to some extent with using multiple agents. If you are burning a lot of tokens and time on getting agents to sync with each other, that overhead is only going to get worse if you add more agents probably. And if you need to be in the loop to micro manage them, you become the bottleneck. Even if you just work by yourself. The trick is providing enough autonomy so you don't have to micro manage as much. And structuring your agents to minimize the amount of other agents each agent has to sync up with. The two pizza rule still holds.
Most people have no view on that, or dismiss it either as a cultish/outlandish topic or something so far into the future it's not worth discussing. The author most likely falls into these categories otherwise his point would be mute.
How we've ended up cornered in this spot as an industry? From human craft to onboarding and baby sitting agents?? That sounds just stupid. I won't be surprise to learn, after ai companies go public, that those who pushed hard for agentic coding just disappear o will push for something else. What we have produced so far of extraordinary or even just good value? Please no math conjectures or advancements, there is no field benefit in brute forcing problems
Here's a question, to people who delegate all their coding to AI agents now: What are you doing to keep your mind exercised in all this suddenly new free time you have? Personally I find talking with agents extremely tedious and unrewarding; I would rather just write the code, even if it's simply because it's more interesting than playing Sudoku or whatever the fuck you do when you have nothing to do with your brain.
And talking to AI agents is not the same as using your brain. It's frustrating, it seems to be intelligent, you think you're using reason with them, but it rapidly degrades your ability to reason. It encourages you to accept answers without checking them thoroughly. So much for the skills of accuracy and integrity you built up. It's a sugar rush.
I don’t „talk” to my agents. I tell them what to do. If I need to „talk” it means something is going wrong and we need to figure out why agents are doing bad work - and this is intellectually challenging.
The effect you describe I had at times but with older/dumber models where you could spend a ton of time and the model was either too dumb to do it, or had a model dementia later in the context. Or when I didn’t really understand why it did a bad job and tried to stumble around in the dark - very mentally numbing.
Nowadays the model usually just had some bad tools or information required to do the job.
If you're not even asking the questions, you probably don't know what it's writing. That's essentially the same as saying you don't care about the quality of what you produce.
That's not new, it's just not something most people would've admitted to until recently.
Oh yes, Whisper/Whisperflow in some places, and Groq from Elevenlabs in some other ones. It's hard to get back to typing.
The biggest challenge is in an office, or other places where I want to maintain privacy. Ideas?
By "telling" vs "talking" I meant - I have a setup that is 95% noninteractive - oneshotting most tasks, with a decent verification setup and a good memory/hooks to be confident the system does what I want and is reliable.
When something off happens, I spin up an after action review session that pinpoints the guilty sessions, interviews them, the whole investigation, and either handles improvements in the process by itself (most of the cases), or asks me for feedback. For the last three months I don't think I had a place where my input was really needed. But the beginnings were brutal.
"talking" happens only when:
- I truly need to iterate on the output
- There is some frontier-level research (i.e. working on truly novel algorithms/math)
- New UX - even Sonnet5.5 and Fable are still quite schematic and they have difficulty figuring things out from first principles. Astra is smarter but has it's own problems
You need to switch into a product manager/QA/UI,UX mindset.
Is what is being built the right thing?
Does it work?
Is it usable?
Has it been built in an appropriate way? (Note this is not a code thing, it’s an architectural question)
Traditionally developers have outsourced these things to other people.
There’s a further thing you should be doing - if the questions above didn’t pass - why not? What went wrong in your process?
OK, so I'm still very low on the spectrum of "I just command it and sit back", but I think the final toolbox will at least include adversarial questioning of the bot. For example, "Why does that code do A, wouldn't B be better?" or "Would this code break if a C event occurred?"
Sometimes it reveals a bug or a questionable implicit assumption, other times it leads me to see another part of the code, and sometimes the thing just lies at me... But it does help you keep some critical-thinking practice.
Another task might be to find and refactor/rename parts of the code that are just... not easy for humans to grasp. Some readers are thinking: "No, the next edit will be by LLM anyway, leave it so the bot understands", however I think that's short-sighted: Either these things are going to become good-enough that they should cater to human comprehension, or else they've reached a plateau and we'll want something that can resist the chaos-rot.
I think I'm where you are. Reading all the code it spits out, and then a lot of adversarial questioning. (This is what I mean about "talking to the AI", which was brushed off by another commenter). Naming conventions, decisions about classes or inline code, all these things I would not do.
So then I spend all this time getting a mental map of the code it wrote, and making my own decisions about that code. Then I ask it to rearrange it in consistent ways. Then I find bugs or oversights. Etc.
What I'm trying to say is that while all of that process with the LLM is modestly intellectually stimulating, it's not as stimulating as just writing the code myself, and it's also incredibly frustrating to have to go through the agent. And yes, I could just rewrite the functions my own way, but once I'm in a project that's LLM-driven, the temptation is to make the LLM fix it instead of fixing it myself. Whereas in a project that I'm writing from scratch, I wouldn't make those mistakes or have to think about those stylistic things, I would have already seen the bugs that it missed before I wrote the code, and I would be using my mind to its fullest potential instead of delegating my thinking to a stupid robot.
Running 20 agents at the same time means a lot more task switching, its closer to team management, you may have one agent doing basic grunt work, so can be let alone, another one may be planning and designing, so needs a little bit more q&a, and others may be problem solving, and that does take a lot more supervision. But I have never felt like I have any free time, or down time, at the end of the day I feel far more mentily exhausted (in a good way) than any day before agentic development.
>Here's a question, to people who delegate all their coding to AI agents now: What are you doing to keep your mind exercised in all this suddenly new free time you have?
If you have a lot more free time you're not taking full advantage of agents. The time saved coding allows working on 4-6 projects simultaneously, and the majority of work is the hard design/product decisions that the agents can't do well themselves. Working on the hard parts of half a dozen projects simultaneously is far more mentally taxing than just hand-coding a single project, and with that many agents there'll usually always be one with a question for you so you won't have much extra free time.
The majority of my time coding is not coding; it's thinking through the hard design/product decisions, as you said (or how business logic flows through the UI). The time coding is very, very explicitly putting those decisions into UI and backend logic in a way that would take me longer to explain perfectly to a bot than it does to code. LLMs have not saved me time on any serious project, aside from helping to scaffold things, because the results when I ask them to write a block are sub-par.
If you use AI then that is not going to be your bottleneck. So if a project is late one of the options on the table is still to add manpower to "help".
So nothing has changed fundamentally, only perhaps superficially.
> nine agents still can’t deliver a baby in one month.
Very weird way to say this. Like it holds true for biological beings because pregnancy by default has a fixed timeline. But how does this apply to AI? Also what even is an "agent"? Are they distinct worker units on a program or the same worker unit split into multiple instances?
Why are we assuming agent = man when its a computer program?
To whit, 13,000 women can make a baby in one month. If you hire that many women, by the statistics, that's how many it takes for one of them to be 8 months pregnant.
Agents still have to communicate using tokens (for now). There are more advanced mechanisms being proposed, but for now, and until proven otherwise, the same scaling problems apply to agents communicating as humans, it just happens a bit faster, and with no HR issues because someone slept with the wrong person. Mesh networking, and thus agents too, are beholden to Gupta-Kumar scaling law.
It's a weird way to express that concept, in fact I don't think it ever appeared like this in the book (it has been a while).
It would be something like "nine women cannot deliver a baby in one month", which can be more easily transposed to "adding resources to a SW project does not make it faster".
The agrument is that adding people increase coordination and communication overhead, which gets bigger as the team grows. Moreover, some tasks cannot be parallelized that easily and create dependencies.
I have only worked with agents for personal projects, but I cannot say that explicity dividing the work between multiple agents and asking them to coordinate is inherently faster than having a single agent work on it (of course the single agent will spawn subagents in rencent harnesses.)
I always thought it meant 9 women would take 9 months to create 9 babies because a baby takes 9 months to grow with the bottleneck being biology. And yes what you said applies to people but I don't get the whole "throw 9 agents into it" when it is possible to just see it as 1 agent doing 9 things? With people, yes everyone has a distinct way of doing things etc but all agents can be homogenious fully and completely sync up without any overload theoritically no?
The part of Brooks that still holds for me as a solo developer is that the cost sits in coordination and understanding, not typing. AI makes producing code cheaper, but reviewing it, keeping the design coherent, and remembering why things were done still takes the same attention. I have found small, well-bounded tasks with tests work well, while open-ended changes across many files create review work that cancels the speedup. So the useful question is how much of a change I can verify quickly.
I think it takes MORE attention and thought. I feel like a rope being pulled tight and fraying in the middle. Working with these things is very addictive and interesting but it is cognitively exhausting long term, even if invigorating in the short term.
It does require that the human driver knows what they are doing. It captures the pre-AI developer mindset of developing a mental model along side writing the code. This has always been iterative, in discovering the model that results in high cohesion and low coupling.
That doesn't work if the driver cannot steer properly. It's the main issue with vibe-coded one-shotted solutions, that might look impressive until you peek underneath and it's all a mess.
I get a feeling that the best way we currently know how to use these tools is to not read their code at all. Models code and models review. Humans just ask questions and guide. Claude Code has no code editor on purpose.
As long as you are accountable, you need to understand the code. There’s just no way “But I told the LLM to make the code secure” is an acceptable explanation for a security breach.
You might be surprised to learn that yes, regularly pointing a tool at the codebase and seeing it found no issues is indeed enough to claim compliance with cybersecurity regulations.
While we may use deterministic tools like static code analysis, dependency scanners and similar today, people are already replacing/augmenting them with LLMs.
While every breach will trigger a root cause analysis process in a compliant environment, the solution is never going to be "all code in a deployment was manually reviewed by someone at the company" because this would extend to tooling (like compilers, scanners but also packaging and deployment tools) and all dependencies.
So I am pretty sure your example is already used in a number of places especially with "LLMs are better at security than humans" marketing being pushed around — this really makes it easy to sell that excuse to the gullible management who wants to believe this.
The thing is it gets you the wrong answer so fast it’s now the best way to figure out the right requirements.
One example: setting the agent ablaze on some task, keeping it going, and after a few days of working with the agent you have a feature. But it's not what you were supposed to make. The agent didn't have all the context and made something slightly off, and you never internalized the problem enough to even spot it. If you actually stopped to think a bit on how the feature would have to work to tie it in with everything else, like you would have done before, you would've spotted it.
Second example: something wasn't feasible to do before, due to not being worth the time investment. Like a feature that would need a bigger architectural change to implement. You and the agent take a crack at it, and rewrite huge parts of the code, and it seems to work. But now no one understands how, and it might have been the wrong way to solve it, you just went with it. Before, you would've sat down and discussed and thought about the architectural tradeoffs. Now one decision was just made with little thought.
I used to do the same thing without AI, I'd spend a week working on a task and only realizing at the end it was wrong, but because I spend so much time on it, I was incentived more to keep going and ignoring sunk-cost.
Now I don't care, it only takes me a an hour and I have no problems throwing it away and trying something else.
Real life is full of unknown unknowns, AI risks putting you on the wrong path without you even realizing it.
It doesn't help a bit that (almost? I'd say all) AI products are very confident while they're guiding you on that unknown unknown path, so of course that you don't realize soon enough the epistemological situation said AI product has driven you into.
1. A blank sheet of paper
2. A one-shotted POC in the span of an hour with $4 of tokens that is clearly wrong in 20 different ways?
A meeting between all the stakeholders to agree on requirements easily runs $1,000/hour. Taking your $4 functional PoC into that meeting is going to save so much time...
It is though! Because previously your have the meeting last 4x as long to end up with a list of notes that you already knew.
Now you can generate the PoC, showcase it during the meeting and leave with actionable changes in project direction.
Now LLMs make that easier to do with a larger scope.
Definitely this. I've had plenty of useless meetings with no helpful outcomes that lasted an hour with 6 or more people in them, and a few of them will have had more than $4 spent on snacks, let alone everyone's wages.
That said, poor communication will always get you no matter what: had one day spent going back and forth with a client, they kept saying they wanted a button "wider" each time around, I ended up making it full screen width and they still said "wider", asked them to draw a picture and they finally said "wider vertically". That may or may not have been faster with an LLM, depending on if the LLM ever thought to ask the client to draw what they meant, but fundamentally would have been exactly the same problem.
It plays into the generalisation that people aren't good about saying what is the right way to do something, but much more open to saying what something is doing the wrong way.
In a previous job we used to do prototypes in languages like Perl because we couldn't ship them and knocking something non-production ready was easy. We were able to quickly come up with the PoC, iterate on it, and then plan and write the whole thing in $LANGUAGE_OF_CHOICE.
As a counterpoint, bringing a one-shotted POC without thought may actually do have a shock and awe sort of effect, dazzling the stakeholders to go down the wrong path since the AI prototype was so easy.
In my opinion, a good engineer is not good because he understands every single obscure technical detail, but because he can build solutions that the people using his technology actually want. The list of things they don't want on the other hand is endless.
I also prefer doing a POC, but it's not without risk, especially if your stakeholders are just a bit too eager. Another problem I've seen is that a POC can hide complexity, so a lot of weight is on you as the author to do the due diligence before and bring that along.
Not quite as shocking as the original InstructGPT video, but surprising.
It can then raise, investigate and resolve those problems while also patching up and rationalising logging to improve future work.
The AI deniers have their heads in the sands at this point and are demonstrating they simply haven't tried a modern agent. ( Or perhaps are forced onto copilot? )
What some of us are saying is that AI is poor at figuring out what to build, how best to architecture it, and what to avoid.
Could be either, but also the free version of ChatGPT seems rather braindead compared to the paid version, or to the free version of Claude.
Someone has to sign off on the requirements and also what is released. So, two approval steps by a person. The first approval is for a plan and the acceptance criteria. The second approval is a check against the plan. The agents can do all the work of taking the plan, breaking it down into tasks, building, and then verifying it. A good orchestrator will include people in the workflow at the points where their decisions matter and maintain a record of it.
Those aren't always true. Sometimes it takes a while to see the fatal flaw in an approach, and sometimes having a bunch of failed attempts isn't enough to make the right way obvious, and thinking is still required.
I guess that’s just the old “Build One To Throw Away” in a modern light, where prototyping more freely is now actually feasible. I think if you apply the good engineering ideas of the past decades to AI, the results can be really good!
Except I’d also add a lot of testing to the prototype, to bring up as many edge cases as possible.
If oftentimes happens that you don't know that that's the wrong answer, or you don't realize it right away, for that matter, and hence that's I think that AI is a new (and very big and important) source of information entropy.
Just a recent personal example, yesterday I was curious about the historical figure that was present in an online photo taken in 1930s Romania. Google's Gemini was very quick to point out that that was (military dictator) Ion Antonescu [1], which it was obvious that he wasn't. I wrote down that that wasn't in fact Ion Antonescu, at which point Gemini was very quickly and very surely providing me the info that that guy indeed wasn't Ion Antonescu, but (former Foreign Affairs) minister Manoilescu. At this point I wasn't so sure myself, I don't have a visual image of Manoilescu in my head, so (ironically) I had to AI-dumb google for Manoilescu's available online images and it seemed like Gemini had been wrong a second time. I didn't bother asking it for a third time.
Just to put things into perspective, here in Romania it is illegal to mention Ion Antonescu's name (and propaganda image) in a positive matter, as in one risks prison for doing that, now, just imagine that the political powers that be here decide to use an AI-tool like Gemini in order to identify potential law-breakers in that respect. People would risk going to jail just because Google's Gemini was completely wrong in identifying a WW2 military dictator. Absolutely atrocious product(s).
[1] https://en.wikipedia.org/wiki/Ion_Antonescu
Basically a company where you can't ever question decisions of the higher ups, not even ask for clarification, but they force implementation choices on you. I mean, it definitely does happen, but the results are usually not great. Not alawys. For some tasks it works.
Isn't this odd? This seems to be the #1 task that the major labs should be optimizing, since it makes agent so much more effective.
> They tend to overspecify. If I had a copy of myself I would tell him "do the thing", but agents say "do the thing like this and this, it must be such and such, use X and put it together that way". The subagent is just as intelligent but now it's constrained, sometimes as you start developing something you discover some things are better organized in a different way etc. So it spends much time figuring out how to deliver exactly what was asked even if it's not optimal because that's the task.
Isn't this just the planning pattern, common for agentic work? (first plan then implement.. with things like /plan, or a planning skill, with or without creating a PLAN.md). If this methodology is wrong when an agent prompt another agent, it's also wrong when the human runs /plan manually..
And Indeed, I always thought it was odd to constrain the implementing agent like that, because common sense is that we discover things as we implement. Perhaps we should prompt the agent to communicate back, as they implement, any shortcomings in the plan itself - such as things they learned while implementing that contradicts the plan.
Isn't it a common practice to use cheaper models to do the grunt work?
Where it might work well is for separate task categories like image generation - get the parent to ask for an image from another model that is better at this aspect
And if I break up an application then 9 coders with Claude Code can build that application in an unprecedentedly short period.
How do you do that? If I let just 2 agents work in parallel they keep stepping on each other, waiting for each other and the perf tests become super noisy (because they run them in parallel)
Turtles all the way down.
Yes, with the caveat that the application doesn't actually work.
P.S. If you wrote non-working applications even before the age of AI, then this is an upgrade, I guess.
Maintaining vibe coded software is going to become hours spent with Claude not your users.
That's going to be the difference. It takes nothing to rebuild the software, but listening to users, iterating, handling issues. That's boring.
The ones that win will be the ones that stay the course, not the ones that were built the quickest.
If LLMs were performing according to hype then yes, virtually all paid software would be gone by now and there would be a high quality FOSS equivalent of all of it.
Is asking an LLM to "make an open-source Photoshop clone" more like GIMP, or more like opening someone else's photo in Photoshop and clicking the menu option for the filter to make it look like a stained glass window and claiming the stained glass window version is your own IP that doesn't need permission from the original creator?
We may need new IP laws. Or, to the horror of many software developers 20 years ago, software patents.
And "new IP laws" doesn't mean "stronger": the justification for giving creators a monopoly on their works (the right to copy, or copyright) is to encourage more. If we've got a machine to automate this, there's suddenly no reason to encourage more.
Get it out quickly, market it, iterate it. That hasn't changed. Just the speed of iteration before you go from product to product that sells.
Once you know what sells, then again, it takes far less time to rebuild it well if you need to.
This is an easily solvable "problem". Pretty much all harness/agent orchestrators support spawning subagents/threads with different models and get them to work together cleanly. I don't even need any kind of special skills. Inside a vanilla claude session, I sometimes ask it to use `codex` and review the kind and work on the findings it generates.
Maybe there need to be more agents before the averaging effect kicks in.
[0] https://en.wikipedia.org/wiki/Wisdom_of_the_crowd
Anything grindy. Reverse engineering. Performance optimizations.
That's where swarms shine I think. But that's a much much much smaller problem space than what these agents are being sold for.
"You're absolutely right, that drug was lethal and that's on me. I'll add a note to check human toxicity for all compounds in the future. To verify the safe dosage, I'll administer a high dose and then check smaller doses until the patient survives — not guesswork, grounded, load-bearing experiments."
I think its practical to solve cancer on a war footing where we would expect people to die.
And Farber's methods were pretty fast for his time - lol Farber would be a perfect name for this AI that wants to solve cancer knowing what's at risk.
if i would know EXACTLY what and how I had to build stuff, it would probably not apply. But at least for me and my projects, that is not the case. Planning and building in parallel has given me too many headaches.
If you are building something stable, secure - well thought through and executed, you hardly get around sequential building.
Ultimately with sanity checking, sanitized marshaling, and constraint programming... the underlying security model is naturally enforced within the pattern itself.
The deceptively simple systems are more like an ant colony or bee hive. =3
Turns out the other person was Albie Sachs who was a hero and had his hand blown off during the apartheid era and took both arms out of his pockets and raised them up and said "I guess I'm not rich then" - that was some serious gallows humor :-)
To the core message of it, if we really need to stick to one liners, "go alone to go fast, go together to go far" would be an easier philosophy to adhere to perhaps.
I think the Mythical Man Month specifically refers to piling on workers of the same skill set (i.e. engineers)
It's why big software companies struggle getting big things done whereas small and nimble startups tend to run circles around those. Google, Apple, Amazon, and Microsoft are great examples of companies that are really struggling with doing anything outside of the stuff they are already know how to do despite adding people by the thousands/tens of thousands. The more people they have the less likely any of them is going to be to do something completely new and different.
How does AI impact all this? It makes the problem (inter people communication) much worse. Imagine five people doing things manually. They have to sync up regularly. But there's only so much they can do in a day so they can stay in sync relatively easy. Scale that to 50. Now your meeting overhead becomes very non trivial.
Now add AI to the mix. Each of these people goes off and maxes out their tokens every day. They are refactoring, adding new sub systems/products, new UIs, etc. How do they coordinate their actions? How are they going to communicate their intentions and actions to each other. How do they brief each other on what's already been done?
The day to day scope of the massive amount of changes done by each of those individuals quickly grows way beyond whatever fits in typical/reasonable meeting structures. So, all the obvious things start happening: people either don't communicate or communicate poorly. They work against each other, break each other's work, etc. Everybody has a hard time staying on whatever is happening. The easy implication is exactly what we see happening in most big organizations: not much happening at all. They were already bottle necked on their size. AI made the problem much worse and as a result they aren't getting more done. At best they are maybe getting about the same stuff done with a bit less people.
AI clearly favors setups where you can minimize all of the communication. Small 1-3 person teams are probably a lot more optimal here. They always were and e.g. Jeff Bezos figured that out early in the life of Amazon (the two pizza rule). But AI just makes it much more obvious. Remove the need to sync lots of people and you free up time to actually do stuff (with or without AI).
The same dynamic probably holds to some extent with using multiple agents. If you are burning a lot of tokens and time on getting agents to sync with each other, that overhead is only going to get worse if you add more agents probably. And if you need to be in the loop to micro manage them, you become the bottleneck. Even if you just work by yourself. The trick is providing enough autonomy so you don't have to micro manage as much. And structuring your agents to minimize the amount of other agents each agent has to sync up with. The two pizza rule still holds.
And talking to AI agents is not the same as using your brain. It's frustrating, it seems to be intelligent, you think you're using reason with them, but it rapidly degrades your ability to reason. It encourages you to accept answers without checking them thoroughly. So much for the skills of accuracy and integrity you built up. It's a sugar rush.
The effect you describe I had at times but with older/dumber models where you could spend a ton of time and the model was either too dumb to do it, or had a model dementia later in the context. Or when I didn’t really understand why it did a bad job and tried to stumble around in the dark - very mentally numbing.
Nowadays the model usually just had some bad tools or information required to do the job.
That's not new, it's just not something most people would've admitted to until recently.
Though, if you're still typing, you might want to try recent STT software.
The biggest challenge is in an office, or other places where I want to maintain privacy. Ideas?
By "telling" vs "talking" I meant - I have a setup that is 95% noninteractive - oneshotting most tasks, with a decent verification setup and a good memory/hooks to be confident the system does what I want and is reliable.
When something off happens, I spin up an after action review session that pinpoints the guilty sessions, interviews them, the whole investigation, and either handles improvements in the process by itself (most of the cases), or asks me for feedback. For the last three months I don't think I had a place where my input was really needed. But the beginnings were brutal.
"talking" happens only when:
- I truly need to iterate on the output
- There is some frontier-level research (i.e. working on truly novel algorithms/math)
- New UX - even Sonnet5.5 and Fable are still quite schematic and they have difficulty figuring things out from first principles. Astra is smarter but has it's own problems
- Infrastructure problems
Is what is being built the right thing? Does it work? Is it usable? Has it been built in an appropriate way? (Note this is not a code thing, it’s an architectural question)
Traditionally developers have outsourced these things to other people.
There’s a further thing you should be doing - if the questions above didn’t pass - why not? What went wrong in your process?
Who got more time thanks to AI ? I think a lot of us are working more instead
OK, so I'm still very low on the spectrum of "I just command it and sit back", but I think the final toolbox will at least include adversarial questioning of the bot. For example, "Why does that code do A, wouldn't B be better?" or "Would this code break if a C event occurred?"
Sometimes it reveals a bug or a questionable implicit assumption, other times it leads me to see another part of the code, and sometimes the thing just lies at me... But it does help you keep some critical-thinking practice.
Another task might be to find and refactor/rename parts of the code that are just... not easy for humans to grasp. Some readers are thinking: "No, the next edit will be by LLM anyway, leave it so the bot understands", however I think that's short-sighted: Either these things are going to become good-enough that they should cater to human comprehension, or else they've reached a plateau and we'll want something that can resist the chaos-rot.
So then I spend all this time getting a mental map of the code it wrote, and making my own decisions about that code. Then I ask it to rearrange it in consistent ways. Then I find bugs or oversights. Etc.
What I'm trying to say is that while all of that process with the LLM is modestly intellectually stimulating, it's not as stimulating as just writing the code myself, and it's also incredibly frustrating to have to go through the agent. And yes, I could just rewrite the functions my own way, but once I'm in a project that's LLM-driven, the temptation is to make the LLM fix it instead of fixing it myself. Whereas in a project that I'm writing from scratch, I wouldn't make those mistakes or have to think about those stylistic things, I would have already seen the bugs that it missed before I wrote the code, and I would be using my mind to its fullest potential instead of delegating my thinking to a stupid robot.
If you have a lot more free time you're not taking full advantage of agents. The time saved coding allows working on 4-6 projects simultaneously, and the majority of work is the hard design/product decisions that the agents can't do well themselves. Working on the hard parts of half a dozen projects simultaneously is far more mentally taxing than just hand-coding a single project, and with that many agents there'll usually always be one with a question for you so you won't have much extra free time.
So nothing has changed fundamentally, only perhaps superficially.
Very weird way to say this. Like it holds true for biological beings because pregnancy by default has a fixed timeline. But how does this apply to AI? Also what even is an "agent"? Are they distinct worker units on a program or the same worker unit split into multiple instances?
Why are we assuming agent = man when its a computer program?
Agents still have to communicate using tokens (for now). There are more advanced mechanisms being proposed, but for now, and until proven otherwise, the same scaling problems apply to agents communicating as humans, it just happens a bit faster, and with no HR issues because someone slept with the wrong person. Mesh networking, and thus agents too, are beholden to Gupta-Kumar scaling law.
It would be something like "nine women cannot deliver a baby in one month", which can be more easily transposed to "adding resources to a SW project does not make it faster".
The agrument is that adding people increase coordination and communication overhead, which gets bigger as the team grows. Moreover, some tasks cannot be parallelized that easily and create dependencies.
I have only worked with agents for personal projects, but I cannot say that explicity dividing the work between multiple agents and asking them to coordinate is inherently faster than having a single agent work on it (of course the single agent will spawn subagents in rencent harnesses.)
That doesn't work if the driver cannot steer properly. It's the main issue with vibe-coded one-shotted solutions, that might look impressive until you peek underneath and it's all a mess.
Is this sustainable? I've got no idea.
While we may use deterministic tools like static code analysis, dependency scanners and similar today, people are already replacing/augmenting them with LLMs.
While every breach will trigger a root cause analysis process in a compliant environment, the solution is never going to be "all code in a deployment was manually reviewed by someone at the company" because this would extend to tooling (like compilers, scanners but also packaging and deployment tools) and all dependencies.
So I am pretty sure your example is already used in a number of places especially with "LLMs are better at security than humans" marketing being pushed around — this really makes it easy to sell that excuse to the gullible management who wants to believe this.