Who Optimizes the Optimizer? Building a Harness for Self-improving Agents

Over the past year or so our team has been working hard on building specialized agents for expertise-heavy use cases, like scientific R&D and industrial engineering. For example, a couple weeks ago we rolled out a Chemistry Specialist Agent for our customers in chemical manufacturing, biochemistry, pharma, etc. The feedback was great, it seems like they were using it a lot, so much so that we had some issues with API rate limits.

But there is a catch: that chemistry agent was entirely built by hand, which took a lot of time, effort and (wasted) tokens.

While this was a really good and successful experience, hand-crafting agents for every specific use case our customers throw at us is not an easy task, and probably not the best use of our time. So we started looking at the question of what we can build that will get us a really good harness for agents on any task, or, putting it in another way, how can we create a generic harness optimizer that will work for most use cases? This, however, brings an even more interesting question: Who optimizes this optimizer? How do we optimize this generic optimizer framework that can build a highly specific agent for any task?

We recently hosted a session during Amsterdam Tech Week to talk through our experiments, what worked, and where the pipelines break. Here’s a look at the data and key takeaways.

The shift from simple RAG to multi-agent systems

Two years ago, everyone was focused on basic RAG. You sent a user query to an LLM, searched a vector index, and got an answer.

In practice, this is evolving fast into something much more complex. A user sends a request to an agent, and that agent might spend hours delegating tasks to sub-agents, inspecting thousands of individual documents, reasoning about the user’s and the organization’s interests, calling external APIs, searching in internal and external documents, etc. In the end, the goal isn’t only to produce a good answer to the user’s request, but to also perform an action, like sending an email, triggering some action in the real world, etc.

That’s where the harness of an agent comes in. In general terms, the harness is everything that surrounds the agent. From the more “mundane” things like how we manage its traces, configurations and actual running loop to the LLM-facing part, like its prompts, skills, tools, memory and search system it has access to. The issue now becomes how to craft the better harness for an agent specialized on a given task. If you have direct access to the LLM’s weights, you could try directly fine-tuning them to your use case, but, in most cases, this is not practical or even possible.

So, our goal was to build a powerful "Proposer" (or optimizer agent) that can look at an agent's execution traces, analyze what it got right (and wrong) and (re-)write the prompt, configurations and even the code of the tools the agent uses. The proposer also needs to be able to test their new proposed harness inside a restricted environment, and iterate on its failures.

The reality of agentic AI optimization

Given the relative success of open-source repos like Andrej Karpathy’s autoresearch, we wanted to do something similar, and initially relied heavily on coding agents (Claude Code + Opus 4.8, most of the time) to build the harness for our proposer, essentially turning it into a “meta-optimizer” This in itself was an interesting experience.

On the positive side and unsurprisingly, Opus was very good at implementing and writing the actual code. It was also great at reading papers (but not finding them), summarizing takeaways, finding new datasets, and hooking up benchmarks and evaluations from Hugging Face or other sources. More importantly, it allowed for automated overnight execution. We could ask Claude to launch an experiment and monitor its execution so, if it crashed at 2 AM, the agent would log the error, fix the bug, and resume the experiment automatically.

At the same time, our experience was definitely less than ideal in other areas:

1. Roughly 90% of the high-level ideas it suggested to improve the optimizer (like new tools, architecture changes or prompt tweaking) simply did not work. New tools it proposed were never used, produced useless, unactionable results to the proposer or simply added bloat to the proposed instead of helping it.

2. It had a severe tendency to over-engineering, producing way more complex and defensive code than needed. It was not uncommon for us to see it trying to work around every possible corner case that would simply never happen in our experiments, while it kept increasing the complexity of the harness (coding agents almost never remove code, they just add code and complexity).

3. Opus (and even Fable, while it was available) was surprisingly bad at reading the traces and logs of the experiments. It would overlook a clearly broken tool or ignore something that the proposer was clearly trying to achieve with the wrong workflow. Many times we had to explicitly tell it what to look for and point out obvious issues in the traces.

4. Opus repeatedly tried to cheat by helping the optimizer. The whole goal of this was to produce a generic optimizer. But the coding agent kept trying to inject the direct answers or “helpful advice” in the proposer prompts that would leak benchmark-specific answers to the proposer. In a few instances, despite being told directly not to do it, it even tried to directly write a new tool for the candidate agents (the one the proposer was supposed to optimize).

In the end, while Claude handled most of the actual implementation, the final harness of the proposer (its tools, the formatting of the traces, how to store and manage different candidates, etc) came from us, not from this “meta-optimizer”. The conclusion is that reading the data and actually thinking through the problem will beat a frontier model in harder, more abstract problems.

The final Proposer harness

After a lot of iterations, we arrived on a quite robust harness for the proposer. It can freely inspect the traces of any previous candidates that wasn’t pruned, search the web, access and write notes for itself so next iterations don’t make the same mistakes, freely write (and test) the code for new tools and edit existing ones, edit the candidate’s configuration and prompt, change the multi-agent architecture by adding and tweaking sub-agents and actively test its new candidate on a small training set to verify how it performs.

When the proposer believes it has a “good” candidate, it submits it to an evaluation on an held-out development test. We gate new candidates based on the pareto frontier of this dev set.

In practice, it means that any new agent the proposer creates needs to land in the pareto frontier when compared against the previous agents generated by the proposer. If an agent doesn’t improve on anything, it is discarded. If a new candidate ties on a bunch of questions, loses on some, but wins on others, we keep it. This allows the system to preserve specialized traits that the proposer can later inspect, reason about what it got right and wrong and combine it with other candidate agents for a stronger final agent. 

Findings and initial results

After arriving at the final (for now) harness for our proposer, we used it to create specialized agents for different chemistry and biology benchmarks. Our findings were clear: Start from a simple harness for the candidate agent you want to optimize and let the proposer do its job (the bitter lesson strikes again), make sure the proposer has the tools it needs to do its job, and maintain good data hygiene (proper train-dev-test splits).

One of our previous findings, from a prompt-only optimization run based on GEP and that we published at an ECIR workshop earlier this year, is that it is always better to start from an empty state.

When you start with a stupidly simple, minimal, one-liner prompt as your harness and let the optimizer build the prompts and tools from scratch, you get better results than trying to optimize an already heavily hand-engineered "expert" harness.

When we let the optimizer run fully automated on our chemistry and biochemistry benchmarks, it successfully created complex sub-agent structures and wrote specialized Python code for multiple tools using domain-specific packages like RDKit without being explicitly told to do so.

Our initial results are promising. Our harness for the proposer significantly improved the agents over a “vanilla” version and even over regular agents with web search capabilities. 

The general behavior of the proposer was also interesting: Start creating general-usage tools and sub-agents that would help in all cases for the training dataset and, in subsequent rounds, keep trying to “patch” where it did not improve:

Wrapping up Amsterdam Tech Week

A big thank you to everyone who joined us for our session during Amsterdam Tech Week.

We had a great turnout, and the technical discussions afterward over pizza and beers around our SDK, evaluation metrics, and how we handle unsupervised targets were excellent.

The beautiful lesson from this research is that you shouldn't think you are smarter than the model when it comes to prompt structures, let the AI do the brute-force optimization and code generation.

But do not delegate the high-level architecture. You have to maintain statistical rigor, build the sandbox boundaries, and do the actual structural thinking yourself.

If you're an engineer or researcher interested in solving these kinds of practical, multi-agent infrastructure problems in Amsterdam, take a look at our open roles.