Chat gpt has often given me the right answer for code after seeing the error trace resulting from its previous attempt.

I also often correct my own mistakes based on clashes with reality - I don't just become more intelligent the second time.

I would argue that you are! You will not try to clash with reality the same way you did before, provided you “remember” and I believe future agents/models will have this kind of contextual memory continuously being getting baked in to improve..just a thought.

I think you could do this with an open model with overnight tuning on the day's errors. Probably very expensive though. Easier to scoop up all the errors on the internet on the first round of pre-training.

Couldn’t agree more! That’s why also maybe they are raising 100 more billions!..:p

You don't need a super agent, you just need two LLM-based systems with errors that aren't too correlated.

How do you "just" accurately evaluate the error state space of an LLM relative to a real business process? Sounds approximately impossible to me.

If you already have the business process robustly defined as code, then the utility of LLM is unclear. The value prop of LLM is in fuzzy business processes like parsing arbitrary helpdesk tickets.

You evaluate it the way we've evaluated production ML for years, with cheap QC layers sampled and checked by more expensive layers (with humans on top.)

LLMs didn't invent stochastic process steps.