The latent space knowledge that the models have is stronger than the inference agent going out and trying to find information to integrate into context.
If you ask why the sky is blue, the model already has the answer. It's corrosive to your conversation to pull a bunch of unknown sources into context so the model can appease your "feels right" request.
If you don't trust the answer, your brain is still way way better at quickly scanning sources to verify the answer.
But the fact of the matter is that these models went from stumbling over "9 + 7 =" three years ago to solving erdos problems today. And benchmarks (that are so saturated we don't even both with them anymore) reveal that the models basically all have total encyclopedic knowledge of every major career field. Which also makes sense because the labs have been purposely drilling hard on building pristine datasets of all this knowledge.
I would challenge you to find one firmly established general academic question that a SOTA model gets wrong. Good luck.
I use it claude and gemini all the time and they get more advanced theory,
motivation, and history wrong all the time.
If you aren’t seeing the errors it is because you are in some really mainstream conversations or because you don’t know what they are saying that is wrong.
This is trivial to demonstrate to yourself for any nontrivial project. A single academic question is easy to get the right answer for. That is not the dominant AI use case for most product people or engineers.
If you ask why the sky is blue, the model already has the answer. It's corrosive to your conversation to pull a bunch of unknown sources into context so the model can appease your "feels right" request.
If you don't trust the answer, your brain is still way way better at quickly scanning sources to verify the answer.
But the fact of the matter is that these models went from stumbling over "9 + 7 =" three years ago to solving erdos problems today. And benchmarks (that are so saturated we don't even both with them anymore) reveal that the models basically all have total encyclopedic knowledge of every major career field. Which also makes sense because the labs have been purposely drilling hard on building pristine datasets of all this knowledge.
I would challenge you to find one firmly established general academic question that a SOTA model gets wrong. Good luck.