I don't sweat sources and almost never check them. I usually prefer to manually check information after it's provided, to prevent the model from borking it's context trying to find sources that justify it's already computed output. Almost all the knowledge is already baked into the latent space of the model, so citing sources generally is a backwards process.
I see it like going to the doctor and asking them to cite sources for everything they tell me. It would be ridiculous and totally make a mess of the visit. I much prefer just taking what the doctor said on the whole, and then verifying it myself afterwards.
Obviously there is a lot of nuance here, areas with sparse information and certainly things that exist post knowledge cut-off. But if I am researching cell structure, I'm not going to muck up my context making it dig for sources for things that are certainly already optimal in the latent space.
Well, I prefer it actually check datasheets so it doesn't go on a wild rabbit hunt to nowhere, since the capabilities it hallucinated for the chip in question doesn't exist.
In my experience, they all do this with dathasheets. Even if they read the actual datasheet, they misunderstand them gravely. I can't relie on them to do unusual setups or chaining stuff properly. It's true I did these attempts a couple of months ago, maybe they're better now.
The latent space knowledge that the models have is stronger than the inference agent going out and trying to find information to integrate into context.
If you ask why the sky is blue, the model already has the answer. It's corrosive to your conversation to pull a bunch of unknown sources into context so the model can appease your "feels right" request.
If you don't trust the answer, your brain is still way way better at quickly scanning sources to verify the answer.
But the fact of the matter is that these models went from stumbling over "9 + 7 =" three years ago to solving erdos problems today. And benchmarks (that are so saturated we don't even both with them anymore) reveal that the models basically all have total encyclopedic knowledge of every major career field. Which also makes sense because the labs have been purposely drilling hard on building pristine datasets of all this knowledge.
I would challenge you to find one firmly established general academic question that a SOTA model gets wrong. Good luck.
I use it claude and gemini all the time and they get more advanced theory,
motivation, and history wrong all the time.
If you aren’t seeing the errors it is because you are in some really mainstream conversations or because you don’t know what they are saying that is wrong.
This is trivial to demonstrate to yourself for any nontrivial project. A single academic question is easy to get the right answer for. That is not the dominant AI use case for most product people or engineers.
I see it like going to the doctor and asking them to cite sources for everything they tell me. It would be ridiculous and totally make a mess of the visit. I much prefer just taking what the doctor said on the whole, and then verifying it myself afterwards.
Obviously there is a lot of nuance here, areas with sparse information and certainly things that exist post knowledge cut-off. But if I am researching cell structure, I'm not going to muck up my context making it dig for sources for things that are certainly already optimal in the latent space.