The different flavors of non-determinism are interesting.
There’s chat-vs-api; same model answers differently depending on input channel.
There’s also statistical. Once in a rare while, a response will be gibberish. Same prompt, same model, same input mode. 70% of the time, sane and similar answers. 0.01% of the time, gibberish. In-between, a sliding-scale — with a ‘cursed middle’ of answers that are mostly viable except for one poisoned thing that’s hard to auto-detect…
There’s chat-vs-api; same model answers differently depending on input channel.
There’s also statistical. Once in a rare while, a response will be gibberish. Same prompt, same model, same input mode. 70% of the time, sane and similar answers. 0.01% of the time, gibberish. In-between, a sliding-scale — with a ‘cursed middle’ of answers that are mostly viable except for one poisoned thing that’s hard to auto-detect…