There were indeed consistent failures to specific IPv6 endpoints, clearly identifiable through curl, while all the IPv4 endpoints were ok.
This happened with pypi (IPv6 BGP routing problem caused by a bad route from one of our peers combined with their fastly CDN not reply to us on IPv6 from the other side of the ocean for some weird reason), but also with yum and apt mirrors (seemingly random problems with the IPv6 service or firewall of the remote endpoint), and various other web resources accessed from pipelines.
The solution always was to temporarily block the bad IPv6 endpoint(s) or temporarily completely disabling IPv6 on the server itself or on the squid proxy server for workloads without direct connectivity.
Obviously it also can be the other way around, but in practice it appears to happen less often with IPv4, and if it does things get addressed quickly instead of taking hours or days or weeks.
Try `ip link set mtu 1280 dev eth0` (or equivalent for your OS).
pMTUd breakage exists on v6 just like it exists on v4, and requires workarounds just like it does on v4. I get the impression a lot of people are applying a workaround on v4 but not on v6, then blaming the resulting failures on v6 without bothering to do any troubleshooting to figure out what's actually wrong.
This was not a path mtu problem since it was random IPv6 endpoints having issues.
And if all the IPv6 endpoints of a major service go down we should pin our mtu to the bare minimum? Nonsense.
The inconvenient truth is that IPv4 will be more stable than IPv6 until the former gets demoted to second class fallback protocol instead of the major driver of the internet. And when that happens no one knows.
People running servers don't seem to be reliably capable of making sure that either pMTUd works or isn't needed on their network, so... it is indeed broken on random servers.
We've mostly decided to go with TCP MSS clamping rather than using the minimum MTU, which is still nonsense but it's nonsense that we already decided to go with for the same problem in v4.
Of course, I don't have enough info to tell if this was the problem or if it was something else, or a mix of the two, but pMTUd does seem to be a leading cause of "fails to load for some weird reason" problems.
This happened with pypi (IPv6 BGP routing problem caused by a bad route from one of our peers combined with their fastly CDN not reply to us on IPv6 from the other side of the ocean for some weird reason), but also with yum and apt mirrors (seemingly random problems with the IPv6 service or firewall of the remote endpoint), and various other web resources accessed from pipelines.
The solution always was to temporarily block the bad IPv6 endpoint(s) or temporarily completely disabling IPv6 on the server itself or on the squid proxy server for workloads without direct connectivity.
Obviously it also can be the other way around, but in practice it appears to happen less often with IPv4, and if it does things get addressed quickly instead of taking hours or days or weeks.