Some people seem to be confused why a hash collision of a cat and a dog matters. Here's a potential attack: share (legal) NSFW pictures that are engineered to have a hash collision with CSAM to get someone else in trouble. The pictures are flagged as CSAM, and they also look suspicious to a human reviewer (maybe not enough context in the image to identify the subject's age). To show that this can be done with real NSFW pictures, here is an example, using an NSFW image from a subreddit's top posts of all time.
Probably not the weird image I posted, which looks obviously suspicious. But maybe someone will make a program to find "cleaner" hash collisions that don't look suspicious.
I posted some examples that look like totally normal images, they're no harder to produce, you just need to noise-shape the gradient descent so that the introduced noise has a spectrum similar to the image. E.g. just feeding back a gaussian highpassed version of the error signal is sufficient.
> Keep in mind that you have to also collide with another perceptual hash function that only Apple has to trigger a match.
If it's another neural network I wouldn't be shocked if the adversarial preimages worked across both-- it's not uncommon for blackbox generalization to work for adversarial examples. It would be very likely if someone (maybe the attacker) made their own version of neuralhash and then generated examples that passed both theirs and apple's public one.
Privacy wise, if there were two perceptual hash functions Apple should have used the more restrictive one on the devices too -- because even if they decide to not report you, your privacy is still invaded if they inspect at your images at all.
The neuralhash function is extremely easy to attack. We should not have any confidence in the competence of its authors, so we shouldn't expect their undisclosed mechanism to provide a great deal of protection.
A secret second hash also will not be secret against a state attacker who will have access to this function by virtue of being trusted to create the databases for Apple.
There is, however, a very simple technique they could use that would provide almost perfect protection: They could stop invading the privacy of their users and refrain from scanning their private content!
Apple's reviewers don't have access to the original CSAM to know if it's a match or not. That stays with NCMEC. If they see some legal porn that looks like it could be illegal, they'd likely flag it as a match.
Roughly thirty images is the threshold for the system to activate, but would they need to review all thirty images to pass it on or would they just need to verify one image looks visually like CSAM in order to pass it along?
It seems unlikely in the event that there was anything that they verified as CSAM they wouldn't pass it on just because they found a false positive in those thumbnails.
Here is the image (NSFW!): https://i.ibb.co/Ct64Cnt/nsfw.png
Hash: 59a34eabe31910abfb06f308