I'm no expert either, I've just used perceptual hashing to find existing files in non-adversarial situations. I share your intuition that finding very similar pics is almost certainly a lot easier than detecting CSAM.
What I don't know is how trivial it is for a motivated attacker (who's making money) to defeat similarity detection. I would *think* that if nothing else, hashing various blur/resamples could help triangulate proximity in adversarial cases but I could be dead wrong.