Because advertisers don't want to deal with a bunch of smaller sites. That's why ad networks and exchanges exist, so you can buy a single impression for the right user wherever they are across the web.
Take away the fine-grained targeting and you end up with big blind campaigns running on a few large sites with the biggest reach. This is the way TV and radio work, wasteful and expensive but great for the channels that get them.
Why can't middlemen still exist, just without the individual targeting? Can't sites still self-report their categories (so that, say, a dog grooming site would say they should get dog ads)?
They can and they do, that's what ad networks are. There are plenty of ways to group sites by category and readership without individual targeting but middlemen add percentage costs to the ad spend which the industry has been slowly pushing out.
The bigger issue is that fine-grained targeting is still wanted so if you remove 3rd-party tracking then advertisers will go towards the biggest sites that can supply it via their own 1st-party tracking. Even then it's still hard to work with many vendors so even more of the money is now going to Google and Facebook who can just do it all with better data than anyone else.
It's definitely real and used for hundreds of billions in annual ad spend. Perhaps you're referring to effectiveness, in which case that's dependent on thousands of factors.
Take away the fine-grained targeting and you end up with big blind campaigns running on a few large sites with the biggest reach. This is the way TV and radio work, wasteful and expensive but great for the channels that get them.