I'm not an expert flyer and I really have had no issues with it. It isn't that hard to fly at all, even in full manual. The "oh shit button" is a godsend.
It's not so much that it's hard, it's that if you want to learn freestyle or racing, you're going to crash. I guess many people just fly it like I fly my quads, cautiously and away from any obstacles.
Nobody is pretending this is a freestyle or racing drone. It really is for people, like me, who want the freedumb of FPV and have the cash to burn on it, but don't have the time to do all the research and put the effort into a "true" FPV drone that I can race and crash (and repair).
After only flying other DJI's, manual mode was mind blowing for me. The concept of having control over the speed of the props changed everything. It is one thing to just fly upwards with the stick and another to point the drone in a direction and apply more "gas".
I "get" FPV now. It is real flying... but that doesn't mean I have the time or energy to invest in it. =)