The rocket equation is nasty. Adding a bit more energy requires exponentially more effort. This is why orbital rockets are staged: past a certain point, you hit diminishing returns with making a single stage bigger and bigger, and it works out better to stack it on top of another stage that does the extra work.
Think of the airplane part of this as an efficient, highly reusable first stage with poor performance.
The converse of this is that the first bit of velocity you save pays off handsomely. Every multiple of the exhaust velocity that you save nets you a factor of e savings in mass ratio. Additionally, getting out of the soup of the lower atmosphere improves the average efficiency of your engine. Again, this is a bigger deal, relatively speaking, when you’re saving the first bit of velocity off the top, as you’re multiplying on a larger launch vehicle.
Think of the airplane part of this as an efficient, highly reusable first stage with poor performance.