Er, one of the lessons from the '80s was the end-to-end principle. Applications are the right place to handle encryption, and network encryption solutions like IPsec are full of layering violations.
One of the other lessons from the 80's in the high-assurance-secure systems was that you can get great ROI from throwing everything you can in terms of assurance at an security-focused kernel and VPN solution that everything else uses. Apps using secure tunnels, mainstream or high-assurance, got hacked way less on crypto part since rolling their own crypto led to more screwups than using solutions by crypto/security-focused developers. You can put the tunnel in network layer, app layer, or whatever. So long as they have a black box with good defaults and documentation on using it.
The first was BLACKER VPN. NSA has their Type 1 Link Encryptors. Mikro-SINA put it in on Intels with more open components. Now, I'm pushing for Wireguard variants in both app and network deployments. There's people laying groundwork for expanded use of it right now. Once that's done, I know some people who might put it in some low-privilege architectures.
Don't you think it ought to be handled by the kernel though? The same way TCP is not handled in userland nowadays. It's a shame that every app has to reinvent the wheel when it comes to crypto (even if most of them de-facto standardize around a few library implementations). I wish SSL/TLS was just one setsockopt away. You'd have one central point to handle certificate authorities, exception, debugging etc...
Absolutely not - there's a ton of complexity in parsing certificates, checking Certificate Transparency signatures, contacting OCSP responders, chasing Authority Information Access references to intermediate certs (the latter two immediately require having a full HTTP client), parsing name constraints, etc. There are already too many ASN.1 decoders in my kernel and I don't want another one. And I can upgrade my userspace libraries much more easily than I can upgrade my kernel.
Importantly, the big reason for TLS to be at the endpoint and not use IPsec/tcpcrypt/etc. is because the important thing to verify in TLS is the hostname, not the IP address, and DNS resolution is in userspace everywhere (as far as I know). You could imagine a design which pushed both DNS resolution and TLS verification into the kernel, but it would be too monolithic of a kernel for even UNIX's tastes.
TCP is a relatively simple protocol (in the sense that it doesn't have too many moving parts, not in the sense that it's uninteresting) and fits fine in the kernel. But even for reliable transport, there are plenty of apps using their own thing over UDP (RTP, Mosh's SSP, QUIC, etc.) that are never going to get into a kernel. And even for TCP alone there's a bunch of potential complexity you can put into the kernel (DCTCP, New Reno, fq_codel, etc.) that have been hard to deploy precisely because they require a kernel patch.
> what has certificate management anything to do with communication encryption?
What's the point of encrypting something if you don't know who you're encrypting to?
> besides, everyone and their dog now has a "certificate" from the likes of let's encrypt, so even their role as identity trust is basically moot.
Yes, everyone should have a certificate, and I'm not sure why you think this means "identity trust" isn't working. Certificates mean that you probably are who you say you are. They don't mean that you're a good person.
Userland TCP/IP stacks are in fact a thing once again, for the sake of performance and scalability in high end systems. It's not common to most systems, but to say it isn't done anymore isn't true. As we move beyond 40Gbps and 100Gbps Ethernet it's only going to become more common.
