An Open Source OS can help, sure, and is a start. A DSP is a programmable hardware device. Both phones to which you linked use variants of ARM processors and then use third-party baseband systems. You're not getting rid of closed-source hardware vulnerabilities by replacing Android or iOS.
There's Osmocom[1] for that. Sadly, it doesn't support modern PHY layers of the modem not modern baseband stack. It demonstrates, though, the possibility. I wish it had more traction.
Agree, we need open source hardware (like RISC-V) to mature in order to eliminate this class of vulnerabilities. I haven't heard much on mobile class RISC-V SOCs though.
RISC-V is an open source ISA, which means anyone is free to implement it, interface with it, customise it etc.
But most RISC-V devices are not open source as far as I know, as least currently. And a mobile class SoC would still be a very complex device, therefore with vulnerabilities (and also therefore with much less motivation for a company to open source the whole design). You'd have a similar problem as now.
That said, if someone wants to work with me on a RISC-V mobile class SoC (or server/supercomputer class) do get in touch, I'd love to do it :-)
Yes, you're not getting rid of closed-source hw vulnerabilities, but you get a lot of control. Librem 5 allows to replace the modem. Both phones ensure that it cannot access anything in the OS. You can also use killswitches if you require location privacy.