Considering that their team likes Go, it seems strange to me that they would consider Rust over Go for the storage layer. A storage layer should be IO-bound, and should hardly trouble the CPU; the choice of language really should not be a determining factor. The big wins in that space are architectural, not language specific.
"A storage layer should be IO-bound, and should hardly trouble the CPU; the choice of language really should not be a determining factor."
This used to be true, but it's out-of-date now. You can now get a network pipe in to a system that a rather beefy multi-core CPU using a user-space TCP stack can barely keep up with, let alone do any real work, and if you can scrape up the PCI express lanes, putting a few of the latest SSDs into a system can start getting you theoretical maximum bandwidth numbers that just a few years ago looked more like what you'd expect for a RAM bandwidth number.
I'm of the opinion that it was already not as true as commonly supposed 5 years ago (in my experience using slow languages on putatively IO-bound tasks was still noticeably slower than using fast languages), but the latest in network pipes and SSDs have really ended it. It's true that on most desktop systems you've still got more CPU than you know what to do with, but as you step into the serious database space that's not true anymore. For a serious database I wouldn't be perturbed if someone looked at Go's performance and just plain discarded it on the spot, even before considering GC issues. It's very fast for a scripting language; it's fairly slow for a compiled language. "The compiler spends hardly any time on optimization" is not what you want to read about your database implementation language.
(I've got one of the nvme SSDs in my laptop, and it is interesting to see just how many CPU bottlenecks there still are in systems nowadays. In some sense, I really shouldn't ever see a "loading" screen because you "ought" to be able to read things off of my SSD fast enough to completely fill my RAM in 5-10 seconds; "merely" loading Firefox ought to be somewhere in the 50ms range. In practice I still see loading screens and load waits, because the CPUs are still doing things. Lots of things that used to be dominated by and hidden in the load time, but aren't anymore.)
You can also do a simple math analysis to see it. If you have an incoming 10Gbps connection, a single core machine has approx. 1/3rd of a cycle per bit to do everything it's going to do with that packet. Even going to a 128 core machine and assuming perfect parallelism with some sort of magical packet muxer gives you a whopping 43-ish cycles per bit. I've never worked on this myself, but I saw a team in my company working with it and were pretty pleased to be able to push ~2Gbps through their 10Gbps network connection with a pretty beefy machine, and just about all they were doing was relatively simple load balancing.
And cache. Compiled languages tend to be more cache friendly, which contributes a lot to speed of execution.
And too many apps still do cause CPU spikes, often for quite a bit of time, and no doubt much potential for optimisation lies there. The popular perception that CPUs are fast enough to deal with nearly every workload is inaccurate. Even an innocuous bit of Javascript on a webpage, probably doing some trivial stuff, causes 100% usage for several seconds.
Developing a storage layer as elaborate as RocksDB from scratch is quite an endeavor, and wanting to just use RocksDB instead of making your own is a smart decision. From there, Go is sort of easy to throw out of the picture: using cgo kills performance and safety. I say this as a person who uses Go as my workhorse, have used it for many years and has a favorable opinion of it.
Ah, my mistake. I misunderstood. I thought that they were replacing the C++-based RocksDB storage with one written in Rust. Yes, it makes perfect sense to use something if it is already proven itself.
Rust would use less memory than Go. (Dropbox also likes Go and used Rust over Go for the storage layer, and when asked, memory usage was their primary reason.)
