"The thing is there’s like 5 companies in the world that run jobs that big. For everybody else… you’re doing all this I/O for fault tolerance that you didn’t really need."
That's not necessarily entirely true. In the early 2000s, I worked for Cadence Design Systems, who at the time needed to build and test the plethora of tools they'd built or acquired on a large variety of systems. I worked on "GridMatrix", sort of like make but using a large, heterogeneous cluster of machines, built on top of the Condor and LFS batch scheduling systems---the same sort of thing underlying MapReduce.
On the other hand, I get where the author's going: cargo-cult design is rampant in enterprise software development. But it's not just cargo-culting Amazon or Google; it also involves fashion and resume padding.
And then, there's "eNumerate multiple candidate solutions. Don’t just start prodding at your favorite!"
Bwahahahahah. <- Unamused laughter.
Our first and only response, as an engineering discipline (if you want to call it that) is to pick the first idea that comes to mind and beat it to death with a stick.
That's not necessarily entirely true. In the early 2000s, I worked for Cadence Design Systems, who at the time needed to build and test the plethora of tools they'd built or acquired on a large variety of systems. I worked on "GridMatrix", sort of like make but using a large, heterogeneous cluster of machines, built on top of the Condor and LFS batch scheduling systems---the same sort of thing underlying MapReduce.
On the other hand, I get where the author's going: cargo-cult design is rampant in enterprise software development. But it's not just cargo-culting Amazon or Google; it also involves fashion and resume padding.
And then, there's "eNumerate multiple candidate solutions. Don’t just start prodding at your favorite!"
Bwahahahahah. <- Unamused laughter.
Our first and only response, as an engineering discipline (if you want to call it that) is to pick the first idea that comes to mind and beat it to death with a stick.