Wouldn't those concerns be resolved by being open source? Could run the new automated system in parallel for some years first, like a second umpire in sports.
How does open source make policy decisions more transparent to the majority of people? Are you assuming that everyone in the country can read and understand the code? I get why programmers and readers of Hacker News could be enchanted by this idea: government by code shifts the power from one set of elites (the wealthy/connected) to another (programmers with a specialized skill set that can read the code). But I'm unclear on how that benefits the nation as a whole.
Why would you need everyone in the country to read and understand the code? You just need a sufficiently large group of academics, programmers, developers etc to identify issues.
> You just need a sufficiently large group of academics, programmers, developers etc to identify issues.
I don't attribute any sort of sainthood or trustability to academics over other segments of society. It's certainly not clear that programmers are especially qualified to craft public policy. Again, instead of one group of elites that can understand what's going on you're substituting another.
Look, with Open Source software, programmers are mostly the consumers of that open source software. The customers of that software can more or less switch and aren't forced to use it at the barrel of a gun (and in the cases where big companies DO have monopoly power, the fact that they use Open Source software doesn't really stop them from exercising it in ways that are clearly for their benefit but less clearly for the benefit of their users). But if you're saying that software is now going to determine who can be arrested and who can go free, how long prison sentences will be or who can go on welfare or receive public money, there is HUGE incentive to game the system. Right now I might find Academics trustworthy because they don't get a huge financial benefit depending on the outcome of their research. But if public policy is controlled by software the incentives to cheat are huge. Now you can say they'll patrol each other, but that just means the incentive will be to collude collectively, as an entire group, for their own benefit against society at large.
