>This is missing the point of a PhD, which is to develop a capacity for doing science (or more generally hard thinking in relevant domain)
These days, it is more about the ability to publish. A few weeks ago I had a conversation with a friend of mine who is wrapping up his PhD. He pointed out that not one of his colleagues is concerned whether anyone can reproduce their work. They use a home grown simulation suite which only they have access to, and is constantly being updated with the worst software practices you can think of. No one in their team believes that the tool will give the same results they did 4 years ago. The troubling part is, no one sees that as being a problem. They got their papers published, and so the SW did its job.
(Should not be surprising to anyone who spends a lot of time in engineering PhD programs).
But those AI programs that started this discussion would have teams of engineers to do that SW correctly.
One interpretation of science is just making theories and testing them with experiments, no solid software engineering required. Another says reproducibility is important but the current incentive structure is not there yet.
These days, it is more about the ability to publish. A few weeks ago I had a conversation with a friend of mine who is wrapping up his PhD. He pointed out that not one of his colleagues is concerned whether anyone can reproduce their work. They use a home grown simulation suite which only they have access to, and is constantly being updated with the worst software practices you can think of. No one in their team believes that the tool will give the same results they did 4 years ago. The troubling part is, no one sees that as being a problem. They got their papers published, and so the SW did its job.
(Should not be surprising to anyone who spends a lot of time in engineering PhD programs).