Where did you see this idea in Coders At Work? Most of them say testing is important . Most of them don't say "semi automated" testing is "essential". So where is your perceived "consensus" of programming legends coming from?

or this

"TDD (writing the tests before the code): good in theory but alien to most folks"

for that matter? Sibel mentions 5 developers in his blog post - Zawinski, Knuth, Norvig, Bloch, Armstrong. I didn't see anyone saying "good in theory but alien to most folks". Only Armstrong says he regularly does (soemthing like) "test driven", but if you read his interview he also says he spends a lot of time doodling and getting his abstractions right before coding, hardly the "YAGNI" style of coding and theory of "emergent design" propounded by most agile consultants.

Who in CaW do you think concludes either of the two statements you put forward as "general conclusions from programming legends".

Or are you saying Robert Martin and co are programming legends? Bit hard to justify I'd think ;-)

You say,

"There still seems to be a general reaction here" .. "I'm too smart and/or busy to write tests" and the corollary "Anyone who writes tests, or advocates testing, isn't a good programmer"."

The former " I am too busy to write (TDD style) tests" is a justifiable argument in certain circumstances (Zawinski says as much) and the latter "Anyone who advocates testing is a poor programmer" is something I don't see anyone here saying. Who said this? Links please? And if someone did, why do you think it is a "general reaction"?

fwiw I see the evolving consensus here ) as "writing unit tests is valuable in certain circumstances (and not so in others) as long as you aren't fanatical about them" and "what is important is not the methodology but the end result".

You seem to be putting forward your beliefs about the value/validity of TDD/testing etc as the "general conclusion, from various programming legends". I am curious as to how you came up with this apparent consensus.

Don't get me wrong, you certainly have a right to your conclusions. I am just saying the CaW interviews and the posts here don't seem to converge to a conclusion along the lines of you say they did.

Zawkinski: created automated tests for a complicated corpus of quasi-standard email headers that he triggered manually. When he rewrote the mailer in Java he unit-tested because it was easier in a Java OO environment. He says there's specific areas where unit-testing would have sped them up, but that they were under so much time-pressure, with short-term ROI horizons that they didn't bother, they just went with the gamble of "getting it right first time". If they had more time then unit-testing was "certainly the way to go". He specifically states that they traded quality for a ship date.

Fitzpatrick: Does "a lot of testing". Tests anything half-way "clever" because others will break it. Forces people who work for him to write tests. "At a certain point, people realize, “Holy crap, it does pay off,” especially maintenance costs later."

Joshua Bloch: automated testing of transactional shared-memory with a "basher". When it failed at high concurrency he had to use unit tests to locate the bug, which wasn't in his code but the underlying Solaris assembly code. This was early 90s and he implies that finding a bug in such code today would probably result in the engineer being shouted at for inadequate testing, though perhaps not in concurrency areas as testing those is still an "art". Argues that writing an API before the code, which he does, isn't technically TDD though I'd say it's in the same ballpark.

Knuth: is "a fan of after-the-fact torture tests".

The blog author interprets the other comments as anti-TDD, though they seem unrelated to me.

Joe Armstrong: Actually does TDD.

Norvig: "I think test-driven design is great. I do that a lot more than I used to do."

I don't understand how you could have come up with a different reading. The originally post seems to be half-heartedly questioning TDD (without much support from the quoted sections), but that is on the far end of a continuum that has automated tests, then unit tests, on the way towards it. Which I thought I made clear with my list where the near end of that spectrum (automating tests for complicated areas) is better supported than the far end (TDD according to the book). Many responses I see here seem to reject testing entirely.

(edit: just to be clear when I say "automated" I mean they wrote a program to test their program. It doesn't necessarily mean that they run it on every change, or check-in or whatever. I just mean it runs and says: pass or fail without a human being having to examine the output, though that output may be helpful when it does fail.)

(EDIT: you first claimed that of the 6 developers from CaW 5-1/2 of 6 supported your claims. Knuth being the "1/2" You've since edited your post)

51/2 for 6?

You must be kidding. You are cherry picking statements that support your viewpoint and ignoring those that go in an opposite direction.

these were what you said "programming legends" had a consensus on

"• (semi-)automated testing for any complex code: essential • unit tests for any long lived code: great • TDD (writing the tests before the code): good in theory but alien to most folks"

Let's see now.

"after the fact torture tests" (Knuth) is hardly support for "TDD' or "semi automated testing is essential". SO that's one down (or 1/2 down :-P) So that yo are down to 5 of 6 right there.

BTW Knuth also said (http://www.informit.com/articles/article.aspx?p=1193856)

"As to your real question, the idea of immediate compilation and "unit tests" appeals to me only rarely, when I’m feeling my way in a totally unknown environment and need feedback about what works and what doesn’t. Otherwise, lots of time is wasted on activities that I simply never need to perform or even think about. Nothing needs to be "mocked up."

Zawinski, Knuth, Norvig etc don't say anything like "semi automated testing is essential" (one of your conclusions) or "TDD is good in theory".

It is one technique among many they sometimes use,certainly.

Norvig (for example) specifically says

"Seibel: What about the idea of using tests to drive design?

Norvig: I see tests more as a way of correcting errors rather than as a way of design. This extreme approach of saying, “Well, the first thing you do is write a test that says I get the right answer at the end,” and then you run it and see that it fails, and then you say, “What do I need next?”—that doesn’t seem like the right way to design something to me"

So that is 4 of 6.

Zawinski says

"Seibel: In retrospect, do you think you suffered at all because of that? Would development have been easier or faster if you guys had been more disciplined about testing?

Zawinski: I don’t think so. I think it would have just slowed us down. There’s a lot to be said for just getting it right the first time. In the early days we were so focused on speed. We had to ship the thing even if it wasn’t perfect. We can ship it later and it would be higher quality but someone else might have eaten our lunch by then."

In other words in retrospect, If speed is a concern, "being disciplined about testing" is NOT important which is hardly "consensus" about the centrality of testing.

So that is 3 of 6.

About Bloch, you say "Argues that writing an API before the code, which he does, isn't technically TDD " and you concluded "it is in the same ballpark" ?! You conclude something 180 degrees opposite to what the interviewer understood?

Josh said

" I do disagree with Martin (Fowler) on one point: I don’t think tests are even remotely an acceptable substitute for documentation. Once you’re trying to write something that other people can code to, you need precise specs, and the tests should test that the code conforms to those specs. "

So write precise specs, write code and then add tests that the code confirms to written specs is "in the same ballpark" as TDD? This is how sw is being developed for ages.

By this logic, any thinking about code before coding is "in the same ballpark" as TDD. A strange argument.

Besides Josh Bloch wrote a test when everything else failed. And when specifically asked if an automated test would have been useful, he says

"I think a good automated unit test of the mutex facility could have saved me from this particular agony, but keep in mind that this was in the early ‘90s. It never even occurred to me to blame the engineer involved for not writing good enough unit tests. Even today, writing unit tests for concurrency utilities is an art form."

More nuanced than your claim that Josh was all for automated testing don't you think?

So 2 of 6.

So how did you come to your conclusions "TDD is great in theory but too strange for most folks" as something they concluded?

You are concluding a few things and cherry picking statements to support this.

Every developer in the world would love an extensive (semi) automated test suite if they didn't have a cost to write and maintain.

You can't take a statement from someone that "tests are great" and ignore "tests are for correcting errors" or "when speed is critical, tests are not important".

My point was that you say "programming legends" conclude various things. As I see it they don't conclude anything beyond "tests are good in certain circumstances for specific purposes". They make some generic statements in support of "testing" but are very cautious of "test driven development" or "you must test every bit of code" por even "you must have an automated test suite".

There is no consensus for the stronger statements you put forward. Nobody says "TDD is good in theory but most people don't get it". Almost everyone says it is a bad idea.

The second part of your original post , that the consensus evolving here on HN is "anyone who advocates testing is abda programmer" is still unsupported. Please, show me links.

EDIT: I see you've edited your post and are now proposing a weaker claim "testing" vs "tdd". Sure if you are saying "programs that test other programs" are a useful technique to have in your arsenal, sure thing.

Programmers have written tests drivers (in other words programs that tests programs) from the beginning of programming. This is hardly an argument for "there is a consensus among programming legends different from teh consensus here on HN" .

I assume that anyone who does TDD thinks unit tests are a useful tool, and that anyone who uses unit tests thinks that automated tests are a useful tool. That seems logical.

Regarding thinking about the API, that's not "thinking about the code", it's thinking about the code that's going to call the code. Unit tests are also code that calls the code. Orthodox TDD theory claims that by writing code that calls the code first you are in fact designing good APIs. Seems related to me.

JWZ's comments on speed are all over the map, probably because he's not talking about sustained speed of development, but a very short shipping deadline based on market forces. A sprint versus a marathon. I think his comments strongly support unit testing in the marathon case (which surprised me based on other things I'd read).

If you have to do something because nothing else works, I'd call that "essential".

Regarding TDD, which seems like the real sticking point, as opposed to unit-testing or automated tests. Two of the six actually claimed to do it personally and like it. Two of the others seemed so enthusiastic about unit-testing that TDD didn't seem like a big leap to me. One designs APIs by considering calling code first, which as I said above, reminds me of some elements of TDD. The last one was talking about writing code in pencil. It didn't seem relevant, though the blog poster thought it was.

I didn't say they did TDD, by the book, 100% of the time, or that they thought it was suitable for every task. But if someone actually says they do TDD then who am I to argue?

I intially commented mostly because of the dissonance caused by the article quotes being very pro-testing, and even had two people flat out claiming to do TDD and yet the overall tone of the original post commentary was anti-TDD, and many comments here (and in previous related posts) were anti-testing in general (not specifically TDD, which I believe the true faithful don't even like referring to as "testing" since it's actually about "design"). That anti-testing feeling seemed to be drawing inspiration from these same heroes, who when I read their own words, struck me as very upbeat about testing.

"useful" is different from "essential". Conflating the two words hardly makes for a coherent argument.

" Two of the others seemed so enthusiastic about unit-testing that TDD didn't seem like a big leap to me."

The problem is in this "leaping" and essentially putting words in their mouths to conclude what they didn't.

When you quote others in support of your conclusions we have to look at what they actually said, not where you landed up after "leaping" from what they said. So you have two of six tdd-ing/ testing/whatever it is you are claiming now.

Unit testing existed long before TDD was put forward as a named practice.

unit testing AFTER writing the code is NOT tdd as its proponents take great pains to make clear. You can't "leap" to conclusions directly opposite to the meaning of terms.

If you read the actual quotes only one saying they do TDD (not automated tests after writing the code) in the accepted meaning of the word "TDD" (TDD == write test code BEFORE you write app code then refactor to get the "design" correct , aka "red green refactor") regularly.

And the one person who writes tests first, Joe Armstrong, does significant "design up front".

This hardly supports your original claims.

"One designs APIs by considering calling code first, which as I said above, reminds me of some elements of TDD. "

By this logic if I think about how a user will use my web app I am doing "TDD". You can't write any code, API or non API without thinking of how it is used. It is just that in API design the "user" is code. Every API designer in the world does this thinking.

Doing this thinking is hardly "TDD". Every programmer in the world is (in your words) "in the ballpark" of TDD practice then. Again ,hardly a strong argument.

"The last one was talking about writing code in pencil."

This must be Knuth. So now you are down to 1.

"I didn't say they did TDD, by the book, 100% of the time, or that they thought it was suitable for every task."

No. But you said there was some sort of consensus among programming legends that "TDD is good in theory but too alien for most folk". (I am quoting you exactly). No one said anything like this or anything that implied this.

All the actual interviews(vs your "leaps") show is that (a) these developers are aware of TDD. (b) Some of them use it occasionally (c) The one person who uses it regularly (Joe Armstrong) does many things different from the theory of "TDD" as propounded by its more mainstream practitioners.

At best you have 1 person of 6 who "tdd"s regularly. This is "consensus" that "TDD is good in theory but alien to most folks"?

Also in your initial post you explicitly juxtaposed these programming legends's "consensus" with a claim that here on HN there was an emerging counter consensus that "anyone who advocates tests is a bad programmer"? I asked for links to anyone saying that. You didn't provide any.

let me ask again. How do you support that?

Peter Seibel has pulled together a blog post on what his interviewers said about TDD and testing. You interpreted fragments of the interview to lead to unsupported conclusions opposite from Seibel's (which are very balanced btw). Then you said the "egenral consensus" on HN is that "anyone who advocates testing is a bad programmer".

When challenged you edit your posts and play with words.

Ahh forget it. This thread is too deep now. And since you edit your posts continuously so my responses don't make sense anymore (apologies to other readers) it is not worth my time trying to keep up.

If you always intended to say "developer testing (including writing unit tests, drivers whatever) is a good thing" in general,then I agree.

Your original post made very strong claims (since edited, very frustrating to responders).That was what I reacted to. and you still haven't supported your claims about the "HN consensus". ;-)

But yeah whatever! Have a nice day.