Joel on Software
Top Five (Wrong) Reasons You Don't Have Testers
by Joel Spolsky Sunday, April 30, 2000
In 1992, James Gleick was having a lot of problems with buggy software. A new version of Microsoft Word for Windows had come out, which Gleick, a science writer, considered to be awful. He wrote alengthy article in the Sunday New York Times Magazine which could only be described as a flame, skewering the Word team for being unresponsive to the requests of customers, and delivering an enormously buggy product.
1992年JamesGleick在遇到了错误百出的软件由此产生了一堆的问题。 新版本的Windows Word推出的时候， 科技作家Gleick觉得这是糟糕透顶的产品。他在纽约时报星期天杂志上写了一篇冗长的文章，这篇文章怒火滔天的抨击了Word团队对用户的请求不利不顾并且推出错误百出的软件产品。
Later, as a customer of a local Internet provider Panix (which also happens to be my Internet provider), he wanted a way to automatically sort and filter his mail. The UNIX tool for doing this is called procmail, which is really arcane and has the kind of interface that even the most hardcore UNIX groupies will admit is obscure.
稍后的时候，作为互联网提供商Panix的一个本地客户（我刚好也用这个互联网提供商）， 他要求一种方式来自动排序和过滤他的电子邮件。 Unix下处理这件事情的工具叫ProcMail，这个工具实在是很神秘，哪怕是最资深的Unix崇拜者也会觉得有点晦涩难懂。
Anyway, Mr. Gleick inadvertently made some kind of innocent typo in procmail which deleted all his email. In a rage, he decided that he was going to create his own Internet access company. Hiring Uday Ivatury, a programmer, he created Pipeline, which was really quite a bit ahead of its time: it was the first commercial provider of Internet access with any kind of graphical interface.
不管怎么说， Gleick不可避免的在用ProcMail的时候输错了一个字符然后导致他所有的email都被删除了。 顿时怒火中烧，他决定创建了他自己的互联网提供商。 他雇佣了Uday Ivatury为公司的程序员，Uday创建了当时还算相当前卫的软件Pipeline：这是当时第一个使用图形接口提供互联网接入服务的商业软件。
Now, Pipeline had its problems, of course. The very first version didn't use any kind of error correction protocol, so it had a tendency to garble things up or crash. Like all software, it had bugs. I applied for a job at Pipeline in 1993. During the interview, I asked Mr. Gleick about the article he wrote. "Now that you're on the other side of the fence," I asked, "do you have a bit more of an appreciation for the difficultly of creating good software?"
当然Pipeline也遇到了它自己的问题。 它的第一个版本没有使用任何的纠错协议， 所有它有一种倾向会把问题不断聚集起来直至软件崩溃。 就像所有其他的软件一样，它也有错误。 1993年的时候我申请了Pipeline的第一份工作。 面试的时候，我问Gleick先生关于他写的那篇文章。 “既然现在你走到围城另一边，”我问道，“你有没有认识到创建一个好的软件的确不是那么容易的？”
Gleick was unrepentant. He denied that Pipeline had any bugs. He denied that it was anything as bad as Word. He told me: "one day, Joel, you too will come to hate Microsoft." I was a little bit shocked that his year of experience as a software creator, not merely a software user, hadn't given him a smidgen of appreciation for how hard it is to really get bug-free, easy to use software. So I fled, turning down the job offer. (Pipeline was bought out, by PSI, the strangest Internet provider on earth, and then unceremoniously taken out and shot.)
Gleick很不高兴。 他拒不承认Pipeline有任何问题。 他拒绝承认任何跟Word一样糟糕的问题。 他告诉我说：“Joel，总归有一天，你也要跟我一样恨Microsoft”。 我当时很吃惊，不仅仅作为软件用户而且作为软件创造者这些年居然没有让他对创造出没有错误，易于使用的软件的困难有一丝丝的认识。 所以我逃走了，拒绝了他的工作邀约。 （Pipeline最后还是被世界上最奇怪的因特网提供商PSI买下，然后没有任何仪式的拖出去毙了。）
Software has bugs. CPUs are outrageously finicky. They absolutelyrefuse to deal with things that they weren't taught to deal with explicitly, and they tend to refuse in the most childish of ways. When my laptop is away from home, it tends to crash a lot because it can't find the network printer it's used to finding. What a baby. It probably comes down to a single line of code somewhere with a teensy tiny almost insignificant bug in it.
软件有错误，CPU是极端的讲究。 它们绝对会拒绝任何他们没有被显式的指明要去做的事情， 而且它们会以最幼稚的方式拒绝。 当我的笔记本搁家里的时候，它常常会崩溃仅仅因为他不能找到它以前发现的网络打印机。 多幼稚啊。 这大概只是源自于某一行代码里的某一个一丢丢不足挂齿的小错误而已。
Which is why you positively, absolutely, need to have a QA department. You are going to need 1 tester for every 2 programmers (more if your software needs to work under a lot of complicated configurations or operating systems). Each programmer should work closely with a single tester, throwing them private builds as often as necessary.
这也是为什么你正好，绝对需要一个QA部门。 你要为每两个程序员配备一个测试人员（ 如果你的软件需要一堆复杂的操作系统配置的话更多）。 每个开发人员都需要紧密的配合测试人员一起工作， 经常扔给他们内部构建版本。
The QA department should be independent and powerful, it must not report to the development team, in fact, the head of QA should have veto power over releasing any software that doesn't meet muster.
QA部门需要独立而强大， 不能向开发部门报告， 事实上QA的头头应该要有权利否决不满足要求的软件的发布。
My first real software job was at Microsoft; a company that is not exactly famous for its high quality code, but which does nonetheless hire a large number of software testers. So I had sort of assumed that every software operation had testers.
我的第一份真正意义上的工作是在微软；该公司并不以高质量的软件代码而著称，但却也雇佣了大量的软件测试人员。 所以我总是假设每一个软件公司都有测试人员。
Many do. But a surprising number do not have testers. In fact, a lot of software teams don't even believe in testing.
You would think that after all the Quality mania of the 80s, with all kinds of meaningless international "quality" certifications like ISO-9000 and buzzwords like "six-sigma", managers today would understand that having high quality products makes good business sense. In fact, they do. Most have managed to get this through their heads. But they still come up with lots of reasons not to have software testers, all of which are wrong.
大部分确实有。 但是也有很多公司没有测试人员。 实际上有很多的软件团队甚至不相信测试。
你大概会以为在经历过80年代的软件质量狂热，经历了各种没有意义的“质量”认证诸如ISO-9000还有诸如“6-sigma”的流行语之后， 现在的经理们应该理解高质量软件产品的商业意义。实际上他们确实明白。 大多数经理是通过他们的脑袋来管理这件事情。 但是他们还是会有一大堆不用软件测试人员的理由， 所有的这些理由当然都是错的。
I hope I can explain to you why these ideas are wrong. If you're in a hurry, skip the rest of this article, and go out and hire one full-time tester for every two full-time programmers on your team.
Here are the most common boo-hoo excuses I've heard for not hiring testers:
我希望我能够向你们解释为什么这些想法都是错的。 如果你很着急， 跳过这篇文章剩下的部分， 去为你团队的每两个全职开发人员雇一个全职测试人员。
下面是我听过的最常见的哭求不用软件测试人员的理由：
1. Bugs come from lazy programmers.
"If we hire testers", this fantasy goes, "the programmers will get sloppy and write buggy code. By avoiding testers, we can force the programmers to write correct code in the first place."
Sheesh. If you think that, you either have never written code, or you are remarkably dishonest about what writing code is like. Bugs, by definition, leak out because programmers did not see the bug in their own code. A lot of times it just takes a second set of eyes to see a bug.
When I was writing code at Juno, I tended to exercise my code the same way every time ... I used my own habits, relying on the mouse a lot. Our marvelous, vastly overqualified tester had slightly different habits: she did more things with the keyboard (and actually rigorously tested the interface using every possible combination of inputs). This quickly uncovered a whole slew of bugs. In fact at times she actually told me that the interface flatly didn't work, 100% did not work, even though it always worked for me. When I watched her repro the bug I had one of those whack-your-forehead moments. Alt! You're holding down the Alt Key! Why didn't I test that?
“如果我们雇测试人员”，这种理论说，“程序员就会变得大意轻心然后写出错误的代码。 通过避免用测试人员， 我们能强迫开发人员一开始就写出正确的代码.”
切。如果你那么想， 要么你从来没写过代码， 要么就是你对写代码是什么样子的绝对的不诚实。 软件错误， 从定义上来说，是因为程序员没有注意到他们代码里的错误。 大多数情况下只要看第二眼就能发现那个错误。
当我在Juno写代码的时候， 我倾向于像往常一样写代码… 我会继承我自己的习惯， 大量的依赖鼠标。 我们聪明绝顶的，大材小用的测试人员有稍微不同的习惯： 她们更多的使用键盘（ 而且实际上严格的使用各种输入组合测试接口）。 这很快就发现了一大滩的错误。 实际上她总是不时地告诉我接口经常不可用， 100%不可用， 哪怕对我来说可用。 当我看着她重现那种我拍着额头说“啊”的错误的时候。 Alt！你按着Alt键！为什么我没这样测试捏？
2. My software is on the web. I can fix bugs in a second.
Bwa ha ha ha ha! OK, it's true, web distribution lets you distribute bug fixes much faster than the old days of packaged software. But don't underestimate the cost of fixing a bug, even on a web site, after the project has already frozen. For one thing, you may introduce even more bugs when you fix the first one. But a worse problem is that if you look around at the process you have in place for rolling out new versions, you'll realize that it may be quite an expensive proposition to roll out fixes on the web. Besides the bad impression you will make, which leads to:
哇哈哈哈哈哈！ 好吧，对的，网络分发让你以比以前软件包分发更快的速度分发你的错误修复包。但是绝对不要低估修复软件的代价和消耗， 哪怕是网络站点， 当项目已经冻结之后。 为了一样事情， 你在修复第一个错误的时候有可能会引入更多的错误。 但更糟的问题是：如果你看看你们现有的软件新版本发布过程规定， 你会意识到要在网上发布一个修复可能是项代价很高的提议。 更不用说你这样做会带来的不良映像，这也会导致：
3. My customers will test the software for me.
Ah, the dreaded "Netscape Defense". This poor company did an almost supernatural amount of damage to its reputation through their "testing" methodology:
when the programmers are about halfway done, release the software on the web without any testing.
when the programmers say they are done, release the software on the web without any testing.
repeat six or seven times.
call one of those versions the "final version"
release .01, .02, .03 versions every time an embarrassing bug is mentioned on c|net.
This company pioneered the idea of "wide betas". Literallymillions of people would download these unfinished, buggy releases. In the first few years, almost everybody using Netscape was using some kind of pre-release or beta version. As a result, most people think that Netscape software is really buggy. Even if the final release was usually reasonably unbuggy, Netscape had so doggone many people using buggy versions that the averageimpression that most people have of the software was pretty poor.
啊，可怕的“网景防御策略”。 这个可怜的公司几乎采用他们的这种“测试”方法论对公司的声誉产生了超自然的破坏打击， 该方法论：
当程序员做到一半的时候，不要任何测试就在网上发布软件
当程序员说他们做好的时候，不要任何测试就在网上发布软件
重复6到7次。
将其中一个版本称为最终版本
当网上发现了一个尴尬的错误的时候发布一个.01 .02 .03版本
该公司开创了“广泛Beta测试”的概念。 字面意义上来说就是让上百万的用户去下载这些未完成的充满错误的发布版。在最初的几年里，几乎每个网景的用户都在用某种预发布的或beta测试版。 因此，大多数的网景用户认为网景软件是真的错误百出，哪怕其实最终版本实际上还是蛮好的， 网景有那么多的用户使用他们的错误百出的软件版本乃至于绝大多数用户对软件产品质量的评价很差。
Besides, the whole point of letting "your customers" do the testing is that they find the bugs, and you fix them. Unfortunately, neither Netscape, nor any other company on earth, has the manpower to sift through bug reports from 2,000,000 customers and decide what's really important. When I reported bugs in Netscape 2.0, the bug reporting website repeatedly crashed and simply did not let me report a bug (which, of course, would have gone into a black hole anyway). But Netscape doesn't learn. Testers of the current "preview" version, 6.0, have complained in newsgroups that the bug reporting website still just doesn't allow submissions. Years later! Same problem!
再者， 让“用户”做测试的意义在于：他们发现错误，然后你们修复。 不幸的是不管是网景还是地球上的其他软件公司都没有能力来筛选两百万用户产生的错误报告然后决定什么是最重要的 。当我给网景2.0报告错误的时候，错误报告网站不停的崩溃，就是不让我报告错误（其实，哪怕就是报告了，也不过就是进了个黑洞而已）。 但网景没有受教。 当前的预览版6.0的测试人员已经在新闻组里抱怨说粗无报告网站还是无法接受提交。几年了！居然还是这个问题！
Of those zillions of bug reports, I would bet that almost all of them were about the same set of 5 or 10 really obvious bugs, anyway. Buried in that haystack will be one or two interesting, difficult-to-find bugs that somebody has gone to the trouble of submitting, but nobody is looking at all these reports anyway, so it is lost.
那些数以万亿计的错误报告中， 我打赌绝大多数都是由5到10个明显的错误造成的， 不管怎样，埋在草堆下面的可能还有一两个有意思的难发现的错误，某些人费劲心思想要提交，但是最后还是没有人看，最后丢失了。
The worst thing about this form of testing is the remarkably bad impression you will make of your company. When Userland released the first Windows version of their flagship Frontier product, I downloaded it and started working through the tutorial. Unfortunately, Frontier crashed several times. I was literally following the instructions exactly as they were printed in the tutorial, and I just could not get more than 2 minutes into the program. I felt like nobody at Userland had even done theminimum amount of testing, making sure that the tutorial works. The low perceived quality of the product turned me off of Frontier for an awfully long time.
这种形式的测试最糟糕的地方还是在于会给你的公司带来难以估量的负面映像。 当Userland发布他们的第一个旗舰版本产品的时候， 我下载下来并且顺着教程开始使用。 不幸的是Frontier崩溃了几次。 而我绝对就是照着他们打印在教程上的指令而已， 花了2分钟还是没办法启动程序。 我觉得Userland公司肯定没有人做过最低限度的测试保证他们的教程是正确的。 这种给人以低质量感觉的软件在花费我相当长的时间之后让我决定了放弃使用Frontier
4. Anybody qualified to be a good tester doesn't want to work as a tester.
This one is painful. It's very hard to hire good testers.
With testers, like programmers, the best ones are an order of magnitude better than the average ones. At Juno, we had one tester, Jill McFarlane, who found three times as many bugs as all four other testers, combined. I'm not exaggerating, I actually measured this. She was more than twelve times more productive than the average tester. When she quit, I sent an email to the CEO saying "I'd rather have Jill on Mondays and Tuesdays than the rest of the QA team put together".
Unfortunately, most people who are that smart will tend to get bored with day-to-day testing, so the best testers tend to last for about 3 or 4 months and then move on.
这条确实很难。 非常难雇到好的测试人员。
对测试人员来说，就像好的程序员， 好的测试员质量要比一般的高一个数量级。 在Juno软件公司， 我们有一个测试人员， JillMcFarlane她发现的错误4倍于其他测试人员的总和。 不是我夸张， 我计算过。 她的效率12倍于平均测试人员。 当她离开的时候， 我发邮件给CEO说“我宁愿Jill仅仅在周一和周二来工作而不要其他所有的测试人员”。
不幸的是， 大多数那么聪明的人会倾向于厌倦于日复一日的测试， 所以最好的测试人员会持续大概3到4个月然后就走人了。
The only thing to do about this problem is to recognize that it exists, and deal with it. Here are some suggestions:
对于这个问题能做的唯一的事情就是承认它然后接受它，下面有几点建议
- Use testing as a career move up from technical support. Tedious as testing may be, it sure beats dealing with irate users on the phone, and this may be a way to eliminate some of the churn from the technical support side.
- Allow testers to develop their careers by taking programming classes, and encourage the smarter ones to develop automated test suites using programming tools and scripting languages. This is a heck of a lot more interesting than testing the same dialog again and again and again.
- Recognize that you will have a lot of turnover among your top testers. Hire aggressively to keep a steady inflow of people. Don't stop hiring just because you temporarily have a full manifest, 'cause da golden age ain't gonna last.
- Look for "nontraditional" workers: smart teenagers, college kids, and retirees, working part time. You could create a stunningly good testing department with two or three top notch full timers and an army of kids from Bronx Science (a top-ranked high school in New York) working summers in exchange for college money.
- Hire temps. If you hire about 10 temps to come in and bang on your software for a few days, you'll find a tremendous number of bugs. Two or three of those temps are likely to have good testing skills, in which case it's worth buying out their contracts to get them full time. Recognize in advance that some of the temps are likely to be worthless as testers; send them home and move on. That's what temp agencies are for.
Here's one way not to deal with it:
这儿还有一个不要做的：
- Don't even think of trying to tell college CS graduates that they can come work for you, but "everyone has to do a stint in QA for a while before moving on to code". I've seen a lot of this. Programmers do not make good testers, and you'll lose a good programmer, who is a lot harder to replace.
And finally. The number one stupid reason people don't hire testers:
最后。 不用测试的最愚蠢的理由：
5. I can't afford testers!
5 . 我雇不起测试！
This is the stupidest, and it's the easiest to debunk.
这是最愚蠢的，最容易攻破。
No matter how hard it is to find testers, they are still cheaper than programmers. A lot cheaper. And if you don't hire testers, you're going to have programmers doing testing. And if you think it's bad when you have testers churning out, just wait till you see how expensive it is to replace that star programmer, at $100,000 a year, who got sick of being told to "spend a few weeks on testing before we release" and moved on to a more professional company. You could hire three testers for a year just to cover the recruiter's fee on the replacement programmer.
Skimping on testers is such an outrageous false economy that I'm simply blown away that more people don't recognize it.
不管找一个好的测试有多难，他们还是比程序员便宜。 便宜多了。如果你不雇测试，那么你就要让程序员做测试。 而且如果你认为测试人员有流动不好的话，你就等着看看要替代明星程序员有多贵了， 10万刀一年， 然后他因为“在发布之前要花几周测试”然后生病了，然后去了一家更专业的公司。 而你如果一年雇了3个测试人员才刚刚能顶上招聘替换人选的费用。
克扣测试人员实在是极端的伪经济学，我只想撵走那些不承认这个事实的人。
我已经无从知道测试劣于开发的这种论断的渊源了， Joel自己论断了测试重要性半天，聪明的测试重要性半天。 但是Joel还是无可避免的戴上了有色眼镜开始歧视测试， 认为解决好的测试人员不足的唯一办法就是BLAH BLAH BLAH… 完全无法认可！ 解决好的测试人员难寻的根本办法只有消除这种对测试的偏见， 绝对不要雇那种便宜的不足的测试来顶替好的测试人员。 同时要把测试的待遇提高到至少跟程序员一致。 这才能避免测试向程序的转型，这实际上根本是不必要的。
Joel也提到了测试必须和开发紧密合作才能提升软件的质量，如果存在着这种歧视偏见和不对等，如何指望测试和开发紧密工作？ 也难怪好的测试要转型。