Therac-25
The Therac-25 case raises questions about responsibility in safety-critical technologies when software, hardware, manufacturers, operators, regulators, and hospitals all play a role. The Therac-25 was a radiation therapy machine that relied heavily on software rather than hardware safety interlocks. In several cases, patients received massive radiation overdoses even though the machine displayed messages suggesting that no dose had been delivered. The error was eventually traced to software problems that could occur when operators entered commands quickly, but the broader failure involved poor software design, insufficient testing, lack of independent review, weak documentation, and misplaced confidence in a new computer-controlled system. The main question of this case is who should be held morally responsible for the injuries and deaths: the programmer, AECL executives, regulators, hospitals, operators, or some combination of all of them. This case highlights how the problem of many hands becomes especially serious in complex technologies, where no single person may fully understand or control the whole system.