Software autopilots in aircraft, and the dangers of over-relying on computers

In short, the aircraft entered a storm and its speed sensors iced over. The computer recognised that it couldn't fly the plane and disengaged the autopilot (as it was designed to do). The pilots took control, but didn't realise that the computer's fly-by-wire safety system, which automatically prevents actions which will endanger the aircraft (in this case, flying too slowly), had also switched off (again, as it was designed to do).
The plane entered a stall. Interestingly, the pilot ignored nearly 100 warnings from a computerised voice telling him that the plane was flying dangerously slow, perhaps because he assumed the error message was false and because the computer normally automatically stops problems like this. The plane stalled and, unaware of the situation (perhaps because they believed it could not happen), the crew did nothing to rectify it (in fact, they made it worse). The aircraft crashed, killing all 228 onboard.

Definite issues with People-Machines interaction here, with the pilots clearly being unaware of what the computer was doing and (more importantly) not doing. The report says the pilots lacked training on the system and were unaware that the safety system could switch off, and also criticised the pilots for lack of communication between themselves.

The article finishes by discussing the issues of ever more automated computer control systems:

"Over the decades, airliners have been built with increasingly automated flight-control functions. These have the potential to remove a great deal of uncertainty and danger from aviation. But they also remove important information from the attention of the flight crew. While the airplane's avionics track crucial parameters such as location, speed, and heading, the human beings can pay attention to something else. But when trouble suddenly springs up and the computer decides that it can no longer cope—on a dark night, perhaps, in turbulence, far from land—the humans might find themselves with a very incomplete notion of what's going on. They'll wonder: What instruments are reliable, and which can't be trusted? What's the most pressing threat? What's going on? Unfortunately, the vast majority of pilots will have little experience in finding the answers."
For more details: READ this article

Source: IBO