Thirty-five years ago a paper was presented at a conference, titled Ironies of Automation, by Lisanne Bainbridge. It included many insightful ideas:
The designer’s view of the human operator may be that the operator is unreliable and inefficient… so should be eliminated from the system. There are two ironies of this attitude. One is that designer errors can be a major source of operating problems… The second irony is that the designer who tries to eliminate the operator still leaves the operator to do the tasks which the designer cannot think how to automate…it means that the operator can be left with an arbitrary collection of tasks, and little thought may have been given to providing support for them.
Physical skills deteriorate when they are not used, particularly the refinements of gain and timing. This means that a formerly experienced operator who has been monitoring an automated process may now be an inexperienced one. If he takes over he may set the process into oscillation…
When manual takeover is needed there is likely to be something wrong with the process, so that unusual actions will be needed to control it, and one can argue that the operator needs to be more rather than less skilled, and less rather than more loaded, than average…
If the human operator is not involved in on-line control he will not have detailed knowledge of the current state of the system. One can ask what limitations this places on the possibility for effective manual takeover, whether for stabilization or shutdown of the process, or for fault diagnosis…
Perhaps the final irony is that it is the most successful automated systems, with rare need for manual intervention, which may need the greatest investment in human operator training.
Driving my new Tesla Model 3 in autopilot mode brings all these ideas to the immediate now. When the automated steering dings and gives up, I have to take over right now, mind 0 to 60, and do something a bit tricky. Lanes merging, lines obscured, or somesuch.
It’s clearly the same in flying. Manual takeover when the automation can’t handle something is very different than happily knowing and agreeing with what’s happening and clicking off the autopilot at your choosing. I wonder if we train enough for the unhappy unchosen manual interventions?
Other than the repeated instructions to hand-fly more, what else can we do as pilots to avoid the ironies of automation?
BONUS SING-ALONG SECTION:
Mr. Play It Safe was afraid to fly
He packed his suitcase and kissed his kids goodbye
He waited his whole damn life to take that flight
And as the plane crashed down he thought
“Well isn’t this nice…”
And isn’t it ironic… don’t you thinkIronic
Alanis Morissette, 1996
The ironies of automation paper is much older than the Alanis Morissette song. Both are still talked about today. Like this paper: The ironies of automation … still going strong at 30? And the ongoing debate about how ironic her lyrics were: Alanis Morissette recognizes it’s not ironic.