Including the Human Factor in Dependability Benchmarks

Including the Human Factor in Dependability Benchmarks
Aaron B. Brown, Leonard C. Chung, and David A. Patterson
Computer Science Division, University of California at Berkeley
387 Soda Hall #1776, Berkeley, CA, 94720-1776
{abrown,leonardc,pattrsn}@cs.berkeley.edu
structure numerically confirm the significance of humanAbstract
error as a primary contributor to system failures [4] [12].
We describe the construction of a dependability benchmark Most existing work on dependability benchmarks has
that captures the impact of the human system operator on ignored the effects of human behavior, positive or nega-
the tested system. Our benchmark follows the usual model tive; this is unfortunate, but perhaps not surprising given
of injecting faults and perturbations into the tested system; that human behavior is typically studied by psychologists
however, our perturbations are generated by the unscripted or HCI specialists, not systems benchmarkers. In this
actions of actual human operators participating in the paper, we present our first steps at bringing consideration
benchmark procedure in addition to more traditional fault of human behavior into the dependability benchmarking
injection. We introduce the issues that arise as we attempt world, and describe our work-in-progress toward building
to incorporate human behavior into a dependability bench- a human-aware dependability benchmark. Although our
mark and describe the possible solutions that we have methodology begins with a ...