Viewing and analyzing multimodal human-computer tutorial d–

Viewing and Analyzing Multimodal Human-computer Tutorial Dialogue: A Database Approach Jack Mostow, Joseph Beck, Raghu Chalasani, Andrew Cuneo, and Peng Jia 1Project LISTEN , 4213 RI-NSH, 5000 Forbes Ave, Carnegie Mellon University, Pittsburgh, USA Telephone: 412-268-1330 (voice) / 268-6436 (FAX); email: Mostow@cs.cmu.edu Abstract. It is easier to record logs of multimodal human-computer tutorial dialogue than to make sense of them. In the 2000-2001 school year, we logged the interactions of approximately 400 students who used the Reading Tutor and who read aloud over 2.4 million words. This paper discusses some of the problems in extracting useful information from such logs and the difficulties we encountered in converting the logs into a more easily understandable database. Once log files are parsed into a database, it is possible to write SQL queries to answer research questions faster than analyzing complex log files each time. The database permits us to construct a viewer to examine individual Reading Tutor-student interactions. This combination of queries and viewable data has turned out to be very powerful. We provide examples of questions that can be answered by each technique as well as how to use them together. Keywords: viewing and analyzing tutorial dialogue, session logs, evaluation tools, Project LISTEN, Reading Tutor, multimodal dialogue, speech I/O, session browser, database queries, perl, MySQL 1 ...