Monitoring Emotions while Students Learn with AutoTutor, NSF Grant

Award Abstract #0325428 ITR: Monitoring Emotions while Students Learn with AutoTutor NSF Org: IIS Div Of Information & Intelligent Systems divider line Initial Amendment Date: September 12, 2003 divider line Latest Amendment Date: July 3, 2007 divider line Award Number: 0325428 divider line Award Instrument: Continuing grant divider line Program Manager: Kenneth Whang IIS Div Of Information & Intelligent Systems CSE Direct For Computer & Info Scie & Enginr divider line Start Date: September 15, 2003 divider line End Date: August 31, 2009 (Estimated) divider line Awarded Amount to Date: $1,256,000.00 divider line Investigator(s): Arthur Graesser a-graesser@memphis.edu (Principal Investigator) Stanley Franklin (Co-Principal Investigator) Robert Reilly (Co-Principal Investigator) Rosalind Picard (Co-Principal Investigator) Barry Kort (Co-Principal Investigator) divider line Sponsor: University of Memphis Administration 315 Memphis, TN 38152-3370 (901)678-3251 divider line NSF Program(s): ITR MEDIUM (GROUP) GRANTS divider line Program Reference Code(s): 1657, 9218, 9251, HPCC divider line Program Element Code(s): 1687 ABSTRACT This research investigates emotions during the process of learning and reasoning while college students interact with complex learning environments. College students learn about introductory computer literacy or conceptual physics on the web by an intelligent tutoring system, called AutoTutor. AutoTutor helps learners construct explanations that answer difficult questions by interacting with them in natural language and by helping them use simulation environments. AutoTutor has an animated conversational agent and a dialog management facility that attempts to comprehend the learner's contributions and to respond with appropriate dialog moves (short feedback, pumps, hints, prompts for information, assertions, answers to student questions, suggestions for actions, summaries). The emotions of the learner are monitored during this learning process by integrating state-of-the-art affect sensing technology with AutoTutor. Confusion, frustration, boredom, interest, excitement, and other learner emotions are classified on the basis of facial actions, body posture, pressure on the mouse, speech acts in dialog, mastery of the material, and the timing of interactions. One strand of research develops the affect-sensing technologies and tests their validity in classifying the learner emotions. A second line of research investigates whether learning gains and learner impressions are influenced by dialog moves of AutoTutor that are constrained by the learner's emotional state. This research will advance education and natural language dialog technologies through a system that promotes deep learning of material in a fashion that is sensitive to the learners' emotions. A learning environment that monitors learner emotions is likely to be more motivating and personally relevant to the learner. PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval). Some links on this page may take you to non-federal websites. Their policies may differ from this site. (Showing: 1 - 10 of 106) Show All Craig, S.D., Graesser, A.C., Sullins, J., & Gholson, B.. "Affect and learning: An exploratory look into the role of affect in learning.," Journal of Educational Media, v.29, 2004, p. 241. Graesser, A.C., Hu, X., Person, P.,Jackson, T., and Toth, J. Modules and information retrieval facilities of the Human Use Regulatory Affairs Advisor (HURAA).. "Modules and information retrieval facilities of the Human Use Regulatory Affairs Advisor (HURAA).," International Journal on eLearning, v.3, 204, p. 29. Graesser, A.C., Lu, S., Jackson, G.T., Mitchell, H., Ventura, M., Olney, A., & Louwerse, M.M.. "AutoTutor: A tutor with dialogue in natural language.," Behavioral Research Methods, Instruments, and Computers, v.36, 2004, p. 180. Hu, X., & Graesser, A.C.. "Human Use Regulatory Affairs Advisor (HURAA): Learning about research ethics with intelligent learning modules.," Behavioral Research Methods, Instruments, and Computers, v.36, 2004, p. 241. Song, K., Hu,X., Olney, A., & Graesser, A.C.. "A framework of synthesizing tutoring conversation capability with web based distance education courseware.," Computers & Education, v.42, 2004, p. 375. Penumatsa, P., Ventura, M., Graesser, A.C., Franceschetti, D.R., Louwerse, M., Hu, X., & Cai, Z.. "The right threshold value: What is the right threshold of cosine measure when using latent semantic analysis for evaluating student answers?," International Journal of Artificial Intelligence Tools, v.12, 2004, p. 257. Bickmore, T. & Picard R.. "Establishing and maintaining long-term human-computer relationships.," IEEE Transactions on Computer-Human Interaction, v.12, 2005, p. 293. Louwerse, M.M., Graesser, A.C., Lu, S., & Mitchell, H.H.. "Social cues in animated conversational agents.," Applied Cognitive Psychology, v.19, 2005, p. 693. Graesser, A.C.. "Views from a cognitive scientist: Cognitive representations underlying discourse are sometimes social.," Discourse Studies, v.8, 2006, p. 59. Graesser, A.C., Chipman, P., Haynes, B.C., & Olney, A.. "AutoTutor: An intelligent tutoring system with mixed-initiative dialogue.," IEEE Transactions in Education, v.48, 2005, p. 612. (Showing: 1 - 10 of 106) Show All CONFERENCE PROCEEDINGS PRODUCED AS A RESULT OF THIS RESEARCH (Showing: 1 - 10 of 14) Show All Jeuniaux, P; Louwerse, MM; Hu, XG. "The role of discourse structure and response time in Multimodal communication," in 6th International Conference on Intelligent Virtual Agents., v.4133, 2006, p. 459-460. View record at Web of Science Hoque, ME; Sorower, MS; Yeasin, M; Louwerse, MM. "What speech tells us about discourse: The role of prosodic and discourse features in speech act classification," in International Joint Conference on Neural Networks., 2007, p. 3004-3009. View record at Web of Science Hoque, ME; Yeasin, M; Louwerse, MM. "Robust recognition of emotion from speech," in 6th International Conference on Intelligent Virtual Agents., v.4133, 2006, p. 42-53. View record at Web of Science Rus, V; Cai, ZQ; Graesser, AC. "Experiments on generating questions about facts," in 8th International Conference on Intelligent Text Processing and Computational Linguistics., v.4394, 2007, p. 444-455. View record at Web of Science Rus, V; McCarthy, PM; Graesser, AC. "Analysis of a textual entailer," in 7th International Conference on Intelligent Text Processing and Computational Linguistics., v.3878, 2006, p. 287-298. View record at Web of Science Cade, WL; Copeland, JL; Person, NK; D'Mello, SK. "Dialogue modes in expert tutoring," in 9th International Conference on Intelligent Tutoring Systems., v.5091, 2008, p. 470-479. View record at Web of Science Chipman, P; Olney, A; Graesser, AC. "The autotutor 3 architecture a software architecture for an expandable, high-availability ITS," in 2nd International Conference on Web Information Systems and Technologies., v.1, 2007, p. 323-332. View record at Web of Science Graesser, AC; Person, N; Lu, ZJ; Jeon, MG; McDaniel, B. "Learning while holding a conversation with a computer," in 2nd Nebraska Symposium on Information Technology in Education., 2005, p. 143-167. View record at Web of Science Witherspoon, AM; Azevedo, R; D'Mello, S. "The dynamics of self-regulatory processes within self-and externally regulated learning episodes during complex science learning with hypermedia," in 9th International Conference on Intelligent Tutoring Systems., v.5091, 2008, p. 260-269. View record at Web of Science Picard, RW. "Future affective technology for autism and emotion communication," in Meeting on Computation of Emotions in Man and Machines., v.364, 2009, p. 3575-3584. View record at Web of Science (Showing: 1 - 10 of 14) Show All BOOKS/ONE TIME PROCEEDING (Showing: 1 - 10 of 171) Show All DiPaolo, R.E., Graesser, A.C., White, H.A., & Hacker, D.J.. "Hints in human and computer tutoring.", 09/01/2003-09/01/2004, , M. Rabinowitz"The design of instruction and evaluation", 2004, "Mahwah, NJ: Erlbaum.". Graesser, A.C., Hu, X., & McNamara,D.S.. "Computerized learning environments that incoporate research in discourse psychology, cognitive science, and computational linguistics.", 09/01/2003-09/01/2004, , A.F. Healy"Experimental Cognitive Psychology and its Applications: Festschrift in Honor of Lyle Bourne, Walter Kintsch, and Thomas Landauer.", 2005, "Washington, D.C.: American Psychological Association.". Hacker, D.J., & Graesser, A.C.. "The role of dialogue in reciprocal teaching and naturalistic tutoring.", 09/01/2003-09/01/2004, , R. Horowitz"Talk about text: Developing understanding about the world through talk and text.", 2005, "Publication of the International Reading Association.". Person, N.P., & Graesser, A.C.. "Pedagogical agents and tutors.", 09/01/2003-09/01/2004, , J.W. Guthrie"Encyclopedia of Education.", 2005, "New York: Macmillan.". Craig, S.D., & Graesser, A.. "Why am I confused: An exploratory look into the role of affect in learning.", 09/01/2003-09/01/2004, , A. Méndez-Vilas and J.A. Mesa González"Advances in Technology-based Education: Towards a Knowledge-based Society", 2003, "Badajoz, Spain: Junta de Extremadura.". Graesser, A.C.. "AutoTutor and other conversational agents that scaffold learning.", 09/01/2003-09/01/2004, , F. Schmalhofer, R.M. Young, and G. Katz"Proceedings of the European Cognitive Science Society", 2003, "Mahwah, NJ: Erlbaum". Kapoor, A., Qi, Y. & Picard, R.. "Fully automatic upper facial action recognition.", 09/01/2003-09/01/2004, "Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures.", 2003, "IEEE". Kort, B. and Reilly, R.. "Educational pedagogy, affective state, and learning.", 09/01/2003-09/01/2004, "Proceedings of the Society for Instructional Technology and Teacher Education.", "SITTE". DiPaolo, R.E., Graesser, A.C., White, H.A., & Hacker, D.J.. "Hints in human and computer tutoring.", 09/01/2004-09/01/2005, , M. Rabinowitz"The design of instruction and evaluation", 2004, "Mahwah, NJ: Erlbaum.". Graesser, A.C., Hu, X., & McNamara,D.S.. "Computerized learning environments that incoporate research in discourse psychology, cognitive science, and computational linguistics.", 09/01/2004-09/01/2005, , A.F. Healy"Experimental Cognitive Psychology and its Applications: Festschrift in Honor of Lyle Bourne, Walter Kintsch, and Thomas Landauer.", 2005, "Washington, D.C.: American Psychological Association.". (Showing: 1 - 10 of 171) Show All