Social signal processing
Nonverbal behaviors such as gestures and facial expressions are unconsciously expressed during conversations, and they function as social signals in communication. Social signal processing (SSP) is a research field of artificial intelligence that analyzes and learns models for social signals in human-to-human and human-to-agent/robot communications. Applying multimodal machine learning approach to social signals, we are developing methods for estimating the characteristics of conversation participants, such as conversational engagement, leadership and communication skills in group discussions, and detecting important utterances in discussions.
Human-Agent Interaction
In order to realize virtual agents and robots that communicate with humans, it is necessary to generate both verbal and nonverbal behaviors of the humanoids. In particular, nonverbal behavior generation includes producing various types of behaviors such as gestures, facial expressions, and postures. We are developing technologies for generating nonverbal behaviors based on the content and the context of the dialogue and the characteristics of the speaker. We are also developing multimodal dialogue systems such as listener agents and customer service robots in robot cafes.
Communication support technologies
As an application of social signal processing and conversational agent technology, we are developing communication support technologies. We have developed systems that visualize characteristics of communication and social signals, and have been investigating the effects of intervening into human conversations by computer systems.