Exploring how humans do, scientists measure and computers compute talking since 2018
I am a social scientist working in Conversational AI. My work focuses on understanding how humans interact with voice technologies in the real world. I aspire to produce useful insights for anyone who designs, builds, uses or regulates talking machines. Late naturalist under pressure. “You must collect things for reasons you don’t yet understand.” — Daniel J. Boorstin
My research interests originate in the study of human-human conversation and story-telling based on video-recordings. Then I became interested in how people aspire to build voice bots that can participate in these activities. What theoretical and computational tools would we need to build voice user interfaces that can hold up their end of conversations?
I am also interested in speech processing of speaker traits and various aspects of social intelligence such as turn-taking coordination, stance, rapport, and humor. What tools, pipelines, and representations would enable us to build voice bots that interact with humans in more useful ways? How can we ensure that this pervasive technology is used for good?
I hope my work will contribute to raise awareness about societal issues related to the rise of voice tech. Let’s democratize the field and make the technologies more accessible.
I also work as a conversation design consultant in the Greater Bay Area and currently write a book chapter on crowd-sourcing in voice technology.
PhD in Conversation Analysis, 2019
Nanyang Technological University, Singapore
We present a modified approach to behavioural profiling that includes a range of frame semantic features that aims to better capture variation of slot fillers of the Mandarin causative construction with rang (讓), shi (使) and ling (令) and their relation to clause structure.
This paper examines gender and age salience and (stereo)typicality in British English talk with the aim to predict gender and age categories based on lexical, phrasal and turntaking features.
Looking back at 70 years of work on talking machines. This is a literature review on the prospects of automating talk now and then.
We compile a collection of existing literature in the tradition of Ethnomethodology and Conversation Analysis on the topic of Artifical Intelligence.
Combining video-based analysis and software development, I explore how current NLU technology fares in real-world contexts.
A series of studies on how to build data-driven pipelines for frame semantics as well as their applications in dialog systems.