Abstract
Despite the thriving research in the field of accessibility, there are several niche area where there is a need for specific solutions. One such area is social interaction. Blind and Low-Vision people (BLV) are limited in identifying emotions through verbal communication alone, which is often biased and unreliable in certain environments. Prior research illustrates the importance of body language in expressions and articulates the mechanism behind human emotions. Drawing inspiratoin from such research and various studies that were focused on emotion recognition systems, our research focused on identifying a portable accessible solution for BLV that classified emotion based on multi-modal input such as facial expressions, body language and tone, with an audio feedback. 2 studies were conducted to understand the effectiveness of using an external aid in identifying emotions with 10 participants who identified as Low-Vision between the ages 31-45. Study one targeted at measuring the success of unassisted identification and Study 2 tested the effectiveness of prototype. BLV participants resonated with the purpose of the prototype and identified certain aspects that could be rendered in a different way. Although BLV were able to successfully detect certain emotions without assistance, most emotions, accompanied with background noise were harder to detect only based on tonality. Overall, the function of the prototype was identified to have a positive impact with minor limitations on the interface and feedback mechanics. As a part of our research we also proposed design considerations that can cater to the utility of such devices in multiple environments and thrive in conditions with multiple speakers. Further, we also proposed a co-design structure that can bestow ownership and enable active participation from BLV.
Library of Congress Subject Headings
Emotion recognition--Automation; Blind--Services for--Design; People with visual disabilities--Means of communication; Computer vision; Body language; Facial expression
Publication Date
12-2024
Document Type
Thesis
Student Type
Graduate
Department, Program, or Center
Information, School of
College
Golisano College of Computing and Information Sciences
Advisor
Tae Oh
Advisor/Committee Member
Roshan Peiris
Advisor/Committee Member
Bryan French
Recommended Citation
Krishna, Gautham, "Making Social Interactions Accessible for the Blind and Low-Vision People by Detecting Emotions through Vocal Tone, Facial Expressions, and Body Language" (2024). Thesis. Rochester Institute of Technology. Accessed from
https://repository.rit.edu/theses/11936
Campus
RIT – Main Campus
Plan Codes
HUMCOMP-MS
