Abstract

Wearable devices and affective computing have gained popularity in the recent times. Egocentric videos recorded using these devices can be used to understand the emotions of the camera wearer and the person interacting with the camera wearer. Emotions affect the facial expression, head movement and various other physiological factors. In order to perform this study we collected dyadic conversations (dialogues between two people) data from two different groups; one where two individuals agree on certain topic and second where two individuals disagree on certain topics. This data was collected using a wearable smart glass for video collection and a smart wristband for physiological data collection. Building this unique dataset was one of the significant contributions of this study. Using this data we extracted various features that include Galvanic Skin Response (GSR) data, facial expressions and 3D motion of a camera within an environment which is termed as Egomotion. We built two different machine learning models to model this data. In the first approach we use an application of Bayesian Hidden Markov model for classifying these individual videos from the paired conversations. In the second approach we use a Random Forest classifier to classify the data based on the Dynamic Time Warping data between the paired videos and individual average data for all the features in individual videos.The study found that in the presence of the limited data used in this work, individual behaviors were slightly more indicative of the type of discussion (85.43% accuracy) than the coupled behaviors (83.33% accuracy).

Library of Congress Subject Headings

Conversation analysis; Video recordings--Data processing; Hidden Markov models; Optical pattern recognition; Computer vision; Context-aware computing

Publication Date

7-2020

Document Type

Thesis

Student Type

Graduate

Degree Name

Computer Science (MS)

Department, Program, or Center

Computer Science (GCCIS)

Advisor

Zack Butler

Advisor/Committee Member

Ifeoma Nwogu

Advisor/Committee Member

Xumin Liu

Campus

RIT – Main Campus

Plan Codes

COMPSCI-MS

Download

Share

COinS