Selfie shots used to monitor mental health

University of Rochester computer scientists have developed innovative software that turns smartphones into mental health monitoring devices. Describing the project at this week’s American Association for Artificial Intelligence conference in Austin, Texas, researcher Jiebo Luo said the program analyzes “selfie” images and videos taken as the user engages with social media.

Rather than direct interaction with the software, Luo explained that his team’s approach is to “quietly observe your behavior” while using either a phone or computer. He adds that the program is; “unobtrusive; it does not require the user to explicitly state what he or she is feeling, input any extra information, or wear any special gear.”

The researchers are able to analyze the image data to extract a number of “clues,” such as heart rate (by monitoring very small, subtle changes in the user’s forehead color), blinking rate, eye pupil radius, and head movement rate. At the same time, the program also analyzed both what the users posted on Twitter, what they read, how fast they scrolled, their keystroke rate and their mouse click rate.

To calibrate the system, Luo and his colleagues enrolled a test group and; “sent them messages, real tweets, with sentiment to induce their emotion.” This allowed the researchers to gauge how subjects reacted after seeing or reading material considered to be positive or negative.

They then compared the combined monitoring with the users’ self reports about their feelings to find out how well the program actually performs. The combination of the data gathered by the program with the users’ self-reported state of mind (called the “ground truth”) allowed the researchers to train the system.

The program then begins to understand from just the data gathered whether the user is feeling positive, neutral or negative. Luo says that he hopes to add extra sensitivity to the program by teaching it to further define a negative emotion as, for example, sadness or anger.

Luo concedes that the program raises ethical concerns that need to be considered. Using this system means; “effectively giving this app permission to observe you constantly,” but adds that the program is designed for the use of the user only and does not share data with anyone else unless otherwise designated by the user.

Related:
Discuss this article in our forum
Scientists map mental illness’ effects on life expectancy
Can probiotics treat depression?
Software scans family photos to diagnose rare disorders
Decoded smartphone movements reveal transport mode

Source: University of Rochester

, , ,

Comments are closed.

1pc New Stainless 20oz Mental Health Matters Cute Retro Tumbler Cup picture

1pc New Stainless 20oz Mental Health Matters Cute Retro Tumbler Cup

$23.99



1pc New Stainless 20oz Mental Health Matters Cute Retro Tumbler Cup picture

1pc New Stainless 20oz Mental Health Matters Cute Retro Tumbler Cup

$23.99



Its OK not to be OK Mental Health Matters Awareness Enamel Pin Badge Brooch MIND picture

Its OK not to be OK Mental Health Matters Awareness Enamel Pin Badge Brooch MIND

$4.30



Mental Health Tapestry Small Feelings Chart Therapy Office Decor Social Emotiona picture

Mental Health Tapestry Small Feelings Chart Therapy Office Decor Social Emotiona

$23.74



Public Affairs Pamphlet No. 196 “Mental Health- Everybody’s Business” 1953 picture

Public Affairs Pamphlet No. 196 “Mental Health- Everybody’s Business” 1953

$8.99



Mental Health Awareness Green Ribbon Lapel Pin picture

Mental Health Awareness Green Ribbon Lapel Pin

$7.00



ANTIQUE NEW YORK DEPARTMENT  MENTAL HEALTH HYGIENE SERVICES MEDICAL  BADGE BX 16 picture

ANTIQUE NEW YORK DEPARTMENT MENTAL HEALTH HYGIENE SERVICES MEDICAL BADGE BX 16

$299.99



NEW YORK STATE NY MENTAL HEALTH EMS EMT Medical Services Patch Iron Sew On picture

NEW YORK STATE NY MENTAL HEALTH EMS EMT Medical Services Patch Iron Sew On

$6.45



Little Sams Art Mental Health Skull Limited Edition Pin Very Rare Nice Glow picture

Little Sams Art Mental Health Skull Limited Edition Pin Very Rare Nice Glow

$99.00



Little Sams Art Mental Health Skull Limited Edition Pin Very RareGREEN VARIANT picture

Little Sams Art Mental Health Skull Limited Edition Pin Very RareGREEN VARIANT

$99.00



Powered by WordPress. Designed by WooThemes