Health & Wellbeing

App screens for autism by reading facial expressions

App screens for autism by reading facial expressions
The app tracks key facial features and is capable of anonymizing participants
The app tracks key facial features and is capable of anonymizing participants
View 1 Image
The app tracks key facial features and is capable of anonymizing participants
1/1
The app tracks key facial features and is capable of anonymizing participants

Autism & Beyond is a new study of childhood mental health powered by Apple's ResearchKit. Intended to detect autism and other developmental challenges, app users can participate in the study which aims to provide new technological tools to help parents screen their children at home.

The study and the associated app are the product of a partnership between the Duke Medical Center and Duke University as well as technologies developed by Apple (ResearchKit) and the Office of Naval Research (ONR).

"We analyze the video to track position and movement of the head and face, including the lips, eyes and nose, all of which indicate emotions," says Dr. Guillermo Sapiro, a professor at Duke University, who developed the algorithm. "For example, while watching stimuli like a funny video, does the child smile, look toward the caregiver or ask the caregiver to view the video as well? We study all of that. Lack of emotion and social sharing are possible characteristics of childhood autism."

Sapiro stresses the app isn't a self-diagnosis resource, but is intended to serve as a potential screening tool for autism and other developmental challenges, and encourage users to contact a physician for specialized testing.

The key technology that makes this app possible, the development of which was sponsored by the ONR, is a complex mathematical algorithm that automatically maps key landmarks on children's faces and assesses emotional responses based on movements of facial muscles.

According to Sapiro, this same tech could be further developed to help screening for a wide array of other issues, including conditions suffered by soldiers such as PTSD, TBI and depression. It is hoped that improved analysis of such facial expressions could lead to more accurate diagnoses by doctors and, consequently, enhanced treatment options for military personnel and veterans.

For families wishing to participate in the study, the Autism & Beyond app can be downloaded from iTunes for free.

You can learn more about the Autism & Beyond application and study in the video below.

Sources: Duke University, Office of Naval Research, Apple Research Kit

About the Autism & Beyond Study

3 comments
3 comments
Daishi
Humans are masters at recognizing facial expressions and emotions compared to where computers are at. There as no way an app looking at peoples faces is going to be more accurate than a dart monkey.
It sounds like someone suckered the "Office of Naval Research" into writing a grant.
OliverStarr
Daishi, I think you're failing to recognize the value of what is being done here. Many medical technologies help screen for various pathologies. They don't deliver a definitive diagnosis but are intended to eliminate other possibilities. The app is simply a first step in developing the enormous libraries that catalog the vast variety in human expression so that it can compile massive amounts of reference data that will show commonalities among people impacted by various cognitive issues.
In particular, diagnosing autism is difficult and an early diagnosis can dramatically impact long term prognosis for many children. Anything that can help parents identify the traits that are common among autistic infants (or people with other problems) is certainly to be of value to the medical community and families everywhere.
In general, it's easy to dismiss the ability of computers to identify problems based upon facial gestures, but remember a decade ago the idea that real-time translation from one language to another seemed highly unlikely, yet most of us use Google Translate regularly today with reasonable to excellent results.
The reason it works now is because Google crowsourced a massive data analysis team (everyone using their originally crappy translation) that has continuously improved their tool. We should expect no less from the medical profession where the cost savings of simple early diagnostic tests can be astronomical when applied against and entire population.
Tommy Hickok
while i'm glad there is an app where is the android version to be-fair many parents with autistic kids are not going to run out and buy apple when you can buy a 10 smart phone for your kid whose going to break it over and over again (experienced autistic person and parent of autistic children) and how about parenting classes for observation purposes i dunno if i'm unique but you can tell when a kids going to have autism from day one hell i can walk around town and point them out even if their parents can't