Science

Can AI detect homosexuality from a facial image? And should it?

A controversial study claims a person's sexual orientation can be identified with a high degree of accuracy from a single facial image
A controversial study claims a person's sexual orientation can be identified with a high degree of accuracy from a single facial image
View 4 Images
Some of the facial landmarks tracked by the neural network
1/4
Some of the facial landmarks tracked by the neural network
One the left is are male and female composite images of what the algorithm determined to be heterosexual while on the right are the male/female homosexual composite images
2/4
One the left is are male and female composite images of what the algorithm determined to be heterosexual while on the right are the male/female homosexual composite images
Image from a 19th century book on physiognomy 
3/4
Image from a 19th century book on physiognomy 
A controversial study claims a person's sexual orientation can be identified with a high degree of accuracy from a single facial image
4/4
A controversial study claims a person's sexual orientation can be identified with a high degree of accuracy from a single facial image

A study published late last week from two Stanford researchers has caused shockwaves around the world. The duo reportedly developed a neural network that could detect the sexual orientation of a person just by studying a single facial image. The startling degree of accuracy achieved by the algorithm has been questioned by some and accused as dangerous by others.

In the controversial study, the researchers trained a neural network on over 35,000 facial images, evenly split between gay and heterosexual. The algorithm tracked facial features that were not only transient (i.e grooming or expressions), but also examined traits that were fixed and perhaps genetically or hormonally founded.

One of the key hinges of the research was a reliance on what is called the prenatal hormone theory (PHT) of sexual orientation. The idea is that either over- or under-exposure in the womb to the key androgens responsible for sexual differentiation is a major driver for same-sex orientation in later life. The research also notes that these specific androgens affect key features of the face, meaning that certain facial characteristics can be linked to sexual orientation.

"Typically, [heterosexual] men have larger jaws, shorter noses, and smaller foreheads. Gay men, however, tended to have narrower jaws, longer noses, larger foreheads, and less facial hair," writes one of the authors of the study Michal Kosinski, in notes recently published defending his work. "Conversely, lesbians tended to have more masculine faces (larger jaws and smaller foreheads) than heterosexual women."

When presented with randomly selected pairs of images, featuring one homosexual man and one heterosexual man, the machine accurately picked the sexual orientation of each subject 81 percent of the time. This rose to 91 percent accuracy when presented with five images of the same person. For women the prediction rate was lower with a base of 71 percent accuracy from one image, rising to 83 percent with five images.

Some of the facial landmarks tracked by the neural network
Some of the facial landmarks tracked by the neural network

The authors of the study seemed to understand the problematic nature of releasing research such as this. In their extensive additional notes they claim the study is intended to make people aware of the potential powers that current visual algorithmic tools have achieved.

"We were really disturbed by these results and spent much time considering whether they should be made public at all," writes Kosinski. "We did not want to enable the very risks that we are warning against."

LGBTQ advocacy organization GLADD and the Human Rights Campaign (HRC) immediately went on the offensive following the study's public outing. Questions were raised over the study's methodology, including the limited scope of the data – all subjects were white, images were plucked from US dating sites, and no distinctions were made between sexual orientation, sexual activity or bisexual identity.

"Technology cannot identify someone's sexual orientation," says GLADD's Chief Digital Officer, Jim Halloran. "What their technology can recognize is a pattern that found a small subset of out white gay and lesbian people on dating sites who look similar. Those two findings should not be conflated."

One the left is are male and female composite images of what the algorithm determined to be heterosexual while on the right are the male/female homosexual composite images
One the left is are male and female composite images of what the algorithm determined to be heterosexual while on the right are the male/female homosexual composite images

The consequences of this kind of research becoming officially applied for nefarious means has concerned many. Just earlier this year horrific stories started to come out of the conservative Russian republic Chechnya. Gay men were apparently being rounded up by authorities, sent to prison camps, beaten and sometimes killed.

Regardless of the accuracy of the actual science at hand, questions of moral responsibility were being asked of the researchers. Why was this even being studied in the first place? Simply to prove a point?

"Imagine for a moment the potential consequences if this flawed research were used to support a brutal regime's efforts to identify and/or persecute people they believed to be gay," says HRC Director of Public Education and Research, Ashland Johnson. "Stanford should distance itself from such junk science rather than lending its name and credibility to research that is dangerously flawed and leaves the world – and this case, millions of people's lives – worse and less safe than before."

There has undeniably been a recurrence in the previously derided field of physiognomy. This idea, circulating for hundreds of years, is that personality traits and characteristics can be identified through the study of physical features. The theory, alongside phrenology, was often used to justify horrifically racist or prejudicial behavior, under the veil of objective science.

Image from a 19th century book on physiognomy 
Image from a 19th century book on physiognomy 

More recently, the field has undergone a small revival underpinned by new genetic understandings and computational advances. In publishing the paper Kosinski defends some of the classic physiognomist claims by making the distinction that, while historically people were not successful at accurately judging others based just on facial features, this doesn't mean that those physical traits were not actually present. The implication is that we simply didn't have the technology to accurately process such micro facial distinctions.

"The fact that physiognomists were wrong about many things does not automatically invalidate all of their claims," writes Kosinski. "The same studies that prove that people cannot accurately do what physiognomists claimed was possible consistently show that they were, nevertheless, better than chance. Thus, physiognomists' main claim – that the character is to some extent displayed on one's face – seems to be correct (while being rather upsetting)."

The science is certainly not as clear cut as Kosinski and his research partner are implying. PHT is generally considered just one of many possible factors that could determine a person's ultimate sexual orientation, and hinging an entire facial recognition system on it is questionable to say the least.

The other, less fixed measures that are tracked by the algorithm seem rather arbitrary and significantly culturally mandated. One note in the study claims that according to the research, lesbians smiled less than their heterosexual counterparts, had darker hair and used less eye makeup.

But regardless of the debatable science, this study raises a significant question about the moral responsibility of the scientist. Is it enough to simply say, as the researchers literally have, "we studied an existing technology – one widely used by companies and governments – to understand the privacy risks it poses."

Is that enough of a justifiable outcome for research that could have incredibly damaging pragmatic results? After all, the study presupposes its underlying assumption that sexual orientation can be clearly tracked through facial characteristics. That presupposition is implied as not even up for debate in the study. That in itself is a dangerous, and potentially irresponsible outcome.

The study was peer-reviewed and is set to be published in the Journal of Personality and Social Psychology.

12 comments
highlandboy
Regardless of whether you approve or disapprove, we have allready seen that the USA Government can, and does, use IT to invade the privacy of individuals, both foreign and domestic, without any process of law. "Secret" science is unhealthy, society needs to know whether something can be done (without necessarily knowing how it is done), in order for healthy debate to occure as to whether it should be done. The power of the weak, poor, marginalised and unrepresented is in numbers. Democracy can only flourish when disscussion is uncensored.
Brian M
Ignoring the dubious/moral use of such as technique, humans have always been able to get at least a hint of someone's sexuality, even the name 'gaydar' has been coined for it. The ability to be able to identify a member of your own group (whatever that is) is probably an important advantage, so can understand why it can be picked up by AI techniques.
owlbeyou
H-boy is correct. In spite of the risk of being branded as homophobes, Kosinski et al feel that an unbiased and scientific objective is the priority here. Technology can and does easily get ahead of us, sometimes to our detriment. Anyone who wants to live under the scrutiny of machines is probably mad or misinformed. If anything, this demonstrates how "mad" this so-called security establishment of the West can be. Conversely, this cultivation of hyper sexuality of everything is not desirable to other cultures. We should not be dictating to them what is moral and what is not, especially as we are not models of morality and virtue ourselves. The West can evolve as it has, but to think that that it can go around and impose its standards on other cultures that are far older is questionable. We celebrate diversity and tolerance and we don't.
MerlinGuy
Interesting conundrum. If an AI can be programmed to correctly identify sexual orientation then it will be built. If a society is open and accepting about a persons sexuality then there really is no danger in the AI. However, that society is only one which would respect people enough to ban the software. If the assumption that sexual orientation has an identifiable medical cause is much more problematic that an AI that can identify sexuality. If there is a medical cause then that discredits the idea a person can choose to be gay. But also it would just be a matter of time before some medical treatment is available to allow parents to choose their child's orientation. Quite the conundrum.
AryehZelasko
I don't know why this is so surprising. I remember reading about a study where people we asked to guess the orientation of someone from a photo and they also had an equally high correct rate. The question of should this technology be used is naive. Should a dozen governments have satellites in orbit with cameras so powerful they can read the licence plate number of a car? Should the NSA have the equipment to monitor every form of electronic activity that occurs on the planet? That is what happens with technology, it is used. This will also be used. What is important to a free society is controlling its use so it does not deprive us of our human and civil rights. Getting those controls in place is much harder than inventing the tech.
Alexaqua
Whilst I agree with the comments about government probably just using this type of technology without our consent I also think that maybe it could be helpful. Many people who are against homosexuals will generally say it is a choice that has been made. Wouldn't this type of technology prove that homosexuality isn't a choice? As a gay male I can't possibly be making changes to my face as I "chose"to be gay. The down side is that no matter how much education is out there about a subject, there will always be people who don't believe the proof and of course this technology could be used to, as the story states, round up gay people to beat and kill them.
Grunchy
This is similar to detecting a grow op in a house merely by looking at it with an infrared camera and seeing a heat signature indicative of indoor hydroponics. Or of detecting marijuana crops by flying over remote land and looking for telltale colors and patterns. Or of buying a parabolic microphone and listening in to 'private' conversations. Or someone has a tattoo of something on their arm that you inadvertently read. Or a woman wears a low-cut top that you're not allowed to look at. There are countless ways of breaching someone's privacy by passively observing them. Google Earth drove past and photographed you or your license plate by accident and broadcast it on the internet. Etc!
JamesDemello
It is technology that will be used. At least we are aware of it. Hiding your head in the sand isn't a valid strategy for dealing with it.
guzmanchinky
Is the tech the problem or how we treat gay people? I've had a chin implant and rhinoplasty, how does it see me, I wonder?
robert14
It has been over a hundred years that lie detectors have been hawked and used and yet they are scarcely more reliable than chance.This is bs.not science.