Yikes — Researchers Have Accidentally Been Making Their Software Sexist

Photo Credit: © Pavel Timofeev / Adobe Stock

By Alex Wilson

READ MORE: Sexual harassment, Feminism, Microsoft, Technology, Unconscious bias, Facebook

As our workplaces continue to increase their reliance on technology, artificial intelligence ensures that newer technologies keep evolving so they remain the best possible resource. But how do we make sure that technologies develop the right kind of perspective and that they avoid the unconscious biases humans have?

Computer science professors at the University of Virginia recently tested the existence of unconscious bias within software they were building. They taught machines using basic photo collections and quickly discovered that the materials they were using were inadvertently teaching machines sexist views of women.

Researchers found that major research-image collections — including one supported by Microsoft and Facebook — displayed a predictable gender bias.  As an example, these images associated photos of coaching with men, while women were tied to images of shopping and washing.

Professor Vicente Ordóñez, who spearheaded the study, told Wired how the software magnified its bias in other functions. “It would see a picture of a kitchen and more often than not associate it with women, not men,” he said. The software would recognize a photo of a person in a kitchen and assume that that person, just because they were in a kitchen, was a woman.

Ordóñez realized that the software didn’t develop its sexist views on its own; the biases displayed by the software were unconsciously injected by the researchers who built it and the data it learned from. Mark Yatskar, a researcher who also worked on the project, stressed that technological unconscious biases must be actively avoided.

“This could work to not only reinforce existing social biases,” he said. “But actually make them worse.”

To his point, machine-learning software didn’t just mirror existing biases; it amplified them. If the software analyzed a photo set that generally associated women with cooking, the software then created an even stronger association between the person and their environment.  As major companies rely on this software to accurately train consumer-facing tech on how to view people, the biases within the data are incredibly concerning.

“A system that takes action that can be clearly attributed to gender bias cannot effectively function with people,” Yatskar said.

Fortunately, these biases can be addressed. Researchers can prevent (and de-program) unconscious biases, but in order to do so they must actively seek out specific, shared prejudices within the software.  This neutralizes the bias, but as larger tech companies like Microsoft have shown, it is a Herculean task. 

“I and Microsoft as a whole celebrate efforts identifying and addressing bias and gaps in data sets and systems created out of them,” Eric Horvitz, director of Microsoft Research told Wired. In response to this, Horvitz's team has developed an ethics code for all of its consumer-facing technology. If the technology doesn't meet those standards, it does not move further in development.

If this strategy sounds vaguely similar to your company’s diversity training program, that’s because it is. Diversity training programs require employees to undergo a lot of self-analysis to determine what their biases are. When aiming to eliminate unconscious bias in machinery, researchers must do the same thing.

Sheryl Sandberg, Facebook COO and author of “Lean In,” acknowledges that technology used as the foundation for consumer products needs to be held to a higher standard. “At Facebook, I think about the role marketing plays in all this, because marketing is both reflective of our stereotypes and reinforces stereotypes,” she told The New York Times. “Do we partner into sexism or do we partner against sexism?”

Sandberg’s decision to partner against sexism is one of the reasons her nonprofit, Lean In, partnered with Getty Images to create the “Lean In Collection” — a series of stock photos that feature diverse women in a multitude of different careers.

“You can’t be what you can’t see,” Sandberg said in reference to the photo collection.

Sandberg’s steps forward are great steps for the immediate future, but ensuring that the data used to train new technology remains bias free remains a pressing issue.  If replicated in larger products, these biases could create erroneous digital ideas about women and eliminate much of the progress we have made.

 

 

Related Articles
Related Community Discussions
Like so many women in tech, I work with a

Like so many women in tech, I work with a LOT of men. I often feel like the outsider, and they're definitely a bit vulgar. I feel like they collaborate with me fine for the most part and handle our interactions professionally. Still, I feel really isolated - and it makes me not want to go to work some time. Does anyone else feel that way? Any ideas on how to make this situation work better?

Widespread sexual harassment in the workplace

I've been reading about the Weinstein sexual harassment allegations and I'm disgusted – yet not totally surprised. It seems like most of my female friends have, at some point, faced sexual harassment at work. Has anyone else been in a situation where they think there's a culture of sexual harassment at their company or in their industry at large? If so, do you have any ideas for how to address it?

Office Mean Girls

I'm working in an office with three other women. They speak two languages, and often communicate in the language of their nationality (I work abroad). I have recently received a promotion and now feel that the Mean Girl environment is getting worse. They exclude me from their conversations and inside jokes, and I feel that it is mainly directed by one jealous woman in particular. She envies the fact that I have gotten a promotion and knows how to push my buttons by being passive aggressively damaging. Basically, she's a trigger. How do I focus on my work, which is more important than ever, without being distracted or sidetracked by this Mean Girl behavior? How can I dissipate it, if at all?

Am I Overreacting?

I work in a very small firm (12 people). Most of us are sharing offices because space is limited. We recently bought a much larger company (60 employees) and the work environment is rapidly changing. I'm the office manager, so a lot of the changes are being coordinated by me, my boss, and the office manager of the other company. A three-way call recently required privacy that our shared office space would not allow for. I ended up in the passenger seat of my boss' pickup while he phoned the other office manager. After the phone call, I briefly mentioned that working with the new manager would be a difficult change. My boss proceeded to tell me that the new manager wouldn't have had the courage to have a phone call in his truck, but it was okay for us, because nothing would ever happen between the two of us because I'm not my boss' 'type.' He told me that he wasn't interested in dating 'little kids.' I'm HAPPILY MARRIED and almost 30. This is the first time in two years of working for him that anything like this has ever come up. Should I be concerned?

Find Out

What are women saying about your company?

Popular Articles
Related Community Discussions
Like so many women in tech, I work with a

Like so many women in tech, I work with a LOT of men. I often feel like the outsider, and they're definitely a bit vulgar. I feel like they collaborate with me fine for the most part and handle our interactions professionally. Still, I feel really isolated - and it makes me not want to go to work some time. Does anyone else feel that way? Any ideas on how to make this situation work better?

Widespread sexual harassment in the workplace

I've been reading about the Weinstein sexual harassment allegations and I'm disgusted – yet not totally surprised. It seems like most of my female friends have, at some point, faced sexual harassment at work. Has anyone else been in a situation where they think there's a culture of sexual harassment at their company or in their industry at large? If so, do you have any ideas for how to address it?

Office Mean Girls

I'm working in an office with three other women. They speak two languages, and often communicate in the language of their nationality (I work abroad). I have recently received a promotion and now feel that the Mean Girl environment is getting worse. They exclude me from their conversations and inside jokes, and I feel that it is mainly directed by one jealous woman in particular. She envies the fact that I have gotten a promotion and knows how to push my buttons by being passive aggressively damaging. Basically, she's a trigger. How do I focus on my work, which is more important than ever, without being distracted or sidetracked by this Mean Girl behavior? How can I dissipate it, if at all?

Am I Overreacting?

I work in a very small firm (12 people). Most of us are sharing offices because space is limited. We recently bought a much larger company (60 employees) and the work environment is rapidly changing. I'm the office manager, so a lot of the changes are being coordinated by me, my boss, and the office manager of the other company. A three-way call recently required privacy that our shared office space would not allow for. I ended up in the passenger seat of my boss' pickup while he phoned the other office manager. After the phone call, I briefly mentioned that working with the new manager would be a difficult change. My boss proceeded to tell me that the new manager wouldn't have had the courage to have a phone call in his truck, but it was okay for us, because nothing would ever happen between the two of us because I'm not my boss' 'type.' He told me that he wasn't interested in dating 'little kids.' I'm HAPPILY MARRIED and almost 30. This is the first time in two years of working for him that anything like this has ever come up. Should I be concerned?