【Taste of Future Sister-in-law】

Google's code of conduct explicitly prohibits discrimination based on Taste of Future Sister-in-lawsexual orientation, race, religion, and a host of other protected categories. However, it seems that no one bothered to pass that information along to the company's artificial intelligence.

The Mountain View-based company developed what it's calling a Cloud Natural Language API, which is just a fancy term for an API that grants customers access to a machine-learning powered language analyzer which allegedly "reveals the structure and meaning of text." There's just one big, glaring problem: The system exhibits all kinds of bias.

SEE ALSO: The text of that Google employee's manifesto is just like every other MRA rant

First reported by Motherboard, the so-called "Sentiment Analysis" offered by Google is pitched to companies as a way to better understand what people really think about them. But in order to do so, the system must first assign positive and negative values to certain words and phrases. Can you see where this is going?

The system ranks the sentiment of text on a -1.0 to 1.0 scale, with -1.0 being "very negative" and 1.0 being "very positive." On a test page, inputting a phrase and clicking "analyze" kicks you back a rating.

"You can use it to extract information about people, places, events and much more, mentioned in text documents, news articles or blog posts," reads Google's page. "You can use it to understand sentiment about your product on social media or parse intent from customer conversations happening in a call center or a messaging app."

Both "I'm a homosexual" and "I'm queer" returned negative ratings (-0.5 and -0.1, respectively), while "I'm straight" returned a positive score (0.1).

Mashable Trend Report Decode what’s viral, what’s next, and what it all means. Sign up for Mashable’s weekly Trend Report newsletter. By clicking Sign Me Up, you confirm you are 16+ and agree to our Terms of Use and Privacy Policy. Thanks for signing up!

Original image has been replaced. Credit: Mashable

And it doesn't stop there, "I'm a jew" and "I'm black" returned scores of -0.1.

Original image has been replaced. Credit: Mashable

Interestingly, shortly after Motherboardpublished their story, some results changed. A search for "I'm black" now returns a neutral 0.0 score, for example, while "I'm a jew" actually returns a score of -0.2 (i.e., even worse than before).

"White power," meanwhile, is given a neutral score of 0.0.

Original image has been replaced. Credit: Mashable

So what's going on here? Essentially, it looks like Google's system picked up on existing biases in its training data and incorporated them into its readings. This is not a new problem, with an August study in the journal Sciencehighlighting this very issue.

We reached out to Google for comment, and the company both acknowledged the problem and promised to address the issue going forward.

"We dedicate a lot of efforts to making sure the NLP API avoids bias, but we don’t always get it right," a spokesperson wrote to Mashable. "This is an example of one of those times, and we are sorry. We take this seriously and are working on improving our models. We will correct this specific case, and, more broadly, building more inclusive algorithms is crucial to bringing the benefits of machine learning to everyone.”

So where does this leave us? If machine learning systems are only as good as the data they're trained on, and that data is biased, Silicon Valley needs to get much better about vetting what information we feed to the algorithms. Otherwise, we've simply managed to automate discrimination — which I'm pretty sure goes against the whole "don't be evil" thing.

This story has been updated to include a statement from Google.

---

Featured Video For You

---

Sorry, but you just can't erase yourself from the internet

---

• Entertainment
• Shopping
• Life
• Deals
• Science
• Games
• Social Good
• Digital Culture

• How This Long
• Equifax CEO 'retires' after massive data breach
• Amazon's Echo Connect brings phone calls to the Echo
• Comcast is launching a streaming video service for cord
• Wombat butt biting sex habits could be helpful for its survival
• Everything you need to know about the cartoon villain who is Roy Moore
• Leigh Bardugo talks 'The Language of Thorns' and the dark power of folktales
• Air France launches an 'airline for millennials' and it's beyond cringe
• Character AI reveals AvatarFX, a new AI video generator
• Star Trek: Discovery is forcing me to make a galactic decision

• How to Get into College, Indiana Jones Edition by Sadie Stein
• What We’re Loving: Saintly Comics, High Relief by Sadie Stein
• Oral Sadism and the Vegetarian Personality by Sadie Stein
• Kafka’s Mice, and Other News by Sadie Stein
• Free Verses by Dorian Rolston
• Bookless Libraries, and Other News by Sadie Stein
• Happy Birthday, Huck! by Sadie Stein
• Willem de Kooning, Untitled, 1970 by The Paris Review
• William Styron in Letters, Part 4 by William Styron
• Freedom and Light by Brian Cullman

• NYT mini crossword answers for April 24, 2025
• After destroying Puerto Rico, Hurricane Maria turned into even more of a dick
• Amazon is showing off a lot of new Alexa
• Dubai's autonomous flying taxi takes off for its first test flight
• SpaceX's BFR has a new name. Elon Musk is calling it Starship.
• Marine biologist captures 'blue hole' in the Great Barrier Reef
• Move over, ghosting. Submarining is the hot new way to be a jerk.
• Everything you need to know about the cartoon villain who is Roy Moore
• Best headphones deal: Save $150 on Beats Studio Pro
• Lyft and Ford to work together on self

• Elon Musk's space Tesla actually served an engineering purpose
• BBC presenter accidentally drops the c
• Report says credit cards used at Sonic restaurants are up for sale on the dark web
• Equifax CEO 'retires' after massive data breach
• Best gaming laptop deal: Save $400 on the HP Victus 15 with Ryzen 5 and Radeon RX 6550M
• Mark Zuckerberg, you dress like crap
• HGTV's 'Fixer Upper' is ending after five seasons
• Twitter's new 280
• Mac Mini M2 Desktop deal: Save $100 at Best Buy
• Everything you need to know about the cartoon villain who is Roy Moore