Free beta access. 5 free AI tokens each week. Optional top-ups available. See what is included
Opening Word document…

AI used to exacerbate racism

Schneier 16 Bruce Schneier, 02-08-2016, " Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World" W. W. Norton & Company, https://ciberativismoeguerra.files.wordpress.com/2017/09/bruce-schneier-data-and-goliath_-2015.pdf pg 780[AJL]

In a fundamental way, companies use surveillance data to discriminate. They place people into different categories and market goods and services to them differently on the basis of those categories. “Redlining” is a term from the 1960s to describe a practice that’s much older: banks discriminating against members of minority groups when they tried to purchase homes. Banks would not approve mortgages in minority neighborhoods—they would draw a red line on their maps delineating those zones. Or they would issue mortgages to minorities only if they were buying houses in predominantly minority neighborhoods. It’s illegal, of course, but for a long time banks got away with it. More generally, redlining is the practice of denying or charging more for services by using neighborhood as a proxy for race—and it’s much easier to do on the Internet. In 2000, Wells Fargo bank created a website to promote its home mortgages. The site featured a “community calculator” to help potential buyers search for neighborhoods. The calculator collected the current ZIP code of the potential customers and steered them to neighborhoods based on the predominant race of that ZIP code. The site referred white residents to white neighborhoods, and black residents to black neighborhoods. This practice is called weblining, and it has the potential to be much more pervasive and much more discriminatory than traditional redlining. Because corporations collect so much data about us and can compile such detailed profiles, they can influence us in many different ways. A 2014 White House report on big data concluded, “… big data analytics have the potential to eclipse longstanding civil rights protections in how personal information is used in housing, credit, employment, health, education, and the marketplace.” I think the report understated the risk. Price discrimination is also a big deal these days. It’s not discrimination in the same classic racial or gender sense as weblining; it’s companies charging different people different prices to realize as much profit as possible. We’re most familiar with this concept with respect to airline tickets. Prices change all the time, and depend on factors like how far in advance we purchase, what days we’re traveling, and how full the flight is. The airline’s goal is to sell tickets to vacationers at the bargain prices they’re willing to pay, while at the same time extracting from business travelers the much higher amounts that they’re willing to pay. There is nothing nefarious about the practice; it’s just a way of maximizing revenues and profits. Even so, price discrimination can be very unpopular. Raising the price of snow shovels after a snowstorm, for example, is considered pricegouging. This is why it is often cloaked in things like special offers, coupons, or rebates. Some types of price discrimination are illegal. For example, a restaurant cannot charge different prices depending on the gender or race of the customer. But it can charge different prices based on time of day, which is why you see lunch and dinner menus with the same items and different prices. Offering senior discounts and special children’s menus is legal price discrimination. Uber’s surge pricing is also legal. In many industries, the options you’re offered, the price you pay, and the service you receive depend on information about you: bank loans, auto insurance, credit cards, and so on. Internet surveillance facilitates a fine-tuning of this practice. Online merchants already show you different prices and options based on your history and what they know about you. Depending on who you are, you might see a picture of a red convertible or a picture of a minivan in online car ads, and be offered different options for financing and discounting when you visit dealer websites. According to a 2010 Wall Street Journal article, the price you pay on the Staples website depends on where you are located, and how close a competitor’s store is to you. The article states that other companies, like Rosetta Stone and Home Depot, are also adjusting prices on the basis of information about the individual user. More broadly, we all have a customer score. Data brokers assign it to us. It’s like a credit score, but it’s not a single number, and it’s focused on what you buy, based on things like purchasing data from retailers, personal financial information, survey data, warranty card registrations, social media interactions, loyalty card data, public records, website interactions, charity donor lists, online and offline subscriptions, and health and fitness information. All of this is used to determine what ads and offers you see when you browse the Internet. In 2011, the US Army created a series of recruiting ads showing soldiers of different genders and racial backgrounds. It partnered with a cable company to deliver those ads according to the demographics of the people living in the house.

Sign in to recut, update, or save this card.