One Month, 500,000 Face Scans: How China Is Using A.I. to Profile a Minority
New York Times, 14 April 2019
By Paul Mozur The Chinese government has drawn wide international condemnationfor its harsh crackdown on ethnic Muslims in its western region, including holding as many as a million of them in detention camps.
Now, documents and interviews show that the authorities are also using a vast, secret system of advanced facial recognition technology to track and control the Uighurs, a largely Muslim minority. It is the first known example of a government intentionally using artificial intelligence for racial profiling, experts said.
The facial recognition technology, which is integrated into China’s rapidly expanding networks of surveillance cameras, looks exclusively for Uighurs based on their appearance and keeps records of their comings and goings for search and review. The practice makes China a pioneer in applying next-generation technology to watch its people, potentially ushering in a new era of automated racism.
The technology and its use to keep tabs on China’s 11 million Uighurs were described by five people with direct knowledge of the systems, who requested anonymity because they feared retribution. The New York Times also reviewed databases used by the police, government procurement documents and advertising materials distributed by the A.I. companies that make the systems.
Chinese authorities already maintain a vast surveillance net, including tracking people’s DNA, in the western region of Xinjiang, which many Uighurs call home. But the scope of the new systems, previously unreported, extends that monitoring into many other corners of the country.
The police are now using facial recognition technology to target Uighurs in wealthy eastern cities like Hangzhou and Wenzhou and across the coastal province of Fujian, said two of the people. Law enforcement in the central Chinese city of Sanmenxia, along the Yellow River, ran a system that over the course of a month this year screened whether residents were Uighurs 500,000 times.
Police documents show demand for such capabilities is spreading. Almost two dozen police departments in 16 different provinces and regions across China sought such technology beginning in 2018, according to procurement documents. Law enforcement from the central province of Shaanxi, for example, aimed to acquire a smart camera system last year that “should support facial recognition to identify Uighur/non-Uighur attributes.”
Some police departments and technology companies described the practice as “minority identification,” though three of the people said that phrase was a euphemism for a tool that sought to identify Uighurs exclusively. Uighurs often look distinct from China’s majority Han population, more closely resembling people from Central Asia. Such differences make it easier for software to single them out.
For decades, democracies have had a near monopoly on cutting-edge technology. Today, a new generation of start-ups catering to Beijing’s authoritarian needs are beginning to set the tone for emerging technologies like artificial intelligence. Similar tools could automate biases based on skin color and ethnicity elsewhere.
“Take the most risky application of this technology, and chances are good someone is going to try it,” said Clare Garvie, an associate at the Center on Privacy and Technology at Georgetown Law. “If you make a technology that can classify people by an ethnicity, someone will use it to repress that ethnicity.”
From a technology standpoint, using algorithms to label people based on race or ethnicity has become relatively easy. Companies like I.B.M. advertise software that can sort people into broad groups.
But China has broken new ground by identifying one ethnic group for law enforcement purposes. One Chinese start-up, CloudWalk, outlined a sample experience in marketing its own surveillance systems. The technology, it said, could recognize “sensitive groups of people.”
“If originally one Uighur lives in a neighborhood, and within 20 days six Uighurs appear,” it said on its website, “it immediately sends alarms” to law enforcement.
In practice, the systems are imperfect, two of the people said. Often, their accuracy depends on environmental factors like lighting and the positioning of cameras.
In the United States and Europe, the debate in the artificial intelligence community has focused on the unconscious biases of those designing the technology. Recent tests showed facial recognition systems made by companies like I.B.M. and Amazon were less accurate at identifying the features of darker-skinned people.
China’s efforts raise starker issues. While facial recognition technology uses aspects like skin tone and face shapes to sort images in photos or videos, it must be told by humans to categorize people based on social definitions of race or ethnicity. Chinese police, with the help of the start-ups, have done that.
“It’s something that seems shocking coming from the U.S., where there is most likely racism built into our algorithmic decision making, but not in an overt way like this,” said Jennifer Lynch, surveillance litigation director at the Electronic Frontier Foundation. “There’s not a system designed to identify someone as African-American, for example.”
The Chinese A.I. companies behind the software include Yitu, Megvii, SenseTime, and CloudWalk, which are each valued at more than $1 billion. Another company, Hikvision, that sells cameras and software to process the images, offered a minority recognition function, but began phasing it out in 2018, according to one of the people.
The companies’ valuations soared in 2018 as China’s Ministry of Public Security, its top police agency, set aside billions of dollars under two government plans, called Skynet and Sharp Eyes, to computerize surveillance, policing and intelligence collection.
In a statement, a SenseTime spokeswoman said she checked with “relevant teams,” who were not aware its technology was being used to profile. Megvii said in a statement it was focused on “commercial not political solutions,” adding, “we are concerned about the well-being and safety of individual citizens, not about monitoring groups.” CloudWalk and Yitu did not respond to requests for comment.
China’s Ministry of Public Security did not respond to a faxed request for comment.
Selling products with names like Fire Eye, Sky Eye and Dragonfly Eye, the start-ups promise to use A.I. to analyze footage from China’s surveillance cameras. The technology is not mature — in 2017 Yitu promoted a one-in-three success rate when the police responded to its alarms at a train station — and many of China’s cameras are not powerful enough for facial recognition software to work effectively.
Yet they help advance China’s architecture for social control. To make the algorithms work, the police have put together face-image databases for people with criminal records, mental illnesses, records of drug use, and those who petitioned the government over grievances, according to two of the people and procurement documents. A national database of criminals at large includes about 300,000 faces, while a list of people with a history of drug use in the city of Wenzhou totals 8,000 faces, they said.
Using a process called machine learning, engineers feed data to artificial intelligence systems to train them to recognize patterns or traits. In the case of the profiling, they would provide thousands of labeled images of both Uighurs and non-Uighurs. That would help generate a function to distinguish the ethnic group.
The A.I. companies have taken money from major investors. Fidelity International and Qualcomm Ventures were a part of a consortium that invested $620 million in SenseTime. Sequoia invested in Yitu. Megvii is backed by Sinovation Ventures, the fund of the well-known Chinese tech investor Kai-Fu Lee.
A Sinovation spokeswoman said the fund had recently sold a part of its stake in Megvii and relinquished its seat on the board. Fidelity declined to comment. Sequoia and Qualcomm did not respond to emailed requests for comment.
Mr. Lee, a booster of Chinese A.I., has argued that China has an advantage in developing A.I. because its leaders are less fussed by “legal intricacies” or “moral consensus.”
“We are not passive spectators in the story of A.I. — we are the authors of it,” Mr. Lee wrote last year. “That means the values underpinning our visions of an A.I. future could well become self-fulfilling prophecies.” He declined to comment on his fund’s investment in Megvii or its practices.
Ethnic profiling within China’s tech industry isn’t a secret, the people said. It has become so common that one of the people likened it to the short-range wireless technology Bluetooth. Employees at Megvii were warned about the sensitivity of discussing ethnic targeting publicly, another person said.
China has devoted major resources toward tracking Uighurs, citingethnic violence in Xinjiang and Uighur terrorist attacks elsewhere. Beijing has thrown hundreds of thousands of Uighurs and others in Xinjiang into re-education camps.
The software extends the state’s ability to label Uighurs to the rest of the country. One national database stores the faces of all Uighurs who leave Xinjiang, according to two of the people.