China is focusing on artificial intelligence to predict who might pose a political risk

ByTech Word News

A Chinese company is trying to develop artificial intelligence-powered technology that would allow authoritarian governments to not only track dissidents, but potentially predict who might become them in the future.

The work, which appears to be in the research phase, is ripped from dystopian science fiction and offers a glimpse of a world in which an authoritarian state is able to crack down on its citizens before any public dissent begins.

Chinese company Geedge Networks sells a commercial version of Great Firewall, the surveillance and censorship software China uses to control online activity. These tools allow governments to monitor Internet traffic and report when someone tries to circumvent traditional Internet censorship.

But according to leaked company documents, the firm is working on new products that use artificial intelligence to examine location and internet usage data to predict who might do or say something critical of the government. according to researchers at Vanderbilt University.

If perfected, such technology would provide authoritarian governments with a powerful tool to use against perceived enemies.

The idea of ​​an authoritarian government using artificial intelligence to suppress dissent is quite disturbing. But using artificial intelligence to predict dissent well in advance of human action has become a nightmare, according to some in the industry.

“This is what happens when mass surveillance meets AI,” said Brett J. Goldstein, director of the Wicked Problems Lab at the Vanderbilt National Security Institute. “Without checks and balances, what China is doing to its citizens is an example of what is possible wherever these tools are left unchecked.”

The Vanderbilt researchers found that Geedge, working with his government-backed research group MESA Lab, was developing technology that would generate profiles of Chinese citizens and then use AI to highlight who might pose a political risk.

But the company’s progress appears to have been hampered by Biden-era controls on exports of US-designed computer chips that power artificial intelligence. This suggests that US restrictions may have slowed China’s development of a new generation of surveillance technology.

Documents from Geedge show that in 2024, when strict US export controls were introduced in China, the company and its lab struggled to find enough computing power for surveillance technology.

Brett V. Benson, a professor of political science at Vanderbilt, said the documents refer to GPU restrictions. GPUs – graphics processing units – are the chips that power artificial intelligence models. As a result of the restrictions, he said, the company started using older, less powerful models and AI chips.

Today, Geedge has access to enough graphics processing units for its current products, according to US officials. But to carry out the most ambitious version of its predictive technology, the company would likely need more advanced chips than China can get, officials said.

Former Biden administration officials have said that intelligence has shown that its policy of restricting China from using high-end US-designed chips has worked to keep America ahead in artificial intelligence and to slow the development of other Chinese technologies.

But whether US export controls will slow China’s development of more punitive use of artificial intelligence remains an open question.

The Trump administration has loosened some Biden-era export controls, leaving restrictions on the most advanced processors from Nvidia, the US company that designs the world’s most powerful artificial intelligence chips. During President Trump’s recent trip to Beijing, US officials said China would have access to a more advanced version of Nvidia’s chips.

But China is trying to wean its AI companies off U.S.-designed chips so that export controls can no longer limit their ambitions.

The new information was gleaned from a trove of data, including 100,000 documents from Geedge that were originally released last September. Working with documents, Wired and other publications outlined how Geedge exported its network security software to countries including Ethiopia, Kazakhstan, Myanmar and Pakistan, allowing them to conduct mass surveillance of mobile networks.

Vanderbilt researchers sifting through the documents focused not on the technology Geedge was selling, but on the next generation of technology he was trying to develop, as well as the limitations that could prevent it from realizing the strategy.

In the early months of 2024, according to Geedge documents, the company’s researchers worked to develop profiles of people’s behavior based on telecommunications, social media and location data. Artificial intelligence models have been used to classify people and to “detect harmful information,” a euphemism often used by the Chinese Community Party to identify political dissent or other material the government wants to suppress.

In one meeting on Feb. 5, 2024, researchers discussed how to create profiles of people to “identify their intent” and “achieve exposure to harmful information,” according to meeting minutes.

Geedge researchers appear to be developing tools to use artificial intelligence to predict who might become critics of the Chinese government, based on data patterns the company has collected using surveillance technology.

“The Geedge research team was doing more than just documenting patterns of behavior. They were trying to predict what citizens might do next and with whom,” Mr Benson said. “These stocks of data on common materials are the raw material for generating profiles that determine who you are and what you will do next.”

Geedge did not respond to a request for comment.

The documents indicate that the company’s team worked to link physical movements with other online activities, including what movies people watched and what books they read, according to the Vanderbilt researchers.

Geedge’s research was similar in some ways to the work of another Chinese company with state ties, GoLaxy. Last year, Vanderbilt and The New York Times outlined GoLaxy’s efforts to develop artificial intelligence-powered software that would push targeted propaganda — topics supported by the Chinese government and against views opposed by Beijing.

In some ways, Geedge’s work also evokes the 2002 film “Minority Report,” in which the government predicts who might commit a crime and a special police unit moves in to arrest people before a breach occurs. Although the “precogs” who predict the future in the film are believed to be infallible, they mistakenly accuse the protagonist, played by actor Tom Cruise, of murder.

China has been using predictive tracking technology for years. But as the Geedge papers show, artificial intelligence models have accelerated the development of such tools.

And Geedge shows how predictive policing technology could soon become a Chinese export.

While the documents do not cover recent months, there is no evidence that Geedge completed or deployed the predictive technology, according to U.S. officials who spoke on condition of anonymity to discuss sensitive assessments.

But officials acknowledged that Chinese firms are working on such technology to fine-tune their surveillance status. And China’s public security authorities have been racing to use DeepSeek, the country’s leading artificial intelligence model, to push other predictive policing technology, according to experts and government officials.

U.S. officials said Geedge and other Chinese surveillance companies likely have enough processing power for their current-generation tools.

But predictive tools, especially if the company ends up incorporating captured phone calls or surveillance video, could run into limits on computing power.