丁香园AV

What got you interested in society鈥檚 obsession with data?

Across the many different debates we have about new technologies, a lot of the familiar arguments really boils down to what we imagine data is and what we imagine it can do. And a lot of it is a kind of aspirational faith, that the machines will provide objectivity to a society and humanity that seems so woefully lacking in it.

For example, this summer the UK government used a very faulty algorithm to generate students鈥� university entrance exam scores. This resulted in endless cases of students losing offers, and massive protests ultimately forced a U-turn on the part of the government. There鈥檚 an underlying belief there that if you throw algorithms and data at a complex, human problem, it鈥檚 going to do provide more accurate and objective results than human decision-making. Or when we hear regular (unsubstantiated) claims that we can use AI to 鈥榩redict鈥� criminality , it鈥檚 riffing off the underlying fantasy that people鈥檚 emotional lives, their intentions, their life trajectories, can be reduced into a series of statistical correlations.

The problem is that, inevitably, these assumptions run into a reality that contradicts them, and there arises the violence. Sometimes, it鈥檚 just too hard to gather high quality data, leading to inappropriate proxies and unreliable assumptions. In the book, I talk about how difficult it proved to get the kind of data that could precisely predict individual cases of terrorist attack 鈥� and so, intelligence agencies would turn to hazy, prejudice-loaded ideas about what constitutes 鈥榮uspicious鈥� behaviour. At one point you鈥檇 even see questions like whether you鈥檝e grown a beard recently, or whether you鈥檝e been subject to assault, as part of an FBI assessment of whether you鈥檙e more likely to commit terror attacks. And when you have the immense political pressure, and a broader societal faith, that says you can use data to predict these things鈥�

In their landmark book Objectivity, Lorraine Daston and Peter Galison say that 鈥渁ll epistemology begins in fear鈥� 鈥� fear that the world is too complicated to be 鈥渢hreaded by reason鈥�. Perhaps our obsession with data today, or what I call in the book 鈥榙ata hunger鈥�, is part of that fear.

What else can you tell us about the book?

The book essentially asks: what tends to happen when big data and AI鈥檚 promises of objectivity and predictivity inevitably fall short? What do intelligence agencies do when the surveillance systems can鈥檛 quite predict the next terrorist attack? What happens to our smart machines when they turn out to be not as smart as they were promised? And the key argument of this book is that when the technology falls short, we often make up the gap with speculations, simulations, and fantasies.

So the irony here is that so much of our technologies are predicated on the promise that they鈥檒l replace human decision-making, and all the uncertainty and guesswork and prejudice associated with that, with something that is more objective, more calculative, and therefore, with greater certainty. But it is precisely this overreach that creates the gaps between what our machines do and the problems we want those machines to address. Incomplete and uncertain data get cobbled together to fabricate a sense of reliable 鈥榩redictions鈥� and objective 鈥榠nsights鈥�.

The problem is that such speculations are themselves neither neutral nor coincidental. The book looks at two broad categories of data-driven surveillance: the first is state surveillance, focusing on the Snowden affair and the American government efforts to predict and prevent terror attacks. The second is self-surveillance, or how the growth of smart machines and IoT tech is producing a new ecosystem for data collection that seeps very deeply into our personal lives 鈥� the home camera, the voice assistant, the wristband, and so on. And across both of these sites, the book shows how the 鈥榝acts鈥� about our lives 鈥� everything from what counts as suspicious behaviour, to what counts as more optimal in our exercise, our work, our sex lives 鈥� is being reconfigured around these technologies and all their flaws.

What is one thing you鈥檇 like people to take away from the book?

That when we encounter data-driven technologies, when we see something that purports to be objective data or a predictive analysis, we should ask: what counts?

If predictive policing companies claim to predict high-crime areas through data, and use that to direct police patrols 鈥� what do they count as a predictor of crime?

If an online proctoring company promises that they can catch cheating students using indicators of 鈥榮uspicious behaviour鈥� 鈥�  what are they counting as suspicious?

This question of 鈥榳hat counts鈥� is inseparable, though, from the follow-up: what counts, and for whom? Because it is no coincidence that these flawed, inaccurate, judgmental technologies of datafication tend to be forced upon the tenants, the workers, the students, the migrants, the poor, the vulnerable.

In the book, we find some folks who find this excess of data to be exciting and empowering. In certain circumstances, it can be an opportunity to know yourself better, and to experiment with yourself into a cool, posthuman future. But for many others, often the most vulnerable among us, to appear correctly in databases is the unhappy obligation on which our lives depend.

Recent blogposts and interviews:

, June 23, 2020
, (starts: 21:42), August 26, 2020
, September 20, 2019