From Robocop to Minority Report, the intersection between policing and artificial intelligence has long captured attention in the realm of high-concept science fiction. However, only over the past decade or so has academic research and government policy begun to focus on it.
Teagan Westendorf’s ASPI report, Artificial intelligence and policing in Australia, is one recent example. Westendorf argues that Australian government policy and regulatory frameworks don’t sufficiently capture the current limitations of AI technology, and that these limitations may ‘compromise [the] principles of ethical, safe and explainable AI’ in the context of policing.
My aim in this article is to expand on Westendorf’s analysis of the potential challenges in policing’s use of AI and offer some solutions.
Westendorf focuses primarily on a particular kind of policing use of AI, namely, statistical inferencing used to make (or inform) decisions—in other words, technology that falls broadly into the category of ‘predictive policing’.
While predictive policing applications pose the thorniest ethical and legal questions and therefore warrant serious consideration, it’s important to also highlight other applications of AI in policing. For example, AI can assist investigations by expediating the transcription of interviews and analysis of CCTV footage. Image-recognition algorithms can also help detect and process child-exploitation material, helping to limit human exposure. Drawing attention to these applications can help prevent the conversation from becoming too focused on a small but controversial set of uses. Such a focus could risk poisoning the well for the application of AI technology to the sometimes dull and difficult (but equally important) areas of day-to-day police work.
That said, Westendorf’s main concerns are well reasoned and worth discussing. They can be summarised as being the problem of bias and the problem of transparency (and its corollary, explainability).
Like all humans, police officers can have both conscious and unconscious biases that may influence decision-making and policing outcomes. Predictive policing algorithms often need to be trained on datasets capturing those outcomes. Yet, if algorithms are trained on historical datasets that include the results of biased decision-making, it can result in unintentional replication (and in some cases amplification) of the original biases. Efforts to ensure systems are free of bias can also be hampered by ‘tech-washing’, where AI outputs are portrayed (and perceived) as based solely on science and mathematics and therefore inherently free of bias.
Related to these concerns is the problem of transparency and explainability. Some AI systems lack transparency because their algorithms are closed-source proprietary software. But it can be difficult to render even open-source algorithms explainable—particularly those used in machine learning—due to their complexity. After all, a key benefit of AI lies in its ability to analyse large datasets and detect relationships that are too subtle for the human mind to identify. Making models more comprehensible by simplifying them may require trade-offs in sensitivity, and therefore also in accuracy. Together these concerns are often referred to as the ‘AI black box’ (inputs and outputs are known, but not what goes on in the middle).
In short, a lack of transparency and explainability makes the detection of bias and discriminatory outputs more difficult. This is both an ethical concern and a legal one when justice systems require that charging decisions be understood by all parties to avoid discriminatory practices. Indeed, research suggests that when individuals trust the process of decision-making, they are more likely to trust the outcomes in justice settings, even if those outcomes are unfavourable. Explainability and transparency can therefore be important considerations when seeking to enhance public accountability and trust in these systems.
As Westendorf points out, steps can be taken to mitigate bias, such as pre-emptively coding against foreseeable biases and involving human analysts in the processes of building and leveraging AI systems. With these sorts of safeguards in place (as well as deployment reviews and evaluations), use of AI may have the upshot of establishing built-in objectivity for policing decisions by reducing reliance on heuristics and other subjective decision-making practices. Over time, AI use may assist in debiasing policing outcomes.
While there’s no silver bullet for enhancing explainability, there are plenty of suggestions, particularly when it comes to developing AI solutions to enhance AI explainability. Transparency challenges generated by proprietary systems can also be alleviated when AI systems are owned by police and designed in house.
Yet the need for explainability is only one consideration for enhancing accountability and public trust in the use of AI systems by police, particularly when it comes to predictive policing. Recent research has found that people’s level of trust in the police (which is relatively high in Australia) correlates with their level of acceptance of changes in the tools and technology used by police. In another study, participants exposed to purportedly successful policing applications of AI technology were more likely to support wider police use of such technologies than those exposed to unsuccessful uses, or not exposed to examples of AI application at all. In fact, participants exposed to purportedly successful applications even judged the decision-making process involved to be trustworthy.
This suggests that focusing on broader public trust in policing will be vital in sustaining public trust and confidence in the use of AI in policing, regardless of the degree of algorithmic transparency and explainability. The goal of transparent and explainable AI shouldn’t neglect this broader context.