The lack of transparency and accountability of machine learning models undermines human trust. And it can have severe consequences; there have been cases of people incorrectly denied parole, wrong decisions leading to the release of dangerous criminals, and generally poor use of limited valuable resources in criminal justice, medicine, finance and in other domains due to the opaque nature of “black box” AI models.

The use of AI tools by law enforcement for investigative and preventive actions has tremendous positive impact, but raises a series of concerns in the minds of the public, and poses challenges both for security agencies as well as the developers of AI solutions. The limits of what is public information, what rights citizens have over their private information and where it is stored or how it is used seems to vary, and the implications of this on personal privacy is a concern that must be addressed. 

Some AI developers are aware of the explainability and interpretability problems, but they aren’t always top of mind. While many have the best of intentions, companies have commercial interests at the forefront of their pursuits, and the issue of ethical AI development and explainability is often the last on the list of priorities when carrying out product development. There is an urgent need to educate AI developers on the impacts of ignoring ethical issues such as model bias, discriminatory algorithms and other prominent problems, and help AI businesses avoid these pitfalls.

While typically policing is absent in popular speculations about “being replaced by robots” or AI automation, there is indeed a degree of skepticism among Law Enforcement Agencies to overcome in this regard. Beyond general mistrust of the accuracy of AI recommendations and a fear of job loss, there is serious concern that police officers using AI for their daily work will leave them exposed to ethical and legal implications as a result of a lack of clear guidelines for its use.