Embedding ethical governance when deploying Artificial Intelligence (AI) in city services

Mikko Rusama, Chief Digital Officer at City of Helsinki
Theo Blackwell, Chief Digital Officer for London

As CDOs of Helsinki and London, since 2019 we’ve been sharing our thoughts and experience on the deployment of artificial intelligence in cities.  We have argued previously that high-level principles are not enough, and that city officials should be equipped with a range of practical tools and questions to shape the trialling and deployment of new technologies.  

This article is the first in a series where we want to illustrate the benefits of AI in practice.

Digitalisation has provided cities with a range of new technologies, including Artificial Intelligence (AI), which can help deliver city services more effectively. Cities are big data custodians: machine learning and AI give rise to a range of novel ethical considerations related to data and AI use cases.  As CDOs of Helsinki and London, in November 2021, we collaborated to support the University of Helsinki’s Ethics of AI online course, which was launched at https://ethics-of-ai.mooc.fi/. This free course is now available to anyone interested in the ethical aspects of AI and helps people to learn the meaning of AI ethics, what can and cannot be done to develop AI in an ethically sustainable way, and how to start thinking about AI from an ethical point of view.

Ethics of AI is an example of the thinking we are supporting from ‘ground up’ to inform the national and supra-national discussions around new rules governing AI.  The European Commission has acknowledged the ethical dimensions of AI and recently proposed a novel  legal framework on AI, which positions Europe in a leading role in this debate globally.  While these details are being deliberated, it is important to note that the current General Data Protection Regulation (GDPR) rules outlining protections and processes related to privacy that provide European and UK citizens with a level of assurance which do not exist in many jurisdictions.

Here we outline two case studies our cities contributed to the Ethics of AI online course, and the future steps our cities are taking to create better frameworks for future technologies.

This also involves the development of new governance principles, which our cities are implementing as part of the MyData Principles in Helsinki and the draft Emerging Tech Charter in London. 

Helsinki case-study: preventative healthcare

The City of Helsinki believes in a health care paradigm shift, which is transforming services from “reactive” to “preventative” models. With the help of sophisticated algorithms it becomes possible to create predictive models for citizen healthcare. By analysing large patient data sets against specific criteria, high-risk individuals can be identified and their care prioritised. These high-risk individuals can be invited to medical appointments to receive proactive treatment.

For example, patients with high blood pressure can be identified proactively to ensure the correct medication is delivered to avoid heart and brain attacks. Another proactive use case is to identify patients who use central nervous system medication and to ensure they have a dedicated doctor who can follow up on their medication and condition.

Preventative healthcare has the potential to improve citizens’ quality of life, as well as to reduce overall treatment costs significantly. 

A range of legal and ethical issues arise from preventative healthcare, including privacy and security considerations in the appropriate use of data. A demarcation line between acceptable prevention and non-acceptable intrusion needs to be set. For example, does the city have the right to use private and sensitive medical data for identifying high-risk patients? 

Establishing and maintaining citizens’ trust is essential in providing proactive and preventative services. Without establishing trust first, obtaining consent from citizens is unlikely and therefore we are unable to deploy innovative AI solutions. Citizens should have the right to deny the use of their data in ways to which they have not consented, unless there is a further, lawful reason for data processing. 

A real ethical dilemma arises when city medical data indicates that an individual needs urgent treatment, but the person has not granted permission for predictive AI to make this determination. Is it right for a city not to act proactively in this situation? Ultimately, citizens have the right to make free will decisions but the city also may have applicable legal duties of care. 

A fundamental question of the city’s role arises here: if the city has knowledge of a potential health risk and does not act upon that data, can the city be considered negligent? Is it possible for citizens to be treated equally in the physical and digital realms? If a person has a medical episode in real life, an ambulance can be called by bystanders without having explicit permission to do so. In an increasingly digital world, proactive approaches can prove more difficult to achieve from the data ethics point of view.

Implementing MyData Principles in Helsinki

The City of Helsinki is working on putting the so-called MyData principles into practice. MyData provides guiding principles to empower individuals regarding the use of their personal data. The human-centric paradigm is aimed at a fair, sustainable, and prosperous digital society, where the sharing of personal data is based on trust as well as balanced and fair relationships between individuals and organisations.

In line with MyData principles, citizens should be able to provide and revoke consent relating to processing of their personal data under specified purposes. On a case-by-case basis, it should always be transparent to citizens: 1) what data has been stored about them and 2) what is the legal basis for processing, even when data processing is not based on a freely-given consent but on other provisions of the GDPR.

London case-study:  Understanding ‘busyness’ during the pandemic

Discussions around sensors, cameras and AI in cities invariably turn to questions around surveillance and the circumstances where this is permissible.  

As the COVID-19 pandemic took hold of the UK in spring 2020, the streets emptied. Shops shut down, and anyone who ventured outside had to follow the ‘two-metre’ rule. In London, the authorities realised that they needed a way to monitor the changes in street activity in real time to gain a better understanding of how lockdown was affecting city life, and what interventions were needed to allow the city’s nine million people to keep socially distanced.

Working with the Greater London Authority (GLA) and Transport for London (TfL), a team at The Alan Turing Institute launched Project Odysseus. This involved a repurposing of an existing project that had been monitoring air pollution around London. The team modified its algorithms, feeding them data from London’s traffic sensors and cameras to estimate pedestrian and vehicle numbers. The authorities used this information to understand changes in activity across London, pinpointing where more social distancing measures were needed.

The majority of the team’s work has focused on analysing the stream of data that flows from TfL’s JamCams – some 30 gigabytes per day. These cameras are positioned at traffic intersections, but they also capture the pedestrians on road crossings and pavements. The team adapted its vehicle detection algorithms to monitor pedestrians, providing authorities with near real-time estimates of pedestrian and vehicle density. Extra information about traffic density also came from the 11,000+ ‘inductive loop’ devices on London’s roads, which detect vehicles that pass over them.

This software was used by TfL to monitor pedestrian density during the pandemic, enabling the authorities to quickly identify where social distancing interventions were required. On Brixton high street, for example, the tool demonstrated that there was too much crowding on pavements, particularly near bus stops. As a result, TfL extended the pavement and moved a bus stop to create more space.  TfL says that it implemented over 700 such interventions at the height of the pandemic’s first wave, and that the Turing Institute’s tool provided key data for those decisions.

Throughout this work, the team has been careful to consider the ethical implications of monitoring CCTV footage by developing algorithms incapable of performing facial recognition or tracking people.The JamCam footage, for example, is also a low enough resolution that there is no risk of being able to identify anyone. The Turing Institute’s Ethics Advisory Group is working to ensure our data and algorithms preserve the privacy of the general public.

The draft Emerging Tech Charter for London

In the future, London wants AI and new technologies collecting personal data to follow the rules-of-thumb set out in London’s Emerging Tech Charter, which represents the next step in thought leadership. The charter covers the trialling as well as deployment of data-gathering technologies and services utilising sensors, cameras, robotics, augmented and virtual reality, through to automated and algorithmic decision-making.    

It provides three principles for implementing technology in London by working in the open, respecting diversity and ensuring trustworthiness with data. These principles act as a guide for the makers and buyers of technology, as well as Londoners and their elected representatives, when new technologies or services are trialled or deployed in the city by public bodies. It is designed to be flexible, so it may also support other organisations when considering deployment of technologies which impact their employees, customers, or the public use of buildings or land. 

Conclusion

In practice, real-world use cases can be difficult to tackle as the above examples demonstrate. It is fair to say that city governments and agencies are on a learning journey here, and sharing our experiences from these discussions will be important to policy makers.  

We as CDOs of Helsinki and London believe that cities should promote innovation which enacts the principles behind GDPR, not just follows the letter of the law. Our aim is to embed ethical data and AI governance principles wherever possible. This will be possible through transparent collaboration with citizens and other cities globally.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s