Our #algorules project, in cooperation with the Bertelsmann Stiftung, has been running for several months – time to take stock. The goal is to develop a catalogue of quality criteria for algorithmic processes, which can serve as a reference point for complaints and reviews.
Here you can get an overview of our past, present and future activities.
What has happened so far:
Research and working paper on existing quality criteria catalogues
There is no need to reinvent the wheel. Especially in the USA, numerous organizations have already attempted to establish quality criteria for algorithms or to update existing catalogues of criteria. In the first phase of the project we thus asked ourselves, what approaches to quality criteria for algorithms already exist and what can we learn from them for further development?
We examined three sets of principles: the FAT/ML Conference’s “Principles for Accountable Algorithms and a Social Impact Statement for Algorithmus”, the Future of Life Institute’s “Asilomar AI Principles”, and the ACM US Public Policy Council’s “Principles for Algorithmic Transparency and Accountability,” and found that a large number of useful quality criteria for algorithms are already in place, from transparency and fairness to responsibility and traceability. What is lacking, however, is an examination of the specific ethical questions raised by the relentless use of algorithms in more and more areas of life, as well as concrete data on the implementation of any set of principles or criteria.
A working paper (available here) summarizes the results of our efforts.
The first expert workshop in Berlin and the initiation of the #algorules process
On the basis of these findings, we created a draft catalogue of quality criteria. An expert workshop was convened in Berlin on 25 May, 2018 in order to discuss the content and direction of our proposals, which profited from the sharing of various perspectives and experiences. Algorithms are used by the private and public sectors in a wide range of fields, from communications services, financial risk assessment and targeted advertising to health and police applications and education. At the same time, the development process of algorithmic systems consists of many phases, each of which involves very different actors. Meeting with key stakeholders from a wide range of fields also marked the official start of our work on the #algorules. The participants provided a number of important impulses for the further development of the catalogue, both for the preamble and for the criteria themselves.
Since the conclusion of this first workshop, our catalogue now contains over a dozen quality criteria. From proven tools such as impact assessments and transparency and accountability mechanisms to basic principles such as fairness and sustainability, we are working on a coherent set of practicable requirements that will now be further developed and refined.
The next phase in the development of the #algorules catalogue consists of a consultation process. We are currently in the process of interviewing around forty high-ranking experts from science, politics, business, civil society, media, and other relevant fields. Not only do we want to gather valuable input for improving and focussing the content of our catalogue. The experiences of consultants from a range of different sectors will also prove indispensable to our implementation strategy. Which requirements must such a set of quality criteria fulfill in order to be accepted by various actors in their respective fields? Who are the important multipliers? How can the catalogue be combined with existing professional ethical guidelines? We anticipate exciting answers to these questions.
Presentation at expert round tables
We have also been gathering important feedback by presenting our project at relevant expert events and panels. For example, we recently delivered a presentation on the #algorules at a workshop on smart assistance systems hosted by the Stiftung Neue Verantwortung (SNV), as well as at an event at the Hanns-Seidel-Stiftung in Munich dealing with the use of algorithms in the process of TÜV technical certification.
Such formats offer us the opportunity to discuss the project and the quality criteria catalogue within a larger group. In addition, they allow us to get a feeling for the applicability of the individual points in clearly defined contexts, be it in medicine or in connection with consumer protection. We then incorporate this feedback in the further development of the catalogue and for the development of an implementation strategy.
Expertise on the commitment of successful professional ethics
Drawing on lessons drawn from existing quality criteria catalogues, we hope to develop an effective strategy for the proper application of our #algorules. A closely related question, for example, concerns how any professional ethic or code can be made binding. Why should responsible actors adhere to a more or less abstract collection of requirements? How can a catalogue of quality criteria for algorithmic processes become a binding standard, analogous, for example, to the press code for journalists?
The authors of our working paper “Ethics for Algorithmists — What we can learn from successful professional codes” try get to the bottom of these questions. Together, Alexander Filipovic, Claudia Paganini and Christopher Koska analyze six professional fields and their codes of ethics: medicine, journalism, social work, advertising, public relations and engineering.
The authors identify ten success factors for professional ethics and offer a practical orientation on how they can be applied to the field of algorithm design. This is done against the background of a detailed description and historical classification of the growing professional field of “algorithmists.”
Second expert workshop in Berlin — developing an implementation strategy
In order to gather more feedback on how our criteria could be implemented, we organized a second expert workshop in Berlin in autumn, 2018. As with the first workshop, we invited representatives from different sectors to gain as broad an insight as possible into the development and application of algorithmic systems. The different perspectives have proven helpful in focusing and improving the quality criteria catalogue and in developing an implementation strategy based on the results of the consultation process and the recommendations of Filipovic, Koska and Paganini.
Broad participation process
By entering a dialogue with chosen experts from their fields in our workshops and consultations, we have laid a solid foundation for the development of our catalogue of quality criteria. In order for this catalogue to achieve the necessary level of societal acceptance, it should also be subjected to a broader debate. Thus, there is now a participation process in place that will involve a larger group of interested people in the development of our criteria. We will continue to publish up-to-date information about this here and on Twitter.
The #algorules process is in full swing. We look forward to continuing our cooperation with the Bertelsmann Stiftung and all those involved and will keep you informed of our progress.
Back to article: Code of conduct for the use of algorithms.