Automatic image recognition offers many possible applications - also in public institutions. In a pilot project, the Heilbronn City Archive trained an AI system to catalogue its stock of photos of people and buildings more easily, quickly and clearly.

Who is the person in this photo? Where is this building? Researching image content is part of everyday life for the staff at the Heilbronn City Archive. The task is huge: over one million photos have been stored in the archive since 1842 - and 5,000 more current or historical photos are added every year. This material has to be provided with the most important information and then entered into a database. Until now, this has been done manually by the employees of the city archive.

First AI project of the city of Heilbronn

In future, they will be supported by an AI solution. The software (DeepVA) from the Freiburg-based company The Chainless helps to recognise people and buildings in the photos and assign them searchable keywords. The Heilbronn city archive tested DeepVA as a pilot project for image recognition between January and October 2020. It was the first AI deployment ever by the city of Heilbronn. The aim was to make up to 1,500 public figures and around 200 prominent buildings in Heilbronn and the surrounding area identifiable through the system.

To achieve this, the AI software was first trained with 25,000 photos. In the first step, it had to learn to distinguish between people and buildings. This was done according to the principles of machine learning: the more photos the software was trained with, the better the recognition rate.

Not all photos were suitable for the process. The prerequisite for the training was that the pictures had a resolution of at least 120x120 pixels and a certain sharpness. For the AI software to recognise a person, at least three portrait photos of that person were necessary. The software had problems several times with profile views of people during training. It recognised people in frontal view much more easily.

Faces easier for AI to recognise than buildings

To recognise buildings, the AI software needed photos of about 20 views of the building, from several angles and in good resolution, to achieve optimal results. Overall, it was easier for the AI to recognise faces than buildings. This is because: to recognise buildings, the software has to learn the front and back of the building separately, as the facades can differ significantly. This is not the case with face recognition.

At the end of the training phase, the AI system recognised 98 percent of the people in the scanned photos. For the buildings, the recognition rate was 75 per cent-here there were not enough photos from different perspectives to increase the recognition rate. However, project manager Miriam Eberlein is confident that the rate will increase in the next few years, because the AI software is constantly learning.

The pilot project of the Heilbronn City Archive was awarded second place in the eGovernment Competition 2020, which was organised by Bearingpoint and Cisco. In the future, the AI software will be used permanently in the Heilbronn City Archive.