The Design of Everyday ATR

A lot of great work has gone into locating the affordances of automated transcription and text recognition in data cleaning, metadata extraction; enhancing collections’ findability and enabling digital scholarship. In thinking about this, Norman (2013: 10-11) within his popular The Design of Everyday Things defines “affordances” as the range of possible actions enabled by a design object - in our case Automated Text Recognition (ATR) and the interfaces underpinning its use. 

Less has been said about the potential hidden affordances both to ATR models and User Interfaces. Designers are taught, and in the case of those at companies like Transkribus do so very well, to signify the main affordances of their objects through controls and prompts. Transkribus’ noticeable ‘training’ button forms a clear example. Some affordances remain hidden, however, and - more often than not - unintentional. That is not to say that they hold no value. They could even hold added value. 

In response to a survey I undertook in 2021 of institutional Transkribus users, namely content-holding institutions - galleries, libraries, archives and museums, as well as academic publishers and even one high school, some respondents suggested that had found low-tech and simpler ways to maximise the platform for their needs, i.e. unintentional / potentially new and valuable affordances. Although the gold standard of collections processing is to capture archival materials in high resolution, ascribe them ISAD(G) aligned metadata and then begin the process of manual transcription to train an ATR model, or correct the results of a pre-built algorithm, these users were utilising affordances outside of this pipeline, however minimally. 

One respondent explained that they tested their XML (how plain text is stored) by using Transkribus’s interactive viewfinder for sample pages, instead of performing transcriptions within the platform. Though not the intended use of the Transkribus platform, this action adds a certain flexibility to ATR work and deviates from established practice. A flexibility that, Norman (2013) suggests, developers should become more comfortable with encouraging. 

What I did not reckon with analysing this 2021 survey was how individual researchers were using both intentional and unintentional ATR affordances.

In leading a Transkribus workshop at the University of Edinburgh’s Futures Institute, sponsored by the Oxford Text Archive (OTA) as part of their Winter Pilot Training Programme, I met some incredible linguistic researchers working across a wide variety of languages and materials - diaries in near-forgotten languages from the tip of South America, Latin dictionaries and Irish Gaelic/Manx documents. This variety proved, yet again, the omnivorous intake that tools like Transkribus support - as well as how broad the user community is becoming. See Megan Bushnell’s CLARIN-UK post for the content of the workshop (Tools for Digitising, Encoding, and Publishing Texts | CLARIN-UK) and Getting Started with Transkribus for the slides used. 

What I didn’t expect to see during the workshop were researchers making use of hidden ATR affordances - such as utilising close-enough Unicode ranges to train the model, then tweaking and retraining to end up with a visual representation mirroring the original manuscript. 

Elsewhere, pre-trained English models (as you probably expect there are a fair few) were used to transcribe Irish (where no public Transkribus model currently exists). It appeared to perform pretty well, though raised questions in the training around dismantling archival biases and colonial weightings with the master’s tools and language. 

After this workshop, I was left thinking more work needs to go into analysing these hidden affordances of ATR and other AI-driven tools. This should go beyond getting ChatGPT to fizz out and lose all sense of reason - although that’s fun too, a DHI colleague is partial to prompting ‘less words, more simple’ again and again and seeing what response they get. Our latest go was to ask whether sharks ever swam in the Colosseum? After one of us watched Gladiator 2. ‘Less words, more simple.’ Some sort of affordance is likely to come out of such mucking around. Maybe?

In all seriousness, though - ATR remains a good case study for detailing unintentional use, especially as we research and develop streamlined workflows for greater efficiency, often leaving little room for these affordances. Once an unintended use is revealed and deemed valuable does it become an intended use? I’m not sure. With systems as complicated as LLMs or transformer-based ATRs, new affordances will always emerge, both intended and unintended. Supporting those who find the latter should not be neglected, but also not necessarily co-opted. In the design of everyday ATR, a certain flexibility should remain.