Maja Bak Herrie and Danish Technological Institute investigate the foundational limitations to the way state-of-the-art AI image generation models respond to different types of visual signification, i.e., their understanding of the relation between the prompt and what that prompt means. While AI image generation models show great promise in simulating high quality imagery, they also tend to fail in subtle and strange ways: producing hands with 6 fingers or text that resembles no language. Applying the semiotics of Charles S. Peirce, specifically his tripartite model of signs, the project explores how successfully different image generation models respond to prompts relying on each of the three different types of signification in Peirce’s model and trace the results back to the underlying techniques used to make the models. The results of these experiments, whether positive or negative, open new questions of the depth or shallowness of AI image generation models’ understanding of signs.

Maja Bak Herrie has received seed funding from Interacting Minds Centre at Aarhus University to undertake the project in 2023.