INTRODUCTION
What do we talk about when we discuss computational art? Do we describe a practice centred around a medium, as when we talk about painting or photography, or do we describe modalities, such as when we talk about visual art or audiovisuals? In reality, both can be accurate answers to this question, although both also fall short of properly defining computational art. Why is that?
Computation is a medium, and like any other medium, it can be used, or misused, to create art (1). Computation can remediate other media forms (2), but its powers extend far beyond that. Computation is a metamedium whose contents are existing media, but also those media forms that have not yet been invented or even that are not physically possible (3). This is possible because of processes of digitisation and imitation that allow systems to effectively resemble the surface features of other media forms, not only looking like them but also, to a large extent, acting similarly. Because of the many affordances of computational systems, and the diversity of ways they can be articulated, there is always a potential for the development of media forms that are new, i.e., without an existing tradition or pattern of usage (4).
For the very same reasons, computation is also a metatool, an everything tool that can be used to create images, sounds, texts, or any other types of signs. In computation, medium and tool, metamedium and metatool, converge in ways where they intersect, merge, and dilute each other’s boundaries. Computational media thus become tools that can create signs but can also keep those signs in memory; they become media that can preserve and transmit signs whilst also being capable of creating information and new signs. They become antinomic systems that are simultaneously able to stabilise information but are compelled to transform it (5).
This is why a modal definition of computational art is problematic. More than being multimodal, computational media tend towards amodality. Computation and computational media exist in a condition across modalities that has the potential to unfold in any or many modalities. Computational media and the computational artworks that use them are computational objects, and, as such, they are in a state of continuous production of signs from data and algorithms.
In computational objects, signs, as such, do not exist. Signs are produced on demand, when and where necessary, in whatever forms or modalities are required. In this process, information, either in data or generated by a computation, is transcoded to analogue signs so it can be communicated to humans and other analogue systems or to digital formats to be communicated with computers and other digital systems (6).
The processes of transcoding liberate computational objects from modality. While specific objects may be tied to modalities, we may not talk of computational objects as being modal-dependent, but very much to the contrary. Computational objects are in all modalities while being in none. They are in the condition of the metainterface (7) of the omnipresence of computation and its centrality for all other processes.
Computational art is developed from computation and computational objects in ways that make computation its raison d’être (8) as well as its material, to paraphrase Henry Flynt’s definition of conceptual art as an art “of which the material are ‘concepts,’ as the material of for ex. music is sound” (9). In computational art, computation is not only a tool, channel, or resource, but also the thing to be experienced, the centre of aesthetic experience. This can happen in cases where it is used in ways that give rise to other signs — thus producing images, texts, sounds, actions, etc. — but also in those cases where it does not — by producing computation without perceivable outputs.
This is the case with works such as Jaromil’s Forkbomb (2002) or Pall Thayer’s Microcodes (2009-2015). Forkbomb is a code without execution because of the destructive potential of its power to recursively copy itself and crash a host computer (10). Microcodes is a set of small poem-like snippets of code that, although they can be executed, are not meant to be, at least not on a computer, but rather through their reading, interpretation, and mental enaction. Even works with perceivable outputs can foster similar aesthetic phenomena because, very often, the aesthetic experience of computational art extends well beyond those perceivable outputs that may be presented and onto a future of potential realisations that one may never directly interact with but nevertheless feels.
Computation can, therefore, be proposed as the work of art itself, even as the sole aesthetic experience. And in those cases where we find that computation is merely used instrumentally — i.e., to remediate, substitute, or replace other tools and media — or when its presence is not aesthetically relevant per se, even if computers and computational systems may be abundantly used, we may be able to talk about “art involving computers”, but not about computational art (11). It is important to keep this in mind at a time when computation is so ubiquitous and when we may not, therefore, think about artworks as instances of computational art simply because they may in some way resort to computational technologies in their production or deployment. The central question is not whether computational technologies were used or are present, but rather whether the artistic proposal of the work is centred on computation at a fundamental level.
Whenever computation is used in a work, more often than not, it will coexist with one or more modalities and media forms and, consequently, with the traditions that have been developed around them, sometimes for millennia. Because of this, computational art can be visual art (12), and it can be literature, or audio-visual, or film, or music, or sound art. It can be all of these whilst remaining computational art, existing in a state of superposition where it is many things at once and never collapses into a single one (13).
COMPUTATIONAL AESTHETICS
Computational art promotes aesthetic experiences that are developed throughout a complex network of layers of information, meaning, and significance (14). Florian Cramer points to commonalities between computational and conceptual art. In Concepts, Notations, Software, Art (15), he proposes an idea of “software art”, a set of practices that “pay attention to how and by whom programs [are] written” (16) and of which the materials are formal instruction code whose “instructions and codedness” are exposed in and by works. In this context, Cramer points to how conceptual and software art find a commonality in the usage of language in the works — either in the form of instructions or concepts — and notes that both also collapse concept notation and execution in works. The definition of computational art to which we resort in this text encompasses Cramer’s software art and even expands it. As such, we find that much similarly to conceptual art, computational art is often anchored by objects and signs that allow communication to happen, but ultimately it is not object-oriented but rather systems-oriented (17) as it turns away from objects and towards relationships.
Conceptual art is often reminiscent of specifications for computational machines — or software — because of its usage of notation or plans to construct forms that, before their materialisation, only exist as concepts and afterwards continue to exist as concepts that retain the potential to be expressed more often and are not bound to a single — and scarce — instantiation. This is why “the idea becomes the machine that makes the art” (18), as the art object becomes a non-essential physical mediator for the art experience.
Computational art is more conceptual than perceptual (19), in ways that go beyond the logical and mathematical. It is cognitive, intuitive, fuzzy, and, again, like conceptual art, its practitioners can be seen as being more mystics than rationalists (20), proposing “conclusions that logic cannot reach” (21) in works that, precisely because of their illogical character, promote new experiences (22).
COMPUTATIONAL READING
This is one of the reasons why the experience of reading computational artworks, and computational media in general, is so different from that which is developed in other media.
When we engage with computational artworks, we do so first through their surface layer. The surface is the level of sensorial effusions of a system, and it is primarily through it that we manage to gain access and start to discover the artwork as a network of signs, actions, and causal processes. Through this analogue layer, we can start discovering the digital and the amodal. We can find a way to access the computational core of the artwork that otherwise would be an impenetrable black box. This computational core is what Frieder Nake calls the subface (23). If the surface is sensorially perceivable, the subface is computational, the system of rules that drives the artwork and on which the surface depends and that we are not able to perceive directly. The interface is a bridge between surface and subface, and it exists in a state of duality, in a superposition of computation and of its materialisation. Through the surface, we may attune to the surface, and we are able to couple structurally with the computation (24).
To do this, we need to expand the homolinear and heterolinear reading modes with a hyperlinear reading (25) that is both procedural and ergodic. This is an active reading process that requires readers to invest a “nontrivial effort” (26) and to bring into play far more than the interpretative functions required by most other media. In ergodic forms, readers may explore, configure, influence, or negotiate with texts in ways that go well beyond what is possible in non-ergodic forms. There can be great hermeneutical complexity in non-ergodic forms, but they are usually characterised by a stability of morphology that is quite rare — if at all possible — in ergodic forms.
Our engagement with computational objects is ergodic, as it is situated, iterative and enactive. It is also, as Penny notes, highly subjective and deictic because when we couple with a computational object, we don’t simply become its source of data or an external element to which it responds. Interaction puts us in a position that affects and shapes the computation and that we can, therefore, understand as a position of programming. Through interaction, we become both programmers and programmed. We become computation, and merge — albeit mostly temporally — with the system we couple with. Therefore, we are unable to adopt an objective view of it, of us, and the engagement. With the immersion of structural coupling comes the logical incoherence of assuming “an observing self that exists outside the engagement with an interactive system” (27).
The mesh we engage with when entangling with a computational artwork is not only formal, semantic, and symbolic, but also an interface. It is ergodic, procedural, and computational, pushing us to read and understand it at this level. The way we do that is by developing a gaze focused on function, program, and the recognition of computational patterns. This computational gaze (28) allows us to develop an active interpretation of the works, to simultaneously see them and see-as them, and to act on them while also acting-as them. This is not simply interpretation, but interpretation as epistemic action (29) an effort at deduction of causal relations, induction of mechanics, and abduction of technological intentionality (30) and an exploration of the aesthetic pleasures at the rift between surface and subface.
While reading through the computational gaze, we iteratively build models that are used to explore the artwork, to pose and answer questions, to fill in information, and to anticipate actions, both those from the system as well as our own. We build models that help us predict the future. By engaging with a computational artwork and interacting with it, if possible, we start to recognise some processes, discover others, and identify causal chains. We may, on occasion, duplicate some of the actual processes in a work, but more often than not, the models that we develop will most likely be approximations of the meaningful processes in a work, conjectures that are good enough because of their predictive power (31).
This process starts from the surface and descends towards the subface through a computational reading during which we develop a Theory of the System (ToS) (32). A ToS is not an emulation of the entire system of an artwork but rather a model of the central processes in its subface. A model of the processes that speak to us and that, therefore, become meaningful for the aesthetic and artistic experience of the artwork. It is a model that includes what we perceive as the most significant actors, objects, and behaviours in a system and the rules that organise their relationships. Therefore, it is a model that includes the system, its context, and those other systems — such as ourselves — that may interact with it. And even in those cases when artworks do not allow direct interaction, a ToS allows us to explore the artworks in ways that can be almost as informative as direct interaction, namely through the potential reprisal, repetition, and iteration of the artwork’s processes.
It is important to note that a ToS is not a general-purpose model. Rather, a unique ToS is developed for each artwork we engage with, and if some of its parts may perhaps be repurposed for other systems, their selection and engagement depend on the perceived relevance of the objects they model within the context of the new artwork.
The development of a ToS is a central part of the aesthetic enjoyment of a computational artwork, and it is the one that deeply attunes to the computational layer of the artwork and that is able to relate it to all other layers. This is an understanding of the work through simulation (33) that is, for the most part, developed subconsciously but that can, on occasion, be the focus of conscious attention and be helped by the direct communication of code, pseudo-code, or procedural descriptions within the work itself or in its paratexts.
DEMATERIALISATION
Through the development of a ToS, we model an artwork; we simulate and even emulate parts of it while abstracting less significant aspects. By doing this, we effectively create a new set of computations within our mind. We do not replicate the entirety of an artwork or its computational layer — except perhaps in the simplest of cases — but we do transform and adapt the artwork in the process of transferring it to our minds.
Computational artworks transform the locus of artistic experience because they create reality. This happens, firstly, through the contingent and situated character of their own existence, but it is also a consequence of how computational artworks involve us with their processes and allow us to bring their fields of meaning to existence through the participative process of creating a ToS, directly enmeshing us in the production of new artistic experiences. Through a ToS we absorb and reproduce the artworks, we dematerialise and detach them from their original substrates. We multiply the artworks so we can take them with us in a process that, seen from the perspective of the works, can almost be described as one of mutation and reproduction. It is a process of attunement to computation and to our capacity to discover computations in artworks and enact them.
This is an auratic process. By developing a ToS and transferring the artworks to our mind, we do not diffuse their auras — their here and now (34) — through the creation of mechanical copies that can be fully reproduced. Rather, we create singular, situated, new dimensions and a “dynamic lifetime” (35) that nourishes their auras. Every ToS has its own here and now and, much like an original, it cannot be reproduced or copied because it is developed through processes that are sub-symbolic, intuitive, and impossible to formalise and communicate. Furthermore, a ToS expands the artwork, adding new layers to its complexity and system. The original layers of mechanics, dynamics and aesthetics that are present in any computational system (36) are expanded by a ToS with two layers that we can describe as of simulated dynamics and simulated mechanics.
Here, computational artworks transcend their objecthood and manifest as artworks that are not bound by a finite set of signs or content, that are not bound by story, narrative, form, or other factors but, crucially, by process.
By dematerialising artworks, removing them from the substrate where they are found and embodying them in our minds, we develop something akin to empathy with them, even a very high level of intimacy. We start seeing and feeling artworks not as objects, formal features of physical materials, not as messages, carriers, or encoded meaning; not as surface but as subface, as information and causal processes that are encountered in a particular setting but may potentially manifest in multiple substrates (37). We develop an ecological understanding of the artworks, intuiting how they may develop outputs that are contingent to their mediation, how they may interface with other systems, including ourselves, and how they become in the irreducible phase space of their computations.
Through this, we manage to understand artworks as rifts between surface and subface, feeling them as objects that withdraw from us, from the world, even from themselves (38). They are objects that need to be interfaced with to be fully realised; objects that exist in a state of suspension — or of superposition — between a past that is captured in code, algorithms, and morphology, and futures that emerge from the potentials of computation. Computational artworks exist in a state between being themselves and not-being themselves (39), with their realisation dependent on their self-destruction through the permanent transformation of their current state. This is not unlike what Graham Harman sees as being at the core of all art, the rift between the real object and its sensual qualities (40), that we explore, attune to and resonate in our aesthetic enjoyment. If this gap exists within all things (41) as an ontological rift between being and appearing (42), it is hardly ever as clear as it is in computational artworks.
REFERENCES AND NOTES
1. Alva Noë, Strange Tools: Art and Human Nature (New York, NY: Hill and Wang, 2015).
BACK
2. Jay David Bolter and Richard Grusin, Remediation: Understanding New Media (Cambridge, MA: MIT Press, 1999).
BACK
3. See Alan Kay and Adele Goldberg, “Personal Dynamic Media”, in The New Media Reader (Cambridge, MA: MIT Press, 2003) pp. 393-404 & Lev Manovich, Software Takes Command: Extending the Language of New Media (New York, NY: Bloomsbury Academic, 2013).
BACK
4. Janet H Murray, Inventing the Medium: Principles of Interaction Design as a Cultural Practice (Cambridge, MA: MIT Press, 2012).
BACK
5. Miguel Carvalhais, “Designing (with) Computational Objects: From Metamedia to Metaenvironments”, Joelho – Journal of Architectural Culture 14, pp. 129-140 (2023).
BACK
6. See Trevor Paglen, “Operational Images”, E-flux 59 (2014) & Jussi Parikka, Operational Images: From the Visual to the Invisual (Minneapolis, MN: University of Minnesota Press, 2023).
BACK
7. Christian U Andersen and Søren B Pold, The Metainterface: The Art of Platforms, Cities, and Clouds (Cambridge, MA: MIT Press, 2018).
BACK
8. Hiroshi Kawano, “What Is Computer Art?”, in Artist and Computer (Morristown, NJ: Creative Computing Press, 1976) pp. 112-113.
BACK
9. Henry Flynt, “Concept Art”, in An Antology of Chance Operations (New York, NY: 1963).
BACK
10. Josephine Bosma, Nettitudes: Let’s Talk Net Art (Rotterdam: NAi Publishers, 2011).
BACK
11. Rosemary Lee, “Art Involving Computers vs. Computational Art”, in Espaço/ Programa (Vila Nova de Cerveira: FBAC, 2022) pp. 41-47.
BACK
12. For an in-depth reading of the processes of image creation and their influence in visual culture, see Rosemary Lee, Algorithm, Image, Art (New York, NY: Atropos Press, 2024).
BACK
13. We should beware that, as LeWitt notes, “When words such as painting and sculpture are used, they connote a whole tradition and imply a consequent acceptance of this tradition, thus placing limitations on the artist who would be reluctant to make art that goes beyond the limitations.” Sol LeWitt, “Sentences on Conceptual Art”, 0-9 5 (1969).
BACK
14. Rosemary Lee, “The Tangential Realism of @Pepitothecat”, in The Book of X: 10 Years of Computation, Communication, Aesthetics & X (Porto: i2ADS & CITAR, 2022) pp. 79-89.
BACK
15. Florian Cramer, “Concepts, Notations, Software, Art”, Netzliteratur (2002) <http://netzliteratur.net/cramer/concepts_notations_software_art.html>, accessed November 2023.
BACK
16. This perspective can be seen as something like a precursor to the idea of “critical code studies” that has been promoted by Mark C. Marino and others. Mark C Marino, Critical Code Studies (Cambridge, MA: The MIT Press, 2020). I find, however, that Cramer’s focus on art that uses code as its material but also “addresses cultural concepts of software” is more attuned than Marino’s to objects such as those we discuss here, where computation is central.
BACK
17. Jack Burnham, “Systems Esthetics”, in Art Forum (1968) pp. 30–35.
BACK
18. Sol LeWitt, “Paragraphs on Conceptual Art”, Art-Language 1 (1969)
BACK
19. Miguel Carvalhais, Art and Computation (Rotterdam: V2_Publishing, 2022).
BACK
20. Even when an artwork may seem strongly rational due to its formal features, we can argue that it may not necessarily be the case.
BACK
21. LeWitt, p. 1. [18.]
BACK
22. Ibid., p. 3.
BACK
23. Frieder Nake, “The Disappearing Masterpiece”, xCoAx – Conference on Computation, Communication, Aesthetics & X, pp. 11-26 (2016).
BACK
24. Simon Penny, Making Sense: Cognition, Computing, Art, and Embodiment (Cambridge, MA: The MIT Press, 2017).
BACK
25. Espen J Aarseth, Cybertext: Perspectives on Ergodic Literature (Baltimore, MD: Johns Hopkins Univ. Press, 1997).
BACK
26. Ibid., p. 1.
BACK
27. Penny, p. 360. [24]; cf the contrary view of Rodrigo Hernández-Ramírez, “Why Distancing Matters for Aesthetics and Technology”, in The Book of X: 10 Years of Computation, Communication, Aesthetics & X (Porto: i2ADS & CITAR, 2022) pp. 67-77.
BACK
28. Miguel Carvalhais, “Breaking the Black Box: Procedural Reading, Creation of Meaning and Closure in Computational Artworks”, in Artificial Intelligence and the Arts: Computational Creativity, Artistic Behavior and Tools for Creatives (Berlin: Springer, 2021) pp. 347-62.
BACK
29. Carvalhais (2022) p. 64. [19.]
BACK
30. Don Ihde, Technology and the Lifeworld: From Garden to Earth (Bloomington, IN: Indiana University Press, 1990).
BACK
31. It is also important to note that the cognitive economy of these models is a very relevant factor. A model is only good if its predictive power is also possible within a reasonable timeframe and with the limited resources one has within the limits of one’s own organism, i.e., if a model takes longer to predict the work than the time it takes the work to compute itself, or if a model demands mustering computational resources that are too demanding within the context in which the work is experienced, then that model is not useful and will certainly be dropped in favour of any alternatives that are more effective.
BACK
32. For a more in-depth overview of this, see Carvalhais (2022). [19]
BACK
33. Melanie Mitchell, Artificial Intelligence: A Guide for Thinking Humans (London: Pelican Books, 2019).
BACK
34. Boris Groys, In the Flow (London: Verso, 2016).
BACK
35. Aarseth, p. 81. [25]
BACK
36. Robin Hunicke, Marc LeBlanc and Robert Zubek, “MDA: A Formal Approach to Game Design and Game Research”, Conference of Artificial Intelligence Proceedings (2004).
BACK
37. And that, much like digital computation, can almost be thought of as being independent of these substrates.
BACK
38. Timothy Morton, Realist Magic: Objects, Ontology, Causality (Ann Arbor, MI: Open Humanities Press, 2013).
BACK
39. Ibid.
BACK
40. Graham Harman, Art and Objects (Cambridge: Polity Press, 2020).
BACK
41. Graham Harman, Object-Oriented Ontology: A New Theory of Everything (London: Pelican Books, 2018).
BACK
42. Timothy Morton, All Art Is Ecological (London: Penguin Books, 2021).
BACK