The 28th edition of ISEA, co-organised by Le Cube Garges and the ÉcoledesArtsDécoratifs–PSL, took place in Paris at the Forum des images from 16 to 21 May 2023, as
well as through an artistic programme in some 50
partner cultural venues throughout France in spring-summer 2023. The videos of which are online here.
Following an international call for proposals, more than 1400 submissions from 70 countries were evaluated by an
artistic and scientific committee of 200 international experts. The theme of this edition was Symbiosis. In this
time of global health, ecological, economic and democratic crisis, symbiosis is a polysemous notion that allows us
to explore in a transversal and interdisciplinary way the mutations and transformations underway in the digital
age, to question the meaning supposedly given to progress, especially in the current environmental and health
context, and to imagine possible and viable futures for our planet and our ecosystems.
The symposium catalog, including the symposium program and the artistic program, is available as
a downloadable pdf here.
Figures
+ 1 500 participants at Symposium conferences and exhibitions at the Forum des
images + 700 participants in online conferences during Symposium week + 7 000
visitors to the Augmented Catalogue in the first 15 days + 1400
proposals + 660 academic proposals +740 artistic
proposals + 70 applicant countries + 50 cultural partner venues
Co-organizers
ISEA2023 was co-organised by Le Cube Garges, executive producer, ÉcoledesArtsDécoratifs–PSL, academic programming director, and the Forum des images, Symposium
venue. ÉcoledesArtsDécoratifs–PSL
is a public institution of higher education under the authority of the Ministry of Culture, whose mission is the
artistic, scientific and technical training of artists and designers, as well as the development of research
creation activities. pioneer in thie field of collaborations between art, design and sciences, it created a
research laboratory, EnsadLab in 2007. Within the University Paris Sciences & Lettres (PSL), towhich it is
affiliated, it set up the PSL SACRe (Science Art Creation Research) doctorate program in 2012. It is the first
practice-based doctorate to be set up within an art and design school in France.
Le Cube Garges is an interdisciplinary and digital cultural innovation center with six facilities covering 8,000
m². Focused on creative renewal, it combines the discovery of new artistic forms, inclusive practices,
digital training and interdisciplinary reflection on societal issues.
SYMBIOSIS, THE MAIN THEME
Symbiosis is essential to life. No living species can survive without cooperation, let alone humans with each
other and with their environments. More than a simple coexistence, symbiosis induces interdependence. It can be
positive, neutral and sometimes negative in certain cases of parasitism. From these interrelations can emerge new
hybrid entities, like the lichen which is a compound of algae and fungi. The first providing photosynthesis and
the second moisture, one benefits from the contribution of the other. Our becoming lichen would be in a way the
cross-cutting question underlying the symposium. Understood in a broad acceptance, the symbiosis is also played
out at the scale of the symposium: it is a transdisciplinary event between creation and research (visual arts,
theater, music, design, cinema, human and social sciences, engineering sciences , natural sciences) and
intersectoral (art, crafts, industry, research, education, cultural innovation).
THE SUB-THEMES
The call for proposals for the symposium was divided into three main sub-themes: symbiotic individuations,
organizations and imaginaries. It was a starting point to structure the call without limiting the scientific
concept of symbiosis, and to encourage cross-cutting proposals outside disciplinary and technical classifications.
The selected interventions thus made it possible to discover works that call on symbiosis as a scientific
concept, but above all projects that used it as a metaphor or more simply as an inspiration. It was an invitation
to discover in a new way, both serious and curious, projects closely associating creation, research and
technologies.
Symbiotic individuations
Which type of “hybrid beings” are coming into existence today, and thereby associating the human and
the non-human? What new forms will they take, and will they be the next inhabitants of our planet? Experiments on
trans-humanism, the emergence of new artificial autonomies, and new representations of natural elements were
examples of the exploratory perspectives that were to be presented and discussed.
Symbiotic organizations
Human societies have founded their equilibrium upon multitudinous and interdependent forms of symbiosis across a
wide swathe of domains: ecology, economy, culture, technology, and more. What happens when a given event destroys
this fragile equilibrium? Researchers, artists, designers, and leading experts offered their perspectives on these
resolutely transdisciplinary approaches, with the shared ambition of contributing to a world that is more
resilient and at ease in its alterity.
Symbiotic imaginaries
In order to exist and prosper, symbiotic relationships—relationships in which each partner derives
reciprocal benefits indispensable to their survival—are essential. Symbiosis is a vital necessity at all
levels of all known ecosystems. But “necessary” does not imply that it should be reduced to its
pragmatism or instrumentality. Symbiosis also opens us to new imaginaries, be they novel or redundant in the
context of their corresponding era. If modes of collective imagination vary according to perspectives, they are
always multiple and in dialogue with reality—what, therefore, constitutes the symbiotic imaginations of
today and how and why invent new ones?
THE “CLUSTERS”
Given the large number of conferences, roundtables and various oral presentations, the academic chairs proposed
clusters to enable the symposium audience to compose its own cross-cutting program to complement the Symbiosis
themes. This
gave meaning to the general programme.
The papers grouped together in each of these clusters could be classified differently and often appeared in
several other clusters. These groupings are therefore retained here, with each cluster listed in alphabetical
order (authors). Each paper is colour-coded to indicate the sub-theme to which the author(s) responded (sub-theme
reminder), together with the cluster in which it was scheduled for the symposium audience and any other clusters
to which it might have been attached. A final glossary by sub-theme and cluster is provided to offer readers
several entry points into this important volume. These classifications in no way restrict the understanding of the
papers, they are simply reading aids.
Academic Chairs and International Programming Committee
Academic Chairs
The Academic Chairs were
responsible for the final programming of the symposium based on the evaluations of the IPC. The names and
biopics of the Academic chair and of the three co-chairs are published below.
The International Programming
Committee (IPC) was composed of 140 internationally renowned experts. The Committee evaluated the 1400
proposals in a double-blind process for full & short papers, panels and roundtable discussions. The names
of the IPC are listed below.
Dr Emmanuel Mahé — ISEA2023 Academic Chair
Dr (HDR) Emmanuel Mahé (France), Director of Research at École nationale supérieure des
Arts Décoratifs (PSL), researcher in information and communication sciences dealing with issues related
to organisations and innovation processes in the fields of arts, sciences and design; scientific officer at the
High Council for the Evaluation of Research and Higher Education. Website of Emmanuel
Mahé
Dr Elena Papadaki — ISEA2023 sub-topics co-chair
Dr Elena Papadaki (UK/Belgium) is a researcher, educator and curator; her research interests lie in the
intersection of technology-reliant artworks, curatorial discourse, interdisciplinary collaboration and audience
reception.
She leads the MA Design at University of Greenwich and currently serves as a Pathway Councillor for the Royal
Society of Arts (RSA), acting as a key advisor and expert to the delivery group “Design for
Life–Students for Change”. Website of Elena Papadaki
Dr Maria Ptqk — ISEA2023 sub-topics co-chair
Dr Maria Ptqk (Spain) is a curator, researcher and cultural producer. With an academic background in law and
economy, she holds a PhD in artistic research with a doctoral thesis on the cyberfeminist collective subRosa.
Her work investigates the ecological and technoscientific realm from a postcultural perspective, in the
crossroads of experimental art practices and collective in/trans/disciplinary knowledge. Website of Maria Ptqk
Prof. François-Joseph Lapointe (Quebec) is an artist and scientist at the University of Montreal,
holding two doctorates (in evolutionary biology and in dance and performance). He has published over 120
articles in the field of molecular systematics, population genetics and metagenomics. As part of his artistic
practice, he applies biotechnology for creative purposes and has created choreogenetics. Website of François-Joseph
Lapointe
International Programming Committee
Aceves Sepulveda Gabriela –
Université Simon Fraser (CA)
Albert Jeffrey –
Université Loyola de la Nouvelle-Orléans (FR)
Almas Almir –
Université de São Paulo (BR)
Anders Peter – Kayvala
(États-Unis)
Badani Pat – Conseil
d'administration de l'ISEA (États-Unis)
Baker Camille – Royal
College of Art, Londres (Royaume-Uni)
Basserau Jean-François
– Ecole des Arts Décoratifs–PSL (FR)
Behar Armand – Centre de
Recherche en Design – ENSCI-Les Ateliers ENS Paris-Saclay (FR)
Bellotto Janet –
Université Zayed (AE)
Berenguer Josep-Manuel –
Sonoscop (ES)
Beyls Peter –
Collège universitaire de Gand Belgique (BE)
Bianchini Samuel – Ecole
des Arts Décoratifs–PSL (FR)
Vatavu Radu-Daniel – Stefan cel Mare University of Suceava (RO)
Vergara Erandy – ISEA International (MX/CA)
Vilca Cecilia – MyAP – Microscopía Electrónica y Aplicaciones en el
Perú (PE)
Warnke Martin – Leuphana University Lüneburg (DE)
Wei Jo – CAFA
(CN)
Young Jiayi –
Université de Californie, Davis (États-Unis)
Zhang Ga – Académie
Centrale des Beaux-Arts, Pékin (CN)
Academic and Artistic Program
At the same time as the academic call for artists and researchers, a specific call for projects was launched,
under the artistic responsibility of Cube-Garges. This selection has been enhanced by the networked artistic
program (50 venues) offers thematic tours around symbiosis in a network of partners in Paris and in France,
during several weeks. It irrigated the territory and thus brought to ISEA2023 both an anchoring in various
places and an opening to diversity. The public discovered a wide variety of installations, virtual or augmented
realities, performances, concerts, video games, design, films or digital books. The tours is also available on a
dedicated online platform here or on our website.
POSTERS
Bazenet Flavien, Plisson Laura and Goeury Alexandre (FR)
Verdeil Marie, Jochem Rebekka, Christian Schulz Jan and Völp Lukas (FR / DE)
Vesac Jean-Ambroise, Duval Hélène, Ouellet-Plamondon Claudiane, St-Onge David, Salter
Chris (CA / CH)
Young Jiayi and Chomaz Jean-Marc (US / FR)
Zannos Iannis, Birringer Johannes, Antoniadis Pavlos, Hirayama Haruka, Jégo Jean-François,
Paschalidou Stella, Nelson Peter and Papachristou Dana (GR / UK / JP / FR)
ISEA International
ISEA International was proud to present the 28th edition of our annual symposia in Paris (France), with an
academic symposium and exhibition predicated as an experimentation incubator driven by the unifying theme
of“Symbiosis”– an essential consideration for the interdependent gathering of the year 2023.
We joined in celebrating ISEA’s return to Paris after 23 years since ISEA2000
“Révélation”, with Le Cube Garges as executive producer and responsible for the artistic
program, and ÉcoledesArtsDécoratifs–PSL
responsible for the symposium’s academic program. As in the previous iteration in Paris, the academic
symposium was held in the heart of the city at Forum des images.
Founded in the Netherlands in 1990, ISEA International (formerly Inter-Society for the Electronic
Arts) is an international non-profit organisation fostering interdisciplinary academic discourse and exchange
among culturally diverse organisations and individuals working with art, science and technology.
The main activity of ISEA International is the annual International Symposium on Electronic Art
(ISEA). The first symposia took place in 1988.
Tributes
Two major figures in art and culture left us in 2023: Peter Weibel, whose tribute was paid during the
Symposium with the entire ISEA community gathered in the large amphitheatre of the Forum des Images, a tribute led
by Pr Roger Malina which elicited many speeches from the audience; and Vera Molnár, who died a few months
after the Symposium and to whom we pay tribute in these proceedings.
Disremembering PeterWeibel: Reminiscences are
more Interesting than Fact
Peter Weibel was a leading figure in the new media art scene. Since 1999, he had been the director of the ZKM in
Karlsruhe and had carried out numerous curatorial activities (Ars Electronica, Venice Biennale, etc.). Among his
most important exhibitions, we can also mention his collaborations with Bruno Latour, who also passed away
recently ("Iconoclash" in 2022, "Making Things Public" in 2005 and "Critical Zones. Horizonte einer neuen
Erdpolitik" in 2020). The sudden departure of Peter Weibel in March 2023 was a shock for the ISEA
communities therefore a tribute was paid to him at this years’ symposium. A lecture was given by Roger
Malina, followed by a session lasting over an hour in which Malina asked the large audience to share memories of
Peter Weibel. Many very moving and interesting words for the ISEA community were expressed. Many deeply moving
words and recollections were expressed by members of the ISEA community.
Tribute to Vera Molnár
Among the numerous reactions following the passing of Vera Molnár at the age of 99, a pioneering artist in
geometric abstraction and generative art, the National Institute of Art History (INHA) in Paris paid tribute to
her. The ISEA Symbiosis committee also wished to honor this iconic figure by transcribing the text from the
INHA here.
Vera Molnár “had established deep ties with the INHA, which were manifested through the donation of
a significant collection of prints in 2022, where the pioneering and generous dimension of her work fully
reflected her personality. Her passing leaves a huge void in the art world, where she was still actively engaging
with young creators who came to visit her. Vera Molnár was born in 1924 in Budapest, where she received a
classical education in Fine Arts and graduated as a professor of art history and aesthetics in 1947. That same
year, she left Hungary and settled in Paris. Quickly choosing her artistic family in geometric abstraction, during
the 1950s she was a radical constructivist who refused to label what she considered, with the theoretical support
of her husband François, a scientist and psycho-physiologist, as "art," viewing it instead as experiments.
In this spirit, she invented an "imaginary machine" in 1959, a proto-computational conceptual process that
involved giving precise composition instructions followed by systematic execution, akin to a machine. Taking a
significant step in 1968, she began using a computer to assist in creating artworks based on programs and
algorithms, becoming a pioneering artist in France in generative drawing. Curious about all materials and mediums,
she drew, glued, painted, sculpted, programmed, photographed, created installations, artist books called
"livrimages"[the French contraction of the words ''book'' and ''image''], and an impressive intimate diary
(1976-2020) in twenty-two volumes of nearly 5000 pages. Whenever possible, Vera Molnár also created prints.
Trained in Fine Arts, she learned wood and linoleum engraving at a young age but later collaborated with
professionals to produce numerous publications. In 2022, she decided to assemble a significant collection to
donate to the INHA library, consisting of 264 prints, 2 portfolios, and 12 preparatory models.
This donation forms an exceptional collection, representing the entirety of the artist's work, unique in French
and foreign public collections. Her works belong to formal families that she worked on, revisited, and reinvested
in for several decades. The Sainte-Victoire Blues directly reference Cézanne's motif, drawn and
painted hundreds of times by Vera Molnár. Lettres de ma mère echo the computer modeling of
the artist's mother's handwriting, who wrote to her from from Budapest, which over the years became increasingly
jagged at the end of each line, described by the artist as "a bit gothic, somewhat hysterical, "yet providing her
with a sensation of "visual bombardment" each time. 4 carrés rouges,
Brèches, 3 triades 3 couleurs, Ordinateur miroir de la main, Hypertransformations hommage
à Klimt, and many other series explore the paradigmatic form of geometric abstraction: the square.
Cut, shifted, overlapped, joined, pushed, covered, giving rise to other shapes, incisions, triangles, quadrangles,
perhaps letters, the squares disassemble, suggesting a figure of balance or generating a perceptual indecision,
leaving it to each individual to contemplate and decide for themselves.
Upon news of her passing, INHA seeks to
perpetuate the living legacy of an immense creator; that of an adventurer of shape, of order and disorder, who
used and challenged tradition. Behind Vera Molnár's programs, algorithms, and systems lie thought and
ideas, as well as tenderness and amusement; in short, life. Today, we bid farewell to an artist generative in
every sense."
With the kind permission of the authors, INHA's director, Éric de Chassey, the director of its library,
Jérôme Bessière, and Vincent Baby, project manager at INHA, close friend and Vera
Molnár specialist.
Symposium Proceedings
Introduction
SYMBIOSIS – ISEA2023 PROCEEDINGS INTRODUCTION
FOR A SYMBIOTIC APPROACH AT ALL SCALES
THE NEED FOR INSTITUTIONAL RECOGNITION OF PRACTICE-BASED RESEARCH
(‘RECHERCHE-CRÉATION’)
KEY STRENGTHS
1. ORGANISATIONAL FORMS ARE ALSO CREATIONS
2. CRITIQUE OF THE OLD BINARITIES THROUGH PRACTICE-BASED RESEARCH
3. EMERGING DE/SUBJECTIVATION PROCESSES
4. GENERIC TOOLS DESIGNED AND SHARED BY COMMUNITIES OF PRACTICE
5. BREAKING DOWN DISCIPLINARY BARRIERS AND THE EMERGENCE OF NEW RESEARCHER PROFILES: EVALUATION AS A
TOOL FOR LEGITIMIZATION
6. PERCEIVE BEYOND OUR USUAL CAPACITIES TO BECOME AWARE OF SOCIETAL AND ENVIRONMENTAL TRANSFORMATIONS
7. WHEN CREATIVE RESEARCH BECOMES CRITICAL ACTION-RESEARCH
CONCLUSION
For a symbiotic approach at all scales
Of the 660 proposals received
for the academic call (and a total of 1,400 including the call for artistic projects), from over 70 countries,
230 were selected, including 99 short and long papers, for the Symposium held at the Forum des Images, in the
heart of Paris.200
international researchers and experts evaluated these applications in a double-blind process. More than 50
global organizations, including schools, universities, research bodies and associations, were selected to
present their research activities and facilitate exchanges among them. In addition, more than 50 partner venues,
such as art and design schools, galleries and institutes, offered a rich complementary program of exhibitions,
performances and debates on a national scale. This mobilization of a range of contributors, evaluators, and a
public of several thousand spectators once again attested to the vitality of ISEA, an annual event initiated
more than 30 years ago by pioneers.
The central theme of the 2023 edition was Symbiosis, as demonstrated by the 99 previously unpublished
papers in these proceedings. The full program, including these papers and all the other symposium formats - artist
talks, panels, workshops, tutorials, posters and demos - can be consulted online. This program was in itself a
form of symbiosis.
But what types of symbiosis are we talking about exactly? How do they manifest themselves? Going
beyond the strict definition of the term and interpreting it as an operative metaphor, how can we envisage and put
into practice a genuine ‘symbiotic’ approach? The aim of the call for entries was to encourage the
reconsideration of any creation, experimentation or research through this prism, even if symbiosis was not
necessarily the main theme. ISEA2023 offered a
sort of ‘core sample’ at a given moment of the different strata of contemporary creation and
research underway in the world.
Our idea was that the symposium's programming should also serve as a collective manifesto, promoting
the fundamental principle of symbiosis to be respected at all times and places: a lasting and mutually
beneficial association between organizations and actors, whether human, non-human or hybrid. Considered in
a broad sense, symbiosis also manifested itself at the level of the symposium itself: it was a transdisciplinary
event combining creation and research (in the fields of visual arts, theatre, music, design, film, human and
social sciences, engineering sciences, natural sciences) and cross-sector (involving art, craft, industry,
research, education and cultural innovation).
Before highlighting what we consider to be some of the strengths of this particularly rich body of
work, in terms of both knowledge creation and innovation, it is important to mention our commitment to promoting
practice-based research in all its forms and formats. This approach fosters the invention of modes of cooperation
between different areas of expertise and enables the emergence of hybrid researcher profiles. Symbiosis does not
lie in the homogenization of its components, but rather in the creation of a hybrid and symbiotic entity, where
each element is essential to the other in devising new ways of existing. Together, they offer innovative ways of
perceiving, understanding and acting on the world.
The need for institutional recognition of practice-based research
(‘recherche-création’)
It often happens that artist-researchers or designer-researchers are valued for their creative
abilities, while at the same time being perceived as illegitimate in their research skills. Indeed, there is a
perception that one cannot be both a researcher, publishing scientific articles for example, and a
creator, exhibiting one’s work. This perception differs from country to country. In countries like
Canada, for example, this is not a problem. In France, for example, despite recent progress, not all research
institutions are yet clear about this. Yet these publications and exhibitions (‘publicizations’) both
stem from the same practice-based research process. New generations of researchers are challenging this
perception, and ISEA is a testament to this, since the event enables creators and researchers to share their
research results in an international A-rank publication that meets the standards of peer-reviewed scientific
journals. Institutional structures have also become aware of this reality, as shown by the doctoral programs
specializing in practice-based research which, through their official recognition, mark a fundamental evolution
both in the academic world (universities, art and design schools) and in the professional sector, and not only in
the cultural field (health, transport, housing, ecology, etc.). However, we can only encourage research and
teaching institutions to take this major development into consideration, by proposing, for example in countries
where it does not yet exist, the status of lecturer, associate or full professor
(‘enseignant-chercheur’ in France) in art and design schools.
The older generations, pioneers who are still very active, were also present at ISEA2023. In Paris,
for example, the pioneers of digital arts active since the 1980s rubbed shoulders with doctoral students just
beginning their research careers. This cross-generational aspect is rare enough to be highlighted and celebrated.
Key strengths
As Academic Chair and co-Chairs, we propose here to highlight what we see as the strategic trends
running through all the papers. This collective work reflects our different disciplinary fields, countries and
educational backgrounds. Our aim is not to summarize such a rich corpus, but to propose several common threads to
encourage more in-depth reading of the articles and, of course, to contradict or consolidate our proposals.
Although this exercise is limited and subjective, partial and biased, it is nonetheless enriching. It
reveals that, despite the diversity of the proposals, it is possible to find convergence points, despite the
divergences, and even, sometimes, diametrically opposed approaches. The theme of symbiosis encourages us to look
not for what unifies, but rather for what the different perspectives or visions proposed, even if they take
different forms and are based on heterogeneous conceptual frameworks, can generate together. After all, symbiosis
thrives on difference.
1. Organisational forms are also creations
Faced with the urgent environmental and democratic challenges of our time, the development of new
collaborative methods is essential. Creative research is an innovative methodological response, offering an
effective means of strengthening and better coordinating interdisciplinary dynamics. It opens the way to a new era
of organizations and individuals working together in symbiosis. Although the approaches may vary, the common
objective is clear: to give increasing importance to research within institutions and to recognize the vital role
of associative networks and companies involved in this process.
Organizations, whether stable or changing, are study entities in their own right. Namely, E. Armand,
A. Asensio and D. Foresta encourage the creation of new cooperative ecosystems between the arts and the
technosciences, underlining the importance of collaborative innovation. Transforming organizations from within is
also seen as a promising way forward. Y. Hu, C. Chou and Y. Kakehi, for example, are calling for the creation of
laboratories within art centers, as they have done by setting up a ‘controlled environment space’ at
the heart of an exhibition. Y. Breuleux, A. Thibault and R. Lapierre, for their part, advocate the idea of
occupying and reappropriating industrial wastelands, which are often included in digital art festivals, to give
them a new lease of life.
Organizational architecture also extends to the digital domain. V. Hilsberg looks at the use of DAO
(Decentralized Autonomous Organization) based on blockchain, arguing that this technology has a
direct influence on artistic creation through distribution methods, perhaps recalling Marshall McLuhan’s
famous adage: ‘the medium is the message’. This specific focus on the medium or media is a key feature
of ISEA.
Print media have always been and remain essential to the evolution of academic structures. C.
Frémontier-Murphy traces the history of the journal Leonardo, published by MIT since 1966, which
became an international benchmark by promoting a then nascent field of research. More recently, S. Bianchini and
G. Lallemand have broken new ground by launching the online academic journal ‘.able’, with the support
of more than 30 international partner institutions and the Arts and Sciences Chair at Polytechnique, EnsAD and the
Daniel and Nina Carasso Foundation. This new image-based medium does not seek to compete with the written word,
but rather emphasizes the importance of the visuals generated by practice-based research projects, referring to
them as ‘publicizations’. This approach is based on the fundamental idea that to publish is to make
public. In the academic context, this means subjecting new knowledge, whether textual or visual, to the critical
scrutiny of peer communities.
When Leonardo was created, organizational innovation was also underway in South America.
J.-C. Mariategui recalls the birth of the Centre for Art and Communication (CAyC), a Latin American network
dedicated to art and cybernetics. Today, international collaboration between researchers has become the norm, as
shown by the example of SEADS (Space Ecologies Art and Design), a transdisciplinary and multicultural collective
on a global scale.
Sometimes developments are the result of unforeseen circumstances, sometimes predictable but poorly
anticipated by public authorities. Recently, the Covid-19 pandemic forced cultural institutions to reinvent
themselves. N. Ricci and M. Agogué share the lessons learnt by the Montréal festivals, illustrating
the resilience and adaptability of organizations in the face of global challenges. These organizational issues
also have their counterparts in the production of works. For example, W. Zhang and S. Su make visible the
‘voids’ between individuals subjected to the social distancing put in place during the Covid-19
pandemic.
All these examples illustrate the growing interest in the creation of new organizational forms in
various fields. The concept of symbiosis originated in the natural sciences. By borrowing the vocabulary of these
sciences, it is possible to suggest that all these organizational forms together give rise to new ecosystems. The
term ‘ecosystem’ was proposed by Arthur George Tansley in 1935 in the journal Ecology. The
concept then gradually migrated into the human and social sciences, particularly economics. Today, it is often
used to designate a complex environment that encourages creation and innovation. Widely adopted by those involved
in entrepreneurial innovation (makers, startupers, entrepreneurs, etc.), this metaphor of scientific origin can
also be applied to art and design research.
These ecosystems, through their interrelationships, are contributing to the emergence of a renewed
‘biosphere’, forming a vast practice-based research network on a global scale. This network can be
seen as a favourable climate for invention, a phenomenon that Nowotny, Scott and Gibbons had already described in
their analysis of knowledge production in contemporary society in 2001. Thus, this organizational and creative
landscape can be compared to a dynamic biosphere, where innovation emerges from interaction and collaboration, as
Wenger, McDermott and Snyder emphasized in 2002 in their study of communities of practice. This is the case in
Québec, but not everywhere. Like any ecosystem, the equilibrium found is fragile and subject to forces that
can destabilize it, as in the case of Symbiotica, which is now threatened with extinction, even though it is a
laboratory with an international reputation for its work in bioart. Ionat Zurr, an artist-researcher and
co-director of this structure, spoke about this during her keynote speech. She also appealed to our
community for help, which we are relaying here.
Each new organizational arrangement is both the product and the result of a process of transformation
which also operates within conceptual frameworks. It is not surprising, therefore, that these frameworks are
themselves put under strain by these arrangements, which ‘reorganize’ - or, depending on one’s
point of view, disorganize - traditional ways of thinking.
2. Critique of the old binarities through practice-based research
The questioning of the traditional dichotomy between culture and nature is now shared by many in the
academic community, particularly in anthropology. Artistic works embody and make perceptible this profound
reconfiguration of paradigms of thought. N. Georgakopoulou, D. Zamplaras, S. Kourkoulakou and C.-Y. Chen explore a
‘sympoetic’ interaction with materials in interactive art, inviting us to question old dichotomies
such as subject/object, mind/body, and nature/culture, in order to embrace a new understanding of materiality. P.
Purg and K. Pranjić oppose a human-centered science, putting forward an ‘emergent symbiotic
mutualism’ that transcends mere species coexistence, from a posthuman perspective. M. Zolotova, for her
part, sees the posthuman turn in many contemporary discourses as a serious attempt to move beyond
anthropocentrism. Such approaches are therefore not identical, and can even sometimes be seen as opposed, but they
all involve a critique of analyses centered solely on the human.
This criticism can be found in other fields too. For example, Bakke questions biocentrism and focuses
on the interactions between minerals and non-human life forms. Adopting an approach of working ‘with’
rather than ‘on’ is another way of challenging these human-centered biases. M. da Rocha Montanari uses
generative art as a collaborative research methodology with indigenous Guarani and Kaiowá communities,
co-creating a ‘poetics of attention’ that draws on sounds and plants in their cosmology. A. Wollensak,
B. Terry and B. Baird present extracts from community testimonies relating to water, accompanied by processed
environmental sounds from Alaska.
For R. Wright and S. Howden, the adoption of indigenous perspectives is crucial to developing
‘sentient machines’ capable of reflecting the complexity of ecological patterns, where computational
ecology fails. C. Schnugg, D. Brill and C. Stary merge aesthetics, performance and digitalization to reinterpret
the world, going beyond the rational production of knowledge. Although these initiatives are diverse, they share a
common goal: by hybridizing different rationalities and subjectivities, they seek to transcend existing approaches
in order to invent new ways of understanding and addressing contemporary crises, both geopolitical and climatic.
3. Emerging de/subjectivation processes
These methodological and epistemological reflections are accompanied by a transformation in the ways
in which we conceive of human ‘identity’, the conceptual as well as practical ‘making’ of
the way in which we define ourselves, or not, as human subjects. Indeed, the processes by which
individuals construct themselves as subjects (a process of subjectivation), or sometimes deconstruct themselves (a
process of de/subjectivation), are present in many practice-based research projects. For example, the performance
by T. Chiaravalloti and A. Meshi’s, explores a symbiotic relationship between humans and an artificial
intelligence algorithm, enabling participants to experience a ‘collective consciousness’. Although
this work aims to highlight the limits of AI’s ability to recognize emotions, it also opens the way to
reflections on future developments in this field and the ethical dilemmas they raise. P. Gemeinboeck and Rob
Saunders, for their part, propose ‘hybrid entanglements’ between humans and robots, based on bodily
empathy between humans and non-humans. S. Cîrcu and C.Y. Chen explore movement to question the
anthropomorphism of humanoids through a mimetic dance between a human and a robotic arm.
These approaches use translation, imitation, differentiation and empathy as means of criticizing, but
also realizing, the hybridization between machines and humans as formalized by a number of artists and theorists
at the Symposium: M. Smigielska uses AI to take account of spectators’ behaviour, E. Amato & E.
Péreny claim a ‘co-individuation’ with our technological avatars, J. Bae and his VR comic strip
explore the relationship between humans and AI, D. Batsis, M. Grigoriadou, or with K. ElRaheb and A. Politis, who
advocate the body as a phenomenon and an extension of ‘hyperreality’.
Even the most fiercely critical creative research into intrusive technologies is unwillingly
participating in this general trend. C. Brunner and J. Fritsch look at the transhumanist movement through the
prism of Gilbert Simondon’s thinking, in particular his notion of ‘human energetics’, and are
inspired by a video work by the Nigerian artist Otobong Nkanga. G. J. Shin, using the concept of the
‘superjet’ in a post-digital context, highlights the emergence of new creative human subjects, capable
in her view of weaving new social relationships.
This critical exploration of the way in which humans conceptualize and experience themselves also
manifests itself in a decentering of the human, highlighting the processes of subjectivation at work in the
‘non-human’ or ‘other-than-human’ world. Adopting a Bruno Latour perspective, elements
such as forests and water are no longer simply reduced to objects of study for humans; they are seen as actors in
their own right, subjects in the active sense of the term. In this vein, G. Trudel, who studies climate change,
integrates data from tree communities, the media arts and forest sciences. She attributes a ‘voice’ to
trees through what she calls ‘individuations of symbiotic modulations’, drawing on and extrapolating
the work of Gilbert Simondon.
Approaches to thinking about and experimenting with symbioses between humans and machines, as well as
between non-human entities, vary widely. However, they all point to a probable reconfiguration of the general
cartography of subjectivation processes: the contemporary human seems ready to move away from the central and
dominant position it has hitherto attributed to itself. At least, that is what the observations and discussions at
the Symposium seem to indicate. The question is what this will mean in the long run, and whether the promise of
emancipation will really be fulfilled.
4. Generic tools designed and shared by communities of practice
As the notions of objects and subjects evolve, research tools and methodologies are also changing.
This transformation is contributing to the emergence of new practices and methods for ‘measuring’
phenomena that were previously difficult to quantify. The evolution of instruments and their impact in the field
of creative research are particularly interesting. Just as there are creators of works of art, there are also
innovators in the organization and design of instruments. Artists, designers, scientists and engineers frequently
collaborate to provide research and creation with platforms and tools that are not only at the service of
creation, but that also stimulate innovation among those who use them.
For example, MARCEL (Multimedia Art
Research Centres and Electronic Laboratories), of which the pioneer D. Foresta is one of the designers and
promoters, offers an online collaborative platform dedicated to the specific formats of the interactive arts, a
project that was born in 1997 and, even then, was the continuation of a process of reflection and practice that
began in 1981. This desire to offer shared tools is obviously not a recent development and is now reflected in a
wide range of proposals, from generalist platforms to tools dedicated to specific fields or uses. S. Thurow, D.
Del Favero, M. Scott-Mitchell, M. Ostwald and H. Grehan have developed a network modelling system which
facilitates the planning of opera rehearsals thanks to an interactive ‘cyber-physical’ spatial
aesthetic, opening the way to new imaginaries. C. Lengelé and P.-A. Gauthier traced the genealogy of
‘Live 4 Live’, an open-source tool designed to simplify the creation and real-time control of
spatialized sound objects. D. Cunin, E. Durand and M. Seta developed generic devices to facilitate the creation of
immersive environments with video tracking and collective interactivity on smartphones.
This trend towards
developing platforms for artists and designers is also to be found in the field of curation. Y. Hofmann, through
the ‘intelligent.museum’ project, has put forward a system based on data analysis for artistic and
cultural institutions. This prototype uses data from a variety of sensors in exhibition spaces to tailor museum
experiences to visitors’ needs. I. Družetić-Vogel, A. Fuchte and M. Bauernfeind proposed
‘ARt chat’, an application for museums that combines augmented reality, art and communication,
allowing visitors to share their impressions in the exhibition space.
These initiatives are not just about providing functional services; they can also take a critical
approach. V. Guljajeva, M. C. Sola and I. J. Clarke, for example, have analyzed several existing tools in the
field of AI, stressing that ‘AI does not create a work at the touch of a button, but requires a deep
understanding of the underlying technology, as well as a creative and critical approach’.
In addition
to the creation of platforms, there is also a trend towards modelling, although approaches vary considerably. The
very term model indicates a desire to provide communities with frameworks based on general principles to be
adopted. Z. Wu, D. Fei, X. Ma, M. Fan and K., for example, see NFT as ‘a sustainable business model for
media arts, involving audience interaction’. Others, such as D. Xu, M. H. Lamers and E. van der Heide,
propose a ‘relational model of co-located interaction’ between spectators and an interactive
work.
These various proposals, although heterogeneous, share a common view: they point to a constant trend
towards the uninterrupted emergence of specialized communities of practice that design generic devices for
instrumental purposes. These communities are also expanding thanks to the instruments they are inventing, enabling
their practices to be disseminated more or less widely, a form of ‘standardization’ with a creative
aim, while at the same time encouraging the integration of their own approaches to uses in the fields of art,
often including the active participation of audiences as a central component of the design of the works.
5. Breaking down disciplinary barriers and the emergence of new researcher profiles: evaluation as a tool for
legitimization
ISEA stimulates multidisciplinary exchanges while consolidating the existence of a specific community
through annual gatherings. The enthusiasm for creation, in synergy with technology and science, forges a solid
link between researchers and creators from a variety of backgrounds. This positive dynamic encourages the exchange
of diverse perspectives, whether practical, theoretical or methodological, around a common subject, a specific
field, or more simply a medium or instrument. The cross-disciplinary aspect of these interactions is particularly
enriching, as it helps to create a diverse community while respecting the disciplinary affiliations of each
individual.
Adopting a double-blind peer review process, ISEA’s call for applications conforms to academic
standards and promotes cross-disciplinary or multidisciplinary approaches. In an academic environment that is
often segmented by discipline, this recognition is crucial. The symposium thus provides a sanctuary for
unclassifiable minds, while giving them international exposure. The evaluation criteria we have chosen enable
artist-researchers and designer-researchers to assert their legitimacy.
The criticism often levelled at the latter - that the dilution of the boundaries between the arts and
sciences would jeopardize the quality of research and creation - ignores the symbiotic nature of these two spheres
of creation. In fact, they are mutually nourishing, while retaining their specificity, whether through collective
work or individual skills, not forgetting the importance of the distinct ecosystems to which these players belong.
Although this does not systematically guarantee the relevance of the results, whether scientific or artistic, this
approach is like that practiced in professional art and science circles, where peer judgement is commonplace.
ISEA’s aim is to bring these hybrid profiles to the fore. Although cross-disciplinarity can be
perceived as a risk of levelling out skills, it is essential for innovating, experimenting with new methods and
tackling issues from a fresh angle. The contribution of these cross-disciplinary approaches can be extremely
beneficial to established disciplines and sectors, enabling them to evolve and renew their own fields of research.
Venturing outside our disciplines through creative research allows us to rediscover them in a new light.
If practice-based research can create new knowledge in science and engineering, it also brings new
practices to the field of creation. Y. Blanchi, for example, links cognitive science and architecture, while S.
Gatz criticises architecture as a means of control by proposing a ‘cosmo-techno-poiesis’. Urbanism as
it exists is questioned by artistic and digital practices, like the infrastructures of networked urban screens
imagined and implemented by M. Thorogood, K. McCulloch and A. Dulic.
In dance, N.N. Correia, D. Souza, I. Nêves and J. Lobato are trying to make the dance process
understandable within a performance using a multisensory approach. I. Teles de Castro e Costa, H. T. Hong and C.
Y. Chen have designed a performance co-created with an autonomous virtual system. In the field of cinema, R.
Sagot-Duvauroux, N. Quaetaert, F. Garnier and R. Ronfard explore how film editing cannot be transposed into
virtual environments and must be reinvented. Questions about the transposition of creative techniques into VR also
arise in the case of Chinese calligraphy with Shum, P.Y.S. & Klein. C. Salter, T. Thomasson and P. Uro are
carrying out what they call a ‘theatrical exploration’ based on augmented reality, with the aim of
making climate change visible.
The ‘adjustment’ between different vocabularies is necessary to give an account of this
research. O. Kobryn, M. Couteau, R. Sagot-Duvauroux, S. Balcon, F. Garnier, R. Ronfard and G. Soulez, researchers
from different disciplines (aesthetics, ergonomics, design, engineering, cognitive sciences), are working to
reconsider the concept of the ‘virtual’.
In the cultural sector, curation and conservation are also invested with methods derived from
practice-based research to redefine or apprehend their activities in a new way. M. Jones, M. Wester and M.
Blottiere present a new approach to curating as practice-based research and propose the acronym ‘CRC’
to establish it as a specific field. D. Irrgang and M. Skłodowska take up the curatorial notion of the
‘exhibition of thought’ co-invented by Bruno Latour and Peter Weibel, who have put it into practice
with curators, artists and researchers at the ZKM.
The circulation of thoughts and practices, the mutual acculturation of different research players and
cross-disciplinary cooperation are all expressions that would remain abstract without these numerous experiments.
These compositional arrangements, to paraphrase Latour, make us see differently.
6. Perceive beyond our usual capacities to become aware of societal and environmental transformations
Making perceptible what naturally escapes our senses is not a new concept in the world of art, or
even in that of design, as the example of decasualization shows. This approach is even less novel in the field of
science, where modifying our perception and understanding of the world forms the basis of a great deal of
research. Nevertheless, the way in which this is done varies from one field to another, with a current tendency to
merge different approaches in order to make discoveries or visions of the world that are not immediately
accessible - often because of their scale - ‘sensitive’, i.e. perceptible to the senses.
In
this context, many research and creation projects aim to reveal new sonic and visual ‘landscapes’,
constituting what might be called an informational aesthetic. There are two main thrusts: the first seeks to
materialise technological artefacts, which are increasingly miniaturized and interconnected; the second aims to
raise awareness of climate change through the collection of biological or geophysical data, with the two thrusts
sometimes overlapping.
In the first category, H. Scurto and A. Chemla-Romeu-Santos generate soundscapes
from data collected in 28 locations around the world via the Locustream online platform, highlighting AI
infrastructures. On the other hand, M. Lukaszuk and S. Bahng exploit gesture and spatiality through
electroacoustic and video improvisation to make intelligible the preponderant role of machine learning.
In the second, Mc Farlane proposes a method of co-creating artistic acoustic ecologies centered on
the Great Lakes in the USA, combining sound data collection and research methodologies enriched by indigenous
knowledge.
In terms of sonification, L. Foo and J. Fritsch create a ‘cryogenic landscape’ to engage
listeners emotionally with climate change, broadcasting the movement and melting of the ice in Greenland almost in
real time, presenting the ice pack as a living being. J.C. Duarte Regino goes a step further by making atmospheric
processes audible, helping us to perceive our atmosphere beyond our usual senses.
At the intersection of
the two, L. Moren and T. Bachvaroff's ‘Under the Bay’ is an augmented reality project that turns
smartphones into ‘microscopes’ to explore marine life invisible to the naked eye. J. Ottavi and J.
Pickett are exploring the interaction between humidity and the decomposition of printed circuits, converting soil
acidity and electronic compost into electrical signals.
Visibility projects can also illustrate
disappearance, a radical type of invisibility. H. Sareen, Y. Fu and Y. Kakehi depict endangered species as
micro-bubbles formed by the nucleation of CO2 in water, which shrink and disappear in a few days to raise
awareness of the threat to biodiversity. B. Ammar-Khodja brings to life the residues of heavy metals in
contaminated urban landscapes, highlighting a kind of point of no return if the pollution persists. G. Legrand, V.
Thornhill, I. Clarke, through the installation ‘Spectral Plain’, explore how information technologies
can integrate and reflect specific socio-cultural beliefs, creating new symbiotic interrelationships with the
non-human and the non-living, thus influencing the participants' perception of their understanding of the world.
These practice-based research projects aim to make complex issues or abstract realities tangible.
Designed to transform our perspective, they encourage us to adopt different behaviours and develop new critical
approaches.
7. When creative research becomes critical action-research
N. Georgakopoulou and her collaborators aspire to create a ‘sonic consciousness’ within
urban spaces. Their project aims to create musical experiences that invite active participation from citizens,
while paying special attention to people with hearing and visual impairments. For his part, R. Ridgway strongly
criticizes the collection of personal data by the main players in the network. In the same vein, B. Gaylor and K.
Hennessy encourage the development of a critical understanding of Big Data to better counter the ‘piracy of
affect’.
From a techno-feminist perspective, G. Lautenschlaeger explores post-human maternity in media art. P.
Costa, L. Ribas and M. Carvalhais analyze the tendency to feminize digital assistants and suggest strategies for
reversing this trend. L. Haute urges us to think about our methods of collaborating with the environment, inspired
by the symbiotic culture of bacteria and yeast. C. Gould and Isabel Stenger urge us to take action to initiate
change in our lifestyles and consumption habits, while J. Garančs explores, through virtual reality,
dystopias based on market values.
Y. Zhang, in his interactive installation, uses machine learning to generate fictional speeches by Xi
Jinping and Donald J. Trump. P.R. İ. Yeginsu highlights the need for artists to find new ways of
collaborating to avoid the negative impacts of commercialization linked to NFTs. The fight against social
prejudice is also a recurring theme, with S. Brueckner, S. Yeung, J. Liu, D. Choberka, K. Shedden, J. Turner, I.
Gillet, M. Lu and Xingwen Wei identifying the biases present in face detection and race classification.
Faced with the environmental crisis, techno-solutionism is criticized through the surrealist and
‘pataphysical’ approach of M.-L. Bourgeois’s project, without systematically rejecting
engineering, as many of the projects are situated at the intersection of the arts and the engineering sciences.
The appeal to various forms of decolonization is also marked, namely with H. Rashtian and G. Aceves-Sepulveda who
question the construction of historical narratives.
These contributions, which oscillate between denunciation and calls for resistance, aim to provide
the public with critical, conceptual, sensory and emotional tools to raise awareness of the issues of pollution,
gender and democracy. The subjects addressed are varied, and the methods proposed are diverse. Although critics of
these works may fear that activism will predominate over rigorous research, it is clear that these contributions,
by their provocative nature, offer food for thought and relevant tools for the scientific community, inviting
ongoing debate among peers.
From this perspective, this type of action research fulfills a fundamental dual critical function: on
the one hand, it is devoted to exploring the invisible, hidden, unexpressed, or even oppressed elements within
society; on the other hand, it simultaneously engages in the struggle against doxas, including those with which it
finds itself unwittingly associated. This bifocal approach is relevant to all fields of study, encompassing both
research and creative work with a critical intent. This approach enables a clear distinction between activist
practices and research activities, thereby amplifying their respective impacts through an enriching critical
symbiosis.
Conclusion
As we have emphasized, the choice of these key strengths represents a challenge, due to its
necessarily limited and subjective nature. Not all the authors are cited in this introduction, due to lack of
space, and we can only encourage readers to read all the articles. We could have highlighted other aspects, such
as the importance attached to feminism, gender issues, the ecological transition, or the various processes of
decolonization. These are essential thematic entries, but they are already very present in current debates, and we
offer additional reading keys. It was also possible to group the proposals according to the type of technologies
used or invented, or by sector of activity. The thematic clusters we proposed during the Symposium could have
encouraged us to do this. However, we have chosen to base ourselves on all the contributions published in these
proceedings and to resume the programmatic work in ‘bottom up’ mode. This approach offers a new
perspective on how everyone can benefit from this work for their own research.
We hope that these guidelines illustrate, if proof were needed, that these proceedings are a rich
source of ideas, methods and formats that are both varied and innovative. It is also important to note that the
proposals go beyond the scope of these proceedings, and we encourage you to explore the online video archive
platform, including artist presentations, demos and roundtables.
These exchanges between art, design, science and technology are essential for generating knowledge
and creating meaningful works. They allow us not only to analyze the present (a form of real-time analysis), but
also to lay the foundations for future discoveries and new approaches that are still being developed. These
initiatives can be seen as precursors, offering a glimpse or intuition of what is emerging and could materialize
in the next ten to twenty years. In a way that complements science fiction, which imagines the future, these
proposals are starting points for extrapolating the present and considering possible future developments. A work
in constant evolution!
Paris, London, Bilbao, Montréal,
Emmanuel Mahé (ÉcoledesArtsDécoratifs–PSL
University, France, EU), Academic Chair
Elena Papadaki (University of Greenwich, UK), sub-thematic co-Chair
Maria Ptqk (Spain, EU), sub-thematic co-Chair
François-Joseph Lapointe (Université de Montréal, Canada), sub-thematic co-Chair
Three personalities were invited to give their views on current developments in the fields of
creation, technologies and sciences, on the role of these in society, and to present their most recent work.
Ionat Zurr (artist-researcher, Symbiotica, Australie) — Symbiosis and the fallacy of a nature-free
existence.
In times of ecological emergency, solutionist fantasies of nature-free human existence promise salvation and
repair. The innovative paradigm offers “products” such as lab-grown (animal free) meat and artificial
automated surrogates to replace reproductive biological bodies.
These so-called innovations require special artificial environments to host, nurture and culturally articulate
this “new” nature-free, decontextualized and colonised life. The entanglement of life with its
surrogate environment/apparatus, echoing human relationships with living and semi-living agents; when control and
care are employed to counter resistance.
Artists, scientists, designers and engineers all play their part in this transformation and its effects on human
relations with life and the environment. This creates a range of ontological conundrums and fantastical
expectations as to what technology can provide and to whom. Using examples of artistic research that deal with
emerging technologies and new knowledge, Ionat Zurr narrated artists’ symbiotic and parasitic relationships
with such post nature.
This talk was framed by the imminent closure of SymbioticA, the first artistic research laboratory based in a
life sciences department. SymbioticA began as a symbiotic act, embodied in an academic institution, to enable
critical, yet mutualistic, relations among artists and scientists. Many of SymbioticA’s alumni have
continued to establish their own laboratories and artistic practice in other academic institutions around the
world, leading to the growth of the field of Biological Arts.
SymbioticA is now being treated as a parasite by a changed host body. Is this a ‘natural’ survivalist
rejection against a foreign body or can we detect symptoms of an autoimmune disorder?
Biography: Dr Ionat Zurr is an artist-researcher. She is the Chair of the Fine Arts Discipline
at the School of Design & SymbioticA academic coordinator at the University of Western Australia. Together
with Oron Catts she established the Tissue, Culture & Art Project in 1996 and their co-authored book Tissues,
Cultures, Art, published by Palgrave McMillan this year. Her collaborative work was exhibited by Pompidou Centre,
MoMA NY, Mori Art Museum, Ars Electronica, National Art Museum of China and more. These ideas and projects reach
beyond the confines of art and the work is often cited as inspiration to diverse areas such as new materials,
textiles, design, architecture, ethics, fiction, and food.
Viktor Ruban (Choreographer, independent culture diplomat, Ukraine) — Coping potential of
creativity and art-practices in times of war: culture diplomacy, fundraising, curatorship and art-therapy force
project.
If symbiosis is an essential notion, it is even more so in today's geopolitical context. War, terror and massive
extinction of people in Ukraine by Russia is a global challenge shaping our global future at this very moment. In
this situation performing artists in Ukraine keep on strong commitment to help in any possible way. Since the
beginning of the full-scale invasion in 2022, the situation is forcing us to search for not only new ways of
dealing with challenges creatively but to discover new ways of implementing our art practices and skills to
completely new levels and spheres, such as resistance in information wars, culture ecology maintenance, new forms
of fundraising and help with physical and psycho-emotional recovery — same as military and civilians
—.
During his session Viktor Ruban, shared information about initiatives that he initiated or is involved in, such
as the European Culture Parliament, and culture diplomacy challenges that he faces on different international
events; Ukrainian emergency performing arts fund and funding challenges for the independent performing arts scene
in Ukraine; international solidarity events and visibility of Ukrainian actual art scene—why it is
important; actual creations in Ukraine and trends seen through the actual national theater prize season; the
development of the project for training “psychological first aid instructors for military from the front
line” and Art therapy force project— a range of activities implementing art-practices and working with
creativity for psycho-emotional health recovery, coping with stress and panic attacks as well as preventing
self-destructive behaviors and PTSD for diverse groups of people.
Biography: Viktor Ruban is choreographer-researcher, curator, performer, educator, independent
culture diplomat and culture activist. Initiator and ambassador of Ukrainian Emergency Performing Arts Fund
initiative, he represents Ukraine in European Culture Parliament. He is also program director and co-founder of
venue #KyivDanceResidency—platform for international studies in somatic, dance and performative practices,
movement-based art and research. Ph.d. student in culture studies of Modern Art Research Institute of the National
Academy of Arts (Kyiv, Ukraine)
Michael Century (musician and cultural theorist, Rensselaer Polytechnic Institute, NY) —
Nonsynchronous Innovation: Periodizing the Digital
While the digital transformation continues to outpace socio-institutional adaptation, the technological arts have
moved on to vastly expand their temporal horizons. Timescapes of artistic research and creation now embrace the
residual as much as the emergent; techno-diversity against digital solutionism; sympoietic rather than linear
models of innovation. But nonsynchronous innovation is hardly unique to the current moment, as revealed in Michael
Century’s recent book Northern Sparks, on Canada’s early experimentation with digital media.
Poised as a “counter-environment” to the great powers, in McLuhan’s phrase, Canada’s
experience of the transitional decades into the information age was grounded in a technological ethos that
emphasized sensorial immediacy, embodied interaction, and improvisatory expression. This alternative ethos was
situated between a pair of distinct yet inextricably bound forces, one national-political and proper to Canada,
the other techno-mediatic and global in scale.
The unraveling of these forces by the late millennium reveals innovation itself as a complexly drawn process
comprised of multiple layers with fluctuating degrees of synchronization. From a cross-media perspective, Northern
Sparks also reveals how the differences between the arts with respect to improvisatory immediacy and discrete
formalization make any neat chronological periodization of the digital problematic.
Biography: Michael Century, musician and cultural theorist, is Professor of New Media and Music
at Rensselaer Polytechnic Institute. His book Northern Sparks: Innovation, Technology Policy, and the Arts in
Canada from Expo 67 to the Internet Age appeared with MIT Press in 2022. At the Banff Centre, he founded the Media
Arts program in 1988. Century’s works for live and electronically processed instruments have been performed
and broadcast in concerts and festivals internationally.
How do art and science participate in writing a world of signs and symbols? What do these two human faculties
and capacities seek and how do they differ radically? Science has progressively freed itself from its purpose
(the search for the why) to model, imitate and master a world in an increasingly abstract manner. If science and
art are performative, what world do they draw and what do they tell us (in terms of discourse) about the human?
As science gradually becomes techno-science and produced by industries, does art have a singular role to play in
the interrelation of these two poles, science and industry, in order to re-inhabit the ethical questions and
meanings that were originally in these other two creative capacities? Can it and should it?
Keywords
Art, Science, Industry, Signs, Sensibility, Technoscience, Blindspot, New cooperative ecosystem, incorporation
of experience.
We will share an assessment of the relations between art-science and industry, the novelty being that with
digital technology the questions have been accelerated, augmented and have now come to integrate the notion of
physicality. Industry is a playground; whether it is science, art, how do we seize the digital? Do we have the
means to think the digital or are we alienated, with the digital no longer as an externality, an interface, but
as an immersion, as a new space of being in the world—particularly with the metaverse—as a
socio-technical space? It is necessary to think of industry as a workbench for virtual universes and as a space
of reconciliation between disciplines via the designers as sensitive mediators and scouts of the new virtual
worlds. We need a new epistemology of the virtual worlds that takes into account the multi-dimensional human. We
are looking for ways between micro-ecologies, localities, craftsmanship and a global, automatic approach, the
two poles that build our "being in the world."
Art could be thought of as the fabrication of new categorizations of an environment¹ that escapes the
norms of language and conventional imaginary modelling. Because the perceived world is always the result of an
interpretation and a shaping which are imposed a posteriori by culture, that the inaugural perceptual
singularity is subsumed by culture. The perceptual thresholds linked to temporal representation structure a
vision of reality constructed by stops and artificial linearity². Yet the imaginary, the perceptual virtual
cannot be stopped at these arbitrary forms of actualization of the world. Whether it be time, imaginary, or
spatial perception, they cannot be detached from a body in action that perceptively actualizes a world that
appears and is created in a living time (dancers or performers give rhythm to time, for example). The
manifestation of the event is an extract of the real, obliterating the infinity of data, infraliminary and
supra-liminary information³ not actualized by language whose readings cannot reach the threshold of a
disciplined thought. It is in this failure of a completeness of reality that art operates.
It is an attempt to reveal hidden information inaccessible to authorized cognitive processing. Other
articulations, forms, relations to the world exist that art tries to bring to light. It is a raw, unanalyzed
grasp⁴ that the act of formalization and expression will seek to bring into existence. Art is, first of
all, an impossible return to an instinctive and affective perception, where the body echoes the world and where
the world is then echoed (exosomatisation). Art in these premises is an anchoring (Harnard Stevan⁵) of
signs of the world kept in memory (retention), attached to and mirroring the body that perceives itself (while
keeping the unspeakable abyss which Pierre Legendre refers to ⁶).
As a first reflexive return, art imprints the world in order to reflect on it and to appear itself (first
distance from the world—see the cave paintings and the work of Carole Fritz⁷). It is a primal
affective reading of a world that is not there for it, and for which it seeks a meaning. It is an attempt to
answer the abysmal question of why and of a world that is not up to his standards, but in which, through the
inscription of signs, he can make his presence last beyond his own finitude.
As the first interface between the body and the environment, art is a sign of principle writing. As a symbol
and practical return of a memory and of an instant kept⁸, it is a means for recalling presences. It is an
imprint of an existence and of a difference made with the self and the environment, the self and the other, who
delimits and recognizes itself in it. Through the diffusion of signs and symbols, language and rituals, the
particularity of a seized moment becomes an abstraction, a practical medium to think about the world and save
the detailed and vivid perception that every physical and phenomenal novelty requires. But if art has to do with
the first symbolic anchors, the first perceptions, it cannot be reduced to them.
Art is a first reading, a projection of human meaning on the world, and in this, science does not differ from
this attempt to understand and mark the world. In science, the purpose is different, the tools are more
sophisticated, no singularity is demanded and the dream of a universal and standardised objectivity.
Art and science are imaginary and projective capacities, part of a process of fabrication and transformation of
the world. This capacity for transformation by modern science has largely emancipated itself from its role as a
myth whose function is to answer the question of why live and to limit unreason in order to support existential
anguish⁹.
But art and science are not only intended to think about the writing of a world and its human legibility, but
also to put it into the world, to give it a stage, to create a narrative for it and for the human being. Their
discourse and imagination are performative.
In contrast to science, art is anchored in the body and the affection of the world, turned towards the
appearance, the distinction, the singular discrimination of signs as a search for actualization. Science in its
hyper-technical devices sets aside this affection for the body, and is based on instrumental rationality.
It is important to underline representation acts as a perceptual framework indicating the phenomenon and giving
it an interpretative form in order to apprehend it.
Science, as a rationalized exo-body, it gives us the opportunity to perceive a phenomenon (a
techno-phenomenology) and a world framed in advance by the laws inserted in it, offering a metric vision and
mathematical logics, influencing in its turn the way of writing and understanding a world surrounded more and
more by technical abstraction, self-referential and discarding the perceptive variability of a sensitive and
feeling body.
Techniques are extensions of organs¹¹ reading and writing the signs of the world. Art has taken an
interest in the world of science and technology out of a spirit of resistance (perhaps) to the homogenization of
bodies and sensibilities, out of a refusal to accept the authority of scientific discourse, which sometimes
imposes "butchering"¹² and reductionist views of the human being. The definition of man varies
according to culture, religion, and time. Feeling human, our relationship to the world is defined according to
the interpretative frameworks conveyed in a culture, which cannot be absolute. We know that the way in which man
is expressed influences in turn the experiential potential and the power of his experience.
“Cogito ergo sum”
“Cogito ergo sum” of Descartes has been the heart of the scientific domination of our
society’s world-view, the way we understand the world defined uniquely through reason. It has been
responsible for an unequaled level of material progress but has reduced humanity into entities befitting
uniquely mechanical formulation, ignoring the whole human being, concentrating only on those aspects of the
person considered important for the task at hand. In recent western history we, as a culture, have tried to deny
this human wholeness by declaring that rationality was the only approach to true knowledge and that all else
must be rejected. Anything resulting from feeling or emotion was unreliable and even dangerous. That added to
the mechanically quantifiable view of how things work has created a distorted perspective, a world-view based on
a diminished idea of the human being. Focusing on the intellect was a useful but limited approach to how
humanity is defined putting aside, "Sensio ergo sum"—I feel therefore I am—which is how most people
first react to sensory input. Recognising that aspect of humanity not only allows us to understand the whole
human by respecting the other half, it also connects our species with the rest of the living world which reacts
principally through feeling. Objectivity is a tool, an important one, but to imagine humans cut off from their
emotional side is dangerous and leads to often undefined but very real frustration as we see more and more
today.
The capacity for symbolic thought has always been part of our makeup in its earliest primitive forms since the
beginning of our genus and most probably well before. It comes from an instinctive and unintellectual operation
of that curiosity which is part of every living being’s functioning as a way of exploring our environment
and surviving in it. When the found object of our ancient ancestors, the first overt manifestation of what we
now know as art, was kept and shared with others it acquired a symbolic sense for the group as well as the
individual who found it, an unarticulated meaning but something felt to be important. With the evolution of our
intellectual capacity, this process becomes more elaborate in providing a «why» to that selection
giving those objects an explanation of their symbolic value and a means of communicating it and often the
beginning of what can be seen as a religious belief system. In the beginning, this was not an intellectual
exercise and the choosing came from feeling rather than understanding. It became more intellectually elaborated
in time through the need to be communicated, probably diluting the experience of the first-person experience but
making it transferable to others and incorporating other reactions to it.
This is the root of artistic creativity and its eventual integration into culture. The symbolic is very much
part of us and ignoring it or pretending it is less important or to be overcome demonstrate a dangerous
misunderstanding of how humans function. To attempt to understand our confrontation with the real, both art and
science are needed for a fuller and more human comprehension of what we are defining. Understanding
intellectually is necessary but stopping there leaves the human wanting more, which defines the emotional
atmosphere we live in today. We cannot expect people to understand when half the person is ignored. In our
mechanical world-view art has been pushed out of its central role of informing into manageable categories to fit
the demands of the market it has been confined to.
Artists, on the other hand, have never ignored science, particularly in a society which has allowed science to
be the sole arbitrator of our reality. Science has thus become the matter of art, our reality subjected to the
questioning of art. Technology, the product of science, has always attracted artists because of a curiosity
about what humans do but also because it is a potential tool, particularly the technologies of communication.
Importantly as well, the artists’ use of technology has often influenced our perception of it and how it
is integrated into society and affects us.
A technology developed through both art and science would, by definition, have more dimensions by being the
product of the full human being. This has been the objective of our project MARCEL, an international group
dedicated to artistic, scientific and educational uses of the network and expanding network technologies through
the demands of art and not just market-driven consumption. It exists and evolves through the share experiences
and technical development from creative uses of the network pushing the boundaries of the possible. This is also
the approach that the TRAS network (transversal network of art-science networks) is trying to put in place in
France. As are the creative platforms in industry that seek to give meaning to technology, since industry is the
heart of its production.
Industry as a blind spot?
Industry is perhaps a blind spot in our times. Principally thought of as an operative process, a technical
milieu and not enough as a thinking object. As a civilization manifestation of a way of seeing the world.
Industry always been a space of intervention and convergence in which an art-science dialectic can flourish.
At the crossroads of the most advanced human creative activities, the industry is projecting a vision,
organizing the work and the economy defining every aspect of everybody’s life. The industry today
manufactures the eye and the prosthetic body of tomorrow, just as science and art have done. To think of working
in theatrical and artistic terms on science, digital environment or technologies that enable all aspect of the
modern world without taking into account the environments in which techno science is constructed may be a
mistake. The question of the representation of the body, of what constructs the human experience, of what links
the human to the world has never been so obvious since the announcement of the promise of virtualization and
modelling of the human being, from his intimate relations to the notion of habitability. Virtual worlds (beyond
the definition of Metaverse) promise us a world in which the human experience could be reconfigured in a bespoke
manner without the canal of affective and invisible human dimension, of the anchoring of the symbolic through
the affective path, of the word that introduces us to subjectivity and otherness, a word that confronts us in
its strangeness and its inexhaustible incompleteness.
The standardization of the perceptible, the logical and scientific determinism could reach its apex in these
new universes that redesign the conception of the human. It is therefore in the interest of artists and
designers to present the eminently open conception of a world thought out by art and designers so as not to
close the vision of the human to an automaton-avatar with which it would be a question of playing and moving in
the way of a puppet. A human representation that disregards the gravity of bodies, the rites of social
interaction, the collective to be woven, the immersion in a living half-place where the segregation of the
visible is decided upstream and selecting what can be seen from what does not deserve to be seen. The human
being is not a pack of information to be grasped, he is a being of dreams and words and it is better to know
this than to ignore it. It is in the interest of art and design, philosophy and anthropology to integrate these
decision-making circles of future representations that will be conveyed with an unequalled power of diffusion
and democratization. There is a stake in everyone being able to grapple with the question of the relationship to
the environment that we can rethink, the adventure of discovery, of the process of making, of surprise and of
creation, which is never elaborated in predetermined data, but which emerges from the unpredictable and the
unknown.
How can we open up these industrial worlds to the civilizational challenges they face today? How can we make
people understand what design can do at the intersection of art, engineering and industry? Design is the
keystone of the human intentions of any industrial project deploying the openness of the human senses, of human
abilities and expectations into a project, in a tangible and accessible manner improving the human benefice
“in fine”... Design is drawing and aim at the same time, a potential direction as a line of conduct
and orientation to be incorporated into any industrial and societal project... How can we reintegrate meaning
upstream of this productive force so that it serves a majority in humility and equity? What are the first tracks
and trials of these co-operations and new ecosystems where art-science-industry work together again? What are
the first attempts made by industry, which must rethink itself at a time of climatic emergency and where the
production of waste objects, of intensive innovation based on the imagination of mass production, of Taylorised
work and progress must be reinvented? How can we reinstate the sensitive in these industries that produce
techno-imaginaries? How can we reconstitute noetic spaces ranging from industrial environments, experiential
hybrid environments, to virtual immersive universes (metaverse)?
Can we imagine and draft new roadmaps to rehabilitate Art without its instrumentalization in the scientific and
industrial spaces where today most of our narratives and actions that construct representations are being
produced? Art is probably the future of industry understood as capacity for representation for its power to
create new bodily and cognitive experiences, and its possibility of creating dreams and new solidarities. This
implies putting art back into life, as medicine of the imagination, as the art of being together. Should it, can
it?
Such regenerative new hybridization and such new forms of cooperation would represent a new symbiosis of the
creative forces of the human being, in which art would, in that case, become design, design as a mindset and
“way to think.” Science and industry could collaborate for human and non-human benefit driven
project. A project which path remains open to matter, to the singular, to the body in “totality,” to
the living, to the unpredictable and immeasurable, which is the nodal point of invention.
This utopian symbiosis we are describing is what we feel is right to focus on, in order to build the narrative
and the bases of a dialogue to be reinvented so that all human faculties are no longer dissociated, but reunited
to serve an imminent civilizational project. Considering all the living in all their diversity and unknowability
as the preciousness of life should be taking into account in the course of our actions. We need a holistic and
systemic approach within the limits of the consequences of the harmful and positive impacts that we can and must
imagine. Industry must reclaim its original role of building a world¹³ to be improved, to be inhabited
for all and in the perspective of diversity. It is a call for a collaborative symbiosis that takes into account
the complexity and the need to work together, despite the stereotypical oppositions that prevent these human
universes from interacting again.
Beyond the civilizational stakes, industry is today a playground for art and science, and dialectic between art
and science is vibrant. It questions their relationship, the tools they manipulate, but also because industry
must regenerate its raison d'être (get out of consumerism, functionalism). It must become
organic. It is when we have a calmed art-science dialectic that industry will be able to resolve these new
issues.
Another question comes to mind: does industry contribute to the discourse of technoscience, or is it the victim
or the keystone? With cultural and economic globalization, its responsibility is taking the main stage... in its
making of the world and tomorrow's world, the industry cannot be reduced to a means of progress nor an
externality functionally assess but as a subject for thought, an object for design... and Art. Shouldn't the
hyper-strategic position of industry be aestheticized as an experience with methods of art and science, i.e.,
art applied to industry which is called design? Industry has reached a level of maturity that is no longer under
the yoke of politics, it is political. As such, industry must reconvene art and science within itself according
to its own responsibilities. An industry that thinks, reflexive to change the world will not arise without the
aesthetic and ethical sense. An industry more in tune with a regenerative mindset will enrich a new form of
relationship with Nature, Human and the world, an artistic approach from within, to nurture a sustainable future
and solutions that undoubtedly will be found.
References
1 Gilbert Simondon, Imagination et Invention, 1965-1966, Paris, Presse
Universitaire de France, 2014.
2 Henri Bergson, L’évolution Créatrice, Paris, Presse
Universitaire de France, 2013.
3 Günter Anders, Et si je suis désespéré que voulez-vous
que j’y fasse, Paris, Allia, 2010, p.72.
4 Charles Sanders Peirce, Écrit sur le signe, Paris, Seuil, 1978, p.23.
6 Pierre Legendre, Les enfants du texte, Paris, Fayard, 1992.
7 Carole Fritz, L’art de la préhistoire, Paris, Citadelles et Mazenod,
2017.
8 Ernst Cassirer, La philosophie des formes symboliques, le langage, Paris, Edition
de Minuit, 2017.
9 Pierre Legendre, Le visage et la main, Paris, Les belles lettres, 2019.
10 Jakob Von Uexkull, Milieu animal et milieu humain, Paris, Broché, 2010.
11 André Leroi-Gourhan, Le geste et la parole, Technique et langage, Paris, Albin
Michel, 1964, p.49.
12 Alain Supiot, La gouvernance par les nombres, Paris, Fayard, 2015.
13 Pierre Musso, La Religion Industrielle, Paris, Fayard, 2017, p.12. Journal article
(online)
5 Stevan Harnard "Le problème de l'ancrage des symboles" Physical Review D, 42, accessed
November 7, 2022, https://archipel.uqam.ca/14111/
Bibliography
Anders Günter, Et si je suis désespéré que voulez-vous que j’y fasse,
Paris, Allia, 2010, p.72. Bergson Henri, L’évolution Créatrice, Paris,
Presse Universitaire de France, 2013.
Cassirer Ernst, La philosophie des formes symboliques, le langage, Paris, Edition de Minuit,
2017. Fritz Carole, L’art de la préhistoire, Paris, Citadelles et Mazenod, 2017.
Harnard Stevan, “Le problème de l'ancrage des symboles", Physical Review D, accessed November
7, 2022, p.42, https://archipel.uqam.ca/14111/
Legendre Pierre, Le visage et la main, Paris, Les belles lettres, 2019.
Legendre Pierre, Les enfants du texte, Paris, Fayard, 1992.
Leroi-Gourhan André, Le geste et la parole, Technique et langage, Paris, Albin Michel,
1964, p.49.
Musso Pierre, La Religion Industrielle, Paris Fayard, 2017, p.12.
Peirce Charles Sanders, Écrit sur le signe, Paris, Seuil, 1978, p.23.
Simondon Gilbert, Imagination et Invention, 1965-1966, Paris, Presse Universitaire de France,
2014. Supiot Alain, La gouvernance par les nombres, Paris, Fayard, 2015.
Von Uexkull Jakob, Milieu animal et milieu humain, Paris, Broché, 2010.
Authors’ Biographies
Edwige Armand is an artist and a teacher-researcher in art and culture at the Purpan engineering school (INP
Purpan) and is attached to the LARA-Seppia laboratory. Passionate about the history of science and art, and
philosophy, she combines these different disciplinary fields around the question of techniques and sciences.
The modifications of the relationship to reality, subjectivity and the body by techniques and sciences are
central axes in her research-creation. The transversality of disciplines allows him to question more broadly
the processes of creation and the transformations of representations that the arts contribute to. To revive
the dynamics of art-science relations, she co-founded and has chaired since 2016 the association Passerelle
Art-Science-Technologie, which works to bring these disciplines closer together, and she is also involved in
the Transversale des Réseaux Arts Sciences.
Anne Asensio, after executive and design management positions in the automotive industry, joined Dassault
Systèmes in 2008 as Vice President Design Experience. She created the Design function of Dassault
Systèmes and the Design Studio, bringing together a multidisciplinary team in innovation strategy
through design, experience design and design research. Advocating a participatory approach to new technologies
and virtual worlds, the Design Studio supports Dassault Systèmes' customers in high-growth industries in
their transformation, digital and sustainable innovation needs towards virtuous design processes. Imagining
alternative scenarios to transform the world we live in into a more sustainable and desirable one, the studio
engages in reflection, experimentation and confrontation of cross-cutting approaches between social and
technological issues through creation.
Don Foresta is an art theorist using new technologies as creative tools. Specialised in art and science, he
has been a professor at ENSAD and the Ecole Nationale Supérieure d'Arts - Paris/Cergy and a research
fellow at the London School of Economics. He has spent 35 years transforming the network as an artistic tool
and is working on the creation of a permanent high-speed network, MARCEL, dedicated to artistic, educational
and cultural experimentation. In 1981, he made his first online exchange between the Center for Advanced
Visual Studies at MIT where he was a fellow and the American Center in Paris where he was director of the
Media Art program. In 1986, as curator of the 42nd Venise Biennale, he made the first computer network used by
artists. He was named Chevalier de l'Ordre des Arts et des Lettres by the Ministry of Culture.
Symbio(geo)sis: When mineral and biological species meet
MonikaBakke
Adam Mickiewicz University Poznań, Poland monika.bakke@amu.edu.pl
Abstract
As biocentrism is increasingly seen as unjustifiable, artists and environmental humanities scholars become more
attentive to feedback between minerals and (non)human life. First, I discuss the urgent need to reinvent
narratives and create new vocabulary about the abundance of life-mineral connections as presented by Mabe
Bethônico in her Stone Statements Editions, which focused on developing "a vocabulary of
proximity." Next, I analyze Magdalena Abakanowicz’s artwork Space of Unknown Growth,
investigating emerging material relations between concrete (an anthropogenic rock) and nonhuman life. I
argue that rather than limiting the environment to a symbiotic community, these artworks encourage the
curiosity and attention necessary to embrace mineral-life connections and envision more inclusive communities of
the future.
Keywords
Art and science, geology, minerals, technofossil, concrete, Long Ecology, biocentrism
Biocentrism is increasingly viewed as unjustifiable. Minerals are everywhere, but in Western thought, nonlife
has always been significantly overshadowed by robust, organic life. The multispecies studies have so far
embraced the diversity of mineral species only in a minimal capacity. Although biocentrism still predominates,
change seems inevitable. The currently emerging trend in the humanities and arts to truly embrace mineral
species (considered nonlife or geos) is growing in significance, and it is attentive not only
to environmental concerns but also to new developments in mineralogy. The latter focuses on the co-evolution of
minerals and life and the unprecedented increase in the diversity of human-made and human-mediated mineral
species and novel materials characteristic of the Anthropocene.
Vocabulary urgencies
To better recognize the complex relations within geo-biotic communities, it is necessary to reinvent
attitudes and narratives about mineral species. This need was addressed by Brazilian artist Mabe
Bethônico, who called for the development of a «vocabulary of proximity» of minerals and
life.
Bethônico created StoneStatements Editions in response to the question: How Will We Live
Together? posited by the curators of the Venice Biennal of Architecture in 2021.
Bethônico’s editorial proposal includes five inspiring book tiles: Geoimaginaries; Conscious
Rocks; Human Invasion in the World of Stones; When Stones Collect Diggers, Robbers, Queens and Kings;
Missing Words for Considering Stones, Rocks, Pebbles and Mountains: A Vocabulary of Proximity. Only the
latter has been developed so far as a collective effort, resulting in a volume of twenty-six short entries. As
Bethônico stated, the remaining books are «imagined, wished for, and suggested to be
developed» as more vocabulary to think about the environment in more than biological terms is urgently
needed.¹ Her project recognizes the
biocentric bias and the conditions of its operation, namely the shortage of concepts, stories, and creative
practices to embrace mineral species.
Anthropogenic rocks
The proximity of life and minerals, especially rocks characteristic of the Anthropocene epoch, is investigated
and anticipated in the artwork Space of Unknown Growth by Polish artist Magdalena Abakanowicz. Located
in the forest of Europos Parkas, this artwork comprises twenty-two concrete boulders in four sizes, spread over
an open-air museum near Vilnius. Concrete, although anthropogenic, is considered a "geological entity" due to
its close relation to the mineral composition of the earth. It is a novel rock and, as geologist Jan Zalasiewicz
suggests, “the most abundant anthropogenic sedimentary rock on the planet.”² Classified as a
technofossil together with a great variety of synthetic compounds, concrete is emerging as the prominent
signature of our geologic epoch. There is, however, a particular risk of succumbing to the lithic allure
recognized by Stacy Alaimo, who calls for caution when suggesting that “attending solely to the lithic
imports delusions of separation and control."³
Space of Unknown Growth speculates about a future that belongs to multispecies communities whose
endurance is anticipated rather than predicted. For surviving, making kin with unexpected others, as Donna
Haraway suggests, is necessary. Staying with the concrete means "staying with the trouble", which demands that
the viewers acknowledge the devastating environmental impact of extracting resources and recomposing
materials.⁴ And yet, Space of Unknown Growth does not convey alarm. Instead, it points out that
adaptation goes together with the transformation of anthropogenic lithic environments, which accommodate more
than human life and is a condition needed to renew planetary species diversity. Living on the ruins is possible
as some organisms can modify the lithic surfaces in their environment. The technofossils that comprise Space
of Unknown Growth indicate the anthropocenic Earth’s habitability. The anticipated growth
speculated on in Abakanowicz’s work is conditioned by the composing and decomposing of biological and
mineral species and the resilience of communities capable of terraforming concrete—the most earthly rock.
Long Ecology
Bringing anthropogenic stone to the forest introduces a perspective of what Jeffrey Jerome Cohen calls Long
Ecology oriented toward the future and vastly exceeding current waste management. However, concrete, as “a
material that mediates transcience and endurance," is not as long-lasting as stone.⁵ It rejects the
conventional belief that geological processes are always long and slow. From Long Ecology's perspective,
although prolific in novel mineral species and formative for technofossils, our epoch might not be capable of
reaching a distant future because anthropogenic minerals might become extinct.⁶
And yet the stones of Space of unknown growth might be considered erratics, conventionally defined as
rocks not matching the local bedrock. Erratics are lithic strangers, newcomers transported to current locations
by glaciers or other natural forces of significant magnitude. Concrete boulders of Space of unknown growth
are synthetic rocks and mineral novelty brought into the environment of a central European forest.
Yet, Abakanowicz’s erratics establish a category of their own as they are art forms, which
nevertheless participate in the currently intense processes of relocating minerals by humans—activity
believed to exceed even glacial action. Abakanowicz’s lithic newcomers, however, with their own material
history, become with their new environments through complex geo-bio relations and processes. These
unexpected others, strangers to their landscapes, arriving from a different time and space, introduce the
possibility of a radically new narrative of connectedness and multispecies kin-making, which stimulates yet
another way for humans to practice attentiveness and care. They demonstrate both vulnerability and resilience,
which characterize earthly worlding and unworlding.
Although the sculptures operate as erratics in the context of Long Ecology, there is no separation between them
and the materiality of their location. Through their lithic ways of being, the works participate in planetary
processes operating, in Cohen’s words, as “...an affectively fraught web of relations that unfolds
within an extensive spatial and temporal range, demanding an ethics of relation and scale.”⁷ In this
sense, Space of unknown growth is deeply ecological in its attentiveness to the vast scales in
which life becomes stone and stone becomes a living environment.
Coda
What we need now is not a belief in the solitude and indifference of stones, rocks, and pebbles but a
celebration of their inextricable connection with life and creative elaborations on their diversity. Both
StoneStatements Editions and Space of Unknown Growth inquire about ways to articulate how
minerals and life actually attract and seduce each other and how minerals open up to life’s intimate
strategies, the outcomes of which cannot be determined. They offer vocabulary and materiality to tell stories of
Symbio(geo)sis. Rather than limiting the environment to sym-biotic community, these works encourage
curiosity and attention needed to embrace mineral-life connections better. Only then will geo-biotic communities
emerge as the answer to the question of how we will live together.
2 C.N. Waters, J. Zalasiewicz, “Concrete: The Most Abundant Novel Rock Type of the
Anthropocene,” in Encyclopedia of the Anthropocene, ed. Dominick A. Dellasala, Michael I.
Goldstein, Oxford, Elsevier, 2018, p.75, https://doi.org/10.1016/B978-0-12-809665-9.09775-5.
3 Stacy Alaimo, Exposed: Environmental Politics and Pleasures in Posthuman Times, U of
Minnesota Press, 2016, p.149.
4 Donna J. Haraway, Staying with the Trouble: Making Kin in the Chthulucene, Duke
University Press, 2016.
5 Richard D. G. Irvine, Anne Bevan, “Concrete Buys Time: Art and Anthropology in the
Anthropocene,” Worldviews: Global Religions, Culture, and Ecology 26, no. 3, October 19, 2022,
p.182, https://doi.org/10.1163/15685357-02603009.
6 Jan Zalasiewicz, Ryszard Kryza, Mark Williams, “The Mineral Signature of the Anthropocene
in Its Deep-Time Context,” Geological Society, London, Special Publications 395, no. 1,
2014, p.112, https://doi.org/10.1144/SP395.2.
7 Jeffrey Jerome Cohen, Stone: An Ecology of the Inhuman, U of Minnesota Press, 2015,
p.41.
Bibliography
Stacy Alaimo, Exposed: Environmental Politics and Pleasures in Posthuman Times, U of
Minnesota Press, 2016.
Jeffrey Jerome Cohen, Stone: An Ecology of the Inhuman, U of Minnesota Press, 2015.
Donna J Haraway, Staying with the Trouble: Making Kin in the Chthulucene, Duke
University Press, 2016.
Richard D. G Irvine, Bevan Anne, “Concrete Buys Time: Art and Anthropology in the
Anthropocene”, Worldviews: Global Religions, Culture, and Ecology 26, no. 3, October 19,
2022, 179–95, https://doi.org/10.1163/15685357-02603009.
C. N. Waters, J. Zalasiewicz, “Concrete: The Most Abundant Novel Rock Type of the
Anthropocene”, In Encyclopedia of the Anthropocene, edited by A. Dominick Dellasala, Michael
I. Goldstein, Oxford: Elsevier, 2018, 75–85, https://doi.org/10.1016/B978-0-12-809665-9.09775-5.
Jan Zalasiewicz, Kryza Ryszard, Williams Mark, “The Mineral Signature of the Anthropocene in Its
Deep-Time Context”, Geological Society, London, Special Publications 395, no. 1, 2014,
109–17, https://doi.org/10.1144/SP395.2.
Author Biography
Monika Bakke is associate professor in the Philosophy Department and director of the Environmental Humanities
Center at the Adam Mickiewicz University, Poznań, Poland. She writes on contemporary art and aesthetics,
particularly interested in posthumanist, transspecies, and gender perspectives. Her curatorial work includes
art exhibitions: Bio-Reminiscences (Poland), Seeing the Forrest Through the Trees (UK),
Boundless Objects (Portugal), and Refugia: Keep (Out of) these Places (Poland). Currently,
her research focuses on nonlife forces and new articulations of mineral becomings in contemporary art.
https://amu.academia.edu/MonikaBakke
4E Cognition for Symbiotic Architecture?
YannBlanchi
BMOffice France yann@bmoffice.fr
Abstract
With the aim of reconsidering the very nature of architecture, we propose a conceptual tool to think of
architectural apparatuses as actors within a continuum composed of both artificial and natural agents. For
this,we look at the cognitive sciences, particularly the 4E cognitive (embodied, embedded, enactive, and
extended). We use examples from contemporary architecture to test our hypothesis and thus attempt to define what
symbiotic architecture could be.
Keywords
4E cognition, Embodied, Embedded, Enaction, Extended, Symbiotic, Architecture, Artificial, Natural
In this paper, we will explore a theoretical tool to think symbiotic architecture. To this end, we consider
whether the transposition of the 4E cognition may be useful in describing the characteristics of what the
symbiotic architecture might be. The paper is organized as follows: Firstly, we set the context and define the
terms used, such as symbiosis and 4E cognition. Then we state our hypothesis of an analogy between 4E cognition
and architecture. Next, we test this idea with examples of recent architectural apparatus, and we conclude.
Context
In 1960, as Gordon Pask stated in his theory of conversation, we moved from a functionalist world to a
mutualist world. Today, in the context of the global crisis, nature is coming back into the loop. The ecocentric
logic would reinscribe us in a network of multiple interactions and concrete connections with the milieu,
engaged in a network of dependencies in a way of decentralized connections.1 Nature and artifice
are bound to co-evolve. Under these conditions architecture can certainly no longer be the static and fixed
object of the moderns but a new type of artefact. Architecture may have entered the age of
naturalization.2
Definitions
Etymologically, the term symbiosis comes from the Greek and means "living with." In biology, symbiosis refers
to associations between hetero specific living organisms. There are three forms of symbiotic
association:parasitism, commensalism and mutualism. It depends on whether the relationship is unidirectional or
bidirectional and beneficial, neutral or harmful. Semiologists study these interactions. On the other hand, in
the field of cognitive sciences, the 4E cognitive approach proposes a conception of cognition as an assembly of
the brain, the body and the environment. The 4E approach is based on four concepts: embodied, embedded,
enactive, and extended.
Embodied cognition is cognition produced in part by structures other than the neural system. Embodied cognition
rejects Cartesian dualism, separating the body from the mind, and includes the body in the cognitive
system.3 The body is an integral part of the cognitive system, for example, we perceive relief
thanks to the stereoscopic vision of our two eyes.
Embedded cognition is cognition coupled with the environment. Thought is not, as it were, secreted by the brain
but by the environment.⁴ The extension of the body into its environment reduces loading and relieves the
brain.⁵Enactive cognition is cognition produced in part by actions.⁶ We note here the importance of
movement in the process. Extended cognition, a cognition situated in the environment, is an externalisation of
processes in our environment. In the context of digital technologies, cyberspace would become an extension of
our brain.⁷
Hypothesis
Our hypothesis is the following: if architectural apparatus supported 4E cognitive activities, then
architecture would become symbiotic. For this purpose, inhabitant/architecture/milieu has to constitute a form
of dynamic coupling, connecting natural and artificial actors. To test our hypothesis, we present below an
architectural project example for each of the four E categories.
Embodied architecture
The Urban Algae Folly, designed by EcoLogicStudio for the EXPO Milano 2015. (1) The microalgae cultures
embedded in the architecture allow to control transparency and shading qualities of the membrane are the result
of algae growth depending on the sunlight and the presence of visitors.
In this example, not only do the algae have the dual role of sensor-actuator, but they are distributed over the
entire surface of the device. All parts of the building body are sensitive and responsive, intelligence is
distributed, embodied in every square centimeter of the material.
Embedded architecture
HygroSkin, Meteorosensitive Pavilion by Achim Menges' team is part of the permanent collection of the Frac
Centre since 2012. (2) This meteo-sensitive architecture is the result of research into materials that react to
variations in their environment. The device is inspired by the pine cone principle. The multi-layered wood
reacts to humidity, the holes open and close depending on the degree of humidity in the air. The heterogeneity
of the material allows the deformation of the surface, as each layer does not have the same expansion
coefficient. In this example, the architecture is coupled to the environment in a unidirectional link that gives
it shape.
Enactive architecture
In 2003 Kas Oosterhuis/ONL agency was invited to create an installation for the exhibition Non-Standard
Architecture. (3) The NSA Muscle is a pneumatic interactive device composed of 72 muscles programmed to be 72
independent members of a single swarm. The 72 inflatable muscles are controlled by individual valves. The device
is programmed to have its own behavioural cycle, but also to react instantaneously to an external stimulus. In
this way, ONL materialises the concept of two-way communication in real time linking two active elements, in
this case, the inflatable device and the users. The users interacting with the Muscle quickly learn how the
Muscle reacts to their actions, and a game is established in this communication.⁸ We are in a case of
bidirectional relationship, in which the movements of each of the protagonists (human and non-human) are part of
the same process.
Extended architecture
Hylozoic Ground by architect and sculptor Philip Beesley was selected to represent Canada at the 2010 Venice
Biennale in Architecture. (4) Beesley's installations are immersive environments that question the boundaries
between the natural and the artificial, the human and the non-human. These hybrid devices are micro
creatures,half environments, half mechanical, half biological. The visitor, by his presence, movement and
breathing,activates a wave of reactions from the device. He is thus inscribed in a respiratory cycle and through
his interactions becomes one with the installation within the same metabolism.
The environment supports a mutual empathic relationship that initiates a reactive movement, an exchange of
particles and an air cycle between the system and the visitor. The visitor's own body limits are questioned.
Philip Beesley named this type of behaviour: diffusive architecture.
Conclusion
The opposition between the living and the artefact is replaced by an isochronism of natural and artificial
phenomena that continues to hybridize. Symbiotic architecture could be seen as a cyber-physical-human system
within the natural artificial continuum. The opposition between the living and the artefact is replaced by an
isochronism of natural and artificial phenomena that continues to hybridize. With these four examples, we have
seen that architectural devices can support relational processes at different scales (within the material itself
or with external elements), in a unidirectional or bidirectional way, beneficial to a single actor or to
several. What the 4E approach shows is that architectural symbiosis needs to be based on a multiplicity of
interactions and actions. Borrowing the concept of the 4 E's (embodied, embedded, enactive, and extended) from
the cogniticians, we wanted to sketch a reading and writing grid for architectural design, our next research
object. It would allow gathering under one concept the different current appellations such as dynamic,
flexible,sensitive, transformable, adaptable, interactive, intelligent and moreover.
3 A. R. Damasio, L’erreur de Descartes : La raison des émotions, M. Blanc
(trad.), Editions Odile Jacob, 2010.
4 A. Prochiantz, Machine-esprit, Editions Odile Jacob, 2000.
5 P. Haselager, J. Van Dijk, I. Van Rooij (s. d.), "A Lazy Brain? Embodied Embedded Cognition and
Cognitive Neuroscience", In Handbook of Cognitive Science, 273-290.
6 E Couchot, La nature de l’art : Ce que les sciences cognitives
nousrévèlent sur le plaisir esthétique, Hermann, 2012.
7 M. Serres, Petites poucettes. Editions le Pommier, 2012.
8 V. Parlac, Surface Change : Information, Matter And Environment. In R. Stouffs &
P.Janssen... (éd.), Proceedings of the 18th International Conference on Computer-Aided
Architectural Design Research in Asia, CAADRIA, 2013.
A [Potential] Cloud War Controversies and conflicts related to climate manipulations.
Marie-JulieBourgeois
Paris-Saclay University, France mail@mariejuliebourgeois.fr
Abstract
The OuCliPo research project (Ouvroir de Climats Potentiels) is a workshop of climate potential,
supported by the Labex LaSIPS, presented by Marie-Julie Bourgeois, PhD in Aesthetics, Science and Technology of
the Arts, researcher in art and design science at University Paris-Saclay. This project is also directed by
Jules-Rémi Bois-Rouge, a (fake) climate geo-engineer, Doctor at the LaPataFlu, a (fake)
Laboratory of Fluid Pataphysics, stemming from the modern surrealist literary movements, proposes a "science of
imaginary solutions which symbolically grants to lineaments the properties of objects described by their
virtuality."1 OuCliPo studies surrealist issues to climate problems; pseudo-scientific
solutions and their implementation in the context of eco-anxiety. The project highlights the ethical and
geopolitical dimensions of solar geo-engineering, as well as the socio-cultural issues associated with these
climate experiments as techno-solutionism.
The 1995 Nobel Prize in Chemistry, Paul Crutzen, warns about the climate situation and the need to find viable
solutions in a seminar2; to the European Parliament, he proposes to seriously study solar
geoengineering as a third way.3
Solar geoengineering is a highly controversial technology that offers a concrete
and—almost—immediate solution to the current problem of global warming. Although theoretical, this
technology is being actively studied in research laboratories and by IPCC experts "Aerosol injection into the
stratosphere is the most convincing solar geoengineering technique, it has the potential to significantly affect
the climate in relation to the ongoing warming. It could offset a rise in global temperatures. It may offer a
faster cooling capacity than CO2 mitigation."⁴ This major technological innovation allows
an efficient cooling of the entire globe by increasing its natural reflective power (albedo).
This theory is based on the study of climate cooling periods following major volcanic eruptions such as the
Pinatubo eruption, which cooled the globe by 0,5°C. In 1991, the release of sulfur particles (SO2) into the
stratosphere increased the reflectivity of the Earth's surface and thus prevented some of the sun's rays from
entering the atmosphere. The temperature of the planet dropped by -0.6°C for fifteen months.⁵,
⁶
Players in geo-engineering
Solar geo-engineering solutions have developed rapidly over the last few years, with the aim of curbing global
warming as a matter of urgency. Laboratories—or startups—specialized in solar geo-engineering, Solar
Radiation Modification (SRM) or Stratospheric Aerosol Injection (SAI) attempt to "fix the climate" by generating
anthropogenic clouds composed of sulfur particles.
These promise to create a protective cloud cover against solar radiation to increase albedo. SAI was patented
in 1991, as a method of Stratospheric Seeding, it involves seeding the stratosphere with small metal oxide
particles, such as thorium, barium or aluminium flakes, with the aim of reflecting sunlight and reducing global
warming. Plans to implement the proposal were not foreseen or publicly known. In the years that have passed
since, global warming has caused major concern and scientists have been looking for solutions to "repair" the
planet. Researchers mainly use numerical modelling for obvious legal, ethical and health reasons. If inhaled,
sulfur dioxyde (SO2) may lead to chronic disease, embedding itself in our lungs. Some laboratories are
conducting risky experiments on a human scale without legal authorization in a frantic race to experiment with
climate⁷:
In 2009, the first known field experiment was carried out in Russia. The IGCE⁸ released
Sulphate aerosols from the ground and from a helicopter to measure solar radiation in the visible wavelength
range.
Between 2010 and 2014, the SPICE⁹ project aimed to assess the feasibility of injecting
particles into the stratosphere from a tethered balloon. The field experiment was canceled due to societal
opposition in the UK.
In 2011, the E-PEACE10 project was conducted off Monterey, California, where salty aerosols
were emitted from an aircraft, along with smoke and exhaust from a research vessel and a container ship.
In 2014, the SCoPEx11 laboratory began small-scale experiments.12 In 2021, this
American laboratory attempted to launch a balloon filled with SO₂ in Sweden; however, the experiment was
shut down after widespread opposition from the Saami Council, via an open letter to the Swedish government and
space corporation13.
Since 2019, the US government project SABRE14 has been studying the impact of introducing
materials into the stratosphere to influence climate and solar radiation. Since 2021, Andrew Lockley has
conducted open-air experiments. British researchers from the SA T AN project launched a high-altitude weather
balloon filled with lifting gas and a hundred grams of SO2 from European Astrotech Ltd in Aylesbury, in
England. Lockley said, "I can only confirm that our craft ascended to the heavens, as intended. I only hope
that this test plays a small part in offering mankind salvation from the hellish inferno of climate
change."1⁵
Founded in 2022, the startup Make Sunsets has gone further, launching a premature high-altitude
implementation, sending balloons with SO2 from Mexico in late 2022 without authorization or observation tools.
The Mexican government reacted by banning the company and solar geoengineering experiments.1⁶
The startup is offering to sell $10 "cooling credits" online1⁷ for releasing one gram
of particles into the stratosphere, to "offset" the warming effect of one ton of carbon, for one year. Even if
this operation is described by its creator like a (bad) joke "We joke slash not joke that this is partly a
company and partly a cult"1⁸ as Isemans says, like a radical rogue guru dressed in an orange
down jacket.
All these projects should be illegal if we respect the moratorium on geoengineering, the UN ministerial meeting
held in Nagoya, Japan, in October 2010 "Any experimentation, private or public, or adventurism aimed at
manipulating the planetary thermostat will constitute a violation of this carefully crafted consensus in the
context of the United Nations," asserted Silvia Ribeiro, ETC Group Director for Latin America.
Seducing words such as (Make Sunsets), the acronyms (SPICE, PEACE, SABRE, SATAN) and the vocabulary
employed by researchers (heaven, hell, inferno, cult...) refer to beliefs and myths. The desire to remedy global
warming by making amends for past pollution is strong. Guilt and the hope of becoming a hero by saving the
planet and its inhabitants from imminent danger guide these geo-engineers / climate sorcerers.
Costs
The cost of deploying these technologies is not excessive. Independent organisations can leverage them without
requiring participation or funding from institutions. According to studies19, 20 it is
sufficient to inject $1500 per ton of particles into the upper atmosphere each year for a gain of 2w/m2, in
order to benefit from a significant decrease of 2°C, and the entire solution would cost between 2.25 and 11
billion dollars per year. The cost seems reasonable in view of the climate issues and international interests.
Risks
The main obstacles are the ethical, causal, and governance issues21 associated with climate
manipulation, as well as the difficulty of large-scale experimentation and impact studies. Expert scientists are
unanimous on the hopes and the risks,22 the danger to the biosphere,23 security
framework,24 news biophysics risks,25 side effects, as well as possible thermal
recovery in the event of an abrupt shutdown.26 Beyond health impacts, the societal risks of deploying
these technologies are a moral issue,27 which provides a miracle "solution" that avoids reducing GHG
emissions, socio-technical lock-in; there are also considerations regarding the irreversibility of the
techniques deployed, and the termination shock.28 Furthermore, this technology, if developed
without international agreements, will cause large-scale damage across borders (GIEC 20122). As the
climate is interdependent on local meteorological events, the effects will be global. It is therefore urgent
that nations position and align themselves in this geoengineering race based on common goals, in order to avoid
wild tests, researching risk and preventing side effects. If experimentation goes unchecked, these solutions
could spark a future global war,29 a scenario that is certainly no longer a fiction given the current
geopolitical context. Since global warming is considered a threat to national security, the US military is
involved in the development of solar geoengineering as part of its militarisation
strategy.30 The prospect of rebalancing the forces belonging to competing powers such as China
or Russia is probable, and the risk of escalation is a consequence of solar counter-geo-engineering
programs31... A potential cloud war32 is brewing.
Look the sky Major volcanic eruptions
The year 536 has been called "the worst year in history"33 by archaeologists. It marks the Little
Ice Age caused by several simultaneous volcanic eruptions. These volcanic eruptions caused 18 months of darkness
on the surface of the globe due to the release of acidic sulfur particles, world temperatures then dropped by
-2.5°C, causing famines, two centuries of plague epidemics, and the fall of several civilizations on all
continents.3⁴ This period also marks the beginning of the Dark Ages within the Middle Ages. The
architect Philippe Rahm3⁵ recalls the links between architecture, climate, volcanic eruption
and epidemics: The eruption of Tambora in 1815 changed the world3⁶ which
benefited cholera. The global temperature dropped by 1°C for several years, prolonging the winter and
disrupting the climate cycle on a global scale, the “year without summer” will be
remembered.
Are we really ready to play with fire by experimenting with particle diffusion in the stratosphere? The health,
meteorological3⁷, agricultural3⁸ and civilizational risks are not well enough
measured or understood. Dangers are not limited to the survival of ecosystems and populations, not all risks
have been studied and identified by geo-engineers.
Science and fiction
That same dark and rainy summer of 1816, in this troublesome climatic context, a group of writers met in
Switzerland; Mary Shelley wrote Frankenstein or the Modern Prometheus3⁹ and Lord
Byron popularised The Vampyre.⁴⁰ We can say that one of the founding myths of
Science Fiction was born in a dramatic climatic context! The following year, in 1817, in the midst of global
climate change, William Turner began to paint coloured sunsets, amongst other Impressionist precursors
influenced by these surrealist red-to-green skies.⁴1 Edvard Munch painted The Scream
after viewing and being influenced by the strange sky "turning blood red."This famous
expressionist portrait and its red sky probably stems from a vision of the Oslo Fjord, after the volcano
eruption of Krakatoa in 1883,⁴2 or may be due to a nacreous cloud.⁴3 Climate
and fiction seem intimately linked in the narrative imagination but also in the psychology of aesthetics and
creative expression. If SRM/SAI technologies are implemented, and the atmosphere becomes opaque and dangerous to
inhale due to particles, will the atmosphere lose its transparency? Our “common sky” will be
whiter.⁴⁴ Today’s blue skies are already an illusion due to Tyndall effect and
Rayleigh scattering and it is based on the size and density of particles... If the atmosphere becomes
concentrated in CO2, will the sky turn orange like the CO2 saturated Martian sky? If our vision becomes
saturated or obstructed by particles, how can we project ourselves towards new horizons, new common futures?
We can conclude that the power of design influences the critical thinking process, whilst the urgency of the
situation also blinds the mind. The Make Sunsets website is far too well-designed for a typical
scientific research project, and its warm orange design (the colour of sunsets) is as seductive as the promises
of rescue it makes. Perhaps the design and aspirations explain its success and ability to raise funds. Despite
the cobbled-together nature of its technology and the danger of its reckless actions. However, these orange-red
colours of the startup should worry us, just as the Scream 😱 reveals the anxiety-inducing
dimension of modern existence.
Realistic satyr and fiction
OuCliPo is an art and design research project that questions the impact of particles on our vision and
our cultures at the scale of landscape, architecture and habitat, by presenting speculative fiction based on
climate manipulation, studies and data.
Nubus⁴⁵ is a (fake) research startup, created to test the credibility of the
technology and also the attractiveness to the general public via a seducing marketing campaign "for a bluer and
a cooler sky." For this purpose, a questionnaire was used to measure support for the Nubus startup and
its promises. The French (fake) startup projects us to the stage of commercialization and industrialization of
SRM/SAI technologies in order to confront us with the mutations they will generate via environmental and society
disturbances. How far are we willing to go, to believe it?
After viewing the Nubus promotional film, a sample of 34 people were interviewed during the exhibition
of the project at the ENS Paris-Saclay and at Evry University:
62% want to help optimize the climate
41% are ready to install a free of charge Nubus module at home
41% are in favour of a cooler climate
29% want a “bluer sky”
In conclusion, a relatively high percentage was inclined to participate in climate optimization in order to
promote cooler and bluer skies. Although a minority, 21% of the respondents, believes that geo-engineering and
SRM/SAI do not seem dangerous for our biosphere, or the climate... When the video was recently shown in an
innovative art and science context, audiences reacted with ambivalence: amused, horrified or seduced, many
believed in it. We decided to exaggerate the tragi-comic potential of the startup’s arguments to defuse
support for the promise of rescue, the urgency of the situation resulted in a strong desire for
solutionism.
Nubusis also developing green kerosene and biofuels to generate
clouds of particles with high nucleation potential, rather than harmful greenhouse gases. Our emissions are
guaranteed to have high clouding potential. We also work with the agrochemical and agricultural industries to
combat soil erosion and desiccation. Our mineral and nutrient stimulants have a high precipitation potential
which is ideal for farmers in times of drought.
"Our partners in the tobacco industry offer electronic cigarettes that contribute on a small scale to the
generation of climate-friendly particle clouds, because there are no small contributions.
Nubus is launching a fund-raising campaign to "save" the planet and offers citizens in search of
meaning the chance to clear their consciences by blue washing our vision of the future.
You can also take a sodium cure with our blue pills, which guarantee accelerated awareness to wipe out
ecological procrastination..." (3)
Even with these arguments, the public still wondered if Nubus really did exist...
Homogenitus are clouds generated by human activities defined by the WMO in the international cloud
atlas.⁴⁶Homogenitus⁴⁷ it is also the name of an
artistic cloud-making machine. Players can generate customizable clouds directly on a web application, choosing
morphology, size, shape and the composition of particles. This artistic installation questions the creation of
clouds by human activity, the ability to control the weather and places the user in a demiurgic position.
The citizen artivist collective fakeCloud⁴⁸ denounced Nubus
activities, it’s SRM/SAI projects and climate manipulations. Nubus is caused of
over-producing clouds related to the Iran accusation against Europe to steal their cloud in
2011.⁴⁹
Research of solutions
In the absence of government control or international rules, companies can deliberately spread or steal clouds.
How will our sky look in a decade? Will a future cloud war occur? This awareness campaign proposed to
approach the ecological problems by addressing these issues through wordplay, irony, detours, fiction, and
references to scientific methodologies and technologies. Designing our potential future atmosphere opacity and
tint aimed to sensitize the public to these rapid ecological solutions, and promote awareness of the seductive
dimension of design.
Our ideologies, our beliefs, our knowledge are challenged by the crises we are experiencing; climate, health,
life, economy, institutions and especially science. At the same time when technologies and social networks allow
collaborative actions, we question the methods of sensitization of the public about the techno-solutionism
challenges. Fictions and post-truths mingle with reality in the scientific field. How can we find a sustainable
way in this mess? Awareness and collective action are the determining factors for reducing greenhouse gases, but
solar geo-engineering projects risk slowing down these efforts by proposing alternative and risky paths.
Climate issues are frequently addressed in a dramatic way, generating various typologies of
reactions⁵⁰: alarmed, worried, cautious, non-committal, unconvinced, and
contradictory. On an individual level, ecological procrastination⁵ is mainly due to
discourses⁵2 that delay massive climate action; techno-solutionism is one of them. Robert
Gifford observes seven psychological barriers to ecological inaction⁵3: limitations of
cognition, ideological views, comparison with significant others, costs, distrust of experts and authority,
perception of risk, and behavioural limitations. For Gifford, these"dragons of inaction"are the social-psychological brakes that impede real and meaningful global action.
Conclusion
OuCliPo proposes to approach climate threats with non-guilty and non-anxious modes of communication
such as fiction and humour. OuCliPo does this by making a mockery of the search for technological
solutions. Between solutions that are “not solutions,” where the cure is worse than the disease,
denial and even counter-action, massive actions, whether positive or negative, have a strong impact on the
climate.
To this end, Nubus offers its customers the chance to redeem their consciences by fixing the planet.
It provides citizens in search of meaning with a conscience cleanser, by washing our vision of the future in
blue for a dual benefit: a thermal solution and the psychological comfort of being able to remedy climate change
by looking at the sky, not to don’t look up! (4). Ideologies, design and the power
of capitalism helps us to believe in these seductive solutions. The market in climate
denial⁵⁴ and the negation of mitigation policy are solutions that benefit climate
skepticism.
Media and digital tools are good ways of approaching social issues with critical distance and speculative
projection, but they can also fuel disbeliefs and conspiracies. Moreover, fiction can play a major role in
raising collective awareness of issues at stake, by accelerating political decision-making. Confronted with the
crises we are experiencing, and the eco-anxieties that are paralysing actions, we need to set off in search of
"eco-quietude" and harmony amongst the disorder. This quasi-spiritual philosophical notion organised the
celestial spheres as early as Pythagoras; between music, numbers, colour and astronomy, harmony was already
giving rhythm to the mental and physical health of mankind in antiquity.⁵⁵
The harmony and symbiosis between man and nature, with sustainability and critical distance, would allow us,
through fiction and humour, to approach climate problems from a truthful background.
(3) ISEA 28th International Symposium on Electronic Art, May 17, 2023, Paris forum des halles
(4)Don’t look up ! Adam McKay 2021
References
Books
1 Alfred Jarry, Gestes et opinions du docteur Faustroll, pataphysicien, Livre II,
chapitre VIII, Paris, Éditions Fasquelle, 1911.
35 Philippe Rahm, Histoire naturelle de l’architecture, Comment le climat, les
épidémies et l'énergie ont façonné la ville et les bâtiments,
Paris, Edition du Pavillon de l’Arsenal, 2020, p.167.
36 Gillen D'Arcy Wood, Tambora – The Eruption That Changed the World,
Princeton University Press, 2015.
39 Mary Shelley, Frankenstein or the Modern Prometheus, Lackington, Hughes, Harding,
Mavor & Jones, 1818.
40 John William Polidori, The Vampyre a Tale, The New Monthly Magazine, 1819.
55 Yuval Noah Harari, Homo Deus, A brief history of tomorrow, London, Vintage, 2017.
Journal article (online)
3 P. J. Crutzen, “Albedo enhancement by stratospheric sulfur
injections,” Climatic Change, vol. 77, no 3-4, July 25, 2006, 211-219,
doi:10.1007/s10584-006-9101-y
5 Mc Cormick and al, “Atmospheric effects of the Mt Pinatubo eruption,” Nature 373,
February 2, 1995, 399–404. doi:10.1038/373399a0
12 J. A. Dykema and al. “Stratospheric controlled perturbation experiment: a small-scale
experiment to improve understanding of the risks of solar geoengineering,” Phil. Trans. R. Soc,
December 28, 2014, doi:10.1098/rsta.2014.0059
19 W. Smith, G. Wagner, “Stratospheric aerosol injection tactics and costs in the first 15
years of deployment,” Environmental Research Letters, Vol. 13, No. 12, November 23, 2018,
doi:10.1088/1748-9326/aae98d
20 W. Smith and al, “A subpolar-focused stratospheric aerosol injection deployment
scenario,” Environmental Research Communications, Vol 4 number 9, September 15, 2022,
doi:10.1088/2515-7620/ac8cd3
22 M. De Guglielmo Weber, “Promesses et risques de la géoingénierie
solaire,” IRIS, Programme climat, énergie & sécurité, February 2023, accessed
August 22, 2023, https://www.iris-france.org/wp-content/uploads/2023/02/19_ProgClimEnerSec.pdf
23 M.G. Lawrence, and al, “Evaluating climate geoengineering proposals in the context of the
Paris Agreement temperature goals,” Nature Communications 9, Septembre 13, 2018, 3734,
doi:10.1038/s41467-018-05938-3
24 Chalecki, and al, “A New Security Framework for Geoengineering,” Strategic
Studies Quarterly, vol. 12, no. 2, 2018, JSTOR, Accessed 22 Aug. 2023, 82–106,
http://www.jstor.org/stable/26430817
25 E. Baughman, et al, “Investigation of the Surface and Circulation Impacts of
Cloud-Brightening Geoengineering,” J. Climate, 25, November 1, 2012, 7527–7543,
doi:10.1175/JCLID-11-00282.1
27 C Preston, “Ethics and geoengineering: reviewing the moral issues raised by solar
radiation management and carbon dioxide removal.” WIREs Climate Change. 4, November 8, 2012,
23-37, doi:10.1002/wcc.198
28 A. Parker, & P. J. Irvine, “The Risk of Termination Shock From Solar
Geoengineering,” Earth's Future, 6, March 11, 2018, 456–467, doi:10.1002/2017EF000735
30 Stephens, and al, “Towards dangerous US unilateralism on solar geoengineering,”
Environmental Politics, December 18, 2022, doi:10.1080/09644016.2022.2156182
41 C. Zerefos, end al, “Further evidence of important environmental information content in
red-to-green ratios as depicted in paintings by great masters,” Vol. 4 issu 6, Atmos. Chem. Phys.
14, March 25, 2014, 2987–3015, doi:10.5194/acp-14-2987-2014
42 Donald W. Olson, and al, "The Blood-Red Sky of the Scream," APS News. American
Physical Society, Vol. 13 number 5, May 2004, accessed August 22, 2023,
https://www.aps.org/publications/apsnews/200405/backpage.cfm
43 F. Prata, and al, “The Sky in Edvard Munch’s The Scream,” Bulletin of the
American Meteorological Society 99, 7, July 1, 2018, doi:10.1175/BAMS-D-17-0144.1
44 B. Kravitz, and al, “Geoengineering: Whiter skies?”, Geophys. Res. Lett.,
39, L11801, June 1, 2012, doi:10.1029/2012GL051652
50 E. W. Maibach, and al, “Identifying Like-Minded Audiences for Global Warming Public
Engagement Campaigns: an AudienceSegmentation Analysis and Tool Development,” PLoS ONE, March,
10, 2011, doi: 10.1371/journal.pone.0017571
51 Richard W. Malotte, “I’ll Save the World from Global Warming—Tomorrow: Using
Procrastination Management to Combat Global," June 1, 2017, doi:10.1007/BF03392214
52 WF Lamb, and al, “Discourses of climate delay,” Global Sustainability 3, e17, June
8, 2020, 1–5, doi:10.1017/sus.2020.13
53 Robert Gifford, “The Dragons of Inaction: Psychological Barriers That Limit Climate
Change Mitigation and Adaptation,” American Psychologist, May 2011, doi:10.1037/a0023566
Magazines and Newspapers (online)
2 European Parliament, “Professor Paul Crutzen: Nobel winner and advocate of a climate
"escape route"”, January 12, 2011, accessed August 22, 2023,
https://www.europarl.europa.eu/news/en/headlines/society/20110103STO11194
6 NASA Observatory, Langley Research Center, Aerosol Research Branch, June 2001, accessed August
22, 2023, https://earthobservatory.nasa.gov/images/1510/global-effects-of-mount-pinatubo
13 Letter from Sam Council to SCoPEx advisory committee, February 24, 2021, accessed August 22,
2023,
https://static1.squarespace.com/static/5dfb35a66f00d54ab0729b75/t/603e2167a9c0b96ffb027c8d/1614684519754/
15 Chris Hatch, “Whose call to shade the Earth?” March, 3, 2023, accessed August 22,
2023, https://www.nationalobserver.com/newsletters/zero-carbon/2023/03/03/whose-call-shade-earth
16 Gobierno de Mexico, “La experimentación con geoingeniería solar no será
permitida en México” January 13, 2023, accessed August 22, 2023,
https://www.gob.mx/semarnat/prensa/la-experimentacion-con-geoingenieria-solar-no-sera-permitida-en-mexico
18 James Temple, “A startup says it’s begun releasing particles into the atmosphere,
in an effort to tweak the climate.” MIT Technology Review, December 24, 2022, accessed
August 22, 2023, https://www.technologyreview.com/2022/12/24/1066041
29 Alexander C. Kaufman, “The Climate Change Solution That Could Spark Global War”
OneZero, June 16, 2019, accessed August 22, 2023,
https://onezero.medium.com/the-climate-change-solution-that-could-spark-global-war-bd1cc9cce1a3
49 Barney Henderson, “Mahmoud Ahmadinejad says Europe 'stealing Iran's
rain’”, The telegraph, May 21, 2011, accessed August 22, 2023,
https://www.telegraph.co.uk/news/worldnews/middleeast/iran/8527455/Mahmoud-Ahmadinejad-says-Europe-stealing-Irans-rain.html
32 “La guerre des nuages, par Mathieu Simonet,” Nouvel Obs, March 28, 2022,
accessed August 22, 2023,
https://www.nouvelobs.com/tribunes/20220328.OBS56297/la-guerre-des-nuages-par-mathieu-simonet.html
54 Marine de Guglielmo Weber, “Crédits de refroidissement : quand la
géo-ingénierie commercialise le déni climatique,” IRIS France, February 17,
2023, accessed August 22, 2023,
https://www.iris-france.org/173813-credits-de-refroidissement-quand-la-geo-ingenierie-commercialise-le-deni-climatique/
Websites
4IPCC, AR5, SYR, 3.3, 2014, accessed August 22, 2023,
https://www.ipcc.ch/report/ar5/syr/
7 Interactive world map on geoengineering, accessed August 22, 2023,
https://map.geoengineeringmonitor.org/srm
8 IGCE, Institute of Global Climate and Ecology accessed August 22, 2023, www.igce.ru
9 SPICE, Stratospheric Particle Injection for Climate Engineering accessed August 22, 2023,
www.spice.ac.uk
14 SABRE, Stratospheric Aerosol processes Budget and Radiative
Effects,https://csl.noaa.gov/projects/sabre/
17 Make Sunsets American startup of solar geo-engineering, accessed August 22, 2023,
https://makesunsets.com/
21 Reynolds, J. et al. “Solar Radiation Modification: Governance gaps and challenges –
Summary.” Carnegie Climate Governance Initiative (C2G), March 2022, New York, www.c2g2.net
26 Felgenhauer, T. et al. “Solar Radiation Modification: A Risk-Risk Analysis - Summary,
Carnegie Climate Governance Initiative” (C2G), February 2022, New York, 2023, www.c2g2.net
45 Nubus is a (fake) French startup of solar geo-engineering, accessed August 22,
2023, https://www.nubus.fr/
46 World Meteorological Organization, International Cloud Atlas,
https://cloudatlas.wmo.int/en/homogenitus.html
47 Homogenitus is an art installation, https://homogenitus.fr/
48 Fake cloud Collective of artivist fighting against climate manipulations, accessed
August 22, 2023, http://fakecloud.fr
Author Biography
The author is a digital artist, designer and PhD researcher in aesthetics and science technologies of the
arts since 2018. She works on ecological, artistic and scientific fictions in society in relation to human
activities. Born in Paris in 1981, she lives and works in Cachan, France. In 2008, she obtained a Master's
degree in New Media at the Ecole Nationale Supérieure de Création Industrielle (ENSCI) and in
2009, she joined the EnsadLab research program. Her research focuses on "solar fictions, devices that
recreate the sun’s path", and she is currently working on climate fictions. She teaches at the
University of Evry (91) since 2013 and co-founded the CondéDesignLab in 2021.
Inhabiting the Edges
CéliaBoutilier
SACRe-PSL ENS/ENSBA — Paris Boutiliercelia@hotmail.fr
Abstract
My doctoral project in art-creation-research, which began in October 2021, is entitled "Imaginal
photography: for an aesthetic of symbiosis." In my practice as a photographic artist, making symbiotic
relationships, unexpected alliances, allows me to provoke a "suspension of evidence" which, through the
intermediary of the image, invites the viewer to share the experience of a different look. From this point on,
the very nature of perception is to admit ambiguity. The interdependencies in ecosystems, the principle of
hybridization between different species are themes addressed by the team of the laboratory of plant symbioses of
the French National Museum of Natural History of Paris with which I collaborate. In my research, I'm
specifically interested in the symbioses present in the orchid with the biologists Florent Martos and Eve
Hellequin. Transcending the notion of organism allows us to reconcile human beings with their natural dimension
of interdependence and interaction. The researchers in this laboratory are the extractors and curators of
natural plant resources, with which they shape new worlds such as the Grande Serre: a space in which I would
like to exhibit works in 2023-2024. These hybrid spaces, co-dependent with humans, seem to me to be fertile
places to dialogue with.
"We inhabit the world from the images we make of it."
This doctoral project starts from the intuition that aesthetics, as a study of sensible knowledge and its
artistic productions, is one of the main foundations of the imagination that allows us to think the world and to
inhabit it. It is from this imagination that imaginative practice is born. Without the imaginative faculty, a
whole area of reality is definitely off-limits to us, lost forever. With the loss of the imaginative faculty,
what we risk is not the loss of fiction, but the loss of reality. We run the risk of compartmentalizing the
sensible and intelligible worlds, of killing them off slowly, precisely because they only come to life and "come
alive" when they are in contact with each other.
Methodology
It seems to me today that the question of representation (of whom speaks and how he or she expresses himself or
herself) actually raises the question of the imaginary, that is to say, that of the fabrication of new
fictionalities. The issue of imagination is crucial, because it opens up perspectives of therapeutic narratives.
These proposals for circulation (in the making itself or in the setting of creations) are multiple hypotheses of
resilience. This is why certain fictionalities can be productive of the real and allow us, by this means, to
increase ourselves. Their power of augmentation is not only intimate, a creation has an impact when it goes
beyond its subject and manages to make language for something else. It’s therefore a question of
photographing this place with the desire to feel it fully, while knowing that from this precise place we will
touch other places, both terrestrial and human.
On the other hand, I’m interested in the interdependencies in ecosystems and the principle of
hybridization between species, because I believe that there is a realaesthetic, philosophical and political
stake in linking Art (culture in general) to Ecology. The laboratory and the studio are the two places where
photographic art was born. Botanical science and in particular the concept of symbiosis seems to me to be able
to achieve this necessary alliance (in biology the word symbiosis defines a lasting interspecific interaction of
mutual benefit). In this sense, I’m conducting this doctoral research project in collaboration with the
team of the plant symbiosis laboratory of the Paris Natural History Museum. I’m specifically interested in
the mycorrhizal symbioses of orchids: those that create networks between individuals and different species,
notably between fungi and plant roots. From the concept of symbiosis in biology, I intend to develop the concept
of symbiosis in the field of aesthetics: "what could be a symbiotic aesthetic? I propose the hypothesis that a
"symbiotic aesthetic" constitutes, on the one hand, a singular way of relating philosophically to the world (at
the source of an enlarged perception), and, on the other hand, a programme of plastic realisation. How can
symbiosis suspend the rational relationship of things to each other? To what extent does this suspension make it
possible to escape from designation and nomination? How can the new representation of the world proposed by
symbiosis change the way we act and interact? This project aims to sketch some answers, both formal and textual,
based on fieldwork with biologists, both in the field and in the laboratories of the MNHN of Paris.
Communication plan
More specifically, I'd like to present a few recent works produced following a scientific mission to Reunion
Island, which fall under what I call a "symbiotic aesthetic." Photographic assemblages are the first
satisfactory way I've found of aesthetically translating hybrid identity, through which the intertwining of
links manifests itself. Boundary images between the "natural," the imaginary and the technological. Different
photographic fragments are linked together by collage. The aesthetic is organic and lively, evoking both art
nouveau and virtual reality, where naturalistic photography meets pictorial inventiveness.
Focus of the work
In my work, it’s a question of "milieu" in that it takes us in its mesh.It’s a question of
resisting any hierarchical placement of knowledge and exploring the plurality of deployments that they allow,
replacing the aim of unity with the problematic of articulation.
I would like my images to be edges. Let them be meeting zones, spaces of contact, symbiotic and twilight places
where the obvious is suspended and where, in a strange familiarity,things enter into relationship. I favour the
observation of transitory and symbiotic forms,changes of state, the confusion of scales (from satellite to
microscope) and all other forms of telescoping practices, insofar as they impact the solidity of bodies, the
sharpness of contours and the fixity of images. The particular is then the bearer of more than just itself, in
that the image is made from an elsewhere that crosses the here, in order to give an account of a here that is
both always open and always worried.
"‘Border identity’ this is the term by which I usually define my own identity. I call it frontier,
anchored, not in a place of rupture, but, on the contrary, in a space of permanent coming together. The border,
as I define it and inhabit it, is the place where worlds touch, tirelessly. It's the place of constant
oscillation: from one space to another, from one sensibility to another, from one worldview to another. It's
where languages mingle, not necessarily thunderously, but naturally impregnating each other, to produce, on the
blank page, the representation of a composite, hybrid universe."¹
Pace of time and ghosts
We inhabit the world through the images we make of it. In this sense, it would seem that constructing ways of
inhabiting the world² also involves ways of making images. In the hollow of my plastic reflections, I apply
myself to modelling a specific temporality: that of a time which has a thickness and in which memory and dreams
are sedimented.
This time—like thought—is not linear (cause/effect), it is made up of feedback loops where the
chain of causality swallows its own tail (systemic thinking). The past is not only what happened, it is also
what was dreamed.
"There is always a shadow of something. And it stays for a long time. Now there is only the shadow of my
mother, who passed on to me the shadow of my grandmother, etc., etc. You see images projected normally and then
you see the same images that become a shadow ofthemselves. That's what Maniac Shadows is all about."³
Uncomfortable landscapes. There is a flaw, a break, a suspension of evidence. We must first remember our
heritage, the memory of the rumbling soil, the organisms and ghosts that inhabit us and that we carry, in spite
of ourselves, like amnesiac children. This requires, first of all, a refusal of postures or practices that are
constituted in a cannibalistic relationship to otherness. This relationship is one of attention, concern and
vigilance. The state of vigilance is a state of slowness, of contemplation that stretches over time. It is a
profound attention that takes the opposite view of the lighted time of immediacy to become a space of common
sharing. It is a care that we cultivate in the relationship that we maintain with the other whether human or
non-human. Taking care of our relationships and their modes of existence goes hand in hand with taking care of
the making of our images." The world to which we belong is first and foremost the world we carry within
us."⁴
References
Books
1 Léonora Miano, "HABITER LA FRONTIÈRE, Paysages francophones -Journée
internationale de la francophonie, Université de Copenhague (Danemark)" conférence de 2009
recueillie dans HABITER LA FRONTIÈRE, L’Arche, 2012, p.25.
2 Bruno Latour, Où atterrir ? - Comment s'orienter en politique, La
Découverte, 2017.
4 Ibid. Léonora Miano, 2012, p.25.
Webvideo interview
3 Chantal Akerman - ManiacShadows, Nuit
Blanche 2013 au Théâtre du Châtelet, Paris.
Bibliography
David Abram, The Spell of the Sensuous: Perception and Languagein a More-Than-Human World,
Vintage, 1997.
Jean-Christophe Bailly, L’imagement, Seuil, 2020.
Jean-Christophe Bailly, Le temps fixé, Bayard, 2009.
Jonathan Crary, Techniques de l’observateur, Vision et modernité au XIXe
siècle, Broché, 1990.
Lorraine Daston, Peter Galison, Objectivité, Les presses du réel, 2012.
Gilles Deuleuze, Logique de la Sensation, 1981.
Gilles Deuleuze, Desert Islands and Other Texts, 1953-1974.
Cambridge, MA: MIT Press, 2004.
John Dewey, L’art comme expérience, 1934.
T. S. Eliot, The Waste Land, Faber & Faber, 2015 [1922].
Cynthia Fleury, "L’imagination renaissante : entreRévélation et Intellection"
in Imagination, imaginaire,imaginal, Coordinated by Cynthia Fleury, PUF, Paris, 2016.
Donna Haraway, Staying with the Trouble: Making Kin in theChthulucene, Duke University
Press, Durham, 2016.
Donna Haraway, When species meet, University of Minnesota Press, Minneapolis, 2007.
William James, Philosophie de l'expérience. Un universpluraliste, 2007.
Bruno Latour, Down to Earth: Politics in the New ClimaticRegime, ed. Cambridge, UK,
Polity Press, 2018.
Bruno Latour, Facing Gaia: Six Lectures on the Political Theologyof Nature, Cambridge, UK, Polity
Press, 2013.
Bruno Latour, Enquête sur les modes d’existence une anthropologie des modernes, La
Découverte, 2012.
Bruno Latour, La science en action - Introduction à la sociologie des sciences, La
Découverte, 2005.
Michael Marder, Plant-Thinking – A Philosophy of VegetalLife, Columbia University Press,
2013.
Michael Marder, La plante du philosophe - Un herbierintellectuel, Mimesis, 2020.
Lynn Margulis, Symbiogenetics - Origin of MitoticCells from Bacterial Communities in the
ProterozoicEon, 2011 [1992].
Lynn Margulis, Symbiosis as a Source of EvolutionaryInnovation: Speciation and Morphogenesis,
(TheMIT Press, 1991).
Giuseppe Penone, Jean-Christophe Bailly, Sève et pensée, Paris, broché, Paris
2021.
Bernard Réquichot, "Métaplastique" dans Écrits, 1955, p.87.
Simon Schaffer, Steven Shapin, Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life,
1985.
Anne Simon, Une bête entre les lignes. Essai dezoopoétique, Marseille, Wildproject,
2021.
Isabelle Stengers, La sorcellerie capitaliste : Pratiques de désenvoûtement, La
Découverte, 2013.
Isabelle Stengers, L’invention des sciences modernes, Flammarion, 1993.
Andreï Tarkovski, Le Temps scellé.
Anna Lowenhaupt Tsing, The Mushroom at the End ofthe World: On the Possibility of Life
inCapitalist Ruins, Old Saybrook, CT, TantorMedia, 2017.
Alfred North, Whitehead, Le concept de nature, J.Vrin, 2006.
Walt Whitman, Leave of grass The Original 1855, Edition, Paris, Broché, 2020.
Journal articles (print)
Carla Hustak, Natasha Myers, "InvolutionaryMomentum: Affective Ecologies and the Sciences ofPlant/Insect
Encounters", Vol. 23, Num. 3, Brown University and differences, A Journal of Feminist Cultural Studies, 2012.
Sabine Plaud, "Vie du langage, vie des images : une marque de continuité dans la philosophie de Ludwig
Wittgenstein".
Katrin Solhdju, "L’expérience "pure" et l’âme des plantes. James lecteur de Fechner",
2007.
Isabelle Stengers, Didier Debaise, L’insistance des possibles. Pour un pragmatisme
spéculatif, Multitudes 2016/4, n° 65, 82-89
Webvideo interview
Bruno Latour, “Agency at the Time of theAnthropocene”, New Literary History 45, 2014, 1-18.
Author Biography
After a DNAP at ENSA Dijon (2016), a year at ERG Brussels (2017), a collaboration with LadHyX Polytechnique
(2014-2019), Célia Boutilier joins the Beaux-Arts de Paris (2017) to defend a DNSAP (2019). Since 2014
she has been collaborating with research laboratories (microfluidics - Polytechnique, and mycorrhizal
symbioses - National Museum of Natural History) where she is interestedin how imaging techniques participate
in theelaboration of knowledge. Since 2016, herwork has been exhibited at fairs and international events,
including the National Gallery in Copenhagen, the Cité Internationale des Arts in Paris and the Maison
des Métallos. It has been quoted inseveral scientific journals, including Physics Today,
The NewYork Times, Le Monde. Since 2020 it has also been included inprivate collections. In
October 2021 Célia Boutilier starts a PhD in "Sciences, Arts,Creation, Research" (SACRe-PSL / ENS) at
the National Superior School of Fine Arts in Paris (ENSBA).
Mytherrella: an interactive installation hallucinating mythological auroral formations
DorinCucicov
University of Bucharest, Romania Bucharest, Romania cucicov@gmail.com
Abstract
Up until very recently, mythological tales predominated in the explanation of the polar auroras. Today, we
understand that the polar lights are a phenomenon brought on by the solar winds' interaction with the
magnetosphere of the Earth leading to precipitation of energetic particles on the topside ionised atmosphere.
However, for a long time, this phenomenon inspired vibrant trans-border stories with a profound impact on local
communities. In our interactive installation, Mytherrella, we aimed to create a dynamic environment in which
scientific data and mythological storytelling unearth new imaginaries about the aurora borealis. The generative
video integrated in the installation uses a custom real-time StyleGAN algorithm that continuously samples from a
model trained on a large set of all-sky auroral images acquired in Kiruna, Sweden. This method enables live
interaction with the generated video, producing novel synthetic auroral formations that are unpredictable while
remaining within the bounds of the learned features. By combining the dataset with a relatively small number of
alternative-style images, the diversity of the generated content is increased, creating a divergent effect that
reinforces the mythological narrative. We share the technique of interactive video generation as well as the
research process behind the creation of the work.
Keywords
interactive installation, mythology, live StyleGAN, aurora borealis, polar lights.
Myths about the aurora borealis are prevalent in the cultural heritage of populations from the polar regions of
the Earth. Surprisingly, these stories have not permeated pop culture in the same way the images of the aurora
borealis did. Many of the stories, despite being from different continents, share many similarities: contact
with the spirit world, vengeance of slain enemies, communication with the dead through whistling or playing ball
games in the sky. For some, the appearance of the aurora borealis is a bad omen; for others, it can be bad only
if disrespected by whistling at it or staying too long outside. Others have more poetic explanations for auroral
formations.5, 6
This paper presents a virtual environment, the result of a creative exercise that leverages both scientific
data and stories from various communities about the polar lights.
Combining them produces novel visions of a future in which humans reinvent themselves through collaboration
with artificial artifacts. Mythology has been at the forefront of explaining the unknown and stimulating human
curiosity for centuries. These stories are refined representations of humanity's deep connection to the natural
world around it. The mystical characters in these tales instilled fear and caution in humans, effectively
negotiating a reverence for natural phenomena. In the past centuries, science has solved numerous natural
enigmas, relieving some anxiety but also making humans less respectful and more entitled.3
Our project, named Mytherrella, addresses the subject of symbiosis on multiple levels. It frames a creative
experiment by integrating empirical data and mythological narratives. The equal treatment of science and myths
aims for a reevaluation of our relationship with natural phenomena. Through the similarities between the stories
of various cultures, cultural symbiosis is highlighted on a global scale, diminishing the significance of
political borders.
Concept and construction
Mytherrella is an endless process of imagination informed by facts from science and driven by myths surrounding
the aurora borealis. Starting from a dataset of all-sky images showing auroral activity recorded by the Swedish
Kiruna auroral observatory, it imagines new auroral formations. It provides control over its hallucinations in
an interactive framework. Both human and machine are in search of the lost/improbable auroras as seen by the
first people that described them.
Mythology of the aurora
One of the most common mythological symbols associated with the polar lights is their connection to the
spiritual world. Polar lights, according to the Greenlandic Inuit, were messages from their deceased friends and
relatives.1; The Meskwaki, a Native American people with homelands in the Great Lakes region, held similar
beliefs. They believed that by whistling to the lights, they could interact with them, thus communicating with
the dead. Some sources indicate that some North American Inuit also used whispering during auroral events to
send messages to the dead.⁵ More active manifestations of the aurora borealis were associated with spirits
playing games in some Greenlandic stories, while on the European continent, in Scotland, Ireland and Norway they
were associated with dancing spirits.1, 4
Some of the more poetic interpretations of the polar lights are found in Sámi peoples, residing in the
north of Finland. They named the aurora - revontulet (fire fox). It is said that the lights are created by a fox
with glittering fur running across the mountain area. 1; In other stories, polar lights are an omen of disaster
and war. These are found in Tlingit indigenous peoples of the Pacific Northwest Coast of North America, Skolt
Sámi in Northern Europe and even in some Chinese and Roman literature.1, 2
There may be several myths about the aurora in a single area, ranging from optimistic interpretations to
terrifying ones. These are dependent on particular historical moments and the social and natural influences that
shaped those times.
Building Mytherrella
The physical construction of the Mytherrella installation was inspired by the early designs of the auroral
observatory in Yerkes, Wisconsin, United States.4 These small sites, fitted with all-sky cameras,
collected images of the auroral events aimed for scientific investigation. In our installation, an immersive
context is created inside a small booth where one can experience and interact with the story in a private
setting. The outside of the booth is painted with selected mythological texts, which acts as a representative of
the old media in contrast with the interactive digital content. An interactive stand inside the booth provides
two analog potentiometers for controlling latitude and longitude on a minimalist digital map. Using these
knobs, a specific location can be identified on the global map. The map is centered on the poles and only
depicts the boundaries between water and land; it does not include any other information, such as names of
nations, borders, seas, or rivers. This choice was made in order to disrupt the influence of the political
borders in the collective perception and suggest a new perspective on interpreting the global map.
Ten locations are hidden on the digital map. They can be discovered by identifying their approximate location
using the navigation controls. These coordinates are linked to locations from which we gathered mythological
stories. Based on these texts, we used text-to-image tool Midjourney (1) to generate unique images. Figure 2
contains one of these images generated based on the mythological texts of the Native Americans describing the
aurora borealis as a means of communication with the spirit world.
The second screen, positioned in front of the control stand, displays a continuously generated video with
auroral formations. This video is created using a StyleGAN model that was trained on a mixed dataset of
scientific and artistic images of the aurora borealis.8 The main corpus consists of 9834 selected
images from a dataset of all-sky images depicting auroral activity provided by the Kiruna Atmospheric and
Geophysical Observatory at the Swedish Institute of Space Physics. All images have a standard round format, as
seen in Figure 3.
(1). Midjourney. https://midjourney.com
Data blending was applied using another 150 artistic images of the aurora borealis in order to extend the
dataset and introduce visual features that break the rigor of the scientific representation. The process of
projecting an image into the latent space produced images that were continuously served to create the video.
When idle, the video would display auroral formations as learned from the main corpus of all-sky images. Finding
an image on the world map while interacting with the installation would begin projecting that image to the
model, disrupting the normal representation by also sampling from the artistic images' features.
Figure 4 describes the approach of continuously serving images for the video projection. A script monitors for
new files that are selected by interacting with the installation. Once a file is detected, it signifies that an
image was identified on the map. This triggers an immediate process of projecting that image to the model. The
resulting images are displayed sequentially creating the effect of video continuity. The 3 processes of the
architecture are independent: generating based on the source image; serving an image for generation; displaying
received images. They are linked only by the configuration of the listeners. This loose coupling makes it simple
to integrate with other systems to serve various seed images or display generated images in various formats.
Discussion
The myths of the aurora borealis are rich and creative stories that have surrounded this natural event for a
long time. Mytherrella is a direct comment on the relationship between science and mythology. Figure 5
exemplifies one sequence of a video generated by projecting the image from Figure 3. It starts from the format
of an all-sky image and navigates the latent space in search of the features from the projected image. By using
the technique of interactive generative video, we sought to create a subtle transition from visuals that have a
scientific aesthetic to a more abstract look that resembles some of the features of the mythological image while
also preserving the features of the aurora. This type of interaction contributed to the immersive character of
the experience by partially delegating the responsibility of the generated image to the person interacting with
it.
Some of the visitors discovered that by rapidly shifting through mythological images on the map, they could
further distort the visual representation of the aurora, taking more control over the generative process. This
encouraged visitors to spend more time with the work and explore both the map and the generative mechanism as
they discovered more layers of interaction. By doing so, they also delved deeper into the mythological stories
about the polar lights, drawing connections between different cultures that transcend political borders.
Without nation-state borders or other human interventions, the world map used in the installation is
purposefully rendered in a minimalist style. The fact that it is centered on the poles further distorts how
visitors typically perceive the world map. The visitors identified common archetypes in geographically remote
regions by experiencing mythological stories from various cultures.
Mytherrella is an experiment in creating a shared space for scientific data and personal stories. Such playful
settings that mix diverse fields of study and research approaches may motivate us to delve deeper and from
different angles into the stories behind them.
Acknowledgements
Mytherrella was developed during a residency in the Space Science Institute from Măgurele, Romania,
organized by Qolony, Bucharest. Interactive sound for the installation was created by Iulia Smeu. Dataset of
all-sky auroral images was provided by Kiruna Atmospheric and Geophysical Observatory at the Swedish Institute
of Space Physics.
References
Books
1 Asgeir Brekke, Egeland Alv, The Northern light: from mythology to space research,
Springer Science & Business Media, 2012.
2; John S. Major, Heaven and earth in early Han thought: Chapters three, four, and five of the
Huainanzi, SUNY Press, 1993, 203-204.
3 James Burke, The impact of science on society, vol. 482, Scientific and Technical
Information Branch, National Aeronautics and Space Administration, 1985, 3-32.
4 D. C. Rose, Keoeeit-The Story of the Aurora Borealis, by W. Petrie, Arctic 17, no. 2,
1964, 26-27.
Journal article (print)
5 Robert Holzworth, "Folklore and the Aurora," Eos, Transactions American Geophysical Union 56,
no. 10, 1975, 686-688.
Journal article (online)
6 Tero Mustonen, "Inuit and Chukchi Star lore: Reflections on Ursa Major, the North Star and
Northern Lights," Snowchange co-op, 2016. Accessed December 4, 2022,
http://www.snowchange.org/2016/11/inuit-and-chukchi-starlore-a-new-discussion-paper-out/
Websites
7 Anniina Jokinen, "Aurora Borealis, The Northern Lights, in Mythology and Folklore," Luminarium,
4 Feb 2007, Accessed December 4, 2022, http://www.luminarium.org/mythology/revontulet.htm
8 Burloiu, Grigore, “controlling StyleGAN in real time,” Accessed December 4, 2022,
https://rvirmoors.github.io/2021/01/04/realtime-stylegan/
At the Sources of an Artistic Mutation towards Science: the First Years of the Journal Leonardo (1968-1981) as
a Forum for the Pioneers of Digital Art
CamilleFrémontier-Murphy
RCM Galerie Paris, France camille@rcmgalerie.com
Abstract
The journal Leonardo was founded in 1968 by Frank Malina, a pioneer of light art in Paris and of aerospace
science in the United States. Leonardo encouraged artists to publish their work in the manner of scientists. It
was a new initiative developed in the revolutionary context of the 1960s and that allowed artists, scientists,
psychologists to exchange on the subjects of art, perceptions, science, society... Many pioneers of digital art
took part in the adventure, including Vera Molnar, Zdenek Sykora, Charles Csuri, and the artist-novelist Herbert
Franke, who became the advocate of the theories and the protagonists of digital art spread out over the four
corners of the planet, just about everywhere a computer could be found. The growing group was defending a more
conceptual approach to art, closer to the spectator, a new form of art rooted in Constructivism and that was in
symbiosis with society’s mutation towards technology.
When the journal Leonardo was founded half a century ago by the artist-engineer Frank J. Malina (1912-1981),
the first generation of artists working with computers and electronic media was still small and far apart.
Leonardo provided them with a permanent forum to exchange, share, and publicize their reflections and the state
of their work with their peers and also a wider audience at a time when international and interdisciplinary
means of communication were not readily available. While some of these pioneers have recently passed away, such
as Herbert Franke, Ken Knowlton, Charles Csuri and Jean-Pierre Hébert, others are still active, such as
Vera Molnar, Joan Truckenbroad, Ruth Leavitt, Jean-François Colonna, Jean-Claude Marquette, Hervé
Huitric and Monique Nahas. Together they have defined the foundations of a community whose concerns and
approaches from a not-so-distant past still resonate today.
The Journal Leonardo and the Buoyant Spirit of 1968
In January 1968, when Leonardo was launched, it was an effervescent time: changes and mutations in society were
crystallizing. A few months before the youth took to the streets of Paris, the ambitious multimedia Living Art
program of the ARC (Animation - Recherche - Confrontation), directed by Pierre Gaudibert, began organizing its
exhibitions and interdisciplinary events at the Paris Musée d'Art Moderne. This initiative was
revolutionary. The museum was no longer perceived as a temple, but as a laboratory that would invite a younger
and more diverse public until 1972. At the same time, an art and technology program began to develop at LACMA
under the direction of Maurice Tuchman. In London, the first major exhibition of electronic art and music took
place at the ICA (Institute of Contemporary Arts) in the late summer of 1968, demonstrating the scope of the
movement. In Zagreb, the series of exhibitions and symposia organized by the New Tendencies under the impetus of
Abraham Moles (1920-1992) opened its fourth chapter dedicated to computer art.
Malina, a Paris-based American artist, was at the center of this ferment. A few years earlier, the Paris
Musée d'Art Moderne had acquired one of his grid paintings, Deep Shadows (1954). He participated in New
Tendencies with a selection of his luminous mobile paintings—Signals, Voyage II, Sink and Source. At
Cybernetic Serendipity, he exhibited his interactive light work Entrechat and presented his new journal
Leonardo. He even helped the exhibition to travel to San Francisco’s Exploratorium, recently founded by
his longtime Caltech friend Frank Oppenheimer.
In fact, Malina himself was a former scientist turned artist. He had designed and supervised the launch of the
first scientific rocket, the WAC Corporal, and had founded the Jet Propulsion Laboratory, in Pasadena,
California, which is now the premier laboratory for the exploration of the solar system. After World War II, he
opposed the use of his rocket as a ballistic missile with a nuclear warhead, and had worked for several years
with Joseph Needham, head of the Natural Sciences Division of UNESCO in Paris, "to reduce barriers to the free
movement of scientists and engineers between nations." Needham also introduced him to György Kepes, a
member of the New Bauhaus in Chicago and the founder of the Center for Advanced Visual Studies at MIT.
Kepes’ book, Language of Vision, was one of the most important artist's manuals ever written
in the United States before Moholy-Nagy’s Vision in Motion (1947). His sociocultural ambition was
clear: "The goal is a new vital structure-order, a new form on a social plane, in which all present knowledge
and technological possessions may function unhindered as a whole"1.Inspired by these texts, and
sometimes frustrated by the bureaucracy of international organizations, Malina had begun an artistic production
at the intersection of art and science. Since the mid-fifties, he had been developing a corpus of moving light
paintings with themes reminiscent of recent space travel.He was actually one of the first artist-scientists of
his time, and soon came up with the idea of creating a society or journal for artist-scientists to meet and
exchange ideas, but he couldn’t find many of them. With the preparation of exhibitions on cybernetics in
London and computer art in Zagreb, a growing community of artists working on the frontiers of science emerged.
Max Bense and Abraham Moles began to apply Norbert Wiener’s cybernetic and information theory to aesthetic
problems. Together they edited the first issue of the magazine Bit international, first published in 1968 by the
Museum of Zagreb within the framework of New Tendencies. Unfortunately, the initiative didn’t last after
1972.
The Emancipated Artist Joins Science
Leonardo, for its part, wanted to be open to all forms of art: "What I was interested in was people who worked
in physics, or chemistry or physiology, or astronautics, or what have you, who introduced into their artworks
the visual experiences and the ideas of their work. And I found many of these people were not doing that. So, I
said that it was premature to try to make a journal for this kind of people. They weren't enough. And since
there was no professional journal for any kind of visual artist, the thing was to make a journal that was open
to any art, and there the artists would write about their work themselves, helped sometimes, and would be by
someone in the direction of relationship between art, science, and technology. (...) And at the end I wanted an
international journal to try to bring about better international cooperation, for peace."
Needham, who became famous for his history of Chinese science, suggested the name of the journal and would be
one of its honorary editorial advisers2. “Creativeness in art does not appear to me to differ
in kind from creativeness in science or any other human activity”, stated Malina in the brochure for his
first solo exhibition in Paris in 1953”3. It was a time when the excessive division of
knowledge reinforced by the specialization of disciplines, began to be questioned. In Gothic
Architecture and Scholasticism (1951), Erwin Panofsky described a common denominator for different forms
of human endeavor in a given period. Creation appeared to be culturally determined by mental habits common to
several disciplines.Leonardo defended the development of art as any other field of research. It was published by
the Pergamon Press of Oxford, which was already responsible for Acta Astronautica, the organ of the
International Academy of Astronautic, co-founded by Malina and von Kármán. English was quickly to
become the dominant language, as it was in the process of doing in the scientific field.Artists, as scientists,
would publish their own articles to discuss their practice, no longer leaving this mission to critics alone.
Artists would become literate persons able to discuss fundamental artistic, social and political
issues.Contributors could be European or American, including African American, Asian, male or female. The
invited authors could be computer artists, but not exclusively. Artists working in a constructivist vein or
kinetic artists, such as Nino Calos or Pol Bury could also contribute. These two artists were also well known
for their writings and will join Joël Stein, Anthony Hill, Vladimir Bonacic and Herbert Franke on the
editorial board of Leonardo.Scientists were also welcome to contribute to Leonardo. The artist was now a social
actor in his own right, able to interact with scientists. The psychologist James Gibson, who was one of the
honorary editorial advisers, wrote a paper on visualization and the point of observation in 1974⁴. Fred
Lawrence Whipple, head of the Smithsonian Astrophysical Observatory, shared in the first edition of Leonardo a
rather interesting way of conceiving paintings from random arrangements of forms and colors. The point was
certainly to clarify the inherent differences between randomness and chaos⁵. Its goal was to build bridges
between different communities, to advocate peaceful international relations. Malina's convictions have not
changed since he left JPL, when he said, “that ideas and effort were really needed now to find ways for
sovereign states to function in peace together, rather than to develop better means of destroying
themselves.”
A dedicated project for a dedicated community, Leonardo was a quite peculiar project of an artist dedicated to
creativity and his peers. It would be a large-scale, self-funded enterprise based in Paris, at a time when no
institution would have supported such an adventure for such a long time. It was made possible by Malina’s
income from the rocket engine company Aerojet.
Different Forms of Expressions for New Ambitions
”Painters during the last several centuries have painted the same things over and over,” Malina
complained in a letter dated October 21, 1953.From György Kepes—now one part of Leonardo’s
editorial advisors and authors —to most digital artists, the same vision prevailed: “the application
of the computer in the realm of art is to see it as part of the technological revolution.”
In 1969, the artist Frederick Hammersley recounted how, tired of classical painting, he enrolled in a computer
drawing class taught by Charles Mattox, a co-editor of Leonardo at the University of New Mexico:“ During
my first semester of teaching at the University of New Mexico in 1968, I was invited by Charles Mattox to attend
a computer drawing class. This happened to coincide with a time in which I had painted myself out, so I welcomed
this new experience. I was shown how to prepare a computer program and how to transfer it to an IBM punch card
by machine.”⁶ Working with these extremely large computers was counterintuitive to most artists of
the 60s, and Mattox decided to supervise a program designed for them. It was the beginning of a long adventure
of programs written for artists. ART1 was written in FORTRAN IV by Richard H. Williams of the Department of
Electrical Engineering and Computer Science. Artists and art students from the University influenced its
development and among them, Katherine Nash, who presented it in Leonardo with illustrations of her works. She
began her paper by stating: “Twenty years of a computerized society make it apparent that twenty years
hence no artists can ignore the computer.”⁷ Herbert Franke was a member of Leonardo’s
editorial advisory board. He explained that with the technological age, the computer could become an instrument
of art like an instrument was for music.The undeniable interest of manipulating such instruments was, without
question, to dissociate artistic practice from the manual arts. Drawing was no longer done by hand, but with the
computer. Art became multimedia and conceptual, rather than the romantic expression of a marginal artist
presented in gilded wooden frames. It could be defined, as in cybernetics, as a signal perceived by the
viewer.At the turn of the 1970s, Franke concluded: “If the day comes, as we are told it will, when every
household is connected to a computer network via a display terminal, anyone will be able to tune in on a large
variety of aesthetic programs. For this purpose, variable programs that permit intervention by the viewer will
be far more suitable than the static programs that are available today. Perhaps in this way the gulf that yawns
between the producer and the consumer will be slowly bridged.”⁸
The work of art created with the computer has been defined with functions similar to those of a book:
distributed in multiple copies, directly accessible at home, and in a special way interactive.Many artists will
share their own approach in Leonardo, such as Zdenek Sykora in 1970, about computer-aided geometric
paintings⁹, or Vladimir Bonacic10, Charles Csuri about facial sketching11, Edvard
Zajec12, Grace Hertlein about the largest computer art exhibition organized in the USA in
197513. The artist Ruth Leavitt and her husband Jay—responsible for heat shields for
spacecraft—presented their reflections on the geometric deformation of a figure, using the same principles
as Moiré14. Vera Molnar also published an article about her work in 1975. Her husband
François had published on art and science in 197415. For some who did not yet have access to
computers, such as Ben Laposky, they presented their skills using analog systems16. Access to a
computer was still extremely complex and, as Colette Bangert nicely summarizes the situation: “Although an
artist need not be knowledgeable in mathematics, he must be prepared to talk to a
mathematician”17.
With all its sometimes dramatic, social, poetic, and technical dimensions, the writings of digital artists in
the journal had become so extensive in the 1970s that Malina decided to collect the articles in a book, Visual
Mathematics and Computer, 1979. It was certainly not exhaustive, but it testifies to the beginning of this new
artistic adventure, just before it reached a low point, which is now being rediscovered. It sheds light on the
roots of today's electronic art lie18.
References
1 György Kepes, Language of Vision, Chicago, 1944, 12-13.
2 Joseph Needham, "Time and History in China and the West,” Leonardo 10, 1977,
233-236.
3 Galerie Henri Tronche, Frank J. Malina, Paris, October, 1953.
4 James J. Gibson, “Visualizing Conceived as Visual Apprehending without any Particular
Point of Observation,” Leonardo 7, 1974, 41-42.
5 Fred L. Whipple, “Stochastic Painting,” Leonardo 1, 1968, 81-83.
6 Frederick Hammersley, “My First Experience with Computer Drawings,” Leonardo 2, 1969,
407-409.
7 Katherine Nash, Richard H. Williams, “Computer Program for Artists: ART 1”,
Leonardo 3, 1970, 439-442.
8 H. W. Franke, “Computers and Visual Art,” Leonardo 4, 1971, 331-338.
10 Vladimir Bonacic, “Kinetic Art: Application of Abstract Algebra to Objects with
Computer-Controlled Flashing Lights and Sound Combinations,” Leonardo 7, 1974, 193-200.
11 Mark L. Gillenson, B. Chandrasekaran, Charles Csuri, Robert Schwartz, “Computer-Assisted
Facial Sketching”, Leonardo 9, 1976, 126-129.
12 Edvard Zajec, “Computer Art: a Binary System for Producing Geometrical Nonfigurative
Pictures,” Leonardo 11, 1978, 13-21.
13 Grace C. Hertlein, “Report on the 2nd International Conference on Computers and the
Humanities, Los Angeles, Calif., U.S.A.”, Leonardo 9, 1976, 43-45.
14 Ruth Leavitt, Jay Leavitt, “Pictures Based on Computer Drawings Made by Deforming an
Initial Design,” Leonardo 9, 1976, 99-103.
15 Vera Molnar, “Toward Aesthetic Guidelines for Paintings with the Aid of a
Computer,” Leonardo 8, 1975, p.185-189, François Molnar, “Experimental Aesthetics
or the Science of Art”, Leonardo 7, 1974, 23-26.
16 Ben F. Laposky, “Oscillons: Electronic Abstractions,” Leonardo 2, 1969,
345-354.
17 Colette S. Bangert, Charles J. Bangert, “Experiences in Making Drawings by Computer and
by Hand,” Leonardo 7, 1974, 289-296.
18 Frank Malina (ed.), Visual Art Mathematics & Computers. Selection from the Journal
Leonardo, Oxford, Pergamon Press, 1979.
Author Biography
Camille Frémontier-Murphy, holds a Ph.D. in History and Civilizations, Centre Alexandre Koyré,
E.H.E.S.S., Paris, a Master's degree in Art History, Paris IV, Sorbonne, and a Master of Science, History of
Science, University of Oxford, Oriel College. Together with Robert Murphy, she runs the RCM Galerie in Paris,
whose exhibition program explores the avant-garde and the relationship between art and science in the second
half of the 20th century.
Sensoriums for the Ephemeral — gamification of values
JānisGarančs
Liepaja University & RIXC Liepaja & Riga, Latvia jg@rixc.lv
Abstract
This paper introduces the author’s current practical investigations, during the creation of a series of
audio-visual installations and VR environments, ‘gamifying’ time-sequenced changes of multiple
values, e.g., from financial data feeds. The work series critically reflect upon gambling tendencies in the
global trading of various, increasingly immaterial assets. Algorithms and emotions of greed, euphoria, and
despair meet in virtual scenery, where almost everything can be offered as a fungible and non-fungible token for
exchange and trade. The project proposes speculative variations of dystopian “hybrid organisms”
representing macroeconomic value exchange as a symbiotic relationship that competes for humanity’s
attention and involvement.
Keywords
Gamification of investment, market society, immersive analytics, ephemeral values, neuroaesthetics
This work series is critically motivated by the re-emergent and growing prominence of gambling factors in
global economic activities—such as institutional promotion of increasingly complex investment products for
masses, crypto-asset trading, online casinos, etc.
Political philosopher Michael J. Sandel describes several last decades as a “drift from ‘market
economy’ to becoming a ‘market society.’”¹ As sociologist Georg Simmel
had already observed in 1900: “Reality and value as mutually independent categories through which our
conceptions become images of the world.”²
Mapping of the behaviour patterns in trader psychology has been an important aspect of the training in the
trading process, besides the implementation and development of various mathematical models. It now appears that
value storage and trading infrastructure increasingly merge with methods of manipulation of human attention and
emotions, and are mediated by computer networks, and increasingly—Machine Learning and AI. There is a
hope, that AI-assisted macroeconomic system could live in symbiosis with human society by working closely with
humans to understand and respond to their needs and goals.
Gamification of investment as a phenomenon
Various schools of economic thought, which emphasize the importance of free markets and individual choice,
might argue that the gamification of investment is a natural and inevitable outcome of a free market economy. In
this view, the gamification of investment is simply a response to the demands and preferences of consumers and
is therefore a legitimate and desirable part of the market process. The most recent examples are various
cryptocurrency and crypto-infrastructure products, meme-stock trading by inexperienced
“investors,” peer-to-peer loan financing, allocations of state pension funds into high-risk
assets, etc. (the most extreme case being introduction in Bitcoin as the legal tender in the country of El
Salvador).
Adversary interactions between technology and legislation in economic activities, especially in
finance—can be described as multilayered symbiotic relationship with society—that can manifest
itself through both mutually beneficial and predatory to society by all involved actors. There have been
historic precedents how rationally motivated mechanisms of value estimation, asset exchange and wealth storing
has caused irrational mass behaviours, financial bubbles. That has severely impacted individual destinies,
families, nations, and whole generations.
It has caught attention not only in sociology, as symptom of serious social “disease”—but
also caused legal interventions of large supervisory bodies managing legislation frameworks for investments and
finance.³Participation in online infrastructure for trading—through easily accessible
interfaces, provides new tools for masses, promising control, but, at the same time, becomes part of
surveillance tool for organised value brokers, setting preconditions for unfair competitiveness, “rigged
game.”
Simulated sensorium for immersive mappings
This artwork series—immersive installations using 3D imagery—feature variations of speculative
illustration of a dystopian ‘hybrid organism’ and simulated macroeconomic system, whose purpose is
transformation of energy and resources through a behaviour and appearance that ranges from ludic (playful) and
awe inducing. One iteration uses live trading data from various sources to create a “virtual
landscape” of trading activity, to observe ‘passively. In another iteration, visitor can make
‘virtual bets’ to experience growth and diminishing of their “investment“ through the
cycles and complex “monetary circuits.“
It is possible that McLuhan today would see immersive analytics for online trading as an example of the
‘extended sensorium’ in a way that media technologies are extending and enhancing human abilities in
the realm of financial decision-making in “global village.“ The concepts of “360°
gaze“ (Stiegler, 2011) or more broadly dispositif (Foucault, 1977) are relevant how interfaces
for value exchange modifies our behaviour of the mediated self under the impression of surveillance. Stiegler
relates to the “data behaviourism“ (Rouvroy, 2012) in the meaning of “producing
knowledge about future preferences, attitudes, behaviours or events without considering the subject’s
psychological motivations, speeches or narratives, [instead] relying on data.“ There are observations,
that visual representations of financial markets and sophisticated products, can be misleading by design choices
or be used “as tools of manipulation” (Krawczyk, 2021).
There has been a range of historic and recent examples of 3D visualization of various data sets—as well
as GUIs for various professional software products and artworks. The projects presented here, however, try to
establish a gradual journey between the extremes of design strategies: intended usability/function and sublime
"dysfunction" as an aesthetic experience. These audio-visual études are practical investigations into the
continuum between immersive analytics and VR/AR artwork. The emerging research area of immersive analytics is
considered a fusion of more recent developments in visualization, auditory displays, computing, and machine
learning.⁴
Between emotional and non-emotional
How do we use technology in hybrid forms to extend our perceptual and sensual abilities? In which way do bodies
merge with technological agencies? In the immersive analytics scenario, the difference between emotional and
non-emotional evaluation of data refers to the use of emotional or affective factors in the analysis and
interpretation of data. In a non-emotional evaluation, data is analysed and interpreted solely based on its
objective characteristics, such as its numerical value or its statistical significance. This type of evaluation
is based on logical and rational reasoning and does not consider the emotional or affective reactions of the
user or the audience.
In contrast, an emotional evaluation of data in an immersive analytics scenario involves considering the
emotional or affective reactions of the user or the audience in the analysis and interpretation of data. Also,
the concept of ideastesia,⁵ which refers to the experience of emotions in
response to abstract ideas, can be applied to the above-mentioned topics in several ways—creating
ephemeral audio-visual representations of “placement,” “reference points”—as
“spatial anchors” in VR and AR setting.
Metaphors and terms
Through algorithmic feedback, virtual instruments with anisotropic properties enable on-screen
manipulation, switching between egocentric and exocentric navigation, and the synchronized off-screen
structuring of the content. The analogy of simulated anisotropy—as an algorithmically simulated
phenomenon of anisotropy known in physics, chemistry, microfabrication, neuroscience as direction-varying of
material, tissue, and space properties.
Anisotropic properties refer to the ability of an interface to have different properties or behaviours in
different spatial directions or contexts.
The notion of the instrument is used both as a generic definition of a task-specific tool (or
toolset, involving appearance and action parameters, that be used to e.g., detect, measure, modify a specific
situation or manipulate a specific object)—and in analogy to the musical, medical, or industrial
instrument. Other terms “ensemble visualization”or “ensemble data” are used in
the context of data visualization as “concrete distributions of data, in which each outcome can be
uniquely associated with a specific run or set of simulation parameters.”⁶
Data staging
For the installation, the data acquisition modules feed from several sources of financial data (stock-
and cryptocurrency exchanges, aggregators, etc.) over their public Internet APIs. Installation accumulates
historic and realtime trading data in the local data cache and uses provided calculations of trading trends and
indexes—further calculations on the time-series data. Various time zoom scales reveal the phases of past,
historic trends, that emphasize the position current trade execution as a spatially expressive metaphor.
Visualisation and sonification modules utilise several specialised financial TA (Technical Analysis) programming
libraries to dynamically calculate sections in the audio-visual score for the real-time 3D graphics and
sonification engine. The use of consistent spatial arrangements helps to create a sense of order and coherence
in the visualization, revealing flows of market movements, and identify relationships and patterns within the
data.
Audio-visual staging
In the scenes, the progressing complexity or visitor-triggered mode shifts induce a challenge to the
audio-visual sensorium: the experience of the conflation within multiple reference systems and plays with the
visitor’s perception effort of “sense-making.”
Various sound properties, such as pitch, timbre, rhythmic elements are juxtaposed in linear-and non-linear
grids and ephemeral relationships are revealed and emphasized by spatially organized audio-visual cues
(perspective, sharpness/blur manipulation), audio-panning, and timbral modulation, allowing the viewer to easily
distinguish between changing context without having to rely on visually “scanning” the chart. This
can be particularly important in complex or dense scenery, where a well-organized spatial arrangement can help
to reduce cognitive overload.
Data representation axis and colours
Axes, colours, and timeline of time series data such as stock or currency trading are typically represented in
a consistent manner across cultures and regions. However, asset value increase usually mean economic gains,
however, in specific trading mode—“shorting,” that is a bet on asset value loss and the gain
is “reversed.” Else, there are some differences that can be attributed to a combination.
Conclusion
Algorithmically manipulated financial trading is an arena where algorithms merge with human emotions and drives
like greed, FOMO (“fear-of-missing out”), euphoria, confusion, and despair into a global hybrid
sensorium. As a phenomenological effort, this work proposes a set of parameters for simulated anisotropy, useful
for designing the structure, notation, and physical, mathematical dimensions in the VR interface and
environment. The framework for simulated anisotropy instruments is envisioned as a contribution to the emergent
fields of immersive analytics and neuroaesthetics, developing guidelines for a hybrid interaction system,
consisting of networked hardware specifications and custom-developed software modules. This project will
implicate experimental strategies with uncertain capacities in the effectiveness of interaction, and impact of
integrated audio-visual language, expanding the vocabulary of spectromorphology.⁷ Future
intentions for this project, using techniques such as sentiment analysis to assess the emotional tone of user
feedback or comments, or using physiological measures, such as heart rate or galvanic skin response, to assess
the user's emotional response to the data.
References
1 Michael J. Sandel, What Money Can't Buy: The Moral Limits of Markets Farrar, Straus and
Giroux, 2012.
2 Georg Simmel, The Philosophy of Money, 3rd edition by Routledge,
2004.
3 Peter WS Newall, Leeonardo Weiss-Cohen, The Gamblification of Investing: How a New
Generation of Investors Is Being Born to Lose, Int J Environ Res Public Health, Apr 28,
2022.
4 Richard Skarbez, Nicholas F. Polys, J. Todd Ogle, Chris North, Doug A.
Bowman, Immersive Analytics: Theory and Research Agenda. Frontiers in Robotics and AI
6, September 10, 2019.
5 Danko Nikolić, “Ideasthesia and
art,” in Digital Synesthesia. A Model for the Aesthetics of Digital Art, ed. Katharina
Gsöllpointner, et al., Berlin/Boston, De Gruyter, 2016.
6 Harald Obermaier, Kenneth I. Joy, Future challenges for ensemble visualization,
IEEE Computer Graphics and Applications, Volume: 34, Issue: 3, May-June 2014.
7
Manuella Blackburn, The visual sound-shapes of spectromorphology: An illustrative guide to composition,
Organised Sound Volume 16 Issue 1 April 2011 of cultural and historical factors, as well as the specific
requirements and conventions of each culture – in few Asian cultures, axis, and colours for
positive/negative chart values (up/down or green/red) are swapped. The discussed experimental artworks
implement perceived relativity of orientation axes and colour schemes to establish ephemeral cognitive
associations, that are enhanced by artificially synaesthetic bindings.
Bibliography
Philipp Chapkovski, Mariana Khapko, Marius Zoican, Does Gamified Trading Stimulate Risk Taking?,
November 25, 2021.
Swedish House of Finance Research, Paper No, 21-25.
Samuel Chabot, Jonas Braasch, “High-density data sonification of stock market information in an
immersive virtual environment,” The Journal of the Acoustical Society of America, 2017,
p.141.
Marcin Marian Krawczyk, From making visible to hiding. Visual representations of financial markets as
tools of manipulation and active and living agents, Visual Studies, 2021.
Stefan Engeser, Falko Rheinberg, “Flow, performance and moderators of challenge-skill
balance,” Motivation and Emotion, 32(3), 2008, 158–172.
Hong Jun Song, Kirsty Beilharz, “Spatialization and timbre for effective auditory
graphing,” AMTA'07 Proceedings of the 8th WSEAS international conference on Acoustics &
music: theory & applications, 2007, 18-26.
Jiahua Xu, Benjamin Livshits, The Anatomy of a Cryptocurrency Pump-and-Dump Scheme, Proceedings
of the 28th USENIX Security Symposium, 2019, 1609-1625.
Antoinette Rouvroy, The End(s) of Critique: Data Behaviourism versus Due Process, In Privacy, Due
Process and the Computational Turn: The Philosophy of Law Meets the Philosophy of Technology, 2012.
Christian Stiegler, The 360° Gaze - Immersions in Media, Society, and Culture,
2021.
Adrianna Zuanazzi, Uta Noppeney, “Additive and interactive effects of spatial attention and expectation
on perceptual decisions,” Sci Rep 8, 2018, 6732.
Mycorrhizal Materialities Positioning the entanglement of human and machine intelligence
KateGeck
RMIT University Melbourne - Australia kate.geck@rmit.edu.au
Abstract
Machine intelligence is increasingly being used in the world with sometimes dramatic effects on human and
other-than-human lives through its decision-making capacity. Much artificial intelligence (AI) and machine
learning (ML) is built on metaphors that centre extraction, competition and control. These also position AI
itself as a resource to be extracted and controlled, paving a troubling path for speculative futures where AI
may gain emergent or ambiguous levels of sentience. These metaphors are part of a historical trend where humans
place themselves above the other-than-human world, and this has formed the basis of an extractive and one-sided
relationship with that world. In light of this, what new metaphors might we employ to platform the relationships
between human and machine intelligences?
Thinking through mycorrhizae could be a productive way to foreground the entangled, generative nature of
exchange between human and machine intelligences. This paper will briefly explore metaphor in human-computer
interaction (HCI) and AI, before making an offering to think about these things through the material of the
mycorrhiza, a symbiotic site of exchange between plants and fungi. It will then briefly detail a creative
project that has emerged from this mycorrhizal thinking to produce machine imagined textiles and embroideries.
It then concludes with a call to embed relational thinking into future practices between human and machine
intelligences in order to create more equitable and even mutualistic outcomes.
Machine intelligence is increasingly being used in the world with sometimes dramatic effects on human and
other-than-human lives through its decision-making capacity. Much artificial intelligence (AI) and machine
learning (ML) is built on metaphors that centre extraction, competition and control. These also position AI
itself as a resource to be extracted and controlled, paving a troubling path for speculative futures where AI
may gain emergent or ambiguous levels of sentience. These metaphors are part of a historical trend where humans
place themselves above the other-than-human world, and this has formed the basis of an extractive and one-sided
relationship with that world. In light of this, what new metaphors might we employ to platform the relationships
between human and machine intelligences? Thinking through mycorrhizae could be a productive way to foreground
the entangled, generative nature of exchange between human and machine intelligences. This paper will briefly
explore metaphor in human-computer interaction (HCI) and AI, before making an offering to think about these
things through the material of the mycorrhiza, a symbiotic site of exchange between plants and fungi. It will
then briefly detail a creative project that has emerged from this mycorrhizal thinking to produce machine
imagined textiles and embroideries. It then concludes with a call to embed relational thinking into future
practices between human and machine intelligences in order to create more equitable and even mutualistic
outcomes.
Metaphor and HCI
Human Computer Interaction Design (HCI) has a long history of employing metaphor to guide and facilitate
interactions between humans and machines. One of the best known examples is the early work of Xerox Parc with
the desktop metaphor. This located computing for the general population within the realm of business
administration and the organisation of documents in a paperless, digital office. The desktop metaphor therefore
helped us understand how computers could augment our administrative productivity. The metaphor provided a shape
to the world of interaction between humans and computers, helping us locate them in our everyday practices and
providing edges to what we assumed was possible with them. This illustrates how metaphors are "imaginative and
creative... capable of giving us a new understanding of our experience."¹ They act as a shorthand for what
is possible with an idea, situation or in the case of technology, with a system. Artificial Intelligence (AI) is
subject to a number of popular metaphors that centre on abundance, extraction and competition². AI emerges
seemingly uncontrolled in ‘explosions of intelligence’; an abundance of "data is mined"; and nations
are engaged in ‘AI arms races’. While some of these metaphors are ecological in nature -
‘oceans of data’ and the so-called "Seed AI"—these are still one-sided tending towards
extraction and domination. There are few, if any relational metaphors offered for the dualistic nature of
ongoing exchange between human and machine intelligence. We seem to work with uneven metaphors: they enable us
to extract without consequence from seemingly endless and abundant data and they point towards neutral and
efficient control of the world at scale. We know that metaphor does so much work, especially in HCI, to enable
us to think with our technology and imagine our futures. Metaphors help us make sense of the world and help us
understand how to act: they "provide coherent structures, highlighting some things and hiding others."³ So
what might we want to highlight in our relationships with machine intelligence moving forward? Which things do
we wish to stop hiding? Could we work with alternate, ecological metaphors that help us imagine mutualistic
relations between human and machine intelligences?
Mycorrhizal Materiality
Mycorrhizae are a relational phenomenon between plants and fungi. They occur when the hyphae of fungi and the
tips of plant roots enter into a mutualistic relationship where they orchestrate the exchange of nutrients
between them. The relationship is dynamic: connections between plant and fungus will continue to form and
re-form to "ceaselessly remodel themselves."⁴ Susan Simard and her team demonstrated this ongoing
mutualistic transfer of resources between plants and fungi through mycorrhizal networks by tracing the movement
of particular carbon isotopes through the network.⁵ As a result, this network is sometimes referred to as
"the wood wide web". Due to their networked nature, mycorrhizae offer an interesting, mutualistic metaphor for
repositioning the relationships between human and machine intelligence. The mycorrhiza is a lively material, a
"polyphonic assemblage" of relations, chemicals and organisms. This "polyphonic assemblage", in the words of
Anna Tsing, is a dynamic, multispecies space that enables a range of voices to be listened to.⁶
The polyphony of mycorrhizae allows us to listen to the different agents and forces within the assemblage as
independent voices, while still enabling them to collect together at particular moments and be heard as a whole.
In the mycorrhizae, we can track the flows of nutrients between life forms and of energy from the sun to the
unfolding of leaves. We can peer under a microscope and examine the interconnected, entangled materialities of
plant and fungus as hyphae penetrate roots, becoming essentially impossible to disentangle. If we approached the
design and deployment of AI systems through this lens, it would enable us to understand from the outset how
biases become entangled within algorithmic decision making.
There is now significant awareness of the role of bias in algorithmic decision making. Mehrabi et al.
systematically outline an extensive list of the types of bias evident in a range of ML applications.⁷ A
vast selection of ways in which bias can emerge is presented, as a whole demonstrating the entangled and
relational nature of exchange between human and machine intelligence. These examples emerge from 3 domains:
‘data to algorithm bias’ where the data may be interpreted by the algorithm in a biased way;
"algorithm to user bias" where algorithms might influence human behaviour; and "user to data bias" where the
human creators of datasets infuse their own biases.⁸ These 3 domains individually and collectively
demonstrate a mycorrhizal structure: data is like a nutrient, flowing from human to machine, sustaining and
limiting the growth of social imaginaries. It is clear from this study and others that it is in fact impossible
to remove bias completely, and so a conceptual ideological shift towards understanding that human and machine
intelligences are entangled provides a base from which to work in further checks and balances on the
implementation of intelligent decision-making systems.
Creative Exchange
We can apply the concept of the mycorrhizae to creative exchanges between human and machine intelligence. There
are a number of (ever increasing) tools available to collaborate with machine intelligence. Generative machine
learning platforms like Midjourney, Stable Diffusion, and Dall-E use text to image prompts to enable artists to
co-create graphic outcomes with diffusion models. Others such as Playform enable artists to work with their own
datasets and Generative Adversarial Networks (GANs) to generate new images. Naturally, these spark criticism and
concern that machines will usurp the role of artists, or that it is not "real art".⁹ However, if we
reframe the relation between human and machine intelligence from the vantage point of mycorrhizae, we can see
that any creative artefact produced is the effect of an entangled assemblage. The artefact can only emerge
through the creative exchanges between human and machine intelligence: from the wording of the prompts, to the
selection of images to move onto further generation, to the curation and presentation of final images that may
be used. A novel example exists in the art project "Mycorrhizal Materialities" (in Figures 1 - 3). A GAN was
trained on 100 hand drawings of Australian fungi created by myself. The images generated by the GAN were refined
through my own aesthetic choices, shaping the "evolution" of the images. Select images were then transposed
using software into embroidery designs and stitched via machine to produce the final works—a series of 4
hangings 50 cm X 180 cm.
This work is an exploration of a co-creative relationship between human and machine intelligence—using
the mycorrhiza as a metaphor for an entangled and generative creative process. The works in this project
consider ML through a relational, material lens with the textile outcomes providing a metaphorical embodiment of
this lens. The relations between human and machine intelligences are interwoven, producing an agential fabric.
The embroidered works explore material entanglement between different intelligences, overlaying the symbiotic
exchanges between human and machine intelligences with those between plants and fungi. These textiles emerge
solely through ongoing creative exchange between me and the algorithms: my drawings generate machine imagined
responses, and further algorithmic exchange occurs as these images are transposed through software into
embroidery designs and again when the machine embroidered onto the fabric. These textiles represent what Karen
Barad calls an "agential cut": produced through the entangled materialities of human and algorithmic
intelligences.¹⁰ These creative works demonstrate a mycorrhizal materiality: ongoing exchange between
human and machine intelligence enable the artefacts to emerge. There are ongoing flows as ideas are offered and
exchanged in a collaborative relation.
Speculative Entanglements
A mycorrhiza involves intentional exchange of nutrients between plants and fungi. In some instances, the fungal
hyphae penetrate the plant root cells, connecting directly to the plant. Human and machine intelligence can be
seen as engaged in direct exchange of knowledge between the two systems. While we do not presently have direct
corporeal interfacing at any scale between human and machine intelligence, it could be said that the transfer of
knowledge, ideas, possibilities and imaginaries are already in considerable practice between human and machine
intelligence across health, justice, administration and the arts. This is having significant effects on human
life as well as on the other-than-human lives of various plants and animals. Mycorrhizal thinking makes space
for care in speculative futures where machine intelligences might gain the capacity to experience the world.
While likely different to human-like consciousness or sentience, there is the capacity for an AI "experience" to
emerge at some point in the future. Systems of machine intelligence often operate with sensing data, processing
this information and making a decision based on particular conditions. At some point, this feedback loop could
complexify in new ways, possibly giving rise to conditions of reflection, whim, or preference: at some point it
might become possible to "feel like an AI"; for an umwelt or zone of experience to emerge around an AI’s
sensing apparatus. This could be impossible entirely, or it could be rudimentary or it could be complex. It
could echo what little we know about the umwelts of plants and insects or it could emerge from sensing
capacities as strange and diverse as those that presently occur in the world. A mycorrhizal network can be
mutualistic or parasitic: the metaphor enables us to think in a holistic and ecological way about the nature of
the relationship between human and machine intelligences. It allows us to ask interconnected questions of our
systems: how might a system place further and unnecessary demands on the earth, such as the use of AI to
essentially enhance the profitability of the mining sector.¹¹ And in a speculative future where
machine intelligences have experiential capacities, how does true mutualism factor into exchange between human
and machine intelligences? Will we continue to base these relationships on extraction, control and domination?
Or might we move toward more symbiogenetic relations, as described by Haraway where "symbogenesis is not a
synonym for the good, but for becoming-with each other in response-ability."¹² Could we weave futures
where human and machine intelligences emerge as companion species, co-evolving through care?
Conclusion
To conclude, machine intelligence is increasingly used in the world. It has sometimes dramatic effects on human
and other-than-human lives through its decision-making capacity. Many of the metaphors that have brought us to
this point centre on extraction, competition and control.
More relational metaphors—like that of the mycorrhiza—might help us think with these technologies
in a more relational way. The creative project detailed in this paper demonstrates an entangled approach to the
creation of new work that remains cognisant of its interconnection between human and machine intelligence. Human
and machine intelligence will always be highly entwined—entangled with one another in an exchange of ideas
and potentialities, just like a mycorrhiza. If we could embed this relational understanding into our practices
with machine learning systems we might be able to better direct their biases and end purposes towards
mutualistic and equitable outcomes.
References
1 George Lakoff, Mark Johnson. Metaphors We Live By, Chicago, Ill, University of
Chicago Press, 1980, 139.
2 Kate Geck, Knitting Algorithmic Assemblages, TEXTILE, November 14, 2022,
1–18, https://doi.org/10.1080/14759756.2022.2114240.
3 George Lakoff, Mark Johnson. Metaphors We Live By, Chicago, Ill, University of
Chicago Press, 1980, 139.
4 Merlin Sheldrake, Entangled Life:How Fungi Make Our Worlds, Change Our Minds and Shape Our
Futures, S.L., The Bodley Head Ltd, 2020, 42.
5 Suzanne W. Simard, David A. Perry, Melanie D. Jones, David Myrold, Daniel M. Durall, Randy
Molina, “Net Transfer of Carbon between Ectomycorrhizal Tree Species in the Field.” Nature
388, no. 6642, August 1997, 579–82. https://doi.org10.1038/41557.
6 Anna Lowenhaupt, Tsing The Mushroom at the End of the World: On the Possibility of Life
in Capitalist Ruins, Princeton, Princeton University Press, 2015, 24.
7 Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan, “A
Survey on Bias and Fairness in Machine Learning.” ACM Computing Surveys 54, no. 6, July
2021, 1–35. https://doi.org/10.1145/3457607.
8 ibid
9 Kevin Roose, “An A.I.-Generated Picture Won an Art Prize. Artists Aren’t Happy",
The New York Times, September 2, 2022,
https://www.nytimes.com/2022/09/02/technology/ai-artificial-intelligence-artists.html.
10 Karen Barad, Meeting the Universe Halfway [Electronic Resource], Durham, N.C.,
Chesham, Duke University Press, 2007.
11 KoBold Metals, “Applying Big Data & Superior Science to Exploration,” n.d.
https://www.koboldmetals.com/.
12 Donna J. Haraway, Staying with the Trouble: Making Kin in the Cthulucene, Durham,
N.C., Chesham, Duke University Press, 2016, 102.
Interspecies Communication — Water bodies
AndreaGogova
ArtSci and Technology, Envi_Art Lab. Department of Environmental Ecology and Landscape
Management Faculty of Natural Sciences, Comenius University in Bratislava Bratislava,
Slovakia andrea.gogova@uniba.sk, andgogo@hotmail.sk
Abstract
In planet Earth, most processes are based on water. Water regulates climate, morphologically influence a
landscape, is a medium of living processes. It is a medium of interaction of organism and mineral parts in
microscopic view, between whole organisms and minerals in macroscale on the Earth, and endless interplanetary
space. Water topic (more than "parasitically" utilization of water resources (Serres, 2007)), in the ArtSci
project is focusing on its communicative possibilities. Equal communicative principal possibilities for all
living—human and other than human bodies—is a metaphor of the epistemological problem of water
protection as a medium of life, biotopes, and ecosystems. Because we are connected through water, we are all
bodies of water. Understanding water through environmental analysis of the territories, philosophy, and fluid
mechanics (chaos-based fluid attractors principle) will bring communicative equality for all bodies of water and
the inevitability of environmental protection. When communication cannot be fluid, then each of us as bodies of
water, will be lost in time-space. The new interdisciplinary methodology of communicative artwork is based on an
ArtSci manifestation of messages mediated in water as an asemic writing or the other kind of communicative
interrelation between human and more than human. The basis is the phenomenological research of water as a medium
of communication which causes a feeling and a togetherness. We and other bodies of water could feel our
togetherness through water.
Keywords
interface, pattern of communication, sympoiesis, structural coupling, water
The problem is approached as artsci research using knowledge of biosemiology and philosophy, environmental
ecology, and landscape analysis to develop a specific interdisciplinary artwork which will support environmental
education.
Water body figuration is related to possibility mediate
interspecies communication to better understanding waterbased life on the planet.¹Water is an
essence and principle of form, which is actualized in the water body figuration. According to principle of
‘structural coupling’ patterned cognition and communication of each communicative actor is related
to their environment. If water is essence of inner and outer environment of actors of communication and water
body figures, their fluid principle, relates to their agency, then relates to the concept of agential cut, and
in differentiation could appear in the meaning which would be presented by the ArtSci work.
In this paper I come out from my last research of transient pattern—visual model of transient patterned
of digital work communication which was based on relation of human and machine intelligent agency (Author,
2021). The biosemiotics view of patterned cognition and communication comes from Maturana and Varela’s
“Structural coupling” theory, in which the recurrent interaction of unit (organism) and medium
(environment) causes structural coupling, which is reflected in a patterned cognition and communication of
autopoietic complex organization (Maturana and Varela, 1972) and Jakob von Uexküll's umwelt theory.
Rereading Darwin`s theory of natural selection Elizabeth Grosz describe it as a complex of parallel randomly
interfere processes of internal dynamism of living beings and assertion of external forces and influences
(Grosz, 2004). Stafford Beer tried to explain Maturana and Varela’s autonomy of autopoiesis that
“...autopoietic systems are environmentally open to material-energetic fluxes or semiotic
mediations” (Beer, 1980) Donna Haraway instead proposes principle of sympoiesis, as enabled to incorporate
a complex interrelation of system as such (Haraway, 2016). The decisive principle is to recognize water as a
communication medium according to the possibilities of structural coupling, but instead of autopoiesis I related
it to the principle to the sympoietic system where the notion of evolutionary changes is crucial hack by
symbiotic transfer, lateral mixing, crossspecies contamination, and viroid life (Neimanis, 2017) Communication
of that holobiontic water based systems then depends on intersection of agential cuts of each body of water of
their unpredictable transient pattern of communication and cognition; according to Neimanis, in common space of
water body (medium) of interconnected ‘bodies of water. My questions are: Could meanings or
feelings emerge in the interrelation of water body figure (water which is an inseparable part of organism) and
water as medium? If yes, then interspecies communication will be interaction of each unpredictable
pattern or umwelt through dialogic membrane—“which is water body medium”, in the semiosphere.
The state of art
Several authors proposed the importance of water as a medium of interconnectivity between organisms (or
minerals) together (as physiological and biochemical processes. Some botanical research describes the
potentiality of mutual interspecies communication mediated by water. For example, the hydraulically lifted
transfer of mycorrhizal network provides a potential pathway between plants based on water (Egerton-Warburton,
Querejeta, Allen, 2007). Water was always a part of the formation of Earth`s mantle, but also originated from
comets and asteroids, that break up into meteorites which fall on the Earth (Crockett, 2015; Hartloch, 2011).
The fact can push the possibility of interspecies communication to the interplanetary space. Water as physical
matter is shaped form the solid, liquid and gas materiality. Fluidity is not only one state of water matter, but
mechanical, formal aesthetical principal of artistic, designer, and architectural practises. The fluidity is
recognised in the relation to Gilles Deleuze and Bernard Cache theory of Objectile1 , which based a theoretical
background to digital artistic form or artwork body figuration. Then water body figures appear in the
differentiations in the repetition processes of multiplicity of water cycles and niches, similarly, as appears
meaning of sign (Derrida, 1967— différance; Deleuze, 1994; Neimanis, 2017). According to Astrida
Neimanis (2017) in differentiation are continuously unfolding embodiments as an expression of eternal return of
the self-same. Water engendering difference “was” an expression of water that “is”, and
its potential “yet-to-come ”. Bodies of water as figuration was already describe within ecofeminism
and anticolonial thinking (Neimanis, 2017, Gaard, 2003, Armstrong, 2006). Lucy Irigaray (1992) related fluidity
to the feminine body as “fluid and ever mobile”, and “secreting a flow.” Fluidity is
diffuse and multiple, overlapping, and interconnected, is repetition in hydro/bio cycles and water figures
acknowledged in differentiation. The dynamic fluid principle relates to the chaos theory, where changes are
based on small intrusion of triggering power, which are resulted in the system changes. The process is based on
perpetual intra-action, entanglement, diffraction, and agential cuts which was described by Karen Barad
“matters agential realism” (Barad, 2007). Then in differentiation is appearing meaning, also any act
of observation is differencing agency and makes a “cut” between what is included and excluded. Every
species has specific, but transient "pattern" of differencing agencies. From the position of a
posthuman approach, then it is possible to see water as a common equal medium of communication between all
bodies of water (human, more-than-human, extra-terrestrial), which can bring water based informational
agency.
Interspecies communication as ArtSci project
The most art research which relates to interspecies communication was based on translation or transcription of
many levels mediated communication principle. Eduardo Kac in his bio art transgenic artwork
“Genesis” (2000) resolves the problem of interspecies communication according to speculative
transcription to human language of Morse code (see https://www.ekac.org/geninfo2.html). The other bio artist
Špela Petrič in her work ‘Institute for Inconspicuous Languages: Reading
Lips’(Petrič, 2018) relates the transcription of water-based physiological interaction of plant to
visual transcript of human sign language (See
https://www.spelapetric.org/#/institute-for-inconspicuous-languages/).... etc., Both of work was related to a
human existing language. Instead of relate communication to human language I propose water body as a common and
direct medium of emotional, also interspecies communication of water body figures. Then understanding appearing
in the differential intersection of agency of each communicative body of water and through a repetition will
come to common emotional “hydro language”.
In the research I focus on water in the role of the medium of emotional communication between all bodies of
water human/more/than human, (extra-terrestrial), both in micro and macro scale. Then cognitive pattern of each
communicative bodies of water relates to its specific agency. The agency relates to agency of water body figures
which are constituting new and reconfiguring systems in the concept of “matters agential realism”
(Barad, 2007), to agents of water-based communication. This principle of shaping the matter is the process of
performative bodymaterial practice which would configure figures and its transient patterns of communication to
generate the vision of the artwork.
Procedural model in relation to interspecies water-based communication
Because water in bodies is flowing in perpetual cycles of water niches, it is related to the procedural model
of communication which was described by Philippe Bootz², but also water is an essence of the artwork.
The essence is related to the water body essence. Then the artwork water body is defined as an
individual water body. The figure of the individual water body relates to agency of interaction (communication).
According to Philippe Bootz`s model of visualization of procedural digital works is work individual body defined
by essence, extensive parts and relations linked essence and extensive parts. Extensive parts are composed by
physical real parts linked with essence, which are managed by agency. Extensive parts compose wider vision of
artwork body³. The essence of artwork is recognized as individual agent which is related to the soul and
the mind (Bootz; Laitano, 2013), they are embodied in sensitive bodies. My body (and the other bodies biological
and technological bodies of water) is a sensitive apparatus putting me in principles of global intuition of
creating transcorporeal matter in the life of “water body languages.” The
performative-material practice is based on the gathering and applying data of extensive parts of artwork in
the context of emotionally build interactive intermedia artwork. Fluid mechanics agents, as AI models, are a
managing agency that introduce characteristics of the water-based world of artwork. The process
relates to extensive parts of all bodies of water which relates to the manifestation and generate emotional
actualization of artwork water body. The reading artwork essence will be put in the middle of
analytical—informative and intuitive feeling. Although flow dynamics modelling—are used to predict
hydrological behaviour, it is not possible to see what will be if human destroys a tiny balance of water flow.
Through the project they are emotionally indrawn in the artistic message: 'We, as bodies of water, can interact
in the water body holobion, we are all equal in the emotional based communication possibility and feeling our
common human and more than human water-based life.
The experimental theoretical and practical transdisciplinary ArtSci research will be build according to the
proposal of Manifesto of Artistic Research⁴ (Henke, Merch, and col, 2020) (farther
“Manifesto”) The goal of the project is to introduce the audience to data- manifestation through
artistic emotional imagination, away from pure data vizualization, to take audience closer to feel water-based
life of equal interaction-communication with other bodies of water. The emotion relates to water-medium
communication of interrelated water bodies. Applying a more intuitive concept could bring emotional
understanding to each body of water and to engage audience to the environmental action.
The problem water crisis is often resolved from an anthropogenic point of view which leads to the other complex
of global environmental problems. By implementation of symbioses, neutral or parasitical relations of sympoietic
system only we will be able understand how systems communicate in the water body medium interface. Using the
analytical natural sciences approaches and human intuition and sensitive feeling will lead to the critical
posthuman ArtSci overview, where chosen agencies could metaphorically relate to cognition and behaviour of
possible communicative process.
The generative artistic approach based on data and performative emotional interaction lead to specific
experimental aesthetical artsci solution and its realization in the artwork. Only clear, unpolluted water is a
medium of understanding water based live. The approach based on the relation of water body figuration to the
agency of unstable patterned interspecies (also interplanetary) communication. According to Deleuze, Haraway,
Neimanis, and other authors the water body figuration is fluid and actual as a characterization of a real;
fluidity, both virtual and actual, can be real aspect of water-based communication. I can see Water as a medium
for bearing the information of life also interspecies communication. Water is matter which enables information
flow from each water-based body to others alongside the common water network. Water is “informable
interactive matter”.
Conclusion
Water is an essence and by water figures relates to the individual water body agency. The interrelation of
water body agency and agency of patterned cognition and communication is possible to find
“languages” of water-based communication. The essence of water based interspecies communication
artwork relates to the water`s essence by its extensive parts and by relations are linked to the agency of water
body figuration. Agency of figures of the water body are related to the agency of interspecies
interaction/communication. The agency comes from the principle of "structural coupling", the principle of the
sympoetic system and the notion of water. Interspecies communication was related to the water interface as a
common medium for all living beings. Water is recognized as an interface, a part of bodies and as an organising
agency of communicative form. Then water is the essence of the interface of communication and figuration of the
water body, and its action is related to the pattern of waterbased communication. The approach leads to
manifestation of possible interspecies communication recorded by the ArtSci project. The understanding of
artwork is related to the audience reading and will move from an analytical to emotional approach. The goal is
to lead the audience closer to an emotional level of water-based communication and go away from the
informational position of data.
The project is tracing possibility how we can understand to water related life on the planet and possible
interspecies communication. To take audience interest of water problems, I tried to ideologically connect them
to the communication with other water bodies. In this context it means more understand, more connected, care,
and to have better environmental justice to all water bodies. The research leads to the importance of intuition
and serendipity in ArtSci's holistic approach to communicating water-based equanimity. This approach would
instill in the audience (professional and general) a better understanding of water not only as a resource, but
also as the inevitability of interconnected life and the necessity of equal access to unpolluted water, to
quality habitats, to a quality and healthy environment for all.
References
1 The first essay presented in this chapter, by Stephen Perrella, is a précis of
end-of-millennium design theory: the Objectile is an open-ended notation which allows for infinite
parametric variations; these can be directly fabricated using file-to- factory technologies, thus enabling
the serial reproduction of non-identical parts, where ranges of limited variations can be mass produced at
no extra cost (Mario Carpo, Stephen Perrella,Bernard Cache 27 March 2013
https://doi.org/10.1002/9781118795811.ch10 in Topological Architecture (1998–2003 ). “Objectile is
an object which is not yet defined by its essential form, but becomes the pure functionality “...
(Deleuze, 1988). and open it to artistic research of digital form. Fluidity as a principle of shaping
architectural form was followed in nonrational geometries of interrelated architectural space and environment
and the other circumstances or parameters (Last,2015).
2 See procedural model of communication for example online
https://mediarep.org/handle/doc/18625
3Understanding to essence is put by Bootz
somewhere in the middle of intuition affected by the reading body of work and analytical approach to body of
work. If the audience is closer to the analytical reading of work they are farther away from emotional
feeling and vice versa (Bootz, 2010)
4 As was declare in Manifesro of artistic research Artistic research is based on more
intuitive and interactive approaches—‘in the form of leaps, digressions, and detours which
continually generate new and unexpected counter/expressions’, ...and trigger irritations and daring
revelations. (Henke, Mersch, and col., 2020)
Bibliography
V. R. Baker, Debates—Hypothesis testing in hydrology: Pursuing certainty versus
pursuinguberty, Water Resour, Res.,53, 2017, 1770– 1778, doi:10.1002/2016WR020078.
K. Barad, Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning,
Durham, Duke University Press, 2007.
K. Barad, "Living in a Posthumanist Materialist World: Lessons from Schroedinger’s Cat", in A. Smelik
and N. Lykke (eds), Bits of Life: Feminism at the Intersections of Media, Bioscience and Technology, 2008,
165-176.
S Beer, University of Washington Press, Bootz, Seattle, 1980.
R. Maturana Humberto, Varela Francisco J., Preface to “Autopoiesis: The Organization of the
Living”, in Autopoiesis and Cognition, The Realization of the Living, Dordrecht, Holland/Boston, D.
Reidel Publishing, 1980.
P. Bootz, M. I. Laitano, Crossreading New Media, Online ELO conference 2013, videné Feb., 2019,
https://doi.org/10.7273/2G71-Q352.
Philippe Bootz, "The Problematic Of Form Transitoire Observable, A Laboratory For Emergent Program med Art",
videné Dec. 2018, Online: www.dichtung-digital.com/2005/1/Bootz.
Philippe Bootz, Sandy Baldwin, Regards Croisés, Perspectives on Digital Literature, West Virginia
University Press, 2010.
R. Braidotti, The Posthuman, Cambridge: Polity Press.Deleuze G, Difference et repetition, Press
Universitaires de France, Paris, 2013.
G. Deleuze, Logika smyslu Karolinum, Praha (original, 1969) pursuinguberty, Water Resour. Res.,53,
2013, 1770–1778, doi:10.1002/2016WR020078.
G. Deleuze & F. Guattari, (1980), Tisíc plošín. Herrmann & synové,
preklad, Praha, 2010.
G. Deleuze, (2016), Spinoza, Praktická filozofie, Óikumené, Praha Drohan Ch.r M,
Atropos press, Deleuze and the Sign, 2009, ISBN-10 : 0981997201
L. Guertin, 2016, online, https://www.paesta.psu.edu/podcast/did-
earths-water-come-outer-space-paesta-podcast-series-episode-17
E. Grosz, The Nick of Time, Durham, Duke University Press, 2004.
monahedayati@yahoo.com Belgium Concordia University, University of Antwerp
Abstract
This paper is an effort to examine the codes of interaction between the carbon-based and the silicon-based,
i.e., the human and the machine, notably the shifting agencies addressed by adopting feminist technoscientific
and new materialist lenses to grapple with the techno-industrial paradigm shift that has been (dis)figuring the
anthropocentric condition. The first part of the paper lays down the qualities of this emerging ecology while
recognizing the importance of human accountability and situatedness. The focal point of this survey is the
anthropologist Lucy Suchman’s classic Human-Machine Reconfigurations which is elaborated upon
through anchor points she posits revisiting Donna Haraway and Karan Barad’s arguments. The last part
engages with the implications of such a coupling for human and machine sensoria in order to envisage the
qualities of a distributive sensorium that this regenerative agency can put forth while alluding to practices of
situated computing.
If another techno-industrial paradigm shift has indeed creeped up on us so as to perfectly confuse the
human-machine boundaries this time around; and that an ontological shift has occurred, as Rosi Braidotti claims,
troubling the contact zones “between the organic and the inorganic, the born and the manufactured, flesh
and metal, electronic circuits and organic nervous systems”; and that the carbon-based and the
silicon-based as a result constantly imbricate, move, and flux inexorably, how can we imagine the agencies
constructed around such a flow? What are the implications of such an entanglement for human-machine
sensoria?¹
This co-evolutionary moment through which human and machine complex systems constantly affect and are affected
by one another’s interrelation to create an unfolding terrain of imbricated becoming is referred to as
technogenesis by N. Katherine Hayles. The implications of this adaptable and generative interrelation
are profound: not only we, as humans invested in the holiness of humanism, have to grapple with constant
adaptation of culturally coded networks of human life, but also with the psychobiological shifts that have
occurred notably in rewiring of intricate neuronal activities of the human brain.²
Detached from the preformulated subject-object binary, the anthropologist Lucy Suchman’s classic
Human-Machine Reconfigurations offers a profound angle on theorizing the conditions of this
human-machine distributive agency. To delve into reconfiguration as an emergent ecology where agencies are
constantly made, unmade and remade, at various points Suchman draws on the scholarship of Donna Haraway as well
as Karen Barad, notably through the notions of figuration and intra-action.
Haraway’s Figuration: A Situated Construct
Through the concept of figuration Haraway intends to foreground the tropic quality of material-semiotic
practices in technoscience that hover in a space of literal-figurativeness. She envisages technologies as
materialized figuration; that is assemblages that are both concerned with meaning making, a figural act, and
physical and hence tangible existence.³
As a lens that particularly zooms in on human-machine interrelation, figuration is a critical framework that
questions the formulation and configuration of technoscientific practices at every instance of occurrence. The
goal is to sidestep fixed universalized paradigms of ‘doing science and technology’ and aim for
specificity of local practices that humans actively shape rather than act as a passive observer within.
According to Suchman, the act of figuration is informed by specific socio-cultural constructs arising from
site-specificity which can reinscribe or challenge the status quo and question the Euro-American imaginaries
built on rationality of the autonomous subject.⁴
Barad’s Intra-Action: Entities in the Making
Barad’s notion of intra-action is based upon their theory of agential realism as an onto-epistemology
that challenges individualist paradigms and insists that intra-acting agencies are always already
inseparable.⁵
While during an interaction two preformulated entities come together for an exchange, an intra-action
underscores how the subjecthood and objecthood gets formed through the encounter. Barad specifically considers
technoscientific practices to be a common site of intra-action where we should recognize the act of boundary
making, objectification, and subjectification as contingent constructs. Barad’s vision is markedly in line
with Haraway’s material-semiotic that considers material constructs and the meaning arising from them as
co-constitutive. They too consider the reality of human-machine boundaries to be cut in particular ways that
follow certain historicity with socio-political consequences.
Suchman’s Reconfiguration
Drawing on the notion of figuration, configuration and intra-action, Suchman proposes reconfiguration as a
creative exploration of human-machine boundaries where what she refers to as “the distributed and enacted
character of agency” as a constantly-regenerative phenomenon should be taken into account.⁶
This view sits against the Western-dominant vision of subjects and objects as fixed entities brought together
to interact and instead points towards a kind of performativity within the encounter where the agents are in
continuous formation, reproduction and transformation; a perspective rooted in Actor Network theory (ANT) as a
social theory based on relational ontology that puts humans and nonhumans alike as actants in an
ever-evolving interrelated network where there are no preconceived positions taken but the positions are rather
assumed through the process of interrelation. In this process the agencies are constantly worked through and
negotiated to actively constitute ontologies based on what Michel Callon (2007), one of the proponents of ANT,
calls “morphology of the relations” through which cyborgs, hybrids and quasi-objects are constructed
and made visible.7(1)
Practices of technology-mediated medicine, including reproductive technoscience, as well as human-computer
interaction are those that Suchman pays particular attention to as sites of human-machine mutually-constituted
agencies. In this space of intra-action the disconcerting fact is that within practices of science and
technology, the technical is formulated in the center while the social is either non-existent or pushed to the
margins. Here, Suchman walks a tightrope of reconceptualizing the human in a way that the inseparability from
the socio-technical substrate is pushed to the fore while recognizing the prominence of accountability but
without assuming the dominion associated with ‘pure’ humanism that views technologies as translators
and assemblers in service of humans. In other words, the question is how to draw a human-machine intra-action
that retains human accountability without telling an essentialist story.
As a feminist construct, the figure of the cyborg, taken across its regenerative stance, can offer one avenue
to explore this notion by radicalizing the human-machine, male-female, and subject-object boundaries, towards an
emergent ecology where socio-materiality is constantly made, unmade and remade. From the Harawayan
goddess-turned-cyborg to the elegant hero/ine and saviour as cultural imaginary and further as an everyday
socio-materiality without a singular body, the figure of the cyborg is omnipresent across the socio-technical
substrate.⁹
By overstepping the isolated shell that contains the human-machine hybrid, in Suchman’s reading, cyborg
not only shatters the glamorized singular figure but “dissolves into a field of complex sociomaterial
assemblages” to open up new ways of theorizing and practicing such an entanglement.¹⁰
Braidotti, takes this destabilization one step further to put forth the figure of the deglamorized everyday
cyborg as “anonymous masses of the underpaid, digital proletariat who fuel the technology-driven global
economy without ever accessing it themselves.”¹¹
Intrinsically, we can see how at every moment of instantiation cyborg cuts the human-machine boundary at a
certain angle and not the other to constitute a shapeshifting intertwinement of the carbon-based and the
silicon-based capable of subverting human-made socio-politico-cultural constructs. This fluid ecology is meant
to transcend deeply etched preconceived notions of intelligent machines as human techno-extensions or the
sensing and sensible human weary of the techno-dystopia, to instead reconceptualize an entanglement among
networked agents, that constitute leaky, generative boundaries.
The stability of the human agency is thereby compromised as according to Suchman “the person figured here
is not an autonomous, rational actor but an unfolding, shifting biography of culturally and materially specific
experiences, relations, and possibilities inflected by each next encounter-including the most normative and
familiar—in uniquely particular ways.”¹²
Within these spaces of constant transformation of the boundaries and redrawing the agencies, the question of
accountability is by no means diffused: we need to recognize that we draw boundaries for meaning making and
these boundaries are always charged with human-centered conceptions and misconceptions that have repercussions.
This accountability is to recognize our position within animation and reanimation of situated encounters. As
Barad states: “we are responsible for the world in which we live, not because it is an arbitrary
construction of our choosing, but because it is sedimented out of particular practices that we have a role in
shaping.”¹³
(Re)(Con)Figuring Hybrids of Sense-Making
During the act of boundary making the question that arises is how can we cut the boundaries in ways that give
rise to hybrids of sense making where intelligibilities and sensibilities are constantly figured, configured and
reconfigured? Reconfiguration in this sense can be read as a possibility to negotiate sensory modalities to
locate sensing and effecting (2) not as autonomous qualities associated with
the human or the machine but traceable within the process of intra-action. If technical practices foreground
machine agency and yet human sensoria and sensibility cannot be reduced to compressed temporal flow of machinic
computation, we need to reckon with contingent encounters that go beyond biological and technological
determinism. Bio-sensibility and machine intelligence in this sense can be deconstructed and diffused to be
reconfigured as dynamic fragments of a curious intelligent sensibility adjustable towards acting in
contingent and context-specific situations.
Such a hybrid, quasi-sensoria made in between the human and the machine offers a heterogeneous
socio-materiality with qualities that are no longer inherent but always negotiated and in formation. This
ontological in-betweenness tends to question universalized presumptions about technical practices centered
around the aptitude of the machine intelligence to offer reliable, definitive, and objective responses to
complex questions that are always rooted in specific situations.
In this sense as Haraway notes the answer does not lie in the dichotomous poles of positivism with its hallmark
of scientific objectivity or relativism with its absolute unfixed orientation but in localized and embodied
partial perspectives.¹⁴
Such a vision relies on the symbiosis of the technical and the sensible to form an interdependent intelligent
sensibility that takes shape within the act of becoming to reckon with procedural, socio-political, and cultural
dimensions of local conditions to compose and assess situated work-flows and responses.
(1) While Suchman praises the idea of “generalized symmetry” proposed by ANT, she
subsequently proposed “dissymmetry” as a framework that recognizes the human-machine
differences. Others including Bowker and Leigh Star, put forth a number of critiques pointed towards
ANT’s networked interrelations. They consider the larger social construct to question the equalizing
effect of such a framework toward human/nonhuman actants and the ethico-political repercussions of a world
operating on this logic, pointing out that ANT "can be read as an uncritical celebration of
the power of modern science and technology.” ⁸ Elsewhere,
others such as Mel Chen, Zakiyyah Jackson, and Tiffany King, among others contend that posthumanist
theories in general discount the human discrimination factors at play due to micro and material nature
of such inquires.
(2) Sensing and effecting are mechanisms of interaction within biological and technical
organisms; while sensors receive information from the environment to relay to the system, effectors act
upon the world based on feedback loops that occur between the two. These notions are derived from
cybernetics, a field of inquiry that studies the principles of communication and control within regulatory
systems.
References
1 Rosi Braidotti, The Posthuman, Cambridge, UK, Malden, MA, USA, Polity Press, 2013,
p.89.
2 N. Katherine Hayles, How We Think: Digital Media and Contemporary Technogenesis,
Chicago, London, The University of Chicago Press, 2012.
3 Donna Jeanne Haraway, Mdest_Witness @Second_Millennium. Female-Man_Meets_OncoMouse: Feminism
and Technoscience, Second edition, New York, NY, Routledge, Taylor & Francis Group, 2018.
4 Lucille Alice Suchman, Human-Machine Reconfigurations: Plans and Situated Actions, 2nd
ed, Cambridge, New York, Cambridge University Press, 2007.
5 Karen Michelle Barad, “Posthumanist Performativity: Toward an Understanding of How Matter
Comes to Matter,” Signs: Journal of Women in Culture and Society 28, no. 3, March
2003, 801–31, https://doi.org/10.1086/345321.
6 Suchman, Human-Machine Reconfigurations, 260.
7 Michel Callon, “Actor-Network Theory, the Market Test,” in Technoscience: The
Politics of Interventions, ed. Kristin Asdal, Brita Brenna, Ingunn Moser, Oslo?, Unipub, 2007.
8 Geoffrey C. Bowker, Susan Leigh Star, “How things (actor-net) work: Classification, magic
and the ubiquity of standards,” 1996, https://www.ics.uci.edu/~gbowker/actnet.html.
9 Donna Jeanne Haraway, “A Manifesto for Cyborgs: Science, Technology, and Socialist
Feminism in the 1980s,” in The Post-modern Turn: New Perspectives on Social Theory, ed. Steven
Seidman, Cambridge, New York, Cambridge University Press, 1994.
10 Suchman, 283.
11 Braidotti, The Posthuman, 90.
12 Suchman, 281.
13 Karen Michelle Barad, Meeting the Universe Halfway: Quantum Physics and the Entanglement of
Matter and Meaning (Durham: Duke University Press, 2007), 105.
14 Donna Jeanne Haraway, “Situated Knowledges: The Science Question in Feminism and the
Privilege of Partial Perspective,” Feminist Studies 14, no. 3, 1988, 575,
https://doi.org/10.2307/3178066.
Bibliography
Marie-Luise Angerer, NONCONSCIOUS: On the Affective Synching of Mind and Machine, S.l.:
MESON PRESS EG, 2022.
Karen Michelle, Barad, Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter
and Meaning, Durham, Duke University Press, 2007.
Karen Michelle Barad, “Posthumanist Performativity: Toward an Understanding of How Matter Comes to
Matter”, Signs: Journal of Women in Culture and Society 28, no. 3, March 2003, 801– 31,
https://doi.org/10.1086/345321.
Geoffrey C Bowker, Leigh Star Susan, “How things (actor-net) work: Classification, magic and the
ubiquity of standards,” 1996, https://www.ics.uci.edu/~gbowker/actnet.html
Rosi Braidotti, The Posthuman, Cambridge, UK, Malden, MA, USA, Polity Press, 2013.
Michel Callon, “Actor-Network Theory, the Market Test.” In Technoscience: The Politics of
Interventions, edited by Kristin Asdal, Brita Brenna, Ingunn Moser. Oslo? Unipub, 2007.
Mel Y Chen, Animacies: Biopolitics, Racial Mattering, and Queer Affect, Perverse Modernities,
Durham, NC, Duke University Press, 2012.
Paul Dourish, Where the Action Is: The Foundations of Embodied Interaction, Cambridge, Mass,
MIT Press, 2001.
Donna Jeanne Haraway, Modest_Witness@Second_Millennium. FemaleMan_Meets_OncoMouse: Feminism and
Technoscience. Second edition. New York, NY, Routledge, Taylor & Francis Group, 2018.
Donna Jeanne Haraway, “A Manifesto for Cyborgs: Science, Technology, and Socialist Feminism in the
1980s”, In The Post-modern Turn: New Perspectives on Social Theory, edited by Steven Seidman,
Cambridge, New York: Cambridge University Press, 1994.
Donna Haraway, “Situated Knowledges: The Science Question in Feminism and the Privilege of Partial
Perspective", Feminist Studies 14, no. 3 1988, p.575, https://doi.org/10.2307/3178066.
N. Katherine Hayles, Unthought: The Power of the Cognitive Nonconscious, Chicago, London: The
University of Chicago Press, 2017.
N. Katherine Hayles, How We Think: Digital Media and Contemporary Technogenesis, Chicago, London:
The University of Chicago Press, 2012.
N. Katherine Hayles, How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and
Informatics, Chicago, Ill: University of Chicago Press, 1999.
Zakiyyah Iman Jackson, “Animal: New Directions in the Theorization of Race and Posthumanism”,
Feminist Studies 39, no. 3, 2013, 669–85, https://doi.org/10.1353/fem.2013.0024.
Tiffany Lethabo King, “Humans Involved: Lurking in the Lines of Posthumanist Flight”,
Critical Ethnic Studies 3, no. 1, 2017, 162–185,
https://doi.org/10.5749/jcritethnstud.3.1.0162.
Bruno Latour, Reassembling the Social: An Introduction to Actor-Network-Theory, Clarendon Lectures
in Management Studies, Oxford, New York, Oxford University Press, 2005.
Andrew Pickering, The Cybernetic Brain: Sketches of Another Future, Chicago, London, University of
Chicago Press, 2010.
Lucille Alice Suchman, “Agencies in Technology Design: Feminist Reconfigurations*, ”In
Machine Ethics and Robot Ethics, by Wendell Wallach, Peter Asaro, edited by Wendell Wallach and Peter
Asaro, 1st ed. Routledge, 2020, 361–75, https://doi.org/10.4324/9781003074991-32.
Lucille Alice Suchman, Human-Machine Reconfigurations: Plans and Situated Actions, 2nd ed.
Cambridge, New York, Cambridge University Press, 2007.
DAOs A blockchain-based application not intervening, but strengthening the agility of contemporary arts
VictoriaHilsberg
vic.hilsberg@hotmail.de
Abstract
This paper discusses the blockchain-based application DAO (Decentralized Autonomous Organization) as a relevant
new mechanism of artistic practice in the contemporary arts. It not only confronts the complexity of the modern
society through its comprehensive application potential. It also channels segments of the contemporary art
discourse such as political qualities, participatory art and cybernetic forces through active use cases. DAOs
enable the interconnection of culture, technology and ecology. Using the most recent publications regarding
blockchain technology as well as in-depth research on current artistic movements, it is possible to observe a
complex manifestation of the matter within the contemporary arts during the concept's comparatively brief
existence since its technical implementation around 2016.
Keywords
Contemporary art, transdisciplinary discourse, blockchain, DAO, Crypto Art, political art history, technology,
participation, ecology, economics, Digital Art.
DAOs, Decentralized Autonomous Organization, are blockchain-based applications. They represent a community or
network of potentially anonymous individuals and/ or institutions, which are connected through a common factor.
Each DAO is collectively owned through a network and does not have a central controlling authority.
¹Governance, coordination and actions take place within defined, self-governing rules, which are
collectively decided upon and in its basics laid out in the White Paper, which is accepted by joining the DAO
network. There is the possibility of implementing a DAO-own cryptocurrency, which can function as a treasury.
One option to govern the organization is the buy-in: Through obtaining DAO-own tokens, a fractionalized
ownership of the DAO is held. In theory, owning all tokens represents ownership of the entity of the DAO. There
must not be this structure in order to provide a functioning DAO as there a several differing technological
implementations already in place.²
Examples of DAOs would be exhibited 2022 alone in renowned exhibition spaces such as Kunstwerke Berlin or
Documenta 15 in Kassel. Why would there be the interest to exhibit this technical development that is so close
to Cryptocurrencies in their early stages of the Hype Cycle?³
Because the one challenge that contemporary art is confronted with is to develop new techniques and therefore a
complexity which matches the one society itself is confronted with. DAOs are opening up different options.
One aspect making DAOs relevant to contemporary art is to define and demonstrate the importance of structure as
a political element. DAOs provide a possibility of testing out agile, seemingly utopian systems and implementing
them long term. One example is the Black Swan DAO, founded by Penny Rafferty, Laura Lotti, Calum Bowden and
others. It caters goals, which were already formulated by art activism groups such as the Art Workers Coalition
in New York in the 1960s. Their intention was to build a platform for the representation of all art sector
associated employees, including artists, and demanded their equal treatment and co-determination.
⁴However, through the missing long-term structure it dissolved over time. The approach of Black Swan DAO
towards equal treatment is built upon blockchain-specific protocol mechanisms working towards a trusted network
within contemporary arts. One of their actions was to act as a platform for democratized cultural funding
processes. Equal treatment symbolizes for Black Swan DAO also equal action rights. Pressing de-hierarchization,
the DAO members decide through a democratic voting mechanism who receives the funding. The decision process does
not focus on evaluating the individual background, but the quality of the project. Additionally, Black Swan DAO
developed through its practice a consensus-building tool for collectives, called Cygnet. ⁵With consistent
optimization, a DAO can be used to organize a complex ongoing structure on an ongoing basis, as in a company, or
foundation-like groupings. The Blockchain expert Shermin V oshmgir compares the function of DAOs with the
control and regulation system of state governance layers, on the one hand, and with that of private companies,
on the other.
She does not connect its functionality to a recognized state. Voshmgir sees the automated DAO system with its
high, democratized adaptation factor as a structural improvement over the rigid forms used to date. Her main
argument here is that digitally secured compliance will radically end hierarchization. All the more so because a
DAO can be adapted from within the structure to the respective needs of the parties involved. ⁶This would
imply that Black Swan DAO could aim successfully to realize missing or not from their perspective successfully
developed elements of the structural support system, regarding state operations and possibly particularly the
support system of the arts.
Through their grassroots democratic ideal, DAOs do face anti-democratic conflicts, despite the complex
technical implementation: Misinformation, bribery, plutocracy or media control. These are, among others, factors
that can manipulatively influence the stakeholders of a DAO in their decision-making during the consensus
process. Corresponding counter-mechanisms are being developed, and partial successes are already in sight.
⁷To develop critical awareness towards misinformation therefore needs to be addressed inside the
structures of DAOs. As shown with the Black Swan DAO or as discussed by Claire Bishop in Artificial Hells
outside of all blockchain intersections, DAOs represent a key element of contemporary arts only strengthened
within the last years and especially since the Covid-19 pandemic: Participation and awareness of collective
practice. ⁸An example is to be found also within the BeeDAO, presented 2022 at Documenta 15. BeeDAO is not
only metaphorically oriented on the swarm organization of a beehive. It works towards the direct or indirect
interest of the alive bees. A physical manifestation takes place through monitored bee hives as direct
representatives of the species which is to be protected, producing public data. On physical assemblies,
proposals towards the shared mission are discussed and decided upon. Using blockchain technology, this
governance process cannot be manipulated through human agency, but provides a technological implemented
liability. ⁹The phrase swarm highlights the participatory practice of institutions and organizations,
referring also to the distinct participatory quality of Digital Art.
This also refers to the third element which highlights the quality of the DAOs. It can not only be seen as a
representation of humanly structured systems. It also can actively develop the idea and styles of
interconnection of planetary elements in anthropocentric times. Terra0 DAO was implemented in 2016 by Paul
Seidler, Paul Kolling and Max Hampshire, the terra0 collective. The DAO represents a forest.
The physical existent piece of land is obtained through initial funding by DAO members. Surplus made from
obtaining the forest, for example by selling the material wood, will be automatically used to buy out the share
of the initial investors. The goal of this project is to develop as much surplus through automated economical
processes enabled with mostly blockchain-based processes, that the forest will be able to buy out all human
investors and govern itself through automated Smart Contracts. ¹⁰This artistic practice scales back
to the cybernetic considerations of the symbioses of men and machine, as for example explained by Joseph C. R.
Licklider, but puts the environmental aspect of this interaction into the forefront. ¹¹Therefore it
develops this interaction further than before discussed, enabled through blockchain technology. It, moreover,
reminds the however involved human parties of two elements, which reoccur in contemporary artistic practice: the
power of the individual is perceptibly recognized. With the metaphorical proposition, that even the smallest
amounts of capital can play a relevant part in the overall economy, an emancipatory practice takes place. This
is underlined through the implementation of democratic mechanisms such as majority voting. Secondly, that the
interconnection of technology, ecology and culture is not a futuristic matter, but is happening in the current
times. Elements such as worker emancipation, trees or bees are rather positively connoted and it might not be a
coincidence that for the first projects operated within the system DAO and contemporary art do use those
accustomed elements in order to embrace engagement. Vitalik Buterin framed a reference system for organizational
structures of DAOs in 2014. The technological implementation took place with The DAO two years later.
¹²Seeing the implementation now already in place in contemporary arts, there is more than a potential
to be observed. Active use cases can be seen in DAOs as a method in contemporary artistic practice. All
discussed DAOs still rely on the physical involvement of members in order to function. There however exists a
structure which can overtime be strengthened and, in the process, develop the participatory aspect of the
collective works. How this will develop is not certain at this point, as little as the future of blockchain is
at this point determined. DAOs however already do confront contemporary conflicts with engaging solutions.
References
1 Shermin Voshmgir, Token Economy: How Blockchains and Smart Contracts Revolutionize the
Economy, Berlin, Blockchain-Hub, 2019, 104 et sqq.
2 Voshmgir, 2019, 119.
3 Vgl. Kathrin Passig, "Neue Technologien, alte Reflexe: Beitrag aus dem Sonderheft 2014 zum Thema
Medienevolution," accessed November 27, 2022,
https://www.medienkorre-spondenz.de/leitartikel/artikel/neue-technologien-altenbspreflexe.html.
4 Lucy R.Lippard, Sixyears: The dematerialization of the art object from 1966 to
1972, Berkeley, Univ. of California Press, 2007, p.IX.
5 Black Swan, “Cygnet Prototype,” in Radical Friends, Decentralised Autonomous
Organisations and the Arts, ed, Ruth Catlow et al.,Torque Editions, 2022, 322-323.
6 Voshmgir, 2019, 104-135.
7 Voshmgir, 2019, 130 et sqq.
8 Claire Bishop, Artificial Hells: Participatory art and the politics of spectatorship,
London, Verso, 2012.
9 "Bee DAO – the inaugural assemblee," Documenta 15, Website, accessed November 18, 2022,
https://documenta-fifteen.de/kalender/beedao-the-inaugural-assemblee/. See also:
https://beedao.zku-berlin.org/
10 Paul Seidler, Paul Kolling, Max Hempshire, “terra0. Can an Augmented Forest Own and
Utilize Itself?,” in Artistic Re:Thinking of the Blockchain, ed. Ruth Catlow et al., Torque
Editions, 2017, 68-72.
11 Joseph C. R. Licklider, "Man-Computer Symbiosis,“ IRE Transactions on Human Factors
in Electronics, HFE-1, 1960, 4-11.
12 Vitalik Buterin, "DAOs, DACs, DAs and More: An Incomplete Terminology Guide,“
Ethereum Foundation Blog, accessed November 5, 2022,
https://blog.ethereum.org/2014/05/06/daos-dacs-das-and-more-an-incomplete-terminology-guide
Bibliography
Claire Bishop, Artificial Hells: Participatory art and the politics of spectatorship, London,
Verso, 2012.
Swan Black, “Cygnet Prototype”, In Radical Friends. Decentralised Autonomous Organisations
and the Arts, edited by Catlow, Ruth et al., Torque Editions, 2022, 322-323.
Vitalik Buterin, "DAOs, DACs, DAs and More: An Incomplete Terminology Guide,“ Ethereum Foundation
Blog, Last accessed, 05.11.2022,
https://blog.ethereum.org/2014/05/06/daos-dacs-das-and-more-an-incomplete-terminology-guide.
Documenta fifteen, "BeeDAO – the inaugural assemblee,“ Last accessed 18.11.2022,
https://documenta-fifteen.de/kalender/ beedao-the-inaugural-assemblee/.
Joseph C. R. Licklider, "Man-Computer Symbiosis,“ IRE Transactions on Human Factors in Electronics,
HFE-1, 1960, 4–11.
Lucy R. Lippard, Six years: The dematerialization of the art object from 1966 to 1972,
Berkeley, Univ. of California Press, 2007.
Kathrin Passig, "Neue Technologien, alte Reflexe: Beitrag aus dem Sonderheft 2014 zum Thema,
Medienevolution,“ Last accessed 04.11.2022,
https://www.medienkorrespondenz.de/leitartikel/artikel/neue-technologien-altenbspreflexe.html.
Paul Seidler, Paul Kolling, Max Hempshire, “terra0. Can an Augmented Forest Own and Utilize
Itself?” In Artistic Re:Thinking of the Blockchain, edited by Catlow, Ruth et al.,
322-323,Torque Editions, 2017, 63-72.
Shermin Voshmgir, Token Economy: How Blockchains and Smart Contracts Revolutionize the Economy,
Berlin, BlockchainHub, 2019.
Lucid Dream: Sensing and Artistic Representation of Plant-Nature Interaction Based on Plants Biosignals
YouyangHu‡, ChiaochiChou*,
YasuakiKakehi‡
‡ The University of Tokyo * National Tsing Hua University Tokyo, Japan Hsinchu,
Taiwan 1huyouyang@xlab.iii.u-tokyo.ac.jp,
2chiaochi@gapp.nthu.edu.tw, 3kakehi@iii.u-tokyo.ac.jp
Abstract
The recent progress of modern science has enabled us to detect the physiological status of organisms through
their bioelectric activity; this technique has been constantly applied to contemporary media art. However,
exploring plants as a subject and allowing them to interpolate and hence perform art through biosignals remains
to be further explored. Lucid Dream is an artwork in which inspection, displacement, and engagement of plant
communities take place, shedding light on the subjective perception of plants, a peripheral subject that rarely
comes to people’s attention. To achieve this, we entered a forest and applied an artificial
intelligence-based learning system capable of interpreting the local plants' responses to wind and rain stimuli
via their biosignals. Subsequently, we established an environmentally controlled space within the art museum.
Here, we simulated and artistically represented the natural elements as perceived by the plants, using
artificial machines driven by the plant-nature interaction model. As viewers enter this space, they can
experience the plant-perceived natural environment, gaining a non-human perspective through direct engagement
with plant life. Lucid Dream not only leverages intelligent computational technologies to comprehend the
perceptual system of plants but also fosters cross-species sensory experiences, enhancing our understanding and
expanding our perspective on the natural environment.
In recent years, as we close the gap between technology and biological systems, artworks utilizing living and
semi-living organisms have become increasingly vigorous in contemporary media art. These artworks have presented
us with another perspective to understand different life forms. With the era of pan-biological materials of bio
art unfolding, artists and scientists who carried multiple identities began to displace the technique of
biosignal detection in art creations. A characteristic of displacement, which is Recontextualization, has
provided a propelling force for Lucid Dream is divided into the working phases of inspecting, displacing, and
engaging with plant communities. We first developed a bio-amplifier to collect biosignals from multiple plants
in a forest, we then the dialectic between artists acting due to their thought or their biological reactions to
stimulus¹, ². Afterward, with the exploration method of using biosignals to construct a new logic for
subjects or environments, people have further paid attention to other living organisms, especially algae and
microbes³, ⁴, ⁵.
In these artworks, non-human lives are presented as lively, having their own “umwelts”⁶.
However, to answer the upcoming question of whether living organisms have a more complex subjective activity and
how to represent it to viewers with more direct and persuasive evidence, we believe that plants can be an
answer. Due to their natural ability to perceive, adapt, and even evolve a particular method to respond to
climate change, plants demonstrate high sensitivity and easily observed feedback to the environment⁷. As a
result, it is of great potential for us to utilize intelligent computational technologies to create systems that
interface with the "umwelts" of plants in nature in such a way as to extend our understanding and broaden our
horizons regarding the natural environment.
Lucid Dream is divided into the working phases of inspecting, displacing, and engaging with plant
communities. We first developed a bio-amplifier to collect biosignals from multiple plants in a forest, we then
applied an artificial intelligence-based learning system that can interpret the plants' perception of the wind
and rain stimuli through their biosignals. The natural environment sensed by plants is displaced to the
environmental-controlled space in another realm to be artistically represented. After being
“Recontextualized,” the environmental data originally subject to scientific methods have been
transformed and enabled a new meaning. When viewers walk into this space, they can experience the natural
environment that plants perceive through their engagement with plants’ life from a non-human perspective.
Extending from art to science, breaking the existing framework of scientific research, and connecting
trans-species sensory experiences to provide a new possible way to respond to the question of representing the
subjective activity of living organisms to viewers.
Biosignals Detection and Analysis
In this project, we focus on the Mimosa in the forest, known for its high sensitivity to environmental stimuli
and commonly used in bioelectric signaling research⁸. We developed a miniature bio-amplifier to detect its
biosignals in response to environmental stimuli. It detects the difference in electric potential between the
substrate and the leaf petiole. As shown in Figure 2, it consists of two integrated amplifiers (TLV27L2 and
ADS1292). Due to the uncertainty of plant resistance value and the electromagnetic interference caused by other
equipment, most of the plant signaling tasks are carried out in the Faraday cage. We overcame this constraint by
using two high input impedance operational amplifiers (TLV27L2) to get less interference and enable detecting
the biosignals from plants in the natural environment. To reduce the size and power consumption of the circuit,
we utilize the ADS1292 which integrated high-resolution analog-to-digital converter (ADC) with a built-in
programmable gain amplifier. We used an STM32 microprocessor to pack the raw data and an ESP32 to transmit the
data to the computer wirelessly based on TCP protocol. On the computer side, there is a program developed in
openFrameworks that receives and processes the biosignals.
We next applied two supervised learning algorithms to conduct a time-domain analysis of the biosignals to get
the Mimosa's status under different environmental stimuli. Figure 3 shows the workflow of this system, A support
vector machine(SVM) classifier was applied to discriminate between wind versus rain-stimulated biosignal
activity. A support vector regression(SVR) model was then used to predict the wind intensity sensed by the
plant. We first reduced the noise of raw signals by a low-pass filter (second-order Butterworth filter) and then
collected the biosignals as 800 dimension vectors detected under two environmental stimuli. As shown in Figure
4, for each stimulus, 50 samples were collected and they were labeled "Wind stimulus" and "Rain stimulus."
We also collected 50 samples that the plant was under no stimulus. We then trained the SVM model that can
classify the status of plants under two environmental stimuli. We also collected the biosignals under three
intensities of wind stimulus. As shown in Figure 5, for each stimulus, 20 samples were collected and they were
labeled with integer values of 10, 127, and 255. We then trained the SVR model that can predict the wind
intensity as an integer value between 0 and 255 that the plants perceived in response to wind stimuli with
different intensities.
In this project, we have developed a plant-based system for environmental state sensing. Despite its inherent
biological variability, which can result in less precision compared to silicon-based sensors, this system holds
significant potential for future bio-hybrid sensing technology. Additionally, we are exploring its creative
applications in the realm of artistic practice, aiming to provide audiences with a novel perspective on their
interaction with the natural environment.
System Implementation and Artistic Representation
Based on our research in biosignals detection and analysis, we created an artwork that artistically represents
the natural environment perceived by plants in an artificial space. As shown in Figure 6, the artwork consists
of a plant observatory in the forest and an environmental-controlled space in the art museum. There are four
Mimosas equipped with bio-amplifiers in the plant observatory. The perceived state of each Mimosa in response to
environmental wind and rain stimuli will be analyzed by our artificial intelligence-based learning system and
remotely simulated in the environment-controlled space. The space is equipped with four sets of air-blowing
systems, each with three fans driven by the SVR models. The input is the biosignals of four Mimosas. The
intensity of the wind perceived by each Mimosa corresponds to the number of active fans in each air-blowing
system. A rain machine is installed in this space, which synchronizes the rain stimuli perceived by Mimosas
based on the SVM model. It consists of a water pipe running along the wall to the ceiling and a pumping motor.
When the system detects that Mimosas senses rain, it will trigger the rain machine to simulate the outdoor rain
environment in this space.
Figure 5 shows the exhibition site in the art museum, with a monitor real-time streaming the environmental
conditions in the forest and the biological state of the four mimosas. The natural environment sensed by plants
is displaced to the environmental-controlled space in another realm to be artistically represented. After being
“Recontextualized,” the environmental data originally subject to scientific methods have been
transformed and enabled a new meaning. When viewers walk into this space, they can experience the natural
environment that plants perceive through their engagement with plants’ life from a non-human perspective.
Conclusion
This paper introduces the artwork Lucid Dream, which uses plants as perceptual subjects to gain insight into
and artistically portray natural elements to viewers. To accomplish this, we have devised a bio-amplifier for
detecting plant biosignals in response to environmental stimuli. Subsequently, within a forest environment, we
have employed an artificial intelligence-based learning system capable of interpreting local plants' responses
to wind and rain stimuli through their biosignals. Additionally, we have established an environmentally
controlled setting to simulate and artistically represent the natural environmental elements as perceived by
these local plants to viewers. Lucid Dream not only delves into the utilization of advanced computational
technologies for understanding plant perceptual systems but also endeavors to create cross-species sensory
experiences, expanding our comprehension and perspective on the natural environment.
References
1 Alvin Lucier, Simon Douglas, Chambers, Scoresby Alvin Lucier, Wesleyan University Press, 2012.
2 David Rosenboom, "Method for producing sounds or light flashes with alpha brain waves for
artistic purposes," Leonardo, 1972, 141-145.
3 Carlos Castellanos, "IntersectionsofLivingandMachine Agencies: Possibilities for Creative AI,"
The Language of Creative AI, Springer, Cham, 2022, 155-166.
4 Sabina Hyoju Ahn, "Sonic transformation with living matter," In proceedings of ISEA symposium,
Inter-Society for the Electronic Arts, 2019.
5 You-Yang Hu, Chou Chiao-Chi, Li Chia-Wei, "Apercevoir: Bio Internet of Things Interactive
System," Proceedings of the 29th ACM International Conference on Multimedia, 2021.
6 Jakob von Uexküll, "An introduction to Umwelt," 2001, 107-110.
7 Jagadis Chandra Bose, The nervous mechanism of plants, Longmans Green, 1926.
8 Alexander G. Volkov, et al. "Mimosa pudica: electrical and mechanical stimulation of plant
movements." Plant, cell & environment 33.2, 2010, p.163-173.
Sounding Softness and the (Artificial) Subject
JonasJørgensen,
MadsBeringChristiansen
SDU Soft Robotics, SDU Biorobotics, The Mærsk Mc-Kinney Møller Institute,
University of Southern Denmark (SDU) Odense, Denmark jonj, mabc@mmmi.sdu.dk
Abstract
The paper discusses the authors’ artwork SONŌ, its artistic motivations, the artistic
research practice underlying its development, and its technical realization. SONŌ is a soft robotics
installation that interrogates the interconnections of soft materiality, sound, and subjectivity. It features a
sessile soft artificial entity capable of expansive movement, which is ceaselessly sounding itself and various
environments using real-time generated audio.
SONŌ is an artwork featuring a soft pneumatically actuated robot manufactured from silicone. The robot
possesses procedurally generated movement and sonification of movements accompanied by a soundscape.
Human and nonhuman animals make utterances that are socially communicative and function to enact a subject
position or form connections with other agents inhabiting the environment. The production of sound, whether
intentional or unintentional, is arguably a basic existential feat of all living organisms. As empirical
phenomena, however, sound by far predates life. In fact, the Universe emerged from what is arguably the ultimate
sonic event (which, paradoxically, no one was around to hear)—the Big Bang, cosmology tells us. Sound
always originates from a source yet is simultaneously transversal and expansive in character and by nature
destined to permeate its surroundings. It is a mediatic phenomenon par excellence—on the material level,
sound appears intangible and perhaps as almost nothing in itself, it only exists parasitic to matter,
manifesting as perturbations and pressure changes travelling in a physical medium. Sound is characterized by a
double movement—it is expansive and enveloping, yet simultaneously local and ephemeral. It is always
subject to dampening and seems to evaporate into thin air on the microscopic level, when its waves are converted
into heat through friction between the molecules of its medium.
Within the Western tradition of logocentric thinking, one of the ways in which sound comes to matter, is
through the privileging of speech over writing.¹ Speech is the primary medium of human thinking and writing
is merely a secondary technology. Hence, sound is positioned as the unbridled carrier of being and
subjectivity—“I sound, therefore I am.” Poststructuralists and their new materialist progeny,
however, champion a different position, that foregoes fixity, in favor of flux and the perpetual process of
becoming, which is perhaps more adequate to the ontology of sound itself. Here, the subject is considered
dynamic and decentered, and the boundaries between self and world permeable. Furthermore, agency is no longer
predicated upon subjectivity nor inherent to the subject itself, but a relational dynamism of forces enveloping
things as well as environments.²
Motivation and Practice
SONŌ (Latin: “[I make] sound”) explores a nexus of sound, soft robotics, and subjectivity.
Through the artwork and its associated practices, we seek to articulate and enact a myriad of complex
interactions between these phenomena and their aesthetic and epistemological capture.
SONŌ addresses what we take to be basic questions of robotic art in general, including, what does it take
to alter or blur the ontological status of an object towards that of a subject, by means of movement and sound?
As soft robotics (robotic morphologies and components constructed from pliable and elastic
materials³) is a key interest in our practice⁴-¹², within the work we were also keen to
query connections between soft materiality and sound through robotics as an aesthetic medium. For instance,
different kinds of matter are capable of producing impact sounds (via resonance) with specific characteristics
in terms of envelopes and frequency spectra¹³. Moreover, materials interact with sound in different
ways, e.g., soft materials tend to dampen sound whereas hard materials reflect it.
SONŌ is influenced by the notion of sound as a naturecultural¹⁴ phenomenon. It
seeks to consider divergent aspects of the material-semiotic conditions of possibility through which sound and
robotic movement can attain agency within specific environments. The physical and physiological properties of
softness and “soft sound” have thus fed into the work, but equally cultural meanings, e.g., notions
about sounds made by fictional soft characters from popular culture and cultural associations of softness as
aligned with, e.g., precariousness and vulnerability. From the outset, we were thus interested to probe the
chimeric character and synesthetic aspects of the concept of “softness”, when used to describe sound
and materials respectively. Definitions and delineations of “soft sound” within sound studies,
psychoacoustics, and musical theory, were, for instance, drawn upon in our explorations of what might constitute
“soft sound” and of the effects of adding “soft” or “hard” sound to a soft
morphology.
Part of the artistic research has been conducted in dialogue with the research field of human-robot
interaction (HRI), wherein sound has recently become subject of increasing interest. Our
practice sought to be receptive of pressing ideas and questions from this research field and consider how they
might gain relevance and be addressed through artistic forms. A body of work within the HRI field has
interrogated how various types of sounds can affect people’s perception and interaction with robots and
found non-verbal audio to be a salient feature with use potential as a deliberate design aspect of, e.g., social
robots. In certain situations and use cases, nonverbal audio is also preferable over synthetic voices, to guide
or facilitate interactions with humans.¹⁵ SONŌ adds to this research on robot sound, by
exploring how sound and embodiment can interact in soft social robots of unconventional nonanthropomorphic and
nonzoomorphic designs, which behaviors that should be accompanied with sound, and what the function of sound
might be within these.
The SONŌ Installation
The soft robot morphology was designed to appear organic yet unfamiliar (see Figs. 1-2). Abstract rounded
shapes and a hue with similarities to Caucasian human skin, or pig skin, with reddish spotted pigmentations were
used. The morphology possesses three independent pneumatic channels. Each of these interconnect four chambers
interspersed across it, which can expand when inflated. Ecoflex 00-30 silicone colored with Silc-Pig pigments
was used to cast the robot in a 3D printed mold (the robot’s design and fabrication is described in more
detail in ¹⁶).
In prior work we have discussed the artistic strategies used to compose the robot’s main sound design
(inspired by the sounds made by fictional soft characters in movies).¹⁷, ¹⁸ We have also
presented results of an empirical study exploring the effects of different sound designs on people’s
perceptions of the robot’s sociality and its interaction affordances.¹⁶ Following these
outcomes, work on presenting the project in the form of an art installation ensued.(1)
Physically, the SONŌ installation (Fig. 3) consists of: 1. The sonified soft robot displayed on a
black plinth
(dimensions 112 x 40 x 40 cm.) 2. A set of external speakers mounted in the room
The plinth features a door that can be opened to operate the robot during exhibition and houses the following
on three shelves (see Fig. 4): an active loudspeaker, an electro-pneu-matic actuation system (microcontroller,
motor shield, pumps, valves etc.) and an audio interface, a laptop PC running a software synthesizer. Along all
four edges of the plate holding the robot morphology, a small opening is present, to allow sound from the
loudspeaker inside the plinth to be transmitted to the exhibition space (Fig. 2). The audio of the installation
consists of two times 2-channel stereo comprising: 1. robot sound – played over the loudspeaker inside the
plinth, 2. a soundscape – played over the external loudspeakers in the room.
The Technical System
A diagram of the technical system is shown in Fig. 5 with the signal paths indicated. An Arduino Uno, which
controls the robot’s movement by activating pumps and valves, functions as the master with a laptop PC
generating the audio running as slave (for details see ¹⁶–¹⁸). A signal to generate
matching robot sound using the FM software synthesizer is sent when a movement phrase is triggered. The robot
does not currently have any sensors.
Robot Behavior and Sound Programming
For the final installation, we built upon existing code already developed for movement and sound generation. We
chose a phrase-based and a categorical approach to designing the robot’s behavior. By “phrase”
we refer to sequences of robot movement and matching sound of a duration up to 30s. The robot operates as a
finite-state machine (FSM) with four mood states (categories). These each correspond to different levels of
arousal (relaxed, medium-relaxed, medium-aroused, aroused). In each mood, the robot generatively combines a
specific set of phrases and pauses matching this mood. Each of the phrases were hand coded and iterated upon for
expressivity (through trial and error) and subsequently matched to one of the four mood states. A total of 13
phrases were used as building blocks that are combined in different ways to generate the robot’s movement
and sound behaviors. All mood states feature a breath-like phrase with the robot performing asynchronous
periodic inflation across the chambers, that use increasing frequencies for increased arousal, in accordance
with findings of our prior work.¹⁹ In addition to the robot sound, the installation features twelve
composed soundscapes that are played parallelly through Ableton Live. These are also triggered by the
microcontroller, but asynchronously with the robot’s movements and sound. The soundscapes consist of
processed synthesized and recorded sounds and select sonic textures combined into ethereal sonic expressions
devoid of temporal structure and timing. The audio was kept spacious and wide to let the robot’s more
erratic utterances come into focus and only add a more subtle affective coloring of these. In accordance with
this, the soundscapes are played over loudspeakers physically separate from the robot’s plinth. The
soundscapes mix slow extensive sounds to create the aural impression of an atmosphere of the installation and to
position the robot (and its visitors) in different sonic worlds of various affective intensities. Played in
random succession, the soundscapes complement or clash with each other, and contribute a sense of emergent
narrative.
Further work
We plan to expand upon SONŌ in an updated version of the installation and in subsequent independent
works. As a next step, we would like to develop a means to have more interactive generation of the robot and
soundscape audio. Currently, the robot switches between its four mood states pseudo-randomly with the
statistical likelihood that the robot will switch its arousal state up or down after a completed phrase cycle as
an adjustable parameter. We hope to add sensors to the installation, e.g., room scale computer vison, to track
activity, behaviors, and affective states of visitors to enable the robot to interact. Furthermore, we are
considering developing and validating a more fine-grained phrase-based or parametric generation of affective
movement and sound with the system that can contribute more variation and nuance to the robot’s
expressions.
1 A supporting video showcasing excerpts of the SONŌ robot performing with the robot sound and
soundscape is available at: https://youtu.be/U0fGXCbcygU
Acknowledgements
We thank Cao Danh Do for help with 3D prints, mechanical design, construction, and illustrations.
References
1 J. Williams, Understanding Poststructuralism, First edition, Chesham, Bucks, Routledge,
2005.
2 K. Barad, Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and
Meaning, Durham, NC, Duke University Press, 2007.
3 D. Rus, M. T. Tolley, “Design, fabrication and control of soft
robots,” Nature, vol. 521, no. 7553, May 2015, 467–475, doi: 10.1038/nature14543.
4 M. B. Christiansen, L. Beloff, J. Jørgensen, A.-S. E. Belling, “Soft Robotics and
Posthuman Entities,” Journal for Artistic Research, no. 22, Dec. 2020, doi:
10.22501/jar.549014.
5 J. Jørgensen, “Leveraging Morphological Computation for Expressive Movement
Generation in a Soft Robotic Artwork,” in Proceedings of the 4th International Conference on
Movement Computing, in MOCO ’17, New York, NY, USA, ACM, 2017, 20:1-20:4, doi:
10.1145/3077981.3078029.
6 J. Jørgensen, “Prolegomena for a Transdisciplinary Investigation into the
Materialities of Soft Systems,” in ISEA 2017 Manizales: Bio-Creation and Peace: Proceedings of the
23rd International Symposium on Electronic Art, University of Caldas, Manizales, Colombia, Department
of Visual Design, Universidad de Caldas, and ISEA International, 2017, 53–160.
7 J. Jørgensen, “Interaction with Soft Robotic Tentacles,” in Companion of
the 2018 ACM/IEEE International Conference on Human-Robot Interaction - HRI ’18, Chicago, IL,
USA: ACM Press, 2018, 38–38, doi: 10.1145/3173386.3177838.
8 J. Jørgensen, “Constructing Soft Robot Aesthetics: Art, Sensation, and Materiality
in Practice,” Ph.D. thesis, IT University of Copenhagen, Copenhagen, 2019.
9 J. Jørgensen, “From Soft Sculpture to Soft Robotics: Retracing Entropic Aesthetics
of the Life-like,” in Shifting Interfaces: An Anthology of Presence, Empathy, and Agency in
21st-Century Media, H. Aldouby, Ed., Leuven, Belgium: Leuven University Press, 2020, 223–242.
10 J. Jørgensen, S. Ploetz, “LARPing Human-Robot Interaction,” in HRI
2020 Workshop on Exploring Creative Content in Social Robotics, Apr. 2020, Accessed: Jun. 19, 2020.
[Online]. Available: https://portal.findresearcher.sdu.dk/da/publications/larping-human-robot-interaction
11 J. Jørgensen, “TeMoG – An Accessible Tool for Creating Custom Soft Robotics
Parts,” in Interactivity and Game Creation, A. Brooks, E. I. Brooks, and D. Jonathan, Eds., in
Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering,
Cham: Springer International Publishing, 2021, 331–342, doi: 10.1007/978-3-030-73426-8_20.
12 J. Jørgensen, “Towards a Soft Science of Soft Robots. A Call for a Place for
Aesthetics in Soft Robotics Research,” J. Hum.-Robot Interact., vol. 12, no. 2, Mar. 2023,
15:1-15:11, doi: 10.1145/3533681.
13 W. Fujisaki, N. Goda, I. Motoyoshi, H. Komatsu, S. Nishida, “Audiovisual integration in
the human perception of materials,” Journal of Vision, vol. 14, no. 4, Apr. 2014,
12–12, doi: 10.1167/14.4.12.
14 D. Haraway, “Situated knowledges: The science question in feminism and the privilege of
partial perspective,” Feminist studies, vol. 14, no. 3, 1988, 575–599.
15 F. A. Robinson, O. Bown, M. Velonaki, “Designing Sound for Social Robots: Candidate
Design Principles,” Int J of Soc Robotics, vol. 14, no. 6, Aug. 2022, 1507–1525, doi:
10.1007/s12369-022-00891-0.
16 J. Jørgensen, M. B. Christiansen, “The Sounds of Softness. Designing Sound for
Human-Soft Robot Interaction,” Frontiers in Robotics and AI, vol. 8, 2021, 1–17,
doi: 10.3389/frobt.2021.674121.
17 M. B. Christiansen, J. Jørgensen, “Augmenting Soft Robotics with Sound,”
in Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, in HRI
’20, New York, NY, USA, Association for Computing Machinery, Mar. 2020, 133–135, doi:
10.1145/3371382.3378328.
18 M. B. Christiansen, J. Jørgensen, “SONŌ,” in Companion of the 2020
ACM/IEEE International Conference on Human-Robot Interaction, in HRI ’20. New York, NY, USA,
Association for Computing Machinery, 639, Mar. 2020, doi: 10.1145/3371382.3378399.
19 T. A. Klausen, U. Farhadi, E. Vlachos, J. Jørgensen, “Signalling Emotions with a
Breathing Soft Robot,” in 2022 IEEE 5th International Conference on Soft Robotics
(RoboSoft), Apr. 2022, 194–200. doi: 10.1109/RoboSoft54090.2022.9762140.
Bibliography
Barad, Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning,
Durham, NC, Duke University Press, 2007. M. B. Christiansen, L. Beloff, J. Jørgensen, A.-S.
E. Belling, “Soft Robotics and Posthuman Entities,” Journal for Artistic Research,
no. 22, Dec. 2020, doi: 10.22501/jar.549014.
B. Christiansen, J. Jørgensen, “Augmenting Soft Robotics with Sound,”
in Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, in HRI
’20, New York, NY, USA, Association for Computing Machinery, Mar. 2020, 133–135, doi:
10.1145/3371382.3378328.
B. Christiansen, J. Jørgensen, “SONŌ,” in Companion of the 2020 ACM/IEEE
International Conference on Human-Robot Interaction, in HRI ’20. New York, NY, USA, Association
for Computing Machinery, Mar. 2020, 639, doi: 10.1145/3371382.3378399.
Fujisaki, N. Goda, I. Motoyoshi, H. Komatsu, S. Nishida, “Audiovisual integration in the human
perception of materials,” Journal of Vision, vol. 14, no. 4, Apr. 2014, 12–12, doi:
10.1167/14.4.12.
Haraway, “Situated knowledges: The science question in feminism and the privilege of partial
perspective,” Feminist studies, vol. 14, no. 3, , 1988, 575–599. J.
Jørgensen, “Leveraging Morphological Computation for Expressive Movement Generation in a Soft
Robotic Artwork,” in Proceedings of the 4th International Conference on Movement Computing,
in MOCO ’17. New York, NY, USA: ACM, 2017, 20:1-20:4. doi: 10.1145/3077981.3078029.
Jørgensen, “Prolegomena for a Transdisciplinary Investigation into the Materialities of Soft
Systems,” in ISEA 2017 Manizales: Bio-Creation and Peace: Proceedings of the 23rd International
Symposium on Electronic Art, University of Caldas, Manizales, Colombia: Department of Visual De-
sign, Universidad de Caldas, and ISEA International, 2017, 153–160.
J. Jørgensen, “Interaction with Soft Robotic Tentacles,” in Companion of the
2018 ACM/IEEE International Conference on Human-Robot Interaction - HRI ’18, Chicago, IL, USA:
ACM Press, 2018, 38–38, doi: 10.1145/3173386.3177838. J. Jørgensen, “Constructing Soft
Robot Aesthetics: Art, Sensation, and Materiality in Practice,” Ph.D. thesis, IT University of
Copenhagen, Copenhagen, 2019. J. Jørgensen, “From Soft Sculpture to Soft Robotics: Retracing
Entropic Aesthetics of the Life-like,” in Shifting Interfaces: An Anthology of Presence, Empathy,
and Agency in 21st-Century Media, H. Aldouby, Ed., Leuven, Belgium: Leuven University Press, 2020,
223–242. J. Jørgensen, S. Ploetz, “LARPing Human-Robot Interaction,”
in HRI 2020 Workshop on Exploring Creative Content in Social Robotics, Apr. 2020,
Accessed: Jun. 19, 2020. [Online]. Available:
https://portal.findresearcher.sdu.dk/da/publications/larping-human-robot-interaction
Jørgensen, “TeMoG – An Accessible Tool for Creating Custom Soft Robotics Parts,”
in Interactivity and Game Creation, A. Brooks, E. I. Brooks, and D. Jonathan, Eds., in Lecture
Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, Cham:
Springer International Publishing, 2021, 331–342, doi: 10.1007/978-3-030-73426-8_20.
Jørgensen, M. B. Christiansen, “The Sounds of Softness. Designing Sound for Human-Soft Robot
Interaction,” Frontiers in Robotics and AI, vol. 8, 1–17, 2021, doi:
10.3389/frobt.2021.674121.
Jørgensen, “Towards a Soft Science of Soft Robots. A Call for a Place for Aesthetics in Soft
Robotics Research,” J. Hum.- Robot Interact., vol. 12, no. 2, Mar. 2023, 15:1-15:11, doi:
10.1145/3533681.
A. Klausen, U. Farhadi, E. Vlachos, J. Jørgensen, “Signalling Emotions with a Breathing Soft
Robot,” in 2022 IEEE 5th International Conference on Soft Robotics (RoboSoft), Apr.
2022, 194–200, doi: 10.1109/RoboSoft54090.2022.9762140.
A. Robinson, O. Bown, M. Velonaki, “Designing Sound for Social Robots: Candidate Design
Principles,” Int J of Soc Robotics, vol. 14, no. 6, 1507–1525, Aug. 2022, doi:
10.1007/s12369-022-00891-0.
Rus, M. T. Tolley, “Design, fabrication and control of soft robots,” Nature, vol.
521, no. 7553, 467–475, May 2015, doi: 10.1038/nature14543. J. Williams, Understanding
Poststructuralism, First edition, Chesham, Bucks: Routledge, 2005.
Author Biographies
Jonas Jørgensen is a researcher and artist based in Copenhagen focusing on practice-based
transdisciplinary work at the nexus of technical science, media art aesthetics, and the posthumanities. He
holds a position as Associate Professor at the Biorobotics section of the University of Southern Denmark where
he is co-director of the Center for Soft Robotics. Webpage: https://jonasjoergensen.org/
Mads Bering Christiansen is a Copenhagen-based designer/artist/researcher whose practice evolves around
designing objects and experiences to explore and speculate about the potential futures emerging from human
technological entanglement. He is currently PhD student at the Center for Soft Robotics at the University of
Southern Denmark with a project that investigates the effects of integrating biomorphic aesthetics in robots
for social human-robot interactions. Webpage: https://madsbering.com/
Gesture and Spatiality in Electroacoustic Improvisation with Digital Video
MichaelLukaszuk,
SojungBahng
Queen’s University DAN School of Drama and Music Kingston, Canada
michael.paul.lukaszuk@gmail.com, sojung.bahng@queensu.ca
Abstract
This paper discusses a multidisciplinary approach to improvisation in which electroacoustic performance is
informed by its capacity to transform and respond to characteristics of digital video, along with visual and
architectural features of virtual spaces. We describe technical and creative strategies that are focused on
leveraging mobile devices in a performance setting for navigating audio-visual relationships in which digital
spaces are conveyed through video playback and processing, while gestural control highlights actual physical
spaces (e.g., gallery spaces). This creates a hybrid venue that benefits live electroacoustic creation with
digital artists whose work extends beyond the domain of sound. The presentation visual analogues to sonic
transformations of gesture and spatiality are used to highlight aspects of electroacoustic performance that
develop out of a gradual exploration between emulation and obfuscation of referential sound objects. Mappings
between electroacoustic instrument and digital video also help delineate the creative use of mobile devices and
controllers as being more than tools. At the same time, the transmission of MIDI and OSC (Open Sound Control)
messages from mobile instruments allows collaborating artists to resource electroacoustic sound in non-aural
contexts.
Keywords
Electroacoustic, Sound and Music Computing, Spatialization, Audio-Visual Improvisation, Digital Instruments.
Digital lutherie has been an effective thread for connecting electroacoustic musicians with video and extended
reality artists. The data stemming from a digital translation of physical gesture to affect sound diffusion can
also be used to convey intersecting spatial relationships for artists who explore proximity and physicality in a
visual context. The motivation for this project is to explore the mobile phone as an instrument for
multidisciplinary improvisation that involves electroacoustic music. Beyond establishing the mobile phone as a
convenient resource for blending artistic or scholarly boundaries, there is a need to consider the
organological implications of routing the device through software environments for sound and visual
media. This depends on making decisions about what kinds of mappings and idiomatic behaviours help assert a
sense of instrumentality, so that a “smartphone-as-instrument” setup conveys compositional drivers
that are germane to electroacoustic creation—such as the spectrum between sounds being presented with a
relationship to a physical source vs. belonging to a more acoustmatic context. In his article
“The Instrumentality of Music,” musicologist Philip Alperson uses the example of Bob Dylan’s
infamous performance with electric guitar at the 1965 Newport Folk Festival to emphasize how instruments help us
understand the taxonomy of musical genres and how instrumentality, as a concept, goes beyond the notion of a
music being “of instruments” to show how instruments place music in relation to larger cultural
trends.¹ Focusing on the mobile phone as a collaborative digital instrument helps disrupt notions of
traditional musical performance practice that render electroacoustic as non-performative. We will discuss
electroacoustic improvisation with digital video in relation to the piece Suffocation
(2022), in which gestural control of a mobile device within a multichannel audio setup facilitates
continuous reframing of sourced objects within 360-degree video.
Related Work
In the 21st century, trends such as the proliferation of laptop orchestras and the emergence of live coding
help establish a context in which seemingly commonplace technologies can be understood as musical instruments
through their use within simulated spaces. For example, in composer Mara Helmuth’s laptop ensemble work,
pieces such as from Uganda...(2013) use a simple graphical user interface in Max/MSP as an instrument
for constructing imaginary sonic environments based on soundscape recordings.² In addition to environmental
references, performers interact with the laptop as an electroacoustic instrument by routing their individual
playback through a multichannel audio setup used within a concert hall.
Interactive musical mobile apps such as Brian Eno and Peter Chilver’s Bloom (2008) use synthetic
or at least ambiguous sampled sounds to create a kind of acousmatic sound world in which there is
intentional ambiguity with the manner in which the source material is presented. Bloom also bridges
mobile phone performance in relation to digital spaces through a 2018 partnership with Microsoft to create a
mixed reality installation in Amsterdam’s Transformatorhuis arts space in which the user interface was
projected on larger touch-sensitive surfaces.³ Conversely, Chilvers’ other musical app projects such
as Air (2011) relies on the reconfiguration of Irish singer Sandra O’Neal’s recorded
vocal performance through gestural interaction with the mobile device. In her PhD Dissertation, Vanessa Chang
writes that “through gesture, contemporary digital art practices borrow and recode forms of presences...to
highlight a more diverse array of creative agencies.”⁴ A “vocal performance” with
“Air” abandons the idea of the voice as a fixed instrument dependent on breath control and becomes
an expression of a cybernetic virtuosity that depends on feedback between physical and simulated bodies.
Suffocation
Suffocation was developed during the fall of 2022, as an improvisational piece stemming from the
authors’ involvement in a mixed-media ensemble, and a desire to explore spatiality using an approach that
blended our respective practices. For the video content used in this piece, a 360-degree camera was placed
inside a small container containing an assortment of tiny objects, such as pins, tissues, pills, and leaves, and
poured or added liquids. The everyday objects and materials are defamiliarized by creating the illusion of
scale. It provokes a paradoxical sensation between unpleasant sentiments and familiar feelings. The audio
content moves through processes of defamiliarization and reimagination of conventional instrumental sounds
within a digital context. This is achieved by sampling commonplace instruments such as pianos and kalimbas,
while also including computer-generated emulations of these sources that could be bent and distorted by
manipulating sound synthesis parameters.
In addition to the use of a quadrophonic (4.0-channel) audio playback system, additional highly directional
parametric speakers were positioned in our gallery space to synchronize with the presentation of height and
depth in the video content.
The setup for project uses pitch, roll, yaw, accelerometer, and keypad button OSC data transmitted from
composer/programmer Kevin Schlei’s smartphone app GyrOSC to affect a Max/MSP patch that acts as a link
between electroacoustic sampling and sound synthesis in the digital audio workstation Reaper, and video playback
and manipulation taking place through an Isadora patch.
Max/MSP sends MIDI control change and note messages that are mapped to affect spatial parameters of projected
video such as scaling and rotation, while simultaneously affecting changes such as triggering recorded audio
samples, musical pitches for virtual synthesizer plugins and audio panning positions. It is the simultaneous
translation of OSC to MIDI data mapped to Isadora for video and Reaper for audio that allows the smartphone to
act as a digital instrument for improvisation.
Theme and “Learned Gestures”
The use of a thematic design for structured improvisation with a digital instrument is an important
consideration for cultivating the mobile device as an electroacoustic instrument for multidisciplinary
collaboration. The use of conceptual bases or programmatic associations helps inform physical gestures that can
help the performer discover new ways of interacting with the mobile device while considering notions of
idiomaticity and how physical movements (e.g., changes in X-Y-Z axis positions) supports the position that
spatialization and abstraction of source material can be principal drivers in audio-visual work. The linking of
audio-visual parameters through MIDI and OSC also acts as a form of research-creation for performance in which
users can rely on previously learned actions (e.g., swiping, tapping, repositioning the device) in performance
to access new forms of creativity. This idea of accessibility issues as being resolved by digital instruments is
echoed in computer music research, such as computer music programmer Ge Wang’s documentation of laptop
orchestra as a pedagogical tool. His research from the Princeton Laptop Orchestra describes such the ensemble as
an experiential workshop environment that allows computer music studies to venture away from a studio/lab
environment that separates composition and performance and instead encourages interdisciplinary
exploration.⁵
This writing also describes how the ensemble’s use of sound spatialization approach references the
hierarchical nature of the orchestra in classical music. While the laptop orchestra model considers spatiality
in reference to its comparison with a conventional symphony orchestra, our use of learned gestures in
electroacoustic improvisation with digital video adds a new perspective to research-creation efforts in
which performance with electroacoustic instruments can also be used for “epistemological intervention into
the ‘regimes of truth’...”⁶ Rather than challenging the orchestra paradigm, we focus on
reconsidering Eurocentric structural materials that foreground melodically oriented constructs of rhythm and
pitch still permeate so much of contemporary sonic performance. By affecting the morphology of sound as
connected to video through fluid physical movements, learned gestures become objects within the performance that
help delineate how sonic and/or visual media are developing.
Conclusion
An exploration of digital instruments as reliant on familiar movements is echoed in computer music research,
such as Thor Magnusson’s Sonic Writing, which emphasizes that “...young people often
enter the world of music through digital technologies (e.g., apps and games).”⁷ By exploring the
cybernetic virtuosity in digital sound and video performance through physical gestures learned through everyday
interactions with mobile devices, we wish to uncover the possible components of a performance practice that is
compatible with live digital art practices such as electroacoustic music—in which spatiality is so
essential to the performance experience. A traditional orchestral performer may have to learn how to use
instrumental techniques such as vibrato to better project the sound of their instrument in a space, the use of a
mobile phone in an improvisational piece such as Suffocation exposes a multiplicity of layers in which
rhythm, cadence and narrative structure are also conveyed by the way that the performer explores the proximity
and distance of their own body as connected to digital audio and visual materials.
References
1 Philip Alperson, “The Instrumentality of Music,” The Journal of Aesthetics and
Art Criticism 66/1, Winter, 2008, 37-51.
2 Michael Lukaszuk, “Soundscapes and Anachronisms in Music for Laptop Ensemble,”
eContact! 21/1, Spring 2022, https://econtact.ca/21_1/lukaszuk_laptop-soundscapes.html
3 Susanna Ray, “Brian Eno, Peter Chilvers create ‘quite magical’ flower garden
of sound in Amsterdam with ‘Bloom: Open Space," Microsoft/Features, Accessed May 15, 2021, February 22,
2018,
https://news.microsoft.com/features/brian-eno-peter-chilvers-create-quite-magical-flower-garden-sound-amsterdam-bloom-open-space/
4 Vanessa Chang, “Tracing Electronic Gesture: A Poetics of Mediated movement," PhD
dissertation, Stanford University, 2017.
5 Ge Wang, et al, “The Laptop Orchestra in the Classroom,” Computer Music Journal
32/1, Spring 2008, 26-37.
6 Owen B Chapman, Sawchuck Kim, “Research-Creation: Intervention, analysis and ‘family
resemblances’,” Canadian Journal of Communication 37/1, April 2012, “Media Arts
Revisited,” 5–26.
7 Thor Magnusson, Sonic Writing, London, Bloomsbury, 2018.
Generated tools: A Defamiliarizing Approach to Creating ML Art
ChloeMcFadden and
OliverBown
School of Art and Design, UNSW New South Wales, Australia chloe.mcfadden@unsw.edu.au,
o.bown@unsw.edu.au
Abstract
In recent years there has been an increase in the adoption of machine learning (ML) systems that can generate
novel images. This increased use may reveal the beginning of a familiarity in which the implications of these
emerging technologies are naturalised or made increasingly invisible. Thus, practices which can disrupt
familiarity may allow us to create experiences of heightened awareness in which we can consider our engagement
with this emerging technology. In this paper, I discuss the outcomes of working with a Generative Adversarial
Network (GAN), using a dataset created from the hand tools section of a popular Australian hardware store.
Through this creative practice, I investigate how artists can use ML as mechanisms for creating artworks that
disrupt, investigate and defamiliarize the known.
As digital technologies are increasingly embedded into the everyday and the ways we construct and interact
within our world perception, it has been noted that they also become increasingly familiar, invisible, or
subconscious.¹ While often perceived as a ‘tool’—an extension of the human hand designed
to fulfil a certain function—our relation to technology is increasingly being understood as a symbiosis of
interdependence.² Furthermore, the co-evolutionary relationship between tools and humans has also been
noted—highlighting the influence these ‘tools’ have in shaping us, as we shape them.³ As
we continue to see, use, and adopt these tools, they become increasingly familiar and every day. However, with
this increased familiarity may come a decrease in active or conscious visibility resulting in the loss of
regular inciting provocation to question how we relate, interact, and construct our perception of the world with
these technologies. Thus, techniques of making the familiar unfamiliar may be a strategy for creating
experiences of heightened perception of or critical engagement with technology.
Generated tools (2021) is a practice-based research project in which I worked with a GAN to create an
installation that features tools as subject matter. In this, I am exploring how we can use ML conceptually to
create defamiliarizing experiences which may allow us to critically reflect upon and re-engage with concepts,
worldviews and ideas which have been naturalised. ML that can generate images is compelling
technology—sparking new ways of making and creative possibilities. However, it also has the potential to
be a reaffirming conservative force, as it is informed by the training dataset and may familiarize and
perpetuate the visuals, ideas and assumptions hosted within.⁴ While this reliance upon the dataset may
pose a risk if we engage uncritically—this also presents the opportunity for artists to disrupt, explore
and denaturalise the known. Thus, through the documentation and discussion of work created with a GAN, this
paper presents the concept of defamiliarization as one approach for understanding the potential impact or
possibilities of working with ML to generate art.
Tools, Technology, and Familiarity
There has been much critical reflection on the relationship between humans and tools—both in how tools
present an invisible familiarity, as well as in the mutual influence we exert. Martin Heidegger explained tools
as "ready-to-hand", meaning as long as they function correctly or to our expectations, they remain concealed
from view, or not in our conscious presence.⁵ However, while tools typically fade into the realm of the
unconscious everyday, they also reveal a co-evolutionary relationship between tools and humans. While we
typically narrativize our relation to tools as being a one-way influence, contemporary philosophical approaches
recognise that it is more of a symbiosis. To solve an issue or to respond to our environment, we create new
tools, which in turn change how we act and lead to the discovery of new problems or use cases for tools as this
cycle repeats itself.⁶
Similar lines of thought have also been applied to technology. It has been suggested that technology is an
extension of ourselves with which we cohabit a shared ecosystem. Through this cohabitation, we are able to
coextend our skills, capabilities, and properties, changing how we act in ways we perceive as beneficial, and in
turn leading to the development and adoption of new technologies.⁷
However, as this cycle of development and adoption continues, this cohabitation becomes increasingly familiar
or subconscious.⁸ While technological innovation feels novel at first, over time with increased adoption
and use, repeated interaction results in a comfortable familiarity that doesn’t necessarily encourage
reflection or critical engagement.
In the past 5 years, there has been increased adoption of ML systems that can generate novel images. While
there is a large amount of general ‘hype’ and awareness, the development of systems like DALLE-2,
Stable Diffusion and Mid Journey, which all have user-friendly demos and applications hosted online, alongside
apps like WOMBO Dream and TikTok’s green screen AI filter, may reveal the beginning of familiarity, or a
future of familiarity with ML systems.
Defamiliarization and visual indeterminacy
If familiarity reduces technology to the subconscious, thus dampening opportunities to be aware of how we
relate to and are shaped by it, unfamiliarity becomes a potential strategy for sparking conscious engagement.
Coined by Viktor Shklovsky in 1917, defamiliarization is understood as a tactic for creating heightened
awareness or perception by halting our automatic assumptions about a subject, allowing us to view it again for
the first time.⁹, ¹⁰, ¹¹ The goal of such a practice is not to
reveal a more objective truth about a subject, but rather to create a heightened perception of how we construct,
understand, and relate to the subject.¹² Defamiliarization has also been identified as a common tactic
employed by digital artists to create experiences of critical distance between audiences and technology, to
temporarily make the familiar unfamiliar for heightened perception to be achieved.¹³
The ways that artists enact this varies broadly, as it is understood that all art forms can generate a
defamiliarizing effect.¹⁴ However, one emerging approach related to ML is visual indeterminacy. It
has been noted that GANs typically create uncanny, strange, or visually indeterminate images.¹⁵
Visual indeterminacy, as well as ambiguity and uncertainty, has also been recognized as useful tools for
prompting multiple interpretations or disrupting an artefact’s socially encoded
properties.¹⁶, ¹⁷ Furthermore, artists may be able to engage the differences
between our logic and the chosen system’s logic to create defamiliarizing effects. For example, when
working with image-based ML, we understand images as representations of objects, scenes, and worldly concepts,
while a system like a GAN is attempting to map the underlying structural logic of the dataset at a pixel level.
Engaging the system’s ability to recreate patterns within the dataset, we can generate coherent and
recognisable forms. However, by navigating their latent spaces, we can push them to create semi-coherent, or
visually indeterminate forms. Thus, working with ML to generate art may present opportunities to denaturalise
our understandings through visual indeterminacy or strangeness, via this difference in structural logic.
Installation description
The installation consists of a tool wall which houses three instances of working with a StyleGAN2 trained using
a dataset of images of hand tools sourced from the Bunnings Warehouse website (an iconic Australian hardware
store).¹⁸ The first instance shows 3D printed tools, modelled and printed using the generated images
by the GAN as reference. The second instance shows 2D “latent space walk” videos projected on the
tool wall, created by incrementally sampling the latent vector space generated by the GAN. The final instance
shows doctored Bunnings Warehouse product catalogues, featuring images of tools generated by the GAN. Audience
members are encouraged to engage with the work by flipping through the product catalogues, as well as being able
to pick up, play with and rearrange the 3D printed objects on the tool wall.
3D Printed tools
While images created by ML systems are becoming increasingly familiar, we most commonly encounter them in
digital spaces, as images or animations. Thus, one goal of the work was to translate the generated images into
3D objects to observe whether this could be a method of creating defamiliarizing experiences. The 3D printed
tools were created by first generating a series of images using a StyleGAN2 trained on a dataset of images of
hand tools sourced from the Bunnings Warehouse website. While the GAN is trained using these images of hand
tools, how these tools exist and function in conceptual space is not captured. Thus, when it generates images of
tools it does so with no consideration of functionality—creating tools that have no clear associated uses.
Simultaneously, we can expect the generated images to be aesthetically tool-like as the GAN is attempting to
replicate the original dataset and find its underlying structural logic. Thus, the work explores whether the
‘uselessness’ of the GAN-produced artefacts is an effective tactic of defamiliarization.
Margaret Boden discusses ‘useless’ artefacts as having the ability to playfully challenge
expectation, while bringing the typical affordance of similar items to the foreground.¹⁹ Tools are
useful—but most of the time they are not being used—hung upon tool boards like artworks, neatly
organised in boxes like collectables, or haphazardly thrown in draws like junk. They often also hold semantic
and sentimental value - tools are given as gifts, passed down through families and lent between friends and
communities. If tools remain concealed from perception so long as they function correctly, how does encountering
inherently useless tools bring these hidden ‘uses’ to the foreground?²⁰
The generated images were then curated and used as references to model and texture 3D tools in Autodesk Maya
and Mudbox. Reference images were selected through a process of working through the generated images and
selecting a broad range of shapes, tool-like iconography and sizes that existed within the possibility space.
The goal when modelling and texturing the tools was to follow the reference images closely, while also
recognizing the inherent role of interpretation when translating from 2D to 3D. These models were then 3D
printed using a variety of grey, black, and aluminium filament. The results are visually and texturally strange,
tool-like artefacts. As they were modelled using the StyleGAN2 curated images as reference, they have a visually
indeterminate quality—with familiar elements (e.g. handles, pointy ends, and bits and pieces of
recognizable tools) combined or blurred in unfamiliar ways. Further, since they are 3D printed, the tools have
unique ridges, divots and holes which create an unfamiliar hand feel. The light-weight material of the filament
creates a unique engagement with the tools which we typically associate with rubbery, metal, and heavy
sensations.
Through the combination of recognizable tool-like elements in unfamiliar ways with the 3D printed materiality,
the familiar functions and feelings that are tied to how we perceive, construct and experience tools
subconsciously are no longer present—opening the space for new interpretations and speculation. As
audiences would approach the strange tools, typically their first reaction was to generate a new use case for
the tool. In short, this element of the work combines visual indeterminate ‘useless’ tools supplied
by the GAN with subversive, textural 3D prints to create a speculative experience for audiences, engaging
difference in structural logics, visual indeterminacy and unfamiliar materiality as tactics of
defamiliarization.
2D latent space walk videos
The projected 2D latent space walk videos were created using the possibility space generated by the StyleGAN2
trained on the dataset of hand tools. Latent space walk videos involve the sampling and incremental changing of
points in the latent vector space which can then be used to create animations. As the system was trained on a
dataset of hand tools, the resulting animations present fluid tools that merge and shape into one another. While
we typically conceptualise tools in a worldly context—with clear boundaries, defined shape language and
affordances that are affirmed via learnt aesthetics (e.g., handles, spouts)—the GAN’s distribution
renders a more fluid visualisation.
Thus, the work is interested in whether the visual indeterminacy created by the GAN is an effective strategy of
defamiliarization. As visual indeterminacy is understood as a tactic to engage audiences with the active nature
of seeing and meaning making, it may engender defamiliarizing experiences.²¹, ²²
Furthermore, visually indeterminate art prolongs perception through the combination of “apparently
detailed and vivid images resist identification.”²³ The generated latent space walk videos thus
present strange tools trained using a highly recognisable visual dataset, which now resist specific
identification via the GAN’s involvement. Whether this resistance of identification invokes the active
nature of seeing, or creates experiences of heightened perception is unclear—does coming into the contact
with the work go beyond strange? Do these splodgey, blurry tools engender an active awareness or reflection on
the symbiotic nature of tools and technology?
Product catalogue
Accompanying the tool wall are doctored product catalogues which have been inserted with images of tools
generated by the GAN. Using Photoshop, the original composition of the catalogue was closely followed as the GAN
generated tools were added based on their perceived visual similarities. The goal was to create a catalogue that
could be believably passed off as real at first glance.
The resulting catalogue is a mixture of the original text, lifestyle images, prices, and generated tools. This
presentation results in a double-take effect. Unlike the 3D printed tools which are outwardly alien or strange
by design, in this presentation, they almost look like real tools inside of a real catalogue at a cursory
glance. The context the product catalogue provides—the text, price tags, product descriptions, branding
and lifestyle images—almost act as visual vouchers for authenticity of these generated tools.
Returning to defamiliarization, Shklovsky positions the purpose of art as to “make forms difficult, to
increase the difficulty and length of perception because the process of perception is an aesthetic end in itself
and must be prolonged."²⁴ Thus, the work engages surprise, understood as violation of
expectation²⁵, and appropriation as potential strategies of defamiliarization. By appropriating the
catalogue and inserting familiar, yet strange tool-like artefacts, the doubletake causes an increase in
perception, calling us to look a little closer and make sense of the nonsensical tools.
In this, the work explores whether the surprise and absurdity of the nonsensical catalogue provides space to
investigate the socially coded meanings present in the subject matter that are familiar and normalised. For
example, does the messaging and imagery surrounding Father’s Day become less natural or familiar through
the introduction of the generated tools?
Conclusion
As the practice of generating images using ML becomes increasingly familiar, we need to be aware and
considerate of the ways it may naturalise or reinforce worldviews, categories, and ideas via this engagement.
However, just as ML has the ability to be conservative, to narrow in on the dataset and to perpetuate the ideas
within—so too can artists employ it to disrupt, investigate and defamiliarize the known.
Through the documentation and analysis of my work, Generated Tools (2021), this paper presents an
experimental approach to generating art with ML to create defamiliarizing experiences. As a part of this
discussion, I have explored how artists can engage with visual indeterminacy and the structural logics of ML
systems, as a method for disrupting the assumed, and reapproaching the known with fresh perspective.
Furthermore, through the analysis of the 3D prints, projected latent space walk videos and doctored product
catalogues, I have investigated how the presentation of ML generated images can create, alter, and deepen the
defamiliarization experience.
In this I am interested in how we can ‘make strange’ in ways that matter. As ML presents unique
opportunities for defamiliarization—how can artists defamiliarize in ways which persist beyond the initial
encounter? How can we blur, splodge and surprise with ML in ways that recover both the subjects of datasets and
technologies themselves from the automatism of perception and use?
Acknowledgements
This project was the first author’s (Chloe McFadden) honours project supervised by the second author
(Oliver Bown); hence the paper is written in the first person singular.
References
1 I. Hwang, M. Guglielmetti, V. Dziekan, "The Familiar": technology-being-with-us," Usage
guidelines, 2016, p.65.
2 É. Brangier, S. Hammes-Adelé, Beyond the technology
acceptance model: Elements to validate the human-technology symbiosis model, in International Conference on
Ergonomics and Health Aspects of Work with Computers, Springer, 2011.
3 J. Navarro, P.A. Hancock, Did Tools Create Humans? Theoretical Issues in Ergonomics
Science, 2022, 1-27.
4 Kate Crawford, Trevor Paglen, "Excavating AI: the politics of images in machine learning
training sets," AI & SOCIETY, 2021, doi: 10.1007/s00146-021-01162-8.
5 Harman Graham, "Technology, objects and things in Heidegger," Cambridge journal of economics 34,
no. 1, 2010, 17-25.
6 J. Navarro, P.A. Hancock, Did Tools Create Humans? Theoretical Issues in Ergonomics
Science, 2022, 1-27.
7 É. Brangier, S. Hammes-Adelé, Beyond the technology acceptance model: Elements to
validate the human-technology symbiosis model, in International Conference on Ergonomics and Health Aspects of
Work with Computers, Springer, 2011.
8 B.C Bruce, M.P. Hogan, The disappearance of technology: Toward an ecological model of
literacy, in Writing in a Technological World, 2019, Routledge, 191-207.
9 V. Shklovsky, Art as technique [1917], The Critical Tradition: Classic Texts and Contemporary
Trends, New York, 2007, 3, 774-784.
10 L. Crawford, Viktor Shklovskij: Différance in Defamiliarization, Comparative
Literature, 1984, 209-219.
11 D.P. Gunn, Making art strange: a commentary on defamiliarization, The Georgia Review,
1984. 38(1), 25-33.
12 R. Bellanova, A.R. Sætnan, How to Discomfort a Worldview?: Social Sciences,
Surveillance Technologies, and Defamiliarization, in Science, Technology, and Art in International
Relations, Routledge, 2019, 29-39.
13 L. Starkand, K. Crawford, The work of art in the age of artificial intelligence: What
artists can teach us about the ethics of data practice, Surveillance & Society, 2019. 17(3/4),
442-455.
14 D.P. Gunn, Making art strange: a commentary on defamiliarization, The Georgia Review,
1984. 38(1), 25-33.
15 Aaron Hertzmann, "Visual indeterminacy in GAN art," Leonardo 53, no. 4, 2020, 424-428.
16 P. Yurman, A.V. Reddy, Drawing Conversations Mediated by AI, in Creativity and
Cognition, 2022.
17 J.J. Benjamin, et al, Machine Learning Uncertainty as a Design Material: A
Post-Phenomenological Inquiry, in Proceedings of the 2021 CHI Conference on Human Factors in Computing
Systems, 2021.
18 Karras Tero, Samuli Laine,Miika Aittala, Janne Hellsten, Jaakko Lehtinen, Timo Aila, "Analyzing
and improving the image quality of stylegan," In Proceedings of the IEEE/CVF conference on computer vision and
pattern recognition, 8110-8119, 2020.
19 M. A Boden, Creativity and art: Three roads to surprise, Oxford University Press, 2010, 63.
20 Graham Harman, "Technology, objects and things in Heidegger," Cambridge journal of economics
34, no. 1, 2010, 17-25.
21 D. Gamboni, Potential images: Ambiguity and indeterminacy in modern art, Reaktion Books, 2002.
22 R. Pepperell, "Seeing without objects: Visual indeterminacy and art," Leonardo 39(5), 2006,
394-400.
23 A. Ishai, et al, "Perception, memory and aesthetics of indeterminate art," Brain Research
Bulletin 73(4), 2007, 319-324.
24 V. Shklovsky, "Art as technique," Literary theory: An anthology 3, 1917, p.16.
25 M. L Maher, et al, Computational models of surprise in evaluating creative
design, Proceedings of the fourth international conference on computational creativity, Citeseer,
2013.
Ecologies of Thought: Generative Art as a Collaborative Research Methodology with Guarani and Kaiowá
Indigenous Communities
MatheusdaRochaMontanari
University of São Paulo, Polytechnic University of Valencia, Multimedia Anthropology
Lab (UCL). São Paulo, Brazil/ Valencia, Spain matheusrmontanari@gmail.com
Abstract
The Ecologies of Thought project aimed to reconceptualize the relationships between ecological and technical
knowledge, seeking an epistemological understanding that pushed beyond nature vs. culture divides. More than a
conceptual and theoretical proposal, which was based on the technodiverse notion of cosmotechnics, the project
developed practical and experimental methodologies in collaboration with the Guarani and Kaiowá Indigenous
communities of Brazil to further this investigation. With an international and interdisciplinary partnership, we
researched relationships between sounds and plants in indigenous cosmology and the ways in which these
relationships can help us to more deeply understand a notion of ecology that is based on the poetics of care.
With that, we created a series of collaborative virtual reality experiments using generative art processes,
which proved to be an interesting methodology of artistic creation as research.
Keywords
Indigenous Communities; Generative Art; Ecology; Virtual Reality; Anthropology; Cosmotechnics; Digital Art; Art
as Research; Multimedia Anthropology; Guarani Kaiowa.
Between March and July 2022, the Ecologies of Thought (1) project established an
international and multidisciplinary partnership between Indigenous communities of Brazil, artists,
anthropologists, botanists, and audiovisual producers to investigate Guarani and Kaiowá ecological thought
through the relationships between sound and plants. The project proposed dialogues among different types and
conceptions of technology, from chanting and the traditional cultivation method, to the use of microcontrollers
and data analysis.
This project is part of the Guarani and Kaiowá Virtual Museum (2), which develops a
virtual reality museum curated by the community elders and shamans – Nhandesys (female), and Nhanderus
(male) in partnership with the Multimedia Anthropology Lab at University College London.
Since the beginning of the project, the community has been challenging and redefining the notion of a "museum,"
the extra-human relationships among plants, animals, instruments, and spirits present us with a complex
challenge: how to maintain a Guarani and Kaiowá cosmotechnical fidelity in the museum?¹ That is, how
to enable the indigenous cosmology, and its technologies, not only to inhabit the virtual space but to build it
as a tekoha, a territory, that allows the exercise of the Guarani and Kaiowá way of being and
living, teko.² That means, how to build a digital Guarani and Kaiowá tekoha?
To our surprise, in the first test of the museum presented to the community with the virtual reality headset,
two Nhandesys, experienced chanters, saw Jaras, spirits, in the virtual space. From this encounter, we
began to investigate different ways to explore the spiritual and cosmological aspects, which are so present in
the Guarani and Kaiowá ecology of thought, within the museum. The relationship between sound and plants
proved to be a fertile field for this research. We began to learn more from the community elders and apprentices
who introduced us to chanting as a cultivation technology: a physical and spiritual dialogue between the plant,
its spirits, and the chanter. To chant is to care for, and the poetics of care is essential for the development
of a healthy territory, be it physical or digital.³
We learned that instruments are also people. And that the Guarani and Kaiowá notion of preservation is
different. For example, if the instrument is kept and not used, even if in a collection, it loses its ability to
produce sound because it loses its ability to create connections when it is removed from its communication
network. Therefore, it became evident the need to explore different ways to activate the collection we have been
building for the museum.
To this end, we developed an experimental strategy. We focused on the creation of virtual reality worlds that,
using generative art processes, explored the Guarani and Kaiowá cosmology from its cosmotechnical aspect.
We used sounds and traditional elements, such as the Chiru, the sacred stick that supports the world, and the
white corn, which is inhabited by one of the most important spirits (Jakairá), to investigate fundamental
elements of the cosmology through an experience, rather than a narrative
representation.⁴ In light of the urgent need to revise and rethink human-ecology relationships on a
global scale, while acknowledging the new epistemic possibilities of data capture and analysis technologies, we
collaborated with Guarani and Kaiowá community members to explore the different and creative kinds of data
that these devices afford, and the ecological knowledge they make possible.
The Guarani and Kaiowá
Mato Grosso do Sul is a state in the mid-western region of Brazil, bordered by Paraguay and Bolivia, in a
transition region between three biomes: Atlantic Forest, Cerrado, and Pantanal. It is the State with the
second-largest indigenous population in the country and the largest outside the Amazon. The Guarani
Ñhandeva and Kaiowá population of the region in 2021 is estimated to be 63.5 thousand people. At the
same time, the State has the second-worst land distribution index in the country, with large properties (>
1,000 ha) occupying 83% of the total area. Thus, it is not surprising that the region is a scenario of serious
conflicts between landowners and indigenous people, especially in the area of cattle ranching, sugarcane, corn,
soybeans, and eucalyptus plantations.⁵
The Guarani and Kaiowá are Guarani speakers, from the Tupi-Guarani linguistic trunk. They have a strong
ancestral territorial bond, both physical and spiritual, which guides their struggle in an especially organized
way since the large assemblies beginning in 1979 (Aty Guasu). These articulations occur mainly between
the religious leaders, Nhanderus and Nhandesys, since the community considers that the simultaneous performance
of the religious ritual (jeroky) "is fundamental to recover the dialogue with the invisible beings and
the guardians of the ancient tekohas," that is, for the recovery of the historical territories of their people,
it is also fundamental to recover the extra-human relations developed in the territories, whether they are
animal, vegetable, and/or spiritual.⁶
These relationships are directly connected to chanting and praying. The chant, besides a dialogue, is part of
the materiality of the world: "In the understanding of the spiritual leaders, the singularities of the physical
world inevitably need singing to continue their continuous existence, otherwise the world will gradually
end."⁷ This is one of the reasons we choose sound, specifically chant, as a source of investigation of the
ecological relations in Guarani and Kaiowá thought.
Artistic creation as an interdisciplinary collaborative methodology
The Guarani and Kaiowá cosmology have a great degree of complexity and diversity, even in the same family
trunk, some stories may vary between one group and another. Moreover, the narratives are full of recursion and
non-linearity, in which each detail branches off into another story, that can take days to be told in the
traditional oral form. So, it was clear from the beginning of the project that there would be limitations on the
form and amount of information we could present. At the same time, we did not want to make a reductionist
narrative translation that would create a shallow representation of the generous exchanges that the project made
possible.
Besides that, the diverse points of view and areas of knowledge involved in the project configure,
simultaneously, its richness and complexity. The challenge of combining shamans, scientists, and artists in the
same project revealed the power of the digital medium as a research tool in itself, and not only as a form of
scientific dissemination. More than that, we set out to develop research in visual arts, rather than
research about visual arts. That is, to reaffirm the processes of creation as valid ways of
researching, thinking, and producing knowledge about the world.⁸ In this sense, artistic creation acts in
the project as a collaborative methodology capable of providing dialogues and investigative interactivity: the
creation processes allowed for the creation of a two-way exchange, in which collaboration affected not only the
production but the understanding and direction of what was being produced.
In this project, art, which occupies a liminal place, is not limited to an art-technology extension of
scientific knowledge. We seek to embrace the mba'e kuaa, the Guarani and Kaiowá technical
knowledge and know-how. In this proposal, we guide ourselves with the concept of cosmotechnics developed by Y uk
Hui. That is, to rediscover the diversity of technical thought beyond the Greek notion of techné:
Technology is not anthropologically universal; its functioning is assured and limited by particular cosmologies
that go beyond mere functionality and utility. Thus, there is no single technology, but a multiplicity of
cosmotechnics.⁹
Generative Art and the Guarani & Kaiowá Cosmology
Generative art can be defined in different degrees of complexity, it is associated with a system that has some
degree of autonomy, in which the artist and the system exercise a series of operations that result in the final
work.¹⁰ Seeking ways to engage with the mythology and respect their characteristics, generative art
methodologies seemed to not only fit with the ontological qualities of the Guarani and Kaiowá cosmology
but to privilege an investigation and audiovisual production that could maintain a cosmotechnical fidelity.
Since both, the Guarani and Kaiowá cosmology and generative art are nonlinear emergent processes with
repetitive elements that are in constant transformation. Working in that direction, we created a series of six
experiments, and in this article, we will be presenting one of them.
Jakairá: white corn, bees, and the maintenance of the world
According to the Kaiowá myth, the deity Jakairá is responsible for the creation of the first
roçado (food garden). Through his wisdom, Jakairá created the white corn, which is at the top of the
food hierarchy, and, from it, all other agricultural products were created. Therefore, Jakairá and white
corn are responsible for the creation and maintenance of almost all food on Earth. Its cultivation follows a
series of rules, from dietary restrictions of the growers to the use of chants to cheer the corn and its
baptism.¹¹ The corn baptism ritual, called Jerosy Puku, is one of the most important
Kaiowá rituals. Through it, the corn becomes fit for consumption by the community in a healthy way, since,
besides being a physical food, it is also considered a spiritual food. The baptism is also a moment of
reaffirmation of the Kaiowá way of being, strengthening the human and extra-human (spirits, plants,
animals) relationships of the community.¹² This specific type of corn is currently in danger of
extinction due to the environmental threats it suffers, the heavy use of pesticides in the GMO corn plantations
that surround the communities, and the consequences of climate change in the region.
Our first experiment arises precisely from this extrahuman ecology. In one of our ethnographic sessions on the
importance of corn and its relationship to sound, we learned that corn has its own sound that is not audible to
all people. However, there is a specific moment when the sound of corn can be heard by everyone. The community
members told us that when the corn, still green (avati kyry), begins to mature and get ready for harvest, it
produces a smell that attracts bees so that "they can make the sound of the corn," warning the cultivator that
it is almost time to harvest. As talking about the white corn is also talking about the beginning of life and
the food that sustains it, this pre-harvest moment, when the bee is attracted to the corn to make its sound,
seemed emblematic to us to investigate an instant of the creation of the world according to the Guarani and
Kaiowá cosmology. This moment that seems, at first sight, to deal with a multispecies relationship between
the plant and the bee, when further investigated, reveals, also, to be a relationship between the spirit owner
of the corn and its cultivator¹³. To continue this investigation, we developed a generative art
experiment using the TouchDesigner programming language: a visual programming language in nodes for the
creation of real-time interactive multimedia content. We started with the image of a white cornfield before the
harvest and analyzed the light incidence in its different areas. From this data analysis, we created a relief
map that was applied to an ellipse for the construction of a terrain. The darker areas of the image created
depressions, and the lighter areas, elevations. Finally, we added two noise layers, one algorithmic, and another
from the sound analysis of the frequencies produced by the bees. The sound frequency analysis controls the
algorithmic noise, which generates motion on the terrain. With that, we exported a 360o video for VR headsets.
In this way, we developed a virtual reality experience in which the ecological relationship between the corn and
the bee produces the creation of a generative world based on the analysis of image and sound data from the
field, guided by the Guarani and Kaiowá cosmology (figure 1).
Even without a naturalistic production of the image, and perhaps precisely because of this characteristic, the
community approved the experiment. Once the step-by-step construction of the generative program was presented,
there was consensus that the experiment had true Guarani and Kaiowá characteristics, mixed with
non-indigenous technology. From this experiment on, the community began to direct the following more directly,
choosing images and elements that should be used for the continuation of the project.
Conclusion
The Ecologies of Thought project aimed to reconceptualize the relationships between ecological and
technical knowledge, seeking an epistemological understanding that pushed beyond nature vs. culture divides.
More than a conceptual and theoretical proposal, which was based on the technodiverse notion of cosmotechnics,
the project developed practical and experimental methodologies in collaboration with Guarani and Kaiowá
communities to further this investigation.¹⁴ With an international and interdisciplinary partnership,
we researched relationships between sounds and plants in indigenous cosmology and the ways in which these
relationships can help us to more deeply understand a notion of ecology that is based on a poetics of care.
Thus, we were able to bring together poetic and technical processes to propose a mixture of knowledge that
beyond an explanation or representation of one form of knowledge into another, develops an experience of its
own. This allows us to fight an idealized naturalistic representation of indigenous culture, which fails to
understand indigenous thought and technologies as valid ways to build and think of a cosmotechnic for our time.
More broadly, this project seeks to undermine the romantic and colonial notion of preservation, which does not
consider the indigenous presence a digital presence, able to act and think about technical processes since its
conception. This means, beyond an illustrative representation of what is believed to be the material and
immaterial cultural heritage of the community, to collaboratively build a digital-indigenous territory, a
digital Guarani and Kaiowá tekoha where a new technological ontology is established, in a way
that the very conception of contemporary technology is affected by indigenous cosmotechnics.
In conclusion, the epistemic research proposed and developed during the project revealed to us the potential
that this collaborative methodology possesses to go beyond epistemological investigations, enabling us to
revisit ontological questions of technology. Collaboration, interdisciplinarity, and the investigation of
creative processes—rather than the proposition of representations—allows us to arrive at questions
related to the level of existence, so that, in the anthropocenic era of the absence of futures, we can
compose a cosmotechnic for a viable, virtual and actual, reality, based on a poetics of care and more-than-human
relations.
Acknowledgments
We would like to acknowledge the University College London Global Engagement Fund which contributed to the
partnership among the Multimedia Anthropology Lab at University College London, the Kunangue Aty Guasu, IDAC,
The School of Botany, The University of São Paulo, The Federal University of Santa Catarina.
We would like to thank and acknowledge the generosity of the Nhandesys and Nhanderus whom we have been working
with during these years. Nhandesy Neuza Alziro, Nhandesy Ivone, Nhandesy Fausta Solano Mendosa e Nhandesy
Roselí Aquino. Nhanderu Tadeu Romeiro Freita, Nhanderu Valdomiro Oswaldo Aquino. Also, the team of
UCL’s Multimedia Anthropology Lab and the Kunangue Aty Guasu: Valdineia Aquino, Scott Hill, Luan Iturve,
Germano Alziro, Jaqueline Gonçalves, Raffaella Fryer-Moreira, Fabi Fernandes, Gerard Oliver, Adriana
Boloc, Stephanie Ferraz, Clara Ciqueira, Luiza Braga and MAL’s translation team. And our partners Karola
Braga, Anderson Santos and Caetano Sordi.
This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de
Nível Superior – Brasil (CAPES) – Finance Code 88887.716693/2022-00.
1 Yuk Hui, The question concerning technology in China: an essay in cosmotechnics,
Falmouth, Urbanomic, 2018.
2 Tonico Benites, “Trajetória de luta árdua da articulação das
lideranças Guarani e Kaiowá para recuperar os seus territórios tradicionais tekoha
guasu,” Revista de Antropologia da UFScar, Vol. 04, No. 02, accessed October 14, 2022,
https://doi.org/10.52426/rau.v4i2.83
3 Matheus Montanari and Gilbertto Prado, “From vigilance to vigil: an introduction to an
alternative paradigm for technology, art, and life,” Diffractions, Vol. 01, No. 05, accessed
October 14, 2022, https://doi.org/10.34632/diffractions.2022.10194
4 Fabio Mura, “A Trajetória dos Chiru na construção da
tradição de conhecimento Kaiowá,” MANA, Vol. 16, No. 01, accessed October 14,
2022, https://doi.org/10.1590/S0104-93132010000100006
5 Anderson de Souza Santos, Luiz Henrique Eloy Amado and Dan Pasca, “"É muita terra
pra pouco índio"? Ou muita terra na mão de poucos? Conflitos fundiários no Mato Grosso do
Sul,” Instituto Socioambiental, accessed October 14, 2022,
https://acervo.socioambiental.org/acervo/documentos/e-muita-terra-pra-pouco-indio-ou-muita-terra-na-mao-de-poucos-conflitos
6 Tonico Benites, “Trajetória de luta árdua da articulação das
lideranças Guarani e Kaiowá para recuperar os seus territórios tradicionais tekoha
guasu,” Revista de Antropologia da UFScar, Vol. 04, No. 02, accessed October 14, 2022,
https://doi.org/10.52426/rau.v4i2.83,171
8 Sandra Rey, “Por uma abordagem metodológica da pesquisa em artes visuais,” in
O meio como ponto zero, ed. Bianca Brites and Elida Tesser, Porto Alegre: Editora da UFRGS, 2002,
123-140.
9 Yuk Hui, Tecnodiversidade, São Paulo, Ubu, 2020, 25.
10 Philip Galanter, “Generative Art Theory,” in A companion to digital art,
ed. Christiane Paul, Hoboken, John Wiley & Sons, 2016.
12 Izaque João, “Jakaira Reko Nheypyrũ Marangatu Mborahéi: Origem E
Fundamentos Do Canto Ritual Jerosy puku Entre Os Kaiowá De Panambi, Panambizinho E Sucuri’y, Mato
Grosso Do Sul”, Master’s. diss., Programa de Pós-Graduação em História da
Faculdade de Ciências Humanas, Programa de Pós-Graduação em História da
Faculdade de Ciências Humanas, 2011.
13 Carlos Fausto, “Donos demais: Maestria e Dominio na Amazônia,” MANA,
Vol. 14, No. 02, accessed October 14, 2022, http://dx.doi.org/10.1590/S0104-93132008000200003
14 Yuk Hui, The question concerning technology in China: an essay in cosmotechnics
Author Biography
Matheus Montanari is a Ph.D. Candidate at the University of São Paulo in Visual Arts, and external
researcher at the Polytechnic University of Valencia. He develops works at the intersection of art, science
and philosophy, investigating ways to rethink technology after art. He is interested in cosmotechnical
diversities, and in combining decolonial ecological thinking with technical know-how. Recently he has been
developing his practice in collaboration with indigenous Guarani and Kaiowá communities in Brazil. He
has exhibited his work in Brazil and abroad, in countries such as: Argentina, Portugal, Austria, Croatia,
China, Spain and Italy. He was awarded the web art category prize of the 67th Contemporary Art Salon of
Paraná.
Emerging Strategies “Under The Bay” in AR/XR
LisaMoren, TsvetanBachvaroff
University of Maryland Baltimore County (UMBC), Institute of Marine and Environmental
Technology (IMET), University of Maryland Center for Environmental Science (UMCES) Baltimore, Maryland
lmoren@umbc.edu, bachvarofft@umces.edu
Abstract
“Under the Bay” is an augmented reality project where anyone can use their cell phone like
a microscope and reveal invisibilities in our world and marine life. When they do a series of animated stories
between humans and non-humans emerge. Images, sounds, and stories are affected by live data streamed in from
sensors located in the largest estuary in North America. Sensors in the Chesapeake Bay relay live pH, oxygen,
temperature, etc. (figure 9). Similar to the water itself, color, speed, audio fluctuate with the water and
marine life, making “Under the Bay” a data-driven narrative with eight scenes that tell a
story of a world beneath the marine surface, and the exciting but frail health of estuaries and oceans
worldwide. The two projects discussed here, “Under the Bay” (2022) and “What
is the Shape of Water?” (2020), are part of Lisa Moren’s series of cross-species artworks
aimed at diminishing human-centered exceptionalism. The collaborations began in 2019 when Lisa was the inaugural
Artist-in-Resident at the Institute for Marine and Environmental Technology (IMET). There, she met researcher
and marine biologist, Dr. Tsvetan Bachvaroff (Tsetso) and the two immediately shared a like-minded vision to
develop a project that exemplified phenomenal exceptionalisms in micro-organisms. In this paper we argue that
novel strategies in nature emerge when a complexity of matter is intermingled with conditions of
differentiation. We explain and identify differentiation in art and architecture, symbiosis in biology, and the
“wobble” in physics as core principles for new forms and creative strategies to emerge. The outcome
is focused on the unusual and significant diversification of dinoflagellate microbes in the Chesapeake Bay and
oceans worldwide. Tsetso directed the live organisms, science and data analysis for the augmented reality
project. Stories are written and told by Lisa, who produced and art directed the animation and AR scenes. The
sound score is by electronic composer Dan Deacon. Dr. Marc Olano led the software engineering and development
with John Boutsikas, for the AR app in IOS and Google Play.
Keywords
Augmented Reality, AR, bio-art, data-driven narrative, emergent strategies, Tao Te Ching, water, symbiosis,
dinoflagellates, Chesapeake Bay, estuary, marine biology, performance, microbes, media art, podcast,
experimental narrative, bio-architecture, Lynn Margulis, Ted Nelson, Jane Bennett, Theodor Schwenk, data-driven
music, philosophy.
In our collaborative work, we borrow the idea of “emergent strategies” to consider strategies in
nature that describe phenomena driving new emerging forms. Inspired by Octavia Butler, Adrian Maree Brown coined
the term “Emergent Strategy,”¹ as “a framework for resistance that is
rooted in the miracles of nature, decentralized, collective leadership, and personal, relational,
organizational, and movement-wide transformation.”² We focused our strategies based in part on
Brown’s idea that anomalous strategies in nature can be a model for the benefit of human communities, from
species longevity to social change. Lisa’s second influence was her participation in a Taoist meditation
group where she studied and meditated on the Tao Te Ching.³ These meditations blended with her observations
of nature, primarily starling bird murmurations described in Scene 05 \\ Lava Lamps in the Sky.
The connections between emergent strategies, organic differentiation, and a Taoist social order became the basis
of this art and science collaboration.
Emergent Strategies
We find the origin of strategic emergence by looking at gene evolution in early microbes. The Darwinian idea of
natural selection has been combined with genome inheritance from two parents into what is now called the "modern
synthesis." While this synthesis has excellent explanatory power, some observations are not a perfect fit to
this synthesis. For example, photosynthesis is scattered across multiple kingdoms, but the process of
photosynthesis itself is unlikely to have been invented multiple times. The solution to this paradox required
two steps (figure 4). First, Lynn Margulis in 1967 proposed that eukaryotes are a combination of two cells from
different bacterial lineages. The first lineage is the archea bacteria which engulfed another bacteria cell that
contributes the mitochondrion. Mitochondria are responsible for the exchange of oxygen and carbon dioxide,
eventually this will be the foundation for breath itself. The mitochondrion is then a "cell within a
cell."⁴ This symbiotic combination of two previously distinct cells into eukaryotes created a new
complexity of features (figure 4). The first scene in our project Scene 01 \\ Origin Stories,
describes that before Margulis, any scientific reference to a symbiosis hypothesis was taboo. To accept
symbiosis, science had to imagine a scenario where a species swallows a giraffe’s eye to acquire longer
eyelashes successfully, but then, they pass those eyelashes onto their offspring. Symbiosis was once a fairy
tale.
But once the formation of eukaryotes via symbiosis of two lineages was accepted, then the symbiosis concept was
extended to include photosynthesis in plants and algae. In this scenario, a previously free-living
cyanobacterium was engulfed by a eukaryotic host (figure 5). The host now contains both mitochondria and
photosynthetic chloroplasts. This new host obtains energy from sunlight via the chloroplast and exploits that
energy in the mitochondria. This innovation ultimately leads to land plants and nearly all photosynthetic
eukaryotes (figure 4).
Sarah Gibbs extended the idea of symbiosis to explain why photosynthesis is scattered across multiple kingdoms.
In this third scenario, one cell with chloroplasts is consumed⁵ by a second, but only the chloroplasts are
retained. This provides access to photosynthesis and explains how photosynthesis could have spread across many
diverse eukaryotic groups.
Therefore, an individual can be shaped by two patterns of evolution: the familiar and dominant expression of
parental genes via vertical descent and a second, less familiar, pattern of horizontal gene transfer from
endosymbiosis (figure 4).
The dinoflagellates are from the eukaryote kingdom and a perfect case study to demonstrate both symbiosis and
the global importance of small eukaryotes to our planet. In global estimates of oceanic photosynthesis
dinoflagellates are one of the top oxygen producers.
The oceans produce oxygen at a scale similar to the contributions from the Amazon rainforest. This is
underscored by modern sequencing which places dinoflagellates as second in abundance to animals and first in
diversity in oceanic surveys. Dinoflagellates also include a wide array of life strategies such as built-in
harpoons used to capture prey.⁶ Other strategies include a symbiotic photosynthetic lifestyle as the algae
found in coral reefs use internal and external types of parasitism, and consuming food for energy. The term
mixotrophy describes the role of many photosynthetic dinoflagellates, which can be both photosynthetic
(autotrophic) and consume prey (heterotrophic).⁷
Mixotrophy and diversity in the ocean combined with the concept of symbiosis represented by Gibbs and Margulis
provides many instances of emergent strategies. For example, the dinoflagellate Dinophysis feeds on
ciliates demonstrating how the chloroplast could be consumed intact from prey by injecting a built-in
“straw-like method” to acquire, or suck, chloroplasts from its prey.⁸ Roughly nine events have
occurred where photosynthetic dinoflagellates contain chloroplasts borrowed or gained from essentially every
available lineage. And this list does not include dinoflagellates with elaborate structural features that
contain external symbiotes such as Ornithocercus.
Within the Dinophysis group there are different types of photosynthesis or symbioses, including the
‘gardening’ Histioneis (figure 6) and Ornithocercus (figure 7), the
‘heterotrophic’ Dinophysis that sucks out chloroplasts, and at least two other types of
internal symbionts. Ornithorcercus will host cyanobacteria in a birdcage-like crown, a strategy that
not only stores nutrients for on-going consumption, but the chains of cyanobacteria reproduce itself creating a
“fruit-on-the- vine-like garden” energy supply (figure 7).
Symbiosis has allowed these the single-celled dinoflagellates to diversify over hundreds of millions of years
into thousands of species that will clearly outlive humans. Therefore, diverse reproductive, energy consumption,
and other strategies assisted the dinoflagellates in obtaining longevity that humans can only dream of. For
example, the dinoflagellate Ceratium can both cell divide and mate for optimum reproduction benefits.
In Scene 01 \\ Origin Stories the viewer will see images of the Ceratium that cell
divide having only one trailing flagellum (a propeller-type tail). However, an identical Ceratium has
two flagella because they had two parents that mated (figure 3).
Other strategies we named included organic differentiation in the structure of the cell walls of the
dinoflagellates. These complex forms, such as Voronoi patterns, use less matter to produce structures that are
lighter in weight than any objects human engineering could produce based on Cartesian principles in
manufacturing. Differentiation can be exemplified in the exoskeleton of a crab or lobster but is most visible in
the repeating hexagon pattern of a turtle shell, where the hexagon shapes repeat, but not perfectly. It’s
in that imperfection, crookedness, or wobble, that creates more strength with less matter. Dinoflagellates,
diatoms, and other microbes also have this efficient design principle of differentiation. Scene 04
\\ Crooked Shelters (figure 8) describes how such differentiation influenced the largest 21st century
algorithmically designed and digitally fabricated architectural form in Seville, Spain, the Metropol
Parasol.⁹ The pavillion’s aerial view appears as a mushroom blooming throughout the gridded city. A
similar architectural example is in Stuttgart, Germany, where one of the algorithmic pavilions based on
differentiation in nature was so strong and lightweight, it blew away.¹⁰
Water Projects
In our first attempt to create a public project for an open house at IMET, we created an artwork demonstrating
the microbial strategy of making light through the bioluminescence of the dinoflagellate (Pyrocystis).
Eventually, the public display was set up as a ceiling tank hooked up to a Max/MSP, Arduino, and AV system for
the Light City Festival in Baltimore’s Inner Harbor. The audience entered a pitch-black room with a large
ceiling tank holding millions of invisible dinoflagellates. The system worked with a voice-activated trigger so
that when a participant spoke into a microphone, for example, “What is the Shape of
Water?” the microscopic organisms above them answered the question in turbulent shapes of
blue bioluminescence (figure 9). Originally, this was influenced by the mesmerizing organic order and
differentiation in the murmuration patterns of starling birds.¹¹ However, Tsetso’s colleague,
Dr. Al Place who studies the motion behavior of dinoflagellates, says that the flocking behavior of the
dinoflagellates will unlikely look as organized as the starlings. Instead, the water agitation produced
turbulent patterns more akin to the wobble, the crookedness or what philosopher Jane Bennett calls murmuring
messiness.¹²
“What is the Shape of Water?” began as a somewhat expedient strategy of displaying
bioluminescence in an art project that took over a year to fully realize. It was an unrealistic approach to
create art installations for the more than two dozen strategies we identified for potential exhibition. We
instead turned to storytelling and produced an augmented reality (AR) project with eight scenes, approximately
10 minutes per scene. Lisa took the strategies we identified for the project and built narratives that meandered
through purposefully fragmented topics related to microbes, water, Taoist meditation, architecture, monuments,
and politics.
The AR project became “Under the Bay” and it’s trailer¹³ illustrates the
animation, narrator, and incoming data. For the data, we worked with the Maryland Department of Natural
Resources (MD DNR) to siphon data from sensors already installed in the largest estuary in North America, the
Chesapeake Bay. In total 36 parameters stream into the project from six locations in the Bay. Locations are from
the Delaware border to Washington, DC, the Eastern Shore of Maryland, and Baltimore City (figure 10). The
parameters pH, temperature, oxygen, salt, chlorophyll, and turbidity (clarity) are updated every 15 minutes.
These parameters then affect the animation’s color, speed, and scale, along with the narration and music
composition. For instance, when the oxygen levels in the Bay are of good quality, the narrator’s voice
sounds normal. However, when the water is anoxic, with little or no oxygen, the voice becomes choppy,
fragmented, as if choking. In this way, the story, images, and sounds change from day to night and season to
season for an ongoing collaborative narrative with the Bay water. The unpredictable variability of the incoming
data becomes an authentic wobble created by the environment.
Listening to the Water
The project propaganda claimed the ambition of cross- species communication specifically “What if we
could hear what the water is saying?” To address skeptics to these claims, we’re borrowing
from water physicist Theodor Schwenk, as we do in the AR project, but also Jane Bennett¹⁴, who
references Graham Harmon, Bruno Latour, and Michel Serres’s The Birth of Physics.¹⁵
If we look at water as an object, we can argue that water is the largest object in the world. On the one hand,
the ocean contains essential elements, H2O, saline, and other matter on the periodic chart. We know that water
and gravity work together to form currents like pipes that braid in distinct patterns and that these flowing
pipes separate into differing physical data such as speed and temperature. These differences become visible when
encountering an obstacle, like a rock in a river, where we observe differentiation in the water shapes bulging
around the rock. Similarly, in the ocean, large and “long waves travel faster than short waves”
and overlap until the larger ones envelop the smaller ones, and the waves repeat the pattern
endlessly.¹⁶ But unlike early CGI waves, the real ocean’s patterns are not perfect or
predictable waves, their equilibrium billows, and exhales asymmetrically, and it’s in that asymmetry that
they wobble. Or, what Bennett calls, they produce a vibratory noise, where the force of repetition
starts to create a surge, an “irregular bombardment of circumstances,” especially when new
physical elements act as an obstacle such as a rock. Here, the current billows and exposes its diverse
temperatures, causing what Serres names a “cauldron of turbulence that thickens into lumps of
phenomena, and the bubbling swirl keeps those shapes upright... while the wobble produces variances of
noise.”¹⁷ A shape derives from the swirl that is both form and vibratory. To Serres, he
calls this vibratory noise “the fluctuating ado that is the strange substance of any discrete,
differentiated shape... (where) the multiplicity of the possible rustles in the midst of the forms
that emerge from it... It is restless matter... (a) percolation.”¹⁸ While much
of this noise dies like seedlings that don’t spawn, the intermingling currents and swirls, overlap with
enough force allowing forms to emerge from it.
While we do not think of water as having agency, free will, or decision-making abilities or even that its
elements are alive, in complex natural events, water does cause a multitude of events to happen, and is
therefore an actant. One of the most significant events that water enacts is sustaining and creating new life,
but also emerging shapes, and unique forms. In this way, water will begin by initiating an abundance of events,
such as a vortex or a whirlpool effect. Any time these elements react or affect one another, there is the
potential for something to emerge, including new life (figure 11).
In Scene 02 \\ Water Moving Around My Fingers, the microbes reflect Schwenks organisms that
take on the negative space of the vortex shape traveling in a reverse corkscrew spiraling up (figure
11).¹⁹ Therefore, when we say we can hear what the water is saying? It’s not that the water, as
pure physical compounds, produces a will or agency that desires to be heard. However, water’s natural
environment has reactions, interactions, and relationships with other phenomena and other physical materials
where unique consequences emerge as shapes, forms, beings, and blooms. Moreover, if we consider the incoming
parameters of the Bay water as a kind of alphabet — to use a human metaphor — the data does arrange
itself to describe a story of the Bay water’s emerging behaviors and forms. This communication is so hard
for humans to understand, especially to predict — that when the data reflects the formation of an algae
bloom, it’s often too late to hear the water telling us we are out of balance. Perhaps the data
acts like a Google knowledge engine, an algorithm anticipating the user’s thoughts when typing a partial
phrase into the search bar and displaying our presumed burning questions. However, in this case, the data
anticipates the thoughts of the Bay water. This concept is so significant that the scores of MD DNR sensors we
use to siphon data are funded solely to predict the emergence of algae blooms in the Chesapeake Bay.
“Under the Bay” observes how the Bay ebbs and flows over time to create an emerging
narrative driven by what the water is enacting as an emergence. We call this murmuration messiness,
listening to what the water is saying.
Conclusion
Once we name this process of emergence, we can apply it to the other scenes in the AR project, addressing
issues of differentiation and emergence not only in nature but also in creative ideas, forms, art, architecture,
monuments, and also politics, protests (we discuss emerging protests including BLM, the beheaded Columbus statue
ending up in the Bay, or the #metoo flocking of women in a bloom of pink hats — figure 12). This brings
Brown’s term emerging strategies full circle where strategies in nature can be a model for social
change. Again, our stories meander through these topics and phrases, like symbiotic waves engulfing one another
leaning on influences from the Tao Te Ching as much as science.
“Under the Bay” will be exhibited at the Peale Museum, in Baltimore, Dec 14 to Feb 1,
2024, for “Chamber of Wonders”, with a public panel on Jan 18. While “Under the
Bay” is freely available to download,²¹ the XR installation will include iPads
interacting with paintings, drawings, and assemblages for a unique museum experience. “What is the
Shape of Water?” and related works will also be on display.
“Under the Bay” Scenes 01-08 include: Origin Stories \\ Water Moving Around My Fingers \\
Chalky Faeries \\ Crooked Shelters \\ Lava Lamps in the Sky \\ Instrumental \\ Vaccine Blooms with Pink Hats \\
Turnover [I Can’t Breathe]
2 Andrea Ritchie, Invisible No More: Police Violence Against Black Women and Women of
Color, Beacon Press, Reprint edition, 2017.
3 Lao-Tzu, translated by Stephen Addiss, Stanley Lombardo, introduction by Burton Watson,
“Tao Te Ching”, Boulder, Hackert Publishing Company, 1993.
4 Lynn Margulis, “On the origin of mitosing cells," Journal of Theoretical Biology,
vol. 14, no. 3, 1967, 225–274.
5 Sarah P. Gibbs, “The chloroplasts of euglena may have evolved from symbiotic green
algae.” Canadian Journal of Botany, vol. 56, no. 22, 1978, 2883–2889.
7 Photosynthetic Fragilidium consuming a large Ceratium
https://www.youtube.com/watch?time_continue=1&v=KFFsnH
VlHyo&embeds_referring_euri=https%3A%2F%2Fmail.google.com%2F&embeds_referring_origin=https%3A%2F%2Fmail.g
oogle.com&source_ve_path=Mjg2NjY&feature=emb_logo
8 Dinophysis sucking the chloroplast from its prey
https://www.youtube.com/watch?v=cq3SBoCHPP4&t=3s
12 Jane Bennett, William Connolly, “The Crumpled Handkerchief.” Time and
History in Deleuze and Serres, Bloomsbury, London, UK, 2013, p.155.
13 “Under the Bay” trailer by Lisa Moren with Tsvetan Bachvaroff, 2022.
https://vimeo.com/796868197
14 Ibid, Bennett, Connolly.
15 Michael Serres, The Birth of Physics, Rowman & Littlefield International,
Ltd., 2018.
16 Theodor Schwenk, “Sensitive Chaos: The Creation of Flowing Forms in Water and
Air,” Sussex, Rudolf Steiner Press, 1996, 33.
17 Ibid, Bennett, Connolly. 157.
18 Ibid.
19 Ibid, Schwenk, p.44.
20 An eight-episode podcast version of “Under the Bay” is available on
Spotify, Google, and Apple podcasts. Links to podcasts and more information can be found on the project
website: http://lisamoren.com/underthebay.
21 Download the AR from Apple IOS or Google Play: https://apps.apple.com/app/id1641553491
Stefanie R. Fishel, The Microbial State: Global Thriving and the Body Politic, University of
Minnesota Press, 2017.
Sarah P. Gibbs, “The chloroplasts of euglena may have evolved from symbiotic green
algae,” Canadian Journal of Botany, vol. 56, no. 22, 1978, 2883–2889.
Douglas Kahn, “Energies in the Arts,” Cambridge and London, MIT Press, 2019,
2–46.
Lao-Tzu, translated by Stephen Addiss, Stanley Lombardo; introduction by Burton Watson, “Tao Te
Ching”, Boulder: Hackert Publishing Company, 1993.
Lynn Sagan, “On the origin of mitosing cells,” Journal of Theoretical Biology, vol. 14,
no. 3, 1967, 225–274, https://doi.org/10.1016/0022-5193(67)90079-3.
Theodor Schwenk, “Sensitive Chaos: The Creation of Flowing Forms in Water and
Air," Sussex, Rudolf Steiner Press, 1996.
Lars Spuybroek, Sympathy of Things: Ruskin and the Ecology of Design, Bloomsbury Academic,
2016.
Acknowledgements
This project is generously supported by the Saul Zaentz Innovation Fund at Johns Hopkins University and the
R.W. Deutsch Foundation. This project originated during an artist-in-residence supported by the Institute of
Marine and Environmental Technology (IMET), the Center for Innovation, Research, and Creativity in the Arts at
the University of Maryland, Baltimore County (UMBC). We’re grateful for the support from Maryland
Department of Natural Resources; John Boutsikas for his API and AR development, Danielle McPhatter and
Harvestworks DigitalMedia Art Center, the IRC and CAHSS, and Antoine Cayrol of Atlas V Immersive Experiences
in AR, XR, VR.
Production Team
Lisa Moren, Co-Principal Investigator, Producer, Art Director, Writer and Narrator; Dr Tsvetan Bachvaroff,
Co-Principal Investigator, Marine Biologist, Researcher and Data Anaylsis; Dan Deacon, Electronic Composer;
Dr. Marc Olano, Co-Principal Investigator, Lead Programmer; John Boutsikas, Programmer and Developer; Austin
Samson Modeler and Animator; William Forrest, Modeler, Animator, UI and Technical Artist; Woody Lissauer,
Voiceover Engineer and Male Narrator 1; Ruskin Nohe-Moren, Male Narrator 2; Aliyah Baruchin Copy Editor and
Fact Checker.
Author’s Biography
Lisa Moren is a multi-disciplinary artist who works with emerging media, bio-matter, public
space, AR and works-on-paper. She has exhibited her work at the Chelsea Art Museum, Creative Time, Drawing
Center (New York), Cranbrook Art Museum (Michigan) and Ars Electronica (Austria), Akademie der Kunste
(Germany), uShaka Museum (South Africa), and the Artists Research Network (Australia). She received the
National Endowment for the Arts award, is a Fulbright Scholar; a multi-year recipient of the Maryland State
Arts Council and CEC Artslink International, is a R.W. Deutsche Award recipient and a Saul Zaentz Innovation
Fellow in Film and Media at Johns Hopkins University. Her writing has appeared in Performance Research;
Visible Language; Inter Arts Actuel; New Media Caucus for “Algorithmic Pollution: Artists working
with Dataveillance and Societies of Control” and “CYBER
IN|SECURITY;” and her books on “Intermedia;” and Issues in
Contemporary Theory for “Command Z: Artists Working with Phenomena and Technology.” Lisa
Moren is a Professor of Visual Art at the University of Maryland Baltimore County (UMBC); is an Affiliate
Faculty at the Imaging Research Center (IRC) UMBC; and taught at FAMU and AVU in Prague; and the University of
California at San Diego (UCSD).
Dr. Tsvetan Bachvaroff’s research is focused on dinoflagellate evolution with special
emphasis on the parasitic dinoflagellates, using large-scale sequencing and phylogenetic methods to describe
the evolutionary history of different types of genes in dinoflagellates. He uses DNA sequence analysis from
data collection, assembly, annotation and phylogeny; has received numerous academic awards, including the
William Trager Award from the Journal of Eukaryotic Microbiology, International Society of Protistologists,
culture independent methods such as single cell PCR, sequencing, and sequence analysis; Establishing
dinoflagellate cultures. He received the Marsho award, Mid-Atlantic Section of the American Society for Plant
Biology, University of Maryland College Park, and the Chemistry Prize, Trinity School, New York. Tsvetan
Bachvaroff received his B.A. degree from Johns Hopkins University and Ph.D. from the University of Maryland
College Park. He was a Postdoctoral Fellow at the Center of Marine Biotechnology and subsequently with the
Smithsonian Institution. He is an Associate Research Professor for the Institute of Marine and Environmental
Technology (IMET) at The University System of Maryland.
Power and Resistance in Digital Degrowth
RaulNieves, JoanSoler-Adillon, EnricMor
Universitat Oberta de Catalunya (UOC) Barcelona, Spain dbsus4@uoc.edu,
jsoleradillon@uoc.edu, emor@uoc.edu
Abstract
Degrowth is increasingly gaining attention as an alternative model to the unfolding eco-social crisis generated
by industrial capitalism, though questions concerning digital technologies have yet to be addressed in degrowth
research. Among the movement of the (digital) commons, whose practices complement degrowth theory, one of the
research areas is the viability of such systems to release spaces from capitalism. As (digital) commons spaces
frequently revert to capitalist logic, we introduce the "technological dramas" model to encompass the reciprocal
and recursive technological production of political power by agonistic entities. We suggest that such a
techno-political perspective could contribute to better frame degrowth-related HCI research.
Planetary chances to restore a sustainable scenario are quickly running low. The UN questions whether we have
already passed the turning point.¹ The unsustainable resource throughput promoted by the industrial model
is (eventually if not yet) incompatible with the materially limited planetary boundaries.² Over the last
decades, scholars and practitioners have started to question perpetual growth, a pivotal paradigm in industrial
capitalism. Resource depletion, claims Turiel, precludes the default problem-solving conveyed through
technological innovation, development, and deployment.³
The Degrowth project entwines a set of proposals to prioritize democratic, social and ecological justice in
pursuit of well-being over economic growth.⁴ Such an economic, political, and social program conflicts
with the capitalist political economy: sustainable growth, for degrowth advocates, is an oxymoron.⁵
Degrowth provides a political-economy frame to rethink human-computer interaction (HCI) theoretical tenets and
practices as key elements to reconfigure the relationship between digital technological activity and eco-social
effects.
Political-economy in HCI
The research field of HCI brings together technological and human-related disciplines to improve the interplay
between users and machines.⁶ Despite HCI’s, deeply entrenched, industrial root logic, examining and
questioning this very logic, especially recently, has connected many authors in HCI sub-communities.⁷
Among them: reflexive HCI⁸, humanistic HCI⁹, or critical HCI¹⁰. Dourish examines the
early Sustainable HCI research production to expose the risks of naturalizing capitalist assumptions and
suggests instead broadening the theoretical approaches in use. The author advocates “dismantle design as
an anti-politics machine.”¹¹ Hence, Ekbia and Nardi urge to incorporate political economy in the
analysis of HCI’s design and practices.¹²
Problematic capitalist core mechanisms, especially productivism and consumerism, take specific forms in HCI-
related practices. Ekbia and Nardi quote Marx “production creates the consumer” relating consumerism
to the paradoxical production of the “user.”¹² This “designed user” is central
to the anthropocentric formulation of problems. Problems, again paradoxically, are produced to fit feasible
technological solutions, feeding the mechanism known as techno-solutionism.¹³ This back-feeding
entanglement is key for the industrial production system. It underpins perpetual growth while consuming
resources and expelling negative externalities, producing new problems.
These analyses within HCI resonate with the degrowth discourse. Despite some proposals (i.e., see ¹⁴
as post-anthropocentrism, or ¹⁵ as post-techno-solutionism) we suggest it’s necessary to
conduct research on how to consistently translate this critique into impact-aware HCI practice. Thus, we turn
towards degrowth-aligned practices to foreground strategic political economy issues arising from the struggle to
transition outside capitalism.
Challenges in (digital) commoning practices
According to Helfrich and Bollier degrowth and the commons movement are complementary to each
other.¹⁶ The commons don’t rely on economic growth and make compatible environmental and social
justice. These authors define the commons in terms of politics and economics as “a vast array of
self-provisioning and governance systems that flourish mainly outside of both the market and the
State.”
While degrowth frames the subject of critique, the commons exhibit social, political, and economic forms of
actualization based on the social practices of commoning: the stewardship practices that a community employs to
manage shared things (virtually anything) in common.
Digitally enabled communities of practice, spawned by the emergence of the internet, have long been at the
center of commons research. Peer-to-peer (P2P) networking architectures fostered a productive model labeled as
commons-based peer production by Benkler.¹⁷ According to P2P advocate Bauwens, this model represents
a generative alternative in front of the extractivist modes of capitalist production.¹⁸ Fuster
defines the digital commons as online communities which share non-exclusive co-created digital
resources.¹⁶
One of the main research concerns in the commons is the relationship between the state, capitalism, and
alternative spaces of resistance. The cartography of such borders seeks to discern the strategic practices to
release and gain back spaces from capitalist control. The commons aim at releasing spaces from capitalism and
confronting the enclosing of old and new commons. Such spaces risk falling back to capitalist logic, a process
referred to as co-optation. While also discussed as “colonization by capital”¹⁹,
“assimilation”²⁰, “incorporation”²¹,
“transvestment”²², or “unwanted corporate appropriation”²³, among
other terms, co-optation is ubiquitous in the commons theory but it is scarcely being examined in depth.
Among the few commons theory authors that specifically examine co-optation, we find De Angelis, Caffentzis, and
Federici. According to De Angelis, capitalism is about to face a social and ecological crisis and will likely
have to leverage or promote the commons to help manage the devastation. ²⁴ De Angelis claims:
“struggles [...] can be absorbed and become part of the system (co-opted), thus renewing it and
sustaining it.” As the logic of the market becomes counterproductive even from the viewpoint of
capital accumulation, precluding the cooperation necessary for an efficient system of production, Caffentzis and
Federici point to "the danger that 'commons' may be co-opted to provide low-cost forms of
reproduction."²⁵
Kostakis et al. review and discuss recent criticisms of peer production, classifying some of them as
co-optation. The study examines whether digital peer production could be emancipatory or instead become part of
capitalism. On one hand, the injection of funds in free open-source software (FOSS) projects and the
multi-million-dollar purchases of FOSS companies increase the risks of appropriation of the commons by corporate
interests. On the other hand, commons’ pro bono production is monetized and exploited by market agents for
profit extraction.²⁶
To inquire into the origins of digital co-optation, we focus here on the ideas of Osseewaarde et
al.²⁷ The authors analyze how digital commoners recurrently transition through alternative spaces, as
they eventually get co-opted. Commoning’s essence opposes the technological rationality of formalization,
standardization, and quantification, yet the emergence of such spaces relies on technological innovation which
is fostered by the growth-oriented efficiency ethos promoted by neo-liberalism. This contradiction, the authors
argue, results in a perpetuating illusion, a form of false consciousness, which is rooted in cynicism.
Ossewarde et al. claim that current standard technologies “are highly opaque because they are often
implicit and part of a formalized design for digital interaction that is in itself an arrangement of
‘false consciousness’.”²⁷ Hence, digital commoning (the generation of
contents, but especially of infrastructures) fails to resist co-optation, supporting capitalist expansion as De
Angelis, Cafentzis, and Federici denote.
While Ekbia et al. point to reification (the assumption of systemic concepts that are actually socially
constructed as inescapable and natural, like the capitalist market) as a hindrance to emancipatory
HCI²⁸, Dourish and Ossewaarde et al. refer to the studies on the reified construction of false
consciousness by Lukacs. Ossewaarde et al. argue that such cynicism cannot be overcome via ideology critique,
but through technology critique when it "is translated into post-capitalist acts of resistance to the dominating
technology design." De Angelis asks “Isn’t this co-evolution between struggle and capital
development really inherently with no end?”²⁹
Introducing a technological power construction model in HCI
Reified notions embed the HCI practices with false consciousness, rendering alternatives prone to get co-opted
by capitalism. To further comprehend this dynamic, we suggest introducing “technological dramas,” a
framework by the anthropologist of technology Bryan Pfaffenberger.³⁰ As technological activity
presents an opportunity to embed political values, Pfaffenberger examines how power and resistance are
constructed through the reciprocal and recursive shaping of artifacts and values in the design process, which
later spread in society.
Pfaffenberger stresses the relevance of myth, ritual, and context in the understanding of the political
dimensions of technological activity. Myths are deployed to suspend skepticism, rituals are associated with
controlled environments produced to pattern human actions, and social contexts are fabricated in parallel. In
this model, technological activity is analyzed as a process of technological communication: “a
technological drama is a discourse of technological statements and counter-statements.”
The model describes three processes that can occur linearly, or under different permutations, in the
construction of politics by technological means:
● Regularization occurs when a design entity (usually part of the establishment)
“creates, appropriates, or modifies a technological production process, artifact, user activity, or
system in such a way that some of its technical features embody a political aim”.
● Adjustment takes shape when impact entities engage in control and alteration
strategies. These strategies attempt to counter the discursively regulated social contexts that regularization
creates, pursuing to counter their effects. This process can lead to technological appropriation
● Reconstitution materializes when impact entities “try to reverse the
implications of a technology through a symbolic inversion process,” labeled as antisignification by
Pfaffenberger. This process can produce, as in some forms of adjustment, appropriation. It can also result in
the fabrication of counterartifacts, “which embody features believed to negate or reverse the
political implications of the dominant system.”
Pfaffenberger employs the term reintegration to refer to co-optation processes: “the
response made by the agents of regularization to the new, problematic counterartifacts. Its goal is to gain
control over these artifacts by bringing them back into the controlled and ordered space of
regularization.” According to the author, some forms of adjustment and reconstitution stages
(resistance processes with relevant degrees of technical intervention) are prone to co-optation. While
co-optation has been previously discussed, it is considered by this model in a wide and complex techno-political
dynamic.
Besides the three above-mentioned processes, Pfaffenberguer introduces a fourth: designification
takes place when the link between technological activity and social meaning-producing discourse dims.
“The artifacts, their contexts, and our social behaviors remain; they become taken for granted,
routine, and part of the natural attitude of everyday life.” According to Pfaffenberger, this is
the stage where technological activity achieves the greatest social penetration. We suggest a connection between
designification and the previously discussed process of reification, here particularly referred to in
techno-political, rather than political economy, terms.
Conclusions
In front of collapse, the political and economic project of degrowth offers an alternative coexistence formula.
The HCI community is already approaching the degrowth frame in order to redefine its tenets and practices, and
political economy analysis has already been adopted by HCI authors to examine reified notions. To expand HCI
research we have examined digital commoning challenges to release spaces from capitalist logic. Reification,
pointed out already by HCI authors, false consciousness, and co-optation processes are discussed by
commons’ authors as strategic issues concerning the transition to post-growth alternative political
economies.
We suggest the relevance of this model in analyzing and prospecting the construction of political power, in its
different stages, in order to deploy strategic practices of transition in front of a ravaging capitalism.
Especially due to the systemic view this model offers to dissect and relate processes like designification,
co-optation, and possibly others, as stages of the permanent struggle through technological activity. We also
suggest leveraging this model as a tool to complement speculative, adversarial, fictional, strategic, or
transitional design techniques.³¹-³⁵ Hence, we expect this contribution to help HCI
researchers better frame degrowth-related practices and research contributions.
This article is part of the R+D+i project PID2021-128875NA-I00, funded by MCIN/AEI/10.13039/501100011033/ "ERDF
A way of making Europe”.
References
1 Matt McGrath, "Climate change: IPCC report warns of ‘irreversible’ impacts of global
warming", Retrieved December 7, 2022, from https://www.bbc.com/news/science-environment-60525591.
2 Nicholas Georgescu-Roegen, The Entropy Law and the Economic Process, Cambridge, MA,
Harvard University Press, 1971.
3 Antonio Turiel, Petrocalipsis, Alfabeto, 2020.
4 Matthias Schmelzer, Aaron Vansintjan, Andrea Vetter, The Future Is Degrowth: A Guide to a
World beyond Capitalism, Verso Books, 2022.
5 Jason Hickel, Giorgos Kallis, "Is Green Growth Possible?", New Political Economy 25
(4), 2020, 469–486.
6 Paul Dourish, Where the Action Is: The Foundations of Embodied
Interaction, Cambridge, MA, USA, MIT Press, 2001.
7 Raul Nieves, Enric Mor, Joan Soler-Adillon, "Decoupling design from the industrial paradigm," In
Proceedings of ISEA2022, the 27th International Symposium on Electronic Art, Barcelona.
8 Phoebe Sengers, John McCarthy, Paul Dourish, "Reflective HCI: articulating an agenda for
critical practice," In CHI ’06 Extended Abstracts on Human Factors in Computing Systems (CHI EA
’06), 1683–1686.
9 Jeffrey Bardzell, Shaowen Bardzell, Humanistic HCI, San Rafael, California: Morgan
& Claypool, 2015.
10 Jeffrey Bardzell, Shaowen Bardzell, Mark Blythe (eds.). Critical Theory and
Interaction Design, Cambridge, MA, USA, MIT Press, 2018.
11 Paul Dourish, "HCI and environmental sustainability: the politics of design and the design of
politics," In Proceedings of the 8th ACM Conference on Designing Interactive Systems (DIS ’10),
1–10.
12 Hamid Ekbia, Bonnie Nardi. "Social Inequality and HCI: The View from Political Economy," In
Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16),
4997–5002.
13 Evgeny Morozov, To Save Everything, Click Here: The Folly of Technological Solutionism,
New York, Public Affairs, 2014.
14 Bill Tomlinson, Bonnie Nardi, Daniel Stokols, Ankita Raturi, "Ecosystemas: Representing
Ecosystem Impacts in Design," In Extended Abstracts of the 2021 CHI Conference on Human Factors
in Computing Systems, 1–10.
15 Eric P.S. Baumer, M. Six Silberman, "When the implication is not to design (technology)," In
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’11),
2271–2274.
16 Giacomo D’Alisa, Federico Demaria, Giorgos Kallis, Degrowth: A Vocabulary for a New
Era, Routledge, 2014.
17 Yochai Benkler, The Wealth of Networks: How Social Production Transforms Markets and
Freedom, New Haven, Conn., Yale University Press, 2007.
18 Michel Bauwens, Vasilis Kostakis, Alex Pazaitis, Peer to Peer: The Commons Manifesto,
University of Westminster Press, 2019.
19 Christian Fuchs, "The Digital Commons and the Digital Public Sphere: How to Advance Digital
Democracy Today," Westminster Papers in Communication and Culture 16 (1), 2021.
20 Maurizio Teli, Marcus Foth, Mariacristina Sciannamblo, Irina Anastasiu, Peter Lyle, "Tales of
Institutioning and Commoning: Participatory Design Processes with a Strategic and Tactical Perspective," In
Proceedings of the 16th Participatory Design Conference 2020 - Participation(s) Otherwise - Volume 1
(PDC ’20), 159–171.
21 Benjamin Birkinbine, Incorporating the Digital Commons: Corporate Involvement in Free and
Open Source Software, University of Westminster Press, 2020.
22 Michel Bauwens, Alekos Pantazis, "The ecosystem of commons-based peer production and its
transformative dynamics," The Sociological Review 66 (2), 2018, 302–319.
23 Benjamin J. Birkinbine, "Conflict in the Commons: Towards a Political Economy of Corporate
Involvement in Free and Open Source Software," The Political Economy of Communication 2 (2), 2015.
24 Massimo De Angelis, "Crises, Capital and Co-optation: Does Capital Need a Commons Fix? | The
Wealth of the Commons," Retrieved May 9, 2022, from
http://wealthofthecommons.org/essay/crises-capital-and-cooptation-does-capital-need-commons-fix.
25 George Caffentzis, Silvia Federici, "Commons against and beyond capitalism," Community
Development Journal 49 (suppl_1), 2014.
26 Vasilis Kostakis, Vangelis Vragoteris, Indra Lal Acharja, "Can peer production democratize
technology and society? A critical review of the critiques." Futures 131, 2021, 102760.
27 Marinus Ossewaarde, Wessel Reijers, "The illusion of the digital commons: ‘False
consciousness’ in online alternative economies," Organization 24 (5), 2017, 609–628.
28 Hamid Ekbia, Bonnie Nardi, "The political economy of computing: the elephant in the HCI room,"
Interactions 22 (6), 2015, 46–49.
29 Massimo De Angelis, "Crises, Movements and Commons," Borderlands 11, 2012.
31 James Pierce, "In Tension with Progression: Grasping the Frictional Tendencies of Speculative,
Critical, and other Alternative Designs." In Proceedings of the 2021 CHI Conference on Human Factors in
Computing Systems (CHI ’21), 1–19.
32 Carl DiSalvo, "Adversarial Design as Inquiry and Practice," In Adversarial Design, MIT Press,
2012, 115–125.
33 Matt Malpass, Critical Design in Context: History, Theory, and Practice, London, New York,
2017.
34 Anna Meroni, "Strategic design: where are we now? Reflection around the foundations of a recent
discipline," Strategic Design Research Journal 1 (1), 2008, 31–28.
35 Terry Irwin, "Transition Design: A Proposal for a New Area of Design Practice, Study, and
Research," Design and Culture 7 (2), 2015, 229–246
Motherplants: Mycelium Network and Artistic Research
JulienOttavi, JennyPickett
Association APO33, Madlab - Cyprus University of Technology Nantes, France - Limassol,
Cyprus julien@apo33.org, jennypickett@apo33.org
Abstract
In 2015, we embarked on an artistic research project involving organic organisms such as plants and fungi,
focusing on their role in the processes of recycling of e-waste. Motherplant explores how these systems create
feedback audio signal transmission through the interplay of moisture and decomposition of the circuit boards.
Specifically, it harnesses the natural decomposition process to transmit audio signals by converting the soil's
acidity and electronic compost into electrical signals.
Keywords
Artistic research project, electronic compost, mycelium networks, soil acidity, symbiosis, mutation of
technologies, e-waste.
The surge in global electronic waste (e-waste) volumes, as reported by the Global E-waste Statistics
Partnership (GESP), is cited by the World Health Organization (WHO). Between 2014 and 2019, there was a
substantial increase of 21% in e-waste generation, resulting in a staggering production of 53.6 million metric
tonnes of e-waste during the latter year. "For perspective, [in the year 2020] e-waste weighed as much as 350
cruise ships placed end to end to form a line 125 km long. This growth is projected to continue as the use of
computers, mobile phones and other electronics continues to expand, alongside their rapid obsolescence."
(Johnson, 2021) The repercussions of this escalating e-waste problem are dire, encompassing adverse effects on
both human health and ecosystems. The e-waste stream comprises a multitude of toxic substances, ranging from
neurotoxins to heavy metals, which pose significant risks (Jain et al., 2023). These hazardous constituents have
the potential to inflict severe harm upon individuals and the environment, underscoring the urgency of
addressing and mitigating the detrimental consequences associated with mounting e-waste volumes.
This project revolves around the intriguing intersections of mycelium networks, audio diffusion, and the
creation of novel computing experiences. Motherplant engages with the notions of symbiosis and mutation of
existing technologies to explore new modes of interaction with recycling, art research and implementation of new
network models based on mycelium communication cooperation systems.
Using dead and discarded motherboards extracted from obsolete computer systems, we attempted a transformative
process with the aim of repurposing these printed circuit boards (PCBs) into "micro-farm" environments. The
primary objective of Motherplant encompassed three key facets: firstly, to experiment with sustainable recycling
of all motherboard components; secondly, to harness their inherent properties for the generation of electrical
current; and lastly, to contribute to the development of an alternative open system designed to facilitate the
exchange of computational data. This fusion of electronic and organic realms introduces a harmonious synergy,
wherein the motherboard's original purpose converges with the natural processes of growth, decomposition, and
nutrient exchange.
Re-creation of the fungal network as an electronic mutation
In Motherplant we are trying to create a novel electronic circuitry that mutates in tandem with a primitive
fungal network. It is important to clarify that our fungal network is a simplified "primitive" representation,
distinct from the extensive rhizomatic networks deployed by mature fungi across vast expanses of soil,
connecting plants and trees. We acknowledge that nature excels in this regard, and our intention is not to
replicate nature's proficiency but rather to cultivate and orchestrate a symbiotic relationship between
electronic circuitry in a localised fungal network.
This begins with a few spores settling down on a nutrient-rich surface. When these spores wake up in close
proximity to one another; they initiate germination at approximately the same time, giving rise to thread-like
cellular structures known as hyphae, which extend outward at comparable rates. This unique fusion of
electronics, powered by solar energy and moisture-rich soil infused with (DIY) mycelium growth, establishes an
intricate information superhighway. As an information superhighway the interactions between a large, diverse
population of individuals speeds up. It allows entities who may be separated to communicate and help each other
out. It also allows them to commit new forms of communication.
This symbiotic relationship between electronic circuitry and a fungal network not only bears relevance to the
creation of a new information exchange paradigm but also holds significant potential in the context of e-waste
recycling. Fungi possess the ability to break down and transform a wide range of organic and inorganic
substances, pesticides, hydrocarbons and heavy metals, making them valuable agents for cleaning up contaminated
environments (Tomer et al., 2021). By virtue of fungal bioremediation, the fungal network may play a pivotal
role in the sustainable degradation and recycling of electronic components.
Schematics for Motherplant
This iteration of Motherplant comprises a redundant, nonfunctional motherboard as its foundational structure,
upon which mycelium is cultivated within a composite medium consisting of straw, soil, and integrated circuitry.
The degradation of the motherboard’s components initiates an acidification process within the medium,
subsequently generating electrical signals. These electrical signals function akin to a battery, delivering an
approximate output of 1.5 volts, which in turn powers an audio circuit. The frequencies generated by this audio
circuit exhibit variability contingent upon the dynamic activity of the mycelium and the pH levels of the
surrounding soil.
Renaturing the Motherboard
The Motherplant concept can be traced back to an earlier experiment initiated at APO33 in Nantes 2015 and
presented during NEAR #1 as part of Nantes Digital Week. During this meeting, the growing need to address
e-waste issues was discussed alongside concerns about the exploitation of workers, recycling components without
protective clothing or in adequate conditions (Ottavi and Pickett, 2015). These first experiments went on to be
presented at the Nomad Village during COP21 in Paris, later on that year (Ottavi, 2015). Electronic waste is
toxic and hazardous to human health, PCB's are hard to recycle and demand a lot of energy and sometimes the
scramble for precious metals and minerals contained therein, sacrifices members of our human community
(Wittsiepe et al., 2015; Beaumont, 2019). Usually, this heavy cost is hidden from our sleek conscientious
recycling certificates. Yet it could take decades, sometimes centuries, before some of these components will be
totally recycled. In 2019 Europe ranked first worldwide in terms of e-waste generation per capita, with 16.2 kg
per capita (Forti et al., 2020).
With Motherplant by employing plants, flowers, and fungal organisms as instrumental agents in the recycling of
electronic waste, we set about composting the motherboard. The heart of any home computer or laptop, the
Motherboard is a concealed symbol of this cyclical consumption and our relentless obsessions for digital speed.
With Motherplant we sought to counteract this hidden world of consumer electronics recycling and make visible
the complexity of such a task if left to nature, as well as the sheer disregard for the problem by the shipping
of our waste elsewhere. The idea of Motherplant was to produce electricity to feed electronic circuits, so that,
in turn, they endlessly sustain a sound installation. This undertaking represented a confluence of art, ecology,
recycling practices and the exploration of alternative energy models. The shape and layout of a motherboard is
called the form factor. The form factor affects where individual components go and the shape of the computer's
case. We design Motherplant in such a way that the classical function of the motherboard decomposes to produce a
new kind of computer. One that won’t consume electricity anymore, but instead will become a receptacle for
plants and spores to grow.
Mycelium computation: a new paradigm in bioart and sound art research
This new kind of Mycelium computation is about to begin, as we search for quantum processors, we are looking
for Mycelium based hijacked computer circuits. The spore computation is not just another form of computer
research in the tradition of engineering production, it's an all-new vision about computers. In 2018, Professor
Andrew Adamatzky, Director of the Unconventional Computing Laboratory at the University of the West of England
in Bristol, UK, wrote in his article "Towards a fungal computer", how a series of scoping experiments, provided
evidence that electrical activity recorded on fruits could potentially serve as a dependable indicator of the
fungi's reaction to thermal and chemical stimulation. (Adamatzky, 2018) It diverges from the prevailing
landscape of technological industrial mass production, presenting a concept characterised by fragility, inherent
hazards, and a multitude of uncharted pathways, and prototypes of computing mycelium bound composites. (Roberts
and Adamatzky, 2022).
A mélange of Bioart (art using fungal bioremediation and technology) with sound art (sound as a profound
production of vibration) serves as a conduit for the cultivation of exchanges and ideas within a networked
context. Within the domain of bioart, the convergence of innovative gardening techniques and the exploration of
mycelium-based experiences culminates in the facilitation of multifaceted networks characterised by a wide array
of functionalities that will serve multiple purposes such as recycling, production of energy, circulation of
electricity but also production of sound waves, feedback transmission and autonomous art installation.
From the perspective of sound-ecology, listening to the world holds significant importance in numerous
practices. However, in the case of Motherplant, it transcends mere observation of nature; it involves a
collaboration with nature to generate a sound installation on a minute scale, characterised by low impedance and
microsounds, all within the dimensions of the sculpture itself. In terms of sound levels, our focus aligns more
closely with the realm of ants and insects than that of elephants or storms. This reveals the potential inherent
in the amalgamation of nature, mycelium, and soil within our artistic patch.
Art frequently detaches itself from its own reality, often transported and displayed within galleries or
dedicated venues, creating a sense of isolation. In contrast, Motherplant seeks to recreate a natural context
intricately linked to the artwork itself, ultimately transforming the context, nature, environment, and
aesthetics into the art itself. Sound plays an integral role in the production process but whilst it doesn't
represent the final objective, it is a vital component within the broader framework. The various iterations of
the art piece over the last 8 years, also contribute to shaping the sculptural essence of Motherplant. The
process that we describe here and the relation between nature, technologies and production of energy is not
separated from the poetic facets of our artistic research.
Recycling plastic and other dirty electronics to produce low currency (power)
Beyond the artistic sculpture of the Motherplant, we had assigned to this project two other important research
perspectives. Whilst these are not the main goals, they resonate with the other aspects (art, sculpture, sound,
visuals...). The first enables us to think about and experiment with the recycling of electronic wastes with the
Motherboard. In this context, the concept of the "mother" assumes a central and symbolic role, representing the
origin from which life emanates. This archetype of the mother extends beyond the human realm to encompass
broader notions such as Mother Earth and Mother Plant, all of which evoke contemplation of the nurturing and
life-giving aspects associated with motherhood. By integrating the concept of the "mother" into the project's
narrative, we establish a profound connection between technology, nature, and the cycle of life. Which
underscores the intricate interplay between electronic devices and the environment. How can we care for
electronic waste management and recycling? Motherplant embraces maternal sustainability and environmental
awareness, towards recycling e-waste and engendering a new computation network. Secondly, recycling e-waste
costs are very high compared to disposing them in landfills, mainly because it often contains hazardous
materials that need to be dismantled and processed properly. Also, the lack of recycling infrastructure poses
problems to handle the large volume of e-waste generated each year. Finally, due to the lack of awareness from
the public, most people have no idea of the importance of recycling their computers and e-waste. This lack of
awareness ends up in huge amounts of illegal dumping or in the wrong landfills and incinerators producing more
pollution but also exposing people to hazardous materials.
We decided, as artists working in electronic arts and media, to engage in this protracted undertaking of the
project Motherplant, and develop our own way of experimenting with living networks. Rather than adhering to the
conventional and often inefficient approach of classical digital art research, which typically involves a
relentless cycle of producing more work and tirelessly pursuing galleries and festivals for exhibitions, we have
chosen a different path. We consider this process in the broader context of "decroissance" or degrowth, lowering
our consumption, decelerating our expectations of results, and embracing the temporality associated with
decomposition. The philosophy of degrowth posits that the relentless pursuit of economic growth is inherently
unsustainable in the long term. It contends that, in order to safeguard the environment and enhance societal
well-being, it is imperative to curtail economic activities and their associated impacts.
The other part of this research centres on the production of electricity. While it's important to note that
this production is characterised by low power output, it has nonetheless proven to be a sustainable source of
electrical current over the past few years, requiring no additional input other than the periodic watering of
the Motherplant. This enduring supply of electricity has so far facilitated a continuous eight-year run of our
sound installation.
The motherboard, serving as the fundamental underpinning of our installation, adheres to a decompositional
timeframe, thereby providing a robust scaffold for the continuous provision of electrical current. This enduring
electrical supply, in a reciprocal fashion, sustains the seamless operation of our sound installation, ensuring
its uninterrupted functionality. This approach to both energy generation and artistic expression accentuates the
intricate interplay amongst technology, ecology, and the dimension of time.
Documentation
View the 2015 video on Climate Solutions: MOTHERPLANT here: https://www.youtube.com/watch?v=CSUeSdR6n1Q. View
the 2018 video on Motherplants here: https://vimeo.com/301201120
Peter Beaumont, “Rotten Eggs: E-Waste from Europe Poisons Ghana’s Food Chain.” The
Guardian, April 24, 2019.
https://www.theguardian.com/global-development/2019/apr/24/rotten-chicken-eggs-e-waste-from-europe-poisons-ghana-food-chain-agbogbloshie-accra.
Vanessa Forti, Peter Baldé, Ruediger Kuehr, Garam Bel, S Adrian, M Drisse, Y Cheng, et al. 2020,
“Quantities, Flows, and the Circular Economy Potential the Global E-Waste Monitor, 2020”,
https://www.itu.int/en/ITU-D/Environment/Documents/Toolbox/GEM_2020_def.pdf.
Jain, Muskan, Depak Kumar, Jyoti Chaudhary, Sudesh Kumar, Sheetal Sharma, Ajay Singh Verma, “Review on
E-Waste Management and Its Impact on the Environment and Society”, Waste Management Bulletin 1 (3),
2023, 34–44, https://doi.org/10.1016/j.wmb.2023.06.004.
Johnson, Ceridwen, “Soaring E-Waste Affects the Health of Millions of Children, WHO Warns”, World
Health Organization, World Health Organization, 2021,
https://www.who.int/news/item/15-06-2021-soaring-e-waste-affects-the-health-of-millions-of-children-who-warns.
Adam Minter, “The Burning Truth behind an E-Waste Dump in Africa”, Smithsonian, Smithsonian.com,
January 13, 2016,
https://www.smithsonianmag.com/science-nature/burning-truth-behind-e-waste-dump-africa-180957597/.
Julien Ottavi, “Climate Solutions: MOTHERPLANT”, www.youtube.com, Solution Zone TV, December
6, 2015, https://www.youtube.com/watch?v=CSUeSdR6n1Q.
Julien Ottavi, and Jenny Pickett, “Near-1 [[NEAR] Nantes Electronic Art Rencontre]”, Apo33.org,
September 22, 2015, https://apo33.org/near/doku.php?id=near-1.
Roberts Nic, Andrew Adamatzky, “Mining Logical Circuits in Fungi”, Scientific Reports 12 (1),
2022, https://doi.org/10.1038/s41598-022-20080-3.
Ajay Tomer, Ramji Singh, Saurabh Kumar Singh, S. A. Dwivedi, Chilkuri Udaykiran Reddy, Malavika Ram Amanthra
Keloth, and Riya Rachel, “Role of Fungi in Bioremediation and Environmental Sustainability”,
Fungal Biology, 2021, 187–200, https://doi.org/10.1007/978-3-030-54422-5_8.
Jürgen Wittsiepe, N. Fobil Julius, Till Holger, Gerd-Dieter Burchard, Michael Wilhelm, and Torsten
Feldt, “Levels of Polychlorinated Dibenzo-p-Dioxins, Dibenzofurans (PCDD/Fs) and Biphenyls (PCBs) in
Blood of Informal E-Waste Recycling Workers from Agbogbloshie, Ghana, and Controls.” Environment
International 79, June 2015, 65–73, https://doi.org/10.1016/j.envint.2015.03.008.
Authors Biographies
Julien Ottavi, holds a Doctor of Arts from the Université de Lorraine. His practice covers sound,
intermedia, and digital network arts. He is the founding member and artistic director of association APO33,
based in Nantes. APO33 has produced numerous ground-breaking explorations in experimental music and electronic
arts, and collaborated with artists and musicians across the world.
Jenny Pickett is a member of the APO33, where she works on projects ranging from interactive installations to
experimental music and performance to international collaboration in art and technology. She is currently a
PhD Candidate at Cyprus University of Technology where she is attached to the Media Arts and Design Research
Lab (MADlab), she is also the Vice Chair for Toolkit of Care a EU COST funded cooperation, as well as an
associate lecturer at the Nantes School of Architecture (ENSA).
Life-as-it-could-be, Symbiosis in Interspecifics’ Codex Virtualis_Genesis
ClaudiaCostaPederson
Wichita State University Kansas, U.S.A. ccp9@cornell.edu
Abstract
This paper discusses Codex Virtualis_Genesis (2020-2022) as an artistic engagement of Artificial Life
(AL) that explores the nature of informational life as a symbiotic process. Created by Mexico City-based
transnational collective Interspecifics (INT), this work follows on from the expanded notions of life central to
the field of artificial life, including organic, inorganic, material, and virtual forms. According to its
founder, Christopher Langton, “there is nothing... that restricts biology to carbon-based life; it is
simply the only kind of life that has been available to study.” From this perspective, Langton proposed in
the late 1990s that AL be dedicated, as he put it, to speculating beyond "life-as-we-know-it" into the
realm of "life-as-it-could-be.” This discussion examines Codex Virtualis_Genesis in light of
Langton’s proposal as a speculative inquiry into a symbiotic view of life, and as well in contrast to
notions of artificial life art as a predominantly technophilic practice. Instead connected to the speculative
imagination, the synthetic life forms of Codex Virtualis_Genesis offer a glimpse into life otherwise:
as an interspecific relation.
Keywords
Artificial Life, Artificial Life Art, Mexico, Symbiosis, Symbiocene.
Interspecifics’ CodexVirtualis_Genesis (2021-2022) was created for the SETI x AI Lab
residency, an art and science program started in 2018 by the Ars Electronica Futurelab in Linz, Austria in
collaboration with the AI Lab (European ARTificial Intelligence Lab) and the SETI Institute in Mountain View,
CA.¹ The program calls on international artists working in digital art to create works “that explore
the evolution of life on earth and universe and critically reflect on anthropocentric world views.”²
Interspecifics won the 2021 Ars’ AI Lab residency call with their proposal to create CodexVirtualis_Genesis as an evolving taxonomic collection of hybrid AL forms.³ In
practice, to create these forms’ shape, color, texture, morphological structures and behavioral dynamics,
Interspecifics used a 2D cellular automata system (a planar cell-based computer graphic system that displays
emergent global behaviors), and two sets of images of living symbiotic organisms. One set is microscope images
(microscopy) of microorganisms living in water bodies that were posted by various scientists on their Instagram
accounts.⁴ Another set was digital photographs of colonies of fungi and bacteria (8,000
distinct species) made available online by researchers at the National Agriculture and Food Research
Organization, NARO, in Japan. Some of the latter organisms are also known as extremophiles because they are
metabolically and biochemically adaptable to harsh environmental conditions such as broad swaths of temperature,
pH, pressure, radiation, salinity, energy, and nutrient limitation. They have therefore been historically used
in astrobiology experiments, including most recently conducted at the ISS (International Space
Station).(1)
The resulting images resemble plant and animal-like microorganisms in a wide range of colors and shapes (Figure
1a and 1b). In a gallery setting, Codex Virtualis_Genesis is shown as a screen-based installation that
as well as displaying selected images of said organisms, also displays images of aspects of the selection
process and evolutionary procedures (Figure 2).
Additionally, in the printed materials provided to visitors at exhibitions, Interspecifics describes Codex
Virtualis_Genesis as titled in a loose nod to Mesoamerican codices and based on “the symbiotic
narrative of evolution.”⁵ Specifically this refers to endosymbiosis, a process of
symbiotic life evolution proposed by the biologist Lynn Margulis in the late 1960s. Endosymbiosis foregrounds
symbiotic mergers (pre-nucleic gene transfer between bacteria and other organisms) as the main sources of
acquisition of new genomes and thus also of speciation (i.e., the formation of new and distinct species in the
course of evolution). From this perspective, random mutation is of marginal consequence as the driver of
evolution and so consequently Margulis argued that Darwinian evolutionary theory based solely on competition
(i.e., the survival of the fittest) is incomplete. According to Margulis, “Life did not take over the
globe by combat, but by networking. Life forms multiplied and complexified by co-opting others, not just by
killing them.”⁶
Interspecifics’ Codex Virtualis_Genesis brings Langton and Margulis in dialogue with an eye to
complement their respective perspectives on the nature of life. In this interest, it extends Margulis’
restricted notion of life (as carbon-based) to as well, as proposed by Langton, include synthetic forms.
Conversely, by linking AL and symbiosis it also clarifies Langton’s speculative vision of AL (i.e., as
“life-as-it-could-be”). (2) According to a member of Interspecifics, Leslie Garcia, this
dialogue speaks to the project’s overarching concept: namely to simultaneously call attention to our
“entanglements” and to stimulate speculative imagings and imaginaries of “non-anthropocentric
and non-anthropomorphic but more organicist, metabolic models of life.”⁷
Cooperate!
Codex Virtualis_Genesis not only demonstrates life as a mutualistic relation on screen but is itself
created with an eye for developing art and science as mutualistic practices: in Interspecifics’ own words,
“to produce in terms of social inclusion, cross-disciplinary practices, and open knowledge.”⁸
To this concern, the project contributes to broadening access to artificial life techniques and consonantly
increasing diversity in artificial life art as an area of the global digital arts. As aforementioned,
Interspecifics drew on freely available images shared by scientists and scientific communities online and social
media to create Codex Virtualis_Genesis’ symbiont organisms. Additionally, Lenia, a readily
available system of cellular automata or artificial life, is integral to the project. Created by the Chinese
software engineer Bert Wang-Chak Chan in 2015, this system is a variation on The Game of Life (GoL) devised by
the British mathematician John Horton Conway in 1970, which was originally played on a simple square grid
without the aid of computers. GoL is a class of mathematical phenomena termed cellular automata by John von
Neumann in his discussion of self-reproducing machines in the mid-1940s.(3) Both GoL and Lenia
are AL systems that focus on life not as a material substrate but as a form of organization and behavior that is
akin to the distributed processes characteristic of the functioning of living organisms. Alternatively, these
systems can also be conceived as no-player games; meaning that after the initial configuration their evolution
does not require further input. The point of cellular automata like GoL and Lenia is in short to demonstrate the
emergence of the variety and complexity of behaviors from a few simple rules. These systems differ insofar the
sophistication of their graphics. In comparison to GoL’s square graphics, which now look outdated,
Lenia’s fuzzy, smooth, and colorful patterns look state of the art. More significantly for artists
broadly, as art funding becomes ever more restricted, and in particular artists that as Interspecifics puts it,
work via-á-vis the context of “precarity” in the global South, Lenia is as an open-source art
and science project. Still requiring some technological expertise, it nevertheless is in theory accessible to a
global public.
Already, in an updated version of his 1989 article on Artificial Life, Langton speculated that the future
evolution of AL would depend on scientists’ imitating nature’s cooperative behaviors (so he urged
biologists and computer technicians to follow its example). (4)Artificial life art, as A-life artist
Simon Penny notes, is similarly an area of digital arts with a high degree of interdisciplinarians and/or
collaborative partnerships between artists and technicians because artificial life techniques are technically
demanding.(5) Because of the emphasis on technical expertise and state-of-the-art technology in the
field (what Penny calls “high nerd quotient” and the art historian María Fernández
calls digital art’s “aesthetic technofetishism”), artificial life art has been dominated by
artists working in the global North.(5)(6)⁹ Artists like Interspecifics are at the
forefront of developing artificial life art as an emerging area of the digital arts in Mexico along with a new
generation of artists in Latin America, including the Argentinian artists Leo Nuñez and Sofia
Crespo and the Peruvian artist Paola Torres Nuñez del Prado. Projects like Linea are significant because
they broaden access to AL, both beyond the sciences and the global North.¹⁰ It is to this
concern that Interspecifics, just as with all their projects, additionally made some of the research,
techniques, and coding created for Codex Virtualis_Genesis freely available online.¹¹
CodexVirtualis
CodexVirtualis_Genesis is yet but the first of the four parts of the total project proposed
by Interspecifics under the title Codex Virtualis. The collective is currently developing the second
part of this project, which is entitled Habitat involves testing the extremophile capabilities of the
organisms created in Genesis by exposing them to inputs that simulate the harsh conditions of
extraterrestrial environments. A third part, which is titled Emergence proposes testing the possibility
of intra-actions (transfers) between artificial and biological organisms. Lastly, Codex Virtualis: Life
involves testing the resilience of resulting hybrid organisms and accordingly selecting one organism to
live virtually in Codex Virtualis.¹²
As a contemporary artificial life artwork, CodexVirtualis follows on a longstanding quest
for mimesis or life-likeness that spans the histories of art, science, and technology, and as well includes
social, spiritual, commercial, intellectual, and military histories. Langton himself traced the origins of AL to
this pursuit in the history of art, starting with paintings and “statuettes” that “capture the
static forms of living things.” He goes on to discuss hydraulic technology such as Egyptian
water clocks or clepsydra as the next step towards imitating nature’s dynamic behavior. This
trajectory developed with the invention of the mechanical escapement in AD 850, ushering in the age of
clockwork technology. Refined throughout the European Middle Ages and the Renaissance, eventually, clockwork
regulation of mechanical devices would not only be used to power them but also to sequence their motions or
behaviors.¹³
Langton refers to this development as technology of process-control, which in the first instance involved
interchangeable cams or drums with movable pegs. A variety of programmable automata are examples, including
lifelike mechanical figures and animals such as the writing and picture-drawing automata built by the
Jaquet-Droz family in Switzerland in the 18th century. In the early part of the twentieth century,
during WWII and the Cold War, as Langton explains, physical programmable controllers gave away to abstract
control structures, or sets of rules or programs. In effect, as he puts it, this is the moment when the
“’logical form’ of the machine was separated from its material basis of construction.”
Today’s computers are a technology of process-control and in this sense the equivalent to an
“algorithm: the logic underlying the dynamics of an automaton.” (7)¹⁴
Artificial life art, which in the contemporary context is facilitated by computer technology, is consequently
heir to these legacies.(8) According to Penny, artificial life art in digital environments spans
three decades, beginning in the late 1980s, as artists turned to “a new type of interactivity” in
search of a “new order of mimesis in which ‘nature’ as a generative system, not as an
appearance, is being represented”. (5) To put it otherwise, artists became interested in
artificial life in pursuit of developing art contrary to ingrained notions of artistic practice. That is, to
create art not as an object but as a process, and in extension, to explore the role of the artist not as one
about control over materials, but as an initiator of an open-ended process. This initial focus on
unpredictability and novelty has since, however, been debated as an illusory pursuit, given that, as Penny adds,
“the mechanism of simulated evolution” appear in comparison to the limitations of biological
evolution more restricted; that is, as bound to the “conditions... established by logic-theoretic
enframing” of mathematical models. (5)
Yet, while Codex Virtualis_Genesis and its larger project, CodexVirtualis shares
interest in interactive aesthetics, this is not the project’s main focus. Its focus is rather apropos of
the curator and cultural critic Edwina Bartlem’s noting as part of her recent discussion of artificial
life art that “A-life discourse contains a prophecy of futuristic and imaginary posthuman or post-organic
life.” (9)Codex Virtualis speaks above all to this imaginary, as according to
Interspecifics, “’Codex Virtualis’ imagines ways of living together, of deep
interspecific relations that may enhance our possibilities of survival.”¹⁵ Garcia further
clarified the project’s concept on another occasion, as a proposal to imagine life and a new world beyond
the Anthropocene, consonant to “the Symbiocene.”¹⁶
Garcia’s reference is to a term coined by the eco-anarchist philosopher, Glenn Albrecht, the Symbiocene.
Symbiocene refutes the fatalistic implications of the Anthropocene, and, as largely inspired by Margulis’
notion of endosymbiosis affirms the possibility of creating a more sustainable world based on
non-anthropocentric mutualism. (10) In this light then CodexVirtualis engages
symbiosis as a metaphor and as a form of dialogue, debate, and exchange (i.e., between art, science, and
politics) for conceptualizing life in a sustainable posthuman age. Because of its ecological focus, it as well
in extension broadens the existing e imaginaries and themes engaged in artificial life art.
Notes
1 Interspecifics was founded in Mexico City in 2013. Core members are Leslie García, Paloma
López Ramírez, Emmanuel Anguiano Hernández, Felipe Rebolledo Carvajal, Carels
Tardío Pi, and Maro Pebo.
2 Ars Electronica is a cultural institute devoted to media art founded in Linz, Austria, in 1979.
A leading organization in the global media arts, as well as managing a museum, multidisciplinary art research
facilities, and an annual festival, it confers the most prestigious award in the genre, the Prix Ars
Electronica. Founded in 1984, the SETI Institute stands for the "search for extraterrestrial intelligence." It
is a non-profit dedicated to the study of life and intelligence in the universe through multidisciplinary
research, education, and partnerships with industry, academia and government agencies, including NASA and NSF.
3 Available at, https://int-lab.cc/codex/?page_id=2.
4 According to Interspecifics, the initial idea was for collaborating (SETI) scientists to share
their images with the group for the project. As it happened, the scientists failed to provide these images in
the end. Leslie Garcia, “Interspecifics, Codex Virtualis: Genesis,” lecture, “Neurotalk
‘Máquinas y seres vivos: Comunicaciones desde el arte’”, Programa ACT (Arte, Ciencia
y Tech- nologías), UNAM, March 2, 2022, available at, https://www.youtube.com/watch?v=NFE_F6HzakY.
5 Interspecifics, CodexVitualis_Genesis, exhibition booklet, 2021, 4, available
at, https://int-lab.cc/codex/?page_id=183.
6 Lynn Margulis and Dorion Sagan, Microcosmos: Four Billion Years of Microbial Evolution
(Berkeley, CA: University of California Press, 1997), 29. Margulis’ speculations have been
corroborated in the last decade as for example by scientists discovering that in ecosystems across the world,
there are immense, mutually beneficial associations of macrofungi with flowering plants in complex, positive,
metabolic, symbiotic relationship to each other. Findings such as these have scientifically overturned the
view that evolution and life are solely founded on competitive struggle between species.
7 Leslie Garcia, “Interspecifics, CodexVirtualis:Genesis,” lecture, “Neurotalk ‘Máquinas y seres vivos: Comunicaciones desde
el arte’.” It is worth to note here that Margulis’ notion of endosymbiosis implies a
critique of heterosexist articulations of life, (i.e., the singularly privileged place of heterosexual
reproduction in evolutionary biology), which presently resonates with Karen Barad’s notion of
“nature’s queer performativity”, which likewise denotes multispecies co-involvements as the
drivers of life’s heterogeneity. See, Karen Barad, “Nature’s Queer Performativity” in
Kvinder, Kønogforskning/Women, GenderandResearch nos. 1-2 (2012): 25-53.
9 Artists working with digital A-life systems mostly reside in North America, Europe, and
Australia, and include Karl Sims, Tom Ray, William Latham, Ken Rinaldo, Bill Vorn, Louise-Philippe Demers,
Troy Innocent, Jon McCormack, Robb Lovell and John Mitchell, Christa Sommerer and Laurent Mignonneau, Jane
Prophet and Gordon Selley, Paul Brown, Richard Brown and Mauro Annunzianto. See also, Fernández,
“’Life-like’: Historicizing Process and Responsiveness in Digital Art”; Sarah Kember,
Cyberfeminism and Artificial Life (London and New York: Routledge, 2003); “; Michael Whitelaw,
Metacreation:Art andArtificialLife (Cambridge, MA: MIT Press, 2004);
and Simon Penny, “Twenty Years of Artificial Life Art”, in DigitalCreativity
21, no. 3, Dec. 2010: 197-204.
10 To date, along with scientists, game designers, filmmakers and artists across the world,
including among them, Interspecifics, have used Lenia to develop their respective projects and in the process
added to the evolving diversity and complexity of behaviours of its life forms. According to Chan, to date,
more than 400 species in 18 families have been identified in Lenia. B. W.-C. Chan, “Lenia: Biology of
Artificial Life,” Complex Systems, 28(3), 2019, pp. 251–286. See also, Siobhan Roberts,
“The Lasting Lessons of John Conway’s Game of Life”, in TheNewYorkTimes, Dec. 28, 2020, available at, https://www.ny-
times.com/2020/12/28/science/math-conway- game-of-life.html.
11 See, https://int-lab.cc/codex/.
12 The duration, location, and conditions of this last stage are yet to be clearly specified as at
the time of writing, Interspecifics is still developing the second stage in the project, Habitat.
13 The Muslim inventor Ismail al-Jazari’s TheBookofKnowledge ofIngeniousMechanicalDevices (1206) is currently considered
by historians of science to be a significant groundwork for modern engineering, hydraulics, and even robotics.
The Book builds on science and wisdom from ancient Greek, Indian, Persian, Chinese and other cultures
and offers a how-to manual of sorts that includes alongside richly detailed illustrations as well as
instructions on how to build al-Jazari’s inventions spanning playful robots to practical contraptions.
14 Likewise, the Latine artist Micha Cárdenas has traced algorithmic culture to pre-digital
technology, citing recipes and rituals as examples. Additionally, Cárdenas reminds us that the word
algorithm is a derivation of the name of the scholar Muhammad ibn Musa al-Khwarizmi (780-850 A.D.), credited
with inventing algebra in his book Dixit Algorismus. Micha Cárdenas, PoeticOperations, Trans of Color Art in Digital Media (Durham, Duke University Press, 2022), 7.
16 Garcia, “Interspecifics, CodexVirtualis:Genesis,”
lecture “Neurotalk ‘Máquinas y seres vivos: Comunicaciones desde el arte’”
References
(1) Cortesão See M., T. Schütze, R. Marx, R. Moeller, V. Meyer, “Fungal
Biotechnology in Space: Why and How?” in H. Nevalinen, ed., Grand Challenges in Fungal
Biotechnology. Grand Challenges in Biology and Biotechnology, New York, Springer, 2020, 501–535.
(2) Christopher Langton, “Artificial Life,” in Ars Electronica: Facing the
Future, edited by Timothy Druckrey, with Ars Electronica, Cambridge, MA, MIT Press, 1999, 126-268.
(3) John von Neumann See, TheoryofSelf-ReproducingAutomata,
Urbana, University of Illinois Press, 1966.
(4) Christopher G. Langton, “Artificial Life,” in The Philosophy of Artificial
Life, edited by Margaret A. Boden, London, UK, Oxford University Press, 1996, 92-93.
(5) Simon Penny, “Art and Artificial Life-A Primer,” in the Proceedings of
DAC09, Digital Art and Culture 2009 conference, University of California, Irvine, 12–15
December 2009, unpaged, available at, https://escholarship.org/uc/item/1z07j77x.
(6) María Fernández, “Postcolonial Media Theory,” in ArtJournal 58, no. 3, Autumn, 1999, 66-69.
(7) Langton, “Artificial Life,” 41-46.
(8) María Fernández See, “’Life-like’: Historicizing Process and
Responsiveness in Digital Art,” in A Companion to Contemporary Art since 1945, edited by Amelia
Jones, Malden, MA, Blackwell Publishing, 2006, 557-581.
(9) Edwina Bartlem, “Immersive Artificial Life (A-Life) Art,” in Backburning:
Journal of Australian Studies 84, edited by Helen Addison-Smith, An Nguyen and Denise Tallis, Perth,
API Network, 2005.
(10) See, Glenn Albrecht, Earth Emotions: New Words for a New World, Ithaca, NY, Cornell
University Press, 2019.
Author Biography
Claudia Costa Pederson is Associate Professor of Art History, author of Gaming Utopia: Ludic Worlds in Art,
Design, and Media (Indiana, 2021); and co-curator of FLEFF’s new media exhibitions at Ithaca College,
NY, since 2017.
Disruptive Avant-Garde Art of Today: Shaping Post-Growth Imaginaries for Symbiotic Futures
KristinaPranjić,
MagdalenaGermek, PeterPurg
University of Nova Gorica Nova Gorica, Slovenia kristina.pranjic@ung.si,
magdalena.germek@gmail.com, peter.purg@ung.si
Abstract
In order to develop new symbiotic relationships and different imaginaries, it is first necessary to critically
restructure the representations of forms of cooperation, which in their positive, desired version usually
represent a certain romantic idea of nature and human, and the possibilities for a harmonious model and holistic
structure of reality. This can be seen in both eco art and activist ecological agendas, which often play on
feelings of harmony and mutual reciprocity, and actually further contribute to a distorted and extremely
one-dimensional image of reality. Using the concepts of conviviality and cosmpolitics, the article aims to offer
new concepts of symbiosis and symbiotic futures that face today's process of defuturing. The second point of the
article is to develop a convincing and solid alternative to the neoliberal view of market-driven models based on
competencies and the logic of growth. Therefore, the actual task for disruptive avant-garde art of today should
be understood as the decolonization of our imaginaries that perceive nature through the logic of growth and the
harmonious model in the direction of shaping post-growth imaginaries for symbiotic futures.
At the end of the epidemic, faced with another European war, and shattered by climate disasters, we seem to be
facing a similar if not even worse zeitgeist to the one of the twenties that called for the avant-garde
art and design school Bauhaus to bring a new social, cultural and creative impetus. However, back then there was
no experience of the failures of the 20th century artistic and social utopias and movements.
For this reason, we can rightly question any revolution or change through artistic and cultural means.
Nevertheless, new artistic expressions that use new technology and scientific methods to conduct true artistic
research are developing novel concepts and opening space for another horizon of experiences that can offer
images and ideas of a different future, in which it will be possible not only to survive, but also to thrive in
symbiosis with other human and non-human beings.
In order to describe the mentioned process through a deep understanding of how current art contributes to the
shaping of our imaginaries for the better future in the age of the Anthropocene, the article will first
introduce key concepts, such as “conviviality”, “symbiosis”, “defuturing”,
“imaginary”, “growth”, and “post-growth/degrowth,” that comprehensively
shape the paradigm, criticized by the artistic projects that are addressing the most burning issues of today,
and rightfully represent the contemporary avant-garde. In the last part of the article, we will show three
distinct examples of current artworks that seek to participate in transforming the human imagination in the
direction of symbiotic futures with a different system of relationship not dependent of industrial and
consumerist systems, rooted in the idea of continuous growth.
Living together: Conviviality and Symbiosis
The term conviviality was incorporated to the humanities vocabular by Ivan Illich, who in his work criticized
the industrial capitalism and productivity, to instead describe a path of human emancipation and autonomy for
reaching convivial life.¹ For Illich conviviality means an “autonomous and creative intercourse
among persons, and the intercourse of persons with their environment; and this, in contrast, with the
conditioned response of persons to the demands made upon them by others, and by a man-made
environment.”² Furthermore, because of his instance of the interdependence among living beings,
Ilich’s thought is particularly productive in the field of posthumanism to describe interdependence and
entanglements between different species, human and non-human agents.
The term “conviviality” comes from Latin convivere, which means “to carouse
together, live together” (com “with, together” and vivere “to
live”). The more common term “symbiosis” that comes from ancient Greek sýn and
bíōsis similarly denotes living together. Today both of these concepts are equally
important in the posthumanism research that emphasizes the importance of exploring relations between human and
non-human beings and other entities or matter, and insisting on the theoretical break with anthropocentrism and
with the dualisms between human/society and nature.
Conviviality does not mean only relationships of cooperation, but it also includes conflict and tensions as
constitutive and necessary part of convivial relations; human beings are understood as a part of the network of
players living in symbiosis with others. ³ Moreover: “To Illich, the word ‘conviviality’
does not mean joy or light-heartedness; it refers to a society in which modern tools are used by everyone in an
integrated and shared manner, without reliance on a body of specialists who control said
instruments.”⁴ For Ilich convivial toll can be used by an individual for a purpose
that they choose, and it needs to bring about more freedom, autonomy and creativity (for all beings).
Perceptions of harmonious interactions of symbiosis were critiqued by Isabelle Stengers’ research in
thermodynamics to include complexity, processes of disturbance and friction that cause a state of crisis and
lead to change.⁵ In her seminal work on “cosmopolitics” Stengers described a “symbiotic
agreement” as a part of an “immanent process of ‘reciprocal capture’;” she
describes “the event” of symbiosis as “the production of new, immanent modes of existence, and
not the recognition of a more powerful interest before which divergent particular interests would have to bow
down. Nor is it the consequence of a harmonization that would transcend the egoism of those
interests.”⁶
We can further find productive thoughts on this topic in Anna Tsing work on rare matsutake mushroom, in which
she presents the complex network of cooperation between human and non-human participants that rejects
stability as a central goal of relationships, and even stability of the term
“naturalness”.⁷ Stengers’ and Tsing’s work not only opens to question the
foundations of objective (natural and humanities) science, but also shows a new way of connecting and generating
knowledge and possible new imaginaries that can arise from different forms of human activity, which is not
necessarily bound to the rationality and authority. Knowledge and science can be, through specific practices and
processes, something that we not only discover, but also shape.
Facing Defuturing
To recognize what is, we first need to establish active knowledge about how our current dysfunctional
relationships of un-sustainment are sustained. One could assume it is truly difficult to be an avant-gardist in
the time of defuturing, as Tony Fry named our condition in his book Defuturing. A New Design
Philosophy.⁸ Fry introduced the concept of defuturing as the destruction of the future by design;
it means that we live in a world that is taking away futures for ourselves and non-human others. He criticizes
Eurocentric, anthropocentric and productivist structures of unsustainability by exploring the history of design
that so importantly yet often subliminally coordinates our activities, and influences our thought.
Because our conditions of existence drastically changed, Fry stresses that we also need to drastically change
our philosophy of existence and of acting. The new direction for making anything in the world is not a direction
within design as-is; within the already established paradigms; it is a direction beyond where design now is. It
is also a direction beyond where thought now is. The alternative to this process of “change so that
nothing changes” and covering of crisis, would be “informed futuring”, which basically refers
to finding the alternative and acquiring the agency in order to create something truly different, an actual
change.⁹
For this kind of futuring, a new avant-garde in doing and thinking is needed. Therefore, it is central to look
into the past to deconstruct and surpass it. The most significant signifiers of how we do and think things, and
how we ascribe value – is growth. Therefore, besides offering a complex image of symbiosis and
establishing new and different relationships, it is important to decolonize our imaginary of growth.
Decolonizing our Imaginaries
We can understand “growth” as a centuries-old matrix of thinking that historically constructs
Western-civilizational models of meaning-making, collectively and individually, from various institutional macro
and microforms of society. The logic of growth is thus not only rooted in the rational categories that shape our
concepts but also in the way we look at the world, how we describe, perceive, and represent it. The stubbornness
to fight growth by reformulating capitalist parameters – for example, by establishing concepts like
social, human, local, and sustainable “development” – can only lead to new ecological,
economic, and social failures.
Agendas such as the New European Bauhaus for example, and the UN 2030 Agenda for Sustainable development do
seem to be signs of a significant shift toward an awareness about social and environmental health, equality, and
justice. However, we need to be critical of the very idea of sustainable development, which remains part of the
Western economic imaginary of growth. We can find this critique especially in the works of Serge
Latouche¹⁰, who claims that the idea of sustainable development is a mystified and ideologized one
that does not bring about any radical turn in thinking but stems from the logic of a compromise.
Consequently, it is unavoidable to start “decolonizing our imaginaries,” a syntagma borrowed from
Latouche¹¹, who refers, on the one hand, to the concept of the imaginary by Cornelius
Castoriadis¹², and on the other hand, to the anti-imperialist concept of decolonization introduced by
anthropologists (e.g., Serge Gruzinski).¹³ If we follow Latouche’s insight, radical change needs
another economy; another view of science that would go beyond the concept of Promethean technoscience; another
conception of life and death; a different conception of wealth and poverty; another notion of time that would no
longer be linear, cumulative, continuous; other conceptions of space; other intergenerational and gender
relations; a different concept of work (placing social relations in the center, instead of, e.g., efficiency or
value accumulation)¹⁴. Latouche integrates this into an eight-point program with listed imperatives:
to re-evaluate, reconceptualize, restructure, redistribute, relocalize, reduce, re-use, and
recycle.¹⁵
Constructing the Imaginary of Post-Growth
One solid set of alternatives to the logics of growth was offered by the introduction of concepts such as
post-growth and degrowth. And even though the concept of degrowth as the most optimal future direction of our
global society is becoming more and more mainstream, it is clear we will not achieve any drastic changes in
human behavior if we do not address its phenomenological and epistemological implications that could provide us
with new models for everyday life.
Neutralizing the disastrous impact of our growth-based society would therefore mean actually using radical
avant-garde methods as a unique combination of chance and plan, happening and controlled work in all aspects of
thinking and creating for uncertain futures. We see the legacy of the historical avant-garde movements, such as
Dadaism and Surrealism, as a plethora of valuable concepts for thinking degrowth that is yet to be researched
and applied from the field of art exclusively to the area of social and environmental care, and precisely to
the way we make and do things. The highly experimental avant-garde presented the most radical
possibilities for changing the logic of thinking and perceiving reality, thus showing the opportunities to let
new forms and relationships emerge.
And although the avant-garde is all too often perceived as art that sought to demolish and destroy everything
that existed before it, it carries within it an extremely affirmative and constructive character: avant-gardes
invented a whole series of new formal languages and strategies that are still in use today as fundamental
techniques of our modern visual or multimodal communication (e.g., photomontage, collage, avant-garde cinema,
advertisements, graphic design typography, new architecture). They sought to formalize the coincidence and
freedom of expression and erase any automatization and pre-given cultural definitions.
The avant-gardists’ proposition appears to have been precisely what we need today for a symbiotic
“metamorphosis in being” as stated by Pasi Haikkurinen, connected with the “practice of
releasement” understood as the “new ethos” for degrowth
society.¹⁶,¹⁷
Artistic Disruption for Radical Innovation
We finally come to the acknowledgment of the importance of artistic disruption and art-thinking as the key
innovative methodology that may provoke radical change in order to create a meaningful difference from what was
(wrong). Which are those new (inter)media and investigative artistic practices involving cutting-edge
technologies, critically confronting the European social and cultural values with the aesthetics-, ethics- and
tech-related legacies of the historical avant-garde?
By way of conclusion, let us try to briefly discuss three of such topical cases selected among the newest
(intermedia) artistic practices:
Vladan Joler's recent hybrid animated info-graphic work entitled New Extractivism (2020) visualizes
the position of the individual as a user of contemporary information technologies, who is subject to systemic
and systematic corporate extraction. Thus, it reveals and critically discusses some of the most pressing issues
of modern platform capitalism. The printed materials, which the author considers to be a kind of sci-artistic
“assemblage”, are presented in the format of a poster and a brochure, and are accompanied by an
animation distributed online, supported by the author's own narrative, which he occasionally also delivers as a
lecture (performance). By combining the elements of a map and a guidebook, and accompanied by abundant and
extremely analytical notes, the work attempts to produce a symbiotic and fractal-like “blueprint of a
machine-like superstructure, or a super allegory.”¹⁸
Often relying on the approaches of speculative design and advanced computer graphics, as well as
info-animations or even physical spatial installations, the works of the interdisciplinary group
Disnovation.org not only vividly show the paradoxes of the contemporary consumer and hyper-informatized
society, but also try to contribute suggestions and tools for resolving them. In the Post Growth Toolkit
(2020), a set of so-called critical games, they offer a mix of scientific and speculative
literary-narrative tools, which should offer the players of this game (or the users of the toolkit) a set of
more sustainable solutions for everyday life. Post Growth Prototypes (2021) complement the mentioned
toolbox with critical (animated video) essays functioning as case studies of advanced and symbiotic concepts
such as solar income, radical energy transition, or transcending the Anthropocene via the post-growth paradigm
shift, introducing a new responsibility of man towards the biosphere.
As the last truly comprehensive example of an artistic practice that radically questions the human role within
ecosystems, we wish to discuss the set of terra0 projects. Since 2018 various prototype environments
have been built on the decentralized (peer-to-peer) blockchain platform Ethereum, which aims to provide
automated frameworks for the resilience of a given ecosystem. By establishing a “Decentralized Autonomous
Organization” on the upper layers of the earth to govern them, terra0 research team aims to
create technologically augmented ecosystems that are both more resilient and more capable of operating within a
predetermined set of rules in the economic sphere, as independent agents: e.g., the forest independently mines
cryptocurrency and decides how it will change its material base. The group believes that modern technologies
such as remote sensing and machine learning provide an opportunity to rethink existing inefficient governance
and regulatory structures. Moreover, they also seem to suggest how, with appropriate art-thinking and
speculative-design based assumptions and interventions, these could play a key role in creating a sustainable,
resilient, symbiotic and biodiverse future.
There are more such artistic practices and they bear witness to the engagement of art in order to answer the
most pressing questions of our time not only by representing what does not work, but above all to construct, in
an avant-garde manner, new symbiotic relationships with human and non- human agents that would contain the full
complexity of the symbiotic organization of different entities jointly working to sustain a functional
environment, without falling back into the old paradigms shaped by the imaginaries and logics of growth,
duality, harmony and stability.
References
1 Ivan Ilich, Tools for Conviviality, New York, Harper and Row, 1973.
2 Ivan Ilich, Tools for Conviviality, 11.
3 Michael Given, “Conviviality and the Life of Soil,” Cambridge Archaeological
Journal 28, 2017, 127–143.
4 Marco Deriu. “Conviviality,” in Degrowth – a new vocabulary for a new era,
ed. Giacomo D’Alisa, Federico Demaria and Giorgos Kallis, London, Routledge, 2015.
5 Ilya Prigogine, Isabelle Stengers, Order Out of Chaos. Man's New Dialogue with Nature,
London, Flamingo, 1984.
6 Isabelle Stengers, Cosmpolitics I, Minneapolis/London, University of Minneapolis Press,
2010, 35-36.
7 Anna L. Tsing, The Mushroom at the End of the World: On the Possibility of Life in
Capitalist Ruins, Princeton/Oxford, Princeton University Press, 2015.
8 Tony Fry, Defuturing, A New Design Philosophy, 2nd Edition, Bloomsbury, Bloomsbury
Publishing, 2020.
9 Tony Fry, Defuturing, A New Design Philosophy, 239.
10 Serge Latouche, Petit traité de la décroissance sereine, Paris, Librairie
Arthème Fayard, 2007.
11Serge Latouche, “Imaginary, Decolonization of,” in Degrowth – a
new vocabulary for a new era, ed. Giacomo D’Alisa, Federico Demaria, Giorgos Kallis, London,
Routledge, 2015.
12Cornelius Castoriadis, L'institution imaginaire de la société,
Paris, Éditions du Seuil, 1975.
13 Serge Gruzinski, La colonisation de l’imaginaire. Sociétés
indigènes et occidentalisation dans le Mexique espagnol (XVIe- XVIIIe siècle) Collection
Bibliothèque des Histoires, Paris, Éditions Gallimard, 1988.
1Corneius Castoriadis, L'institution imaginaire de la société,
Paris, Éditions du Seuil, 1975.
2 Marco Deriu, “Conviviality,” in Degrowth – a new vocabulary for a new era,
ed. Giacomo D’Alisa, Federico Demaria, Giorgos Kallis, London, Routledge, 2015.
3 “Disnovation.org”, accessed December 6, 2022, https://disnovation.org/index.php.
4 Tony Fry, Defuturing, A New Design Philosophy. 2nd Edition, Bloomsbury: Bloomsbury
Publishing, 2020.
5 Michael Given, “Conviviality and the Life of Soil,” Cambridge
Archaeological Journal 28, 2017, 127–143.
6 Serge Gruzinski, La colonisation de l’imaginaire. Sociétés
indigènes et occidentalisation dans le Mexique espagnol (XVIe- XVIIIe siècle) Collection
Bibliothèque des Histoires, Paris, Éditions Gallimard, 1988.
7 Donna Haraway, “Cyborgs and Symbionts: Living Together in the New World Order,”
in: The cyborg handbook, ed. Chris Gray, New York, Routledge, 1995, xi–xx.
8 Donna Haraway, Staying with the trouble: Making kin in the Chthulucene, Durham,
Duke University Press, 2016.
9 Pasi Heikkurinen, “Degrowth: A metamorphosis in being,” Environment and
Planning E: Nature and Space Vol. 2, issue 3, 2019, 528–547.
10 Pasi Heikkurinen, “Degrowth by means of technology? A treatise for an ethos of
releasement,”Journal of Cleaner Production 197, 2018, 1654–1665.
11 Ivan Ilich, Tools for Conviviality, New York, Harper and Row, 1973.
13Serge Latouche, “Imaginary, Decolonization of,” in Degrowth
– a new vocabulary for a new era, ed. Giacomo D’Alisa, Federico Demaria, Giorgos Kallis,
London, Routledge, 2015.
14 Serge Latouche, Petit traité de la décroissance sereine, Paris,
Librairie Arthème Fayard, 2007.
16 Ilya Prigogine and Isabelle Stengers, Order Out of Chaos. Man's New Dialogue with
Nature, London, Flamingo, 1984.
17 Isabelle Stengers, Cosmpolitics I, Minneapolis/London, University of Minneapolis
Press, 2010.
18 “terra0,” accessed December 6, 2022, https://terra0.org/.
19 Anna L. Tsing, The Mushroom at the End of the World: On the Possibility of Life in
Capitalist Ruins, Princeton/Oxford, Princeton University Press, 2015.
Drivers for Resilience in Cultural Organizations: lessons from the Montreal festivals in the face of the
COVID-19 pandemic.
NicolasRicci, MarineAgogué
Center for Management Science, Mines - Paris HEC - Montréal
nicolas.ricci@mines-paristech.fr, marine.agogue@hec.ca
Abstract
Crises such as the COVID-19 pandemic represent appropriate moments to innovate. Many organizations in the
cultural sector have thus proposed numerous changes in their activities trying to develop new forms of
symbiosis, bringing back the notion of resilience. Beyond its buzz word aspect, resilience has essentially been
associated with a set of organizational capacities to adapt and innovate in the face of a disruption in the
cultural environment, leaving little consideration to question the main drivers of resilience in cultural
organizations.
We propose then to study the adaptation of the Montreal festivals offer, building on primary data from 8
interviews with festival directors or managers and secondary data from internal and external documentation. We
therefore mobilize the concept of the business model to identify and discuss the drivers for resilience in
cultural organizations. We show a trend for festivals to come back to their formal business model despite the
deployment of different innovations and identify role and purpose as the two main drivers for the resilience of
festivals.
Finally, we call for a comparison with other cultural organizations to discuss the preserving and reconfiguring
aspects of their resilience.
Keywords
Resilience, COVID-19 Pandemic, Festivals, Cultural organizations, Business models, Purpose.
The COVID-19 pandemic and the associated restrictions have generated a disruption in the offer and consumption
patterns of cultural products⁴. The transition to online has allowed a favorable development for cultural
sectors that did not depend on the physical presence of their public. On the other hand, very dependent sectors
such as cultural events in a broad sense, have faced significant difficulties despite various innovations in the
format and distribution of their cultural offer.⁶
Internally, the teams—with a high turnover rate in some cultural events—have been working
intensively since the start of the pandemic to reinvent themselves while preserving their identity. This has led
to the emergence of a tension between identity and constrained innovation.¹⁰ We offer with this paper
to clarify how this conflict was expressed within cultural organizations by questioning a well-known concept in
management when dealing with adaptation under constraints: organizational resilience.
Literature review
Organizational Resilience as a development of capacities to adapt and innovate in crisis
period
From its original definition in material science to the first ones in psychology, resilience has been defined
through the idea of a capacity of adaptation to a shock.¹ In management science, the development of the
concept of Organizational Resilience followed the same logic. For instance, Gibson and Tarrant define resilience
as an “adaptive capacity and how we better understand and address uncertainty in our internal and external
environments.”³ This observation can be confirmed by systematic review on the concept of
organizational resilience.¹¹
The essential of the literature thus focus on the description of this capacity. Begin and Chabaud¹ propose
a typology to describe the dimensions of organizational resilience:
An ‘absorption capacity’ associated with the survival efforts of an organization.
A ‘renewal capacity’ describing a reflective moment from the actors.
An ‘appropriation capacity’ referring to the new knowledge developed from the crisis.
Organizational Resilience as a reasoning moment for organizations
However, building on Kraemer definition of resilience in social work studies⁷, resilience can also be
defined with a phenomenal lens rather than the notion of capacity. According to the author, an acceptable
description of resilience depends on the context of your study (cultures, time, people, etc.) Thus, the fact
that resilience involves a phenomenon and a cognitive process from the actors to tackle it, constitutes the
common point between every form of resilience.
Organizational resilience can then be described through a succession of moments² illustrating successive
cognitive processes.
By considering a psychodynamic approach of resilience, Winkler¹² investigate actors thinking and
reasoning during a crisis period, leading to the consideration of individual drivers for resilience.
Yet, we lack such an approach of organizational resilience allowing us to identify the phenomenon of
resilience⁵ and discuss the main drivers for the reasoning of actors from cultural organizations in crisis
periods.
Methodology
Field of study: Why considering festivals to reflect the phenomenon of resilience within cultural
organizations?
Among the variety of cultural organizations, we propose to study festivals. Their activities deeply depend on
the physical gathering of publics and artists in a same place. Since the start of the pandemic, festivals have
faced deep changes in their business models and modified their formats in digital forms or cancelled their
editions and constitute then an interesting panel of forms of resilience. Moreover, festivals play a central
role in the exhibition of several artworks and represent a real platform in the life of cultural ecosystems in
general.
Data collection
We started by building a database registering most of the different festivals of Montreal (n=71) and
information found on websites and social media such as attendance, followers, period of the year, duration, type
of festival (i.e., music, cinema, visual arts, etc.), format of 2020/2021/2022 editions.
We contacted 68 of these festivals through direct mail to directors, contact mail, LinkedIn.
We interviewed 8 directors and managers between December 2021 and June 2022, with a variety within the
festivals in terms of size, duration, and types. The semi-structured interviews lasted between 30 minutes and 1
hour, we developed themes such as the adaptation of the festival team, the format before, during and after the
pandemic, their relations with the different stakeholders.
We completed this primary data with internal documents (audience studies, annual reports, internal
communication) provided by the directors and managers to complete our study. We also collected information from
websites and local press articles.
Data analysis
This study follows the principles of a grounded theory: we use data collected directly in our field of study to
develop intermediate theorization. To do so, we proposed an open code of the interviews content.⁸
We identified two major categories in the content: one part referring to the business model of the festival
(format, stakeholders, funding, etc.), a second one relative to remarks on the role of the festival, its
purpose. Then, we proposed to encode the information on the business model in canvas format for business
models.⁹ We then identified several codes for the drivers of reasoning that we gathered into two concepts:
“role” and “purpose.”
Results
Comparison of the Montreal festivals business models before, during and after the COVID19 pandemic: a trend for
a “back to normal” Gathering our coding from each festival interviewed, we propose three general
business model of Montreal festivals before, during and after the pandemic.
Pre-pandemic business model.
We built the pre-pandemic business model (Figure 1 (1)) around a value proposal common to all the festivals we
interviewed: offer a moment and a place for exhibit of specific artworks to a public. We identified a general
trend for the revenue streams (public fundings, private sponsors, event sales) and the cost structure (rent of a
physical place, equipment and furniture for the event, salaries). Both interviews and secondary data helped us
to define the public of festivals, channels, and public relationship management practices. We finally
highlighted recurrent public and private partners, key activities (artistic program, event organizing, public
management) and key resources (artists, representation place, festival team and volunteers).
Pandemic business model.
The pandemic business model evolved on almost every aspect (Figure 2). Most festivals turned into a digital
format, requiring a reconfiguration of resources and skills around online broadcast technologies. The value
proposal remains unchanged, illustrating the lack of will to change the concept of festival and their identity.
Nevertheless, the business model is only sustainable thanks to public support.
Post-pandemic business model.
The post-pandemic business model regains almost all its pre-pandemic properties (Figure 3). The digital turn is
hardly preserved. However, there is a desire to keep the public engagement tools that proved to be rather
effective during the 2020 and 2021 editions, such as additional content or contests.
Role and purpose as a driver for the resilience process
The comparison of the different business models reveals a clear absence of evolution in the value proposition
of festivals. The festival directors insisted on their wish to preserve the identity of the festival, directly
linked to the idea of “festival as a platform for artworks and artists promotion.”
Some directors admitted that they think a festival mainly for artists rather than for the public. The
willingness to keep strong links with the artists illustrates the role of festivals as a service for artists in
both exhibition and socialization and its associated benefits (collaboration, inspiration, artistic movement
evolution, etc.). As a result, the key point for the shift to online services delivered during the pandemic was
to preserve both moments of exhibition for public and moments between artists and practitioners.
Nevertheless, online festivals remained an unsatisfactory solution (not only for economic reasons), revealing
purpose as the second driver of the resilience of festivals. The directors associated the digital format to the
lack of informal moments and places for both festival teams, artists and public. The significant part of
informality and materiality in these events leaves little adhesion for remote formats, making the physical
format part of the festival purpose.
Discussion and conclusion
Our results show that role and purpose are fundamental drivers of resilience within festival teams involving
that:
Business models are intended to preserve the value proposition that
can then be described as representative of the festival's identity
The will of the teams is to "return to normal" as soon as the
restrictions linked to the pandemic are over
In sum, we were able to highlight a preservative dimension of organizational resilience in the case of
festivals. This leads us to question the different forms of preserving resilience that the COVID-19
pandemic has brought to the forefront in the cultural and artistic communities. Conversely, it is quite possible
to imagine forms of reconfiguring resilience in other cultural sectors, the study of which could allow
for an interesting theoretical confrontation.
Finally, we have seen that the business model is a suitable
tool for diagnosing and discussing the phenomenon of resilience, and that rather than an approach based on
adaptive capacities, we can think of resilience in terms of the drivers of the actors whose role and purpose
constitute.
(1) Figures of the business models are located at the end of this document
References
1 Bégin, Lucie, Didier Chabaud, "La résilience des organisations. Le cas d’une
entreprise familiale," Revue française de gestion 200 (1), 2010, 127-42.
2 Conz, Elisa, Giovanna Magnani, "A Dynamic Perspective on the Resilience of Firms: A Systematic
Literature Review and a Framework for Future Research," European Management Journal 38 (3), 2020,
400-412, https://doi.org/10.1016/j.emj.2019.12.004.
3 Carl A. Gibson, Michael Tarrant. s. d. "A “Conceptual Models” Approach to
Organisational Resilience," The Australian Journal of Emergency Management 25 (2), 6-12,
https://doi.org/10.3316/informit.084520139241216.
4 Julia V. Gnezdova, Vladimir S. Osipov, et Igor V. Hrip- tulov, "Creative Industries: A Review of
the Effects of the COVID-19 Pandemic," In Post-COVID Economic Revival, Volume II: Sectors, Institutions,
and Policy, édité par Vladimir S. Osipov, 2022, 159-71, Cham: Springer International
Publishing. https://doi.org/10.1007/978-3-030-83566-8_10.
5 Julia Hillmann, et Guenther Edeltraud, "Organizational Resilience: A Valuable Construct for
Management Research?" International Journal of Management Reviews 23 (1), 2021, 7-44,
https://doi.org/10.1111/ijmr.12239.
6 Olena Khlystova, Yelena Kalyuzhnova, Maksim Belitski, "The Impact of the COVID-19 Pandemic on
the Creative Industries: A Literature Review and Future Research Agenda," Journal of Business Research
139, février, 2022, 1192-1210, https://doi.org/10.1016/j.jbusres.2021.09.062.
7 Sebastian Kraemer, "Promoting Resilience. Changing Concepts of Parenting and Child Care,"
International Journal of Child & Family Welfare 4 (3), 2000, 273-87.
8 Ann Langley, "Strategies for Theorizing from Process Data," Academy of Management Review
24 (4): 1999, 691-710, https://doi.org/10.5465/amr.1999.2553248.
9 Alexander Osterwalder, Yves Pigneur, Business model generation: a handbook for visionaries,
game changers, and challengers, Vol. 1, John Wiley & Sons, 2010.
10 Michael G Pratt, Majken Schultz, Blake E. Ashforth, Davide Ravasi, The Oxford Handbook of
Organizational Identity. Oxford University Press, 2016.
11 Tennakoon Niranjala, Nadira Janadari, "Organizational Resilience: What it is and what it
isn’t? A Conceptual Review," Wayamba Journal of Management, 12 juillet, 2021, 171,
https://doi.org/10.4038/wjm.v12i1.7520.
12 Astrid Winkler, "Resilience as Reflexivity: A New Understanding for Work with Looked-After
Children," Journal of Social Work Practice 28 (4), 2014, 461-78,
https://doi.org/10.1080/02650533.2014.896784.
Metabolism and Art
HannahRogers, AdamBencard
University of Copenhagen, Denmark hannahstarrogers@gmail.com, adam@sund.ku.dk
Abstract
Metabolic arts should be added to the emerging interdiscipline of metabolic humanities and this paper will
discuss ways of defining metabolism that might be productive in helping to produce tools and touchstones for
metabolic readings of contemporary art before presenting examples of artworks which might be interestingly
illuminated by light of this sign, taking time to relish the process of these materially oriented internal
analysis coupled with how the work might be considered in terms of its broader implications for the concept of
metabolism.
Among the many challenges of our Anthropocene moment, is the new attention that must be paid to how humans and
other species are embedded within, exposed to, and even composed of the very materiality of a planet that is
rapidly changing. By focusing on metabolism, understood as “the chemical processes that occur within a
living organism in order to maintain life; the interconnected sequences of mostly enzyme catalyzed chemical
reactions by which a cell, tissue, organ, etc., sustains energy production, and synthesizes and breaks down
complex molecules” (Landecker 2011). As a fundamental biochemical process of life, metabolisms are
ubiquitous and multilayered, ranging from distinct multispecies bodily processes, to a variety of chemical
transformations across many organisms on a planetary scale, atmospheric and respiration relations between human
and plant metabolisms, the body as an environment, to notions of metabolism in a more purely metaphorical sense
in aesthetics. Tracing the workings of metabolism is a method for connecting change and dysfunction on a bodily,
social, and earth-wide scale. As John Bellamy Foster (1999) explains, Marx conceived of a dysfunction in the
form of a metabolic rift as a separation “between humanity and the soil, reflected in the antagonism of
town and country” (399).¹
Metabolic humanities appear to be a rising sign under which medical humanities, environmental humanities, and
agricultural corners might be united, and metabolicarts may be a productive stage for the three-legged stool of
ASTS to balance in tension around the subject.²
Metabolism might be taken to mean many things from distinct multi-species bodily processes, to a range of
chemical transformations across many organisms on a planetary scale, atmospheric and respiration relations
between human and plant metabolisms, the body as an environment, to notions of metabolism in a more purely
metaphorical sense in aesthetics. This is not to suggest that metaphor is absent in what is generally understood
to be the scientific notion of metabolism, indeed sociologist Hannah Landecker has shown how very entangled
social conditions and metabolic science are and have been.³ Landecker suggests that metabolism was
foreclosed by understanding the body as a machine, with a ledger, but this could also point to understanding the
body as a business, with immediate implications about capital and a relationship to Foster's Marx.
Metabolisms are ubiquitous but may be most noticeable in states of dysfunction. These dysfunctions are the
basis of medical sciences, the source of new ways of conceiving of ecological relations in a climate-changed
world, and are a longstanding way of diagnosing philosophical indigestions. Metabolism can be the constant that
brings methodological tendencies to the fore as a shared subject, and yet the variable historical understandings
of metabolism alone give rise to ages possible metabolic unfoldings so that our studies need not be bounded by
the current state of metabolic science and indeed, as ASTS scholars would argue, this is best understood in its
social, political, philosophical, and historical contexts. Metabolism serves so many possible needs for
scientists, scholars, and artists by providing metaphors, models, puzzles, solutions and balances. Metabolic
science is built on a stack of ever-modified metaphors,⁵ including the metabolism as an engine or motor
(fast/slow and often an emphasis on the notion of fuel/energy sources), furnace (hot/cold), “chain
reactions,” “chemical cascades,” the “chemical carnival,” and many others.⁶
Science is implicated in metaphor thinking, as even the classic experiment asks us to make correspondence
between the specific findings on the bench and the broader world, and this extends out to our public
understandings of science, often with further analogies which both simplify and make more culturally complex
these concepts. We suggest a focus on Landecker’s etymology for the term Stoffwechsel, translated as
“total metabolism,” with a further emphasis around the "stoff'' or in English “stuff” of
the process to emphasize the insistence bioartists have shown in exhibiting the stuff of living things and
parts.⁷ This emphasis on stuff has been important in bioart as such work has avoided distance and
representation except by a part of the actual living tissue, bacteria thing and at the same time it has insisted
on pulling against conceptual art to attempt to produce in material those ideas. Being in the presence of stuff
has been a hallmark of bioart and an emphasis on process may well be the hallmark of the metabolic arts.
The dual use Marx and Landecker make of, on one hand, critiquing soil chemistry or medical genetics, and on the
other hand, using those subjects as the basis for their thought, is a situation shared by bioartists who are
often using the very biological and biotechnical materials they are critiquing. Enter metabolic arts: drawing on
metabolism from the sciences, embodied experience of metabolism, and the potential of laboratory and home
metabolism practices to create encounters with life and life processes, many artists and art-aligned
practitioners have created art and art-science works that relate to metabolism. We observe and anticipate that
many artists may return to Marx's original thinking about metabolism, particularly, his interest in the way that
capitalist systems attempt to disentangle plants from animals including people, one natural and functional
system was separated into two sets of problems with planetary implications: urban/rural waste and food
insecurity/soil depletion.
Metabolic arts
The metabolic gaze can enrich understandings or experiences of a range of contemporary life-science engaged
artworks. Tissue Culture & Art (TC&A), made up of Oron Catts and Ionat Zurr, along with collaborators
specific to individual works, have offered a trajectory of artworks spanning from 2016 to the present, including
iterations of COMPOSTUBATOR and Sunlight Soil and Shit (De-)Cycle (3SDC).⁸
While each work is distinct, they represent the formation of the artists’ thinking on these subjects over
a number of instantiations. In the COMPOSTUBATOR series, a compost heap powers an incubator for
sustaining a group of cells, generally through microbes heating water in conduit which flows past the incubator
chamber and regulates the temperature. 3SDC was designed as a series of engagements around a set of
agricultural tools with an emphasis on considering their implications and the philosophies behind these and
other proposed system changes. 3SDC was launched with the COMPOSTUBA TOR in Freemantle, Australia,
positioned by the artists on their project website in relation to agricultural systems: “The heated
incubator sustains the growth of cells in a tissue culture flask to create what is today known as
“lab-grown meat.” This type of “meat” is the cornerstone of what is called Cellular
Agriculture – growing animal products without the animal.” A second portion of the project is the
use of Alkaline Hydrolysis (also Aquamation or Biochemiation), based on an 1888 process for turning
farm animals into crop fertilizer. The artists created their own version by hacking brewing equipment to break
down much smaller animal bodies (meat and fishing waste) to fertilize the Hydroponic Garden, which is
ironically supported by artificial lights driven by solar panels when possible. All of these elements are
connected by the Control Room which gathers data from sensors across the project (thermometers,
CO2 levels, pH monitors, cameras, etc.) The project aims to highlight the problem of increasing metabolic rifts
in order to solve agricultural issues, something that seems implicit in many of the lab-based food systems
proposed in the public sphere today. The artists write: “SymbioticA’s 3SDC builds resources to
enable the community to accelerate metabolic rifts in agricultural innovation. This project considers whether
the precursor to sustainable food systems will be the creation of a metabolic rift – where the means of
production will grow ever distant from nature.” The artists directly invoke Marx’s metabolic rift as
the target of their investigations by explaining that the exhibition is durational and will be changed over the
course of the exhibition. Their aim at “maintaining the utmost clarity and transparency of our process is
the key topromote understanding of the impact of metabolic rifts.” TC&A are working with metabolisms,
in its nuanced form at the microbial level and at the broader level of our food system.
Baum & Leahy’s Cometabolise: A Holobiont Dinner (2021) was a living sculpture and an
exploration in making the idea of the holobiont more familiar for viewers (Figure 1).⁹ A holobiont is an
assemblage of a host and other species living in or around it that together form a discrete ecological unit.
From a holobiont perspective, our bodies are permeable living environments for our cells and the cells (and
whole bodies) of other living things. It expands and blurs the notion of the host as a unit which could ever be
extracted from this entangled set of relations. The artwork insists on the overlapping metabolic processes of
the multispecies beings which help to metabolize our food. The artwork emphasizes the idea that bodies are
porous, multispecies entities, highlighting the fact that humans and microbes eat and drink together. This
bespoke dining set contains a sourdough culture held in a spherical glass carafe reminiscent of a bioreactor. A
closed container for a starter culture is, of course, one of our most familiar bioreactors. The piece reminds us
of the domestic nature of metabolism and invites thoughts of kitchens and laboratories. The artists emphasize
the performative nature of the work as the microbes are constantly metabolizing and are fed while visitors are
offered bread baked from the starter at the communal dining set.
George Gessert and Violet Ray’s BREATHE (2022) is a video piece investigating plant metabolism
and its circular processing with our human breathing. Given his history with plants as a primary subject and
medium for his artwork, Gessert’s work with plant metabolism is an obvious extension of those concerns.
The yet unexhibited filmic work which was created with Gessert’s longtime interlocutor but new
collaborator media artist Violet Ray, poetically explores the metabolism of plants through photosynthesis by
exhibiting the process at a cellular level and using text to invite audiences to connect their own breath to the
cycle of the plant’s photosynthesis and respiration. The artists ask us to pace ourselves with plants. As
Gessert puts it, as we think of metabolism, “Why stick to humans and animals? Photosynthesis creates the
air we breathe and is a key part of the planetary metabolism that supports most life on
earth.”¹⁰ This focus on the larger cycles that metabolism is implicated in overlaps with the
concerns of scale that appear across the metabolic arts.
Tagny Duff’s Wastelands (2015-2018) explores shit as an energy source in a speculative future
without fossil fuels. Duff explores a deep future 500 years away when humans, through collaboration with
bacteria and viruses, use their own feces as an energy source in small, portable bioreactors.¹¹ The
Wastelands Project relies on Duff's many years of experience working with biotechnologies in its artistic
practice, with a particular focus on viruses and on White Heather Hunter’s co-invention of a new
bioplastic with art conservator Courtney Books. The latter provided the basis for the biomaterial development
used to construct the bioreactor bags for the project. Metabolic ubiquity can create complications since living
things always involve metabolisms but all art with living things may not query those processes or engage them
directly. Yet, the search for metabolisms may provide new insights and new places of tension for contemporary
arts about life and the Anthropocene.
References
1 J. Foster, “Marx’s Theory of Metabolic Rift: Classical Foundations for Environmental
Sociology,” American Journal of Sociology 105, no. 2, 1999, 366–405.
2 H. Rogers, M. Halpern, Hannah D., K. Ridder-Vignone, The Routledge Handbook of Art,
Science, and Technology Studies, London, Routledge, 2021.
3 H. Landcker, “Food as exposure: Nutritional epigenetics and the new metabolism,”
BioSocieties 6 No. 2, Palgrave MacmillanUK, 2016, 167-194.
4 H. Landecker, “The Metabolism of Philosophy, in Three Parts,” In: Cooper, I. &
Malkmus, B., (eds.), Dialectic and Paradox: Configurations of the Third in Modernity, New York,
Peter Lang, 2013.
5 M. Black, Models and Metaphors, Ithaca, Cornell UniversityPress, 1962.
6 Taylor C See, B. M. Dewsbury, “On the Problem and Promise of Metaphor Use in Science and
Science Communication,” Journal of Microbiology & Biology Education, 19(1), 19.1.46 (2018),
Rabinbach, A. The Human Motor: Energy, Fatigue, and the Origins of Modernity, Basic Books, 1990.
7 A. Bencard, Stofsk(r)ifter: Metabolic Machines by Thomas Feuerstein. Hauser, J.
(ed.), A. Bencard (ed.), M. Grünfeld, L. Whiteley, 1 ed. Copenhagen, Medicinsk Museion, Jun 2020.
8 See the Tissue Culture & Art’s pages at Compostcubator 0.1 & 0.2 – The
Tissue Culture & Art Project (tcaproject.net) and 3SDC Sunlight Soil and Shit, sunlightsoilshit.systems.
9 For more, see Baum & Leahy’s art pages at Baum & Leahy, baumleahy.com.
10 Personal correspondence with the artists, February-April 2022.
11 For more, see Tagny Duff’s project page at Wastelands – Thoughts, images and
experiments considering human-microbial relations on Earth in 2517, wordpress.com, and Whitefather
Hunter’s documentation page at biotechnofeminism: laboratory craft | whitefeatherhunter
Ephemera: Bubble Representations as Metaphors for Endangered Species
HarpreetSareen1, YiboFu2, YasuakiKakehi3
1The University of Tokyo, 2,3 Parsons School of Design Bunkyo City,
Tokyo, Japan, New York, NY USA sareen@xlab.iii.u-tokyo.ac.jp, fuy644@newschool.edu, kakehi@iii.u-tokyo.ac.jp
Abstract
The effects of a hierarchical relationship of humans with non-humans are now more pronounced than ever.
Anthropogenic ecological stressors, including high levels of carbon dioxide, water scarcity, habitat
fragmentation have led to disruption of climate systems, in turn endangering many local and global species.
ephemera is an installation composed of glass vessels that show bubble images representing animals from all
continents and ecologies currently under threat as per the IUCN Red list. These self-assembling bubble pictures,
formed by nucleation of CO2 bubbles in water, are in a homeostasis at the beginning of the installation and
shrink each hour to eventually disappear in a few days. The tension between the present endangerment and the
urgency of the future action, manifests in the shrinking of these bubbles, invoking unnatural ephemerality due
to the human effect. The fauna pictures in this installation, composed of carbon dioxide bubbles, symbolize the
transitoriness of now threatened species.
In 2015, Hawaiʻi was called as the extinction capital of the world.¹ Occupying a small land mass
(0.25%), the islands had 25% of the endangered species in the US.² In intertwined ecosystems, direct and
indirect effects human activities—monoculture, greenhouse emissions, extraction, and urbanization—on
the biological diversity in a different location are significant³ but often hidden from sheltered lives of
humans in another place.⁴, ⁵ Separated from nature in this manner, our thinking of
endangered species is cultural, and public engagement with endangered species going forward depends on new
structures of imagination. How then should we invest culturally in the fate of endangered species? What emotions
do we collectively engender?
This work, an installation composed of ephemerality and melancholy, focuses not particularly on the beauty of
imperiled fauna but on their current transience. Glucksmann’s philosophy of the ephemeral⁶ is
particularly relevant where the moment is not static, but modulated and resonant. Drawing parallels with
flux-images, where the process takes precedence, Glucksmann calls out the aesthetics of the ephemeral as fluid
and polysensorial. In the context of this installation, images of various endangered species form with carbon
dioxide bubbles—bubbles that shrink in a few days gradually waning the entire image. Materials within this
installation’s cultural context are designed not for absolute control but for its relegation.
Concept
Jean-Louis Boissier called out the roles of images as an interface,⁷ which we can act upon, manipulate,
and transform—an image that vectorizes a relationship. We focus on such a relationship through evocative
material with embedded cultural context of greenhouse gases and their effect on fauna. Specifically, the focus
is on the ephemerality of the images. In prior transient works
⁸,⁹,¹⁰,¹¹,¹² durability
is determined by the intrinsic material properties in combination with the surrounding ecosystem, often
deliberately designed to offer only partial or imperfect control. Material semantics - original meaning of a
material - shape the perception and overall experience. The ambiguous ontological status of the ephemeral also
makes it a “powerful metaphor for expressing nuances of memory, time and knowledge.”¹³
Interpretation of ephemeral meanings and consequences is thus beyond the object itself.¹⁵
In this work, images of endangered animals form on glass vessels through thousands of carbon dioxide bubbles.
These bubble formations shrink by the hour to eventually disappear in a few days. The bubbles are formed by
nucleation of CO2 bubbles in water and are representational images of animals from continents and ecologies
currently under threat.
The installation is composed of ten glass vessels that are hydrophobically treated for controlled
micro/nanostructures on surfaces. When carbonated liquid media is poured onto the surface, bubble generation is
activated and bubbles of carbon dioxide of various sizes stick at defined points on the surface. Inspired by
Sylvester et al., who mentioned bubbles as a material¹⁶ between “neither real nor fully
virtual,” we extend the temporality of bubbles in this work. The process from initial nucleation to
stabilization takes an average of twenty minutes, depending on the saturation of carbon dioxide in the liquids.
Bubble images and patterns are stable for one day before they start to gradually shrink, completely disappearing
in five days.
System Design
We created a novel fabrication technique to selectively modify hydrophobicity properties of surfaces
(glass/plastic) such that bubbles in carbonated liquids nucleate at specific positions on surfaces.
This turns stochastic nucleation into controlled nucleation of bubbles for images, patterns, and text on
various surfaces. Such surface modifications are initially invisible to the human eye. On pouring carbonated
water in containers of various shapes (horizontal/vertical) and sizes, bubbles nucleate, coalesce, and grow to
accurate sizes (1.0 mm – 5.5 mm) thus creating patterns, images, or text on surfaces. Such bubbles are
highly stable and can keep sticking to the surfaces for longer than a week without significant disturbance.
To fabricate the glass vessels, we use Regular (Home Depot, 599047) glass as a substrate which is initially
hydrophilic (wettable) in nature. An illustration software (Adobe Illustrator) or our custom design tool is used
to create desired patterns and print out an image mask using a vinyl cutter (Cricut Maker 3). The sticker mask
is pasted on the surface and a ceramic hydrophobic coating is applied manually on the glass surface using a
zigzag technique before drying for 24 hours. This creates a surface with distinct wettability regions that are
invisible to the human eye.
When carbonated water is poured into the containers, small bubbles nucleate first coalescing into each other
becoming large in size corresponding with the size of the coating on the surface. These large bubbles may become
buoyant at first due to the saturation level of CO2 in the liquid. As levels of gas normalize, bubbles stabilize
at nucleation sites adhering to the hydrophobic regions with an attractive force¹⁷. Such stabilized
configurations of bubbles last for five to seven days.
To design the bubbles patterns for this installation, we create a custom tool based on p5.JS that allows
automatically generated wettability patterns by consolidating image import (color or grayscale), grayscale
conversion and corresponding bubbles pixelation in a single pipeline. User imported images are analyzed against
the background. Our algorithm analyzes the input image to create a comparative brightness map, following which
darker areas of images are tagged as proportionately dense in bubble density and lighter areas are sparse in
bubble density. For easy accessibility, the tool may be used online and outputs a .svg directly compatible with
the fabrication machine for cutting the masks.
Exhibition and Experience Walkthrough
Purpura et al. have previously called ephemeral art been as “good to think with.”¹⁴
Their impermanence is a constitutive part of their aesthetic, and of the ways in which they come to act on the
world. In the context of the exhibition shown at a major venue, our intention was to generate emotions of
sadness or mourning among the viewers, over the endangerment of species. We reviewed the IUCN Red List of
imperiled species at various threatened levels and chose ten species from marine, land and amphibious domains
across all continents. The vessels were fabricated as per techniques described before, bottom lit for contrast
of bubbles in liquid and setup in an L-shape. These were specifically separated from each other and laid on the
floor for two key purposes: a) For viewers to bow their heads down or kneel as if paying homage, and b) To
portray the vessels as final mementos of threatened species.
Such an exhibition design was shaped with a perspective of higher-levels of abstraction, focusing on carbon
dioxide bubbles as material from a cultural perspective. This work is thus not temporary, rather has a directive
intent to survive in the memory.
Conclusion
We presented ephemera, an installation where ephemeral images of endangered species are composed
through bubbles of carbon dioxide. Bubbles with extended lifetimes slowly disappear in five to seven days,
representing the images of endangered species as final mementos for humans. Through our exhibition, we intended
the viewers to observe images in a drifting state and to be involved in the relationship depicted by the image.
Glass vessels arranged onto the floor and viewed by the audience while kneeling are meant to invoke mourning or
sadness among viewers—emotions that were listed over aspects of nature during the rapid modernization
progress. Through this work, we explore a new ephemeral material and its aesthetic of affect, and believe that
the ephemerality in this work represents a shift from the art object to a communicative act.
Acknowledgements
This work was initiated and primarily supported by JSPS KAKENHI (Grant Number 20H05960). We also thank the
Provost Office Fund at The New School, and Labex Digi-Cosme (ANR11LABEX0045DIGICOSME) operated by ANR as part of
Idex Paris-Saclay program (ANR11IDEX000302) for their partial supports.
References
1 Washington Post, (n.d.), Is Hawaii ‘the extinction capital of the world’? Exhibit A:
The alala bird. Retrieved December 8, 2022, from
https://www.washingtonpost.com/national/health-science/is-hawaii-the-extinction-capital-of-the-world-exhibit-a-the-alala-bird/2016/04/25/3f45c6ac-f210-11e5-89c3-a647fcce95e0_story.html
2 Jason D. Baker, Charles L. Littnan, David W. Johnston, "Potential effects of sea level rise on
the terrestrial habitats of endangered and endemic megafauna in the Northwestern Hawaiian Islands," Endangered
Species Research 2, 2006, 21-30.
3 Chen Xinli, Han YH Chen, Chen Chen, Ma Zilong, Eric B. Searle, Yu Zaipeng, Huang Zhiqun,
"Effects of plant diversity on soil carbon in diverse ecosystems: A global meta‐analysis,"
Biological Reviews 95, no. 1, 2020, 167-183.
4 Judith Littleton, Gina McFarlane, Melinda S. Allen, "Human–animal entanglements and
environmental change: Multispecies approaches in Remote Oceania," In The Routledge Handbook of the
Bioarchaeology of Climate and Environmental Change, 493-510, Routledge, 2020.
5 K. Anggi Hapsari, Tim C. Jennerjahn, C. Lukas Martin, Volker Karius, and Hermann Behling,
"Intertwined effects of climate and land use change on environmental dynamics and carbon accumulation in a
mangrove‐fringed coastal lagoon in Java, Indonesia," Global change biology 26, no. 3, 2020,
1414-1431.
6 Christine Buci-Glucksmann, Emanuele Quinz, "For an Esthetics of the Ephemeral. Interview with
Christine Buci-Glucksmann," Hybrid, Revue des arts et médiations humaines 1, 2014.
7 Margaret C. Flinn, "Jean-Louis's Moments of Jean-Jacques." Studies in French Cinema 10, no. 2,
2010, 141-154.
8 Daisuke Uriu, Naohito Okude, "Thanato Fenestra: photographic family altar supporting a ritual to
pray for the deceased," In Proceedings of the 8th ACM conference on designing interactive systems,
2010, 422-425.
9 Virolainen Antti, Arto Puikkonen, Tuula Kärkkäinen, Jonna Häkkilä, "Cool
interaction with calm technologies: experimenting with ice as a multitouch surface," In ACM International
Conference on Interactive Tabletops and Surfaces, 2020, 15-18.
10 Elizabeth Diller, Diana Murphy, Ricardo Scofidio, Blur: the making of nothing, Harry N
Abrams Incorporated, 2002.
11 Suzaan Boettger, "34. Within and Beyond the Art World: Environmentalist Criticism of Visual
Art," Handbook of Ecocriticism and Cultural Ecology, 2016, 664-682.
12 Parés Narcís, Jaume Durany, Anna Carreras, "Massive flux design for an interactive
water installation: WATER GAMES," In Proceedings of the 2005 ACM SIGCHI International Conference on
Advances in computer entertainment technology, 2005, 266-269.
13 Allyson Purpura, "Framing the ephemeral," African Arts 42, no. 3, 2009, 11-16.
14 Marilyn Ekdahl Ravicz,"Ephemeralart: A Casefor the Functions of Aesthetic stimuli," 1980,
115-134.
15 Daniel Miller, "Artefacts and the meaning of things." In Companion encyclopedia of
anthropology, 430- 453, Routledge, 2002.
16 Axel Sylvester, Tanja Döring, Albrecht Schmidt, "Liquids, smoke, and soap bubbles:
reflections on materials for ephemeral user interfaces," In Proceedings of the fourth international
conference on Tangible, embedded, and embodied interaction, 2010, 269-270.
17 Chen Shi, Xin Cui, Xurui Zhang, Plamen Tchoukov, Liu Qingxia, Noemi Encinas, Maxime Paven, et
al. "Interaction between air bubbles and superhydrophobic surfaces in aqueous solutions," Langmuir
31, no. 26, 2015, 7317-7327.
Jellyeyes – Symbiosis, Evolution and Vision
JillScott
ZHdK, Zurich, Switzerland info@jillscott.org
Abstract
Jellyeyes is an augmented reality installation especially created for the viewer based on symbiosis, touch, and
biomimicry. Jellyeyes is inspired by narratives from the theories of evolution, and these stories involve the
characters of a hunter-diver-tourist and a marine biologist. These narratives are inspired by the personal
diving experiences of the writer and residencies in three different science research centers that focus on
evolution in relation to symbiosis, marine biology, and neuro-visual systems.
In 1967, Lynn Margulis created the concept of endosymbiosis. In this scientifically renowned theory, she
claimed that bacteria was the agent of change.¹ Specifically symbiotic bacteria invaded the primary cells
to create mitochondria and chloroplasts, the powerhouse parts of the cell as we know it today. Jellyeyes is
about the focus of this theory on the evolution of light cells or fluorescent chloroplasts. They are found in
fluorescent bacteria that live in the symbiont algae or Zooxanthellae inside any coral reef. These bacteria
called Cyanobacterial Endosymbiont evolved to create fluorescent light that acts as a sun block for the coral
and is sensitive to temperature change. These chloroplasts are also found in the pigmented cells of our eyes
called Rhodopsin, that move through the photoreceptors of our retinas in reaction to incoming light. (Figure
1)
This research in evolution and symbiosis led to discussions at conferences with marine biologists in
Australia.² In the Great Barrier Reef, symbiotic relationships between fluorescent algae and modern corals
are currently being affected by human impacts on the lives of species in this environment. In fact, the health
of these symbiotic relationships between the algae and corals, can shift the very roles they play in aquatic
ecosystems, for example, as helpers or guardians against pathogens or enemies. The death of symbiont algae can
be seen in the latest sad map about coral bleaching in Australia. (Figure 2) Unfortunately, a warmer climate not
only affects chloroplasts in algae but symbiont levels of co-habitation as well as some species levels of visual
acuity.
Neuroscientists study the effects of excess light on photoreceptors, and their pigmented chloroplasts.³
Human photoreceptors have evolved through adaptation and are very similar to the retinas of the box jellyfish
and the squid. Classified in evolution under the label of camera-based eyes, all three species share two
jelly-like substances; aqueous humor or the moving fluid in the eye that is affected by temperature and vitreous
humor: the jellylike shapes with fibers that are attached to their retinas. Hence the title: Jellyeyes!
Because liquids in these eyes are affected by UV exposure, an increase in bad conditions cause cellular damage
and very dry eye problems.³ Photoreceptors need healthy chloroplasts to function accurately. In humans they
control low and high light levels and the photoreceptors are formed from neural tissues and directly attached to
the brain (optic nerve). In squid: they control low light levels in photoreceptors that are also attached to
their brains, but in jellyfish, they use basic pigmented cells to assess light and these are attached to each
other by fibrous tissues. This last species has no brain but what is called a distributed brain. (Figure 4) But
how can such information on evolution in neurocience, marine biology and symbiont behaviour be interpreted in an
artwork? Could Augmented Reality and biomimicry help us “see” and understand the reef environment
through the light receptors of the eyes of other species?
Artwork
In summary, Jellyeyes is such an augmented reality investigation. It consists of a large photograph of a dead
barrier reef and an iPad built into a sculpture of the human optic nerve. (Figure 3) The image on the iPad
brings the photo alive! Sounds of water and bubbles immerse the viewer into the environment. The aim is to
explore the delicate balance of this aquatic ecosystem through the 24 eyes of the box jellyfish, our own human
eyes and eyes of squid or calamari. But the viewer can choose a temperature gauge to warm the water, and this
changes the environmental conditions! Ocean warming affects the predator-prey relationship of these species and
the symbionts algae leave the coral to die. The viewer can select the magnifying glass to discover how the loss
of this symbiont caused a loss of a food for the coral to eat, a weakened immune system, bleaching, disease, and
death. Warming also creates other food-algae and more jellyfish breed in the reef. In another scenario they can
choose to have an empathetic and parabolic look through the eyes of the Australian box jellyfish at the changing
predator-prey relationships of species in the reef. The viewer can choose the black and white view of the
squid’s eye to discover stories about the human hunter or they can use the human eye to see different
narratives.
The whole artwork is also available on an iPhone with a downloadable screen-based photograph. Here the aim is
to help tourists understand the ecology and the climate change problems on the reef.
But how exactly are these narratives chosen by the viewers?
Evolution and Roleplaying
In Jellyeyes, these narratives are divided into three sections or menus that the viewer can explore:
Co-Evolution, Structural Evolution and Comparative Evolution.
Co-Evolution:Here bacteria are seen as the agents of change.⁴ In this tribute to Lynn
Margulis, Jennifer Margulis (her daughter) is turned into the character of a marine biologist, who collects
evidence about the state of symbiont death in the coral and refers to her mother’s theories. She collects
fluorescent bacteria from the algae (zooxanthellae) and measures the corals level of nutrition and growth, how
they block excessive sunlight, control toxic compounds, stress levels and even ward off pathogens who attack the
algae. The viewer’s interaction causes acidity and symbiotic destruction in this environment. (Figure 5)
Structural Evolution:This menu takes Charles Darwin’s theory of adaptation⁵ and
applies it to the camera-based eye. How has this eye evolved in relation to its lineal ancestors. The result is
a simplified tree of the evolution of visual perception. (Figure 8) For example, the jellyfish is an early
example of our eye’s development and the squid another stage in the evolution of the light receptors of
the camera-based eye. In this narrative, an ignorant, destructive tourist steals too much food. Sometimes, even
digs into the coral to the horror of the scientist who tries to teach respect for this environment. (Figure 7)
Comparative Evolution:James Lovelock once said that life has created the conditions for its
own existence.⁶ In this menu, the viewer can choose either the eye of the box jellyfish, the squid, or
their own eye to see the relationship between the behavior of these three species in the barrier reef
environment. The ignorant diver tourist plays around with the poisonous Australian box jellyfish and every day
more of these very stingers swim in the reef. Ocean warming shifts the ecology of the reef’s predator-prey
relations and behaviors like these will have a backlash and harm our own existence. (Figure 7)
As in previous artworks,⁷ Jellyeyes uses the sensorial strategies of touch and biomimicry, to create
performative roleplaying to augment the above narratives and their repercussions. In Co-Evolution the role of
the participant becomes the CO2 emitter and through touch he or she or they, learn about the reactions of
symbiont bacteria, the survival of the coral, depreciation of the symbiont algae, squid, and reproduction of the
box jellyfish, as well as the interdependencies between these species. In Structural Evolution, the participant
becomes the investigator and learns through touch about the similarities of vision between species and about the
evolution and health of the light receptors in the photoreceptors. In Comparative Evolution, the viewer becomes
an empathizer, they see through the eyes of these species and can change these views with touch to identify what
they see. Here, the biomimicry of sight is used to witness the human environmental impacts on the reef
itself.
Conclusion
Perhaps, augmented reality art might be an effective catalyst for climate activism and education. Here the aims
are to reveal the influence of evolution on species survival in relation to the health of their habitats, to the
need for inherited variations and to the mutual benefits of keeping symbiosis alive. It is dangerous for us and
them not to mitigate climate change. Jellyeyes clearly shows the effects of our fossil fuel emissions on the
survival of the species in the reef. It offers a novel creative interpretation of structural evolution and
ecological interconnections. Here, the viewer becomes immersed in the symbiotic sensory relationship between
algae, vision and modern corals. Viewers say that Jellyeyes encouraged them to think about variations and
reproduction problems as well as symbiotic mutualism. They empathized with non-humans by “seeing”
through the eyes of other species. Afterall, the future of the barrier reef on this planet is in the eyes of
many species, but it is now in our hands!
References
1 L Fester, R. Margulis, Symbiosis as a Source of Evolutionary Innovation Speciation and
Morphogenesis, MIT Press, 1991.
2 ACRS Australian Coral Ref Society Conferences,
https://australiancoralreefsociety.org/conference/.
4 L Margulis, Symbiotic Planet: A New Look at Evolution Basic BVooks, 1999.
5 Charles Darwin, On Origin of Species, Pub John Murry, 1859.
6 James Lovelock, GAIA: A new look at Life on Earth, Oxford Uni Press, WW Norton, 1988.
7 Neuro_Eco _Media, www.jillscott.org
Partners
HFF, The Munich Film Academy. Munich, Germany Dr. Lisa-Ann Girshwin (CSIRO Australian Marine Stinger
Advisory Services, Australia)
Scientific Consultants
Prof. Dr. Stephan Neuhauss (Institute for Molecular and Cellular Research, Neurobiology, University of
Zurich, Switzerland) Prof. Christopher Robinson (Aquatic Research EAWAG, ETHZ, Switzerland)
Funding
Pro Helvetia, The Swiss Arts Council HFF, The Munich Film Academy. Munich, Germany
Production Credits
Concept & Directing: Jill Scott Production: Marille Hahne Camera: Julia Daschner Light: Georg Nikolaus
Animation: Natascha Jankovski Postproduction: Moritz Huber Programming: Nikolaus Völzow
Biography
Jill Scott is a media artist, a writer and art and science researcher. She is professor emerita at the Zurich
University of the Arts (ZHdK) in Zürich and founded their Artists-in-Labs Program in 2000. Her own
artwork spans 44 years of production about the human body and body politics. In the last 20 years she has
focused human health based on research into molecular biology, neuroscience, and ecology. She has had many
international exhibitions in both art and science venues. She also directs LASER Salon in Zurich for the
Leonardo Society USA and writes books on art and science (Springer and de Gruyter).
Technoshamanism: Symbiotic Techniques of Art and Healing
EdwardShanken
University of California, Santa Cruz United States eshanken@ucsc.edu
Abstract
Technoshamanism combines traditional shamanic technologies with emerging technologies based in silicon (dry),
biology (wet) and hybrid (moist), in the service of healing and sustaining life. This paper explores how
contemporary artists pursue expanded forms of consciousness by symbiotically joining technoscientific tools and
shamanic techniques.
More than ever before, contemporary artists, theorists, scientists, and activists need to pay more
attention to socalled indigenous knowledge. – Guillermo Gomez-Peña
Introduction
Mircea Eliade defined shamanism as “techniques of ecstasy.” He noted that, across cultures, it is
fundamentally a technology of healing that is "at once mysticism, magic, and religion."¹ Anthropologist
Michael Harner claimed that shamanic traditions around the world have developed a broad range of technologies -
from sonic drivers (drums) to plant medicine rituals—in order to achieve trance states that offer insights
beyond those available to typical waking consciousness.²
Shamans are both of this world and of the world(s) beyond. They communicate with spirits and ancestors in other
dimensions, learn from them, guide members of the community to them, and harness their power to heal and protect
individuals and the community. The shaman can embody the consciousness of other beings, including other animals.
A shaman also can release an errant spirit that has stricken a member of their community with illness, sending
the uninvited “hitchhiker” on its way. This can be dangerous, so shamans must be very strong,
capable of self-healing, and masters of their practice.
Shamanism may heal at scales ranging from the individual to the global. Indeed, following the Huni Kuin concept
of Xinã bena–“New Time”—some shamans/pajés of the Amazon have
decided that the preservation of the Earth demands sharing their cultural insights and healing technologies more
widely than in the past. This strategy has even led to the creation of Huni Kuin—Beya Sina Bena,
a videogame about the ancient stories of these indigenous people, scheduled for release in 2024. The developer,
Philosophical School of Games, claims that the game was produced in collaboration with the Huni Kuin communities
of the Jordão River, in Acre, Brazil.
Brazilian cultural critic Fabiane Borges notes that “apart from any possible encounter between technology
and shamanism, shamanism is itself a technology for the production of knowledge. ³ Similarly, I take
technoshamanism to join a combination of traditional shamanic technologies with emerging technologies based in
silicon (dry), biology (wet) and hybrid (moist), all in the service of healing and sustaining life.
The same can be said of art, historically interwound with techne, the Ancient Greek word for art and
the etymological root of technology. To risk stating the obvious, the use of shamanism for healing vastly
predates the application of science to allopathic medicine. The recent decriminalization of plant medicines and
the legalization of psychedelic-assisted therapies in some jurisdictions demonstrate increasing acceptance of
ancient indigenous technologies and marks a merger of diverse healing methodologies.
In a parallel manner, the engagement of contemporary art with shamanic traditions bridges knowledge domains and
augments art’s potentials to heal and to envision the future. These goals characterize a wide range of
artistic practices, from Ernesto Neto’s A Sacred Space to Roy Ascott’s “shamantic
web” of dual consciousness, to Anandha Ray’s Covenant VR virtual reality
ceremony/performance. A shaman was even part of Refik Anadol’s cohort of artists and software engineers at
the Google Artists and Machine Intelligence program in 2016.
Nonetheless, there is a dearth of literature that addresses the nexus of contemporary art and shamanism. Until
recently, the work of Joseph Beuys dominated these discourses, with a few notable exceptions. Jack
Burnham’s “Artist as Shaman” (1974) applied a structuralist method to interpret shamanic
aspects of Dennis Oppenheim’s work. In “Art in the Dark” (1983) Thomas McEvilley interpreted
performance by artists from Gunter Brus to Kim Jones as shamanic acts of expression and catharsis.
“Weaving the Shamantic Web...” (Ascott, 1998) claimed that “this ancient ritual mirrors our
contemporary artistic aspirations using digital technologies.” ⁴ Further essays by Ascott and his
circle, especially Lila Moore and myself, have followed in the 2010s and 2020s.⁵
In 2017, Christine Macel curated The Pavilion of Shamans, one of nine “trans-pavilions”, bringing
shamanism into the center of mainstream contemporary art (MCA) discourses at the 2017 Venice Biennale. Notable
works included Juan Downey’s Circle of Fires Vive (1979), Jeremy Shaw’s Liminals
(2017), and Neto’s A Sacred Place (2017), the centerpiece of the pavilion. Despite scant
media attention directed to this pavilion, art critic Tess Thackara (2017) highlighted it and discussed many
additional international artists, indicating broad interest in the topic, if not by MCA critics, then by makers.
⁶ Indeed, my 2020 Facebook post about technoshamanism yielded over one hundred comments, primarily by
artists about works of new media art (NMA) that fit the term. More recent MCA scholarship includes critiques of
Mircea Eliade’s and Michael Harner’s universalizing of shamanism and Claude Levi-Strauss’ and
Burnham’s psychopathology of the shaman, and is informed by Viveiros de Castro’s concept of
“equivocation,” offering a more nuanced and theoretical approach to shamanism and contemporary art.
⁷
Borges and others use the term “technoshamanism” to problematize the destructiveness of western
technoscience and its incursion on indigenous technologies of shamanism, which, by contrast, honor the sanctity
of the Earth. Technoshamanism names the messiness of cultural hybridity and the commodification of shamanic
traditions, including ayahuasca tourism and the gamification of Huni Kuin culture, resulting from colonization
and globalization. The visual arts of indigenous cultures, including Shipibo textile patterns, Papunya Tula dot
paintings, and Vodun rituals, are also being hybridized with, and appropriated by,
international contemporary art and visual culture. Such appropriations raise vital questions about
Technoshamanism as a form of aesthetic practice: How are contemporary artists engaging with the cosmologies,
technologies, and intellectual property of shamanic cultures in ways that honor and benefit the indigenous and
mestizo peoples that have cultivated and preserved shamanic traditions? How are indigenous artists engaging with
shamanic (and postindustrial technologies) in ways that strengthen their communities and contribute to the
discourses of contemporary art?
Mindful of these issues, my research is primarily concerned with the following speculative prospects:
How can artists embrace visionary consciousness?
How can art support entheogenesis (becoming divine together) by joining ancient shamanic techniques and
contemporary technoscientific tools?
How can art catalyze greater awareness of what Thich Nhat Hanh calls interbeing (the unity of all things) to
help heal the Earth?
Early Technoshamans
In 1997, British artist Roy Ascott participated in ayahuasca ceremonies in Brazil, which has had a profound
impact on his praxis. He theorized parallels between the dual consciousness that emerges in shamanic ceremonies
and the expanded field of consciousness afforded by emerging technologies. “In many respects,” he
wrote in 1998, “this ancient ritual mirrors our contemporary artistic aspirations using digital
technologies.” Claudia Jacques claims that this dual consciousness can only be manifested through
“indirection, shared participation, and metaphor,” core tenets of the telematic art that he
pioneered. This is the realm, she adds, that “visionary thinkers, creative artists, and shamans alike
aspire to experience and explore.” ⁸
Ascott (1998) describes the shaman as “the one who ‘cares’ for consciousness, for whom the
navigation of consciousness for purposes of spiritual and physical wholeness is the subject and object of
living.” He witnessed the pajé “passing through different layers of reality, through different
realities...” The shaman inhabits a state of dual consciousness, of “seeing at once both inward
realities and the outward surfaces of the world.”
Although it predates Ascott’s participation in shamanic plant-medicine ceremonies, his 1989 telematic
artwork, Aspects of Gaia: Digital Pathways Across the Whole Earth, embodies a technoshamanic, dual
consciousness. This installation integrates the global relationality of digital technologies with the shamanic
commitment to healing the Earth. Aspects of Gaia took its inspiration from atmospheric scientist James
Lovelock’s holistic Gaia Hypothesis (1979), which proposes that the Earth (Gaia) is a living
organism, a self-regulating, complex system that maintains the conditions for life on the planet. Artists,
scientists, shamans, musicians, visionaries, and indigenous artists were invited to participate by sharing their
feelings and beliefs about Gaia, which became work’s content. Ascott likened the participants to healers
who access the meridians of the earth’s nodes and creatively interact with the flow of data to perform a
type of “global acupuncture.” Their contributions suggested, for Ascott, a telematic
“noosphere,” an emergent field of consciousness generated from interconnected individual
consciousnesses, which might help harmonize and heal the planet.” ⁹
For American composer and electronic music pioneer Pauline Oliveros, cultivating expanded forms of
consciousness was the primary focus of her career. For this early technoshaman, the wisdom of the body is
crucial to accessing expanded states of consciousness. “I have progressed through many changes in music
technology from the end of the 1950s to the present,” she wrote. “Along the way I developed a bodily
relation to machines for making music. It has always been necessary.... because of the essential knowledge of
the body that is preconscious and nonverbal.” ¹⁰ Her experience of performing improvised music
parallels shamanic double-consciousness: Oliveros became a channel through with spirits from other dimensions
could communicate with and heal us: “This altered state of consciousness in performance is exhilarating
and inspiring,” she explained. “The music comes through as if I have nothing to do with it but allow
it to emerge through my instrument and voice.” ¹¹ Oliveros’ ideal attributes for a future
artificial intelligence “chip”—with which she could make music—are refreshingly
expansive. They include abstract psychic abilities, that seem to parallel shamanic intentions of achieving unity
and healing on a cosmic level. These include: the ability to understand the relational wisdom that comprehends
the nature of musical energy; the ability to perceive and comprehend the spiritual connection and
interdependence of all beings and all creation as the basis and privilege of music making; the ability to create
community and healing through music making; the ability to sound and perceive the far reaches of the universe
much as whales sound and perceive the vastness of the oceans. This could set the stage for inter-dimensional
galactic improvisations with yet unknown beings.¹²
Wake me up when ChatGPT can offer these features! For Oliveros, expanded consciousness on a galactic scale was
the foundation of healing, a form of service that she performed within the frame of a feminist ethics of care.
Chilean-American artist Juan Downey is another early technoshaman. His 1979 video, The Laughing
Alligator, resulted from the many months he and his family lived with the Yanomami people in southern
Venezuela, on the border of Brazil. He participated with his Yanomami hosts in rituals using yopo, a plant
medicine that contains a variety of DMT alkaloids. The artist came to regard shamanism as one of the most
powerful elements in the bond that linked the Yanomami to their surroundings, to the earth, and to each other as
part of a larger unity. As he wrote, A white and round place opens up in the front of my brain. Excretions
of light that vaguely align into circles, the intensity of a spiral or the infinite peace of a mauve color.
[...] I want to enter into the white space of my empty consciousness. ¹³
From spending a lifetime in trance-like states, Downey expanded his consciousness. He tuned into the
consciousness of others. And he helped others do the same. As a result, his work plays an important role in
stimulating new ways of thinking that are the prerequisite to healing society’s pathologies and to
recreating the world in a more sensitive, inclusive, and caring way.
Contemporary Technoshamans
American dancer and choreographer Anandha Ray’s virtual reality screen-dance, Covenant VR
(Figure 1), was a revelation. Joining shamanic healing rituals, modern choreography and dance
performance, and exquisite VR cinematography by Gary Yost, it created a unique container for embodied catharsis
mediated through virtual reality.
After putting on the headset, the next thing I knew, BAM! I was suddenly transported to a dance rehearsal
studio, at a virtual distance no more than a meter from a striking, dark-skinned dancer (Linda Steele II.) I
felt right there in the action, far closer than front row seats, inhabiting a perspective and proximity
typically seen only by other dancers. Because this is a 360-degree VR experience, one’s view is not
limited to a frontal perspective.
As viewers enmeshed in the VR environment, we witness the shamanic ceremony from a privileged vantage, so close
that we can virtually touch the protagonist, a proximity that heightens our sense of being there, of feeling the
healing and of experiencing it ourselves. Yost’s cinematographic wizardry is the ideal match for
Ray’s shamanic approach to dance, which “allow[s] movement to open portals of inquiry to better
understand the state of being human.” ¹⁴ This tour-de-force of Technoshamanism offers
an enthralling and cathartic experience.
Korea has a rich, living tradition of shamanism that continues to inspire artists, including technoshaman Kim
Jeong Han. His high-tech multimedia installation BirdMan (2005) is deeply informed by Korea’s
shamanic and Buddhist traditions. The artwork asks fundamental questions: How do human beings conceptualize the
world? How do birds conceptualize it? Can a hybrid world that joins human and nonhuman qualia (the internal,
subjective physiological component of sense perceptions) transform perception beyond the limits of human
physiology? If, as Donna Haraway claims, new perceptions create new metaphors, can the experience of another
species’ perceptual reality help create hybrid perspectives, marked by greater empathy and ecological
sensitivity?
The concept of a hybrid bird-man appeared to Kim in a dream. Due to a traumatic childhood experience, even as
an adult, Kim’s fear of birds prevented him from helping a one-winged bird that was suffering and dying.
He dreamed that he learned bird language from a monster with a bird head and only one wing. The dream and the
artwork can thus be interpreted as an effort by the artist to attain catharsis and heal a trauma through
aesthetic and shamanic means. As Kim and his co-authors noted:
“In Korean tradition, some shamans can share their own bodies with the deceased soul. Whenever a shaman
is possessed by the spirit of the dead, s/he acts, speaks and senses like another person, as if borrowing the
perception of the deceased. This moment looks like a coexistent state of the living body and the dead in which
perception and identity of the two is hybridised.” ¹⁵
The Buddhist idea that “the ‘Self” is not different than the “Other” is another
prevailing concept in BirdMan. The work offers the audience an opportunity to experience a form of
hybrid perception that joins human and avian “qualia.” (internal and subjective physiological
component of sense perceptions). Kim’s work does not explicitly represent accoutrements or scenes of
shamanic healing. As a result, the audience is not likely to visually identify shamanic elements in it. Rather,
BirdMan is driven implicitly by a shamanic perspective. Kim leads his own ritual of self-healing
through his artistic practice. By enabling us to metaphorically become one with bird, by offering us an
experience of hybrid avian-human perception, his work enables us to expand our consciousness beyond the limits
of our embodied human minds by joining self and other. It enables us to create new identities in between humans
and non-humans. And, as a result of this, it enables us to create new metaphors to live by and to live
with.
Such ideas echo Donna Haraway’s eco-feminist theories of multispecies worlds. She advocates a concept of
kinship or “making kin” that joins all beings: “all earthlings are kin in the deepest sense
[...]. All critters share a common ‘flesh,’ laterally, semiotically, and genealogically.” She
applies the term “sympoietic” to emphasize the collective process of poetic emergence in which all
beings are collaborators in the process of the Earth’s becoming. “Who and whatever we are, we need
to make-with—become- with, compose-with [...].” Taking care of the Earth, for Haraway, demands
caring for the diversity of beings, and “multispecies ecojustice” must be not only a goal but a
means to living well, together, as kin. By “staying with the trouble,” she proposes, “[m]aybe,
but only maybe, and only with intense commitment and collaborative work and play with other terrans [inhabitants
of Earth], flourishing for rich multispecies assemblages that include people will be possible.”
¹⁶
Buddhist luminary Thich Naht Hanh has taught that, “Our own awakened consciousness is what can
heal the Earth.” Indeed, the pressing and enduring concerns of global warming and the abuses of technology
demand that we expand our perceptual domain in order to heal ourselves, our kin, and our planet. To do so, we
must expand our metaphors and our means and channels of communication. We must summon the full power of art and
technology, while we equally harness ancient shamanic technologies and other modes of entering trance, expanding
consciousness, experiencing ecstatic states, fostering communication among all beings. If we want to have a
future, the artists of the future must serve as beacons of hope and as active participants in healing the world
and our relationship to it. Art, as a “psychic dress-rehearsal for the future,” to quote Jack
Burnham, must embrace kinship and harmonious co-emergence with all beings. We must become divine together. Today
there is no greater artistic calling, no greater aesthetic necessity than helping to heal and preserve
Earth/Gaia’s biodiversity for posterity. Technoshamanismis a potent strategy to move in that
direction.
References
1 Mircea Eliade, Shamanism: Archaic techniques of ecstasy. c. 1951, Princeton University
Press, 2004, xvii–xxvii.
2 Michael Harner, "My Path in Shamanism." International Journal of Transpersonal Studies
31.2, 2012.
3 Fabiane Borges, “Seminal Thoughts for A Possible Tecnoshamanism”
nd, np, c. 2014, Quote edited for clarity.
4 Roy Ascott, “Weaving the Shamantic Web: Art and Technoetics in the Bio-Telematic
Domain.” c. 1998, In Telematic Embrace, ed. Edward Shanken, Berkeley, University of California
Press, 2004, p. 356-62, Subsequent references to and quotations from Ascott are from this source.
5 Lila Moore, “The Shaman of Cybernetic Futures: Art, Ritual and Transcendence in Fields of
the Networked Mind” Cybernetics and Human Knowing 25, 2018, p. 2-3, 119-41.
6 Tess Thackara, Why Shamanic Practices Are Making a Comeback in Contemporary Art, Artsy,
2017, accessed June 13, 2023, https://bit.ly/48hsvk4
7 This literature includes Robert Wallis, “Art and Shamanism: From Cave Paintings to the
White Cube," 2019, Camila Maroja, “Persistence of Primitivism: Equivocation in Ernesto Neto’s
A Sacred Place and Critical Practice”, 2019, and Karen Gonzalez-Rice, “Revisiting "Art in
the Dark", 2023.
8 Claudia Jacques, “Forward: A Tribute to the Messenger Shaman: Roy Ascott,”
Cybernetics and Human Knowing 25, 2018, 5-15.
9 For more on Ascott, art, technology, and shamanism, see Lila Moore and Edward Shanken,
“Roy Ascott” in Charlie Gere and Francesca Franco, eds. Bloomsbury Encyclopaedia of New
Media Art, London, Bloomsbury Press, forthcoming.
10 Pauline Oliveros, “Improvising Composition: How to Listen in the Time Between,” c.
2012, in Gillian Siddall and Ellen Waterman, eds. Negotiated Moments: Improvisation, Sound, and
Subjectivity, Durham, Duke University Press, 2016, 83.
11 Pauline Oliveros, Deep Listening: a Composers Sound Practice, New York, iUniverse Inc.
2005, xix.
12 Pauline Oliveros, “Quantum Improvisation: The Cybernetic Presence,” in
"Sounding the Margins: Collected Writing 1992-2009, Kingston NY, Deep Listening Publications, 2009,
53.
13 Juan Downey, Dibujando con los Yanomami, (Caracas: Galería Adler Castillo,
1977): np. Quoted in Edward Shanken, “Pushing the Limits. Surrealism, Possession, and the Multiple Self:
Juan Downey and The Laughing Alligator” in Maricris Herrera, ed. Juan Downey,
1940-1993. Mexico City, Ediciones MP, 2019, 527-42.
15 Kim Jeong Han, Kim Hong-Gee, Lee Hyun Jean, "The BirdMan: hybrid perception," Digital
Creativity 26, 2015, 1, 56-64.
16 Donna J. Haraway, "Anthropocene, Capitalocene, Plantationocene, Chthulucene: Making Kin,"
Environmental Humanities 6, 2015, 1, 159-165.
17 Thich Nhat Hanh, Love Letter to the Earth, Berkeley, Parallax Press, 2013,
56.
Symbiotic Collaborators: The New Creative Subject in Postdigital Participatory Art
GyungJinShin
Hong Kong Baptist University, School of Creative Arts Hong Kong gjshin@hkbu.edu.hk
Abstract
As the Internet promotes participatory culture, contemporary network-driven participatory art, which I refer to
as “postdigital participatory art” (PPA), has introduced additional revolutionary creative subjects.
PPA induces a distinctive type of collective agency beyond mere collaboration among individuals by means of the
participatory architecture of the web. These multiple participants distribute the authority power of creation
throughout the network, transcending the limitations of time and space. In this paper, I attempt to theorize the
attributes of these new creative subjects, which I refer to as “participant-superjects,” with the
concept of superject serving here to indicate “power by modulation.” I outline the attributes of
these diffuse creative subjects and gauge their radical possibilities in terms of the agenda of experimental
art. I argue that, based on the new sense of relationality, materiality, and ontological perception associated
with the postdigital environment, these unique creative subjects are able to open up a new dimension of
creativity that differs from the modernist model, which emphasizes the creativity of the individual. I
hypothesize that the fluid power driven by this new creative subject exerts a latent force in building new
social relations outside the logic of the capitalist system.
With the death of the modern subject in the twentieth century, the divinization of single authorship on which
modern art relied has faced challenges and slowly collapsed along with the notion of “open
work,”¹ the postwar avant-garde of the 1960s, and author discourse.², ³ Artists
and theorists of the previous century attempted to undermine the traditional relationship between artist and
spectator, arguing instead for open-ended interpretations. These developments spurred a renegotiation of the
artist-spectator relationship and, in turn, the invention of new types of creative subjects. From this new
perspective, the status of the audience, as a “viewer-turned-participant,” has become almost equal
to that of the artist, and recent forms of contemporary art that are co-authored, collaborative, and
participatory have inherited the preference for antiauthorship. Such forms of art, including “relational
art”⁴ and “participatory art”⁵, ⁶, ⁷ have
flourished since the 1990s as artists have attempted to distribute the authority power traditionally associated
with the singular artist by encouraging the participation of viewers and envisioning new social relations.
In a hyperconnected environment that facilitates participation and collaborative creation, contemporary
participatory art has become ubiquitous and heterogeneous, transcending the boundaries of online and offline.
Recognizing that Bishop’s concept of “participatory art” overlooks the influence of
technology, I seek to help fill the resulting conceptual gap by coining a new term, “postdigital
participatory art” (PPA), to describe digitally mediated co-creation that relies on digital networks to
encourage audience participation. This new form of digitally mediated co-creation reflects changes in the
perceptions of time and space that have been described as “postdigital.”⁸, ⁹
In exploring PPA, I pay particular attention to the emergence of new authority power that is fluid and
ephemeral. Though such unique participatory creative subjects are profoundly observable in recent digitally
mediated participatory art, they have yet to receive careful analysis in terms of scope, motives,
characteristics, and patterns. Accordingly, I consider here the manifestation and radical possibilities of this
unique creative subject, which involves anonymous, networked participants mass-produced on networks.
Participant-Superject: The Unique Authors in the Network
As I conceptualize it, network-driven PPA relies heavily on collaboration among participants and often lacks a
teleology. Creators, materials, artworks, and spectators, in the absence of a predetermined blueprint for
object-making, co-emerge in the processing of inconclusive events. This development gives rise, in
turn, to the distinctive characteristics of a new creative subject that is ephemeral and fleeting, untethered as
it is to a fixed plan and, often, anonymous and unrecognized despite contributing significantly to the act of
creation.
The Jogging (https://thejogging.tumblr.com/), an ongoing, network-driven project, exemplifies the
attributes and modalities of these authors. [Figure 1] Utilizing a scroll-down thread on Tumblr that has
continued over several years, anonymous volunteers on the network Photoshop, reproduce, and reblog the images
associated with the original images uploaded by artists Brad Troemel and Lauren Christensen in 2009. Troemel
described his role as “initiating” and said in an interview that “‘Jogging’ refers
to a work flow, constantly moving, and not really focusing on any one thing, but rather to just continue
forward.”¹⁰ Without an individual artist furnishing a predetermined intention or conclusion,
such postdigital participatory projects are observable as the inconclusive, event-embracing agencies, networks,
and raw digital materials (e.g., information or data) involved in a work.
From Author-Subject to Participant-Superject
This new material variability in postdigital participatory art thus entails a reframing of the traditional
notion of author-subject. In a digital environment that strengthens temporality rather than spatiality, digital
participants have the potential to become temporally modulated subjects, or superjects, bearing latent
power by modulation rather than the power of individuals. As the antithesis of the author-subject
status, I coin the term participant-superject to describe the diffuse creative subjects that result
from the blurring of the artist-spectator boundary.
The concept of the superject, which refers here to “power by modulation,” has proved useful in
describing the special modalities of multiple agents that exist digitally when they are involved in PPA as
opposed to the modalities of physical participation. Deleuze adopted Whitehead’s concept of the superject
in the 1990s to illustrate the shift in subjectivity accompanying the rapid advance of digital technology at the
time.¹¹ According to Deleuze, a subject, when perceiving an object that is continually changing, can
be defined as a point of view that is likewise in a state of variation rather than fixed or determined
in advance.
The superject implies a “plural” subjectivity that cannot be fully described using the concepts of
a constant I and we and, instead, implicates a new status of the subject, a temporal I
and we. The superject, in Deleuze’s language, is an inherent multiplicity folded into a
collective unity.
The Three Phases of the Participant-Superject
Before gauging the radical possibilities of the participant-superject, it is useful to consider the
concept’s unique characteristics. Following Whitehead, Deleuze, and Savat, I describe the three main
phases of the superject as 1) temporal, 2) affective, and 3) condividual. The
temporal phase relates to the eventual nature of the superject. Referring to this phase,
Deleuze described the superject as an event rather than an essence. Adopting this view, Savat explicated the
specifics of the temporal aspect of the superject in the context of his analysis of digitality. According to
him, under the new temporal and spatial conditions of the digital environment, a superject exists as what
Deleuze called a “dividual,” an event that is ceaselessly modulated as a code, in contrast with an
embodied or spatial “individual.”¹² In this context, the manner of existence becomes
continuous, fluid, and momentary.
The second phase highlights the affective nature of the superject. Whitehead envisaged this facet of
the superject in an attempt to separate the subject per se from its experience of the
world.¹³ Thus, he proposed, the superject simultaneously emerges through the intake of
“data” in the form of sensory perceptions that precede consciousness, unlike the subject that
accumulates data. In this context, feelings mediate superjects and their associated data. This aspect of the
superject focuses on the moment of affective experience shared throughout the network.
Third, the superject implies the consolidation of collective agents, that is, condividuals. To be
specific, unlike the individual, which retains distinctive characteristics, the dividual tends to connect with
other similar components and combine with other dividuals to form condividuals).¹⁴ Existing
as metadata or data in the digital milieu, the dividuals can be used to build superjects in the network.
The participants in PPA, or participant-superjects, embody these phases of the superject—again,
temporal, affective, and condividual—which overlap and are interrelated. Rather
than being constrained to a physical venue for performance or exhibition, the participant-superjects appear
temporarily, take part in the creative process, and disappear in an instant. As the sequence of the action does
not “take place,” the actors rely heavily on temporality rather than spatiality. Further, the
participants in PPA tend to manifest momentary feelings or emotions rather than the conscious experiences
inscribed in a work. As The Jogging well shows, the intuitive and improvisational responses of the
participants manifest in such functions as liking, retweeting, and sharing on social media. Lastly, these
participants, as dividuals, merge easily and unite readily with others thanks to the anonymity and easy access
that the Internet affords.
Symbiotic Collaboration Among the Participant-Superjects
This new creative subject of PPA has the potential to fall into the trap of capitalism or to support resistance
to it. Stiegler used the term pharmakon to describe the aspect of technology that intoxicates and
cures, and postdigital participation has a similar “pharmakonic” effect, serving as simultaneously
toxin and remedy.¹⁵ In the 1990s and 2000s, the expectation was that the participatory environment of
the web would promote widespread self-awareness, the formation of grassroots communities, and co-individuation
by connecting individuals without respect to age, location, or gender. Since the network has also been viewed as
an arena for the fulfilment of capitalist desires, however, the participants in it found that they were treated
as consumers and unpaid labourers. Most of the major social networking platforms have reverse-engineered the
very structure of the participatory web to produce profits from the databases generated as a by-product of the
daily behaviours of online users. Moreover, governments and internet giants alike use the network as a source of
big data for analysing superjects and recognizing and forecasting social trends.
Regarding the paradoxical loop of postdigital participation problematized above, I argue that the modulatory
power of the participant-superject could radicalize the agenda of experimental works by altering the form that
power takes. This new form of power, again as Deleuze pointed out, refers to the modulation of
dividuals.¹⁶ For Deleuze, of course, such power deserves criticism, but it also serves as a positive
starting point for the arguments presented here. Considering the dividual a latent power of struggle,
Raunig claimed that the dividuality emerging as self-division in contemporary social media could promote new
forms of dispersed resistance against machinic capitalism.¹⁷ In the same vein, Savat insisted that a
new mode of politics involving the dividual, which he called the “politics of fluids,” has become a
more critical field of participation than the modern “solid politics” involving
individuals.¹⁸ Thus, he affirmed that the potential inherent in “fluid action” can be
utilized and realized in constructive ways because dividuality represents individuals’ ways of being in
the network.
Following Gaunig and Savat, I affirm that the fluid power of the superjectification seen in PPA—again, in
contrast with the solid power of individuals—exerts a latent force in terms of inventing social
relationships apart from the logic of the capitalism that finances and reifies human bodies and even social
relations. The characteristics of superjectification—being temporal, dispersed, affective, and
condividual—are more radical and effective in terms of “pre-empting” the looming problems
associated with this logic. Furthermore, the unique sense of affinity helps dispersed cognitions cling together
contagiously, in turn affecting real politics (e.g., Ushahidi software, the MeToo and Occupy movements, and the
activist group Anonymous).
Thus, PPA projects tend to encourage affinity-driven co-individuation outside the context of the labour system
and manipulative relationships on the web.¹⁹ Public_Public_Address: A Nationwide Virtual Protest
(2020-), for example, is an ongoing virtual protest for which Jason Lazarus, Stephanie Syjuco, and Siebren
Versteeg have been accepting submissions of selfie videos of individuals holding protest pickets in support of
the Black Lives Matter movement. [Figure 2] The participants include those who had been marginalized in society
and those unable to protest in person because of a disability. The artists integrated the video submissions into
live-streaming on YouTube and simultaneously in the window of a New York gallery in 2020. Seemingly marching
toward the viewers, the images of these anonymous participants have been incorporated into a bizarre new form of
space-time. Rather than isolating, separating, and pixelizing the agency of the participants in the network,
this project reveals the aesthetics of fluid power that the superjective participants generate in the
postdigital environment by transcending the online-offline binary.
Conclusions
As the exploitation of participation in the network becomes increasingly sophisticated, PPA has the potential
to offer space in which to imagine new social relationships by questioning freely the dominant logic and weaving
together digital objects, technological materials, and human agency symbiotically. In this regard, art systems
centred on individual authorship fall short when it comes to describing a new dynamism that embraces the energy,
matter, force, objectiles, and superjects that digitally mediated co-creation can bring about. In the consistent
flow of the network, the conventional binaries of matter and form, subject and object, physical and digital, and
artist and spectator co-emerge, co-concretize, and intra-act. Then, the new creative subject of PPA, or
participant-superject, having emerged in the context of the shift in digital materiality and subjectivity, can
radicalize the author-subject model and re-envision the agendas of open-ended and participatory aspects of art.
Superjective authors, then, with their mutual resonances, cause events and collective experiences to
proliferate and add further from the postdigital environment. These authors are omnipresent and already part of
humans’ ontological status. As Literat observed regarding many cases of online crowdsourced art, however,
“the crowd is still a crowd, not yet a community” because the digital fibres and technological
structures that mediate digital encounters can hinder efforts to achieve artistic collaboration or build a sense
of community.²⁰ Thus, further studies are needed to assess the potential of PPA to invent meaningful
communities apart from the profit-making algorithms of the various online platforms.
References
1 Umberto Eco, "The poetics of the open work," The Open Work 251, no. 1., 1962.
2 Roland Barthes, "The Death of the Author," Contributions in Philosophy 83, [1967],
2001, 3-8.
3 Michel Foucault, "What is an Author?," in Textual Strategies, ed. J.V. Harari,
Ithaca, Cornell University Press, [1969] 1979, 141-60.
4 Nicolas Bourriaud, Relational aesthetics. Trans. Simon Pleasance, Fronza Woods,
Mathieu Copeland, Dijon, Les Presses Du Réel, 2002, Originally published as Esthétique
Relationnelle, Dijon, Les Presses Du Réel, 1998.
5 Claire Bishop, “Introduction: Viewers as Producers,” in Participation,
London, Whitechapel/Cambridge, MIT Press, 2006, 10-17.
6 Claire Bishop, “Social Turn: Collaboration and Its Discontents,”
in Artforum 44, no. 6, 2006, 178–83.
7 Claire Bishop, Artificial Hells: Participatory Art and the Politics of Spectatorship,
London, New York, Verso Books, 2012.
8 Robert Pepperell, Michael Punt, The Postdigital Membrane: Imagination, Technology and
Desire, Bristol, Intellect Books, 2000.
9 Sy Taffel, “Perspectives on the Postdigital,” Convergence: the
International Journal of Research Into New Media Technologies 22 (3), 2016, 324–38.
10 Brad Troemel, “Interview with Rob Walker, The Jogging, a Tumblr at the intersection of
BuzzFeed, 4chan and weirdo experimental art,” Yahoo News, October 24, 2013.
11 Gilles Deleuze, The Fold: Leibniz and the Baroque, London, The Athlone Press,
1993.
12 David Savat, “(Dis)Connected: Deleuze’s Superject and the Internet,”
in International Handbook of Internet Research, 11:423– 36, Dordrecht, Springer
Netherlands, 2010, 432.
13 Alfred N. Whitehead, Process and Reality: An Essay in Cosmology. eds Griffin,
David Ray and Sherburne, Donald W, New York, Free Press, [1929] 1978.
14 Marco Deseriis, Improper Names: Collective Pseudonyms from the Luddites to Anonymous,
Minneapolis, University of Minnesota Press, 2015.
15 Bernard Stiegler, What makes life worth living: On pharmacology, Cambridge,
Polity, 2010.
16 Gilles Deleuze, “Postscript on the Societies of Control,” October 59,
Winter, 1992, 3–7.
17 Gerald Raunig, Dividuum. Machinic Capitalism and Molecular Revolution, Volume 1
(trans. Aileen Derieg), Los Angeles, Semiotext(e)/MIT Press, 2016.
18 David Savat, Uncoding the Digital, Basingstoke, Palgrave, 2013.
19 Marco Deseriis, “The Politics of Condividuality”, transversal.at., 2018,
7-10, accessed August 31, 2021, https://transversal.at/transversal/0318/deseriis/en
20 Ioana Literat, “The Work of Art in the Age of Mediated Participation: Crowdsourced Art
and Collective Creativity”, International Journal of Communication 6, 2012, 2962–84.
Bibliography
Claire Bishop. “Participation and Spectacle: Where Are We Now?,” in Living as form: Socially
engaged art from 1991-2011, ed. Nato Thompson (Massachusetts: MIT Press, 2012), 34-45. Kyle Chayka,
“Art in the Corporatized Sphere: The Impact of Commercial Social Media on Online Artistic
Practice,” in A Companion to Digital Art, ed. Christiane Paul (New York: John Wiley & Sons,
2016), 413–425. Christian Fuchs, Digital Labour and Karl Marx (London: Routledge,
2014) Ioana Literat, “Facilitating Creative Participation and Collaboration in Online Spaces: the
Impact of Social and Technological Factors in Enabling Sustainable Engagement,” Digital Creativity
28 (2), (2017): 73–88.
Acknowledgements
The overall structure and conception of this paper draw on my PhD dissertation published in June 2022
Proteus 3.0 - Interacting with the cloud
MariaSmigielska*,
CompMonks*
mariasni.com; compmonks.com Zurich Switzerland maria@mariasni.com,
compmonks@compmonks.com *authors contributed equally to the project
Abstract
This paper presents the Proteus 3.0 project as an interactive and generative video installation, which
tries to remain in conversation with its audience and learns from its engagement to produce new and potentially
meaningful sequences of images. This particular version of a series is focused on digitized material as
ferrofluid and its digital surrogate generated with a reaction-diffusion algorithm, projected onto a room-scale
oval oculus allowing for a collective/immersive interaction. By employing state-of-the-art custom reinforcement
learning models coupled with human intelligence in a symbiotic fashion, it aims to make the interaction more
meaningful by understanding better human behaviour during an interaction. Additionally, it intends to maintain
the interaction as conversations, rather than playful attractions.
This paper presents the Proteus 3.0 project as an interactive and generative video installation, which
tries to remain in conversation with its audience and learns from its engagement to produce new and potentially
meaningful sequences of images. Proteus, as a series (Figure 1), has systematically evolved with its
ferrofluid displays and modes of magnetic activation at multiple resolution ranges, particular sequences of
digital and analogue layers as well as types of interaction. This version is particularly focused on moving the
ferromaterial to the digital realm with its digital surrogate generated from a reaction-diffusion algorithm,
scaling up from an object-size analogue display to an architectural oculus of oval shape at room scale, and
which, at the same time allows to shift from an intimate interaction to a collective/immersive one. By employing
state-of-the-art reinforcement learning models coupled with human intelligence in a symbiotic fashion, it also
intends to build an interaction that is based on conversations, rather than playful attractions. Most frequent
and straightforward approaches towards interaction design are based on implicit principles and clear
instructions built in a closed loop to promote immediate engagement. These oversimplified views on interaction
neglect its long-term impact¹ or the ones stemming from more traditional discourses, and which define any
interaction with an artwork purely as a mental act.² In that lens, the project revisits the
Conversation Theory of Gordon Pask³ in its scope of human-machine interaction as a form of open
and emulative conversation. A dynamic process in which both ends learn and adapt from each other. As such, the
project aims to construct an interaction that goes beyond the boundaries of predefined scenarios that follow the
strict categories of an audience’s behavioral patterns and tends towards individualized and dynamic
responses.
Background Conversational interaction
What most generally describes successfully an interaction is the degrees of responsiveness and control an
artwork and an audience are communicating to each other. An interactive artwork is essentially a composite made
of an artificial component (the artwork itself as an object) and a human component (the audience) that both
define its nature and behavior⁴. These degrees of communication are then located somewhat between total
randomness and total predictability, without reaching any of these extremums. A total random response could only
qualify as part of a reactive process, and a totally predictable one as part of a controlled manipulation. This
does not necessarily involve a stable exchange and most frequently concurrent dynamics are at play from both
ends as human intentions and artificially designed systems may have different and evolving goals over time.
While this constitutes a great challenge for an artist to frame and produce such ideas, it also represents the
richness and openness that an interaction may provide in qualifying an object. Interactivity is, by design, the
relational property of an object and not an intrinsic one. It results that the value of such artwork is
consequently constantly in a state of volatility and becoming, evolving throughout interactions. One might argue
that this is necessarily the fate of every artwork, and more generally every object. But this becomes
fundamental when interaction becomes itself an object of design in the artwork. Although, the most frequent and
straightforward approaches towards interaction design are based on implicit principles and clear instructions
built in a closed loop to promote immediate engagement and playful attractions. Similarly, most theoretical
intent to define interactivity in art has been partially treating aspects of interactions that revolve around
the topic of controllability. In general, the ability to improve control makes novelty the source of most types
of interactivity⁵. In the general perspective of control, interaction can be seen as a learning stage in
order to reach control. Once sufficient learning is achieved, interaction becomes redundant. Interaction becomes
endowed with the feature of being instructional. However, even during the early stages of cybernetics, the
theory of control was seen as only a chapter in the theory of messages⁶. Interactive art must be
dissociated from the grasp of control theory [ref] in order to avoid any pessimistic perspectives⁷ or
shortcomings in the envisioned artistic power of such artistic interventions. On the same level, Input/Output
theory of interaction with computers⁸ simply defines interactions as a unilateral transactional process of
information and therefore fall short on creative content. On another aspect, when interactivity is defined as
procedural and participatory⁹, the definition lacks enough abstraction and generalisation to talk about
the systemic design of an interaction and the informational novelty that go beyond the sole responsiveness of
participations to an artwork. On another end, the evolution of machine perception in human computer interaction
can be seen as trivialising the term of interactivity in new media a rts, and eventually rendering the sum of
its tentative description as too loose to be useful to qualify an artwork¹⁰. This oversimplified
views on the interaction neglects its long-term impact¹¹ or the one stemming from traditional art,
which defines interaction with art purely as a “mental act”¹². It also resonates with the
cognitive approach of the power of an artwork being located in the beholder’s response
¹³.
Proteus 3.0 within the series of Protei
Proteus is a series developed as a duo since 2018 driven by the curiosity in exploring aspects of
interacting with matter, tensions between analogue and digital, as well as relations between human and machine
intelligences¹⁴. The series evolves through the continuous visual and material reference of a colloid
compound called ‘ferrofluid’ as a main character ¹⁵. When exposed to magnetic
fields and interacting with another liquid carrier, it reveals its intricate and lively nature by presenting
strongly contrasted and ever-changing complex patterns ranging from discrete patches of dots, meandering
streams, to larger coagulated blobs. The volatility and organicity of this material behaviour in the series is
used as an allegory of the Greek mythology of a deity called Prōteus and his ability to assume
many forms and provoke uncanny encounters ¹⁶ that remain evasive to the rational mind but open-ended
to the visual imagination. Over the course of its serial declination, Proteus has iteratively evolved
with varied organic and deep black patterns to serve as an emulator for visual human intelligence. That of
seeking meaning in a constantly changing flow of images and unpredictable symbolic relations which can only be
found in the human mind, just like gazing at a cloud. But the historical problem in such a romantic
contemplation is often emphasized through the notion of distance and the lack of direct communication between a
cloud and its beholders. If the cloud could talk back in response to one’s efforts to find visual meaning,
would the conversation stabilize to a consensus, or would it constantly move to explore and seek novelty? This
problem of distance and communication reveals itself when reframed in the artwork as a generative/interactive
installation and its architectural embodiment.
Methods
Proteus 3.0 is an interactive generative 4K video installation, rear-projected onto a black screen of oval
shape. The screen, of dimensions approximating 3.5 x 2.5 meters, is rotated 45 degrees and integrates an HD wide
angle and infrared camera to estimate body and head poses from the audience and to detect regions of visual
interest located on the screen. This information is continuously tracked and serves to build markers of
attention to modify the generated visual frames (Figure 3).
Material surrogate
Because to this day, no digital model has been able to accurately simulate the physical behaviour of ferrofluid
material in real time ¹⁷, a specific GLSL shader has been developed for the procedural generation of
a digital surrogate. Based on a Gray-Scott reaction-diffusion model between two substances, a series of image
filters have been designed as an overlay to mimic the dynamics of the ferrofluid and its pattern formations.
Our experiment was conducted in two following steps: (1) initiate predefined interaction scenarios run in the
exhibition context to collect the visitors’ data, (2) building adaptive and evolving interaction models
with reinforcement learning. The details of those two phases are described below.
Predefined interaction scenarios
At the first launch, the initial interaction consisted of 5 predefined interaction scenarios running in the 80
seconds loop. Interaction scenarios consisted of generative visuals coupled with the most common human
behavioural patterns. The visitors' data responding to these scenarios were collected during eight months of the
BioMedia exhibition at ZKM Karlsruhe, starting in December 2021.
In order to build Regions of Interests (ROI) of each interaction, skeleton data of max 6 visitors
simultaneously were captured with a Kinect 2 camera to extract their torso and face orientation projected back
on the oculus canvas. Each interaction has been recorded into a large dataset organized by timestamp with raw
numerical information (visitors' ROI, skeleton data, location in relation to the screen) as well as
low-resolution images (like RD noise map and visitors' ROI) for further training. The construction of datasets
considers that any relevant personal information is decoupled from the recorded data.
Generative interaction scenarios with reinforcement learning (RL)
The scenarios further evolved into generative characters with the support of a machine learning model
programmed in Python, specifically a reinforcement learning model, to learn from collected data and
extracted markers of attention. It is being trained discretely offline and concurrently for a real-time
generation to support the progressive evolution of the artwork that can adapt to different exhibition contexts
and visitors' behaviours. The custom model has been developed in python with a PyTorch library.
The real-time performance of the work and the offline training are orchestrated by two combined processes. For
real-time performance, a TouchDesigner file manages both visual inputs coming from the camera and generated
outputs going to the display. The generated images are sent to two video projectors, mapped and blended for a
smooth projection onto the screen. Daily, the recorded data is updated to a remote server and feeds new offline
training sessions for the reinforcement learning model. The updated model is then sent back to the local
computer controlling the installation to update Proteus' behaviour.
Engagement
Engagement can be measured with simple metrics like the amount of time spent looking at art¹⁸ or
more complex summative indexes, like Sweep Rate (SRI) and Diligent Visitors (DV) representing exhibitions
“Thorough Use”¹⁹ to improve audience engagement in museums at large. However, those
static metrics allow us to look at visitors’ response only through a pinhole and we do not know
the exact impact of the interaction on the visitor (long-term, imaginary, etc). Therefore, we need more data
collected during the interaction itself and more sophisticated tools to analyze them. “The Plant”
project²⁰ proves that reinforcement learning can improve some of measures of
engagement, like visitors touch count during interaction, but not average duration of the interaction. This
project allows us to draw a hypothesis that AI-based tools combined with generative models can increase the
quality of engagement and not just its quantitative factors like duration.
In the next developmental step of Proteus project, we aim to utilize the developed RL model in the future
exhibition and conduct comparative studies to evaluate this hypothesis.
Conclusions
Proteus 3.0 is an interactive and generative video installation, which, once in place within the walls of its
exhibition and switched on, tries to remain in conversation with its audience and learns from the visitors'
engagement to produce new and potentially meaningful sequences of images (Figure 2). Until the end of its
exhibition, when it is switched off and set frozen in time. By then, its last learned state is saved and becomes
the new generative beginning for its next exhibition. The learning and interactive process continues endlessly
during the artwork’s lifetime for each exhibition and conditions its existence. Its generative imagery
remains open-ended and its content to be modulated by its beholders. This project is an attempt to utilize
AI-supported generative systems, which facilitate an interaction that goes beyond predefined behavioral models
directly controlled by a few, easily comprehensible parameters and therefore might increase the engagement in
the audience.
References
1 Edmonds E, Candy L, “Interacting: Art, Research and the Creative Practitioner,”
2011.
2 Paul C, “Digital Art” Third edition, London, Thames & Hudson, 2015.
3 Pask G, “Conversation Theory Applications in Education and Epistemology” Amsterdam,
Elsevier, 1976.
4 Margaret A. Boden, Ernest A. Edmonds, From Fingers to Digits: An Artificial
Aesthetic, Illustrated edition, Cambridge, Massachusetts: The MIT Press, 2019.
5 “Aaron Smuts, What Is Interactivity?—PhilPapers”, 2023, Accessed September 15.
6 Norbert Wiener, The Human Use Of Human Beings: Cybernetics And Society. Revised
ed. edition. New York, N.Y: Da Capo Press, 1950.
7 Terrence Rafferty, “Everybody Gets a Cut,” The New YorkTimes, May 4, sec.
Magazine, 2003, https://www.nytimes.com/2003/05/04/magazine/everybody-gets-a-cut.html.
8 David Z. Saltz, “The Art of Interaction: Interactivity, Performativity, and
Computers,” The Journal of Aesthetics and Art Criticism 55 (2), 1997, 117–27,
doi:10.2307/431258.
9 Janet H. Murray, Hamlet on the Holodeck, Updated Edition: The Future of Narrative in
Cyberspace, Updated ed. edition, Cambridge, Massachusetts: The MIT Press, 2017.
10 Manovich Lev, The Language of New Media, MIT Press, 2002.
11 Ernest Edmonds, “Art, Interaction and Engagement,” In 2011 15th International
Conference on Information Visualisation, London, United Kingdom, IEEE, 2011, 451–56,
doi:10.1109/IV.2011.73.
12 Christiane Paul, Digital Art. Third edition. London, Thames & Hudson, 2015.
13 Eric
Kandel,The
Age of Insight: The Quest to Understand the Unconscious in Art, Mind, and Brain, from Vienna 1900 to the
Present,
1st edition, New York, Random House, 2012.
14 https://proteusproject.ch/
15 https://en.m.wikipedia.org/wiki/Ferrofluid
16 https://en.m.wikipedia.org/wiki/Proteus
17 Han Shao, Hang Libo, Dominik L. Michels, “A Current Loop Model for the Fast Simulation of
Ferrofluids,” IEEE Transactions on Visualization and Computer Graphics, 2022, 1-12,
doi:10.1109/TVCG.2022.3211414.
18 Jeffrey K. Smith, Lisa F. Smith, “Spending Time on Art.” Empirical Studies
of the Arts 19 (2), 2001, 229–36.
19 Beverly Serrell, “The Aggregation of Tracking-and-Timing Visitor-Use Data of Museum
Exhibitions for Benchmarks of ‘Thorough Use.’” Visitor Studies 23 (1), 2020,
1–17, doi:10.1080/10645578.2020.1750830.
20 Zoe Tong, Dana Kulic, “Learning to Engage in Interactive Digital Art,” In 26th
International Conference on Intelligent User Interfaces, College Station TX USA, ACM, 2021,
275–79, doi:10.1145/3397481.3450691
Acknowledgments
Artwork by: Maria Smigielska, Compmonks Created as part of the intelligent.museum project at the ZKM |
Center for Art and Media Karlsruhe and at the Deutsches Museum. Funded by the Digital Culture Programme of the
Kulturstiftung des Bundes (German Federal Cultural Foundation). Funded by the Beauftragte der Bundesregierung
für Kultur und Medien (Federal Government Commissioner für Culture and Media). Funded by BioMedia
ZKM | Center for Art and Media Karlsruhe.
Authors’ Biographies
Maria Smigielska is an architect and researcher based in Zurich, currently working at DBT, ETH. She is
interested in enhancement of potentials for creation of architectural elements, design objects and mixed media
installations, by using digital and interactive technologies for encoding and modulating materials properties,
custom design and fabrication methods. Mariasni.com
Compmonks (PhD) is an architect and a researcher in technology for architecture based in Switzerland. He
currently leads research on interactive and generative design with brain-computer interfaces at ETH
Zürich. His artistic work follows a series of design objects and mixed-media installations focused on
the power of combining humans with computers.
1 University of Antwerp, 2 University of Copenhagen, 3
Technical University of Delft, 4 University of the Philippines, 5 EDESTA-University Paris
8, 6 University of Plymouth, 7 SEADS (Space Ecologies Art and Design) Antwerpen, Belgium
pieter.steyaert@uantwerpen.be
Abstract
Co-creative practices including non-human actors give rise to a series of challenges and critical issues.
Ēngines of Ēternity is an ongoing artwork which seeks to attribute agency to microscopic animals
called rotifers and to the unique environment of outer space. Scientific experiments involving these animals, as
well as the evolving artwork that accompanies them, have gone through multiple iterations having been flown to
space and consequently returned. This article reflects on how transdisciplinary approaches can provide a vehicle
to connect knowledge and enquiries from art and sciences in the context of real-world problems. We then focus on
the case study of Ēngines of Ēternity and how these interactions played out during this process. We
conclude the article with a critical reflection on non-human agency, using the aforementioned project as a case
study.
The bidirectional exchange of knowledge, ideas and methodologies can foster co-creative practices and allow for
a genuine blurring between disciplinary boundaries to occur.¹ Such holistic conditions may act as a
catalyst for the synthesis of a deeper understanding with regard to the nature of the universe we inhabit and
with it the emergence of a new post-Anthropocene era.² In order to achieve a sustainable future, we need to
understand the mechanics and challenges of human and non-human interactions. This article explores and assesses
the process of ArtScience co-creation, through a transdisciplinary project entitled Ēngines of
Ēternity, acting as a case study. The work's narrative incorporates questions regarding manmade power
structures and cultural immortality, through the use of non-human organisms, whilst highlighting the benefits
and issues that can occur as a consequence. The author of this work and of this article is SEADS, a
transdisciplinary and cross-cultural collective of artists, scientists, engineers and activists. The
collective’s mission statement notes that transdisciplinary approaches are key to unlocking collective
intelligence, which is a prerequisite for generating more diversified and inclusive futures.³ They
incorporate co-creation methodologies in a range of their projects, amongst others are community art projects
such as Bio-modd and Seeker.⁴, ⁵ The Ēngines of Ēternity project explores new
forms of co-creation between different entities: humans, biological organisms and algorithms within the context
of outer space. Before introducing the project, we outline the concept of creativity in the post-Anthropocene,
which has been key to functioning as an underlying theoretical framework for this case study.
Creativity and Agency in the post-Anthropocene
If the Anthropocene is understood to be the geological epoch in which human activities are the dominant
influence on the environment, the post-Anthropocene can be understood to be the epoch that dawns when human
activities and their effects are no longer the dominant influence on the environment.⁶, ²
The exact nature that the post-Anthropocene will take is difficult to predict because it depends on many
factors, including how humanity responds to the challenges currently faced in the Anthropocene itself. On a
global scale, we are not able to predict when and how the post-Anthropocene concept will become a dominant
global framework, or indeed if this in fact will take place. However, there may be triggers that are capable of
driving societies (both human and non-human) towards it, such as the ongoing global warming crisis and
deterioration of the environment, societal cultural shifts, or technological advancements that may even allow
humanity to live in greater harmony with Earth's ecosystems, but this is currently speculative. Merging
disciplines can contribute to identifying and solving complex problems through the collaborative exchange of
ideas and methodologies.⁷ Indeed, it has become apparent that many of the major problems facing the world
today are hugely complex in nature and will not be solved by any one single discipline alone.⁸ Addressing
complex societal and scientific problems requires involving multiple disciplines. However, there is a danger of
oversimplifying the collaborative benefits between these disciplines. We argue that commonalities or filling up
gaps can ignore or overlook the need to find new creative ways of dialogue between the disciplines, thus
creating new knowledge or lines of inquiry that can transform the conversation on various global and scientific
issues.
Differing from alternative approaches to merging disciplines, transdisciplinarity concerns both what exists,
and is between and beyond all disciplines, blending into an empirical and theoretical exploration of
co-creation.¹, ⁹, ¹⁰ One form of transdisciplinarity involves
combining art and science (ArtScience) to create new co-creation practices that utilize methods and
methodologies from both fields. Such art interventions can take place in present and future shared spaces and
may contribute to the constitution of new symbolic orders and organisations of human and non-human connections,
staging the mise en forme of post-Anthropocene co-existence. Seen in this way, ArtScience co-creation can be
seen as an integral part of rethinking humanity’s relations to nature.
Bruno Latour argues that non-humans, such as animals, plants and objects, are active participants in social
life and should receive equal attention in the study of society.¹¹ In this way of thinking non-humans
should be seen as actors in their own right, rather than simply as objects that humans act upon; non-humans can
have agency, and influence the course of events in the world. Latour’s conceptualization of agency does
not strictly correspond with notions of sentience nor free will; neither does it have to be confined within
physical boundaries. Agency can be an emergent property of multi-actant configurations.¹¹ According to
Latour, social-cultural practices are enunciative and performative actions through which the new
“subjects” can test different semiotic, sociological and alterity modes of existence, that lie
between “being-as-being and being-as-others.”¹² In a similar vein, we refer to the notion
of a “multispecies roundtable” for co-creating systems. This involves bringing together plants,
computers, and people in mutually beneficial ways and placing them in a situation where they must arrive at a
consensus.⁴, ¹³
Speculative Realism, Posthumanism, New Materialism, and the non-human are just a few examples of concepts that
challenge the limits of what Donna Haraway calls the “fantasy of human exceptionalism.” These ideas
reject the notion that humans are somehow separate, beyond, or more advanced than our earthly
co-habitants.¹⁴ The theorists working within these fields help to illuminate the limits of human
experience and thought, highlighting the capacities, experiences, and potentials of other living, non-living,
and non-human entities. Their examples embrace a radically holistic and inclusive approach to the future of
humanity and are established concepts in critical cultural thinking.
The Engines of Eternity project puts these ideas into practice by directly interacting with microscopic
organisms, computational algorithms, and the physicality of outer space. These practical engagements hint at new
opportunities and insights but also confront the limitations of an extended co-creation practice. Continuous
hands-on approaches and their analysis allow us to move beyond established thinking and gain new critical
insights through hands-on artistic research.
Ēngines of Ēternity
Ēngines of Ēternity is a transdisciplinary project that explores how creativity and agency might
function in a post-Anthropocene context. It is a work in progress and is intended to be a series of mixed-media
installations. The project is a joint effort between SEADS and scientists from Karine Van Doninck's laboratory
at UNamur. The foundational idea of the project is to take the biological phenomena of cloning and DNA repair as
metaphorical departure points for an art installation about humanity’s obsession with cultural
immortality.¹⁵ By metaphorical departure point, we mean that we use the fascinating biological
characteristics of the rotifer animal to question creating a perfect society. At the core of the project is an
artwork that evolves through complex systems of interplay and co-creation between biological organisms, humans,
and technology. The project explores dominating ideologies about evolution and cultural identity within the
space environment. The discussion on cultural immortality is discussed in a separate article³, as this one
focuses on the methods and their implications that were crucial to developing the artwork. Here, we instead
describe the background of our ArtScience co-creation activities: space biology experiments with the Bdelloid
rotifer.
The Bdelloid rotifer or ‘wheel animal’ typically lives in freshwater environments and is part of
the greater ecosystem that exists there. On the surface, rotifers seem to represent a biological culture
perfected through evolution, cloning themselves in endless repetition. However, during this cloning process,
genetic material actually gets reshuffled and diversity is generated through the process of horizontal gene
transfer. This method of generating diversity can be defined as the non-sexual movement of genetic information
between organisms, and genetic material from organisms as diverse as fungi, bacteria and plants have been
discovered lurking inside the rotifer’s genome.¹⁵
Through a series of space biology experiments, scientists from UNamur sent rotifers to the International Space
Station (ISS) in both 2019 and 2020. The goal of these experiments is to investigate rotifer DNA repair
mechanisms and cloning in the conditions of outer space. In December 2019, the rotifers going to ISS were
accompanied by an artistic "seed" in the form of a specially designed visual code that accompanied the rotifers.
The code was developed by the Ēngines of Ēternity team and serves as a seed for a series of
3D-printed sculptures. These sculptures show the physical changes to the rotifers in response to environmental
conditions and experimental techniques. The sculptures are based on a mapping of recorded experimental data 3D
sculptures.
The work is driven by two main objectives: the evolution of the "code" throughout different missions and the
translation of these seeds into three-dimensional sculptures. Analysis was conducted on both the bags that went
up into space and the bags that served as a control group on Earth in order to reveal the differences between
protein expressions. Each individual bag that was sequenced was assessed against the average from the control
group bags, resulting in discrete sets per bag of large quantities of raw data. These sets were then grouped and
ordered per function, accumulating the differences compared to the control group. The result was a visualization
per bag showing the impact of the space environment on rotifer animals. These visualizations were then used to
evolve the original visual "code" that accompanied these bags by using it as an attractor to manipulate the
pixels of the original seed.
The second objective was to translate these seeds, both the original and the evolved version, into a
three-dimensional sculpture. The glyphs that make up the seeds were transformed into a circle and a linear
regression algorithm was applied to layer multiple versions of the original visual on top of each other and
connect them. This resulted in a three-dimensional sculpture where the base visualises the overall shape and
every layer shifts towards a higher level of detail. In this way, the final sculpture shows both the impact of
the environment on the rotifer animals at a higher order and on the smaller, individual protein clusters.
The “code” and sculptures will gradually evolve into the final artwork over several missions based
on the changes in the rotifers. Hence, the context of outer space is not merely treated as a passive gallery
space to exhibit art in, but rather as a dynamic environment that can actively shape the evolution of the work.
Co-creation revisited Co-creation in Art and Science
The historical development of scientific and artistic disciplines has resulted in the development of
organisations and structures in which these two areas do not organically engage or have dialogue. The
organisational and procedural infrastructure that exists within these two fields is therefore sometimes seen as
opposites. Science is often associated with the use of strict methodologies and systems to produce empirical
data, unbiased information, and facts that help us better understand the physical and natural world around us.
On the other hand, art is often viewed as a more subjective practice that allows individuals or groups to
express their creativity and imagination.¹⁶ While these definitions are not entirely inaccurate, such
reductionist viewpoints are unhelpful when beginning to ask deeper questions. ²³ Indeed, in recent
times a variety of Art-Science networks have emerged that act as places where such communities can interact, and
begin to create imaginaries for developing questions regarding our place in the wider world, whilst reaching
large audiences. Interactions between the fields of science and art can be regarded as a one, or two-way, form
of communication between the two. When considered as a one-way form of interaction, there are two possibilities
for one to influence the other, either art provides science with some form of benefit, or vice versa.
The SEADS team developed tools using techniques and knowledge that differ from those of the pure scientific
team. The goal was to gain insights into the impact of the alien environment of space on living creatures in an
exploratory context. Initially met with skepticism, the collaboration with the pure science team evolved
throughout the project. By communicating about our results, the team was able to establish new ways of trust and
collaboration. The SEADS collective comprises members with backgrounds in both art and science tapping into
collective intelligence. This allows them to push boundaries when collaborating within and outside of the
network. For instance, in the Ēngines of Ēternity project, the artists received hard drives
containing all the raw transcriptomic data to explore. This not only resulted in the aforementioned sculptures
but also allowed the team to analyze their findings and compare them to those of the pure science team. It
appears that new insights emerged from this transdisciplinary approach, which has yet to be validated and might
lead to a future publication. In some ways, it is also possible to view the process of horizontal gene transfer
as a biological counterpart to transdisciplinarity itself. The generation of novel genetic material through the
incorporation of foreign DNA into the rotifers, from such phylogenetically distant organisms, echoes the process
of numerous distinct disciplines joining together to create a more efficient system. Much like
transdisciplinarity itself, it is not merely the merging of disciplines (or in this case genetic material) that
makes this comparison applicable, but the fact that the process has wider implications on a much deeper level as
well.
If transdisciplinary practices are considered to be a combination of, translational and methodological
approaches from various disciplines, utilised to address a problem, then the horizontal integration of genomic
elements from cross-species sources can be said to function in a similar manner. DNA segments once integrated
into the rotifer’s genome can be translated into new useful proteins, that are capable of carrying out a
variety of functions, and in turn, contribute to the evolutionary adaptation of the organism. The whole is
greater than the sum of its parts, in both the case of transdisciplinary inquiry and biological evolution.
Posthuman co-creation
A fundamental question in this project is whether true co-creation between humans and non-humans has been
established, more specifically between the art-science team and the rotifers. How much agency did the rotifers
actually have in participating and contributing to the artwork? In addition to exploring human-human
interactions, we also investigate the agency of non-human entities, specifically the Rotifer animal. The
evolution of the artwork is driven by changes in protein expression. However, this approach raises the question:
are these non-human entities truly being given agency?
There are two ways to approach this question. One way is instrumentalist, which would argue that regardless of
the artists' intentions, the rotifer animals are being exploited for both science and art. In fact, there is
even an element of cruelty, as the animals are taken from their natural habitat and subjected to experimentation
in an extreme environment without any choice in the matter. From a pragmatic standpoint, it is difficult to
refute this way of framing the issue.
We present a second approach as follows: The artist team explores the intrinsic attributes of the rotifer
animal and co-creates the artwork based on their findings. In this instance, horizontal gene transfer is viewed
as a form of creation. By incorporating the protein expressions from these animals as the main driving force,
and using complex bio-informatics algorithms, some level of agency over the artwork is given to these non-human
entities. Extreme environments have been fundamental driving forces for the evolution of rotifers. However, in
the case of the artwork discussed here, the driving force of its evolution is not solely an attempt to depict
the intrinsic attributes of these animals. Instead, it also considers the impact of the alien environment of
near-zero gravity in space on these extremophiles. This marks the first time in the history of the species that
they have been exposed to the environment of space.
In this sense, the two levels of co-creation within this case study can be extended to include "The alien" as
an environmental contributing factor. Additionally, the extensive use of algorithms and predictive computational
methods has played a significant role in the visual outcome of the project. As a result, this project explores
new forms and levels of co-creation between humans, biological organisms, algorithms, and outer space.
Horizontal gene transfer and parthenogenesis are used not only as a metaphor, as was discussed in the previous
section, but as an act of self-creation and self-transformation which instruments served as contributing factors
in the formation of the artwork.
Conclusion
Co-creative practices including those involving non-human actors were explored through the creation of an
ongoing artwork project entitled Ēngines of Ēternity. The nature of such transdisciplinary
methodologies was investigated by a small community of artists and scientists involved in the project, whose
evolving discourse during the process of developing and reflecting on the project has helped to provide a
framework to inform conversations regarding the future of humanity, and whether and how we might move towards a
more post-Anthropocene stage in the future. Establishing an ethical post-Anthropocene requires approaching the
Other from a humble and empathic perspective. This case study provides insight into a practical approach towards
this objective. For example, by focusing on the intrinsic qualities of creation present in the non-human agents.
Nonetheless, it is evident that there are challenges that need to be addressed to successfully accomplish this
goal.
References
1 Erich Jantsch, "Inter-and transdisciplinary university: A systems approach to education and
innovation", Higher education 1, no. 1, 1972, 7-37.
2 Erle C. Ellis, Anthropocene: a very short introduction, Vol. 558, Oxford University Press,
2018.
3 Angelo CJ Vermeulen, Diego Maranan, Pieter Steyaert, Nassim Versbraegen, Ann Peeters, Jeroen Verschuren,
Arise Wan et al. "Ēngines of Ēternity: An Artistic Inquiry into Space Settlement Ideology Using
Rotifer Experiments on Board the ISS," In 73rd International Astronautical Congress, 2022.
4 Pamela G. Cajilig, Diego S. Maranan, Arlene Sy, Oliver Salva, Angelo Vermeulen, "Multispecies Roundtable
for Climate Impact: A Speculative Proposal," 2017.
5 Diego S. Maranan, Angelo CJ Vermeulen, "When ideas migrate: A postcolonial perspective on Biomodd [lba2],"
2015.
6 A. C. J. Vermeulen, Caroline Nevejan, Frances Brazier, "Seeker: Co-Creating Diversified Futures,"
Studio Time: Future Thinking in Art and Design, Black Dog Press, London, 2018, 172-182.
7 Van der Bijl-Brouwer Mieke, Giedre Kligyte, Tyler Key, "A co-evolutionary, transdisciplinary approach to
innovation in complex contexts: Improving university well-being, a case study," She Ji: The Journal of
Design, Economics, and Innovation 7, no. 4, 2021, 565-588.
8 Solomon Bililign, "The need for interdisciplinary research and education for sustainable human development
to deal with global challenges," International Journal of African Development 1, no. 1, 2013, p.8.
9 Farzam Ranjbaran, Cristina Marras, "European Peer Review Guide: Integrating Policies and Practices into
Coherent Procedures," ESF Member Organisation Forum on Evaluation of Publicly Funded Research, 2011.
10 Bruno Latour, Reassembling the social: An introduction to actor-network-theory, Oup Oxford, 2007.
11 Bruno Latour, "On technical mediation," Common Knowledge 3, no. 2, 1994, 29-64.
12 Bruno Latour, Enquête sur les modes d'existence: une anthropologie des modernes, Paris, La
découverte, 2012.
13 A. C. J. Vermeulen, "Living computers, Mars simulations and DIY Starships: Advancing cross-disciplinary
and cross-cultural collaboration," Link 3 Workshop Space, Plymouth University, 2016.
14 Donna J. Haraway, When species meet, Vol. 3. U of Minnesota Press, 2013.
15 Matthieu Terwagne, Emilien Nicolas, Boris Hespeels, Ludovic Herter, Julie Virgo, Catherine Demazy, Anne-
Catherine Heuskin, Bernard Hallet, Karine Van Doninck," DNA repair during nonreductional meiosis in the
asexual rotifer Adineta vaga", Science advances 8, no. 48, 2022, eadc8829.
16 Norman G. Lederman, "Nature of science: Past, present, and future," In Handbook of research on science
education, Routledge, 2013, 831-879.
17 Jonathan Osborne, Collins Sue, Mary Ratcliffe, Robin Millar, Rick Duschl, "What
“ideas-about-science” should be taught in school science? A Delphi study of the expert community,"
Journal of research in science teaching 40, no. 7, 2003, 692-720.
Acknowledgements
This project is funded by the European Union’s Horizon 2020 research and innovation programme under the
Marie Sklodowska-Curie grant No 860470.
Author Biography
SEADS (Space Ecologies Art and Design) is a transdisciplinary and cross-cultural collective of artists,
scientists, engineers and activists. Its members come from all corners of the world. SEADS is actively engaged
in deconstructing dominant paradigms about the future and develops alternative models through a combination of
critical inquiry and hands-on experimentation.
SEADS employs its own signature methodology which is centered around community building, co-creation and
bottom-up design. SEADS believes that these approaches are key to unlocking collective intelligence, a
prerequisite for generating more diversified and inclusive futures. Furthermore, SEADS also embraces a hacking
and open-source ethos, with the goal of engaging as many people as possible in the activities and ideas that
they initiate. Since 2009, the collective has co-created more than 40 art projects, together with local
communities all over the world.
Spectral Plain: A case study for exploring the world-building potential of co-creative systems that combine
text generation models with game mechanics.
1École des Arts Décoratifs – PSL (EnsadLab), University of
Applied Sciences and Arts Northwestern Switzerland (IXDM) 2KU Leuven, LUCA School of Arts,
3Hong Kong University of Science and Technology 1guillemette.legrand@ensad.fr,
guillemette.legrand@fhnw.ch, 2vincentsimon.thornhill@kuleuven.be,
3ijclarke590@connect.hkust-gz.edu.cn *These Authors contributed equally.
Abstract
This paper describes the game installation ‘Spectral Plain’; introducing a case study of an
interactive artwork that intersects algorithmic, sensing, and gaming technologies to explore new forms of
generative world-building, which are different from the world-imaginations created by commercial and political
narratives. Through this case study, we aim to explore how information technologies might embed specific
socio-cultural beliefs. Here, we investigate the imaginaries created and disseminated through dominant
representations of information technologies, while simultaneously searching for ways to create new multiscalar
imaginaries of the earth through the mobilisation of these technologies. This case study describes how physical
and digital interactions can be combined with AI and randomising technologies to foster forms of co-creation
with and between humans. It also explores the capacity for AI-integrated game environments to decenter
anthropogenic perspectives by creating new symbiotic interrelations with the non-human and the non-living that
can impact participants’ perception of their understanding of the world.
Keywords
World-building, cosmologies, game mechanics, interactive installation, autoregressive language model,
co-creative system.
This project started with an intention to examine, and communicate the capacity of information technologies to
construct imaginaries of the "world." The initial technological context for this investigation was the
commercial and political representation of 5G wireless technology; the latest in a lineage of "network"
imaginations of a world coinciding with the development of communication technologies. Professor of Anthropology
Shannon Mattern argues for a need to acknowledge how marketing fantasies and policy scenarios have limited our
imaginations, not just of what the technology is, but also of how we organise and live in the world through
these imaginations.¹ Philosopher of Science John Tresch scales Mattern’s concerns to the imagination
of the planet, stating that “when modern technology is linked to a ‘cosmology’ or a
‘world(s) view,’ we are offered a much thinner picture than that suggested by anthropology’s
presentations of the ways humans have organized nature and society—a ‘mechanization of the world
picture’ simplifying all experience to utilitarian building blocks of masses and forces, functionalist
means and ends.”² Between the marketing and political narratives of 5G, and the technical operations
of this infrastructure, we saw the potential to build a new reading of wireless technology.
This rereading was pursued through decentering wireless technology from the regulatory tradition of allocating
and auctioning ‘bandwidth’ of electromagnetic frequency to commercial and geopolitical actors.
Instead, we wanted to reposition this technology by placing it within a wider spectrum of frequencies that human
and non-human entities are intertwined with. Imagining electromagnetic energy in new scales and alignments
prompts us to think at a cosmological dimension, a concept derived from Timothy Morton’s description of a
"Spectral Plain;" an infinite surface where “you can’t really distinguish very easily between alive
and not alive, between sentient and non-sentient, between conscious and non-conscious.”³ Through
enriching and diversifying the comprehension and representation of electromagnetic frequencies, the project aims
to open up new imaginaries of the earth.
Research Process
The research process centred on testing ways to produce narratives about electromagnetic energy that cross
between human and nonhuman scales, from the atomic, to the terrestrial, to the cosmic. To do this, we created a
series of working sessions through which the project was developed through participation with varied audiences
and formats, while iterating the development of technological systems for generating narratives. These
experimentations took the shape of online and physical workshops and interactive performances that were set up
to test different forms of collective world-building; methods that were later implemented in the final artwork.
We used the game environment as a site for experimentation, which enabled us to test ways that various modes of
representation and points of view could coexist or create friction. It also allowed us to play with multi-scalar
experiences; from micro to macro, from the local (storing of individual data about the player locally) with the
planetary (data processed through large distributed algorithmic models). Through recombining different
information technologies, we created an experimental ground for a collective to engage with world-building. We
use the exhibition periods as a moment to present the work to a public, but also a moment to extend our research
through further testing of the system we created, while collecting additional feedback on the players'
experiences.
Of the different technologies involved, our research into the use of GPT-3 had the most impact on how we were
able to reconsider the ways collective world-building processes could be constructed in co-creation with AI
technology. Several games have explored the use of GPT-3 to generate open-ended narratives, such as AI
Dungeon.⁴ However, rather than using this technology to create divergent storylines, we sought to use the
intrinsic properties of this linguistic model to establish connections and patterns between different bodies of
knowledge about electromagnetic energy.
Project Description
Our artistic research into co-creative systems through the medium of the game environments led us to develop
the artwork as an interactive game-installation, called Spectral Plain (figure 2).
In this interactive installation, three players can navigate three levels of an environment (the skies, the
terrestrial and the underground). The game story takes place in a period following a fictive geomagnetic
reversal (a change in the planetary magnetic field), where ‘spectral energy’ is unleashed, and
players are engaged in discovering new perspectives on electromagnetic energies.
The installation uses a real-time data capture and a text-generation algorithm (GPT-3) via an integrated
blueprint in the game engine (Unreal Engine) to log in various datum (time stamps, players' statement
selection), which is then used to inform the final audio-visual cosmogram.
The three levels of the game are developed in a systematic way with each environment being constructed around
five “spectra” of electromagnetic frequencies (figure 3) that are named: aqua, light, geology,
matter, and device. These frequencies manifest as different entities within the game environment, for example,
the light spectra manifests as chlorophyll in the terrestrial level.
The presence of the three players in the physical environment of the installation creates a unique disturbance
in the gamma radiation that is present around the installation. In the game infrastructure, this disturbance
triggers the Geiger counter to produce a "true random number." This number is then used within the game engine
blueprint to randomise the graphics that the players will interact with and the parameters within GPT-3.
The game environment is developed to give agency over the creation of a new world-imaginary of spectral energy
that is different from the dominant 5G narratives. Here, we align with philosopher Federico Campagna’s
position towards the concept of "world;" not using the word to describe an ontological reality, but rather
seeing the world as an artificial construct of the imagination, “a metaphysical process that creates
discontinuities, separations between things, individual identities, ‘somethings’ out of a plane of
pure existence where no clear divisions are already inscribed.”⁵
At the end of the players’ journey, a cosmogram is automatically generated (figure 4). This cosmological
diagram is constructed by gathering data about the players’ movement and interactions within the levels of
the game. From this cosmogram, unique cosmological readings of the unified journey are then created through
fine-tuning the GPT-3 language model. These readings are then "spoken" through the use of speech synthesis
software. In this process of co-creation between player and algorithms, we are interested in how automated AI
technology can be used to generate connections between diverse perspectives and narratives of energetic entities
(scientific, mythologic, commercial narratives). The reading is not an attempt to accurately describe
relationships between these perspectives, but rather, to create an artistic and lyrical intervention.
Technical Realisation
In the Spectral Plain interactive game-installation, we combine physical and virtual player interaction,
together with a combination of predictive (GPT-3) and randomising information technologies (Geiger counter
radiation sensor as true random number generator (TRNG). The input and output of these different interactions
and technologies are linked and moderated through Unreal Engine’s Blueprint Visual Scripting
system.⁶ The node-based interface of this system allowed us to map and program interactions in the
back-end of game creation software in a visual manner.
At the intersection of cosmology, and generative technologies, we also became interested in logic and structure
of different divination techniques from different cultures. In Spectral Plain, we develop an analogy to
techniques such as birth chart readings, and Yi Ching, by approaching their logic as a form of programming. In
our technical realization of the project, processes of symbolisation, inclusion of chance, and geometric
calculation can be seen as referential to ancient divination techniques that link the individual to the
planetary.
The section attempts to walk through the different steps taken in the process of playing Spectral Plain by
annotating the diagram (figure 5 below):
The presence of the three players in the physical environment of the installation creates a unique
disturbance in the gamma radiation that naturally exists in the immediate vicinity. This disturbance triggers
the geiger tube in a gamma radiation sensor in a (literally) unique way, producing a random number
analogically, rather than digitally. As it is produced analogically, it is considered a "true random number."
This true random number is then used within the Unreal Engine blueprint to randomise statements generated by
GPT-3. These statements are fragments of stories about electromagnetic ‘entities’ that are located
throughout the different layers of the game environment. The players log their choice of statement into the
game engine blueprint by pressing one of the graphic
symbols ‘✕’ ‘◯’ ‘▢’ ‘△’ on their
controller.
Information about what buttons are pressed, and information about the time spent at each entity is logged
into the blueprint. The combination of this information establishes an order of importance in the entities
the player’s encounters during their journeys in the game.
These two sets of information are processed by a program that draws a vector space between the three most
important entities per player. These vector space triangles are placed on top of a pre-rendered cosmogram
drawing (figure 4) which spatialise the different entities in the game.
By overlapping the triangles of each player, we determine a middle point that will be used as a seed for
GPT-3. This middle point gives the meta direction to the generated text, while the most important entities
for each player are described within the body of the generated texts.
The structure of each cosmogram reading is generated based on a prompt that requests a title, a description
and an advice.
The text is then processed by a voice synthesiser to produce an audio file that is played when the players
reach the visual cosmogram that is automatically inserted into the concluding environment of the game.
Conclusion
Spectral Plain is an interactive game-installation that builds upon concepts present within ancestral
technologies (birth chart construction, Yi Ching reading instruments, etc.) through the use of current
technologies in order to enable a space for co-creation, where symbiotic interrelations between disciplines
(art, geoscience); the non-human (e.g., glowing micro-algae, chlorophyll photosynthesis); and the non-living
(e.g., AI, geological elements) can occur. In this technical and conceptual description of the work and its
intended effects, we aim to contribute to the discourse surrounding the ontologies of information technologies.
This paper also aims to create a case study to allow for continued questioning of how artistic research can
challenge, mobilise, and enrich the imaginary potential of these technologies.
By advancing the possibilities to engage with and through narrative-based technologies within
game-environments, we follow Alenda Y. Chang’s proposition to explore "the untapped potential of games to
create meaningful interaction within artificially intelligent environments, to model ecological dynamics based
on interdependence and limitation, and to allow players to explore manifold ecological futures—not all of
them dystopian.”⁷
The project requires further technical and conceptual development in order to explore ways to balance and
represent generative-based components, chance factors, and collective agency within the game-narrative. It is
also important to question the ability of large language models like GPT-3 to generate a truly novel story, and
find ways to negotiate with its capacity to homogenise culture and beliefs through language parsing and
recomposing.
References
1 Shannon Mattern, “Networked Dream Worlds,” Real Life , accessed December 7, 2022,
https://reallifemag.com/networked-dream-worlds/
2 John Tresch, “Technological World-Pictures: Cosmic Things and Cosmograms,” Isis Vol.
98, No. 01, accessed December 7, 2022, http://dx.doi. org/10.1086/512833 (online)
3 Timothy Morton, Being Ecological, Cambridge Massachusetts, MIT Press, 2018, 126.
4 Latitude “AI Dungeon (2019),” accessed December 7, 2022, https://play.aidungeon.io
6 Unreal Engine, “Anatomy of a blueprint (2022),” accessed December 7, 2022,
https://docs.unrealengine.com/4.27/en-US/ProgrammingAndScripting/Blueprints/Anatomy/
7 Alenda Y. Chang, Playing Nature: Ecology in Video Games, University of Minnesota Press, 2019,
16.
Ecotechnologies of Practice: in-forming changing climates
GisèleTrudel
École des arts visuels et médiatiques, Université du Québec à
Montréal. MÉDIANE Canada Research Chair in Art, Ecotechnologies of Practice and Climate Change.
Montréal, Canada. trudel.gisele@uqam.ca
Abstract
How do ecotechnologies of practice actualize? This paper traces the material/theoretical operations of an
ongoing research-creation concerned with changing climates. It mixes information in experimental approaches from
collectivities of trees, media arts and forest sciences. Through individuations of symbiotic modulations, the
paper is a thinking-with Balsam Fir, Diana Beresford-Kroeger, Camera, Domingo Cisneros, Dendrometer, Erin
Manning, Isabelle Stengers, Jack Pine, Gilbert Simondon, Light Emitting Diodes, Numbers, Microphone, Recorder,
Sapflow, Sensings, Speakers, Sugar Maple, Temperature, Yellow Birch.
Keywords
Research-creation, changing climates, trees, ecotechnologies of practice, information, individuation,
art/sci/phi.
An interdisciplinary sharing is at work. Ælab joins forest engineer Christoforos Pappas who is explaining
to them how to attach a sapflow sensor to a tree in the boreal forest (Figure 1). The tree already senses with
the surroundings, the sapflow sensor brings the tree's operations into human understandings, as sociologist
Jennifer Gabrys has discussed.¹ Inter-cross-transdisciplinary approaches activate philosopher Isabelle
Stengers' concept of “ecologies of practice” in modern science.² To move outward of
disciplinary silos, she states openness is required while still maintaining tension, to question how practices
produce knowledges (plural added; savoirs in French). Stengers argues for knowledges production in
interdependence, within the practices of each scientific field, and in-between them. Each field can contribute a
differential worldview to one another, a distinguishing plurality.
Scientific fields work actively to try to solve climate changes. Yet, what else can occur in-between the
sciences, trees, art and machines? There is an important increase of sensual and affective artistic explorations
with trees, among others by Jane Tingley, Domingo Cisneros, Susan Turcot, Rasa Smite and Raitis Smits, Agnes
Meyer-Brandis. In this research-creation project, scientific and tree knowledges are expressed in outdoor media
art installations. How can knowledges and practices resonate together, within and through a multitude of fields,
given the concerns for the “more-than of life-living” of the blue planet?³
Experimental in-forming
The concepts of information and individuation offered by Gilbert Simondon provide insight for the encounter of
ecologies of practices.⁴ In his philosophy, information is more-than what is generally considered as
factual data. Information is operation. It activates change through perturbation, by provoking or inducing an
encounter between disparate elements/factors, as event. Potential is propelled into actualization through a
dynamic play of forces. This is of particular importance in the context of climate change where fearful and
helpless affects predominantly circulate. How else does change occur?
The first outdoor media installation by MÉDIANE to explore the potential of changing climates is bois
eau métal (2021), an artwork concerned with the ecophysiology of three trees, with numerical data
from Smartforests Canada research site, including immediate weather sensing. Presented in the Arboretum, the
scaffolding structures are a site of temporary co-constructions (chantier), mini pavilions without
walls for experiencing three types of immersive sounds setups, video imagery and graphic visualizations. Tree
data from a forest is co-composed with colors from a custom-made weather kit. The whole manifests as rhythms of
physical processes in feedback loops. This installation acts as information, pushing the exploratory
differential encounters between fields of knowledges and experiences in a shared reality with trees. Stengers
analyzed science domains which often perpetuate their own methods of safeguarding knowledges production.
Contemporary art often relies on the unwavering codes of modernist heritage to produce and present artwork as
autopoiesis, production of self “in and of itself”. To promote changes, in Stengers' words,
“this is the reason why, through the exploration of knowledges, there is also a type of ethical
experimentation” (our translation).⁵ Respecting and challenging, one and other. Shifts occur, along
with the tree's own experience; despite the tree being rooted in one location, seemingly immobile,
the sensor data shows the tree is constantly moving and changing at microscales.
Changing climates warrant to be addressed as a “problem” in and of change, within
research fields and in their differential gathering, and in dialogue with diverse publics. How can
ecotechnologies of practice address the problem of changing climates, understood here as an obstacle for thought
in and into action, rooted generally in fear, anxiety and helplessness? This is an obstacle needing new thought,
a coproduction of knowledges in practice, in experience. Stengers and Simondon provide compelling offerings to
think-with and act-with problems, in experimentation. In Simondon's concept of individuation, no individual
(human, instrument or otherwise) preexists to the encounter. Individuation occurs through the meeting of forces,
garnered little-by-little and in proximity, gatherings that build and change in time. Propelled into change by
the perturbation of information, triggered by the tree's mode of living, change happens. For better or for
worse, without moral imperative, it is a propositional ongoing, a doing, building with each tiny
difference.⁶ This approach is core to MÉDIANE's ongoing research-creation process, whereby each
installation proposes change, building with the tree, data gathering, analysis, and by the gradual spreading of
knowledges production and circulation in-between trees, forest science, out-of-doors art and publics.
An orientation, not a beginning
The slogans of the FridaysForFuture global strike movement (FFF) repeat that governments, corporations and
publics need “to listen to the science, to listen to the scientists.”⁷ Scientists are directly
convened to address climate change, outside of the silos of fundamental knowledges production. Undeniably,
scientists are needed and respected to do so. However, it points to the domination of science as holding
exclusivity to resolve climate emergencies. Peoples (plural) have perhaps given over climate change issues to
science and to government to solve, forfeiting distributed agencies. How can a combination of practices
contribute to new actions? With Stengers, it becomes possible to extend the ecologies of practices in scientific
realms to a mixing of approaches by ways of practice, in-between collectivities. A new knowledges
circulation, a thinking-with trees, machines, materialities, artists, scientists and publics, even if only
temporarily. This problem for thought in/to action, perturbed by climate changes, is addressed by
ecotechnologies of practice. In Figures 5 and 6, the 2022 media installation entitled Orée des bois
addressed these questions through the study of the birch tree's phenology, the seasonal changes from Spring
to Fall which affect growth and connections with flora and fauna.
A problem of & with & in practice
For Stengers, ecology is always double: scientific and political (italics added).⁸ This doubling
can be activated in contemporary art, design and well as social sciences, charting the trajectories of actants.
In Greek, technè is art, skill, and craft or the method employed resulting in an
object.⁹ Logos tries to lay claim to (human) language, to “collect, gather, in
derivatives of word, speech, statement, discourse, computation, account.”¹⁰ To consider
technology only in a prepotent human way provides humans with vector to dominate and control, to justify the
means to an end, often neglecting entangled elements/lives/factors. Technology understood in this dominant way
severely limits approaches, because if it is seated only in the human as an exclusive semiotic rationality, it
prevents or even opposes new informations to coalesce or collide with more-than and other-than. It is pertinent
to note that in sustainable architecture, a definition for ecotechnologies is holistic systems or machines with
the environment, including reuse, passive housing, renewable energies, crade-to-cradle designs.¹¹
Simondon proposes another technology, neither thing nor method fashioned by human will. In his philosophy,
individuation is actualized through “phases” of relations between physical environments, instruments
and humans. Technology is in-forming, changing, extending another expression of logos, the cosmos
expressing its manifold in movement with animals, minerals, plants¹². A
“life-living.”¹³ It is also the combined strengths of trees, scientists, artists and
publics in changing climates.
This ongoing research-creation proposes ecotechnologies of practice in a crafting of
“response-abilities.”¹⁴ The tree crafts problem-solving operations with the
surroundings: tackling water preservation or evapotranspiration as needed, regulating sap flow, making clouds,
coordinating burgeoning and senescence timings, going dormant in northern winters, turning sapflow into
antifreeze to prevent embolisms.¹⁵ Furthermore, trees produce photosynthesis in a cosmic relation of
sun and soil, they yield oxygen and even grow stronger and bigger with the extraction of carbon dioxide from the
atmosphere.¹⁶ They communicate through VOCs (volatile organic compounds) and mycelium
networks.¹⁷ These are but a few of the tree's wondrous techniques, parts of a larger continuum to
coevolve their technologies over 350 million years.¹⁸ To partake gently in their knowledges,
Indigenous artist Domingo Cisneros proposes eating and drinking parts of coniferous trees without harming the
plant, a celebration to share in the strength of the longevity of their problem-solving techniques.¹⁹
In a similar celebratory fashion, medical biochemist, botanist, climate activist and author Diana
Beresford-Kroeger is as adamant as she is hopeful in times of changing climates. Her “Bioplan” makes
simple statements. She advocates to protect trees in the immediate vicinity as well as native forests in local
communities.²⁰ So efficient, it's alarming. Her approach changes the habitual urgent call
associated with resiliency and adaptation of diverse communities that are foregrounded through catastrophic
discourse. Change—with life-living trees—is summoned as operation. As Stenger states, consensus is
stasis, refusing (to) change. Beresford-Kroger's bioplan extends symbiotic intimacy with tree technologies.
Ongoing Doings
Ecotechnologies of practice entail a perturbation in the habitual conception of the relation of ecology and
technology, explored here as operative process. The tree's ecological practice, in conjunction with
that of arts and sciences, investigates changing climates with creative problem-solving in dialogue with
publics. The differential fields of knowledges bring new expression by their combination, a strength emerging
in-between, pushing forth modular alliances. Crafting changes is ecotechnologies of practice: even if ephemeral,
potential is actualized through encounters, soliciting variation and individuating collectivities.
Acknowledgements
The research-creation is conducted on the unceded lands of the Kanienʼkehá ka Nation. The
MÉDIANE team gratefully acknowledges funding from SSHRC, CFI, FRQSC and support from Hexagram-UQAM.
References
1 Jennifer Gabrys, Program Earth. Environmental Sensing Technology and the Making of a
Computational Planet, Minnesota, Minnesota University Press, 2016.
2 Isabelle Stengers, La guerre des sciences, Cosmopolitiques 1, Paris, La
Découverte, 1997.
3 Erin Manning, Out of the clear, Colchester and New York, Minor Compositions, 2022.
4 Gilbert Simondon, L'individuation à la lumière de notions de forme et
d'information, Grenoble, Jérôme Millon, 2005.
5 Isabelle Stengers, La guerre des sciences, 8.
6 Gilbert Simondon, L'individuation à la lumière de notions de forme et
d'information.
7 Accessed November 4, 2022, https://fridaysforfuture.org.
8 Isabelle Stengers, La guerre des sciences, 58.
9 Accessed November 6, 2022,
https://www.oxfordlearnersdictionaries.com/definition/english/technology.
10 Accessed November 6, 2022, https://www.etymonline.com/word/-logy?ref=etymonline_crossreference.
11 Accessed November 6, 2022, https://leverageedu.com/blog/ecotechnology/.
12 Carl Sagan, “Cosmos (1980),” IMDb website, accessed November 1, 2022,
https://www.imdb.com/title/tt0081846/characters/nm0755981#quotes.
13 Erin Manning, Out of the clear.
14 Donna J. Haraway, Staying with the Trouble: Making Kin in the Chthulucene, Durham,
Duke University Press, 2016.
15 Jehová Lourenço Junior, Community Ecology - Wood Anatomy - Plant Physiolog,
Accessed November 3, 2022,
https://jehovajr.wixsite.com/science/post/the-hydraulic-architecture-of-trees-in-a-climate-changing-scenario
16As Carbon Dioxide Grows More Abundant, Trees Are Growing Bigger, Study Finds,
Accesssed October 10, 2022, https://e360.yale.edu/digest/carbon-dioxide-climate-change-bigger-trees
17 Suzanne Simard, Finding the Mother Tree, Allen Lane, Penguin Random House Canada,
2021.
18 Serge Muller, Germinal Rouhan, “Comment les arbres sont apparus sur Terre,”
Museum, November 30, 2021, accessed November 1, 2022,
https://www.mnhn.fr/fr/actualites/comment-les-arbres-sont-apparus-sur-terre.
19 Ælab and CREAF (Centre de recherche et d'expérimentation des arts forestiers),
“Les trois frères (2020),” https://vimeo.com/channels/472281/452356412).
20 Accessed November 15, 2022, https://dianaberesford-kroeger.com/mission/.
Author Biography
Gisèle Trudel and Stéphane Claude founded the experimental documentary artist research unit
Ælab in 1996 (aelab.com). Collectivities actualize the potential of MÉDIANE (2020-2025,
mediane.uqam.ca).
Water Stories: Visual Poetics and Collective Voices
AndreaWollensak,
BrettTerry, BridgetBaird
Ammerman Center for Arts & Technology, Connecticut College Sonalysts Emerita Computer
ScienceDepartment, Connecticut College New London [authors 1 and 2] and Waterford [author 3]
CT,USA ajwol@conncoll.edu brettcarrollterry@gmail.com bbbai@conncoll.edu
Abstract
Water Stories: Visual Poetics and Collective Voices is a two-part project that brings together
multiple points of view from local youth, community, and poets in Alaska to share what water means in their
life. Visual Poetics combines a live poetry reading by Alaskan poets and interactive video in which the
poets’ voices trigger generative visual elements. Collective Voices is a sound work
featuring excerpts of community voices sharing water-based memories against a backdrop of processed
environmental sounds of Alaskan waterways. Water Stories is part of a year-longartist residency
(2021-2022) with the Anchorage Museum culminating in a series of listening sessions broadcast at the Anchorage
Museum and Out North Radio, live interactive poetry readings at the museum,and video projections on the museum
façade from November 2022 through January 2023.
Keywords
Generative, digital poetics, audio-reactive software, community engagement, collaborative, ecosystems,resilient
societies, multiple scales.
Alaska’s climate is projected to warm several degrees by the end of the century, more than twice the
warming compared to the rest of the country. The record-breaking temperatures experienced by Alaska communities
have resulted in thawing permafrost, thinning sea ice, and more wildfires. The Water Stories: Visual Poetics
and Collective Voices project engages the Alaskan public to consider how climate change alters their
experiences of the land, and to consider the role of water in their lives.
Background
Water Stories is the most recent artistic collaboration between the authors, and is part of aseries of
works focused on climate change and the environment. Reading the Wrack Lines (2021),an outdoor audio
video projection on a lighthouse, featured community-sourced textual reflections on climate
change. Open Waters (2020) was a multiple-media installation exhibited at museums in Rhode Island,
and New York, connecting the five-hundred-year imaginative history of the open polar sea and northwest passage
with current geopolitics and arctic climate change. Another previous audiovisual collaboration, Ice
Core Modulations: Performative Digital Poetics (2017) included generative imagery and poetic texts
controlled via historical CO2 data taken from ice core samples made available from the National Snow and Ice
Data Center (NSIDC). All three of these works have been presented at past ISEA Conferences.
These collaborations have sought to create engaging works that address complex environmental issues through a
plurality of artistic, poetic, and scientific perspectives. The collaborators have brought to bear individual
expertise in audiovisual generative computer programming, visual art media, poetry, and electroacoustic sound
composition to synergistically create works unified by shared source materials that include place-based personal
narratives, historical source material, raw scientific data, local audiovisual media collected on site, and
original poetic texts.
Goals, Objectives and Artistic Dimensions
Water Stories seeks to amplify voices of Alaska residents through storytelling, poetry, and local
landscape imagery. The overarching project goals and objectives include:
Collaborating with local community partners.
Revealing evidence of a changing climate in lived spaces through creative voices and artistic media.
Using narrative storytelling as a way to bear witness to our changing environment through personal
experience.
Seeking to draw out the poetic from connections between visual arts, language arts,science/technology.
Presenting community voices via innovative multi-sensory formats.
Creating a reflective and engaging digital poetics experience thematically focused on environmental change
and disappearance of place.
The artistic dimensions of Water Stories are framed by community storytelling and creative
writing that focus on the local water and environment in Alaska. Based on these objectives, Water
Stories resulted in two distinct works showcasing community voices from Alaska:
Collective Voices is an audio composition that amplifies voices of Alaskan youth and community
through stories of what water means in their life.
Visual Poetics is a dynamic audiovisual work (Figure 1) developed with Processingsoftware in which
poetic fragments, background photography and abstract visual elements interact in response to a live poetry
reading.
Ecosystems of Collaboration
There are two synergistic types of collaboration taking place within the WaterStories work. In the
first, the collaboration between the creators of the work allows artists to bring their own unique perspectives
and disciplinary skill sets together. As the collaborators have previously worked together on site-based work
exploring themes of climate change, several methods already developed to facilitate this collaboration were
leveraged for this effort, in particular the use of original poetic phrases to provide asemantic glue unifying
different visual, sonic, and textual elements. In this case, the performative nature of the collaboration was
underscored by having a live poetry reading be a central element of the work. With the poet present, hearing
their words while seeing the work respond to their speech connected the underlying collaborative synergy between
text and image on the surface, and at a deeper level, with the connections implicit in the semantic connotations
and subject matter of the poetry itself.
The second level of collaboration in the work connects the audience and creative producers,capturing community
narratives through interviews and recorded reflections. CollectiveVoices empowers individual voices
through community collaboration, allowing them to interplay and be heard in a public creative context, fostering
a reflective audio-based ecosystem through which the listener experiences a plurality of narrative and poetic
perspectives. Through its use of intertwined recorded personal reflections, the work bridges the private/public
gap, and was shared publicly via broadcast on radio, internet, and as part of an outdoor audio installation at
the Anchorage Museum.
Visual Poetics
Alaskan poets Erin Coughlin Hollowell and Jen Stever each composed ten poems as part of this collaboration.
Framed by her Iñupiaq culture and history, Stever’s poems focus on the local environment, family,
and the changing climate. Hollowell’s poems explore careful observations of the coastline and reflections
of visual language and form. Distinct phrases from each poem were incorporated into the Processing software in
several different visual representations. Randomly selected phrases fade in and subsequently fade out within the
audiovisual work, with some phrases coalescing from and dispersing into granulated pixels.
During several trips to Alaska, photographic and video source materials were collected and used as background
imagery within the work (Figure 2). Adding to the text and shifting background imagery, different visual
elements and behaviors were developed in Processing for each poem, using a vocabulary of moving generative forms
include triangular networks, shards, and ellipses, each with distinctive color palettes tied to the background
imagery (Figure 3). A reactive audio system was developed to listen to the poet’s voice, using amplitude
and frequency to trigger and modulate the generative forms, with different behaviors for each poem.
Two versions of the work were developed. The first version was a live performance in which both poets were
present reading their works, accompanied by the audio reactive Processing software running in real-time to
generate visuals projected behind the poet. The second version (Figure 4) was made to project nightly on the
façade of the Anchorage Museum for several months, and was made by recording a performance of the work as
a video file. The two versions thus exhibit differences of scale, accessibility, and connection to the spoken
voice.
Collective Voices
To create the eight-minute audio composition Collective Voices, several recording sessions took place
at the Anchorage Museum and OutNorth Radio in which community participants shared personal narratives about
place and local waterways. The authors would like to gratefully acknowledge community partners Indra Arriaga
Delgado and OutNorth, a progressive arts organization with a mission to advance contemporary art in Anchorage,
support underrepresented voices, and promote cultural dialogue by amplifying the alternative voice. In order to
promote a diversity of voices, many different community groups were invited to participate including high school
students enrolled in the Anchorage Museum’s afterschool Teen Climate Communicators program and members of
the general public invited to record their stories at the recording session venues. Selected excerpts from these
narratives were edited into an audio work using Logic software and combined with multiple layers of local
processed environmental sounds from the Anchorage Museum’s audio collection. Collective Voices was
presented at a listening station at the Anchorage Museum during their November First Friday community event,
broadcast on Out North Radio, and made available for internet listening via SoundCloud.
Conclusion
Water Stories: Visual Poetics and Collective Voices engages multiple points of view from local youth,
community, and poets in Alaska to share what water means in their life. Grounded in a year-long artist residency
at the Anchorage Museum, this interdisciplinary project explores the changing environment through place-based
ecosystems of collaboration.
Authors Biographies
Andrea Wollensak is a multimedia artist, designer and educator. Her work spans media from traditional and
digital fabrication to generative-interactive systems and includes frequent collaborations across disciplines.
Wollensak’s work has been exhibited internationally, most notably at the Göteborg International
Biennial of Contemporary Art and the Brno Biennial of Design. Her work has been supported by the Rockefeller
Foundation, the International Artist Studio Program in Sweden, Anchorage Museum, Banff Centre for the Arts,
and the National Science Foundation. She has presented her artwork at numerous venues including ISEA, CAiiA,
Generative Art, and College Art Association. At Connecticut College, she is on the faculty in the Studio Art
Department, and an Associate Fellow at the Ammerman Center for Arts and Technology where she served as
Director from 2012 to 2020.
Brett Terry is a composer and sound artist, when not busy with his daily life as a software engineer. His
electro-acoustic, choral and chamber compositions have been performed at venues such as SEAMUS, ICMC, ISEA,
CAiiA, and Sound Culture in addition to collaborating with visual artists on numerous audiovisual works. As an
associate editor of Computer Music Journal (MIT Press), he has curated a special issue on Visual
Music.
Bridget Baird is a Professor Emerita in Computer Science and Mathematics at Connecticut College and a past
Director of the Ammerman Center for Arts and Technology. Her research examines the intersections among the
arts and various technologies. Projects include investigating an archaeological site in Ecuador through
virtual reality, exploring music and dance through motion capture and multiple modalities, using digital
techniques and algorithms to better understand and mine historical documents, and more recently, addressing
climate change and environmental concerns by using generative art. Baird collaborated, as a Fulbright scholar,
with colleagues in both Mexico and Ecuador. Involvement with the local community has also been important to
her and a constant interest has been to increase the number of women in the sciences.
Nga manawataki o te koiora: biological rhythms, posthuman design and decolonial thought.
RewaWright, SimonHowden
Queensland University of Technology Independent Artist Meanjin/Brisbane, Australia
rewa.wright@qut.edu.au
Abstract
Western science, in fields such as computational ecology, has grown to accept the truths that Indigenous
culture have long known: that computational ecology accepts that ecological models are too complex to be
summarised in computational form. Since this complexity evades the codification of mere indexing, how then,
should we work with computational companions (code, algorithms, programs, platforms). What new ways of
intra-acting can we develop alongside computational frameworks, which bring us one more step closer to sentient
machines? Most importantly, how can ethical ways of thinking and doing motivate transformations in the
computational space, in areas such as machine learning where extreme problems of bias are now embedded? This
research does not answer these complex questions, for they are genuinely ‘wicked problems’ that
reach toward wider issues of equity, sustainability, and economy. Our aim is to use creative practice to
generate gestures and markings that tentatively trace a way forward. This research contributes to new modalities
of human computer interaction that attempt to restore the dynamic pathways developed by Indigenous thinking,
challenging artificial boundaries such as nature/culture, instead giving respect to concepts of interconnection.
Examining some of the differences between Western epistemology and Indigenous thinking opens a pathway toward
Indigenous Futures that are crafted in support of a decolonial ecology.
Through imaging technologies such as electron microscopes, quantum physics has been slowly proving to itself
that all particles in the universe are deeply entangled. Now, the curious situation exists where Western science
has ‘validated’ what mātauranga Māori (Māori knowledge) has known and preserved
for thousands of years. A close analogy for the unseen movements of quanta is mauri (transitory and shifting
states of being), while the structure these quanta form with other particles, emerges in wairua (felt
interconnections between all living things). In the Māori world view, people and nature are inseparable
and share genealogy. This resonates with current scientific opinion which has only come around too lately to the
perspective that plants are to be as valued as humans. Indigenous thinking is in fact far ahead of Western
science, in that we never lost the intuitive ways of knowing and being, and we never blindly accepted the
artificial separation of mind and body of Descartes’ Enlightenment edict. Through quantum imaging in the
1990s, physicists finally saw that the whole world was connected, every nanoparticle and atom is entangled with
every other. This was not new to mātauranga Māori, which for thousands of years had preserved that
intuitive knowledge of deep entanglement in our concepts of wairua (life force, shared between all living
things) and mauri (transitory and shifting states of being). Underscoring the radical cosmological contingency
that materially binds humans to all other entities, the concept we know as mauri is expressed to a degree in
quantum physics. That is, the principle that nano level particles indeed bind together very entity in the
universe. Our work is about the entangled connections that are intuitively felt, and join humans,
nonhuman/more-than-human kin and algorithms in a constant state of becoming. Our research thinks alongside
evolutionary biologists such as Monica Gagliano and Stefano Mancuso, who argue that plants are intelligent
[2,4]. Situated agency emerges as a tangled network of augmented reality infrared vision, gestural and plant
bio-electrical data, audible through a bespoke sensing network of hardware devices and custom software.
Understanding the invisible world humans cannot see is important, therefore, to weave an understanding of
interconnection back into the process of everyday life. In our work, plants signal to one another as part of
their growth process, and we captured these signals as MIDI, an audible expression of their invisible movements.
The arhythmic and micro-temporal signals were then incorporated as a foundation to co-create a sound scape with
our plant companions. Contact/Sense explores the entangled connections that are intuitively felt between humans
and other life forms, sentient or not. Framing the eco-digital within our artistic practice is the scientific
context of evolutionary plant biology, on the one hand, and the spiritual Indigenous approach to knowledge as
beyond human, assembled in complex networks that are not only empirical but acknowledge unseen unquantifiable
forces and the insights it gives as to human behavior and thinking in relation to plants. Agential realist
accounts such as those elaborated by Barad [2008], go some way toward destabilizing, at least within posthuman
philosophy, the Humanist/Enlightenment duality, opening space for extended speculation on matter and meaning.
However, these Western systems occlude the cosmology of knowledge systems like mātauranga Māori.
Underscoring the radical cosmological contingency that materially and spiritually binds humans to all other
entities, mātauranga Māori traverses both material and spiritual realms, pointing toward a universe
that is not entirely knowable, where empirical claims to deep knowledge about the production of matter by
material forces are only partially relevant. Permeated with temporal and spatial relations that defy the "arrow
of time" and granulize past, present and future into a non-linear series of instances, mātauranga
Māori affirms the validity of techniques and methods that trace speculative knowledge.
According to plant neurobiologists Mancuso and Viola, plants are not "passive machines" for processing light,
water and food: In fact, they are "intelligent" and show this by constantly signaling to one another and to the
world, for example, giving off scents that attract or repel insects and even sharing resources, such as water,
amongst their communities [4]. Tracing the connections between quanta and nanoparticles as mauri and wairua as a
starting point for a discussion of new media art in a decolonial framework, this paper reflects upon
intersectional threads within art, physics, data and posthuman design, to weave an interconnected path between
Western and Indigenous science, philosophy and cosmology. We use our practice in media art blending
computational networks, human bodies, and plant signals to explore the metaphysical and embodied knowledge
manifest through a decolonial symbiosis or science, art and technology, and inclusive to all humans as well as
our "more-than-human" interlocutors (plants and algorithms). Concepts of symbiosis may seem attractive since
they indicate hybridity, however, as art practitioners we need to be mindful of the power relations in any
merger: so that in our writing we approach symbiosis in a decolonial sense, to alleviate the tendency for a more
powerful force to overtake a lesser one.
"More-than-human" creativity
"More-than-human" creativity is an emergent thread that reaches toward forming relations with more-than-human
kin, organic and silicon, such as plants and algorithms. Through research that bridges plant signaling with
augmented and extended reality, machine learning, and interaction design for virtual environments, this project
explores the entangled quality of human creativity in the computational networks and living ecologies. This
embeddedness, or interdependency, or entanglement, between the organic and the silicon enables new forms of
creativity, where artists’ partner with nonhuman kin via iterative acts of co-creation.
This approach destabilizes traditional principles of design, which emanate from the intentionality of the
artist or designer. A more-than-human approach allows for an open space of generative creativity where unplanned
events and phenomena can unfold, shift, and disrupt the final work. Through practice-based and traditional
research, the installation project Nga manawataki o te koiora: Biorhythms, has several interconnected
aspects. Indigenous art is conventionally seen as manifesting tradition and not technology. However this work
overturns that power relation, where embodied cultural knowledge is allowed to flow through computational
networks.
Embodied knowledge and algorithmic design
Nga manawataki o te koiora: Biorhythms, is a projection mapping and installation piece that takes you
on a journey into a computational transduction of the forest, rivers and oceans of Aotearoa/New Zealand.
Visually, an interconnected natural ecology is translated into the real-time world of audio reactive geometries
and mesh topologies. The concept was to convey the feeling of these things, without literal interpretation.
Traditional kete (woven baskets) inspire fluid movements which become pixel topologies. Animated motion made
with noise oscillators, shift from reimagined nets used to catch eel (hiinaki), to seed pods exploding from
pixel plants, such as the red pōhutukawa. The soft blue/green of kina (sea eggs) become fluffy vectors
transparently overlaid on a fluid mesh of waves. Following and modifying the tradition of naturalism and
curvilinear geometry that marks traditional Māori art, this piece visually encapsulates the feeling of the
natural world without being a literal representation that vested in Western pictorial traditions of realism.
This 20-minute piece, recorded as live audio reactive in Touch Designer, is sonically a composite of
human-nonhuman music, alternately interspersed and mixed together. From the "human" side, music consists of
three original electronic compositions by Simon Howden. From the plant side, we intersperse original recordings
of plant sonics captured in research since 2019. We consider plant to be co-composers of this work. A previous
installation, Contact/Sense was performed at the SIGGRAPH Asia Art Gallery in Brisbane 2019, and
combined plant sonics with mixed reality. Donna Haraway introduced the notion of "companion species" to describe
non-human organic life forms that we cohabit alongside in society and culture. Plants and humans have lived
alongside one another for thousands of years, in a co-dependent relationship of care and cultivation. Applying
the decolonial philosophy of mātauranga Māori (Māori epistemology), combined with posthuman
lens to art and science, our hope is that through this artwork, people will feel a little closer to the hidden
bio-electrical processes of plants, and consider plants not as a resource for extraction, but as a
‘companion species’ in a sustainable ecology.
The plant sonics were recorded during earlier live audio-visual performances with an agave attenuate,
a series of vines, several palms, and a multitude of grasses, tropical and sub-tropicals since 2018. Rewa Wright
has developed a unique mode of mixed reality performance with plants and algorithms networked with the human
body. The plant signals are essentially bio-electrical impulses, which we then assign to MIDI and apply sound
design to, so that soft wooden drums with loose skins and resonant tapping highlight the micro temporality and
asymmetry of plants, whose signals sound unstructured to the human ear. Plant rhythm is phenological, and traces
their processes such as photosynthesis and osmosis as they follow patterns of growth. Several plant
neurobiologists have noted the signals that plants emit are akin to intentional communication and sentience, and
this is now a recognized area of scientific study.¹, ²
Plant neurobiology speaks to the material reality, unraveled by quantum physics, that every particle in the
universe is connected to its nearest neighbors, and through those neighbors to all other things. Quantum imaging
has provided visual proof of this connection, as discussed extensively by philosopher Karen Barad,³ whose
concepts of "situated knowing" and "intra-action", both grounded in quantum (meta-)physics, have resonance with
Indigenous notions of deep time/space and interconnectivity. As a foundational concept, the deep interconnection
of life forces (i.e., quantum entanglement) has always been known to indigenous people, and in my culture,
Māori from Aotearoa/ New Zealand, we understand this unbroken link between vegetal, organic, silicon,
geological forms (to name just a few categories) to underpin all material and cosmological reality.
Understanding the invisible world humans cannot see is important, therefore, to weave an understanding of
interconnection back into the process of everyday life. In our work, plants signal to one another as part of
their growth process, and we captured these signals as MIDI, an audible expression of their invisible
movements. The arhythmic and micro-temporal signals were then incorporated as a foundation to co-create a sound
scape with our plant companions. Using computer art to translate emotions about place from bush into pixels,
this work explores the potential for a symbiosis of data, plants, ecology and algorithms. This work is deeply
influenced by Rewa’s cultural background as a First Nations Māori artist, from the Ngai Tawake, Te
Kaimaroke, and Te Uri o Hau hapu of Aotearoa/New Zealand. Her ancestry recognizes human and plant relations
going back thousands of years, and Rewa is committed to multi-species justice and sustainable living on Planet
Earth. Mixing the biorhythms of plants and the calls with human produced electronic music, reveals a co-creative
mesh of human and nonhuman kin, an entanglement that is fundamental. For example, in Rewa’s pepeha
(personal genealogy), she traces lineage to the maunga (mountain) called Tokatoka, the awa (river) called
Wairoa, and the moana (sea) called Kaipara. These connections are at the base of her identity and this is the
same for all Māori, since our genealogy is tied to the whenua (land). Transducing this physical connection
with the land into computational space is only ever partial. However, hints at the ways Indigenous knowledge
might be embodied as data and algorithms, nurturing a framework that advances decolonial thought and gestures
toward Indigenous futures. While quantum imaging has provided visual proof of this connection between all
vibrating atoms and matter, it is a concept that has always been known to indigenous people, and in my culture,
Māori from Aotearoa/ New Zealand, we understand this connection between vegetal, organic, to underpin all
material and cosmological reality.
References
Books
1 Paco Calvo, Monica Gagliano, Gustavo M. Souza, Anthony Trewavas, Plants are intelligent,
here’s how, Annals of Botany 125, no. 1, 2020, 11-28.
2 Stefano Mancuso, Alessandra Viola, Brilliant green: the surprising history and science of
plant intelligence, Island Press, 2015.
3 Karen Barad, Meeting the universe halfway: Quantum physics and the entanglement of matter
and meaning, duke university Press, 2007.
Authors Biographies
Dr. Rewa Wright (Ngai Tawake/Te Uri o Hau/ Te Kaimaroke) is Senior Lecturer in Film, Screen and Animation at
the Queensland University of Technology, Australia. Simon Howden is a sound designer and independent music
producer. Both humbly reside in Meanjin/Brisbane on the lands of the Turrbal and Yugara, First Nations owners.
We recognise that these lands have always been places of teaching, research and learning, and that sovereignty
was never ceded.
The Dark Side of the NFTs: The artists’ need for new systems of collaboration
İpekYeginsu
Kadir Has University Faculty of Communication, Department of Visual Communication Design
Istanbul, Turkey ipekyeginsu@khas.edu.tr
Abstract
NFTs are often imagined as the initiators of a democratic revolution in the art world, and yet their effects
are much more complicated than meets the eye. While the financial independence and the new exposure channels
they offer to the artists are undeniable, they also disturb the art world’s current equilibrium, resulting
in the emergence of a chaotic art ecosystem that makes the independent artists’ already existing
vulnerabilities even more pronounced. This paper defends the position that artists necessitate new systems of
collaboration to sidestep these negative effects, overviews some promising examples of such attempts and
discusses the potential solutions for the future.
Keywords
Art ecosystem, digital art, NFT, blockchain, crypto art, artist communities, collaboration.
In today’s art ecosystem radically reshaped by the conditions of the Covid-19 pandemic, the artists
working with digital technologies have more exposure channels available to them compared to those working with
traditional media. The former are able to exhibit their works online without facing the criticisms the artists
using conventional media often receive for choosing a digital exposure platform, and they are free from the
costs associated with transporting, insuring and installing a physical exhibition. Neither do they necessitate
the partnership of a gallery or a museum to present their works; social media platforms and NFTs allow them to
sidestep the gatekeepers in the mainstream arts and culture institutions, contributing to the emergence of a
more democratic and decentralized art world. In fact, Sarah D. McDaniel and Denny Galindo observe that
“art galleries and auction houses have been the market makers of the art world”, and “NFTs and
their marketplaces disrupt this process by allowing more artists to go straight to market and sell directly to
buyers.”¹
The NFTs’ market success empowers the artists in other ways as well. For instance, Bernard Marr draws
attention to the individual power the NFTs have bestowed upon Pplpleasr, the artist also known as Emily Yang,
who uses her NFT sales’ revenue to support the other artists and especially those of Asian descent.²
Artists without a digital background are also able to access this unfamiliar territory thanks to a new line of
service providers facilitating migration into the digital realm without being lost in technicalities. A good
example for such endeavors is Vive Arts, a company specialized in the sales and exhibition of art “in all
digital formats”, assisting its clients starting from the artwork’s creation phase.³ On the
other hand, for the digital artists wishing to develop their skills, Visa launched the “Visa Creator
Program” to “tutor a curated selection of digital artists through the NFT process, helping them
maximize their talent and use the blockchain space as both a creative tool and marketplace.”⁴
Although NFTs emerge as a convenient funding source for independent artists, they are also criticized for their
adverse effects on the environment due to their heavy energy consumption and the subsequent carbon emission. But
a more subtle phenomenon challenges the artists in other ways that might threaten their career sustainability in
the long run unless they succeed at establishing new systems of alliance and collaboration. The following
section addresses both the NFTs’ ecological impact and these relatively implicit effects in more detail.
NFTs and Chaos in the Art Ecosystem
NFTs and Climate Change
The NFTs’ impact on the environment has been a source of ardent debate since the beginning. Some artists
have been among the leading opponents of their use in the art world due to their adverse effects on the planet,
and they have also been expressing their concerns publicly. For instance, when media artist Joanie Lemercier
discovered that the sale of his NFT series consumed the same amount of energy needed to run his studio for two
years, he decided to cancel his two upcoming NFT releases.⁵ Another media artist, Memo Akten, designed a
system to assess Ethereum’s carbon footprint in numeric values and created the website
“cryp-toart.wtf” to share the most up-to-date information about the issue.⁶ Nowadays, he
continues to produce NFTs on “eco-friendly blockchains.”⁷ However, others like Eric James
Beyer claim that “criticism is exaggerated and conflates hype with the truth.” Beyer compares the
NFTs’ yearly energy consumption with that of the small mining industries, giant tech’s data storage
facilities or residential air conditioning devices, and reports that the former is relatively lower. He adds
that new measures to reduce carbon emission rates are also being implemented, such as Ethereum’s migration
from a PoW (Proof-of-Work) to a PoS (Proof-of-Stake) system, and mentions various ecofriendly trading platforms
as plausible alternatives.⁸ Raisa Bruner is another optimist mentioning the success of the Canadian
company CurrencyWorks in “turning oil waste into environmentally friendly energy that powers crypto
mining”, and the Israeli company StarkWare that was able to invent a technique to store more information
in one block unit, reducing the system’s overall energy consumption.⁹ Even Mike Winkelmann, the
artist widely known as Beeple, ensures the public that he will eventually manage to “completely offset
emissions from his NFTs by investing in renewable energy, conservation projects, or technology that sucks CO2
out of the atmosphere.”¹⁰ Efforts at building synergy between the NFT min(t)ers and renewable
energy sectors are already plenty. But whether the artists are ready to face the challenges brought about by
this new technology reshaping the art ecosystem is seldom addressed.
The NFTs’ Negative Effects on the Artists
The problems regarding the NFTs are not limited to their ecological impact; in addition to their advantages,
they have adverse effects on the artists’ careers as well. In his article on The Atlantic, Anil Dash
observes that “the current NFT market is drawing an extraordinary range of grifters and spammers”
and the artists’ works are regularly being hijacked on social media and converted to NFTs and sold by
other people without their permission, so much so that, when an app designed for blocking user groups on Twitter
was released, its leading customers were the artists “using it to block NFT spammers from hijacking their
works and monetizing them as NFTs without permission.”¹¹ The artists’ vulnerabilities in
the digital environment are also pointed out by Cass Marshall, who explains that the artists exhibiting work on
social media and worrying about hijackers ultimately decide to avoid online presence altogether. And if the
hijackers are able to generate the right type and amount of speculation around the work, they are able to obtain
much higher revenue from the same work compared to the original artist.¹² Thus, as Simon Spichak says,
it is no surprise that “much of the pushback against NFTs is coming from the artists that they were
supposed to help”, due to the unbridgeable difference between the copyright culture of the latter and
those of the newly emerging NFT fans. He adds that only 50% of the artworks in the NFT market are sold for
prices above 200 US dollars, and no correlation exists between the work’s artistic quality and financial
worth. To the contrary, the majority of the astronomic sales prices belong to works without artistic
sophistication.¹³ Andy Storey identifies yet other ways in which NFTs negatively impact the
artists’ careers, such as speculative and fluctuating prices hurting the artists’ overall market
value, the NFT platforms’ high entry costs, low respect for the works’ artistic value, and the
possibility of the work being used in contexts the artist does not wish to be associated with.¹⁴
The NFTs have also become a source of conflict for various actors in the art world. The artists continuing to
produce NFTs and the NFT platforms offering their works for sale are often pressurized by the anti-NFT artists
to stop doing so. The pressure can be so serious that, for instance, the artist portfolio platform ArtStation
had to cancel its upcoming NFT initiative due to intense online criticism by its users based on environmental
concerns.¹⁵ The platform was even compelled to release an official statement apologizing for
“all the negative emotions” that their actions caused.¹⁶ Similarly, the artists
participating in the Art Wars project declared that the project’s initiator, artist and curator Ben Moore
did not ask for their permission when producing their works’ NFT versions. Consequently, the NFT
marketplace OpenSea decided to remove the project from their inventory. And complaints do not only come from the
artists either; they target the artists instead. Hermès, a giant brand, sued the artist Mason Rothschild
for releasing “MetaBirkins” or “100 NFTs resembling the iconic Hermès Birkin
handbag.” The result of the lawsuit will also impact the collectors who have already bought items from the
collection.¹⁷ Rothschild defended himself by arguing that under the First Amendment, he had the right
to “create art based on [his] interpretations of the world around [him]” and drew an analogy between
his work and Andy Warhol’s Campbell Soup Can, but it was not sufficient for the court to dismiss the
case.¹⁸ Put differently, the NFTs emerge as an important source of chaos and conflict in the art
world, particularly in relation to issues around ecology and copyright.
In sum, the NFTs constitute an additional income source and an alternative exposure channel especially for the
independent artists, allowing them to bypass the gatekeepers in the museums, galleries and auction houses. But
they simultaneously give rise to a more chaotic art ecosystem in which artists have to choose between increased
visibility and the violation of their intellectual property rights, or producing critical work referencing
iconic images and facing corporate lawsuits. Environmental concerns associated with NFT minting and transactions
add further complexity to the issue, increasing the tension between pro-and anti-NFT artists. Moreover, due to
the medium’s accessibility to everyone which appears to be an advantage, the artists have to compete
against the NFT creators without an artistic background who prioritize financial return over artistic quality.
New Systems of Alliance and Collaboration for the Artists
As a response to the challenges at hand, artists necessitate new systems of alliance and collaboration allowing
them to navigate this chaotic art ecosystem more safely. Apparently, one of the basic strategies with fruitful
results is to join forces with other artists in the form of co-productions. In fact, fine art photographer
Gabriel Dean Roberts explains that he finds NFT collaborations with other artists and even cross-disciplinary
collaborations with musicians highly beneficial, as each side brings their own audience into the picture while
the uniqueness of the collaboration itself is a form of added value. Joint ventures also contribute to the sense
of community among NFT artists by highlighting interconnectedness.¹⁹ Such a solidarity project took
place in 2021 when a hundred artists jointly created a single NFT composed of a hundred parts that sold out
within a few minutes. According to the project’s curator Loopify, the artist list consisted of a few
established figures and “one that is new to NFTs”, and the idea emerged in response to “the
current limitations for lesser-known artists to mint and sell their own NFTs such as high gas fees, limited
understanding of the tech, and little visibility”. Moreover, to ensure that the artists were exempt from
paying additional fees, the revenue was first transferred into USD coin, from which each artist received an
equal share. ²⁰ These examples illustrate that artist alliances around NFT creation have the
potential to enhance the visibility of every individual artist while reducing the costs associated with NFT
production in comparison to doing so single-handedly, and they particularly benefit the emerging artists with
limited experience in crypto art. These, in turn, could increase diversity in the art world, ultimately
contributing to its long-term sustainability. Collective production also reduces the carbon emissions per capita
associated with individual NFT minting, which would minimize the NFTs’ negative impact on the environment
and further contribute to the sustainability of the NFT art ecosystem.
The problems related to copyright, intellectual property theft and hijacking require even more organized action
on the part of the artists and a closer collaboration with the legislative parties as well as with legal
professionals specializing in this subject. Jessica Rizzo observes that, ultimately, the authority resides in
the hands of the courts whose exemplary decisions will gradually establish the principles dominating the
creation, distribution and trading of the NFTs. According to Juliet Moringiello from Widener University, the
laws of intellectual property in the physical world are actually valid in the web3 environment as well, and she
refers to the 1996 “Declaration of the Independence of Cyberspace” by John Perry Barlow as a similar
example where his attempts at justifying the bypassing of the existing laws ultimately failed.²¹ Yet,
the legal framework is still insufficient for the artists even in the United States. For instance, on its
official website, Thienel Law LLC refers to The Visual Artists Rights Act of 1990 (VARA) as well as The Lanham
Act (Trademark Act) and the Copyright Act as only partially protective documents.²² And as the current
legal framework around NFTs is still in formation, the artists themselves should be more directly involved in
the establishment of these principles. One way in which they have already begun to make a positive difference is
to hold the minting and sales platforms accountable for regulating their clients’ activities with regards
to intellectual property. A good example for such systems is provided by OpenSea; in 2022, the platform
implemented an anti-plagiarism mechanism consisting of both algorithmic detection and review by actual
humans.²³ OpenSea also enabled a stolen art reporting system thanks to which the original artist can
file a takedown request against the plagiarist, although the system does not guarantee that the request will be
accepted.²⁴ Another solution could be to have the NFT market platforms involve the artists themselves
in the artwork curation and elimination processes, so that unethical behavior by a member of the community is
directly sanctioned by that very community, e.g., by banning the user from the platform following the receipt of
a certain number of complaints. In short, collective action on the part of the artists can be effective in
pushing all NFT platforms to take the necessary measures, which would also accelerate the establishment of a
basic legal, or at least ethical framework for NFT art creation and transaction around the world, and would help
reduce the chaos in the NFT art ecosystem.
As illustrated by the MetaBirkin case, brands are another crucial party in the legal disputes over NFTs. One of
the most difficult dilemmas emerges around the artists’ right to borrow iconic imagery from popular
culture vs. the brands’ urge to protect their intellectual property. This is one of the most sensitive
territories for the future of artistic creativity. If the courts decide to overprotect the brands, they will
also have acknowledged that at least half of the seminal artworks from the 20th century are copyright
violations. Such a verdict would condemn all the artistic practices involving appropriation, collage and
pastiche, and it would do so because it “can” thanks to the digital technology facilitating
surveillance and tracking. This, in turn, would seriously deprive the NFT world from the ability to produce art
with a political or critical stance, almost “sterilizing” it into a corporate-friendly environment.
The dilemma is further complicated by the fact that brands complain about the NFT artists in the same way that
artists complain about their online hijackers, and technically differentiating the two contexts is not always
easy. Introducing itself as “the first blockchain ecosystem built by the art community, for the art
community,” Arcual is a promising initiative in this regard. Jointly founded by Luma Foundation, Art
Basel’s parent company MCH Group and BCG Digital Ventures, the platform aims at creating a clean NFT
trading environment and offering the artists additional revenue from secondary sales by using “smart
contracts with embedded resale terms.”²⁵ Again, the main dilemma presents itself as the
trade-off between creative freedom and financial/legal safety, for companies founding such initiatives also have
vested interests in the corporate world especially in the form of sponsorships, and whether in the case of a
dispute between an artist and a brand they would come in defense of the former instead of the latter leaves a
big question mark.
Conclusions and Ideas for Future Research
As illustrated by the overview above, the NFT market’s global outreach and independent environment is
extremely beneficial for the artists, but it also intensifies their existing vulnerabilities especially in terms
of intellectual property rights by destabilizing the art ecosystem as well as contributing to the deterioration
of the ecological one. As the ethical framework of this newly emerging territory has not yet fully developed,
the artists might still seize the opportunity to directly shape its future if they manage to act in solidarity,
hold the NFT trading platforms accountable and insist on directly participating in the ongoing legislative
processes. Further research is necessary to deepen our understanding of the dynamics dominating the NFT art
world, the existing legislation at the national, regional and/or global level and how artist communities do and
can operate in the face of the existing challenges. Most importantly, further study is necessary to explore how
broader artist communities can be mobilized for collective action.
References
1 Sarah D. McDaniel, Denny Galindo, “Democratizing Art: How NFTs Are Reshaping the Art
World”, Morgan Stanley website, accessed December 1, 2022,
https://www.morganstanley.com/articles/nft-art-market-nft-collectibles#:~:text=NFTs%20and%20their%20marketplaces%20disrupt,galleries%20and%20artists%20get%20paid
2 Bernard Marr, “Web3, NFTs, And The Future Of Art”, Forbes, August 19,
2022, accessed December 2, 2022,
https://www.forbes.com/sites/bernardmarr/2022/08/19/web3-nfts-and-the-future-of-art/?sh=6731712f1e05
3 Vive Arts, “Combining industry-leading metaverse technologies with bespoke services, Vive
Arts marketplace showcases and sells art in all digital formats”, Vive Arts website, accessed December
1, 2022, https://www.vivearts.com/about-platform
4 Mark Hunter, “Visa Launches NFT Artist Support Program”, FullyCrypto,
October 14, 2021, accessed December 1, 2022, https://fullycrypto.com/visa-launches-nft-artist-support-program
5 Gregory Barber, “NFTs Are Hot. So Is Their Effect on the Earth’s Climate”,
Wired, March 6, 2021, accessed November 30, 2022,
https://www.wired.com/story/nfts-hot-effect-earth-climate
6 Memo Akten, “The Unreasonable Ecological Cost of #CryptoArt (Part 1)”,
Medium, December 14, 2020 (updated December 2021), accessed November 30, 2022,
https://memoakten.medium.com/the-unreasonable-ecological-cost-of-cryptoart-2221d3eb2053
7 Memo Akten, “NFT”, www.memo.tv, accessed June 16, 2023,
https://www.memo.tv/works/nft/
8 Eric James Beyer, “NFTs and the Environment: Why the Anger Is Unjustified”,
nftnow, September 16, 2022, accessed November 30, 2022,
9 Raisa Bruner, “Environmental Concerns Have Cast Doubt on NFTs—But That’s
Changing”, Time, November 18, 2021, accessed November 30, 2022,
https://time.com/6120237/nfts-environmental-impact/
10 Justine Calma, “The climate controversy swirling around NFTs”, The Verge,
May 15, 2021, accessed November 29, 2022,
https://www.theverge.com/2021/3/15/22328203/nft-cryptoart-ethereum-blockchain-climate-change
11 Anil Dash, “NFTs Weren’t Supposed to End Like This”, The Atlantic,
April 2, 2021, accessed December 2, 2022,
https://www.theatlantic.com/ideas/archive/2021/04/nfts-werent-supposed-end-like/618488/
12, 15 Cass Marshall, “NFTs are generating huge paydays for some artists, others feel under
siege”, Polygon, March 12, 2021, accessed December 1, 2022,
https://www.polygon.com/22327806/nft-artists-online-theft-non-fungible-token
13 Simon Spichak, “Independent Artists Say NFTs Are the Bane of Their Existence”,
Futurism, January 24, 2022, accessed December 2, 2022, https://futurism.com/why-artists-hate-nfts
14 Andy Storey, “9 Reasons Why NFTs Are Bad For Artists”, Postergrind, May 7,
2022, accessed December 2, 2022,
https://postergrind.com/9-reasons-why-nfts-are-bad-for-art-ists/#:~:text=Artists%20claim%20that%20their%20art-work,is%20a%20lot%20of%20effort
16 ArtStation Team, “A Statement from ArtStation”, ArtStation, March 8, 2021,
accessed December 1, 2022, https://magazine.artstation.com/2021/03/a-statement-from-artstation
17 Simon Fitzpatrick, Rosie Adcock and Sophie Mellor, “February NFT Litigation Roundup: Art
Wars, Hermes "MetaBirkins", and more...”, Lexology, February 23, 2022, accessed December 3,
2022, https://www.lexology.com/library/detail.aspx?g=666c41a0-7815-4d21-a0f6-6da07ec1388e
18 Andrew Rossow, “The Hermès Lawsuit May Dictate the Future of NFTs”,
nftnow, May 19, 2022, accessed December 4, 2022,
https://nftnow.com/guides/how-the-hermes-lawsuit-could-deter-mine-the-future-of-trademark-rights-in-nfts
19 Gabriel Roberts, “NFT Collabs: Why You Should Work with Other Artists”, YouTube
video, 3:32. April 5, 2021, https://www.youtube.com/watch?v=AeW3dm8xU4Q
20 Joshua Mapperson, “100-artist NFT collaboration sells out in minutes, increases 7X in
price in 24 hours”, Coin Telegraph, March 4, 2021, accessed December 3, 2022,
https://cointelegraph.com/news/100-artist-nft-collaboration-sells-out-in-minutes-increases-7x-in-price-in-24-hours
21 Jessica Rizzo, “The Future of NFTs Lies with the Courts”, Wired, April 3,
2022, accessed December 4, 2022, https://www.wired.com/story/nfts-cryptocurrency-law-copyright
22 Thienel Law LLC, “NFTs and Artists: Understanding Intellectual Property Laws in the
Metaverse”, Thienel Law LLC web- site, July 1, 2022, accessed December 5, 2022,
https://www.thienel-law.com/blog/2022/6/23/nfts-and-artists-un-derstanding-intellectual-property-laws-in-the-metaverse
23 Protos Staff, “Humans and computers to fight NFT plagiarism on OpenSea”,
Protos, May 12, 2022, accessed June 16, 2023,
https://protos.com/humans-and-computers-to-fight-nft-plagiarism-on-opensea/
24 Jack Morse, “How to report plagiarized NFTs as stolen art”, Mashable,
April 13, 2022, accessed June 16, 2023, https://mashable.com/article/how-to-report-plagiarized-nft-stolen-art
25 Arcual Art website, accessed December 6, 2022, https://www.arcual.art
STAND BY/ME
YuZhang
STUDIO Ü Eindhoven, the Netherlands yuzhang.nl@gmail.com
Abstract
This paper describes initial intention, concept, concerns in design process, production, and technical details
of an artwork named “STAND BY/ME”. STAND BY/ME is an interactive installation that uses
machine-learning models to generate speeches for Xi Jinping and Donald J. Trump, and uses randomness to build
virtual conversations for the political spectrum, while connecting the information flow with the lived reality
of the everyday. Visitors are exposed to narratives of both digital communism and digital capitalism that is
randomly controlled. Next to the visual flow of randomly generated speeches, the work involves seemingly
mundane, yet “super-charged” electrical household items, i.e., power sockets, as the actuality of
human-technology confrontation. This work (1) allows visitors to feel like they are on “stand by”
and (2) triggers questions about how technologism impacts individuals' views and information consumption, while
(3) people face randomly generated political speech as a quiet, mundane confrontation. In sum, visitors are
invited to be bystanders of randomized political speech, visualized through streams of text ad made tangible
through everyday technology.
Keywords
Information representation, information flow, digitals and physicals, social media, technologism, politicism.
With the advent of smart and learning systems, connected technology is no more a simple luminous torch to shine
upon our wishes or serve our needs. It senses a multitude of signals from our environment, and shapes our
thoughts, behavior and social interactions. We find ourselves at the moment when technologism becomes an almost
unavoidable aspect of the everyday. Most often our true realization is overshadowed by technological wonder or
even deeply influenced by a form of techno-mysticism. When we place the finger on the screen and swipe it, an
“informative” world starts and soon it is filled with bits and glimpses of ever-faster news cycles.
As individuals, we consume the abundance of news that produced to capture our attention, move us emotionally and
sometimes even take action. Over time, we reach a state of being alert and at the same time wholly incapable of
action, especially when consuming news about politics. The initial intention of the STAND BY/ME installation
(Figure 1) is to synthesize the experience of “stand by,” in the context of technologism and
politicism, where each individual can view cycles of randomized and virtual information from two political
“speakers” as a bystander. “Stand by” expresses a human inability to act socially, as an
inhibition of the mind, a perversion of mindfulness. It is a deeply tiring state. The deeper intention behind
this art scene is to state: when an individual is incapable of action, digital data still changes its meaning
and form just depending on its medium of distribution, communication and interface.
“Stand by” and STAND BY/ME
STAND BY/ME focuses on the concept of “stand by” and transforms feelings of confusion,
in-betweenness and randomness into an immersive audio-visual experience. This experience is designed to trigger
visitors’ reaction to human and technological expressions of “stand by”, and visitors’
feeling when encountering a situation gripped by a general sense of “stand by.”
The physical part of this installation is composed of 30 linked power sockets (Figure 2), which are
technologically extended and enhanced with light and sound actuation. Apart from the layer of connected physical
sockets, the installation incorporates all public speeches by Chinese president Xi Jinping since
2014¹, ², ³ and thousands of tweets by U.S. former president Donald J.
Trump since 2016⁴ until January 2021. A recurrent neural network (RNN) was trained with the speeches and
tweet corpus, and we fine-tuned the parameter space to obtain synthetic speeches that express the current
acceleration of digital communism and digital capitalism. The two “speakers” are then visually
juxtaposed with a simulation of randomized draws of numbers (balls) from the National Lottery.⁵ The
lottery serves as the metaphorical random actor in this installation: conceptual and technical randomness
controls relations between recurring concepts in each politician’s generated speeches. The content and the
processing of approaching information data are displayed as a digital information interface (Figure 3).
This work projects the patterns of information flow from the machine-generated speeches first, and connects
with the lived reality of the everyday. The ordinary power sockets, “super-charged” household items,
visualize a connected data system that enacts the randomness in the actuality of human-technology confrontation.
Throughout the design process, this work touches on four symbolic and logic assumptions in the concept
“stand by.”
Two Politicians
As two “speakers,” Mr. Xi and Mr. Trump have different ways of approaching the public, quite
oppositely.⁶ In 2020, we have read a lot about “truth” by analyzing and interpreting the
information and actions from Mr. Xi and Mr. Trump. The citizens have been (still) confused about information and
have perceived them as a strongly regional particularity. STAND BY/ME uses a format of public stances and, at
the same time, is filled with random content—formal posing as a facade. The concept of STAND BY/ME was
accomplished around March of 2020. At the moment of writing this paper, the new conflicts in the world started
to develop —presidential election result for the 2020 US election became a turning point for American
foreign policy regarding Beijing, followed by Xi has secured a precedent-breaking third term as the Communist
Party’s leader and Donald Trump announced a White House bid for 2024. This work is filled with future
projections including the efforts to establish dialogues between different cultures and regions. The result of
such efforts seems very “virtual,” either in this work or in the reality.
National Lottery
National lottery symbolizes the daydream of picking easy money, yet participating in a random gamble. The
format of National lottery used in this work contains three main contents: jackpot size, six ball numbers and
one bonus ball number. STAND BY/ME uses this format but creates new feeds that are machine-generated random
numbers.
In STAND BY/ME, the lottery is a metaphor for randomness and offers each visitor only a passive role who are
not able to take action in the process—intelligent agents submitting to luck.
Power Socket
The power socket in its most ordinary form is a distribution hub for electricity. Here, its function is
transcended: it is a physical connection hub which is available all the time to connect devices that can send
and receive all information packages. Power sockets often embody the most common notion of indicating
“stand by,” a glowing LED. The use of power sockets in this work metaphorically refers to the action
“to be on charge” and similarly as an active participant in propagating information and visualizes
the information flow in the audio-visual patterns. Through multiple power sockets, STAND BY/ME embodies a sense
of “something somehow happening,” and visitors are incapable of actions.
Interaction
Interaction in this installation went through different stages: from a means to engage visitors in exploring
and discovering the emptiness behind the façade to the final implementation where interaction is
deliberately withheld from the visitor. The interaction becomes that visitors only remain passive in the
installation to simplify the dynamics of light and sound (reacting only to the data processing of the National
Lottery). By observing the data processing instead of being part of it, visitors are drawn into the processes of
how information spreads or hits a levee or how light and sound trigger the translation of the digital speech
data. As visitors realize that they are bystanders, in the back of their minds, they will feel the pressure and
obligation to act, without being empowered.
Technical Details
In this installation, a central single-board computer (SBC) runs a Processing sketch that (1) displays the
information interface with virtual speeches and conversations from two politicians and projects the results as a
live “dashboard” of speeches, highlights and lottery drawings (Figure 3), and (2) translates into 30
linked power sockets that are extended with light and sound output (Figure 2). Each of the 30 power extension
sockets is internally altered, while keeping its original appearance. Inside each socket, WIFI-connected
electronics (based on ESP32 boards) allow for decentralized control of each two LED lights and one speaker
module. The installation generates its own data stochastically modeled after patterns of information flows from
social networks. The patterns are mapped and distributed to all sockets via a message bus running on the SBC.
The dynamics of light and sound of each altered socket are driven by each sockets’ internal programming
and designed to be responsive to the data in the system.⁷ Let’s dive in a bit deeper. The
information display in figure 3 shows three columns. The rightmost column is the data display from National
Lottery. It contains four main parts—(1) data source and its introduction, (2) real-time lottery
information (jackpot size, six ball numbers and one bonus ball number) in technical randomness, (3) the latest
coming six ball numbers, (4) the keywords from speeches by each “speaker” that generated from the
latest coming bonus number. In the same figure 3, the two “speakers” are shown in the left and
middle columns with each four parts of content: (1) data source introduction, (2) real-time RNN generated
speeches, (3) up to six highlighted keywords from speeches selected by the latest draw from the National
Lottery, and (4) the statement selected from the speeches with shared keywords selected by the latest draw of
the bonus ball. The speeches differ in content and important repeating keywords, which are extracted in a ranked
list per speaker. These lists are used to draw keywords from by means of random numbers from the Lottery
draws.⁸
Conclusion
STAND BY/ME creates an immersive experience set up as interplay between humans, machine and artificial entitles
in a cognitively overwhelming way using text, light and sound. Although STAND BY/ME seems calm in general, it
puts visitors in an almost passive position when facing the complexity of the machine-generated progress. At the
same time, visitors soon discover the need to process each incoming scenario of an entirely modified and virtual
information for both digital communism and digital capitalism under the control of gambling randomness. The
installation unpacks the concept “stand by” in a collaboration of data sources, data system,
technologies, multiple electrical household items, and the digital interface. The design process of this work
dives into the process of refining audiovisual complexity with special attention to the details of information
representations, metaphorical or not. This work provides the prompts for different perceptions and enables
visitors to have a felt experience of the connection, distraction and confusion when digital data changes its
meaning and form at every point, depending on its medium and the current “mood” of randomness.
Acknowledgments
The grant of work was awarded by ARGEkultur in the frame of DIGITAL SPRING FESTIVAL. STAND BY/ME was shown in
the group exhibition “POLATECH 1” in the period of 27th of August until 6th of September, 2020 at
Albert van Abbehuis, Eindhoven. The work was also invited and exhibited at Athens Digital Arts Festival 2022.
This installation was built with much inspiration and support from Mathias Funk from the Industrial Design
department at Technology University of Eindhoven.
References
1 China.org.cn, Bilingual Texts, accessed October 30, 2022,
http://www.china.org.cn/english/china_key_words/node_7216079.html
2 Mission of the People’s Republic of China to the European Union, Speeches (Resources),
accessed October 30, 2022, http://www.chinamission.be/eng/zywj/ZYYJ/
3 Ministry of Foreign Affairs of the People’s Republic of China, Speeches (Policies and
Activities), accessed October 30, 2022, https://www.fmprc.gov.cn/mfa_eng/wjdt_665385/zyjh_665391/
4 Donald J. Trump, Tweet (text), accessed December 30, 2020, https://twitter.com/realdonaldtrump
5 THE NATIONAL LOTTERY, Lotto results, accessed October 30, 2022,
https://www.nationallottery.co.uk/results/lotto/drawhistory
6 Chris Buckley, “Trump’s and Xi’s Differences Magnify Uncertainties Between
U.S. and China (2016),” accessed October 30, 2022,
https://www.nytimes.com/2016/12/19/world/asia/china-donald-trump-power.html
Dr. Yu Zhang has a background in fine arts and design. In her Ph.D. research she investigates the theory and
artistic practice of interactive technologies for public, large-scale installations. Over the past years, she
has designed and researched interactive systems that respond to everyday phenomena, environmental concerns,
child-system interaction, online collaboration platforms, and uncertainty in data visualization. Yu has
participated in various international Art Residencies of the last years, and her work has been exhibited at
galleries, museums, and festivals world-wide. Besides, Yu’s teaching experience covers a broad range
from traditional classrooms and workshops to designled project-based learning activities. Her book
“Coding Art,” co-authored with Mathias Funk, was published by Apress/Springer in 2021.
https://yuzhang.nl/
Centre for Art as Forum, Department of Arts and Cultural Studies, University of Copenhagen
Copenhagen, Denmark tanyaravnag@gmail.com
Abstract
The world of art has always been occupied with art’s temporal relations to the future. In the current
artistic landscape, we see a wealth of exhibition themes and titles concerned with ‘the future’ in
responses to the dominant narratives of a contemporary technological world driven by algorithmic systems and
prediction models. This momentous future-orientation is my cue to rethink art’s relation to the future, by
zooming in on its temporal modes of existence. With a take-off in the notion of art as “time-based,”
as conceptually based in the time, duration, and/or the function of a medium and the experience it mediates, I
propose a different intratemporal mode of existence for art. This concerns how art co-exists with,
evolves through, and co-produces temporal relations in between humans and technology. This proposal of an
intratemporal perspective on art might contribute to further investigations into art epistemologies in which art
becomes a part of larger narratives in which human beings and communities co-evolve—and have always
co-evolved—with technics. It might offer inroads to study art on its new paths of exploration in
collaboration with science and technology and when art is occupied with the very making of the future through
participation in innovation projects.
My temporal investigation in this paper takes off in the catalog Alchemists of the Future published
for the Ars Electronica Future Lab’s 25 years anniversary in 2021. In the concluding chapter,
“Perspectives,” we can read about how the Future Lab’s activities of visionary prototypes and
innovative collaborations between art and science in 1996 were initiated to contribute with future narratives to
address urgently needed paradigm changes.¹ The visions expressed in the Future Lab
catalog, about art’s involvement in our greater societal narratives of technological change, echo the
bringing together of art, science, and technology with the conception of the New York-based organization
EAT—Experiments in Art and Technology—in 1967, which was founded by engineers Billy Klüver and
Fred Waldhauer and artists Robert Rauschenberg and Robert Whitman. The visions of creative and experimental
research processes between artists and engineers for bringing artists closer to the materials of technology and
more in touch with the forces shaping contemporary society, the use of projection and new communications
technology to achieve this, which entailed the exploration of new roles for art in the changing
‘technological environment’ of the late 1960s/early 1970s, and the migration of these practices from
an art to non-art contexts,² altogether paved an explorational path for art’s evolvement through
changing relations to the future.
Today, as we find when searching through numerous recent titles of exhibitions, knowledge forums, and events of
art, (1) the future orientation has saturated the broader field of art. This future-orientation
should interest us as more than a thematic trend.
The legacy of E.A.T., and the catalyzation of ideas of collaboration between art, science, and technology
through the Future Lab, among many more initiatives, informs a fast-growing discourse in art whereby the art is
treated, funded, and appropriated as a catalyst for change. For example, when art migrates into cultures and
contexts of technological innovation; when artists are invited into residencies, technology and science labs of
corporate technology companies; or, where art becomes a protagonist in major creative funding schemes and
innovation programs and is granted support as a catalyst for, for example, industrial innovation, urban
development, or human rights. For one example amongst many, the call “Art-driven use experiments and
design” under the Horizon Europe Framework, which explicitly allocates a strategic role for art in
technological innovation culture. These movements in art, whereby art has gained new roles in strategic projects
of future-oriented and future-shaping technological innovation, require new approaches to grasp and assess
art’s modes of existence, which I propose that we understand through its relations to the future as an
epistemological and methodological compass.
My inquiry is guided by the following line of questions: Why, in the context of our contemporary technological
environment, is the orientation towards the future in art so momentous? What characterizes art’s relation
to the future in our current technological environment? If the occupation with the future in art concerns a
temporal orientation towards how everyday lives, cultures and societies will or might evolve with technology,
then how does art participate in the temporal processes that will bring us there? Why does art’s relation
to the future matter to the roles that art pursues and gains within technological innovation—as a locus
for human symbiotic imagination (about the future) and our technocultural making of it?
My overall suggestion is that we need to grant more attention to art’s temporal modes of existence as
simultaneously a matter of object functionality and environment, human and intersubjective experience,
technocultural context, and cultural evolution. With a point of departure in the conception of art as
“time-based,” I engage an alternative, intratemporal mode of existence for art, with which
I understand art to be a part of a larger temporal complex: art is not based in time but existing through
intratemporal infrastructures and relations with its contemporary technological environment, which in our
current age is characterized by and evolving through data-driven algorithmic processes. I unfold the
intratemporal perspective on art through three temporal dimensions—object temporality, worldly
temporality, and deep temporality—that relate art to epistemologies on how human experience changes with
technological culture.
Art and temporality - beyond “time-based” media
When art is described and categorized in a temporal perspective, it is commonly referred to as
“time-based.” My claim in what follows is, however, that this temporal conception of art and the
epistemological framework that it engages is insufficient to grasp art’s behavioural modes of existence
and interdisciplinary evolvement today.
The conception of art as “time-based” is broadly used by museums with reference to artworks that
rely on technology, such as video, film, audio, slide, installation artworks, as well as artworks that function
only for the duration of their time on display, like computer-based and mechanical works of art. The conception
of time-based art ties time to the expressive and functional qualities of the medium. Time-based art is
conceptually rooted in “time-based media,” a term coined by museum conservators for durational works
of art that unfold over a period of time. It is used widely by art institutions to describe art that is
‘dependent on technology and has a durational dimension’ (Tate), that ‘unfold to the viewer
over time’ (Guggenheim), and that are ‘dependent on time, duration, or function’ (National
Gallery of Australia). Time-based media has a run-time enabled by the form or medium that limits and contains
the experience. The medium enables the inscription of the spectator in different experiences of time. By looking
at art as “time- based,” we focus on how the art facilitates meetings between different durations.
For example, between the durations of human experience and the durations of a rationalized society. This
understanding is fueled by a broad theoretical interest in temporary multiplicity in the writings of among
others Henri Bergson, Alfred North Whitehead, Gilles Deleuze, Michel Foucault, and Michel de Certeau.
Christine Ross’ examination on art and temporality in The Past Is The Present, It’s The Future
Too exemplifies this understanding of art as facilitating meetings between durations. She describes
various ‘durational’ temporal strategies in art as aesthetic counter reactions to the forwardness of
the modern era. These are temporal strategies of, for example, endlessness, ephemerality, repetition, real-time,
contingency, randomness, slowmotion, condensation, acceleration, extension, abbreviation, speeding up,
hesitation, disruption, fissuration, extendibility, and interminability—all temporal strategies for
suspending linear conceptions of time that confirm one universal temporal logic.³ Such temporal strategies
of suspension evoke a tendency emerging in contemporary art of the 1960s, which is described by Pamela M. Lee in
terms of “chronophobia”—a sense of unease or maybe even rebellion in art against temporal
societal narratives that dominated during the middle of the 20th century and which translated into a critical
consciousness in artistic expressions of performance, conceptual art, sound art, installation practices and land
art.⁴ The art of the 1960s that both Ross and Lee write about reacted against a relation between
temporality and historicity, namely one dominant narrative about technological progress that characterizes
Western modernity, which celebrated technological transformation, automatization, acceleration and
standardization. The dominant narrative reflects a universal conception of time as linear, structured around
past, present and future, and organized based on classical physics’ ideas about absolute mathematical time
and ground principles of natural science about relativity. In this narrative, time and space are compressed by
technological and mechanical processes—what David Harvey has named “time-space compression”
which refers to how global communications technologies and information economy compress barriers and distances,
which is a function of late capitalism.⁵ This global, temporal narrative is structured around a singular
temporal scale characterized by rules of regulation, discipline, speed, effectivity, immediacy and
progression—as Jonathan Crary describes in the book 24/7.⁶
The time-based conception rests on a philosophical notion rooted in the ideas of Plato and a substantivist and
absolutist conception of time, treating time as an empty container with rules and logics, that is, temporal
rules and logics that are ready for art to critically engage with. We recognize this conception of contained
time when art is accounted for as an aesthetic, conceptual, critical manifestation capable of presenting and
representing alternative temporal modes to those driving capitalism by which to inscribe people into different
experiences of time. This temporal containment, however, delimits art’s relation to the future as
representational or reflective material that eventually becomes confirmative of the future narrative that it
speaks to.
I would like to propose a different temporal condition for art. Because, although art is situated in a specific
temporal slot and has a particular duration, and although it might depend on the phone or a mobile device that
enables specific temporal qualities of the experience, the work is not delimited to a temporal capsule. It is
not delimited to exist “based in time,” as if in a form of a temporal container that we can
individually step into for a direct experience with represented image or concept. The time-based conception
relies on a direct experience between the human and the artwork. This does not correspond to the ways in which
we experience and exist with temporalities through the ways in which most of us engage with technology today.
Nor does the time-based conception account for the human-perceptual and technocultural effects of these temporal
experiences. Time is articulated in technical systems but only in connection with human engagement with
technics, as we learned from the writings of Gilbert Simondon.⁷
In the following, I will propose the contours of an alternative temporal conception of art to that of
time-based; one that considers an intratemporal mode of art’s existence. This involves the conception of
time as something that the art is relationally entangled with, through which it evolves, and which the art
contributes to generating.
Art’s Intratemporal Mode of Existence
Art’s intratemporal relation to the future today bears traces of future-orientations in art of the past.
Along with the ongoing critical discourse in art continuing the critical occupation with the forwardness of the
modern era that Ross locates in art of the 1960s, we recognize a trajectory from the futurist art movement of
the early twentieth century that sought to capture in art the dynamism, energy and movement of the modern world
and modern life. Preceding future-oriented movements unfolded in effect and response to a technological
environment before the internet, social media networks, and data-driven distributions and accumulation of
registrations of our behavior. Today, however, the technological environment conditions a different
intersubjective condition than that of the 1960s.
An intratemporal perspective on art might immediately evoke Martin Heidegger’s understanding of phenomena
and objects in terms of temporal relations rather than substance, in Being and Time. (2)
⁸ Heidegger contrasts intratemporality with authentic temporality, seeing intratemporality as an
existential structure to Dasein determined by calculation and measuring instruments. My use of the term
“intratemporality” takes a different reference, in Yuk Hui’s connection of intersubjectivity
(subject-context relation) and interobjectivity (object-milieu-relation) in the term. In Hui’s theory,
intratemporality is a dimension of changing f temporal relations between objects and neurosensory evolution that
happens through our networked and synchronous co-evolvement. ⁹ In my adaptation of this understanding of
intratemporality to this inquiry on art, I consider art as temporally related with temporalities of technologies
and technological cultures well beyond the medium, the art experience, and the discourse of the art environment.
In the following, I will draw some perspectives on how intersubjectivity, as a matter of temporal relations, is
conditioned by the temporalities of digital objects and worldly connectivity as well as by deep temporalities of
our cognitive and cultural heritage from technocultural pasts. These temporal dimensions combine in art’s
intratemporal mode of existence and tie art to the concept of the future in new ways.
Object temporality: In Hui’s account of the conditions of digital objects and extension on
Heidegger’s notion of intratemporality in this regard, he notes how data-driven temporal processes mediate
between intersubjective and inter-objective relations and influence temporal experiences in our everyday lives.
The ways in which things are quickly shared, behavior and ideas are quickly adopted, and experiences are
synchronized, effects an organization of consciousness about how things are temporally related to each other.
This reorganization of consciousness with object temporality is what N. Katherine Hayles addresses in How We
Think: Digital Media and Contemporary Technogenesis.¹⁰ Hayles writes about how the media
interface (e.g., the screen), for example, might seem like it correlates directly to human modes of sensory
experience and cognitive processing while it only indirectly correlates to these modes of experience, since it
involves technical operations to which we lack a direct access. This is because different time scales of human
cognition and machine cognition intermesh. As computational processes occur at time frames that are below the
threshold constitutive of human perceptual experience and introduce levels of operationality that impact our
experience but do not have any perceptual correlate, we are not conscious of their consequences to our actions.
The mutual interference between temporalities of machinic systems and human-temporal functioning of
consciousness means that our creation of (abstractions, forms, content, systems, meanings) is not rooted in a
direct human relation with what we create but depends on unconscious processes. This unconscious aspect of
perception with object temporality connects us to worldly temporality.
Worldly temporality: Global effects—from economical dynamics at a macro level to machinic
operations at a micro level—reach us through object temporality. The unconscious cognitive processes at
work in our engagement with object temporality not only concern an engagement with digital objects (in and
beyond art). They condition intersubjective experience as object temporality relates to the temporalities of
networked media technologies, which is entangled with human experience. As Mark Hansen notes in Feedforward:
On The Future Of Twenty-First-Century Media, with digital media, the external world has become a part of
individual experience, while experience has become externalized and environmentalized in contemporary forms of
mediation.¹¹ While we physically exist in the phenomenal world, our thinking, behavior, and the
effects of our actions are also of a worldly context, conditioned by mediating factors of different
temporalities affected by environmental and global connectivity, and experienced across timespaces.
What this meshing of human and machine temporality results in are operational processes that function as a kind
of technical “memory,” which becomes a cultural support structure, and which affects how perception
and intersubjective imagination are at work. This technical support structure evolves from a long process of
evolutionary adaptation of technical tendencies and their logics, whereby art is intratemporally entangled with
a sense of deep temporality.
Deep temporality: Art’s experiences can amplify and resonate through volumes of people, connect
us to our ancient past and memories of cultural rhythms, rituals, and practices, and throw us into
uncertain futures. An intratemporal dimension of deep temporality links art to cultural patterns, which have
shaped the ways in which we use and develop tools and technology since our human origin and the perceptual
habits we have evolved and enact when experiencing something, including art. Collective memories, cultural
programs and imaginations have been transmitted via habit and repetition through communities and historical
epochs. With reference to technoanthropological ideas from the philosophical writings on human technogenesis of
Bernard Stiegler¹² based on the anthropology of André Leroi-Gourhan that roots human
co-evolvement with technics in the origin of human civilizations,¹³ we can consider how art has a part
in the shaping of cultural memory, symbols and rituals that we have adapted from ancient pasts. These have
formed through civilizations, cultures and generations to manifest in the cultural codes, meanings and logics we
navigate by today. These cultural adaptations of technocultural aesthetics and behavior inform how human
cognition meets machinic operations today.
With this intratemporal dimension of deep temporality, I wish to emphasize a technicity in the art as having a
function with regards to our cultural evolution with technics. Art, as a human aesthetic expression, has evolved
with evolutional adaptations that carry the past into the present—and entwine with the
future—through technological tools and the cultural and cognitive memory structures they engage. From
studies on ancient human pasts, we know that art, and the technicity with which it operates, in the ancient
shapes of rituals, ornamentation, craftwork traces of human gatherings, and more, has taken on various roles as
a cultural transmitter and as an aesthetic mechanism of societal organization. For example, as a kind of
mediator of societal imaginaries; a vehicle for intelligence, memory, language, forms of expression and pattern
recognition to travel through generations; as both depiction and facilitation of rituals (practical, cultural
and spiritual); as a connector of human beings to their past and origin and a basis for collective consciousness
and emotional intelligence; as a connector of humans to the materials and environments of our world and its
ecosystems; as embodying conceptions and philosophies of science as a foundation for furthering civilizations,
among many others. These are observations from my research on various intersubjective functions art has had in
ancient societies.¹⁴
Intratemporality concerns temporalities that are within us, among us, beyond us and preceding us—and
which entangle in our tenement towards the future. The intratemporal mode of art’s existence therefore
cues a re-examination of art’s relation to the future.
Art’s intratemporal relation to the future
The current future orientation in art is not either confirming or resisting a utopian desire. What should
interest us are also not the future destinations that art offers or is used to test, project, or speculate upon,
which are ideas that inform scenario-based design and conceptual attention to "possible futures." It is also not
the critical comment on future-driven regimes. Future-oriented art of today does more than make room for
reimagining the future.
My proposal here is that art’s intratemporal relation to the future concerns how art and its experiences
are entangled with temporalities that relate us to our everyday engagement with (digital) technologies, with the
flows and dynamics of worldly (data) processes, and which engage cultural adaptations and intersubjective
evolvement through intuitions and perceptions that precede our experience today. This perspective ties
art’s relation to the future to contexts of technological cultures beyond that of Western rationalization.
The intratemporal perspective on art concerns what kinds of temporalities the art engages and connects in our
bodies, objects and surroundings, and in which ways (by the use of which techniques and aesthetic means). It
concerns how art intervenes in our experience of those temporalities. This perspective writes art into a larger
narrative in which human experience is changing with technology and in which art has always played a role in the
ways in which human beings have co-evolved with technics.
This reconception of time-based art can help us to grasp the new routes and roles art pursues through temporal
engagements with technological innovation culture. When art collaborates with science and technology in the
domain of innovation, it not only envisions, problematizes, or proposes but also co-produces our
futures. This involves a change in perspective, from how art represents and responds to the future, to how art
has a constitutive relationship to the future. This is because art engages with human intuitions, desires, and
aspirations from where our futures emerge. Art becomes a part of larger intratemporal processes of human
co-existence and co-evolvement with technology. This calls for further examination of futurity in art, how the
art’s techniques and experience is temporally entangled with future-driven systems and processes of human
co-evolvement with technics.
(1) Some of the many recent future-oriented exhibition contexts. I’ve come across in my
research, which exhaust and understate my argument that the attention to and conceptualization of the future in
art is momentous: Possibles (ISEA2022); Futures Implied (Media Architecture Biennale 2020); Writing the History
of the Future (ZKM – Center for Art and Media Karlsruhe 2019); Futures (Smithsonian 2022); Future and the
Arts: AI, Robotics, Cities, Life - How Humanity Will Live Tomorrow (Mori Art Museum 2019); The Future Starts
Here (V&A South Kensington 2018); Possible Spaces (Danish Architecture Center 2018); Future Shock (180
Studios 2022); WHO Futures Art Exhibition: Envisioning the Future of Health in 2050 (World Health Organization
2022); Future World (Art Science Museum Marina Sands 2022); Future U (RMIT Gallery 2021); Hope for the future
& meaning of life (Kawaguchi Art Museum 2021); Sampling The Future (National Gallery of Victoria 2022);
Edible Futures (The Dutch Institute of Food & Design 2022); Future Food Today (Space10 Gallery 2022); The
Future We Create (Art Works for Change 2022); Remembering the Future: 100 Years of Inspiring Art (Heard Museum
2022); Future Perfect (worldwide 20192022); Future Retrieval: Close Parallel (Cincinnatti Art Museum 2021);
TECH/KNOW/FUTUREU/ From Slang to Structure (Montclair State University 2021); Past Present Futures: Notions of
Time in Twentieth-Century Art (Blanton Museum of Art 2001); Future Is Today (Al-Tiba9 Global 2020);
Decriminalised Futures (Institute of Contemporary Arts 2022); Futureritual (Institute of Contemporary Arts
2022); The Future of Now:; Contemporary Art in Our Unstable World (Emmanuel Art Gallery 2022); Designs for
Different Futures (Philadelphia Museum of Art 2020); The Future States (Latvian National Museum of Art 2018);
Remember the Future Orleans House Gallery 2021); Designs for Different Futures (Walker Art Museum 2021).
(2) Heidegger’s attention to intratemporality concerns an existential structure of Dasein that
is inauthentic and measured by technological instruments and by calculation. In Heidegger’s optics,
intratemporality denotes an inescapable horizon for Western history of being.
References
1 Horst Hörtner, Roland Haring, Hideaki Ogawa, Andreas Hirsch, Alchemists of the Future: Ars
Electronica Futurelab: The First 25 Years and Beyond, Hatje Cantz, 2022.
2 Christophe Leclercq, “The Legacy of Experiments in Art and Technology (E.A.T.): An Environmental
Aesthetics,” paper presentation at ISEA2011: 17th International Symposium on Electronic Art, 2011.
3 Christine Ross, The Past is the Present; it's the Future Too: The Temporal Turn in Contemporary Art,
New York, Continuum, 2012.
4 Pamela M. Lee, Chronophobia: On Time in the Art of the 1960s, Cambridge, The MIT Press, 2006.
5 David Harvey, The Condition of Postmodernity: An Enquiry into the Origins of Cultural Change,
Hoboken, Wiley- Blackwell, 1991.
6 Jonathan Crary, 24/7, London and New York, Verso, 2014.
8 Martin Heidegger, Being and Time, New York: State University of New York Press, 2010 [1927].
9 Yuk Hui, On the Existence of Digital Objects, Minneapolos, University of Minnesota Press, 2015.
10 N. Katherine Hayles in How We Think: Digital Media and Contemporary Technogenesis.
11 Mark. B. N. Hansen, Feed Forward: On The Future Of Twenty-First-Century Media, Chicago,
University of Chicago Press, 2015.
12 Bernard Stiegler, Technics and Time, 1: The Fault of Epimetheus, Stanford, Stanford University
Press, 1998.
13 André Leroi-Gourhan, Gesture and Speech. Translated by Anna Bostock, Berger. Cambridge,
Mass., MIT Press, 1993.
13 André Leroi-Gourhan, Evolution et techniques I: L'homme et la matiere (Man and Matter),
Paris, Albin Michel, 1943.
14 Tanya Ravn Ag, “Media Art in the Hybrid City – Why?” keynote at Art in the Smart City
symposium, hosted by Art Republic at Stavanger Art Museum, Norway, November 2018.
Bibliography
Jonathan Crary, 24/7, London, New York: Verso, 2014. Hansen, Mark. B. N. Feed Forward: On The Future
Of TwentyFirst-Century Media, Chicago, University of Chicago Press, 2015.
N. Katherine Hayles, How We Think: Digital Media and Contemporary Technogenesis, Chicago, University
of Chicago Press, 2012.
Martin Heidegger, Being and Time, New York, State University of New York Press, 2010 [1927].
Horst Hörtner, Roland Haring, Hideaki Ogawa, Andreas Hirsch, Alchemists of the Future: Ars
Electronica Futurelab: The First 25 Years and Beyond, Hatje Cantz, 2022.
David Harvey, The Condition of Postmodernity: An Enquiry into the Origins of Cultural Change,
Hoboken, Wiley-Blackwell, 1991.
Yuk Hui, On the Existence of Digital Objects. Minneapolos, University of Minnesota Press, 2015.
Christophe Leclercq, “The Legacy of Experiments in Art and Technology (E.A.T.): An Environmental
Aesthetics,” Paper presentation at ISEA2011: 17th International Symposium on Electronic Art, 2011.
Pamela M. Lee, Chronophobia: On Time in the Art of the 1960s, Cambridge, The MIT Press, 2006.
André Leroi-Gourhan, Gesture and Speech, Translated by Anna Bostock Berger, Cambridge,
Mass., MIT Press, 1993.
Christine Ross, The Past is the Present; it's the Future Too: The Temporal Turn in Contemporary
Art, New York, Continuum, 2012.
Bernard Stiegler, Technics and Time, 1: The Fault of Epimetheus, Stanford, Stanford University
Press, 1998.
Author Biography
Tanya Ravn Ag is a curator and scholar on art, technology and digital culture, focused on how art changes
with contemporary technogenesis and algorithmic culture. She is the editor of Digital
Dynamics in Nordic Contemporary Art (Intellect, 2019) and co-editor of What Urban Media Art Can
Do – Why, When, Where, and How? (av edition, 2016) based on which she co-founded the globally
networked Urban Media Art Academy in 2017. Her curatorial engagements with urban, media-based art include the
Screen City Biennial 2017 in Stavanger, the SP Urban Digital Festival in São Paulo in 2013 and 2014, and
Nordic Outbreak presented in New York City and across the Nordic region by the Streaming Museum in 2013-2014.
She has served on the ISEA Board since June 2023 after chairing the IIAC (ISEA International Advisory
Committee) since 2020.
Acknowledgements
The research informing this paper has been supported by the Carlsberg Foundation’s Reintegration Grant
for the project Art of our Times: Deep Temporalities of Art and Algorithmic Culture at the Department of Arts
and Cultural Studies, University of Copenhagen.
For a more symbiotic co-individuation with our technological avatars: how to go, with the Sciences and the
Arts, beyond hybridizations?
EtienneArmandAmato & EtiennePéreny
Laboratoire DICEN-IDF, Université Gustave Eiffel & Université Paris 8
Paris, France etienne-armand.amato@univ-eiffel.fr ; pereny@gmail.com
Abstract
With interactive computing, the metaphorical use of biological notions of hybridization and symbiosis has
become widespread. They refer to the possibilities of mixing as well as to the conditions of emergence of
relationships ranging from mutual benefit to instrumentalization between technologies and humans. In order to
better understand the relevance of such analogies with the living, this article draws on scientific and artistic
research concerning the interactive avatar. These seem particularly instructive on our relationship to
technology because this virtual being hybridises the living and the artificial, while constituting the key and
the condition of access to digital spaces, to co-evolve with other users, themselves avatarised, or other
autonomised agents. The article distinguishes between "cyber" avatars populating persistent universes, video
games, virtual realities, i.e., cybermedia environments of simulation and interaction. The "hyper" avatars are
those of the Web, online services, 2.0 platforms or socio-numerical networks that are juxtaposed in networked
informational and documentary hypermedia. From then on, the challenge is to reinvest the sense and
responsibility of their potential for augmentation or simulation, in particular by promoting cooperative
interactions through science, art and technology, which are themselves by their very nature synergistic with
each other.
Introduction: symbiotic design at the origin of interactive computing
Since the great convergence of telecommunications, computer science and miniaturized electronics, the expansion
of networked digital environments has been accelerating, which never ceases to raise questions, so much so that
it guides the civilizational changes underway. Digital art has been able to contribute to these developments
from the outset by appropriating the different waves of inventions, software and tools: network art, code art,
interactivity, Web art and Game Art, etc. However, due to a lack of recognition, these influences have remained
underground and unofficial, as industrial and commercial creations dominate. After decades of intellectual and
artistic, academic and cultural engagement, initiatives such as ISEA or numerous Arts & Sciences projects,
have made common the idea that artists contribute to create an alternative technological culture to those
stimulated by consumption, market and entertainment, in order to produce new practices and regenerative
imaginaries. By choosing the theme of "Symbiosis," ISEA 2023¹ sends a strong signal to the communities in
charge of digital cultures and electronic arts, a kind of encouragement to favour a transformation of our
relationship with the digital, to move towards more responsibility and reciprocal benefits, while comfortable
consensuses and evidences are cracking. However, the matter is not so simple. Admittedly, the symbiotic approach
has the advantage of offering a qualitative leap forward and a step back, integrating the latest scientific
knowledge on cooperative interdependencies, at all scales of life and at levels previously invisible, and just
which have been revealed by technical images.² However, this notion does not have the same cultural and
political effects depending on whether we take it from an apparently neutral angle; with the Anglo-Saxon
meaning, which considers as symbiotic all forms of interactions between species, including the most aggressive
and predatory, parasitic and deadly ones; or whether we take it in its French and European meaning, which
favours synergistic, mutualist or commensal symbioses, offering mutual or at least unilateral benefits.
But in a facetious and creative way, let us open our remarks with an enigma offered to the reader of this
article, before he or she dives into the heart of digital systems and environments by following the path of
co-evolution made possible by avatars.
From whom does this excerpt come? "The potential of the human-machine symbiosis is easily visible in the arts,
where computing technologies have enabled the creation of previously unrealizable forms of expression. Computing
technology has empowered a new legion of artists working in mediums such as immersive and augmented reality
games, animated feature films, and music composition and performance (...) In these areas, we are beginning to
see humans and machines as complete partners in artistic creation."³ It is striking that an eminent
researcher in computer science, having joined the most demanding military programs of the DARPA in terms of
success, recognizes artistic activity as one of the best contexts for the emergence of new partnerships for the
benefit of an original creation. To agree with this, it has been widely shown that artists, through their close
contact with techniques and their potentialities, in their exploration of the logic and capacities implemented
in their tools, which have become software with computers, constitute an avant-garde that clears out the
possible and opens up new appropriations. If the works in art history have shown this, it is up to thinkers more
in tune with the arts and technologies of the image, such as the Frenchman Edmond Couchot, to show it in the
course of his numerous works.
But beyond the man-computer symbiosis and the place of the Arts in its advances, let's go back for a moment to
the origin and the evolutions of this new paradigm of microbiological symbiosis that appeared following the work
of Lynn Margulis⁴ concerning eukaryotes in the 1960s, co-author of the "Gaia Hypothesis" with James
Lovelock.⁵, ⁶ This research eventually produced major scientific advances concerning
plants, forest and soil, but also the intestinal microbiota in humans, emphasizing the symbiotic interdependence
of living things, both inside and out their bodies.
For our purposes and references concerning the coevolution of man and his technique, we will retain the
teachings of Leroi-Gourhan⁷ and Gilbert Simondon⁸. The former analyzed hominization through "gesture
and speech," techniques that preform both our morphology and our cognition, the latter insists on the
"individuation process" and its "associated environment," which organize the individual and collective mode of
existence of man, as well as that of technical objects.
But the difficulty remains in finely articulating the human and the artificial, in respecting proven frameworks
of thought while going beyond them, updating them, making their complexity a little more "simplex," following
the teachings of Alain Berthoz⁹ supported by the phenomenology of Merleau-Ponty¹⁰ and the
enaction of Varela.¹¹ By proposing an emerging concept, that of the symbiotic co-individuation of man
and his avatar, that is to say an individuation—not to be confused with individualization—which is
made joint by their common process of co-evolution, it is indeed a question of situating its context and
exposing the premises of a vast site of studies and creations which remains to be undertaken through an approach
seeking to create a new synergy between the Sciences and the Arts.
I) Symbiosis and hybridization: biological metaphors in vogue in the computer and digital field, from the
beginning of computing to the present day.
a) A novel concept of a possible partnership between humans and computers
Anyone interested in the links between symbiosis and computer science will identify the inaugural 1960 article
"Man Computer symbiosis"¹² written by one of the acknowledged founders of modern interactive
computing, J.C.R. Licklider. Both a psychologist and a computer scientist, he developed a way of thinking about
the partnership with the computer, and later helped coordinate, within military programs, the transition from
the first computer networks to the Internet. Such an early reference to and mobilization of a concept from the
life sciences, and moreover in connection with the military-industrial complex, may surprise and even arouse
suspicion. A few points need to be made here. Firstly, it bears witness to a context of technical emergence,
that of computers finally operating in real time and becoming sufficiently programmable and accessible thanks to
their sensitive interfaces (screen, optical pen, and keyboard) to envisage a genuine cooperative interaction
that would bring together the best of the two worlds, living and artificial. Secondly, it also highlights a
typically cybernetic vision of putting human and computer in parallel, becoming colleagues and partners, who,
far from being reduced to each other, assert their specificities and differences, but pull their strengths and
weaknesses. Thus, by associating and communicating according to the right methods, they will constitute a tandem
achieving what no one alone would manage to do. Finally, this positive, even idealistic, vision must be
understood as a "political move" opening up another perspective at the heart of the technocratic system to
confront the promoters of the all-machine approach and of a progress leading eventually to the advent of
artificial intelligences superior to humans. However, several controversies could immediately invalidate such an
idea and send it back to the side of naive approximations. In fact, symbiosis is observed in the field of the
living, not the artificial. Applying it to computers would mean granting this class of logical machines the
status of a species endowed with certain properties of the living, such as evolution, adaptation and
reproduction. But in our western conception of the world, practising a “naturalistic ontology” as
French anthropologist Philippe Descola call it,¹³ the highest faculties are the prerogative of
humanity. If we want to continue the analysis with Gilbert Simondon,¹⁴ in the absence of integration
with culture, technology is conceived as separate and external to man, who at best uses it, while attributing to
it his own misdeeds. When in fact, technology, this human creation, has the role of mediating between nature and
culture. However, the metaphorical use of symbiosis has a few merits: better respect for each stakeholder in the
tandem; understanding of their interdependencies; and the establishment of a more balanced cognitive and
sensitive dynamic than the sole technical and strictly functional approach, that of a mechanically extended
human, "mechanically extended man" or "humanly extended machines" (quoted in ¹²). With this approach,
enlightened by the living, a perspective is developed that is neither completely anthropocentric nor
technocentric. It will indeed influence a whole community of inventors and engineers to seek mutually beneficial
relationships. This begins in the field of complex problems, extends to ergonomic interfaces, in connection with
the interfaces and work methods invented by Engelbart ¹⁵ with his "mother of all demos,"¹⁶
and will continue with the first technological research involving computer networks, at the origins of the
premises of the Internet.
b) Instrumentalization and hybridization: what association between the living and the
artificial?
It is clear that more than 60 years later, this cooperative conception has not really governed the deployment
of the "digital." The triumphant regime is that of a generalized and excessive instrumentalization of some by
others. It is also based on older, naturalistic metaphors that have become dogma: the struggle for survival,
selection, competition and domination, all stimulated by a misunderstood and ideologized Darwinism of
Anglo-Saxon and capitalist origin that fetishizes the market and competition. This cultural overdetermination
tends to reduce both humans and computers to a few utilitarian functions, rather than mutually emancipating
them, or just amplifying them (Licklider¹²) or increasing them (Engelbart¹⁵). For example,
workers turn out to be replaceable as soon as a task can be automated, while the computer itself is devalued,
becomes obsolete, as soon as a new, more powerful and "modern" generation arrives on the market. In other words,
technological "progress" is based on the division and specialization of tasks to the detriment of others: models
implementing interests, reconducting the existing and thus rigidifying socio-technical systems. (1) Following an
instinct of conservation hiding under the imperative of innovation, it is essentially a matter of optimizing and
stabilizing rather than solving and changing; of intensifying short-term productivity rather than changing the
level of symbolic integration and moving towards "a better ecology of the mind."¹⁷
Another notion, also imported from biology, is proliferating in the field of new info-communication
technologies, and continues to be very much in vogue on the cultural level.
It is the notion of hybridization, synonymous, depending on the context, with convergence, fusion, mixing,
blending, and association, concerning both the technologies themselves and the interpenetration or interweaving
of heterogeneous dimensions: the human and the artificial, or the real and the informational. It carries an
enthusiastic conception of the possible crossings between various technical lineages as well as of what results
from them, their offspring. It probably has a positive echo in our anthropological collective unconscious
because of the reproductive experiments dating back to the Neolithic period, carried out by our species on
plants and animals to make them edible, domesticate them, use them, and thus develop ever more new hybrids to
improve our living conditions.
As a result, there is a tendency to explore and implement all possible configurations through clever and
inspiring mixtures, according to a principle of free experimentation that does not take into account the stakes
and consequences from the outset, but rather evaluates them, possibly and only after the fact. Believing himself
to be the "master and possessor" of his technique (2), humans let themselves go to "hubris", to this excess of
power, to this attractive vertigo that is generated by a too continuous success. The conquest of all
possibilities guides our collective strategy. Most forms of exploitation, first and foremost of the living
through technology, are justified, pending the next techno-solution that will correct the effects. In this vein,
transhumanist conceptions assume that humanity must continue to mutate by itself thanks to Technology and
Science, assisted by an Art that would become its official foil. But do they realize that the solutions obtained
by a definitive hybridization (the cyborg) or by deep environmental modifications (the geoengineering) remain
prisoner of the same paradigm that caused the problems it tries to solve? Although some forms of hybridization
may be viable and desirable, with lines that improve through heterosis, which is the alliance of the best of two
species, we will try to think according to a higher general principle, that of an evaluation and regulation of a
symbiotic nature, in the French sense of the term, and therefore mutualistic, which could frame, anticipate and
therefore limit any possible deleterious effects.
II) "Cyber" and "hyper" avatarizations: two main ways of entering the digital world
a) The cybernetic avatar: a body situated to experience virtual technologies
The fundamental phenomenon that networked microcomputing has made possible, but which was not immediately
grasped or understood as such, concerns the "existential" entry of the human into the machine and into the heart
of its software. If this is realized in many ways, the avatar is the most widespread and concrete emblematic
figure, especially the one that networked video games have made known to us. Indeed, since its
baptism,¹⁸ the avatar has made it possible to inhabit an immersive environment with other human
persons and programmed entities, and even to cooperate with artificial beings or with other "semi-living"
beings, which have yet to be better qualified and understood, both scientifically and artistically.
The original symbiotic vision already mentioned had stopped at the face-to-face meeting of the human being and
the computer, forming an interdependent and winning team. With the integration of networked computers, an
informational, quasi-corporeal vector was needed, if only to manifest itself to other distant people, caught in
the same technological constraints. In a shared place, benefiting from a system of exchanges and actions, the
human invests his avatar to interact in a more complex way than with conversational interactivity based on
dialogue and instruction. Collectively, the first interactivity (reactive, sequential, asymmetrical) is overcome
and gives rise to the second interactivity (proactive, simultaneous, mutual). The first occurrences of avatars
(3), retrospectively identified by semiologists, computer scientists and other analysts,²⁰ result
from the marginal appropriations of the very first transistor computers by new adepts of creative subcultures,
looking for a way to divert and explore the capacities of new electronumerical machines, at a more human size
and finally becoming more accessible.
Since then, despite their improvement and growing sophistication, playable avatars, those of the simulated
worlds of video games, can still be analyzed in terms of "living-artificial complexes." Indeed, the technical
part, with the programmed functioning, and the living part, with the human behaviours adjusting in real time to
the action in progress, are mixed in the same virtual being. In this respect, avatars are similar to
techno-human or bio-technical "hybrids," depending on the point of view adopted. In this respect, field studies
in the human and social sciences show that online participants who use avatars know how to consider any "player
character" they encounter in a persistent universe from both sides. They evaluate both their operational
capacities and the living intentionality that drives them. In addition, the hybrid avatar has several remarkable
characteristics that relate to its mode of operation and attendance. On the one hand, it results from a "soft"
hybridization, because it is temporary and reversible (unlike the cyborg), which preserves the integrity of both
halves, the human and the artificial. On the other hand, this humanized technical hybrid is unified and
maintained by two parallel processes that can be analyzed through the symbiotic metaphor. On one side of the
screen, in the ordinary world, there is a human-machine-computer coupling which, for the time being, joins the
Lickliderian vision of a dynamic symbiosis. Except that here, it is even more marked because it is based on a
total interdependence: the human needs a formalized avatar to evolve in a simulated world while, without its
player, the avatar is only an empty shell, identifiable by its looped attitudes, as if waiting for a soul to
re-function. Both interface and vehicle, it is an entity that articulates the real and the virtual. On the other
side of the screen, this time in the simulated world, the avatar must compose, negotiate and build its
partnerships, if possible symbiotic to last and grow, both with its fellow avatarized, and with other
inhabitants fully programmed and more or less autonomous. Its survival and development often depend on
relationships of co-construction and cooperation, even in a competitive context. By associating as a group, by
combining their talents, by carrying out collective strategies, while benefiting from the support of artificial
beings, the human beings hybridized by the avatar adapt to a place lived in common. They develop a powerful
intersubjectivity, a feeling of living together that is confirmed by their decisions and actions. Unified by a
particular spatio-temporality and physicality, this virtual but concrete place mobilizes their situated
cognition; that is, sufficiently spatialized to transform the environment and its components or actants into
resources. As for the corporeality of their avatar, it engages a cognition here virtually embodied, but
otherwise instantiated according to principles partly analogous to our human condition. Finally, through their
avatars, humans enter into a very strange trade with certain autonomized agents, typical of their adventure
worlds: pets, helpers and assistants, drones or robots, supernatural allies... With these, a distributed
cognition develops, i.e., distributing the processing of information over these autonomized entities, for
example, the tracking, surveillance or management of this or that process. All of this is linked to the ongoing
cooperation with the other comrades in the game, the other avatars. In this way, many intimate and vital
relationships are established and unraveled with programmed or hybrid technical beings, whose radical otherness
is forgotten, by dint of being dressed up in the colors of the narrative, of the game or of simple co-existence,
as within Second Life-type metaverse based on collective creativity.²¹
This diagram visualizes the way in which an avatarial hybridization takes place, framed and framed by a double
relationship that can potentially be interpreted in symbiotic terms: on the left, that of the human coupled to
the nested triad computer/software/avatar; on the right, that of a cybernetic "avatar" comparable to a symbiont,
as soon as it is engaged in relationships of interdependence with mutual benefit with its other congeners, or
even with various entities entirely programmed and eligible for a relationship of interest. It is in these
virtual worlds that symbiotic co-individuation could take place not only between the human and his avatar, but
also with the empowered virtual beings that also inhabit them.
b) The hypermedia avatar: an aggregate of data in the service of an identity model in hybrid spaces in
two ways
In contrast, we must address, even briefly, the other immeasurable domain of hypermedia, because the way in
which humans must avatarize themselves in order to exchange and cooperate with their fellow human beings is
quite different from the previous one.²² This involves the establishment of a "profile," with oriented
functionalities. This is in fact a particular identity model, a sort of explicit information sheet specifying
the aspects useful for the activities specifically allowed by the platform. By filling in fields and forms, by
providing qualities and quantities, by uploading media (photos, thumbnails, videos, visuals, sounds), the user
explicitly creates his profile, which will then be fed more or less invisibly by the "datas," "metrics" and
other "analytics" that are generated according to his activities. Because the activities are more disembodied,
more informational than simulational, the participant feeds the platform where he acts by his punctual acts, by
his simple consultations, requests, clicks and orders, which have become his editorial and media contributions.
There are certainly some services that focus on cooperation, such as Wikipedia for collaborative writing, or
GitHub for collective programming. However, most platforms, especially commercial or social ones, juxtapose
hyper avatars with no common space other than a customizable interface. They offer specific functionalities, for
example, for shopping or conversation, which require a simple interactivity, of command and impulse: summoning
contents, pages or profiles; clicking to act; typing and sending asynchronous messages to exchange; just
exciting by notes, scores or likes, or on the contrary, inhibiting by blocking and unsubscribing.
In the inaugural vision, it is within this informational, documentary and media hyperdimensionality; from
Vanevar Buch's Memex to Ted Nelson's precursor "hypertext"²³ ; that positive symbiotic relationships
were supposed to unfold. But we are struggling to find them, so much so that the algorithms that model and
influence behaviour encourage compulsive buying, isolation in filtered bubbles, the excitement of passions and
cognitive biases. Thus, these hyperspaces have tended to reduce the human to a set of models and
data,²⁴ made of functions and behaviors that integrate with software. User-consumers are thus
influenced and made predictable by traceability, profiling and probabilistic anticipations. The mainly
utilitarian relationships that are currently developing within the "hyper" digital environment have shown their
danger, even toxicity for democracies, potentially influencing crucial voting processes. At issue is the
modelling of stakeholders, which is often done without their knowledge, and the monitoring and stimulation of
behaviour guided and motivated by forms of economic, cultural and political predation that go beyond the logic
of advertising or the attention economy alone.
Such extreme pressure and overkill can be explained