Reexamination of "fidelity" in sound and image

Department of Acoustic Design, Faculty of Design
JO Kazuhiro, Associate Professor

At the JO Laboratory, Faculty of Design, Kyushu University, we conduct practice-based research mainly through the creation of artworks with a background in acoustics and interaction design. We critically confront media technology and explore ways of expression that go beyond demonstrations of new technologies as an example of the humanization of technology.

These are the words that appear at the beginning of our laboratory's website[1]. However, we think that these words alone are not enough to explain what we are doing, so we will introduce some specific examples along with one of our recent research projects below.

This research, entitled "Reexamination of 'fidelity' in sound and image (JSPS Grant-in-Aid for Scientific Research (B) 23H00591)," is based on the following idea

"In this research, we will realize a technique to generate sound and images that are processed as digital data (information) in the form of samples or pixels inside a computer, as phenomena in other natural systems. The research will consider techniques to generate acoustic and visual expressions that go beyond mere natural phenomena by intervening in the process of reproduction, rather than simply observing natural phenomena as they are. Through the creation of works that entrust the final expression to the incomputable part of each natural system and the creation of a dialogue through the public display of these works, the concept of "fidelity" in sound and image will be reexamined."

This is what the project is all about. To explain a little more, the development of AI technology in the past few years (as of March 2024, suno. ai[2] for sound and sora[3] for image are probably the most obvious examples) has made it possible to computationally generate almost any kind of sound and image inside a computer. In light of the current situation, we are trying to produce sound and images that focus on elements in various natural phenomena, more specifically the phenomenon of microbial reproduction, that cannot be computed.

Below are specific examples of sound and image.

- Fermentation of microorganisms (sound)
Here, we are examining a technique that uses fermentation as a principle for sound generation. Fermentation is an act in which microorganisms metabolize organic matter to obtain energy. In order to produce sound through fermentation, we focus on the sound of the burst of carbon dioxide gas produced by the fermentation of alcohol. Since alcohol is produced along with the generation of sound, we applied for a license to test brews in accordance with the Liquor Tax Law [4], and just recently, on March 1, 2024, after more than six months of processing, we were able to register the food lab, recording studio, and various other facilities on our Ohashi campus as a manufacturing facility (see figure). Currently, we plan to use makgeolli, a traditional Korean alcoholic beverage, and mead, which is said to be the oldest alcoholic beverage known to mankind, as materials to create works that leave the development of sound to microorganisms that are in continuous motion, independent of human hands.

- Luminescent bacteria (image)
For the generation of images, we are focusing on a microorganism called luminescent bacteria, which emit light in the dark by itself. Here, an image produced in advance inside a computer is output as a high-definition silk screen using a machine called a digital screen-engraving machine. Luminescent bacteria are then applied to the agar medium via the screen as a substitute for ink, so to speak, to produce a fine pointillism that is self-luminescent. The bacteria applied to the medium reproduce under the influence of temperature and humidity, producing an image whose content shifts indeterminately along a timeline (Figure)[5][6].

As described above, this research aims to create sounds and images that are difficult to faithfully reproduce in principle, by actively accepting the uncertainty inherent in the phenomenon of microbial reproduction. Through these practices related to sound and image, which are based on human intention but entrusted to microorganisms to change, we will reexamine the idea of "fidelity".

References

[1] JO Laboratory, Faculty of Design, Kyushu University https://www.design.kyushu-u.ac.jp/~jo/

[2] Suno.ai https://www.suno.ai/

[3] sora https://openai.com/sora

[4] Liquor Tax Law https://elaws.e-gov.go.jp/document?lawid=328AC0000000006 (In Japanese)

[5] Takumi Saeki and Kazuhiro Jo, "A Medium for Images or Luminous Bacteria," A Study of Images Using Luminous Bacteria and Digital Screen Printing, Interaction 2023, pp. 344-347, 2023. (In Japanese)

[6] Takumi Saeki and Kazuhiro Jo. 2024. 'イ'(1926) by BioLuminescent Bacteria. In Proceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '24). Association for Computing Machinery, New York, NY, USA, Article 95, 1-3. DOI: https://doi.org/10.1145/3623509.3635320

■Contact
Department of Acoustic Design, Faculty of Design
JO Kazuhiro, Associate Professor