Following text has been written during the fellowship:
Beques per a la recerca i la creació en els àmbits artístics, del pensament i dels nous sectors creatius i d’interdisciplinarietat. Osic / Departament de cultura / Generalitat
Auditory perception is a very important awareness next to shape, material, color, light, temperature, etc. I think that architecture is completely sound material related to architecture and the environment, forming a synergy. The influence of space in our inner senses connects with our inner sound inside our environment. We could experience a building with more nuances, if we had the opportunity to hear it. This research deals about the crossing between architecture and sound through data translation. Together with a theoretical research are some experiments that show in detail the translation of sound to three-dimensional models in the digital domain and vice versa. Some of these experiments are produced by means of 3D printing in order to make a public presentation of the pieces. This part will be exempt from this application since it belongs to the artistic production and project execution. The goal of this research is to show how the sound can generate forms and the forms can be heard, establishing relationships between sound and space. Passing from shape to sound and from to sound creates an indicating loop about art being a continuous transformation and that things can have more than one appearance.
Connection between the virtual and material space
We are at a historic moment of great significance for human evolution. The layer of intangible reality that involves knowledge, information and data, has blended in the physical reality of the tangible. Information technology and communication have generated a new symbiotic stage. This draws an augmented reality capable of destabilizing the pre-existing Status- Quo, comparable in magnitude to the cultural historical moment of the printing press appearance. In the last twenty years, various fields of human knowledge have entered the intangible computer space to articulate and mediate representation in the material space. The intangible layer representing the digital world constitutes a (meta) tool that helps us to realize our ideas. As an instrument in constant development, software and networks have enabled the emergence of a new reality, which in its essence is sustained by a binary language (1) and vast amounts of complex arithmetic. This computerized numerical processing has become a second mind, which enables simultaneous information and the ability to unlimited reproduction. Digitalism reduces the linear equation of time, allowing real-time simulations in numerous processes and thus optimizing analytical calculations in compressed time. It could take years to resolve them through human calculation.
Somehow, this virtual layer works similarly to the general systems theory. This theory deals with finding the properties common to all systems. That is, it tries to find the values of different academic disciplines in an interdisciplinary meaning by a new interlaced conceptual language. Its purpose is the development of tools that enable the various branches of science in their research practice. Originally it is a theory that appeared in the biology field, but rather inspired other fields and was called Science Systems, specializing in cybernetics, information theory, game theory, chaos theory, the theory of catastrophes, etc. This new language is also found in the digital world. It is a phenomenon of contemporary communication and information technologies, which since the mid-twentieth century marked a new stage. With the emergence of this paradigm we process almost all the works of different disciplines. The structure inherent in any software is similar when we perform an architectural project, compose a piece of music, write a text, produce a visual art installation, elaborate scientific theories, etc.
If all these different genres begin to interact with the same link (on digital media, any software has a common structural root binary), we can use this transdiscipline to obtain new achievements between knowledge, science, art, music, media and architecture.
Usually hybrids occur between subgenres of a single discipline or field; for example hybrids between biology and medicine and biology and physics or in the arts field, artistic installations (from genres such as sculpture, sound art and multimedia). In different historical periods it has been reported that hybridization between different areas has generated an expansion of the knowledge field. In the architecture field of recent decades, a paradigm is bio-construction, hybridizing fields of biology, ecology and construction. These hybrids are not only a new discipline but also inspiring breakthroughs in certain specialties. For example in music, granular synthesis techniques (2) are purely inspired by the theories of quantum physics. However, hybrids may also be formed by more than two disciplines. There are many possibilities of recombination, once used the same link or common language. The intersection between architecture and sound in the digital realm could be another hybrid.
As mentioned before, designing objects and composing sound may be deductible into similar digital parameters. This makes it possible to transfer disciplines based on quantitative and qualitative parameters. The purpose of this research is to discover new sound translations from sound to form and form and to sound. Later, it will be shown how this matter has been experienced from different angles. The experiments deal with the collection and processing of sound data, for subsequent transfer to a three-dimensional model and vice versa. The symbiotic relationships sound / form experiments remain at a logical level because from initial sound we go to a formal three-dimensional model with architectural potential. However, if we analyze the issue as a matter where the symbiotic relationships between sound and form coexist, then it is a more complex issue. Complexity is also marked by pairing (sound) intangibility-(object) tangibility. In the intangible field (in the computational world any operation becomes part of this frame because it does not possess any corporeal entity) a property absent in the field of the tangible is acquired. This property has the potential of nonphysical form and is indeterminate, whereas in the tangible domain each object is given and determined. Computer can perform complex operations, not only in regard to the object itself, but the representation and media coverage of it. We could get a series of data stored in a memory, which can then be incorporated into many different elements, a musical composition, a text interpretation, statistics, three-dimensional model, etc.
New digital technologies provide a possibility to influence the sound architectural space in an innovative way. The sound can directly influence the tectonic and not just the atmosphere. Auditory perception is the most important awareness of shape, material, color, light, temperature, etc. I think that sound architecture is completely linked to material architecture and the environment, forming a synergy. The influence of space in our inner senses connects with our inner sound inside our environment. Could we experience a building with more nuances, if we had the opportunity to hear it?
Studies of quantum physics, especially string theory says that the tiniest particles of any material are filaments that vibrate in more dimensions. Vibrating they produce a sound, albeit inaudible to our ears. So everything that is in continuous vibration has its own sound. In the case of a living being cells are generated, others are born and die, or regenerate. Clearly there is already a vibration at the cell level. Professor Jim Gimzesky detected the sounds of cells. He differentiated the cancer from the healthy ones, because they sounded different. Diseased cells sounded with a very high pitch. This finding is important because it represents another way to detect illness besides the image.
But what happens with inert materials? According to string theory the smallest particles also vibrate and, consequently, produce a sound. Apparently, it is a matter of scales of reality, not being able to hear the simple sounds these particles emit at quantum level. However, all this theory is still under investigation. A quantum computer and special measure machines have not been jet developed in order to do it. However, technological advancement is increasing fast. The day will come when we will know how our house sounds, our mattress, our table, etc. But what serves to know how our house or our furniture sounds?
There are different answers to this question. First, that it could provide fast detection of material damage, often not seen from the perfect outside. When a living being is sick it complaints; why can a concrete wall not do the same. Another answer is that by listening to these sounds, we will be aware of different states of reality that are hidden to the naked eye or ear. By recognizing sound we go deeper into the reality of every living being or inanimate, discovering new layers of perception. Discovering an object through another sense is like giving a new way of looking at it.
What is possible with today’s technology is an approach between an object and its sound, a translation of a 3D model to sound and vice versa. As in any translation, there are several ways of saying the same thing. Therefore, there will be different songs or sounds that describe the original object, depending on the angle it has taken.
Information and software
‘The complexity of the world and our culture requires the type of thinking that programming enables. We need to think about models and simulations. We need to build systems rather than fix relationships or objects. We need to grasp and analyze difficult complex systems. We need to break them into smaller manageable pieces in order to understand them. We need to build relational machines rather than an object or an idea.’ Casey Reas
The codes are binary encoding of data. These represent only some information and become something through interpretation and computational logic. Its quality is essential plasticity, the ability to take on any configuration request. Therefore, they are a truly universal means but lack of materiality itself. The software processes the contrary, if they have material qualities. When coding, algorithms are elected by their behavior. Optimized parameters settings produce a number of desirable results and create interfaces for their suitability to the task in question. So materiality software actually represents the most logical process. Choosing all the properties of the software and its combination of factors will determine the outcome, which can be either parametric data-driven, interactive or performative. All aspects of the software are defined through code. When software interfaces mimic the physical world it’s because we want to do so, because creativity computing has the potential to be really huge and sometimes even bizarre.
Immateriality – data between visibility and invisibility
The intangibility as material is being discovered today, opening a new field in which poetic narration with space and information is being held. Are electronic screens the only appropriate means to display this new material? What is the shape of these data? Now the fact that location-based data floating through space are redefining contexts and places, a new field for designers has opened: How can we profitably integrate the information (data) into physical space? We have to get used that information has more than one appearance, and not a tangible one. In interpreting it through texts, images, 3D shapes, etc., we make it tangible. Perhaps the way to show such information or data, in addition to physically display screens, could be transforming it into other types of formal data, sound, touch, taste, smell, etc. The translation of data is a change of perception, which can be very useful, first to understand it better and second to become aware of the importance of it.
If we get the description of a work of art from another point of view, we perceive that there are other ways of understanding this work. We will perceive other nuances that enrich the comprehension. Quantum mechanics physics is letting us see the plurality of a different world. A micro-world, where there can be two or more states simultaneously of the same entity, is connected with the thought about this data translation, because the original data information is kept. This thought may seem like a crazy idea. However, every day we are becoming more aware about the fact that everything is somehow connected with each other. Before entering the issue of translating form to sound and vice versa, I would like to describe the meaning of data and information in the digital world.
Sound engine as formal genesis
In the current scenario, data and information have become central. The irreversible inclusion layer of technology in all fields has opened the possibility of studying new branches of our disciplines and how they are interconnected with each other. The precise innovation lies in finding new mechanisms for creating forms. The experiments of sonomorphism and sonification are a method of generating forms and spaces. These translation techniques provide opportunities to create three-dimensional forms and to encourage mutability, interchangeability, multi functionality and hybridization as new design paradigms.
Unexpected and interesting ways appear during these processes, opening up new design approaches, generating possible impact on the collective imagination given the unconventional crossing of this discipline. As mentioned before, in the digital medium, data remain in the intangible domain, ie any operation becomes part of this domain because it doesn’t possess any corporeal entity. This lack of physicality becomes through physical interfaces something concrete so we can manage to make it visible. These interfaces can be input (keyboard, mouse, joystick, etc.) and output (screens, projectors or other display devices). Despite this intangibility, the data may have different specifications (an image, text, a modelo3D, etc.).
A computer can perform complex operations, not only in regard to the object itself, but to its representation.
During the process of translating data, some information is collected on the theme about a virtual element and then customized into another digital entity. Later this body of information on the subject can become tangible through various techniques (3Dprinting, CNC cutting, etc.).
What is interesting about the transfer of data is that it increases relationships between information and matter. Using different methods, the sound can become formal representations 3D. Hybridization of areas and frameworks is a process that is achieved with relative plasticity through customized digital tools. This in turn represents a more holistic view of human knowledge. Data translations enable information management and contextual analysis to be done. This can be introduced into design, increasing its process in unexpected and interesting ways. Therefore, these tools are interpreting large amounts of data that give a better awareness about complexity. Indeed they can accelerate knowledge and understanding of complex systems, otherwise indecipherable. Coming from a different direction, the introduction of data systems from other frameworks enables design to re -contextualize with communication.
For example, take the data structures of biological systems and translate them into physical 3D models to understand complex systems, or being able to hear the weather forecast with a melodic sequence. A new approach simplifies data communication and understanding.
This research includes experiments that show the relationship between the morphology and vibration phenomenology. The experiments are the basis of this research project. It will check if the sound can generate forms and spaces, thus being able to create symbiotic frameworks of architecture and the arts. In other words, it is a methodology of approach between formal design (sculptural / architectural / modeling) and sound, where there is a whole new field of research on the inclusion of elements invisible to the volumetric and spatial conformation. Consequently, the sound could be an intangible building material, opening new paths of exploration interest.
Sonification: translation from form to sound
A sonification allows clearly scientists, musicians and the public to interact with data very differently, especially compared with the more numerous techniques that involve vision. In fact this is, because the functions of hearing are very different that of vision. Sonification offers an alternative type of understanding of data (sometimes more accurate), which would not be possible using eyes alone.
Feeling is multidirectional, because our ears have not to be pointing to a sound source to hear it. Also the frequency response of our hearing is thousands times more accurate than our vision. To play a moving image the sample rate (called frame-rate) for the film is 24 frames per second while the audio must be sampled at 44,100 frames per second to accurately reproduce sound. Auditory perception also works as something that can be enjoyed over time scales of multiple simultaneous data streams of audio in many different dynamics. While our pupils dilate and constrict, limiting the amount of visual data, they can absorb sound at once. Our ears are also more advanced in detecting patterns of regular time and the data; how often we hear these patterns, like the frequency, the harmonious relations and timbre.
As sonification becomes an increasingly common tool, it is important to understand that aesthetic decisions are inevitable and even essential in any type of data representation. We are so used to looking at visual representations of information maps that we forget that these are also arbitrary transcoding. Even photography is not an unambiguous record of reality; camera mechanics and photographer’s art control performance. Sonification processes carry also a part of subjectivity due to the use of certain software and the author’s choice. Instead of seeing these ambiguities as a nuisance, we have to embrace the freedom that allows us to highlight certain characteristics over others and / or discover previously unseen patterns in the visual world. Innovation starts with experimentation. The landscape of creative practice is changing radically new technologies for the design and manufacture are allowing us to do more, faster, while disrupting the way we move imagine and make the world around us.
Design process is now more dynamic, global and integrated than ever, since the factories and retail experiences are intertwined. Therefor encryption becomes omnipresent and matter becomes renovated as programmable means. People and organizations leading this massive change exhibit a new way of working that ask more often “what if” … than try expectations. Software giants are building art manufacturing laboratories, while engineering companies and manufacturing are building industry-leading software tools. How is the rhythm of this changing field kept, and most importantly, the necessary impetus for its vision of tomorrow? During the past six years, the Lab format has been refining the tools and processes at the center of this question.
Digital manufacturing production changes dramatically through the democratization of access to industrial tools, as well as changing the way they produce objects, opening a door for the creation of customized objects. In combination with the code it is possible to connect directly the process of parametric software for instant manufacturing workflow, making atoms out of bits and introducing an approach that is radically different from traditional 3D modeling.
Marius Watz, coder and artist speaks about how generative systems change the approach of static models to computational logic. Objects are here understood as mere instances of family forms, produced by a specific interaction of parameters. Such forms can be created based on data or through interactive media, adapting to the conditions coded into the system. The artist becomes a” gardener “of possible forms collecting desirable results in an iterative coding and prototyping.
Sonomorphism: translation from sound to form
Sonomorphism is a term coined by me to describe the process of translation of sound into a form. This form can then become a real sculptural object using a 3D printing.
I made the translation using software SoundPlot. This appliance allows translation of sound samples to three-dimensional maps, imported from different 3D CAD modeling software (eg. Rhino). Soundplot is a software product researched by engineer at MIT Steven Pilam and architects Herwig Baumgartner and Scott Uriu, studio B + U in Los Angeles.
Steven Pliam (http://web.media.mit.edu/~pliam/res/efolio/resources/sdpt1.htm)
The software interprets the different changes of value in a particular waveform. This is translated into various available geometric models (points, curves, poly lines and surfaces). This data interpretation of the waveform assigned to these geometric figures generates a three-dimensional map, translating various values of wave amplitude and a resolution concerning configurable sampling. The resulting model will have a volume that depend on a mathematical function (logarithmic, exponential, product, etc.) interpreting the information for each simple waveform.
The procedure and process by Soundplot has a fixed nature. For example, taking a fixed sound fragment, like an uncompressed audio file (.wav formats, or AIFF) results a ‘frozen’ map, available in different 3D CAD programs. Therefore it is a procedure in which the intrinsic time of any sound manifestation is analyzed, quantified and translated into topology of a three-dimensional model. In synthetic terms, a fixed topological or frozen form is extracted from a sound sample fixed in time. The procedure is not real-time and requires rendering. The renderer can then become a sculpture with a 3D printer.
However, I have seen that a more appropriate way to do these translations between sound and form is to program customized software or make that two or three software interact. This means it’s important to learn programming and coding. I have used this research to immerse myself in the programming language Processing. (3) With processing I’ve found another way to convert sound into 3Dforms as explained below in the experimental part.
(1) The binary code is the substrate that articulates cybernetics and computing world. Through a reductionist encoding 0s and 1s, complex expressions containing programs to operate, written in different programming languages being a link between the human and the machine. These languages are synthesized by compilers and assemblers that reveal complex instruction out of the 1s and 0s. This is the link between the intangible world of languages and the world of tangible circuits and electronic components. The 1s and 0s constitute literally physical logic gates, letting or not letting pass respectively the current through complex electronic circuits that make up a computer.
(2) Granular Synthesis conceptually comes from quantum physics, understanding the sound flow not like a continuous where singularities occur in time, but as magma of sound Quantum proliferation or micro particles with resizable density and measures. The sound effect produced by this technique, nebula or ‘swarm’ Quantum is based on the selection of an existing sound piece. It is as if under a microscope we could increase the sound sample to an extent where no sound events could develop in linear time, entering the domain of timbre and microsounds. A thorough analysis of this paradigm can be found in the book of Microsound Curtis Roads (2001) Cambridge: MIT Press.
(3) Processing associate software concepts with visual, movement and interaction principles. It integrates a programming language, environment development and teaching methodology in a unified system. Processing language is a text programming specifically designed to generate and modify images, and belongs to the open source software movement.