Media Arts and Technology, UC Santa Barbara

Events

Symbiotic Architectures: Synergistic Compositions in Extended Reality, a Masters presentation by Iason Paterakis. Tuesday, December 12th, 2023 at 11am PST. transLAB (Room 2615, Elings Hall).

Symbiotic Architectures: Synergistic Compositions in Extended Reality

Abstract

The coming wave of new media technologies relying on generative composition and Artificial Intelligence (AI) has introduced novel opportunities in transmodal synthesis and creative synergies within real-time composition networks. The term ‘perforated systems’ as proposed by Marcos Novak, is used to describe the collaborative systems that arise from dialogues between diversified environments operating through flowing fields of data. Such systems allow for the creation of complex artificial, physical, and hybrid environments in Extended Reality (XR) through real-time collaboration.

This master thesis delves into the intersection of XR technologies, transmodal environments, and generative AI in speculative architectural composition, unveiling paradigms of hybrid ‘perforated systems’ in the form of interior and urban installations. These installations propose alternative environments through a fusion of tangible artifacts, olfactory, audio, and visual components, and explore the interplay between dynamic and diverse data-driven compositions in XR.

Ranging from biodata-driven abstract worlds in Virtual Reality (VR), to Mixed Reality (MR) installations like the ‘Synaptic Time Tunnel’ presented in SIGGRAPH 2023, and AI-generated urban projection mapping performances, these endeavors fuse cutting-edge technologies with generative AI tools to achieve maximal complexity with minimal initial conditions.

In the age of data, networks, and AI, the potential for collaboration between different fields and creators is vast and profound. The present thesis aims to demonstrate how transdisciplinary ‘perforated’ networks can generate complex and abstract compositions that emerge from simple elements and expand into Extended Reality. The objective is to harness the potential of merging "perforated systems" with AI, XR technologies, and the built environment, to foster the development of immersive and collaborative experiences that push the boundaries of worldmaking.

Past Events

News

The Second Edition (2023) of a book by Professor Curtis Roads, "The Computer Music Tutorial" has been released.

The Computer Music Tutorial, Second Edition (2023) by Curtis Roads

Curtis Roads, professor in Media Arts and Technology and affiliate faculty in Music at UCSB, has announced the publication of an expanded, updated, and fully revised Second Edition of his textbook The Computer Music Tutorial (2023, The MIT Press, 1257 pages).

Essential and state-of-the-art, The Computer Music Tutorial, Second Edition is a singular text that introduces computer and electronic music, explains its motivations, and puts topics into context. Curtis Roads's step-by-step presentation orients musicians, engineers, scientists, and anyone else new to computer and electronic music.

The new edition continues to be the definitive tutorial on all aspects of computer music, including digital audio, signal processing, musical input devices, performance software, editing systems, algorithmic composition, MIDI, and psychoacoustics, but the second edition also reflects the enormous growth of the field since the book's original publication in 1996. New chapters cover up-to-date topics like virtual analog, pulsar synthesis, concatenative synthesis, spectrum analysis by atomic decomposition, Open Sound Control, spectrum editors, and instrument and patch editors. Exhaustively referenced and cross-referenced, the second edition adds hundreds of new figures and references to the original charts, diagrams, screen images, and photographs in order to explain basic concepts and terms.

Features include:

New chapters on virtual analog, pulsar synthesis, concatenative synthesis, spectrum analysis by atomic decomposition, Open Sound Control, spectrum editors, instrument and patch editors, and an appendix on machine learning.

Two thousand references support the book's descriptions and point readers to further study.

Mathematical notation and program code examples used only when necessary.

Twenty-five years of classroom, seminar, and workshop use inform the pace and level of the material.

As Prof. Roads states: "I finished writing the first edition in 1993. It finally came out in 1996, the year I joined the UCSB Music faculty as a Visiting Associate Professor. Writing the Second Edition required going through the research literature in the field since 1993. It often felt overwhelming but I just had to keep going. In 2017 I devoted all my creative time to the project. I promised myself I would finish it in 2020, and at 10 PM on 31 December 2020 I finished writing. Time for Champagne! The production process took all of 2021 and most of 2022. In a way it was a perfect project for the pandemic lockdown, as it gave me a daily purpose in a time of isolation. The textbook has been the core of my teaching at UCSB."

An article about the release of the 2nd edition was published in the UCSB Current:

news.ucsb.edu/2023/021272/core-textbook-computer-music-distills-vast-information-space-definitive-tutorial

The book can be found at MIT Press:

mitpress.mit.edu/9780262044912/the-computer-music-tutorial

Professor Roads's previous books include Microsound (2001, The MIT Press) and Composing Electronic Music: A New Aesthetic (2015, Oxford University Press).

MAT PhD student Sabina Hyoju Ahn and MAT alumni Myungin Lee will present their paper "Parasitic Signals: Multimodal Sonata for Real-time Interactive Simulation of the SARS-CoV-2 Virus" at the IEEE VISAP 2023 - Perpetual Presence conference. October 4 - 29, 2023, Library at the Dock Gallery in Melbourne, Australia.

"Parasitic Signals: Multimodal Sonata for Real-time Interactive Simulation of the SARS-CoV-2 Virus"

Abstract

This project aims to transform the nano-scale of a striking biological phenomenon, the relationship between SARS-CoV-2 coronavirus and human molecules, into an interactive audiovisual simulation. In this work, the interaction data between the spike protein of SARS-CoV-2 and human cellular proteins is measured by Atomic Force Microscopy, which can touch and image a single molecule. We are creating an interactive audiovisual installation and performance from a set of interaction data.

Co-authors of the paper are Yoojin Oh, senior scientist at the Institute of Biophysics, Johannes Kepler University Linz, and JoAnn Kurchera-Morin, Director and Chief Scientist of the AlloSphere Research Facility and professor of Media Arts and Technology and Music at the University of California Santa Barbara.

visap.net/2023/contributions/parasitic-signals-multimodal-sonata-for-real-time-interactive-simulation-of-the-sars-cov-2-virus

visap.net/2023

www.myunginlee.com/covid

A team of MAT students led by MAT Chair and transLAB Director Professor Marcos Novak designed and installed an immersive and interactive exhibit called the "Synaptic Time Tunnel" for the 2023 SIGGRAPH Conference at the Los Angeles Convention Center.

Synaptic Time Tunnel, SIGGRAPH 2023.

Sponsored by Autodesk, the Synaptic Time Tunnel was a tribute to 50 years of innovation and achievement in the field of computer graphics and interactive techniques that have been presented at the SIGGRAPH conferences.

An international audience of more than 14,275 attendees from 78 countries enjoyed the conference and its Mobile and Virtual Access component.

Contributors:

Marcos Novak - MAT Chair and transLAB Director, UCSB
Graham Wakefield - York University, UCSB
Haru Ji - York University, UCSB
Nefeli Manoudaki - transLAB, MAT/UCSB
Iason Paterakis - transLAB, MAT/UCSB
Diarmid Flatley - transLAB, MAT/UCSB
Ryan Millet - transLAB, MAT/UCSB
Kon Hyong Kim - AlloSphere Research Group, MAT/UCSB
Gustavo Rincon - AlloSphere Research Group, MAT/UCSB
Weihao Qiu - Experimental Visualization Lab, MAT/UCSB
Pau Rosello Diaz - transLAB, MAT/UCSB
Alan Macy - BIOPAC Systems Inc.
JoAnn Kuchera-Morin - AlloSphere Research Group, MAT/UCSB
Devon Frost - MAT/UCSB
Alysia James - Department of Theater and Dance/UCSB

More information about the Synaptic Time Tunnel can be found in the following news articles:

Forbes.com: SIGGRAPH, Autodesk Take Time Tunnel Through Computer Graphics History on 50th Anniversary

Forbes.com: SIGGRAPH 2023 Highlights

PR Newswire: SIGGRAPH 2023 Conference Commemorates 50 Years of Innovations With Growth in Contributed Works and In-person Attendees

s2023.siggraph.org

translab.mat.ucsb.edu/S50_SynapticTimeTunnel

translab.mat.ucsb.edu

MAT Professor Emeritus Matthew Turk is a featured panelist in a discussion titled "Generative AI: Revolutionizing Today, Imagining Tomorrow", at the Cabrillo Pavillion in downtown Santa Barbara on October 18th from 5:30pm to 8pm. The event is moderated by MAT alumni Marco Pinter.

Delve deep into the transformative world of Generative AI in this enlightening panel discussion. Spanning the rich tapestry of academia to real-world applications in national and local industries, our distinguished panelists will shed light on current practices, novel innovations, and the future horizons of Generative AI. From the integration of this groundbreaking technology in sectors like healthcare and construction to its evolving academic research, join us for a comprehensive journey into the heart of AI's generative revolution. Witness firsthand the synthesis of theoretical insights with hands-on expertise, as we explore what the future holds for this dynamic field.

Link to the Event on Eventbrite

MAT alumni Andres Burbano has been selected as chair of the 51st annual ACM SIGGRAPH 2024 conference.

ACM SIGGRAPH is the premier conference and exhibition on computer graphics and interactive techniques. This year they celebrate their 50th conference and reflect on half a century of discovery and advancement while charting a course for the bold and limitless future ahead.

Burbano is a native of Pasto, Colombia and an associate professor in Universidad de los Andes’s School of Architecture and Design. As a contributor to the conference, Burbano has presented research within the Art Papers program (in 2017), and as a volunteer, has served on the SIGGRAPH 2018, 2020, and 2021 conference committees. Most recently, Burbano served as the first-ever chair of the Retrospective Program in 2021, which honored the history of computer graphics and interactive techniques. Andres received his PhD from Media Arts and Technology in 2013.

Read more from the ACM SIGGRAPH's website and this article on the ACMSIGGRAPH Blog.

The next ACM SIGGRAPH conference is in August 2023 and will be held in Los Angeles, California s2023.siggraph.org.

Past News

Alumni Testimonials

"With a deep interest in exploring sound, not only creatively, but technically and scientifically, I began my studies at MAT at UCSB. It was here, through academic discipline and perseverance, alongside professors and colleagues, that I was able to hone my skills and begin my successful career.

Years later, I am leading development of game audio and augmented reality technology at Apple.

The strong academic foundations that I gained in spatial audio, sound synthesis, and audio engine architecture have allowed me to achieve even greater success in my career. I will always be grateful to those who helped me take those first steps to success!"

David Thall
Lead Audio Programmer
Game and Augmented Reality Technology
Apple
"As a UCSB MAT Master's student (2002-2004), I benefited greatly from the required and elective courses taught by professional leaders with broad perspectives, both technical and humanistic. This helped me to improve my research skills, and to refine my academic vision, which culminated in a PhD (2009) from UCSB Electrical and Computer Engineering specializing in Digital Signal Processing. Since that time I have been employed at several academic institutions around Europe, where my broad technical and humanistic perspective has been unique and advantageous."

Bob L. Sturm, Associate Professor
School of Electronic Engineering and Computer Science
Royal Institute of Technology KTH
Stockholm, Sweden
"As one of the first graduates of the Media Arts and Technology PhD program, I have found the experiences provided by the MAT community invaluable to both my professional development and personal enjoyment of creating technology-based experiences, throughout life. I have been employed for more than 10 years now in the department of Architecture, Design and Media Technology at Aalborg University Copenhagen, where I teach both interaction design and real-time signal processing for audio and music performance. As Associate Professor, I guide students to create their own software and hardware tools, concepts, and practices for creating new sonic and embodied interactions, as well as running the Augmented Performance Lab. Together with my colleagues, I helped start the Sound and Music Computing masters program in 2014, and co-chaired the international New Interfaces for Musical Expression (NIME) conference in 2017."

Dan Overholt
Associate Professor
Aalborg Univeristy, Copenhagen
Denmark
"MAT is a rare graduate program in that it operates more like an creative ecosystem -- your work as a researcher is significant and connects to others. People care a lot about what you are doing, and invest thought into your own thinking. Throughout my eight years of study, from 2007 to 2015, and as a postdoctoral student afterwards, this supportive environment provided the technological, financial, and intellectual resources necessary for me to take risks in my work. As a result, I was able to dive deep into subjects I thought were off limits to my cognitive function, and achieve the great satisfaction of learning how to learn better. It is a testament to the culture fostered there that an experimental filmmaker such as myself entering the program with very basic programming skills can now make regular contributions to mathematics journals, work as a real-time computer graphics programmer in the DTLA arts district, be asked to consult on science fiction television dramas, and innovate novel geometrical algorithms of use in architecture. I think the MAT program at UCSB is a hidden gem -- you enter it with one direction in mind, and come out brilliantly refracted into a spectrum of possibilities."

Pablo Colapinto
Oblong Industries Inc & Independent Artist
Los Angeles
"The education and mentorship I received as an MAT student gave me the skills to discuss my work critically and in the context of the rich history of media arts; and has been pivotal in having my work accepted in exhibitions around the world, including Dubai, Montreal, Tehran, Hong Kong, New York and cities throughout the US. Further, my PhD in MAT was instrumental in securing my position as Curator of Interactive Media for MOXI, the Wolf Museum of Exploration and Innovation. In that capacity, almost half of the artists I have curated into MOXI have been MAT alumni, MAT faculty, MAT students, and distinguished media artists who I met as part of my work in the MAT program. My collaborations with MAT students and alumni are ongoing, and I continue to be grateful for the education, mentorship and community of the UCSB MAT program."

Marco Pinter
Computational Artist
Curator of Interactive Media, MOXI (The Wolf Museum of Exploration + Innovation)
Director of Software Innovation, InTouch Health

Hive	CHON	Angkasa
Melange	Lavin	Turn