Schedule MAT251 01F
Refs: Zannos, Viola, Murakami
1 Guest lecture: Ioannis Zannos iani@create.ucsb.edu
Issues of perceptual congruity/incongruity between modes of sense and interaction in multimedia (multimodal) installations. In natural environments perceptual traces are perceived as intrinsically coherent, e.g. one usually instinctively associate the sound coming from a source with the visual perception of the same object. In multimedia installations this intrinsic physical link is absent. How does one reconstruct it and what is the potential opened by artificial reconstruction?
Adaptivity of installation (or everyday) environments over different time scales. i.e. Installations that interact with the environment both immediately and according to data accumulated over long periods of time (hours, days, weeks)
Telepresence/augmented reality. I.e. Transcending the sense of physical confinement in an installation space by offering processed sensory input to exterior spaces or other locations.
There is a brief description of the installation at http://www.canon.co.jp/cast/artlab/pros3/infoMain.html
2 Bill Violas He Weeps for You, 1976
Video/sound installation
A drop of water emerging from a small brass valve is magnified by a video camera and projected on a large screen. The close-up image reveals that the viewer and a portion of the room are visible inside each forming drop. The drop swells and finally falls, and a loud sound is heard when it lands on an amplified drum. The entire room and persons in it are subject to the cadence of the falling drops, which continue in infinite repetition and reflection.
3 Taisuke Murakami Contact Water 1998 mixed realities
Explanation of the array used to keep track of changes
The sequence of TTC steps in the TTC Array.dir:
1 the following starts the trackthemcolors XTRA
in startmovie function:
open the trackthemcolors XTRA with
openTTC()
also initialize the array with
array = new (script "ArrayScript",gH,gV)
2 the following tells Trackthemcolors what x,y ratio to use in createGrid() function:
initialize the video input with
initTTC(w,h)
3 showing the video image on screen in the show video behavior attached to sprite 4
showTTC (50,300) -- puts the image on screen at 50 pixels from left, 300 from top)
4 Getting a reference data from the empty room: in get ref behavior atttached to sprite 3
getColors(TrackObj,gCurList) into gRefList
5 WHERE THE INTERACTIVE WORK IS DONE
in the scorescript in frame 4 (Control the grid) Continuous getting of current image and comparing it to the ref image list:
-- getting the current image and putting the r,g,b values into "theresult"
theresult=getColorsTTC(gCurList)
-- getting the difference in templist
templist= theresult-gRefList
-- averaging the r,g,b values into 1 as we are only interesetd in the gray values
average=(abs(templist[i]) + abs(templist[i+1]) + abs(templist[i+2]))/3
-- getting the current array value for this location
j=getcell(array,x,y) -- current value
-- if the difference is greater than tolerance then increase the value in the database array by iValue (which is currently preset for 6)
-- but only up to a max of 600
if abs(average)>tolerance then
if j<600 then j=j+iValue -- increase value
-- otherwise decrease by dValue (currently preset at 3)
else if j>5 then j=j-dValue -- decrease value
put updated value back into the array
setcell(array,x,y,j) -- change the value in the array
6 REMOVE OBJECTS FROM RAM AT END OF RUNNING THE MOVIE:
this takes place in the movie script in
stopmovie()
2 OCT.15 PRESENT TO THE GROUP A doctored VERSION OF the TTC ARRAY.DIR
Visiting Lecture Announcement from Prof. Matthew Turk:
Francis Quek, from the Computer Science and Engineering Dept. at Wright State University, is visiting on October 15th and 16th, and will be giving a talk at 3pm (see abstract below). Francis works in multimodal interaction, investigating gesture, speech, and gaze and how they can be used in human-computer interaction. He has multidisciplinary collaboration with people from several places working in psycholinguistics, natural language processing, anthropology, neurology, and other areas.
He is interested in talking with people at UCSB with potential common interests. I thought some of you would be interested. If you'd like to talk with him on Monday 10/15 or Tuesday 10/16, please let me know when you're available. Ditto for your students or others you think would be interested.
