Annual Review 2010 > Division of Information Systems

Computer Arts Laboratory

Michael Cohen

Professor

Pang Wai Man

Assistant Professor

Yannis Haralambous

Visiting Researcher

Most of the courses taken by engineers and computer science students emphasize scientific discipline and the accumulation of "truth." The Computer Arts Lab. activities include such technically objective factors, but also encourage original expression, subjectively motivated by aesthetics rather than "correctness," sometimes "putting the art before the course!" Unlike many other labs' activities that try to converge on a "right answer" sharable by everyone else, artistic disciplines encourage originality, in which the best answer is one that is like no one else's.

The Computer Arts Lab., through its resident Spatial Media Group 1 is researching projects including practical and creative applications of virtual reality and mixed (augmented, enhanced, hybrid, mediated) reality and virtuality; panoramic interfaces and spatially-immersive displays (especially stereotelephonics, spatial sound, and stereography); wearable and mobile applications, computing, and interfaces; and networked multimedia, with related interests in cve (collaborative virtual environments), groupware and cscw (computer-supported collaborative work); hypermedia; digital typography and electronic publishing; force-feedback displays; telecommunication semiotics (models of teleconferencing selection functions); information furniture; way-finding and navigation (including using a Segway personal transporter); entertainment computing; ubicomp (ubiquitous computing), calm (ambient), and pervasive technology. We are particularly interested in narrowcasting commands, conference selection functions for adjusting groupware situations in which users have multiple presence, virtually existing in more than one space simultaneously. We explore realtime interactive multimedia interfaces— auditory, visual, haptic, and multimodal:

Other activities:

We host an annual symposium, the Int. Symposium on Spatial Media,33 inviting experts to share their knowledge and passion regarding such themes as "Spatial Sound and Spatial Telepresence" ('01), "Magic in Math and Music" ('02), "Advanced Multimedia and Virtual Reality" ('03), "Spatial Sound" ('04), "Hearing and Sound Installations" ('05), "Sound, Audio, and Music" ('06), "Interactive Media, Security, and Stereography" ('06), "Music XML and the Structure of Swing, Understanding Color Media, Media Grid, and Visualization Tools" ('07), "Multimedia Computing" ('08), "Systems and Applications" ('09'10) and "Multimodal Interfaces" ('10'11).

Our lab sponsors several student performance circles, including the Yasakoi Dance Circle,34 and Disco Mix Club.35 We also sponsor a couple of other student circles, the Dual Boot (Ultimate Frisbee) Flying Disc Club,36 and the Furiten Mah Jongg Circle.37

We are working with Aizu Yougo Gakko,38 the special education school next to the university, to develop multimedia interfaces that can encourage and entertain students with special needs. We have consulted on deployment of switch-adapted media players, have deployed some iPad accessibility applications, and have developed a song selection program using a "step-scan, select" affordance.

Through the research & development, the deployment & integration, of stereographic, spatial sound, haptic, and mobile applications, including virtual and mixed reality, we nurture scientific and artistic interest in advanced computerhuman and humanhuman communication. Our ultimate domain is the exploration of interfaces and artifacts that are literally sensational.

Some relevant links:

Audio Courseware: http://sonic.u-aizu.ac.jp
Spatial Media: http://sonic.u-aizu.ac.jp/spatial-media/Videos/cohea.html
English: http://sonic.u-aizu.ac.jp/spatial-media/Videos/coheen.mpg
Japanese: http://sonic.u-aizu.ac.jp/spatial-media/Videos/cohejp.mpg
Multimedia and Virtual Reality Videos: http://sonic.u-aizu.ac.jp/spatial-media/Videos
Mobile control of rotary motion platform: http://sonic.u-aizu.ac.jp/spatial-media/Videos/keitai+Schaire2.mov
Dual Driving Simulator: http://sonic.u-aizu.ac.jp/spatial-media/Videos/DualDrivingSimulator.mov
"VMP My Ride": http://sonic.u-aizu.ac.jp/spatial-media/Videos/VMPMyRide.mp4
Mixed Reality Videos: http://sonic.u-aizu.ac.jp/spatial-media/mixedreality/VideoClips
Cluspi Control of Rotary Motion Platform: http://sonic.u-aizu.ac.jp/spatial-media/Videos/CLUSPI_demo-QT.mov
Sudden Motion Sensor Control of Collaborative Virtual Environment: http://sonic.u-aizu.ac.jp/spatial-media/Videos/SMS-CVE.mov
QuickTime Virtual Reality: http://sonic.u-aizu.ac.jp/spatial-media/QTVR/
U. of Aizu Panorama: http://sonic.u-aizu.ac.jp/spatial-media/QTVR/Aizu_Daigaku.mov
Object Movie: http://sonic.u-aizu.ac.jp/spatial-media/QTVR/shoe.mov
Hideo Noguchi + Akabeko: http://sonic.u-aizu.ac.jp/spatial-media/QTVR/Noguchi+Akabeko.mov
Rotational Degrees of Freedom: http://sonic.u-aizu.ac.jp/spatialmedia/QTVR/Rotational-DsoF.mov
Press and Mass Media Coverage:
Fukushima Minpo, June 10, 2010 http://www.u-aizu.ac.jp/~mcohen/ scrapbook/FukushimaMinpo-10.6.10.jpg
"Nikkei": Nihon Keizai Shimbun, Nov. 5, 2010 (p. 35) http://www. u-aizu.ac.jp/~mcohen/scrapbook/NihonKeizaiShimbun-2010-11-5p.35.png
"Switch", Teleview Fukushima, Jan. 4, 2011 http://gallery.me.com/ xilehence#101561/MAH04434-edited
Fukushima Minpo, February 18, 2011 (p. 9) http://www.minpo.jp/ view.phpview.php?pageId=4107&mode=0&classId=0&blockId=9789060& pageId=4107&mode=0&classId=0&blockId=9789060& newsMode=article
University Newspaper, Apr. 8, 2011 http://www.u-aizu.ac.jp/~mcohen/scrapbook/UniversityNewspaper-8.4.11.pdf
1sonic.u-aizu.ac.jp/spatial-media/Videos/coheen.mpg
2www.ubic-u-aizu.jp/shisetsu/kengaku.html
3www.u-aizu.ac.jp/~mcohen/spatial-media/groupware/clients/HKB/welcome-J.html, http:// www.ubic-u-aizu.jp/images/stories/pdf/coh-02-01.pdf
4sonic.u-aizu.ac.jp
5www.u-aizu.ac.jp/~mcohen/welcome/courses/AizuDai/graduate/Sound+Audio/syllabus.html
6web-int/~j-huang/Lecture/3DSound/3dsound.html
7www.u-aizu.ac.jp/official/curriculum/syllabus/3_E_000.html
8www.gimp.org
9www.u-aizu.ac.jp/~mcohen/welcome/courses/AizuDai/undergraduate/Computer_Music
10www.apple.com/ilife/garageband, www.pgmusic.com/band.htm
11sonic.u-aizu.ac.jp/~mcohen/welcome/courses/AizuDai/undergraduate/Computer_Music/Compositions/2009/
12www.u-aizu.ac.jp/~mcohen/welcome/courses/AizuDai/graduate/Computer_Music/syllabus.html
13www.u-aizu.ac.jp/~mcohen/welcome/courses/AizuDai/Mma.html
14www.u-aizu.ac.jp/~mcohen/spatial-media/stereograms.html
15www.chromatek.com
16mpropc4.u-aizu.ac.jp
17sonic.u-aizu.ac.jp/spatial-media/QTVR/
18www.u-aizu.ac.jp/~mcohen/welcome/courses/AizuDai/undergraduate/HI&VR/VirtualTour/
19www.u-aizu.ac.jp/~mcohen/welcome/publications/SMS-CVE.mov
20www.sensable.com/products-freeform-systems.htm
21www.zcorp.com/Products/3D-Printers/spage.aspx
22sonic.u-aizu.ac.jp/spatial-media/mixedreality/VideoClips/KuruKuru-pitcher-long.mov
23sonic.u-aizu.ac.jp/spatial-media/mixedreality/VideoClips/CITMixedReality_Demo.wmv
24sonic.u-aizu.ac.jp/spatial-media/mixedreality/VideoClips/keitai+Schaire2.mov
25openwonderland.org
26www.u-aizu.ac.jp/~mcohen/spatial-media/VMPMyRide
27www.thedesignium.com
28www.aizu.com
29www.gclue.com
30www.segway.com
31web-int.u-aizu.ac.jp/~mcohen/welcome/courses/AizuDai/undergraduate/HI"&VR
32www.alice.org
33www.u-aizu.ac.jp/~mcohen/welcome/ISSM/10-11/
34www.u-aizu.ac.jp/circles/yosakoi
35dmconline.co.cc
36www.u-aizu.ac.jp/circles/dualboot
37web-ext.u-aizu.ac.jp/circles/furiten
38www.aizu-sh.fks.ed.jp

Refereed Journal Papers

[mcohen-01:2010]

Juliän Villegas and Michael Cohen. Roughness Minimization Through Automatic Intonation Adjustments. JNMR: J. of New Music Research, 39(1):75-92, 2010.

http://www.tandf.co.uk/journals/nnmr. We have created a reintonation system that minimizes perceptual roughness of parallel sonorities as they are produced. Intonation adjustments are performed by finding, within a user-defined vicinity, a combination of fundamental frequencies that yields minimal roughness. The vicinity imposition limits pitch drift and eases realtime computation. Prior knowledge of the temperament and notes being played is not necessary for the operation of the algorithm. We test a proof of concept prototype adjusting equal temperament intervals reproduced with a harmonic spectrum towards pure intervals in realtime. Pitch drift of the rendered music is not prevented but limited. This prototype exemplifies musical and perceptual characteristics of roughness minimization by adaptive techniques. We discuss the results obtained, limitations, possible improvements, and future work.

[wmpang-01:2010]

Wai-Man Pang, Jing Qin, Yuqiang Lu, Yongming Xie, Chee-Kong Chui, and Pheng-Ann Heng. CUDA-Accelerated 3D Algebraic Computed Tomography Reconstruction with Motion Compensation. International Journal of Computer Assisted Radiology and Surgery, 6(2):187-199, March 2011.

Purpose To accelerate the simultaneous algebraic reconstruction technique (SART) with motion compensation for speedy and quality computed tomography reconstruction by exploiting CUDA-enabled GPU. Methods Two core techniques are proposed to fit SART into the CUDA architecture: (1) a ray-driven projection along with hardware trilinear interpolation, and (2) a voxel-driven back-projection that can avoid redundant computation by combining CUDA shared memory. We utilize the independence of each ray and voxel on both techniques to design CUDA kernel to represent a ray in the projection and a voxel in the back-projection respectively. Thus, significant parallelization and performance boost can be achieved. For motion compensation, we rectify each ray's direction during the projection and back-projection stages based on a known motion vector field. Results Extensive experiments demonstrate the proposed techniques can provide faster reconstruction without compromising image quality. The process rate is nearly 100 projections s − 1, and it is about 150 times faster than a CPU-based SART. The reconstructed image is compared against ground truth visually and quantitatively by peak signal-to-noise ratio (PSNR) and line profiles. We further evaluate the reconstruction quality using quantitative metrics such as signal-to-noise ratio (SNR) and mean-square-error (MSE). All these reveal that satisfactory results are achieved. The effects of major parameters such as ray sampling interval and relaxation parameter are also investigated by a series of experiments. A simulated dataset is used for testing the effectiveness of our motion compensation technique. The results demonstrate our reconstructed volume can eliminate undesirable artifacts like blurring. Conclusion Our proposed method has potential to realize instantaneous presentation of 3D CT volume to physicians once the projection data are acquired

[wmpang-02:2010]

Jing Qin, Wai-Man Pang, Yim-Pan Chui, Tien-Tsin Wong, and Pheng-Ann Heng. A Novel Modeling Framework for Multilayered Soft Tissue Deformation in Virtual Orthopaedic Surgery. Journal of Medical Systems, 34:261-271, June 2010.

Realistic modeling of soft tissue deformation is crucial to virtual orthopedic surgery, especially orthopedic trauma surgery which involves layered heterogenous soft tissues. In this paper, a novel modeling framework for multilayered soft tissue deformation is proposed in order to facilitate the development of ortho- pedic surgery simulators. A multilayered 3D mass-spring model is constructed based on the segmented Chinese Visible Human (CVH) dataset. We adopt a bilinear elasticity scheme to simulate the passive nonlinear biome- chanical properties of skin and skeletal muscle. An optimization approach is employed in conguring the parameters of the mass-spring model with experimental data in biomechanics literature as benchmarking references. The computation intensive physics simulation is accelerated by the newly released Physical Processing Unit (PPU), a consumer-level hardware tailor made for physically based computation. With an integration of high Send oprint requests to:? Present address: Insert the address here if needed quality volume visualization, our framework serves as an interactive and intuitive platform for virtual orthopedic surgery simulation. A prototype developed based on this framework as well as a series of experiments performed on it demonstrate the feasibility of the proposed framework in providing interactive and realistic tissue deformation for orthopedic surgery simulation.

[wmpang-03:2010]

Wai-Man Pang, Jing Qin, Yim-Pan Chui, and Pheng-Ann Heng. Fast Prototyping of Virtual Reality Based Surgical Simulators with PhysXenabled GPU. Transaction in Edutainment IV, pages 176-188, 2010.

We present our experience in fast prototyping of a series of important but computation-intensive functionalities in surgical simulators based on newly released PhysX-enabled GPU. We focus on soft tissue deformation and bleeding simulation, as they are essential but have previously been difficult to be rapidly prototyped. A multilayered soft tissue deformation model is implemented by extending the hardware accelerated mass-spring system (MSS) in PhysX engine. To ensure accuracy, we configure spring parameters in an analytic way and integrate a fast volume preservation method to overcome the volume loss problem in MSS. Fast bleeding simulation with consideration of both patient behavior and mechanical dynamics is introduced. By making use of the PhysX built-in SPH-based fluid solver with careful assignment of parameters, realistic yet efficient bleeding effects can be achieved. Experimental results demonstrate that our approaches can achieve both interactive frame rates and convincing visual effects even when complex models are involved.

[wmpang-04:2010]

Wing-Yin Chan, Dong Ni, Wai-Man Pang, Jing Qin, Yim-Pan Chui ajd Simon Chun-Ho Yu, and Pheng-Ann Heng. Learning Ultrasound-Guided Needle Insertion Skills through an Edutainment Game. Transactions on Edutainment IV, pages 200-214, 2010.

Ultrasound-guided needle insertion is essential in many of minimally invasive surgeries or procedures, such as biopsy, drug delivery, spinal anaesthesia, etc. Accurate and safe needle insertion is a difficult task due to the high requirement of hand-eye coordination skills. Many proposed virtual reality (VR) based training systems put their emphasis on realistic simulation instead of pedagogical efficiency. The lack of schematic training scenario leads to boredom of repetitive operations. To solve this, we present our novel training system with the integration of game elements in order to retain the trainees' enthusiasm. Task-oriented scenarios, time attack scenarios and performance evaluation are introduced. Besides, some state-of-art technologies are also presented, including ultrasound simulation, needle haptic rendering as well as a mass-spring-based needle-tissue interaction simulation. These works are shown to be effective to keep the trainees up with learning.

Refereed Proceedings Papers

[mcohen-02:2010]

Senaka Amarakeerthi, Rasika Ranaweera, and Michael Cohen. Speechbased Emotion Characterization Using Postures and Gestures in CVEs. In Proc. Int. Conf. on Cyberworlds, page (electronic proceedings), Singapore, oct 2010.

www3.ntu.edu.sg/SCE/cw2010. Collaborative Virtual Environments (CVEs) have gained an increasing popularity in past two decades. Most CVEs use avatar systems to represent each user logged into a CVE session. Some avatar systems are capable of expressing emotions with gestures, postures, and facial expressions. In previous studies, various approaches have been explored to convey emotional states to the computer, including voice, facial movements, and emotions. We propose a technique to extract emotions from the voice of a speaker and animate avatars to reflect extracted emotions in real-time. The system has been developed in "Open Wonderland," a Java-based open-source framework for creating collaborative 3D virtual worlds. In our prototype, six primitive emotional states – anger, dislike, fear, joy, sadness, and surprise – were considered. An emotion classification system, which uses short time log frequency power coefficients (LFPC) to represent features and hidden Markov models (HMMs) as the classifier, was modified to build an emotion classification unit. Extracted emotions were used to activate existing avatar gestures in Wonderland.

[mcohen-03:2010]

Hiromitsu Sato and Michael Cohen. Using Motion Capture for Realtime Augmented Reality Scenes. In Vitaly Kluev and Michael Cohen, editors, Proc. HC-2010: 13th Int. Conf. on Humans and Computers, pages 58-61, Aizu-Wakamatsu, December 2010.

http://sparth.u-aizu.ac.jp/hc2010, isbn 978-4-900721-01-2. The purpose of this study is improvement of real-time human-computer interface in using augmented reality. In this article, we suggest avatar animated in real-time with motion capture system and augmented reality. Development of ubiquitous computing is impressive in late years. Augmented reality is an instance of ubiquitous computing technology. Augmented reality is related to virtual reality, characterized by composition of real and virtual, and showing of virtual object as electronic information and giving the real world addition of the information. Because the computer's performance is continually upgrading, we can show complicated 3D models, and it is possible easily to add information to a computer such as a mobile telephone. Core technologies of augmented reality are position recognition, image recognition, and orientation recognition. In the past, we needed high performance machine in order to process these. Using modern hardware, we can process such data using mobile phone as well as notebook. Recently, Sekai Camera became famous as augmented reality software that runs on iPhone and Android. Assuming continual computer performance upgrade, there will appear a lot of augmented reality software we can use in the future. However, the means to affect information in augmented reality have not developed yet.

[mcohen-04:2010]

Juliän Villegas and Michael Cohen. "GABRIEL": Geo-Aware BRoad casting for In-Vehicle Entertainment and Localizability. In AES 40th Int. Conf. "Spatial Audio: Sense the Sound of Space", page (electronic proceedings), Tokyo, October 2010.

http://www.aes.org/events/40/. We explore the potential of modern locationaware multimedia applications to tour services and driver navigation advisories, the deployment of telematic resources through spatial sound. We have retrofitted a commercial tour bus with location-aware spatial sound advisories, delivered via wireless headphones (for passengers) and bone conduction headphones (for the driver). The prototype is intended to serve as a "proof of concept" as well as a testbed for anticipated future research in which geo-located bilingual tourist information, navigation instructions, and traffic advisories are rendered simultaneously.

[mcohen-05:2010]

Akira Inoue and Michael Cohen. Time-aware Geomedia Browsing Integration with Virtual Environment. In Vitaly Kluev and Michael Cohen, editors, Proc. HC-2010: 13th Int. Conf. on Humans and Computers, pages 47-54, Aizu-Wakamatsu, dec 2010.

http://sparth.u-aizu.ac.jp/hc2010, isbn 978-4-900721-01-2, We propose a pap application named "Human History Maps" to use data of human history information and MultiSound Manager as rich contents Map application. We provided Human History Maps that aims to learn more easily the historical famous person. Web services that combine map with person of history. It is service that makes tracks of a person's life visible on the map. The System memorizes the person that having met, the place, time, and so on. The system can visualize the connection of multiple people of the person and the person at the same time. Human History maps constructed to display Maps by Google Maps API that was developed and offered by Google. In that, we focus to the feature of Google Street View, one of the functions of Google Maps. And we provide and approach some extended feature of Google Street View. There are functions to combine street view with sound and to use AR (Augmented Reality) with which a user can control web applications interactively. The learning effect a rises when a user learns to use visualized history and recognized with a sound. As a result, it is possible to be interested in Human (person) of history for user that become not passive but more active to learn. Moreover, in the future, it will be possible to apply it for various services and the researches by recording in our data base, and sharing the data of Human History Maps among users.

[mcohen-06:2010]

Prabath Weerasinghe, Rasika Ranaweera, Senaka Amarakeerthi, and Michael Cohen. "Emo Sim": Expressing Voice-Based Emotions in Mobile

Interfaces. In Vitaly Kluev and Michael Cohen, editors, Proc. HC-2010: 13th Int. Conf. on Humans and Computers, pages 28-31, Aizu-Wakamatsu, dec 2010. http://sparth.u-aizu.ac.jp/hc2010, isbn 978-4-900721-01-2. Human interaction with mobile devices is currently a very active research area. Speech enriched with emotions is one of the major ways of exchanging ideas, especially via telephony. By analyzing a voice stream using a Hidden Markov Model (HMM) and Log Frequency Cepstral Coefficients (LFPC) based system, different emotions can be recognized by a separate system. Using a simple Java client, the recognized emotions are delivered to a sever as an index number. A mobile client then retrieves the emotion and displays it through colored icons. Each emotion is mapped to a particular color, as it is natural to use colors to represent various expressions. We believe that with the help of this application one could conceivably change one's way of talking or avoid chatting with somebody whose emotional state is negative!

[mcohen-07:2010]

Mamoru Ishikawa, Takeshi Matsuda, and Michael Cohen. Guitarto-MIDI Interface: Guitar Tones to MIDI Notes Conversion Requiring No

Additional Pickups. In Proc. AES 129th Conv., pages 28-31, San Francisco, nov 2010. Preprint 8209, http://www.aes.org/events/129/. Many musicians, especially guitarists (both professional and amateur), use effects processors. In recent years, a large variety of digital processing effects have been made available to consumers. Further, desktop music, the "lingua franca" of which is MIDI, has become widespread through advances in computer technology and DSP. Therefore, we are developing a "Guitar to MIDI" interface device that analyzes the analog guitar audio signal and emits a standard MIDI stream. Similar products are already on the market (such as the Roland GI-20 GK-MIDI Interface), but almost all of them need additional pickups or guitar modification. The interface we are developing requires no special guitar modification. We describe a prototype platformed on a PC that anticipates a self-contained embedded system.

[mcohen-08:2010]

Norbert Gyërbirë, Henry Larkin, and Michael Cohen. Spaced Repetition Tool for Improving Long-term Memory Retention and Recall of Collected Personal Experiences. In Proc. ACE, Int. Conf. on Advances in Computer Entertainment Technology, pages 28-31, Taipei, nov 2010.

http://ace2010.ntpu.edu.tw. A variety of electronic displays available for home use creates opportunities for intelligent applications. This paper presents a semipassive photo reviewing tool for consolidating memories of experiences utilizing personal picture libraries. A form of spaced repetition algorithm is used to create visual journeys which link photos together around a user-chosen central theme. Systematically reviewing images from positive personal experiences can be useful to remember significant events, as well as to balance out stressful events in our lives. The design exploits existing digital home displays and aims to improve usage of media collections.

[mcohen-09:2010]

Owen Noel Newton Fernando, Michael Cohen, and Adrian David Cheok. Multipresence-Enabled Mobile Spatial Audio Interfaces. In Proc. ICEC: Int. Conf. on Entertainment Computing, pages 434-436, Seoul, September 2010.

http://icec2010.or.kr/xe/. Mobile telephony offers an interesting platform for building multipresence-enabled applications that utilize the phone as a social or commercial assistant. The main objective of this research is to develop multipresence-enabled audio windowing systems for visualization, attention, and privacy awareness of narrowcasting (selection) functions in collaborative virtual environments (CVEs) for mobile devices such as 3rd- and 4th-generation mobile phones. Mobile audio windowing system enhances auditory information on mobile phones and encourages modernization of office- and mobile-based conferencing.

[mcohen-10:2010]

Norbert Gy}orbirë, Henry Larkin, and Michael Cohen. Long-term Memory Retention and Recall of Collected Personal Memories. In Computer Graphics (Proc. acm Siggraph 2010), pages 28-31, Los Angeles, July 2010.

A novel photo view and review personal information system is proposed to increase retention and recall of collected memories by utilizing the spacing effect in learning and presenting a context-aware selection of photos as the "learning material." The primary aim is to improve memory connections by automatically creating "visual photo journeys."

[mcohen-11:2010]

Norbert Gy}orbirë, Henry Larkin, and Michael Cohen. Collaborative Capturing of Significant Life Memories. In CHI Workshop "Know Thyself: Monitoring and Reflecting on Facets of One's Life", page (electronic proceedings), Atlanta, April 2010.

http://personalinformatics.org/chi2010. The continual advancement in technology and form-factor of personal devices have made possible "memory archiving" and "life logging," the recording of a person's life through wearable and ubiquitous computing technologies. The purpose of this research is to minimize the total data recorded while maximizing the number of most significant or user-preferred media. This media is captured automatically based on the person's arousal level and collaboratively employing nearby camera phones. The recording groupware application analyzes collective emotion for more accurate capture. Keywords: Collaborative life logging, significant memories, mobile groupware.

[mcohen-12:2010]

Rasika Ranaweera, Michael Cohen, and Michael Frishkopf. Virtual World Music— Music Browser in Wonderland. In iED: Immersive Education Initiative Boston Summit, page (electronic proceedings), Boston, April 2010.

http://mediagrid.org/summit/2010 Boston Summit program full.html, As immersive virtual environments and online music networks become increasingly popular, it behooves researchers to explore their convergence: groupware music browsers populated by figurative avatars. Collaborative virtual environments (CVEs), like Second Life, offer immersive experiential network interfaces to online worlds and media. We developed a virtual environment, based upon and similar to the "Music in Wonderland" proof-of-concept by Sun Microsystems, that enables a place where avatar-represented users can go to browse musical databases. Music is a medium that is used for a wide range of purposes in different situations in very different ways in real life. Different systems and interfaces exist for the broad range of needs in music consumption. Locating a particular recording is well supported by traditional search interfaces via metadata, but improving search techniques via different strategies is becoming a growing need. World music browser is a cylinder in which a transparent rectangular map of the world is coated, and tracks are placed according to origins of tracks, enabling location-aware browsing. Hemisphere is a well-known structure which can be easily defined and projected in the 3D environment either declaratively or procedurally. Since Wonderland does not permit recumbency, the avatar always stands vertically. A sphere limits angle of view more and more as objects approach the poles; but on the other hand a cylinder can be effectively used to indicate geographical location using a suitable projection. This would be ideal as the avatars in Wonderland can fly! The avatar can move in to cylinder, and click to listen track samples, if s/he is inside the hearing range. The selected tracks are highlighted and also multiple tracks can be heard at the same time when the user moves near from one track to another and in between sound may always overlap. The system is collaborative: multiple users can hear the same music together, and they can hear each others' speech via voice chat. Wonderland supports stereophonic audio communication, with a headset, participants get realtime immersive stereo audio with distance attenuation, means the voices of others present in Wonderland will become louder when approached and decays when apart. Audio spatialization supports stereo (2 channels) and only left/right positioning including delay effect. The stereo effect is done at the server side by the Audio Treatment Component, two interleaved channels are sent in each packet. One channel is delayed by 0-0.63 ms, depending on the location of the source relative to the receiver. As this is done on the server it is possible to mix many audio sources appropriately for each receiver which also reduces the required bandwidth. Technically, the music collection is an XML file which keeps the list of albums along with the tracks (songs) as children nodes. Using SAX (Simple API for XML) parser, which invokes callback methods as it reads XML, generic, artistic, and geographic details about each album can be parsed and displayed in client browser.

[mcohen-13:2010]

Norbert Gy}orbirë, Henry Larkin, and Michael Cohen. Collaborative Capturing of Significant Life Memories. In CHI Workshop "Know Thyself: Monitoring and Reflecting on Facets of One's Life", page (electronic proceedings), Atlanta, April 2010.

http://personalinformatics.org/chi2010. The continual advancement in technology and form-factor of personal devices have made possible "memory archiving" and "life logging," the recording of a person's life through wearable and ubiquitous computing technologies. The purpose of this research is to minimize the total data recorded while maximizing the number of most significant or user-preferred media. This media is captured automatically based on the person's arousal level and collaboratively employing nearby camera phones. The recording groupware application analyzes collective emotion for more accurate capture. Keywords: Collaborative life logging, significant memories, mobile groupware.

[mcohen-14:2010]

Juliän Villegas, Michael Cohen, Ian Wilson, and William Martens.

Influence of Roughness on Preference Regarding Musical Intonation. In 128th Conv. of the Audio Engineering Society, page Convention Paper 8017, London, May 2010. http://www.aes.org/events/128/. An experiment to compare the acceptability of three different music fragments rendered with three different intonations is presented. These preference results were contrasted with those of isolated chords also rendered with the same three intonations. The least rough renditions were found to be those using Twelve-Tone Equal-Temperament (12-tet). Just Intonation (ji) renditions were measured as the roughest and least preferred. A negative correlation between preference and psychoacoustic roughness was also found.

[mcohen-15:2010]

Kyoko Katono and Michael Cohen. DollyZoom Camera Perspective with Alice. In Vitaly Kluev and Michael Cohen, editors, Proc. HC-2010: 13th Int. Conf. on Humans and Computers, pages 55-57, Aizu-Wakamatsu, December 2010.

http://sparth.u-aizu.ac.jp/hc2010, isbn 978-4-900721-01-2. This paper describes achievement of the "Vertigo" cinematographic effect in "machinima" software that can make 3D animation named "Alice." By composing virtual camera gestures of zoom in and zoom out with camera motion, "dolly zoom effect" can be realized.

[mcohen-16:2010]

Rasika Ranaweera, Michael Cohen, and Michael Frishkopf. Virtual World Music— Music Browser in Wonderland. In iED: Immersive Education Initiative Boston Summit, page (electronic proceedings), Boston, April 2010.

http://mediagrid.org/summit/2010 Boston Summit program full.html, As immersive virtual environments and online music networks become increasingly popular, it behooves researchers to explore their convergence: groupware music browsers populated by figurative avatars. Collaborative virtual environments (CVEs), like Second Life, offer immersive experiential network interfaces to online worlds and media. We developed a virtual environment, based upon and similar to the "Music in Wonderland" proof-of-concept by Sun Microsystems, that enables a place where avatar-represented users can go to browse musical databases. Music is a medium that is used for a wide range of purposes in different situations in very different ways in real life. Different systems and interfaces exist for the broad range of needs in music consumption. Locating a particular recording is well supported by traditional search interfaces via metadata, but improving search techniques via different strategies is becoming a growing need. World music browser is a cylinder in which a transparent rectangular map of the world is coated, and tracks are placed according to origins of tracks, enabling location-aware browsing. Hemisphere is a well-known structure which can be easily defined and projected in the 3D environment either declaratively or procedurally. Since Wonderland does not permit recumbency, the avatar always stands vertically. A sphere limits angle of view more and more as objects approach the poles; but on the other hand a cylinder can be effectively used to indicate geographical location using a suitable projection. This would be ideal as the avatars in Wonderland can fly! The avatar can move in to cylinder, and click to listen track samples, if s/he is inside the hearing range. The selected tracks are highlighted and also multiple tracks can be heard at the same time when the user moves near from one track to another and in between sound may always overlap. The system is collaborative: multiple users can hear the same music together, and they can hear each others' speech via voice chat. Wonderland supports stereophonic audio communication, with a headset, participants get realtime immersive stereo audio with distance attenuation, means the voices of others present in Wonderland will become louder when approached and decays when apart. Audio spatialization supports stereo (2 channels) and only left/right positioning including delay effect. The stereo effect is done at the server side by the Audio Treatment Component, two interleaved channels are sent in each packet. One channel is delayed by 0-0.63 ms, depending on the location of the source relative to the receiver. As this is done on the server it is possible to mix many audio sources appropriately for each receiver which also reduces the required bandwidth. Technically, the music collection is an XML file which keeps the list of albums along with the tracks (songs) as children nodes. Using SAX (Simple API for XML) parser, which invokes callback methods as it reads XML, generic, artistic, and geographic details about each album can be parsed and displayed in client browser.

[mcohen-17:2010]

Yuya Sasamoto and Michael Cohen. Spatial Sound Control with Yamaha Tenori-On. In Vitaly Kluev and Michael Cohen, editors, Proc. HC-2010: 13th Int. Conf. on Humans and Computers, pages 62-65, AizuWakamatsu, dec 2010.

http://sparth.u-aizu.ac.jp/hc2010, isbn 978-4-900721-01-2. In this research, we explore control of spatial sound using the Yamaha Tenori-On and the University of Aizu Busi- ness Innovation Center (UBIC) 3D Theater speaker array. The TenoriOn is a new type of musical interface that combines control and display functions in a matrix of 16 × 16 LED buttons. The 16× 16 Tenori-On matrix can be operated easily and intuitively, so that anyone can begin playing the instrument right away. Light animation is generated in synchronization with the performance and sound.

[wmpang-05:2010]

Wai-Man Pang. Towards Fast Gabor Wavelet Feature Extraction for Texture Segmentation by Filter Approximation. In Proceeding of 9th IEEE/ACIS International Conference on Computer and Information Science (ICIS2010), pages 252-257, 2010.

Abstract--Gabor wavelet transform is one of the most effective feature extraction techniques for textures. As the Gabor wavelets are believed to be rather consistent to the response of Human Vision System (HVS), and many successful examples are reported in the areas of texture analysis. However, computational complexity of the feature extraction is still high even for computers nowadays, especially large sized image is involved. This paper attempts to break through the bottleneck in the whole extraction process, that is to accelerate the convolutions by approximating the originally non-separable Gabor filter kernels to separable ones. Although the final computed features are not exactly the same as original ones, we prove that acceptable results can be achieved for segmentation purpose. While the acceleration ratio is as satisfactory as a gain of about 30% in time in the worst case with a MATLAB implementation.

[wmpang-06:2010]

Xinmin Tang, Jing Qin, and Wai-Man Pang. Automatic and Accurate 3D Face Registration under the Guidance of Intra-class Difference Measurement. In Proceeding of IEEE TENCON 2010, pages 173 178, 2010.

The accuracy of 3D face registration algorithm greatly influences the effect of 3D face recognition. While a lot of efforts have been dedicated in developing automatic and accurate registration methods, these methods usually cannot be compared and evaluated on a fair basis because of the lack of a standard quantitative measurement. In this paper, we propose a practical quantitative analysis method based on intra-class difference to evaluate the accuracy of face registration methods, and apply it to guide the procedures of an automatic nose symmetry plane (NSP) method. After every step, we calculate the mean and standard deviation (STD) of intraclass pose differences for all involved face images to assess the effect of this step and determine how to further improve the accuracy in the following steps. Extensive experiments have been conducted using the FRGC (V1.0 and V2.0) benchmark 3D face dataset to demonstrate the feasibility of our guided registration method.

[wmpang-07:2010]

Jing Qin, Wai-Man Pang, Binh P. Nguyen, Dong Ni, and Chee-Kong Chu. Particle-based Simulation of Blood Flow and Vessel Wall Interactions in Virtual Surgery. In Proceeding of Symposium on Information and Communication Technology 2010 (SoICT2010), pages 365-368, 2010.

We propose a particle-based solution to simulate the interactions between blood flow and vessel wall for virtual surgery. By coupling two particle-based techniques, the smoothed particle hydrodynamics (SPH) and mass-spring model (MSM), we can simulate the blood flow and deformation of vessel seamlessly. At the vessel wall, particles are considered as both boundary particles for SPH solver and mass points for the MSM solver. We implement an improved repulsive boundary condition to simulate the interactions. The computation of blood flow dynamics and vessel wall deformations are performed in an alternating fashion in every time step. To ensure realism, parameters of both SPH and MSM are carefully configured. Experimental results demonstrate the potential of the proposed method in providing real-time and realistic interactions for virtual vascular surgery systems.

[wmpang-08:2010]

Ka-Wai Ng, Hon-Cheng Wong, Un-Hong Wong, and Wai-Man Pang. Probe-volume: An exploratory volume visualization framework. In 3rd International Congress on Image and Signal Processing (CISP), 2010, pages 2392 2395, 2010.

Direct volume rendering has been an important technique for visualizing volume data. An unobstructed view on the inspected features of interest can help the user to get the insight of the volume data being explored. In this paper an exploratory volume visualization framework named Probe-Volume is presented. Probe-Volume provides a moveable and sizeable probing box for the user to examine an interested volumetric space of the data enclosed by the box. High-quality volume rendered results with such a feature can be achieved efficiently by providing two different user specified transfer functions for regions inside and outside of the probing box. In addition, our framework allows the user to visualize the volume with four rendering modes, namely, Full, MIP, MIDA, and Contour. Combining with the probing box, the user can select different rendering effects for regions included and excluded by the box in order to improve the visual effects. Experimental results show that Probe-Volume can provide informative direct volume rendered images of raw volume data efficiently and effectively.

[wmpang-09:2010]

Wai-Man Pang and Hon-Cheng Wong. Compression of Pre-computed Per-pixel Texture Features using MDS. In Proceeding of 28th Picture Coding Symposium (PCS2010), pages 390 393, December 2010.

There are many successful experiences on employing texture analysis to improve the accuracy and robustness on image segmentation. Usually, a per-pixel based texture analysis is required, this involves intensive computation especially for large images. While, precomputation and storing of the texture features involves large file space which is not cost effective. To adopt to this novel needs, we propose in this paper the use of multidimensional scaling (MDS) technqiue to reduce the size of perpixel texture features of an image, while preserving the textural discrminiability for segmentation. As per-pixel texture features will create very large dissimilarity matrix, and make the solving of MDS intractable. A sampling-based MDS is therefore introduced to tackle the problem with a divide-and-conquer approach. A compression ratio of 1:24 can be achieved with an average error lower than 7%. Preliminary experiments on segmentation using the compressed data show satisfactory results as good as using the uncompressed features. We foresee that such a method will enable texture features to be stored and transferred more effectively on low processing power devices or embedded system like mobile phones.

[wmpang-10:2010]

J. Qin, C.-K. Chui, J. Zhang, T. Yang, W.M. Pang, S.H. Teoh, and V. Sudhakar. Wall stress analysis of non-stented/stented abdominal aortic aneurysm based on fluid structure interaction simulation. In Proceeding of 24th International Congress on Computer Assisted Radiology and Surgery (CARS 2010), pages 154-155, 2010.

Purpose: To estimate the wall stress distribution of non-stented and stented Abdominal Aortic Aneurysm (AAA) based on fluid-structure interaction (FSI) simulation. Methods: The 3D geometric models of AAA are reconstructed from computed tomography angiographic (CTA) images. A combined logarithm and polynomial strain energy equation is applied to model the elastic properties of arterial wall while a novel reduced quasi-linear viscoelastic model is proposed to describe the strain dependent relaxation behavior. The blood flow across the aneurysm is modeled as an incompressible laminar flow. The obtained pressure of blood flow is applied as a load on the AAA mesh with and without stent-graft and then the wall stress distribution is calculated by FSI solver equipped in ANSYS. Results: A series of in vitro experiments including uniaxial tensile tests and relaxation tests are conducted on pieces of human abdominal arteries. Both the combined logarithm and polynomial model and the modified reduced relaxation function fit the experimental data well. Two sets of geometric models with different shapes of AAA and stent-graft are employed to evaluate the proposed method. Our simulation results are consistent with the diagnosis of experienced radiologists. Conclusions: The proposed method can be used as both a new criterion in evaluating the rupture risk of AAA and a tool for preoperative planning of Endovascular Aneurysm Repair (EVAR). It also has potential to be applied in designing an effective patient-specific stent-graft.

Unrefereed Papers

[mcohen-18:2010]

Michael Cohen. Keynote Address: The Future of Immersive Education— Virtual Worlds, Simulators, Games, and Augmented/Mixed Reality in K12 and Higher Education. In FutureCampus Forum, Singapore, May 2010.

http://www.futuregov.net/events/futurecampus-forum-2010/. Immersive education has the potential to redefine the learning space, combining 3D and virtual reality technology with digital media to immerse and engage students. This keynote reviews the power of experiential learning and Dale's "Pyramid of Learning," outlines the potential of multimedia courseware and digital interactive story-telling, and describes "Alice," object-oriented programming environment for `machinema,' and "Project Wonderland," a "Second Life"-style groupware for conferencing, chat spaces, distance learning and immersive virtual learning environments.

[mcohen-19:2010]

Michael Cohen and Juliän Villegas. From Whereware to Whence- and Whitherware: Augmented Audio Reality for Position-Aware Services. In ISVRI: Proc. Int. Symp. on Virtual Reality Innovations, Singapore, March 2011.

http://isvri2011.org, Since audition is omnidirectional, it is especially receptive to orientation modulation. Position can be defined as the combination of location and orientation information. Location-based or location-aware services do not generally require orientation information, but position-based services are explicitly parameterized by angular bearing as well as place. "Whereware" suggests using hyperlocal georeferences to allow applications location-awareness; "whenceand whitherware" suggests the potential of position-awareness to enhance navigation and situation awareness, especially in realtime high-definition communication interfaces, such as spatial sound augmented reality applications. Combining literal direction effects and metaphorical (remapped) distance effects in whence- and whitherware position-aware applications invites over-saturation of interface channels, encouraging interface strategies such as audio windowing, narrowcasting, and multipresence.

[mcohen-20:2010]

Juliän Villegas and Michael Cohen. Hrir~: Modulating Range in Headphone-Reproduced Spatial Audio. In VRCAI: Proc. of the 9th Int. Conf. on Virtual-Reality Continuum and Its Applications in Industry, Seoul, December 2010.

isbn 978-1-4503-0459-7, http://vrcai2010.org. Hrir , a new software audio filter for Head-Related Impulse Response (HRIR) convolution is presented. The filter, implemented as a Pure-Data object, allows dynamic modification of a sound source's apparent location by modulating its virtual azimuth, elevation, and range in realtime, the last attribute being missing in surveyed similar applications. With hrir users can virtually localize monophonic sources around a listener's head in a region delimited by elevations between [−40; 90]ffi, and ranges between [20, 160] cm from the center of the virtual listener's head. An application based on hrir is presented to illustrate its benefits.

[mcohen-21:2010]

Michael Cohen. Under-explored Dimensions in Spatial Sound. In VRCAI: Proc. of the 9th Int. Conf. on Virtual-Reality Continuum and Its Applications in Industry, Seoul, December 2010.

isbn 978-1-4503-0459-7, http://vrcai2010.org. An introduction to spatial sound in the context of hypermedia, interactive multimedia, and virtual reality is presented. Basic principals of relevant physics and psychophysics are reviewed (itds: interaural time differences, iids: interaural intensity differences, and frequencydependent attenuation capturable by transfer functions). % especially including acoustical intensity, spatial hearing, Doppler Shift, and hrtfs. Modeling of sources and sinks (listeners) elaborates such models to include such as intensity, radiation, % auditory source width, distance attenuation & filtering, and reflections & reverberation. %, and refraction around obstacles. Display systems— headphones and headsets, loudspeakers, nearphones, stereo, home theater and other surround systems, discrete speaker systems, speaker arrays, % [systems, discrete speaker systems, speaker arrays, % [?], wfs (wave field synthesis), ], wfs (wave field synthesis), and spatially immersive displays— are described. Distributed applications are surveyed, including stereotelephony, chat-spaces, and massively multiplayer online role-playing games (mmorpgs), with references to immersive virtual environments.

[mcohen-22:2010]

Aaron Walsh, Nicole Yankelovich, Michael Gardner, and Michael Cohen. Panel: The Future of Immersive Education— Virtual Worlds, Simulators, Games, and Augmented/Mixed Reality in K12 and Higher Education. In iED: Immersive Education Initiative Boston Summit, Boston, April 2010. http://mediagrid.org/summi/2010 Boston Summit program full.html.

[mcohen-23:2010]

Juliän Villegas and Michael Cohen. Workshop: Spatial Sound and Entertainment Computing. In ICEC: Int. Conf. on Entertainment Computing, Seoul, September 2010.

http://icec2010.or.kr/xe/. This half-day tutorial introduces the theory and practice of spatial sound for entertainment computing, including psychophysical (psychoacoustic) basis of spatial hearing; outlines the mechanism for creating and displaying spatial sound the hardware and software used to realize such systems and display configurations; and reviews some applications of spatial sound to entertainment computing, especially multimodal interfaces featuring spatial sound. Many case studies reify the explanations; animations, videos, and live demonstrations are featured. %Chromastereoptic eyewear will be distributed to each attendee. Participants in the course can expect to acquire a solid introduction to spatial sound and a survey of some applications of spatial sound to entertainment computing, hopefully stimulating some new ideas that they can take back to their labs and hack on their own.

Chapters in Book

[mcohen-24:2010]

Juliän Villegas and Michael Cohen. Principles and Applications of Spatial Hearing, chapter Mapping Musical Scales Onto Virtual 3d Spaces. World Scientific, 2010.

eisbn 978-981-4299-31-2, http://eproceedings.worldscinet.com/9789814299312/ 9789814299312 005%6.html, http://www.worldscibooks.com/lifesci/7674.html. We introduce an enhancement to the Helical Keyboard, an interactive installation displaying three-dimensional musical scales aurally and visually. The Helical Keyboard features include tuning stretching mechanisms, spatial sound, and stereographic display. The improvement in the audio display is intended to facilitate pedagogic purposes by enhancing user immersion in a virtual environment. The newly developed system allows spatialization of audio sources controlling the elevation and azimuth angles at a fixed range. In this fashion, we could overcome previous limitations on the auditory display of the Helical Keyboard, for which we heretofore usually displayed only azimuth.

Academic Activities

[mcohen-25:2010]

Michael Cohen, October 2011.

Program Committee, NIME 2011 (Int. Conf. on New Instruments for Musical Expression), http://www.nime2011.org

[mcohen-26:2010]

Michael Cohen, October 2011.

Program Committee, IFIP ICEC 2011 (Int. Conf. on Entertainment Computing), http://www.icec2011.org

[mcohen-27:2010]

Michael Cohen, March 2011.

Co-chair, IEEE 3DUI (Symp. 3D User on Interfaces) Grand Contest, http://conferences.computer.org/3dui/3dui2011/cfp-contest.html

[mcohen-28:2010]

Michael Cohen, March 2010-11.

Executive Committee, IEEE Computer Society Technical Committee on ComputerGenerated Music

[mcohen-29:2010]

Michael Cohen, 2010-11. Voting Member, IEEE MMTC (Multimedia Communications Technical Committee)

[mcohen-30:2010]

Michael Cohen, Dec. 2010.

Program Committee, HC-2010: Thirteenth Int. Conf. on Human and Computer (Hamamatsu and Aizu-Wakamatsu and Hamamatsu), http://sparth.u-aizu.ac.jp/hc2010

[mcohen-31:2010]

Michael Cohen, September 2010.

Program Committee, IEEE ICEC 2010 (Int. Conf. on Entertainment Computing), http://www.icec2010.or.kr

[mcohen-32:2010]

Michael Cohen, October 2010.

Program Committee, IEEE VSMM (Int. Conf. on Virtual Systems and Multimedia), http://www.vsmm2010.or.kr

[mcohen-33:2010]

Michael Cohen, 2010.

Member, Audio Engineering Society Technical Committee on Spatial Sound

[mcohen-34:2010]

Michael Cohen, 2010-11. Reviewer and Scientific Committee, J. of Virtual Reality and Broadcasting

[mcohen-35:2010]

Michael Cohen, 2010.

Reviewer, Computer Music Journal, http://www.mitpressjournals.org/cmj, http://www.computermusicjournal.org

Ph.D., Master and Graduation Theses

[mcohen-36:2010]

Sasamoto Yuya (s1150100). Graduation Thesis: "Spatial Sound Control with the Yamaha Tenori-On", School of Computer Science and Engineering, March 2011.

Thesis Adviser: Michael Cohen

[mcohen-37:2010]

Saze Shougo (s1150101). Graduation Thesis: "Augmented Reality Interface for Spatial Sound", School of Computer Science and Engineering, March 2011.

Thesis Adviser: Michael Cohen

[mcohen-38:2010]

Shimizu Shinichirou (s1150114). Graduation Thesis: "Setting Rigging for Maya with a MEL Scrip", School of Computer Science and Engineering, March 2011.

Thesis Adviser: Michael Cohen

[mcohen-39:2010]

Hiromitsu Sato (m5131106). Master Thesis: "Using Motion Capture for Realtime Augmented Reality Scenes", Graduate School of Computer Science and Engineering, March 2011.

Thesis Adviser: Michael Cohen

[mcohen-40:2010]

Shun Shiratori (s1150117). Graduation Thesis: "Comparison of Images Generated by Mentalray and RenderMan", School of Computer Science and Engineering, March 2011.

Thesis Adviser: Michael Cohen

[mcohen-41:2010]

Kyoko Katouno (s1150061). Graduation Thesis: "ZoomDolly Camera Perspective with Alice", School of Computer Science and Engineering, March 2011.

Thesis Adviser: Michael Cohen

[mcohen-42:2010]

Mamoru Ishikawa (m5131103). Masters Thesis: "Dancing Wiper in a Driving Simulator", School of Computer Science and Engineering, March 2011.

Thesis Adviser: Michael Cohen

[mcohen-43:2010]

Norbert Gyërbirë (d8092101). Doctoral Thesis: Novel Interaction Methods for Recording and Recollecting Significant Personal Experiences", Graduate School of Computer Science and Engineering, March 2011.

Adviser: Michael Cohen

[mcohen-44:2010]

Akira Inoue Akira (m5131102). Masters Thesis: "Time-aware Geomedia Browsing Integration with Virtual Environments", Graduate School of Computer Science and Engineering, March 20 11.

Thesis Adviser: Michael Cohen