Human Interface Laboratory

/ Masahide Sugiyama / Professor
/ Michael Cohen / Associate Professor
/ Susantha Herath / Associate Professor
/ Minoru Ueda / Assistant Professor

Using our communication channels (sense organs: ears, mouth, eyes, nose, skin, etc) we can communicate each other, including between human and human, human and machine, and human and every information sources. Because of disability of the above channels in software or hardware sense, sometimes it becomes to be difficult for human to communicate. Research area of Human Interface Laboratory covers enhancement and generation of various human interface channels.

In order to advance the above research on human interface, we adopt the following research principle:

• Theoretical Our target is human interface and our study has possibility to do try-and-error, heuristic, too practical business. Based on our experimental results, and experiences we try to establish the theory, unified insight, generalization and analytical viewpoints.
• Practical Our target is not theory generation for theory. We extract the concept, theory in order to clarify in experimental and quantitative viewpoint.

We organized second workshop IWHIT98 on Nov. 11th, 12th and 13th (International Workshop on Human Interface Technology 1998) which was sponsored by the International Affairs Committee of the University of Aizu. The workshop had 5 sessions (1.Object Location and Tracking in Video Data, 2.Subjective Factors in Handling Images, 3.Visual Interfaces, 4.Visual and Body Perception, 5.Tools for Language Generation; 15 lectures).

We promoted 5 SCCPs for students (``Speech Processing and Multimedia", ``Sign Language Processing System'', ``GAIA -- Planet Management'', ``Computer Music'', ``Aizu Virtual City on InterNet") and 2 Research Projects (``Object Location and Tracking in Video Data'', ``Spatial Media: Sound Spatializiation''). We received 4 commissioned research funds; IPA on ``Development of Japanese Dictation Software'' , HITOCC on ``Study on Computer Security using Speaker Recognition", Fukushima Prefectural Foundation for the advancement of Science and education on ``Environment computer activity project'', Telecommunication Advancement Organization of Japan Fund on ``Sign Language Communication Between Different Languages''.

We exhibited our research activities in the open campus in University Festival (Oct 31st, Nov.1st) and Fukushima Sangyo Fair (Nov. 29th and 30th). We promoted Lab Open House for Freshmen on April 3rd.

On our research activity we presented 6 papers in academic journals and 10 refereed papers in International Conferences.

One of members organized working group on ``Blind and Computer" and about 30 people attended to the working group and received the support from NHK Wakaba Fund.

We have the homepage of Human Interface Lab to open our research and education activities to the world.

http://www.u-aizu.ac.jp/labs/sw-hi/.

Refereed Proceeding Papers

1. T. Muramoto and M. Sugiyama. Visual and Audio Segmentation for Video Streams. Proc. of ICME2000, pp. 1547-1550, IEEE, Aug. 2000.

2. K. Watanabe, N. Fukada and M. Sugiyama. Design of Video Caption Markup Language VCML and Development of VCML Player. Proc. of ICME2000, pp. 163-166, IEEE, Aug. 2000.

3. M. Sugiyama. Model Based Voice Decomposition Method. Proc. of ICSLP2000, pp. PAe(16,17)-J-09, Oct. 2000.

4. T. Uchida, M. Yamashita and M. Sugiyama. Voice/Music Segmentation using Cepstrum Flux. Technical Report of Speech, editor: SP2000-17, pp. 9-16, ASJ/IEICE, IEICE, June 2000.

5. T. Narita and M. Sugiyama. Algorithms for Fast Retrieval of Music. Technical Report of Speech, editor: SP2000-16, pp. 1-8, ASJ/IEICE, June 2000.

6. T. Narita and M. Sugiyama. Study on Fast Music Retrieval, -- Evaluation of Normalization of acoustic distortion and Retrieving Efficiency. Technical Report of Hearing, editor: MA00-41, pp. 1-8, ASJ/IEICE, June 2000.

7. T. Akatsuka and M. Sugiyama. Generation of Iroha-Uta using Frequency Distribution of Kana. Proc. of IPSJ Spring Meeting, pp. 1Q-6, IPSJ, Mar. 2001.

8. T. Suzuki and M. Sugiyama. Improvement of VCML and Player. Proc. of IPSJ Spring Meeting, pp. 7Q-02, IPSJ, Mar. 2001.

9. T. Akatsuka and M. Sugiyama. Generation of Iroha-Uta. Proc. of ECEI2000, pp. 2H-24, ICEI, Aug. 2000.

10. T. Suzuki, K. Kaneda and M. Sugiyama. Improvement of VCML displaying Video Caption. Proc. of ECEI2000, pp. 2H-24, ICEI, Aug. 2000.

11. T. Uchida and M. Sugiyama. Construction of CampusWave Sound Database. Proc. of ECEI2000, pp. 2A-6, ICEI, Aug. 2000.

12. T. Imai and M. Sugiyama. Sound Effect Toolkit. Proc. of ECEI2000, pp. 2A-7, ICEI, Aug. 2000.

13. T. Muto and M. Sugiyama. Model Based Voice Decomposition Method under Time Constraint. Proc. of ASJ, editor: ASJ, pp. 3-3-21, 135-136, ASJ, Mar. 2001.

14. K. Kaneda and M. Sugiyama. Detection of Laughter for Auditory Scene in Caption Display. Proc. of ASJ, editor: ASJ, pp. 3-P-3, 165-166, ASJ, Mar. 2001.

15. T. Narita and M. Sugiyama. Music Retrieval using Power Information. Proc. of ASJ, editor: ASJ, pp. 3-8-4, 145-146, ASJ, Mar. 2001.

16. S. Takeuchi, M. Yamashita, T. Uchida and M. Sugiyama. Optimization of Speech/Music Segmentation. Proc. of ASJ, editor: ASJ, pp. 3-P-1, 169-170, ASJ, Mar. 2001.

17. K. Watanabe and M. Sugiyama. Segmentation of the Long Duration Speech Data into the Reading Segments. Proc. of ASJ, editor: ASJ, pp. 3-P-2, 167-168, ASJ, Mar. 2001.

18. Takashi Mikuriya, Masataka Shimizu and Michael Cohen. A Collaborative Virtual Environment Featuring Multimodal Information Controlled by a Dynamic Map. Proc. HC2000: Third Int. Conf. on Human and Computer, 3D Forum, pp. 77--80, 2000.

    In recent years, there has been extensive research on collaborative virtual environments. It is important to construct a robust system for developing a multimodal multiclient environment. In this paper, the basic planned structure of a virtual collaborative environment is described. To realize the system, {\sc Java} with the {\sc jsdt} (Java Shared Data Toolkit) is being used to leverage off its ease of constructing collaborative virtual environments. The description of the system which has a potential for multicasting is expressed. A {\sc 2.5d} dynamic map client is presented as an example of an interface. Legacy clients can also be integrated using a proxy between server and client. The framework for interacting shared space with multi-clients multi-users is described. We are developing a suite of multicasting clients to support audio, video, and text chat applications, including multiple views of shared spaces, featuring spatialized audio (via both a speaker array system and nearphones), haptic input/outut (via a swivel chair with a servomotor and force sensors), and photo-realistic immersive interfaces (via {\sc qtvr} [QuickTime Virtual Realty] panoramic movies). Applications are expected to include distance learning and collaborative computing (internet telephony complemented by video and distributed whiteboard feature), telepresence and telerobotics (through a mobile robot), and gaming.

19. Michael Cohen and Kenta Sasa. An Interface for a Soundscape-Stabilized Spiral-Spring Swivel-Seat. Proc. WESTPRAC VII: The Seventh Western Pacific Regional Acoustics Conference, pp. 321--324, Kumamoto, Japan, 2000.

    ISBN 4-9980886-1-0 and 4-9980886-3-7. We have programmed a Java applet to display and control the state of `The Internet Chair,' a multimedia-enhanced swivel seat rotated by a force-feedback servo-motor. As a user interface of the Internet Chair, this applet graphically displays a virtual spiral spring, dynamically calculating its projected force, using a natural extension of Hooke's Law (stress proportional to strain), which values are used to set the resistance of the servo-motor. Further, by using spectral colors in the 2D display, virtual depth is visually suggested by Chromastereoptic eyewear. By adjusting audio panning (intensity stereo) through the Java media layer of a virtual source with respect to the orientation of the Internet Chair, a virtual soundscape may be rotated and stabilized, registrable with the actual physical space (for eventual alignment of auditory cues with real-life events or locations). Currently capable of rendering HDD-resident .au and .wav sound files in either `one-shot' or looped mode, the applet will be extended to also allow audio display of network-streamed signals. Another visual display will be used as a viewport into a 3D scene (using modeling and rendering technology like QTVR, Java3D, or VRML), also fixable against the real space across yawing rotation, complementing and reinforcing the stabilized soundscape. A Java framework integrates these three different spatial media modalities: audio, visual, and haptic. Groupware deployment will allow multiple users across the internet to share the sights, sounds and physical sensations of virtual and real worlds.

20. Michael Cohen. A Design for Integrating the Internet Chair and a Telerobot. Proc. IS2000: Int. Conf. on Information Society in the 21st Century, pp. 276--280, Invited Paper, IPSJ, IEICE, IEEE, 2000.

    We are integrating the Internet Chair with a telerobot. The goal is high-level radio- and \initials{ip}-based tele\"{e}xistence with an advanced, virtual-reality based frontend (control system) with realtime audio and video telepresence and force rendering. Azimuthal sensors and effectors in the chair will allow it to be used to steer the robot, or, when the robot runs autonomously, to monitor its orientation propriocentrically, which sensation complements the visual and auditory telepresence modalities. We plan to deploy synesthetic displays, compositing information cross-modally, like inferred auditory scene details rendered graphically.

1. Michael Cohen. Received Japanese patent No. 3042731, Voice Reproduction System with Nobuo Koizumi and Shigeaki Aoki, 2000.

   A sound reproduction apparatus which reproduces sound by driving a plurarity of loudspeakers using a sound signal comprising; head-related transfer function filtering parts which simulate transfer functions of sound transmission path from a sound source to a human head, sound field correction filtering parts which alleviate effects of sound field in reproduction parts, a chair which comprises a tilting-angle-variable back with a set of loudspeakers and a turnable seat, an azimuth detecting part which detects an azimuthal angle due to turning of said seat an elevation angle detecting part which detects an elevation angle due to tilting of said tilting-angle-variable back, and a transferring part which transfers outputs from said both detecting parts to said head-related transfer function filtering parts.

1. Masahide Sugiyama. IPA, Software Development Fund for Japanese Speech Dictation System.

1. Masahide Sugiyama. Member of Human Interface Committee in IPSJ.

2. Masahide Sugiyama. Member of Spoken Language Processing in IPSJ.

3. Masahide Sugiyama. Vice president of Speech Research Committee in IEICE/ASJ.

4. Masahide Sugiyama. Referee of IEICE and ASJ (Acoustic Society of Japan).

5. Masahide Sugiyama. Editor of Special Issue ``System Development" in IEICE Transaction.

6. Masahide Sugiyama. Member of ASJ program committee.

7. Masahide Sugiyama. Member of ISS committee in IEICE.

1. Michael Cohen. Digital Posters SCCP. University of Aizu, 2000.

2. Michael Cohen and Jens Herder. Virtual Reality Audio SCCP. University of Aizu, 2000.