Visual Media Adaptation for AToM

This research develops an adaptation system for visual media such as image & video under the various contexts of content semantics, user preference, network characteristics, and terminal capability. The system is compatible with Part 7 (Digital Item Adaptation) of the MPEG-21 standard. Multimedia documents and various metadata (user preference, terminal/network characteristics, etc.) are created in the form of Digital Items (DIs) of MPEG-21 and validated against the standardized schemas. The presentation of an adapted document at a terminal is controlled by a XSL (eXtensible Stylesheet Language) stylesheet. More information of this project can be found at http://hemera.icu.ac.kr/DML/main.html 

Fig. : Adaptation System Architecture

Based on this framework, our group further considers different aspects of adaptation/delivery, such as modality conversion, visual accessibility.

Modality Conversion

In a UMA system, content adaptation is the key solution to provide the best possible presentation under constraints of various kinds of terminals and network connections available today. Content adaptation has two aspects: one is modality conversion that converts content from one modality (e.g. video) to different modalities (e.g. image), the other is content scaling that changes the quality (e.g. bit rate) of the contents without converting their modalities. So far, most researches on content adaptation have focused on content scaling. Our research focuses on the systematic methods to convert the contents’ modalities in the adaptation process. Two important issues are being tackled: modality conversion preference and modality conversion QoS.

• Modality Conversion Preference

Content adaptation process may convert variously the modalities of contents. For each content object, there would be many conversion possibilities, whereas the user may prefer or even can hardly perceive (e.g. blind users) some modalities. The role of modality conversion preference is to let user specify his choices on modality conversion. Without this tool, user has to accept any adaptation solution provided by the provider. This tool should be flexible to support various practical cases and demands from user. Also the user preferences should be smoothly integrated into the adaptation process.

• Modality Conversion QoS

Intuitively, given some resource constraint of terminal/network, the provider will (down) scale the contents to meet the resource constraint while still providing the best possible quality to the user. However, in some cases, the quality of the scaled contents is unacceptable or not as good as that of a substitute of a different modality. A possible alternative solution for this problem is to convert the contents into other modalities. For example, when the bandwidth is too low, sending a sequence of “important” images would be more appropriate than streaming a scaled video with low quality. From the QoS point of view, the most important question in modality conversion is “at which resource constraint should the current modality be converted to another modality?” However, currently modality conversion is often carried out just when the terminal cannot support some modality. Our research tries to answer this question when the terminal can support the modalities but the resource constraint is limited.

Fig. : The comparison of different versions having different modalities, namely video, image (sequence), audio, and text.

Quality Evaluation Across Different Modalities 

This research is well related to Modality conversion QoS. In heterogeneous computing environments, multimedia contents would be adapted drastically in terms of quality as well as modality. There is a strong need to measure the content quality in this context. However, the quality is often considered within a single modality (e.g. video, audio). There has been very little work on quality evaluation when the content's modality is variously converted (e.g. a video converted to a sequence of "important" images or explanatory text). This research aims at evaluating the content quality across different scaling levels and modalities. We show that the quality consists of two key aspects: the perceptual quality and the semantic quality. The former refers to user's satisfaction in perceiving the content, regardless of what information the content contains; the latter, which is crucial in modality conversion, refers to the amount of conveyed information, regardless of how the content is presented.

Accessibility for Low Vision Users

Ethical and legal requirements have made accessibility a crucial feature for any information systems. The Word Wide Web Consortium (W3C) has issued the Web content accessibility guidelines, listing the necessary tricks to make Web content accessible to people with disabilities. Recently, many governments officially adopted these accessibility guidelines and requested government agencies to employ information technology that is accessible to disabled users (e.g. Section 508 of the Rehabilitation Act of US law). This fact has resulted in a hot competition for media accessibility technologies and related products. However, the current accessible technologies are rather simple and just suitable for either mild impairments or total impairments (e.g. blind, deaf). This research deals with the visual content adaptation, in the context of MPEG-21 standard, to help low vison users have better accessibility to the contents. Low vision means that even with regular glasses, medicine, or surgery, the patients still have difficulties in everyday tasks, of course including accessing the existing multimedia contents. The common symptoms of low vision include loss of fine detail, lack of contrast, light sensitivity, need of light, losses of visual field.

More details of this project can be found at http://accessibility.icu.ac.kr/

Fig. : Simulation tool of image adaptation for low vision users.

Video Summarization

Our goal is to develop flexible and efficient solutions to generate summary for sports video data. To this end, we focus on incorporating the standard audio-visual features (MPEG-7 descriptors) with domain-specific information in a common framework, and thus locate and extract interesting events. Currently, our test dataset are soccer videos. More details of this project can be found at http://hemera.icu.ac.kr/soccer/index.htm

Video Streaming

MPEG video streaming over Mobile Internet leads to the degradation of MPEG video quality. When a handoff of a mobile node occurs, it is difficult to guarantee seamless video quality due to the change of routing path towards the mobile node. In this research, we study MPEG video streaming with a novel scheme, called Concatenation and Optimization for Reservation Path (CORP), so as to guarantee QoS of MPEG streaming service in the Mobile Internet. A demo this research can be found here.