Current Challenges

In this age of mobilization the cell phones now offer full video capabilities, including recording and play back. Along with the popularity of YouTube and explosion of user created content shot on low end equipment (such as mobile phones) and the evolution of high resolution displays there is a huge gap in the resolution of the content and display capabilities. Moreover, the content created over the past decades for VHS and Analog TV requires up-scaling. In addition, there is a huge demand for streaming HD quality video over WiFi and 3G networks.

The developers of Notebooks and Smart Phones are struggling to successfully implement streaming video; the primary issue with WiFi and 3G network is that they are optimized for the transmission of data in a way that is not ideal for the transmission of video.

The available bandwidth and the quality of the connection have a major impact on the video viewing experience. Avieon’s VISP™ IP utilizes video post processing algorithms which is an ideal solution to overcome these challenges.

Why Super Resolution?

Images with high pixel density are desirable in many applications, such as high resolution (HR), high quality video conference, high definition television broadcasting and, Blu-ray movies, etc. Unfortunately most of the multimedia content is recorded/shot from using low resolution (LR) cameras such as cell phone cameras or webcams. Several resolution enhancement techniques have emerged to improve image quality, color tone, contrast and reduce blur and blocking artifacts. However, the performance of simple interpolation methods is poor due to the aliasing, zigzag edges, and blocking effects.

Existing Super Resolution (SR) methods can be divided into three categories, specifically the reconstruction-based method, the learning-based method, and the functional interpolation method.

Reconstruction-Based SR

The reconstruction-based SR method builds an HR image from a sequence of LR images. The limitation of reconstruction-based methods is that, as the image magnification factor increases, the reconstruction constraints provide less and less reduce the amount of useful information. In addition, it requires that the image pairs are related by global parametric transformations, and/or the parameter of the camera’s Point-Spread-Function is known in advance. This limits its applicability and requires expensive hardware devices.

Learning-based SR

The second category technique is , the learning-based methods SR, which involves learnings of the co-occurrence prior between HR and LR image patches or coefficients, and processing the LR input along with appropriate smoothness constraint to generate HR image. Although learning-based SR method is suitable for both multi-frame and single-frame SR, it is somewhat dependent on the training set. Hence the result is not stable and sometimes produces artifacts in real applications.

Functional Interpolation SR

The third category is the functional interpolation SR approach, which applies an existing function on the LR input. In this line, This technique utilizes some well-known image interpolation methods can be found, such as bi-cubic and polynomial spline interpolation. However, simple interpolation function cannot recover the high frequency components and often blurs the discontinuities of interpolated images. This method generates SR images with good perceptual quality, but is very computational expensive.

Research on super resolution has been ongoing during since the past decade. However the challenge of applying super resolution techniques continues to be the practical balance of various constraints such as; computational resources, the cost of other components in the architecture; such as memory, number of gates, the available real-estate available and the power consumption to support the processing requirements. Most of the super resolution algorithms are impractical for real-time embedded application either due to computatione requirement or memory size / bandwidth.