Super Resolution – The Next Step in High End Imaging Technology

Single image super-resolution (SISR) is a common and challenging vague problem, which aims to obtain a high-resolution (HR) output from one of its low-resolution (LR) versions. To solve the SISR problem, recently powerful deep learning algorithms have been employed and achieved the state-of-the-art performance.

Diagram showing Super Resolution

The end goal is to recover the high resolution image, which one can resample depending on the input images and imaging model. It is very important for the imaging model for the super resolution to be accurate. An incorrect modeling can degrade the image quality. Deep learning can be easily applied to perform the Super resolution rather than using the traditional approach of using images from multiple cameras or video sequence.

Applications of Super Resolution

A high resolution image is of utmost importance in the field of medical imaging for diagnosis of any healthy issues. Other fields requiring the execution of super resolution imaging include surveillance, forensic imaging, and satellite imaging applications. The only issue is that setup for high resolution imaging is a bit expensive, and not feasible for many owing to limitation of sensor, and optical manufacturing technology. There are basically two commonly used Super Resolution frameworks including SRGAN(based on SRResNet) and SRCNN.

A close-up shown in varying qualities using Super Resolution Technology

What is the ESRGAN?

The Enhanced Super Resolution Generative Adversarial Networks (ESRGAN) is a seminal work capable of generating realistic textures via single image super resolution. The only drawback is that the details often include unnecessary artifacts. The ESRGAN provides better visual quality with more realistic and natural texture as compared to the traditional SRGAN.

ESRGAN used to make old games look amazing.

Apart from the breakthroughs in accuracy and speed of single image super-resolution using faster and deeper CNNs (Convolutional Neural Network), the main pending issue is how would one recover the finer texture details when super-resolving at large upscale factors. We present to you the GAN (Generative Adversarial Networks), the framework capable of inferring photo-realistic natural images for a 4x upscaling factor.

SRGAN Model Architecture.

There is a discriminator network trained to differ between the super-resolved images and original photo-realistic images. The original authors also utilized a content loss motivated by perceptual similarity instead of the similarity in pixel space. The deep residual network recovers photo-realistic texture from the heavily down-sampled images.

The Difference Between Quality of Original and SRGAN image

Super Resolution CNN?

The Super Resolution Convolutional Neural Network, also known as the SRCNN is a deep-learning based super resolution method for enhancing image resolution in CT images. The success of this model has been proven when 89 chest CT cases were divided among 45 training cases and 44 external test cases.

Using the SRCNN, a high resolution image was constructed from a low resolution image, which was down-sampled from an original test image. The two image quality metrics were compared with the conventional linear interpolation methods. The image restoration quality of the SRCNN scheme was higher as compared to that of the liner interpolation method.

The high resolution image reconstructed using the SRCNN was restored and comparable to the original image used for reference for a x2 magnification. It indicates that the SRCNN scheme significantly outperformed the linear interpolation methods for improving the image resolution in CT scan images. In short, SRCNN has the potential to create high resolution CT images from standard CT images You can know more about this via the image super resolution technology including:

Table displaying the Top 3 Image Super Resolution Frameworks.

SRCNN Vs SRGAN - A Simple Comparison

SRCNN is a deep learning network which consists of three parts namely patch extraction and representation, non-linear mapping, and reconstruction. On the other hand, the SRGAN (Super Resolution Generative Adversarial Network) is a seminal work capable of generating realistic textures during single image super resolution. The details are often followed by unpleasant artifacts.

It important to note that SRGAN can be quite creative in its upscaling that it would be easy to get fooled as to believing the output as ground truth. One should be careful when using this tech for security purposes as it can be used to apprehend a look-alike innocent person as opposed to the perpetrator.

SRGAN is there but not quite...

Summary

The Super Resolution framework reconstructs a higher resolution image or a sequence from an observed low resolution images. SR has been under development from last three decades for both multi and single frame applications in daily life. Thus, Super Resolution is surely the next step in the evolution of high resolution imaging services in the medical and other sectors. However, it needs to grow to become more accurate, efficient as well as deployable for real-time applications.

To Learn More AI - Check this Link.

Ritesh Kanjee

Augmented Startups

Comments

Post a Comment

Popular posts from this blog

Movidius NCS (with Raspberry Pi) vs. Google Edge TPU (Coral) vs. Nvidia Jetson Nano – A Quick Comparison

Image
Lately, there has been a lot of talk regarding the possibility of machines learning to do what human beings do in factories, homes, and offices. With the advancement in artificial intelligence , there has been widespread fear and excitement about what AI, machine learning, and deep learning is capable of doing. What is really cool is that Deep Learning and AI models are making their way from the cloud and bulky desktops to smaller and lower powered hardware. In this article, we will help you understand the strengths and weaknesses about three of the most dominant deep learning AI hardware platforms out there. The Intel Movidius Neural Compute Stick (NCS) with Raspberry Pi Developed by Intel Corporation, the Movidius Neural Compute Stick can efficiently operate without an active internet connection. Its computing capabilities come from the Myriad 2 Vision Processing Unit (VPU) . It offers profiling, tuning, and compiling a Deep Neural Network (DNN) on a development computer with the ri...
Read more

DeepSORT - Deep Learning applied to Object Tracking

Image
Introduction So in this article, Im going to give to you a clear and simple explanation on how Deep SORT works and why its so amazing compared to other models like Tracktor++, Track-RCNN and JDE. But to understand how DeepSORT works, we first have to go back, waaay back and understand the fundamentals of object tracking and the key innovations that had to happen along the way, for DeepSORT to emerge. Before we get started, if you are interested in developing object tracking apps, then checkout my course in the link down below, where I show you how you can fuse the popular YOLOv4 with DeepSORT for robust and real-time applications. Okay so back to Object Tracking. Now lets imagine that you that you are working for Space X, and Mr. Musk has tasked you with ensuring that on launch, the ground camera is always pointing at the Falcon 9 as it thrust into the atmosphere. As much as you are excited to be personally chosen by Elon to work on this task you ask yourself, “How will I go about th...
Read more

OpenCV AI Kit - An Introduction to OAK-1 and OAK-D

Image
Introduction Okay so I just got my OpenCV AI Kit and I am really excited to play with. But even though Iv seen a bunch of features on Kickstarter, I don’t quite understand the true capabilities of this device and why I should even care . Well in this video we going take a deep dive into this device and find out exactly what its capable off, and at the end of the video I will share my thoughts on if this device is worth all the hype as well how it matches up to the competition. What is the OpenCV AI Kit So before we get into that, lets get into the first question of What exactly is the OpenCV AI Kit or OAK. According to the creators, it is a tiny, powerful, open source Spatial AI system. So you can think of it as if an embedded 4k camera and neural compute stick had a baby, make that two babies OAK-1 and OAK-D Difference between the OAK 1 – OAK D Now What sets these two devices apart is that the OAK-1 has automatic motion-based lossless digital zooming which means that the sensor has ...
Read more