YOLACT - Real Time Instance Segmentation

Instance Segmentation[1] is one of the hottest areas for research in the field of Artificial Intelligence. The process in its present form relatively computationally expensive, therefore the present architectures that are able to generate the best results are not very suitable for “real-time” applications. The networks that yield good results in instance segmentation are FCIS[2], Mask-R CNN [3], RetinaMask[4], PA-Net[5] etc. These frameworks although perform relatively well but the inference obtained from them can’t be used in “real-time” due to the computational complexity that is involved in the creation of such systems, the sheer number of parameters makes it impossible for these network to perform on machines with lower computational capability. The task therefore requires a different architecture that is able to perform in the “real time” computations.

YOLACT to the rescue

YOLACT [6] (You only look at the coefficients) is a more optimized version for instance segmentation, which has secured a good reputation for its speed and accuracy trade-offs. It is able to achieve 29.8 mean Average Precision (mAP) on MS COCO at 33 frames per second, much faster than the other competitive frameworks.

So what is YOLACT


YOLACT uses ResNet-101 [7] with FPN (Feature Pyramid Networks) [8] that helps in creating pyramids of feature maps of high resolution images rather than the Conventional Pyramid of Images approach, therefore reducing the time and requirements of computational capabilities. The computational cost is cut further with the use of only higher level extracted feature layers that have higher semantic value (The resolution and semantic value are both inversely proportional in the given architecture as demonstrated in the figure below). The architecture has a top-down pathway that allows for the construction of the higher resolutional layers from these semantic rich layers, in order to keep the detection abilities for the locations of the objects after upsampling and downsampling there are lateral connections between the reconstructed layers and the corresponding feature maps.

Source: [9]

The up-sampled layers then interact with Prototypical Neural networks [10] and a Prediction head that has the mask coefficients for the respective detections, all the frames obtained from the prediction head are then put through a Non-maximum suppression where we only receive the detections that have the highest coefficients of detection. The combined output is then cropped and the colors for different instances are then generated from thresholding the final output.

How it compares with the existing architectures:

Speed

YOLACT has a number of advantages over the existing architectures, one of the most significant ones will be the speed of predictions. As mentioned earlier YOLACT is the only existing architecture that can actually deliver “real-time” inference.
The following figure compares the performance of different architectures multiple variations of the YOLACT:

As is clear from the comparison chart above YOLACT is quicker with its inference than all the other nearest competitors in all its variants. The accuracy and speed are a trade-off and some variations of YOLACT are able to beat FCIS in terms of the accuracy as well.

Higher Quality of Masks

The masks generated from YOLACT are of higher quality than the existing architectures, as the masks actually use the entire image space domain and that therefore doesn’t cause any loss from the repooling.

Source : [11]

Summary

In conclusion the YOLACT architecture has a little disadvantage over the existing architecture in terms of accuracy, but it more than make up for that with its speed in inference.

There are plenty of real world applications where this kind of behavior is desirable for example with: object detection on mobile devices (apps that use filters for images etc), autonomous vehicles, Geo-sensing, precision agriculture (for its ability to generate better masks), face segmentation, e-commerce etc. These areas have considerable scope for the real-time implementation of YOLACT where it outperforms the existing architectures in terms of its ability to generate better masks and speed. YOLACT is a relatively young architecture and the results at this early stage are promising, with time as more and more people are able to work on it, the performance will show significant improvements.

To Learn More AI, Click Here

References:

[1]. https://www.theregister.co.uk/2018/11/28/aws_self_driving_car/

[2]. https://arxiv.org/abs/1611.07709

[3]. https://arxiv.org/abs/1703.06870

[4]. https://arxiv.org/abs/1901.03353

[5]. https://arxiv.org/abs/1803.01534

[6]. https://arxiv.org/abs/1904.02689

[7]. https://medium.com/@14prakash/understanding-and-implementing-architectures-of-resnet-and-resnext-for-state-of-the-art-image-cf51669e1624

[8].https://medium.com/@jonathan_hui/understanding-feature-pyramid-networks-for-object-detection-fpn-45b227b9106c

[9]. https://cdn-images-1.medium.com/max/800/1*_kxgFskpRJ6bsxEjh9CH6g.jpeg

[10]. https://arxiv.org/pdf/1703.05175.pdf

[11]. https://img-blog.csdnimg.cn/20190410164829996.png

Comments

Post a Comment

Popular posts from this blog

Movidius NCS (with Raspberry Pi) vs. Google Edge TPU (Coral) vs. Nvidia Jetson Nano – A Quick Comparison

Image
Lately, there has been a lot of talk regarding the possibility of machines learning to do what human beings do in factories, homes, and offices. With the advancement in artificial intelligence , there has been widespread fear and excitement about what AI, machine learning, and deep learning is capable of doing. What is really cool is that Deep Learning and AI models are making their way from the cloud and bulky desktops to smaller and lower powered hardware. In this article, we will help you understand the strengths and weaknesses about three of the most dominant deep learning AI hardware platforms out there. The Intel Movidius Neural Compute Stick (NCS) with Raspberry Pi Developed by Intel Corporation, the Movidius Neural Compute Stick can efficiently operate without an active internet connection. Its computing capabilities come from the Myriad 2 Vision Processing Unit (VPU) . It offers profiling, tuning, and compiling a Deep Neural Network (DNN) on a development computer with the ri...
Read more

DeepSORT - Deep Learning applied to Object Tracking

Image
Introduction So in this article, Im going to give to you a clear and simple explanation on how Deep SORT works and why its so amazing compared to other models like Tracktor++, Track-RCNN and JDE. But to understand how DeepSORT works, we first have to go back, waaay back and understand the fundamentals of object tracking and the key innovations that had to happen along the way, for DeepSORT to emerge. Before we get started, if you are interested in developing object tracking apps, then checkout my course in the link down below, where I show you how you can fuse the popular YOLOv4 with DeepSORT for robust and real-time applications. Okay so back to Object Tracking. Now lets imagine that you that you are working for Space X, and Mr. Musk has tasked you with ensuring that on launch, the ground camera is always pointing at the Falcon 9 as it thrust into the atmosphere. As much as you are excited to be personally chosen by Elon to work on this task you ask yourself, “How will I go about th...
Read more

OpenCV AI Kit - An Introduction to OAK-1 and OAK-D

Image
Introduction Okay so I just got my OpenCV AI Kit and I am really excited to play with. But even though Iv seen a bunch of features on Kickstarter, I don’t quite understand the true capabilities of this device and why I should even care . Well in this video we going take a deep dive into this device and find out exactly what its capable off, and at the end of the video I will share my thoughts on if this device is worth all the hype as well how it matches up to the competition. What is the OpenCV AI Kit So before we get into that, lets get into the first question of What exactly is the OpenCV AI Kit or OAK. According to the creators, it is a tiny, powerful, open source Spatial AI system. So you can think of it as if an embedded 4k camera and neural compute stick had a baby, make that two babies OAK-1 and OAK-D Difference between the OAK 1 – OAK D Now What sets these two devices apart is that the OAK-1 has automatic motion-based lossless digital zooming which means that the sensor has ...
Read more