Multisensor video magnification

Abstract

This study was performed with the three main goals. The first of which includes the implementation of “Eulerian Video Magnification” for the normal videos to observe the subtle changes that cannot be seen through the naked eyes. The second goal was to implement interpolation on thermal infrared videos. Finally, the third goal is to combine both the videos to amplify complex variations to reveal important aspects of the world around us. Using Eulerian magnification, the experiments were performed for the series of videos that includes different human body parts. The specific region were face, hand and palm. Not only the human behaviour was observed but also the environmental objects was considered to see the similar types of subtle changes and reveal some more important facts and display them in indicative manner. Eulerian magnification amplifies color changes at the pulse rate to make the subtle color variation due to blood flow that are visible to the human eye. Simulated and real video of human skin are processed to reveal blood flow in the face, wrist and hand using a MATLAB implementation of the Eulerian magnification algorithm. From the research, it shows that it is very susceptible to impairment from motion and camera configuration. The next experiment was done with the videos from thermal camera. The task was to increase the frame rate for the video where we implemented the Butterflow algorithm by Duong Pham [18]. The algorithm uses the library from OpenCV and uses the concept of “Two-Frame Motion Estimation Based on Polynomial Expansion” by Gunner Farneback [5]. The algorithm takes low frame rate videos and convert it into the required frame rate for the user. Finally, both the tested videos are overlapped in Qt-framework to reveal the changes in object behaviour.