4224

Algorithm for Stabilizing Fast Forward of Videos, Especially for Wearable Cameras

Peleg Shmuel, HUJI, School of Computer Science and Engineering, Computer Science
Relies on Ego-Sampling, a novel approach for frame sampling

Categories

Computer Science & Engineering, Imaging / Computer Graphics

Development Stage

Proof of concept

Patent Status

Patent application filed in the United States

Highlights

  • The videos captured by such egocentric cameras such as GoPro and Google Glass are long, boring, and difficult to watch from start to end.

  • Attempting to browse the video faster by fast forwarding based on frame sampling is problematic in egocentric videos because the shake introduced by natural head motion gets accentuated rendering the video useless.

  • There is thus a need for automated tools that enable faster access to the information in such videos.

  • This solution provides a novel and lightweight approach for creating fast forward videos for egocentric videos and can be used as a method to create stereo sequences from monocular egocentric video.

Our Innovation

Ego-Sampling, a novel frame sampling technique to produce stable fast forward for egocentric videos

31-2015-4224_-_pic1.jpg

Representative frames from a fast forward from a camera wearer riding a bike and preparing to cross a road. Top row: uniform sampling of the input sequence leads to a very shaky output as the camera wearer turns his head sharply to the left and right before crossing the road. Bottom row: Ego-Sampling prefers forward looking frames and therefore samples the frames non-uniformly so as to remove the sharp head motions. The stabilization can be visually compared by focusing on the change in position of the building (circled in yellow) appearing in the scene. The building does not even show up in two frames of the uniform sampling approach, indicating the extreme shake.

Key Features

  • Most egocentric cameras are worn on the head or attached to eyeglasses. This leads to significant shaking of the camera due to the wearer’s head motion. Camera shaking is higher when the person is “in transit” (e.g. walking, cycling, driving, etc.).
  • For fast forward viewing, it is preferable for consecutive output frames to have similar viewing direction as if they were captured by a camera moving forward on rails.
  • This frame sampling technique selectively picks frames with similar viewing directions, resulting in a stabilized fast forward egocentric video.
  • The fast forward produced by Ego-Sampling can be post-processed by traditional video stabilization techniques to further improve the stabilization.

The Opportunity

  • The market for wearable cameras is an early stage and experiencing rapid growth as the use cases for wearable cameras expand. Market research firm Tractica forecasts that wearable camera shipments will increase from 5.6 million in 2014 to 30.6 million units annually by 2020, equivalent to a compound annual growth rate (CAGR) over the period of about 32 percent.
  • New uses are being found for these cameras. “While GoPro is driving the market for sports and adventure enthusiasts, we expect usage of consumer lifelogging cameras like those from Narrative to mature over time to capture specific moments and support video streaming.
  • The public safety sector is also experiencing growth in the adoption of body-worn cameras for police officers provided by companies like Taser, Vievu, and Digital Ally. In addition, enterprise users are experimenting with applications like user experience research in retail and hospitality.”

Patent Status

Granted US 9,672,626

Contact for more information:

Aviv Shoher
SVP BUSINESS DEVELOPMENT
+972-2-6586635
Contact ME:
Image CAPTCHA