adobe logo cmu logo neurips 2022 logo

VideoMap: Video Editing in Latent Space

David Chuan-En Lin1, Fabian Caba Heilbron2, Joon-Young Lee2, Oliver Wang2, Nikolas Martelaro1

1Carnegie Mellon University, 2Adobe Research

📄 Paper📝 Citation (BibTeX)

Abstract

Video has become a dominant form of media. However, video editing interfaces have remained largely unchanged over the past two decades. Such interfaces typically consist of a grid-like asset management panel and a linear editing timeline. When working with a large number of video clips, it can be difficult to sort through them all and identify patterns within (e.g. opportunities for smooth transitions and storytelling). In this work, we imagine a new paradigm for video editing by mapping videos into a 2D latent space and building a proof-of-concept interface.

Example Application - Match Cut

Shape Lens

Shape match cut interface

Color Lens

Color match cut interface

Semantic Lens

Summary video interface

Example Application - Summary Video

Semantic search interface Search video interface