Algorithm combines images from different cameras into panorama

Disney Research has developed an algorithm that allows the creation of high-quality video panoramas with different cameras. In the future, for example, people may be able to combine video streams from different smartphones into one panorama.

Creating a panorama of several photos is not that complicated technically. By using techniques such as multi-level blending, a nice picture can be formed with little effort, the Disney researchers write in their paper. With video, however, the phenomenon called parallax causes much greater problems than with a static photo.

Parallax has to do with the apparent position of an object relative to another object, which is different from different positions. Trees in the distance, for example, hardly move as seen from the train, while the catenary poles pass very quickly. This makes it difficult to create a single moving image with different camera images that no longer contains visible artifacts.

The problem arises because there is no unity of time and place in the space of the various recording devices. In the paper, the authors discuss how to locally ‘warp’ the image in order to remove parallax and thus no longer have blur, ghosting or other distortions in the image. The algorithm corrects for the apparent difference in an object’s position caused by the different camera angles.

The video feeds must overlap. The technology also automatically aligns and determines the position of the camera, making manual calibration unnecessary. In addition, warping of straight lines is detected and corrected, so that, for example, a road remains straight. The technique also corrects for small differences in the timing of the frames between the different cameras.

With their technique, the researchers were able to stitch together images from fourteen different camera types, with which they were able to make a panoramic video of more than 100 megapixels: