Vision Based Markerless Tracking, Sensor Fusion And 3D Reconstruction Algorithms

Markerless visual tracking methods track the position of a camera based solely on video information in unprepared environments. They can be used for realistic placement of virtual objects into video in augmented reality, for example. Typical solutions use an automatically reconstructed 3D map containing the relative positions of some identifiable surface points in the scene to find and track the camera position. The reconstruction process consists of identifying common points across images, triangulating common points and finding camera positions and simultaneously optimizing camera and point positions over multiple images in a process called bundle adjustment.

Building 3D reconstructions automatically and adapting the solution to different tasks is difficult requiring implementing custom solvers to
complex mathematical optimization problems and combining several computer vision algorithms. In creation of this deliverable a new software package for this purpose was developed in Aalto University. The package contains a generic sparse optimization toolkit,
implementations of several basic algorithms in geometric computer vision and a sample reconstruction pipeline for a single uncalibrated camera. The generic design makes it relatively easy to implement or adapt a reconstruction method for specific cases (e.g. varying calibrated and uncalibrated camera models, fixed reference points and triangulated correspondences). The package has been used successfully in the joint entry of VTT and Aalto University in the ISMAR tracking contest to build and align a map to reference coordinates for real-time visual tracking of a camera.

Parts of the tracking solutions used in the ISMAR tracking contest, as well as other development items such as template based tracking, are integrated in VTT's ALVAR subroutine library. The Alvar Basic and Alvar Pro packages are currently available as version 1.5, using OpenCV 1.0. Currently they are being merged into Alvar Desktop 2.0 library, using OpenCV 2.x. The Alvar Desktop library will be released publicly later in 2011. The latest version of Alvar Basic libraries can be downloaded from www.vtt.fi/multimedia.