Depth-based sparse bundle adjustment.

It is demonstrated in this paper that due to error model inconsistency, a certain degree of accuracy loss would be incurred to the estimated parameters when the traditional bundle adjustment method is directly applied to the scenario where a fraction of observations is implicitly error free (e.g., the reference image points in commonly used least squares matching refinement). To this end, a depth-based object point model and corresponding depth-based sparse bundle adjustment method are proposed in this paper, in which the position of an object point is represented by its 1D depth relative to its reference image. A corresponding projection model is derived, the sparse block structures of normal equations are studied depending on whether there are shared image parameters to be optimized or not, and corresponding sparse solutions of the normal equations and parameter covariance matrices are derived. Compared with the traditional sparse bundle adjustment method, simulated experiments demonstrate that our method matches the error model of the target scenario, and thus can avoid further accuracy loss. Moreover, both simulated and real data experiments demonstrate that our method can effectively improve computational efficiency.