Seamless image stitching in the gradient domain

. The algorithm of seamless image stitching in the gradient domain based on the variational derivative estimation method. The functionality of this algorithm is confirmed by computational experiments with real images of different types.


Introduction
Nowadays, for a wide range of digital image processing tasks, «stitching» one image from several is used. This task is called «creating panoramic images» or «stitching». The task of stitching arises not only when creating panoramas, but also when inserting an object into the image, when creating texture, etc.
Methods and algorithms are known that allow to get panoramic images, some of these methods are implemented in the photographic equipment. However, it should be noted that in some cases the image stitching methods and algorithms used do not give satisfactory results. Often this fact is due to the quality of input images, which may contain noise component of different origin, images may have different illumination, geometric mismatches and much more.
The purpose of any stitching algorithm is to obtain the resulting image, which will have the following properties: -elements of the resulting image should be as similar as possible to the input images; -the resulting image should not show the stitch on which the stitching is made.
The quality of the direct stitching of the image can be improved by considering not the image itself, but its gradient domain when stitiching. Moreover, at the first and fourth stages of creation of the panoramic image there is a necessity of an estimation of derivatives. The problem can arise in the event that the image contains noise of different origin. In this case, when searching for key points using derivative estimations, there are many false key points. At the fourth stage of creation of the panoramic image quality of calculation of derivatives significantly affects the result of stitching.

2.Research objects and experimental methods
During the course of this research, it is proposed to search for key points on the source images and to use a variation method of derivative evaluation for direct stitching [1,2].
This method of evaluating derivatives of any order is based on the formula known from mathematical analysis, which expresses a differentiable function through its derivative: To represent the derivative, use the expression: where Ω -frequency span, -Fourier transform, which can be represented as

∫
The expression that allows interpolation of the signal using the Fourier transform of the derivative is obtained using expression (2) on the right side of relation (1).
In this case, interpolating equalities can be written as Using the variational principle of minimizing the Euclidean norm of the first derivative it is easy to obtain an expression for estimating the first derivative of the signal. In this way, for the solution of the initial task the constraints presented in expression (4) are used. The solution of the main task will be the following expression where . For calculation , the left-hand sides of relation (7) must be written in accordance with expression (4). Thus, the unknown coefficients of the system are calculated by the expression: Furthermore, the matrix represented by expression (9) is a positive definite matrix. However, the determinant of matrix A will be more than zero only if the integration interval in (9) satisfies the primary condition So, considering expressions (7) and (9), we obtain an expression for calculating the estimate of the derivative Given that the pixels of the images are an equidistant set of values, and using the system of linear algebraic equations (8), the computational formula can be represented as: For an optimization of calculations, matrices B and A can be preliminarily calculated. The expression for the second derivative can be obtained by differentiating expression (2) and using the variational principle of minimizing the Euclidean norm of the derivative. The desired expression will look like If the signal values are equidistant, it is easy to obtain next computational relations The considered method is proposed to be used for calculation of the derivatives used for image stitching, because it is resistant to the noise component of the image, which is confirmed in the works [1 -5].
To confirm the efficiency of the presented method of evaluation of derivatives in the problem of image stitching, computational experiments with images of different types were carried out. As the first stage of obtaining panoramic images is to search for special points, this method was used when they were detected. Search for special points is carried out in two ways: by the algorithm SURF, and when using in the process of implementation of the algorithm SURF estimates of derivatives obtained by the proposed method. Furthermore, this method will be referred to as SURF* for simplicity. Also, the original images were subjected to artificial noise before the search and comparison of special points. The results of the experiment are presented in Table 1.   noise  SURF  5875  5397  6394  5148  7352  7148  8375  7925  SURF*  5826  5389  6398  5137  7123  6921  8224  7895 Both algorithms find a huge number of key points, which can be angles, brightness differences, color boundaries and much more. When searching for special points on images with the noise component of such elements becomes more, because the noise present in the image can be perceived by the algorithms of searching for special points as a special point. These points will be false and will not give a positive result in the comparison. Table 1 shows that when comparing images with and without noise component, both algorithms find more special points on the noisy image. However, the proposed algorithm finds a smaller number of key points on the noisy image in relation to the original algorithm, which suggests the presence of a smaller number of «false» points among those found.
However, that the number of special points found does not provide adequate information about the quality of the algorithms, because the main parameter is the number of compared special points and the quality of their comparison. This is especially important when using images with a noise component. Therefore, the second part of the experiment was to compare the key points on the images of different quality.
In the experiment, the work was carried out with images that have a shift relative to each other, and with images containing the noise component. White noise was used as the noise component. The results of the experiment are presented in tables 2 -4, and in figures 1 -3.     The results confirm that in the modified SURF* algorithm which is presented in this work, as compared to the usual algorithm, allows you to compare a greater number of key points when shifting the image. When analyzing images containing noise component, it was found that the modified SURF* algorithm coped with stitching and recovery better than conventional SURF.
The conducted experiments allow us to conclude that the use of a variation method of evaluation of derivatives in the calculation of the image gradient when searching for key points to create a panoramic image is reasonable, since the quality of the resulting image depends directly on the quality of the search and the number of special (key) points. The efficiency of the modified SURF* algorithm has also been proved to solve the problem of searching for key points on images containing a noise component.

Conclusion
The algorithm of creating a seamless (panoramic) image using the variation method of derivative evaluation is proposed. This method is used at two stages of image stitching: firstly, to search for key points on the original images, secondly, to image stitching in the gradient domain. Using this algorithm is reasonable for images containing noise component, since the proposed method of evaluation of derivatives is resistant to noise.