Algorithm for automated visual inspection of MMIC using a classifier based on neural networks

. We present the algorithm for automated visual inspection of microwave monolithic integrated circuits (MMIC) using computer vision and artificial neural networks. The artificial neural network classifies each pixel of a microphotograph to a certain photomask area. The algorithm detects defectiveness of an MMIC according to classification result and photomask comparison.


Introduction
The visual inspection of an MMIC is the control quality of a manufactured MMIC according to their appearance using special optical devices [1]. We evaluate a manufactured MMIC in some aspects by visual properties. The most important aspects are the photomask repetition accuracy, an MMIC elements surfaces qualitative analysis and an absence of damage and pollution [2]. After analyzing the defectiveness degree, we conclude the MMIC suitability in appearance.

The automated visual inspection algorithm
Visual inspection automation is the development of an algorithm and software that increases the speed and quality of defects detection in MMICs.
The input data for the algorithm is a vector photomask file in the ".dxf" [3] format and MMICs microphotographs set obtained in automatic mode. The algorithm contains the following steps for converting the input data: 1) Searching a few alignment points previously specified by searching for microphotograph key parts (for example, a rectangle angle of an MMIC, its metallization angle) [4][5]. The algorithm calculates the alignment points (x; y) position by searching the minimum difference coefficient between image pixels of the alignment point and search area pixels [4]: where are RGB format color components of alignment point search area pixels (a range of values is from 0 to 255); are RGB format color components of alignment point neighborhood image pixels (a values range from 0 to 255).
2) Calculating the vector photomask scale and position. To simplify the task, we removed the angular component at the getting microphotography stage.
The photomask scale can be calculated by using the following equations: where I x , I y are the microphotograph alignment points coordinates (in pixels) and F x , F y are the alignment points coordinates on the photomask (in relative units) [4].
Knowing one alignment point coordinates and photomask scale, we can calculate the relative photomask position to the microphotograph: (3) The scale may have a negative value, which indicates a photomask mirror image.
3) Extract layers from the vector photomask file, that determine the MMIC appearance. Next, the algorithm rasterizes and fills the vector photomask. 4) Extract interest areas from an MMIC microphotograph and a matching rasterized photomask. This allows the algorithm to localize areas and set special requirements for critical MMIC parts. The interest region position and size are set in photomask relative units.
5) Converting a microphotograph by a classifier based on an artificial neural network. Each converted microphotograph pixel stores information about affiliation to a certain photomask area. An artificial neural network [6] is a multilayer fully connected neural network. The Figure 1 depicts used architecture. The network consists of three layers: input layer with three neurons (RGB channels number); hidden layer with 5-10 neurons; output layer with 4-8 neurons (the classified photomask areas number). The neural network is preliminarily trained to classify microphotographs pixels to a certain photomask area by the backpropagation method. The conversion result is a matrix of the probability vectors of a microphotography pixel affiliation to a certain photomask area, may be presented as an affiliation map.  Each pixel intensity defect map is the pixel distance from the converted MMIC microphotograph pixels to the nearest pixel of the same accessory rasterized photomask (if the pixel is farther from the area to which it belongs, then the defect map pixel is lighter (Fig. 2)). The algorithm searches the nearest pixel using an expanding search square of size from 1 to 255. The maximum value is "255", which allows to obtain the defect map image without additional transformations and to limit the search area, speeding up the whole algorithm. Fig. 2. Calculating the pixel distance example of the microphotograph classified section for a rasterized photomask area. 7) Defect map threshold converting (binarization) using a predetermined conversion level. This step allows separating normally located MMIC topology elements from defects. 8) MMIC binary defect map bright pixels analysis for the maximum allowable defect size. The algorithm searches for a cluster by a simple breadth-first search for linked pixels graph. Next, the algorithm searches for the most distant pixels vertically and horizontally. The difference in the coordinates of these pixels determines the cluster size. Since the microphotograph scale is known to advance, the algorithm calculates the cluster size in microns. The algorithm correlates this size with the maximum allowable defect size and concludes that an MMIC is good.

Conclusion
We have developed a test software based on the considered algorithm for testing it. The software allows to configure each step of the automated visual inspection algorithm, to perform automated and manual microphotograph MMIC visual inspection. The software based on the visual inspection algorithm accelerates the visual inspection process by 5 times, the automated visual inspection accuracy is 93% higher relative to manual visual inspection.