AN-1021: MMSE-Based Multipoint Calibration Algorithm for Touch Screen Applications

Introduction

Modern devices typically use LCDs with touch screen technology as user interfaces. Due to their simple construction and well-known operation, resistive-type touch screens are the most popular in cost-conscious designs. However, the mechanical misalignments and scaling factors of resistive-type touch screens affect the X and Y coordinates produced by the touch screens. Therefore, it is difficult to perfectly align the coordinates of the touch screen to the display (LCD or otherwise) behind it. When the final product that includes a touch screen comes out of the box, a calibration algorithm must first be done.

The classical calibration algorithm for a touch screen is a three-point calibration algorithm that uses three reference points. The classical three-point calibration algorithm is both efficient and effective, but its performance is low when the touch screen is comparatively large. This application note proposes the minimum mean square error (MMSE)-based multipoint calibration algorithm, which uses more than three reference points, for resistive-type touch screens. Both mathematical deduction and experimentation prove that this algorithm is more accurate than the classical three-point calibration algorithm.

Mathematics

Figure 1 shows a circle in red where its ideal center is O (origin) and its ideal radius is R. Consider that the red circle represents an image shown by the LCD under the touch screen. The blue ellipse represents an exaggerated set of points produced by the touch screen when the user traces the red circle shown by the LCD. Therefore, the reconstructed image appears rotated, translated, and scaled by a different factor in each axis because of mechanical misalignments and scaling factors of resistive-type touch screens. The challenge for the calibration algorithm is to translate the set of coordinates reported by the touch screen into a set of coordinates that accurately represent the displayed image.

If there is a point where its ideal coordinates are P(x, y), and the coordinates produced by touch screen are P’(x’, y’). (Point P is on the red circle, and that Point P’ is the corresponding point on the blue ellipse.) Suppose that P’(x’, y’) could return to its ideal coordinates, P(x, y), after it rotates by θ, is scaled by KX and KY in each axis, and is translated by TX and TY in each axis. The purpose of the calibration algorithm is to calculate the coefficients of θ, KX, KY, TX, and TY. The coefficients can then be used to calibrate the coordinates directly produced by the touch screen.

To make the analysis easy to understand, assume that P’(x’, y’) are the coordinates directly produced by the touch screen, and its equivalent representation in polar coordinates is P’(Rcos θ₀, Rsin θ₀). In addition, assume that the corresponding ideal coordinates of P’(x’, y’) are P(x, y). Because of the relationship between P’(x’, y’) and P(x, y), it is reasonable to get the following:

P(x, y) = P(K_XRcos(θ₀ + θ) + T_X, K_YRsin(θ₀ + θ) + T_Y)

According to the trigonometric function,

cos(θ₀ + θ) = cosθ₀ cosθ − sinθ₀ sinθ

and

sin(θ₀ + θ) = sinθ₀ cosθ + cosθ₀ sinθ

The following equations can then be obtained:

and then

where:

θ, K_X, K_Y, T_X, and T_Y are all constants.

Let:

cosθK_X = KX₁

−sinθK_X = KX₂

T_X = KX₃

sinθK_Y = KY₁

cosθK_Y = KY₂

T_Y = KY₃

Then the previous equations can be rewritten as:

These equations can be used to calibrate the P’(x’, y’) back to P(x, y). There are three unknown coefficients in each equation for either the X axis or the Y axis.

Classical Three-Point Calibration Algorithm

The previous analysis produces the calibration equation for either the X axis or the Y axis. Each equation has three unknown coefficients for either the X axis or the Y axis. Therefore, if the information for the three irrelative reference points was available, one can construct a linear system of equations and get the solution for the unknown coefficients by solving the equations.

Assuming that the ideal coordinates of the three reference points are (x₀, y₀), (x₁, y₁), and (x₂, y₂), and their corresponding sampled coordinates are (x’0, y’0), (x’1, y’1), and (x’2, y’2), then the equations for both the X axis and the Y axis are:

and

The equations can then be rewritten as matrix type:

and

The calibration coefficients, KX₁, KX₂, KX₃, KY₁, KY₂, and KY₃, can then be calculated by solving the equations.

The results calculated by using the elimination method are as follows:

Let k = (x'₀ − x'₂)(y'₁ − y'₂) − (x'₁ − x'₂)(y'₀ − y'₂)

MMSE-Based Multipoint Calibration Algorithm

Using the coefficients calculated by the classical three-point calibration algorithm, the three reference points can be calibrated to the exact ideal position. However, for other points that are not close to the reference points, the calibration performance is not good enough, especially when the size of the touch screen is comparatively large. The experimental result in the Examples section also proves this problem. Therefore, consider using more than three reference points to get the best calibration coefficients.

Assuming that there are N + 1 reference points (N + 1 > 3), whose ideal coordinates are (x₀, y₀), (x₁, y₁), …, (x_N, y_N), and whose corresponding sampled coordinates are (x’₀, y’₀), (x’₁, y’₁), …, (x’_N, y’_N), then the equations are as follows:

and

Note that the number of equations in each equation system (N + 1) is larger than the number of unknown coefficients (3).

The purpose is to then calculate the best calibration coefficients to fit all the (N + 1) reference points. The method to get the best coefficients is to follow the MMSE rule. This is why the algorithm is called MMSE-based multipoint calibration algorithm.

Taking the X axis as an example, define an aim function:

where FX is the sum of the square error for the reference points.

The best coefficients of KX₁, KX₂, and KX₃ are the coefficients that can minimize the aim function, FX. Therefore, the following equations can be used:

that is:

These equations can be simplified as:

Let:

The equations can then be rewritten in matrix type as R • KX = BX, and the best coefficients, KX = R⁻¹ • BX, can then be calculated by solving the previous equations system.

Similar to the X axis, let:

then the best coefficients for the Y axis, KY = R⁻¹ • BY, can be calculated by solving the equation system, R • KY = BY.

The results calculated by using the elimination method are as follows:

Let:

The equation systems R • KX = BX and R • KY = BY can then be rewritten as:

and

It is easy to see that the equation systems have the same format as the former equation systems for the classical three-point algorithm:

The same formula can then be used to calculate the results of coefficients.

Let:

k = (a₀ − a₂)(b₁ − b₂) − (a₁ − a₂)(b₀ − b₂)

Then:

Analysis for MMSE-Based Multipoint Calibration Algorithm

By the a₀, a₁, a₂; b₀, b₁, b₂; c₀, c₁, c₂; and d₀, d₁, d₂ calculation, it is clear that a₀, a₁, a₂ can be treated as the weighted averages of x’_iby three different methods, and b₀, b₁, b₂ can be treated as the weighted averages of y’_i by three different methods. Similarly, c₀, c₁, c₂ can be treated as the weighted averages of x_i by three different methods, and d₀, d₁, d₂ can be treated as the weighted averages of y_i by three different methods.

Because the final equation systems have the same format as the previous equation systems for a classical three-point algorithm, the algorithm-based MMSE rule can be consider as another three-point algorithm. However, the difference in the MMSE-based algorithm is that, instead of direct information of any reference point being used, the weighted average information of the reference points is used. That is, first calculate three weighted average points (a₀, b₀), (a₁, b₁), (a₂, b₂) of N + 1 direct sampled points (x’₀, y’₀), (x’₁, y’₁), …, (x’_N, y’_N). The corresponding ideal coordinates of the three weighted average points are considered (c₀, d₀), (c₁, d₁), and (c₂, d₂). The MMSE-based algorithm is then equivalent to the classical three-point algorithm for these three weighted average points.

Steps for MMSE-Based Multipoint Calibration Algorithm

Complete the following steps for the MMSE-based multipoint calibration algorithm:

1. Select N +1 (N + 1 > 3) reference points (x₀, y₀), (x₁, y₁), … , (x_N, y_N).
2. Get the sampled coordinates of the reference points that were produced by the touch screen, (x’₀, y’₀), (x’₁, y’₁), …, (x’_N, y’_N).
3. Use the formulas given in this application note to calculate the calibration coefficients, KX and KY. These include the following:
   
   
4. In normal operation, use the calibration coefficients (KX, KY) and the following equations to calculate the calibration for point P’(x’, y’).
   
   

Examples

As shown in Figure 2, nine points were selected on the touch-screen. Their ideal coordinates are (3931, 3849), (2047, 3849), (164, 3849), (3931, 2047), (2047, 2047), (164, 2047), (3931, 246), (2047, 246), and (164, 246). The corresponding sampled coordinates produced by the touch screen are (3927, 3920), (2054, 3936), (193, 3943), (3911, 2119), (2054, 2127), (195, 2164), (3915, 331), (2050, 354), and (189, 371). Obviously, there are very large errors between the ideal coordinates and the corresponding sampled coordinates.

Three experiments follow that used the classical three-point algorithm, MMSE-based five-point algorithm, and MMSE-based nine-point algorithm:

• The classical three-point algorithm selects the three references as Point 1, Point 6, and Point 8 in Figure 2. The calibration coefficients are:
  
  KX₁ = +1.011238, KX₂ = −0.003952, and KX₃ = −24.638760 KY₁ = +0.009894, KY₂ = +1.005168, and KY₃ = −130.112700
  
  By using the previous coefficients, the results after calibration are listed in Table 1.
• The MMSE-based five-point algorithm selects the five references as Point 1, Point 3, Point 5, Point 7, and Point 9 in Figure 2. The calibration coefficients are:
  
  KX₁ = +1.009899, KX₂ = −0.002260, KX₃ = −23.715720 KY₁ = +0.008494, KY₂ = +1.006247, KY₃ = −121.821000
  
  By using the previous coefficients, the results after calibration are listed in Table 2.
• The MMSE-based nine-point algorithm, selects the nine references as Point 1, Point 2, Point 3, Point 4, Point 5, Point 6, Point 7, Point 8, and Point 9 in Figure 2. The calibration coefficients are:
  
  KX₁ = +1.011161, KX₂ = −0.001887, KX₃ = −25.777180 KY₁ = +0.009718, KY₂ = +1.006107, KY₃ = −126.258100
  
  By using the previous coefficients, the results after calibration are listed in Table 3.

Conclusion

As shown in the results of the previous experiments, no matter which calibration algorithm was used, the calibrated coordinates are much better than the direct sampled coordinates. In addition, by comparing these three experiments, it was concluded that

• The classical three-point calibration algorithm helps calibrate the three reference points to the ideal position. In addition, its performance is very good for the points close to the three reference points. However, for the points not close to the three reference points, the performance of the classical three-point calibration is not good enough. The sum of the square error for this algorithm was the largest of the three tested algorithms. Therefore, the classical three-point calibration algorithm is not a good choice for applications where touch screen sizes are comparatively large.
• For certain points (the points close to reference points), the MMSE-based multipoint calibration algorithm has worse performance than the classical three-point calibration algorithm. However, when looking at the whole touch screen, the MMSE-based multipoint calibration algorithm has a smaller sum of square error than the classical three-point algorithm because it uses the information of more than three reference points. Therefore, its performance is better than the classical three-point algorithm from a holistic view.
• For the MMSE-based multipoint calibration algorithm, the more reference points used, the better the performance is.

The results of these experiments are consentaneous with the mathematics.

Code Implementation

The code implementation of the calibration algorithm in C language is shown in the Coding section. When there are three reference points, the code implements the classical three-point calibration algorithm. When there are more than three reference points, the code implements the MMSE-based multi-point calibration algorithm. The code was tested with the ADuC7026 (the ADuC7026 is an MCU product from Analog Devices, Inc.). The results of the three example experiments were calculated by this code.

Coding

    #define N 9 // number of reference points for calibration 
algorithm 
signed short int ReferencePoint[N][2]; // ideal position of reference points 
signed short int SamplePoint[N][2]; // sampling position of reference points 
double KX1, KX2, KX3, KY1, KY2, KY3; // coefficients for calibration algorithm 
void Do_Calibration(signed short int *Px, signed short int *Py) // do calibration for point 
(Px, Py) using the calculated coefficients 
{ 
 *Px=(signed short int)(KX1*(*Px)+KX2*(*Py)+KX3+0.5); 
 *Py=(signed short int)(KY1*(*Px)+KY2*(*Py)+KY3+0.5); 
} 
int Get_Calibration_Coefficient() // calculate the coefficients for calibration 
algorithm: KX1, KX2, KX3, KY1, KY2, KY3 
{ 
 int i; 
 int Points=N; 
 double a[3],b[3],c[3],d[3],k; 
 if(Points<3) 
 { 
 return 0; 
 } 
 else 
 { 
 if(Points==3) 
 { 
 for(i=0; i< Points; i++) 
 { 
 a[i]=(double)(SamplePoint[i][0]); 
 b[i]=(double)(SamplePoint[i][1]); 
 c[i]=(double)(ReferencePoint[i][0]); 
 d[i]=(double)(ReferencePoint[i][1]); 
 } 
 } 
 else if(Points>3) 
 { 
 for(i=0; i<3; i++) 
 { 
 a[i]=0; 
 b[i]=0; 
 c[i]=0; 
 d[i]=0; 
 } 
 for(i=0; i< Points; i++) 
AN-1021 Application Note
Rev. 0 | Page 10 of 12 
 { 
 a[2]=a[2]+(double)(SamplePoint[i][0]); 
 b[2]=b[2]+(double)(SamplePoint[i][1]); 
 c[2]=c[2]+(double)(ReferencePoint[i][0]); 
 d[2]=d[2]+(double)(ReferencePoint[i][1]); 
 a[0]=a[0]+(double)(SamplePoint[i][0])*(double)(SamplePoint[i][0]); 
 a[1]=a[1]+(double)(SamplePoint[i][0])*(double)(SamplePoint[i][1]); 
 b[0]=a[1]; 
 b[1]=b[1]+(double)(SamplePoint[i][1])*(double)(SamplePoint[i][1]); 
 c[0]=c[0]+(double)(SamplePoint[i][0])*(double)(ReferencePoint[i][0]); 
 c[1]=c[1]+(double)(SamplePoint[i][1])*(double)(ReferencePoint[i][0]); 
 d[0]=d[0]+(double)(SamplePoint[i][0])*(double)(ReferencePoint[i][1]); 
 d[1]=d[1]+(double)(SamplePoint[i][1])*(double)(ReferencePoint[i][1]); 
 } 
 a[0]=a[0]/a[2]; 
 a[1]=a[1]/b[2]; 
 b[0]=b[0]/a[2]; 
 b[1]=b[1]/b[2]; 
 c[0]=c[0]/a[2]; 
 c[1]=c[1]/b[2]; 
 d[0]=d[0]/a[2]; 
 d[1]=d[1]/b[2]; 
 a[2]=a[2]/Points; 
 b[2]=b[2]/Points; 
 c[2]=c[2]/Points; 
 d[2]=d[2]/Points; 
 } 
 k=(a[0]-a[2])*(b[1]-b[2])-(a[1]-a[2])*(b[0]-b[2]); 
 KX1=((c[0]-c[2])*(b[1]-b[2])-(c[1]-c[2])*(b[0]-b[2]))/k; 
 KX2=((c[1]-c[2])*(a[0]-a[2])-(c[0]-c[2])*(a[1]-a[2]))/k; 
 KX3=(b[0]*(a[2]*c[1]-a[1]*c[2])+b[1]*(a[0]*c[2]-a[2]*c[0])+b[2]*(a[1]*c[0]-
a[0]*c[1]))/k; 
 KY1=((d[0]-d[2])*(b[1]-b[2])-(d[1]-d[2])*(b[0]-b[2]))/k; 
 KY2=((d[1]-d[2])*(a[0]-a[2])-(d[0]-d[2])*(a[1]-a[2]))/k; 
 KY3=(b[0]*(a[2]*d[1]-a[1]*d[2])+b[1]*(a[0]*d[2]-a[2]*d[0])+b[2]*(a[1]*d[0]-
a[0]*d[1]))/k; 
 return Points;
 } 
} 

References

Vidales, Carlos E. “How to Calibrate Touch Screens, Embedded Systems Design.” Embedded.com, May 31, 2002. Embedded Systems Design. May 27, 2009.

作者

Ning Jia

Ning Jia 是ADI亚洲客户应用中心的一名应用工程师，负责整个亚洲地区关于众多模拟产品的技术支持。Ning毕业于北京邮电大学，获得信号与信息处理硕士学位，于2007年加入ADI。