Development a Vision Based Seam Tracking System for None Destructive Testing Machines

45

Development a Vision Based Seam Tracking

System for

None Destructive Testing Machines

A. Nasser moradi, B. Mashalah abasi dezfulli, C. Seyed Enayatollah alavi

Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Khouzestan,Iran,nasser418822@yahoo.com

Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Khouzestan,Iran,abbsi_masha@yahoo.com

Department of Computer Engineering, shahid chamran university, Khouzestan,Iran Se_alavi@yahoo.com

Abstract

The automatic weld seam tracking is an important challenge in None Destructive Testing (NDT) systems for welded pipe inspection. In this Study, a machine vision based seam tracker, is developed and implemented, instead of old electro-mechanical system. A novel algorithm based on the weld image centered is presented, to reduce Environment conditions and improve the seam tracking accuracy. The weld seam images are taken by a camera arranged ahead of the machine and the centered is extracted as a parameter to detect the weld position, and offset between this point and central axis is computed and used as control parameter of servomotors. Adaptive multi step segmentation t technique is employed to increase the probable of real edge of the welds and improve the line fitting accuracy. This new approach offers some important technical advantages over the existing solutions to weld seam detection: It’s based on natural light and does not need any auxiliary light. The adaptive threshold segmentation technique applied, decrease Environmental lighting condition. It's accurate and stable in real time NDT testing machines. After a series of experiments in real industrial environment, it is demonstrated that accuracy of this method can improve the quality of NDT machines. The average tracking error is 1.5 pixels approximately 0.25mm..

Keywords:NDT, seam tracker, feature extraction, morphological filter, weld seam.

I. Introduction

46

in these systems, the vision sensors consists of a laser structural light, the laser-based vision sensor module incorporates a structured-light laser generator.

although laser scanner has found its important use in welding seam tracking, but disadvantage of these systems is that it needs an accessory device to produce the laser light, thus the sensor is complicated, and another one is that it's not suitable for very narrow seam weld (Fang et al,2012), and the laser scanner system is still relatively expensive ( Liu et al,2009), this limitation motivated us to develop a new vision based system without external special illumination.

In Some researches , fuzzy techniques implied for estimating weld seam position, but their time-consuming algorithm was a big problem in those system(Kuo et al, 2002),( Liqin et al ,2010), although some researchers proposed template matching methods (XizhangChen et al,2006), the measurement accuracy of seam extraction couldn't response real time needs.

In our approach the vision sensor is a CCD camera without auxiliary light source(fig.1), The camera installed a head of testing machine, capture the weld seam image for seam tracking process and, NDT instruments(ultrasonic probes) transmit and receive ultrasonic wave at weld seam for inspection of welded area. The position of seam with a fixing distance (to calibrate the CCD for first time, operator can change this distance) ,the deviation of NDT probes exist only in horizontal direction thus a servomotor coupled at a gearbox and facilitate automatic adjustment of the moving vehicle in horizontal direction. In control system the camera communicate weld seam image with an industrial PC(IPC) through the capture card (frame rate=30fps),weld seam edge and deviation is computed by image processing algorithm in IPC ,the servomotor used as actuator and get offset from IPC through a control device then adjust the system ,fig.2(a).

Figure .1 the structure of seam tracker

47

Figure .2 (a) hardware block diagram, (b) real implemented seam tracker

II. Image Processing Algorithm

The System is based on natural lighting, and many disturbances such as electrical noise, water effects exist. (Water is needed for creating a guidance region for ultrasonic wave transport in NDT ultrasonic test),.How to robustly extract the image feature in this condition is critical for the performance of image processing algorithm, figure .3 shows algorithm diagram.

A. Preprocessing

In real Industrial environment it may be a little difference between two serial samples, thus to Reduce computational time of execution, the change is computed and compare with an experience threshold, if the change is lower than threshold the old parameters is used. .

) (t+1)-f(t)

f=median(f (1)

Where and denote two consecutive frames of a video

   = 0 1 ) (t motion T f T f < >= 2 2 || || || || ) 2 ( ) 3 (

∑∑

2 | ( , )|

|| ||

, Indicate changes only when the is greater than a threshold (T is an experience threshold), then motion (t) is "0", the system will skip the next frame. Median filter is a nonlinear spatial filter whose response is based on ordering of the pixels contained in image area encompassed by the filter and replacing the central pixel with one having the median gray value, this filter is very effective to reduce impulse noise [1].

48 B. Edge based segmentation

The next step of our approach is to find the parallel weld seam edge, we try to save generality in local area, the gradient mask for horizontal and vertical edge is applied, this mask is effective to extract weld edge information, many edge detection techniques have been proposed, our experiments in real weld image all these technique have unsuccessfully results and some of them time-consuming(Canny).In our approach we used the horizontal and vertical gradient masks, and to save negative-edge sum square of these masks(coefficient 0.25 and sobel masks)

Figure .4: vertical and horizontal masks

Where im denote result picture,mydenote vertical mask, mx denote horizontal mask.

Fig 5, the result of applied method ,its show that edge extracted successfully but result combined real and false edge.

II. KJJJJHJHJ

Figurer.5: Edge segmentation Result ,(a).sample weld image,(b)vertical edge,(c) horizontal edge,(d) result sum (a) and)(b)

C. Threshold based segmentation

Selecting a proper threshold value is a key point in image segmentation [1].once the image gradient magnitude has been determined for each pixel in the image, it is then Necessary to determine which pixels will be classified as real edges, it is clear that pixels with the largest magnitude are more likely, to be edges, it is often impossible to select a single threshold that will

2

2 _{( * )}

) *

(f m_x f m_y

49

identify all edges while avoiding false edge responses, Then the edge detection process can be characterized by the probability of correct edge detection(Pratt , 2001)

∫

∞

=

t

D p G edge dG

P ( | ) (5)

And the probability of false detection

∫

∞

− =

t

F pG no edgedG

P ( | ) (6)

Where t is the edge detection threshold and p(G|edge) and p(G|no-edge) are the conditional probability densities of the edge gradient.

Figure.6: probable real and false edge

This error will be minimum if the threshold is chosen such that an edge is deemed present when

) | ( ) | ( ) | ( ) | ( edge G p edge no G p edge no G p edge G

p _≥ −

− (7)

Application of a statistical decision rule to determine the threshold value requires knowledge of the a priori edge probabilities and the conditional densities of the edge gradient. The value of a gradient magnitude is quite abstract and can vary significantly according to the input depth data, unless it has been normalized in some manner. It is therefore usually necessary to change the threshold for different images or even within the Seam image. One threshold technique, that is commonly used to choose a threshold on a per-image base, is to use the average of gradient strengths across the entire image to determine a threshold value. For that image. This is based on the idea that the mean of the gradient magnitude squared is roughly proportional to the signal-to-noise ratio (SNR) and that the cut-off threshold should be the root-mean-square (RMS) of this estimate (AdriaanMuller, 2013), as

2 2 ) * ( ) * (

4mean f mx f my

T ≅ + (8)

50

(a) (b)

Figure.7:(a) result sum vertical and horizontal masks,(b)result of Threshold based segmentation

D. Adaptive threshold

According to NDT pipe test standards (API 5L), the system must be in real time and automatic mode without operator interferences. Thus the segmentation technique must adapt itself with variable condition; the basic approach is based on visual inspection of the histogram. But in this seam image the gray level difference between the seam and the backgrounds not large enough, therefor, Some classical adaptive threshold approach (Rafael et al,2008),( Otsu N,1979),was applied. But they are not fit for this weld seam image.

The weld seams are approximately parallel with horizontal axis in seam image, thus the projection value of rows can be adapted, and obviously the rows of pixel with the seams have the max projection, thus the adaptive threshold value for segmenting from background can computed as d coefficient of max gray value of the pixel in the rows of the seam.

(9)

Where v(i) denote sum of gray values in row i, I (i,j) denote gray value in (i,j).

T =A*max(v(i)) (10)

T denote adaptive selected threshold, Denote coefficient (in our approach is 0.45)

Figure. 8: (a) Binary image after edge segmentation, (b) projection of the gray value in rows,

To test the effectiveness of proposed adaptive threshold, a sample weld seam image was used in fig.9, the proposed threshold and the threshold result using atsu[12] and canny method, from the result it can be seen that the proposed method is much better than other method.

∑

=

= m

j j i I i

v

1

) , ( )

51

Figure. 9: Adaptive thresholding result,(a)sample weld image,(b) edge segmentation previews step,(c)atsu method,(d) our method.

The output of adaptive threshold cannot be used directly for feature extraction to increase seam tracking accuracy and decrease executing time the seam must be thinned. The skelet finding procedure cost too much time due to its iterative working principle, and add some unwanted edge to the thinned picture, thus in our approach we only use erosion function with SE=[1 1 1].

E. feature Extraction

The position of the camera with respect to the head of machine is fixed any deviation of machine head from the seam cause the seam to move the left and right in image, thus the image feature must reflect the movement of the seam in row direction. The coordinate of the intersection points of horizontal midline of the image (red line in fig.9 and the two seam lines are image feature .Fig .10 shows the image feature defined in the weld seam, Red line interests the seams at the points which are defined as the image feature points.(f1 and f2),and point fc is center of line intercept f1 and f2, line with function f=c is the central axis and calibrated with operator. Since the image feature is defined as the coordinate of the intersection points of the horizontal midline of image and seam lines, thus the seam lines must be first extracted. Three methods are widely used to extract feature lines. Random sample consensus (RANSAC)[1] ,Hough transform[1] and least square method. RANSAC is iterative and much costly method. The feature extracted using hough transform is only a rough one [10] to increase the accuracy linear least square line fitting liner fitting technique is adopted.

52 Suppose two weld seams are

(11)

Where and are the slope and intercept of the feature lines. The seams are approximately parallel with horizontal axis thus sum of horizontal distance from feature line written as:

∑

= N

i yi kxi b

f

1

2 )

( (12)

After Minimize function f and Differentiate it with respect to parameters k and b the result can be written as follows:

       − = − − − − = x k y b X X X X Y Y X X k T T T ) ( ) ( ) ( )

( (13)

Where k and b denote the parameters of fitted line and X=[x1,x2,x3,…,xn]T , Y=[y1,y2,y3,…,yn]T ,X=x[1,1,…,1]T ,

Y =y[1,1,1,…,1]T and {(x1,y1),(x2,y2),…,(xn,yn)} are real edge seam pixels and point (x,y) is the average coordinates of edge points.

F. Calculating the Offset

The image features are calculated as:

   + = + = 2 2 2 1 1 1 b x k s b x k s (14)

and denote seam up-edge function and the seam down-edge, function, respectively. So the seam center function is calculated easily, represented as follows:

=0.5[( + )x+( )] (15)

As mentioned the feature points are intersection points vertical midline and and , thus ,( L denote length of weld image) thus the intercepts points respected as.

, (16)

53

, denote center of up-edge and down-edge lines and c denote system center point, calibrated by operator, dr is deviation from system center.

System is stable at real produce line has acceptable response in noised industrial environments. Fig 12. Show stability in weld seam extraction in damaged welded images. After a series of experiments in real industrial environment.

III .Results and Analysis

To test the performance of the proposed seam tracker system, NDT test and seam tracking experiment have been conducted using our implemented system (our seam tracker hardware and mechanically was implemented in real industrial environment in Ahwaz pipe mills) and feasibility of tracking system was evaluated. According to pipe produce standards (API 5L) a 56" welded metal pipe prepared. The camera system, calibrated and scaling factor computed (h=0.168mm) that denote relation between real offset and pixel size, the pipe tested at real industrial Environment and test repeated many time, table show test conditions. The result for 50 fames is shown in fig.11 it can be seen that max tracking error form real weld center is [-5,+3] pixel, and average error computed 1.5pixel,and 0.252 in millimeter. The seam tracking algorithm precision can meet most industrial NDT systems.

Table 1

Experiment And result

Figure .11: Tracking image space error in pixel

Test condition Seam tracker experiment result

Pipe diameter

test speed Frame rate

Frame size

Pipe length

Scaling factor (mm/pixel)

standard deviation(in

pixel)

Error average (pixel)

Error average (mm)

54

(a) (b) (c)

Fig.12: (a) unstable weld seam image (effected noise and water), (b):pre-processing,(c): Feature extraction and line fitting with proposed

IV. Conclusions

55

line has acceptable response in noised industrial environments. After a series of experiments in real industrial environment, it is demonstrated that accuracy of this method can improve the quality of NDT machines. The average tracking error is 1.5 pixels approximately 0.25mm.

Acknowledgements

The authors gratefully appreciate the help from AHWAZ PIPE MILLS for providing a real environment for NDT testing; comments and Suggestions from any anonymous reviewers are very valuable in the improvement of the quality of this paper

References

i. Bae Ky,Park JH(2006) “A study on development of inductive sensor for automatic weld seam tracking”,ourna of materials Processing Technology journal,vol.176,pp.111-116.

ii. Hong Yue,Kai Li( 2009) "Vision-based pipeline girth-welding robot and image

processing of weld seam", industrial Robot: An international Journal, Embedded Group Publishing Limited , Vol. 36 .issue 3,pp.284-289.

iii. Hsing-Chia Kuo,Li-Jen Wu( 2002) ,"An Image Tracking System For Welded Seams Using Fuzzy Logic", Journal of material processing technology,vol.120,pp.169-185.

iv. J.Liu,Z. Fan Olsen,K.Christensen and J.Kristensen,(2009) ”Using Active Contour Model for Feature Extraction in Camera –Based Seam Tracker of Arc Welding, ,IEEE/RSJ international conference on intelligent Robots and systems .October 11-15,Louis, USA,pp.5948-5954.

v. Mohamed Aly(2008) "Real time Detection of Lane Markers in Urban Streets" ,IEEE Intelligent Vehicles Symposium Eindhoven University of Technology, Eindhoven, The Netherlands , June 4-6.

vi. Otsu N(1979)" a threshold selection method from gray-level histogram", IEEE Transaction on systems, Man And Cybernetics, vol.9, pp.62-66.

vii. Rafael CG,Richard EW(2008)."Digital image processing". Publishing House of Electronics Industry, Beijing.

viii. Shi YH,WangGR,LiGi(2007)"Adaptive Robotic Welding System Using Laser Vision Sensing For Underwater " Engineering proceeding of IEEE International Conference on Control And Automation, ,china, pp.1213-1218.

ix. Simon AdriaanMuller( 2013)"Planar Segmentation of Range Images ", Thesis Master of Science in Applied Mathematics at Stellenbosch University, Department of Mathematical Sciences, Faculty of Science March.

56

xi. William K. Pratt(2001) ," Digital Image Processing: PIKS Inside, Third Edition book", John Wiley & Sons.

xii. XizhangChen .ShanbenChen . Tao Lin .Yucheng Lei(2006), "Practical method to locate the initial weld position using visual technology",Int J AdvManufTechnol ,vol. 30,pp. 663–668.

xiii. Zao jun Fang, Min. Tam(2010), "Visual Seam Tracking System For Butt Weld Of Thin Plate", Int AdvManut Technol journal, ,vol.49,pp.519-526.

xiv. Zao Jun Fang,DeXu(2012), "Application of Vision Sensor to Seam Tracking of butt in Container Manufacture", Embedded Visual System and its Applications on Robots jurnal",Bentham Science Publisher ltd,pp. 56-82.