Trajectory-Based Ball Detection
and Tracking with Aid of Homography and Tracking with Aid of Homography
in Broadcast Tennis Video
Xi Y Ni j Ji
Xinguo Yu, Nianjuan Jiang, Ee Luang Ang
Visual Communications and Image Processing 2007 Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 6508 Proc. of SPIE IS&T Electronic Imaging, SPIE Vol. 6508
Present by komod
I t d ti
Introduction
• The ball is the most important object in tennis (and in many kind of sports) in tennis (and in many kind of sports)
• Very challenging problem
– Camera motion
– presence of many ball-like objects presence of many ball like objects
– small size and the high speed of the ball Obj t i di ti i h bl
– Object-indistinguishable
I t d ti
Introduction
• Method
– Trajectory-based
• the ball is the “most active” object in tennis video
• previous work： A Trajectory-based ball detection and tracking algorithm in broadcast tennis video, Proc of ICIP
Proc. of ICIP
– Homography
• Goal Goal
– find projection locations of the ball on the ground
– find landing positions
I t d ti
Introduction
I t d ti
Introduction
F t P i t E t ti
Feature Point Extraction
• Court Segmentation
• Court Segmentation
– Find the court color range and paint all the pixels in this range with a single color
pixels in this range with a single color
– find the lines separating the audience from the playing field
the playing field
• detecting the change pattern of color for each row and column of the image
– paint the audience area in the court color.
F t P i t E t ti
Feature Point Extraction
• Straight Line Detection
gridding Hough transform – gridding Hough transform
• Court Fitting
– Detect the net and use it as reference
– find the intersection of lines find the intersection of lines
H h A i iti
Homography Acquisition
• Standard Frame
whose lookat is the cluster center of all – whose lookat is the cluster center of all
lookats of all the frames in the considered clip
considered clip
– The lookat of frame is a point in the
l ld th t d t th
real world that corresponds to the
center of the frame
H h A i iti
Homography Acquisition
• Disparity Measure of Two Court Images
– For i = 1 to 9
• Measure Function
– Let CLet Cstdtd be the court in the standard frame and Cbe the court in the standard frame and Cttrn denote the transformed court from the segmented court in frame F
– For given H and FFor given H and F
H h A i iti
Homography Acquisition
• Initial Matrix
– transforms an image point X' (x
1', y
2', 1) to a i t X ( 1) i th i
point X (x
1, y
2, 1) in another image – X = HX‘
• Tuning of Homography
• Tuning of Homography
– The homograph matrix computed based on feature points
A small hough space enclosing it
– A small hough space enclosing it
H h A i iti
Homography Acquisition
T i d
• Tuning procedure
• Frame transform Frame transform
B ll L ti I Hitti F
Ball Location In Hitting Frame
• Hitting frame detection
Find the sound emitted by the racket – Find the sound emitted by the racket
hitting
M X t l C ti di k d f
• M. Xu et al, Creating audio keywords for event detection in soccer video, In Proc. of ICME
ICME
• Hitting racket detection
– Maybe player tracking
B ll C did t D t ti
Ball Candidate Detection
• Object segmentation from standard frame
• Four sieve are used for non-ball object removal
– Court Sieve Θ1
• filter out audience area
• filter out court linesfilter out court lines – Ball Size Sieve Θ2
• filter out the objects out of the ball-size range
• homography from ground model to standard frame
• homography from ground model to standard frame
• use a range of allowable ball sizes (estimate error) – Ball Color Sieve Θ3
• filter out the objects with too few ball color pixels – Shape Sieve Θ4
• filter out objects out of the range of width-to-height ratio
• 2.5 is suggested in previous paper
B ll C did t D t ti
Ball Candidate Detection
– Each sieve is a Boolean function on domain Ο(F)
– The set of remaining objects is C(F)
• C(F) = {o : o אO(F), Θi(o)=1 for i = 1 to 4}
• Candidate Classification
Three features are use – Three features are use
• Size, color, and distance from other objects
– The ball-candidates are classified into 3 CategoriesThe ball candidates are classified into 3 Categories
C did t T j t G ti
Candidate Trajectory Generation
• No detail explanation in this paper
– X. Yu et al, Trajectory-based ball detection and
tracking of broadcast soccer video IEEE Transactions tracking of broadcast soccer video, IEEE Transactions on Multimedia, issue 6, 2006.
• Candidate Feature Plots (CFPs)
– CFP-y CFP l – CFP-l
• The algorithm is actually works on the CFP-l which are 3-D plots
C did t T j t G ti
Candidate Trajectory Generation
T j t P i
Trajectory Processing
• Trajectory Confidence Index
Let T be a candidate trajectory – Let T be a candidate trajectory
– and λ
1,λ
2,…,λ
m, be all properties of
t j t T
trajectory T
– confidence index Ω(T)
T j t P i
Trajectory Processing
• Trajectory Discrimination
T j t P i
Trajectory Processing
• Ball Projection Location
y = an
3+ bn
2+ cn + d – y = an
3+ bn
2+ cn + d.
• Ball Land Detection
– form a ball position function against frame number i, y = f(i) , y ( )
– find the maximum of f '(i) between each pair of hittings
each pair of hittings
E i t l R lt Experimental Results
• 5 clips
t t d f 2 704 576
• extracted from mpeg2 704x576
• average time for acquiring ball g q g candidates
• ALG for a frame is 86 15s on a
• ALG new for a frame is 86.15s on a P4/1.7Ghz PC with 512MB RAM
• ALG old is 19.21s
E i t l R lt Experimental Results
BPL
E i t l R lt
Experimental Results
E i t l R lt Experimental Results
• average discrepancy of all detected balls from the groundtruth
balls from the groundtruth
i l
previous result
E i t l R lt Experimental Results
• frames with inserted 3D projected virtual content
• frames with inserted 3D projected virtual content
• Homography in home surveillance video
C l i d F t W k Conclusion and Future Works
• The previous algorithm mainly alleviated the challenges raised by causes besides camera motion
• The algorithm presented in this paper additionally g p p p y
counteracts the challenges brought to us by the camera motion
• The contributions of this paper are two-fold
– it develops a procedure to robustly acquire an accurate homograph matrix of each frame
– it forms an improved version of ball detection and tracking algorithm
algorithm
• Two future works
– evolve the algorithm into an end-to-end system for ball detection and tracking of broadcast tennis videog
– analyze the tactics of players and winning-patterns, and hence produce rich indexing of broadcast tennis video by making use of the ball position
Any Question? y
Thank You
E i t l R lt Experimental Results
• 7 segments, total 120 s, mpeg1 video, Men’s Final of FRENCH OPEN 2003 Men s Final of FRENCH OPEN 2003
back
