### Trajectory-Based Ball Detection

### and Tracking with Aid of Homography and Tracking with Aid of Homography

### in Broadcast Tennis Video

### Xi Y Ni j Ji

### Xinguo Yu, Nianjuan Jiang, Ee Luang Ang

Visual Communications and Image Processing 2007 Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 6508 Proc. of SPIE IS&T Electronic Imaging, SPIE Vol. 6508

*Present by komod*

## I t d ti

## Introduction

### • The ball is the most important object in tennis (and in many kind of sports) in tennis (and in many kind of sports)

### • Very challenging problem

### – Camera motion

### – presence of many ball-like objects presence of many ball like objects

### – small size and the high speed of the ball Obj t i di ti i h bl

### – Object-indistinguishable

## I t d ti

## Introduction

### • Method

### – Trajectory-based

• the ball is the “most active” object in tennis video

• previous work： A Trajectory-based ball detection and tracking algorithm in broadcast tennis video, Proc of ICIP

Proc. of ICIP

### – Homography

### • Goal Goal

### – find projection locations of the ball on the ground

### – find landing positions

## I t d ti

## Introduction

## I t d ti

## Introduction

## F t P i t E t ti

## Feature Point Extraction

### • Court Segmentation

### • Court Segmentation

### – Find the court color range and paint all the pixels in this range with a single color

### pixels in this range with a single color

### – find the lines separating the audience from the playing field

### the playing field

• detecting the change pattern of color for each row and column of the image

### – paint the audience area in the court color.

## F t P i t E t ti

## Feature Point Extraction

### • Straight Line Detection

### gridding Hough transform – gridding Hough transform

### • Court Fitting

### – Detect the net and use it as reference

### – find the intersection of lines find the intersection of lines

## H h A i iti

## Homography Acquisition

### • Standard Frame

### whose lookat is the cluster center of all – whose lookat is the cluster center of all

### lookats of all the frames in the considered clip

### considered clip

### – The lookat of frame is a point in the

### l ld th t d t th

### real world that corresponds to the

### center of the frame

## H h A i iti

## Homography Acquisition

### • Disparity Measure of Two Court Images

– For i = 1 to 9

### • Measure Function

– Let CLet C_{std}_{td} be the court in the standard frame and Cbe the court in the standard frame and C_{t}_{trn}
denote the transformed court from the segmented
court in frame F

– For given H and FFor given H and F

## H h A i iti

## Homography Acquisition

### • Initial Matrix

### – transforms an image point X' (x

_{1}

### ', y

_{2}

### ', 1) to a i t X ( 1) i th i

### point X (x

_{1}

### , y

_{2}

### , 1) in another image – X = HX‘

### • Tuning of Homography

### • Tuning of Homography

### – The homograph matrix computed based on feature points

### A small hough space enclosing it

### – A small hough space enclosing it

## H h A i iti

## Homography Acquisition

### T i d

### • Tuning procedure

### • Frame transform Frame transform

### B ll L ti I Hitti F

### Ball Location In Hitting Frame

### • Hitting frame detection

### Find the sound emitted by the racket – Find the sound emitted by the racket

### hitting

### M X t l C ti di k d f

### • M. Xu et al, Creating audio keywords for event detection in soccer video, In Proc. of ICME

### ICME

### • Hitting racket detection

### – Maybe player tracking

## B ll C did t D t ti

## Ball Candidate Detection

### • Object segmentation from standard frame

### • Four sieve are used for non-ball object removal

– Court Sieve Θ_{1}

• filter out audience area

• filter out court linesfilter out court lines
– Ball Size Sieve Θ_{2}

• filter out the objects out of the ball-size range

• homography from ground model to standard frame

• homography from ground model to standard frame

• use a range of allowable ball sizes (estimate error)
– Ball Color Sieve Θ_{3}

• filter out the objects with too few ball color pixels
– Shape Sieve Θ_{4}

• filter out objects out of the range of width-to-height ratio

• 2.5 is suggested in previous paper

## B ll C did t D t ti

## Ball Candidate Detection

*– Each sieve is a Boolean function on domain Ο(F) *

*– The set of remaining objects is C(F)*

*• C(F) = {o : o אO(F), Θ*_{i}*(o)=1 for i = 1 to 4}*

### • Candidate Classification

Three features are use – Three features are use

• Size, color, and distance from other objects

– The ball-candidates are classified into 3 CategoriesThe ball candidates are classified into 3 Categories

### C did t T j t G ti

### Candidate Trajectory Generation

### • No detail explanation in this paper

– X. Yu et al, Trajectory-based ball detection and

*tracking of broadcast soccer video IEEE Transactions*
*tracking of broadcast soccer video, IEEE Transactions *
*on Multimedia, issue 6, 2006.*

### • Candidate Feature Plots (CFPs)

– CFP-y CFP l – CFP-l

• The algorithm is actually works on the CFP-l which are 3-D plots

### C did t T j t G ti

### Candidate Trajectory Generation

## T j t P i

## Trajectory Processing

### • Trajectory Confidence Index

### Let T be a candidate trajectory – Let T be a candidate trajectory

### – and λ

_{1}

### ,λ

_{2}

### ,…,λ

_{m}

### , be all properties of

### t j t T

### trajectory T

### – confidence index Ω(T)

## T j t P i

## Trajectory Processing

### • Trajectory Discrimination

## T j t P i

## Trajectory Processing

### • Ball Projection Location

### y = an

^{3}

### + bn

^{2}

### + cn + d – y = an

^{3}

### + bn

^{2}

### + cn + d.

### • Ball Land Detection

### – form a ball position function against frame number i, y = f(i) , y ( )

### – find the maximum of f '(i) between each pair of hittings

### each pair of hittings

## E i t l R lt Experimental Results

### • 5 clips

### t t d f 2 704 576

### • extracted from mpeg2 704x576

### • average time for acquiring ball g q g candidates

### • ALG for a frame is 86 15s on a

### • ALG _{new} for a frame is 86.15s on a P4/1.7Ghz PC with 512MB RAM

### • ALG _{old} is 19.21s

## E i t l R lt Experimental Results

BPL

## E i t l R lt

## Experimental Results

## E i t l R lt Experimental Results

### • average discrepancy of all detected balls from the groundtruth

### balls from the groundtruth

i l

previous result

## E i t l R lt Experimental Results

### • frames with inserted 3D projected virtual content

### • frames with inserted 3D projected virtual content

### • Homography in home surveillance video

### C l i d F t W k Conclusion and Future Works

• The previous algorithm mainly alleviated the challenges raised by causes besides camera motion

• The algorithm presented in this paper additionally g p p p y

counteracts the challenges brought to us by the camera motion

• The contributions of this paper are two-fold

*– it develops a procedure to robustly acquire an accurate *
homograph matrix of each frame

– it forms an improved version of ball detection and tracking algorithm

algorithm

• Two future works

– evolve the algorithm into an end-to-end system for ball detection and tracking of broadcast tennis videog

– analyze the tactics of players and winning-patterns, and hence produce rich indexing of broadcast tennis video by making use of the ball position

## Any Question? y

## Thank You

## E i t l R lt Experimental Results

### • 7 segments, total 120 s, mpeg1 video, Men’s Final of FRENCH OPEN 2003 Men s Final of FRENCH OPEN 2003

back