[cvPONG] A 3-D Pong Game Controlled Using Computer Vision
Techniques Quan Yu and Chris Wagner
Slide 2
[Project Idea] 3-D adaptation of the classic Pong game Score by
bouncing the ball past the opponents paddles Avoid letting the ball
go past your paddles Calculate the users paddle positions based on
the positions of the users hands and head Total of 3 Paddles
Implementation: Unity3D + EmguCV (a OpenCV wrapper)
Slide 3
Slide 4
[Methods Used] Background Subtraction - Chris Differentiate
user from background Skin Detection Chris, Quan Find skin pixels in
image Erosion & Dilation - Chris Clean image Finding Components
- Quan Find head and hands
Slide 5
[Background Subtraction] Use background subtraction to obtain
mask region for skin detection 1. Background Subtraction 1 Compare
object pixels intensities in capture frame with those of previously
captured background image How do we choose a good T?
Slide 6
[Background Subtraction] Use background subtraction to obtain
mask region for skin detection 2. Mixture of Gaussians Each pixel
modeled by a mixture of K Gaussian distributions Different
Gaussians represent different colors Mixture weights determined by
time proportions that colors stay in scene Learns probable
background colors are the ones which stay longer and are more
static Implemented in OpenCV: BackgroundSubtractorMOG()
Slide 7
[Background Subtraction] Use background subtraction to obtain
mask region for skin detection 2. Mixture of Gaussians An Improved
Adaptive Background Mixture Model for Real-time Tracking with
Shadow Detection P. KaewTraKulPong and R. Bowden
Slide 8
[Skin Detection] Apply skin detection in masked area to locate
users hands and head Elliptical Boundary Model Convert RGB color
space to normalized 2-D chrominance space by eliminating intensity
(handles varying illumination conditions) Most non-skin
chrominances concentrate on a single point (gray point) Want skin
chrominance classification distribution to avoid overlapping this
point (false positives) Skin chrominance distribution fits nicely
into a skewed normal distribution towards the gray point
Slide 9
[Skin Detection] Apply skin detection in masked area to locate
users hands and head Elliptical Boundary Model Chrominance
DistributionSingle Gaussian ModelElliptical Boundary An Elliptical
Boundary Model for Skin Color Detection Jae Y. Lee and Suk I.
Yoo
Slide 10
[Finding Components] Skin detection gives binary image in which
we can find three largest components (2 hands, head) 1. Erode then
dilate image Removes small components and noise 2. Calculate
contour boundaries of remaining components Use OpenCV method,
findContours 3. Calculate bounding box around three largest
contours 4. Take center of bounding boxes as position of users
paddles
Slide 11
[Demo]
Slide 12
[Problems] Trade-off between accuracy and performance
Shadows/Illumination Glasses! Similarly colored objects in
foreground Speed 8 fps in debug mode Erosion/Dilation removes too
many skin pixels Low-fidelity between frames EmguCV is not
completely compatible with Unity3D
Slide 13
[Outcome and Lessons Learned] A controlled environment is
better than a good algorithm in skin detection Vision-based input
is a more fun experience than the keyboard/mouse
Speed/interactivity is important in video games Simple background
subtraction does not work well Simple skin detection works pretty
good Noise is hard to remove
Slide 14
[Future Work] Incorporate MoG background subtraction to prune
skin detection areas Utilize shape analysis/face detection to
separate hands/head Smoother paddle movement between frames GPU
programming to improve performance Add more game elements
