This repository contains the project for the "Vision and Cognitive Systems" course.
This project provides a comprehensive suite of computer vision tools for detailed analysis of tennis matches. Our system is designed to extract key insights from tennis videos by detecting and tracking players and the ball, identifying important events like racket hits and ball bounces, and analyzing game dynamics. We also developed a novel method to predict player score probabilities based on a ball and score heatmap
The core contributions of this project include:
-
Ball Bounce Detection: We developed a new deep learning method that treats bounce detection as a visual pattern recognition problem. Our approach uses a lightweight Convolutional Neural Network (CNN) combined with DBSCAN clustering to achieve an
$84$ % accuracy in detecting true bounces within a 2-frame tolerance. -
Racket Hit Detection: We created an audio-visual approach that combines audio signals with visual cues to detect when a player hits the ball. This method demonstrated an
$84.74$ % accuracy within a 6-frame range. - Score Probability Prediction: Our system introduces a computer vision-based method to predict shot-by-shot score probabilities by analyzing the ball's estimated landing position and the opponent's location.
Our work represents a significant step forward in automated sports analysis, providing tools that can give viewers and analysts much deeper insights into tennis matches.
Our model achieved an 84.74% accuracy in detecting racket hits within 6 frames of the actual event and a 72.11% accuracy within a 2-frame tolerance. The system missed some hits in
We developed a custom dataset for bounce detection and formulated the problem as a binary classification task. The model's performance was evaluated using precision, recall, and F1-score. Our CNN model, when combined with DBSCAN clustering for exact frame prediction, achieved an 84% accuracy in detecting true bounces with a tolerance of
| Precision | Recall | F1 Score |
|---|---|---|
Using a fine-tuned ResNet-34 model, we predicted the coordinates of 14 key court points. The model was trained with MSE loss and achieved a total loss of 1.16 on the training set and 9.26 on the validation set.
The complete project report can be found here:
git clone https://github.com/danielevirzi/Tennis_Sport_Analysis.gitpip install -r requirements.txtpython main.pymain.py: Entry point for running the analysis pipeline.data_scrapert/: Scripts for scraping video data.ball_landing/: Code for ball landing detection.court_line_detector/: Code for detecting and predicting court lines.data/: Contains datasets used for training and evaluation.mini_court/: Code for court detection and line prediction.models/: Pre-trained models and architecture definitions.output/: Stores output videos and results.other/: Contains images, logos, and other media files.trackers/: Player and ball tracking algorithms.training/: Scripts and notebooks for training models.utils/: Utility functions for data processing and visualization.
For contributions, suggestions, or inquiries, feel free to reach out.
This project is open-source under the MIT License. See LICENSE for more information.