In baseball, a pitcher’s ability to conceal their intentions is as important as the quality of their arsenal. However, pitchers may occasionally exhibit subtle behavioral cues (tells) that can reveal pitch type or location before release.
Traditionally, identifying such cues has required painstaking manual review by coaches and scouts, a process that is both time-intensive and prone to confirmation bias. There is a natural opportunity to automate this discovery process using modern computer vision.
Recent advances in pose estimation, self-supervised visual representation learning, and object segmentation have made it feasible to extract rich, structured representations from sports video without large labeled datasets. Tools such as YOLO-Pose, DINO, and SAM have demonstrated strong generalization across diverse visual domains, yet their application to fine-grained behavioral analysis in baseball remains largely unexplored.
In this work, we present an automated pipeline for pitcher tell discovery from delivery clips. Given only pitch outcome labels, our system tracks the pitcher, localizes a semantically meaningful keyframe, extracts features, and applies a linear probe to test for predictive signal.