Towards Unconstrained 2D Pose Estimation of the Human Spine

---

Abstract: We present SpineTrack, the first comprehensive dataset for 2D spine pose estimation in unconstrained settings, addressing a crucial need in sports analytics, healthcare, and realistic animation. Existing pose datasets often simplify the spine to a single rigid segment, overlooking the nuanced articulation required for accurate motion analysis. In contrast, SpineTrack annotates nine detailed spinal keypoints across two complementary subsets: a synthetic set comprising 25k annotations created using Unreal Engine with biomechanical alignment through OpenSim, and a real-world set comprising over 33k annotations curated via an active learning pipeline that iteratively refines automated annotations with human feedback. This integrated approach ensures anatomically consistent labels at scale, even for challenging, in-the-wild images. We further introduce SpinePose, extending state-of-the-art body pose estimators using knowledge distillation and an anatomical regularization strategy to jointly predict body and spine keypoints. Our experiments in both general and sports-specific contexts validate the effectiveness of SpineTrack for precise spine pose estimation, establishing a robust foundation for future research in advanced biomechanical analysis and 3D spine reconstruction in the wild.

Overview

Official repository for the CVPR 2025 workshop paper "Towards Unconstrained 2D Pose Estimation of the Human Spine" by Muhammad Saif Ullah Khan, Stephan Krauß, and Didier Stricker. This project provides an easy-to-install Python package, pretrained model checkpoints, the SpineTrack dataset, and evaluation scripts to reproduce our results.

• Inference
• SpineTrack Dataset
• Results and Evaluation

If you use our models or dataset, please cite our work as described in the Citation section.

---

Inference

Recommended Python Version: 3.9–3.12

For quick spinal keypoint estimation, we release optimized ONNX models via the spinepose package on PyPI:

pip install spinepose

On Linux/Windows with CUDA available, install the GPU version:

pip install spinepose[gpu]

Using the CLI

usage: spinepose [-h] (--version | --input_path INPUT_PATH) [--vis-path VIS_PATH] [--save-path SAVE_PATH] [--mode {xlarge,large,medium,small}] [--nosmooth] [--spine-only]

SpinePose Inference

options:
  -h, --help            show this help message and exit
  --version, -V         Print the version and exit.
  --input_path INPUT_PATH, -i INPUT_PATH
                        Path to the input image or video
  --vis-path VIS_PATH, -o VIS_PATH
                        Path to save the output image or video
  --save-path SAVE_PATH, -s SAVE_PATH
                        Save predictions in OpenPose format (.json for image or folder for video).
  --mode {xlarge,large,medium,small}, -m {xlarge,large,medium,small}
                        Model size. Choose from: xlarge, large, medium, small (default: medium)
  --nosmooth            Disable keypoint smoothing for video inference (default: enabled)
  --spine-only          Only use 9 spine keypoints (default: use all 37 keypoints)

For example, to run inference on a video and save only spine keypoints in OpenPose format:

spinepose --input_path path/to/video.mp4 --save-path output_path.json --spine-only

This automatically downloads the model weights (if not already present) and outputs the annotated image or video. Use spinepose -h to view all available options, including GPU usage and confidence thresholds.

Using the Python API

import cv2
from spinepose import SpinePoseEstimator

# Initialize estimator (downloads ONNX model if not found locally)
estimator = SpinePoseEstimator(device='cuda')

# Perform inference on a single image
image = cv2.imread('path/to/image.jpg')
keypoints, scores = estimator(image)
visualized = estimator.visualize(image, keypoints, scores)
cv2.imwrite('output.jpg', visualized)

Or, for a simplified interface:

from spinepose.inference import infer_image, infer_video

# Single image inference
results = infer_image('path/to/image.jpg', vis_path='output.jpg')

# Video inference with optional temporal smoothing
results = infer_video('path/to/video.mp4', vis_path='output_video.mp4', use_smoothing=True)

SpineTrack Dataset

SpineTrack is available on HuggingFace. The dataset comprises:

• SpineTrack-Real A collection of real-world images annotated with nine spinal keypoints in addition to standard body joints. An active learning pipeline, combining pretrained neural annotators and human corrections, refines keypoints across diverse poses.

• SpineTrack-Unreal A synthetic subset rendered using Unreal Engine, paired with precise ground-truth from a biomechanically aligned OpenSim model. These synthetic images facilitate pretraining and complement real-world data.

To download:

git lfs install
git clone https://huggingface.co/datasets/saifkhichi96/spinetrack

Alternatively, use wget to download the dataset directly:

wget https://huggingface.co/datasets/saifkhichi96/spinetrack/resolve/main/annotations.zip
wget https://huggingface.co/datasets/saifkhichi96/spinetrack/resolve/main/images.zip

In both cases, the dataset will download two zipped folders: annotations (24.8 MB) and images (19.4 GB), which can be unzipped to obtain the following structure:

spinetrack
├── annotations/
│   ├── person_keypoints_train-real-coco.json
│   ├── person_keypoints_train-real-yoga.json
│   ├── person_keypoints_train-unreal.json
│   └── person_keypoints_val2017.json
└── images/
    ├── train-real-coco/
    ├── train-real-yoga/
    ├── train-unreal/
    └── val2017/

All annotations are in COCO format and can be used with standard pose estimation libraries.

Results and Evaluation

We benchmark SpinePose against state-of-the-art lightweight pose estimation methods on COCO, Halpe, and our SpineTrack dataset. The results are summarized below, with SpinePose models highlighted in gray. Only 26 body keypoints are used for Halpe evaluations.

Method	Train Data	Kpts	COCO		Halpe26		Body		Feet		Spine		Overall		Params (M)	FLOPs (G)
			AP	AR	AP	AR	AP	AR	AP	AR	AP	AR	AP	AR
SimCC-MBV2	COCO	17	62.0	67.8	33.2	43.9	72.1	75.6	0.0	0.0	0.0	0.0	0.1	0.1	2.29	0.31
RTMPose-t	Body8	26	65.9	71.3	68.0	73.2	76.9	80.0	74.1	79.7	0.0	0.0	15.8	17.9	3.51	0.37
RTMPose-s	Body8	26	69.7	74.7	72.0	76.7	80.9	83.6	78.9	83.5	0.0	0.0	17.2	19.4	5.70	0.70
SpinePose-s	SpineTrack	37	68.2	73.1	70.6	75.2	79.1	82.1	77.5	82.9	89.6	90.7	84.2	86.2	5.98	0.72
SimCC-ViPNAS	COCO	17	69.5	75.5	36.9	49.7	79.6	83.0	0.0	0.0	0.0	0.0	0.2	0.2	8.65	0.80
RTMPose-m	Body8	26	75.1	80.0	76.7	81.3	85.5	87.9	84.1	88.2	0.0	0.0	19.4	21.4	13.93	1.95
SpinePose-m	SpineTrack	37	73.0	77.5	75.0	79.2	84.0	86.4	83.5	87.4	91.4	92.5	88.0	89.5	14.34	1.98
RTMPose-l	Body8	26	76.9	81.5	78.4	82.9	86.8	89.2	86.9	90.0	0.0	0.0	20.0	22.0	28.11	4.19
RTMW-m	Cocktail14	133	73.8	78.7	63.8	68.5	84.3	86.7	83.0	87.2	0.0	0.0	6.2	7.6	32.26	4.31
SimCC-ResNet50	COCO	17	72.1	78.2	38.7	51.6	81.8	85.2	0.0	0.0	0.0	0.0	0.2	0.2	36.75	5.50
SpinePose-l	SpineTrack	37	75.2	79.5	77.0	81.1	85.4	87.7	85.5	89.2	91.0	92.2	88.4	90.0	28.66	4.22
SimCC-ResNet50*	COCO	17	73.4	79.0	39.8	52.4	83.2	86.2	0.0	0.0	0.0	0.0	0.3	0.3	43.29	12.42
RTMPose-x*	Body8	26	78.8	83.4	80.0	84.4	88.6	90.6	88.4	91.4	0.0	0.0	21.0	22.9	50.00	17.29
RTMW-l*	Cocktail14	133	75.6	80.4	65.4	70.1	86.0	88.3	85.6	89.2	0.0	0.0	8.1	8.1	57.20	7.91
RTMW-l*	Cocktail14	133	77.2	82.3	66.6	71.8	87.3	89.9	88.3	91.3	0.0	0.0	8.6	8.6	57.35	17.69
SpinePose-x*	SpineTrack	37	75.9	80.1	77.6	81.8	86.3	88.5	86.3	89.7	89.3	91.0	88.9	89.9	50.69	17.37

For evaluation instructions and to reproduce the results reported in our paper, please refer to the evaluation branch of this repository:

git clone https://github.com/dfki-av/spinepose.git
cd spinepose
git checkout evaluation

The README in the evaluation branch provides detailed steps for setting up the evaluation environment and running the evaluation scripts on the SpineTrack dataset.

---

Citation

If this project or dataset proves helpful in your work, please cite:

@InProceedings{Khan_2025_CVPR,
    author    = {Khan, Muhammad Saif Ullah and Krau{\ss}, Stephan and Stricker, Didier},
    title     = {Towards Unconstrained 2D Pose Estimation of the Human Spine},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2025},
    pages     = {6171-6180}
}

License

This project is released under the CC-BY-NC-4.0 License. Commercial use is prohibited, and appropriate attribution is required for research or educational applications.