PyTorch maDLC analysis crashes post-inference with `ValueError: need at least one array to stack`

Is there an existing issue for this?

  • I have searched the existing issues

Operating System

AlmaLinux 9.6 (Sage Margay) (HPC)

DeepLabCut version

3.0rc10/rc12

What engine are you using?

pytorch

DeepLabCut mode

multi animal

Device type

Nvidia L40S 48GB

Bug description 🐛

After training up a PyTorch model, I ran the analysis step in order to extract outlier frames.
The analysis step run to 100% and then threw an error.

100%|██████████| 1296914/1296914 [15:18:47<00:00, 23.53it/s]

Traceback (most recent call last):
  File "/home/ts/scripts/DLC_analyze_and_extract.py", line 71, in <module>
    sys.exit(main())
  File "/home/ts/scripts/DLC_analyze_and_extract.py", line 37, in main
    deeplabcut.analyze_videos(
  File "/home/ts/miniforge3/envs/dlc/lib/python3.12/site-packages/deeplabcut/compat.py", line 953, in analyze_videos
    return analyze_videos(
  File "/home/ts/miniforge3/envs/dlc/lib/python3.12/site-packages/deeplabcut/pose_estimation_pytorch/apis/videos.py", line 689, in analyze_videos
    _generate_assemblies_file(
  File "/home/ts/miniforge3/envs/dlc/lib/python3.12/site-packages/deeplabcut/pose_estimation_pytorch/apis/videos.py", line 883, in _generate_assemblies_file
    unique_bpts = np.stack(keypoint_preds[num_bodyparts:])
  File "/home/ts/miniforge3/envs/dlc/lib/python3.12/site-packages/numpy/core/shape_base.py", line 445, in stack
    raise ValueError('need at least one array to stack')

ValueError: need at least one array to stack

I tried with both autotrack = true and false, tried stitching the tracklets after the analysis.
All resulted in the same error.
Even a short 2 minute clip caused the same error.

With some trial and error I figured out that the _full.pickle file is not written, but the _full.pickle.dat, _full.pickle.dir, _full.pickle.bak, and _meta.pickle exist.

Steps To Reproduce

  1. Train pytorch model
  2. run analysis
  3. see failure once 100% is reached
  4. No _full.pickle

Relevant log output

Anything else?

Tried it both with rc10 and rc12, both with the same error.
Also both dekr_w48 and dlcrnet_stride32_ms5 show the same behavior.
Pytorch is 2.8.0+cu128

I can share a reproducible example via email with a 2 min video.

Code of Conduct