IndexError: list index out of range when using analyze_videos
Is there an existing issue for this?
- I have searched the existing issues
Bug description
Hi!
This is my first time using python and DeepLabCut, so any suggestions or comments are greatly appreciated! When I was trying to analyze the video file, it was running fine until it failed halfway. Then I tried the suggestion in #1527 but the same error occured again. Due to the capacity of the gpu, I am running on batchsize=1.
Operating System
Ubuntu 20.04.6 LTS
DeepLabCut version
2.3.8
DeepLabCut mode
multi animal
Device type
gpu
Steps To Reproduce
- Enter conda environment by conda activate DEEPLABCUTAUD
- Switch to ipython
- import deeplabcut
- Analyze videos
- Error appears after analyzing two videos
Relevant log output
In [28]: deeplabcut.analyze_videos('/home/fishlab4/Fish_audrey-Audrey-2023-10-31/config.yaml',['/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos'],videotype='.MP4',auto_track=True,allow_growth=True) Using snapshot-500000 for model /home/fishlab4/Fish_audrey-Audrey-2023-10-31/dlc-models/iteration-0/Fish_audreyOct31-trainset95shuffle1 /home/fishlab4/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/tensorflow/python/keras/engine/base_layer_v1.py:1694: UserWarning: `layer.apply` is deprecated and will be removed in a future version. Please use `layer.__call__` method instead. warnings.warn('`layer.apply` is deprecated and ' Activating extracting of PAFs 2023-11-15 11:33:07.319245: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1371 MB memory: -> device: 0, name: Quadro P620, pci bus id: 0000:18:00.0, compute capability: 6.1 Analyzing all the videos in the directory... Starting to analyze % /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Non-Resident1.MP4 Loading /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Non-Resident1.MP4 Duration of video [s]: 712.71 , recorded with 59.94 fps! Overall # of frames: 42720 found with (before cropping) frame dimensions: 2704 1520 Starting to extract posture from the video(s) with batchsize: 1 2023-11-15 11:33:09.072472: W tensorflow/core/common_runtime/bfc_allocator.cc:290] Allocator (GPU_0_bfc) ran out of memory trying to allocate 592.19MiB with freed_by_count=0. The caller indicates that this is not a failure, but this may mean that there could be performance gains if more memory were available. 0%| | 0/42720 [4:56:08<?, ?it/s] 34%|████████████████████████████████████████████████████▉ | 14493/42720 [4:50:29<9:24:09, 1.20s/it 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 42720/42720 [14:15:16<00:00, 1.20s/it] Video Analyzed. Saving results in /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos...██████████████████████████████████████████████████████████████████████████| 42720/42720 [14:15:16<00:00, 1.20s/it] Using snapshot-500000 for model /home/fishlab4/Fish_audrey-Audrey-2023-10-31/dlc-models/iteration-0/Fish_audreyOct31-trainset95shuffle1 Processing... /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Non-Resident1.MP4 Analyzing /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Non-Resident1DLC_dlcrnetms5_Fish_audreyOct31shuffle1_500000.h5 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 42720/42720 [00:22<00:00, 1871.29it/s] 42720it [00:51, 826.14it/s] The tracklets were created (i.e., under the hood deeplabcut.convert_detections2tracklets was run). Now you can 'refine_tracklets' in the GUI, or run 'deeplabcut.stitch_tracklets'. Processing... /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Non-Resident1.MP4 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2595/2595 [00:22<00:00, 115.13it/s] Starting to analyze % /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident1.MP4 Loading /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident1.MP4 Duration of video [s]: 711.71 , recorded with 59.94 fps! Overall # of frames: 42660 found with (before cropping) frame dimensions: 2704 1520 Starting to extract posture from the video(s) with batchsize: 1 63%|█████████████████████████████████████████████████████████████████████████████████████████████████▊ | 26746/42660 [8:55:39<5:17:41, 1.20s/it] 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 42660/42660 [14:16:36<00:00, 1.20s/it] Video Analyzed. Saving results in /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos... Using snapshot-500000 for model /home/fishlab4/Fish_audrey-Audrey-2023-10-31/dlc-models/iteration-0/Fish_audreyOct31-trainset95shuffle1 Processing... /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident1.MP4 Analyzing /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident1DLC_dlcrnetms5_Fish_audreyOct31shuffle1_500000.h5 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 42660/42660 [00:20<00:00, 2042.56it/s] 42660it [00:41, 1019.74it/s] The tracklets were created (i.e., under the hood deeplabcut.convert_detections2tracklets was run). Now you can 'refine_tracklets' in the GUI, or run 'deeplabcut.stitch_tracklets'. Processing... /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident1.MP4 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1490/1490 [00:14<00:00, 100.95it/s] Starting to analyze % /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Non-Resident1.MP4 Loading /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Non-Resident1.MP4 Duration of video [s]: 712.71 , recorded with 59.94 fps! Overall # of frames: 42720 found with (before cropping) frame dimensions: 2704 1520 Starting to extract posture from the video(s) with batchsize: 1 4%|██████▍ | 1728/42720 [34:31<13:42:12, 1.20s/it]--------------------------------------------------------------------------- IndexError Traceback (most recent call last) Cell In[28], line 1 ----> 1 deeplabcut.analyze_videos('/home/fishlab4/Fish_audrey-Audrey-2023-10-31/config.yaml',['/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos'],videotype='.MP4',auto_track=True,allow_growth=True) File ~/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/predict_videos.py:628, in analyze_videos(config, videos, videotype, shuffle, trainingsetindex, gputouse, save_as_csv, in_random_order, destfolder, batchsize, cropping, TFGPUinference, dynamic, modelprefix, robust_nframes, allow_growth, use_shelve, auto_track, n_tracks, calibrate, identity_only, use_openvino) 623 from deeplabcut.pose_estimation_tensorflow.predict_multianimal import ( 624 AnalyzeMultiAnimalVideo, 625 ) 627 for video in Videos: --> 628 AnalyzeMultiAnimalVideo( 629 video, 630 DLCscorer, 631 trainFraction, 632 cfg, 633 dlc_cfg, 634 sess, 635 inputs, 636 outputs, 637 destfolder, 638 robust_nframes=robust_nframes, 639 use_shelve=use_shelve, 640 ) 641 if auto_track: # tracker type is taken from default in cfg 642 convert_detections2tracklets( 643 config, 644 [video], (...) 651 identity_only=identity_only, 652 ) File ~/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/predict_multianimal.py:184, in AnalyzeMultiAnimalVideo(video, DLCscorer, trainFraction, cfg, dlc_cfg, sess, inputs, outputs, destfolder, robust_nframes, use_shelve) 172 PredicteData, nframes = GetPoseandCostsF( 173 cfg, 174 dlc_cfg, (...) 181 shelf_path, 182 ) 183 else: --> 184 PredicteData, nframes = GetPoseandCostsS( 185 cfg, 186 dlc_cfg, 187 sess, 188 inputs, 189 outputs, 190 vid, 191 nframes, 192 shelf_path, 193 ) 195 stop = time.time() 197 if cfg["cropping"] == True: File ~/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/predict_multianimal.py:511, in GetPoseandCostsS(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, shelf_path) 503 frame = rgba2rgb(frame) 504 dets = predict.predict_batched_peaks_and_costs( 505 dlc_cfg, 506 np.expand_dims(frame, axis=0), (...) 509 outputs, 510 ) --> 511 db[key] = dets[0] 512 del dets 513 elif counter >= nframes: IndexError: list index out of range In[30] deeplabcut.analyze_videos('/home/fishlab4/Fish_audrey-Audrey-2023-10-31/config.yaml',['/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Non-Resident1.MP4','/home/fishlab4/Fish_audr ...: ey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Predator1.MP4','/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Resident1.MP4','/home/fishlab4/Fish_audrey-Audrey-2023-10- ...: 31/videos/LD1_(Tank51)_Non-Resident2.MP4'],videotype='.MP4',auto_track=True) Using snapshot-500000 for model /home/fishlab4/Fish_audrey-Audrey-2023-10-31/dlc-models/iteration-0/Fish_audreyOct31-trainset95shuffle1 /home/fishlab4/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/tensorflow/python/keras/engine/base_layer_v1.py:1694: UserWarning: `layer.apply` is deprecated and will be removed in a future version. Please use `layer.__call__` method instead. warnings.warn('`layer.apply` is deprecated and ' Activating extracting of PAFs 2023-11-17 12:16:38.828896: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1371 MB memory: -> device: 0, name: Quadro P620, pci bus id: 0000:18:00.0, compute capability: 6.1 Starting to analyze % /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Non-Resident1.MP4 Loading /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Non-Resident1.MP4 Duration of video [s]: 712.71 , recorded with 59.94 fps! Overall # of frames: 42720 found with (before cropping) frame dimensions: 2704 1520 Starting to extract posture from the video(s) with batchsize: 1 --------------------------------------------------------------------------- | 1728/42720 [34:33<13:36:07, 1.19s/it] IndexError Traceback (most recent call last) Cell In[30], line 1 ----> 1 deeplabcut.analyze_videos('/home/fishlab4/Fish_audrey-Audrey-2023-10-31/config.yaml',['/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Non-Resident1.MP4','/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Predator1.MP4','/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Resident1.MP4','/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident2.MP4'],videotype='.MP4',auto_track=True) File ~/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/predict_videos.py:628, in analyze_videos(config, videos, videotype, shuffle, trainingsetindex, gputouse, save_as_csv, in_random_order, destfolder, batchsize, cropping, TFGPUinference, dynamic, modelprefix, robust_nframes, allow_growth, use_shelve, auto_track, n_tracks, calibrate, identity_only, use_openvino) 623 from deeplabcut.pose_estimation_tensorflow.predict_multianimal import ( 624 AnalyzeMultiAnimalVideo, 625 ) 627 for video in Videos: --> 628 AnalyzeMultiAnimalVideo( 629 video, 630 DLCscorer, 631 trainFraction, 632 cfg, 633 dlc_cfg, 634 sess, 635 inputs, 636 outputs, 637 destfolder, 638 robust_nframes=robust_nframes, 639 use_shelve=use_shelve, 640 ) 641 if auto_track: # tracker type is taken from default in cfg 642 convert_detections2tracklets( 643 config, 644 [video], (...) 651 identity_only=identity_only, 652 ) File ~/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/predict_multianimal.py:184, in AnalyzeMultiAnimalVideo(video, DLCscorer, trainFraction, cfg, dlc_cfg, sess, inputs, outputs, destfolder, robust_nframes, use_shelve) 172 PredicteData, nframes = GetPoseandCostsF( 173 cfg, 174 dlc_cfg, (...) 181 shelf_path, 182 ) 183 else: --> 184 PredicteData, nframes = GetPoseandCostsS( 185 cfg, 186 dlc_cfg, 187 sess, 188 inputs, 189 outputs, 190 vid, 191 nframes, 192 shelf_path, 193 ) 195 stop = time.time() 197 if cfg["cropping"] == True: File ~/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/predict_multianimal.py:511, in GetPoseandCostsS(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, shelf_path) 503 frame = rgba2rgb(frame) 504 dets = predict.predict_batched_peaks_and_costs( 505 dlc_cfg, 506 np.expand_dims(frame, axis=0), (...) 509 outputs, 510 ) --> 511 db[key] = dets[0] 512 del dets 513 elif counter >= nframes: IndexError: list index out of range
Anything else?
Below is my config file
# Project definitions (do not edit) Task: Fish_audrey scorer: Audrey date: Oct31 multianimalproject: true identity: true # Project path (change when moving around) project_path: /home/fishlab4/Fish_audrey-Audrey-2023-10-31 # Annotation data set configuration (and individual video cropping parameters) video_sets: /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Non-Resident1.MP4: crop: 0, 2704, 0, 1520 /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Predator1.MP4: crop: 0, 2704, 0, 1520 /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Resident1.MP4: crop: 0, 2704, 0, 1520 /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident1.MP4: crop: 0, 2704, 0, 1520 /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident2.MP4: crop: 0, 2704, 0, 1520 /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Predator1.MP4: crop: 0, 2704, 0, 1520 /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Resident1.MP4: crop: 0, 2704, 0, 1520 /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Non-Resident1.MP4: crop: 0, 2704, 0, 1520 /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Predator1.MP4: crop: 0, 2704, 0, 1520 /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Resident1.MP4: crop: 0, 2704, 0, 1520 individuals: - Cleaner - ClientPredator - ClientResident - ClientNonResident uniquebodyparts: [] multianimalbodyparts: - HeadTerminalMouth - HeadBasisUp - HeadBasisDown - TailBottomCorner - TailUpperCorner - TailJunctionBody - BodyMidUp - BodyMidDown bodyparts: MULTI! # Fraction of video to start/stop when extracting frames for labeling/refinement start: 0 stop: 1 numframes2pick: 20 # Plotting configuration skeleton: - bodypart1 - bodypart2 - bodypart2 - bodypart3 - bodypart1 - bodypart3 skeleton_color: black pcutoff: 0.6 dotsize: 12 alphavalue: 0.7 colormap: rainbow # Training,Evaluation and Analysis configuration TrainingFraction: - 0.95 iteration: 0 default_net_type: dlcrnet_ms5 default_augmenter: multi-animal-imgaug default_track_method: ellipse snapshotindex: -1 batch_size: 1 # Cropping Parameters (for analysis and outlier frame detection) cropping: false #if cropping is true for analysis, then set the values here: x1: 0 x2: 640 y1: 277 y2: 624 # Refinement configuration (parameters from annotation dataset configuration also relevant in this stage) corner2move2: - 50 - 50 move2corner: true
Code of Conduct
- I agree to follow this project's Code of Conduct