Home Original page

hed-python/examples at main · VisLab/hed-python

Jupyter notebooks for annotation

This directory contains Jupyter notebooks that provide interactive workflows for annotation, validation, and analysis.

📖 For detailed documentation, see the Jupyter notebooks section

Available Notebooks

Notebook Purpose
extract_json_template.ipynb Create JSON sidecar template from all event files in dataset
find_event_combinations.ipynb Extract unique combinations of column values
merge_spreadsheet_into_sidecar.ipynb Merge edited 4-column spreadsheet back into JSON sidecar
sidecar_to_spreadsheet.ipynb Convert JSON sidecar to 4-column spreadsheet for editing
summarize_events.ipynb Summarize event file contents and value distributions
validate_bids_dataset.ipynb Validate HED annotations in a BIDS dataset
validate_bids_dataset_with_libraries.ipynb Validate with HED library schemas
validate_bids_datasets.ipynb Batch validate multiple BIDS datasets

Quick Start

Prerequisites

  • Python 3.10+ required
  • These notebooks are only available in the GitHub repository, not in the PyPI distribution

Installation Option 1: Clone Repository

# Clone the repository
git clone https://github.com/hed-standard/hed-python.git
cd hed-python/examples

# Install dependencies
pip install hedtools jupyter notebook

# Launch Jupyter
jupyter notebook

Installation Option 2: Download Examples Only

# Download just the examples directory
svn export https://github.com/hed-standard/hed-python/trunk/examples

cd examples
pip install hedtools jupyter notebook
jupyter notebook

Typical Annotation Workflow

A recommended workflow for annotating a new BIDS dataset:

  1. 📊 Summarize events → Understand your event structure
  2. 📝 Extract template → Create initial JSON sidecar
  3. 📋 Convert to spreadsheet → Easier editing format
  4. ✏️ Edit annotations → Add HED tags in spreadsheet
  5. 🔄 Merge back → Convert spreadsheet to JSON
  6. ✅ Validate → Check for errors, iterate until valid

Notebook descriptions

Extract JSON template

Creates a JSON sidecar template based on all unique values found across all event files in your BIDS dataset.

Key parameters: dataset_path, exclude_dirs, skip_columns, value_columns

Find event combinations

Extracts all unique combinations of values across specified columns, useful for understanding column relationships or creating recoding mappings.

Key parameters: dataset_path, key_columns, output_path

Merge spreadsheet into sidecar

Merges a 4-column spreadsheet (with edited HED annotations) back into a BIDS JSON sidecar file (which could be empty).

Key parameters: spreadsheet_path, sidecar_path, description_tag, output_path

Sidecar to Spreadsheet

Converts the HED content of a JSON sidecar into a 4-column spreadsheet for easier viewing and editing.

Output format: column_name | column_value | description | HED

Summarize events

Provides an overview of event file structure, column names, and value distributions across the entire dataset.

Key parameters: dataset_path, exclude_dirs, skip_columns, value_columns

Validate BIDS dataset

Validates all HED annotations in a BIDS dataset against the specified schema.

Key parameters: dataset_path, check_for_warnings

Note: This validates HED only, not full BIDS compliance.

Validate BIDS dataset with libraries

Demonstrates validation using multiple schemas (base schema + library schemas like SCORE).

Use case: Datasets using specialized vocabularies beyond the base HED schema.

Validate multiple datasets

Convenience notebook for batch validation across multiple BIDS datasets.

Additional resources

Notes

  • BIDS structure required: These notebooks expect standard BIDS directory structure
  • Inheritance handling: Automatically processes BIDS sidecar inheritance
  • Exclude directories: Always exclude derivatives, code, stimuli, sourcedata
  • Skip columns: Typically skip onset, duration, sample (BIDS-predefined)
  • Value columns: Columns with continuous data (annotated with # placeholder)