Usage#

Basic usage#

Activate idtracker.ai’s Conda environment and launch the Segmentation app:

conda activate idtrackerai
idtrackerai

In the app you select your video(s), adjust the segmentation parameters until every animal is detected cleanly, and start tracking. When tracking finishes, the results are in a new session_[SESSION_NAME] folder next to your video (see Output structure).

Terminal usage#

idtracker.ai can also run without any graphical interface. This is useful for batch processing, automation, and tracking on remote machines over SSH.

Parameter files#

Click Save parameters in the Segmentation app to store the current configuration in a TOML file:

example.toml#
name = 'example'
video_paths = ['/home/user/idtrackerai/video_A.avi']
intensity_ths = [0, 155]
area_ths = [100.0, inf]
tracking_intervals = ""
number_of_animals = 8
use_bkg = false
check_segmentation = false
track_wo_identities = false
roi_list = ['+ Polygon [[138.0, 50.1], [992.9, 62.1], [996.9, 878.9]]']
exclusive_rois = false

Load the file to reopen the app exactly as you left it:

idtrackerai --load example.toml

Add --track to skip the app and start tracking directly:

idtrackerai --load example.toml --track

A parameter file can also hold any of the Advanced parameters. Basic parameters stored in a file act as defaults for the app. For example, if you always track 8 animals, put number_of_animals = 8 in a settings.toml file. Then idtrackerai --load settings.toml opens the app with 8 animals already set.

Combining files and command-line arguments#

Any parameter can also be passed on the command line as --parameter_name VALUE, and --load accepts several files:

idtrackerai --load my_settings.toml example.toml --number_of_animals 15 --track

Later sources override earlier ones:

  1. Files passed to --load are read in order, so example.toml overrides my_settings.toml.

  2. Arguments on the command line override every loaded file.

Tip

When the parameter file was created on a different computer (for example, when tracking on a remote server), override the paths that differ:

idtrackerai --load example.toml --video_paths path/in/remote/computer.avi --track

To track many videos in batch, call idtrackerai --load <file> --track once per video from a shell script or a loop.

Tracking log#

idtracker.ai reports its progress in the log. The log is shown live in the terminal (Anaconda Prompt on Windows) and written to idtrackerai.log in the current working directory. A copy is also saved in the session folder when the session ends.

Read the log, especially at the start of a session. It lists:

  • every parameter that was loaded, so you can confirm that your files and arguments were read as intended,

  • warnings about the video or the parameters,

  • the result of each tracking step.

If tracking stops with an error, the last lines of the log explain what went wrong. For common errors right after installing, see Installation Troubleshooting. For anything else, send the log file to info@idtracker.ai.

Advanced parameters#

Besides the basic parameters set in the Segmentation app (see example.toml), idtracker.ai has advanced parameters that you can set in a TOML file or on the command line. The defaults work well for most videos.

Important

  • Parameter names are case-insensitive: output_dir and OUTPUT_DIR are the same parameter.

  • In TOML files, write paths in 'single quotes'. In "double quotes", backslashes (\) start escape sequences, which breaks Windows paths (see the TOML documentation).

  • The TOML value '' is read as Python’s None, which means “use the automatic value”.

Output#

output_dir

Directory where the session folder is created. Default: '', the directory of the input video.

trajectories_formats

List of formats to save the trajectory files in. Options: "h5", "npy", "csv", "csv_tidy", "pickle" and "parquet" (see Formats). Default: ["h5", "npy", "csv"].

bounding_box_images_in_ram

Bounding box images are an intermediate step in creating identification images. If true, they stay in RAM until they are no longer needed. If false, they are written to disk and read back later. Only set it to true on very slow disks (HDDs), to speed up segmentation. Default: false.

data_policy

How much intermediate data to keep in the session folder after a successful tracking. Keeping more data enables more tools afterwards; keeping less saves disk space. Default: "idmatcher.ai".

Each policy deletes everything the previous one deletes, plus one more folder:

Data policies#

Policy

Deleted after tracking

Tools that still work

"all"

Nothing

All

"idmatcher.ai" (default)

bounding_box_images, crossings_detector

All

"knowledge_transfer"

Also identification_images

Video Generator, Validator, Knowledge transfer

"validation"

Also accumulation

Video Generator, Validator

"trajectories"

Also preprocessing

Video Generator

Here, “all tools” means the Video Generator, Validator, Knowledge transfer, idmatcher.ai and Cluster inspection. The folders are described in Output structure.

Background subtraction#

With background subtraction on, idtracker.ai estimates the background from a sample of frames, subtracts it from each frame, and thresholds the difference to find the animals.

background_subtraction_stat

Statistic used to compute the background from the sample of frames:

  • "median": works well in most cases (default).

  • "mean": the default in version 4.

  • "max": recommended for dark animals on a bright background.

  • "min": recommended for bright animals on a dark background.

Instead of a statistic, you can give the path to a background image (PNG) that you computed beforehand.

number_of_frames_for_background

Number of frames sampled to compute the background, equally spaced across the tracking intervals. More frames give a more accurate background but take more time and RAM. Default: 50.

Tip

Compute a background once with idtrackerai_background and reuse it in every tracking run:

idtrackerai_background video_A.avi -o background.png

Then set background_subtraction_stat = 'background.png'. Run idtrackerai_background -h for the other options (--stat, --n_frames and --tracking_intervals).

Tracking checks#

check_segmentation

If true, idtracker.ai stops the session when any frame has more blobs than animals. Extra blobs are shadows, reflections, dust or other objects detected as animals; they contaminate the tracking and lower identification accuracy. In the segmentation app this is the Stop tracking if #blobs > #animals checkbox. Default: false.

Parallel processing#

Segmentation and the creation of identification images run in parallel. The video is split into chunks (episodes), and each chunk is processed by an independent worker.

number_of_parallel_workers

Number of workers. Default: 0.

  • 0: half of the CPUs, up to a maximum of 4 (using more than 4 rarely speeds things up).

  • Positive: exactly that many workers. 1 disables multiprocessing.

  • Negative: the number of CPUs minus that many. For example, -2 on a 12-CPU computer gives 10 workers.

Warning

During segmentation, each worker can use up to 4 GB of RAM, so many workers can fill your memory quickly. If your computer has many cores (more than 10) or little RAM, monitor memory usage (with htop or free on Linux, Task Manager on Windows, or Activity Monitor on macOS) and lower the number of workers if needed.

frames_per_episode

Maximum number of frames per chunk. Fewer frames per episode means more, smaller chunks. Default: 500.

Knowledge transfer#

You can start training the identification network of a new session from the model trained in a previous session. This speeds up training when the videos are very similar: same lighting, same camera-to-arena distance, same type and size of animals.

knowledge_transfer_folder

Path to a previous session folder or its accumulation folder, for example '/home/username/session_test' or '/home/username/session_test/accumulation'. Default: '' (no transfer; the model starts from random weights and identities are assigned arbitrarily).

When it is set, idtracker.ai:

  • initializes the new identification model with the weights from that folder,

  • reuses the id_image_size and resolution_reduction of the previous session,

  • tries to transfer the identities, so that animal 1 in the new session is animal 1 in the previous one.

Note

  • Identities only transfer reliably when the recording conditions are nearly identical. Otherwise, identities can be swapped between animals.

  • If the folder has no trained model, idtracker.ai logs a warning and continues without transfer.

id_image_size

Side length, in pixels, of the square identification images. Default: '' (set automatically from the size of the animals).

resolution_reduction

Scale factor applied to the identification images, from 0 (maximum reduction) to 1 (no reduction). Use it to reduce computation when animals are large. Default: '' (automatic: images are only reduced if they would be larger than 80 pixels per side).

Note

The two automatic values depend on each other:

  • If neither is set (default), id_image_size is computed from the median animal size, and resolution_reduction is only applied if that size exceeds 80 pixels.

  • If only id_image_size is set, the images are only reduced if the animals are larger than that size.

  • If only resolution_reduction is set, id_image_size is computed from the rescaled animal size.

We recommend leaving both automatic, or inheriting them from a previous session with knowledge_transfer_folder.

Tip

idmatcher.ai is another way to match identities across sessions. It needs all sessions to share the same identification image size and resolution reduction, so knowledge_transfer_folder or id_image_size is useful there too.

Contrastive#

Since version 6.0.0, idtracker.ai identifies animals mainly with contrastive learning (described in the 2026 eLife publication):

  1. A ResNet is trained to place images of the same animal close together in an embedding space. It learns from positive pairs (two images from the same fragment) and negative pairs (images from two different fragments that coexist in time).

  2. Training stops when the silhouette score of the embeddings reaches a target value, or when it stops improving.

  3. The embeddings are clustered and each cluster becomes an identity.

  4. If enough images were identified, tracking is complete. Otherwise, the classical accumulation protocol of idtracker.ai’s idCNN takes over, starting from the contrastive results.

disable_contrastive

If true, skip contrastive learning and go directly to the accumulation protocol. Default: false.

contrastive_min_accumulation

Minimum fraction of images that contrastive learning must identify to be considered sufficient. Below this fraction, the accumulation protocol runs. Default: 0.5.

contrastive_batchsize

Number of image pairs per training batch. A batch of size \(N\) has \(N\) positive and \(N\) negative pairs, so \(4N\) images. Larger batches need more GPU memory. Default: 400.

contrastive_silhouette_target

Silhouette score (from 0 to 1) at which training stops. Set it to 1, which is never reached, to keep training until contrastive_patience runs out; this gives the best possible model at the cost of time. Default: 0.91.

contrastive_patience

Number of training steps without improvement of the silhouette score before training stops early. Default: 30.

contrastive_max_mbytes

Maximum RAM, in megabytes, used to preload identification images during training. Default: '' (a quarter of the available memory).

Advanced hyper-parameters#

Warning

These parameters change how the networks are trained or run. Use them with care.

threshold_early_stop_accumulation

Fraction of accumulated images at which the accumulation protocol stops early. Default: 0.999.

maximal_images_per_animal

Maximum number of images per animal used to train the idCNN in each accumulation step. Default: 3000.

device

Device where the neural networks run, passed to torch.device(). Typical values are "cpu", "cuda" and "cuda:0" (see the PyTorch documentation). Default: "" (chosen automatically).

torch_compile

If true, all models are compiled with torch.compile. This can make tracking faster, but does not work on every device. Default: false.

Settings file with all defaults#

The file below sets every parameter to its default value, so loading it has no effect. Copy it as a starting point and keep only the lines you want to change.

settings.toml#
# Segmentation app
name = ''
video_paths = ''
intensity_ths = [0, 130]
area_ths = [50.0, inf]
tracking_intervals = ""
number_of_animals = 0
use_bkg = false
check_segmentation = false
track_wo_identities = false
roi_list = ''
exclusive_rois = false

# Output
output_dir = ''
trajectories_formats = ["h5", "npy", "csv"]
bounding_box_images_in_ram = false
data_policy = 'idmatcher.ai'

# Background subtraction
background_subtraction_stat = 'median'
number_of_frames_for_background = 50

# Parallel processing
number_of_parallel_workers = 0
frames_per_episode = 500

# Knowledge and identity transfer
knowledge_transfer_folder = ''
id_image_size = ''
resolution_reduction = ''

# Contrastive
disable_contrastive = false
contrastive_min_accumulation = 0.5
contrastive_batchsize = 400
contrastive_silhouette_target = 0.91
contrastive_patience = 30
contrastive_max_mbytes = ''

# Advanced hyper-parameters
threshold_early_stop_accumulation = 0.999
maximal_images_per_animal = 3000
device = ""
torch_compile = false

idtrackerai -h#

Run idtrackerai -h to print every command-line argument:

Output of idtrackerai -h
options:
-h, --help

show this help message and exit

--version

show program’s version number and exit

General:
--load <path …>

A list of .toml files to load session parameters in increasing priority order. Later files override earlier ones for the same parameter.

--track

Start tracking directly from the terminal, bypassing the graphical user interface (GUI).

--name <str>

Name of the session. Defaults to the name of the video files if not specified.

--video_paths <path …>

List of paths to the video files to track. (default: []).

--intensity_ths <float float>

Blob intensity thresholds. If using background subtraction, the second value is the background difference threshold. (default: None).

--area_ths <float float>

Blob area thresholds (min, max). (default: None).

--tracking_intervals <pair_of_ints …>

Tracking intervals in frames. Examples: “0,100”, “[0,100]”, “[0,100] [150,200] …”. If not set, the whole video is tracked. (default: None).

--number_of_animals <int>

Number of different animals that appear in the video. (default: 0).

--use_bkg <bool>

Compute and extract background to improve blob identification. (default: False).

--resolution_reduction <float>

Video resolution reduction factor for identification images. 0 = maximum reduction, 1 = no reduction. (default: None).

--exclusive_rois <bool>

Treat each separate ROI as a closed group of identities. (default: False).

--track_wo_identities <bool>

Track the video without assigning identities (no AI identification). (default: False).

--roi_list <str …>

List of polygons defining the Region Of Interest (ROI). (default: None).

Output:
--output_dir <path>

Output directory where the session folder will be saved. Defaults to the parent directory of the video files. (default: None).

--trajectories_formats <str …>

List of formats for saving trajectory files. (choices: h5, npy, csv, csv_tidy, pickle, parquet) (default: [‘h5’, ‘npy’, ‘csv’]).

--bounding_box_images_in_ram <bool>

If true, bounding box images are kept in RAM until no longer needed. Otherwise, they are saved to disk and loaded as needed. (default: False).

--data_policy <str>

Data retention policy for the session folder after successful tracking. (choices: trajectories, validation, knowledge_transfer, idmatcher.ai, all) (default: idmatcher.ai).

Background Subtraction:
--background_subtraction_stat <str>

Statistical method to compute the background (choices: median, mean, max, min). Set a file path to load a custom background image. (default: median).

--number_of_frames_for_background <int>

Number of frames used to compute the background. More frames increase accuracy but also memory and computation time. (default: 50).

Parallel processing:
--number_of_parallel_workers <int>

Maximum number of parallel workers for segmentation and identification image creation. Negative: CPUs minus value. Zero: half of CPUs (max 4). One: no multiprocessing. (default: 0).

--frames_per_episode <int>

Maximum number of frames per video chunk (episode) for parallel processing. (default: 500).

Knowledge and identity transfer:
--knowledge_transfer_folder <path>

Path to a previous session or accumulation folder to transfer model knowledge from. (default: None).

--id_image_size <int>

Size (in pixels) of the identification images used for tracking. (default: []).

Checks:
--check_segmentation <bool>

Abort tracking if any frame has more blobs than animals (to avoid non-animal blobs contaminating identification). (default: False).

Contrastive:
--disable_contrastive <bool>

Skip contrastive learning and go directly to the accumulation protocol. (default: False).

--contrastive_max_mbytes <float>

Maximum RAM (in megabytes) for preloading identification images during contrastive training. (default: None).

--contrastive_min_accumulation <float>

Minimum fraction of images that need to be accumulated for taking the contrastive step as sufficient. (default: 0.5).

--contrastive_batchsize <int>

Number of image pairs per batch in contrastive training. Larger values require more GPU memory. (default: 400).

--contrastive_silhouette_target <float>

Minimum silhouette score (0 to 1) required for contrastive training to finish. (default: 0.91).

--contrastive_patience <int>

Number of training steps without improvement before early stopping contrastive training. (default: 30).

Advanced hyperparameter:
--threshold_early_stop_accumulation <float>

Fraction of accumulated images needed to early stop the accumulation process. (default: 0.999).

--maximal_images_per_animal <int>

Maximum number of images per animal used to train the identification CNN in each accumulation step. (default: 3000).

--device <str>

Device name for torch.device() (e.g., ‘cpu’, ‘cuda’, ‘cuda:0’, see https://pytorch.org/docs/stable/tensor_a ttributes.html#torch-device). Leave empty for automatic selection.

--torch_compile <bool>

If true, models are compiled with torch.compile for faster execution (may not be compatible with all devices, see https://pytorch.org/tutorials/intermediat e/torch_compile_tutorial.html). (default: False).

Deprecated:
--add_time_column_to_csv <str>

The parameter ‘add_time_column_to_csv’ has been removed.

--convert_trajectories_to_csv_and_json <str>

The parameter ‘convert_trajectories_to_csv_and_json’ has been removed.

--protocol3_action <str>

The parameter ‘protocol3_action’ has been removed.

--threshold_acceptable_accumulation <str>

The parameter ‘threshold_acceptable_accumulation’ has been removed.

--maximum_number_of_parachute_accumulations <str>

The parameter ‘maximum_number_of_parachute_accumulations’ has been removed.

--max_ratio_of_pretrained_images <str>

The parameter ‘max_ratio_of_pretrained_images’ has been removed.

--identity_transfer <str>

The parameter ‘identity_transfer’ has been removed.

Usage analytics#

idtracker.ai collects anonymous usage analytics to help us improve the software. Data is sent each time you run an idtracker.ai command from the terminal (not when you use idtracker.ai from Python). It contains general information only, such as your operating system, the idtracker.ai version and the command you ran. It contains no personal or sensitive information and is used only for research. You can read the source code to see exactly what is collected.

To opt out, do either of the following:

  • Open the About menu in the top-left corner of any idtracker.ai app and uncheck the analytics box.

  • Set the environment variable IDTRACKERAI_DISABLE_ANALYTICS to true or 1:

    • Linux/macOS: export IDTRACKERAI_DISABLE_ANALYTICS=true

    • Windows: set IDTRACKERAI_DISABLE_ANALYTICS=true

Either way, analytics stay disabled until you turn them back on.