Usage#

Basic Usage#

Activate idtracker.ai’s Conda environment using conda activate idtrackerai, then run the command:

idtrackerai

to launch the Segmentation app, a graphical application where you can select videos, set parameters, and start the tracking process.

Terminal usage#

From the Segmentation app, you can either start tracking directly or save your parameters to a TOML file for reuse and automation:

example.toml#

name = 'example'
video_paths = ['/home/user/idtrackerai/video_A.avi']
intensity_ths = [0, 155]
area_ths = [100.0, inf]
tracking_intervals = ""
number_of_animals = 8
use_bkg = false
check_segmentation = false
track_wo_identities = false
roi_list = ['+ Polygon [[138.0, 50.1], [992.9, 62.1], [996.9, 878.9]]']
exclusive_rois = false

This file contains the full configuration defined in the Segmentation app and it can be loaded anytime with:

idtrackerai --load example.toml

to recover the app as you left it. Add the --track flag to start the tracking process directly from the terminal:

idtrackerai --load example.toml --track

This way, idtracker.ai can be run without a graphical interface, directly from the terminal — useful for automation, batch processing, or remote SSH tracking.

Parameter log

Every loaded parameter will be notified in the Tracking log, always read it while checking your parameters have been properly read.

Advanced parameters can also be loaded from a .toml file using the same --load argument (see Advanced parameters below).

Any parameter can also be passed directly on the command line as --PARAMETER VALUE.

An example of an advanced idtracker.ai command could be:

idtrackerai --load my_basic_settings.toml example.toml --track_wo_identities true --number_of_animals 15 --track

Note

Parameter files specified with --load are processed in the order they are listed, with later files overriding earlier ones for the same parameter. For example, in the command above, settings in example.toml will take precedence over those in my_basic_settings.toml. Additionally, any parameter specified directly on the command line will override all settings from the loaded files.

Tip

In the case of running idtracker.ai in remote (where the session parameters may have been created in another computer), it could be helpful to override, for example, the video paths from example.toml:

idtrackerai --load example.toml --video_paths path/in/remote/computer.avi --track

Tracking log#

Take care of your machine

Pay attention to your computer status during tracking (CPU, RAM and GPU usage). idtracker.ai can be very memory-intensive in some parts (see Parallel processing) and your computer can struggle on very long high resolution videos.

During tracking, idtracker.ai reports its progress through the log. It is displayed live in the terminal (Anaconda Prompt on Windows) and written to idtrackerai.log in the current working directory. Keep an eye on it — it reports loaded parameters, warnings, and the result of each tracking step.

When a critical error occurs, the last lines of the log describe what went wrong. You can send the log file to info@idtracker.ai for support. For common post-installation errors, see Installation Troubleshooting.

Advanced parameters#

In addition to the basic parameters shown in example.toml, idtracker.ai supports the following advanced parameters.

Important

All parameter names are case-insensitive.
Define path variables using 'single quotes' instead of "double ones" in the toml files to avoid backslashes (\) to trigger special characters (see TOML documentation to know more)
The value '' in a toml file is loaded as a Python’s None in idtracker.ai.

Output#

OUTPUT_DIR. Sets the directory path where the output session folder will be saved, by default it is the input video directory.
TRAJECTORIES_FORMATS. The output trajectory files can be saved in four different formats: H5DF ("h5"), Numpy ("npy"), Python’s pickle ("pickle"), CSV ("csv" or "csv_tidy"), and Parquet (parquet). Use this parameter to indicate the desired format(s) as a list of strings. Know more about these formats in Trajectory files.
BOUNDING_BOX_IMAGES_IN_RAM If true, bounding box images (an intermediate step in generating identification images) are kept in RAM until no longer needed. Otherwise, they are saved to disk and loaded when needed. Only set this to true when working with very slow disks (HDDs) to speed up segmentation.
DATA_POLICY. The tracking algorithm stores a significant amount of intermediate data alongside the trajectory files. This setting controls how much of it is kept — retaining more enables more post-processing tools, while retaining less saves disk space.

The available options are: "all", "idmatcher.ai", "knowledge_transfer", "validation" and "trajectories".

The default, "idmatcher.ai", balances storage efficiency with tool availability.

The table below details which data is preserved under each policy and which tools remain functional.

Data availability for different data policies#
	`all`	`idmatcher.ai`	`knowledge_transfer`	`validation`	`trajectories`
Trajectories	✅	✅	✅	✅	✅
Pre-processing folder	✅	✅	✅	✅	❌
Identification model folder	✅	✅	✅	❌	❌
Identification images	✅	✅	❌	❌	❌
Crossing detector folder	✅	❌	❌	❌	❌
Bounding box images	✅	❌	❌	❌	❌

Tool availability for different data policies#
	`all`	`idmatcher.ai`	`knowledge_transfer`	`validation`	`trajectories`
Video Generator	✅	✅	✅	✅	✅
Validator	✅	✅	✅	✅	❌
Knowledge transfer	✅	✅	✅	❌	❌
idmatcher.ai	✅	✅	❌	❌	❌
Cluster inspection	✅	✅	❌	❌	❌

Output defaults#

output_dir = ''
trajectories_formats = ["h5", "npy", "csv"]
bounding_box_images_in_ram = false
data_policy = "idmatcher.ai"

Background subtraction#

Animal segmentation works by subtracting the background from each frame and applying a threshold to the difference image. A stack of sample frames is used to estimate the background with a statistical method.

BACKGROUND_SUBTRACTION_STAT. Sets the statistical method used to compute the background from a sample of frames. Options are: "median" (default, works well in most cases), "mean" (default in version 4), "max" (recommended for dark animals on bright backgrounds), and "min" (for bright animals on dark backgrounds). You can also supply a file path to a pre-computed background image (PNG) instead of one of these keywords.

Tip

Pre-compute the background of a video with the command idtrackerai_background. The resulting PNG can be passed directly to BACKGROUND_SUBTRACTION_STAT, avoiding recomputation on every tracking run.
NUMBER_OF_FRAMES_FOR_BACKGROUND. Sets the number of frames used to generate the stack of sample frames. These are equally spaced along the tracking intervals. More frames means more accuracy but also more computing time and RAM usage.

Background subtraction defaults#

background_subtraction_stat = "median"
number_of_frames_for_background = 50

Tracking checks#

CHECK_SEGMENTATION. Frames containing more blobs than animals indicate a bad segmentation — non-animal blobs (shadows, reflections, dust, etc.) have been detected. These can contaminate the tracking algorithms and degrade identification accuracy. Set this to true to have idtracker.ai abort the session if a bad segmentation is detected.
```
check_segmentation = false
```
Note

This parameter appears on the segmentation app as Stop tracking if #blobs > #animals.

Parallel processing#

Some parts of idtracker.ai are parallelized — specifically, segmentation and identification image creation. The video is sliced into chunks, each processed by an independent worker.

NUMBER_OF_PARALLEL_WORKERS. Sets the number of workers used in the parallel parts of the application.
- A negative value indicates using as many workers as the total number of CPUs minus the specified value.
- A value of zero means running half of the total number of CPUs in the system. If the system has more than 8 cores, defaults to 4 workers, as using more than 4 cores does not provide significant speed-up.
- A positive value explicitly sets the number of workers to the specified value.
- One means not using multiprocessing at all.
The default value is 0.

Warning

During segmentation, every worker can use up to 4GB of memory, using too many workers might fill your RAM memory very fast. Computers with a large number of CPU cores (>10) should be monitored and the number of parallel workers should be adjusted accordingly. For users with limited RAM, consider reducing the number of parallel workers. Additionally, use monitoring tools like htop, top, or free on Linux, Task Manager or Resource Monitor on Windows, and Activity Monitor on macOS to keep an eye on your system’s resource usage.
FRAMES_PER_EPISODE. Sets the size of the video chunks (episodes). Less frames per episode means more parallel chunks.

Parallel processing defaults#

number_of_parallel_workers = 0
frames_per_episode = 500

Knowledge transfer#

You can use the knowledge acquired by the identification model of a previous video as a starting point for the training of the current one. This speeds up the identification training when the videos are very similar (same light conditions, distance from camera to arena, type and size of animals).

KNOWLEDGE_TRANSFER_FOLDER. Path to a previous session or accumulation folder whose trained model will be used to initialize the current session. Examples: '/home/username/session_test' or '/home/username/session_test/accumulation'.

When set, idtracker.ai will:
- load the model weights from the specified folder as a starting point for training the current identification model,
- adopt the same ID_IMAGE_SIZE and RESOLUTION_REDUCTION values from the previous session,
- attempt to transfer identities automatically when conditions are sufficiently similar (lighting, camera distance, animal size/appearance).
Note
- Video conditions must be nearly identical for reliable identity transfer; otherwise identities can be misassigned.
- If the folder does not contain the expected trained model, idtracker.ai will log a warning and proceed without transfer.
- By default (empty value), no knowledge transfer is performed and identification models start training from random weights with arbitrary identity assignments.
ID_IMAGE_SIZE. Identification images are square; by default their size is set automatically to match the animals’ size in the video. Set this to an integer to override the automatic size (pixels per side).
RESOLUTION_REDUCTION. When animal images are large (> 80 pixels per side), this parameter scales them down to reduce computation. It ranges from 0 (maximum reduction) to 1 (no reduction).

Note

When set automatically, ID_IMAGE_SIZE and RESOLUTION_REDUCTION interact as follows:

Neither defined (default): ID_IMAGE_SIZE is set from the average animal size; RESOLUTION_REDUCTION is applied only if the result would exceed 80 pixels.
Only ID_IMAGE_SIZE defined: resolution reduction is applied only if the average animal size exceeds the specified image size.
Only RESOLUTION_REDUCTION defined: ID_IMAGE_SIZE is set from the rescaled average animal size.

We recommend letting idtracker.ai set both automatically, or using KNOWLEDGE_TRANSFER_FOLDER to inherit them from a previous session.

Knowledge transfer defaults#

knowledge_transfer_folder = ''
id_image_size = ''
resolution_reduction = ''

Tip

There are alternative ways of transferring identities between tracking sessions. Check our tool idmatcher.ai, it requires the identification image size and the resolution reduction factor to be equal for all the sessions.

Contrastive#

Since version 6.0.0, idtracker.ai uses contrastive learning as its primary identification algorithm (described in the 2025 eLife publication). A ResNet network is trained to map animal images into an embedding space where images of the same individual cluster together. Training uses positive pairs (images from the same fragment) and negative pairs (images from different, co-existing fragments). As training progresses, the silhouette score of the resulting clusters rises toward the target threshold.

Once the target silhouette score is reached (or the patience limit is hit), the embedded images are clustered and identities are assigned. If the accumulated fraction of identified images is sufficient, tracking is complete. Otherwise, the classical accumulation protocol takes over, using the contrastive results as a warm start for idtracker.ai’s idCNN.

DISABLE_CONTRASTIVE. Skips the contrastive step to go directly to accumulation protocol.
CONTRASTIVE_MIN_ACCUMULATION. Minimum fraction of images that need to be accumulated for taking the contrastive step as sufficient. If the fraction of accumulated images is lower than this value, the accumulation protocol will be run.
CONTRASTIVE_BATCHSIZE. Number of pairs of images contained in a contrastive training batch. The more pairs of images, the more GPU memory will be needed. A batch of size \(N\) will contain \(N\) positive and \(N\) negative pairs of images, so \(4N\) images in total.
CONTRASTIVE_SILHOUETTE_TARGET. Minimum silhouette score required for contrastive to finish training. This score, since coming from a K-Means clustering, is ranged from zero to one. Set it to one (unachievable value) to maximize the quality of the contrastive model and stopping the training only because of the triggering of the patience.
CONTRASTIVE_PATIENCE. Number of steps without an improvement on the silhouette score to trigger the patience and early stopping the training during contrastive learning.
CONTRASTIVE_MAX_MBYTES. Maximum RAM (in megabytes) reserved for preloading identification images during contrastive training. By default (empty value), a quarter of the available system memory is used.

Contrastive defaults#

disable_contrastive = false
contrastive_min_accumulation = 0.5
contrastive_batchsize = 400
contrastive_silhouette_target = 0.91
contrastive_patience = 30
contrastive_max_mbytes = ''

Basic parameters#

The assignment of any basic parameter (like the ones in example.toml) in the settings file acts as a default. For example, if you always track videos with 8 animals, you can set number_of_animals = 8 in your settings file and, when running idtrackerai --load settings.toml, the segmentation app will open with 8 animals as the default.

Advanced hyper-parameters#

Warning

These parameters change the way the CNN is trained, use with care.

THRESHOLD_EARLY_STOP_ACCUMULATION. Fraction of accumulated images needed to early-stop the accumulation process.
MAXIMAL_IMAGES_PER_ANIMAL. Maximum number of images per animal used to train the CNN in each accumulation step.
DEVICE. Device name passed to torch.device() to specify where machine learning operations run, typically "cpu", "cuda", "cuda:0", etc. See Torch documentation. (default: empty string, automatic device selection).
TORCH_COMPILE. If set to true, all models will be compiled with torch.compile. This can make the software run faster but may not be compatible with all devices.

threshold_early_stop_accumulation = 0.999
maximal_images_per_animal = 3000
device = ""
torch_compile = false

File example#

An example settings file with all parameters set to their defaults (no effect):

example settings.toml#

# Segmentation app defaults
name = ''
video_paths = ''
intensity_ths = [0, 130]
area_ths = [50.0, inf]
tracking_intervals = ""
number_of_animals = 0
use_bkg = false
check_segmentation = false
track_wo_identities = false
roi_list = ''
exclusive_rois = false

# Output
output_dir = ''
trajectories_formats = ["h5", "npy", "csv"]
bounding_box_images_in_ram = false
data_policy = 'idmatcher.ai'

# Background subtraction
background_subtraction_stat = 'median'
number_of_frames_for_background = 50

# Parallel processing
number_of_parallel_workers = 0
frames_per_episode = 500

# Knowledge and identity transfer
knowledge_transfer_folder = ''
id_image_size = ''
resolution_reduction = ''

# Contrastive
disable_contrastive = false
contrastive_min_accumulation = 0.5
contrastive_batchsize = 400
contrastive_silhouette_target = 0.91
contrastive_patience = 30
contrastive_max_mbytes = ''

# Advanced hyper-parameters
threshold_early_stop_accumulation = 0.999
maximal_images_per_animal = 3000
device= ""
torch_compile = false

`idtrackerai -h`#

Use the command idtrackerai -h to print the list of all possible command line arguments in your terminal:

Output of idtrackerai -h

options:

-h, --help: show this help message and exit
--version: show program’s version number and exit

General:

--load <path …>: A list of .toml files to load session parameters in increasing priority order. Later files override earlier ones for the same parameter.
--track: Start tracking directly from the terminal, bypassing the graphical user interface (GUI).
--name <str>: Name of the session. Defaults to the name of the video files if not specified. (default: ).
--video_paths <path …>: List of paths to the video files to track. (default: []).
--intensity_ths <float float>: Blob intensity thresholds. If using background subtraction, the second value is the background difference threshold. (default: None).
--area_ths <float float>: Blob area thresholds (min, max). (default: None).
--tracking_intervals <pair_of_ints …>: Tracking intervals in frames. Examples: “0,100”, “[0,100]”, “[0,100] [150,200] …”. If not set, the whole video is tracked. (default: None).
--number_of_animals <int>: Number of different animals that appear in the video. (default: 0).
--use_bkg <bool>: Compute and extract background to improve blob identification. (default: False).
--resolution_reduction <float>: Video resolution reduction factor for identification images. 0 = maximum reduction, 1 = no reduction. (default: None).
--exclusive_rois <bool>: Treat each separate ROI as a closed group of identities. (default: False).
--track_wo_identities <bool>: Track the video without assigning identities (no AI identification). (default: False).
--roi_list <str …>: List of polygons defining the Region Of Interest (ROI). (default: None).

Output:

--output_dir <path>: Output directory where the session folder will be saved. Defaults to the parent directory of the video files. (default: None).
--trajectories_formats <str …>: List of formats for saving trajectory files. (choices: h5, npy, csv, csv_tidy, pickle, parquet) (default: [‘h5’, ‘npy’, ‘csv’]).
--bounding_box_images_in_ram <bool>: If true, bounding box images are kept in RAM until no longer needed. Otherwise, they are saved to disk and loaded as needed. (default: False).
--data_policy <str>: Data retention policy for the session folder after successful tracking. (choices: trajectories, validation, knowledge_transfer, idmatcher.ai, all) (default: idmatcher.ai).

Background Subtraction:

--background_subtraction_stat <str>: Statistical method to compute the background (choices: median, mean, max, min). Set a file path to load a custom background image. (default: median).
--number_of_frames_for_background <int>: Number of frames used to compute the background. More frames increase accuracy but also memory and computation time. (default: 50).

Parallel processing:

--number_of_parallel_workers <int>: Maximum number of parallel workers for segmentation and identification image creation. Negative: CPUs minus value. Zero: half of CPUs (max 4). One: no multiprocessing. (default: 0).
--frames_per_episode <int>: Maximum number of frames per video chunk (episode) for parallel processing. (default: 500).

Knowledge and identity transfer:

--knowledge_transfer_folder <path>: Path to a previous session or accumulation folder to transfer model knowledge from. (default: None).
--id_image_size <int>: Size (in pixels) of the identification images used for tracking. (default: []).

Checks:

--check_segmentation <bool>: Abort tracking if any frame has more blobs than animals (to avoid non-animal blobs contaminating identification). (default: False).

Contrastive:

--disable_contrastive <bool>: Skip contrastive learning and go directly to the accumulation protocol. (default: False).
--contrastive_max_mbytes <float>: Maximum RAM (in megabytes) for preloading identification images during contrastive training. (default: None).
--contrastive_min_accumulation <float>: Minimum fraction of images that need to be accumulated for taking the contrastive step as sufficient. (default: 0.5).
--contrastive_batchsize <int>: Number of image pairs per batch in contrastive training. Larger values require more GPU memory. (default: 400).
--contrastive_silhouette_target <float>: Minimum silhouette score (0 to 1) required for contrastive training to finish. (default: 0.91).
--contrastive_patience <int>: Number of training steps without improvement before early stopping contrastive training. (default: 30).

Advanced hyperparameter:

--threshold_early_stop_accumulation <float>: Fraction of accumulated images needed to early stop the accumulation process. (default: 0.999).
--maximal_images_per_animal <int>: Maximum number of images per animal used to train the identification CNN in each accumulation step. (default: 3000).
--device <str>: Device name for torch.device() (e.g., ‘cpu’, ‘cuda’, ‘cuda:0’, see https://pytorch.org/docs/stable/tensor_a ttributes.html#torch-device). Leave empty for automatic selection. (default: ).
--torch_compile <bool>: If true, models are compiled with torch.compile for faster execution (may not be compatible with all devices, see https://pytorch.org/tutorials/intermediat e/torch_compile_tutorial.html). (default: False).

Deprecated:

--add_time_column_to_csv <str>: The parameter ‘add_time_column_to_csv’ has been removed.
--convert_trajectories_to_csv_and_json <str>: The parameter ‘convert_trajectories_to_csv_and_json’ has been removed.
--protocol3_action <str>: The parameter ‘protocol3_action’ has been removed.
--threshold_acceptable_accumulation <str>: The parameter ‘threshold_acceptable_accumulation’ has been removed.
--maximum_number_of_parachute_accumulations <str>: The parameter ‘maximum_number_of_parachute_accumulations’ has been removed.
--max_ratio_of_pretrained_images <str>: The parameter ‘max_ratio_of_pretrained_images’ has been removed.
--identity_transfer <str>: The parameter ‘identity_transfer’ has been removed.

Usage Analytics#

idtracker.ai collects usage analytics to improve the software. This data is anonymized, does not include any sensitive or personally identifiable information, and is collected in every command line call to an idtracker.ai module (not collected for API calls). It is used solely for research purposes and contains general information such as the operating system, the version of idtracker.ai being used, and the command used to run it.

You may check out the source code to see exactly what data is collected.

If you want to opt-out of this data collection, go to the “About” menu located in the top-left corner of any idtracker.ai graphical application and uncheck the box for analytics. Alternatively, set the environment variable IDTRACKERAI_DISABLE_ANALYTICS to true or 1. For example:

On Linux/macOS: export IDTRACKERAI_DISABLE_ANALYTICS=true
On Windows: set IDTRACKERAI_DISABLE_ANALYTICS=true

Any of these actions will keep analytics disabled until you enable them again.