idtrackerai

Identification network

idCNN

This module contains the main Tensorflow operations needed to train and the idCNN

class id_CNN.ConvNetwork(params, training_flag=True, restore_index=None)[source]

Manages the main Tensorflow graph for the convolutional network that identifies images (idCNN)

Attributes

image_width (int) Width of the input image to the network
image_height (int) Height of the input image to the network
image_channels (int) Number of channes of the input image to the network
params (<NetworkParams object>) Object collecting the hyperparameters and other variables of the network
restore_index (int) Checkpoint index to the model that has to be restored
layers_to_optimise (list) List of strings with the names of the layers to be optimized
training (bool) Flag indicating whether the network is going to be used for training or to do fordward passes
ops_list (list) List with tensorflow operations
session (tf.Session()) Tensorflow session used to run the operations
is_restoring (bool) Flag indicating whether the network model should be restored from a previous checkpoint
restore_folder_conv (string) Path to the folder with the convolutional part of the model to be restored
restore_folder_fc_softmax (string) Path to the fully connected layers of the model to be restored
is_knowledge_transfers (bool) Flag indicating whether the knowledge from a previously trained model should be used
summary_op (tf.operation) Operation the merge the Tensorflow summaries to be visualized in the tensorboard
summary_writer_training (tf.operation) Summary writer for the training
summary_writer_validation (tf.operation) Summary writer for the validation
x_pl (tf.tensor) Tensor placeholder for the input images [None, image_width, image_height, image_channels]
y_target_pl (tf.tensor) Tensor placeholder for the labels of the input images [None, number_of_animals]
keep_prob_pl (tf.tensor) Tensor placeholder for the ‘keep probability’ in case of dropout
loss_weights_pl (tf.tensor) Tensor placeholder for the weights of the loss
y_logits (tf.operation) Output of the last fully connected layer
softmax_probs (tf.operation) Softmax probabilities computed from y_logits
predictions (tf.operation) Predictions of the given input images computed as the argmax of softmax_probs
loss (tf.operation) Loss function
optimisation_step (tf.operation) Optimization function
global_step (tf.global_step) Global tensorflow counter to save the checkpoints
accuracy (tf.operation) Accuracy of the predictions computed from the y_logits
individual_accuracy (tf.operation) Individual accuracy of the predictions for every class computed from the y_logits
weights (ndarray) Proportion of labels of every class in the collection of images in an epoch
fc_vector (tf.operation) Output of the second to last fully connected layer of the network
saver_conv (tf.operation) Operation to save the convolutional layers of the model
saver_fc_softmax (tf.operation) Operation to save the fully connected layers of the model
save_folder_conv (string) Path to the checkpoints of the convolutional layers of the model
save_folder_fc_softmax (string) Path to the checkpoints of the fully connected layers of th model

Methods

check_checkpoint_path(restore_folder)
compute_batch_weights(batch_labels) Returns the weights vector (batch_weights) for a batch of labes
compute_loss_weights(training_labels) Computes the label weights for all the images that will be train in an epoch
create_summaries_writers() Create the summary writers for thw tensorboard summaries
evaluation() Computes and returns the accuracy and individual accuracy of the predictions given by the model for the input images
get_feed_dict(batch) Returns the dictionary feed_dict with the variables and parameters needed to perform certain operations for a give batch
get_fully_connected_vectors(batch) Runs in the tensorflow session session the fc_vector operation
get_layers_to_optimize() Set layers of the network to be optimised during the training process
predict(batch) Runs in the tensorflow session session the softmax_probs and the predictions operations
reinitialize_conv_layers() Reinizializes the weights in the conv layers indicated in the list conv_layers_list.
reinitialize_softmax_and_fully_connected() Reinizializes the weights in the two last fully connected layers of the network
restore() Restores a previously trained model from a checkpoint folder
restore_classifier() Restores the weights of the convolutional layers from a checkpoint file of a previously trained model
restore_convolutional_layers() Restores the weights of the convolutional layers from a checkpoint file of a previously trained model
save() Saves the models in the correspoding save_folder_conv and save_folder_fc_softmax folders using the saver saver_conv and saver_fc_softmax
set_optimizer() Sets the optimization operation to be perform during the training process
set_savers() Create or retrieves the paths where the checkpoint files will be saved and initialize the objects to save them
train(batch) Runs in the tensorflow session session an optimization_step and the operations in ops_list for a given batch
validate(batch) Runs in the tensorflow session session the operations in ops_list for a given batch
weighted_loss() Loss function minimised during the training process
write_summaries(epoch_i, feed_dict_train, …) Writes the summaries using the summary_str_training and summary_writer_validation to be visualized in the Tensorboard
compute_batch_weights(batch_labels)[source]

Returns the weights vector (batch_weights) for a batch of labes

compute_loss_weights(training_labels)[source]

Computes the label weights for all the images that will be train in an epoch

create_summaries_writers()[source]

Create the summary writers for thw tensorboard summaries

evaluation()[source]

Computes and returns the accuracy and individual accuracy of the predictions given by the model for the input images

get_feed_dict(batch)[source]

Returns the dictionary feed_dict with the variables and parameters needed to perform certain operations for a give batch

get_fully_connected_vectors(batch)[source]

Runs in the tensorflow session session the fc_vector operation

get_layers_to_optimize()[source]

Set layers of the network to be optimised during the training process

predict(batch)[source]

Runs in the tensorflow session session the softmax_probs and the predictions operations

reinitialize_conv_layers()[source]

Reinizializes the weights in the conv layers indicated in the list conv_layers_list. The list can include the names ‘conv1’, ‘conv2’ or ‘conv3’

reinitialize_softmax_and_fully_connected()[source]

Reinizializes the weights in the two last fully connected layers of the network

restore()[source]

Restores a previously trained model from a checkpoint folder

restore_classifier()[source]

Restores the weights of the convolutional layers from a checkpoint file of a previously trained model

See also

saver_fc_softmax()

restore_convolutional_layers()[source]

Restores the weights of the convolutional layers from a checkpoint file of a previously trained model

See also

saver_conv()

save()[source]

Saves the models in the correspoding save_folder_conv and save_folder_fc_softmax folders using the saver saver_conv and saver_fc_softmax

set_optimizer()[source]

Sets the optimization operation to be perform during the training process

set_savers()[source]

Create or retrieves the paths where the checkpoint files will be saved and initialize the objects to save them

See also

createSaver()

train(batch)[source]

Runs in the tensorflow session session an optimization_step and the operations in ops_list for a given batch

See also

get_feed_dict()

validate(batch)[source]

Runs in the tensorflow session session the operations in ops_list for a given batch

See also

get_feed_dict()

weighted_loss()[source]

Loss function minimised during the training process

write_summaries(epoch_i, feed_dict_train, feed_dict_val)[source]

Writes the summaries using the summary_str_training and summary_writer_validation to be visualized in the Tensorboard

id_CNN.compute_accuracy(labels, logits)[source]

Computes the accuracy for a set of labels and logits

Parameters:

labels : tf.tensor

Tensor of shape [batch size, number of classes] with the labels of the input images in dense format

logits : tf.tensor

Tensor of shape [batch size, number of classes] with the output of the last fully connected layer

Returns:

accuracy : tf.constant

Accuracy for the given set of logits correspoding to the labels

id_CNN.compute_individual_accuracy(labels, logits, classes)[source]

Computes the individual accuracy for a set of labels and logits given the number of classes

Parameters:

labels : tf.tensor

Tensor of shape [batch size, number of classes] with the labels of the input images in dense format

logits : tf.tensor

Tensor of shape [batch size, number of classes] with the output of the last fully connected layer

classes : int

Number of classes

Returns:

individual_accuracies : tf.tensor

Tensor of shape [number of classes] with the accuracy for every class

id_CNN.createSaver(name, exclude_fc_and_softmax)[source]

Returns a tensorflow saver with a given name which will only save the convolutional layers if exclude_fc_and_softmax is True, and it will only save the fully connected layers if exclude_fc_and_softmax is False

id_CNN.get_checkpoint_subfolders(folder_name, sub_folders_names)[source]

Create if it does not exist the folder folder_name and the subfolders sub_folders_names. It returns the list of sub_paths created or

Network parameters

class network_params.NetworkParams(number_of_animals, cnn_model=0, learning_rate=None, keep_prob=None, use_adam_optimiser=False, scopes_layers_to_optimize=None, restore_folder=None, save_folder=None, knowledge_transfer_folder=None, image_size=None, number_of_channels=None, video_path=None)[source]

Manages the network hyperparameters and other variables related to the identification model (see idCNN)

Attributes

video_path (string) Path to the video file
number_of_animals (int) Number of animals in the video
learning_rate (float) Learning rate for the optimizer
keep_prob (float) Dropout probability
_restore_folder (string) Path to the folder where the model to be restored is
_save_folder (string) Path to the folder where the checkpoints of the current model are stored
_knowledge_transfer_folder (string) Path to the folder where the model to be used for knowledge transfer is saved
use_adam_optimiser (bool) Flag indicating to use the Adam optimizer with the parameters indicated in _[2]
scopes_layers_to_optimize (list) List with the scope names of the layers to be optimized
_cnn_model (int) Number indicating the model number to be used from the dictionary of models CNN_MODELS_DICT in id_CNN
image_size (tuple) Tuple (height, width, channels) for the input images
number_of_channels (int) Number of channels of the input image
kt_conv_layers_to_discard (str) convolutional layers to discard when performing knowledge transfer
.. [2] Kingma, Diederik P., and Jimmy Ba. “Adam: A method for stochastic optimization.” arXiv preprint arXiv:1412.6980 (2014).  

Stop training criteria (idCNN)

class stop_training_criteria.Stop_Training(number_of_animals, epochs_before_checking_stopping_conditions=10, check_for_loss_plateau=True, first_accumulation_flag=False)[source]

Stops the training of the network according to the conditions specified in __call__()

Attributes

num_epochs (int) Number of epochs before starting the training
number_of_animals (int) Number of animals in the video
epochs_before_checking_stopping_conditions (int) Number of epochs before starting to check the stopping conditions
overfitting_counter (int) Counts the number of consecutive overfitting epochs during training
check_for_loss_plateau (bool) Flag to check for a plateu in the loss of the validation
first_accumulation_flag (bool) Flag to indicate that it is the first step of the accumulation

Methods

__call__(loss_accuracy_training, …) Returns True when one of the conditions to stop the training is satisfied, otherwise it returns False

Store accurcay and loss

class store_accuracy_and_loss.Store_Accuracy_and_Loss(network, name, scope=None)[source]

Store the loss, accuracy and individual accuracy values computed during training and validation

Parameters:

_path_to_accuracy_error_data :

Path to save the lists loss, accuracy, individual_accuracy

name : string

‘Training’ or ‘Validation’

loss : list

List with the values of the loss

accuracy : list

List with the values of the accuracy

individual_accuracy : list

List with the values of the individual accuracies

number_of_epochs_completed :

Number of epochs completed

scope : string

‘Training’ if the class is instantiated during the training of the accumulation. ‘Pretraining’ if the class is instantiated during the training

Methods

append_data(loss_value, accuracy_value, …) Appends the loss_value, accuracy_value and individual_accuracy_value to their correspoding lists
load() Load the values stored in case there are any saved
plot([axes_handles, index, color, …]) Plots the accuracy and the individual accuracies for every epoch
plot_global_fragments(ax_handles, video, …) Plots the global fragments used for training until the current epoch
save(number_of_epochs_completed) Saves the values stored
append_data(loss_value, accuracy_value, individual_accuracy_value)[source]

Appends the loss_value, accuracy_value and individual_accuracy_value to their correspoding lists

load()[source]

Load the values stored in case there are any saved

plot(axes_handles=None, index=0, color='r', canvas_from_GUI=None, legend_font_color=None)[source]

Plots the accuracy and the individual accuracies for every epoch

plot_global_fragments(ax_handles, video, fragments, black=False, canvas_from_GUI=None)[source]

Plots the global fragments used for training until the current epoch

save(number_of_epochs_completed)[source]

Saves the values stored

Get data

class get_data.DataSet(number_of_animals=None, images=None, labels=None)[source]

Contains the images and labels to be used for training a particular model

Attributes

images (ndarray) Array of shape [num_images, height, width, channels] containing the images of the dataset
num_images (int) Number of images in the dataset
labels (ndarray) Array of shape [num_images, number_of_classes] containing the labels corresponding to the images in the dataset
number_of_animals (int) Number of classes in the dataset

Methods

consistency_check() Checks that the length of images and labels is the same
convert_labels_to_one_hot() Converts labels from dense format to one hot format See Also ——– shuffle_images_and_labels().
consistency_check()[source]

Checks that the length of images and labels is the same

convert_labels_to_one_hot()[source]

Converts labels from dense format to one hot format See Also ——– shuffle_images_and_labels()

get_data.dense_to_one_hot(labels, n_classes=2)[source]

Convert class labels from scalars to one-hot vectors.

get_data.duplicate_PCA_images(training_images, training_labels)[source]

Creates a copy of every image in training_images by rotating 180 degrees

Parameters:

training_images : ndarray

Array of shape [number of images, height, width, channels] containing the images to be rotated

training_labels : ndarray

Array of shape [number of images, 1] containing the labels corresponding to the training_images

Returns:

training_images : ndarray

Array of shape [2*number of images, height, width, channels] containing the original images and the images rotated

training_labels : ndarray

Array of shape [2*number of images, 1] containing the labels corresponding to the original images and the images rotated

get_data.shuffle_images_and_labels(images, labels)[source]

Shuffles images and labels with a random permutation, according to the number of examples

get_data.split_data_train_and_validation(number_of_animals, images, labels, validation_proportion=0.1)[source]

Splits a set of images and labels into training and validation sets

Parameters:

number_of_animals : int

Number of classes in the set of images

images : list

List of images (arrays of shape [height, width])

labels : list

List of integers from 0 to number_of_animals - 1

validation_proportion : float

The proportion of images that will be used to create the validation set.

Returns:

training_dataset : <DataSet object>

Object containing the images and labels for training

validation_dataset : <DataSet object>

Object containing the images and labels for validation

Epoch runner

Computes a given operation for an entire epoch divided in batches

class epoch_runner.EpochRunner(data_set, starting_epoch=0, print_flag=True, batch_size=None)[source]

Runs an epoch divided in batches for a given operation and a given data set (DataSet).

Attributes

epochs_completed (int) Number of epochs completed in the current step of the training
starting_epoch (int) Epoch at which the training step started
print_flag (bool) If True prints the values of the loss, the accucacy, and the individual accuracy at the end of the epoch
data_set (<DataSet object>) Object containing the images and labels to be passed through the network in the current epoch (see DataSet)
batch_size (int) Number of images to be passed through the network in each bach.

Methods

next_batch(batch_size) Returns the images and labels for the next batch to be computed.
run_epoch(name, store_loss_and_accuracy, …) Performs a given batch_operation for an entire epoch and stores the values of the loss and the accurcaies in a Store_Accuracy_and_Loss object for visualization.
next_batch(batch_size)[source]

Returns the images and labels for the next batch to be computed. Images and labels are extracted from a get_data.DataSet object

Parameters:

batch_size : int

Number of examples to be passed through the network in this batch

Returns:

images : ndarray

Array of shape [batch_size, height, width, channels] containing the images to be used in this batch

labels : ndarray

Array of shape [batch_size, number_of_classes] containing the labels corresponding to the images to be used in this batch

See also

get_data.DataSet

run_epoch(name, store_loss_and_accuracy, batch_operation)[source]

Performs a given batch_operation for an entire epoch and stores the values of the loss and the accurcaies in a Store_Accuracy_and_Loss object for visualization

Parameters:

name : string

A string to be printed in the epoch information. Typically ‘Training’ or ‘Validation’.

store_loss_and_accuracy : <Store_Accuracy_and_Loss object>

Object collecting the values of the loss, accurcay and individual accuracies (see store_accuracy_and_loss.Store_Accuracy_and_Loss)

batch_operation : func

Function to be run in the epoch

Returns:

feed_dict : dict

Dictionary with the parameters and variables needed to run the batch_operation. It is used to save the Tensorflow summaries if needed

Get predictions

Retrieves the predictions of the model idCNN (ConvNetwork) for a set of images

class get_predictions.GetPrediction(data_set)[source]

Manages the inference of the identities of a set of images

Attributes

data_set (<DataSet object>) Object containing the images whose labels has to be predicted (see DataSet)
softmax_probs (ndarray) Array of shape [number of images, number of animals] with the softmax vectors for every image in the data_set
predictions (ndarray) Array of shape [number of images, 1] with the predictions computed as the argmax of the softmax vector of every image (in dense format).
predictions_KNN (ndarray) Array of shape [number of images, 1] with the predictions for every image computed after a clustering of the second to last fully connected layer with KNN algorithm (in dense format)
_fc_vectors (ndarray) Array of shape [number of images, number of neurons in second to last fully connected layer]
batch_size (int) Size of the batch to send the images through the network to get the predictions

Methods

get_predictions_fully_connected_embedding(…) Not used in the current version of the algorithm.
get_predictions_softmax(batch_operation) Runs a batch_operation (typically predict()).
next_batch(batch_size) Return the next batch_size examples from this data set.
get_predictions_fully_connected_embedding(batch_operation, number_of_animals)[source]

Not used in the current version of the algorithm. Only for analysis.

get_predictions_softmax(batch_operation)[source]

Runs a batch_operation (typically predict())

Parameters:

batch_operation : func

Function to be run in the epoch

next_batch(batch_size)[source]

Return the next batch_size examples from this data set.

Parameters:

batch_size : int

Number of images to get for the next batch

Returns:

images : ndarray

Array of shape [batch_size, height, width, channels] with the images of the batch

get_predictions.kMeansCluster(vector_values, num_clusters, max_num_steps, stop_coeficient=0.0)[source]

Not used in the current version of the algorithm. Only for analysis.