idtrackerai

Identification network

idCNN

This module contains the main Tensorflow operations needed to train and the idCNN

class id_CNN.ConvNetwork(params, training_flag=True, restore_index=None)[source]

Manages the main Tensorflow graph for the convolutional network that identifies images (idCNN)

Attributes:
image_width : int

Width of the input image to the network

image_height : int

Height of the input image to the network

image_channels : int

Number of channes of the input image to the network

params : <NetworkParams object>

Object collecting the hyperparameters and other variables of the network

restore_index : int

Checkpoint index to the model that has to be restored

layers_to_optimise : list

List of strings with the names of the layers to be optimized

training : bool

Flag indicating whether the network is going to be used for training or to do fordward passes

ops_list : list

List with tensorflow operations

session : tf.Session()

Tensorflow session used to run the operations

is_restoring : bool

Flag indicating whether the network model should be restored from a previous checkpoint

restore_folder_conv : string

Path to the folder with the convolutional part of the model to be restored

restore_folder_fc_softmax : string

Path to the fully connected layers of the model to be restored

is_knowledge_transfers : bool

Flag indicating whether the knowledge from a previously trained model should be used

summary_op : tf.operation

Operation the merge the Tensorflow summaries to be visualized in the tensorboard

summary_writer_training : tf.operation

Summary writer for the training

summary_writer_validation : tf.operation

Summary writer for the validation

x_pl : tf.tensor

Tensor placeholder for the input images [None, image_width, image_height, image_channels]

y_target_pl : tf.tensor

Tensor placeholder for the labels of the input images [None, number_of_animals]

keep_prob_pl : tf.tensor

Tensor placeholder for the ‘keep probability’ in case of dropout

loss_weights_pl : tf.tensor

Tensor placeholder for the weights of the loss

y_logits : tf.operation

Output of the last fully connected layer

softmax_probs : tf.operation

Softmax probabilities computed from y_logits

predictions : tf.operation

Predictions of the given input images computed as the argmax of softmax_probs

loss : tf.operation

Loss function

optimisation_step : tf.operation

Optimization function

global_step : tf.global_step

Global tensorflow counter to save the checkpoints

accuracy : tf.operation

Accuracy of the predictions computed from the y_logits

individual_accuracy : tf.operation

Individual accuracy of the predictions for every class computed from the y_logits

weights : ndarray

Proportion of labels of every class in the collection of images in an epoch

fc_vector : tf.operation

Output of the second to last fully connected layer of the network

saver_conv : tf.operation

Operation to save the convolutional layers of the model

saver_fc_softmax : tf.operation

Operation to save the fully connected layers of the model

save_folder_conv : string

Path to the checkpoints of the convolutional layers of the model

save_folder_fc_softmax : string

Path to the checkpoints of the fully connected layers of th model

Methods

compute_batch_weights(batch_labels) Returns the weights vector (batch_weights) for a batch of labes
compute_loss_weights(training_labels) Computes the label weights for all the images that will be train in an epoch
create_summaries_writers() Create the summary writers for thw tensorboard summaries
evaluation() Computes and returns the accuracy and individual accuracy of the predictions given by the model for the input images
get_feed_dict(batch) Returns the dictionary feed_dict with the variables and parameters needed to perform certain operations for a give batch
get_fully_connected_vectors(batch) Runs in the tensorflow session session the fc_vector operation
get_layers_to_optimize() Set layers of the network to be optimised during the training process
predict(batch) Runs in the tensorflow session session the softmax_probs and the predictions operations
reinitialize_conv_layers() Reinizializes the weights in the conv layers indicated in the list conv_layers_list.
reinitialize_softmax_and_fully_connected() Reinizializes the weights in the two last fully connected layers of the network
restore() Restores a previously trained model from a checkpoint folder
restore_classifier() Restores the weights of the convolutional layers from a checkpoint file of a previously trained model
restore_convolutional_layers() Restores the weights of the convolutional layers from a checkpoint file of a previously trained model
save() Saves the models in the correspoding save_folder_conv and save_folder_fc_softmax folders using the saver saver_conv and saver_fc_softmax
set_optimizer() Sets the optimization operation to be perform during the training process
set_savers() Create or retrieves the paths where the checkpoint files will be saved and initialize the objects to save them
train(batch) Runs in the tensorflow session session an optimization_step and the operations in ops_list for a given batch
validate(batch) Runs in the tensorflow session session the operations in ops_list for a given batch
weighted_loss() Loss function minimised during the training process
write_summaries(epoch_i, feed_dict_train, …) Writes the summaries using the summary_str_training and summary_writer_validation to be visualized in the Tensorboard
check_checkpoint_path  
compute_batch_weights(batch_labels)[source]

Returns the weights vector (batch_weights) for a batch of labes

compute_loss_weights(training_labels)[source]

Computes the label weights for all the images that will be train in an epoch

create_summaries_writers()[source]

Create the summary writers for thw tensorboard summaries

evaluation()[source]

Computes and returns the accuracy and individual accuracy of the predictions given by the model for the input images

get_feed_dict(batch)[source]

Returns the dictionary feed_dict with the variables and parameters needed to perform certain operations for a give batch

get_fully_connected_vectors(batch)[source]

Runs in the tensorflow session session the fc_vector operation

get_layers_to_optimize()[source]

Set layers of the network to be optimised during the training process

predict(batch)[source]

Runs in the tensorflow session session the softmax_probs and the predictions operations

reinitialize_conv_layers()[source]

Reinizializes the weights in the conv layers indicated in the list conv_layers_list. The list can include the names ‘conv1’, ‘conv2’ or ‘conv3’

reinitialize_softmax_and_fully_connected()[source]

Reinizializes the weights in the two last fully connected layers of the network

restore()[source]

Restores a previously trained model from a checkpoint folder

restore_classifier()[source]

Restores the weights of the convolutional layers from a checkpoint file of a previously trained model

See also

saver_fc_softmax()

restore_convolutional_layers()[source]

Restores the weights of the convolutional layers from a checkpoint file of a previously trained model

See also

saver_conv()

save()[source]

Saves the models in the correspoding save_folder_conv and save_folder_fc_softmax folders using the saver saver_conv and saver_fc_softmax

set_optimizer()[source]

Sets the optimization operation to be perform during the training process

set_savers()[source]

Create or retrieves the paths where the checkpoint files will be saved and initialize the objects to save them

See also

createSaver()

train(batch)[source]

Runs in the tensorflow session session an optimization_step and the operations in ops_list for a given batch

See also

get_feed_dict()

validate(batch)[source]

Runs in the tensorflow session session the operations in ops_list for a given batch

See also

get_feed_dict()

weighted_loss()[source]

Loss function minimised during the training process

write_summaries(epoch_i, feed_dict_train, feed_dict_val)[source]

Writes the summaries using the summary_str_training and summary_writer_validation to be visualized in the Tensorboard

id_CNN.compute_accuracy(labels, logits)[source]

Computes the accuracy for a set of labels and logits

Parameters:
labels : tf.tensor

Tensor of shape [batch size, number of classes] with the labels of the input images in dense format

logits : tf.tensor

Tensor of shape [batch size, number of classes] with the output of the last fully connected layer

Returns:
accuracy : tf.constant

Accuracy for the given set of logits correspoding to the labels

id_CNN.compute_individual_accuracy(labels, logits, classes)[source]

Computes the individual accuracy for a set of labels and logits given the number of classes

Parameters:
labels : tf.tensor

Tensor of shape [batch size, number of classes] with the labels of the input images in dense format

logits : tf.tensor

Tensor of shape [batch size, number of classes] with the output of the last fully connected layer

classes : int

Number of classes

Returns:
individual_accuracies : tf.tensor

Tensor of shape [number of classes] with the accuracy for every class

id_CNN.createSaver(name, exclude_fc_and_softmax)[source]

Returns a tensorflow saver with a given name which will only save the convolutional layers if exclude_fc_and_softmax is True, and it will only save the fully connected layers if exclude_fc_and_softmax is False

id_CNN.get_checkpoint_subfolders(folder_name, sub_folders_names)[source]

Create if it does not exist the folder folder_name and the subfolders sub_folders_names. It returns the list of sub_paths created or

Network parameters

class network_params.NetworkParams(number_of_animals, cnn_model=0, learning_rate=None, keep_prob=None, use_adam_optimiser=False, scopes_layers_to_optimize=None, restore_folder=None, save_folder=None, knowledge_transfer_folder=None, image_size=None, number_of_channels=None, video_path=None)[source]

Manages the network hyperparameters and other variables related to the identification model (see idCNN)

Attributes:
video_path : string

Path to the video file

number_of_animals : int

Number of animals in the video

learning_rate : float

Learning rate for the optimizer

keep_prob : float

Dropout probability

_restore_folder : string

Path to the folder where the model to be restored is

_save_folder : string

Path to the folder where the checkpoints of the current model are stored

_knowledge_transfer_folder : string

Path to the folder where the model to be used for knowledge transfer is saved

use_adam_optimiser : bool

Flag indicating to use the Adam optimizer with the parameters indicated in _[2]

scopes_layers_to_optimize : list

List with the scope names of the layers to be optimized

_cnn_model : int

Number indicating the model number to be used from the dictionary of models CNN_MODELS_DICT in id_CNN

image_size : tuple

Tuple (height, width, channels) for the input images

number_of_channels : int

Number of channels of the input image

kt_conv_layers_to_discard : str

convolutional layers to discard when performing knowledge transfer

.. [2] Kingma, Diederik P., and Jimmy Ba. “Adam: A method for stochastic optimization.” arXiv preprint arXiv:1412.6980 (2014).

Stop training criteria (idCNN)

class stop_training_criteria.Stop_Training(number_of_animals, epochs_before_checking_stopping_conditions=10, check_for_loss_plateau=True, first_accumulation_flag=False)[source]

Stops the training of the network according to the conditions specified in __call__()

Attributes:
num_epochs : int

Number of epochs before starting the training

number_of_animals : int

Number of animals in the video

epochs_before_checking_stopping_conditions : int

Number of epochs before starting to check the stopping conditions

overfitting_counter : int

Counts the number of consecutive overfitting epochs during training

check_for_loss_plateau : bool

Flag to check for a plateu in the loss of the validation

first_accumulation_flag : bool

Flag to indicate that it is the first step of the accumulation

Methods

__call__(loss_accuracy_training, …) Returns True when one of the conditions to stop the training is satisfied, otherwise it returns False

Store accurcay and loss

class store_accuracy_and_loss.Store_Accuracy_and_Loss(network, name, scope=None)[source]

Store the loss, accuracy and individual accuracy values computed during training and validation

Parameters:
_path_to_accuracy_error_data :

Path to save the lists loss, accuracy, individual_accuracy

name : string

‘Training’ or ‘Validation’

loss : list

List with the values of the loss

accuracy : list

List with the values of the accuracy

individual_accuracy : list

List with the values of the individual accuracies

number_of_epochs_completed :

Number of epochs completed

scope : string

‘Training’ if the class is instantiated during the training of the accumulation. ‘Pretraining’ if the class is instantiated during the training

Methods

append_data(loss_value, accuracy_value, …) Appends the loss_value, accuracy_value and individual_accuracy_value to their correspoding lists
load() Load the values stored in case there are any saved
plot([axes_handles, index, color, …]) Plots the accuracy and the individual accuracies for every epoch
plot_global_fragments(ax_handles, video, …) Plots the global fragments used for training until the current epoch
save(number_of_epochs_completed) Saves the values stored
append_data(loss_value, accuracy_value, individual_accuracy_value)[source]

Appends the loss_value, accuracy_value and individual_accuracy_value to their correspoding lists

load()[source]

Load the values stored in case there are any saved

plot(axes_handles=None, index=0, color='r', canvas_from_GUI=None, legend_font_color=None)[source]

Plots the accuracy and the individual accuracies for every epoch

plot_global_fragments(ax_handles, video, fragments, black=False, canvas_from_GUI=None)[source]

Plots the global fragments used for training until the current epoch

save(number_of_epochs_completed)[source]

Saves the values stored

Get data

class get_data.DataSet(number_of_animals=None, images=None, labels=None)[source]

Contains the images and labels to be used for training a particular model

Attributes:
images : ndarray

Array of shape [num_images, height, width, channels] containing the images of the dataset

num_images : int

Number of images in the dataset

labels : ndarray

Array of shape [num_images, number_of_classes] containing the labels corresponding to the images in the dataset

number_of_animals : int

Number of classes in the dataset

Methods

consistency_check() Checks that the length of images and labels is the same
convert_labels_to_one_hot() Converts labels from dense format to one hot format See Also ——– shuffle_images_and_labels()
consistency_check()[source]

Checks that the length of images and labels is the same

convert_labels_to_one_hot()[source]

Converts labels from dense format to one hot format See Also ——– shuffle_images_and_labels()

get_data.dense_to_one_hot(labels, n_classes=2)[source]

Convert class labels from scalars to one-hot vectors.

get_data.duplicate_PCA_images(training_images, training_labels)[source]

Creates a copy of every image in training_images by rotating 180 degrees

Parameters:
training_images : ndarray

Array of shape [number of images, height, width, channels] containing the images to be rotated

training_labels : ndarray

Array of shape [number of images, 1] containing the labels corresponding to the training_images

Returns:
training_images : ndarray

Array of shape [2*number of images, height, width, channels] containing the original images and the images rotated

training_labels : ndarray

Array of shape [2*number of images, 1] containing the labels corresponding to the original images and the images rotated

get_data.shuffle_images_and_labels(images, labels)[source]

Shuffles images and labels with a random permutation, according to the number of examples

get_data.split_data_train_and_validation(number_of_animals, images, labels, validation_proportion=0.1)[source]

Splits a set of images and labels into training and validation sets

Parameters:
number_of_animals : int

Number of classes in the set of images

images : list

List of images (arrays of shape [height, width])

labels : list

List of integers from 0 to number_of_animals - 1

validation_proportion : float

The proportion of images that will be used to create the validation set.

Returns:
training_dataset : <DataSet object>

Object containing the images and labels for training

validation_dataset : <DataSet object>

Object containing the images and labels for validation

Epoch runner

Computes a given operation for an entire epoch divided in batches

class epoch_runner.EpochRunner(data_set, starting_epoch=0, print_flag=True, batch_size=None)[source]

Runs an epoch divided in batches for a given operation and a given data set (DataSet).

Attributes:
epochs_completed : int

Number of epochs completed in the current step of the training

starting_epoch : int

Epoch at which the training step started

print_flag : bool

If True prints the values of the loss, the accucacy, and the individual accuracy at the end of the epoch

data_set : <DataSet object>

Object containing the images and labels to be passed through the network in the current epoch (see DataSet)

batch_size : int

Number of images to be passed through the network in each bach.

Methods

next_batch(batch_size) Returns the images and labels for the next batch to be computed.
run_epoch(name, store_loss_and_accuracy, …) Performs a given batch_operation for an entire epoch and stores the values of the loss and the accurcaies in a Store_Accuracy_and_Loss object for visualization
next_batch(batch_size)[source]

Returns the images and labels for the next batch to be computed. Images and labels are extracted from a get_data.DataSet object

Parameters:
batch_size : int

Number of examples to be passed through the network in this batch

Returns:
images : ndarray

Array of shape [batch_size, height, width, channels] containing the images to be used in this batch

labels : ndarray

Array of shape [batch_size, number_of_classes] containing the labels corresponding to the images to be used in this batch

See also

get_data.DataSet

run_epoch(name, store_loss_and_accuracy, batch_operation)[source]

Performs a given batch_operation for an entire epoch and stores the values of the loss and the accurcaies in a Store_Accuracy_and_Loss object for visualization

Parameters:
name : string

A string to be printed in the epoch information. Typically ‘Training’ or ‘Validation’.

store_loss_and_accuracy : <Store_Accuracy_and_Loss object>

Object collecting the values of the loss, accurcay and individual accuracies (see store_accuracy_and_loss.Store_Accuracy_and_Loss)

batch_operation : func

Function to be run in the epoch

Returns:
feed_dict : dict

Dictionary with the parameters and variables needed to run the batch_operation. It is used to save the Tensorflow summaries if needed

Get predictions

Retrieves the predictions of the model idCNN (ConvNetwork) for a set of images

class get_predictions.GetPrediction(data_set)[source]

Manages the inference of the identities of a set of images

Attributes:
data_set : <DataSet object>

Object containing the images whose labels has to be predicted (see DataSet)

softmax_probs : ndarray

Array of shape [number of images, number of animals] with the softmax vectors for every image in the data_set

predictions : ndarray

Array of shape [number of images, 1] with the predictions computed as the argmax of the softmax vector of every image (in dense format).

predictions_KNN : ndarray

Array of shape [number of images, 1] with the predictions for every image computed after a clustering of the second to last fully connected layer with KNN algorithm (in dense format)

_fc_vectors : ndarray

Array of shape [number of images, number of neurons in second to last fully connected layer]

batch_size : int

Size of the batch to send the images through the network to get the predictions

Methods

get_predictions_fully_connected_embedding(…) Not used in the current version of the algorithm.
get_predictions_softmax(batch_operation) Runs a batch_operation (typically predict())
next_batch(batch_size) Return the next batch_size examples from this data set.
get_predictions_fully_connected_embedding(batch_operation, number_of_animals)[source]

Not used in the current version of the algorithm. Only for analysis.

get_predictions_softmax(batch_operation)[source]

Runs a batch_operation (typically predict())

Parameters:
batch_operation : func

Function to be run in the epoch

next_batch(batch_size)[source]

Return the next batch_size examples from this data set.

Parameters:
batch_size : int

Number of images to get for the next batch

Returns:
images : ndarray

Array of shape [batch_size, height, width, channels] with the images of the batch

get_predictions.kMeansCluster(vector_values, num_clusters, max_num_steps, stop_coeficient=0.0)[source]

Not used in the current version of the algorithm. Only for analysis.