Identification network¶

idCNN¶

This module contains the main Tensorflow operations needed to train and the idCNN

class id_CNN.ConvNetwork(params, training_flag=True, restore_index=None)[source]

Manages the main Tensorflow graph for the convolutional network that identifies images (idCNN)

Attributes: image_width : int Width of the input image to the network image_height : int Height of the input image to the network image_channels : int Number of channes of the input image to the network params : Object collecting the hyperparameters and other variables of the network restore_index : int Checkpoint index to the model that has to be restored layers_to_optimise : list List of strings with the names of the layers to be optimized training : bool Flag indicating whether the network is going to be used for training or to do fordward passes ops_list : list List with tensorflow operations session : tf.Session() Tensorflow session used to run the operations is_restoring : bool Flag indicating whether the network model should be restored from a previous checkpoint restore_folder_conv : string Path to the folder with the convolutional part of the model to be restored restore_folder_fc_softmax : string Path to the fully connected layers of the model to be restored is_knowledge_transfers : bool Flag indicating whether the knowledge from a previously trained model should be used summary_op : tf.operation Operation the merge the Tensorflow summaries to be visualized in the tensorboard summary_writer_training : tf.operation Summary writer for the training summary_writer_validation : tf.operation Summary writer for the validation x_pl : tf.tensor Tensor placeholder for the input images [None, image_width, image_height, image_channels] y_target_pl : tf.tensor Tensor placeholder for the labels of the input images [None, number_of_animals] keep_prob_pl : tf.tensor Tensor placeholder for the ‘keep probability’ in case of dropout loss_weights_pl : tf.tensor Tensor placeholder for the weights of the loss y_logits : tf.operation Output of the last fully connected layer softmax_probs : tf.operation Softmax probabilities computed from y_logits predictions : tf.operation Predictions of the given input images computed as the argmax of softmax_probs loss : tf.operation Loss function optimisation_step : tf.operation Optimization function global_step : tf.global_step Global tensorflow counter to save the checkpoints accuracy : tf.operation Accuracy of the predictions computed from the y_logits individual_accuracy : tf.operation Individual accuracy of the predictions for every class computed from the y_logits weights : ndarray Proportion of labels of every class in the collection of images in an epoch fc_vector : tf.operation Output of the second to last fully connected layer of the network saver_conv : tf.operation Operation to save the convolutional layers of the model saver_fc_softmax : tf.operation Operation to save the fully connected layers of the model save_folder_conv : string Path to the checkpoints of the convolutional layers of the model save_folder_fc_softmax : string Path to the checkpoints of the fully connected layers of th model

Methods

 compute_batch_weights(batch_labels) Returns the weights vector (batch_weights) for a batch of labes compute_loss_weights(training_labels) Computes the label weights for all the images that will be train in an epoch create_summaries_writers() Create the summary writers for thw tensorboard summaries evaluation() Computes and returns the accuracy and individual accuracy of the predictions given by the model for the input images get_feed_dict(batch) Returns the dictionary feed_dict with the variables and parameters needed to perform certain operations for a give batch get_fully_connected_vectors(batch) Runs in the tensorflow session session the fc_vector operation get_layers_to_optimize() Set layers of the network to be optimised during the training process predict(batch) Runs in the tensorflow session session the softmax_probs and the predictions operations reinitialize_conv_layers() Reinizializes the weights in the conv layers indicated in the list conv_layers_list. reinitialize_softmax_and_fully_connected() Reinizializes the weights in the two last fully connected layers of the network restore() Restores a previously trained model from a checkpoint folder restore_classifier() Restores the weights of the convolutional layers from a checkpoint file of a previously trained model restore_convolutional_layers() Restores the weights of the convolutional layers from a checkpoint file of a previously trained model save() Saves the models in the correspoding save_folder_conv and save_folder_fc_softmax folders using the saver saver_conv and saver_fc_softmax set_optimizer() Sets the optimization operation to be perform during the training process set_savers() Create or retrieves the paths where the checkpoint files will be saved and initialize the objects to save them train(batch) Runs in the tensorflow session session an optimization_step and the operations in ops_list for a given batch validate(batch) Runs in the tensorflow session session the operations in ops_list for a given batch weighted_loss() Loss function minimised during the training process write_summaries(epoch_i, feed_dict_train, …) Writes the summaries using the summary_str_training and summary_writer_validation to be visualized in the Tensorboard
 check_checkpoint_path
compute_batch_weights(batch_labels)[source]

Returns the weights vector (batch_weights) for a batch of labes

compute_loss_weights(training_labels)[source]

Computes the label weights for all the images that will be train in an epoch

create_summaries_writers()[source]

Create the summary writers for thw tensorboard summaries

evaluation()[source]

Computes and returns the accuracy and individual accuracy of the predictions given by the model for the input images

get_feed_dict(batch)[source]

Returns the dictionary feed_dict with the variables and parameters needed to perform certain operations for a give batch

get_fully_connected_vectors(batch)[source]

Runs in the tensorflow session session the fc_vector operation

get_layers_to_optimize()[source]

Set layers of the network to be optimised during the training process

predict(batch)[source]

Runs in the tensorflow session session the softmax_probs and the predictions operations

reinitialize_conv_layers()[source]

Reinizializes the weights in the conv layers indicated in the list conv_layers_list. The list can include the names ‘conv1’, ‘conv2’ or ‘conv3’

reinitialize_softmax_and_fully_connected()[source]

Reinizializes the weights in the two last fully connected layers of the network

restore()[source]

Restores a previously trained model from a checkpoint folder

restore_classifier()[source]

Restores the weights of the convolutional layers from a checkpoint file of a previously trained model

saver_fc_softmax()

restore_convolutional_layers()[source]

Restores the weights of the convolutional layers from a checkpoint file of a previously trained model

saver_conv()

save()[source]

Saves the models in the correspoding save_folder_conv and save_folder_fc_softmax folders using the saver saver_conv and saver_fc_softmax

set_optimizer()[source]

Sets the optimization operation to be perform during the training process

set_savers()[source]

Create or retrieves the paths where the checkpoint files will be saved and initialize the objects to save them

train(batch)[source]

Runs in the tensorflow session session an optimization_step and the operations in ops_list for a given batch

validate(batch)[source]

Runs in the tensorflow session session the operations in ops_list for a given batch

weighted_loss()[source]

Loss function minimised during the training process

write_summaries(epoch_i, feed_dict_train, feed_dict_val)[source]

Writes the summaries using the summary_str_training and summary_writer_validation to be visualized in the Tensorboard

id_CNN.compute_accuracy(labels, logits)[source]

Computes the accuracy for a set of labels and logits

Parameters: labels : tf.tensor Tensor of shape [batch size, number of classes] with the labels of the input images in dense format logits : tf.tensor Tensor of shape [batch size, number of classes] with the output of the last fully connected layer accuracy : tf.constant Accuracy for the given set of logits correspoding to the labels
id_CNN.compute_individual_accuracy(labels, logits, classes)[source]

Computes the individual accuracy for a set of labels and logits given the number of classes

Parameters: labels : tf.tensor Tensor of shape [batch size, number of classes] with the labels of the input images in dense format logits : tf.tensor Tensor of shape [batch size, number of classes] with the output of the last fully connected layer classes : int Number of classes individual_accuracies : tf.tensor Tensor of shape [number of classes] with the accuracy for every class
id_CNN.createSaver(name, exclude_fc_and_softmax)[source]

Returns a tensorflow saver with a given name which will only save the convolutional layers if exclude_fc_and_softmax is True, and it will only save the fully connected layers if exclude_fc_and_softmax is False

id_CNN.get_checkpoint_subfolders(folder_name, sub_folders_names)[source]

Create if it does not exist the folder folder_name and the subfolders sub_folders_names. It returns the list of sub_paths created or

Network parameters¶

class network_params.NetworkParams(number_of_animals, cnn_model=0, learning_rate=None, keep_prob=None, use_adam_optimiser=False, scopes_layers_to_optimize=None, restore_folder=None, save_folder=None, knowledge_transfer_folder=None, image_size=None, number_of_channels=None, video_path=None)[source]

Manages the network hyperparameters and other variables related to the identification model (see idCNN)

Attributes: video_path : string Path to the video file number_of_animals : int Number of animals in the video learning_rate : float Learning rate for the optimizer keep_prob : float Dropout probability _restore_folder : string Path to the folder where the model to be restored is _save_folder : string Path to the folder where the checkpoints of the current model are stored _knowledge_transfer_folder : string Path to the folder where the model to be used for knowledge transfer is saved use_adam_optimiser : bool Flag indicating to use the Adam optimizer with the parameters indicated in _[2] scopes_layers_to_optimize : list List with the scope names of the layers to be optimized _cnn_model : int Number indicating the model number to be used from the dictionary of models CNN_MODELS_DICT in id_CNN image_size : tuple Tuple (height, width, channels) for the input images number_of_channels : int Number of channels of the input image kt_conv_layers_to_discard : str convolutional layers to discard when performing knowledge transfer .. [2] Kingma, Diederik P., and Jimmy Ba. “Adam: A method for stochastic optimization.” arXiv preprint arXiv:1412.6980 (2014).

Stop training criteria (idCNN)¶

class stop_training_criteria.Stop_Training(number_of_animals, epochs_before_checking_stopping_conditions=10, check_for_loss_plateau=True, first_accumulation_flag=False)[source]

Stops the training of the network according to the conditions specified in __call__()

Attributes: num_epochs : int Number of epochs before starting the training number_of_animals : int Number of animals in the video epochs_before_checking_stopping_conditions : int Number of epochs before starting to check the stopping conditions overfitting_counter : int Counts the number of consecutive overfitting epochs during training check_for_loss_plateau : bool Flag to check for a plateu in the loss of the validation first_accumulation_flag : bool Flag to indicate that it is the first step of the accumulation

Methods

 __call__(loss_accuracy_training, …) Returns True when one of the conditions to stop the training is satisfied, otherwise it returns False

Store accurcay and loss¶

class store_accuracy_and_loss.Store_Accuracy_and_Loss(network, name, scope=None)[source]

Store the loss, accuracy and individual accuracy values computed during training and validation

Parameters: _path_to_accuracy_error_data : Path to save the lists loss, accuracy, individual_accuracy name : string ‘Training’ or ‘Validation’ loss : list List with the values of the loss accuracy : list List with the values of the accuracy individual_accuracy : list List with the values of the individual accuracies number_of_epochs_completed : Number of epochs completed scope : string ‘Training’ if the class is instantiated during the training of the accumulation. ‘Pretraining’ if the class is instantiated during the training

Methods

 append_data(loss_value, accuracy_value, …) Appends the loss_value, accuracy_value and individual_accuracy_value to their correspoding lists load() Load the values stored in case there are any saved plot([axes_handles, index, color, …]) Plots the accuracy and the individual accuracies for every epoch plot_global_fragments(ax_handles, video, …) Plots the global fragments used for training until the current epoch save(number_of_epochs_completed) Saves the values stored
append_data(loss_value, accuracy_value, individual_accuracy_value)[source]

Appends the loss_value, accuracy_value and individual_accuracy_value to their correspoding lists

load()[source]

Load the values stored in case there are any saved

plot(axes_handles=None, index=0, color='r', canvas_from_GUI=None, legend_font_color=None)[source]

Plots the accuracy and the individual accuracies for every epoch

plot_global_fragments(ax_handles, video, fragments, black=False, canvas_from_GUI=None)[source]

Plots the global fragments used for training until the current epoch

save(number_of_epochs_completed)[source]

Saves the values stored

Get data¶

class get_data.DataSet(number_of_animals=None, images=None, labels=None)[source]

Contains the images and labels to be used for training a particular model

Attributes: images : ndarray Array of shape [num_images, height, width, channels] containing the images of the dataset num_images : int Number of images in the dataset labels : ndarray Array of shape [num_images, number_of_classes] containing the labels corresponding to the images in the dataset number_of_animals : int Number of classes in the dataset

Methods

 consistency_check() Checks that the length of images and labels is the same convert_labels_to_one_hot() Converts labels from dense format to one hot format See Also ——– shuffle_images_and_labels()
consistency_check()[source]

Checks that the length of images and labels is the same

convert_labels_to_one_hot()[source]

Converts labels from dense format to one hot format See Also ——– shuffle_images_and_labels()

get_data.dense_to_one_hot(labels, n_classes=2)[source]

Convert class labels from scalars to one-hot vectors.

get_data.duplicate_PCA_images(training_images, training_labels)[source]

Creates a copy of every image in training_images by rotating 180 degrees

Parameters: training_images : ndarray Array of shape [number of images, height, width, channels] containing the images to be rotated training_labels : ndarray Array of shape [number of images, 1] containing the labels corresponding to the training_images training_images : ndarray Array of shape [2*number of images, height, width, channels] containing the original images and the images rotated training_labels : ndarray Array of shape [2*number of images, 1] containing the labels corresponding to the original images and the images rotated
get_data.shuffle_images_and_labels(images, labels)[source]

Shuffles images and labels with a random permutation, according to the number of examples

get_data.split_data_train_and_validation(number_of_animals, images, labels, validation_proportion=0.1)[source]

Splits a set of images and labels into training and validation sets

Parameters: number_of_animals : int Number of classes in the set of images images : list List of images (arrays of shape [height, width]) labels : list List of integers from 0 to number_of_animals - 1 validation_proportion : float The proportion of images that will be used to create the validation set. training_dataset : Object containing the images and labels for training validation_dataset : Object containing the images and labels for validation

Epoch runner¶

Computes a given operation for an entire epoch divided in batches

class epoch_runner.EpochRunner(data_set, starting_epoch=0, print_flag=True, batch_size=None)[source]

Runs an epoch divided in batches for a given operation and a given data set (DataSet).

Attributes: epochs_completed : int Number of epochs completed in the current step of the training starting_epoch : int Epoch at which the training step started print_flag : bool If True prints the values of the loss, the accucacy, and the individual accuracy at the end of the epoch data_set : Object containing the images and labels to be passed through the network in the current epoch (see DataSet) batch_size : int Number of images to be passed through the network in each bach.

Methods

 next_batch(batch_size) Returns the images and labels for the next batch to be computed. run_epoch(name, store_loss_and_accuracy, …) Performs a given batch_operation for an entire epoch and stores the values of the loss and the accurcaies in a Store_Accuracy_and_Loss object for visualization
next_batch(batch_size)[source]

Returns the images and labels for the next batch to be computed. Images and labels are extracted from a get_data.DataSet object

Parameters: batch_size : int Number of examples to be passed through the network in this batch images : ndarray Array of shape [batch_size, height, width, channels] containing the images to be used in this batch labels : ndarray Array of shape [batch_size, number_of_classes] containing the labels corresponding to the images to be used in this batch
run_epoch(name, store_loss_and_accuracy, batch_operation)[source]

Performs a given batch_operation for an entire epoch and stores the values of the loss and the accurcaies in a Store_Accuracy_and_Loss object for visualization

Parameters: name : string A string to be printed in the epoch information. Typically ‘Training’ or ‘Validation’. store_loss_and_accuracy : Object collecting the values of the loss, accurcay and individual accuracies (see store_accuracy_and_loss.Store_Accuracy_and_Loss) batch_operation : func Function to be run in the epoch feed_dict : dict Dictionary with the parameters and variables needed to run the batch_operation. It is used to save the Tensorflow summaries if needed

Get predictions¶

Retrieves the predictions of the model idCNN (ConvNetwork) for a set of images

class get_predictions.GetPrediction(data_set)[source]

Manages the inference of the identities of a set of images

Attributes: data_set : Object containing the images whose labels has to be predicted (see DataSet) softmax_probs : ndarray Array of shape [number of images, number of animals] with the softmax vectors for every image in the data_set predictions : ndarray Array of shape [number of images, 1] with the predictions computed as the argmax of the softmax vector of every image (in dense format). predictions_KNN : ndarray Array of shape [number of images, 1] with the predictions for every image computed after a clustering of the second to last fully connected layer with KNN algorithm (in dense format) _fc_vectors : ndarray Array of shape [number of images, number of neurons in second to last fully connected layer] batch_size : int Size of the batch to send the images through the network to get the predictions

Methods

 get_predictions_fully_connected_embedding(…) Not used in the current version of the algorithm. get_predictions_softmax(batch_operation) Runs a batch_operation (typically predict()) next_batch(batch_size) Return the next batch_size examples from this data set.
get_predictions_fully_connected_embedding(batch_operation, number_of_animals)[source]

Not used in the current version of the algorithm. Only for analysis.

get_predictions_softmax(batch_operation)[source]

Runs a batch_operation (typically predict())

Parameters: batch_operation : func Function to be run in the epoch
next_batch(batch_size)[source]

Return the next batch_size examples from this data set.

Parameters: batch_size : int Number of images to get for the next batch images : ndarray Array of shape [batch_size, height, width, channels] with the images of the batch
get_predictions.kMeansCluster(vector_values, num_clusters, max_num_steps, stop_coeficient=0.0)[source]

Not used in the current version of the algorithm. Only for analysis.