brainspy.utils.performance package#

Module contents#

Package for training a perceptron after the signal of a Processor class or one of its children.

Submodules#

brainspy.utils.performance.accuracy module#

File for training a perceptron after the signal of a Processor class or one of its children.

brainspy.utils.performance.accuracy.evaluate_accuracy(inputs, targets, node)[source]#

To evaluate the accuracy of the Perceptron algorithm on the input data ( which is the outut of the DNPU or DNPU architecture ) based on the inputs and target values provided. The accuarcy is evaluated by passing the normalised output of the DNPU or DNPU architecture through the trained perceptron, and compare the output against the binary targets

inputstorch.Tensor

The inputs to the perceptron algorithm, which are the outputs of the DNPU or DNPU architectures that you want to evaluate the accuracy against.

targetstorch.Tensor

Binary targets against which the output of the perceptron algorithm is compared.

nodeOptional[torch.nn.Module]

Is the trained linear layer of the perceptron. Leave it as None if you want to train a perceptron from scratch. (default: None)

Returns:

  • accuracy - int - the accuracy calculated from the data provided

  • labels - bool - if the predictions of the noda are greatrer than 0

brainspy.utils.performance.accuracy.get_accuracy(inputs, targets, configs=None, node=None)[source]#

To calculate the accuracy of the device on a binary classification task. Binary classification is the task of classifying the elements of a set into groups on the basis of a classification rule. This function helps finding a binary threshold for separating the output of a DNPU or DNPU architecture, and assigns a True/False label to the output depending on whether if the signal is above or below the discovered threshold. In this way, this function helps calculating the number of correctly classified binary targets out of the total number of targets, for a given dataset.

To calculate the accuracy, a perceptron is trained using binary cross entropy on the output of the DNPU or DNPU architecture for the training dataset. The model trained consists of a single linear layer, where the sigmoid of the perceptron is included in the binary cross entropy loss function. For calculating the accuracy of the test and validation datasets, the already trained perceptron is passed to the function, and the method calculates the accuracy based on that.

Refer to https://www.upgrad.com/blog/perceptron-learning-algorithm-how-it-works/ to see how a Perceptron works.

The specific method for calculating the accuracy is:

  1. Normalises the input data (which is the output data of the DNPU or DNPU architecture)

2. (Optional) Train a perceptron, only needed when using the normalised output of the DNPU or DNPU architecture corresponding to the training dataset of a particular task. To use it leave the option node=None.

3. Pass the normalised output of the DNPU or DNPU architecture through the trained perceptron, and compare the output against the binary targets. This comparison is used to calculate the accuracy of the solution.

  1. Store all the data including results in a dictionary and return it

Refer to: https://pytorch-lightning.readthedocs.io/en/1.2.6/_modules/pytorch_lightning/metrics/classification/accuracy.html to see how accuracy is calculated in PyTorch.

Parameters:

  • inputs (torch.Tensor) – The inputs to the perceptron algorithm, which are the outputs of the DNPU or DNPU architectures that you want to evaluate the accuracy against. Only input tensors with a single dimension are supported with the default node.

  • targets (torch.Tensor) – Binary targets against which the outuut of the perceptron algorithm is compared. Only target tensors with a single dimension are supported with the default node.

  • configs (dict, optional) –

    Configurations of the model to get the accuracy using the perceptron algorithm. The configs should contain a description of the hyperparameters that can be used for get_accuracy. By default is None. It has the following keys:

    1. epochs: int Number of loops used for training the perceptron. (default: 100)

    2. learning_rate: float Learning rate used to train the perceptron. (default: 1e-3)

    1. data:

    3.1 batch_size: int Batch size used to train the perceptron. (default: 256)

    3.2 worker_no: int How many subprocesses to use for data loading. 0 means that the data will be loaded in the main process. (default: 0)

    3.3 pin_memory: boolean (default: False) If True, the data loader will copy Tensors into CUDA pinned memory before returning them.

  • node (Optional[torch.nn.Module]) – Is the trained linear layer of the perceptron. Leave it as None if you want to train a perceptron from scratch. (default: None) The default perceptron only supports one dimensional outpus.

Returns:

  • Dictionary containing the results/accuracy of the algorithm in a dictionary with the following keys

  • 1. accuracy_value (float)

  • Percentage accuracy obtained by calculating the number of

  • correct binary outputs against the targets.

  • 2. node (torch.nn.Linear)

  • The linear layer of the trained/used perceptron.

  • 3. predicted_labels

  • Predicted labels after passing the normalised inputs through

  • the perceptron.

  • 4. norm_threshold (Normalised threshold used for the classification.)

  • configs (Configurations of the node. It has the following keys:)

  • 4.1 epochs (int)

  • Number of loops used for training the perceptron. (default (100))

  • 4.2 learning_rate (float)

  • Learning rate used to train the perceptron. (default (1e-3))

  • 4.3 data

  • 4.3.1 batch_size (int)

  • Batch size used to train the perceptron. (default (256))

  • 4.3.2 worker_no (int)

  • How many subprocesses to use for data loading. 0 means that the

  • data will be loaded in the main process. (default (0))

  • 4.3.3 pin_memory (boolean (default: False))

  • If True, the data loader will copy Tensors into CUDA pinned

  • memory before returning them.

brainspy.utils.performance.accuracy.get_default_node_configs()[source]#

To get a default configuration of the node of a perceptron. This method is used in the get_accuracy method if a node is not provided.

Returns:

  • Dictionary containin the configurations of the perceptron to calculate the accuracy

  • on the input data (which is the output of the DNU or DNPU architecture). The dictionary

  • contains the following keys

  • 1. epochs (int)

  • Number of epochs.

  • 2. dataloader (torch.utils.data.Dataloader)

  • The normalised output from the DNPU or DNPU architecture, already in a dataloader

  • format.

  • 3. optimizer (torch.optim.Optimizer)

  • Optimization algorithm. (default (torch.optim.Adam))

  • 4. loss_fn (Optional[torch.nn.modules.loss._Loss])

  • Loss functions are used to gauge the error between the prediction output and the

  • provided target value, by default torch.nn.BCEWithLogitsLoss()

  • 5. node (Optional[torch.nn.Module])

  • Is the trained linear layer of the perceptron. Leave it as None if you want to

  • train a perceptron from scratch. (default (None))

brainspy.utils.performance.accuracy.init_results(inputs, targets, configs)[source]#

To initialize the results of the accuracy test and the results of the Perceptron algorithm. The method initializes the results dictionary for evaluation of accuracy , and initializes the perceptron dataset from thge Dataloader (Refer to data.py for the Perceptron dataloader).

Parameters:

  • inputs (torch.Tensor) – The inputs to the perceptron algorithm, which are the outputs of the DNPU or DNPU architectures that you want to evaluate the accuracy against.

  • targets (torch.Tensor) – Binary targets against which the output of the perceptron algorithm is compared.

  • configs (dict, optional) –

    Configurations of the model to get the accuracy using the perceptron algorithm. The configs should contain a description of the hyperparameters that can be used for get_accuracy. By default is None. It has the following keys:

    epochs: int

    Number of loops used for training the perceptron. (default: 100)

    learning_rate: float

    Learning rate used to train the perceptron. (default: 1e-3)

    batch_size: int

    Batch size used to train the perceptron. (default: 256)

Returns:

  • dict - initialized data for evaluation of accuracy

  • torch.utils.data.Dataloader (results of the Perceptron dataloader)

brainspy.utils.performance.accuracy.plot_perceptron(results, save_dir=None, show_plots=False, name='train')[source]#

Plot the results of the perceptron algorithm. You can choose to see how the data has been plotted and also save the results to a specified directory.

Parameters:

  • results (dict) – Results/accuracy of the algorithm in a dictionary with the following keys: 1. accuracy_value : Percentage Accuracy of the Algorithm 2. node : the transformation applied to the dataset 3. predicted_labels : predicted labels used to calculate accuracy 4. norm_threshold : Threshold probability value for accuracy calculation of the Perceptron 5. configs : configurations of the node

  • save_dir (str, optional) – Directory in which you want to save the results. (default: None)

  • show_plots (bool, optional) – To see how the perceptron plotted the data, by default False

  • name (str, optional) – To train the data, by default “train”.

Returns:

A new figure contaning the results.

Return type:

matplotlib.pyplot.figure

brainspy.utils.performance.accuracy.train_perceptron(epochs: int, dataloader: DataLoader, optimizer: Optimizer, loss_fn: _Loss = BCEWithLogitsLoss(), node: Module | None = None, stop_at_max_accuracy: bool = False)[source]#

To train the Perceptron obtain a set of weights w that accurately classifies each instance in our training set. In order to train our Perceptron, we iteratively feed the network with our training data multiple times.

The perceptron is used as a linear classifier to facilitate supervised learning of binary classifiers. The objective of this learning problem is to use data with correct labels for making predictions for training a model. This supervised learning include classification to predict class labels.

Refer to: https://medium.com/biaslyai/pytorch-introduction-to-neural-network-feedforward-neural-network-model-e7231cff47cb for an example of training a Perceptron in PyTorch.

Parameters:

  • epochs (int) – Number of epochs.

  • dataloader (torch.utils.data.DataLoader) – The normalised output from the DNPU or DNPU architecture, already in a dataloader format.

  • optimizer (torch.optim.Optimizer) – Optimization algorithm. (default: torch.optim.Adam)

  • loss_fn (Optional[torch.nn.modules.loss._Loss]) – Loss functions are used to gauge the error between the prediction output and the provided target value, by default torch.nn.BCEWithLogitsLoss()

  • node (Optional[torch.nn.Module]) – Is the trained linear layer of the perceptron. Leave it as None if you want to train a perceptron from scratch. (default: None)

  • stop_at_max_accuracy (boolean) – Decides to immediately stop after achieving 100% solution if true. If false, it will keep training to improve the threshold separation. Recommended to be true when doing many runs at the same time in tasks like capacity test or searcher.

Returns:

  • accuracy (int - accuracy of the perceptron)

  • node (torch.nn - node of the perceptron)

brainspy.utils.performance.accuracy.zscore_norm(inputs, eps=1e-05)[source]#

To calculate the standard normal distribution from the input data. The standard normal distribution, represented by the letter Z, is the normal distribution having a mean of 0 and a standard deviation of 1. Refer to https://towardsdatascience.com/the-surprising-longevity-of-the-z-score-a8d4f65f64a0 to read about z-score and normalisation of data in PyTorch.

Parameters:

  • inputs (torch.Tensor) – The inputs to the perceptron algorithm, which are the outputs of the DNPU or DNPU architectures that you want to evaluate the accuracy against.

  • eps (int , optional) – Value of epsilon. (default: 1e-5)

Returns:

Normalised inputs.

Return type:

torch.Tensor

brainspy.utils.performance.data module#

Package for describing the dataset for training a perceptron after the signal of a Processor class or one of its children.

class brainspy.utils.performance.data.PerceptronDataset(inputs, targets, device=None)[source]#

Bases: Dataset

This class is an instace of the Pytorch Dataset. It passes all the information onto the Pytorch dataset. The dataset stores the samples and their corresponding labels, and DataLoader wraps an iterable around the Dataset to enable easy access to the samples.

Refer to https://pytorch.org/tutorials/beginner/basics/data_tutorial.html to see how Pytorch datasets are created and used.

brainspy.utils.performance.data.get_data(results, batch_size)[source]#

Initialises the perceptron Dataset and loads the dataset into the Pytorch Dataloader. The dataloader loads the data into the memory according to the batch size. Refer to https://pytorch.org/tutorials/beginner/basics/data_tutorial.html for DataLoaders in PyTorch.

The data can be shuffled. After each epoch the data is shuffled automatically. This is by design to accelerate and improve the model training process.Because of this, the learning algorithm is stochastic and may achieve different results each time it is run. Refer to https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html to see how epochs are used when training a Classifier.

Parameters:

  • results (dict) –

    These contain the input and target values of the perceptron alogorithm. It also contains the normalised input data from which the Pytorch dataloader is created.

    It has the following keys:

    inputstorch.Tensor

    The inputs to the perceptron algorithm, which are the outputs of the DNPU or DNPU architectures that you want to evaluate the accuracy against.

    norm_inputstorch.Tensor

    Standard normal distribution of the input data. To calculate this, the zscore_norm function can be used in brainspy.utils.performance.accuracy

    targetstorch.Tensor

    Binary targets against which the outuut of the perceptron algorithm is compared.

  • propagated (batch size- The batch size defines the number of samples that will be) – through the network.

Returns:

Dataloader of the perceptron algorithm

Return type:

torch.utils.data.Dataloader