classification_report

Package Contents

class classification_report.Config(**kwargs)[source]

Bases: object

This class can be used to create and store data for an type of configuration.

Parameters:**kwargs – Arbitrary keyword arguments.
Examples::
>>> from classification_report import Config
>>> training_config = Config(lr=0.1, batch_size=32, device="GPU")
>>> model_config = Config(number_layers=3, num_head=32)
>>> model_config.number_layers
3
__setattr__(self, name, value)
__getattr__(self, name)
__delattr__(self, name)
update(self, **kwargs)

To update the config class attributes values.

This will add new attribute for a non existing attribute in the Config class or replace the value with a new a value for an existing attribute.

Parameters:**kwargs – Arbitrary keyword arguments.
Examples::
>>> from classification_report import Config
>>> training_config = Config(lr=0.1, batch_size=32, device="GPU")
>>> training_config.lr
0.1
>>> training_config.update(lr=0.2,precision:16)
>>> training_config.lr
0.2
append_values(self, **kwargs)

This method can be used to append values to an existing attribute.

For Example if using Lr Scheduler then this can be use to track all lr values by appending in a list.

Note

The attribute should be prexisiting.

Parameters:**kwargs – Arbitrary keyword arguments.
Examples::
>>> from classification_report import Config
>>> training_config = Config(lr=0.1, batch_size=32, device="GPU")
>>> training_config.lr
0.1
>>> training_config.append_values(lr=0.2)
>>> training_config.lr
[0.1,0.2]
>>> training_config.append_values(lr=[0.3,0.4])
>>> [0.1,0.2,0.3,0.4]
__str__(self)
save_config_json(self, path: str)

Save the Configuration in json format.

Parameters:path – The file path to save json file.
Examples::
>>> from classification_report import Config
>>> training_config = Config(lr=0.1, batch_size=32, device="GPU")
>>> training_config.save_config_json("training_config.json")
Configuration Saved
classmethod load_config_json(cls, path: str)

Loading the saved configuration.

Parameters:path – The file from json config will be loaded.
Returns:A Config Class is returned with attributes set from json file
Return type:Config
Examples::
>>> from classification_report import Config
>>> training_config = Config.load_config_json("training_config.json") # Execute the saving code first.
>>> training_config.lr
0.1
get_dict_repr(self)
Returns:The Dictionary representation of Config class
Return type:dict
Example::
>>> from classification_report import Config
>>> training_config = Config(lr=0.1, batch_size=32, device="GPU")
>>> training_config.get_dict_repr()
{"lr":0.1,"batch_size":32,"device":"GPU"}
class classification_report.HyperParameters(**configs)[source]

Bases: classification_report.config.Config

It stores collections of Config in one place. It inherits the Config class.

Parameters:**config – Arbitary Config objects
Raises:AssertionError – Pass only Config class object.
Examples::
>>> from classification_report import Config, HyperParameters
>>> model_config = Config(**{'hid_dim': 512,'n_layers': 8,'n_heads': 8,'pf_dim': 2048,'dropout': 0.1})
>>> training_config = Config(num_epochs=15, max_lr=0.09, batch_size=64)
>>> inference_config = Config(batch_size=16)
>>> hyper = HyperParameters(model_config = model_config, training_config=training_config,infer_config=inference_config)
>>> hyper.model_config.hid_dim
512
>>> hyper.save_config_json("hyper.json")
Configuration Saved
>>> hyper = HyperParameters.load_config_json("hyper.json")
Saved Configuration Loaded
static flatten(d: dict, parent_key: str = '', sep: str = '_')

Flatten the nested dictionary using recusrion.

Parameters:
  • d – The dictionary to flatten.
  • parent_key – The parent key.
  • sep – The sep to be used to join parent_key with nested_key
Returns:

Flattened dictionary.
Return type:

dict
class classification_report.Report(classes: TrainType, dir_name: str = None)[source]

Generating Report for classification model by tracking Model training and giving different types of metrics to evaluate the Model.

For any classification problem during Model’s training it is very important to track Model’s Weight Biases and Gradients. After training the important part is the model evaluation where we evaluate the model performance. This Report class simplify the evaluation part where all the evaluation metrics are automatically generated for the model.It uses Tensorboard to visualize all these.

write_a_batch(self, loss: LossType, batch_size: int, actual: ActualType, prediction: PredictionType, train: bool = True)

This methods records the batch information during train and val phase.

During training and validation record the loss, batch actual labels and predicted labels.

Note

For prediction don’t pass raw_logits pass softmax output.

Parameters:
  • loss – The batch loss.
  • batch_size – The batch size on which the loss was calculated. The batch_size may change during last iteration so calculate batch_size from data.
  • actual – The actual labels.
  • prediction – The predicted labels.
  • train – True signifies training mode and False Validation Mode.
Returns:

Report class instance.
update_actual_prediction(self, actual: ActualType, prediction: PredictionType, train_type: TrainType)

Stores actual and predicted labels seperately for training and validation and after every batch call the values are appended. :param actual: The actual labels. :param prediction: The predicted labels. :param train_type: The labels belong to train or valid.

Returns:Report class instances.
update_loss(self, loss: LossType, batch_size: int, train_type: TrainType)

Accumlates loss for every batch seperately for training and validation.

Parameters:
  • loss – The batch loss.
  • batch_size – The batch size on which the loss was calculated. The batch_size may change during last iteration so calculate batch_size from data.
  • train_type – The Labels belong to train or valid.
Returns:

Report class instance.
update_data_iter(self, batch_size: int, train_type: TrainType)

Accumlates the iteration count and data point count for training and validation.

Parameters:
  • batch_size – The batch size on which the loss was calculated. The batch_size may change during last iteration so calculate batch_size from data.
  • train_type – The Labels belong to train or valid.
Returns:

Report class instance.
plot_an_epoch(self, detail: bool = False)

Plot an epoch method simplifies ploting standard things which are needed to be plotted after an epoch completion for granular control use this which detail = False and call other methods on top of it.

Parameters:detail – whether to use detail mode or not.
Returns:Report class instance.
init_data_storage(self)

Clean the data storage units after every epoch.

change_data_type(self, data: LossType, required_data_type: LossType)

Change the data type of input to required data type.

Parameters:
  • data – Input data type.
  • required_data_type – Change the data type to given format, can be either np or f.
Returns:

The data in required data type.
close(self)

Close the tensorboard writer object.

After calling this method report will not track anything.

write_to_tensorboard(self)

This methods call various other method which write on tensorboard.

plot_loss(self)

Plots loss at the end of the epoch.

Returns:Report class instance.
plot_model(self, model: torch.nn.Module, data: torch.Tensor)

Plot model graph.

Parameters:
  • model – The model architecture.
  • data – The input to the model.
Returns:

Report class instance.
plot_confusion_matrix(self, at_which_epoch)

Plots confusion matrix.

Parameters:at_which_epoch – After how many epochs the confusion matrix should be plotted. For example if the model is trained for 10 epochs and you want to plot confusion matrix after every 5 epoch then the input to this method will be 5.
Returns:Report class instance.
plot_precision_recall(self)

Plots Precision Recall F1-score graph for all Classes with Weighted Average and Macro Average.

Returns:Report class instance.
plot_missclassification_count(self, at_which_epoch)

Plot Misclassification Count for each class.

Bar graph for False Positive and False Negative Count.

Parameters:at_which_epoch – After how many epochs the Misclassification Count should be plotted. For example if the model is trained for 10 epochs and you want to plot this after every 5 epoch then the input to this method will be 5.
Returns:Report class instance.
calculate_fp_fn(self, actual: ActualType, pred: PredictionType)

Calculates False Postive and False Negative Count per class.

Parameters:
  • actual – The actual labels.
  • pred – The predicted Labels.
Returns:

Report class instance.
plot_mcc(self)

Plots Mathews Correlation Coefficient.

Returns:Report class instance.
plot_pred_prob(self, at_which_epoch: int)

Plots scatter plot for the predicted probabilites for each class.

Parameters:at_which_epoch – After how many epochs the predicted probabilites should be plotted. For example if the model is trained for 10 epochs and you want to plot this after every 5 epoch then the input to this method will be 5.
Returns:Report class instance.
plot_model_data_grad(self, at_which_iter: int)

Plot Histogram and Distribution for each layers of model Weights, Bias and Gradients.

Parameters:at_which_iter – After how many iteration this should be plotted. The ideal way to plot this to plot after every one-half or one-third of the train_iterator.
Returns:Report class instance.
Examples::
>>> report.plot_model_data_grad(at_which_iter = len(train_iterator)/2)
plot_hparams(self, config: HyperParameters)

Plot Hyper parameters for the model. This method should be called once training is over.

Parameters:config – Hyperparameter Configs.
Returns:Report class instance.
classification_report.convert_prob_to_label(data: np.ndarray) → np.ndarray[source]

Convert Probability to labels

Parameters:data – (np.ndarray): Probability output of softmax. The size of data (batch_size, num_classes)
Returns:Returns pobability converted to labels by finding maximum in last dimension. The output shape is (batch_size,)
Return type:np.ndarray
classification_report.__version__ = 1.0.0[source]