redbnn.nn package

Submodules

redbnn.nn.base module

class redbnn.nn.base.baseNN(architecture, num_classes)

Bases: torch.nn.modules.module.Module

Deterministic Neural Network classifier. Used as the baseline model for reduced Bayesian Neural Network (redBNN).

evaluate(dataloader, device, *args, **kwargs)

Evaluate self.network on test data.

Parameters

  • dataloader (torch.dataloader) – Test dataloader.

  • device (str) – Device chosen for testing.

  • *args – Variable length argument list.

  • **kwargs – Arbitrary keyword arguments.

Returns

Test accuracy.

Return type

(float)

forward(inputs, softmax=False)

Forward pass of the inputs through self.network.

Parameters

  • inputs (torch.tensor) – Input images.

  • softmax (bool) – If True computes the softmax of each output tensor.

Returns

Output predictions.

Return type

(torch.tensor)

get_activation(x, layer_idx)

Get self.network activation corresponding to the chosen layer index.

Parameters

  • x (torch.tensor) – Input image.

  • layer_idx (int) – Index of the chosen layer.

Returns

Input activation at the chosen layer.

Return type

(torch.tensor)

load(filename, savedir)

Loads the learned parameters.

Parameters

  • filename (str) – Filename.

  • savedir (str) – Output directory.

save(filename, savedir)

Saves the learned parameters as torch.tensors on the CPU.

Parameters

  • filename (str) – Filename.

  • savedir (str) – Output directory.

to(device)

Sends self.network to the chosen device.

Parameters

device (str) – Name of the chosen device.

train(dataloaders, device, num_iters, feature_extract=True, use_pretrained=True, is_inception=False)

Trains self.network with stochastic gradient descent, using Adam optimizer.

Parameters

  • dataloaders (dict) – Dictionary containing training and validation torch dataloaders.

  • device (str) – Device chosen for training.

  • num_iters (int) – Number of training iterations.

  • feature_extract (bool) – If True only updates the parameters with requires_grad=True, otherwise updates all parameters. Defaults to True.

  • use_pretrained (bool) – If True loads a pre-trained model from torchvision library with the chosen architecture. Defaults to True.

  • is_inception (bool) – Special case for training torchvision inception network. Defaults to True.

Returns

History of validation accuracies.

Return type

(list)

training: bool
zero_grad(*args, **kwargs)

Set all gradients in self.network to zero.

redbnn.nn.reduced module

class redbnn.nn.reduced.redBNN(architecture, num_classes, inference, reduction, bayesian_idx)

Bases: redbnn.nn.base.baseNN

Reduced BNN is a Neural Network classifier with a single Bayesian block or a single Bayesian Layer, depending on the chosen reduction method.

evaluate(dataloader, n_samples, device)

Evaluate self.network on test data.

Parameters

  • dataloader (torch.dataloader) – Test dataloader.

  • n_samples (int) – Number of posterior samples drawn during the evaluation.

  • device (str) – Device chosen for testing.

Returns

Test accuracy.

Return type

(float)

forward(inputs, n_samples=10, sample_idxs=None, expected_out=True, softmax=False)

Forward pass of the inputs through the network using the chosen number of samples.

Parameters

  • inputs (torch.tensor) – Input images.

  • n_samples (int, optional) – Number of samples drawn during the evaluation.

  • samples_idxs (list, optional) – Random seeds used for drawing samples. If samples_idxs is None it is defined as the range of integers from 0 to the maximum number of samples.

  • expected_out (bool, optional) – If True computes the expected output prediction, otherwise returns all predictions as a torch.tensor.

  • softmax (bool, optional) – If True computes the softmax of each output tensor.

Returns

Output predictions

Return type

(torch.Tensor)

guide(x_data, y_data=None)

Variational distribution for SVI inference method.

Parameters

  • x_data (torch.tensor) – Input data points.

  • y_data (torch.tensor, optional) – Labels of the input data.

load(filename, savedir, hmc_samples=None)

Loads the learned parameters.

Parameters

  • filename (str) – Filename.

  • savedir (str) – Output directory.

  • hmc_samples (str, optional) – Number of samples drawn during HMC inference, needed for loading models trained with HMC.

model(x_data, y_data)

Stochastic function that implements the generative process and is conditioned on the observations.

Parameters

  • x_data (torch.tensor) – Observed data points.

  • y_data (torch.tensor) – Labels of the observed data.

save(filename, savedir, hmc_samples=None)

Saves the learned parameters as torch.tensors on the CPU.

Parameters

  • filename (str) – Filename.

  • savedir (str) – Output directory.

  • hmc_samples (str, optional) – Number of samples drawn during HMC inference, needed for saving models trained with HMC.

to(device)

Sends self.network to the chosen device.

Parameters

device (str) – Name of the chosen device.

train(dataloaders, device, use_pretrained=True, is_inception=False, num_iters=2, svi_iters=10, hmc_samples=100, hmc_warmup=100, eval_samples=10)

Freezes the deterministic parameters and infers the Bayesian paramaters using the chosen inference method.

Parameters

  • dataloaders (dict) – Dictionary containing training and validation torch dataloaders.

  • device (str) – Device chosen for training.

  • use_pretrained (bool, optional) – If True loads a pre-trained model from torchvision library with the chosen architecture.

  • is_inception (bool, optional) – Special case for training torchvision inception network.

  • num_iters (int, optional) – Number of iterations for fine-tuning the pre-trained network.

  • svi_iters (int, optional) – Number of iterations for Stochastic Variational Inference.

  • hmc_samples (int, optional) – Number of Hamiltonian Monte Carlo samples.

  • hmc_warmup (int, optional) – Number of Hamiltonian Monte Carlo warmup samples.

  • eval_samples (int, optional) – Number of posterior samples drawn during the evaluation.

Raises

NotImplementedError – If training is not implemented for the chosen inference method.

training: bool

redbnn.nn.subnetwork module

class redbnn.nn.subnetwork.SubNetwork(architecture, num_classes)

Bases: redbnn.nn.base.baseNN

Subset of a Neural Network architecture, given start and end layers idxs.

initialize_model(reduction, start_idx, end_idx)

Build subnetwork architecture from start_idx to end_idx (both idxs included) of the original model.

Parameters

  • original_model (torchvision.models) –

  • reduction (str) – Reduction method can be either layers or blocks depending on the desired structure.

  • start_idx (int) – Index of the first layer/block in the subnetwork.

  • end_idx (int) – Index of the last layer/block in the subnetwork.

training: bool

Module contents