tomopt.optimisation.wrapper package

Submodules

tomopt.optimisation.wrapper.volume_wrapper module

class tomopt.optimisation.wrapper.volume_wrapper.AbsVolumeWrapper(volume, *, partial_opts, loss_func=None, partial_scatter_inferrer, partial_volume_inferrer, mu_generator=None)

Bases: object

Abstract base class for optimisation volume wrappers. Inheriting classes will need to override _build_opt() according to the detector parameters that need to be optimised.

Volume wrappers are designed to contain a Volume and provide means of optimising the detectors it contains, via their fit() method.

Wrappers also provide for various quality-of-life methods, such as saving and loading detector configurations, and computing predictions with a fixed detector (predict())

Fitting of a detector proceeds as training and validation epochs, each of which contains multiple batches of passive volumes. For each volume in a batch, the loss is evaluated using multiple batches of muons. The whole loop is:

1. for epoch in n_epochs:

   1. loss = 0

   2. for p, passive in enumerate(trn_passives):

      1. load passive into passive volume

      2. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. Irradiate volume with mu_bs muons

         2. Infer passive volume

      3. Compute loss based on precision and cost, and add to loss

      4. if p`+1 % `passive_bs == 0:

         1. loss = loss/passive_bs

         2. Backpropagate loss and update detector parameters

         3. loss = 0

      5. if len(trn_passives)-(p`+1) < `passive_bs:

         1. Break

   3. val_loss = 0

   4. for p, passive in enumerate(val_passives):

      1. load passive into passive volume

      2. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. Irradiate volume with mu_bs muons

         2. Infer passive volume

         3. Compute loss based on precision and cost, and add to val_loss

      3. if len(val_passives)-(p`+1) < `passive_bs:

         1. Break

   5. val_loss = val_loss/p

In implementation, the loop is broken up into several functions:

• _fit_epoch() runs one full epoch of volumes and updates for both training and validation

• _scan_volumes() runs over all training/validation volumes, updating parameters when necessary

• _scan_volume() irradiates a single volume with muons multiple batches, and computes the loss for that volume

The optimisation and prediction loops are supported by a stateful callback mechanism. The base callback is Callback, which can interject at various points in the loops. All aspects of the optimisation and prediction are stored in a FitParams data class, since the callbacks are also stored there, and the callbacks have a reference to the wrapper, they are able to read/write to the FitParams and be aware of other callbacks that are running.

Accounting for the interjection calls (on_*_begin & on_*_end), the full optimisation loop is:

1. Associate callbacks with wrapper (set_wrapper)

2. on_train_begin

3. for epoch in n_epochs:

   1. state = “train”

   2. on_epoch_begin

   3. for p, passive in enumerate(trn_passives):

      1. if p % passive_bs == 0:

         1. on_volume_batch_begin

         2. loss = 0

      2. load passive into passive volume

      3. on_volume_begin

      4. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. on_mu_batch_begin

         2. Irradiate volume with mu_bs muons

         3. Infer scatter locations

         4. on_scatter_end

         5. Infer x0 and append to list of x0 predictions

         6. on_mu_batch_end

      5. on_x0_pred_begin

      6. Compute overall x0 prediction

      7. on_x0_pred_end

      8. Compute loss based on precision and cost, and add to loss

      9. if p`+1 % `passive_bs == 0:

         1. loss = loss/passive_bs

         2. on_volume_batch_end

         3. Zero parameter gradients

         4. on_backwards_begin

         5. Backpropagate loss and compute parameter gradients

         6. on_backwards_end

         7. Update detector parameters

         viii. Ensure detector parameters are within physical boundaries (AbsDetectorLayer.conform_detector) viv. loss = 0

      10. if len(trn_passives)-(p`+1) < `passive_bs:

          1. Break

   4. on_epoch_end

   5. state = “valid”

   6. on_epoch_begin

   7. for p, passive in enumerate(val_passives):

      1. if p % passive_bs == 0:

         1. on_volume_batch_begin

         2. loss = 0

      2. on_volume_begin

      3. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. on_mu_batch_begin

         2. Irradiate volume with mu_bs muons

         3. Infer scatter locations

         4. on_scatter_end

         5. Infer x0 and append to list of x0 predictions

         6. on_mu_batch_end

      4. on_x0_pred_begin

      5. Compute overall x0 prediction

      6. on_x0_pred_end

      7. Compute loss based on precision and cost, and add to loss

      8. if p`+1 % `passive_bs == 0:

         1. loss = loss/passive_bs

         2. on_volume_batch_end

      9. if len(val_passives)-(p`+1) < `passive_bs:

         1. Break

   8. on_epoch_end

4. on_train_end

Parameters:

• volume (Volume) – the volume containing the detectors to be optimised

• partial_opts (Dict[str, Callable[[Iterator[Parameter]], Optimizer]]) – dictionary of uninitialised optimisers to be associated with the detector parameters, via _build_opt

• loss_func (Optional[AbsDetectorLoss]) – Optional loss function (required if planning to optimise the detectors)

• partial_scatter_inferrer (Type[ScatterBatch]) – uninitialised class to be used for inferring muon scatter variables and trajectories

• partial_volume_inferrer (Type[AbsVolumeInferrer]) – uninitialised class to be used for inferring volume targets

• mu_generator (Optional[AbsMuonGenerator]) – Optional generator class for muons. If None, will use from_volume().

fit(n_epochs, passive_bs, n_mu_per_volume, mu_bs, trn_passives, val_passives, cbs=None, cb_savepath=Path('train_weights'))

Runs the fitting loop for the detectors over a specified number of epochs, using the provided volumes or volume generators. The optimisation loop can be supported by callbacks.

Parameters:

• n_epochs (int) – number of epochs to run for (a training and validation epoch only counts as one ‘epoch)

• passive_bs (int) – number of passive volumes to use per volume batch (detector updates occur after every volume batch in training mode)

• n_mu_per_volume (int) – number of muons to use in total when inferring the target of a single volume

• mu_bs (int) – number of muons to use per muon batch; multiple muon batches will be used until n_mu_per_volume is reached

• trn_passives (PassiveYielder) – passive volumes to use for optimising the detector

• val_passives (Optional[PassiveYielder]) – optional passive volumes to use for evaluating the detector

• cbs (Optional[List[Callback]]) – optional list of callbacks to use

• cb_savepath (Path) – location where callbacks should write/save any information

Return type:

List[Callback]

Returns:

The list of callbacks

get_detectors()

Return type:

List[AbsDetectorLayer]

Returns:

A list of all AbsDetectorLayer s in the volume, in the order of layers (normally decreasing z position)

get_opt_lr(opt)

Returns the learning rate of the specified optimiser.

Parameters:

opt (str) – string name of the optimiser requested

Return type:

float

Returns:

The learning rate of the specified optimiser

get_opt_mom(opt)

Returns the momentum coefficient/beta_1 of the specified optimiser.

Parameters:

opt (str) – string name of the optimiser requested

Return type:

float

Returns:

The momentum coefficient/beta_1 of the specified optimiser

get_param_count(trainable=True)

Return number of parameters in detector.

Parameters:

trainable (bool) – if true (default) only count trainable parameters

Return type:

int

Returns:

Number of (trainable) parameters in detector

load(name)

Loads saved volume and optimiser parameters from a file.

Parameters:

name (str) – file to load

Return type:

None

opts: Dict[str, Optimizer]

predict(passives, n_mu_per_volume, mu_bs, pred_cb=<tomopt.optimisation.callbacks.pred_callbacks.PredHandler object>, cbs=None, cb_savepath=Path('train_weights'))

Uses the detectors to predict the provided volumes The prediction loop can be supported by callbacks.

Parameters:

• passives (PassiveYielder) – passive volumes to predict

• n_mu_per_volume (int) – number of muons to use in total when inferring the target of a single volume

• mu_bs (int) – number of muons to use per muon batch; multiple muon batches will be used until n_mu_per_volume is reached

• pred_cb (PredHandler) – the prediction callback to use for recording predictions

• cbs (Optional[List[Callback]]) – optional list of callbacks to use

• cb_savepath (Path) – location where callbacks should write/save any information

Return type:

List[Tuple[ndarray, ndarray]]

Returns:

The object returned by the pred_cb’s get_preds method

save(name)

Saves the volume and optimiser parameters to a file.

Parameters:

name (str) – savename for the file

Return type:

None

set_opt_lr(lr, opt)

Sets the learning rate of the specified optimiser.

Parameters:

• lr (float) – new learning rate for the optimiser

• opt (str) – string name of the optimiser requested

Return type:

None

set_opt_mom(mom, opt)

Sets the learning rate of the specified optimiser.

Parameters:

• mom (float) – new momentum coefficient/beta_1 for the optimiser

• opt (str) – string name of the optimiser requested

Return type:

None

class tomopt.optimisation.wrapper.volume_wrapper.ArbVolumeWrapper(volume, *, opts, loss_func=None, partial_scatter_inferrer=<class 'tomopt.inference.scattering.ScatterBatch'>, partial_volume_inferrer=<class 'tomopt.inference.volume.PanelX0Inferrer'>, mu_generator=None)

Bases: AbsVolumeWrapper

Arbitrary volume wrapper in which the user supplies pre-instantiated optimisers for whatever paramters should be optimised.

Wrappers also provide for various quality-of-life methods, such as saving and loading detector configurations, and computing predictions with a fixed detector (predict())

Fitting of a detector proceeds as training and validation epochs, each of which contains multiple batches of passive volumes. For each volume in a batch, the loss is evaluated using multiple batches of muons. The whole loop is:

1. for epoch in n_epochs:

   1. loss = 0

   2. for p, passive in enumerate(trn_passives):

      1. load passive into passive volume

      2. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. Irradiate volume with mu_bs muons

         2. Infer passive volume

      3. Compute loss based on precision and cost, and add to loss

      4. if p`+1 % `passive_bs == 0:

         1. loss = loss/passive_bs

         2. Backpropagate loss and update detector parameters

         3. loss = 0

      5. if len(trn_passives)-(p`+1) < `passive_bs:

         1. Break

   3. val_loss = 0

   4. for p, passive in enumerate(val_passives):

      1. load passive into passive volume

      2. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. Irradiate volume with mu_bs muons

         2. Infer passive volume

         3. Compute loss based on precision and cost, and add to val_loss

      3. if len(val_passives)-(p`+1) < `passive_bs:

         1. Break

   5. val_loss = val_loss/p

In implementation, the loop is broken up into several functions:

_fit_epoch() runs one full epoch of volumes

and updates for both training and validation

_scan_volumes() runs over all training/validation volumes,

updating parameters when necessary

_scan_volume() irradiates a single volume with muons multiple batches,

and computes the loss for that volume

The optimisation and prediction loops are supported by a stateful callback mechanism. The base callback is Callback, which can interject at various points in the loops. All aspects of the optimisation and prediction are stored in a FitParams data class, since the callbacks are also stored there, and the callbacks have a reference to the wrapper, they are able to read/write to the FitParams and be aware of other callbacks that are running.

Accounting for the interjection calls (on_*_begin & on_*_end), the full optimisation loop is:

1. Associate callbacks with wrapper (set_wrapper)

2. on_train_begin

3. for epoch in n_epochs:

   1. state = “train”

   2. on_epoch_begin

   3. for p, passive in enumerate(trn_passives):

      1. if p % passive_bs == 0:

         1. on_volume_batch_begin

         2. loss = 0

      2. load passive into passive volume

      3. on_volume_begin

      4. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. on_mu_batch_begin

         2. Irradiate volume with mu_bs muons

         3. Infer scatter locations

         4. on_scatter_end

         5. Infer x0 and append to list of x0 predictions

         6. on_mu_batch_end

      5. on_x0_pred_begin

      6. Compute overall x0 prediction

      7. on_x0_pred_end

      8. Compute loss based on precision and cost, and add to loss

      9. if p`+1 % `passive_bs == 0:

         1. loss = loss/passive_bs

         2. on_volume_batch_end

         3. Zero parameter gradients

         4. on_backwards_begin

         5. Backpropagate loss and compute parameter gradients

         6. on_backwards_end

         7. Update detector parameters

         viii. Ensure detector parameters are within physical boundaries (AbsDetectorLayer.conform_detector) viv. loss = 0

      10. if len(trn_passives)-(p`+1) < `passive_bs:

          1. Break

   4. on_epoch_end

   5. state = “valid”

   6. on_epoch_begin

   7. for p, passive in enumerate(val_passives):

      1. if p % passive_bs == 0:

         1. on_volume_batch_begin

         2. loss = 0

      2. on_volume_begin

      3. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. on_mu_batch_begin

         2. Irradiate volume with mu_bs muons

         3. Infer scatter locations

         4. on_scatter_end

         5. Infer x0 and append to list of x0 predictions

         6. on_mu_batch_end

      4. on_x0_pred_begin

      5. Compute overall x0 prediction

      6. on_x0_pred_end

      7. Compute loss based on precision and cost, and add to loss

      8. if p`+1 % `passive_bs == 0:

         1. loss = loss/passive_bs

         2. on_volume_batch_end

      9. if len(val_passives)-(p`+1) < `passive_bs:

         1. Break

   8. on_epoch_end

4. on_train_end

Parameters:

• volume (Volume) – the volume containing the detectors to be optimised

• opts (Dict[str, Optimizer]) – Dict of strings mapping to initialised optimisers

• loss_func (Optional[AbsDetectorLoss]) – optional loss function (required if planning to optimise the detectors)

• partial_scatter_inferrer (Type[ScatterBatch]) – uninitialised class to be used for inferring muon scatter variables and trajectories

• partial_volume_inferrer (Type[AbsVolumeInferrer]) – uninitialised class to be used for inferring volume targets

• mu_generator (Optional[AbsMuonGenerator]) – Optional generator class for muons. If None, will use from_volume().

classmethod from_save(name, *, volume, opts, loss_func, partial_scatter_inferrer=<class 'tomopt.inference.scattering.ScatterBatch'>, partial_volume_inferrer=<class 'tomopt.inference.volume.PanelX0Inferrer'>, mu_generator=None)

Instantiates a new PanelVolumeWrapper and loads saved detector and optimiser parameters

Parameters:

• name (str) – file name with saved detector and optimiser parameters

• volume (Volume) – the volume containing the detectors to be optimised

• opts (Dict[str, Optimizer]) – Dict of strings mapping to initialised optimisers

• loss_func (Optional[AbsDetectorLoss]) – optional loss function (required if planning to optimise the detectors)

• partial_scatter_inferrer (Type[ScatterBatch]) – uninitialised class to be used for inferring muon scatter variables and trajectories

• partial_volume_inferrer (Type[AbsVolumeInferrer]) – uninitialised class to be used for inferring volume targets

• mu_generator (Optional[AbsMuonGenerator]) – Optional generator class for muons. If None, will use from_volume().

Return type:

AbsVolumeWrapper

class tomopt.optimisation.wrapper.volume_wrapper.FitParams(**kwargs)

Bases: object

Data class used for storing all aspects of optimisation and prediction when working with AbsVolumeWrapper

Parameters:

kwargs (Any) – objects to be stored

cb_savepath: Optional[Path] = None

cbs: Optional[List[Callback]] = None

cyclic_cbs: Optional[List[CyclicCallback]] = None

device: device = device(type='cpu')

epoch: int = 0

epoch_bar: Optional[ProgressBar] = None

loss_val: Optional[Tensor] = None

mean_loss: Optional[Tensor] = None

metric_cbs: Optional[List[EvalMetric]] = None

metric_log: Optional[MetricLogger] = None

mu: Optional[MuonBatch] = None

mu_bs: Optional[int] = None

n_epochs: Optional[int] = None

n_mu_per_volume: Optional[int] = None

passive_bar: Union[NBProgressBar, ConsoleProgressBar, None] = None

passive_bs: Optional[int] = None

pred: Optional[Tensor] = None

sb: Optional[ScatterBatch] = None

skip_opt_step: bool = False

state: Optional[str] = None

stop: Optional[bool] = None

trn_passives: Optional[PassiveYielder] = None

tst_passives: Optional[PassiveYielder] = None

val_passives: Optional[PassiveYielder] = None

volume_id: Optional[int] = None

volume_inferrer: Optional[AbsVolumeInferrer] = None

warmup_cbs: Optional[List[WarmupCallback]] = None

class tomopt.optimisation.wrapper.volume_wrapper.HeatMapVolumeWrapper(volume, *, mu_opt, norm_opt, sig_opt, z_pos_opt, loss_func, partial_scatter_inferrer=<class 'tomopt.inference.scattering.ScatterBatch'>, partial_volume_inferrer=<class 'tomopt.inference.volume.PanelX0Inferrer'>, mu_generator=None)

Bases: AbsVolumeWrapper

Volume wrapper for volumes with DetectorHeatMap-based detectors.

Volume wrappers are designed to contain a Volume and provide means of optimising the detectors it contains, via their fit() method.

Wrappers also provide for various quality-of-life methods, such as saving and loading detector configurations, and computing predictions with a fixed detector (predict())

Fitting of a detector proceeds as training and validation epochs, each of which contains multiple batches of passive volumes. For each volume in a batch, the loss is evaluated using multiple batches of muons. The whole loop is:

1. for epoch in n_epochs:

   1. loss = 0

   2. for p, passive in enumerate(trn_passives):

      1. load passive into passive volume

      2. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. Irradiate volume with mu_bs muons

         2. Infer passive volume

      3. Compute loss based on precision and cost, and add to loss

      4. if p`+1 % `passive_bs == 0:

         1. loss = loss/passive_bs

         2. Backpropagate loss and update detector parameters

         3. loss = 0

      5. if len(trn_passives)-(p`+1) < `passive_bs:

         1. Break

   3. val_loss = 0

   4. for p, passive in enumerate(val_passives):

      1. load passive into passive volume

      2. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. Irradiate volume with mu_bs muons

         2. Infer passive volume

         3. Compute loss based on precision and cost, and add to val_loss

      3. if len(val_passives)-(p`+1) < `passive_bs:

         1. Break

   5. val_loss = val_loss/p

In implementation, the loop is broken up into several functions:

_fit_epoch() runs one full epoch of volumes

and updates for both training and validation

_scan_volumes() runs over all training/validation volumes,

updating parameters when necessary

_scan_volume() irradiates a single volume with muons multiple batches,

and computes the loss for that volume

The optimisation and prediction loops are supported by a stateful callback mechanism. The base callback is Callback, which can interject at various points in the loops. All aspects of the optimisation and prediction are stored in a FitParams data class, since the callbacks are also stored there, and the callbacks have a reference to the wrapper, they are able to read/write to the FitParams and be aware of other callbacks that are running.

Accounting for the interjection calls (on_*_begin & on_*_end), the full optimisation loop is:

1. Associate callbacks with wrapper (set_wrapper)

2. on_train_begin

3. for epoch in n_epochs:

   1. state = “train”

   2. on_epoch_begin

   3. for p, passive in enumerate(trn_passives):

      1. if p % passive_bs == 0:

         1. on_volume_batch_begin

         2. loss = 0

      2. load passive into passive volume

      3. on_volume_begin

      4. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. on_mu_batch_begin

         2. Irradiate volume with mu_bs muons

         3. Infer scatter locations

         4. on_scatter_end

         5. Infer x0 and append to list of x0 predictions

         6. on_mu_batch_end

      5. on_x0_pred_begin

      6. Compute overall x0 prediction

      7. on_x0_pred_end

      8. Compute loss based on precision and cost, and add to loss

      9. if p`+1 % `passive_bs == 0:

         1. loss = loss/passive_bs

         2. on_volume_batch_end

         3. Zero parameter gradients

         4. on_backwards_begin

         5. Backpropagate loss and compute parameter gradients

         6. on_backwards_end

         7. Update detector parameters

         viii. Ensure detector parameters are within physical boundaries (AbsDetectorLayer.conform_detector) viv. loss = 0

      10. if len(trn_passives)-(p`+1) < `passive_bs:

          1. Break

   4. on_epoch_end

   5. state = “valid”

   6. on_epoch_begin

   7. for p, passive in enumerate(val_passives):

      1. if p % passive_bs == 0:

         1. on_volume_batch_begin

         2. loss = 0

      2. on_volume_begin

      3. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. on_mu_batch_begin

         2. Irradiate volume with mu_bs muons

         3. Infer scatter locations

         4. on_scatter_end

         5. Infer x0 and append to list of x0 predictions

         6. on_mu_batch_end

      4. on_x0_pred_begin

      5. Compute overall x0 prediction

      6. on_x0_pred_end

      7. Compute loss based on precision and cost, and add to loss

      8. if p`+1 % `passive_bs == 0:

         1. loss = loss/passive_bs

         2. on_volume_batch_end

      9. if len(val_passives)-(p`+1) < `passive_bs:

         1. Break

   8. on_epoch_end

4. on_train_end

Parameters:

• volume (Volume) – the volume containing the detectors to be optimised

• mu_opt (Callable[[Iterator[Parameter]], Optimizer]) – uninitialised optimiser to be used for adjusting the xy position of Gaussians

• norm_opt (Callable[[Iterator[Parameter]], Optimizer]) – uninitialised optimiser to be used for adjusting the normalisation of Gaussians

• sig_opt (Callable[[Iterator[Parameter]], Optimizer]) – uninitialised optimiser to be used for adjusting the scale of Gaussians

• z_pos_opt (Callable[[Iterator[Parameter]], Optimizer]) – uninitialised optimiser to be used for adjusting the z position of panels

• loss_func (Optional[AbsDetectorLoss]) – optional loss function (required if planning to optimise the detectors)

• partial_scatter_inferrer (Type[ScatterBatch]) – uninitialised class to be used for inferring muon scatter variables and trajectories

• partial_volume_inferrer (Type[AbsVolumeInferrer]) – uninitialised class to be used for inferring volume targets

• mu_generator (Optional[AbsMuonGenerator]) – Optional generator class for muons. If None, will use from_volume().

classmethod from_save(name, *, volume, mu_opt, norm_opt, sig_opt, z_pos_opt, loss_func, partial_scatter_inferrer=<class 'tomopt.inference.scattering.ScatterBatch'>, partial_volume_inferrer=<class 'tomopt.inference.volume.PanelX0Inferrer'>, mu_generator=None)

Instantiates a new HeatMapVolumeWrapper and loads saved detector and optimiser parameters

Parameters:

• name (str) – file name with saved detector and optimiser parameters

• volume (Volume) – the volume containing the detectors to be optimised

• mu_opt (Callable[[Iterator[Parameter]], Optimizer]) – uninitialised optimiser to be used for adjusting the xy position of Gaussians

• norm_opt (Callable[[Iterator[Parameter]], Optimizer]) – uninitialised optimiser to be used for adjusting the normalisation of Gaussians

• sig_opt (Callable[[Iterator[Parameter]], Optimizer]) – uninitialised optimiser to be used for adjusting the scale of Gaussians

• z_pos_opt (Callable[[Iterator[Parameter]], Optimizer]) – uninitialised optimiser to be used for adjusting the z position of panels

• loss_func (Optional[AbsDetectorLoss]) – optional loss function (required if planning to optimise the detectors)

• partial_scatter_inferrer (Type[ScatterBatch]) – uninitialised class to be used for inferring muon scatter variables and trajectories

• partial_volume_inferrer (Type[AbsVolumeInferrer]) – uninitialised class to be used for inferring volume targets

• mu_generator (Optional[AbsMuonGenerator]) – Optional generator class for muons. If None, will use from_volume().

Return type:

AbsVolumeWrapper

class tomopt.optimisation.wrapper.volume_wrapper.PanelVolumeWrapper(volume, *, xy_pos_opt, z_pos_opt, xy_span_opt, budget_opt=None, loss_func=None, partial_scatter_inferrer=<class 'tomopt.inference.scattering.ScatterBatch'>, partial_volume_inferrer=<class 'tomopt.inference.volume.PanelX0Inferrer'>, mu_generator=None)

Bases: AbsVolumeWrapper

Volume wrapper for volumes with DetectorPanel-based detectors.

Volume wrappers are designed to contain a Volume and provide means of optimising the detectors it contains, via their fit() method.

Wrappers also provide for various quality-of-life methods, such as saving and loading detector configurations, and computing predictions with a fixed detector (predict())

Fitting of a detector proceeds as training and validation epochs, each of which contains multiple batches of passive volumes. For each volume in a batch, the loss is evaluated using multiple batches of muons. The whole loop is:

1. for epoch in n_epochs:

   1. loss = 0

   2. for p, passive in enumerate(trn_passives):

      1. load passive into passive volume

      2. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. Irradiate volume with mu_bs muons

         2. Infer passive volume

      3. Compute loss based on precision and cost, and add to loss

      4. if p`+1 % `passive_bs == 0:

         1. loss = loss/passive_bs

         2. Backpropagate loss and update detector parameters

         3. loss = 0

      5. if len(trn_passives)-(p`+1) < `passive_bs:

         1. Break

   3. val_loss = 0

   4. for p, passive in enumerate(val_passives):

      1. load passive into passive volume

      2. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. Irradiate volume with mu_bs muons

         2. Infer passive volume

         3. Compute loss based on precision and cost, and add to val_loss

      3. if len(val_passives)-(p`+1) < `passive_bs:

         1. Break

   5. val_loss = val_loss/p

In implementation, the loop is broken up into several functions:

_fit_epoch() runs one full epoch of volumes

and updates for both training and validation

_scan_volumes() runs over all training/validation volumes,

updating parameters when necessary

_scan_volume() irradiates a single volume with muons multiple batches,

and computes the loss for that volume

The optimisation and prediction loops are supported by a stateful callback mechanism. The base callback is Callback, which can interject at various points in the loops. All aspects of the optimisation and prediction are stored in a FitParams data class, since the callbacks are also stored there, and the callbacks have a reference to the wrapper, they are able to read/write to the FitParams and be aware of other callbacks that are running.

Accounting for the interjection calls (on_*_begin & on_*_end), the full optimisation loop is:

1. Associate callbacks with wrapper (set_wrapper)

2. on_train_begin

3. for epoch in n_epochs:

   1. state = “train”

   2. on_epoch_begin

   3. for p, passive in enumerate(trn_passives):

      1. if p % passive_bs == 0:

         1. on_volume_batch_begin

         2. loss = 0

      2. load passive into passive volume

      3. on_volume_begin

      4. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. on_mu_batch_begin

         2. Irradiate volume with mu_bs muons

         3. Infer scatter locations

         4. on_scatter_end

         5. Infer x0 and append to list of x0 predictions

         6. on_mu_batch_end

      5. on_x0_pred_begin

      6. Compute overall x0 prediction

      7. on_x0_pred_end

      8. Compute loss based on precision and cost, and add to loss

      9. if p`+1 % `passive_bs == 0:

         1. loss = loss/passive_bs

         2. on_volume_batch_end

         3. Zero parameter gradients

         4. on_backwards_begin

         5. Backpropagate loss and compute parameter gradients

         6. on_backwards_end

         7. Update detector parameters

         viii. Ensure detector parameters are within physical boundaries (AbsDetectorLayer.conform_detector) viv. loss = 0

      10. if len(trn_passives)-(p`+1) < `passive_bs:

          1. Break

   4. on_epoch_end

   5. state = “valid”

   6. on_epoch_begin

   7. for p, passive in enumerate(val_passives):

      1. if p % passive_bs == 0:

         1. on_volume_batch_begin

         2. loss = 0

      2. on_volume_begin

      3. for muon_batch in range(n_mu_per_volume//mu_bs):

         1. on_mu_batch_begin

         2. Irradiate volume with mu_bs muons

         3. Infer scatter locations

         4. on_scatter_end

         5. Infer x0 and append to list of x0 predictions

         6. on_mu_batch_end

      4. on_x0_pred_begin

      5. Compute overall x0 prediction

      6. on_x0_pred_end

      7. Compute loss based on precision and cost, and add to loss

      8. if p`+1 % `passive_bs == 0:

         1. loss = loss/passive_bs

         2. on_volume_batch_end

      9. if len(val_passives)-(p`+1) < `passive_bs:

         1. Break

   8. on_epoch_end

4. on_train_end

Parameters:

• volume (Volume) – the volume containing the detectors to be optimised

• xy_pos_opt (Callable[[Iterator[Parameter]], Optimizer]) – uninitialised optimiser to be used for adjusting the xy position of panels

• z_pos_opt (Callable[[Iterator[Parameter]], Optimizer]) – uninitialised optimiser to be used for adjusting the z position of panels

• xy_span_opt (Callable[[Iterator[Parameter]], Optimizer]) – uninitialised optimiser to be used for adjusting the xy size of panels

• budget_opt (Optional[Callable[[Iterator[Parameter]], Optimizer]]) – optional uninitialised optimiser to be used for adjusting the fractional assignment of budget to the panels

• loss_func (Optional[AbsDetectorLoss]) – optional loss function (required if planning to optimise the detectors)

• partial_scatter_inferrer (Type[ScatterBatch]) – uninitialised class to be used for inferring muon scatter variables and trajectories

• partial_volume_inferrer (Type[AbsVolumeInferrer]) – uninitialised class to be used for inferring volume targets

• mu_generator (Optional[AbsMuonGenerator]) – Optional generator class for muons. If None, will use from_volume().

classmethod from_save(name, *, volume, xy_pos_opt, z_pos_opt, xy_span_opt, budget_opt=None, loss_func, partial_scatter_inferrer=<class 'tomopt.inference.scattering.ScatterBatch'>, partial_volume_inferrer=<class 'tomopt.inference.volume.PanelX0Inferrer'>, mu_generator=None)

Instantiates a new PanelVolumeWrapper and loads saved detector and optimiser parameters

Parameters:

• name (str) – file name with saved detector and optimiser parameters

• volume (Volume) – the volume containing the detectors to be optimised

• xy_pos_opt (Callable[[Iterator[Parameter]], Optimizer]) – uninitialised optimiser to be used for adjusting the xy position of panels

• z_pos_opt (Callable[[Iterator[Parameter]], Optimizer]) – uninitialised optimiser to be used for adjusting the z position of panels,

• xy_span_opt (Callable[[Iterator[Parameter]], Optimizer]) – uninitialised optimiser to be used for adjusting the xy size of panels,

• budget_opt (Optional[Callable[[Iterator[Parameter]], Optimizer]]) – optional uninitialised optimiser to be used for adjusting the fractional assignment of budget to the panels

• loss_func (Optional[AbsDetectorLoss]) – optional loss function (required if planning to optimise the detectors)

• partial_scatter_inferrer (Type[ScatterBatch]) – uninitialised class to be used for inferring muon scatter variables and trajectories

• partial_volume_inferrer (Type[AbsVolumeInferrer]) – uninitialised class to be used for inferring volume targets

• mu_generator (Optional[AbsMuonGenerator]) – Optional generator class for muons. If None, will use from_volume().

Return type:

AbsVolumeWrapper