pugh_torch.losses package

Submodules

pugh_torch.losses.hetero_cross_entropy module

pugh_torch.losses.hetero_cross_entropy.hetero_cross_entropy(preds, targets, availables, *, super_index=0, ignore_index=-100, alpha=0, weight=1)[source]

Cross Entropy Loss for Heterogenous Datasets.

Basically a cross-entropy-loss for when you are training on multiple datasets where some of the datasets may have some of the classes labeled as a single superset (for example, background).

Wordy logic:

  1. Lets say Dataset A has classes [“cat”, “dog”, “bird”].

  2. Lets say Dataset B has classes [“cat”, , “other”]. i.e. Dataset B is missing class “dog” and “bird”, and those pixels fall into the “other” category.

  3. For Dataset A, hetero_cross_entropy is exactly the same as normal cross entropy loss.

  4. For Dataset B, a “dog” will be labeled into the “other” category, but we don’t want to penalize the network for predicting dog, because it could be correct. However, we absolutely know that the network should not be predicting “cat”.

  5. To solve this on dataset B, we apply cross entropy loss over the pixels not in the “other” class in the target. In addition to this, we want to maximize the probability that the network guessed either “dog” or “bird” for the pixels labeled as “other” in the target.

Described in “Combining Heterogeneously Labeled Datasets For Training Segmentation Networks”

TODO: this could be extended to multiple supersets, but the API might get complicated, and I personally don’t have a use yet.

Parameters

  • preds (torch.Tensor) – (N, C, …) where C is the number of classes. Typically this is the logits coming out of your network.

  • targets (torch.Tensor) – (N, …) T

  • availables (torch.Tensor) – (N, C) One-hot vector of which classes are availble from the dataset for this exemplar.

  • ignore_index (int) – Pixels with this target value will be not contribute to the loss what so ever. Typically this value might be what you pad the target segmentation image with if you are performing random crops. May be negative.

  • super_index (int) – Pixels with this target value will be encouraged to be of classes not in the dataset it came from. Typically this is a “background”, “unknown”, or “other” class. May be negative.

  • alpha (float) – If nonzero, enables label smoothing. This will divide up the smoothing weight amongst the superclass labels such that the total weight is equivalent to a single class not in the superclass.

  • weight (float or numpy.ndarray or torch.Tensor) – (N, C, …) If C==1, will broadcast across all classes. Applied AFTER the log operation, but before the reduction.

Returns

Scalar loss value

Return type

torch.Tensor

pugh_torch.losses.label_smoothing module

pugh_torch.losses.label_smoothing.label_smoothing(input, target, weight=None, size_average=None, reduce=None, reduction='mean', ignore_index=-100, alpha=0.1, num_classes=None)[source]

Computes the smoothed labels and applies cross-entropy loss.

The smoothed target label is computed as:

y_ls = (1 - alpha) * y_hot + alpha / n_classes

Parameters

  • input (tensor.Tensor) – (B, C, …) Network logits

  • target (tensor.LongTnsor) – (B, …) Ground truth label indices

  • weight (tensor.Tensor) – (C, …)

  • alpha (float) – Smoothing value in range [0,1). If this value is 0, then this is the same as vanilla cross-entropy. If this value is 1, then you get a uniform distribution, which wouldn’t be very useful. Defaults to 0.1

  • num_classes (int) – [optional] Used to compute smoothing amount. if not provided, defaults to the number of classes inferred by the input shape.

Module contents

pugh_torch.losses.get_functional_loss(s)[source]
pugh_torch.losses.reduction_str(s)[source]