pugh_torch.optimizers package¶

Subpackages¶

class pugh_torch.optimizers.base.Optimizer(params, defaults)[source]¶

Bases: torch.optim.optimizer.Optimizer

Only used to automatically register optimizers functions.

Lookahead Optimizer Wrapper. Implementation modified from: https://github.com/alphadl/lookahead.pytorch Paper: Lookahead Optimizer: k steps forward, 1 step back - https://arxiv.org/abs/1907.08610

class pugh_torch.optimizers.lookahead.Lookahead(base_optimizer, alpha=0.5, k=6)[source]¶

load_state_dict(state_dict)[source]¶

Loads the optimizer state.

Arguments:

state_dict (dict): optimizer state. Should be an object returned: from a call to state_dict().

state_dict()[source]¶

Returns the state of the optimizer as a dict.

It contains two entries:

state - a dict holding current optimization state. Its content
differs between optimizer classes.
param_groups - a dict containing all parameter groups

step(closure=None)[source]¶

Performs a single optimization step (parameter update).

Arguments:

closure (callable): A closure that reevaluates the model and: returns the loss. Optional for most optimizers.

Note

Unless otherwise specified, this function should not modify the .grad field of the parameters.

pugh_torch.optimizers.lookahead.LookaheadAdam(params, alpha=0.5, k=6, *args, **kwargs)[source]¶

class pugh_torch.optimizers.ralamb.Ralamb(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0)[source]¶

step(closure=None)[source]¶

Performs a single optimization step (parameter update).

Arguments:

closure (callable): A closure that reevaluates the model and: returns the loss. Optional for most optimizers.

Note

Unless otherwise specified, this function should not modify the .grad field of the parameters.

pugh_torch.optimizers.rangerlars.RangerLars(params, alpha=0.5, k=6, *args, **kwargs)[source]¶