Optimization Module

The optimization module provides optimizers for parameter training.

descent.optim

Custom parameter optimizers.

PyTorch Optimizers

DESCENT also supports all standard PyTorch optimizers:

  • torch.optim.Adam

  • torch.optim.AdamW

  • torch.optim.SGD

  • torch.optim.LBFGS

See the PyTorch documentation for details.

Learning Rate Schedules

DESCENT supports all PyTorch learning rate schedulers:

  • torch.optim.lr_scheduler.StepLR

  • torch.optim.lr_scheduler.ReduceLROnPlateau

  • torch.optim.lr_scheduler.CosineAnnealingLR

  • torch.optim.lr_scheduler.ExponentialLR

See the PyTorch optimizer documentation for details.