Optimizers (pyeddl.optim)¶
The classes presented in this section are optimizers to modify the SGD updates during the training of a model.
The update functions control the learning rate during the SGD optimization
Stochastic gradient descent optimizer. |
Stochastic Gradient Descent¶
This is the optimizer by default in all models.
-
class
pyeddl.optim.SGD(lr=0.01, momentum=0.0, decay=0.0, nesterov=False, **kwargs)[source]¶ Stochastic gradient descent optimizer.
Includes support for momentum, learning rate decay, and Nesterov momentum.
- Args:
lr: float >= 0. Learning rate. momentum: float >= 0. Parameter that accelerates SGD
in the relevant direction and dampens oscillations.
decay: float >= 0. Learning rate decay over each update. nesterov: boolean. Whether to apply Nesterov momentum.