Mini toolsDefinition
A hyperparameter that controls how much to adjust the model in response to the estimated error each time the model weights are updated.
Detailed Explanation
The learning rate determines the size of the steps taken during optimization. Too large a learning rate can cause unstable training and overshooting optimal values while too small a rate can result in slow convergence or getting stuck in local minima. Various scheduling techniques like learning rate decay and warm-up can help optimize training dynamics.
Use Cases
Neural network training gradient descent optimization transfer learning
