TAAFT
Free mode
100% free
Freemium
Free Trial
Create tool
Glossary
Deep Learning
Activation Functions

Activation Functions

[ˌæktɪˈveɪʃən ˈfʌŋkʃənz]
Deep Learning
Last updated: December 9, 2024

Definition

Mathematical functions that determine the output of a neural network node based on its input

Detailed Explanation

Activation functions introduce non-linearity into neural networks, allowing them to learn complex patterns. Common functions include ReLU, sigmoid, and tanh, each with different properties affecting network training and performance.

Use Cases

Used in all neural network architectures

Related Terms

Gradient Descent

An optimization algorithm that iteratively adjusts parameters to minimize a cost function by moving in the direction of steepest descent.

Deep Learning

Batch Normalization

Technique that normalizes the input of each layer to improve training stability and speed

Deep Learning

Dropout Regularization

Technique to prevent overfitting by randomly deactivating neurons during training

Deep Learning