Neural Network Backpropagation Lab

Observe the chain rule of calculus in action. Step through forward activation passes, compute binary cross-entropy loss, and propagate analytical partial derivatives ∂L/∂W to update synaptic weights in real-time.

Dataset Shape

Activation Function

Hidden Layer Neurons4

Learning Rate (α)0.15

Feedforward & Backpropagation Architecture

Epoch: 0Loss: 0.5000

Analytical Chain Rule Equations:BCE Loss + RELU

1. Output Delta:

δ² = (&ycirc; − y)

2. Hidden Delta:

δ¹ = (W²·δ²) &odot; σ'(z¹)

3. Weight Update:

W ← W − α (∂L/∂W)

2D Decision Boundary Space

Span: [-1.2, 1.2]

Class 0 (y = 0)

Class 1 (y = 1)

Binary Cross-Entropy Loss CurveLast 40 Epochs

Press "Train Network" to begin loss recording.

Universal Approximation

By combining non-linear activation functions (ReLU, GELU) with stacked linear matrix transformations, a single hidden layer can approximate any continuous decision boundary.

Gradient Flow & Vanishing

Sigmoid saturates at both tails with σ'(z) ≈ 0, causing backpropagated gradients to exponentially vanish in deeper networks. ReLU solves this with constant unit derivative ∂a/∂z = 1 for z > 0.

Step vs Batch Optimization

Full Batch Gradient Descent computes exact average gradients over all N data points, providing smooth and deterministic convergence towards local loss minima.

Loading AI::COSMOS...