Backpropagation (On a High-Level)
(unfinished, will come back and edit later)
I'm a second-year student at the University of Texas at Austin with an interest in engineering, math, and machine learning.
So far, we've looked at forward propagation. Now, it's time to explore back propagation.
Recall that forward propagation was used for propagating outputs of one layer to the input of the next. Backpropagation moves in the opposite direction and is a crucial part of training a neural network.
Note that neural networks are recursive structures. In other words, the training process consists of a lot of self-repeating steps. Backpropagation plays a major role in this training process by recursively fine-tuning the weights of the neural network based on the error/loss incurred during the prior epoch. It is essentially an implementation of gradient descent but for neural networks.
There is some calculus involved here that I am not fully sure of and that may be difficult to explain in a short article. For now, I am linking some good resources to explore this topic further; after I have properly understood the concept, I will come back and update this article.
Resources:
