https://pytorch.org/docs/stable/notes/autograd.html
Autograd mechanics — PyTorch 1.10.0 documentation
Autograd mechanics This note will present an overview of how autograd works and records the operations. It’s not strictly necessary to understand all this, but we recommend getting familiar with it, as it will help you write more efficient, cleaner progr
pytorch.org