Hi, this is Harold. I document here my learning notes. The content spans many aspects of ML research and engineering.
Good notes
- Golden rules for scaling deep neural networks
- The diffusion process
- GPU memory tiling
- Activation checkpointing
- Floating point numbers
Resources I keep pointing people to
- Machine Learning Engineering Open Book
- CUDA MODE youtube channel
- Understanding Diffusion Models: A Unified Perspective
- Sasha Rush youtube channel