(Time Stamp at 1H:50M but the explaination is 20 minutes long) in he really greatly explains about context size, flops
5 hour long interview with Dylan Patel is the founder of SemiAnalysis, a research & analysis company specializing in semiconductors, GPUs, CPUs, and AI hardware. Nathan Lambert is a research scientist at the Allen Institute for AI (Ai2) and the author of a blog on AI called Interconnects
This is a much watch.