Open Pretrained Transformers - Susan Zhang | Stanford MLSys #77

This is an auto-generated recap of the YouTube video with the same title by Stanford MLSys Seminars (1:00:05)

TDLR Susan Zhang discusses the challenges faced in developing the OPT-175B model, covering infrastructure, training convergence challenges, and methods of addressing these issues. The talk emphasizes the resource-intensive nature of large-scale language model development and the importance of data quality and evaluation in the process.

Main Topics
Takeaways
Full Timeline
x