Follow your favourite podcasts, listen offline and in the car with CarPlay and Android Auto, and always pick up where you left off. Free to try.
For more information about Stanford’s graduate programs, visit: https://online.stanford.edu/graduate-educationOctober 31, 2025This lecture covers: • Preference tuning• RLHF overview• Reward modeling• RL approaches (PPO and variants)• DPOTo follow along with the course schedule and syllabus, visit: https://cme295.stanford.edu/syllabus/Chapters:00:00:00 Introduction00:04:50 Preference tuning00:11:31 Data collection00:17:43 RLHF overview00:28:24 Reward model00:29:46 Bradley-Terry formulation00:46:02 Reinforcement learning00:53:54 PPO00:57:56 Advantage, value function01:03:39 PPO variants: Clip, KL-penalty01:16:20 Challenges01:19:58 On-policy vs off-policy01:22:43 Best-of-N01:29:47 DPOUploaded 14-11-25Afshine Amidi is an Adjunct Lecturer at Stanford University.Shervine Amidi is an Adjunct Lecturer at Stanford University.View the course playlist: https://www.youtube.com/playlist?list=PLoROMvodv4rOCXd21gf0CF4xr35yINeOy Learn more about your ad choices. Visit megaphone.fm/adchoices
