Introduction

• We will learn:

  • How to prepare a large dataset from the Hub
  • How to use Trainer API to fine-tune a model
  • How to use a custom training loop
  • How to leverage the Huggingface Accelerate library to easily run that custom training loop on any distributed setup

• Learned:

  • Learned about datasets in the Hub
  • Learned how to load and preprocess datasets, including using dynamic padding and collators
  • Implemented your own fine-tuning and evaluation of a model
  • Implemented a lower-level training loop
  • Used HuggingFace Accelerate to easily adapt your training loop so it works for multiple GPUs or TPUs