A hands-on project to understand how large language models are adapted to a domain.
- Tokenize a JSONL news dataset and save it to disk
- Fine-tune GPT-2 with the Hugging Face
Trainer - Generate and evaluate news-style text from prompts
I wrote up what I learned in Generative AI Large Language Model: Train From Pre-trained Model.