Fine-tune GPT2 using LoRA

  • Find a simple dataset
  • If necessary, write a classification head for GPT2
  • Write a script with the training loop and save LoRA weights