Fine-tuning TinyLLaMA
Published:
2024 · PEFT and LoRA
Fine-tuned TinyLLaMA on the CUAD legal-contract dataset using LoRA adapters and Kaggle GPUs, with Weights & Biases experiment tracking.
The recorded project results were 1.1219 validation loss and 1.0250 training loss after 2,800 steps.
Technologies: Python, PyTorch, Transformers, PEFT, LoRA, Weights & Biases.