Updated on 14th August 2024

Fine-tuning Large Language Models

📝Article • Models & Datasets on: 🤗Hugging Face

This repository provides a collection of Jupyter notebooks that demonstrate how to fine-tune large language models using various tools and techniques.

fine-tuning or instruction tuning is the process where the pre-trained model is further trained on the smaller dataset to adapt its knowledge for a specific task or domain. This process tweaks the model’s parameters to perform specific tasks. In fine-tuning, there are two methods:

Supervised fine tuning (SFT):

In SFT, the model is trained on a labeled dataset. The labeled dataset typically contains examples of instruction (input) and response (output) pairs relevant to the task. In this process, the model learns how to respond to specific instructions.

Reinforcement Learning from Human Feedback (RLHF):

In RLHF, the model interacts with users, generates responses, and receives feedback in the form of reinforcement signals. Basically, the model learns and improves its performance based on the feedback it receives.

Details of each notebook:

LLMs	Description	Dataset	Notebooks	Tuned Model
Phi-2	This model has been fine-tuned on a Medical dataset to answer questions related to diseases and symptoms. (used SFT method)	Dataset		phi-2-medquad
llama-2	This model has been fine-tuned on a dataset with human-generated prompts to answer questions related to general knowledge. (used SFT method)	Dataset		llama-2-finance

Diagram: