On this tutorial, we exhibit the way to effectively fine-tune a big language mannequin utilizing Unsloth and QLoRA. We deal with constructing a steady, end-to-end supervised fine-tuning pipeline that handles widespread Colab points similar to GPU detection failures, runtime crashes, and library incompatibilities. By rigorously controlling the setting, mannequin configuration, and coaching loop, we present the way to reliably practice an instruction-tuned mannequin with restricted assets whereas sustaining robust efficiency and fast iteration velocity.
import os, sys, subprocess, gc, locale
locale.getpreferredencoding = lambda: "UTF-8"
def run(cmd):
print("n$ " + cmd, flush=True)
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, textual content=True)
for line in p.stdout:
print(line, finish="", flush=True)
rc = p.wait()
if rc != 0:
elevate RuntimeError(f"Command failed ({rc}): {cmd}")
print("Installing packages (this may take 2–3 minutes)...", flush=True)
run("pip install -U pip")
run("pip uninstall -y torch torchvision torchaudio")
run(
"pip install --no-cache-dir "
"torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 "
"--index-url
)
run(
"pip install -U "
"transformers==4.45.2 "
"accelerate==0.34.2 "
"datasets==2.21.0 "
"trl==0.11.4 "
"sentencepiece safetensors evaluate"
)
run("pip install -U unsloth")
import torch
attempt:
import unsloth
restarted = False
besides Exception:
restarted = True
if restarted:
print("nRuntime needs restart. After restart, run this SAME cell again.", flush=True)
os._exit(0)We arrange a managed and appropriate setting by reinstalling PyTorch and all required libraries. We be certain that Unsloth and its dependencies align appropriately with the CUDA runtime accessible in Google Colab. We additionally deal with the runtime restart logic in order that the setting is clear and steady earlier than coaching begins.
import torch, gc
assert torch.cuda.is_available()
print("Torch:", torch.__version__)
print("GPU:", torch.cuda.get_device_name(0))
print("VRAM(GB):", spherical(torch.cuda.get_device_properties(0).total_memory / 1e9, 2))
torch.backends.cuda.matmul.allow_tf32 = True
torch.backends.cudnn.allow_tf32 = True
def clear():
gc.gather()
torch.cuda.empty_cache()
import unsloth
from unsloth import FastLanguageModel
from datasets import load_dataset
from transformers import TextStreamer
from trl import SFTTrainer, SFTConfigWe confirm GPU availability and configure PyTorch for environment friendly computation. We import Unsloth earlier than all different coaching libraries to make sure that all efficiency optimizations are utilized appropriately. We additionally outline utility features to handle GPU reminiscence throughout coaching.
max_seq_length = 768
model_name = "unsloth/Qwen2.5-1.5B-Instruct-bnb-4bit"
mannequin, tokenizer = FastLanguageModel.from_pretrained(
model_name=model_name,
max_seq_length=max_seq_length,
dtype=None,
load_in_4bit=True,
)
mannequin = FastLanguageModel.get_peft_model(
mannequin,
r=8,
target_modules=["q_proj","k_proj],
lora_alpha=16,
lora_dropout=0.0,
bias="none",
use_gradient_checkpointing="unsloth",
random_state=42,
max_seq_length=max_seq_length,
)
We load a 4-bit quantized, instruction-tuned mannequin utilizing Unsloth’s fast-loading utilities. We then connect LoRA adapters to the mannequin to allow parameter-efficient fine-tuning. We configure the LoRA setup to steadiness reminiscence effectivity and studying capability.
ds = load_dataset("trl-lib/Capybara", cut up="train").shuffle(seed=42).choose(vary(1200))
def to_text(instance):
instance["text"] = tokenizer.apply_chat_template(
instance["messages"],
tokenize=False,
add_generation_prompt=False,
)
return instance
ds = ds.map(to_text, remove_columns=[c for c in ds.column_names if c != "messages"])
ds = ds.remove_columns(["messages"])
cut up = ds.train_test_split(test_size=0.02, seed=42)
train_ds, eval_ds = cut up["train"], cut up["test"]
cfg = SFTConfig(
output_dir="unsloth_sft_out",
dataset_text_field="text",
max_seq_length=max_seq_length,
packing=False,
per_device_train_batch_size=1,
gradient_accumulation_steps=8,
max_steps=150,
learning_rate=2e-4,
warmup_ratio=0.03,
lr_scheduler_type="cosine",
logging_steps=10,
eval_strategy="no",
save_steps=0,
fp16=True,
optim="adamw_8bit",
report_to="none",
seed=42,
)
coach = SFTTrainer(
mannequin=mannequin,
tokenizer=tokenizer,
train_dataset=train_ds,
eval_dataset=eval_ds,
args=cfg,
)
We put together the coaching dataset by changing multi-turn conversations right into a single textual content format appropriate for supervised fine-tuning. We cut up the dataset to take care of coaching integrity. We additionally outline the coaching configuration, which controls the batch measurement, studying charge, and coaching period.
clear()
coach.practice()
FastLanguageModel.for_inference(mannequin)
def chat(immediate, max_new_tokens=160):
messages = [{"role":"user","content":prompt}]
textual content = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to("cuda")
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
with torch.inference_mode():
mannequin.generate(
**inputs,
max_new_tokens=max_new_tokens,
temperature=0.7,
top_p=0.9,
do_sample=True,
streamer=streamer,
)
chat("Give a concise checklist for validating a machine learning model before deployment.")
save_dir = "unsloth_lora_adapters"
mannequin.save_pretrained(save_dir)
tokenizer.save_pretrained(save_dir)We execute the coaching loop and monitor the fine-tuning course of on the GPU. We swap the mannequin to inference mode and validate its habits utilizing a pattern immediate. We lastly save the educated LoRA adapters in order that we will reuse or deploy the fine-tuned mannequin later.
In conclusion, we fine-tuned an instruction-following language mannequin utilizing Unsloth’s optimized coaching stack and a light-weight QLoRA setup. We demonstrated that by constraining sequence size, dataset measurement, and coaching steps, we will obtain steady coaching on Colab GPUs with out runtime interruptions. The ensuing LoRA adapters present a sensible, reusable artifact that we will deploy or lengthen additional, making this workflow a strong basis for future experimentation and superior alignment methods.
Try the Full Codes right here. Additionally, be at liberty to observe us on Twitter and don’t neglect to affix our 120k+ ML SubReddit and Subscribe to our E-newsletter. Wait! are you on telegram? now you possibly can be a part of us on telegram as effectively.
