Fine-tune LLMs with LlamaFactory on Google Colab

What this is, in plain English

Every chatbot you’ve used (ChatGPT, Gemini, Claude) is a “large language model,” or LLM: software that read enormous amounts of text and learned to predict what words come next. “Fine-tuning” means taking one of these models and training it a little further on your own examples, so it gets reliably good at one narrow job, in your wording, your format, your domain.

LlamaFactory is a free, open-source tool that runs that training for you across 100+ different models, so you never write the training code yourself. Google Colab is a free Google website that lends you a powerful computer in the cloud, including a GPU (the specialised chip AI training needs and most laptops don’t have).

Be honest with yourself first: this is an advanced workflow. You don’t need a computer-science degree, but you do need to be comfortable running code cells you don’t fully understand and occasionally reading an error message. There is no single magic prompt to paste. The exact, up-to-date instructions live inside the Colab notebook itself and shift as the tool updates, so this page does two jobs: it shows you what fine-tuning is genuinely good for (so you can decide if it’s worth your time), and it walks you to the right notebook and through the parts that trip people up.

What you can use it for

Fine-tuning earns its keep when a plain prompt keeps giving inconsistent results and you have examples of what “right” looks like. Common uses:

• A support bot that sounds like your business. Train a model on your past customer chats so it answers in your tone and knows your products, instead of generic replies.
• A specialist Q&A model. Feed it a body of material (course notes, a manual, legal or medical guidelines) so it answers in that narrow area more accurately than a general chatbot.
• Consistent formatting at scale. When you need output always in the same shape (a fixed JSON structure, a set report layout), a fine-tuned model obeys far more reliably than one nudged by prompt alone.
• A private, offline model. Fine-tune a small open model you can later run on your own machine, so sensitive data never leaves your device.
• Classification and tagging. Sort tickets, messages, or documents into your own categories with your own labels.

Tools you need

• LlamaFactory (free): the open-source tool that runs the fine-tuning. Nothing to install locally; it runs inside the notebook.
• Google Colab (free): the cloud computer that supplies the GPU. A free Google account is all you need to start.
• Your examples (free): a set of example inputs and the ideal outputs you want the model to learn from. A few hundred good examples can already make a difference; quality matters more than quantity.

How it actually works

You are not writing code from scratch. You are running an existing notebook cell by cell and changing a few settings. The realistic path:

1. Open the official LlamaFactory Colab notebook (linked from the project’s GitHub page) and click File, then “Save a copy in Drive” so you have your own editable version. You should now be editing a copy titled “Copy of…”.
2. Switch on the free GPU. Go to Runtime, then “Change runtime type,” choose a GPU, and save. Without this, training is painfully slow or fails outright.
3. Run the setup cell. The first cell installs LlamaFactory. Click the play button to its left and wait for the green check. You’ll see install logs; “Successfully installed” near the end means it worked.
4. Point it at your examples. The notebook shows the exact data format it expects (usually a simple file of input and ideal-output pairs). Match your data to that format and upload it. The notebook, not this page, is the source of truth for the precise format, so follow it.
5. Pick your model and method. You’ll choose a base model and, almost always, a method called LoRA or QLoRA (explained below), because it’s the only kind of fine-tuning that fits in a free Colab’s memory.
6. Start training and watch the “loss.” Run the training cell. A number called the loss should trend downward over time; that’s the model improving. Exact settings depend on your data, so start with the notebook’s defaults before changing anything.
7. Save and download the result. When training finishes, the notebook gives you cells to save the fine-tuned model and download it. Do this before closing the tab, or you lose it.

Where exact values aren’t given here, that’s deliberate: they depend on your data and chosen model, and the notebook’s defaults are a safe starting point.

Words you’ll see, explained

• LLM / VLM: a large language model handles text; a vision-language model (VLM) also handles images.
• Fine-tuning: extra training on your own examples to specialise a model.
• GPU: the chip that makes AI training fast. Colab lends you one free.
• LoRA / QLoRA: lightweight fine-tuning methods that train a small add-on instead of the whole model, so it fits on free hardware. QLoRA is the even lighter version. Pick these unless you have a reason not to.
• Epoch: one full pass through all your examples. More epochs means more learning, but too many makes the model memorise instead of generalise.
• Loss: a score for how wrong the model currently is. Lower is better; you want it falling during training.

Original source

Based on the LlamaFactory project by hiyouga, an open-source framework for fine-tuning over 100 language and vision-language models, presented at ACL 2024. The project and its ready-to-run Colab notebook are hosted on GitHub.

Notes & variations

• Do you even need this? If you only need the model to follow instructions for an occasional task, you probably don’t need fine-tuning at all. A well-written prompt, or feeding the model your documents at question time (a method called “RAG”), is cheaper and faster. Reach for fine-tuning when prompting alone keeps falling short and you have real examples.
• Free-tier limits: Colab’s free GPU has time and memory caps and can disconnect on long runs. Keep datasets and models small to start, and stick to QLoRA. If you outgrow it, paid Colab or another cloud GPU lifts the limits.
• Common pitfall: forgetting to save a copy of the notebook to your Drive first, or closing the tab before downloading the trained model. Either one loses your work.