Local LoRA Training for Specialised Domain Data

2026-06-22 · Source: LLM on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, medium

Summary

A personal project demonstrated local LoRA fine-tuning of a Large Language Model (LLM) on a 24GB MacBook Air M5 using LoRA and Apple's MLX framework. The objective was to override the base model's default refusals and enforce adherence to niche domain terminology, specifically from the "Honkai: Star Rail (HSR)" game. The author utilized `mlx-lm` to train a LoRA adapter on a `Qwen2.5-1.5B-Instruct` model. Data preparation involved creating `train.jsonl` and `valid.jsonl` files in a specific JSONL format, with diverse user questions and concise assistant answers, often bootstrapped synthetically. The training process successfully completed 1,000 iterations with a batch size of 2, 16 layers for LoRA adapters, and a learning rate of 2e-5, enabling the model to accurately answer HSR lore questions that the base model previously refused.

Key takeaway

For Machine Learning Engineers or AI Students seeking to customize LLMs for specialized domains without cloud reliance, this demonstrates that local LoRA fine-tuning on Apple silicon is highly effective. You can override base model refusals and inject niche terminology using tools like `mlx-lm` on consumer hardware like a MacBook Air M5. Focus on meticulously prepared, granular JSONL training data, even bootstrapping it with other LLMs, to achieve precise domain alignment while managing compute resources.

Key insights

Local LoRA fine-tuning on Apple silicon effectively overrides base model refusals for niche domain terminology using minimal resources.

Principles

• LoRA customizes models offline, preserving privacy.
• Quality over quantity in training data is crucial.
• Fine-tuning risks degrading generalized reasoning.

Method

Train LoRA adapters using `mlx-lm` on Apple silicon, preparing data in JSONL format with diverse user prompts and concise assistant responses, then deploying the adapter with the base model.

In practice

• Use `mlx-lm` for local LLM fine-tuning.
• Bootstrap datasets synthetically with LLMs.
• Structure data as `train.jsonl` and `valid.jsonl`.

Topics

• LoRA Fine-tuning
• Apple MLX
• LLM Customization
• Domain-Specific AI
• Local Inference
• Data Preparation

Code references

• ml-explore/mlx-lm

Best for: Software Engineer, Machine Learning Engineer, AI Student

Related on AIssential

• System Report for CCL25-Eval Task 5: New Dataset and LoRA-Fine-Tuned Qwen2.5 — Domain-specific datasets and LoRA fine-tuning significantly enhance LLM performance in specialized tasks like classical poetry analysis.
• How to Run LLMs Locally (Great For Learning and Privacy) — Specialized tools make running powerful LLMs locally feasible for privacy, learning, and production needs.
• Auditing Training Data in Domain-adapted LLMs: LoRA-MINT — LoRA-MINT uses perplexity and synthetic samples to audit training data membership in LoRA-fine-tuned LLMs.
• Federated LoRA Fine-Tuning for LLMs via Collaborative Alignment — CLAIR enables robust federated LoRA fine-tuning by identifying shared low-rank structures and contaminated clients from local model differences.
• A Practical Guide to LLM Fine Tuning — PEFT methods like LoRA and QLoRA efficiently adapt LLMs by updating minimal parameters, preserving base model knowledge.
• Building LLMs from the Ground Up: A 3-hour Coding Workshop — Building LLMs from scratch, including data preparation, architecture, pre-training, and fine-tuning, enhances fundamental understanding.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by LLM on Medium.