Spinning Up a CPU-Only Micro-LLM with LoRA for Literary Style

Developed by Kashif Salahuddin and Samiya Kashif

We are a literature nerd who also happens to enjoy training models at home. Over a few late nights (and several cups of tea), we built SamKash-Tolstoy, a lightweight LoRA adapter that specializes in the voice, cadence, and moral tension of 19th-century Russian literature. Think Tolstoy’s patience and Dostoevsky’s inner storm.

This post shares the why, what, and how, plus how you can try it or fine-tune it further.

TL;DR

· What: A small LoRA adapter on top of deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B, tuned for Russian-literature style and analysis.

· Data: ~475 public-domain books (Project Gutenberg, Russian literature category) + critics/university articles filtered from the OSCAR web corpus. No copyrighted books.

· Why: A hobby LLM that reliably writes in a Tolstoy/Dostoevsky-inflected voice and answers short literary prompts without rambling.

· Where: Hugging Face → salakash/SamKash-Tolstoy

· How to use: Load the DeepSeek base + this LoRA adapter (code below).

Motivation

General LLMs can imitate classics, but they’re inconsistent, tone slips, moral focus fades, and the prose drifts modern. I wanted something that stays in that world: long sentences when needed, psychological pressure when appropriate, and a feel for everyday moral detail (Tolstoy) versus acute conscience under stress (Dostoevsky).

What makes it different

· Focused corpus: The model’s “home turf” is Russian literature. It reads like it lives there.

· Reasoning-forward base: DeepSeek-Qwen distill is compact yet holds structure well for longish paragraphs which is great for scenes, summaries, and notes on themes.

· Ethical sourcing: Only public-domain books from Gutenberg, plus critics/university articles from OSCAR filtered by relevance.

· Lightweight training: Parameter-efficient LoRA on an Apple M1 Max (or CPU) is enough to steer style without a data center.

Dataset at a glance

· Project Gutenberg (“Russian Literature”): 475 English translations of Russian classics; boilerplate removed and normalized to a single text column.

· OSCAR web corpus (filtered): articles by academics and critics about Russian literature — used sparingly to anchor themes, motifs, and interpretation.

I also did basic keyword filtering (authors/works) and avoided obviously off-topic pages. If I scale up later, I’ll add deduplication and a small classifier for “about literature” vs. “random mentions.”

Model & training recipe

· Base model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B

· Fine-tune method: LoRA (PEFT) on all linear layers

· Sequence length: 1,024–2,048 tokens (packed)

· Optimizer: AdamW (lr ≈ 2e-4)

· Precision: fp16 on MPS (Apple Silicon) or fp32 on CPU

· Tricks: gradient checkpointing; small batch + grad accumulation

· Objective: next-token prediction for domain adaptation + follow-up SFT on short instruction pairs Become a member

What it’s good at (and not)

· It’s good at: Short scenes in a Tolstoy/Dostoevsky-ish tone; quick summaries of themes/motifs; concise “study guide” style answers.

· It’s not for: Factual Q&A without sources; safety-critical or policy-sensitive use; imitating living authors or copyrighted books.

Tip: Add RAG (retrieval) if you want grounded, factual answers, this adapter pairs well with a small FAISS/Qdrant index of your notes.

Try it (5 lines)

import torch from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline from peft import PeftModel

base_id = “deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B” adpt_id = “salakash/SamKash-Tolstoy” # the LoRA adapter on the Hub

device = “mps” if torch.backends.mps.is_available() else “cpu” dtype = torch.float16 if device == “mps” else torch.float32

tok = AutoTokenizer.from_pretrained(base_id, use_fast=True) base = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=dtype); base.to(device) model = PeftModel.from_pretrained(base, adpt_id); model.config.use_cache = True

gen = pipeline(“text-generation”, model=model, tokenizer=tok, device=-1) print(gen(“Write a short scene in the style of Crime and Punishment: a feverish student crosses a Petersburg bridge at night.”, max_new_tokens=180, do_sample=True, temperature=0.7, top_p=0.9)[0][“generated_text”])

One-liners to try: • “Summarize the theme of conscience in War and Peace in 3 sentences.”

• “Rewrite this sentence in a Tolstoyan reflective tone: ‘He made a quick decision.’”

• “Give a two-line character sketch of Sonya from Crime and Punishment.”

Improving responses (fastest paths)

· Decoding controls: lower temperature (0.4–0.7), set max_new_tokens, consider do_sample=False for concise outputs.

· SFT on your prompts: a tiny instruction dataset (200–1,000 pairs) cleans up tone and structure quickly.

· RAG: for factual questions, retrieve short passages and instruct: “Answer only from context; otherwise say ‘I don’t know.’”

How I trained it at home (M1 Max)

· Why LoRA: keep compute tiny and still steer style. LoRA trains only small adapter weights, not the full base.

· Runtime: a few hours for pilot runs with 1–2k sequence length and modest steps; manageable on a laptop.

· Gotchas: tokenizer mismatch ruins results, always load the base model’s tokenizer; pack sequences to keep batches dense.

Ethics & license

· Books: only public-domain titles from Project Gutenberg.

· Web: OSCAR pages filtered for relevance; the repo hosts adapters only, not raw pages.

· License: Adapters and code under Apache-2.0; the base model and datasets retain their original licenses.

· No endorsements: Personal hobby project. No affiliation with publishers or estates.

Roadmap

· Add a small RAG demo Space for grounded Q&A.

· Expand instruction tuning with more style repair and concise summaries.

· Release a tiny dataset card for the filtered literary subset.

· Optional: a merged-weights offline build (subject to base-license constraints).

Links

Hugging Face model: https://huggingface.co/salakash/SamKash-Tolstoy Base model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B

Closing

If you love Russian literature, or teach it, or just want a writing buddy with a 19th-century conscience, please give SamKash-Tolstoy a try. We would love feedback, examples, and gentle critiques. This is a hobby project made from curiosity and respect for the canon; We hope it’s useful to you, too.