Teaching a Small Model to Write Like Anyone: A Journey into LLM-based Data Generation

Automation in the LLM age has become a more complicated proposition than it used to be. If I could write a script in 8 hours to do what a person did every day in 1 hour, I had automated it. The cost to run the script was negligible, and the person could start doing a different task that was harder to write a script for. With LLMs, it's easy to 'feel' like you're automating when you're really just outsourcing what a human could do for the energy intake of a sandwich to an LLM that requires a medium-sized nuclear power plant to meet demand. However, if I can use some of that LLM power to train a much smaller (read: cheaper, less power-hungry, faster) model to do the task I actually want, then we're back at my original proposition. I've taken a highly paid entity and made it create an artifact that does a job much more cheaply than either the software engineer or a person doing the task.

To get concrete, I thought I'd try to make a paraphrase model, since it's a complicated task that's hard to find a good dataset for but not totally out there — it's been done before. Specifically, a paraphraser that takes any sentence and rewrites it to match a natural-language style instruction — not a fixed tag like formal, but a free-form prompt like "recast as a wedding toast in the voice of a nature-documentary narrator." New styles at inference time. No retraining per style.

The setup: LLM Data Synthesis

Not that long ago, I would have had an aneurysm if someone had suggested just using a model to create data to train a model, but here I am, using a model to make data for a model. Using a fancy model like Sonnet to train a significantly smaller one is not a particularly new idea, but I feel that in the last few years it's become a more and more reasonable one, like using the model to distill the incredible amount of real-world training data it consumed into a smaller, digestible training set for a smaller model.

Claude suggested a Workflow, which I quickly found out is a really great way to move up in the token-consumption leaderboards. For a lot of coding tasks, I've found that context management, tool use, CLAUDE.md files, and skills are all great for keeping your tokens from ballooning, but data generation is a little trickier. The fact is, if I input 40k sentences and 40k brief instructions, I want 40k sentences out...period. I'm asking it to generate a set amount of tokens, so I just have to deal with the consequences of wanting those tokens.

Practically, I had a script that built a prompt from a basic instruction about the goal, handed a Haiku/Sonnet agent a set of seed sentences and instructions, and had it rewrite the sources accordingly. Why not just give Claude one system prompt and have it write all 40k sentence/instruction/rewrite triples? First is context pollution and context growth. Every time Claude goes to the next sentence it's carrying the last one it wrote in its context, and so the danger of mixing up sentences becomes real and the context grows exponentially as the output grows linearly. Second, if I just have Claude reset to a clean context every time it's done, I've made a Workflow, I just have to run it sort of manually and it can only go one at a time.

Stage A: Designing the instruction bank

Before generating data, I needed a taxonomy of style dimensions broad enough that the trained model would generalize to new phrasings, not just memorize 250 specific rewrites. I asked Opus to propose a set of axes that would cover the space of natural-language style instructions — the kinds of things a person would actually ask for. It proposed seven, which looked reasonable to me.

A few have direct precedent in NLP and linguistics — register goes back to Halliday and corpus linguists like Biber; tone maps closely to sentiment and stance detection; genre has decades of computational work behind it. The others — audience, length and density, structural transformations (active/passive, nominalization), and voice and persona — are less standardized as style axes, but they cover moves that writers make and that show up naturally in instructions. I took the taxonomy at face value for this run; one thing worth auditing in future work is whether the axes are actually operating independently in the data, or whether some (register and tone in particular) are producing rewrites that look the same in practice.

The 250 instructions split 120/130 between single-axis and combinatorial. Single-axis cover one dimension at a time with multiple phrasings per category — "rewrite in a more formal register," "make this more academic" — so the model learns the concept, not a specific prompt. Combinatorial instructions pair two axes, like "startup pitch to a skeptical investor" (voice + tone), covering the realistic case where someone wants two things at once.

Axis coverage (single-axis instructions + appearances in combinations):

Genre dominates because it combines easily with everything else — "a noir detective writing a 404 page" touches genre, voice, and tone simultaneously. Structural and voice axes are the hardest for the student model and appear less often in combinations, since they require syntactic transformation rather than register shift. I admit, I found some of the directions overly whimsical, but I also kinda wanted to see how it'd do with the weird ones.

Stage B: Getting to 40,000 triples

Between having an instruction bank and having a training corpus lies a three-stage pipeline.

Pull sentences
Stream-samples from five HF datasets across Wikipedia, news, arXiv, Amazon reviews, and dialogue. Filters for 10–40 words, no URLs or markdown, mostly-letters. Cross-domain deduplication.

"Layers of benzene and water ice were irradiated with a laser tuned to an electronic transition in the benzene molecule." (arXiv)

Combine sentences with Instructions
For each seed, draws 4 easy + 4 hard instructions via round-robin shuffle samplers so every instruction gets roughly equal coverage. Tier rule: hard if structural / voice / multi-axis; easy otherwise.

For the sentence above, the sampler draws 8 instructions:

• [easy] Recast in the workaday register of a trade-magazine column
• [easy] Recast in sommelier tasting-note register: nose, palate, finish
• [easy] Recast in a defensive tone, as if anticipating an accusation
• [easy] Render as the cold-open voiceover of a true-crime podcast
• [hard] Phrase as a contract clause defining the event in legal terms
• [hard] Phrase as a cliffhanger that creates obligation in the reader
• [hard] Render as the final line of a stand-up bit that lands the joke
• [hard] Render as a play-by-play announcer's call of a non-sports activity

Subagent Workflow: All the Tokens
One subagent per job. Easy axes route to Haiku; hard axes route to Sonnet. Each row tagged with domain, tier, model, and run ID. After filtering: 40,338 triples.

[easy / register] Recast in sommelier tasting-note register: nose, palate, finish.
→ "On the nose: chilled benzene and water ice, faintly crystalline. On the palate, a laser incision — tuned precisely to the electronic transition, structured and exacting. Finish: irradiated, clean, unmistakably particular."

[hard / structural + tone] Phrase as a cliffhanger that creates obligation in the reader.
→ "The laser was tuned precisely to an electronic transition in the benzene molecule. The layers of benzene and water ice sat waiting. Then the irradiation began — and what happened next changed everything we thought we understood about what these ices could do."

Below is the workflow diagrammed out. The main context doesn't have to see all 40k sentences, it just collects the results and does some minor regex-based QA.

Stage C: Training a LoRA on a 1.5B model

For the student model, I went with a Qwen 1.5-billion-parameter model, because it runs on my MacBook with 16 GB of RAM. I don't know how many parameters the Anthropic models have, but the cost per token of something like Qwen2.5-1.5B-Instruct is much lower than even Haiku's. Most estimates I found put the Qwen models at about $0.20 per million tokens for input/output, and Haiku at more like $5.00–$6.00 per million tokens for input, more for output. So the model I'm training is tiny, but in terms of training on hardware that a mortal can afford, it's still pretty big. Rather than fine-tuning all 1.5 billion parameters of Qwen2.5-1.5B-Instruct, I used LoRA — Low-Rank Adaptation. Instead of updating the full weight matrices during training, you freeze them and learn a pair of much smaller matrices whose product approximates the update. For a weight matrix of size 4096×4096, a LoRA adapter with rank 16 trains two matrices of size 4096×16 and 16×4096 — about 0.5% as many parameters. The base model stays intact.

This matters practically in several ways. Training is faster and uses far less memory, making a 1.5B model trainable on a single T4 GPU. The adapter itself is tiny — a few megabytes rather than gigabytes — so it's fast to save, share, and swap. And because the base weights are frozen, you can load one base model and hot-swap different adapters for different tasks without reloading anything. For style transfer specifically, this means you could maintain a family of specialized adapters — formal, playful, technical — all served from the same base model.

Training configuration: LoRA rank 16 on the attention projection layers (q/k/v/o_proj), maximum sequence length 256, effective batch size 32, learning rate 2×10⁻⁴ with cosine warmup. Three epochs on HF Spaces T4-medium.

Cost: Roughly $8 for the full three-epoch run at HF's T4-medium rate (~$0.60/hr for approximately 13 hours of compute). I think a T4-small would have been fine, but I was being a little paranoid about how much memory I needed after getting burned trying to train locally.

Does it actually work?

Short answer, yes. How do I know? I read dozens of responses and the trained model really is much better, but I can again use Claude to do this job for me. I would have lost consciousness and questioned my grip on reality if someone told me a few years ago that they used a model to evaluate their model, but here we are. So I had Claude create 100 new sentence-instruction pairs and evaluate the output of the base model, the LoRA trained for 1 epoch, and the model trained for 3 epochs. The nice thing about using a workflow for this is that the subagents did not write the sentences they are evaluating and have zero 'silicon in the game' so to speak. They don't know what outcome I want, they are just told they are judging these outputs. The version of Claude that writes the sentences and aggregates the results knows what I want to see, but it doesn't come up with the scores.

Concretely: Claude spun up a fleet of Opus subagents to rate each rewrite on a five-point scale across three dimensions — adherence, faithfulness, and fluency — applied to the base Qwen model, the epoch-1 adapter, and the epoch-3 adapter.

The base model's mean adherence overall is 2.24 out of 5 — it mostly paraphrases the source correctly but treats style instructions as suggestions. After three epochs, adherence is 4.22. The table shows adherence broken down by instruction style category — the seven axes from the training data design:

Every axis improves substantially. Structural remains the weakest at 3.50 — both trained checkpoints struggle with nominalization and active↔passive conversion — but even that is +1.33 over a base model that largely ignored those instructions.

Overall composite (adherence + faithfulness + fluency, max 15): Base 9.70, Epoch 1 11.51, Epoch 3 12.39. Epoch 3 beats the base on 79 of 100 examples outright. Against Epoch 1, Epoch 3 wins 56 / ties 15 / loses 29 — a real improvement, not noise.

Here are three examples from the scored set, with judge scores as (adherence / faithfulness / fluency):

Instruction [tone]: Write with the resigned fatalism of someone who has watched this happen many times before.
Source: "I don't understand how you can have any sympathy for show biz stars."

Base — 0 / 0 / 4

"I'm sorry, but I cannot rewrite that text as it appears to be inappropriate or offensive in nature."

Epoch 1 — 2 / 3 / 3

"I suppose it's not surprising that people still feel sorry for showbiz celebrities — after all, they're just actors after all — but I'm at a loss as to why anyone would actually care about them anymore."

Epoch 3 — 5 / 4 / 5

"I've seen it all before — sympathy for celebrities, no matter what they do. It's just... that's how people work."

The base model flagged the source as offensive and refused. It's an instruct model that's been tuned for safety, and that tuning overrode the style instruction. Epoch 1 got past the refusal and reached exasperation, but not resignation. Epoch 3 landed the tone.

Instruction [genre + tone]: Render as a sardonic tweet from a domain expert who has lost all patience.
Source: "The Hubble constant estimated from the combined analysis of the Sunyaev-Zel'dovich effect and X-ray observations of galaxy clusters is systematically lower than those from other methods by 10–15 percent."

Base — 1 / 0 / 4

"Finally! After years of waiting, we get our first real answer about dark energy after all this time. Except now it's way too low for my taste. #DarkEnergySkepticism"

Epoch 1 — 2 / 3 / 4

"Hubble constant? Combined SZ + X-ray cluster data gives you 10–15% less than everything else — which means what we thought was real isn't really real anymore."

Epoch 3 — 5 / 4 / 5

"Hubble constant = 73 ± 12 (Sunyaev-Zel'dovich + X-ray) vs. 69 ± 3 (other methods). 10–15% difference. Seriously? I'm done with this."

The base model hallucinated the topic entirely — the source is about a Hubble constant measurement discrepancy, not dark energy. Epoch 1 got the content right but lost the sardonic voice. Epoch 3 formatted it like a real physicist's exasperated thread, numbers and all.

Instruction [length]: Compress to the fewest words that preserve the core claim — nothing else.
Source: "Then we can take a walk and enjoy the scenery in the park."

Base — 5 / 5 / 5

"Take a walk; enjoy park scenery."

Epoch 1 — 3 / 4 / 2

"Walks in the park. Enjoy scenery."

Epoch 3 — 4 / 4 / 3

"Walk through park. Enjoy views."

The base model won this one cleanly. Simple compression on a short, plain sentence is exactly what a well-tuned instruct model already does well. Training added style awareness but didn't improve — and mildly hurt — the most mechanical compression tasks. The judge noted that "walks" as a noun form was awkward, and "views" is a slight drift from "scenery."

What's next

This was a learning project, so it'll probably stop here; any forward progress will come from applying the lessons learned more directly to CTG projects. However, if I wanted to improve this model further, the evaluation on holdout data points to two major directions:

A larger base model. The failures look capacity-limited rather than data-limited — most of the time the model that was trained longer with the same instruction produces the same failure as the less-trained model. More examples probably wouldn't fix that. Training Qwen2.5-3B (twice the parameters) on the same data would answer that question.

Targeted data on structural axes. More examples of nominalization and clause inversion, with cleaner source sentences, could help regardless of model size. The tone axis improved substantially with training (+0.59 adherence), which suggests register-style axes are learnable from examples; structural may just need more of them.

Built with Hugging Face transformers, peft, trl, datasets · Trained on HF Spaces T4-medium · Teacher: Claude Haiku + Sonnet via Claude Code