The Best AI Model for Creative Writing (I Blind Tested 5 of Them on 50 Samples)
Parag T.
0:00 / 0:00
The Best AI Model for Creative Writing (I Blind Tested 5 of Them on 50 Samples)
151 просмотр · 3 недели назад
Parag T.
65 подписчиков
151 просмотр · 3 недели назад
Writing Craft, a skill for better creative writing: https://withpt.ai/resources/writing-c...
Upcoming live workshop, build AI systems that research, write and deliver: https://withpt.ai/workshop
Same model, same prompt, better copy. These 9 rules came out of a blind test: 50 pieces of writing, scored by 3 judges who had no idea who wrote what. Then the skill went through a second blind test of its own, and the writing scored higher on every prompt.
Set the skill up once, then ask for what you wanted anyway. A post, a newsletter, an email, hooks, a landing page, a rewrite of something flat. It has no house style of its own, so it sits next to whatever style guide you already use and your rules win. Two files, and it works in Claude Code, Codex, Gemini, or pasted into any chat.
Here's where it came from.
5 models, 5 prompts, everything written twice. 50 samples, scrambled and stripped, scored blind by 3 judges. One of them swept all 5 rounds. Then it argued against its own win.
The 5 blind tested prompts were everyday writing jobs: a LinkedIn post from a rough thought, a newsletter opener, 3 angles on a topic, a punch up of flat copy, and 10 hooks. The 3 judges came from 2 different labs, and 2 of the 3 had their own writing sitting in the pile. The one with every reason to disagree with the result agreed anyway.
Then the part I didn't expect. Each judge was asked, in their own words with no vocabulary supplied, what separated the best writing from the worst, and the 3 of them came back with 9 things. I put those 9 into a skill and ran the whole blind test a second time, one model with the skill and the same model without it. With it, the writing got better on every one of the 5 prompts, none worse, and took a perfect score on the 3 angles prompt. Same model, same prompts, same judge, one change.
A lot of what you think is the model is the instructions.
Inside this video:
The 5 guardrails that kept it blind, from scrambled filenames to sealed model identity
Which model wins which job, and where each one loses ground
Whether a model prefers its own writing when it can't see the label
What a 15 out of 15 looks like sitting next to a 6, on the same prompt
The 4 signals all 3 judges named independently, without seeing each other's answers
How the 9 signals became Writing Craft, and the 3 a script settles before you ever see the draft
The same model run with the skill and without it, across all 5 prompts
What the winning model said when I asked if it deserved to win
ABOUT PT
A decade in product and growth, now building AI systems and sharing what works. I run my own products, including Roam Radar (https://roamradar.ca) and AI Matters (https://aimatters.cc). The workflows, prompts and tools here come out of that work.
WHAT'S NEXT
The blind test keeps running. As new models launch they go through the same 5 prompts, the same 3 questions and the same blind set, so the scores line up against these ones.
Subscribe and you get the next one, plus the prompts, skills and workflows I actually run on real work.
Sponsorship and work inquiries:
pt@withpt.ai
TIMESTAMPS
0:00 The question, and the trap
0:52 50 files, nobody knew who wrote them
1:50 The 5 prompts, exactly how you type
3:15 Keeping it honest
5:28 The judges
6:26 The only 3 questions that counted
7:09 Fable scores first
7:43 Sol, the generous one
8:32 Opus, the other flagship
9:13 The scoreboard
9:55 Where each one wins and loses
10:52 Do models mark their own homework
11:23 The one Sonnet won
12:22 What good actually looked like
13:21 The hedging tax
14:25 Who was actually fast
14:57 Why I ran it all twice
16:01 Which one to use for what
16:51 The 9 signals the judges named
17:26 Turning the signals into a skill
17:59 Blind testing the skill
18:17 46% of the gap, closed
19:16 The hooks went from 7 to 12
19:55 I asked the winner if it deserved it
21:14 What you can use today
Find me here:
Instagram: / withpt.ai
X: / withpt_ai
TikTok: / withpt.ai
YouTube: / @withpt_ai