Перейти к содержимому

The Best AI Model for Creative Writing (I Blind Tested 5 of Them on 50 Samples)

Parag T.

0:00 / 0:00

The Best AI Model for Creative Writing (I Blind Tested 5 of Them on 50 Samples)

151 просмотр · 3 недели назад
Parag T.
65 подписчиков
151 просмотр · 3 недели назад
Writing Craft, a skill for better creative writing: https://withpt.ai/resources/writing-c... Upcoming live workshop, build AI systems that research, write and deliver: https://withpt.ai/workshop Same model, same prompt, better copy. These 9 rules came out of a blind test: 50 pieces of writing, scored by 3 judges who had no idea who wrote what. Then the skill went through a second blind test of its own, and the writing scored higher on every prompt. Set the skill up once, then ask for what you wanted anyway. A post, a newsletter, an email, hooks, a landing page, a rewrite of something flat. It has no house style of its own, so it sits next to whatever style guide you already use and your rules win. Two files, and it works in Claude Code, Codex, Gemini, or pasted into any chat. Here's where it came from. 5 models, 5 prompts, everything written twice. 50 samples, scrambled and stripped, scored blind by 3 judges. One of them swept all 5 rounds. Then it argued against its own win. The 5 blind tested prompts were everyday writing jobs: a LinkedIn post from a rough thought, a newsletter opener, 3 angles on a topic, a punch up of flat copy, and 10 hooks. The 3 judges came from 2 different labs, and 2 of the 3 had their own writing sitting in the pile. The one with every reason to disagree with the result agreed anyway. Then the part I didn't expect. Each judge was asked, in their own words with no vocabulary supplied, what separated the best writing from the worst, and the 3 of them came back with 9 things. I put those 9 into a skill and ran the whole blind test a second time, one model with the skill and the same model without it. With it, the writing got better on every one of the 5 prompts, none worse, and took a perfect score on the 3 angles prompt. Same model, same prompts, same judge, one change. A lot of what you think is the model is the instructions. Inside this video: The 5 guardrails that kept it blind, from scrambled filenames to sealed model identity Which model wins which job, and where each one loses ground Whether a model prefers its own writing when it can't see the label What a 15 out of 15 looks like sitting next to a 6, on the same prompt The 4 signals all 3 judges named independently, without seeing each other's answers How the 9 signals became Writing Craft, and the 3 a script settles before you ever see the draft The same model run with the skill and without it, across all 5 prompts What the winning model said when I asked if it deserved to win ABOUT PT A decade in product and growth, now building AI systems and sharing what works. I run my own products, including Roam Radar (https://roamradar.ca) and AI Matters (https://aimatters.cc). The workflows, prompts and tools here come out of that work. WHAT'S NEXT The blind test keeps running. As new models launch they go through the same 5 prompts, the same 3 questions and the same blind set, so the scores line up against these ones. Subscribe and you get the next one, plus the prompts, skills and workflows I actually run on real work. Sponsorship and work inquiries: pt@withpt.ai TIMESTAMPS 0:00 The question, and the trap 0:52 50 files, nobody knew who wrote them 1:50 The 5 prompts, exactly how you type 3:15 Keeping it honest 5:28 The judges 6:26 The only 3 questions that counted 7:09 Fable scores first 7:43 Sol, the generous one 8:32 Opus, the other flagship 9:13 The scoreboard 9:55 Where each one wins and loses 10:52 Do models mark their own homework 11:23 The one Sonnet won 12:22 What good actually looked like 13:21 The hedging tax 14:25 Who was actually fast 14:57 Why I ran it all twice 16:01 Which one to use for what 16:51 The 9 signals the judges named 17:26 Turning the signals into a skill 17:59 Blind testing the skill 18:17 46% of the gap, closed 19:16 The hooks went from 7 to 12 19:55 I asked the winner if it deserved it 21:14 What you can use today Find me here: Instagram:   / withpt.ai   X:   / withpt_ai   TikTok:   / withpt.ai   YouTube:    / @withpt_ai