Temperature, Top P, and Other Settings That Make AI Behave (or Go Rogue)
You expect brilliance. You get nonsense.
One moment, AI is writing solid, structured prose. The next, it’s throwing out Shakespearean monologues in the middle of your cozy romance. Or worse. It just keeps repeating the same damn sentence like it’s stuck in a time loop.
What’s going on? AI doesn’t “think” like we do. It follows patterns, probabilities, and the rules you set for it. And if those rules aren’t quite right, your AI assistant can turn into an unpredictable mess.
At Future Fiction Academy, we believe creators need control. That means knowing how to tweak temperature, Top P, and frequency penalties—the settings that decide whether AI is your obedient writing assistant or a chaotic gremlin churning out nonsense.
Let’s break it all down so you can steer AI in the right direction and get it working for you, not against you.
What Are AI Hyperparameters (And Why Should Writers Care)?
Hyperparameters are the knobs and dials that control how AI generates text. They tell AI:
- How creative or predictable to be
- How much randomness to allow in its responses
- How often to repeat words or phrases
Knowing how to tweak these settings lets you fine-tune AI’s output—whether you need structured, logical storytelling or wild brainstorming sessions. Because let’s be real: Sometimes you want a reliable ghostwriter, and sometimes you want a caffeinated improv actor making up plot twists on the fly.
Let’s look at the three biggest AI writing settings you can control.
1. Temperature: The Creativity Dial
Think of temperature as AI’s spontaneity level. It controls how predictable or unpredictable the generated text is.
- Low temperature (0 – 0.3): AI plays it safe. Great for fact-based writing, technical descriptions, or structured storytelling (but can get repetitive). Think of it as the AI equivalent of a cautious accountant.
- Medium temperature (0.4 – 0.7): A balanced mix of logic and creativity. Perfect for fiction writing, dialogue, and world-building that stays coherent.
- High temperature (0.8 – 1.5): AI goes full chaos gremlin. Great for brainstorming, surreal concepts, and unexpected twists, but it can go completely off the rails if you’re not careful.
If AI ever starts hallucinating wildly, throwing in random alien abductions in your cozy mystery, try lowering the temperature.
2. Top P: Controlling the AI’s Focus
Top P (or Nucleus Sampling) determines how broad or narrow AI’s word choices are. If Temperature is the wildness dial, Top P controls how “out there” AI is willing to go with word selection.
- Low Top P (0.1 – 0.3): AI picks from only the most likely words. Great for technical accuracy, but can feel repetitive and boring—like a textbook.
- Medium Top P (0.4 – 0.7): AI considers a wider pool of words. Works well for most fiction writing.
- High Top P (0.8 – 1.0): AI picks from a much larger variety of words. This leads to more diverse, sometimes odd responses—great for poetic prose or unpredictable storytelling.
If AI keeps giving you the same predictable phrases, nudging up Top P can help make it feel more organic. If it starts going on tangents that don’t make sense, lowering Top P reins it in.
3. Frequency & Presence Penalty: Avoiding Repetitive AI Writing
Ever had AI get stuck in a loop, repeating the same phrase like a broken record? That’s where Frequency Penalty and Presence Penalty come in.
- Frequency Penalty: Stops AI from overusing the same words/phrases.
- Presence Penalty: Encourages AI to introduce new concepts.
If AI keeps regurgitating the same line (“His piercing blue eyes stared into her soul.” five times in a row), increasing the Frequency Penalty forces it to mix things up. On the other hand, Presence Penalty makes AI introduce new ideas instead of sticking to the same topic.
What About Top K?
Some AI models also use Top K, which is similar to Top P but works slightly differently. Instead of filtering based on probability, Top K limits the number of words AI can pick from at any given moment.
- Low Top K (5-10): AI sticks to safe, obvious choices.
- High Top K (40+): AI has way more word choices, meaning more creativity but also more risk of going off track.
OpenAI’s GPT models don’t typically use Top K, but other platforms like Amazon Bedrock and some OpenRouter models do.
Which Settings Should Writers Use?
It depends on what you’re writing! Here’s a quick guide:
| Writing Style | Temperature | Top P | Frequency Penalty |
|---|---|---|---|
| Factual/Non-Fiction | 0 – 0.3 | 0.2 – 0.5 | High |
| General Fiction | 0.5 – 0.7 | 0.5 – 0.8 | Medium |
| Experimental Writing | 0.8 – 1.2 | 0.7 – 1.0 | Low |
| Idea Brainstorming | 1.0 – 1.5 | 0.9 – 1.0 | Low |
💡 Bonus Note: While most LLMs follow the expected temperature behavior (low = structured, high = creative), Google’s Gemini model tends to require a much higher temperature (1.2 – 1.25) to produce usable, engaging fiction. If you’re using Gemini and getting rigid, formulaic responses, try increasing the temperature beyond what you’d typically use with other models.
Want to Try It Yourself?
If you’re ready to see an LLM in action, check out RaptorWrite—our free AI writing tool that lets you bring your own Openrouter API key and work directly with models like OpenAI, Claude, and Mistral. No middlemen, no fluff, just pure AI writing power in your hands.
And stay tuned for our next post, where we’ll break down what tokens are, why AI charges by the token, and how to make every word count!
Got questions? Drop them in the comments.
Prefer to Learn in a Different Format?
📺 Prefer a visual guide? Check out our free Introduction to AI Basics for Fiction Writers course: Watch here
📖 Want everything in one place? Grab our AI Basics for Fiction Authors book: Get it on Amazon
TL;DR (Because Writers Love a Summary)
- Temperature controls creativity—low for structure, high for chaos.
- Top P filters word choices—low for predictable, high for variety.
- Frequency & Presence penalties reduce repetition.
- Different settings suit different writing styles.
Next up: How to Experiment with AI and Improve Your AI Prompts!
Last Post: Why AI Counts Every Word: A Author’s Guide to Tokens & Costs
Want to read more of our blog posts? Check out the archive.
