Stable Diffusion Prompts: Structure, Weights, Negatives

Prompting Stable Diffusion is different from prompting a hosted chat model, because nothing rewrites your text before the model sees it. What you type is what the model gets, in the order you typed it, with the weights you gave it. That makes the prompt a precise instrument once you learn its grammar. This guide covers the structure that works across SD 1.5, SDXL and SD 3.5, the weighting and negative-prompt syntax, how LoRA trigger words fit in, and a method for testing changes so you actually learn from each run.

Front-load the prompt

Stable Diffusion gives more attention to words near the start of the prompt, and very long prompts get truncated or split into chunks depending on the interface. Put the thing that matters most first:

1. Subject. "a weathered fisherman in a yellow raincoat"
2. Action or pose. "mending a net"
3. Setting. "on a stone harbour wall, small boats behind"
4. Composition. "medium shot, eye level, off-centre"
5. Light. "overcast morning, soft diffuse light"
6. Medium and finish. "35 mm photograph, natural colour" or "gouache illustration, visible brush strokes"

For SD 1.5 and SDXL, comma-separated phrases in that order work well. SD 3.5 was trained on more natural captions and follows a plain sentence or two more faithfully, so write it as prose and keep commas for genuine lists.

Quality tags such as "masterpiece, best quality, 8k" became a habit with SD 1.5 fine-tunes that were trained with them. They help on some of those checkpoints and do nothing or harm on SDXL and SD 3.5. Check the checkpoint’s model card; if its examples use them, use them, otherwise leave them out.

Weighting syntax

In the web UI and most ComfyUI text encoders you can change how much a phrase counts:

• (word:1.3) increases emphasis; (word:0.7) reduces it. Stay between roughly 0.6 and 1.4; beyond that, images break.
• (word) on its own is a small boost and [word] a small reduction in the web UI. Prefer explicit numbers so your prompts are readable later.
• Order still matters even with weights. A weighted phrase at the end does not match an unweighted one at the start.

Weighting is for nudging, not rescuing. If a subject refuses to appear, the fix is usually a clearer noun or a different checkpoint, not a weight of 1.8.

Negative prompts

The negative prompt lists concepts to steer away from. For SD 1.5 and SDXL a short standard set covers most cases: "blurry, low quality, deformed hands, extra fingers, text, watermark, signature". Add specifics per job: "cartoon" when you want photography, "photograph" when you want illustration, "crowd" when you want one person.

Keep it short. Twenty negatives dilute each other and can remove things you wanted. SD 3.5 generally needs far fewer negatives, and some fast or distilled variants ignore the negative prompt entirely because they run at CFG 1; read the model card.

LoRA trigger words and style phrases

Many LoRAs are trained with a trigger word that activates the concept. The model card tells you the word and a suggested weight; include the trigger near the start of the prompt and load the LoRA at the recommended weight. If a style LoRA overwhelms your subject, reduce its weight before you touch the prompt. The settings guide covers weights, CFG and the other controls that interact with prompting.

Test one thing at a time

The method that separates people who understand their prompts from people who get lucky:

• Fix the seed. With everything else held, the only difference between two images is the word you changed.
• Change one phrase per run. Swap "overcast" for "golden hour" and look. Then swap it back and change the medium.
• Use a grid. The web UI’s X/Y/Z plot and ComfyUI batch nodes let you vary a word against CFG or a weight in one go.
• Save the full set. Prompt, negative, checkpoint, LoRAs, seed, sampler, steps, CFG and size. Most interfaces embed these in the PNG; keep that file, not a re-exported JPG.

Settings at a glance

Try it

Positive: a weathered fisherman in a yellow raincoat mending a net, on a stone harbour wall, small boats behind, medium shot, eye level, overcast morning, soft diffuse light, 35 mm photograph, natural colour
Negative: blurry, low quality, deformed hands, extra fingers, text, watermark, cartoon
Checkpoint: SDXL   Seed: 2024 (fixed)   Steps: 25   CFG: 6   Size: 1024 x 1024

Run it, then change only "overcast morning, soft diffuse light" to "golden hour, long shadows" and compare. Then try (yellow raincoat:1.3) to see how weighting shifts attention without changing anything else.