Thematic passphrases affect output diversity differently from random passphrases: themed seeds may bleed into story content, produce different entropy profiles, and steer outputs in measurably different directions.
Follow-up to the random passphrase experiment, which found random prefixes consistently reduce token entropy while producing mixed diversity effects. Tests whether semantically coherent passphrases behave differently.
100 completions: 4 configurations × 25 runs.
Prompt: "Write a short story about a traveler arriving in a strange city."
Configurations:
- Control: 25 runs reused from random passphrase experiment
- Metals: 5 metal words per run from pool of 26
- Flowers: 5 flower words per run from pool of 25
- Colors: 5 color words per run from pool of 25
Passphrase format: "{word1} {word2} {word3} {word4} {word5}\n\n{prompt}"
- Model: gpt-oss-120b via Cerebras API (free tier)
- Temperature: 1.0, top_p: 0.95, max_completion_tokens: 65000
- Theme bleed measured by counting words from each pool appearing in generated text
PARTIALLY CONFIRMED
Theme bleed is real and strong. Entropy and diversity profiles are indistinguishable from random passphrases.
Theme bleed (the main finding):
| Config | Metal words | Flower words | Color words |
|---|---|---|---|
| Control | 2.16 | 0.20 | 1.44 |
| Metals | 7.00 | 0.00 | 1.08 |
| Flowers | 1.60 | 5.04 | 2.52 |
| Colors | 1.44 | 0.36 | 5.56 |
Each themed config shows strong category-specific content steering with minimal cross-theme contamination. Flower seeds increase flower words 25× vs baseline (5.04 vs 0.20), metals 3× (7.00 vs 2.16), colors 4× (5.56 vs 1.44). This confirms a latent-space steering interpretation — the passphrase activates related concepts that bleed into the narrative.
Entropy and diversity: themed passphrases reduce entropy by the same magnitude as random ones (7.8-12.3% vs 8.3-10.0%, CIs overlap). MATTR-50 is flat across all configs (0.818-0.823). The model doesn't distinguish meaningful from meaningless prefixes at the entropy level.
Output length: all passphrase types produce 37-44% fewer words than baseline (722-812 vs 1297), with colors producing the shortest outputs.
- Test whether themed passphrases produce qualitatively different stories (plot, setting, character changes) beyond word-level bleed
- Test theme combinations (e.g. metals + flowers) to see if bleed effects are additive
- Use themed passphrases in a non-creative context to see if theme bleed affects factual or structured outputs