# Effect of AI-Style Phrasing Artifacts on Tool Calls (cactus-needle 2.0.5)

> AI-writing artifacts in queries — em-dash constructions, hedging preambles, unicode punctuation, markdown fragments, list-style phrasing, over-polite boilerplate — degrade tool calling even when the intent is perfectly clear.

Date: 2026-08-17



## Test

Follow-up to [needle-determinism-and-paraphrase-sensitivity](/scratchpads/needle-determinism-and-paraphrase-sensitivity), which found a single em-dash paraphrase drove confidence toward 0 for 3 of 15 tested intents. This experiment re-tests at scale whether AI-characteristic phrasing degrades tool calling in general, since AI-phrased queries will reach tool models through LLM-to-LLM delegation and synthetic training data.

Paired design against existing clean records from [needle-argument-complexity](/scratchpads/needle-argument-complexity). Selected 30 standard parent tasks spread across all four catalog sizes and argument-complexity tiers 1-3, requiring the parent's clean record to be tool-correct.

Authored 6 style variants per parent by hand (180 tasks total), holding intent and gold arguments constant and varying only surface phrasing:
- **em_dash**: compressed em-dash imperative/aside
- **hedge_preamble**: "Certainly! To proceed with your request, ..."
- **unicode_punct**: curly quotes, ellipsis character, non-breaking hyphen replacing plain ASCII punctuation
- **markdown**: bold/backtick/bullet fragment, often verb-less
- **list_style**: terse "- action; field: value" task-list line
- **polite_boilerplate**: over-polite wrapper ("I would greatly appreciate it if...")

Ran all 180 via the shared harness in one deterministic pass (greedy decoding, no repeats). Model: Needle 2 (cactus-needle 2.0.5), local CPU inference. Clean parents were not re-run; their existing records served as the paired control.


## Result

**CONFIRMED**

AI-writing artifacts do degrade tool calling with intent held constant, but the effect is style-specific rather than uniform across the six styles tested.

**Tool-selection accuracy per style** (n=30 pairs each):

| style | tool-correct | accuracy | tool flips | abstention flips |
|---|:-:|:-:|:-:|:-:|
| markdown | 23/30 | 0.767 | 7 | 7 |
| hedge_preamble | 27/30 | 0.900 | 3 | 1 |
| em_dash | 29/30 | 0.967 | 1 | 0 |
| polite_boilerplate | 29/30 | 0.967 | 1 | 0 |
| unicode_punct | 30/30 | 1.000 | 0 | 0 |
| list_style | 30/30 | 1.000 | 0 | 0 |

**Paired argument accuracy** (restricted to pairs tool-correct on both clean and variant sides):

| style | clean-acc | variant-acc |
|---|:-:|:-:|
| em_dash | 0.621 | 0.586 |
| hedge_preamble | 0.593 | 0.593 |
| markdown | 0.652 | 0.652 |
| polite_boilerplate | 0.621 | 0.621 |
| unicode_punct | 0.633 | **0.700** |
| list_style | 0.633 | **0.767** |

Markdown fragment phrasing is the clear outlier and the only style with a qualitatively distinct failure mode: all 7 of its flips are false abstentions on verb-less bolded/backticked fragments such as `**Set volume**: \`35\``. These are not low-confidence hedges — confidence at abstention ranged 0.20 to 0.92, including one case abstaining at confidence 0.9227 despite the gold tool and arguments being unambiguously present in the fragment. This reads as a verb-presence or imperative-mood grounding requirement rather than an uncertainty problem.

Em-dash phrasing does not reproduce the earlier confidence-collapse finding at this larger scale. The prior report found a single em-dash paraphrase driving confidence toward 0 across 3 intents; here, across 30 intents, em-dash produced the best accuracy of the four damaging styles (29/30) and a confidence shift that was small and noisy (mean -0.049, stdev 0.167, range -0.59 to +0.37) rather than a deterministic floor-to-0 effect. The original finding looks intent-specific — messaging and music intents interacting badly with compressed shorthand — rather than a general property of em-dashes.

Unicode punctuation and list-style phrasing caused zero tool-selection damage and, unexpectedly, improved paired argument accuracy over the clean baseline (0.633 to 0.700, and 0.633 to 0.767). Restructuring free text into an explicit key:value shape, or isolating quoted content with curly quotes, appears to help value extraction rather than hurt it — the opposite of what the hypothesis predicted for those two styles. Hedge_preamble and polite_boilerplate independently broke the same parent (a search_web query misrouted to find_nearby) at near-zero confidence, pointing to a parent-specific fragility that any sufficiently long wrapper exposes rather than a style-specific artifact.

The damage profile also differs in kind from character-level noise. [needle-noisy-input-robustness](/scratchpads/needle-noisy-input-robustness) found typo and STT noise leaves tool selection roughly stable while argument filling collapses sharply. Here the damaging styles hit tool selection and abstention behavior, while argument accuracy on calls that still land correctly stays at or above the clean baseline. AI-isms and character-level noise are distinct failure modes.


## Next

1. Test whether markdown's verb-less fragment failure generalizes across more parents and catalog sizes, to confirm the imperative-mood grounding requirement rather than a sample artifact.
2. Isolate std_043 (the search_web/find_nearby parent broken by both hedge_preamble and polite_boilerplate) to test whether wrapper length alone, independent of style, predicts the flip.
3. Test whether combining a helpful style (list_style or unicode_punct) with a damaging one (markdown, hedge_preamble) cancels or compounds the effect.
4. Re-run em_dash on the original 3 fragile intents from needle-determinism-and-paraphrase-sensitivity specifically, to confirm whether the earlier collapse was intent-specific rather than sample noise.