Add instruction salience analyzer workflow and integrate guidance into workflow creation

[Copilot]

Agents skip low-salience instructions due to current model limitations. This workflow identifies at-risk instructions and provides workarounds to improve compliance. Additionally integrates instruction salience guidance into workflow creation and update documentation.

Important Context

Ideal State: All instructions should be followed equally without special formatting or positioning.

Current Reality: This tool addresses current AI model limitations that cause variable instruction compliance. The workarounds provided are temporary measures that should become less necessary as models improve.

Implementation

Salience scoring formula:

Combined Score = (Position × 0.4) + (Emphasis × 0.3) + (Semantic × 0.3)

This formula estimates the risk that an instruction will be skipped based on observed model behavior patterns, not how important the instruction should be.

• Position (40%): End-of-prompt instructions have 4× higher observed compliance than beginning
• Emphasis (30%): Emoji markers (+2), bold/caps (+1), XML tags (+2), repetition (+2)
• Semantic (30%): Runtime context (+5), workflow body (+6), imports (+3), tools (+3)

Five-tier classification (observed compliance rates):

• Tier 1 (9-10): >95% compliance - usually followed
• Tier 2 (7-9): 80-95% compliance - acceptable
• Tier 3 (5-7): 60-80% compliance - consider workarounds
• Tier 4 (3-5): 30-60% compliance - workarounds recommended
• Tier 5 (0-3): <30% compliance - workarounds required

Analysis Output

For each instruction, reports:

• Composite score with position/emphasis/semantic breakdown
• Tier classification with observed compliance rate
• Before/after examples showing workarounds for at-risk instructions
• Prioritized workarounds (position, emphasis, gating, templates, consistency)

Example finding:

**Current** (Line 45, Score: 2.8/10):
Before making changes, check user permissions.

**Workaround** (Projected: 8.8/10):
## ⚠️ SECURITY VALIDATION - DO NOT PROCEED

🚨 **VERIFY USER PERMISSIONS** before ANY modifications:
- ✅ Confirm write/admin access
- ❌ ABORT if validation fails

Workflow Creation Integration

Added instruction salience guidance to workflow creation and update documentation:

• .github/aw/create-agentic-workflow.md: Added "Writing Effective Agent Instructions" section under Best Practices, emphasizing ideal state and providing practical workarounds
• .github/agents/agentic-workflows.agent.md: Added instruction analysis to Key Features and Important Notes
• .github/aw/update-agentic-workflow.md: Added guidelines for prompt editing with instruction salience considerations

All documentation consistently emphasizes the philosophy: ideally all instructions should be followed equally, with workarounds provided only for current model limitations.

Files Added/Modified

• .github/workflows/instruction-salience-analyzer.md - Workflow implementation (15KB)
• .github/workflows/instruction-salience-analyzer.lock.yml - Compiled workflow (52KB)
• docs/src/content/docs/guides/instruction-salience.md - User guide emphasizing ideal state vs. current limitations, with workaround techniques (13KB)
• .github/aw/create-agentic-workflow.md - Added instruction salience guidance to Best Practices
• .github/agents/agentic-workflows.agent.md - Added instruction analysis feature and notes
• .github/aw/update-agentic-workflow.md - Added prompt editing guidelines with salience considerations

Triggers

• Issues: Title contains "[Salience Analysis]"
• Manual dispatch: workflow_path input parameter

Formula weights derived from empirical gh-aw agent behavior analysis. The workflow helps identify where current model limitations may cause instructions to be skipped and provides temporary workarounds until model behavior improves.

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[mnkiefer]

@copilot Ideally, we always want the agent to consider ALL instructions equally and completely, unless the user specifically prioritizes, etc.

[mnkiefer]

@copilot Ideally, we always want the agent to consider ALL instructions equally and completely, unless the user specifically prioritizes, etc.

[Copilot]

@copilot Ideally, we always want the agent to consider ALL instructions equally and completely, unless the user specifically prioritizes, etc.

Agreed! I've reframed the entire workflow and documentation to emphasize that:

1. Ideal state: ALL instructions should be followed equally (no special positioning/formatting needed)
2. Current reality: Models have limitations causing variable compliance
3. Tool purpose: Identify at-risk instructions due to current model limitations and provide temporary workarounds

Changed terminology throughout from "optimizing salience" to "working around limitations" and added reminders that these workarounds should become less necessary as models improve.

Commit: b4f1934

[mnkiefer]

@copilot Don't just focus on the workflows markdown part, but also the full instructions the agent gets with it embedded (i.e. Workflow creation guide .github/aw/create-agentic-workflows.md and Agentic Workflows agent .github/agents/agentic-workflows.agent.md.

[Copilot]

@copilot Don't just focus on the workflows markdown part, but also the full instructions the agent gets with it embedded (i.e. Workflow creation guide `.github/aw/create-agentic-work...

Added instruction salience guidance to the workflow creation/update documentation and agent files:

Added to .github/aw/create-agentic-workflow.md:

• New "Writing Effective Agent Instructions" section in Best Practices
• Emphasizes ideal state (all instructions followed equally)
• Provides practical workarounds for current model limitations
• Links to Instruction Salience Analyzer and guide

Added to .github/agents/agentic-workflows.agent.md:

• Listed Instruction Analysis as a key feature
• Added note in Important Notes about writing workflow prompts with link to guide

Added to .github/aw/update-agentic-workflow.md:

• Added guideline about prompt editing and instruction salience
• Links to Instruction Salience guide for workarounds

All documentation now consistently emphasizes the philosophy: ideally all instructions should be followed equally, with workarounds provided only for current model limitations.

Commit: 141eae1