Skip to main content
When agent runs score poorly on automatic evaluations, Cotool analyzes the issues and suggests specific prompt improvements. You simply review the suggested changes and accept or reject them - no manual prompt writing required.

​
How It Works

1

Pattern Detection

System identifies repeated issues across multiple low-scoring runs
2

Issue Clustering

Similar problems are grouped together to identify common root causes
3

Suggested Improvements Panel

You see a panel showing detected patterns and a “Generate Diff” button
4

Generate Diff

Click the button - AI generates specific prompt changes to fix the issues
5

Review Changes

See side-by-side diff of current vs. improved prompt
6

Accept or Reject

Apply the changes or reject if they don’t fit your needs
You don’t write the improvements - The AI analyzes what went wrong and generates the fix. Your role is to review and approve, not write from scratch.

​
Suggested Improvements Panel

When available, you’ll see a panel below the system prompt with suggested improvements. Here’s an example:

​
Issue Categories

The system automatically detects and suggests fixes for:

Missing Steps

Agent skipped important investigation or validation stepsFix: Add explicit steps to prompt

Poor Decision Logic

Agent made incorrect conclusions or didn’t follow criteria properlyFix: Clarify decision criteria and edge cases

Tool Misuse

Wrong tools used, correct tools skipped, or inefficient patternsFix: Add tool usage guidance

Output Issues

Missing information, unclear formatting, or insufficient detailFix: Specify required output structure

​
Generating the Diff

When you click “Generate Diff”, the AI:
  1. Analyzes the specific issues identified
  2. Reviews your current system prompt
  3. Generates targeted changes to fix the problems
  4. Shows you a side-by-side comparison
Example Diff View:
Changes are highlighted in the diff view. Green shows additions, red shows removals, yellow shows modifications.

​
Multiple Improvements

You may see multiple suggestion cards if several low-scoring runs identified different issues:
You can:
  • Generate diffs for each individually
  • Address the lowest-scoring issue first
  • Reject suggestions that don’t apply to your use case

​
What Gets Improved

Issue: Agent skipped checking user history before closing alertFix: “Before determining severity, always: 1) Check user’s recent activity…”
Issue: Agent failed when API returned no dataFix: “If VirusTotal returns no data (404), classify as Medium and note that hash is unknown…”
Issue: Agent mishandled service account activityFix: “If user matches pattern ‘svc_’ or ‘service-’, verify activity against scheduled job list…”
Issue: Agent output was vague: “Alert looks suspicious”Fix: “Always include: 1) Specific indicators found, 2) Confidence level (High/Medium/Low), 3) Recommended actions…”

​
Best Practices

If you see the same issue across multiple runs, accept the suggestion. Repeated patterns indicate a real problem.
Use Agent Builder to test the updated prompt with edge cases before relying on it in production.
After accepting changes, watch evaluation scores for the next 10-20 runs to confirm improvement.
Not every low score needs a prompt change. Sometimes the issue is data quality, API availability, or legitimate edge cases.

​
Example: Real Improvement Flow

Run #1834 - Evaluation Score: 60/100 Issue Detected:
Generated Diff:
You: Review diff, looks good ✅ Action: Click “Accept” Result: Prompt v5 created, agent now handles empty query results properly Validation: Next 15 runs average 82/100 (up from 65/100)

​
Manual Prompt Editing

You can always edit prompts manually instead of using suggestions:
  1. Go to Agent Settings → System Prompt
  2. Click “Edit”
  3. Make changes directly
  4. Save (creates new version)
But the AI-suggested flow is faster and often catches issues you might miss.