The Problem with Generic AI Tools for Detection Authoring
If you’ve ever tried ChatGPT, Claude, or Cursor to write detections, you’ve likely experienced:1. Doesn't Compile
Detection language syntax is wrong or it uses log schemas which don’t match yours
2. False Positives
Detection technically works but generates unusable noise
3. Missing Data
Detection requires log sources you don’t ingest, so it never fires
How It Works
1
Start Detection Authoring
Begin a new detection authoring session and specify the target platform
2
Explore Environment
Cotool can use platform-specific tools to understand your environment:
- Query available indexes/data sources
- Check field names and log structure
- Review existing detection patterns
- Understand platform capabilities
3
Draft Detection
Cotool proposes an initial detection based on your requirements and platform conventions. The detection appears in a side-by-side code editor.
4
Test Query (For Applicable Platforms)
For platforms where query language = detection language (Splunk, Scanner, SentinelOne, Elastic), the Cotool can run the query to validate:
- Syntax is correct
- Query executes without errors
- Expected fields are available
5
Iterative Refinement
Collaborate with Cotool to refine the detection:
- Edit code directly in the code editor
- Select portions of code and ask for adjustments
- Request changes via chat
- Cotool updates the detection and re-renders
6
Deploy
Once satisfied, you can:
- Save to the platform directly
- Save to GitHub as code
- Copy and manually deploy
Example Workflow
Scenario: Detect Suspicious PowerShell Activity in Splunk
1
Start Session
You: “I want to create a Splunk detection for suspicious PowerShell execution”Cotool identifies platform (Splunk) and starts exploring
2
Cotool Explores
Cotool uses Splunk tools to:
- Check available indexes
- Verify Windows log sources are ingested
- Review field names (e.g.,
process_name,CommandLine)
3
Cotool Drafts
Cotool generates initial SPL query:Cotool renders this in the code editor
4
Cotool Tests
Cotool runs the query against Splunk to verify syntax
5
You Refine
You: Select the
match pattern and type “Add exclusions for known admin scripts”Cotool updates the query with exclusion logic and re-renders6
Deploy
You: “Save this as a saved search in Splunk”Cotool clarifies parameters (cron schedule, alert actions) and deploys
Key Features
Environment Awareness
Cotool doesn’t generate detections in a vacuum. It can:- Query your platforms to understand available data
- Check log ingestion and field availability
- Review existing detection patterns for consistency
- Understand platform-specific syntax and capabilities
Iterative Collaboration
Unlike one-shot generation, detection authoring is collaborative:- Edit code directly and Cotool adapts
- Select portions of code for targeted improvements
- Provide feedback and refine over multiple iterations
- Build detections incrementally
Syntax Validation
For applicable platforms, Cotool runs queries to ensure:- Code compiles and executes
- Required fields exist
- No syntax errors
Version History
The diff editor shows changes between detection versions, making it easy to:- See what changed
- Revert to previous versions
- Understand AI’s modifications
Frequently Asked Questions
Does Cotool deploy detections automatically?
Does Cotool deploy detections automatically?
No. Cotool proposes the detection and helps refine it, but you control when and how it’s deployed. You can save to the platform, save to GitHub, or copy and deploy manually.
What if I don't have the required log sources?
What if I don't have the required log sources?
If Cotool explores your environment and finds missing data, it will inform you. You can either:
- Enable the required log sources
- Adjust the detection to use available data
- Choose a different detection approach
How does Cotool know my platform conventions?
How does Cotool know my platform conventions?
Cotool has general knowledge of platform capabilities. Additionally, it can use tools to review your existing detections and learn your organization’s patterns and naming conventions.
Tips for Effective Detection Authoring
Provide Threat Context
Provide Threat Context
Attach threat reports, blog posts, or IOCs as files in the chat. This gives Cotool concrete examples of what to detect.
Use Log Samples
Use Log Samples
If you have sample logs showing the behavior, attach them. Cotool can craft detections that match those exact patterns.
Leverage Platform Tools
Leverage Platform Tools
Ask Cotool to explore your environment first. “Check what Windows logs we ingest” or “Show me existing PowerShell detections” helps ground the detection in reality.
Test Incrementally
Test Incrementally
On test-enabled platforms, have Cotool run the query after each major change. Catch issues early.
Be Specific with Exclusions
Be Specific with Exclusions
When reducing false positives, provide specific examples: “Exclude user accounts starting with ‘svc-’” rather than vague “reduce false positives.”