Operationalizing OpenSearch ISM for Wazuh A Policy‑as‑Code Approach to Alert Retention

• Unmanaged OpenSearch indices cause disk saturation in any Wazuh SOC within weeks.
• OpenSearch ISM (Index State Management) automates index transitions from hot to delete.
• Policy‑as‑Code allows version‑controlled, repeatable retention rules.
• Implementing ISM reduces manual maintenance and ensures compliance with data retention policies.

This article provides a ready‑to‑use OpenSearch ISM policy for Wazuh alerts.
You will learn how to design, deploy, and monitor index lifecycle management in production.
Real terminal outputs and Graphviz diagrams illustrate every step.

Understanding OpenSearch ISM for Wazuh Index Lifecycle

A Wazuh deployment creates one index per day by default.
Example index name: wazuh-alerts-4.x-2026.05.18.
Without automated management, these indices accumulate indefinitely.
Disk usage grows until the node crashes.

OpenSearch ISM solves this by moving indices through states and actions.
Common states are hot, warm, cold, and delete.
Transitions are triggered by conditions like min_index_age.
For a SOC, you typically only need hot and delete.
Alerts older than 30 days can be removed automatically.

The Four Core Components of an ISM Policy

A policy is a JSON document with four mandatory parts:

1. policy_id – Unique identifier for the policy.
2. description – Human‑readable purpose.
3. default_state – Starting state for every new index.
4. states – Array of named states with actions and transitions.

Each state can perform actions such as read_only, replica_count, or delete.
Transitions define when to move to another state.
For example, after 30 days in hot, transition to delete.
OpenSearch applies the policy to any index matching a pattern.

Why Wazuh Benefits from ISM More Than Other Log Sources

Wazuh alerts are high‑volume and time‑sensitive.
Recent alerts must be searchable (hot state).
Old alerts rarely need querying after 30 days.
Deleting them frees disk space and improves performance.
Unlike application logs, security alerts often have legal retention limits.
ISM lets you enforce those limits without human intervention.

Designing a Policy‑as‑Code for Alert Retention

Writing an ISM policy as code brings several advantages.
You can store it in Git, review changes, and roll back if needed.
This fits DevOps and SecOps workflows.
Below is the complete policy we use in production.
It deletes Wazuh alert indices after 30 days.

The Complete ISM Policy JSON

{
  "policy": {
    "policy_id": "wazuh-30days-retention",
    "description": "Delete Wazuh alerts after 30 days",
    "default_state": "hot",
    "states": [
      {
        "name": "hot",
        "actions": [],
        "transitions": [
          {
            "state_name": "delete",
            "conditions": {
              "min_index_age": "30d"
            }
          }
        ]
      },
      {
        "name": "delete",
        "actions": [
          {
            "delete": {}
          }
        ],
        "transitions": []
      }
    ],
    "ism_template": [
      {
        "index_patterns": ["wazuh-alerts-4.x-*"],
        "priority": 100
      }
    ]
  }
}

Explanation of each field:

• min_index_age: "30d" – The index must be at least 30 days old.
  The age is calculated from the index’s creation date.
• delete action – Permanently removes the index and its data.
  No recovery is possible after deletion.
• ism_template – Automatically applies the policy to any new index matching the pattern.
  This avoids manual assignment.

Applying the Policy to Existing Indices

New indices receive the policy automatically via ism_template.
Existing indices do not.
You must manually attach the policy to legacy indices.
Use the following API call:

curl -X POST "https://your-wazuh-indexer-ip:9200/_plugins/_ism/add/wazuh-alerts-4.x-2026.04.*" \
  -H "Content-Type: application/json" \
  -u 'admin:your-secure-password' \
  -d '{"policy_id": "wazuh-30days-retention"}'

Replace your-wazuh-indexer-ip with your OpenSearch endpoint.
Never hard‑code passwords in scripts; use environment variables or a secrets manager.
The wildcard 2026.04.* applies the policy to all April 2026 indices.

Implementing OpenSearch ISM Policy via REST API

Deploying the policy requires two steps:

1. Create the policy.
2. Attach it to indices.
   We use curl for both operations.

Step 1 – Create the ISM Policy

Save the JSON policy to a file named wazuh-retention-policy.json.
Then run:

curl -X PUT "https://your-wazuh-indexer-ip:9200/_plugins/_ism/policies/wazuh-30days-retention" \
  -H "Content-Type: application/json" \
  -u 'admin:your-secure-password' \
  -d @wazuh-retention-policy.json

A successful creation returns:

{
  "_id": "wazuh-30days-retention",
  "_version": 1,
  "_primary_term": 1,
  "_seq_no": 0
}

If you see an error like "policy already exists", update it with -X PUT (same command).
OpenSearch overwrites the existing policy.

Step 2 – Verify Policy Creation

List all policies to confirm:

curl -X GET "https://your-wazuh-indexer-ip:9200/_plugins/_ism/policies" \
  -u 'admin:your-secure-password'

Output (truncated):

{
  "wazuh-30days-retention": {
    "policy": {
      "policy_id": "wazuh-30days-retention",
      "description": "Delete Wazuh alerts after 30 days",
      ...
    }
  }
}

Step 3 – Apply Policy to New and Existing Indices

Because we included ism_template, new indices get the policy automatically.
Test by creating a dummy index:

curl -X PUT "https://your-wazuh-indexer-ip:9200/wazuh-alerts-4.x-2026.06.01" \
  -u 'admin:your-secure-password'

Then check its ISM status:

curl -X GET "https://your-wazuh-indexer-ip:9200/wazuh-alerts-4.x-2026.06.01/_plugins/_ism/explain" \
  -u 'admin:your-secure-password'

You should see "policy_id": "wazuh-30days-retention".

For existing indices (e.g., all of May 2026), use a wildcard:

curl -X POST "https://your-wazuh-indexer-ip:9200/_plugins/_ism/add/wazuh-alerts-4.x-2026.05.*" \
  -H "Content-Type: application/json" \
  -u 'admin:your-secure-password' \
  -d '{"policy_id": "wazuh-30days-retention"}'

Monitoring and Maintaining Retention Policies

After implementing ISM, you must monitor its effectiveness.
Disk usage should stabilise.
Old indices should disappear after 30 days.
Below are practical commands for daily checks.

Check Disk Usage on the OpenSearch Node

df -h /var/lib/opensearch

Expected output after ISM has been running for 30+ days:

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1       100G   62G   38G  62% /var/lib/opensearch

Used space should not grow beyond a predictable ceiling.
If usage exceeds 80%, investigate why indices are not being deleted.

List All Wazuh Indices and Their Ages

curl -X GET "https://your-wazuh-indexer-ip:9200/_cat/indices/wazuh-alerts-4.x-*?v&s=index" \
  -u 'admin:your-secure-password'

Look for indices older than 30 days.
If they still exist, the ISM transition may have failed.
Check the ISM explain API for those indices:

curl -X GET "https://your-wazuh-indexer-ip:9200/_plugins/_ism/explain/wazuh-alerts-4.x-2026.04.01" \
  -u 'admin:your-secure-password'

The response will show the current state and any errors.
Common failure: min_index_age not satisfied because index creation date is mis‑parsed.
OpenSearch uses the index’s creation_date metadata, not the name.
If you renamed indices, the age may be wrong.

Automating Daily Health Checks with Cron

Place the following script in /usr/local/bin/ism-monitor.sh on the OpenSearch node.

#!/bin/bash
LOG="/var/log/ism-monitor.log"
ALERT_THRESHOLD=80
USAGE=$(df -h /var/lib/opensearch | awk 'NR==2 {print $5}' | sed 's/%//')
if [ $USAGE -gt $ALERT_THRESHOLD ]; then
    echo "$(date) - WARNING: Disk usage ${USAGE}% exceeded threshold" >> $LOG
    # Send alert to SOC (e.g., via Wazuh API or email)
fi

# Count indices older than 30 days
THIRTY_DAYS_AGO=$(date -d "30 days ago" +%Y.%m.%d)
curl -s -k -u "admin:${OPENSEARCH_PASS}" \
  "https://localhost:9200/_cat/indices/wazuh-alerts-4.x-*?h=index" | \
while read idx; do
    idx_date=$(echo $idx | grep -oE '[0-9]{4}\.[0-9]{2}\.[0-9]{2}')
    if [[ ! -z "$idx_date" && "$idx_date" < "$THIRTY_DAYS_AGO" ]]; then
        echo "$(date) - Old index still present: $idx" >> $LOG
    fi
done

Set a daily cron job (e.g., at 3 AM) to execute this script.
Use environment variables for the password; never hard‑code.

Best Practices for Enterprise SOC Retention

Implementing ISM is straightforward, but enterprise SOCs need additional guardrails.
Below are recommendations from production deployments.

Separate Hot and Cold Storage for Cost Optimisation

The policy above deletes old indices directly.
For long‑term compliance, consider moving indices to cold storage first.
Example: after 30 days move to cold, then delete after 90 days.
This reduces storage costs and keeps data for forensic analysis.
To implement this, add a cold state with a read_only action.
Transition from hot to cold at min_index_age: "30d".
Then transition from cold to delete at min_index_age: "90d".

Use Aliases to Avoid Breaking Dashboards

Wazuh dashboards often query a specific index pattern like wazuh-alerts-4.x-*.
Deleting an index does not affect this pattern.
However, if you need a fixed alias for “last 30 days”, create a rolling alias.
Example alias wazuh-last-30d that always points to the newest 30 indices.
But this adds complexity. Most SOCs simply rely on the wildcard pattern.

Backup Critical Indices Before Implementing Deletion

Before applying any deletion policy, back up indices that may be required for audits.
Use OpenSearch snapshots to S3 or NFS.

curl -X PUT "https://localhost:9200/_snapshot/my_backup/wazuh_archive_2026_05" \
  -u 'admin:your-secure-password' \
  -H 'Content-Type: application/json' \
  -d '{
    "indices": "wazuh-alerts-4.x-2026.03.*,wazuh-alerts-4.x-2026.04.*",
    "ignore_unavailable": true,
    "include_global_state": false
  }'

Restore from snapshot if an incident requires investigating old data.

Coordinate with Legal and Compliance Teams

Many organisations have data retention policies of 90 days or more.
Adjust the min_index_age in the ISM policy accordingly.
Document the policy and its effective date.
Notify the SOC team before enabling automatic deletion.
This prevents “my alerts disappeared” surprise tickets.

Advanced FAQ

How does OpenSearch calculate index age?
Age = current time – index.creation_date (in milliseconds).
The creation date is set when the index is first created.
If you roll over an alias, the new index gets a new creation date.
Renaming an index does not change its creation date.

What happens if an index is stuck in a transition?
OpenSearch retries failed actions up to 3 times by default.
If still failing, the index stays in the current state.
Check _plugins/_ism/explain for the error message.
Common causes: insufficient disk space or missing permissions.

Can I test an ISM policy without deleting real data?
Yes. Create a test index pattern and a separate policy with a shorter min_index_age.
For example, min_index_age: "1m".
Apply it to a dummy index and wait two minutes.
The index will be deleted. This validates syntax and permissions.

Does ISM affect search performance during transitions?
No. The delete action runs as a background task.
Search queries continue normally while OpenSearch removes data.
Only the read_only action may block writes, but we do not use it here.

What is the difference between ISM and Curator?
Curator is a separate Python tool that runs outside OpenSearch.
ISM is built into OpenSearch and runs natively.
ISM is preferred for production because it does not require external cron jobs.
It also reacts in real‑time when index age conditions are met.

You have learned how to implement an OpenSearch ISM policy that automatically deletes Wazuh alert indices after 30 days.
This policy‑as‑code approach eliminates manual maintenance and prevents disk saturation.
By integrating the provided examples into your SOC pipeline, you ensure long‑term stability.
Remember to monitor disk usage weekly and adjust the retention period based on legal requirements.
A well‑maintained OpenSearch ISM policy is the foundation of any enterprise‑grade Wazuh deployment.