Monitor not sending notification on missing data datadog

Question

facing some issue with notification when data is missing. We receive metrics every 1 hour. We have set a monitor to notify if no metrics received for 70 minutes. But even if we fail to send the metrics, no notification is sent to us after 70 minutes. Why might this be happening?

{
    "id": "<ID>",
    "name": "Test Monitor",
    "type": "metric alert",
    "query": "sum(last_1m):count:jobs.operation_time{*} by {job_type}.as_count() > 20  (this is an impossible condition)",
    "message": "{{#is_no_data}}\nNo data received\n{{/is_no_data}}@[email protected]",
    "tags": [
        "tag1"
    ],
    "options": {
        "notify_audit": false,
        "locked": false,
        "timeout_h": 0,
        "silenced": {},
        "include_tags": true,
        "no_data_timeframe": 70,
        "require_full_window": true,
        "new_host_delay": 300,
        "notify_no_data": true,
        "renotify_interval": 0,
        "escalation_message": "",
        "thresholds": {
            "critical": 20
        }
    },
    "priority": 1,
    "classification": "metric"
}

Could you try set require_full_window at false? On the UI, this parameter is recommended when the metric is sparsed. — XYZ123
– XYZ123, Commented Aug 9, 2021 at 8:13
The primary reason is that the monitor is never going into no data state even if metric is not getting received for hours. I don't understand why it remains in ok state. — Vibhav
– Vibhav, Commented Aug 10, 2021 at 12:53

Vibhav · Accepted Answer · 2021-08-12 15:21:04Z

0

Turns out that it was an issue from datadog side. Its been fixed now. https://status.datadoghq.com/incidents/jymbdkfx0y0h

answered Aug 12, 2021 at 15:21

Vibhav

2012 gold badges6 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Monitor not sending notification on missing data datadog

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related