how to reset "alerting" state in Grafana

Question

I have set up a Grafana alert, that alerts when my values in a graph reaches a specific value.

And that value was reached a month ago and ever since then the state has been "ALERTING".

I obviously want to reset this state to "OK", since the incident happened a month ago and has been resolved.

Any idea how to do that?

I've taken some screenshots of the graph/panel and the settings for the alert, so you can see, what I've set it to.

The main goal is to get a alert everytime the values reach over 468.

Barbershop

Answer 1

Well it is obvious and I guess you just get better at Grafana when trying...

I've found out that you should simple go to the page "Alert Rules" and from there pause and then restart the alert.

When you re-enable it it will change status from "alerting" to "pending", the alert will then check after some time (check what time you've set it too) and if the query no longer calls the alert, it will change to "OK", if not - it will again send out a new alert.

Grafana should normally do this itself - but I've experienced that I've had to manually do this.

Should you not have this possibility - meaning that the query should not call the alert, but should show "OK" and it instead keeps showing "Alerting", then I couldn't find another way around than to delete the alert and set it up again. Not the best solution, but pausing and re-enabling didn't work and I haven't had any difficulties since.

If anyone have had the same experience and found a better solution, then please feel free to post it.

I can also see that this has been a bug/known error as stated here

Barbershop

2021: Wow, two years later and this bug is still annoying – Unpolled

Unfortunately this seems to have changed, at least in Grafana 9. There's no pause button on the alert rules screen, so you'll just have to deal with getting alerts forever, at least until you delete them from your config files (e.g. loki-alerting-rules.yaml or whatever). – Roue

Answer 2

In the newer versions of grafana, you can configure custom recovery threshold. Simply add a threshold expression and set your required values and it should work. Attached screenshot for reference

Tetraploid