We are running a large number of DAGS, and as a result, thousands of task instances for each calendar date. We have sporadic days that have failed and we are attempting to re-run those dates as efficiently as possible.
The method we attempted to use is using the UI's search feature to narrow down the full list of individual dates that failed, and clicking 'delete' (which, to my understanding, was equivalent to 'clearing' a task instance), shown in this screenshot.
We are using catchup = True in order to automatically re-run cleared dates, but after checking individual DAGs that should be re-running those deleted dates, the dates no longer appear at all. Not even in a 'cleared' or 'running' state. The dates just aren't present. For example, we deleted 3/30/2020 and 3/31/2020, so in the tree view, the dates jump from 3/29/2020 to 4/01/2020.
Is there an additional setting or anything to keep in mind when deleting/clearing instances using the UI, from the search results page in particular?