python apscheduler - skipped: maximum number of running instances reached

Question

I am executing a function every second using Python apscheduler (version 3.0.1)

code:

scheduler = BackgroundScheduler()
scheduler.add_job(runsync, 'interval', seconds=1)
scheduler.start()

It's working fine most of the time but sometimes I get this warning:

WARNING:apscheduler.scheduler:Execution of job "runsync (trigger: interval[0:00:01], next run at: 2015-12-01 11:50:42 UTC)" skipped: maximum number of running instances reached (1)

1.Is this the correct way to execute this method?

2.What does this warning mean? Does it affect the execution of tasks inside the function in anyway?

3.how to handle this?

Domestic

Answer 1

It means that the task is taking longer than one second and by default only one concurrent execution is allowed for a given job. I cannot tell you how to handle this without knowing what the task is about.

Milagrosmilam

Hi Alex, The Task check for a folder inside a FTP location and if the folder is found it starts fetching data from those folders which can take upto 24 hours. – Domestic

Since this process takes longer than a second, you are going to be seeing those warnings a lot. You don't need to do anything about it though. – Aberdare

Answer 2

Increase max_instances

If the particular use case permits it, simply increase max_instances as shown below.

import apscheduler.schedulers.background

scheduler = apscheduler.schedulers.background.BackgroundScheduler({'apscheduler.job_defaults.max_instances': 2})

There are three apscheduler configuring styles. These are described in the documentation. Above is the dictionary/JSON style, here is the argument style:

import apscheduler.schedulers.background

scheduler = apscheduler.schedulers.background.BackgroundScheduler(job_defaults={'max_instances': 2})

Marcellemarcellina

Answer 3

If you want concurrently running instances of the same job and avoid the warning, you can include the max_instances argument in the scheduler's add_job() method. The default value is one.

Yale

Answer 4

I am pretty sure my task is not taking more than the interval. I just followed this answer instead and switched to apscheduler==2.1.2 as suggested here.

Lastminute

Answer 5

You can increase the maximum number of allowed running instances when adding the job using the max_instances parameter:

scheduler.add_job(runsync, 'interval', seconds=1, max_instances=2)

Cramped

Answer 6

It's possible you are running flask in debug mode, which starts two instances of the app at once; the second is hitting the max_instances error you see.

Ciera