Bull Queue is not getting completed

Question

Am new to Bull.I have tried running bull based on their documentation code. The Process are starting but my job is not getting completed, or am not sure whether its triggering complete event or not? Am not sure where am making a mistake

Attaching my code below

const Queue = require('bull');

const myFirstQueue = new Queue('my-first-queue', 
{
  redis: { 
      port: Config.redis.port, 
      host: Config.redis.host, 
      password: Config.redis.password 
  },
});



(async function ad() {
    const job = await myFirstQueue.add({
    foo: 'bar',
  });
})();

myFirstQueue.process(async (job, data) => {
  log.debug({ job, data }, 'Job data');
  let progress = 0;
  for (let i = 0; i < 10; i++) {
    await doSomething(data);
    progress += 10;
    job.progress(progress).catch(err => {
      log.debug({ err }, 'Job progress err');
    });
    log.debug({ progress }, 'After await');
  }

  return job;
});

const doSomething = data => {
  return new Promise((resolve, reject) => {
    return resolve(data);
  });
};

myFirstQueue.on('completed', (job, result) => {
  log.debug(`Job completed with result ${job}`);
});


myFirstQueue.on('progress', (job, progress) => {
log.debug(`Job progress with result ${job} ${progress}`);
});

I can see the logs which is inside the progress event handler but complete event is not getting triggered. Any help is appreciated

Extracurricular

Answer 1

you need to call done() from process then only completed event will trigger.

myFirstQueue.process(async (job, done) => {
  const data = job.data;
  let progress = 0;
  for (let i = 0; i < 10; i++) {
    await doSomething(data);
    progress += 10;

    job.progress(progress).catch(err => {
      log.debug({ err }, 'Job progress err');
    });
  }

  done();
});

Relic

can you please answer this as well #64367047 – Impertinent

Answer 2

you can fetch data from job Object itself, no need to pass data externally, process call back needs job and data() as params. Call data() callback at last to complete a job. You can pass data and also Error if it fails some validations. Better Explanation https://github.com/OptimalBits/bull/blob/develop/REFERENCE.md#queueprocess

  // call done when finished
  done();

  // or give a error if error
  done(new Error('error transcoding'));

  // or pass it a result
  done(null, { framerate: 29.5 /* etc... */ });

You can directly use promise if you don't want to call done() call back, ref-https://github.com/OptimalBits/bull#using-promises

Did small modification to your code, Hope it helps,

const Queue = require('bull');

const myFirstQueue = new Queue('my-first-queue');



(async function ad() {
    const job = await myFirstQueue.add({
    foo: 'bar',
  });
})();

myFirstQueue.process(async (job, done) => {
  log.debug( 'Job data ' + job.data);
  let progress = 0;
   for (let i = 0; i < 10; i++) {
    await doSomething(job.data);
    progress += 10;
    job.progress(progress).catch(err => {
      log.debug({ err }, 'Job progress err');
    });
    log.debug({ progress }, 'After await');
  }

  done(null, {done: job.data}); //we need this if we are not using promise
});

const doSomething = data => {
  return new Promise((resolve, reject) => {
    return resolve(data);
  });
};

myFirstQueue.on('completed', (job, result) => {
  log.debug(`Job completed with result ${job}`);
});


myFirstQueue.on('progress', (job, progress) => {
log.debug(`Job progress with result ${job} ${progress}`);
});

Harlequinade

can you please answer this as well #64367047 – Impertinent