EMR activity stuck in Waiting_For_Runner state
Asked Answered
M

1

9

I am creating a data pipeline to export dynamoDB table to S3 bucket.I used the standard template to use for this in data pipeline console. I ha verified that the runsOn field is set to the name of EMR cluster to be started. However, The EMR activity status is still as Waiting_For_Runner...Any ideas why is this so??

Thanks!!!

Master answered 8/5, 2014 at 7:21 Comment(2)
solved the issue.. authorization problem :(Master
How did you fix this authorization issue?Frech
D
1

Waiting_For_Runner means datatpipeline is trying to connect to EMR.

Few reasons you can check:

  1. IAM permissions between EMR and Data pipeline(Roles). Here's a link!
  2. Check Task runner is running or not on master instance
    $ps -ef | grep workerGroup (Master Instance)
  3. Check --workgroup name in EMR(you can see in Task runner process) and compare it with the name of workgroup in datapipeline.
Demagogy answered 2/6, 2017 at 9:26 Comment(2)
What are the IAM permissions supposed to be?Tachycardia
@AidanConnelly you can refer below AWS Doc link for IAM stuff: docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/…Demagogy

© 2022 - 2024 — McMap. All rights reserved.