Login/Register
Azure Data Factory CI npm validate step suddenly crashing
Asked Answered
Solved azure-devopsazure-data-factorycicdazure-pipelines-yaml
T

3

5

We've been using the "new" CICD setup for Azure Data Factory as descibed by the documentation page: https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery-improvements. We're doing this using Azure DevOps pipelines written in YAML.

This has been working quite well for us, until it suddenly started crashing at the validation step this morning. No code has been changed on our side. I'm wondering if there's some issue with version number or incompatability but I can't find anything.

The Azure YAML code in question is this:

  - task: Npm@1
    displayName: 'Validate ADF json'
    inputs:
      command: 'custom'
      workingDir: $(packageJsonFolder)
      customCommand: 'run build validate $(Build.Repository.LocalPath)/DataFactories/${{ parameters.folderName }} $(adfAzureResourceId)'

The full log of error messages is as following:

node  /home/vsts/work/1/s/DataFactories/downloads/main.js validate /home/vsts/work/1/s/DataFactories/internaldataload /subscriptions/xxxxxxxxxxxxxxxx/resourceGroups/xxxxxxxxxxxxxx/providers/Microsoft.DataFactory/factories/xxxxx

/home/vsts/work/1/s/DataFactories/node_modules/@microsoft/azure-data-factory-utilities/lib/bundle.manager.js:53
                    throw "Execution failed with exit code: " + exitCode;
                    ^
Execution failed with exit code: 1
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @ build: `node node_modules/@microsoft/azure-data-factory-utilities/lib/index "validate" "/home/vsts/work/1/s/DataFactories/internaldataload" "/subscriptions/xxxxxxxxxxxxxxxx/resourceGroups/xxxxxxxxxxxxxx/providers/Microsoft.DataFactory/factories/xxxxx"`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the @ build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/vsts/.npm/_logs/2022-06-16T09_04_15_237Z-debug.log




=====ERROR=====
Error: Command failed: node  /home/vsts/work/1/s/DataFactories/downloads/main.js validate /home/vsts/work/1/s/DataFactories/internaldataload /subscriptions/xxxxxxxxxxxxxxxx/resourceGroups/xxxxxxxxxxxxxx/providers/Microsoft.DataFactory/factories/xxxxx
/home/vsts/work/1/s/DataFactories/downloads/main.js:2
(()=>{var e={82257:(e,t,r)=>{e.exports=r(53605)


    at Object.94616 (/home/vsts/work/1/s/DataFactories/downloads/main.js:2:4897020)
    at r (/home/vsts/work/1/s/DataFactories/downloads/main.js:2:7274976)
    at /home/vsts/work/1/s/DataFactories/downloads/main.js:2:7275513
    at /home/vsts/work/1/s/DataFactories/downloads/main.js:2:7953499
    at Object.<anonymous> (/home/vsts/work/1/s/DataFactories/downloads/main.js:2:7953503)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)

Execution finished....
Found npm debug log, make sure the path matches with the one in npm's output: /home/vsts/.npm/_logs/2022-06-16T09_04_15_237Z-debug.log
0 info it worked if it ends with ok
1 verbose cli [ '/opt/hostedtoolcache/node/10.24.1/x64/bin/node',
1 verbose cli   '/opt/hostedtoolcache/node/10.24.1/x64/bin/npm',
1 verbose cli   'run',
1 verbose cli   'build',
1 verbose cli   'validate',
1 verbose cli   '/home/vsts/work/1/s/DataFactories/internaldataload',
1 verbose cli   '/subscriptions/xxxxxxxxxxxxxxxx/resourceGroups/xxxxxxxxxxxxxx/providers/Microsoft.DataFactory/factories/xxxxx' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prebuild', 'build', 'postbuild' ]
5 info lifecycle @~prebuild: @
6 info lifecycle @~build: @
7 verbose lifecycle @~build: unsafe-perm in lifecycle true
8 verbose lifecycle @~build: PATH: /opt/hostedtoolcache/node/10.24.1/x64/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/home/vsts/work/1/s/DataFactories/node_modules/.bin:/opt/hostedtoolcache/node/10.24.1/x64/bin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/home/vsts/.local/bin:/opt/pipx_bin:/home/vsts/.cargo/bin:/home/vsts/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/vsts/.dotnet/tools:/snap/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
9 verbose lifecycle @~build: CWD: /home/vsts/work/1/s/DataFactories
10 silly lifecycle @~build: Args: [ '-c',
10 silly lifecycle   'node node_modules/@microsoft/azure-data-factory-utilities/lib/index "validate" "/home/vsts/work/1/s/DataFactories/internaldataload" "/subscriptions/xxxxxxxxxxxxxxxx/resourceGroups/xxxxxxxxxxxxxx/providers/Microsoft.DataFactory/factories/xxxxx"' ]
11 silly lifecycle @~build: Returned: code: 1  signal: null
12 info lifecycle @~build: Failed to exec build script
13 verbose stack Error: @ build: `node node_modules/@microsoft/azure-data-factory-utilities/lib/index "validate" "/home/vsts/work/1/s/DataFactories/internaldataload" "/subscriptions/xxxxxxxxxxxxxxxx/resourceGroups/xxxxxxxxxxxxxx/providers/Microsoft.DataFactory/factories/xxxxx"`
13 verbose stack Exit status 1
13 verbose stack     at EventEmitter.<anonymous> (/opt/hostedtoolcache/node/10.24.1/x64/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:332:16)
13 verbose stack     at EventEmitter.emit (events.js:198:13)
13 verbose stack     at ChildProcess.<anonymous> (/opt/hostedtoolcache/node/10.24.1/x64/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack     at ChildProcess.emit (events.js:198:13)
13 verbose stack     at maybeClose (internal/child_process.js:982:16)
13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
14 verbose pkgid @
15 verbose cwd /home/vsts/work/1/s/DataFactories
16 verbose Linux 5.13.0-1029-azure
17 verbose argv "/opt/hostedtoolcache/node/10.24.1/x64/bin/node" "/opt/hostedtoolcache/node/10.24.1/x64/bin/npm" "run" "build" "validate" "/home/vsts/work/1/s/DataFactories/internaldataload" "/subscriptions/xxxxxxxxxxxxxxxx/resourceGroups/xxxxxxxxxxxxxx/providers/Microsoft.DataFactory/factories/xxxxx"
18 verbose node v10.24.1
19 verbose npm  v6.14.12
20 error code ELIFECYCLE
21 error errno 1
22 error @ build: `node node_modules/@microsoft/azure-data-factory-utilities/lib/index "validate" "/home/vsts/work/1/s/DataFactories/internaldataload" "/subscriptions/xxxxxxxxxxxxxxxx/resourceGroups/xxxxxxxxxxxxxx/providers/Microsoft.DataFactory/factories/xxxxx"`
22 error Exit status 1
23 error Failed at the @ build script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]

##[error]Error: Npm failed with return code: 1
Finishing: Validate ADF json

Our package.json file is exactly as from the documentation page.

{
    "scripts":{
        "build":"node node_modules/@microsoft/azure-data-factory-utilities/lib/index"
    },
    "dependencies":{
        "@microsoft/azure-data-factory-utilities":"^0.1.5"
    }
}

Any ideas what we can do?

Tufted answered 16/6, 2022 at 12:16 Comment(0)
H
7

Fixed after updating the 'Install Node.js' task to match the reference from doc:

# Installs Node and the npm packages saved in your package.json file in the build

- task: NodeTool@0
  inputs:
    versionSpec: '14.x'
  displayName: 'Install Node.js'
Handbook answered 16/6, 2022 at 16:15 Comment(3)
Aha, I see they updated it. Thanks! – Tufted
Thank you; this has made my Friday afternoon better :) – Marjie
Thank you, the same problem and this solutions work fine. I have versionSpec older. Its works to 21/09. – Subject
S
9

If you are reading this in 2024, the solution is to update node.js to 18

# Installs Node and the npm packages saved in your package.json file in the build

- task: NodeTool@0
  inputs:
    versionSpec: '18.x'
  displayName: 'Install Node.js'
Sunshinesunspot answered 13/6 at 8:12 Comment(3)
Yes, my Build Pipelines started failing this afternoon as I was using 14.x. Updated to 18.x and the issue was resolved. Thank you. – Panchromatic
Reference from Microsoft here. – Panchromatic
Upgrading to 18.x fixed it for me as well - thank you! – Piragua
H
7

Fixed after updating the 'Install Node.js' task to match the reference from doc:

# Installs Node and the npm packages saved in your package.json file in the build

- task: NodeTool@0
  inputs:
    versionSpec: '14.x'
  displayName: 'Install Node.js'
Handbook answered 16/6, 2022 at 16:15 Comment(3)
Aha, I see they updated it. Thanks! – Tufted
Thank you; this has made my Friday afternoon better :) – Marjie
Thank you, the same problem and this solutions work fine. I have versionSpec older. Its works to 21/09. – Subject
L
0

After changing to Node 18 it is resolved.

Labiche answered 2/7 at 8:55 Comment(1)
Please don't post "thank you" as an answer. – Debar

© 2022 - 2024 — McMap. All rights reserved.