AWS CodeBuild - EnvironmentVariables specified in action config does not match expected format

Question

I'm trying to add an environment variable to a Pipeline action that uses AWS Codebuild. However, no matter what I add, if I choose a type of Secret Manager, the step fails with the following error:

Invalid action configuration

EnvironmentVariables specified in action config does not match expected format, the expected format is JSON array adhering to the following format: [{"name": "string", "type": "string", "value": "string"}]

This is what I'm entering in the UI:

And the JSON that CodePipeline is generating looks like this:

[{"name":"SERVICE_CREDS","value":"my-secret:service_creds","type":"SECRETS_MANAGER"}]

What is going on here?? I don't know what I could possibly be getting wrong on my end. I'm entering text into the boxes they provide. The JSON that Pipelines produces from they input boxes looks valid to my eye. So, I have no idea why it is saying that the environment variables aren't matching the expected format!

Broadcasting

"Value" cannot have a ":" in it as you are putting there. It will result in invalid json under the hood. – Figurehead

I'm not sure that that is the issue. The JSON produced is perfectly valid (this can be verified with jsonformatter.curiousconcept.com). Additionally, the : are how you specify the Secret Manager path, so they cannot be dropped. – Broadcasting

I have the same issue, did you found a solution? – Pearle

Answer 1

If anyone comes to this page after searching for the error:

EnvironmentVariables specified in action config does not match expected format, the expected format is JSON array adhering to the following format

This is a recurring issue when your have a CodePipeline which feeds an environment variable '#{SourceVariables.CommitMessage}' from Source action to CodeBuild action and if the CommitMessage contains a quote or is multi line, then the action will fail due to internal json parser failure.

Note: CodeCommit always adds a '\n' so this issue will always occur with CodeCommit. For GitHub, it will only occur if you use the extended commit message.

For now to workaround this issue without loosing the 'COMMIT_MESSAGE' environment variable, please follow these steps:

Workaround:

• Remove the 'COMMIT_MESSAGE' Environment Variable from CodePipeline configuration on the CodeBuild action.

• Make sure your CodeBuild project's service role has permission to do 'ListPipelineExecutions' on the Pipeline.

• Add the following in Buildspec 'Install' phase to install 'jq' utility [1]:

  - apt-get install jq
  

• Add the following in Buildspec where you need to get the commit message (please update to the name of the pipeline):

  - COMMIT_MESSAGE=$(aws codepipeline list-pipeline-executions  --pipeline-name <Pipeline_Name> --max-items 1 | jq -r '.pipelineExecutionSummaries[0].sourceRevisions[0].revisionSummary')
  - export COMMIT_MESSAGE
  - echo $COMMIT_MESSAGE # debug command only
  - printenv # debug command only
  

Using this method, we are using the 'list-pipeline-executions' [2] AWS CLI call to retrieve the recent pipeline execution and parse the commit message from this execution. The 'COMMIT_MESSAGE' variable will include the complete commit message with any quotes or newlines.

References:

[1] jq - https://stedolan.github.io/jq/

[2] list-pipeline-executions - https://docs.aws.amazon.com/cli/latest/reference/codepipeline/list-pipeline-executions.html

Figurehead

Answer 2

In continuation with the above answer, instead of using/installing jq in buildspec.yml, we can also use the AWS inbuilt query flag/method as shown below.

- COMMIT_MESSAGE=$(aws codepipeline list-pipeline-executions --pipeline-name <Pipeline_Name> --max-items 1 --query 'pipelineExecutionSummaries[0].sourceRevisions[0].revisionSummary')
- echo COMMIT_MESSAGE $COMMIT_MESSAGE # debug command only

Ref - https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-filter.html

Mandrake

When using this approach with a GitHub Enterprise connection I ran into the issue that the 'revisionSummary' was actually a piece of json: "{\"ProviderType\":\"GitHubEnterpriseServer\",\"CommitMessage\":\"<actual message>\"}". So I still needed jq to get the message – Kulak

yeah that hit me too, this is what I used in the end: aws codepipeline list-pipeline-executions --pipeline-name <Pipeline_Name> --max-items 1 --query 'pipelineExecutionSummaries[0].sourceRevisions[0].revisionSummary' | jq -r '. | fromjson | .CommitMessage' – Pears