AWS CLI V2 "AWS firehose put-record" complaining about Invalid base64:

Question

I have used to be able to send a record to firehose without any problem like this

aws firehose put-record --delivery-stream-name my-stream --record='Data="{\"foor\":\"bar\"}"'

But since I have updated my cli to version 2 I am getting this error:

Invalid base64: "{"foor":"bar"}"

I have tried to convert {"foor":"bar"} to based64 but then I got InvalidArgumentException

aws firehose put-record --delivery-stream-name my-stream --record='Data=e1wiYXNkZlwiOlwidGVzdGFtXCJ9Ig=='


An error occurred (InvalidArgumentException) when calling the PutRecord operation: This operation is not permitted on KinesisStreamAsSource delivery stream type.

Cubital

Answer 1

In My case, I was using Kinesis via Windows Powershell CLI.

aws kinesis put-record --stream-name First-Kinesis --data "Sign In" --partition-key user222 --cli-binary-format raw-in-base64-out

Adding below cofiguration resolved my issue.

--cli-binary-format raw-in-base64-out

I hope this helps your purpose.

Dragonet

This helped for me. Thank you – Putnem

Answer 2

If you're using v2 of the AWS CLI, use this command to invoke a putRecord call to Firehose:

aws firehose put-record --delivery-stream-name STREAM-NAME --cli-binary-format raw-in-base64-out --record="{\"Data\":\"1\"}"

Firstling

Answer 3

I received the same error message:

An error occurred (InvalidArgumentException) when calling the PutRecord operation: This operation is not permitted on KinesisStreamAsSource delivery stream type.

... and none of the above solutions helped.

There is another cause of this error message: it happens when your delivery stream is configured to use a Kinesis stream as a source.

When you create your stream, there is a choice of the stream source. It can be either:

• "Amazon Kinesis Data Streams" - choosing this will give you the error,

or

• "Direct PUT" - choosing this option will let you write to your stream with the CLI.

Thurnau

Thank you. More about the difference here: #62588069 – Acanthopterygian

Answer 4

Try using "--cli-binary-format raw-in-base64-out" with your original command (the one without the base64 encoded record). AWS CLI version 2 passes binary parameters as base64-encoded strings by default.

Refer to the following link: https://docs.aws.amazon.com/cli/latest/userguide/cliv2-migration.html#cliv2-migration-binarypara

Physicist

An error occurred (InvalidArgumentException) when calling the PutRecord operation: This operation is not permitted on KinesisStreamAsSource delivery stream type. – Cubital

Answer 5

When creating a Data Firehose, the library you use on the code side depends on the source you select.

If you select Amazon Kinesis Data Stream as the source, you should use the Amazon.Kinesis library.

If you select Direct PUT as the source, you should use the Amazon.KinesisFirehose library.

If you use the wrong library, you will get an error.

Legionnaire