Base64 encoding is adding a new line

Question

I'm trying to encode a database string using base64 on the command line in linux.

Once I do I add the value to a secret in kubernetes but my application is failing to connect to the database due to the db string not being accepted. There seems to be a newline getting added when I check the value in lens and this is not there in a similar cluster in the same secret

jdbc:postgresql://test.xxxxxxxx.eu-west-2.rds.amazonaws.com/test

deirdre$ echo jdbc:postgresql://test.xxxxxxxx.eu-west-2.rds.amazonaws.com/test | base64 | tr -d "\n"
amRiYzpwb3N0Z3Jlc3FsOi8vdGVzdC54eHh4eHh4eC5ldS13ZXN0LTIucmRzLmFtYXpvbmF3cy5jb20vdGVzdAo=

Is there something I am doing wrong? or is there an issue with the /?

Polycrates

Answer 1

You can fix those easy with

echo -n "string" | base64

"echo -n" removes the trailing newline character.

You can also see my last answer i gave to following Question Kubernetes secrets as environment variable add space character

Hellenist

Answer 2

the problem is that base64 adds the newline in order to be compatible with older systems that have a maximum line width. you can add the -w 0 option to the base64 command to change the behavior so that it no longer adds new lines.

in your example this would be

echo "jdbc:postgresql://test.xxxxxxxx.eu-west-2.rds.amazonaws.com/test" | base64 -w 0

which results in

amRiYzpwb3N0Z3Jlc3FsOi8vdGVzdC54eHh4eHh4eC5ldS13ZXN0LTIucmRzLmFtYXpvbmF3cy5jb20vdGVzdAo=

edit:

printf "%s" jdbc:postgresql://test.xxxxxxxx.eu-west-2.rds.amazonaws.com/test | base64 -w 0 

produces the correct output which adds an additional newline in the base64 encoded string which is apparently required for the url to be recognized as properly ended

Glendoraglendower

Hi there, thanks for the reply. I tried that actually but it produces the same encoded value, see how your output is the same as mine above? its really odd – Polycrates

hi. i am sorry, i seem to have missread the question. because base64 also adds additional '\n' characters every 76 characters. can you try using the output of: "printf "%s" jdbc:postgresql://test.xxxxxxxx.eu-west-2.rds.amazonaws.com/test | base64 -w 0" if that doesn't work I am currently out of other ideas – Glendoraglendower

Answer 3

So please use echo -n to remove the line break before redirecting to base64; and use base64 -w 0 to prevent base64 itself to add line break into the output.

me@host:~ 
$ echo -n mypassword | base64 -w 0
bXlwYXNzd29yZA==me@host:~ # <<<<<<<<<<<<<<< notice that no line break added after "==" due to '-w 0'; so me@host is on the same line
$ echo -n 'mypassword' | base64 -w 0
bXlwYXNzd29yZA==me@host:~ # <<<<<<<<<<<<<<<<<< notice adding single quotes does not affect output, so you can use values containing spaces freely

A good way to verify is using od -c to show the actual chars.

me@host:~ 
$ echo -n bXlwYXNzd29yZA== | base64 -d | od -c
0000000   m   y   p   a   s   s   w   o   r   d
0000012

You see no "\n" is added. But if you use "echo" without "-n", od will show "\n":

me@host:~ 
$ echo mypassword | base64 -w 0
bXlwYXNzd29yZAo=me@host:~ 
$ echo bXlwYXNzd29yZAo= | base64 -d | od -c
0000000   m   y   p   a   s   s   w   o   r   d  \n
0000013

---

At last, create a function will help you in the future:

base64-encode() {
    if [ -z "$@" ]; then
        echo "Encode string with base64; echoing without line break, and base64 does not print line break neither, to not introducing extra chars while redirecting. Provide the string to encode. "
        return 1
    fi
    echo -n "$@" | base64 -w 0  # here I suppose string if containing space is already quoted
}

Leuctra

base64 doesn't seem to have a -w option on my system, what does it stand for (whitespace)? – Asymptomatic

I am using Ubuntu and base64 (GNU coreutils) 8.28. It stands for -w, --wrap=COLS wrap encoded lines after COLS character (default 76). Use 0 to disable line wrapping – Leuctra

ok. this isn't an option on the macOS version I have. thanks! – Asymptomatic

Out of curiosity, in that case do you have similar option/switch? Or you need to instead install GNU base64? – Leuctra

I think probably the latter, but I haven't tried it. – Asymptomatic

macOS by default is using -w0 so I think you are fine without. Search here in stackoverflow and you will see it – Leuctra

Huh, I was searching here in stackoverflow, but anyway thanks – Asymptomatic

here from Google. Search here does not work well. – Leuctra

thanks, somehow google was unable to turn that up for me in a general search (which is how I ended up on this page) – Asymptomatic

Answer 4

Another way to encode the string is using cat instead of echo, which makes your string be recorded in your bash history ($HOME/.bash_history).

With this command your string will not be recorded in your history, which may be useful if your string has credentials.

cat | tr -d "\n" | base64

Type your string, then press ENTER, then press CTRL+D.

Example:

user@computer$ cat | tr -d "\n" | base64
myuser:mypassword
bXl1c2VyOm15cGFzc3dvcmQ=

Note that tr -d "\n" is removing the new line (LF) character.

Meandrous

Answer 5

In my case -n didn't work when written in shell script file while working when calling the same thing from Terminal on macOS.

The way I worked it around was truncating like so:

echo "username:password" | tr -d '\n' | base64

Note: It also worked by calling using zsh or bash, e.g., zsh ./my_script.sh or bash ./my_script.sh however that option was not useful for me.

Kibitka