SocGholish Malware Attack - UPDATED 08/03/24 – Haley Marketing

UPDATE - 08/03/24
A postmortem review was held with WP Engine on Monday, 7/29/24. WP Engine is Haley Marketing's current primary hosting partner. The results of that meeting were unsatisfactory. Here is what we know:

Only sites hosted at WP Engine were impacted. We host several hundred sites elsewhere; none of which were infected. We support many clients who self-host; none of these were impacted.
Sites hosted at WP Engine are isolated, and the servers on which they are hosted are also isolated. There should be no mechanism for the spread of malware from site to site, yet it appears this happened. We do not have an explanation from WP Engine as to how this could have occurred.
Several sites had advanced security and firewalling enabled through Sucuri. These sites were also impacted.
Our forensic testing did not provide any explanation of how this malware was able to infect the sites on WP Engines' servers.
WP Engine did not support us in a timely manner, did not notify us of the issue, did not help eliminate the issue, and the only solution they offered was to install Wordfence to scan websites. We were told the process of installing Wordfence and scanning websites would require 10+ days.
To expedite a solution for our clients, our DevOps team developed a script to programmatically update every website we host at WP Engine to eliminate the malware. Within 24 hours, the script completed running, and the malware had been remediated from these sites. After we were done, WP Engine asked how we resolved the issue so quickly and requested a copy of our proprietary script.

As your hosting company, our job is to provide you with as reliable and secure an environment for your website as we can. That’s why we partnered with WP Engine, a top-tier hosting company for WordPress websites, and why we have a Client Success Team available to provide support when issues arise.

Unfortunately, as we have just experienced, there is no foolproof way to defend against bad actors. And when things go wrong, that’s what you are really paying for—to have us mitigate issues with your website as quickly as possible. This is exactly what our team did for you.

In the process of resolving and mitigating future malware infections, we have:

Installed Wordfence on every site we host and have it running scheduled scans to notify us of any issues found across all sites.
Developed a script that allows us to automatically and expediently make WordPress Core and plugin updates or as needed.
Cleaned every site that we host on WP Engine by reinstalling WordPress Core and every plugin.
Additionally, we cleaned the database, removing all spam comments and entries to further safeguard your team from potential risks.
Consulted with WP Engine and Sucuri regarding malware removal and residual impact on email.
Double-verified that all sites reported as potentially having residual issues were, in fact, malware-free. These sites were tested using an enhanced security scan via Sucuri.
Updated and/or added security headers to all hosted sites, including all WordPress sites, Career Portal, HaleyMail, and Talent Showcase sites.

On July 15, after all the malware had been cleaned, several clients reported ongoing issues with email delivery to and from their clients. Research into this issue led us to discover that a cybersecurity company called Proofpoint was listing our web server IP addresses and our clients' domains as at risk for malware.

While Haley Marketing has no relationship with Proofpoint (and neither do any of our clients), and email delivery is completely independent of website hosting, we continue to work with our clients to help get this issue addressed. We took the following steps:

Contacted Proofpoint multiple times through their whitelisting request form, email, contact form, and by calling their support team.
Consulted with our clients to contact their clients and provide instructions for getting their domain whitelisted on Proofpoint.

As we move forward, we are further mitigating the risk of future malware infections by evaluating alternative website hosting providers. The hosts we are looking at include premium-level firewalls (Cloudflare Enterprise WAF) and better security and vulnerability testing (Immunify360.)

If a new host is chosen, we will contact you to move your site. Please understand that this will require a DNS change, which we will assist with or provide you instructions on how to do it yourself. Please watch for this email.

UPDATE - 07/19/24
Tested and verified all Haley Marketing server IP addresses using Proofpoint's Dynamic Reputation IP lookup. All IPs were reported as "not blocked" by ProofPoint.

UPDATE - 07/18/24
Email issues persist for some clients. The suggested resolution is to contact the business using Proofpoint for security and request that they whitelist your domain in the Proofpoint system. Additionally, having a backup domain from which emails are sent is an option.

Please note that Haley Marketing has no affiliation or relationship with Proofpoint, and although we have attempted to content them numerous times, it has not yielded results.

UPDATE - 07/16/24
Some clients report issues with email deliverability sent to highly sensitive businesses like healthcare, financial, or government institutions. These businesses use a 3rd party security service provided by Proofpoint. Proofpoint appears to have marked domains as infected and is unresponsive to requests for recan and delisting. A suggested workaround is to remove any links to the website URL from email signatures.

UPDATE - 07/15/24
All sites hosted by Haley Marketing have been scanned and are clear of malware.

What Happened:
SocGholish, also known as FakeUpdates, is a sophisticated and prevalent malware campaign designed to exploit vulnerabilities in websites to compromise their security. Unfortunately, some of our hosted websites have been affected by this malware. This is not an isolated issue but part of a larger, global attack targeting various web hosting services and websites.

Our Response:
Please rest assured that resolving this security issue is our top priority. Our team, in collaboration with our hosting and security partners, is working around the clock to ensure that all affected websites are thoroughly cleaned and secured. Here are the steps we are taking:

Malware Removal: Identifying and removing the Socgholish malware from all affected websites.
Security Measures: Implementing additional security protocols to prevent future infections.
Ongoing Audits: Conducting comprehensive security audits on all hosted websites to maintain their safety and reliability.

Learn More:
For more detailed information about the Socgholish malware and the steps being taken to combat it, please refer to this detailed post by one of our security partners, Sucuri: Socgholish Malware Information.

Your Support:
While we are managing the technical aspects, your vigilance can also contribute to maintaining security. We recommend:

Regularly updating your passwords with strong, unique combinations.
Keeping all third-party plugins and themes updated.
Avoiding suspicious links and unverified downloads.

Ongoing Communication:
We understand the disruption this may cause to your operations and appreciate your patience and understanding. Please know that we are doing everything within our power to restore normalcy and secure your website as quickly as possible. We will keep you informed with regular updates on our progress.

If you have any questions or need further assistance, our support team is here to help. Do not hesitate to contact us.

TIMELINE OF EVENTS

Tue July 9	We became aware that two clients had an issue with their websites that was impacting a paid advertising campaign.
Wed July 10	We first discovered that this issue impacted more than two clients and appeared to be spreading. Our Success Team escalated to our DevOps Team to investigate what was happening and why.
Thu July 11	We heard from WP Engine, the company that provides our hosting services, that over 100,000 websites were infected with SocGhoulish malware. Please note that WP Engine has not admitted this publicly, and the person who told us about the incident was removed as our Account Manager. Our team began manually cleaning websites as we identified which were infected. Due to malware on their websites, some of our clients had issues with not being able to send or receive email while their site was infected and even after it was cleaned due to the policies of their email service provider. Please note that we do not manage email delivery for any of our clients and have no control over their policies. As we received requests from clients through our help center or via email, we continued to remediate their websites and contact those clients. We did not send a general message to all our website hosting clients because we did not know the cause or extent of the problem, and our primary focus was on remediating the infected websites. We continued to reach out to WP Engine to determine the cause of the malware and their recommendations for remediation (to prevent sites from getting infected a second time). We received no response from them.
Fri July 12	We continued to clean sites. To notify clients about this issue, we posted an update to our help center and emailed all our clients about the event. Friday afternoon, we pulled in team members across the company to remediate the issues on client websites. Our team worked until 3 a.m. in the morning to support clients, and we continued Saturday morning at 7 a.m.
Sat July 13	Our team was able to develop a solution to automate the remediation of impacted websites rather than the manual cleaning we were doing. The process began to run Saturday afternoon, and by Sunday evening, most sites were cleaned.
Sun July 14	Continued automated remediation and manual updates to the remaining sites that could not be remediated with the automated process.
Mon Jul 15	All websites had been cleaned. We posted an update on our help center: https://helpme.haleymarketing.com/hc/en-us/articles/28413323899796-SocGholish-Malware-Attack-UPDATED-7-15-24, and we sent an email to all hosting clients regarding this update.
Tues Jul 16	Several clients reported issues sending emails to highly secure clients (e.g., government, medical, and insurance companies). We identified an issue with the 3rd party Proof Point blocklisting some server IPs and requested delisting.
Wed Jul 17	Email issues persist. For most clients, the workaround of removing the link to their websites in their email signatures allowed emails to be delivered. We tested moving sites to alternative hosts and having companies clear block lists, but the listing appears to be by domain, not IP. We continued to respond to clients and look for solutions. Proof Point sent our request to other departments. WP Engine has not responded other than to confirm that the email was not sent through the WP Engine server IP.
Thurs Jul 18	Proof Point is requesting PTR records on the web server IP, which WP Engine refuses since no email is sent from that IP. Escalated to Proof Point for advice. The only resolution to highly secure industry email appears to be one-off removal from blacklists by providing extensive proof. Additional security measures were put in place to change all HMG admin user passwords on all sites implemented.

FREQUENTLY ASKED QUESTIONS

When did Haley Marketing first become aware of the malware?
The first client inquiry was on Tuesday, July 9. We became aware that this was not an isolated incident on Wednesday, July 10.
Why weren’t clients immediately notified?

We did not initially know the extent of the infection. Once we became aware of the magnitude of the malware problem, we needed to focus on remediation.

The reality of the situation is that we knew that the minute we sent out a broadcast email, we would be inundated with questions from clients, and the time required to respond to these questions would deter us from working to fix the problem.

We understand that some clients disagree with this decision, but we were concerned that the malware would continue to spread if we did not prioritize resolving the problem on Thursday.
Why was WordPress core not updated on my site?
Our hosting partner, WP Engine, maintains WordPress Core updates, which are applied when they determine the update is applied. Effective immediately, Haley Marketing will take over this responsibility and add WP Core to our nightly updates.
Why can’t I update this plugin? Doesn't Haley Marketing take care of this for me?
Haley Marketing programmatically updates all plug-ins nightly. However, some plug-in updates may fail. They may require a manual entry of a license key or, on occasion, even a full manual re-install. Some plug-ins will no longer update if your site has not moved to PHP8, as the developer may no longer support PHP7.4. If you notice a plugin that will not update, please open a ticket, and we will investigate.
Is there a way to tell if ours has been affected?
You can check your site using SIteLock's free website scanner: https://www.sitelock.com/free-website-scan/. Don’t worry, though—we are checking every site and making sure it is clear of malware.
Was there a data breach involved?
No, there was no data accessed in this attack. Additionally, if you use our Career Portal software, it was not impacted by the malware issue at WP Engine. Because our Career Portal is a software application, not a website plug-in, your job data and candidate applications are securely stored on a separate server away from your website. One of the benefits of our Career Portal is that it provides your business information with an extra layer of protection.
Was this attack targeted at Haley Marketing?
No, this attack was widespread and not targeted at Haley Marketing or any of our clients.
Why did some of my emails not get delivered?
As a security measure, many ISPs and email service providers check links in emails. If your website URL is linked in your company signature and your website has malware, they may block your email. This should be clear now as well, but we recommend that you verify that important emails were received, as they may have had a higher-than-usual bounce rate during this issue.
Why didn't Haley Marketing prevent this from happening?
We understand your frustration and truly empathize with your experience. While it may be easy to place blame on us or WP Engine, it's important to remember that the true culpability lies with the malicious group who created the malware and staged this attack. We are constantly vigilant and invest significant resources in defending against such threats. However, the bad actors behind these attacks are always looking for new ways to compromise websites.
How did you solve the problem?
Initially, WP Engine was using Sucuri to mitigate the problem. That stopped (we were not given any reason why this was stopped), and they promised to install and run Wordfence, a security plug-in, on all sites.
Friday evening, WP Engine told us they were running a script to install Wordfence on all their client websites, which they did. However, this script did not run the scan or fix any of the issues it found. That was still up to our team. WP Engine was not moving fast enough, so we began manually installing Wordfence on client sites. Then we ran a scan of each website, and when issues were found, we addressed them one by one.

On Saturday, our team decided that the pace of remediation was too slow, and we developed a program to uninstall any plug-ins that are likely to have been infected and reinstall clean versions. This script was launched Saturday afternoon and ran until Sunday afternoon. It successfully remediated the issue for the majority of the sites we host.

For the remaining sites, we made modifications to the process and reran until all our clients' sites were cleaned, manually making updates as needed
How did the malware get on the websites?
We do not know the answer to this question, and WP Engine has not provided any explanation. We believe this was likely a specific attack on WP Engine since they had so many websites impacted, and other web hosts are not reporting similar issues with this malware.
We continue to try to get an answer from WP Engine, and we have a call scheduled with their team to ask this very question.
How will Haley Marketing prevent an issue like this in the future?
The short answer is that there is no way to prevent issues like this from ever happening. With AI and automation, we anticipate the volume of attempts to compromise websites will increase.
What most of our clients do not know is that we successfully block attacks like this all the time. Our clients’ websites are under attack daily. In the past, things like Distributed Denial of Service (DDoS) attacks would take down our servers and our clients' sites, but we upgraded our infrastructure so that events like these are much less likely to impact our clients.

We also moved from our past hosting company to WP Engine because they are seen as best-in-class for hosting WordPress websites. They host more than 1.5 million websites, and they provide:

- Auto-renewing SL and SSH access
- Security patching and plug-in risk scans
- Advanced DDoS and managed WAF
- Automated WordPress updates
- Automatic backups
- Caching for improved website performance
- Global CDN with more than 200 data centers
- 99.99% uptime SLA

In addition to what WP Engine offers, your hosting fee pays for our Client Success Team and Help Center, along with support from our DevOps team. You have real people you can contact for help throughout the workday.

Going forward, we are re-evaluating our relationship with WP Engine to determine if this is the best option for hosting. While their performance has been excellent for years, their response to this incident was poor—we received very little support, and the solution they offered would have required more than 10 days to implement. Our team reduced this to one day.
What can be done to ensure our email gets through regardless of our website?

There is no easy answer to this question, but there are several things you can do to protect your company and ensure you can get email to your clients and candidates:

- If you suspect an issue with your website, report it to our Success Team immediately. Tell us what you are seeing and why you suspect there is a problem.
- Run a malware scan on your devices to ensure the infection has not spread throughout your organization.
- Remove your company URL from your email signature. This can be done temporarily.
- Have a backup email solution. In the event that your email is ever blocked for any reason, have a backup email account, like a Gmail account, that you can use with your clients and candidates until your primary email addresses are restored.
- Have a landing page site on a separate domain that you can switch to by changing your DNS in the event that anything goes wrong with your main website. You can then redirect your primary domain to the landing page site while your main site is repaired.
- If you serve businesses in highly sensitive industries, such as government, healthcare, insurance, etc., your email may be blocked, and proof that the issue has been resolved may be required before your email is allowed through their firewalls. You will need to have your IT team reach out to these companies directly for assistance clearing the block.
Are there other ways Haley Marketing can protect our company?
Yes. As you may be aware, we offer website success packages that include advanced security through a 3rd party service. We created these packages to offer an affordable solution to clients who want more advanced security, advanced ADA compliance, and lower costs for regular website updates.
If you have a large website, run mission-critical applications via your website, or serve businesses in highly sensitive industries, we recommend you contact us to learn more about the success packages.

Related articles