UK online store, WooCommerce on a VPS
WooCommerce store overloaded by scrapers
17.8→~2server load
- Problem
- CPU and memory kept spiking and the store slowed down for real customers.
- Cause
- A site-copying scraper spread across roughly 4,400 IP addresses, plus aggressive crawlers, and WP-Cron running on every page view.
- What I did
- Identified and blocked the scraper network from the access logs
- Rewrote robots.txt to rein in crawlers that respect it
- Moved wp-cron to a real system cron job
- Result
- Server load dropped from 17.8 to about 2.