Scrubbing logs for bad IPs
Periodically, log files should be scrubbed for bad bots and malicious IPs. Let's do that.
This will work with any log file in which you have one IP address per line.
The first thing we want to do is pull out all the IP addresses in our log file. Let's go ahead and do that for lawsonry's log file right now:
grep -E -o '(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4]
[0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)
\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)' lawsonry.dashingwp.com.access.log
>> ipslist.txt
Make sure you stick all that on a single line!
Unfortunately, if you cat ipslist.txt you'll see a bunch of duplicate entries.
To get rid of them, let's sort the file:
sort -u ipslist.txt >> ipslist
Now remove our old file with rm -rf ipslist.txt because we don't need it
anymore.
Go ahead and cat ipslist to see the list of unique IPs.
Alternatively, you could go straight to iptables (which is what I do) and just block them at the server level:
BLOCK_THIS_IP="x.x.x.x"
iptables -A INPUT -s "$BLOCK_THIS_IP" -j DROP