Simple program to report successful/fail ip logins and sorted by count

[arukashi]

JDunphy, Thanks, great job with this script. But i think something has changed with log format in Zimbra 9 and log lines about web login attempts are not shown. So i had to change "http" to "qtp" at line 158, and "invalid" to "authentication failed" here

And then It comes to me that some automation is missing here, so i made one based on check_login.pl reports. It is pretty clumsy, but do what i want.
We must have an ipset module for iptables and netfilter-persistent package installed. Hope that helps anyone.

Code: Select all

#!/bin/bash

# SYNOPSIS
#
# This script updates ipset list with IPs that were considered as bruteforcers addresses
# IP list is updates constantly so old abusers IP stays in the list
# bruteforce_monkeys is the target ipset

mv_time=$(date "+%Y.%m.%d-%H.%M.%S")
tmp=/tmp/ipsets.tmp
tmp2=/tmp/ipsets2.tmp
ipset=/etc/iptables/ipsets

# List of valid IP addresses, e.g. known users who may typed invalid password by mistake
whitelist=/scripts/whitelist_for_bruteforce_ips.txt

# Grep existing IP
awk '/add bruteforce_monkeys/,0 AND {print$3}' $ipset > $tmp

# Getting fresh list from Zimbra logs using check script and add them to existing list
./check_login.pl --gethost none --fail ip | awk '{print$6}' | grep -v total | grep -v -e '^[[:space:]]*$' >> $tmp

# Sort by uniques and grep over whitelist, just in case.
sort -u $tmp | grep -v -E -f $whitelist > $tmp2

# Add some syntax magic to temporary file
sed -i -e 's/^/add bruteforce_monkeys /' $tmp2

# Backing up ipset conf
cp $ipset /scripts/ipset_bckp/ipsets.bckp_$mv_time

# Another syntax magic
sed -i -e '/^add bruteforce_monkeys/d' $ipset

# Final update to ipset conf
cat $tmp2 >> $ipset

# Apply updated list
systemctl restart netfilter-persistent.service

# Remove garbage
rm $tmp $tmp2

At the end of iptables rules add this

Code: Select all

iptables -A INPUT -p tcp -m set --match-set bruteforce_monkeys src -j DROP

Don't know if it would be that effective as i expect, but would do no harm either, haha

[JDunphy]

Very nice and pretty cool!

Jim

[thomas.klaube]

Hi JDunphy,

it would be really cool, if this script would count unsuccessful sasl login attempts as well because this is what we see most on our servers: scripts that try to guess user passwords through SMTP AUTH. You can grep like this:

grep -i sasl /var/log/zimbra.log | grep -i "auth failed" | less

I find hundreds if not thousends of those login attempts. But they are not reflected in audit.log (at least not each of them).

Regards
Thomas

[JDunphy]

Hi Thomas,

Hopefully this get's posted and not moderated. I have had to re-edit long and effectively remove posts on these new forums. This is my second try here.

I think I might have something like that. Last fall check_login.pl was rewritten after I noticed that failures could also be found in other logs and it didn't appear to be parsing correctly as zimbra has progressed. It's a lot different because it now uses a SQLite db to hold the ip's which I needed for another project where we have automated and build ip lists on hacker activity in real-time based on learned behavior from previous ip attacks. I need to get that version cleaned up so I can share it. That version could also have a zimlet to display all the known good ip's for a user and how many bad ip's for password health since the zimlet could pull from the SQLite db. Lot's of possibilities.

In the mean-time, there is always this which I use to learn about attack patterns. In a tuned (search for STEP or '%%%' in check_attacks.pl) version, it only reports %100 scores for hacker activity and not any local users. It can print lists of ip's which might work together with your scripts:

Code: Select all

% check_attacks.pl --help
        [--fcolor=<color name (i.e. RED)>]
        [--srcip=<ip address>]
        [--search='regex of search']
        [--localUser ]
        [--logDir=`pwd` ]
        [--file=nginx.access.log ]
        [--IPlist ]
	[--statuscnt]
	[--display="date|upstream|bytes|port|referrer]
	[--usertype=<attacker|local|all>
	[--pstatus=<regex of status codes>
        [--help]
        [--version]
    where:
        --srcip|src: print only records matching ip addresses
        --search|sea: print all requests from an ip that has a search term hit
	--statuscnt: prints out the count for each status return code found
        --help|h: this message
examples:  (-- or - or first few characters of option so not ambigous)
         % check_attacker.pl -srcip 10.10.10.1      #only this ip address
         % check_attacker.pl -srcip  '10.10.10.1|20.20.20.2'      #only these ip addresses
         % check_attacker.pl -search  'python|POST'     # if an ip had search term, all requests printed for ip
         % check_attacker.pl -search  '.jsp|.php|bot' # if an ip had search term, all requests printed for ip
         % check_attacker.pl -statuscnt  #print status codes
         % check_attacker.pl --statuscnt  #print status codes  #same
         % check_attacker.pl --localUser #include local users accounts
         % check_attacker.pl --IPlist   # print list of ips
         % check_attacker.pl --IPlist --localUser   # print list of ips from local users
         % check_attacker.pl --IPlist --ipset  # print list of ips in ipset format
         % check_attacker.pl --IPlist -pstatus='40.' --ipset  # print list of ips in ipset format with status code 400..409
         % check_attacker.pl --localUser --IPlist   # print list of local ips used by local users
         % check_attacker.pl --IPlist --ipset  | sh # install ip's into ipset 
         % check_attacker.pl --initIPset  # show how to create ipset 
         % check_attacker.pl -fc RED  #change color 
         % check_attacker.pl --usertype=local  # print out strings of only local users
         % check_attacker.pl --pstatus='4..'  # print out only those requests with a code of 4XX (ie 403, 404, 499)
         % check_attacker.pl --usertype=all --pstatus='403|500'  # print out only those requests with a code of 403 or 500 for all types (local & attacker)
         % check_attacker.pl --display=date      # default is to display the user agent
         % check_attacker.pl --display=referrer  # default is to display the user agent

    Status Codes - https://en.wikipedia.org/wiki/List_of_HTTP_status_codes

You could do something like this:

Code: Select all

% check_attacks.pl --search '(auth\s*failed|python)'
% check_attacks.pl --pstatus='401'
%  check_attacks.pl --search '.php|python'
% check_attacks.pl --srcip '(90.151.171.106|98.118.50.242)'

If you like the output results, then add --IPlist and --ipset

Code: Select all

% check_attacks.pl --search '(auth\s*failed|python)' --IPlist --ipset

Note: All output goes to stdout so if you either put it in a file or pipe the output directly to sh. ie)

Code: Select all

% check_attacks.pl --pstatus='401' --ipset --IPLIST | egrep -v '(X.X.X.X|Y.Y.Y.Y)' | sh

Note: IPLIST seems to add all attackers ip's based on a 100% ratings score. I need to fix that since it probably should use search. Note: Real users can have 403, 404's, 500's, etc but I have never seen a 401 status code for example. It's main purpose was to isolate real user activity from hacker activity so I could focus on what was happening to our servers.
Ref: https://github.com/JimDunphy/ZimbraScri ... attacks.pl

I was playing around with ChatGPT4 a few days ago to see what it could do and had it build me a tool. It can assign dates to ip's and then let you query and pull lists of ip's based on how old they are. So one could query your maillogs, look for dictionary attacks, unfinished smtp sessions, etc and use this to track ip's for other programs. For example, query based on how new the ip's should be and build ip lists for ip reputation milters for ipsets in FW's.
Ref: https://github.com/JimDunphy/ZimbraScri ... manager.pl


There are commands like iprange that can take lists of ip's and build cidr's so your ipsets are a lot more efficient. All of USA,CAN,GB, AUS when filtered through iprange with a prefix of /8 is only 24,651 cidr's. That is our normal server traffic and we have a milter on our MX's to add a header to any incoming email to allow our spam rules to score higher from other countries. For other users we have a zimbra filter that automatically tags any email as foreign for countries outside that range. Just an extra FYI these came from what we think is a foreign address for normal email traffic to us. Not blocking just more information to the users.
Ref: https://github.com/firehol/iprange

[ThyTe]

Hi JDunphy,

it would be really cool, if this script would count unsuccessful sasl login attempts as well because this is what we see most on our servers: scripts that try to guess user passwords through SMTP AUTH. You can grep like this:

grep -i sasl /var/log/zimbra.log | grep -i "auth failed" | less

I find hundreds if not thousends of those login attempts. But they are not reflected in audit.log (at least not each of them).

Regards
Thomas

Great idea!

And if possible, add a variable to the script to be able to filter the result by adding an IP together with the domain...