Back to your phpLogCon instance
A Little Background: Our FTP server is being attacked on a regular basis. Under an attack there are upwards of 60 hits per minute, usually from half way around the globe. I don't really know what they think is on our server, but they're determined to get in, or at least not let anybody else get in. The first time I saw the attacks there were over 25K connection attempts in less than 6 hours. I had no way of stopping them. We run a Cisco ASA firewall and their threat detection wasn't doing the trick. So I decided to use a little Linux trickery and write a small script to help thwart their efforts. Basically, an expect script to logon to the firewall via command line and shun the offending IP address until the attack stopped. First, I had to get info on when the server was being attacked. Initially I setup a separate log file for the firewall that rotated every 5 minutes and had the program scan it for attacks. It was the only thing I could think of since our firewall log files grow to over 4 million entries a week. This was quite messy and eventually unacceptable. Enter pla2 (PIX logging architecture.) A database driven software that allows viewing of the PIX activities. Finally, a simple query to pull records based in time. This worked well for the first few weeks, but again the database grew way too big. Daily pruning of 7,000 records took over an hour, which left the FTP server vulnerable until the operation finished. Again unacceptable. I'm not adept at perl programing, so modifying or extracting what I needed from pla2 in a timely fashion wasn't really an option. So began the hunt for something that would go straight from system to a database.
Four days ago I found rsyslog on the net. My disto(s) of Linux also had a package for it. I installed it, but the package was quite a few versions old, 2.0.1. As of this writing there's up to at least 3.21.4. Needless to say it didn't work. So back to the website. I downloaded the latest stable package 3.18.4. This was promising, but I could not get the regular expressions to pull the data I needed. They say the 3rd times the charm. So it was. I was hesitant to use a development version on a production server, but was not disappointed! It took quite a bit of lurking around the forums here and gathering info elsewhere, but I'm now able to save only the FTP traffic to our server into a database while still continuing to log all firewall traffic to the normal log file. This cuts down the database size by quantum measures. The only real problem I had were pulling only the fields I wanted since Cisco's log format is so unique(being polite.)
Here's what I ended up with. This will pull the date, time, source and destination IP and ports. The config file discards anything other than traffic to the server on port 21... a way cool feature I might add.
- Code: Select all
$template std_format, "%timestamp% %FROMHOST% %msg%\n"
local6.* /var/log/firewall.log;std_format
:msg, !contains, "OurFTPServer.com(21)" ~
$template ftp,"insert into traffic_log (log_time, log_src_ip, log_src_pt, log_dst_ip, log_dst_pt) values ('%timegenerated:::date-mysql%', '%msg:R,ERE,1,BLANK,0:(\b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b)--end%', '%msg:R,ERE,1,BLANK:Internet/.*\((.*)\) ->.*--end%', '%msg:R,ERE,1,BLANK,0:DMZ\/(\b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b)--end%', '%msg:R,ERE,1,BLANK:Internet/.*\((.*)\) hit-cnt.*--end%')",sql
local6.* :ommysql:localhost,Database,User,Secret;ftp
If anyone knows of a better way to do this... I'm all ears! I searched and tried all kinds of regex's to get the IP, but this was the only one, that I could find that worked.
I'm only on day one of production logging of the FTP traffic, but I can foresee no headaches on the horizon.
Thanks Again, I think you may have saved my bacon and my FTP server.
Best Regards,
M.P.H.
). I would like to double-check if there is an easier way to extract (I doubt...) but therefore I need to see one.
After setting this up yesterday and having a couple of FTP attacks last night, I've decided I need more info for tracking, etc... I started working on putting more fields in the database this morning. I'm trying not to use the field functionality as I may add other log entry types into the mix later. I think that a careful combination of Expression-Based Filters and templates should do the trick.
I will check it out once I get this settled. I'll also be looking at the WinSyslog for our critical servers sometime here in the future.