Parsing a log file can be a simple but effective way to monitor the health of your application in a production environment and is very cost effective considering data center can grow to many thousands of servers. A single server with a one or many business critical applications can provide multiple log files to help monitor the health of the server and application. On *nix the directory /var/log/ contains system and *nix application log files. Let's say your interested in seeing the http requests of your incoming transactions through your Apache http server.

tail -f /var/log/httpd/access_log

This tail gives you the ability to view real-time http server activity which can be an enormous amount of data. To limit the data append a " grep index.php?id= " to the tail to only view index.php?id CGI transactions. A known security issue from the access_log file can be that incoming requests contain bogus CGI parameters indicating an external attack.

Well obviously you don't want to sit in front of the screen waiting for errors to scroll by, you should have better things to do with your time. As described in earlier chapters enterprise monitoring solutions provide log file adapters for your log files. These log file monitors automate the process of manually sitting in front of the screen or later on digging through the logs searching for possible problems. The general idea here is to be informed when critical errors occur through E-mail, SMS or sent to an event correlation server to determine the severity.

Here are a few tools that specialize in log file monitoring that have required features such as auto-discovery and low CPU utilization when parsing multiple log files on the same server.

- Splunk http://www.splunk.com/ auto-discovery, low CPU utilization
- Nimsoft http://www.nimsoft.com/ very low CPU utilization

Additional helpful features to look out for are how to manage new feature error codes on application releases. Changing error codes in the application log files can be a daunting task without the enterprise level management of the error code in the log monitoring application. Managing error codes in logs includes template changes for editing a group of servers with the same applications, centralized console for log file monitor changes, alert suppression in cases where you don't want to be flooded by events where the error code occurs multiple times within a short period of time.

The real challenge with log file monitoring, is we are waiting for an error to occur then respond to the issue rather than preemptively responding to arising critical situation. If you really think about it, the application error codes are inserted by the application developer who consider what could happen to a certain method, for example java.io.Exception. Why hasn't this been fixed in the first place, why anticipate a production outage if at this place in the code an error were to possibly occur? The next chapter explains additional benefits and the value of preemptive monitoring though production level application code analysis at run time.