Wednesday, October 15, 2008

Log data outside your database.

As described in my previous blog about building a server environment that's scalable and reliable one of the ideas is to log outside your database. In this blog I will describe the advantages of doing this.
Logging is essential to manage your infrastructure. Within an application you can divide the data you have into 2 types:
1) Transactional data. For instance, an financial entry, customer information, ....
2) Log data. For instance Date time that the financial entry is created. Time spend to create the financial entry, .....

What are advantages of logging data outside the database. This means you will have only transactional data in your database.
1) No bottleneck in your database to insert log records.
2) The database transaction log will be much smaller because most insert, update actions are done on log tables. Backup of transaction log will be smaller and faster, including the restore if needed.
3) Backup of data log can be less frequent as transactional data. Loss of log data will not hurt the users. It will hurt the manageability of the system.
4) Better use of data cache on the server. Cache is only filled with transactional data. Data records are not part of the data cache anymore.

Of course every advantage has his own disadvantage. For instance:
1) Separate database(s) for logging records, which results in more databases to manage.
2) If log data is logged in local files, how to aggregate this local files to one log table. You need to build a mechanism for it.

No comments: