Wednesday, August 19th, 2009

How to create a new log file for each time the application runs?

Recently I got a question from my blog reader Kuruba, regarding log4j logging. Kuruba is having a batch program with Log4j logging and he wants to send the log messages to a new log file each time his batch runs. This can be achieved by extending Log4j’s FileAppender, which is used for logging to a file, and adding our own customization to the file appender. Below I’m explaining how to do it.

Extending FileAppender to create a new log file for each application run:

Here’s the pseudo code for the logic we are going to implement.

  1. Create a new Class NewLogForEachRunFileAppender (or any name as you wish), and extend it from the class org.apache.log4j.FileAppender.
  2. Override the activateOptions() function for setting the new log file name every time the logger is instantiated.
  3. For simplicity, we will append the current timestamp to each log file name. So, each time when you run the application, you will get a fresh log file.

Customized FileAppender: NewLogForEachRunFileAppender:

Here is the code for the customize FileAppender, which will create a new log file, appender with the current timestamp, for each run of the application. Please note that this code is tested in Windows environment. So, please be cautious and do a testing when running this code in Unix or other environment.

package com.veerasundar.dynamiclogger;

import java.io.File;
import java.io.IOException;

import org.apache.log4j.FileAppender;
import org.apache.log4j.Layout;
import org.apache.log4j.spi.ErrorCode;

/**
* This is a customized log4j appender, which will create a new file for every
* run of the application.
*
* @author veera | http://veerasundar.com
*
*/
public class NewLogForEachRunFileAppender extends FileAppender {

public NewLogForEachRunFileAppender() {
}

public NewLogForEachRunFileAppender(Layout layout, String filename,
		boolean append, boolean bufferedIO, int bufferSize)
		throws IOException {
	super(layout, filename, append, bufferedIO, bufferSize);
}

public NewLogForEachRunFileAppender(Layout layout, String filename,
		boolean append) throws IOException {
	super(layout, filename, append);
}

public NewLogForEachRunFileAppender(Layout layout, String filename)
		throws IOException {
	super(layout, filename);
}

public void activateOptions() {
if (fileName != null) {
	try {
		fileName = getNewLogFileName();
		setFile(fileName, fileAppend, bufferedIO, bufferSize);
	} catch (Exception e) {
		errorHandler.error("Error while activating log options", e,
				ErrorCode.FILE_OPEN_FAILURE);
	}
}
}

private String getNewLogFileName() {
if (fileName != null) {
	final String DOT = ".";
	final String HIPHEN = "-";
	final File logFile = new File(fileName);
	final String fileName = logFile.getName();
	String newFileName = "";

	final int dotIndex = fileName.indexOf(DOT);
	if (dotIndex != -1) {
		// the file name has an extension. so, insert the time stamp
		// between the file name and the extension
		newFileName = fileName.substring(0, dotIndex) + HIPHEN
				+ +System.currentTimeMillis() + DOT
				+ fileName.substring(dotIndex + 1);
	} else {
		// the file name has no extension. So, just append the timestamp
		// at the end.
		newFileName = fileName + HIPHEN + System.currentTimeMillis();
	}
	return logFile.getParent() + File.separator + newFileName;
}
return null;
}
}

log4j.properties file

Since we created a custom file appender, we need to tell the Log4j to use our custom file appender. For demo purpose, I configured this custom FileAppender as a rootLogger. So, all log messages will go to our file appender. Below is the sample log4j.properties file.

log4j.rootLogger = DEBUG, fileout
log4j.appender.fileout = com.veerasundar.dynamiclogger.NewLogForEachRunFileAppender
log4j.appender.fileout.layout.ConversionPattern = %d{ABSOLUTE} %5p %c - %m%n
log4j.appender.fileout.layout = org.apache.log4j.PatternLayout
log4j.appender.fileout.File = D:/temp/dynamiclogger/logs.log

If you look at the property log4j.appender.fileout, our customized file appender is associated with this appender. We have also given a log file name “log.log” in the above configuration file. So, with the above configuration, when you run your application, the log files will be generated in this fashion: logs-1250696128437.log, logs-1250696142828.log.

{ 28 comments… read them below or add one }

govind August 20, 2009 at 10:49 am

Thanks Veer,

Reply

Veera August 20, 2009 at 5:40 pm

@Govind

Welcome. I hope that I answered your question.:)

Reply

Tanzy September 10, 2009 at 10:35 am

Hi Veera,

This is really a very useful content.
Thanks for posting it.

Well i have a query regarding this.
How about if i want to use MaxBackupIndex properties. Can we do it? if yes then how?

Thanks,
Tanzy.

Reply

Veera September 10, 2009 at 4:38 pm

@Tanzy

Yes. It’s possible but with a small code change in the above solution. We just need delete the old log files if the total number of log files reached the maxBackupIndex threshold. This is doable. Will try to post the solution for this later. :)

Reply

Robin February 2, 2010 at 3:26 pm

Thanks for making this code publicly available. It`s very useful and works straight ‘out of the box’ for me on Ubuntu.

Regards

Reply

Veera February 2, 2010 at 4:45 pm

you are welcome.

Reply

KiranB August 30, 2010 at 8:59 am

Hi Veera,
The code works to create multiple log files each time it is called. But am using a Thread java class which calls this log fiel genrate everytime a thread starts, say for example my code is used to run 5 threads parallely and inside my process i tried calling this NewLogForEachRunFileAppender class it generates 5 different log files but writes all the logger.info into single log file inside of writing each process logger.info into its specific log files. Please let me what needs to be done?

Reply

Veera September 1, 2010 at 9:41 am

Hi Kiran,

The above example code was made for different runs of the application, without having threads in mind. That’s why it didn’t work properly to you.

Anyway, I’ll work out a solution and will inform u soon.

Reply

Chandu April 24, 2012 at 5:34 am

Hi Veera,

Were you able to craft a solution keeping threads in mind?

Thanks,
Chandu

Reply

Nancy February 7, 2011 at 8:19 am

Hello,

I couldn’t find the log4j.properties file in any of the log4j jar files, so I created one, but I am not sure where to place it. I put it in the same directory as the one that has the log4j jar files, but my logger would still log out to the console instead to the customized appender. I am not sure what the fix to this issue is, but I created the same customized appender that you have and created a logger in my class that I want to log. It still did not log out to the appender. It seems the logger is not tied to the appender yet. Is there a fix to this issue?

Thanks

Reply

Veera February 7, 2011 at 7:54 pm

Nancy,

Initially you need to create the log4j.properties file for yourself, which you already did correctly. After creating the file, place it inside the WEB-INF folder (if your’s is a web application), otherwise place it inside the source/src folder so that log4j can pick up this file and configures the logging accordingly.

Reply

Innokenty September 6, 2011 at 12:22 pm

actually, the easiest way to do this is just write:
log4j.appender.FILE.append = false
in log4j.properties!!! I have been googling it for a 20 already, and then i got it, and it worked perfectly! nobody writes about this solution and suggests some brainfucking stuff!

Reply

Sridhar Reddy October 13, 2011 at 10:56 am

Thank you very much Veera….i understand very clearly through this …Thank you once again..

Reply

Rajesh October 17, 2011 at 2:28 pm

Mikka nandri Veera, it was so use full for me to meet the requirements. once again thanks much.

Reply

Tabiul November 28, 2011 at 8:12 am

Thanks a lot man. It helped me a lot as I had the same requirement for my project

Reply

Eric December 1, 2011 at 12:01 pm

Hi Veera, i was wondering how you can disable this custom file appender of yours at runtime. I tried removeAppender, but still the logger uses this appender. I’m stuck on this.

Reply

Tarik Makhija March 11, 2012 at 5:37 pm

We are using Weblogic 9.2 and Log4j for logging the application logs
Our Application is used by Multiple testers.
All the Application Logs go to common Log file “APP.log”
So each tester have to scroll down through the file to find his/her each piece of log.

For Weblogic 9.2 Server We have file called “config.xml” , This File gets updated when anything is changed or updated from the Weblogic Console.

In this “config.xml” File , We have a Server setting like below :
-Dlog4j.configuration=file:/srvrs/dev/apacheLog4jCfg.xml

This Setting is referring to File called “apacheLog4jCfg.xml” Which means that Weblogic Server is using apache Log4j to do the logging.

In “apacheLog4jCfg.xml” File We have the setting like below :

Appender Name is APPLOGFILE and class is org.apache.log4j.RollingFileAppender
param name is File and value is APP.log

The Setting “” will log all the application logs to the File “APP.log”
i.e All the testers will hit the application and all the logs will get stored in the File “APP.log”

I have two questions below :
Q1 : So Using log4j , Is it possible that for each Tester testing the application , a separate log file should be created, So that he/she can see their own set of logs ?

Q2 : Is it possible to instruct the Weblogic Container or do Server side setting to use log4j at one point of time for logging to samelog file for all users OR to to instruct the Weblogic Container to use to sl4j , if we want to log to different log files for different users ?

Reply

Rafael March 14, 2012 at 7:44 pm

Great post.
Just one question: How do I do it in a web application, say, to create a new log file each time an user logs in ?

Reply

Veera March 15, 2012 at 6:23 am

I doubt will it be a efficient option to create a new file for each user login..

Reply

Rafael March 15, 2012 at 2:12 pm

Hi, Veera. I developing a web application and I have no experience with logging strategies. I thought a loggin by user was supposed to be good. Thus, what do you advise as good one?

Reply

Veera March 16, 2012 at 3:28 am

Here’s what we followed: We use a daily rolling file appender – every day a new log file will be created. Then for each and every request our application receives, we create a correlation ID which is specific to that request alone and we log it with every log statement so that we can identify which request the log is belongs to.. we also log the user name so that we can associate the log to the corresponding user.

Reply

Rafael March 16, 2012 at 1:11 pm

good, thanks

Reply

Madhu March 20, 2012 at 1:08 pm

Hey Veer,

your post helped me very much. Appreciate your effort
Thanks very much

Madhu

Reply

Anil April 20, 2012 at 8:36 am

Thanks man! Doin gr8 job!
When ever i need some info regarding log4j i will open your site!
Explained in a very clear and understandable way!

Reply

padmaja April 23, 2012 at 2:52 am

Hi,
I’ve created a properties file like this.But the log file is not created in the specified location.Please suggest me a solution for this.

log4j.rootLogger = DEBUG, induction
log4j.appender.induction = com.chrysler.action.NewLogForEachRunFileAppender
log4j.appender.induction.layout.ConversionPattern = %d{ABSOLUTE} %5p %c – %m%n
log4j.appender.induction.layout = org.apache.log4j.PatternLayout
log4j.appender.induction.File = D:/logs.log

Reply

voahar April 24, 2012 at 10:00 am

Hi
Thanks for this post, it has been very useful for me :p
Have a nice day
Cheers

Reply

NJ May 3, 2012 at 4:13 am

HI,

Thanks for this information,it resolve my issue.

Reply

Raj May 7, 2012 at 11:07 am

Hi veer,

how to incorporate this fileappender class with my application ???

Reply

Leave a Comment

Previous post:

Next post: