Geeks With Blogs
The Unstable Mind of a .Net Developer

To be honest, while I was writing the original Adventures in System Diagnostics post, I had no intentions of turning it into a series.  Since then, however, I have given consideration to implementing in a production environment (already written, the sequel) and also to developing custom listeners (not yet written, soon to be the threequel?).  With these last two titles, I had thought that would be the end of this topic.  It turns out, I was wrong.

Just this past week, we started seeing issues with an application in which I had used TraceSource extensively.  This particular application is long running and does a lot of work processing data within a database.  Because of this, a lot of exceptions are caught (insufficient privileges, missing tables, etc.), written to TraceSource and then subsequently ignored to be reviewed post process.  The issue we started seeing was that it would encounter one of these throwaway exceptions and then terminate execution.  The entire execution is handled in its own try..catch and we could see where there was a problem, but no exception details were given.  What was worse, when reviewing the code, the exception was occurring between a call to the TraceSource.TraceEvent method and a return statement.

 

 public bool SomeMethod() {
    try {
        // Do something }
    catch (Exception ex) {
        // Log the message to the trace and move on
       
ts1.TraceEvent(TraceEventType.Error, 0, "Log some exception");
        return false; <-- Error occurred between the statement above and here
    }

    return true;
}

 

It turns out that the unknown exception was occurring because of the TraceSource (Well, at least the TraceSource configuration).  The nature of the application requires messages be written to three different locations, a text file (verbose messages for later troubleshooting), the console (informational messages for users tracking progess) and the event log (error messages to be reviewed by network ops, dbas).  The following is a how the trace listeners are configured:

      <add name="fileTrace" type="System.Diagnostics.TextWriterTraceListener">
        <
filter type="System.Diagnostics.EventTypeFilter" initializeData="Verbose"/>
      </
add>
      <
add name="eventTrace" type="System.Diagnostics.EventLogTraceListener">
        <
filter type="System.Diagnostics.EventTypeFilter" initializeData="Error"/>
      </
add>
      <
add name="consoleTrace" type="System.Diagnostics.ConsoleTraceListener">
        <
filter type="System.Diagnostics.EventTypeFilter" initializeData="Information"/>
      </
add>
 

When you consider how the TraceSource is initialized, it makes sense these listeners are triggered in order.  So, in the case of the message above, it would be written to the text file first, the event log second and the console last.  Herein lies the problem and the solution.  The  message was being written to the text file, but not to the event log or console.  Likewise with the top level exception message.  Now, you may get this right away, but I must admit it took us a couple of moments to catch on to the only scenario (or at least one of very few) where the message was written successfully to the text file, but not to the event log or console and an exception was thrown in the interim.

5, 4, 3, 2, 1 …

That’s right, the event log was full!  Uugghhh!  The Framework was throwing an exception because the TraceSource listener could not write to the event log, which was then triggering the top level exception handler.  No details were provided because the TraceSource.TraceEvent method in the top level exception handler was also failing causing the TraceSource.TraceData method call to never be executed.

At this point, the event log was cleared out and the application restarted.  This time, when the application hit the same throwaway exception, it ignored it like it was designed to do and proceeded to hum along.  Problem solved!

So, when using TraceSource and the event log trace listener, be sure to check the permissions and configuration of the Application event log on the server where your application is deployed to avoid having the same or similar issues.

 

Ralph Wheaton
Microsoft Certified Technology Specialist
Microsoft Certified Professional Developer

Posted on Thursday, September 24, 2009 9:53 AM ASP.Net , System.Diagnostics , TraceSource , TraceEvent | Back to top


Comments on this post: Adventures in System.Diagnostics – The Intermission

# re: Adventures in System.Diagnostics – The Intermission
Requesting Gravatar...
Thanks, we do forget such minor issues and keep on struggling with the problem longer than needed. Do appriciate for highlighting this.
Left by Asad Ali Butt on Mar 16, 2010 5:08 PM

Your comment:
 (will show your gravatar)


Copyright © Ralph Wheaton | Powered by: GeeksWithBlogs.net