Geeks With Blogs
A Technical Debtor Toward continuous improvement

I'm really not obsessed with performance -- honest!

However, when a co-worker asked me today exception handling was an acceptable way of coding defensively, my reaction was rather predictable. Exceptions are pure evil, and should be... well, exceptional.

Yes, you guessed it. The next question was "How bad is try/catch really?"

The short answer is that is involves minimal overhead... unless an exception is thrown. In that case, the .NET exception handling mechanism does a few nice things, like providing the stack trace as part of the exception. Great for debugging... lousy for performance. Of course, if exceptions are treated as being exceptional -- that is, they're only raised when something you haven't predicted occurs -- then you really shouldn't care about performance. Since your application is most likely left in an unknown or unstable state after an exception, the odds are you're going to reset a substantial part of your application, if not shut it down entirely.

As a rule, you can predict that users will provide data that is, to put it politely, suspect. This isn't a reflection on the users; if anything, it's more a comment on the tendency of developers to know how their own code works, and to continually feed it good data. But, both for this reason, and for security reasons, the ASP.NET mantra of "never trust any input" applies in pretty much any application. Code defensively against user input, so that you don't have to rely on try/catch blocks to deal with exceptions.

How significant is the overhead of catching an exception? Here's the code for a console application to illustrate:

  1: Module Module1
  2:     Sub Main()
  3:         TestNaive(1)
  4:         TestDefensive(1)
  5:         TestWithTryCatch(1)
  6:         TestWithException(1)
  7: 
  8:         Dim maxCount As Integer = 1000000
  9: 
 10:         Console.WriteLine("Naive: " & TestNaive(maxCount) & " ticks")
 11:         Console.WriteLine("Defensive coding, no try/catch: " & TestDefensive(maxCount) & " ticks")
 12:         Console.WriteLine("Naive with try/catch: " & TestWithTryCatch(maxCount) & " ticks")
 13:         Console.WriteLine("Exception: " & TestWithException(maxCount) & " ticks")
 14:         Console.ReadLine()
 15:     End Sub
 16: 
 17:     Private Function TestNaive(ByVal iterationCount As Integer) As Double
 18:         Dim sw As New Stopwatch
 19:         sw.Start()
 20:         For i As Integer = 1 To iterationCount
 21:             Dim x As Double = i / (iterationCount - i + 1)
 22:         Next
 23:         sw.Stop()
 24: 
 25:         Return sw.ElapsedTicks / iterationCount
 26:     End Function
 27: 
 28:     Private Function TestDefensive(ByVal iterationCount As Integer) As Double
 29:         Dim sw As New Stopwatch
 30:         sw.Start()
 31:         For i As Integer = 1 To iterationCount
 32:             Dim y As Integer = (iterationCount - i + 1)
 33:             If Not y = 0 Then
 34:                 Dim x As Double = i / y
 35:             End If
 36:         Next
 37:         sw.Stop()
 38: 
 39:         Return sw.ElapsedTicks / iterationCount
 40:     End Function
 41: 
 42:     Private Function TestWithTryCatch(ByVal iterationCount As Integer) As Double
 43:         Dim sw As New Stopwatch
 44:         sw.Start()
 45:         For i As Integer = 1 To iterationCount
 46:             Try
 47:                 Dim x As Double = i / (iterationCount - i + 1)
 48:             Catch ex As Exception
 49: 
 50:             End Try
 51:         Next
 52:         sw.Stop()
 53: 
 54:         Return sw.ElapsedTicks / iterationCount
 55:     End Function
 56: 
 57:     Private Function TestWithException(ByVal iterationCount As Integer) As Double
 58:         Dim sw As New Stopwatch
 59:         sw.Start()
 60:         For i As Integer = 1 To iterationCount
 61:             Try
 62:                 Dim x As Double = i / 0
 63:             Catch ex As Exception
 64: 
 65:             End Try
 66:         Next
 67:         sw.Stop()
 68: 
 69:         Return sw.ElapsedTicks / iterationCount
 70:     End Function
 71: End Module

My results tend to vary a little bit, since the work done in each loop so trivial that it is susceptible to noise. The first three tests are generally pretty close to each other. The fourth test -- testing with an exception raised -- is always much longer. Generally it's somewhat more than 5x longer than the other tests.

TryCatchPerfTest

TryCatchPerfTest2

What I find really interesting is that the inclusion of the try/catch block appears to make the naive implementation run faster. I don't have the .NET debugging symbols loaded on my machine, and dotTrace really didn't provide any insight. Anyone out there who has dug deeply into the CLR and can provide insight, it would be much appreciated.

However, the moral of the story is straight-forward. Treat exceptions as exceptional. Code defensively rather than catching exceptions wherever possible. No surprise there.

Posted on Monday, December 1, 2008 5:06 PM Tips and Tricks , VB , DevCenter | Back to top


Comments on this post: Try/Catch Performance

# re: Try/Catch Performance
Requesting Gravatar...
The reason why your try/catch function is faster is easy:
The defensive version has an additional if clause which causes an conditional jump which ruins the day of the branch prediction unit of the processor ;-).


32: Dim y As Integer = (iterationCount - i + 1)
33: If Not y = 0 Then
34: Dim x As Double = i / y

46: Try
47: Dim x As Double = i / (iterationCount - i + 1)
48: Catch ex As Exception
49:
50: End Try

Yours,
Alois Kraus
Left by Alois Kraus on Dec 02, 2008 6:31 AM

# re: Try/Catch Performance
Requesting Gravatar...
Alois,

Perhaps I wasn't clear enough. I was referring to the difference between the naive test and the naive test with try/catch. There's no defensive coding there, but the version with try/catch appears faster.

Jeff
Left by Jeff Certain on Dec 02, 2008 6:34 AM

# re: Try/Catch Performance
Requesting Gravatar...
Not sure I can explain it off the top of my head, but compiled for AnyCPU or x64 on my x64 Windows results in the following timings:

Naive: 0.446562 ticks
Defensive coding, no try/catch: 0.344532 ticks
Naive with try/catch: 0.442742 ticks
Exception: 0.339604 ticks

An x86 build results in timings closer to yours:

Naive: 0.59108 ticks
Defensive coding, no try/catch: 0.205472 ticks
Naive with try/catch: 0.205776 ticks
Exception: 5.797074 ticks

Not that you should really care either way, as even at 6 ticks you're still doing around 1.5 million/second. Not really the bottleneck in any real world app, I'd think.
Left by Mark Brackett on Dec 02, 2008 9:25 AM

# re: Try/Catch Performance
Requesting Gravatar...
Mark,

Thanks for the feedback.

I am indeed building on an x86 box.

As far as this not being the bottleneck, I'd agree... at least on modern hardware, in most scenarios.

However, if you're coding in a constrained environment, it's sometimes necessary to eke out every bit of performance possible.

From a more philosophical point of view, I'd say that defensive coding is worthwhile for two reasons. The first is that I find it easier to understand what's going on, as well as the developer's intent. The second is more an issue of craftsmanship. While not as important as it used to be, writing code that performs well is worth doing in it's own right. And, in some cases, such as trying to optimize the throughput of a large number of concurrent processes, performance still matters.

Jeff
Left by Jeff Certain on Dec 02, 2008 9:46 AM

# re: Try/Catch Performance
Requesting Gravatar...
I think you also need to be careful about what you are testing - you have established that exception handling code is really slow WHEN the exception is always raised, but that is hardly a surprising result..

I played around with your defensive coding functions and tested with the following:

1) removed the +1 from the denominator calculation, to ensure the branch/exception code is called once
2) added two "1inN" functions to test various percentages of exceptions.

E.g.,
For j = 1 to iters / N
for i = 1 to N
//exception or If Not code, no +1

Results for x86:
Defensive coding, NO try/catch: 0.187116 ticks
Defensive coding and try/catch: 0.185249 ticks

Defensive coding, NO try/catch, 0.1% div/0: 0.197068 ticks
Defensive coding, try/catch and 0.1% div/0: 0.184994 ticks

Defensive coding, NO try/catch, 1% div/0: 0.184682 ticks
Defensive coding, try/catch and 1% div/0: 0.189369 ticks

Defensive coding, NO try/catch, 10% div/0: 0.104836 ticks
Defensive coding, try/catch and 10% div/0: 0.270024 ticks

Defensive coding, NO try/catch, 25% div/0: 0.061308 ticks
Defensive coding, try/catch and 25% div/0: 0.436903 ticks

For AnyCPU / x64:
Defensive coding, NO try/catch: 0.211974 ticks
Defensive coding and try/catch: 0.209349 ticks

Defensive coding, NO try/catch, 0.1% div/0: 0.209431 ticks
Defensive coding, try/catch and 0.1% div/0: 0.20953 ticks

Defensive coding, NO try/catch, 1% div/0: 0.198279 ticks
Defensive coding, try/catch and 1% div/0: 0.197792 ticks

Defensive coding, NO try/catch, 10% div/0: 0.126249 ticks
Defensive coding, try/catch and 10% div/0: 0.139832 ticks

Defensive coding, NO try/catch, 25% div/0: 0.078965 ticks
Defensive coding, try/catch and 25% div/0: 0.092372 ticks

Conclusion: When testing for performance, measure the same things, and make sure you are focused on the appropriate use cases, and the appropriate code. For example, I assumed from previous experience in other languages that a major bottleneck in your code was the Dim inside the for loop. I tested it, and not so on my machine.

YMMV
Left by Sean Stapleton on Dec 03, 2008 3:21 AM

# re: Try/Catch Performance
Requesting Gravatar...
Sean,

I deliberately hit the exception repeatedly to see what the impact of hitting it once was. (That is, the number I display is the average cost of raising the exception). Really, the point was to show that defensive coding performs better than catching exceptions -- and your tests seem to bear that out nicely.

Jeff



Left by Jeff Certain on Dec 08, 2008 9:30 AM

# re: Try/Catch Performance
Requesting Gravatar...
What my results show is that *on my machine, x86* if you are doing *10M* divsion ops in a tight loop, *and* 10% of the operations are div/0, defensive coding will save you 0.17 seconds.

Conclusion: When testing for performance, make sure you are focused on the appropriate use cases, and the appropriate code
Left by Sean Stapleton on Dec 11, 2008 4:48 AM

# re: Try/Catch Performance
Requesting Gravatar...
I did similar tests a while back that yielded that the cost of try/catch, although not insignificant, was -- for the benefit given -- acceptable.

The code used, and results, can be found at:

http://skysigal.xact-solutions.com/Blog/tabid/427/EntryId/318/C-The-cost-of-try-catchs.aspx
Left by Sky Sigal on Dec 17, 2008 8:08 PM

Your comment:
 (will show your gravatar)


Copyright © Jeff Certain | Powered by: GeeksWithBlogs.net