Andreas Hausladen generously took the time to make some detailed comments on my previous post, one of which prompted me to throw together some further performance test cases for String types specifically. The results were something of a mixed bag and contained some surprises.
The Tests
Methodology
In any discussion of performance testing inevitably methodology comes under scrutiny – I don’t intend getting into that discussion – relative differences are the point of interest here and in terms of the code under test I have tried to be careful to level the playing field as far as possible. Of course, if anybody finds anything that completely invalidates any of the tests, that’s a different matter entirely.
Compiler settings were the same in all cases Specific settings of note to be mentioned are:
- For Delphi 2009, the new $STRINGCHECKS compiler setting is OFF. It is worth noting that this is ON by default and incurs a performance penalty that is unnecessary unless your Delphi projects also make use of, or are made use of by, C++ Builder.
- For all versions of Delphi tested, FastMM 4.90 was used.
The memory manager in Delphi 2009 seems actually to be marginally more efficient even than this latest version of FastMM, which was interesting in itself, and for practical purposes of course we would use the built-in memory manager if that is the most efficient.
But for the purposes of comparative performance testing, using the same memory manager with all compilers ensures that results aren’t influenced by differences in memory manager implementations.
Areas Under Test
Andreas’ comments mentioned use of const parameter declarations specifically, so I looked at these in particular, comparing calls to a procedure with a const parameter and to a procedure with no const declaration on the same parameter. Subsequent comments discussing the $STRINGCHECKS setting shed some light on Andreas’ initial findings, but still the results from these tests contained a surprise so I left them in.
Simple tests were contrived for string assignment, concatenation and RTL routines Copy() and Delete().
I included IntToStr() in order to repeat the tests that previously gave me cause for concern w.r.t ANSI string performance in Delphi 2009.
I also tested using Pos() to find a substring in a string – both where the substring of interest did exist within the subject string, and also where it did not.
The final test performed in each compiler version was for the Replace() function.
A number of additional tests were then run in specific compilers.
For Delphi 7 and Delphi 2007 I also included tests of the FastStrings FastPos() and FastReplace() functions. I did not bother including these tests in Delphi 2009 following Andreas comments (confirmed by the tests) about the FastStrings having been superceded by improvements in the RTL.
For Delphi 2009 I included repeats of the various tests using strings declared as ANSIString – no explicit conversions to Unicode were involved in any of these tests – all variables, parameters etc were declared consistently as ANSIString.
It was suggested that a comparison with WideString might be of interest. I’m not convinced that this really is going to provide much insight – we already know that WideString is a very inefficient type that is not fundamentally changed in Delphi 2009, and that UnicodeString is vastly more efficient – nevertheless I included these tests to satisfy those with curiosity in this area.
Also for Delphi 2009 I incorporated a repeat of the Concat test using a TStringBuilder, just to see how performance of this class compared with regular string building operations (specifically concatenation in this case).
The Tests and The Results
 |
download:
StringPerformance - Source (1.45KB) added: 17/09/2008 clicks: 654 description: Source code for string performance tests. |
Without my Smoketest framework (coming soon, I promise!) this will not compile let alone produce any test results but I provide the source so that anyone can find artefacts in my test code that they feel might explain test results they believe to be inaccurate or unrepresentative.
 |
download:
StringPerformance - Test Results Data (19.21KB) added: 17/09/2008 clicks: 709 description: Test results data with pretty Excel formatting. |
This isn’t how Smoketest emits the results by the way (I wish!). The test project is compiled as a CONSOLE app and emits performance results as CSV output to the console which I redirected to files that I then imported into Excel. As a result of this exercise I already have in mind some enhancements to Smoketest to make such comparative testing easier in the future.
For those who have no interest in the complete raw test results, here’s a capture of the summary tab showing the actual results data for Delphi 2009 and relative comparison of those results with those for Delphi 7 and Delphi 2007 (the result data for which is on separate sheets in the workbook).
The colour scheme should be fairly obvious, and follows a gradient scale from red (worse performance) to green (improved performance). The third column of comparisons shows the relative performance in Delphi 2009 itself of ANSIString and WideString against String (i.e. UnicodeString).
First, The Good News
UnicodeString
On the basis of the tested operations, Unicode string performance appears – with some exceptions – to be generally only slightly slower than for ANSI strings in Delphi 7. The gap between Delphi 2009 and 2007 is greater however, presumably because Delphi 2007 incorporates improvements over Delphi 7 as Andreas had suggested.
Unsurprisingly it is char-wise operations that suffer most – Copy() and Pos() for example.
ANSIString
Some surprising extremes here – some things are appreciably faster than Delphi 7, but some crucial operations are quite significantly slower and in some cases I’m at a loss to explain why. The lack of an ANSI version of IntToStr() hurts badly if you are using ANSIString explicitly.
WideString
No real surprises here. WideString is slow and performance isn’t really any different in Delphi 2009 compared to either Delphi 7 or Delphi 2007.
Two notable exceptions to this are WideChar indexing into a WideString and the Replace() function, both of which do seem to be improved compared to Delphi 7, but not Delphi 2007. i.e. these improvements came in Delphi 2007 and remain in Delphi 2009, rather than being a Delphi 2009 improvement per se.
FastStrings
Not highlighted in the above image, but available in the test result data, the comparative performance of FastStrings against the RTL routines in Delphi 7 and Delphi 2007 confirmed something that Andreas had suggested – FastStrings is no longer as generally useful as it once was.
Admittedly my tests only exercised FastPos() and FastReplace() as these are the routines most relevant to myself. Of these FastPos() has clearly been superceded by improvements in the RTL, but not so FastReplace(), which is still twice as fast as the RTL StringReplace() routine (but the FastStrings version is admittedly slightly more cumbersome to invoke).
The Not-So Good News
As mentioned, overall UnicodeString performance is comparable to string handling performance in Delphi 7 and somewhat less efficient than Delphi 2007.
There are however a number of cases where the string handling in Delphi 2009 is not just slightly, but significantly worse.
The RTL Copy() routine is the most obvious and possibly significant example, being fully three times slower than the Delphi 2007 implementation.
The Pos() routine seems to have suffered quite badly in Delphi 2009, being only half as efficient as that in Delphi 2007.
The Bad News – ANSI Strings
When concerns about the impact of Unicode on applications has been raised, one suggestion has been to “ANSIfy” any necessary code. That is, make string (and char etc) declarations explicitly ANSI in order to maintain previous ANSI string behaviour in those areas of an application where is is felt necessary.
Unfortunately if performance is a factor in such cases, this might not fly.
Conclusion
First it should go without saying that the significance of any of these results to you will vary according to your application needs.
For myself the findings were predominantly encouraging.
My two greatest concerns with the Unicode implementation were performance and memory footprint. Having adopted UTF16, there is simply no avoiding the memory footprint issue, but it seems that the performance issue is – to a large extent – not a significant concern.
But this is does not seem to be necessarily a universal truth.
If performance of string handling code is crucial to your application then it would perhaps be advisable to specifically test your implementation approach and ensure you are getting the most from the compiler. When it was a simple question of ANSIString vs WideString it was pretty easy.
With UnicodeString and ANSIString it could be more complex.
For my own part, I am no longer too concerned about the FastStrings situation in Delphi 2009, although I might ANSIfy the FastStrings interfaces for my own use just in case, particularly FastReplace().
The project for which these concerns were most relevant for me is currently still in Delphi 5, so I suspect that an eventual migration to Delphi 2009 will actually show an overall improvement, relative to that Delphi 5 base (even though we already use FastStrings and FastMM).
Parting Thought:
- Whereforeart thou TStringBuilder?
The results for a simple string concatenation using TStringBuilder were staggering. You will note from the test code that I was careful to not muddy the test results with construction and destruction of the TStringBuilder itself, but of course in real world usage some additional overhead is bound to be incurred by the need to erect and tear down TStringBuilder instances as needed.
I haven’t tested the advantage that TStringBuilder delivers in Delphi.NET, and if colleagues experiences with the C# equivalent are anything to go by, it could be huge. But this simply doesn’t seem to apply on the Win32 side and in fact TStringBuilder seems to exact nothing but penalties – it results in harder to read code and seems to be slower – dramatically so – than “raw” string operations.
Certainly, if you were to use TStringBuilder with performance in mind, it looks like you could be making a terrible mistake, although the test in this exercise hardly constituted a comprehensive test of all TStringBuilder functionality. Perhaps there are other operations where it is faster.
But it looks to me as if TStringBuilder is there primarily as a .NET compatibility fixture, rather than to provide any real benefit to developers of Win32 applications, with the possible exception of developers wishing or needing to single-source a Win32/.NET codebase where string handling performance isn’t a concern.
