Scripting Performance

Alex Angelopoulos (aka at mvps dot org)

How I DON'T Measure Script Performance

I don't compare five different languages based on how long it takes to perform an arbitrary operation.

This approach has merit: it is doable, and it has meaning for certain types of tasks.  In the world of admin scripting, however, it has two problems.  First, it overlooks the key point that most admin scripters are in some sense committed already to a programming paradigm and thus comparing languages does little for them.  Second, it neglects the key fact that performance issues quite often center around scaling - in other words, if I do X to one machine, does it take 10 times as long with 10 machines?  Only twice as  long? Or  100 times as long?

What Metrics Should Be Used?

There are several metrics which might be of value for attempting to assess script performance or "goodness" for achieving a goal. In a broad sense, the metrics might include the following.

Execution Time and Variance

How long does it take a script to execute, start to finish?  Does this vary dependent on task or CPU load?

CPU Usage Profile

What kind of CPU usage happens?

Memory Usage Profile

How much memory is consumed?

Composition Time

Programmer/Administrator time to create a script is one of the most important costs to consider.  How long does it take to write the code for a certain task?

Storage Space

LOC

Terminal Count

Tweakability

Issues In WSH Performance Measurement

Variability

Some Performance Scripts

Savage Benchmark

Modified from a Pascal implementation, this is an interesting script for demonstrating how Moore's Law makes script performance acceptable for many applications which might have been considered hardcore computing in the recent past.  Running on a P3-600, I had execution times which were roughly 35% of the "original" implementation of this code on a P-133 in Pascal in 1997.  Obviously this is slower in terms of execution-time/per-CPU-speed

Tests of Internal Constructs in VBScript

Retrieving a Class Property Value

Tests