For those of you who don't know what a standard deviation (SD) is, it is a measure of how much your data deviates from the expected value (average value). Standard deviation and expected value are the statistical terms for describing the bell curve that represents your data. A low expected value means your sample data is very close to the center. A high expected value means your data varies heavily. In the context of process runtime, it could mean your process is not very repeatable or reliable. One way to improve a process is to reduce the standard deviation of the process. By knowing the standard deviation, it can be easier to identify the process exeuctions that deviate heavily from the expected value. These outliers tend to be the executions that cause the most problems. The Standard deviation Wikipedia page has some graphs that explain it farther and more background information.
The most common method of calculating the SD is: $$\sqrt{E(x^2)-E(x)^2}$$. E(y) is the average value of y. Since you need to calculate the $$E(x^2)$$, I have heard people say that you need to know every value of x to calculate the SD of x. For the $$E(x)^2$$ term, you can keep a running sumation of x and the count, then divide the summed x by the counter to get the average. I actually had a developer use that as an excuse on why he couldn't calculate the SD in real time. The begs the question, why can't you do the same trick with x^2. In reality, you can keep a running summation of your count, your value and the square of your value. With these three pieces of information, you can calculate the SD and average of your dataset.
// example SD calculation
// we have an Enumeration. We don't know all
// the values at once
final Enumeration< Double > e = getEnumeration();
// declare our 3 sumation variables
double sum_value = 0.0;
double sum_value_2 = 0.0;
int count = 0;
// iterate, only knowing a single 'value' at one time
while ( e.hasMoreElements() )
{
final double value = e.nextElement();
// sumate
count++;
sum_value += value;
sum_value_2 += value * value;
}
// calculate the standard deviation
final double avg = sum_value / count;
final double avg_2 = sum_value_2 / count;
final double variance = avg_2 - avg;
final double sd = Math.pow( variance, 0.5 );
