Calculating Standard Deviation of Angles?

Question

So I'm working on an application using compass angles (in degrees). I've managed to determine the calculation of the mean of angles, by using the following (found at http://en.wikipedia.org/wiki/Directional_statistics#The_fundamental_difference_between_linear_and_circular_statistics) :

 double calcMean(ArrayList<Double> angles){
      double sin = 0;
      double cos = 0;
      for(int i = 0; i < angles.size(); i++){
           sin += Math.sin(angles.get(i) * (Math.PI/180.0));
           cos += Math.cos(angles.get(i) * (Math.PI/180.0)); 
      }
      sin /= angles.size();
      cos /= angles.size();

      double result =Math.atan2(sin,cos)*(180/Math.PI);

      if(cos > 0 && sin < 0) result += 360;
      else if(cos < 0) result += 180;

      return result;
 }

So I get my mean/average values correctly, but I can't get proper variance/stddev values. I'm fairly certain I'm calculating my variance incorrectly, but can't think of a correct way to do it.

Here's how I'm calculating variance:

 double calcVariance(ArrayList<Double> angles){

      //THIS IS WHERE I DON'T KNOW WHAT TO PUT
      ArrayList<Double> normalizedList = new ArrayList<Double>();
      for(int i = 0; i < angles.size(); i++){
           double sin = Math.sin(angles.get(i) * (Math.PI/180));
           double cos = Math.cos(angles.get(i) * (Math.PI/180));
           normalizedList.add(Math.atan2(sin,cos)*(180/Math.PI));
      }

      double mean = calcMean(angles);
      ArrayList<Double> squaredDifference = new ArrayList<Double>();
      for(int i = 0; i < normalizedList.size(); i++){
           squaredDifference.add(Math.pow(normalizedList.get(i) - mean,2));
      }

      double result = 0;
      for(int i = 0; i < squaredDifference.size(); i++){
           result+=squaredDifference.get(i);
      }

      return result/squaredDifference.size();
 }

While it's the proper way to calculate variance, I'm not what I'm supposed to use. I presume that I'm supposed to use arctangent, but the standard deviation/variance values seem off. Help?

EDIT: Example: Inputting the values 0,350,1,0,0,0,1,358,9,1 results with the average angle of 0.0014 (since the angles are so close to zero), but if you just do a non-angle average, you'll get 72...which is way off. Since I don't know how to manipulate individual values to be what they should be, the variance calculated is 25074, resulting in a standard deviation of 158 degrees, which is insane!! (It should only be a few degrees) What I think I need to do is properly normalize individual values so I can get correct variance/stddev values.

Lemuellemuela

I did not analyze fully, but this code seems to need Math.atan2(y,x) – Carreno

@Carreno - I had done this originally (and have put it back in recently) and the results are the same. I tried the method above along with atan2 and the results I get are the same within 12 or 13 orders of magnitude. – Lemuellemuela

EDIT: Looks like using atan2 addresses Chechulin's post. I'll edit my question. – Lemuellemuela

Answer 1

By the Wikipedia page you link to the circular standard deviation is sqrt(-log R²), where R = |mean of samples|, if you consider the samples as complex numbers on the unit circle. So the calculation of standard deviation is very similar to the calculation of the mean angle:

double calcStddev(ArrayList<Double> angles){
      double sin = 0;
      double cos = 0;
      for(int i = 0; i < angles.size(); i++){
           sin += Math.sin(angles.get(i) * (Math.PI/180.0));
           cos += Math.cos(angles.get(i) * (Math.PI/180.0)); 
      }
      sin /= angles.size();
      cos /= angles.size();

      double stddev = Math.sqrt(-Math.log(sin*sin+cos*cos));

      return stddev;
 }

And if you think about it for a minute it makes sense: When you average a bunch of points close to each other on the unit circle the result is not too far off from the circle, so R will be close to 1 and the stddev near 0. If the points are distributed evenly along the circle their average will be close to 0, so R will be close to 0 and the stddev very large.

Cicala

@Cicala - I'm not so mathematically inclined. To calculate the standard deviation using the variance here, I'd use "double stddev = Math.sqrt(Math.log(1/Math.pow(Math.sqrt(sinsin + coscos), 2)))*180/Math.PI;" ? From the wiki page, it seems like you get R the same way but you won't be subtracting from 1. – Lemuellemuela

You can simplify by removing the square and square root and you get stddev = sqrt(-log(sin*sin+cos*cos))*180/pi – Cicala

Mind the units. The function as written takes angles in degrees as input and returns the standard deviation in radians. – Idola

Caution! Circular standard deviation is not an angular quantity! Its values range from 0 to infinity, so "converting to/from radians" doesn't make any sense, and will incorrectly scale results. – Fracture

Current good way to deal with this is through scipy already implemented functions, Cf my answer below : https://mcmap.net/q/927717/-calculating-standard-deviation-of-angles – Jonellejones

Answer 2

When you use Math.atan(sin/cosine) you get an angle between -90 and 90 degrees. If you have 120 degrees angle, you get cos=-0.5 and sin=0.866, then you get atan(-1.7)=-60 degrees. Thus you put wrong angles in your normalized list.

Assuming that variance is a linear deviation, I'd recommend you to rotate your angles array by the -calcMean(angles) and add/subtract 360 to/from angles above/below 180/-180 (damn my writing!)) while finding maximum and minimum angle. It will give you desired deviations. Like this:

    Double meanAngle = calcMean(angles)
    Double positiveDeviation = new Double(0);
    Double negativeDeviation = new Double(0);
    Iterator<Double> it = angles.iterator();
    while (it.hasNext())
    {
        Double deviation = it.next() - meanAngle;
        if (deviation > 180) deviation -= 180;
        if (deviation <= -180) deviation += 180;
        if (deviation > positiveDeviation) positiveDeviation = deviation;
        if (deviation > negativeDeviation) negativeDeviation = deviation;
    }
    return positiveDeviation - negativeDeviation;

For average squared deviations you should use your method (with angles, not "normalized" ones), and keep looking for (-180, 180) range!

Letitialetizia

Using atan2 addresses the issues brought up in this answer. It doesn't change the variance results I get though. Thanks. – Lemuellemuela

do you check for normalizedList.get(i) - mean to be in -180:180 range? Because if you have it be 300 it means that you should treat it as a -60. – Letitialetizia

it looks like that may have done the trick. I'm not positive but I'll check more closely when I get the chance....or I'll check future replies. Thanks! – Lemuellemuela

Your code seems to calculate the difference between the smallest and largest absolute difference in angles. This is not equivalent to the mathematical definition of the standard deviation. – Dylane

Answer 3

The math library function remainder is handy for dealing with angles.

A simple change would be to replace

normalizedList.get(i) - mean

with

remainder( normalizedList.get(i) - mean, 360.0)

However your first loop is then redundant, as the call to remainder will take care of all the normalisation. Moreover it's simpler just to sum up the squared differences, rather than store them. Personally I like to avoid pow() when arithmetic will do. So your function could be:

double calcVariance(ArrayList<Double> angles){
 double mean = calcMean(angles);

  double result = 0;
  for(int i = 0; i < angles.size(); i++){
   double diff = remainder( angles.get(i) - mean, 360.0);
        result += diff*diff;
  }

  return result/angles.size();
 }

Cordierite

Answer 4

The current good way to deal with this is now the two functions already implemented in scipy :

• circmean : https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.circmean.html
• circstd : https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.circstd.html

Couple of great things included :

• vectorization for fast computing
• nan dealing
• high, low thresholds, typically for angles between 0 and 360 degrees vs between 0 and 2 Pi.

Jonellejones

Answer 5

The accepted answer by Joni does an excellent job at answering this question, but as Brian Hawkins noted:

Mind the units. The function as written takes angles in degrees as input and returns the standard deviation in radians.

Here's a version that fixes that issue by using degrees for both its arguments and its return value. It also has more flexibility, as it allows for a variable number of arguments.

public static double calcStdDevDegrees(double... angles) {
    double sin = 0;
    double cos = 0;
    for (int i = 0; i < angles.length; i++) {
        sin += Math.sin(angles[i] * (Math.PI/180.0));
        cos += Math.cos(angles[i] * (Math.PI/180.0)); 
    }
    sin /= angles.length;
    cos /= angles.length;

    double stddev = Math.sqrt(-Math.log(sin*sin+cos*cos));

    return Math.toDegrees(stddev);
}

Urtication

Circular standard deviation is not an angular quantity! Its values range from 0 to infinity, so "converting to/from radians" doesn't make any sense, and will incorrectly scale results. – Fracture