Weird behaviour of the magnetic field measurements while rotating the device

Question

The situation:

I'm trying to create an indoor localisation system with the use of the magnetic field variations inside a building via fingerprinting. As a sensor I'm using my mobile phone a Galaxy Nexus.

Basically the idea to get my fingerprints is to measure the magnetic field in x,y and z direction respectively and rotate the vector by the orientation of the mobile to bring the vector into the world coordinate system. To locate someone he also measures the magnetic field, rotates the vector and send it to the server where I'll run my locating algorithms.

My problem:

When I measure the magnetic field at the same spot in different orientations I don't get the same result after rotating the vector. Even the field strength isn't the same.

To show this I have made four measurements at the same position and rotated the mobile around his z axis (standard Android specification) by 90, 180 and 270 degrees counter-clockwise respectively. Results can be seen on that 'table' and on the Matlab plots (follow link) below (if you want to have other views let me know)

Edit: To be more precise: What I really did was that I defined my own coordinate system called lets say the room coordinate system. I took one wall of the room and defined it as north. The y-axis of my phone aliens with that wall (at the first position) while it was flat on the table. Therefore the positive z-axis points to the sky and the x-axis is defined as the walls in 90 degrees. The vectors named with 'Raw' are the raw values of the phone and so they are in the coordinate system of the phone. The Vector named with 'Rotated' are rotated into that 'room coordinate system'. In the plot the dotted vectors are the raw vectors and the solid ones are the rotated ones.

╔═════════════╦════════╦════════╦════════╦══════════╗
║ Vector      ║    X   ║   Y    ║    Z   ║ Strength ║
╠═════════════╬════════╬════════╬════════╬══════════╣
║ 0°Raw       ║  17.55 ║   8.21 ║ -36.83 ║  41.62   ║
║ 0°Rotated   ║  17.55 ║   8.21 ║ -36.83 ║  41.62   ║
║ 90°Raw      ║  15.74 ║ -11.36 ║ -36.86 ║  41.66   ║
║ 90°Rotated  ║  11.36 ║  15.74 ║ -36.86 ║  41.66   ║
║ 180°Raw     ║ -65.06 ║ -16.18 ║ -44.79 ║  80.63   ║
║ 180°Rotated ║  65.06 ║  16.18 ║ -44.79 ║  80.63   ║
║ 270°Raw     ║   6.38 ║  61.95 ║ -46.37 ║  77.64   ║
║ 270°Rotated ║  61.95 ║  -6.38 ║ -46.37 ║  77.64   ║
╚═════════════╩════════╩════════╩════════╩══════════╝

As you can see even after rotating them they doesn't really match. Do you have any idea why this could be? Anything I might haven't considered yet? Thanks a lot in advance!

Edit2:

Second experiment

I did another experiment with way better results but still not as good as I expected. This time the direction is not as constant as it should be. This time I did my measurements outdoor. I aliened the y-axis to the true north an put it flat on the ground. I rotated it roughly 3 times 90 degrees counter-clockwise. I took with another camera a picture of the position of the mobile and measured the real rotation with photoshop. The results you can see on the table and on the plots below:

╔═════════════╦════════╦════════╦════════╦══════════╗
║ Vector      ║    X   ║   Y    ║    Z   ║ Strength ║
╠═════════════╬════════╬════════╬════════╬══════════╣
║ 0°Raw       ║   1.22 ║  20.31 ║ -57.53 ║  61.02   ║
║ 0°Rotated   ║   1.22 ║  20.31 ║ -57.53 ║  61.02   ║
║ 104°Raw     ║  25.92 ║   5.66 ║ -57.53 ║  63.35   ║
║ 104°Rotated ║ -11.93 ║  23.70 ║ -57.53 ║  63.35   ║
║ 204°Raw     ║   5.42 ║ -16.86 ║ -59.67 ║  62.24   ║
║ 204°Rotated ║ -11.90 ║  13.12 ║ -59.67 ║  62.24   ║
║ 290°Raw     ║ -12.18 ║   0.73 ║ -57.85 ║  59.12   ║
║ 290°Rotated ║  -3.64 ║  11.64 ║ -57.85 ║  59.12   ║
╚═════════════╩════════╩════════╩════════╩══════════╝

Additional notes:

The values are averaged over a couple of seconds. So it's not a problem of noise.

I know I can't use the function getOrientation because it uses the magnetic field to calculate the orientation and that wouldn't make sense since I want to find the variations of the magnetic field. For the example above I created the rotation quaternion manually.

Strangles

Where do you test this? Indoor? outdoor? What coordinate system are you using for the number above? device coordinate system? – Unguentum

Its tested indoor. The values marked with 'Raw' are the values directly from the device, so in the coordinate system of the device. I defined the first vector to be 0 degrees (only the yaw) and rotated all of the others in the respect of that direction. So I kinda brought them into the world coordinate system (marked with 'rotated') – Strangles

The first 4 rows look OK, the last 4 seems to wrong and may be due to magnetic interference. You should test outdoor and see if there is any difference first. The values in row 4 seems right to me. – Unguentum

Shouldn't it be regardless of whether its indoor or outdoor? I mean as long as the position is the same it should be the same magnetic field regardless of the orientation of the sensor. Only the direction should change. If there is an interference (which probably is since its indoor) it should stay the same at this spot. Or am I wrong? – Strangles

I am not sure, the magnatic field reverse direction but should not double the field strength. – Unguentum

It kinda looks like the sensor is more sensitive in some directions – Strangles

That why It does not hurt to try outdoor. I does not make sense that you have to test outdoor, but one never know. Also, you should convert to world coordinate system to see how far off the values are. – Unguentum

I'll try it outdoor tomorrow and post my results. The rotated values are in the world coordinate system. My mobile was lying flat on the table and I only rotated it around the z axis (yaw). So I'm already in the world coordinate system like Android defined it (Link) The only little trick I used is that I defined the north somewhere (the yaw of the first vector). But as long as I put all other vectors into the same coordinate system it doesn't matter. If I would know where the true north is, all rotated vectors would be rotated the same way – Strangles

No your are not in the world coordinate system, the value returns are with respect to the device y, x and z axis. So if you rotate your device the y and x values will change. Let say the magnetic vector does not change when you rotate, but the coordinate system change so that the values returned would change. – Unguentum

I know. That's why I rotated the vectors. Vectors named by 'raw' are the raw values from the device in the coordinate system of the device. After rotating them by the rotation of the device you get a vector in the world coordinate system which I named 'rotated' Take a look at the plots I made with Matlab. Rotated are solid and raw values are dotted. To distinguish the vectors I coloured them in the same colour. – Strangles

Something is very wrong then, the magnetic vector is suppose to lie on the y-z plane. You have very large x-value, huge magnetic interference. – Unguentum

Line your phone up so that the y axis point toward North and see what the x-value is. – Unguentum

Tomorrow I'll go out and try the same somewhere where I don't have much magnetic impact. Also first I need to find the true north :D I'll update my post as soon as possible. Since then thanks! – Strangles

Answer 1

Make sure that you're monitoring the accuracy of the sensors via the onAccuracyChanged(Sensor sensor, int accuracy) method. I find that with my Nexus 10 tablet, the accuracy regularly changes, and when it's low or unreliable, I get results similar to yours.

Trapp

The first little test showed that you are right. It changes regularly. So that might be the problem. Tomorrow or the day after I'll test everything only with high accuracy data. I'll let you know about the results. Thank you! – Strangles

Thank you very much! I think this was the solution! The first tests I've made showed that if I take care using only the sensor data with highest accuracy the results are much better! But also it showed that the magnetic field accuracy changes really often and its hard to get good measurements. Do you know how to improve the accuracy of the magnetic field sensor? I mean without moving my device in a 8-shape? Or do you know what causes the bad accuracy? – Strangles

I'm actually in the middle of a complaint to google about exactly this issue. I have a Nexus 10 and I think I've proved to them that the magnetic sensor in the device that I returned was unreliable. Their most recent response said: "The potentially misleading information is the fact that the HAL is reporting SENSOR_STATUS_UNRELIABLE when in fact it should probably be SENSOR_STATUS_ACCURACY_LOW. As soon as the Nexus 10 moves, the accuracy drops to 0 (unreliable). Once the tablet stops moving around, it goes back up to 3(high)." This is still ongoing, so not sure what the final outcome will be! – Trapp

Well that sounds really odd. For my device (Galaxy Nexus) its a bit different. For me it is just really hard to get measurements on SENSOR_STATUS_ACCURACY_HIGH. Usually I'm getting SENSOR_STATUS_ACCURACY_MEDIUM or SENSOR_STATUS_ACCURACY_LOW. Even if I moved the phone in an 8-shape. If I get SENSOR_STATUS_UNRELIABLE it usually turns to MID or LOW quite soon. Is your complaint somewhere available for public? e.g. in a forum? Could be interesting for me to follow. – Strangles

It's just a conversation that evolved over email with [email protected] as a result of me returning the device to them. They sent me a replacement device, but it behaves exactly like the device that I sent back to them! Anyway, I'll try and remember to put a comment here when there's a final conclusion. – Trapp

That would be great! Good luck with you device and many thanks again! – Strangles

FYI, I asked googleplay-support "is there any prospect of me being able to get a Nexus10 which won't switch from high to low/unreliable just because it's moving", and I've finally got the answer "I forwarded on your message. I was told to go ahead and process a remorse return for you, which tells me that unfortunately the answer is no". So I guess there are some problems with Nexus 10 at the moment, although no idea whether it also relates to any other Android products. – Trapp