Simplification / optimization of GPS track

Question

I've got a GPS track produced by gpxlogger(1) (supplied as a client for gpsd). GPS receiver updates its coordinates every 1 second, gpxlogger's logic is very simple, it writes down location (lat, lon, ele) and a timestamp (time) received from GPS every n seconds (n = 3 in my case).

After writing down a several hours worth of track, gpxlogger saves several megabyte long GPX file that includes several thousands of points. Afterwards, I try to plot this track on a map and use it with OpenLayers. It works, but several thousands of points make using the map a sloppy and slow experience.

I understand that having several thousands of points of suboptimal. There are myriads of points that can be deleted without losing almost anything: when there are several points making up roughly the straight line and we're moving with the same constant speed between them, we can just leave the first and the last point and throw away anything else.

I thought of using gpsbabel for such track simplification / optimization job, but, alas, it's simplification filter works only with routes, i.e. analyzing only geometrical shape of path, without timestamps (i.e. not checking that the speed was roughly constant).

Is there some ready-made utility / library / algorithm available to optimize tracks? Or may be I'm missing some clever option with gpsbabel?

Breadwinner

Answer 1

Yes, as mentioned before, the Douglas-Peucker algorithm is a straightforward way to simplify 2D connected paths. But as you have pointed out, you will need to extend it to the 3D case to properly simplify a GPS track with an inherent time dimension associated with every point. I have done so for a web application of my own using a PHP implementation of Douglas-Peucker.

It's easy to extend the algorithm to the 3D case with a little understanding of how the algorithm works. Say you have input path consisting of 26 points labeled A to Z. The simplest version of this path has two points, A and Z, so we start there. Imagine a line segment between A and Z. Now scan through all remaining points B through Y to find the point furthest away from the line segment AZ. Say that point furthest away is J. Then, you scan the points between B and I to find the furthest point from line segment AJ and scan points K through Y to find the point furthest from segment JZ, and so on, until the remaining points all lie within some desired distance threshold.

This will require some simple vector operations. Logically, it's the same process in 3D as in 2D. If you find a Douglas-Peucker algorithm implemented in your language, it might have some 2D vector math implemented, and you'll need to extend those to use 3 dimensions.

You can find a 3D C++ implementation here: 3D Douglas-Peucker in C++

Your x and y coordinates will probably be in degrees of latitude/longitude, and the z (time) coordinate might be in seconds since the unix epoch. You can resolve this discrepancy by deciding on an appropriate spatial-temporal relationship; let's say you want to view one day of activity over a map area of 1 square mile. Imagining this relationship as a cube of 1 mile by 1 mile by 1 day, you must prescale the time variable. Conversion from degrees to surface distance is non-trivial, but for this case we simplify and say one degree is 60 miles; then one mile is .0167 degrees. One day is 86400 seconds; then to make the units equivalent, our prescale factor for your timestamp is .0167/86400, or about 1/5,000,000.

If, say, you want to view the GPS activity within the same 1 square mile map area over 2 days instead, time resolution becomes half as important, so scale it down twice further, to 1/10,000,000. Have fun.

Paddle

If you are looking for a PHP class, that does this, here are 2 that I found: github.com/david-r-edgar/RDP-PHP github.com/gregallensworth/PHP-Geometry Here you can test your file simplification online, play with the config and instantly see the results and changes: labs.easyblog.it/maps/gpx-simplify-optimizer – Tosh

Answer 2

Have a look at Ramer-Douglas-Peucker algorithm for smoothening complex polygons, also Douglas-Peucker line simplification algorithm can help you reduce your points.

Silicosis

Thanks, I thought that there must be already some algorithm available, just missed it! Am I right that the only thing I need here is adaptation of this algorithm in 3D space (i.e. ''(x, y, t)'' coordinates) and somehow define "orthogonal distance" in such space? – Breadwinner

Yeah you will need to map the time parameter somehow :) – Silicosis

Answer 3

You want to throw away uninteresting points. So you need a function that computes how interesting a point is, then you can compute how interesting all the points are and throw away the N least interesting points, where you choose N to slim the data set sufficiently. It sounds like your definition of interesting corresponds to high acceleration (deviation from straight-line motion), which is easy to compute.

Appaloosa

I don't know my target number of points a priori. But generally, yes, the fairly simple solution is to have a simple metric for each point of how exactly interesting it is (preferably based on a sliding window of ''k'' points before it and ''k'' points after it). Do you have an idea how this metric would probably look like - I've experimented a bit and I can't come up with one? – Breadwinner

Let the point of interest be x_2, the previous point be x_1 and the next point be x_3. Let the times at which you were at those points be t_1, t_2, t_3. Then the velocity before the point is v_12 = (x_2 - x_1) / (t_2 - t_1) and the velocity after is v_23 = (x_3 - x_2) / (t_3 - t_2). The acceleration vector is a = (v_23 - v_13) / (t_3 - t_1). You would want to use the magnitude of that vector, |a|. – Appaloosa

That would be an approximation of instantaneous acceleration, not a weighted average acceleration in a sliding window. I'm not sure that will work - i.e. it will reveal exactly non-interesting points but guarantee that all interesting points would stay. – Breadwinner

Answer 4

OpenSource GeoKarambola java library (no Android dependencies but can be used in Android) that includes a GpxPathManipulator class that does both route & track simplification/reduction (3D/elevation aware). If the points have timestamp information that will not be discarded. https://sourceforge.net/projects/geokarambola/

This is the algorith in action, interactively https://lh3.googleusercontent.com/-hvHFyZfcY58/Vsye7nVrmiI/AAAAAAAAHdg/2-NFVfofbd4ShZcvtyCDpi2vXoYkZVFlQ/w360-h640-no/movie360x640_05_82_05.gif

This algorithm is based on reducing the number of points by eliminating those that have the greatest XTD (cross track distance) error until a tolerated error is satisfied or the maximum number of points is reached (both parameters of the function), wichever comes first.

An alternative algorithm, for on-the-run stream like track simplification (I call it "streamplification") is: you keep a small buffer of the points the GPS sensor gives you, each time a GPS point is added to the buffer (elevation included) you calculate the max XTD (cross track distance) of all the points in the buffer to the line segment that unites the first point with the (newly added) last point of the buffer. If the point with the greatest XTD violates your max tolerated XTD error (25m has given me great results) then you cut the buffer at that point, register it as a selected point to be appended to the streamplified track, trim the trailing part of the buffer up to that cut point, and keep going. At the end of the track the last point of the buffer is also added/flushed to the solution. This algorithm is lightweight enough that it runs on an AndroidWear smartwatch and gives optimal output regardless of if you move slow or fast, or stand idle at the same place for a long time. The ONLY thing that maters is the SHAPE of your track. You can go for many minutes/kilometers and, as long as you are moving in a straight line (a corridor within +/- tolerated XTD error deviations) the streamplify algorithm will only output 2 points: those of the exit form last curve and entry on next curve.

Appendicitis

Answer 5

I ran in to a similar issue. The rate at which the gps unit takes points is much larger that needed. Many of the points are not geographically far away from each other. The approach that I took is to calculate the distance between the points using the haversine formula. If the distance was not larger than my threshold (0.1 miles in my case) I threw away the point. This quickly gets the number of points down to a manageable size.

I don't know what language you are looking for. Here is a C# project that I was working on. At the bottom you will find the haversine code.

http://blog.bobcravens.com/2010/09/gps-using-the-netduino/

Hope this gets you going.

Bob

Tried

Thanks, but your example is much simpler than what I need. If I understand your code correctly, you just filter out points that doesn't differ much than a given distance from a previous one. I want to simplify more, by checking that we're maintaining the same constant speed for some time. Last, but not least, you're using Haversine formula, while it's better to use Vincenty's formula - OpenLayers includes implementation of that one. Thanks for the answer anyway :) – Breadwinner

@Breadwinner Why is it better to use Vicinity's formula rather than Haversine? – Eckart

Haversine is much less accurate: it assumes that planet is sphere, while Vincenty assumes oblate spheroid, which is much closer to what Earth really is. – Breadwinner

@Breadwinner Accuracy is almost irrelevant for this context. You don't need accuracy when you're already discarding most of your data in the first place :) Over the tiny distances (sub 1 mile) we're talking about for simplifying GPS tracks, does it really matter if the point is a few feet out when deciding if the track point is needed or not? Haversine is faster... – Eckart

True, but if we'll return to the context of original problem, we'll end up not using any of them anyway - in fact, just using polygon coordinates + time in Douglas-Peucker 3D algorithm yields great results. – Breadwinner

For points that are really close to each the other the haversine formula is absolutely okay to calculate distances. You could even use a simple Pythagoras for this. mkompf.com/gps/distcalc.html – Jaquiss

Answer 6

This is probably NP-hard. Suppose you have points A, B, C, D, E.

Let's try a simple deterministic algorithm. Suppose you calculate the distance from point B to line A-C and it's smaller than your threshold (1 meter). So you delete B. Then you try the same for C to line A-D, but it's bigger and D for C-E, which is also bigger.

But it turns out that the optimal solution is A, B, E, because point C and D are close to the line B-E, yet on opposite sides.

If you delete 1 point, you cannot be sure that it should be a point that you should keep, unless you try every single possible solution (which can be n^n in size, so on n=80 that's more than the minimum number of atoms in the known universe).

Next step: try a brute force or branch and bound algorithm. Doesn't scale, doesn't work for real-world size. You can safely skip this step :)

Next step: First do a determinstic algorithm and improve upon that with a metaheuristic algorithm (tabu search, simulated annealing, genetic algorithms). In java there are a couple of open source implementations, such as Drools Planner.

All in all, you 'll probably have a workable solution (although not optimal) with the first simple deterministic algorithm, because you only have 1 constraint.

A far cousin of this problem is probably the Traveling Salesman Problem variant in which the salesman cannot visit all cities but has to select a few.

Ellett

If we strive to find an really optimal solution (with a given target number of points that should be left), then yes, it's probably NP-hard, but ''good enough'' solution is fine for me. Basically, it boils down to finding a good enough approximation of a line in a 3D space of ''(x, y, t)'', composed using straight segments. – Breadwinner

Answer 7

Try this, it's free and opensource online Service:

https://opengeo.tech/maps/gpx-simplify-optimizer/

Pensive

Could you explain where can I find sources for that service? – Breadwinner

Ah, so it's RDP implementation. Thanks! – Breadwinner

yes... fork the github repo code if you want try another algoritms! – Pensive

Answer 8

I guess you need to keep points where you change direction. If you split your track into the set of intervals of constant direction, you can leave only boundary points of these intervals.
And, as Raedwald pointed out, you'll want to leave points where your acceleration is not zero.

Ferrari

Answer 9

Not sure how well this will work, but how about taking your list of points, working out the distance between them and therefore the total distance of the route and then deciding on a resolution distance and then just linear interpolating the position based on each step of x meters. ie for each fix you have a "distance from start" measure and you just interpolate where n*x is for your entire route. (you could decide how many points you want and divide the total distance by this to get your resolution distance). On top of this you could add a windowing function taking maybe the current point +/- z points and applying a weighting like exp(-k* dist^2/accuracy^2) to get the weighted average of a set of points where dist is the distance from the raw interpolated point and accuracy is the supposed accuracy of the gps position.

Vudimir

Answer 10

One really simple method is to repeatedly remove the point that creates the largest angle (in the range of 0° to 180° where 180° means it's on a straight line between its neighbors) between its neighbors until you have few enough points. That will start off removing all points that are perfectly in line with their neighbors and will go from there.

You can do that in Ο(n log(n)) by making a list of each index and its angle, sorting that list in descending order of angle, keeping how many you need from the front of the list, sorting that shorter list in descending order of index, and removing the indexes from the list of points.

def simplify_points(points, how_many_points_to_remove)
  angle_map = Array.new
  (2..points.length - 1).each { |next_index|
    removal_list.add([next_index - 1, angle_between(points[next_index - 2], points[next_index - 1], points[next_index])])
  }
  removal_list = removal_list.sort_by { |index, angle| angle }.reverse
  removal_list = removal_list.first(how_many_points_to_remove)
  removal_list = removal_list.sort_by { |index, angle| index }.reverse
  removal_list.each { |index| points.delete_at(index) }
  return points
end

Harlin