How do I display 400,000 or more points in Openlayers 3 using less than 200MB of memory?
Asked Answered
I

2

7

I created a standalone map to test this out for myself. I took a heap snapshot, using Chrome Developer Tools, of the page upon loading it and found it was using 882MB of memory. I'm looking to plot about an hours worth of lightning data and I would like for the user to be able to interact with it so Openlayers makes sense here. However its taking up a ton of memory and need a solution that is much more memory efficient.

Below is the code I used to do this:

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="chrome=1">
    <meta name="viewport" content="initial-scale=1.0, user-scalable=no, width=device-width">
    <script src="https://cdnjs.cloudflare.com/ajax/libs/ol3/3.6.0/ol.js"></script>
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/ol3/3.6.0/ol.css">
  </head>
  <body>

    <div class="container">

      <div class="row">
        <div class="md-12">
          <div id="map" class="map"></div>
        </div>
      </div>
      <div id="span12"> 
      </div>
    </div>


    <script>
var iconInfo = [{
    points: 4,
    radius: 3,
    radius2: 0,
    angle: 0
}, {
    points: 4,
    radius: 3,
    radius2: 0,
    angle: 0
}, {
    points: 4,
    radius: 3,
    radius2: 0,
    angle: 0
}, {
    points: 4,
    radius: 3,
    radius2: 0,
    angle: 0
}];

var i;

var iconCount = iconInfo.length;
var icons = new Array(iconCount);
for (i = 0; i < iconCount; ++i) {
  var info = iconInfo[i];
  icons[i] = new ol.style.RegularShape({
    points: info.points,
    radius: info.radius,
    radius2: info.radius2,
    angle: info.angle,
    fill: new ol.style.Fill({color: 'rgba(0, 0, 0, 0.9)'}),
    stroke: new ol.style.Stroke({width: 2, color: 'rgba(0, 0, 0, 0.9)'}),
  });
}

var featureCount = 350000;
var features = new Array(featureCount);
var feature, geometry;
var e = 25000000;
for (i = 0; i < featureCount; ++i) {
  geometry = new ol.geom.Point(
      [2 * e * Math.random() - e, 2 * e * Math.random() - e]);
  feature = new ol.Feature(geometry);
  feature.setStyle(
      new ol.style.Style({
        image: icons[i % (iconCount - 1)]
      })
  );
  features[i] = feature;
}

var vectorSource = new ol.source.Vector({
  features: features
});
var vector = new ol.layer.Vector({
  source: vectorSource
});


var map = new ol.Map({
  layers: [vector],
  target: document.getElementById('map'),
  view: new ol.View({
    center: [0, 0],
    zoom: 5
  })
});

var overlayFeatures = [];
for (i = 0; i < featureCount; i += 30) {
  var clone = features[i].clone();
  clone.setStyle(null);
  overlayFeatures.push(clone);
}

var featureOverlay = new ol.layer.Vector({
  map: map,
  source: new ol.source.Vector({
    features: overlayFeatures
  }),
  style: new ol.style.Style({
    image: icons[iconCount - 1]
  })
});

map.on('click', function(evt) {
  var info = document.getElementById('info');
  info.innerHTML =
      'Hold on a second, while I catch those butterflies for you ...';

  window.setTimeout(function() {
    var features = [];
    map.forEachFeatureAtPixel(evt.pixel, function(feature, layer) {
      features.push(features);
      return false;
    });

    if (features.length === 1) {
      info.innerHTML = 'Got one butterfly';
    } else if (features.length > 1) {
      info.innerHTML = 'Got ' + features.length + ' butterflies';
    } else {
      info.innerHTML = 'Couldn\'t catch a single butterfly';
    }
  }, 1);
});

map.on('pointermove', function(evt) {
  if (evt.dragging) {
    return;
  }
  var pixel = map.getEventPixel(evt.originalEvent);
  var hit = map.hasFeatureAtPixel(pixel);
  map.getTarget().style.cursor = hit ? 'pointer' : '';
});

</script>
</body>
</html>

Any suggestions on how I could achieve better memory efficiency?

Irritative answered 16/7, 2015 at 17:1 Comment(0)
T
13

Short answer

OpenLayers 3 uses about 2 kB per Point feature (see below), so while there are some optimizations possible you have to keep the number of features down. 400 000 features will require about 800 MB of memory.

Load your features dynamically, or use MultiPoint geometries.

Move the style from the geometry to the layer.

Longer answer

Style

When i tested, removing the style from the feature and replacing it with a simple property reduced the memory footprint by 290 B per feature. See http://jsfiddle.net/vkm2rg46/3/:

var vector = new ol.layer.Vector({
    source: vectorSource,
    style: function (feature, resolution) {
        var i = feature.getProperties().styleId;
        return [new ol.style.Style({
            image: icons[i]
        })];
    }
});

and to help the style function:

feature.set('styleId', i % (iconCount - 1));

Spatial index

You could set useSpatialIndex to false on the vector source. The source keep a spatial index to quickly retrieve features within a given extent, which seems to need about 200-250 bytes per feature. However, removing the index could have bad performance consequences with this amount of features.

Reduce feature count##

Your best bet is probably to load fewer features. There are several solutions to this.

Load on demand

It's most commonly solved by letting the server take care of the data, and dynamically load it when needed. You probably don't want to display 400 000 points at lower zoom levels, and the users wont pan everywhere.

This could be done by vector tiles or with a normal vector source using a bbox or tile.

It could also be done client side, by creating features/geometries from your own dataset in the vector source loader function.

Multipoints

A MultiPoint geometry with 10 or 100 points hardly take any more space than a single Point geometry. If you group you lightning strikes into MultiPoint geometries, memory could be a non-issue. You would however loose some semantics, and the possibility to attach metadata to each single point.

JsFiddle: http://jsfiddle.net/vkm2rg46/8/

Memory usage

I created http://jsfiddle.net/g7qduy1w/3/ to test the memory use of the geometry, features and source. You can take snapshot at the different stages (notably, the event listening data increases when adding a geometry to a feature, and a feature to a source). With a simple point geometry added to a feature without extra properties, and added to a source, the memory use per feature is: 288 B geometry event listener
424 B rest of geometry data
752 B feature event listeners
184 B rest of feature data
261 B source (share of total memory using 100 000 features)

Tieck answered 23/7, 2015 at 22:32 Comment(0)
G
4

Have a look at this example from camptocamp

https://www.camptocamp.com/en/actualite/drawing-large-amounts-of-points-with-openlayers-3-and-webgl

OpenLayers Symbols with WebGL:

http://openlayers.org/en/master/examples/symbol-atlas-webgl.html

It displays 100k points very efficiently!

Greenebaum answered 24/7, 2015 at 11:56 Comment(1)
Yes WebGL is the way to go. Also avoid transparency and the fancy stuff.Lugworm

© 2022 - 2024 — McMap. All rights reserved.