Adding Markers in Background while Looping through ArrayList

Question

I add Markers to a googleMap from an ArrayList of Objects. There are about 250 markers; I even have to convert them to bitmaps to customize them. It's a pretty resource intensive task. BUT It seriously blocks my UI thread.

Here is how I do it:

    final HashMap<Marker, NearLocation> markerIdMap = new HashMap<Marker, NearLocation>();
    for (final NearLocation result : MainActivity.nearLocationList) {

            // Do all the hard work here

    }

How can I do this somehow dynamically, after the map loads and it populates them as they are produced? I am not sure if I can do that by doing some of the work in the background, then when the marker is complete move it to the UI thread to add.

I know how to do this INDIVIDUALLY with an AsyncTask. Not sure though while I am looping through...

Dardar

Answer 1

As far as I know, adding the markers cannot be done outside the UI thread.

What you can do is perform all the preparations in background (create marker, convertion to bitmap, etc..). To spare the UI thread a bit when adding markers you could zoom in and use https://code.google.com/p/android-maps-extensions/ to only show the visible markers or cluster markers to bring down the amount, though 250 is not alot imo.

Here is a SO answer on the topic: Add markers dynamically on Google Maps v2 for Android

I have an app that has around ~4500 markers running fairly well using the first method (as long as it is not rapidly zoomed out that is). It should be noted that here I have chosen to make a splash screen with a progress bar, preparing all the markers before the user has any chance of opening the map.

If you want a simple mechanism for selecting surrounding markers without using 3rd part library you can do something like this: Android Maps v2 - animate camera to include most markers

An idea that comes to mind just now, if the creation of markers is really that expensive, is to add EventBus https://github.com/greenrobot/EventBus to your project.

Using EventBus you could do a long running preparation of markers inside an onEventAsync() method. Inside that method, whenever a marker is ready, post the new marker to UI EventBus.getDefault().post(marker) and catch it in onEventMainThread(Marker marker). Here you could save the marker in a list of all the prepared markers and if the map is currently open, add it.

Here is some of the code I use to prepare the markers in the app I mentioned earlier. It is used to show hydrants in an app used by the fire dept. At first startup all the hydrants are read from a set of CSV files and MarkerOptions for all ~4500 hydrants are created. Undoubtedly a lot of the code is of no use to you, just leaving it in case there is something that you can benefit from:

private List<HydrantHolder> mHydrants;
private Map<HydrantType, List<MarkerOptions>> mMarkers;

private class ReadHydrantsFiles extends
        AsyncTask<Void, Integer, List<HydrantHolder>> {

    File[] hydrantsFiles = new File[0];

    // Before running code in separate thread
    @Override protected void onPreExecute() {

        loadStarted();

        String filepath = paths.PATH_LOCAL_HYDRANTS;

        File hydrantsPath = new File(filepath);
        hydrantsFiles = hydrantsPath.listFiles(new FilenameFilter() {

            @Override public boolean accept(File dir, String filename) {
                return filename.toLowerCase(Locale.ENGLISH).contains(
                        Constants.FILETYPE_CSV);
            }
        });

        int lineCount = 0;

        if (hydrantsFiles == null || hydrantsFiles.length == 0) {
            if (!preferences.isFirstStartUp()) {
                // TODO notify user
            }
            Log.e(TAG, "Missing hydrants");
            if (moduleCallback != null) {
                moduleCallback.doneLoadingModule(toString());
            }
            this.cancel(false);
        } else {

            for (File file : hydrantsFiles) {
                // store linecount for visual progress update
                lineCount += Files.lineCount(file);

            }

        }

    }

    // The code to be executed in a background thread.
    @Override protected List<HydrantHolder> doInBackground(Void... args) {
        List<HydrantHolder> all_hydrants = new ArrayList<HydrantHolder>();
        // Directory path here
        int totalLineProgress = 0;

        // // required
        int indexLatitude = modulePreferences.indexLatitude();
        int indexLongitude = modulePreferences.indexLongitude();
        // optional
        int indexAddress = modulePreferences.indexAddress();
        int indexAddressNumber = modulePreferences.indexAddressNumber();
        int indexAddressRemark = modulePreferences.indexAddressRemark();
        int indexRemark = modulePreferences.indexRemark();
        // decimals
        int latitude_decimal = modulePreferences.latitude_decimal();
        int longitude_decimal = modulePreferences.longitude_decimal();

        if (indexLatitude <= 0 || indexLongitude <= 0)
            return all_hydrants;

        for (File file : hydrantsFiles) {
            BufferedReader in = null;
            try {

                String hydrantspath = paths.PATH_LOCAL_HYDRANTS;
                File hydrantsPath = new File(hydrantspath);

                // Read File Line By Line
                in = new BufferedReader(new InputStreamReader(
                        new FileInputStream(file), "windows-1252"));
                String strLine;
                while ((strLine = in.readLine()) != null) {

                    totalLineProgress++;

                    String[] hydrantParts = strLine.split(";", -1);

                    String errors = "";
                    final String hydrantType = file.getName().replace(
                            Constants.FILETYPE_CSV, "");

                    File[] iconFiles = hydrantsPath
                            .listFiles(new FilenameFilter() {

                                @Override public boolean accept(File dir,
                                        String filename) {
                                    if (filename.contains(hydrantType)
                                            && filename
                                                    .contains(Constants.FILETYPE_PNG)) {
                                        return true;
                                    }
                                    return false;
                                }
                            });

                    HydrantHolder.Builder hb = new HydrantHolder.Builder();
                    if (hydrantParts.length >= 5) {

                        hb.setHydrantType(hydrantType);
                        if (iconFiles.length != 0) {
                            hb.setIconPath(hydrantspath
                                    + File.separatorChar
                                    + iconFiles[0].getName());
                        }
                        hb.setLatitude(hydrantParts[indexLatitude],
                                latitude_decimal);
                        hb.setLongitude(hydrantParts[indexLongitude],
                                longitude_decimal);

                        if (indexAddress > 0)
                            hb.setAddress(hydrantParts[indexAddress]);
                        if (indexAddressNumber > 0)
                            hb.setAddressNumber(hydrantParts[indexAddressNumber]);
                        if (indexAddressRemark > 0)
                            hb.setAddressRemark(hydrantParts[indexAddressRemark]);
                        if (indexRemark > 0)
                            hb.setRemark(hydrantParts[indexRemark]);

                        if (hb.getErrors().isEmpty()) {
                            HydrantHolder hydrant = hb.build();
                            all_hydrants.add(hydrant);
                        } else {
                            // TODO write error file to Dropbox if possible,
                            // otherwise locally
                            Log.d(TAG, errors);
                        }
                    } else {
                        errors = "Missing information";
                    }

                    publishProgress(totalLineProgress);
                }
            } catch (InvalidPathException e) {
                Log.e(TAG, e.getMessage(), e);
            } catch (IOException e) {
                Log.e(TAG, e.getMessage(), e);
            } finally {
                if (in != null) {
                    try {
                        in.close();
                    } catch (IOException e) {
                    }
                }
            }
        }
        Log.d(TAG, "" + all_hydrants.size());
        return all_hydrants;
    }

    // Update the progress
    @Override protected void onProgressUpdate(Integer... values) {
        // set the current progress of the progress dialog
        // if (progressDialog != null && values != null && values.length >
        // 0) {
        // progressDialog.setProgress(values[0]);
        // }
    }

    @Override protected void onPostExecute(List<HydrantHolder> hydrants) {
        // Device.releaseOrientation((Activity) context);

        Log.d(TAG, "Saved " + hydrants.size() + " hydrants");
        mHydrants = hydrants;

        new PrepareMarkerOptionsTask(hydrants).execute();

        super.onPostExecute(hydrants);
    }

}

private class PrepareMarkerOptionsTask extends
        AsyncTask<Void, Integer, Map<HydrantType, List<MarkerOptions>>> {
    // Before running code in separate thread
    List<HydrantHolder> mHydrants;

    public PrepareMarkerOptionsTask(List<HydrantHolder> hydrants) {
        this.mHydrants = hydrants;
        mMarkers = new HashMap<HydrantsModule.HydrantType, List<MarkerOptions>>();
    }

    @Override protected void onPreExecute() {

    }

    // The code to be executed in a background thread.
    @Override protected Map<HydrantType, List<MarkerOptions>> doInBackground(
            Void... arg) {

        for (HydrantHolder hydrant : mHydrants) {

            final String hydrant_type = hydrant.getHydrantType();
            final String hydrant_icon_path = hydrant.getIconPath();
            double latitude = hydrant.getLatitude();
            double longitude = hydrant.getLongitude();

            final LatLng position = new LatLng(latitude, longitude);

            final String address = hydrant.getAddress();
            final String addressNumber = hydrant.getAddressNumber();
            final String addressremark = hydrant.getAddressRemark();
            final String remark = hydrant.getRemark();

            // Log.d(TAG, hydrant.toString());

            BitmapDescriptor icon = BitmapDescriptorFactory
                    .defaultMarker(BitmapDescriptorFactory.HUE_RED);

            if (!hydrant_icon_path.isEmpty()) {
                File iconfile = new File(hydrant_icon_path);
                if (iconfile.exists()) {
                    BitmapDescriptor loaded_icon = BitmapDescriptorFactory
                            .fromPath(hydrant_icon_path);
                    if (loaded_icon != null) {
                        icon = loaded_icon;
                    } else {
                        Log.e(TAG, "loaded_icon was null");
                    }
                } else {
                    Log.e(TAG, "iconfile did not exist: "
                            + hydrant_icon_path);
                }
            } else {
                Log.e(TAG, "iconpath was empty on hydrant type: "
                        + hydrant_type);
            }

            StringBuffer snippet = new StringBuffer();
            if (!address.isEmpty())
                snippet.append("\n" + address + " " + addressNumber);
            if (addressremark.isEmpty())
                snippet.append("\n" + addressremark);
            if (!remark.isEmpty())
                snippet.append("\n" + remark);

            addHydrantMarker(
                    hydrant_type,
                    new MarkerOptions().position(position)
                            .title(hydrant_type)
                            .snippet(snippet.toString()).icon(icon));

            // publishProgress(markers.size());
        }
        return mMarkers;
    }

    // Update the progress
    @Override protected void onProgressUpdate(Integer... values) {
    }

    @Override protected void onCancelled() {
    }

    // after executing the code in the thread
    @Override protected void onPostExecute(
            Map<HydrantType, List<MarkerOptions>> markers) {

        Log.d(TAG, "Prepared " + markers.size() + " hydrantMarkerOptions");
        mMarkers = markers;

        loadEnded();

        super.onPostExecute(markers);

    }

}

public Set<HydrantType> getHydrantTypes() {
    return new HashSet<HydrantType>(mMarkers.keySet());
}

private void addHydrantMarker(String typeName, MarkerOptions marker) {
    HydrantType type = new HydrantType(typeName, marker.getIcon());
    if (mMarkers.containsKey(type)) {
        List<MarkerOptions> markers = mMarkers.get(type);
        markers.add(marker);
    } else {
        List<MarkerOptions> markers = new ArrayList<MarkerOptions>();
        markers.add(marker);
        mMarkers.put(type, markers);
        enableHydrantType(type);
    }
}

public class HydrantType {

    private final String name;
    private final BitmapDescriptor icon;

    public HydrantType(String name, BitmapDescriptor icon) {
        this.name = name;
        this.icon = icon;
    }

    public String getName() {
        return name;
    }

    public BitmapDescriptor getIcon() {
        return icon;
    }

    @Override public int hashCode() {
        return name.hashCode();
    }

    @Override public boolean equals(Object o) {
        if (o instanceof HydrantType) {
            if (((HydrantType) o).name.equals(name)) {
                return true;
            }
        }
        return false;
    }

} 

Based on the comments I add a bit more text and code.

Yes I add all MarkerOptions at once. Though, because I zoom in before adding all the markers GoogleMaps-extensions (first link) only spend CPU power on adding the visible ones to the map. If the user pans the map, or zooms out, more visible markers are automatically added.

To make the map lazy-load markers:

@Override
public void onMapReady(GoogleMap map) {
    Log.i(TAG, "onMapReady");
    if (map != null) {
        map.setClustering(new ClusteringSettings().enabled(false)
                .addMarkersDynamically(true));
    }
}

And the code I use to add hydrants (now this is overly complicated in many cases, but if you read carefully, it simply zooms to an address and adds all hydrants only after zoom has completed):

public void addHydrantsNearAddress(final AddressHolder addressHolder,
        final boolean zoomToAddress) {

    final GoogleMap map = googlemaps.getMap();


    final OnCameraChangeListener addHydrantsAfterZoom = new OnCameraChangeListener() {

        @Override public void onCameraChange(CameraPosition cameraPosition) {
            Log.d(TAG, cameraPosition.target.toString());
            Log.d(TAG, addressHolder.position.toString());

            final GoogleMap map = googlemaps.getMap();
            // if (Location.distanceBetween(cameraPosition.tar,
            // startLongitude, endLatitude, endLongitude, results)) {
            new Handler().postDelayed(new Runnable() {

                @Override public void run() {
                    addAllHydrants();
                }
            }, 500);

            map.setOnCameraChangeListener(null); // unregister
            // }
        }
    };

    if (map == null) {
        // wait for the map to be ready before adding hydrants
        googlemaps.setGoogleMapsCallback(new GoogleMapsCallback() {

            @Override public void mapReady(GoogleMap map) {

                if (zoomToAddress) {
                    map.setOnCameraChangeListener(addHydrantsAfterZoom);
                    googlemaps.zoomTo(addressHolder);
                } else {
                    addAllHydrants();
                }

                googlemaps.setGoogleMapsCallback(null); // unregister
            }
        });
    } else {

        if (zoomToAddress) {
            // only setOnCameraChangeListener if cammera needs to move
            LatLng cammeraPos = map.getCameraPosition().target;
            LatLng addressPos = addressHolder.position;

            float[] results = new float[1];
            Location.distanceBetween(cammeraPos.latitude,
                    cammeraPos.longitude, addressPos.latitude,
                    addressPos.longitude, results);

            // Log.d(TAG, "distance " + results[0]);

            if (results[0] > 1) {
                map.setOnCameraChangeListener(addHydrantsAfterZoom);
                googlemaps.zoomTo(addressHolder);
            } else {
                googlemaps.zoomTo(addressHolder);
                addAllHydrants();
            }

        }

    }

}

Another reason for this to be more complicated than needed for the sake of example is that I let the user filter between hydrant types. Hope that you can see the idea despite this.

addAllHydrants() is as simple as it sounds, iterating MarkerOptions and adding them:

public void addAllHydrants() {

    enableAllHydrants();

    GoogleMap map = googlemaps.getMap();

    map.setTrafficEnabled(modulePreferences.showTraffic());
    map.setMapType(modulePreferences.getMapType());

    addHydrants(map);
}

private void addHydrants(GoogleMap map) {

    Log.d(TAG, "addHydrants");

    if (mHydrants == null || mHydrants.isEmpty()) {
        Toast.makeText(context,
                context.getString(R.string.missing_hydrants),
                Toast.LENGTH_LONG).show();
        return;
    } else {
        for (MarkerOptions marker : getEnabledHydrantMarkers()) {
            map.addMarker(marker);
        }
    }

}

I really think that you would be over complicating things for yourself trying to calculate, fetch and add visible markers yourself, rather than just.

1. Prepare MarkerOptions (example in first blob of code)
2. Wait for map to be ready using onMapReady callback (examples of this is in the official examples from Google)
3. Set addMarkersDynamically(true) on the newly created map
4. Zoom in on an address or area of interest (Device location for example)
5. Add all MarkerOptions
6. See Markers automagically being added as you move around the map

To answer qustion about onMapReady callback

If you use XML to add the map, then I suspect that you might not need to add the callback. Try and use the map directly in onCreate() of your Activity. As long as the call to getExtendedMap() is not null, you should be fine.

I am creating the fragment in code to be able to put controller code together with the map. Thus my SupportMapFragment looks something like this:

public class GoogleMapFragment extends SupportMapFragment {

    private OnGoogleMapFragmentListener mCallback;

    public GoogleMapFragment() {
        mCallback = null;
    }

    public static interface OnGoogleMapFragmentListener {
        void onMapReady(GoogleMap map);
    }

    @Override public void onAttach(Activity activity) {
        super.onAttach(activity);
        try {
            mCallback = (OnGoogleMapFragmentListener) activity;
        } catch (ClassCastException e) {
            throw new ClassCastException(getActivity().getClass().getName()
                    + " must implement OnGoogleMapFragmentListener");
        }

    }

    @Override public View onCreateView(LayoutInflater inflater,
            ViewGroup container, Bundle savedInstanceState) {
        View view = super.onCreateView(inflater, container, savedInstanceState);
        if (mCallback != null) {
            mCallback.onMapReady(getExtendedMap());
        }

        return view;
    }

}

Qnp

Thanks! definitely upvoting this; if and when I get it working I'll mark correct. You have 4,500 markers? Wow! So it sounds like you load them all at once... however, this may change what I need to do. I only want to show the 250 closes to the user and then when they scroll, use the setOnCameraPositionChange method and go get more markers from my database. I'm having a rough time with all of these things. The app, GasBuddy does this the best I've seen. – Dardar

One more thing; Using the setOnCameraChangeListener (typo above) is nice, but it is called for every single map change -- meaning a TON of DB calls, no matter how small of a change.. it would be nice to have a timeout with it. – Dardar

Have made an edit. Note that I call map.setOnCameraChangeListener(null); right after the change and that addHydrantsNearAddress() is invoked by a button click. That is, the setOnCameraChangeListener in the code is ever only invoked once per map load. After that Google-map-extension takes over, adding markers when the visible area changes due to panning or zooming. – Qnp

thanks! I'm just going to mark this correct because this is exactly what I was looking for. It's up to me to implement now, thanks again! (If there is anything else I may be back...) – Dardar

No problem - I will gladly answer more questions along the way. I am still sorry that my code examples are not more clear than this, but it is not easy to extract from existing code. Happy coding and good luck :) – Qnp

Question: What interface do you have to implement to use public void onMapReady(GoogleMap map) {? Also, can you show me the code where you initialize the map using extensions? I have successfully imported it to my project but not this pretty basic info anywhere.... (I am NOT using the Support library in my project, but it seems I have to use SupportMapFragment? I changed it to that in my xml. – Dardar

Added new edit, and yes,.. now you mention it I had some problems due to the forcing of the support library by the SupportMapFragment. Had forgotten about that. I create the interface myself, as shown in the code. Then to use it let the Activity adding the map implement OnGoogleMapFragmentListener – Qnp

Answer 2

Just wanted to share a simple solution that I came up with for showing all markers without reloading every time.

Basically I rely on onCameraChangeListener to check if the user's view of the map has changed, and only show the markers within that view, otherwise the markers will be hidden... You will still have the longer load time the first time you load your data, but it's pretty quick after that.

First you need to add all the markers to your map, and store all the markers in a Map

// Put it in onCreate or something
map.clear(); // Clears markers
markers.clear(); // private Map<Marker, Item> markers = new HashMap<Marker, Item>()
for (Item item : items) {
    MarkerOptions markerOptions = new MarkerOptions()
        .position(new LatLng(item.getLatitude(), item.getLongitude())
        .title(item.getTitle())
        .snippet(item.getSnippet());
    Marker marker = map.addMarker(marketOptions);
    markers.put(marker, item);
}

Then you need to setup the listener:

// Put in onCreate or something
map.setOnCameraChangeListener(new GoogleMap.OnCameraChangeListener() {
    @Override
    public void onCameraChange(final CameraPosition cameraPosition) {
        showMarkers(cameraPosition.target);
        LatLngBounds bounds = map.getProjection().getVisibleRegion().latLngBounds;
        for (Map.Entry<Marker, Item> entry : markers) {
            Marker marker = entry.getKey();
            if (bounds.contains(marker.getPosition()) {
                marker.setVisible(true);
            } else {
                marker.setVisible(false);
            }
        }
    }
});

This should trigger at the end of animationMap and any form of camera movement (programatically, or by user dragging)

You may consider persisting the data using onSaveInstanceState to make it even more responsive when switching between apps

Garlaand