Removing brackets [] from ends of geojson in R

Question

I'm trying to import a geojson into R as a spatial object (i.e., sp) from a USGS web service (streamstats) and am having trouble getting it into the right format for R.

library(jsonlite)
mydata <- fromJSON("https://streamstats.usgs.gov/streamstatsservices/watershed.geojson?rcode=NY&xlocation=-74.524&ylocation=43.939&crs=4326&includeparameters=false&includeflowtypes=false&includefeatures=true&simplify=true")

This returns the feature in a geojson plus a bunch of other things I don't need. I can select just the data frame I need and write it out with:

tmp <- mydata$featurecollection[2,2]
write_json(tmp, 'test.json')

[{"type": "FeatureCollection", ...bunch of other stuff}]

If I manually remove the brackets "[]" on each end of the json file I can then import it as a spatial object with:

library(geojsonio)
test <- geojson_read('test.json', method='local', what='sp')

Otherwise I get this error:

Error in rgdal::ogrListLayers(input) : Cannot open data source

Is there a way to remove the brackets on each end? Maybe there's even a simpler solution I'm missing from where I select out the required data frame.

Ravine

Answer 1

Another geojsonio author here...

I don't think it's a problem with any of the libraries/packages (geojsonio, rgdal, etc), but the fact that the square brackets make it not a proper geojson object according the spec (square brackets denote an array).

That url returns a json object that contains an array (confusingly called featurecollection) that then contains two objects, each one containing a name and a feature, and each feature is a proper geojson FeatureCollection. Those FeatureCollections are what we want to pull out - One a Point (probably the centroid of the watershed?), and one a Polygon defining the watershed boundary.

library(jsonlite)
library(geojsonio)
library(sp)

# Don't simplify the results so that it's easier to pull out the full geojson objects
mydata <- fromJSON("https://streamstats.usgs.gov/streamstatsservices/watershed.geojson?rcode=NY&xlocation=-74.524&ylocation=43.939&crs=4326&includeparameters=false&includeflowtypes=false&includefeatures=true&simplify=true", 
                   simplifyVector = FALSE, simplifyDataFrame = FALSE)

# Extract each geojson object and 'auto_unbox' to remove square brackets from 
# arrays of length 1:
point_geojsonsting <- toJSON(mydata$featurecollection[[1]]$feature, 
                             auto_unbox = TRUE)
poly_geojsonsting <- toJSON(mydata$featurecollection[[2]]$feature, 
                            auto_unbox = TRUE)

# Read directly to sp without writing to disk first:
point_sp <- geojson_sp(point_geojsonsting)
poly_sp <- geojson_sp(poly_geojsonsting)

plot(poly_sp)
plot(point_sp, add = TRUE, col = "red", pch = 21, cex = 5)

Created on 2018-05-09 by the reprex package (v0.2.0).

Showpiece

auto_unbox = TRUE solved a similar problem I faced with multiple brackets for jsonlite::write_JSON as well. – Eelworm

Answer 2

This answer was assuming that saving the file was a necessary step (for example, to read it with another Rscript). In this way, you make the slow external call only one time, and then you can test by only requesting your local file, which is much faster. Also the external request may not be always available.

As a workaround you may remove the write_json command line and add this one:

stringJson <- toJSON(tmp)
write(substr(stringJson, 2, nchar(stringJson) - 1), file = 'test.json')

Another alternative is to use rjson library to write the json (but be careful because both libraries have functions with the same name):

library(jsonlite)
library(rjson)
mydata <- jsonlite::fromJSON("https://streamstats.usgs.gov/streamstatsservices/watershed.geojson?rcode=NY&xlocation=-74.524&ylocation=43.939&crs=4326&includeparameters=false&includeflowtypes=false&includefeatures=true&simplify=true")

tmp <- mydata$featurecollection[2,2]
write(rjson::toJSON(tmp), file = 'test.json')

Hope it helps :)

Meridithmeriel

Building on answer from @tk3, you can avoid even writing out and read directly to an sf object by skipping the write line in answer and instead doing new <- substr(stringJson, 2, nchar(stringJson) -1) final <- st_read(new) – Probable

Answer 3

geojsonio maintainer ...

So there was a change in rgdal::readOGR where any geojson string longer than a valid file path gets truncated and thus not read. See https://github.com/ropensci/geojsonio/issues/130

So we did a workaround for geojsonio::geojson_sp, but not for geojsonio::geojson_read

Here's a way around it though with jqr

library(jqr)
library(geojsonio)
library(sf)
myurl <- "https://streamstats.usgs.gov/streamstatsservices/watershed.geojson?rcode=NY&xlocation=-74.524&ylocation=43.939&crs=4326&includeparameters=false&includeflowtypes=false&includefeatures=true&simplify=true"
jq(url(myurl), ".featurecollection[1]") %>% st_read(quiet = TRUE, stringsAsFactors = FALSE)

where jq lets us dip into the JSON without having to write to disk first

Furtado

This is using the development branch of jq which has a method for connection objects (version 1.0.9200). The latest release version (1.0.0) can't accept URLs. – Mohammadmohammed