How to download and parse a csv file in Racket?

Question

How do I download and parse a csv file in Racket?

Sneak

Answer 1

Use get-pure-port to download the file, and use the Planet library (require (planet neil/csv)) to parse it.

The following example downloads and parses a csv file containing data on the size of the various Galapagos islands and how many species were found on each island.

#lang racket
(require (planet neil/csv:1:=7) net/url)

(define galapagos-url 
  (string->url
   "http://www.stat.washington.edu/~handcock/536/Data/galapagos.csv"))

(define make-galapagos-csv-reader
  (make-csv-reader-maker
   '((separator-chars              #\,)
     (strip-leading-whitespace?  . #t)
     (strip-trailing-whitespace? . #t))))

(define (all-rows url make-reader)
  (define next-row (make-reader (get-pure-port url)))
  (define (loop)
    (define row (next-row))
    (if (empty? row)
        '()
        (cons row (loop))))
  (loop))

(all-rows galapagos-url make-galapagos-csv-reader)

The first rows returned are:

'(("Island"
   "Observed.species"
   "Native.species"
   "Area(km^2)"
   "Elevation(m)"
   "Distance.nearest.island(km)"
   "Distance.Santa.Cruz(km)"
   "Area.adj.island(km^2)")
  ("Baltra" "58" "23" "25.09" "" "0.6" "0.6" "1.84")
  ("Bartolome" "31" "21" "1.24" "109" "0.6" "26.3" "572.33")
  ("Caldwell" "3" "3" "0.21" "114" "2.8" "58.7" "0.78")

Sneak

Answer 2

Neil has a new library, csv-reading, so use that instead.

First, install the package using raco:

raco pkg install csv-reading

To convert a CSV file to a list, here is a helper function:

(require csv-reading)

(define (csvfile->list filename)
  (call-with-input-file filename
                        csv->list))

To download a CSV file and convert to a list, do this:

(require net/url)
((compose csv->list get-pure-port string->url) "http://example.com")

See here for the csv-reading library: csv-reading library, it is the latest version (and the other answers are using deprecated ones).

Escalate

Answer 3

This answer got me where I was looking to go, but I thought I'd chime in for future intrepid explorers. There is an easier invocation (perhaps with newer versions of the csv library?) for doing what you ask. This of course assumes you want the comma delimiter, and to strip leading / trailing whitespace as in the above example.

#lang racket
(require (planet neil/csv:2:0) net/url)

;; Continuing with the UW data sources examples
(define iver-url 
  (string->url
   "http://faculty.washington.edu/cadolph/vis/iver.csv"))

(csv->list (get-pure-port iver-url))

In this version, the csv->list function automatically creates a csv-reader with the aforementioned defaults. You can override the defaults (e.g. if you had a different delimiter or didn't want to strip trailing and leading whitespace) by instead passing a your custom csv reader constructed with make-csv-reader to csv->list.

Cynosure

Answer 4

Use get-pure-port to download the file, then use read-csv-file (from the 2htdp/batch-io library included in Racket) to parse the data.

Download the file

#lang racket
(require net/url)

;; Get the data.
(define the-url (string->url "http://www.example.com/data.csv"))
(define the-data (port->bytes (get-pure-port the-url)))

;; Write the data to a file.
(define outfile (open-output-file "data.csv"))
(write-bytes the-data outfile)
(close-output-port outfile)

Reference: https://www.monolune.com/how-to-download-files-using-racket/

WARNING: By default, when using HTTPS, Racket will not verify the server's certificate. Read the documentation (get-pure-port) on how to enable server certificates verification when using HTTPS.

Parse the data

#lang racket
(require 2htdp/batch-io)

(read-csv-file "data.csv")  ; Returns a list.

Foy