d3.js or rxjs error? this.svg.selectAll(...).data(...).enter is not a function

Question

This is a weird one. It's also a bit long so apologies in advance. update - it ended up being 2 problems see my answer below.

Here's my error: EXCEPTION: this.svg.selectAll(...).data(...).enter is not a function

I have an angular-cli client and a node api server. I can retrieve a states.json file from a service using an observable (code below). d3 likes the file and displays the expected US map.

The moment I change the target of the service in my api server from a file to a bluemix-cloudant server I get the error above in my client.

When I console.log the output in a variation using ngOnInit, initially mapData prints as an empty array and the error gets thrown. This is the obvious source of the error since there's no data, but the Chrome debugger shows the get request pending. When the request completes, the data prints as expected in the console.

• angular-cli version 1.0.0-beta.26
• angular version ^2.3.1
• d3 version ^4.4.4
• rxjs version ^5.0.1

map.component.ts:

import { Component, ElementRef, Input } from '@angular/core';
import * as D3 from 'd3';
import '../rxjs-operators';

import { MapService } from '../map.service';

@Component({
  selector: 'map-component',
  templateUrl: './map.component.html',
  styleUrls: ['./map.component.css']
})
export class MapComponent {

  errorMessage: string;
  height;
  host;
  htmlElement: HTMLElement;
  mapData;
  margin;
  projection;
  path;
  svg;
  width;

  constructor (private _element: ElementRef, private _mapService: MapService) {
    this.host = D3.select(this._element.nativeElement);
    this.getMapData();
    this.setup();
    this.buildSVG();
  }

  getMapData() {
    this._mapService.getMapData()
      .subscribe(
        mapData => this.setMap(mapData),
        error =>  this.errorMessage = <any>error
      )
  }

  setup() {
    this.margin = {
      top: 15,
      right: 50,
      bottom: 40,
      left: 50
    };
    this.width = document.querySelector('#map').clientWidth - this.margin.left - this.margin.right;
    this.height = this.width * 0.6 - this.margin.bottom - this.margin.top;
  }

  buildSVG() {
    this.host.html('');
    this.svg = this.host.append('svg')
      .attr('width', this.width + this.margin.left + this.margin.right)
      .attr('height', this.height + this.margin.top + this.margin.bottom)
      .append('g')
      .attr('transform', 'translate(' + this.margin.left + ',' + this.margin.top + ')');
  }

  setMap(mapData) {
    this.mapData = mapData;
    this.projection = D3.geoAlbersUsa()
      .translate([this.width /2 , this.height /2 ])
      .scale(650);
    this.path = D3.geoPath()
      .projection(this.projection);

    this.svg.selectAll('path')
      .data(this.mapData.features)
      .enter().append('path')
        .attr('d', this.path)
        .style('stroke', '#fff')
        .style('stroke-width', '1')
        .style('fill', 'lightgrey');
  }
}

map.service.ts:

import { Http, Response } from '@angular/http';
import { Injectable } from '@angular/core';
import { Observable } from 'rxjs/Observable';

@Injectable()
export class MapService {
  private url = 'http://localhost:3000/api/mapData';
  private socket;

  constructor (private _http: Http) { }

  getMapData(): Observable<any> {
    return this._http.get(this.url)
      .map(this.extractData)
      .catch(this.handleError);
  }

  private extractData(res: Response) {
    let body = res.json();
    return body.data || {};
  }

  private handleError(error: any) {
    let errMsg = (error.message) ? error.message :
      error.status ? `${error.status} - ${error.statusText}` : 'Server error';
    console.error(errMsg);
    return Promise.reject(errMsg);
  }
}

Is this a function of being Async and the call to the data takes too long for d3?

I had hopes that this question Uncaught TypeError: canvas.selectAll(...).data(...).enter is not a function in d3 would offer some insight but I don't see any.

Any help or insight is greatly appreciated!

EDIT: Here's a screenshot of the headers section from Chrome per Marks request below. The response tab shows the data properly coming across as a GeoJSON object. I've also copied that response into a file locally and used it as a map source with positive results.

Data Tests so far: GeoJSON file (2.1mb)

• Local file, local server: Success (response time 54ms)
• Same file, remote server: D3 errors before data returned to browser (750ms)
• API call from remote server: D3 errors before data returned to browser (2.1 s)

Moribund

Can you show the output of logging mapData.features? – Rentschler

@Rentschler - it's the parts of the GeoJSON data that define the borders of the states. Here's a sample: "features": [ { "type": "Feature", "properties": { "GEO_ID": "0400000US01", "STATE": "01", "NAME": "Alabama", "LSAD": "", "CENSUSAREA": 50645.326000 }, "geometry": { "type": "MultiPolygon", "coordinates": [ [ [ [ -88.124658, 30.283640 ], [ -88.086812, 30.259864 ], [ -88.074854, 30.249119 ], [ -88.075856, 30.246139 ], [ -88.078786, 30.245039 ], ...},{next state and so on...}] – Moribund

Did you make sure that this.mapData.features is what you expect (must be an array) inside the setMap function? – Rentschler

@Rentschler It is exactly the same content from a file on the node server. As I said, d3 likes that file and displays the map. this.mapData.features is the array d3 needs to produce the map. – Moribund

Does setMap() get called twice? Once with an empty array, and once with the expected array? – Ivett

@wilburr90 I console logged every function to see the execution sequence and setMap(mapData) only showed executing once. – Moribund

Hi, how about wrapping the chart html with and *ngIf and set it to true only when you have the api response? – Bobseine

Hi Bruce, see this question is still not resolved. Can you make a codepen/playground/jsfiddle somewhere for us to look and debug? – Sympathetic

Everyone is focused on your angular code below. My guess, though, is that your data coming from you API is somehow malformed, times out or is otherwise bogus. Can you post a screenshot of the header tab from the chrome network tool, when you make the request? – Sollows

Added the screenshot. @Rentschler asked about the data earlier and I added the test approach I used to validate the data coming from the api. – Moribund

Answer 1

Wow. This has been a trip!

Here's the tl;dr - I had two issues I was dealing with: the format of the data being returned and data latency.

1. Data format: when my json file was on the server the api call would wrap it in a { data: } object but when it was served up from an api calling my clouodant database the wrapper wasn't there. @PierreDuc, thank you for that.
2. I found this SO answer to solve the latency problem -> Queue/callback function after fetching data in an Observable in Angular 2

Here's the modified code and the tl part:

map.component.ts:

import { Component, ElementRef, Input, AfterViewInit, ChangeDetectorRef } from '@angular/core';
import * as d3 from 'd3/index';
import '../rxjs-operators';

import { MapService } from '../shared/map.service';

@Component({
  selector: 'map-component',
  templateUrl: './map.component.html',
  styleUrls: ['./map.component.css']
})
export class MapComponent implements AfterViewInit {

  errorMessage: string;
  height;
  host;
  htmlElement: HTMLElement;
  mapData;
  margin;
  projection;
  path;
  svg;
  width;

  constructor (
    private _element: ElementRef, 
    private _mapService: MapService,
    private _changeRef: ChangeDetectorRef
  ) { }

  ngAfterViewInit(): void {
    this._changeRef.detach();
    this.getMapData();
  }

  getMapData() {
    this._mapService.getMapData().subscribe(mapData => this.mapData = mapData, err => {}, () => this.setMap(this.mapData));
    this.host = d3.select(this._element.nativeElement);
    this.setup();
    this.buildSVG();
  }

  setup() {
    console.log('In setup()')
    this.margin = {
      top: 15,
      right: 50,
      bottom: 40,
      left: 50
    };
    this.width = document.querySelector('#map').clientWidth - this.margin.left - this.margin.right;
    this.height = this.width * 0.6 - this.margin.bottom - this.margin.top;
  }

  buildSVG() {
    console.log('In buildSVG()');
    this.host.html('');
    this.svg = this.host.append('svg')
      .attr('width', this.width + this.margin.left + this.margin.right)
      .attr('height', this.height + this.margin.top + this.margin.bottom)
      .append('g')
      .attr('transform', 'translate(' + this.margin.left + ',' + this.margin.top + ')');
  }

  setMap(mapData) {
    console.log('In setMap(mapData), mapData getting assigned');
    this.mapData = mapData;
    console.log('mapData assigned as ' + this.mapData);
    this.projection = d3.geoAlbersUsa()
      .translate([this.width /2 , this.height /2 ])
      .scale(650);
    this.path = d3.geoPath()
      .projection(this.projection);

    this.svg.selectAll('path')
      .data(this.mapData.features)
      .enter().append('path')
        .attr('d', this.path)
        .style('stroke', '#fff')
        .style('stroke-width', '1')
        .style('fill', 'lightgrey');
    }

  }

map.service.ts:

import { Http, Response } from '@angular/http';
import { Injectable } from '@angular/core';
import { Observable } from 'rxjs/Observable';

@Injectable()
export class MapService {
// private url = 'http://localhost:3000/mapData'; // TopoJSON file on the server (5.6 ms)
// private url = 'http://localhost:3000/mapDataAPI'; // GeoJSON file on the server (54 ms)
// private url = 'http://localhost:3000/api/mapData'; // get json data from a local server connecting to cloudant for the data (750ms)
private url = 'https://???.mybluemix.net/api/mapData'; // get GeoJSON from the cloud-side server api getting data from cloudant (1974 ms per Postman)

constructor (private _http: Http) { }

getMapData(): Observable<any> {
    return this._http.get(this.url)
      .map(this.extractData)
      .catch(this.handleError);
  }

  private extractData(res: Response) {
    let body = res.json();
    return body; // the data returned from cloudant doesn't get wrapped in a { data: } object
    // return body.data; // this works for files served from the server that get wrapped in a { data: } object
    }

  private handleError(error: any) {
    let errMsg = (error.message) ? error.message :
      error.status ? `${error.status} - ${error.statusText}` : 'Server error';
    console.error(errMsg);
    return Promise.reject(errMsg);
  }
}

I really appreciate everyone's input - I still have some cleanup to do on the code - there may still be some things to do but the data creates the map. My next tasks are adding data and animation. I'm shooting for a presentation similar to this: http://ww2.kqed.org/lowdown/2015/09/21/now-that-summers-over-what-do-californias-reservoirs-look-like-a-real-time-visualization/

You can find the code for it here: https://github.com/vicapow/water-supply

Moribund

Answer 2

My guess is that angular messes up the reference to your map element between the constructor and the time that your request comes back. My advice is to start building the svg inside ngAfterViewInit or even better, when the response from the server has arrived. I believe this issue is mainly based on timing. If of course the data received from the server is not malformed and you can actually log a nice array of mapping data in your console.

Also the document.querySelector('#map').clientWidth will return 0 or undefined if the view is not ready yet, and when the #map is inside the map.component.html.

When you are working on elements inside the template, always use the ngAfterViewInit life cycle hook.

Besides that, it doesn't seem like you are using any of angular's change detection inside your component. I would advice you, to prevent any interference with your elements, to detach from the ChangeDetectorRef:

@Component({
  selector: 'map-component',
  templateUrl: './map.component.html',
  styleUrls: ['./map.component.css']
})
export class MapComponent implement AfterViewInit {

  private mapData;

  constructor (
     private _element: ElementRef, 
     private _mapService: MapService,
     private _changeRef: ChangeDetectorRef
  ){}

  ngAfterViewInit(): void {
     this._changeRef.detach();
     this.getMapData();
  }

  getMapData() {
    this._mapService.getMapData().subscribe((mapData) => {
       this.mapData = mapData;
       this.setup();
       this.buildSvg();
       this.setMapData();
    });
  }

  setup() {
     //...
  }

  buildSVG() {
    //...
  }

  setMapData(mapData) {
    //...
  }

}

addendum

On the other hand, when analyzing your steps:

• you create a svg
• append a g to it
• then you do a selectAll('path')
• and try to add data to this selection
• and only after that you try to append a path

Can you try appending the path first and after that add data to it? Or use

this.svg.selectAll('g') 

Makes more sense to me, or perhaps I don't really understand how selectAll works.

2nd addendum

I think I really got it now for you :D can you change your extractData function to this:

private extractData(res: Response) {
    return res.json()
} 

My guess is that your webserver doesn't return the mapdata in an object with a data property, but just the object immediately, and your implementation seems to be straight from the angular.io cookbook :)

Rosalynrosalynd

It is definitely an issue of timing. I moved the code into AfterViewInit as suggested but still same result: error message. Interesting points on ChangeDetectorRef - I had not known about that so I appreciate learning something new! – Moribund

@BruceMacDonald I've updated my answer with an addendum. Perhaps that will work, combined with what i stated above – Rosalynrosalynd

selectAll will select all DOM elements that match the string or function you pass to it. Much like a CSS selector, it can work with tags, classes, id's, etc. I've been comfortable with the succession of d3 steps to build the map but you've caused me to re-think the entire lifecycle as angular and d3 interact to change the DOM and the event-firing sequence - especially with a high latency data source. I have more learning to do... – Moribund

@BruceMacDonald but you are doing a selectAll('path') even though you haven't added a path element yet. Sounds counterintuitive to me :) – Rosalynrosalynd

That's the beauty and mystique of d3! The enter().append('path') actually is doing the heavy lifting by adding the paths needed for the map from the data being passed to it even though the DOM elements do not yet exist. – Moribund

But it already fails at the enter, which makes me think that the selectAll returns an empty selection, which in turn makes the data fail. There is no path in your svg yet. Just a g – Rosalynrosalynd

See @MattDionis answer below – Moribund

@BruceMacDonald Ive updated my answer once more. Seems to me this is the only logical solution :D – Rosalynrosalynd

Answer 3

This is more of a "band-aid", but try changing getMapData to this:

getMapData() {
  this._mapService.getMapData()
    .subscribe(
      mapData => {
        if (mapData.features) {
          this.setMap(mapData);
        }
      },
      error =>  this.errorMessage = <any>error
    )
}

This will guard against setMap being called without mapData.features.

Rahal

I changed getMapData - Here are the results (no joy): When I serve the us-states.json file from the server the response is GET /mapData 304 5.589 ms - - and the map will display. But serving the contents from the api to cloudant, the response is GET /api/mapData 200 702.774 ms - - and no map is displayed. How do I stall the setMap(mapData) function until the data arrives from the slow server? – Moribund

@BruceMacDonald, I added setTimeout to my GET /mapData endpoint to simulate a slow response, but no matter how long I set the timeout for it still works (the map successfully loads after the timeout period). – Rahal

Thanks for checking, I still have some research to do. The change to getMapData at least is stopping the d3 error but I still don't see a map from the api call. – Moribund

I'm going to continue hacking away at it as well. – Rahal

Answer 4

Wouldn't it work with a Promise instead of an Observable? Something like

In your service:

getMapData (): Promise<any> {
  return this._http.get(this.url)
                  .toPromise()
                  .then(this.extractData)
                  .catch(this.handleError);
}

You could also directly extract your Data in this function, something like:

.then(response => response.json().data)

and in your component:

getMapData() {
    this._mapService.getMapData()
        .then(
            mapData => mapData = this.setMap(mapData),
            error =>  this.errorMessage = <any>error
         )
}

My only concern is where to call the setMap function in the code above. Since I can't test it, I hope it can help.

Cachepot

I will try it and post the results – Moribund

Same results. On the bright side the switch to Promise is as easy as advertised on angular.io. The only thing I had to hunt for was adding import 'rxjs/add/operator/toPromise'; to map.service.ts but everything else was straightforward. – Moribund

Answer 5

Have you tried moving your functions from the constructor to ngOnInit, something like:

import { Component, ElementRef, Input, OnInit } from '@angular/core';
import * as D3 from 'd3';
import '../rxjs-operators';

import { MapService } from '../map.service';

@Component({
  selector: 'map-component',
  templateUrl: './map.component.html',
  styleUrls: ['./map.component.css']
})
export class MapComponent implements OnInit {

  errorMessage: string;
  height;
  host;
  htmlElement: HTMLElement;
  mapData;
  margin;
  projection;
  path;
  svg;
  width;

  constructor (private _element: ElementRef, private _mapService: MapService) {}

  setup() {
    this.margin = {
      top: 15,
      right: 50,
      bottom: 40,
      left: 50
    };
    this.width = document.querySelector('#map').clientWidth - this.margin.left - this.margin.right;
    this.height = this.width * 0.6 - this.margin.bottom - this.margin.top;
  }

  buildSVG() {
    this.host.html('');
    this.svg = this.host.append('svg')
      .attr('width', this.width + this.margin.left + this.margin.right)
      .attr('height', this.height + this.margin.top + this.margin.bottom)
      .append('g')
      .attr('transform', 'translate(' + this.margin.left + ',' + this.margin.top + ')');
  }

  setMap(mapData) {
    this.mapData = mapData;
    this.projection = D3.geoAlbersUsa()
      .translate([this.width /2 , this.height /2 ])
      .scale(650);
    this.path = D3.geoPath()
      .projection(this.projection);

    this.svg.selectAll('path')
      .data(this.mapData.features)
      .enter().append('path')
        .attr('d', this.path)
        .style('stroke', '#fff')
        .style('stroke-width', '1')
        .style('fill', 'lightgrey');
  }

  ngOnInit() {
      this.host = D3.select(this._element.nativeElement);
      this.setup();
      this.buildSVG();

      this._mapService.getMapData()
        .subscribe(
           mapData => this.setMap(mapData),
           error =>  this.errorMessage = <any>error
        )
   }
}

Now, I'm not sure it will change anything, but it is considered good practice to use the life cycle hook (OnInit) instead of the constructor. See Difference between Constructor and ngOnInit.

Cachepot

Same results after moving calls from the constructor to ngOnInit. – Moribund