d3.js Zoomable Sunburst visualization from self-referencing CSV input

Question

I'm a newbie at d3.js and need help to adapt Zoomable Sunburst to make it work with self-referencing CSV data. Sample lines from the input CSV:

id,parentId,name,size
ROOT,NULL,Root,
RE,ROOT,General > Revenue Expenditure,
RE11,RE,Main supervision recovery etc.,
RE11A109K,RE11,Shivjayanti celebrations and lighting,170000
RE11H108,RE11,Electicity for import tax naka,2550000
RE11J,RE11,Maintaince for main building,
RE11J101A,RE11J,Electricity expenditure,11475000
RE11J101 C,RE11J,Power lift,2125000

As you can see, there are variable levels of depth. At some places the data is coming at 3rd level, at others we might have parent-child relationships going 9 levels deep, and so on. That's government budgets for you! While there are columns in addition to these 4 that aren't critical to the visualization (so omitted here), I would be displaying their contents in a side pane on mouseover. So while non-critical, any additional columns do need to carry through and not get dropped.

I looked into many d3.nest() examples but those doesn't seem to work for parent-child self-referencing columns and data with variable levels of depth.

I'm presently using a workaround to convert this into hierarchical JSON in the flare.json format, using this DataStructures.Tree project . But looking for a more direct solution. Almost there, but not able to mix up code from different sources. Would be grateful to be shown a full top-to-bottom solution. Thanks in advance!

Sparry

Answer 1

Got it. We include these scripts from the DataStructures.Tree project linked in the question : base.js, DataStructures.Tree.js. (you'll find them in /js/lib/ and /js/vendor/)

<script type="text/javascript" src="base.js"></script>
<script type="text/javascript" src="DataStructures.Tree.js"></script>
<script src="http://d3js.org/d3.v3.min.js"></script>

Then, we replace this line,

d3.json("flare.json", functon(error, root) {

..with these lines:

d3.csv("electrical5.csv", function(data){
var tree = DataStructures.Tree.createFromFlatTable(data),
           root = tree.toSimpleObject(function(objectToDecorate, originalNode) {
                objectToDecorate.size = originalNode.size;
                if (objectToDecorate.children && objectToDecorate.children.length == 0) {
                    delete objectToDecorate.children;
                }
                return objectToDecorate;
            });
//console.log(JSON.stringify(root));

leave everything else just as it is.

---

Uncomment the console.log line for debugging; it will put the json code in the browser console when you load the page. You can also make a textarea in the webpage and output the json code in that.

document.getElementById("SHOWME").value = JSON.stringify(root);

[at bottom of page]

<textarea id="SHOWME"></textarea>

it will be unformatted json code, so copy-paste it to http://codebeautify.org/jsonviewer and that ought to give you a well-formatted json.

---

To get more columns in apart from the regular [id,parentId,name,size] , we have to edit DataStructures.Tree.js

Locate these lines:

simpleChildRepresentation.push(decorateNode({
    "name" :  node.name,
    "children" : children
}, node));

And insert the extra columns in the same format as the node.name line.

simpleChildRepresentation.push(decorateNode({
    "name" :  node.name,
     //CUSTOM COLUMNS
    "workcode" : node.id,
    "parentcode" : node.parentId,
    "department" : node.department,
    "totalorextract" : node.totalorextract,
    "total" : node.total,
    "pages" : node.pages,
    //CUSTOM COLUMNS DONE
    "children" : children
}, node));

You can now directly visualize self-referencing csv data on any d3.js visualization that uses flare.json . Animations get a bit clunkier, though.

Sparry

Answer 2

An answer for something quite similar

From csv file : sample.csv

id,value
data,
data.aa,
data.aa.bb,
data.aa.bb.cc,123
data.aa.bb.dd,345
data.aa.ee,
data.aa.ee.ff,789

To

{
"name": "root",
"children": [{
    "name": "data",
    "children": [{
        "name": "aa",
        "children": [{
            "name": "bb",
            "children": [{
                "name": "cc",
                "children": [],
                "size": "123"
            }, {
                "name": "dd",
                "children": [],
                "size": "345"
            }]
        }, {
            "name": "ee",
            "children": [{
                "name": "ff",
                "children": [],
                "size": "789"
            }]
        }]
    }]
}]
}

With this Python

import json
import csv

with open('sample.csv') as f:
    columns = ("id", "value")
    reader = csv.DictReader(f, columns)
    next(reader)

    json_data = {
        "name": "root",
        "children": []
    }

    for data in reader:
        current_node = json_data
        cols = data['id'].split('.')
        last_value = data['value']
        last_cell_name_with_value =  ""
        if last_value != '':
            last_cell_name_with_value = cols[len(cols)-1]
        for i in range(len(cols)):
            found_last_cell_with_value = False
            cell = cols[i]
            if cell == last_cell_name_with_value:
                found_last_cell_with_value = True
            if cell:
                found_child = False
                for child in current_node["children"]:
                    if child["name"] == cell:
                        current_node = child
                        found_child = True
                        break
                if not found_child:
                    if found_last_cell_with_value == False:
                        new_node = {
                            "name": cell,
                            "children": []
                        }
                    else:
                        new_node = {
                            "name": cell,
                            "children": [],
                            "size": last_value
                        }
                    current_node["children"].append(new_node)
                    current_node = new_node


with open('hierarchical.json', 'w') as json_file:
    json.dump(json_data, json_file)

Thera