ruby: Extracting fields from nested json

Question

I am trying to teach myself ruby and solve a problem at work. My ultimate goal is to extract out three of the many fields in JSON response from an API, manipulate and dump to CSV for executive reporting.

The structure of the JSON is:

{
  "status": 200,
  "data": {
    "total": 251,
    "alerts": [
      {
        "dataPoint": "x",
        "ackedBy": "x",
        "dataSourceInstance": "x",
        "dataSource": "x",
        "host": "x",
        "endOn": 0,
        "ackedOn": 1385085190,
        "dataSourceInstanceId": 588384,
        "hostId": 935,
        "type": "alert",
        "dataSourceId": 694,
        "ackedOnLocal": "2013-11-21 17:53:10 PST",
        "id": 6170384,
        "startOn": 1385084917,
        "thresholds": "!= 1 1 1",
        "endOnLocal": "",
        "level": "error",
        "ackComment": "x",
        "value": "No Data",
        "hostDataSourceId": 211986,
        "acked": true,
        "hostGroups": [{
          "alertEnable": true,
          "createdOn": 1362084592,
          "id": 21,
          "parentId": 78,
          "description": "",
          "appliesTo": "",
          "name": "2600hz",
          "fullPath": "x"
        }],
        "startOnLocal": "2013-11-21 17:48:37 PST"
      },

To be specific I want to extract out dataPoint,startOn,ackedOn.

I am thinking I need to extract the total value first, so I know how many alerts I have. That will help me loop through the alert instances.

*url = "https://xyz"
uri = URI(url)
http = Net::HTTP.new(uri.host, 443)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
req = Net::HTTP::Get.new(uri.request_uri)
response = http.request(req)

# Make sure we had a proper web response
if response.code == '200' then

# Set up that the total is two levels deep in the json
path = JsonPath.new('$.total')

# Extract out total field into variable
totalAlerts = path.on(response)

# Print hash to confirm value
pp totalAlerts

end*

I am stuck trying to just extract out the total. Output shows nothing:

sudo ruby alerts.rb
[]

Is there a better way to do this?

Spielman

Answer 1

Try this:

# just for demonstration, you would probably do json = JSON.parse(response)
json = {
  "status": 200,
  "data": {
    "total": 251,
    "alerts": [
      {
        "dataPoint": "x",
        "ackedBy": "x",
        ...

For just the total:

total = json['data']['total']

For the other values you asked about:

json['data']['alerts'].each do |alerts|
  # do whatever you want with these...
  alerts['dataPoint']
  alerts['startOn']
  alerts['ackedOn']

Now the question is, what do you want to do with the results? Do you want to collect them into a new hash? The json['data']['alerts'] is an array so you have to decide how you want to collect them. You could do:

collected_alerts = { 'dataPoints' => [], 'startOns' => [], 'ackedOns' => [] }
json['data']['alerts'].each do |alerts|
  collected_alerts['dataPoints'] << alerts['dataPoint']
  collected_alerts['startOns'] << alerts['startOn']
  collected_alerts['ackedOns'] << alerts['ackedOn']
end

Now you can get all those values in collected_alerts

Phthalein

I added this to test jsonResponse = JSON.parse(response) total = jsonResponse['data']['total'] pp total and get the following error: sudo ruby alerts.rb /usr/local/lib/ruby/2.1.0/json/common.rb:155:in initialize': no implicit conversion of Net::HTTPOK into String (TypeError) from /usr/local/lib/ruby/2.1.0/json/common.rb:155:in new' from /usr/local/lib/ruby/2.1.0/json/common.rb:155:in parse' from alerts.rb:37:in <main>' I ultimately want to load the values into hash or array to manipulate before putting it into a csv thanks! – Spielman

make it JSON.parse(response.body) – Phthalein

I have two more problems and then i am done. # Calculate mins it took to acknowledge alert and store it in timeToAcks # Formula is Ack time - Start time which would give us seconds / 60 to give mins. timeToAck = (alerts['ackedOn']-alerts['startOns'])/60 collected_alerts['timeToAcks'] << timeToAck When i run it i get this error alerts.rb:57:in -': nil can't be coerced into Fixnum (TypeError)from alerts.rb:57:in block in <main>' from alerts.rb:47:in each' from alerts.rb:47:in <main>' – Spielman

Whoa there cowboy, for the community to benefit the most from this, you need to a) mark this answer if it solves your problem. b) start a new question to get a new answer. – Phthalein

oh ok. i thought they are related to the original question. mark this as answered – Spielman

Answer 2

In my case, I had to convert a Logstash event into a JSON array to get a value from one of its objects, so what did it for me was writing these lines on a Ruby filter:

sources_arr = event.get('[message][body][sources]')
sources_json = JSON.parse(sources_arr.to_json)
event.set('my_source_uri', sources_json[0]['uri'])

Notice the JSON.parse(<array>.to_json)

Tight