RestClient strips out the array of hashes parameter with just the last hash?

Question

I have a condition where i need to pass a parameter as an array of hashes which looks like this:

The following is the Rack::Test post method for API call.

post "#{url}.json",
:api_key => application.key,
:data => [{"Company"=>"Apple,Inc","Website"=>"Apple.com"},{"Company"=>"Google","Website"=>"google.com"}],
:run => { :title => "The First Run" }

And this is the log of the rails app.

Parameters: {"api_key"=>"6a9acb84d0ea625be75e70a1e04d26360606ca5b", "data"=>[{"Company"=>"Apple,Inc", "Website"=>"Apple.com"}, {"Company"=>"Google", "Website"=>"google.com"}], "run"=>{"title"=>"The First Run"}, "line_id"=>"4e018e2c55112729bd00000a"}

Now, this is the RestClient post method I'm using to call the API.

RestClient.post("/lines/#{@line.id}/runs.json", {:run => {:title => @title}, @param_for_input => @param_data})

And this is the log of the rails app.

Parameters: {"run"=>{"title"=>"run name"}, "data"=>{"Company"=>"Google", "Website"=>"google.com"}, "api_key"=>"f488a62d0307e79ec4f1e6131fa220be47e83d44", "line_id"=>"4e018a505511271f82000144"}

The difference is in the data parameter.

When sending with Rack::Test method, the data is passed as "data"=>[{"Company"=>"Apple,Inc", "Website"=>"Apple.com"}, {"Company"=>"Google", "Website"=>"google.com"}]

but via RestClient way, the parameter data array is stripped out and only the last hash is passed as "data"=>{"Company"=>"Google", "Website"=>"google.com"}

Why the RestClient is stripping out the array of hashes to just a last hash of the array?

Isochroous

Are you sure @param_data is what you think it is? I'm using RestClient and haven't seen any issues like this. – Twospot

This is the value of @param_data = [{:Company => "Google", :Website => "google.com"}, {:Company => "Times", :Website => "times.com"}] – Isochroous

Answer 1

I suspect it's to do with differences in how they convert a hash into params. Rack::Test will probably be using Hash#to_param, which gives the following results:

> params = {:api_key => "12345", :data => [{"Company"=>"Apple,Inc","Website"=>"Apple.com"},{"Company"=>"Google","Website"=>"google.com"}], :run => { :title => "The First Run" }}

> paramstring = params.to_param
 => "api_key=12345&data%5B%5D%5BCompany%5D=Apple%2CInc&data%5B%5D%5BWebsite%5D=Apple.com&data%5B%5D%5BCompany%5D=Google&data%5B%5D%5BWebsite%5D=google.com&run%5Btitle%5D=The+First+Run" 

> URI.unescape(paramstring)
 => "api_key=12345&data[][Company]=Apple,Inc&data[][Website]=Apple.com&data[][Company]=Google&data[][Website]=google.com&run[title]=The+First+Run" 

This is the troublesome part:

data[][Company]=Apple,Inc&data[][Website]=Apple.com&data[][Company]=Google&data[][Website]=google.com

The rails uri parser has to read this and turn it back into a hash. In my opinion putting an array of hashes into your params is asking for trouble as it creates a string, like the above, which is fundamentally difficult to parse. For example, presented with these two params

data[][Company]=Apple,Inc
data[][Company]=Google

The parser may decide that both of them are describing the Company variable in the first hash in the array called "data", and so overwrite the first with the second, which is what's happening with you.

It sounds like your problem is at the generation stage rather than the intepretation stage, but still, i would try to create a cleaner scheme for your parameters, in which arrays are only ever used as the final part of the param name, (ie use a hash instead of an array to hold company data) and you instead insert some unique keys to differentiate the company hashes from each other. Something like this:

{:api_key => "12345", 
 :data => {1 => {"Company"=>"Apple,Inc","Website"=>"Apple.com"}, 2 => {"Company"=>"Google","Website"=>"google.com"}}, 
 :run => { :title => "The First Run" }}

1 and 2 could be the actual ids of some company record, or they could just be some numbers you put in to make unique keys, which are chucked away at the other end. This will generate params like this:

data[1][Company]=Apple,Inc
data[2][Company]=Google

Which are now in no danger of overwriting each other.

In your subsequent controller action, it's just a change from doing this:

params[:data].each do |company_hash|
  #do something with company hash
end

to

params[:data].each do |k, company_hash|
  #do something with company hash and optionally k if you want, or ignore k
end

Unmanly

This might have something to do with RestClient itself. Coz when I make the post request using HTTParty, it works as expected without any tweak in my code. options = { :body => { :api_key => Api_key, :data =>{:run => { :title => "runsdsd" }, :inputs => [{"Company"=>"Apple,Inc"},{"Company"=>"Google"}] } } } run = HTTParty.post("#{Url}/api/v1/lines/kapil/line2/runs.json",options) – Isochroous

Answer 2

I ran into the same problem with our rails application. I found the following workarounds to work with RestClient + Rails backend.

Rails is expecting data[][Company]. Use 'data[]' instead of 'data' as the key. For example:

RestClient.post 'http://archive.greenviewdata.com/containers', { 
  :run => {:title => 'something'}, 
  'data[]' => [
    {"Company"=>"Apple,Inc","Website"=>"Apple.com"},
    {"Company"=>"Google","Website"=>"google.com"}
  ]
}

In our case, we had an array nested two levels deep in the hash. The above workaround doesn't fix the problem because of the way RestClient formats the parameters. So, if you have an array that's nested deeper than the top level of the hash passed to RestClient, you have to use the following workaround:

RestClient.post 'http://archive.greenviewdata.com/containers', {
  :run => {:title => 'something'}, 
  :nested => {
    'data' => {
      '' => [
        {"Company"=>"Apple,Inc","Website"=>"Apple.com"},
        {"Company"=>"Google","Website"=>"google.com"}
      ]
    }
  }
}

Insphere

this didn't work for me. we found that that native json format doesn't handle arrays as expected and were forced to use the {"1"=>{...data...},{"2"=>{...data2}} method mentioned in the accepted answer which is a native "array" in json – Dagan

The second solution(deeply nested) worked for us. I am wondering how you managed to figure out this craziness?. Anyway thank you. – Timberhead