Rails pagination in API using Her, Faraday

Question

I've been trying to figure this out all day, and it's driving me crazy.

I have two rails apps, ServerApp and ClientApp. ClientApp gets data from ServerApp through an API, using the Her gem. Everything was great until I needed pagination information.

This is the method I am using to get the orders (this uses kamainari for pagination and ransack for search):

# ServerApp
def search
  @search = Order.includes(:documents, :client).order('id desc').search(params[:q])
  @orders = @search.result(distinct: true).page(params[:page]).per(params[:per])

  respond_with @orders.as_json(include: :documents)
end

It returns an array of hashes in json, which Her uses as a collection of orders. That works fine.

# Response
[
  {
    "client_id": 239,
    "created_at": "2013-05-15T15:37:03-07:00",
    "id": 2422,
    "ordered_at": "2013-05-15T15:37:03-07:00",
    "origin": "online",
    "updated_at": "2013-05-15T15:37:03-07:00",
    "documents": [
      { ... }
    ]
  },
  ...
]

But I needed pagination information. It looked like I needed to send it as metadata with my json. So I change my response to this:

respond_to do |format|
  format.json do
    render json: { orders: @orders.as_json(include: :documents), metadata: 'sent' }
  end
end

This does indeed send over metadata, so in my ClientApp I can write @orders.metadata and get 'sent'. But now my orders are nested in an array inside of 'orders', so I need to use @orders.orders, and then it treats it like an array instead of a Her collection.

After doing some reading, it seemed sending pagination info through headers was the way a lot of other people did this (I was able to get the headers set up in an after_filter using this guide). But I am even more lost on how to get those response headers in my ClientApp - I believe I need a Faraday Middleware but I just am having no luck getting this to work.

If anyone knows how I can just get this done, I would be very grateful. I can't take another day of banging my head against the wall on this, but I feel like I am just one vital piece of info away from solving this!

Caning

evt. this might also be an issue for the Her project. You can open an issue there, and see if you get feedback from the project maintainers – Heliocentric

I had considered posting it to the Her github page, but I thought this was more a misunderstanding on my part and not an issue on their part. – Caning

Answer 1

I encountered the same issue and solved it by adding my own middleware and rewriting the "parse" and "on_complete" methods without that much hassle and avoiding the use of global variables.

Here's the code:

   class CustomParserMiddleware < Her::Middleware::DefaultParseJSON
      def parse(env)
         json = parse_json(env[:body])
         pagination = parse_json(env[:response_headers][:pagination_key]) || {}
         errors = json.delete(:errors) || {}
         metadata = json.delete(:metadata) || {}
         {
           :data => json,
           :errors => errors,
           :metadata => {
              :pagination => pagination,
              :additional_metadata => metadata
            },

     end

      def on_complete(env)
        env[:body] = case env[:status]
           when 204
             parse('{}')
           else
            parse(env)
         end
      end
    end

then, you can access the pagination as follows:

    model = Model.all
    model.metadata[:pagination]

Laurentium

Answer 2

I finally got this working. The trick was to use a global variable in the faraday on_complete - I tried to find a better solution but this was the best I could do. Once again, I got the header code from here. Here's the full guide to how to get pagination working with Her:

First, on my server side, I have the Kaminari gem, and I pass page and per as params to the server from the client. (This is also using ransack for searching)

def search
  @search = Order.order('id desc').search(params[:q])
  @orders = @search.result(distinct: true).page(params[:page]).per(params[:per])

  respond_with @orders.as_json(include: :items)
end

My client makes the request like so:

@orders = Order.search(q: { client_id_eq: @current_user.id }, page: params[:page], per: 3)`

Back on the server, I have this in my ApiController (app controller for api):

protected
  def self.set_pagination_headers(name, options = {})
    after_filter(options) do |controller|
      results = instance_variable_get("@#{name}")
      headers["X-Pagination"] = {
        total_count: results.total_count,
        offset_value: results.offset_value
      }.to_json
    end
  end

In the server orders_controller.rb, I set the pagination headers for the search method:

class OrdersController < ApiController
  set_pagination_headers :orders, only: [:search]
  ...
end

Now to receive the headers we need a Faraday middleware in Her on the client.

# config/initializers/her.rb
Her::API.setup url: Constants.api.url do |c|
  c.use TokenAuthentication
  c.use HeaderParser # <= This is my middleware for headers
  c.use Faraday::Request::UrlEncoded
  c.use Her::Middleware::DefaultParseJSON
  c.use Faraday::Adapter::NetHttp
  c.use Faraday::Response::RaiseError
end

# lib/header_parser.rb
# don't forget to load this file in application.rb with something like:
# config.autoload_paths += Dir[File.join(Rails.root, "lib", "*.rb")].each { |l| require l }

class HeaderParser < Faraday::Response::Middleware
   def on_complete(env)
    unless env[:response_headers]['x-pagination'].nil?
      # Set the global var for pagination
      $pagination = JSON.parse(env[:response_headers]['x-pagination'], symbolize_names: true)
    end
  end
 end

Now back in your client controller, you have a global variable of hash called $pagination; mine looks like this:

$pagintation = { total_count: 0, offset_value: 0 }`

Finally, I added Kaminari gem to my client app to paginate the array and get those easy pagination links:

@orders = Kaminari.paginate_array(@orders, total_count: $pagination[:total_count]).page(params[:page]).per(params[:per_page])`

I hope this can help someone else, and if anyone knows a better way to do this, let me know!

Caning

Answer 3

• You can pass header options to Faraday when setting up the connection, see the docs at http://rubydoc.info/gems/faraday/0.8.7/Faraday/Connection:initialize

• Sometimes it helps to do a curl request first, esp. use -vv option for verbose output where you will see all headers. (Maybe you can attach some log outputs from the Server too)

• You can use e.g. clogger (http://clogger.rubyforge.org/) do monitor header information on the Rails server side

Heliocentric

I am actually already adding headers in Faraday using the middleware that is posted on Her's github page. The problem is I don't fully understand how to write the on_complete middleware to get the headers being sent back from the API, and then make them available to my controller. And I am using Postman Chrome extension, and the headers are there - I just am not sure how to use them! The furthest I've seen a guide go in describing what to do with headers involved just writing to a log. – Caning