Using GraphQL machinery, but return CSV

Question

A normal REST API might let you request the same data in different formats, with a different Accept header, e.g. application/json, or text/html, or a text/csv formatted response.

However, if you're using GraphQL, it seems that JSON is the only acceptable return content type. However, I need my API to be able to return CSV data for consumption by less sophisticated clients that won't understand JSON.

Does it make sense for a GraphQL endpoint to return CSV data if given an Accept: text/csv header? If not, is there a better practise way to do this?

This is more of a conceptual question, but I'm specifically using Graphene to implement my API. Does it provide any mechanism for handling custom content types?

Goldsberry

Answer 1

Yes, you can, but it's not built in and you have to override some things. It's more like a work around.

Take these steps and you will get csv output:

1. Add csv = graphene.String() to your queries and resolve it to whatever you want.

2. Create a new class inheriting GraphQLView

3. Override dispatch function to look like this:

   def dispatch(self, request, *args, **kwargs):
       response = super(CustomGraphqlView, self).dispatch(request, *args, **kwargs)
       try:
           data = json.loads(response.content.decode('utf-8'))
           if 'csv' in data['data']:
               data['data'].pop('csv')
               if len(list(data['data'].keys())) == 1:
                   model = list(data['data'].keys())[0]
               else:
                   raise GraphQLError("can not export to csv")
               data = pd.json_normalize(data['data'][model])
               response = HttpResponse(content_type='text/csv')
               response['Content-Disposition'] = 'attachment; filename="output.csv"'
   
               writer = csv.writer(response)
               writer.writerow(data.columns)
               for value in data.values:
                   writer.writerow(value)
       except GraphQLError as e:
           raise e
       except Exception:
           pass
       return response
   

4. Import all necessary modules

5. Replace the default GraphQLView in your urls.py file with your new view class.

Now if you include "csv" in your GraphQL query, it will return raw csv data and then you can save the data into a csv file in your front-end. A sample query is like:

query{
  items{
    id
    name
    price
    category{
        name
    }
  }
  csv
}

Remember that it is a way to get raw data in csv format and you have to save it. You can do that in JavaScript with the following code:

req.then(data => {
    let element = document.createElement('a');
    element.setAttribute('href', 'data:text/csv;charset=utf-8,' + encodeURIComponent(data.data));
    element.setAttribute('download', 'output.csv');

    element.style.display = 'none';
    document.body.appendChild(element);

    element.click();

    document.body.removeChild(element);
})

This approach flattens the JSON data so no data is lost.

Jennijennica

Answer 2

I have to implement the functionality of exporting list query into a CSV file. Here is how I implement extending @Sina method.

my graphql query for retriving list of users (with limit pagination) is

query userCsv{
    userCsv{
        csv
        totalCount
        results(limit: 50, offset: 50){
            id
            username
            email
            userType
        }
    }
}

Make CustomGraphQLView view by inheriting from GraphQLView and overide dispatch function to see if query has a csv also make sure you update graphql url pointing to this custom GraphQLView.

class CustomGraphQLView(GraphQLView):
    def dispatch(self, request, *args, **kwargs):
        try:
            query_data = super().parse_body(request)
            operation_name = query_data["operationName"]
        except:
            operation_name = None
        response = super().dispatch(request, *args, **kwargs)
        csv_made = False
        try:
            data = json.loads(response.content.decode('utf-8'))
            try:
                csv_query = data['data'][f"{operation_name}"]['csv']
                csv_query = True
            except:
                csv_query = None
            if csv_query:
                csv_path = f"{settings.MEDIA_ROOT}/csv_{datetime.now()}.csv"
                results = data['data'][f"{operation_name}"]['results']
                # header = results[0].keys()
                results = json_normalize(results)
                results.to_csv(csv_path, index=False)
                data['data'][f"{operation_name}"]['csv'] = csv_path
                csv_made = True
        except GraphQLError as e:
            raise e
        except Exception:
            pass
        if csv_made:
            return HttpResponse(
                status=200, content=json.dumps(data), content_type="application/json"
        )
        return response

Operation name is the query name by which you are calling. In previous example given it is userCsv and it is required because the final result as a response came with this key. Response obtained is django http response object. using above operation name we check if csv is present in the query if not present return response as it is but if csv is present then extract query results and make a csv file and store it and attach its path in response.

Here is the graphql schema for the query

class UserListCsvType(DjangoListObjectType):
    csv = graphene.String()
    class Meta:
        model = User
        pagination = LimitOffsetGraphqlPagination(default_limit=25, ordering="-id")

class DjangoListObjectFieldUserCsv(DjangoListObjectField):
    @login_required
    def list_resolver(self, manager, filterset_class, filtering_args, root, info, **kwargs):
        return super().list_resolver(manager, filterset_class, filtering_args, root, info, **kwargs)


class Query(graphene.ObjectType):
    user_csv = DjangoListObjectFieldUserCsv(UserListCsvType)

Here is the sample response

{
  "data": {
    "userCsv": {
      "csv": "/home/shishir/Desktop/sample-project/media/csv_2021-11-22 15:01:11.197428.csv",
      "totalCount": 101,
      "results": [
        {
          "id": "51",
          "username": "kathryn",
          "email": "[email protected]",
          "userType": "GUEST"
        },
        {
          "id": "50",
          "username": "bridget",
          "email": "[email protected]",
          "userType": "GUEST"
        },
        {
          "id": "49",
          "username": "april",
          "email": "[email protected]",
          "userType": "GUEST"
        },
        {
          "id": "48",
          "username": "antonio",
          "email": "[email protected]",
          "userType": "PARTNER"
        }
      ]
    }
  }
}

PS: Data generated above are from faker library and I'm using graphene-django-extras and json_normalize is from pandas. CSV file can be download from the path obtained in response.

Trike

Answer 3

GraphQL relies on (and shines because of) responding nested data. To my understanding CSV can only display flat key value pairs. This makes CSV not really suitable for GraphQL responses.

I think the cleanest way to achieve what you want to do would be to put a GraphQL client in front of your clients:

+------+  csv  +-------+  http/json  +------+
|client|<----->|adapter|<----------->|server|
+------+       +-------+             +------+

The good thing here is that your adapter would only have to be able to translate the queries it specifies to CSV.

Obviously you might not always be able to do so (but how are you making them send GraphQL queries then). Alternatively you could build a middleware that translates JSON to CSV. But then you have to deal with the whole GraphQL specification. Good luck translating this response:

{
  "__typename": "Query",
  "someUnion": [
    { "__typename": "UnionA", "numberField": 1, "nested": [1, 2, 3, 4] },
    { "__typename": "UnionB", "stringField": "str" },
  ],
  "otherField": 123.34
}

So if you can't get around having CSV transported over HTTP GraphQL is simply the wrong choice because it was not built for that. And if you disallow those GraphQL features that are hard to translate to CSV you don't have GraphQL anymore so there is no point in calling it GraphQL then.

Sememe