Django Tastypie slow POST response
Asked Answered
H

2

6

I'm trying to implement a Tastypie Resource that allows GET & POST operations following a per user-permission policy, the model is pretty simple (similar to the Note model in Tastypie documentation) and the resource itself is also pretty simple, I just have an extra override_urls method to implement search with Haystack.

My main problem now is that although running the project locally seems to be working fine, requests are fast and everything. Once I deployed the project (On Linode, using Nginx, Gunicorn, Runit), I discovered that POST requests are too slow, taking about a 1.1 min to come back with a 201 status. GET requests on the other hand are working well and as expected.

I ran a Python Hotshot profiler on the request and it's showing that the entire POST request is taking 0.127 CPU seconds. I'm not really sure what's happening here.

I should mention that I'm using ApiKeyAuthentication and DjangoAuthorization for my Tastypie resource.

Here's a screenshot from Chrome Inspector for the request: http://d.pr/i/CvCS

It would be great if anyone can direct me into the correct direction to look for an answer for this problem.

Thanks!

Edit:

Some code:

Models & Resource:

class Note(models.Model):
    timestamp = models.DateTimeField('Timestamp')
    user = models.ForeignKey(User)
    page_title = models.CharField("Page Title", max_length=200)
    url = models.URLField('URL', verify_exists=False)
    summary = models.TextField("Summary")
    notes = models.TextField("Notes", null=True, blank=True)

    def __unicode__(self):
        return self.page_title

    def get_absolute_url(self):
        return self.url


class NoteResource(ModelResource):
    user = fields.ForeignKey(UserResource, 'user')

    class Meta:
        queryset = Note.objects.all()
        resource_name = 'note'
        list_allowed_methods = ['get', 'post']
        detail_allowed_methods = ['get']
        always_return_data = True
        authentication = ApiKeyAuthentication()
        authorization = DjangoAuthorization()
        # authentication = Authentication() #allows all access
        # authorization = Authorization() #allows all access

        ordering = [
            '-timestamp'
        ]

    def override_urls(self):
        return [
            url(r"^(?P<resource_name>%s)/search%s$" % (
                self._meta.resource_name, trailing_slash()),
                self.wrap_view('get_search'), name="api_get_search"),
        ]

    def obj_create(self, bundle, request=None, **kwargs):
        return super(NoteResource, self).obj_create(bundle,
                                                        request,
                                                        user=request.user)

    def apply_authorization_limits(self, request, object_list):
        return object_list.filter(user=request.user)

    def get_search(self, request, **kwargs):
        self.method_check(request, allowed=['get'])
        self.is_authenticated(request)

        sqs = SearchQuerySet().models(Note).filter(
                                        user=request.user
                                    ).auto_query(
                                        request.GET.get('q', '')
                                    )

        paginator = Paginator(sqs, 100)

        try:
            page = paginator.page(int(request.GET.get('page', 1)))
        except InvalidPage:
            raise Http404("Sorry, no results on that page.")

        objects = []

        for result in page.object_list:
            bundle = self.build_bundle(obj=result.object, request=request)
            bundle.data['score'] = result.score
            bundle = self.full_dehydrate(bundle)
            objects.append(bundle)

        object_list = {
            'objects': objects,
        }

        self.log_throttled_access(request)
        return self.create_response(request, object_list)

Gunicorn Conf:

bind = "0.0.0.0:1330"
workers = 1

Nginx Conf (included in the main nginx.conf):

server {
        listen 80;
        server_name domain.com example.com;
        access_log  /path/to/home/then/project/access.log;
        error_log /path/to/home/then/project/error.log;

        location / {
                proxy_pass   http://127.0.0.1:1330;
        }

        location /static/ {
                autoindex on;
                root /path/to/home/then/project/;
        }
}
Hwu answered 1/3, 2013 at 20:12 Comment(7)
Courtesy generally dictates that you should identify that this has been cross posted on the google group as well. Do you have any signal processing rules set on the Model? In particular post_save?Dellinger
Thanks @BrandonBertelsen, you're right, I should've mentioned that. Sorry. I do have a post_save signal, but it's pretty simple, basically just filling the Summary field.Hwu
Can you run a test excluding your post_save action?Dellinger
@BrandonBertelsen i did that and i got the same result, that's why i didn't mention it in the post in the first place because it couldn't be causing itHwu
Figured it out. In the main nginx.conf file (/etc/nginx/nginx.conf), turns out I had keepalive_timeout set on 65, which is considered too much. I switched it to 0 and everything worked ok.Hwu
@DarthBaker perhaps you should add the solution as an answer, so that this issue doesn't look like an unsolved one ?Vally
Is there any reason why you're using a single worker in your Gunicorn config?President
A
3

OP: Figured it out. In the main nginx.conf file (/etc/nginx/nginx.conf), turns out I had keepalive_timeout set on 65, which is considered too much. I switched it to 0 and everything worked ok.

Sorry, I spend a couple of minutes with this question, then realized there were more comments and then realized the OP did find a solution :( and did not mark it as answered.

Acyl answered 25/3, 2013 at 7:9 Comment(0)
H
0

While changing the keepalive_timeout will work, it doesn't fix nginx's underlying Content-Length header bug that is causing it. This bug was fixed in version 0.8.32 of nginx, but if you have an older version you can either:

  1. Change the keepalive_timeout=0 on your server
  2. Upgrade nginx on your server to a version >= 0.8.32
  3. Fix the issue in your server side code as explained here

Hopefully this helps anyone else who stumbles across this problem.

Hitherto answered 21/5, 2015 at 16:19 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.