How to test twisted web resource with trial?

Question

I'm developing a twisted.web server - it consists of some resources that apart from rendering stuff use adbapi to fetch some data and write some data to postgresql database. I'm trying to figoure out how to write a trial unittest that would test resource rendering without using net (in other words: that would initialize a resource, produce it a dummy request etc.).

Lets assume the View resource is a simple leaf that in render_GET returns NOT_DONE_YET and tinkers with adbapi to produce simple text as a result. Now, I've written this useless code and I can't come up how to make it actually initialize the resource and produce some sensible response:

from twisted.trial import unittest
from myserv.views import View
from twisted.web.test.test_web import DummyRequest

class ExistingView(unittest.TestCase):
    def test_rendering(self):
        slug = "hello_world"
        view = View(slug)
        request = DummyRequest([''])
        output = view.render_GET(request)
        self.assertEqual(request.responseCode, 200)

The output is... 1. I've also tried such approach: output = request.render(view) but same output = 1. Why? I'd be very gratefull for some example how to write such unittest!

Misread

Answer 1

Here's a function that will render a request and convert the result into a Deferred that fires when rendering is complete:

def _render(resource, request):
    result = resource.render(request)
    if isinstance(result, str):
        request.write(result)
        request.finish()
        return succeed(None)
    elif result is server.NOT_DONE_YET:
        if request.finished:
            return succeed(None)
        else:
            return request.notifyFinish()
    else:
        raise ValueError("Unexpected return value: %r" % (result,))

It's actually used in Twisted Web's test suite, but it's private because it has no unit tests itself. ;)

You can use it to write a test like this:

def test_rendering(self):
    slug = "hello_world"
    view = View(slug)
    request = DummyRequest([''])
    d = _render(view, request)
    def rendered(ignored):
        self.assertEquals(request.responseCode, 200)
        self.assertEquals("".join(request.written), "...")
        ...
    d.addCallback(rendered)
    return d

Denotative

Thanks! It seems to work, but twisted never returns the result - I've checked with print, it waits after request.notifyFinish(). The slug I've tested works well when accessed. What could possibly be wrong? – Misread

It stops on 'request.setHeader' and request.setHost - most probably they're not implemented in DummyRequest (or one of them). – Misread

Yup it fails on: setHeader and getCookie - I think subclassing DummyRequest and creating cookie and setHeader functionality will solve this :) – Misread

Is there a way to make deferred which fires after request.finish() to be fired in this test enviorment too? It adds stats to db :] – Misread

I'm not sure what you mean. Anything using request.notifyFinish should work fine in this setup. Is that the kind of "deferred which fires after request.finish()" you mean? – Denotative

Sorry, it was a design flaw in my code. Your solution works perfectly :) – Misread

Answer 2

Here is a DummierRequest class that fixes almost all my problems. Only thing left is it does not set any response code! Why?

from twisted.web.test.test_web import DummyRequest
from twisted.web import server
from twisted.internet.defer import succeed
from twisted.internet import interfaces, reactor, protocol, address
from twisted.web.http_headers import _DictHeaders, Headers

class DummierRequest(DummyRequest):
    def __init__(self, postpath, session=None):
        DummyRequest.__init__(self, postpath, session)
        self.notifications = []
        self.received_cookies = {}
        self.requestHeaders = Headers()
        self.responseHeaders = Headers()
        self.cookies = [] # outgoing cookies

    def setHost(self, host, port, ssl=0):
        self._forceSSL = ssl
        self.requestHeaders.setRawHeaders("host", [host])
        self.host = address.IPv4Address("TCP", host, port)

    def addCookie(self, k, v, expires=None, domain=None, path=None, max_age=None, comment=None, secure=None):
        """
        Set an outgoing HTTP cookie.

        In general, you should consider using sessions instead of cookies, see
        L{twisted.web.server.Request.getSession} and the
        L{twisted.web.server.Session} class for details.
        """
        cookie = '%s=%s' % (k, v)
        if expires is not None:
            cookie = cookie +"; Expires=%s" % expires
        if domain is not None:
            cookie = cookie +"; Domain=%s" % domain
        if path is not None:
            cookie = cookie +"; Path=%s" % path
        if max_age is not None:
            cookie = cookie +"; Max-Age=%s" % max_age
        if comment is not None:
            cookie = cookie +"; Comment=%s" % comment
        if secure:
            cookie = cookie +"; Secure"
        self.cookies.append(cookie)

    def getCookie(self, key):
        """
        Get a cookie that was sent from the network.
        """
        return self.received_cookies.get(key)

    def getClientIP(self):
        """
        Return the IPv4 address of the client which made this request, if there
        is one, otherwise C{None}.
        """
        return "192.168.1.199"

Misread

It actually does set 404. Is it possible that I have to set code 200 manually in my Resource class? – Misread

request.setResponseCode() is usually called in ResourceSubclass.render() or .processingFailed() methods. – Ozuna

In _render() function above we call request.render(). Might it be caused by request using NOT_DONE_YET & deferreds? – Misread