Twisted Tutorials – VII

This article is going to deal with another aspect of Deferreds, which normally is a bit non-trivial to realise,so we are going to present a complex situation, which will demonstrate the problem, and how deferred solves the problem.

To motivate our discussion we’re going to add a hypothetical feature to our poetry client. Suppose some hard-working Computer Science professor has invented a new poetry-related algorithm, the Byronification Engine. This nifty algorithm takes a single poem as input and produces a new poem like the original, but written in the style of Lord Byron. What’s more, our professor has kindly provided a reference implementation in Python, with this interface:

class IByronificationEngine(Interface):

    def byronificate(poem):
        Return a new poem like the original, but in the style of Lord Byron.

        Raises GibberishError if the input is not a genuine poem.

Like most bleeding-edge software, the implementation has some bugs. This means that in addition to the documented exception, the byronificate method sometimes throws random exceptions when it hits a corner-case the professor forgot to handle.

We’ll also assume the engine runs fast enough that we can just call it in the main thread without worrying about tying up the reactor. This is how we want our program to work:

  1. Try to download the poem.
  2. If the download fails, tell the user we couldn’t get the poem.
  3. If we do get the poem, transform it with the Byronification Engine.
  4. If the engine throws a GibberishError, tell the user we couldn’t get the poem.
  5. If the engine throws another exception, just keep the original poem.
  6. If we have a poem, print it out.
  7. End the program.

The idea here is that a GibberishError means we didn’t get an actual poem after all, so we’ll just tell the user the download failed. That’s not so useful for debugging, but our users just want to know whether we got a poem or not. On the other hand, if the engine fails for some other reason then we’ll use the poem we got from the server. After all, some poetry is better than none at all, even if it’s not in the trademark Byron style.

Here’s the synchronous version of our code:

    poem = get_poetry(host, port) # synchronous get_poetry
    print >>sys.stderr, 'The poem download failed.'
        poem = engine.byronificate(poem)
    except GibberishError:
        print >>sys.stderr, 'The poem download failed.'
        print poem # handle other exceptions by using the original poem
        print poem


This sketch of a program could be make simpler with some refactoring, but it illustrates the flow of logic pretty clearly. We want to update our most recent poetry client (which uses deferreds) to implement this same scheme. But we won’t do that until Part 10. For now, instead, let’s imagine how we might do this with client 3.1, our last client that didn’t use deferreds at all. Suppose we didn’t bother handling exceptions, but instead just changed the got_poem callback like this:

def got_poem(poem):

What happens when the byronificate method raises a GibberishError or some other exception? Looking at Figure 11 from Part 6, we can see that:

  1. The exception will propagate to the poem_finishedcallback in the factory, the method that actually invokes the callback.
  2. Since poem_finished doesn’t catch the exception, it will proceed to poemReceived on the protocol.
  3. And then on to connectionLost, also on the protocol.
  4. And then up into the core of Twisted itself, finally ending up at the reactor.

As we have learned, the reactor will catch and log the exception instead of crashing. But what it certainly won’t do is tell the user we couldn’t download a poem. The reactor doesn’t know anything about poems or GibberishErrors, it’s a general-purpose piece of code used for all kinds of networking, even non-poetry-related networking.

I think the concept is rather simple, and dealing with it just requires plain reading. So I won’t be going after copying the content. You can refer to the website for rest of the stuff (


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s