Is there any performance comparison between Perl web frameworks?
Asked Answered
O

4

15

I have seen mentions (which sounded like unsubstantiated opinions, and dated ones at that) that Embperl is the fastest Perl web framework.

I was wondering if there's a consensus on the relative speed of the major stable Perl web frameworks, or ideally, some sort of fact-based performance comparisons between implementations of the same sample webapps, or individual functionalities (e.g. session handling or form data processing), etc...?

UPDATE: This question is specifically about the speed comparison of different frameworks, executing identical/equivalent tasks. I appreciate the good intentions, but I already know that speed is not the only criteria I should be looking at. I wasn't asking for philosophical advice. And believe it or not, being frameworks, you CAN actually compare their speed on an apple-to-apple basis by running identically purposed tasks/code/apps on them (e.g. render a given form with a given set of templated inserts etc...), even if the full functionality of each framework is not 100% the same.

Ostium answered 14/1, 2011 at 1:16 Comment(7)
you would not compare a trailer to a sports car, would you? If you need a kitchen the speed of the sports car does not matter at all. So: It depends ;)Asternal
You might have to narrow it down a little.. "Web framework" is such a huge category.. and, of course, speed depends on how you use it etc. Looking at the code might help, but I guess other decisions affects performance more then the "speed" of the code..Primordial
Regarding your update: i still dont think it makes much sense to put a lot of efford into comparisons where the results render mostly useless. So i dont think anyone has done it.Asternal
@Mugen - if you have a choice of 3-4 frameworks and a list of functional requirements that are supported by ALL of them, speed matters. To make the example in your first comment more realistic, if you need 2 seats, a seatbelt, and enough space to store a gym bag, all over the sudden sports car sounds better than a trailer since they both qualify. If I needed a kitchen, I'd phrase my question very specifically around "Which Perl web frameworks have a kitchen?"Ostium
Øyvind Skaar - You are somewhat correct (there are other factors to performance) but some underlying platform differences DO exist (see for example the link in bvr's answer for proof).Ostium
-1 You really aren't too clued up about this. If you ask only for the speed of a framework with no regard to the type of task it is performing then you will get wildly differing results, and there will be no winner. You must define a typical workload, and from there benchmark tests will offer you an initial selection. You must also remember that, if the internet is involved, then the connection speed will swamp any minor differences in processing the requests. And if you are using a database then your attention is probably best focussed there than on the web server. Voting to closeDedededen
@Dedededen - No internet. No DB (in-memory cached data on an app server as backend). The types of tasks are wildly varied. If you don't understand why someone needs to know the answer, doesn't make it a bad question. Spouting 100% obvious things that are already noted in the question under "but I already know that speed is not the only criteria I should be looking at" isn't a way to display your cluefulness.Ostium
S
9

Here is one comparison between perl frameworks, in terms of speed (startup) and memory consumed by framework itself. It is a bit old (2008), so it does not compare new stuff like Plack.

http://mark.stosberg.com/blog/2008/11/startup-benchmarks-for-mojo-catalyst-titanium-httpengine-and-cgiapplication.html

Strobilaceous answered 14/1, 2011 at 14:6 Comment(3)
thanks!!! +1 and I'm accepting this for now unless something majorly more comprehensive/useful gets added in the future.Ostium
and all: Keep in mind the benchmarks are for a cgi environment. For most of the frameworks, you would want to use mod_perl.Tahsildar
I wrote the comparison above, and would not choose a framework based the above. Once you have a real application for your own purpose, benchmark that in combination. You may find that in the big picture, the contribution that the framework plays is small, and a slower framework may be a better choice because of other characteristics besides speed.Phallus
A
12

I don't want to go into the interpretation discussion (for most real world scenarios these overheads have no impact at all) - but here are my tests:

1. Pure Plack

zby@zby:~/progs/bench$ cat app.psgi 

sub {
   my ( $env ) = @_;
   return [
       200,
       [ 'Content-Type' => 'text/text' ],
       [ 'Hello World' ]
       ];
}
zby@zby:~/progs/bench$ plackup
HTTP::Server::PSGI: Accepting connections at http://0:5000/

With simple ab -n 10000 I get

Requests per second: 2168.05 [#/sec] (mean)

2. Dancer

zby@zby:~/progs/bench$ cat dancer.pl 
 #!/usr/bin/perl
           use Dancer;

           get '/' => sub {
               return "Why, hello there";
           };

           dance;
zby@zby:~/progs/bench$ perl dancer.pl 
>> Dancer server 1950 listening on http://0.0.0.0:3000
== Entering the development dance floor ...

With similar ab test I get Requests per second: 1570.49 [#/sec] (mean)

3. Mojolicious::Lite

zby@zby:~/progs/bench$ cat mojo.pl 
 # Using Mojolicious::Lite will enable "strict" and "warnings"
    use Mojolicious::Lite;

    # Route with placeholder
    get '/' => sub {
        my $self = shift;
        $self->render(text => "Hello!");
    };

    # Start the Mojolicious command system
    app->start;
zby@zby:~/progs/bench$ perl mojo.pl daemon
Sat Jan 22 20:37:01 2011 info Mojo::Server::Daemon:320 [2315]: Server listening (http://*:3000)
Server available at http://*:3000.

Result: Requests per second: 763.72 [#/sec] (mean)

4. Catalyst.

Unfortunately the code is much too long to be presented here in its entirety, but the Root controller contains:

sub index :Path :Args(0) {
    my ( $self, $c ) = @_;

    # Hello World
    $c->response->body( 'Hello World' );
}

The result is:

Requests per second: 727.93 [#/sec] (mean)

5. WebNano

zby@zby:~/progs/bench$ cat webnano.psgi

{
    package MyApp;
    use base 'WebNano';
    1;
}

{
    package MyApp::Controller;
    use base 'WebNano::Controller';

    sub index_action {
        my $self = shift;
        return 'This is my home';
    }
    1;
}    
MyApp->new()->psgi_callback;
zby@zby:~/progs/bench$ plackup webnano.psgi 
HTTP::Server::PSGI: Accepting connections at http://0:5000/

And the result:

Requests per second: 1884.54 [#/sec] (mean)

This will change after some more features are added.

Argillite answered 22/1, 2011 at 20:35 Comment(3)
This is somewhat meaningless, as you are really benchmarking the web server. Try your Plack app running on Feersum (instead of HTTP::Server::PSGI), and you'll probably get 10x the number of requests per second. Try actually doing something in your application, and you'll find that the performance is the same on all the frameworks.Troublemaker
OK - I've added some more disclaimers. It's just data - it has as much meaning as your interpretation will put into it. For example you can use this data to support your thesis that for real-world applications the overhead generated by the framework is negligible :)Argillite
You are comparing web server (plackup, mojo daemon, etc)Heaton
S
9

Here is one comparison between perl frameworks, in terms of speed (startup) and memory consumed by framework itself. It is a bit old (2008), so it does not compare new stuff like Plack.

http://mark.stosberg.com/blog/2008/11/startup-benchmarks-for-mojo-catalyst-titanium-httpengine-and-cgiapplication.html

Strobilaceous answered 14/1, 2011 at 14:6 Comment(3)
thanks!!! +1 and I'm accepting this for now unless something majorly more comprehensive/useful gets added in the future.Ostium
and all: Keep in mind the benchmarks are for a cgi environment. For most of the frameworks, you would want to use mod_perl.Tahsildar
I wrote the comparison above, and would not choose a framework based the above. Once you have a real application for your own purpose, benchmark that in combination. You may find that in the big picture, the contribution that the framework plays is small, and a slower framework may be a better choice because of other characteristics besides speed.Phallus
L
7

I know this doesn't answer you directly but I don't think an up to date comparison exists and I know a comprehensive one doesn't. It would take a couple weeks of work, at least, to do a thorough benchmark because there are so many frameworks in Perl right now with so many DB/Template/Server permutations and different kinds of usage patterns of the app could make major performance differences too.

I do believe you will be missing a lot by taking Mark's simple 2008 benchmarks as the answer to your quest. Deployment matters as much as if not more than the web framework for speed. For example, Catalyst is not going to win a raw "hello world" speed war but the BBC's video Catalyst application can serve 1,000 concurrent videos. Flexibility, scalability, and support of different deployments becomes a big factor in picking a web framework.

Plack is new and major. In just one year it's seen a huge adoption, middleware/plugin growth, and support from just about every framework. The Starman engine for plack apps is surprisingly fast and supports hot reloads and worker process increment/decrement. Since almost all the Perl frameworks can run as .psgi now you could run whatever you want on Starman + nginx (or lighttpd). There are dozens of good deployment combinations and quite a few changes and new entries in the web framework space in the last two years.

If you're doing modern web stuff, make sure to pick a kit with websocket support. That alone will increase performance dramatically over traditional Ajax; small requests/responses can be a factor of 100 times smaller/lighter with websockets.

Sidenote: modperl is probably not the best persistent deployment to pick at this point unless you have a need for the deep hooks into the request cycle. It has many caveats and wrinkles and ties you to apache (a great server but not the fastest option by a long shot).

Happy hunting!

Update 20 October 2016: uWSGI is a fantastic match for PSGI apps in Perl.

Lorenzoloresz answered 14/1, 2011 at 19:47 Comment(3)
Apache/mod_perl may not be a flexible option for my search, but since I don't know for sure yet, I'm glad you mentioned other options. +1 for very thorough and thoughtful answer.Ostium
websockets sounds very intriguing - thanks for mentioning that idea. However, practically, they appear to have pretty much no cient side support (No IE, no production FF, and turned off due to security concerns in FF4 and Opera (en.wikipedia.org/wiki/WebSockets#Browsers_supporting_WebSocket)Ostium
@DVK, thanks! Re:websockets, yes, it's early but it's wonderful and coming quickly, IE 9 looks to have very good html5 support for example. I haven't tried this yet and some of the code looks a little naïve but a super cool idea: The Graceful WebSocket — lets you write websocket code that will degrade to Ajax when necessary.Lorenzoloresz
H
5

TechEmpower have a some good comparisons of framework performace. See http://www.techempower.com/benchmarks/

As of now, they have included 4 perl frameowkrs (Dancer, Kelp, Mojolicious, Web::Simple) as well as many frameworks from other languages. The results can be filtered to show only Perl frameworks if desired.

Huntlee answered 18/6, 2013 at 9:52 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.