What criteria should I use to evaluate a Perl "app server" (mod_perl replacement)?

Question

Short version:

What criteria should I use to evaluate possible candidates for a Perl "app server" (mod_perl replacement)?

We are looking for some sort of framework which will allow executing various Perl programs repeatedly (as a service) without the costs of:

1. re-launcing perl interpreter once per each execution

2. loading/compiling Perl modules once per execution

(both of which are the benefits that running mod_perl provides)

Notes:

• We do NOT much care about any additional benefits afforded by mod_perl too much, such as deep Apache integration.

• This will be a pure app server, meaning there is no need for any web specific functionality (it's not a problem if the app server provides it, just won't be needed).

• We will of course consider the obvious criteria (raw speed, production-ready stability, active development, ability to run on OSs we care about). What I'm interested in is less trivial and subtle things that we may wish from such a framework/server.

Background:

At $work, the powers that be decided that they want to replace a current situation (simple webapps being developed in Embperl and deployed via Apache/mod_perl).

The decision was made to use a (home-grown) MVC system that will have a Java Spring front end for the View; and the Controller will parsel out back-end service requests to per-app services that perform Model duties (don't get hung up on details of this - it's not very relevant to the main question).

One of the options for back-end services is Perl, so that we can leverage all our existing Perl IP (libraries, webapp backend code) going forward, and not have to port 100% of it to Java.

To summarize:

    | View    | Model/app | Model loaded/executed by:                          |
================================================================================
OLD | Empberl | Model.pm | mod_perl has Model.pm loaded, called from view.epl  |
NEW | Java    | Model.pm | perl generic_model.pl -model Model (does "require") |
================================================================================

Now, those of you who did Perl Web development for a while, will immediately notice the most glaring problem with the new design:

    | Perl interpreter starts  | Perl modules are loaded and compiled |
=======================================================================
OLD | Once per mod_perl thread | Once per mod_perl thread
NEW | Once per EVERY! request  | Once per EVERY! request              |
=======================================================================

In other words, in the new model, we no longer have any performance benefits afforded by mod_perl as a persistent server side app container!!!

Therefore, we are looking at possible app containers to serve the same function.

(as a side note, yes, we thought about simply running an instance of Apache with mod_perl as such an app container, as a viable possibility. However, since web functionality is not required, I'd like to see if any other options may fit the bill).

Annmarieannnora

What protocol will be used to make the java and the perl parts talk to each other? – Decompose

But can't you keep the "parallelism" of one start up by just continuing to run the services in an apache mod_perl environment (or PSGI) and talking to them all from your new Java Swing thingie? Is that too slow? At least the perl interpreter is then running and waiting and ready. – Dendrochronology

Sorry I meant my previous comment to be a further question/response regarding to your side note. Have you tested this mod_perl app container approach? It seems that www.apache.org ran or runs Qpsmtpd to help handle their mailing list delivery in this way (i.e. embedded as a SMTP service in an apache mod_perl instance). So apache and mod_perl ... "they're not just for the web" :-) – Dendrochronology

@Decompose - protocol isn't terribly relevant (as in, flexible). The current app-server-less design was simply command line interface, passing equivalent of a query strring on STDIN and emitting JSON responses to STDOUT. – Annmarieannnora

@G.Cito - Yes, mod_perl is one of the possible replacements. However we need to see if any better alternatives exist. – Annmarieannnora

Potential community wiki candidate? – Dendrochronology

Answer 1

Starman is a High-performance preforking PSGI/Plack web server that may be used in that context. It's easy to build a REST application that serves stateless JSON objects (this is a simple use case).

Starman is a production-ready server and it's really easy to install a set of Starman instances behind a reverse-proxy (this SO question may helps you), for scaling purposes

Gabion

Answer 2

I think you've already identified what you need to know and what to test: execution time versus memory. You also need to evaluate the flexibility and ease of deployment that you get with mod_perl and the big win of preserving the usefulness of software you've already developed (and the accumulated experience of your staff). Remember you can easily separate services by CPU and machine if your new front end is going to be talking to your applications inside your own network. A lot depends on how "web-ish" you can make your services and if they can be efficiently "daemonized". Of course there's lots of ways for web front ends to talk to other services and perl can handle pretty much all of them ... TIMTOWTDI.

Since you mention "tuits" (i.e. "manpower") as a constraint, perhaps the best approach in the short term is to set up a separate apache - mod_perl instance as an "application container" and run your applications that way (since they run well that way already, is this correct?). After all, apache (and mod_perl) are well known, battle tested, and eminently tweakable and configurable. The deployment options are pretty flexible (same machine, different machine(s), failover, load balancing, cloud, local, VMs) and they have been well tested as well.

Once you get that running you could then begin experimenting with various "low manpower required" approaches to magically daemonizing your applications and services without apache. Plack and Starman have been mentioned, Mojolicious:: is another framework that is capable of working with JSON websockets etc (and Plack). These too have been well tested but are perhaps less familiar than mod_perl and Apache. Still if you are a perl shop you shouldn't have difficulty working with these "modern" tools. One day, if you do end up with more resource, you could build out a sophisticated networked platform for all your perl based services and use all the cool "new" (or at least newer than mod_perl) stuff on CPAN like POE, Plack, etc. You might have a lot of fun developing cool stuff as you solve your business problem.

To clarify my earlier comment: Ubic (see Ubic on MetaCPAN) can daemonize (and thus precompile) your perl tools and offers some monitoring and management facilities that come for free with the framework. There is one Ubic module available designed for use with Plack called Ubic::Service::Plack. In and of itself Ubic does not provide an easy solution for your new Java/Swing application to talk to your perl applications but it might help manage/monitor whatever solution you come up with.

Good luck and have fun!

Dendrochronology

" if they can be efficiently "daemonized"." - they can not (reasons aren't techical, but manpower requreiments to write and test daemons), otherwise we wouldn't need an app server, obviously :) – Annmarieannnora

Note that with Ubic you can "daemonize" pretty much anything (just as you can with runit, daemontools, s6, etc). I'll edit my answer to clarify that. A bit of research comparing the responsiveness of a perl script running as a "daemon" under Ubic versus starting it up per request or under mod_perl might help. But to make that useful we'd need to know how you plan to have the web service and the backends communicate with one another. As @Miguel notes it would be fairly easy to use JSON for this. – Dendrochronology

Answer 3

You can create a simple daemon using HTTP::Daemon, and have all benefits of compiling necessary parts of your code later (require), or in advance, before daemon starts.

Smoothen