Faster way to compile Factor Programs
Asked Answered
Y

1

8

I really love the Factor language. But I find that compiling programs written in it is incredibly slow, and thus it's not feasible to create real projects with Factor.

For example, compiling the sample Calculator WebApp takes about 5 minutes on my laptop (i3 processor, 2GB RAM, running Fedora 15).

I've searched around but couldn't find a faster way to compile Factor programs than using the interpreter (the main factor binary executable).

It becomes ridiculous when you attempt to only use the interpreter for each run and not "deploy" your program to a native binary file (which doesn't even work on many programs). It means that every time I want to run the Calculator, for example, I have to wait a 5 minute cold-start duration.

I'd like to know whether this is a common issue, and whether there's a good way to tackle it.

Yerxa answered 6/10, 2011 at 11:5 Comment(5)
would you please mention on what platform this is? Since 5 minutes seems overly pathetic for the linked Calculator example I'd say there is something wrong with the platform/infra, not with the compiler mechanics (It. Couldn't. Possibly. Be. That. Slow?)Elater
Yes, sorry. I'm running Fedora 15, fully updated. I downloaded the "stable" release of Factor 2 days ago.Yerxa
Ok, will have a look on linux late tonightElater
I'm sorry can't figure out what went wrong with my Fedora setups. Trying some other linux distro instead...Elater
Re: which doesn't even work on many programs - I had to export LD_PRELOAD=/usr/lib/libgtkglext-x11-1.0.so even only to start the UI listener. Factor is dynamically loading all it's dependencies. You might want to strace -e trace=dlopen,dlsym to find out what failsElater
E
15

I admit that before today, I had never heard of Factor. I took the opportunity to play with it. It is looking nice (reminds me of squeak-vm and lisp at the same time). I'll cut the smalltalk (pun very much intended) and jump to your question.

Analysis

It appears that the way Factor works, results in loading vocabularies being slow.

I compiled Factor on my 64 bit quadcore linux system (from git revision 60b1115452, Thu Oct 6). Putting everything on tmpfs the build dir takes 641Mb, of which 2x114Mb is in the factor.image and its backup (factor.image.fresh).

When strace-ing the calculator app loading, there is a huge list of factor files being loaded:

  • 3175 factor files are touched.
  • compilation of these takes roughly 30 seconds on my box
  • the memory usage maxes out on just under 500Mb (virtual) and 300Mb reserved set

I'm strongly suspecting your box is low on memory, and might be getting very swappy
This would definitely explain compilation taking 5 minutes

Can you confirm whether this is the case (likely if you are running some kind of shared host or VPS appliance). If you run a virtual machine, consider increasing the available system memory.


Saving Heap Images (snapshots)

I already mentioned the factor.image file (114Mb) before. It contains a 'precompiled' (bootstrapped, actually) heap image for the Factor VM. All operations in the VM (working on the UI listener or compiling factor files) affects the heap image.

To avoid having to recompile your source files time and time again, you can save the end-result into a custom heap image:

http://docs.factorcode.org/content/article-images.html

Images

To start Factor with a custom image, use the -i=image command line switch; see Command line switches for the VM.

One reason to save a custom image is if you find yourself loading the same libraries in every Factor session; some libraries take a little while to compile, so saving an image with those libraries loaded can save you a lot of time.

For example, to save an image with the web framework loaded,

USE: furnace
save

New images can be created from scratch: Bootstrapping new images

Deploying applications

Saving heap images results in files that will (typically) be bigger than the original bootstrap image.

The Application deployment tool creates stripped-down images containing just enough code to run a single application

The stand-alone application deployment tool, implemented in the tools.deploy vocabulary, compiles a vocabulary down to a native executable which runs the vocabulary's MAIN: hook. Deployed executables do not depend on Factor being installed, and do not expose any source code, and thus are suitable for delivering commercial end-user applications.

Most of the time, the words in the tools.deploy vocabulary should not be used directly; instead, use Application deployment UI tool.

You must explicitly specify major subsystems which are required, as well as the level of reflection support needed. This is done by modifying the deployment configuration prior to deployment.

Concluding

I expect you'll benefit from (in order of quickest win):

  • increasing available RAM (only quick in virtual environments...)
  • saving a heap image with
USE: db.sqlite
USE: furnace.alloy
USE: namespaces
USE: http.server
save

This step brought the compilation on my system down from ~30s to 0.835s

  • deploying the calculator webapp to a stripped heap image (refer to the source for deployment hints)

In short, thanks for bringing Factor to my attention, and I hope my findings will be of any help, Cheers

Elater answered 7/10, 2011 at 0:22 Comment(2)
Swapping might just be the reason. Even when only running several pages in Google Chrome I'm already using up a few bytes of swap. Thanks a lot for the in-depth answer, it was a joy and and inspiration to read.Yerxa
@YamMarcovic: thanks. Did you try the save image way? In retrospect that is even easier than expanding memory (but you might still want to do that :)) It brought down the compile time to 0.8s when I tried itElater

© 2022 - 2024 — McMap. All rights reserved.