Preprocessing PHP to remove functionality from built files

Question

I've been reading about Phing and Ant and I'm not sure which, if either, of these tools are most useful for this scenario.

It could easily be debug statements etc, but I'll give you our literal scanario.

We have a free and premium version of a downloadable PHP app, and rather than including just a variable hidden somewhere and then doing:

if($premium == true) {
   echo 'some additional functionality';
} else {
    echo 'basic functionality';
}

Obviously someone could then take the source and change that variable, and bang - they've stolen our code. And something like Ioncube etc is just totally unwieldy in my experience and support on hosting companies is just not good enough.

I would prefer something.. perhaps similar to this:

## if premium ##
echo 'some additional functionality';
## else ##
echo 'basic functionality';
## endif ##

And then I would run two builds, one setting the premium to true and one to false, which would generate two files of simply:

echo 'some additional functionality';

and

echo 'basic functionality';

It would also be very helpful to be able to only include entire files based on this same condition passed to the build app.

I can't find a way to do this but I am open to any alternative ideas if possible.

Help would be outstanding,

UPDATE

Using the C preprocessor is great and looks like it does everything I need. However, I can't find how to do the following 3 things.

#1 I need to remove the comments generated into the output files. Below is an example of those.

# 1 "./index.php"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "./index.php"

I haven't found an example of how to do this in the manual page you linked me to.

#2 I need to make it recursively go through every discovered file. When I run my current code I get an error: ../target/./folder/test.php: No such file or directory

So basically I have my 'source' folder which I'm in, which contains a subfolder called 'folder' and it doesn't recreate that, nor the files inside it (test.php)

#3 I'm sure this one is easy - how can I get it to process .js files and probably .html just to be safe as well? In one call, I mean. I assume running it on .jpg etc etc files is a bad idea..

Thanks again!

Bluecollar

How you deploy your packages? If you are using PEAR for packaging, just create two different packages.xml (of course you cant name them both package.xml). – Whole

We actually don't use PEAR for packaging. Deployment is simply zipping the file and uploading it to our server. It's very basic in this area and simplicity is the key - there's only one source so versioning isn't needed etc. But I can't manually keep track of a bug in an unrelated area and merging that fix into god-knows how many customisations we'll end up with. – Bluecollar

Answer 1

It's pretty low-tech, but there is of course the C preprocessor which does exactly what you want; just bang in a couple of makefiles to call it with find or grep -R and you get a simple, easy-to-understand solution with syntax you probably know.

More detail

You probably have gcc installed already on any *nix host. Otherwise, it'll be a standard package. Some distributions provide it separately to to gcc (like Debian's cpp package).

The program has some simple instructions; the wiki page is a good start, and the manual has more detail than you need. Basically, it's a matter of calling it on each file with the -E option just to do the macro processing, and then copying the output some build directory.

You can write a one-liner script to do that with find, along of the lines of find <proj dir> -type f -name '*.php' -exec cpp -E -D <FULL or RESTRICTED> {} -o <build dir>/{} \; and reference the macros FULL and RESTRICTED in your PHP, like

#ifdef FULL
    <lines for paying customers>
#endif

UPDATE To make the paths work nicely, try this out:

#!/bin/bash
cd /.../phing/source/
find . -type f -name '*.php' -exec cpp -E -D FULL {} -o ../target/{} \;

Then ../target/{} should be expanded to ../target/./index.php.

UPDATE

Added -P to remove the linemarkers (#1). Added a line to copy directory structure (#2). Changed filename match to run on js on html (#3).

#!/bin/bash
cd /.../phing/source/
find . -type d -exec mkdir -p ../target/{} \;
find . -type f -regex '.*\.(php|html|js)' -exec cpp -E -P -D FULL {} -o ../target/{} \;

Approximation

That sounds like it might be perfect but do you think you could give me a bit more info as to what I'd need to install and roughly how my simple example would happen? – Bluecollar

How's that? A bit more detail. I suggest you look into git for versioning too. It's very lightweight, doesn't take too long to learn, and is a great way for keeping track of changes and rolling back regressions by finding what caused them. I use it also to manage content between different machines, so I can upload only the stuff which changed (see nicholaswilson.me.uk/view-source/system/scripts/site-bundle for an example). – Approximation

Versioning isn't a huge need for me right now (Although I've used SVN in the past). The detail is outstanding, however, I get an error opening the output file. find /home/marc/Server/phing/source/ -type f -name '*.php' -exec cpp -E -D FULL {} -o /home/marc/Server/phing/target/{} \; This gives me cc1: fatal error: opening output file /home/marc/Server/phing/target//home/marc/Server/phing/source/index.php: No such file or directory Your help so far has been outstanding - have I just messed up my directories? – Bluecollar

You could use sed to fiddle the path, or to keep it ultra-simple, just cd into the directory your source is in. If you know SVN, you might be pleasantly surprised by git, but it would take a bit of time to learn, so fair enough. – Approximation

So far this is absolutely brilliant - if I may, I have two final questions. Can I include combinations in one build call? (To satisfy multiple ifdef calls? For instance, FULL and DEBUG) And, if I just have an ifdef and I want to NOT define it (So I need to build one where FULL isn't defined - nothing is), how would I do that? – Bluecollar

And can I remove the embedded comments in the target? (Sorry to keep asking you but you've been incredibly helpful so far!) – Bluecollar

Yes: use -D FULL -D DEBUG. To check for a macro not defined, use #ifndef (see wikipedia for more examples). Stripping comments is even easier: that happens automatically anyway! – Approximation

See the -C and -CC options to cpp if you want to keep any comments (from the manual). – Approximation

That isn't quite what I meant; sorry for not being clear. Let's say I have an ifdef FULL and I want to just pass in.. nothing, so that that evaluates to false, how would I do that? Is it just removing the -D FULL completely? Also I meant, can I remove the comments that end up at the top of the generated file? – Bluecollar

Removing the output comments is actually vital as it breaks PHP execution if it's included in HTML. I've updated my original post to clarify my current questions! – Bluecollar

I've sorted the first question there. So, how can I remove the comments that end up at the top of the file? And also, how do I make it recurse through and prevent the error I added in my update? Last thing I need to solve! – Bluecollar

Got it recursing just by making another command to duplicate the directory structure before running this. Now, come on.. comments from the top of the newly-made file - how do I get rid of them? – Bluecollar

I don't check SO all that often. -P removes the output at the top. (Use Google and search SO: it's in this question). Answer updated. Keeping the other files in sync (images, css, etc) is a job for VCS or rsync, or whatever system you use to handle commits between devs. – Approximation

Brilliant. Couldn't find that. I have another question - is there a way to disable processing for a block of code? For instance, I have some inline CSS in my .php file and it uses a #some-id selector. That obviously breaks and messes up my code. If not, could I do an #include on a non-parsed filetype? Or does that get included and then parsed as well? – Bluecollar

No, sorry. CPP's an awesome general purpose macro-language, but it does need vaguely C-like syntax. There are plenty of CSS preprocessing languages like LESS, but I don't know if any handle conditional inclusion. It might be down to playing around with an existing tool to add that in and calling that from your build script. I think we've kind of done this question to death now though. Hope your project goes well. – Approximation

Answer 2

Sorry for digging up this topic, but I had the same need and the CPP approach had too many side-effects for my use.

So I developed a basic pre-processor filter for phing that does the trick:

#ifdef premium
  echo 'some additional functionality';
#else
  echo 'basic functionality';
#endif

The filter is available on github: https://github.com/tmuguet/PreProcessorFilter

Metatarsus

Answer 3

I have come to an idea, the answer before this is accepted but I want to share it anyways. So here is the deal:

Store all the payed features in seperate files. With a hook class, you can check and add the features where you want. Do not include the payed features on free version of the build. You will have 2 zip files but one source.

Here is very simple Hook class that might help you:

Class Hook {
    var $features_enabled;
    public function __construct() {
        // Construction codes goes here.
        // You can check features files. If there is, require them once
        // if there isnt any, define a variable:
        if (file_exists("./features/feature1.php")) {
            require_once './features/feature1.php';

            // set this true. so we can check the features and run it
            $this->features_enabled = TRUE;
        }
        else {
            // there is no feature avaliable. So there is no need to check functions.
            $this->features_enabled = FALSE;
        }
    }

    /**
     * Check the Feature.
     *
     * @param string The feature name
     * @param array|string|... The arguments that you want to pass.
     */
    public function check($feature_name, $args = NULL) {
        // if features cannot be included on constructor, do not need to check.
        if ($this->features_enabled == FALSE)
            return;

        // if feature is a function: check it then run it
        if (function_exists($feature_name))
            $feature_name($args);

        // if feature is a Class: create it then return it.
        if (class_exists($feature_name))
            return new $feature_name($args);
    }
}

Then you can check them in anywhere on your code. Example:

$hook = new Hook();

//.... Codes goes here.

// This will check for a function of class. If there is, execute it
$hook->check('feature_badges');

I know it's very simple and need to develop in many other ways. But If you can manage it:

• You will seperate the features. This way, you can create different packages.
• You will check the features with class. Even if user see the feature name, it doesn't matter. Because he cannot see the feature's code.

Aguie

Also it's a great idea, absolutely. However the app already exists and refactoring it in that way (i.e. classes providing functionality) would be difficult due to the scale. – Bluecollar

Answer 4

Well, Phing is the Ant for PHP. I've never used Phing, but I assume that it's tuned for stuff like this.

Use Phing to create two separate packages: One premium app and one for the free app.

You could use Ant too, but Ant is tuned for Java while Phing is tuned for PHP. The only reason you might want to use Ant over Phing is that there are much more resources available to Ant than Phing. But, if you're a PHP shop, learn to use Phing if for nothing else than it looks nice on a resume. ("Yes, I'm a senior PHP developer. I even know Phing").

Collocate

Absolutely, but very specifically how would that work in my source files? I'm exploring the CPP idea above but potentially a more robust solution in Phing might be better for the future. I haven't found a proper example of this, only straight token replacement (Doesn't help) or regular-expression matching, which feels.. pretty unwieldy to me. – Bluecollar

Perhaps the best solution using Phing would be to make two build targets, and use it to manage your preprocessing by calling the script or cpp directly. – Approximation

With Phing (as with Ant) you can use a filter to change the source as it's processed. I'd use that before cpp which is really part of C and not PHP. No reason why cpp won't work, but it's not part of PHP. Because of that, you're going to have some support/education issues when hiring new developers. Take a look at <ReplaceTokens> and <ReplaceTokensWithFile>. That'll probably be a better option. – Collocate

Phing's built-in filters unfortunately don't handle conditionals very well; that's the whole point. CPP doesn't really have anything to do with C or C++; it's just a general macro language for preprocessing, which is bundled with C because the system of packaging was primitive and they needed a way to include header files. CPP is simple and transparent, so perhaps calling it from Phing really is the easiest solution. – Approximation

Answer 5

You could just build such preprocessor by yourself - PHP has built-in support for parsing PHP files: http://www.php.net/manual/en/function.token-get-all.php

I have used it to built an obfuscator and must say it is very easy to use. Just print the tokens you wish and remove the tokens that are within "## if premium ##" blocks.

UPDATE: I guess C preprocessor idea is better. Leaving this answer for completeness though. :)

Sixfold

Cool idea but just the sentence 'you could build your own preprocessor' feels like it'd be false economy! I guess I prefer the idea of doing it in PHP, but hey. The C idea is working well except for my 3 questions above which I'm sure I'll get help with soon! – Bluecollar

Answer 6

Disclaimer: I am totally sure this is not the most efficient way to do this!

--

I do this in 4 parts:

• Clone the source directory structure in target
• Parse the PHP and JS files from source into target
• Copy over all the jpg, gif, css, etc files from source to target
• Remove the comments added by the cpp call from all PHP and JS files in target

--

find ./ -type d -exec mkdir ../target/{} \;
find ./ -regextype posix-awk -regex "(.*.php|.*.js)" -exec cpp -E -D COMMERCIAL {} -o ../target/{} \;
find ./ -type f -regextype posix-extended -regex "(.*.jpg|.*.gif|.*.png|.*.css|.*.html|.*.txt|.*.xml|.*.ttf)" -exec cp -p --parents "{}" ../target \;
cd ../target
find ./ -regextype posix-awk -regex "(.*.php|.*.js)" -exec sed -i ./{} -e 's/#.*//;/^\s*$/d' \;

--

As I said, I'm sure this isn't hugely inefficient but that there is one .sh file that I execute and bang - I have my builds!

Bluecollar

Obviously I execute that from the scope of the source directory. I hope this helps someone in a similar situation. – Bluecollar

Line 3, copying the other files: you can do -not -regex '.*\.(php|js)' to copy everything except those. Also see comment above about -P. If some of the files are big, the natural way to do this would be with rsync. I'm glad you got it working for you. – Approximation