C++ genetic programming: Invoking a linker/compiler, executing the compiled program and piping input / output

Question

This is a generic question, and although I'm pretty sure some of it's parts have been answered I want opinions rather than a broad discussion. I intend to do a master thesis on evolutionary computation and genetic programming, and I would like opinions from experts on Linux/C++ whether it would be possible to create source code files from the genetic program, invoke gcc to compile them, if they fail to compile to capture those reasons why they failed, if they do compile to execute the compiled program (as a new process) and then to be able to send input to that program and capture output and any raised exceptions (or crashes). I understand that the topic is too broad, but I would like to know if anyone thinks that this is undoable, silly to try, or even if there could be a better way on doing this.

Raceme

Not silly at all, but extremely time consuming. The intellectually challenging part is to come up with a good c++ dna to evolve. What are the topics/goals of your primordial soup? – Pistol

Is there a restriction to do this is C++? I think that there maybe be better options if you are going to have a software the changes itself... – Disharmonious

This isn't going to work. Having a mutation survive and eventually contribute to improvement is extremely unlikely. Compilers don't put up with single letter changes, they were designed to do the opposite. – Brightman

The dna is all the possible operators, statements, class, variable and assignment words we use in programming. The reason why I want to create a source file and compile it rather than execute a sequence of instructions (in linear or tree-based fashion) is because I believe it would be more realistic and easier to handle illegal programs, uncompilable programs, etc. There is no restriction to do this in C++, it's just the language i know best, have already done something similar and I believe will provide better performance than say, Java or Python. Mutations work on source code, not execution. – Hallel

Answer 1

Yes, it is possible to do this, and actually quite simple. You output source code to a temporary file (mkstmp), you fork/exec a compilation process (that outputs to a temporary file), you fork/exec the resulting program, before that you dup2 and pipe to plug an input and an output. It is basic Unix programming, nothing really complicated to do in C here.

The code generation itself may be more difficult to get right, but this heavily depends on the project.

Also, we have modern tools since a couple of months : I believe that Clang may definitively be something to look at for this kind of stuff. If the code generation you plan to do is simple (or not simple, but structured), you can also directly output LLVM code. It is not hard, and enables you to generate efficient just in time compiled code.

Haldi

Thank you very much for the links! It had not crossed my mind to use some other compiler. The code generation it's self will be a process of GP evolution, so if it does not compile, essentially it dies. If it compiles and crashes, segments, raises exceptions, etc it dies. Even if it compiles and runs smoothly, it still has to do what it is required in each scenario to do, in order to survive. I guess I could instead use Clang. The code generation will entirely depend on the generations and complexity of required task. – Hallel

Answer 2

What are you trying to optimize in this genetic program? Other than just looking for programs that run, what criteria would you be looking for? I don't quite get the point...

To be clear, the reason I ask this is because I only understand using genetic algorithms in an attempt to solve some sort of optimization problem. In this case you would have some sort of heuristics where you can evaluate all children of the process and then breed new offspring based on the heuristic and a selection criteria. I don't understand what the desired outcome of generating this source would be or how you would create heuristics to evaluate it.

Yuji

It is not a genetic algorithm trying to optimize something, it is a machine learning - artificial life, experiment to see if it is possible for a machine to evolve complex computer code, from which it can learn and evolve even more complex code. The programs functions, classes, etc will be evaluated should conform to special criteria in each case, and doing so will hopefully enable a creation of a "rule-set". For example in one of the first steps I would try evolving a function which returns which of the two inputs is greater. Later on, the engine should be able to use that function . – Hallel

Well, if you want to use a genetic algorithm, at some point in time you are going to have to form your problem in terms of something you want to optimize. You will also need to formalize heuristics that evaluate the source code that you are generating. I don't really see why what you are doing has anything to do with genetic algorithms. However cool it may seem to try to "evolve" a computer by randomly changing lines of source code, there doesn't really seem to be any method behind this. – Yuji

There is a method! Also, it is not an optimization algorithm, its an evolutionary algorithm in order to create viable, meaningful and usable code. Evaluation takes place by: a) compilation error checking b) exception checking c)failing to accomplish goal (input & output checking) whatever that may be. Like I said, I have done this before, my concern is not evaluation per se, but rather communication with the compiler and the invoked process in order to do the actual evaluation. Also, It is not a GA, it's a GP, no string encoding, fixed strings, etc. see: genetic-programming.org – Hallel