Porting Autodesk Animator Pro to be cross platform

Question

a previous relevant question from me is here Reverse Engineering old paint programs

I have set up my base of operations here: http://animatorpro.org wiki coming soon.

Okay, so now I have a 300,000 line legacy MSDOS codebase. It's sort of a "be careful what you wish for" situation. I am not an experienced C programmer. I'm not entirely inexperienced either, but for all intents and purposes I'm a noob to the language and in particular the intricacies of its libraries. I am especially ignorant of the vagaries of the differences between C programs written specifically for MSDOS and programs that are cross platform. However I have been studying this code base for over a year now, and this is what I know about Animator Pro:

Compilers and tools used:

• Watcom C compiler
• tcmake (make program from Turbo C)
• 386asm, a specialised assembler for the Phar Lap dos extender
• and of course, the Phar Lap dos extender itself.
• a selection of obscure dos utilities

Much of the compilation seems to be driven by batch files. Though I have obtained copies of all these tools, I have not yet succeeded at compiling it. (though I have compiled its older brother, autodesk animator original.

It's got a plugin system that replicates DLL before DLL's were available, based on REX. The plugin system handles:

• Video Drivers (with a plethora of included VESA drivers)
• Input drivers (including wacom tablets, and keyboards)
• Drawing Tools
• Inks (Like photoshop's filters, or blending modes)
• Scripting Addons (essentially compiled scripts)
• File formats

It's got its own script interpreter named POCO, based on the C language- The scripting language has enough power to do virtually all the things the plugin system can do- Just slower.

Given this information, this is my development plan. Please criticise this. The source code is available in the link above, so you can easily, if you are so inclined, assess the situation yourself.

1. Compile with its original tools.
2. Switch to using DJGPP, and make the necessary changes to get it to compile with that, plus the original assembler.
3. Include the Allegro.cc "Game" library, and switch over as much functionality to that library as possible- Perhaps by simply writing new video and input drivers that use the Allegro API. I'm thinking allegro rather than SDL because: there is a DOS version of Allegro, and fascinatingly, one of its core functions is the ability to play Animator Pro's native format FLIC.
4. Hopefully after 3, I will have eliminated most or all of the Assembler in the project. I say hopefully, because it's in an obscure dialect that doesn't assemble in any modern free assembler without significant modification. I have tried them all. Whatever is left gets converted to assemble in NASM, or to C code if I can define the assembler's actual function.
5. Switch the dos extender from Phar Lap to HX Dos http://www.japheth.de/HX.html, Which promises to replicate as much of the WIN32 api as possible. Then make all the necessary code changes for that to work.
6. Switch to the win32 version of Allegro.cc, assuming that the win32 version can run on top of HXDos. Make any further necessary changes
7. Modify the plugin system to use some kind of standard cross platform plugin library. What this would be, I have no idea. Maybe you can offer some suggestions? I talked to the developer who originally wrote the plugin system, and he said some of the things it does aren't possible on modern OS's because of segmentation restrictions. I'm not sure what this means, but I'm guessing it means all the plugins will need to be rewritten almost from scratch.
8. Magically, I got all the above done, and we can try and make it run in windows, osx, and linux, whilst dealing with other cross platform niggles like long file names, and things I haven't thought of.

Anyone got a problem with any of this? Is allegro a good choice? if not, why? what would you do about this plugin system? What would you do different? Is this whole thing foolish, and should I just rewrite it from scratch, using the original as inpiration? (it would apparently take the original developer "About a month" to do that)

One thing I haven't covered above is the text/font system. Not sure what to do about that, but Animator Pro has its own custom font format, but also is able to use Postscript Type 1 fonts, and some other formats.

Plasm

Answer 1

My biggest concern with your plan, in a nutshell: Your approach seems to be to attempt to keep the whole enormous thing working at all times, tweaking the environment ever-further away from DOS. During each tweak to the environment, that means you will have approximately a billion subtle assumptions that might have broken at once, none of which you necessarily understand yet. Untangling them all at once will be incredibly painful.

If I were doing the port, my approach would be to disable as much code as possible to get SOMETHING running in a modern environment, and bring the parts back online, one piece at a time. Write a simple test harness program that loads a display driver and draws some stuff, and compile it for DOS to make sure you understand the interface. Then write some C code that implements the same interface, but with Allegro (or SDL or SFML), and make that program work under Windows or Linux. When the output differs, you have a simple test case to work from.

Your entire job on this port is swapping out implementations of various interfaces and functions with completely new ones. This is a job that unit testing excels at. Don't write any new code without a test of some kind that runs on the old code under DOS! Make your potential problems as small and simple as you possibly can. Port assembly code instead of rewriting it only if you're reasonably confident that it will actually make your job easier (ie, algorithmic stuff that compiles fine with few tweaks under NASM). Don't bite off a bigger piece than you can comfortably fit in your brain at once.

I, for one, look forward to seeing your progress! I think what you're attempting to do is great. Thanks for doing it.

Chordate

Good advice. I have also thought about that approach. There's something of an advantage in that the build process actually builds 3 programs- The main monsterous paint program itself, one that does nothing but read in a list of FLICs and play them in sequence (with some other tricks to make it useful for "power point" type tasks), (player) which is probably the simplest, and one that does nothing more than load in images in various formats, and save in various other formats. (Crop). My thought was to do the above sequence, but with Player first, which would familiarise me with the codebase... – Plasm

enough to know what I can cut out and still have the thing work. – Plasm

As for tests- Not familiar enough with C or DOS to know the best way to set up a test harness. At this point, I'm just trying to get it to compile at all, since just the passage of time has been enough to change the Watcom C compiler enough that this source code doesn't compile without a fair amount of modification. (A huge bulk of the functions don't even have prototypes! I get tons of warnings about that alone) ...But there are some strange "tests" built into various parts of the code, that try to initialise a struct with a character array, the number of members are determined... – Plasm

by a relational expression... for example {char x0[sizeof(Fli_head) == 128];} ... If the size of Fli_head isn't 128, then it tries to initialize a character array with 0 members, which throws a cryptic compiler error, drawing my attention to that bit of code. The assumption that's broken here is the size of the int types being used in the file format- and in a zillion other places. I haven't figured out how to solve this problem yet. – Plasm

Looks like the original implementor was forward-thinking enough that you could tweak the typedefs in INC\STDTYPES.H to solve that particular issue, thank goodness. The code appears to assume that SHORT is a 16-bit integer, and LONG is a 32-bit integer. Lucky for you, it looks rather like this codebase was intended to be somewhat portable, which makes sense given that it started life on the Atari ST. – Chordate

@Jeremy Penner that's what I thought, so I changed the typedefs over to some stuff from stdint.h and that appears to have only confused it more, due partly to the inconsistent use of types from STDTYPES.H. A bunch of other stuff has suddenly intervened the past couple days so I haven't had a chance to take a close look at what's going on there. – Plasm

If your C compiler has stdint.h, then it might be worth importing that, then be explicit about the bit widths - use int32 instead of LONG and int16 instead of SHORT. – Glisten

Answer 2

Hmmm - I might approach it by writing an OpenGL video "driver" for it. and todays machines are fast enough with tons of ram that you could do all the pixel specific algorithms on main CPU into a back buffer and it would work. As the "generic" VGA driver just mapped the video buffer to a pointer this would be a place to start. There was a zoom mode in the UI so you can look at the pixels on a high res display.

Selfassured

Answer 3

It is often very difficult to take an existing non-trivial code base that wasn't written with portability in mind - you mention a few - and then try to make it portable. There will be a lot of problems on the way. It is probably a better idea to start from scratch and rewrite the code using the existing code as reference only. If you start from scratch you can leverage existing portable UI solution in your new project like Qt.

Pushing

I guess my fear in doing it that way is losing the original character of the program. – Plasm

@Breton. Humbly speaking, any kind of tool that was designed to work on low res & limited colours IS going to lose its character once ported to modern setups. the amazing 640 x 480 back then looks really tiny on any modern HD screen. – Pennyweight

@Pennyweight I get what you mean- However in this particular case the original program is in fact fully capable of running at fully resolution on a modern setup, as long you can write a video driver for it. In my mind it's like really liking star wars episode IV and wanting to avoid making an Episode 1. Or say, taking the original super mario bros and remaking it using 3D models and cheesy voice acting. – Plasm

in retrospect my last comment here really made no sense. Disregard. :D – Plasm

to be honest I think still that in your situation you are better off starting from scratch. porting an app that wasn't designed for it is often like trying to push a square through a round hole. very frustrating. – Pushing