Delphi style: How to structure data modules for unit-testable code?

Question

I am looking for some advice about structuring Delphi programs for maintainability. I've come to Delphi programming after a couple of decades of mostly C/C++ though I first learned to program with Turbo Pascal, so I'm not uncomfortable with the basic language. In my previous experience with C++ and C#, I became a TDD convert through using cxxtest and NUnit.

I have inherited this program that I am now responsible for maintaining. It consists mainly of forms, and a couple of data modules. The application business logic and data access is mainly scattered about the forms, and the data modules are mostly just places for global ADO objects to live. The database access is generally done by referring to a global instance of TADOQuery or TADOCommand, formatting SQL text right into the relevant property of the object, and calling its Open or Execute method.

I am trying to get the business logic into a degree of encapsulation where it can be unit-tested. I've seen this answer and it makes perfect sense as far as abstracting logic from forms. I am wondering what the best practices are for data access. My thinking is that the data modules should expose a sort of app-specific mini-API (probably with all virtual methods) so that they can be replaced with mock objects for testing. The link at this other answer shows some examples that lead me to believe I'm on the right track, but I'm still interested in seeing some kind of best practices document about data modules. Most of the pages that I can find through Google present the same kind of examples about all the cool stuff you can do at design time with hooking up data-bound controls to queries and that sort of thing, which I'm not very interested in at the moment.

Schlosser

Do the existing forms use data aware controls (TDBEdits) or standard controls (TEdits)? – Paracasein

No data aware controls so far. I've been thinking about that. – Schlosser

Data aware controls tend to make an application harder to unit test. Although I'm a fan of them myself I'd recommend you steer clear of them at this stage. – Paracasein

Answer 1

Personally I'm not a fan of TDataModule. It does very little to encourage good OO design principles. If all it was used for was a convenient container for DB components that would be one thing but far too often it becomes a dumping ground for business logic that would be better off in a domain layer. When this happens it winds up becoming a god class and a dependency magnet.

Add to this a bug (or maybe its a feature) that's continued to exist since at least Delphi 2 that causes a form's data aware controls to lose their data sources if those data sources are located in a unit that isn't opened before the form.

My suggestion

• Add a domain layer between your UI and your database
• Push as much of your business logic into domain objects as possible.
• Make your UI and your data persistence layers as shallow as possible by using design and architectural patterns to delegate decision making to the domain layer.

If you're not familiar with it the technique is referred to as domain driven design. Its certainly not the only solution but its a good one. The basic premise is that the UI, business logic and database change at different rates and for different reasons. So make the business logic a model of the problem domain and keep it separated from the UI and database.

How does this make my code more testable?

By moving the business logic to its own layer you can test it without interference from from either the UI or the database. This doesn't mean your code will be inherently testable simply because you put it in its own layer. Making legacy code testable is a difficult task. Most legacy code is tightly coupled so you will spend a good deal of time pulling it apart into classes with clearly defined responsibilities.

Is this the Delphi style?

That depends on your perspective. Traditionally, most Delphi applications were created by developing the UI and the database in tandem. Drop a few db aware controls on the form designer. Add/update a table with fields to store the control's data. Sprinkle with a liberal amount of business logic using event handlers. Viola! You just baked an application. For very small applications this is a great time saver. But lets not kid ourselves, small applications tend to turn into big ones and this design becomes an unsustainable maintenance nightmare.

This really isn't the fault of the language. You find the same quick/dirty/shortsighted designs from hundreds of VB, C# and Java shops. These kinds of applications are the result of novice developers that don't know any better (and experienced developers that should know better), an IDE that makes it so easy to do and pressure to get the job done quickly.

There are those in the Delphi community (as there are in other communities) that have been advocating better design techniques for a long time.

Dingo

Data Modules aren't at fault for "big ball of mud" designs. A data module is simple a class that can support non visual controls. What you do with that is up to you. Don't blame the guy who made a shovel, if you dig yourself into a pit. – Columbuscolumbyne

A poor analogy. It takes less time to fill in a pit than it does to dig it. The same cannot be said for a poorly designed application. My point is the lack of encapsulation actually encourages poor design. In the same way the ease and convenience of double-clicking a form control to create event handlers encourages coupling business logic directly to the UI. The reason there is so much crappy code is because the tools make it so easy to write it that way and in some cases actually hinder you from good design. – Dingo

Answer 2

I think you need (and in fact, most delphi database developers are going to need) a Mock Dataset (Query, table, etc etc) Component that you could use, and substitute them at module-init time, for your current ADO dataset objects to this mock dataset, for test purposes. Instead of forcing Interfaces into your design, which are one way to provide a substitution capability, consider the fact that by Liskov substitution principle, you should be able to (at test fixture setup time), inject into your data module, the set of mock-datasets that you want to use, and simply replace the ADO datasets that you are using, at test execution time, with some other functionally equivalent entity (a mock dataset, or a file-backed table dataset).

Perhaps you could even remove the datasets completely from the data module, and have them hooked up at runtime (in your main application) to the correct ADO dataset objects, and in unit tests, attach your mock datasets.

Since you did not write the ADO dataset, you don't need to unit test it. However, mocking up such a dataset might be difficult.

I would suggest you consider using a JvCsvDataSet or a ClientDataSet as the basis for your fixture (mock) datasets. You would be able to then use these to make sure that all your database platform dependencies (stuff that writes remote procedures or database SQL) is abstracted out into other classes, which again you are going to have to mock up. Such an effort might not only be required to make your business logic unit testable, it might also be a step towards becoming multiple-database-platform friendly in your business logic.

imagine you have an ADOQuery called CustomerQuery, rename the object that you dropped onto your data module, to CustomerQueryImpl, and add this to your data module class declaration:

  private
        FCustomerQuery:TADOQuery;

  published
        property CustomerQuery:TADOQuery read FCustomerQuery write FCustomerQuery;

then in your data module on create event, hook up the property to the objects:

   FCustomerQuery := CustomerQueryImpl

Now you can write unit tests, which will 'hook' in and replace CustomerQuery with its own test fixture (mock object) at runtime.

Columbuscolumbyne

Answer 3

Firstly before you change anything you need some unit tests so you can ensure you don't break anything. I would attempt to write unit tests against the current GUI without changing anything. DUnit has support for GUI testing (along with traditional unit testing) and although it's a little clunky and can't handle modal dialogs it is functional.

Next, since your forms don't use data aware controls I would approach this by introducing another layer of data modules, a service layer if you will, between the forms and the existing global data modules.

For every form in your application I would create a corresponding new service layer data module. This may sound like a lot of data modules but they're very lightweight and you can consolidate them later if you want.

You could use ordinary TObjects rather than TDataModules for the service layer if you liked however using data modules gives you the flexibility of being able to place non-visual components on them later, for example a TClientDataSet and TDataSource if you went down the data-aware controls route at a later date.

Initially each service layer data module would merely act as a proxy for accessing the global data modules. Your goal at this point would be simply to remove the direct dependency of the forms on the global data modules.

Once the forms only indirectly accessed the global data modules via the service layer data modules then I would start to move functionality from the forms into the service layer. With this functionality in the service layer data modules you will find it much easier to write unit tests for new and existing code.

At this point you could also start consolidating the per-form service layer data modules. It will be much easier to consolidate them now after the logic extraction from the forms is complete than if you try do do it during that process.

Paracasein

Answer 4

Please read this article, its about Unit Testing and Mock Objects including the theory of mock objects, localizing UT and interfaces discovery.

hope you enjoy it.

Burke

You don't need dependency inversion and interface discovery to mock things in Delphi. Unit name aliases are usually enough. Delphi already has an Interface section. You don't have to ISomethingEverything to make mock objects (fixtures) work. – Columbuscolumbyne