managing complex web forms

Question

I work on an application, the core of which is a very complex set of web forms. The source of this complexity

1. scale, some forms contain about 50 pages, which up to 30 questions on each page
2. complex rules, e.g. if the user answers "no" to question 4, then questions 5-15 do not apply, so should be removed from the form

Currently we use Angular Schema Form for the forms, but it doesn't provide a compelling way to manage large number of complex rules among form fields.

I don't have much experience with rules-based programming, but it seems to me that this might provide a better way of managing the rules among form fields. For example, using a rules-based approach we could define the rules associated with each form field, and the rules engine could use these to decide which field(s) to display next. Using our current approach, we achieve this with a massive amount of imperative JavaScript which is almost impossible to test and maintain.

If anyone has experience with developing such complex web forms, I'd be interested to hear about their experience e.g. tools/libraries they could recommend. Our current stack is based on the JDK (Java, Groovy, Grails) and JavaScript (Angular, Node) so tools/libraries/frameworks that run on those platforms would be of particular interest.

Mckeown

Have you compared to using angular formly? – Helico

@Helico no, I'm aware that formly is another very popular Angular library for developing forms, but amn't aware of the pros and cons compared with Angular Schema Form – Unwrap

Hard to really assess what would help without understanding the issues in more detail but it sure sounds to me like you might benefit by having multiple routes for each "form" wherein each route is it's own form determined by the decision tree and each sub route binds to a shared service. Might want to try testing out angular formly also and see if it helps you with more features that would benefit you – Helico

Try looking at the strategy pattern – Cannular

@IvanChaer the forms in question are not surveys – Unwrap

@AKADER I'm familiar with this pattern, but have no idea how it would help me solve this problem – Unwrap

This question is far too broad to be answered in the space allotted by SO. You're also looking for a recommendation, which at your rep you should know is off-topic. – Note

@Mckeown At the time of that comment (and this one), the question has an open bounty. Even someone with your reputation should know that means I can't vote to close. Someone with your reputation should also know that asking for recommendations is off-topic, so maybe our assumptions on what people should know at certain reputation levels are not quite correct. – Note

Answer 1

I have experience of building a solution for this type of process - admittedly not 50 form pages at a time but my use case did have long technical forms where sets of questions became hidden based on an earlier answer.

I wrote a design-time tool for an expert to set the questions. This tool presented the questions as a tree and allowed questions to be grouped into blocks to make the show/hide rules easier to manage for the author. These rules were encoded as simple 'Q10 > 123' meaning when answer to Q10 is greater than 123 then show this question (or block of questions). The answer required by the questions could be text input, drop down selections, etc.

The result of the design-time activity by the author was an XML definition of the question tree, question details (prompt, mandatory, type, options etc), and visibility rules.

At runtime I fed this to a JavaScript 'player' embedded in a web page. The player consumed the question definitions, set up a model, created the required display elements etc, generating the the HTML required to render the page. This included running a visibility check based on those rules. This check ran again when an answer was given so that any dependent questions could be hidden / come into view based on the given answer.

At the point where the user completed the form we saved the data to the DB by posting a hidden form to the server. This form included an 'isVisible' marker for each question which was also stored in the DB as a means to be able to quickly display the resulting answers and knowing which were hidden on the basis that if they were hidden in the form at the end of the session then they should be hidden when the results were displayed, printed or emailed etc.

I built mine originally in VBScript (IE was THE browser back then!), then moved it to JavaScript, and more recently updated with JQuery. It's been used millions of times and seems to be robust and practical.

In summary, you need a tool to help your experts set the questions for the forms; you need a means to save and transfer that information; you need a way to process the form & question definition for the user; you need a way to store the answers given; and you need a way to quickly re-display the results without needing the re-calculate show/hide on all the questions.

Ballman

OMG, I could REALLY use that tool you said you designed! Currently, I'm working on a system that is, ultimately a form, but builds a report based on responses. The form will probably have close to 400-500 questions and sub-questions based on the answers. Is this thing you made for sale or something? – Recession

No sorry but I have a side project in cold storage which is the same thing as a standalone service. – Ballman

Answer 2

If you want to use a rule engine ask yourself these questions. If the answers apply use it.

Is your algorithm a bunch of computations or is there major decision-making capability involved?

• If you have many conditions or your rules aren't very tabular(You might be able to store the rules in a DB) you can even take a mixed approach but it's a little harder to maintain because you have rules in different sections, but easier to program because you're only programming specific rules.

How complex are your decisions?

• If you have more than a couple of conditional statements

• You can clearly define the rules

How volatile are your rules?

• You don't know your rules yet or your rules need to be flexible and will change over time.

Does your algorithm need custom tweeking?

• If you don't need to tweek for things like performance use a rule engine

Cannular