Test Driven Development (TDD) for User Interface (UI) with functional tests
Asked Answered
M

7

29

As we know, TDD means "write the test first, and then write the code". And when it comes to unit-testing, this is fine, because you are limited within the "unit".

However when it comes to UI, writing functional tests beforehand makes less sense (to me). This is because the functional tests have to verify a (possibly long) set of functional requirements. This may usually span multiple screens (pages), preconditions like "logged in", "having recently inserted a record", etc.

According to Wikipedia:

Test-driven development is difficult to use in situations where full functional tests are required to determine success or failure. Examples of these are user interfaces, programs that work with databases, and some that depend on specific network configurations.

(Of course, Wikipedia is no "authority", but this sounds very logical.)

So, any thoughts, or better - experience, with functional tests-first for UI, and then code. Does it work? And is it "pain"?

Milagro answered 11/1, 2011 at 13:59 Comment(1)
I've used TDD successfully in many complex scenarios. TDD is certainly not limited to unit testing. Do you have a specific scenario you think would be painful to TDD?Spectator
B
18

Try BDD, Behavior Driven Development. It promotes writing specification stories which are then executed step by step stimulating the app to change it's state and verify the outcomes.

I use BDD scenarios to write UI code. Business requests are described using BDD stories and then the functionality is being written to turn the stories i.e. tests green.

Britzka answered 11/1, 2011 at 14:3 Comment(3)
So the specifications are written before the UI is built, but the automated UI test probably written after the UI was built, right?Wagram
That's the beauty of BDD @Steven, specification expressed in BDD scenarios is executable.Womera
Lightweight BDD for UI as a TDD strategy, to keep steps small and help define code modularity is good. You can augment them with auto-generated snapshot / replay tests for pixel perfect / signed off design tests.Nichols
F
11

The key to testing a UI is to separate your concerns - the behavior of your UI is actually different than the appearance of your UI. We struggle with this mentally, so as an exercise take a game like Tetris and imagine porting it from one platform (say PC) to another (the web). The intuition is that everything is different - you have to rewrite everything! But in reality all this is the same:

  • The rules of the game.
  • The speed of the blocks falling.
  • The logic for rows matching
  • Which block is chosen
  • And more...

You get the idea. The only thing that changes is how the screen is drawn. So separate how your UI looks from how it works. This is tricky, and usually can't be perfect, but it's close. My recommendation is to write the UI last. Tests will provide the feedback if your behavior is working, and the screen will tell if it looks right. This combination provides the fast feedback we're looking for from TDD without a UI.

Favianus answered 19/12, 2015 at 16:6 Comment(3)
btw as long as you mention some affiliation to a site, like "this is my domain that ..." probably no one is going to mind. TDD is one of your things. You can mention the link off your profile page too. Far be it from me to keep someone from sharing useful infoAragats
Tetris is a terrible example. Because it takes text input so you can get much more coverage with unit testing than with an app. The problem with user interface testing. Is that the input is clicks and the output is a pixel configuration. Now think about this, with an API the input is strings and the output is strings.Microminiaturization
I think though that TDD doesn't necessary mean unit tests or even automated tests. With UI it still makes sense to write the tests you will pass down, and then run those tests, and also rely on unit tests where possible.Microminiaturization
A
6

TDD for functional tests makes sense to me. Write a functional test, see it failing, then divide the problem into parts, write a unit test for each part, write code for each part, see unit tests pass and then your functional test should pass.

Here is a workflow suggested in the book Test-Driven Development with Python by Harry Percival (available online for free):

tdd workflow

P.S. You can automate your functional tests using e.g. Selenium. You can add them to the Continuous Integration cycle as well as unit tests.

Aneroid answered 1/8, 2013 at 20:21 Comment(0)
B
3

ATDD is very useful if you have a very good grasp on how the UI should behave and also ensure that the cost of change to the functional tests that you build upfront is less.

The biggest problem with doing this, of course, is most often the UI is not fully speced out.

For example, if you are building a product, and are still doing rapid iterations of getting a UI that goes through a usability testing and the feedback incorporated, you do not want the baggage of fixing your functional tests with every small change to the UI.

Given that functional tests are generally slow, the feedback cycle is high and its very painful to keep them green with the changes to UI.

I work on a product team where we had this exact same decision to make. A colleague of mine has summarized our final approach very well here. (Disclaimer: Please ignore the tool specific details there.)

Blodgett answered 27/2, 2011 at 13:48 Comment(0)
S
2

I've done Acceptance TDD with a UI. We would assert the common header and footer were used via xpath'ing for the appropriate ids. We also used xpath to assert the data appeared in the correct tag relative to whatever ids we used for basic layout and structure. We would also assert the output page is valid html 4.01 strict.

Schick answered 11/1, 2011 at 15:7 Comment(0)
H
1

Programmatic UI tests is a salvation when you want to be confident your UI does what expected. The UI tests are closer to BDD (behavior driven development) than to TDD. But the terminology is cloudy and however you call 'em they are useful! I have got rather good experience using cucumber. I use it to test flex applications but it's mostly used to test web apps. Click the link! The site have got really nice examples of methodology.

Hernandez answered 11/1, 2011 at 14:12 Comment(6)
programatic UI tests are a must, but I argue that they shouldn't be written before the code.Milagro
It depends. I prefer to write 'em before the code. This lets me progress in small understandable steps, stay focused and have something working all the time. The behavior descriptions make the communication with business owner, client, manager easier. Another use case which is less of BDD but more of functional testing is programming of visual components.Hernandez
The link is dead. It seems to be good if I need some japanese doctor .Toledo
The link is fine. Check it please :)Hernandez
As always, it depends. If the TDD UI tests become too brittle, and they tend to quickly. The team faces pain/dilemma when they need to make changes. This is true of all TDD, of course, it just tends to hurt more with a UI that's still in flux. For many applications, time in flux while progressing from MVP to "final/gold" can be in the order of months, or years.Nichols
In the 10 years after posting the answer, my opinions changed several times until they converged into "it depends on the context."Hernandez
N
0

Advanced UI is not compatible with straight TDD.

Use XP practices to guide you.

It is unwise to slavishly follow TDD (or even BDD) for UI, it's worth thinking about the root goals of both practices. In particular, we want to avoid using general purpose TDD tools for UI. (If) They're simply not designed for that use case, you end up binding yourself to tests that you have to iterate on as rapidly as the UI code (I call this, UI test-lock).

That's not the purpose of TDD. We don't do it to go slower, we use to go fast (and NOT break things!).

We want to achieve a few things with TDD:

  • Code / Algorithmic Correctness
  • Minimal code to complete a test
  • Fast feedback
  • Architectural separation of concerns to the units (modular parts under test, no more, no less)
  • Formalize specification in a useful/usable way
  • To some extent, document what the module/system does (e.g. provide an example of use.)

We can achieve these goals in UI development, but how and where we apply practices is important to avoid test-lock.

Applying these principles to UI

What benefits can we get from testing UI? How can we avoid test-lock.

With usual TDD one major benefit is that we can think about the abstractions, and generally design the shape of a test subject, as we think of names, relationships etc.

For UI, much of what we want to do, especially if it's non-trivial, is only reasoned about effectively when we can see it manifested on screen.

Writing tests that describe what we expect to see in a UI, specifically what data, can be TDD tested. For example, if we have a view that should show an account balance, a test that expects it to appear in an on-screen element that can be addressed by some form of ID, is good. We will know our app/view is displaying content that's expected, in standard UI library elements.

If, on the other hand, we want to create a more complex UI and wish to apply TDD to it, we will have some problems. Chances are, we won't even know how to code it, if it's novel enough.

For this we would prototype and iterate with tools which give us fast feedback, but don't saddle us with any need to write tests first. When we reach a point where implementation becomes clear, we can then switch up to test first for the production code.

This keeps us fast. Designers and Product owners in the team can also see results, provide inputs, and tweaks.

The code should be well-formed. As soon as we identify small potential pieces which will make it to production code, we can migrate them to be under test. Using TCR or a similar method to add tests, or simply use the TDD method to re-write. Remember once it works, you can apply snapshot/record-playback testing, to keep regression errors at bay.

Do what works, keep it simple.


Addendum

Notes about the scope of UI discussed above

There will be a need to figure out how to make things work as expected in any advanced UI. There is also a likelihood that specs change and tweak in extremely arbitrary / capricious ways. We need to ensure any tests we apply to these test subjects are flexible, or extremely easy to re-generate based on a working model. (replay tests are gold for this.)

Good development practices, in a nutshell, code separation. Is going to help you ensure the code is at least testable, and simpler to debug, maintain and reason about.

Don't be a slave to ritual.

It doesn't make you correct. Just slower.

Nichols answered 17/9, 2022 at 6:35 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.