How are tests handled by Scrum using TFS 2012?

Question

We are, for the first time, trying to implement scrum on our company using TFS 2012. So far the process is not doing very well since we have questions that no one could find the answers so far.

Our main concern is how to handle the tests phase. Here are or scenario (in terms of people/jobs):

• We have 6 programmers
• We have a scrum master
• We have 2 testers (that are not programmers)

That is what we have until now:

• All desires go to the board
• We have the sprint meeting where we add tasks to those desires
• We prepare the sprint
• People start to do their jobs

Our definition of Done clarifies that a story can only be considered done when the story goes to the test guys and one of them (in that case, me) says that the story is done. So far so good.

We have a Test Server where all tests are executed and that server is similar to the production server (web app).

As I said, out main concern is how to handle tests:

• Since all developers can commit their code (using SVN), when should they commit? When a task is done or when a backlog item is done?
• When a test release should be released?
• When should the tests begin? Should we start testing after a task is done or after a backlog item is done? How can we get notified when we should begin testing?
• Should we create a Deployment task and a Test Task on every backlog item?

Any help would be nice.

Greenwell

Answer 1

- Since all developers can commit their code (using SVN), when should they commit? When a task is done or when a backlog item is done?

Ans: I think you should commit as soon as code is ready. If you have tasks created under a user story and a task covers some small development you can submit and close the task. So like wise you have smaller tasks for development of the user story. Once all tasks are completed User story (backlog item) is completed.

To test the these commits, what you can do is to have a automation test suite which runs against the CI environment. So you can cover the smoke test and the regression testing. Which type to test suite to run can be determined based on time. For ex, you can run the regression test suite weekly and smoke test suite nightly.

- When a test release should be released? - When should the tests begin? Should we start testing after a task is done or after a backlog item is done? How can we get notified when we should begin testing?

Ans: There shouldn't be strict deadline such to release a test build. Testing can be started while a user story develops. How a tester can help is testers can have 'show me session' on the code that developer works with and give some feedback. and start feature testing once the user story is resolved (you can have this state in tfs). But make sure to complete testing before the sprint ends.

If the testing debt is high you can have a hardening sprint.

-Should we create a Deployment task and a Test Task on every backlog item?

Ans: Yes there should be two different tasks for each user story. one task covers the feature testing (user story testing). This is important to do to estimations and also to get the correct team velocity.

Rummage

Answer 2

There is nothing specific to TFS in the way tests are handled in Scrum. In fact, there's nothing particular to Scrum for handling tests! Your team has agreed that a story (or backlog item) is Done, only after it has been tested, and the only thing that Scrum "cares about" is that you meet your Definition of Done.

With regard to your specific questions:

• You should commit your changes early, and often. The more you commit, the easier it is for you to communicate changes to the rest of the team, and in case of a disaster (you screw up the code, or your hard drive fails) - the easier it is for you to recover. In my experience it is best to commit whenever your code is stable and safe enough to add to the repository. This should be done at least when you complete a task
• You should release early and often (see a recurring theme here?). This means that as soon as your product has a valuable increment, and it is stable and safe enough to expose to stakeholders outside your team. After your first release, you should be able to do so whenever you complete a backlog item - every Minimally Viable Product increment.
• You should test - you guessed - early and often. This means, as soon as the changes to your product are sufficient to test. For developer unit tests, this means pretty much every time you add a scenario (if you're doing test driven development). For the "QA" tests, it should be done as early as possible - whenever a task is completed, if you can manage it. In fact, you may wish - if necessary - to rethink how you break backlog items into tasks, in order to increase the tasks' testability. If, for example, you break the tasks by the component or code-layer, it is impossible to test the code before the entire story is code-complete. On the other hand, if you break the backlog items by use case or scenario, you will be able to test whenever you complete a task - and the developers will get their feedback faster - immediately after they complete the tasks!
• If you need to deploy every backlog item (and you do), the work involved should be accounted for. If every deployment is slightly different (one story might need to be published to the web, another might mean uploading an app to a mobile-app store), in a non-trivial and non-obvious way, then you should have a task for it, explaining how to deploy. If, however, every deployment task's title looks like "deploy" and the estimations are the same, then you should consider not wasting your time, and simply add a state to the backlog items' workflow (something like: To Do --> In Progress --> Testing --> Deploy --> Done)

Cuthbertson

Nice answer. Do you know how to add a new state for backlog items and tasks? – Greenwell

Answer 3

Overall, Scrum is not going to give you the process to follow. It only provides a framework within which you can experiment with your process empirically to find what works best for your team.

Since all developers can commit their code (using SVN), when should they commit? When a task is done or when a backlog item is done? Can you achieve an integrated, done increment without committing? If not, commit when it makes sense. Pick one and talk about the impact it had. What happens when you batch all of your changes until the backlog item is "done"? Can a backlog item be done without committing?

When a test release should be released? When the Product Owner feels they have enough value to ship or there is value in shipping. Your the increments you create in a sprint should be releasable.

Maybe you are asking can you ship mid sprint. Sure! Consider it another empirical experiment to be executed within the Scrum framework. You can try this expriment and inspect, empirically, on the results.

When should the tests begin? Should we start testing after a task is done or after a backlog item is done? Again, pick one and use Scrum to inspect on the impact. As a recommendation, don't wait too long. Try to get yourself into a position of building tests/test cases in parallel. Collaborate between those skills.

How can we get notified when we should begin testing? You have 6 Development Team members. Ask the other person. Email, Skype, turn your head, stand up and walk over. Maybe you can use the Daily Scrum as a place to plan the day

Should we create a Deployment task and a Test Task on every backlog item? Again, some teams do. If this helps you a) understand the remaining work and b) execute your process then try it. Maybe "Deployable via xyz script" is a new Definition of Done item in the making.

Zinazinah