Unit Testing or Functional Testing? [closed]

Question

I have recently heard of Functional Testing over Unit Testing.

I understand that Unit Testing tests each of the possibilities of a given piece of code from its most atomic form. But what about Functional Testing?

This sounds to me like only testing if the code works, but is it as reliable as Unit Testing?

I've been told there was two school of thoughts for the matter. Certains would prefer Unit Testing, others Functional Testing.

Is there any good resources, links, books, any references or one of you all who can explain and elighten my path on the subject?

Thanks!

Retroact

Does functional testing == integration testing? – Wojcik

Everyone has a very good answer to my question and brings proper clarity to the explanations brought. Thanks to you all! I chose @TrueWill's answer as he provides references, in addition to what all of you said. Please know that I all upvoted your answers and reread and reread your answers as per care to fully understand your points. Thanks! – Retroact

@JoshKodroff, not quite (though you'll find differing opinions and definitions). Functional testing, as I understand it, tests the expected behavior for a use case, without much regard for what's happening behind the scenes. It tests units of code working together that are already unit tested. Integration tests, on the other hand, test that everything works properly from end to end with realistic inputs and outputs. I believe that integration testing will do a lot less mocking of external dependencies and data sources and will actually do the work that would occur in the real application. – Holystone

Answer 1

Jason's answer is correct. Different types of tests have different purposes, and can be layered for best results (good design, meeting specifications, reduced defects).

• Unit testing = drives design (with Test-Driven Development, or TDD)
• Integration testing = do all the pieces work together
• Customer acceptance testing = does it meet the customer's requirements
• Manual testing = often covers the UI; dedicated testers can find what automation misses
• Load testing = how well does the system perform with realistic amounts of data

There is some overlap between these categories; unit tests can specify behavior, for instance.

And there are others; for more than most people care to know, see Software Testing.

One point people missed is that unit testing is testing pieces of code in isolation. Good unit tests don't hit the database, for instance. This has two advantages: it makes the tests run fast so you'll run them more often, and it forces you to write loosely coupled classes (better design).

You asked for resources; I recommend Roy Osherove's book The Art of Unit Testing with Examples in .NET. While no book is perfect, this one gives many excellent pointers on writing good tests.

EDIT: And for writing tests against existing software, nothing beats Michael Feathers' book Working Effectively with Legacy Code.

Arel

Unit testing does not necessarily drive design. In TDD it does, in other paradigms its there to make sure your code is behaving as you expect, and that changes to the internals of the class don't break the individual pieces of functionality – Knowall

@Knowall - agreed, there are other perspectives, but this is mine. :) Here's another worth investigating: en.wikipedia.org/wiki/Behavior_Driven_Development – Arel

A few other types of testing (just for the sake of discussion): Security testing = is the attack surface of your code appropriate. Are all attack vectors handled well? Performance / Concurrency testing = how well does your application handle several (thousand) users at once? – Priestess

After a few years of unit testing, integration testing, functional testing, etc. I do agree with @Arel about Behaviour-Driven Developement (BDD). I have done a self-teaching project using BDD, and it proved to be great to work with. Now that I better understand the different tests, I also say each type of test has its own purpose. None is best than the other as it serves a different purpose. Aside, when the behaviour of a feature is described, it is easier to focus on what should the code, and state the acceptance criterion for the unit tests. – Retroact

Answer 2

Unit testing versus functional testing is not an xor, but rather an and. Unit testing is about testing units in isolation while functional testing is about testing the whole in integration (do all the units works together properly?).

Both are necessary components of good software engineering practices.

Equalizer

@Jason: Thanks for the swiftness of your answer! This is lean and swift as we like them! It is a good point you have there insisting on the Software Engineering Practices. – Retroact

Answer 3

Unit testing tests your code units (methods, etc) to make sure they do what you expect them to.

Functional testing tests your system design to make sure the pieces interact correctly. If you write a command that takes and int and returns a string and test it fully, you can be sure it works. But if you don't have system tests, you may never notice that the rest of the code thinks it can accept a null but it can't.

Both types of testing are important.

edit: To add a slightly different view to what gbjbaanb said:

• Unit test = my code works
• Functional test = my design works
• Integration test = my code is using your 3rd party stuff correctly (databases, etc)
• Factory Acceptance Test = my system works
• Site Acceptance Test = your code sucks, this totally isn't what I asked for!?!

Swihart

I like your Site Acceptance Test remark! =P Thanks for these precisions. It already redirects my mind and thoughts about the differences and specialities of each of them. As a matter of fact, I had never noticed there was other tests than Unit and Functional Testings. When I come to think of it, it makes much sense! – Retroact

Answer 4

• Unit test = lowest, granular level.
• Functional test = middling, modular level.
• Integration test = higher application level.
• Factory Acceptance Test = see it all work
• Site Acceptance Test = see it all fail :)

All the above are useful but they're not mutually exclusive. You should be doing most of them but the amount of time you spend on each part depends on the results you get from them, that's all. If your code is too modular to be easily unit tested, then spend your efforts on the functional tests. If you're writing a library of small components, spend your time on unit testing them, and if you're writing control systems for military missiles you should definitely be site acceptance testing them (as explosions even when it fails is fun :) )

Melpomene

I've generally seen the term Customer Acceptance Test, where the customer/users/business analyst specifies the criteria. There's also Load Testing. All the layers have value. – Arel

+1 for good answer, -1 for "necessary" – Nasalize

I guess he didn't mean necessary as you are obliged to perform all of these tests within each and every single project, but necessary as they are useful when required to use them depending on a specific situation. Notice he says "you should be doing most of them", if I am not mistaken. – Retroact

@TrueWill: I think there are tools for Load Testing, and these tests are mostly to be performed by the insfracstructure personnel, am I am right? – Retroact

@gbjbaanb: Thanks for your answer! You show more precisely the role of each of them and really help for a good vision of them. – Retroact

@Will - Load (or Performance) testing is useful too - see LoadRunner, or WAST for two examples. Also there's Usability testing (plonk a user in front of it and watch them scratch their heads). – Melpomene

@Will - yes, there are tools as gbjbaanb just mentioned. Who is responsible for a testing layer depends on the organization. Having developers doing testing and having dedicated testers are both beneficial, and are not mutually exclusive. :) – Arel

Answer 5

Functional testing, also called System testing, aims at testing the complete system, and verifying the functional requirements are satisfied.

Unit testing aims at testing the "units", i.e. the functions or methods the system is build from in isolation. It's sometimes called Developer testing. Unit testing can be hard after the fact, that's why TDD writes the test before the code.

Those are complementary as the units can work independently and not when integrated all together, or they can pass the unit tests, and not fulfill all the product requirements.

Yttrium

+1 for mentioning that functional and system tests are usually the same thing. – Holystone

Answer 6

Unit Testing and Functional Testing have two different results.

Unit Testing verifies that a small piece of code works as expected. It is usually done by the developer to ensure that the code works correctly. They are usually automated by a testing-framework as well.

Functional Testing verifies that a feature works as expected by going through a certain pathway through the program. They are usually executed by a person on the software ensuring that the program will work they way it is supposed to for the user. It, as such, is higher level, and thus tests several units at once.

I think both are important. If you have limited resources, though, and have to pick/choose techniques, and I think it depends on the products you create, but for what I do (automotive control products used by humans through some buttons) functional tests are most important. It checks, and ensures, that when the user gets the product, it does what it is supposed to do. This doesn't mean we should opt out of unit testing, but if push-comes-to-shove, functional is the most important to ensure great user experience and getting the product out the door.

If you produce, say, a database engine (or some other product that isn't necessarily user-facing), unit testing may be what you really ought to do.

Friesian

What do you mean by testing-framework? Are there any example or documentation about the topic? – Retroact

@sheepsimulator: Every product is user-facing, otherwise there would be no point in developing the product. In other words, if there's no one to use the product, why develop it? – Equalizer

@Jason: I guess that what he meant is related to his sector of work. sheepsimulator seems to work mainly with programmable automates (PLCs). So, the system is mostly self-sufficient, if I may say so, or if your prefer, perhaps simply requires less attention from human beings (users). – Retroact

@Will A testing framework is a peice of software, generally in the form of a library and a test runner to assist the developer in writing and executing automated tests, usually these frameworks are used to automate unit tests. There are many testing frameworks out there and which one you use depends on your target language/environment and your preference. Check out NUnit or XUnit.NET for the .net environment, if you like java there is JUnit, most languages have at least one. – Mosera

@Crippledsmurf: So when using testing from within the Visual Studio Team System environment, there is a Testing-Framework associated with it, necessarily... I didn't think of NUnit as a testing-framework before. Introduced the way you do, it makes perfect sense. I didn't know about XUnit.NET and JUnit, though. Thanks! – Retroact

@Will - there are a LOT of frameworks. See xprogramming.com/software.htm For .NET, several of these (including the excellent NUnit) can be integrated with Visual Studio using ReSharper, TestDriven.NET, etc. – Arel

@Will - We use CppUnit with our codebase. It works pretty well. The big thing about writing unit tests, I've found, is you have to write your classes/software with that in mind. It's a change in design philosophy, compared to what I'm used to. It's a learning process. – Friesian

@Jason - I was trying to make a distinction between testing by human beings / testers and machines. Some tests really need to be run by human beings. Others could perhaps be run by machines. – Friesian

@Will To be clear, yes the testing bits that come with VS Team System is a test framework, it's called MSTest. In my prior comment I mentioned the concept of a test runner, a test runner is the component that executtes and reports the results of tests. Most frameworks provid their own external test runners that aren't intergrated with the IDE but many also intergrate with VS either through a plug in such as TestDriven.net or by writing a provider to intergrate with the Visual Studio test UI as MSTest does. – Mosera

@Crippledsmurf: Thanks for this information. It is gratefully appreciated. I shall remind it along with the other information I have learned from testing with others' responses. – Retroact

Answer 7

A Unit Test tests a piece of Code and confirms for a programmer that another piece of code is doing what it is supposed to. In Test Driven Development, the unit test is written first and observed to fail, before the code is written causing the test to pass. Programmers are interested in Unit Tests. Unit Test are quick to execute.

A Function Test tests your black box requirement and demonstrates that a piece of user functionality is in place. For example, if I press the big red button, the bell begins to ring. The functional test mightn't even be testing code. Perhaps there is a mechanical process that cause the bell to ring having pressed the button. Customers are interested in Functional Tests as they confirm that a high level process if working in a manner that they understand. They are often slow to execute.

Capone

Business partners or product owners may be interested in functional tests...but isn't functional testing more granular than what a customer or end user wants to see? Wouldn't that be Acceptance or Usability testing? – Holystone

Answer 8

There is a place for both in most development work.

Unit testing is there to test small units of code, to see that they work as expected.

Functional testing is there to test that the overall functionality of the system is as expected.

They are at different levels and both should be used.

Prophecy