What is the difference between integration and unit tests?
Asked Answered
G

21

354

I know the so-called textbook definition of unit tests and integration tests. What I am curious about is when it is time to write unit tests... I will write them to cover as many sets of classes as possible.

For example, if I have a Word class, I will write some unit tests for the Word class. Then, I begin writing my Sentence class, and when it needs to interact with the Word class, I will often write my unit tests such that they test both Sentence and Word... at least in the places where they interact.

Have these tests essentially become integration tests because they now test the integration of these 2 classes, or is it just a unit test that spans 2 classes?

In general, because of this uncertain line, I will rarely actually write integration tests... or is my using the finished product to see if all the pieces work properly the actual integration tests, even though they are manual and rarely repeated beyond the scope of each individual feature?

Am I misunderstanding integration tests, or is there really just very little difference between integration and unit tests?

Groundwork answered 14/8, 2008 at 6:27 Comment(0)
O
344

The key difference, to me, is that integration tests reveal if a feature is working or is broken, since they stress the code in a scenario close to reality. They invoke one or more software methods or features and test if they act as expected.

On the opposite, a Unit test testing a single method relies on the (often wrong) assumption that the rest of the software is correctly working, because it explicitly mocks every dependency.

Hence, when a unit test for a method implementing some feature is green, it does not mean the feature is working.

Say you have a method somewhere like this:

public SomeResults DoSomething(someInput) {
  var someResult = [Do your job with someInput];
  Log.TrackTheFactYouDidYourJob();
  return someResults;
}

DoSomething is very important to your customer: it's a feature, the only thing that matters. That's why you usually write a Cucumber specification asserting it: you wish to verify and communicate the feature is working or not.

Feature: To be able to do something
  In order to do something
  As someone
  I want the system to do this thing

Scenario: A sample one
  Given this situation
  When I do something
  Then what I get is what I was expecting for

No doubt: if the test passes, you can assert you are delivering a working feature. This is what you can call Business Value.

If you want to write a unit test for DoSomething you should pretend (using some mocks) that the rest of the classes and methods are working (that is: that, all dependencies the method is using are correctly working) and assert your method is working.

In practice, you do something like:

public SomeResults DoSomething(someInput) {
  var someResult = [Do your job with someInput];
  FakeAlwaysWorkingLog.TrackTheFactYouDidYourJob(); // Using a mock Log
  return someResults;
}

You can do this with Dependency Injection, or some Factory Method or any Mock Framework or just extending the class under test.

Suppose there's a bug in Log.DoSomething(). Fortunately, the Gherkin spec will find it and your end-to-end tests will fail.

The feature won't work, because Log is broken, not because [Do your job with someInput] is not doing its job. And, by the way, [Do your job with someInput] is the sole responsibility for that method.

Also, suppose Log is used in 100 other features, in 100 other methods of 100 other classes.

Yep, 100 features will fail. But, fortunately, 100 end-to-end tests are failing as well and revealing the problem. And, yes: they are telling the truth.

It's very useful information: I know I have a broken product. It's also very confusing information: it tells me nothing about where the problem is. It communicates me the symptom, not the root cause.

Yet, DoSomething's unit test is green, because it's using a fake Log, built to never break. And, yes: it's clearly lying. It's communicating a broken feature is working. How can it be useful?

(If DoSomething()'s unit test fails, be sure: [Do your job with someInput] has some bugs.)

Suppose this is a system with a broken class: A system with a broken class

A single bug will break several features, and several integration tests will fail.

A single bug will break several features, and several integration tests will fail

On the other hand, the same bug will break just one unit test.

The same bug will break just one unit test

Now, compare the two scenarios.

The same bug will break just one unit test.

  • All your features using the broken Log are red
  • All your unit tests are green, only the unit test for Log is red

Actually, unit tests for all modules using a broken feature are green because, by using mocks, they removed dependencies. In other words, they run in an ideal, completely fictional world. And this is the only way to isolate bugs and seek them. Unit testing means mocking. If you aren't mocking, you aren't unit testing.

The difference

Integration tests tell what's not working. But they are of no use in guessing where the problem could be.

Unit tests are the sole tests that tell you where exactly the bug is. To draw this information, they must run the method in a mocked environment, where all other dependencies are supposed to correctly work.

That's why I think that your sentence "Or is it just a unit test that spans 2 classes" is somehow displaced. A unit test should never span 2 classes.

This reply is basically a summary of what I wrote here: Unit tests lie, that's why I love them.

Oceanid answered 14/8, 2008 at 6:28 Comment(7)
A really good answer! However I just want to add that mocking is not for unit test only. It could also be very useful in a lot of integration test cases.Fe
Great answer! I just don't quite agree with two points: 1) that integration tests are "of no use in guessing where the problem could be"; and 2) that "a unit test should never span 2 classes". I created lots of integration tests, and when they break it's usually not hard to pinpoint the source of the problem, provided you get a full stack trace, or a single failed assertion (in which case it can be harder to find the source, but not so much, as the integration test provides a contained context for debugging). (continues)Precedency
Unit tests can exercise multiple classes, provided they are not public classes which should have have their own separate unit tests. One case is when a public tested class uses other non-public helper classes which only exist to support the public class; in this case, the "unit" comprises two or more classes. Another case is that most classes use third-party classes (the String/string class, collection classes, etc.) which don't make sense to be mocked or isolated from; we simply regard them as stable and reliable dependencies which are outside the testing scope.Precedency
With integration tests it's a bit harder to find the root problem, but you could still debug it and find the root problem. Assuming unit tests don't fail often, then maybe seldom it'll take a bit more time to fix bugs if you have only integration tests, but then you get the added value of testing components integration also and you save you the time of writing the unit tests. I think this claim (which came from my boss) is wrong, but I'm not sure how can I convince him, any ideas?Lubricator
By the reasoning in this answer, one could argue that it might be more time effective to skip writing unit tests and spend the time saved by locating the source of failing integration tests when they fail.Latakia
Arialdo Martini, not sure if you'll receive notification about that comment since it's now community wiki post, but could you please provide working link on your Unit tests lie, that's why I love them article. It looks like provided link is broken, but I would like to read it. (Damn you, StackOverflow, when you will finally implement private messaging?)Briquet
user74754, right. Maybe the answer should also mention that unit testing, when applied with TDD, is also a design technique, as it enables emergent design. In some cases, when a unit test has provided its value, as a design tool, it could be even deleted. Of course, in general, spotting an issue with a failing unit test is way faster than debugging the code.Oceanid
F
65

When I write unit tests I limit the scope of the code being tested to the class I am currently writing by mocking dependencies. If I am writing a Sentence class, and Sentence has a dependency on Word, I will use a mock Word. By mocking Word I can focus only on its interface and test the various behaviors of my Sentence class as it interacts with Word's interface. This way I am only testing the behavior and implementation of Sentence and not at the same time testing the implementation of Word.

Once I've written the unit tests to ensure Sentence behaves correctly when it interacts with Word based on Word's interface, then I write the integration test to make sure that my assumptions about the interactions were correct. For this I supply the actual objects and write a test that exercises a feature that will end up using both Sentence and Word.

Fahlband answered 1/9, 2008 at 19:33 Comment(0)
C
53

In unit test you test every part isolated: enter image description here

in integration test you test many modules of your system:

enter image description here

and this what happens when you only use unit tests (generally both windows are working, unfortunately not together):

enter image description here

Sources: source1 source2

Calamite answered 31/7, 2019 at 5:28 Comment(4)
You have three images but only two sources.Savoyard
@Savoyard take a look into first source - two pictures are from thereCalamite
Love this answer 👏Mauer
picture with the whole car hitting the wall seems to me more like system testingAlienor
O
47

My 10 bits :D

I was always told that Unit Tests is the testing of an individual component - which should be exercised to its fullest. Now, this tends to have many levels, since most components are made of smaller parts. For me, a unit is a functional part of the system. So it has to provide something of value (i.e. not a method for string parsing, but a HtmlSanitizer perhaps).

Integration Tests is the next step up, its taking one or more components and making sure they work together as they should.. You are then above the intricacies of worry about how the components work individually, but when you enter html into your HtmlEditControl , it somehow magically knows wether its valid or not..

Its a real movable line though.. I'd rather focus more on getting the damn code to work full stop ^_^

Ouphe answered 14/8, 2008 at 8:59 Comment(0)
H
23

Unit tests use mocks

The thing you're talking about are integration tests that actually test the whole integration of your system. But when you do unit testing you should actually test each unit separately. Everything else should be mocked. So in your case of Sentence class, if it uses Word class, then your Word class should be mocked. This way, you'll only test your Sentence class functionality.

Hypognathous answered 3/8, 2009 at 17:7 Comment(3)
I know this is an old post but I just came across it. What if you had a third class called Font which the Sentence class interacts with and you want to test the functionality between Word and Sentence class, then you had to mock the Font class but this will not make it a unit test. So what I'm saying is that using mocks doesn't necessarily makes it a unit test, mocks can also be used in integration tests.Fe
Of course mocks can be used in integration tests, but order for a unit test to actually be as such everything external to the unit should be simulated. If integration tests use mocks, then they're likely partial integration tests (if such term exists). THere are of course partial integration tests which are top-down or bottom-up ones. Latter usually don't require mocks while former do.Hypognathous
Not all unit tests require mocking. In fact, it is generally good to design your system such that code can be tested without mocks.Pym
U
18

I think when you start thinking about integration tests, you are speaking more of a cross between physical layers rather than logical layers.

For example, if your tests concern itself with generating content, it's a unit test: if your test concerns itself with just writing to disk, it's still a unit test, but once you test for both I/O AND the content of the file, then you have yourself an integration test. When you test the output of a function within a service, it's a unit-test, but once you make a service call and see if the function result is the same, then that's an integration test.

Technically you cannot unit test just-one-class anyway. What if your class is composed with several other classes? Does that automatically make it an integration test? I don't think so.

Unmoving answered 14/8, 2008 at 6:34 Comment(3)
"Technically you cannot unit test just-one-class anyway. What if your class is composed with several other classes?" Well, a "strict" unit test would just mock/stub all the dependencies. However, it is debatable whether this is always practical...Woollyheaded
That's true sieske -- the important bit is to be able to keep dependencies to an absolute minimum.Unmoving
-1, a unit test doesn't test a single feature, but a single software function or class, i.e. it tests a logical unit of software.Fallon
M
15

using Single responsibility design, its black and white. More than 1 responsibility, its an integration test.

By the duck test (looks, quacks, waddles, its a duck), its just a unit test with more than 1 newed object in it.

When you get into mvc and testing it, controller tests are always integration, because the controller contains both a model unit and a view unit. Testing logic in that model, I would call a unit test.

Mycetozoan answered 14/8, 2008 at 6:32 Comment(0)
U
12

In my opinion the answer is "Why does it matter?"

Is it because unit tests are something you do and integration tests are something you don't? Or vice versa? Of course not, you should try to do both.

Is it because unit tests need to be Fast, Isolated, Repeatable, Self-Validating and Timely and integration tests should not? Of course not, all tests should be these.

It is because you use mocks in unit tests but you don't use them in integration tests? Of course not. This would imply that if I have a useful integration test I am not allowed to add a mock for some part, fear I would have to rename my test to "unit test" or hand it over to another programmer to work on.

Is it because unit tests test one unit and integration tests test a number of units? Of course not. Of what practical importance is that? The theoretical discussion on the scope of tests breaks down in practice anyway because the term "unit" is entirely context dependent. At the class level, a unit might be a method. At an assembly level, a unit might be a class, and at the service level, a unit might be a component. And even classes use other classes, so which is the unit?

It is of no importance.

Testing is important, F.I.R.S.T is important, splitting hairs about definitions is a waste of time which only confuses newcomers to testing.

Udale answered 21/7, 2013 at 0:0 Comment(4)
-1 Definition is what makes people able to use the same terms without always explaining what they mean, and is essential to collaboration. As such it is essential to understand the difference between both notions.Fallon
As @Fallon mentioned it is important so no need to explain every time or find everyone has a different definition causing confusion. Tests will be written differently and ran differently, but this does not suggest both should not be done by wanting to define the differences.Cher
The answer's conclusion may be extreme, but most of its points are quite valid: Unit tests and integration tests are mostly the same thing except for their granularity -- and it is not obvious where a line should be drawn between them.Ensile
This doesn't help when creating a common language in a professional environment. While mostly, you're right, it doesn't matter that much, not having a common language will create misunderstandings and confusion amongst the team. I think the best option is to have the team agree on on their terms and definitions.Undercast
E
11

The nature of your tests

A unit test of module X is a test that expects (and checks for) problems only in module X.

An integration test of many modules is a test that expects problems that arise from the cooperation between the modules so that these problems would be difficult to find using unit tests alone.

Think of the nature of your tests in the following terms:

  • Risk reduction: That's what tests are for. Only a combination of unit tests and integration tests can give you full risk reduction, because on the one hand unit tests can inherently not test the proper interaction between modules and on the other hand integration tests can exercise the functionality of a non-trivial module only to a small degree.
  • Test writing effort: Integration tests can save effort because you may then not need to write stubs/fakes/mocks. But unit tests can save effort, too, when implementing (and maintaining!) those stubs/fakes/mocks happens to be easier than configuring the test setup without them.
  • Test execution delay: Integration tests involving heavyweight operations (such as access to external systems like DBs or remote servers) tend to be slow(er). This means unit tests can be executed far more frequently, which reduces debugging effort if anything fails, because you have a better idea what you have changed in the meantime. This becomes particularly important if you use test-driven development (TDD).
  • Debugging effort: If an integration test fails, but none of the unit tests does, this can be very inconvenient, because there is so much code involved that may contain the problem. This is not a big problem if you have previously changed only a few lines -- but as integration tests run slowly, you perhaps did not run them in such short intervals...

Remember that an integration test may still stub/fake/mock away some of its dependencies. This provides plenty of middle ground between unit tests and system tests (the most comprehensive integration tests, testing all of the system).

Pragmatic approach to using both

So a pragmatic approach would be: Flexibly rely on integration tests as much as you sensibly can and use unit tests where this would be too risky or inconvenient. This manner of thinking may be more useful than some dogmatic discrimination of unit tests and integration tests.

Ensile answered 22/9, 2014 at 14:49 Comment(0)
E
5

Unit Testing is a method of testing that verifies the individual units of source code are working properly.

Integration Testing is the phase of software testing in which individual software modules are combined and tested as a group.

Wikipedia defines a unit as the smallest testable part of an application, which in Java/C# is a method. But in your example of Word and Sentence class I would probably just write the tests for sentence since I would likely find it overkill to use a mock word class in order to test the sentence class. So sentence would be my unit and word is an implementation detail of that unit.

Ejective answered 14/8, 2008 at 10:26 Comment(0)
M
4

I think I would still call a couple of interacting classes a unit test provided that the unit tests for class1 are testing class1's features, and the unit tests for class2 are testing its features, and also that they are not hitting the database.

I call a test an integration test when it runs through most of my stack and even hits the database.

I really like this question, because TDD discussion sometimes feels a bit too purist to me, and it's good for me to see some concrete examples.

Marcellamarcelle answered 14/8, 2008 at 6:37 Comment(0)
P
4

I do the same - I call them all unit tests, but at some point I have a "unit test" that covers so much I often rename it to "..IntegrationTest" - just a name change only, nothing else changes.

I think there is a continuation from "atomic tests" (testing one tiny class, or a method) to unit tests (class level) and integration tests - and then functional test (which are normally covering a lot more stuff from the top down) - there doesn't seem to be a clean cut off.

If your test sets up data, and perhaps loads a database/file etc, then perhaps its more of an integration test (integration tests I find use less mocks and more real classes, but that doesn't mean you can't mock out some of the system).

Proprietary answered 14/8, 2008 at 6:47 Comment(0)
H
4

Integration tests: Database persistence is tested.
Unit tests: Database access is mocked. Code methods are tested.

Hemelytron answered 13/7, 2012 at 18:56 Comment(0)
G
3

Unit testing is testing against a unit of work or a block of code if you like. Usually performed by a single developer.

Integration testing refers to the test that is performed, preferably on an integration server, when a developer commits their code to a source control repository. Integration testing might be performed by utilities such as Cruise Control.

So you do your unit testing to validate that the unit of work you have built is working and then the integration test validates that whatever you have added to the repository didn't break something else.

Geraldina answered 14/8, 2008 at 6:36 Comment(0)
B
3

Simple Explanation with Analogies

This answer will focus purely on examples.

Integration Tests

Integration tests check if everything is working together.

Unit Tests

They tell you whether one specific thing is working.

Examples

Consider a car:

  • Integration test for a car: e.g. does the car drive to Pondicherry and back? If so, the car as whole is working. If it fails, you won't really know where. Was it the radiator, transmission, engine, or carburetor?

  • Unit test for a car: Is the engine working? This tests just the engine; nothing else. If this test fails, then you can be confident that there is a bug in the engine....This ties in closely with the concept of "fakes". You might need some keys in order to start the engine - except, you don't want to go to the hassle of an actual ignition (with a lock)...instead, you would hotwire the car to start it....in other words you would use a "fake" key.

Similarly, in unit testing, you would use "fakes" in order to make the engine work a particular way. And then you could simply test: "is it running".

Biography answered 6/12, 2016 at 0:58 Comment(0)
A
2

I call unit tests those tests that white box test a class. Any dependencies that class requires is replaced with fake ones (mocks).

Integration tests are those tests where multiple classes and their interactions are tested at the same time. Only some dependencies in these cases are faked/mocked.

I wouldn't call Controller's integration tests unless one of their dependencies is a real one (i.e. not faked) (e.g. IFormsAuthentication).

Separating the two types of tests is useful for testing the system at different levels. Also, integration tests tend to be long lived, and unit tests are supposed to be quick. The execution speed distinction means they're executed differently. In our dev processes, unit tests are run at check-in (which is fine cos they're super quick), and integration tests are run once/twice per day. I try and run integration tests as often as possible, but usually hitting the database/writing to files/making rpc's/etc slows.

That raises another important point, unit tests should avoid hitting IO (e.g. disk, network, db). Otherwise they slow down alot. It takes a bit of effort to design these IO dependencies out - i can't admit I've been faithful to the "unit tests must be fast" rule, but if you are, the benefits on a much larger system become apparent very quickly.

Armond answered 14/8, 2008 at 12:19 Comment(0)
T
1

A little bit academic this question, isn't it? ;-) My point of view: For me an integration test is the test of the whole part, not if two parts out of ten are going together. Our integration test shows, if the master build (containing 40 projects) will succeed. For the projects we have tons of unit tests. The most important thing concerning unit tests for me is, that one unit test must not be dependent on another unit test. So for me both test you describe above are unit tests, if they are independent. For integration tests this need not to be important.

Temper answered 14/8, 2008 at 6:37 Comment(0)
I
1

Have these tests essentially become integration tests because they now test the integration of these 2 classes? Or is it just a unit test that spans 2 classes?

I think Yes and Yes. Your unit test that spans 2 classes became an integration test.

You could avoid it by testing Sentence class with mock implementation - MockWord class, which is important when those parts of system are large enough to be implemented by different developers. In that case Word is unit tested alone, Sentence is unit tested with help of MockWord, and then Sentence is integration-tested with Word.

Exaple of real difference can be following 1) Array of 1,000,000 elements is easily unit tested and works fine. 2) BubbleSort is easily unit tested on mock array of 10 elements and also works fine 3) Integration testing shows that something is not so fine.

If these parts are developed by single person, most likely problem will be found while unit testing BubbleSoft just because developer already has real array and he does not need mock implementation.

Ironstone answered 25/9, 2008 at 9:37 Comment(0)
F
1

In addition, it's important to remember that both unit tests and integration tests can be automated and written using, for example, JUnit. In JUnit integration tests, one can use the org.junit.Assume class to test the availability of environment elements (e.g., database connection) or other conditions.

Forewarn answered 10/9, 2013 at 17:12 Comment(0)
S
1

I get asked this a lot in interviews. Until now I'd ramble on pretentiously about my expertise and pontificate about component and acceptance testing.

For years I'd understood only integration and unit tests. I could, but didn't always bother to, write unit tests as a solo developer honing my skills.

Unit tests

That is a crucial difference. Unit tests are easy to implement and execute, requiring, ideally, no dependencies. That is what mocks are for. It is often easier to not mock everything, particularly where you gain coverage of other functions you wrote. Easier, maybe, but that isn't the idea of unit testing.

I'll reiterate, unit tests are meant to be easy to run and small. Their failure provides immediate insight into where a bug has been introduced.

Here is the hierarchy of tests, from cheap and plentiful at the bottom to slow, expensive, and few, at the top:

hierarchy of test expense and breadth.

Several more layers can be conceptualised, but were omitted for clarity.

Integration tests

With integration tests you would consider bringing in serious external dependencies, such as VMs, virtual networks and appliances. Possibly you could use actual modems, routers, and firewalls where the expense was justified.

These wouldn't be run locally but on a build server. A mixture of local Jenkins and cloud based CI providers fulfil this need.

Other test terminology

That is my understanding that has served me for several years in industry. We could talk about component tests, and get a definition, but if the definition isn't in common circulation then it loses value.

Acceptance tests were what we would call business unit or customer requirements. These would lead the direction of everything and sit at the top of the pyramid (picture a dollar sign).

E2E, or end to end testing was used synonymously with integration tests, but I noticed online it is placed above. I guess it could have more relevance to acceptance tests the integration tests, which would tend to be more detailed with less interest from stakeholders (though immense interest internally in the department).

Synthesize answered 24/11, 2022 at 11:13 Comment(0)
A
0

If you're a TDD purist, you write the tests before you write production code. Of course, the tests won't compile, so you first make the tests compile, then make the tests pass.

You can do this with unit tests, but you can't with integration or acceptance tests. If you tried with an integration test, nothing would ever compile until you've finished!

Angulate answered 2/11, 2018 at 17:8 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.