Difference between acceptance test and functional test?

Question

What is the real difference between acceptance tests and functional tests?

What are the highlights or aims of each? Everywhere I read they are ambiguously similar.

Absent

Answer 1

In my world, we use the terms as follows:

functional testing: This is a verification activity; did we build a correctly working product? Does the software meet the business requirements?

For this type of testing we have test cases that cover all the possible scenarios we can think of, even if that scenario is unlikely to exist "in the real world". When doing this type of testing, we aim for maximum code coverage. We use any test environment we can grab at the time, it doesn't have to be "production" caliber, so long as it's usable.

acceptance testing: This is a validation activity; did we build the right thing? Is this what the customer really needs?

This is usually done in cooperation with the customer, or by an internal customer proxy (product owner). For this type of testing we use test cases that cover the typical scenarios under which we expect the software to be used. This test must be conducted in a "production-like" environment, on hardware that is the same as, or close to, what a customer will use. This is when we test our "ilities":

• Reliability, Availability: Validated via a stress test.

• Scalability: Validated via a load test.

• Usability: Validated via an inspection and demonstration to the customer. Is the UI configured to their liking? Did we put the customer branding in all the right places? Do we have all the fields/screens they asked for?

• Security (aka, Securability, just to fit in): Validated via demonstration. Sometimes a customer will hire an outside firm to do a security audit and/or intrusion testing.

• Maintainability: Validated via demonstration of how we will deliver software updates/patches.

• Configurability: Validated via demonstration of how the customer can modify the system to suit their needs.

This is by no means standard, and I don't think there is a "standard" definition, as the conflicting answers here demonstrate. The most important thing for your organization is that you define these terms precisely, and stick to them.

Christlike

plus 1 for the nice answer and "aka, Securability, just to fit in" :) . Funny thing :) SO team did not take into consideration the fact that in the real world someone may replace the + sign with the real word like I did. So they do not allow to type +1 as first word in the comment but they allow "plus 1" :). So functionally, they failed to test this properly :). Myabe they just tried acceptance tests :) – Hearty

Answer 2

I like the answer of Patrick Cuff. What I like to add is the distinction between a test level and a test type which was for me an eye opener.

test levels

Test level is easy to explain using V-model, an example: Each test level has its corresponding development level. It has a typical time characteristic, they're executed at certain phase in the development life cycle.

1. component/unit testing => verifying detailed design
2. component/unit integration testing => verifying global design
3. system testing => verifying system requirements
4. system integration testing => verifying system requirements
5. acceptance testing => validating user requirements

test types

A test type is a characteristics, it focuses on a specific test objective. Test types emphasize your quality aspects, also known as technical or non-functional aspects. Test types can be executed at any test level. I like to use as test types the quality characteristics mentioned in ISO/IEC 25010:2011.

1. functional testing
2. reliability testing
3. performance testing
4. operability testing
5. security testing
6. compatibility testing
7. maintainability testing
8. transferability testing

To make it complete. There's also something called regression testing. This an extra classification next to test level and test type. A regression test is a test you want to repeat because it touches something critical in your product. It's in fact a subset of tests you defined for each test level. If a there's a small bug fix in your product, one doesn't always have the time to repeat all tests. Regression testing is an answer to that.

Havard

This is the best answer to this question and "distinction between a test level and a test type" is something that most answers miss here and you are right it is "eye opener" – Oogenesis

Answer 3

The difference is between testing the problem and the solution. Software is a solution to a problem, both can be tested.

The functional test confirms the software performs a function within the boundaries of how you've solved the problem. This is an integral part of developing software, comparable to the testing that is done on mass produced product before it leaves the factory. A functional test verifies that the product actually works as you (the developer) think it does.

Acceptance tests verify the product actually solves the problem it was made to solve. This can best be done by the user (customer), for instance performing his/her tasks that the software assists with. If the software passes this real world test, it's accepted to replace the previous solution. This acceptance test can sometimes only be done properly in production, especially if you have anonymous customers (e.g. a website). Thus a new feature will only be accepted after days or weeks of use.

Functional testing - test the product, verifying that it has the qualities you've designed or build (functions, speed, errors, consistency, etc.)

Acceptance testing - test the product in its context, this requires (simulation of) human interaction, test it has the desired effect on the original problem(s).

Kentish

This is my favourite answer on this question. The distinction between a problem and a solution helps to make this distinction a lot clearer. – Bolt

Finally a clear explanation – Presbyterate

Answer 4

The answer is opinion. I worked in a lot of projects and being testmanager and issuemanager and all different roles and the descriptions in various books differ so here is my variation:

functional-testing: take the business requirements and test all of it good and thorougly from a functional viewpoint.

acceptance-testing: the "paying" customer does the testing he likes to do so that he can accept the product delivered. It depends on the customer but usually the tests are not as thorough as the functional-testing especially if it is an in-house project because the stakeholders review and trust the test results done in earlier test phases.

As I said this is my viewpoint and experience. The functional-testing is systematic and the acceptance-testing is rather the business department testing the thing.

Multiplicity

I like this answer :) They're pretty much the same thing. – Emission

UAT is yes ultimately done by the "paying" customer. However, it is most of time first done by a QA person that is "good" with testing and "trying" to break the the system and looking for all the "little" things BEFORE the "paying" customer gets their hands on it. Selenium automation for repeating tedious things can also be used along with true UAT testing by a QA tester, but never to repeat true testing to test all functionality expected with all expected browsers. UAT is pretty self explanatory. I think most of the functional testing descriptions seem to be to robotic and dictionary. – Jefferyjeffie

As I said this is my experience how the terms are interpreted. – Multiplicity

That is fine. When I noticed this vague definition... I just had to comment "functional-testing: take the business requirements and test all of it good and thorougly from a functional viewpoint." – Jefferyjeffie

Haha, yes, now I understand you. Okay, this is something you could write a whole book about it. I did not want to go into this too much the moment I wrote it. – Multiplicity

Answer 5

1. Audience. Functional testing is to assure members of the team producing the software that it does what they expect. Acceptance testing is to assure the consumer that it meets their needs.

2. Scope. Functional testing only tests the functionality of one component at a time. Acceptance testing covers any aspect of the product that matters to the consumer enough to test before accepting the software (i.e., anything worth the time or money it will take to test it to determine its acceptability).

Software can pass functional testing, integration testing, and system testing; only to fail acceptance tests when the customer discovers that the features just don't meet their needs. This would usually imply that someone screwed up on the spec. Software could also fail some functional tests, but pass acceptance testing because the customer is willing to deal with some functional bugs as long as the software does the core things they need acceptably well (beta software will often be accepted by a subset of users before it is completely functional).

Cologne

Answer 6

Functional Testing: Application of test data derived from the specified functional requirements without regard to the final program structure. Also known as black-box testing.

Acceptance Testing: Formal testing conducted to determine whether or not a system satisfies its acceptance criteria—enables an end user to determine whether or not to accept the system.

Perri

Answer 7

In my view the main difference is who says if the tests succeed or fail.

A functional test tests that the system meets predefined requirements. It is carried out and checked by the people responsible for developing the system.

An acceptance test is signed off by the users. Ideally the users will say what they want to test but in practice it is likely to be a sunset of a functional test as users don't invest enough time. Note that this view is from the business users I deal with other sets of users e.g. aviation and other safety critical might well not have this difference,

Coulter

Acceptance testing will determine whether or not a system satisfies the acceptance criteria of a given use case or all imaginable use cases. It is usually performed by a expert-user to determine whether or not the system is acceptable. In aeronautics a test pilot is an aviator who tests new aircraft by fling specific maneuvers. Top pilots, navigators and engineers conduct flight tests and at the end of the test missions they will provide evaluation and certification data. – Nyala

Answer 8

Acceptance testing:

... is black-box testing performed on a system (e.g. software, lots of manufactured mechanical parts, or batches of chemical products) prior to its delivery.

Though this goes on to say:

It is also known as functional testing, black-box testing, release acceptance, QA testing, application testing, confidence testing, final testing, validation testing, or factory acceptance testing

with a "citation needed" mark.

Functional testing (which actually redirects to System Testing):

conducted on a complete, integrated system to evaluate the system's compliance with its specified requirements. System testing falls within the scope of black box testing, and as such, should require no knowledge of the inner design of the code or logic.

So from this definition they are pretty much the same thing.

In my experience acceptance test are usually a subset of the functional tests and are used in the formal sign off process by the customer while functional/system tests will be those run by the developer/QA department.

Savoury

Answer 9

The relationship between the two: Acceptance test usually includes functional testing, but it may include additional tests. For example checking the labeling/documentation requirements.

Functional testing is when the product under test is placed into a test environment which can produce variety of stimulation (within the scope of the test) what the target environment typically produces or even beyond, while examining the response of the device under test.

For a physical product (not software) there are two major kind of Acceptance tests: design tests and manufacturing tests. Design tests typically use large number of product samples, which have passed manufacturing test. Different consumers may test the design different ways.

Acceptance tests are referred as verification when design is tested against product specification, and acceptance tests are referred as validation, when the product is placed in the consumer's real environment.

Mitsue

Answer 10

Acceptance testing is just testing carried out by the client, and includes other kinds of testing:

• Functional testing: "this button doesn't work"
• Non-functional testing: "this page works but is too slow"

For functional testing vs non-functional testing (their subtypes) - see my answer to this SO question.

Cavuoto

Answer 11

They are the same thing.

Acceptance testing is performed on the completed system in as identical as possible to the real production/deployement environment before the system is deployed or delivered.

You can do acceptance testing in an automated manner, or manually.

Duffey

While automation with Selenium and Watin (or Watir) etc... are very valuable first line of defense, nothing beats a trained QA person that is set on "breaking the system. Automation is great, but with modern development of AJAX and javascript framework and the changing of output on a page, to automate everything is a scripting updating nightmare. They are NOT the same thing – Jefferyjeffie