What is a smoke testing and what will it do for me?

Question

I thought I understood that a smoke server was used for continous integration to measure the performance and test coverage of a project. Is this basically correct? What does it mean to smoke-test a program? Does it simply mean to continously apply the tests defined on the smoke server to be able to pinpoint performance degradation to a particular time frame?

This other question does describe the difference between sanity testing and smoke testing but does not ask to define smoke testing. This question is many years older than the reference question.

Mash

Does this answer your question? What is the difference between smoke testing and sanity testing? – Rambouillet

Answer 1

Smoke testing is a set of basic cheap to run tests that precede actual testing. It aims to verify that the build is deployed successfully and that all test env. aspects are running and ready for the actual test process. It saves you bringing the full extent of your testing wrath down a faulty build and just realizing that you have been testing on a bad env. or erroneously deployed build possibly too late.

Clintonclintonia

I was totally off. Cheap to run... basically an indication of fire? – Mash

@ojblass: Yes, what you might me referring to is soak-testing - en.wikipedia.org/wiki/Soak_testing. – Aday

I'd say it doesn't nessecarily preceed the actual testing. You might use smoke test also to test if a deployment was really successful. – Hydnocarpate

Also, smoke tests aren't just for checking that env is ok, but you can also check if -in th case of a desktop app for example- that the apps opens correctly and doesn't crash. – Butterbur

@VineetReynolds No, it's definitely not "soak testing". That is testing for a long period of time. Smoke testing is very quick... when you turn it on, does smoke come out? That's it. Soak testing looks for more subtle problems that can only occur after a lot of time "soaking". – Patroon

I think the term actually stems from real world testing of chimneys - it's a simple safety check to ensure there's nothing preventing smoke leaving a chimney. – Foreshow

Wow, I was completely wrong up to now. I thought a smoke test tests a system under heavy load. Thank you for correcting me. – Honeysucker

Answer 2

Turn it on, see if any smoke comes out.

Staley

This is where the name comes from, actually. It's a term from hardware engineers that got reused in software – Constitute

Answer 3

The term originates in hardware repair and has been applied to software. It's intended to be a quick test to see if the application "catches on fire" when run for the first time. As stated above it's just to make sure you don't waste a bunch of folks time by setting them loose on something that's obviously broken.

Circumsolar

Answer 4

Smoke testing isn't performance testing or regression prevention testing.

Smoke testing is a set of your highest priority fully automated tests. Even if your regular build succeeds, it's possible for your product to be broken in very fundamental ways that make it 100% unusable. Smoke tests are designed to test that level of core functionality. A build that doesn't pass smoke tests is very likely to fail a majority of all of your automated tests.

If a smoke test fails that generally means that you're losing productivity because the build is not usable, can't be tested, etc. Smoke test failures tend to require immediate fixes, on the order of hours not days.

The important aspect about smoke tests is that they can be run quickly. You should ALWAYS be running smoke tests of some sort. Many groups have sufficient resources to run a larger suite of tests on their daily or continuous integration builds (which is goodness), but smoke tests should be considered the bare minimum.

Lagasse

I don't believe smoke tests are necessarily automated - it can also refer to basic manual tests - which may cover things that cannot really be tested automatically such as layout. – Meta

True, but we are getting to the point where almost anything can be automated. Be lazy, automate. – Celestyn

Answer 5

The wikipedia page on smoke testing is actually quite good. It includes a pointer to Steve McConnel's ("Code Complete") paper in the best practices section of IEEE Software 13(4), 1996 titled "Daily Build and Smoke Test".

I'd say smoke testing is a precondition to unit and other forms of testing: if the smoke test fails there's no point in even starting up a unit test.

Adaadabel

Answer 6

We use the term smoke test to refer to a testing that does quick basic tests to an application or product to make it qualify for further testing like integration testing or functionality testing. It is of little use to do integration testing or other adavanced testing if the application fails to do some of the elementary things required for these advanced testing. For instance we have a web based application and we do smoke test to ensure we can load all the web pages, lets users to login/logout etc. These are the basic things that should work with the application to make it testable. If the pages do not even load or if the users could not even login, we cannot do much with functionality or integration testing. We do smoke testing with every build followed by functionality and integration testing.

Waterage

Answer 7

Smoke testing is the first step of integration testing. As a developer, you've put all the parts together and you're making sure your app will run before giving it to your QA team for "real" testing.

I wanted to write more, but MahdeTo beat me to a correct answer... :)

Intumesce

Answer 8

*checking the basic and critical feature of an application before proceeding the testing is known as smoke testing. *In smoke testing we check the positive flow of basic and critical feature to check if the build is testable or not.

Petterson

Answer 9

I agree with some of the previous answers. To me, Smoke Testing is simply a check if all my X thousands of sub-pages and redirects are still working.

Working means that the status code of each sub-page is "200 OK".

I run Smoke Tests as late as possible. After I deployed my code to the server and before I take this server active.

Today there is even an online service (SaaS) doing the job for me. https://www.smest.it helps me with Smoke Testing. I just entered once my private domain on their homepage and ever since I get my website checked daily. No stress at all.

Antisemite

Answer 10

Smoke testing can also be called build verification testing or build acceptance testing, if smoke testing fails it means build is not usable and can not be accepted , this defect needs to be fixed immediately on high priority

Sleek

Answer 11

From Wikipedia

a smoke test may address basic questions like "does the program run?", "does the user interface open?", or "does clicking the main button do anything?" The process of smoke testing aims to determine whether the application is so badly broken as to make further immediate testing unnecessary.

Postconsonantal