Test accounts and products in a production system

Question

Is it worth designing a system to expect test accounts and products to be present and active in production, or should there be no contamination of production databases with test entities, even if your shipping crew knows not to ship any box addressed to "Test Customer"?

I've implemented messaging protocols that have a test="True" attribute in the spec, and wondered if a modern schema should include metadata for tagging orders, accounts, transactions, etc. as test entities that get processed just like any other entity--but just short of the point where money gets spent. Ie: it fakes charging an imaginary credit card and fakes the shipment of a package.

This isn't expected to be a substitute for a fully separated testing, development, and QA database, but even with those, we've always had the well-known Test SKU and Test Customer in the production system. Harmless?

Nitride

Answer 1

Having testing accounts in production is something I usually frown upon because it opens up a potential security hole. One should strive to duplicate as much of the production environment in testing as possible but there are obviously cases where that isn't possible. Expensive production only hardware is a prime example. I would say as a general practice it should be discouraged but as with all things if you can provide a reason which makes sense to you then you might overlook a hard and fast rule.

Binding

Personally I think having a mechanism to simulate transactions is essential to not only smoke test a production system, but also to help analyse specific production issues. – Description

Answer 2

I imagine the Best Practice Police would state the mantra "never ever test in prod" and maybe even throw in "developers should not have access to prod".

However, I work on a mainframe-based system where there are huge differences between production and test/qa/qc; the larger the system, the more likely such a situation is. Additionally, the more groups that have a stake in the application, the more likely this is.

I need more than two hands to count how many times we could only duplicate a problem in the production environment. The option then becomes creating test tables/users/data or using live customer data.

At times we do also create test records in production tables, as some users/clients like having something they can search/retrieve that is always there.

So my advice is that it is OK to put test accounts/products into production if it will help to troubleshoot after go-live.

Martinmartina

Answer 3

If your database is created from scripts in an automated fashion, then this becomes a non-question.

In my environment we use cruise control for continuous builds. The SQL Scripts for generating the database are checked into CVS with everything else, and the database is rebuilt from those scripts on a daily basis.

Our test data is a second set of sql scripts, which are run for the test database and are not run for the production database.

Given our environment test data never touches the production database.

This solution really works great for us.

Toile

Answer 4

I wouldn’t put any test data in a production system nor would I want to have access to this system as a developer.

I’m working in an industry with very sensitive medical and financial information and having such information would make it impossible to distinguish productive from data out of the testing system.

IMHO the best practice is to completely separate these two worlds and invest in setting up a procedure to prepare a comprehensive testing environment.

Hearten

a noble goal, but not always feasible in my experience, as @Martinmartina points out – Birth

I successfully managed not ever touching a production system and don't allow it for any of my developers. I only permit sitting next to somebody in the operations or support team advising on some interventions on live data. I treat it like a holy grail, this makes everybody much more responsible. – Hearten

Answer 5

In out ERP systems (internally accessible only) we have test data so that when we move changes from test to production environments we can test the whole process. I view that data as a necessary evil, since subtle configuration differences between systems can cause catastrophic results, so once a change is in production we test is fully before "releasing" it to the users.

As I said though, these are internal apps only, so the security risks are lessened somewhat - that's a very valid concern.

Birth

Answer 6

Never ever test in prod, even though that is where all the revenue is generated/stats are collected/magic happens...?

Always have a production test plan. There are going to be problems that happen on prod, or, if you are unlucky, only happens on prod. If you don't have anything in place, the first time you need to test on prod (which are usually high-stress cases) you'll be up the creek without a paddle.

It's not harmless to have test data on prod, you do need to be careful.

Downpour