What information do you request when resolving an issue on a client's box using a desktop app?

Question

We write a Windows desktop application that is installed on a few thousand computers that we have no access to. When one of these users reports a bug, even if the description is quite thorough, there is other information which may be helpful.

We are currently working on an automated feedback agent, which means we will have access to very detailed information. But like a kid in a candy store, we're not sure where to start. What information would you include in your feedback package in the same case? What is useful to help us reproduce there error?

What we've got so far is:

• The version number of our app,
• OS and OS version number
• Proxy information
• .NET version and update number
• Information specific to our app (eg. data versions)
• Access and error logs

Note that this is similar to this question, however we're not so much interested in getting info when the program crashes as we are as when the user experiences a bug.

Edit: Clarification: not asking for what information the user should give in the case of a bug, but rather what information should we gather programmaticaly?

Unless

Related: #6762603 – Unless

Answer 1

Even if you have all of that information, reproducing the issue can be difficult. When users describes how they created the bug they're often mistaken about key steps - it's hard for them to know what areas are critical when they don't know the inner workings of the application. You might want to implement a way of tracking user actions, such as root-level event handling or some other means - if you have undo/redo functionality then I'm sure this would suffice. You could then include the last (x) steps of the action chain in the error report.

Ostensive

+1 for using the built in undo functionality. Excellent idea. – Unless

Using undo functionality, if available, really seems like a good candidate for tracking. Best idea so far. – Fibriform

Answer 2

Edit: I think I misread your question. I thought you're talking about what information to get from a customer reporting a bug, rather than already planning what I describe here. I'll leave it for reference anyway.

In a similar situation, despite fewer users, our app got a "Package logs for support" button which will create a zip file with all log files, and the currently opened project file, if any. All other information you described is already part of one of the log files. This way, a customer can conveniently send the ZIP file to us, which can be done from the main application window without opening a project file or connecting to a network interface, which are the two main points where something can go wrong. This makes it so much easier than to rely on the user providing feedback "by hand".

Other than that, the exact steps to reproduce the problem must be available. Most of that is typically in the project file itself (which is in the ZIP file we get), with only a few steps missing.

Things that seemed to be important for this, other than what you already listed:

• User/account information. This may help with permission issues. You might want to include things like timezone, locale, Windows theme.
• Application configuration, including where it is installed.
• As already said, the current file the user is working with, because this may be the cause of the problem.
• User settings, i.e. the data the application stores per user. Seen weird things with these. MRU list may also be helpful.

Fibriform

Answer 3

Aside from the basic state of the application (version, configuration, etc.), the most important information to get is:

• The step(s) required to reproduce the error or bug, including the inputs used if possible
• The expected output
• The actual output
• Contact information in case you need to follow up (if your software is consumed anonymously)

That will be enough information to solve 99.9% of the issues. For the rest, follow up and get any detailed information you think will help solve the problem (which, hopefully, will be far better understood at this point).

Freya

However, if we can get some set of detailed information automatically at no expense to the user, why not bundle it up when they make a request and avoid a two way trip up and down our (unfortunately but necessarily) long communication chain to the end user? – Unless

@Robert: Of course, the more automation the better. However, this incurs a potentially huge cost setting it up in code. You would have to log every piece of information entered into the system, which would essentially duplicate your underlying data set. Unless the system is unstable (in which case there are bigger issues to deal with), it should be sufficient to ask for the relevant information only when it's needed. – Freya