Contributing R test scripts

Question

Although tools like RUnit, svUnit, and testthat are good for package developers, I think it would be useful to have some means of uploading test scripts or even just usage examples for particular packages. Users who do continuous integration (e.g. Jenkins) or even basic unit testing may already have such tests and could find it beneficial to contribute scripts for package maintainers' use.

Does such functionality exist, either through CRAN or RForge, or via other sites, such as github? If so, is there a major example of using a repository, e.g. github, to allow users to contribute tests?

---

(Thanks to @mariotomo for reminding me of svUnit.)

Halves

This is a really good idea, for R and for other languages. – Bort

I've had a few people contribute tests through pull requests on github – Commemoration

do not forget svUnit! :) it does quite nice within Jenkins. – Upward

Answer 1

I would suggest to refrain from overcomplicating things. Why not just something like this:

• Look at a given package's sources (ie on r-forge, rforge, github, ... or straight CRAN sources),

• understand its testings scheme (ie tests/ directory, examples in manual pages, or one of the three unit testing frameworks from CRAN), and

• contribute new tests.

That's really all there is too it. Same for contributing documentation, demo scripts, new code, .... We can and should focus on the open in open source.

Obovoid

+1 Good advice. I had overlooked the tests/ directory. I had meant for tests to be visible to other users, too, whether or not they affect the release (e.g. if the author intends for behavior to change, even if certain tests break - visibility for that is useful to others). I made an oversight on the user side and had not integrated the test/ results as part of the validation phase before using an upgraded package. Thanks for a very good reminder. – Halves

Pleasure. Your call for small incremental contributions is good in its focus on small, actionable things. – Obovoid

One wrinkle I just discovered in a small sample of packages, albeit several that are critical for my work, is that those with a tests/ directory are, more likely than not, written by R Core or at least very exemplary developers. The rest were written by very solid researchers, but they may not have much development expertise. I'll need to mull on whether it is better to evangelize testing to many package maintainers or just post tests for a bunch of packages in a repository. – Halves

Another idea is if you had a foo-tests repo on github, you could look at the fork graph and see who else had added tests. – Bort

@Dirk: I don't know whether it's proper for me to edit your answer, but I think it is almost complete. One addition I'd recommend regards the example code question. Within a package the Writing R Extensions manual says the standard locations are the demo/ and test/Examples/ subdirectories, and, as you point out, tests/ for test code. – Halves

tests/Examples/ is, as I read this, just for reference output of the examples in the help pages. Whereas tests/ itself can already contain tests and their reference output --- and this is what I did, or still do, for packages not converted yet to proper unit tests. – Obovoid

Thanks for the answer & follow up. It's not quite what I was hoping for, but you've pointed out a useful & simpler path that I'd overlooked. – Halves

Well I didn;t mean to discourage you from starting a great and grand new scheme. But I am simply trying to remain pragmatic: it is not that the community efforts like the R Wiki, Crantastic, ... have been runaway successes. So I'd consider small and incremental schemes too. Either way, more tests is good. – Obovoid

Sorry if my tone seemed discouraged - I'm not. I now see oodles of packages that need a tests/ subdirectory! :) – Halves