The cpan tool that ships with core Perl includes a -l option, that directs it to provide a list of installed modules. For example, the last 10 items in the list on my system:
$ cpan -l 2>/dev/null |tail -10
Test2::Event::Encoding 1.302073
Test2::Event::Bail 1.302073
Test2::Event::Exception 1.302073
Test2::Event::Subtest 1.302073
Test2::Event::Skip 1.302073
Test2::Event::Info 1.302073
Test2::Event::Diag 1.302073
Test2::Event::TAP::Version 1.302073
JSON::PP 2.27400_02
JSON::PP::Boolean undef
As demonstrated here you get a list of modules and version numbers. Sometimes the tool doesn't find the version in META, and therefore will return undef for the version numbers. CPAN authors should be on the lookout for these sorts of mistakes, as they make it harder for tools that wish to identify the version without compiling the module itself.
Once you have the modules and version numbers you can use the cpanm tool (provided by App::cpanm) with its --test-only option to pull down a module of a specific version and test it. You can request a specific version like this:
cpanm Some::[email protected]
(Pulls down only version 0.9990 of the target module).
Where things get tricky are the following: Perl ships with a bunch of modules, and some of them also receive updates via CPAN. The cpan -l tool will list all installed modules, including the ones that ship with Perl.
Also some of the modules listed are simply part of a larger distribution.
Another tool that can be useful to you, which has been bundled with Perl since 5.8.9, is corelist. If you run this:
corelist File::Temp
You will get: "File::Temp was first released with perl v5.6.1"
If you do this:
corelist JSON
You will get: "JSON was not in CORE (or so I think)"
So it is pretty simple to determine if a module you're looking at in your list is one that ships with Perl or not. Use that information as you see fit.
Another thing you'll have to solve for is what to do about shared dependencies. If the first thing you test is a Moose upgrade, you'll pull in half of CPAN (that's an exaggeration), and that will dirty your environment for testing other modules. To mitigate this effect you have a few choices. One is to leverage App::perlbrew and its lib option to set up throwaway library space. That way you can install a module and its dependencies in a destination designated by perlbrew lib and perlbrew use, and then throw it away when done to move on to the next library for testing.
However, there may be a better solution, of which I am not adequately familiar to document here: The toolchain used by CPAN smoke testers. See CPAN::Testers if you wish to pursue this strategy. The smoke testers have worked out relatively lightweight approaches to pulling down and testing modules with their dependencies in an automated fashion.
Finally, another issue you will encounter is that CPAN authors are the ones who decide which versions of their modules live on CPAN and which ones get deleted. A few years ago CPAN authors were encouraged to keep their CPAN repositories clean by deleting old versions. I don't know if this practice is still encouraged, but it does mean that you cannot count on a specific version number still existing. To solve for this problem you should probably maintain your own repository of the tarballs for all versions you have installed at a given moment in time. The CPAN module framework Pinto helps with keeping versions of a module, pinning some to not update, and other useful tricks.
cpan -lgives you a list of installed cpan modules and versions, but the list includes modules which are bundled with Perl. That's not necessarily a bad thing if the goal is to test everything. Thecpanmutility facilitates a--test-onlymode, and allows you to specify a version. – Chillycpan/cpanp/cpanmorprove? What about modules I update because they are outdated, but don't actually use? – Godforsaken