restore all R packages after installing a new version of R?

Question

When you are doing an R update, what is the best approach to re-installing and updating all packages that were already installed on your previous R version when some of your packages are on CRAN but the rest are on github (or other sources)?

In the past, I've followed this approach:

Open old version of R (e.g. R 3.6) and make a copy of all installed packages:

installed <- as.data.frame(installed.packages())
#save a copy
write.csv(installed, 'previously_installed.csv')

Then install and open new version of R (e.g. R 4.1), read in the old package names and install (from default: CRAN):

previously_installed <- read.csv('previously_installed.csv')
package_list <- as.character(previously_installed$Package)
package_list

install.lib <- package_list[!package_list %in% installed.packages()]   
for(lib in install.lib) install.packages(lib, dependencies = TRUE)

This works really well but will only install packages that are on CRAN so all packages that are only on github won't be installed. Is there a way to automatically install these packages from github?

You could work out which packages weren't installed (e.g. remaining github packages):

git_packages_not_installed <- install.lib[!install.lib %in% installed.packages()] 

I think you need to know the authors name to install all github packages though so I'm not sure how to automate this (e.g. devtools::install_github("DeveloperName/PackageName"). I know you can supply two repository options but I'm not sure this helps or see here.

What is best practice in this situation?

thanks

Accidence

You could try looking at the URL field in the description to see if it lists the repo: installed.packages(fields ="URL"). In the future you should keep a list of packages that you install from github somewhere else to make it easier to reinstall them. Maybe consider something like renv to manage package dependencies rather than just trying to copy over entire package libraries. – Swithin

@Swithin thanks, what do you mean by look in the description to see if there's a URL, where would I find that? Keeping a list doesn't seem very R like! Instead of coping entire package libraries, what is your approach, do you only install packages as you need them after an update? Surely you have a core number of packages you always need to install? – Accidence

I mean run installed.packages(fields ="URL") to find the URLs given in the description files for your installed packages (on your old R). You can grep for "github" in the URL to see if they included the repo where the package came from. Note that many CRAN packages will still also include the github repo link in the URL, so just do this for packages that fail. I always start over with major R releases and install on demand. This helps me avoid reinstalling things I'll never use. I work on so many different machines that it better for me to manage packages at the project level, not system. – Swithin

Package pak should take care about installing either from CRAN or from GitHub in a single command. But still authors names are needed. – Chipman

Answer 1

If you were only using CRAN packages, my advice would be similar to @CaptainHat's, but with one extra step: first copy all the old packages to the new location, but then call update.packages(checkBuilt = TRUE, ask = FALSE). This will update the ones that were built for an incompatible earlier version of R. (Only copy ones that aren't already there. If you copy base packages, that will break R.)

Unfortunately, that doesn't know how to update packages you installed from Github. I believe remotes::update_packages() should be able to handle those, but I've never actually tried it.

Exosmosis

thanks, as a matter of interest, do you install all packages that you previously used or do you follow @Swithin advice above and just install as you need? For github packages, it sounds like you follow this approach anyway? – Accidence

As a rule, I don't use github packages except to check whether a bug in the CRAN version has already been fixed before I submit a patch for it. For CRAN packages, I do what @Swithin does: install as needed. For some projects, I have a needsPackages function at the start that installs everything I know is needed. – Exosmosis

interesting. when i say github packages, I'm referring to packages that are not on CRAN at all, e.g. simple ones that contain addins etc – Accidence

I'm sure some packages that aren't on CRAN or BioC are good quality, but I don't figure it's worth the time to work out which ones, so I just avoid them all. – Exosmosis

Answer 2

Don't delete your old 'library' folder, and copy the contents into your new 'library' folder.

Eg. copy the contents of C:\Program Files\R\R-4.0.2\library into C:\Program Files\R\R-4.0.3\library. That's where R will look for them.

Doss

I'm afraid this isn't what I want. I want to update (re-install) all the packages. Your approach just copies the old ones – Accidence

Bu default R stores libraries only for major releases like R-4.0 vs R-4.1. And just copying the files without recompiling for the different version will often cause breaking errors, especially going from R-3-6 to R-4-0. This may work for R only packages without compiled code, but it still not really a recommend way to go between version (copying from one computer to another with the same R version would probably be safer) – Swithin

ah I see. Thanks – Doss