Updating packages in Emacs

Question

I have the following setup for packages (not sure if there is a better recommended one):

(require 'package)
(setq package-archives '(("ELPA" . "http://tromey.com/elpa/") 
                          ("gnu" . "http://elpa.gnu.org/packages/")
                          ("marmalade" . "http://marmalade-repo.org/packages/")))

; Apparently needed for the package auto-complete (why?)
(add-to-list 'package-archives
             '("melpa" . "http://melpa.milkbox.net/packages/") t)

(package-initialize)
(setq url-http-attempt-keepalives nil)

I have three questions related to the installation and updating of packages.

Q1. Is there a way to update the list of available packages (and most recent versions) and update a specific package?

Q.2 What is the difference between the following package sources?:

• ELPA,
• GNU
• marmalade
• melpa

Q.3 Does it matter the order in which they are added to package-archives?

Thunderstruck

Do you use the M-x package-list-packages ? It shows all the available packages and their most recent version. You can also install from this interface. – Timoteo

MELPA is needed for auto-complete since obviously none bothered to make auto-complete available on Marmalade. Do not bother, but rather clean up your package-archives. Get rid of the ELPA archive, which is not maintained anymore, and of the marmalade archive, which is a mess of out-dated and duplicated packages without clear guidance and maintenance. Use only the standard GNU archive, and MELPA> – Pulmonic

Thanks @lunaryorn! That's very helpful. Do you know how I could stay informed on things like this? (i.e. "good-things-to-know" like these as Emacs evolves) – Thunderstruck

Information is freely flowing around in the web, just catch it: Follow /emacs on Reddit, follow blogs and twitter of renowned Emacs people like Nic Ferrier, Bozhidar Batsov, Magnar Sveen, John Wiegley, etc., join #emacs on Freenode IRC, and many more. – Pulmonic

You should (setq package-enable-at-startup nil) if you're going to call (package-initialize) yourself. – Crosslegged

user815423426: See https://mcmap.net/q/137900/-emacs-24-package-system-initialization-problems – Crosslegged

Answer 1

1. In order to automatically update the list of packages, only if there is no package list already, use the following:

   (when (not package-archive-contents)
       (package-refresh-contents))
   

   In order to update all installed packages, type package-list-packages, which will take you to the *Packages* buffer (and also update the list of packages), and then type U x.

   package-refresh-contents unconditionally tries to download a package list from all repos you've added to package-archives; package-archive-contents is non nil if you have already downloaded the package list.

2. ELPA is the original. I don't think it's really maintained anymore, but I'm not sure. I don't use it.

   GNU is "official". It's maintained along with Emacs, which means things should always work but updates and new packages don't come very often.

   Marmalade is basically a website where you can upload a complete package, and it will be added to the marmalade repo. You don't just submit a link to the package's upstream, and it doesn't quite automate the creation of the package completely. I think this is the Right Thing, because you don't necessarily want to track upstream. Unfortunately, it has been unmaintained for a while, but someone recently took it over so it should be back and better at some point.

   Melpa takes a URL to e.g. the EmacsWiki lisp area or a github repo, and builds a package automatically from it. Thus it is usually at most a day behind whatever it is tracking. Although it tracks upstream, I've never had a problem in practice, and this is where most of my packages are from. There is also Melpa Stable, which is like Melpa but grabs tagged revisions of the upstream repo instead of the latest revision. Melpa stable has fewer packages than Melpa. Probably Melpa should be preferred over Melpa Stable.

   Org mode has its own package.el repo (http://orgmode.org/elpa/).

   All of the package repos work the same, you just add them to your package-archives.

   Here's a more in-depth blog post about this subject, which I mostly agree with.

3. I'm not sure, but I think if a package is duplicated in different repos, the order the repos appear in in package-archives determines precedence. I don't know if higher precedence is at the beginning or end of the list.

Update: In Emacs 25, there is a variable package-archive-priorities that you can use to prioritize your package repos (e.g. prefer ELPA over MELPA).

---

Here is the relevant section of my init.el, if you're interested:

(setq jpk-packages
      '(
        ac-dabbrev
        ...
        yasnippet
        ))

(package-initialize)
(add-to-list 'package-archives
             '("melpa" . "http://melpa.org/packages/"))
(add-to-list 'package-archives
             '("org" . "http://orgmode.org/elpa/"))

;; install any packages in jpk-packages, if they are not installed already
(let ((refreshed nil))
  (when (not package-archive-contents)
    (package-refresh-contents)
    (setq refreshed t))
  (dolist (pkg jpk-packages)
    (when (and (not (package-installed-p pkg))
             (assoc pkg package-archive-contents))
      (unless refreshed
        (package-refresh-contents)
        (setq refreshed t))
      (package-install pkg))))

(defun package-list-unaccounted-packages ()
  "Like `package-list-packages', but shows only the packages that
  are installed and are not in `jpk-packages'.  Useful for
  cleaning out unwanted packages."
  (interactive)
  (package-show-package-list
   (remove-if-not (lambda (x) (and (not (memq x jpk-packages))
                            (not (package-built-in-p x))
                            (package-installed-p x)))
                  (mapcar 'car package-archive-contents))))

Jeniferjeniffer

Thanks! What does when (not package-archive-contents) check ? – Thunderstruck

The conditional prevents refreshing every time I start emacs (I usually only want it to check when I tell it to). It only checks if there are no package archives, which should only be the case the first time I clone my config to a new machine. – Jeniferjeniffer

Just letting you know of @Brandy Trainor's answer, which seems to complement your answer - in case it is helpful. – Thunderstruck

I think this needs to call (refresh-package-contents) unconditionally before downloading new packages, as otherwise Emacs may try downloading an old package version (which is by then a 404), fail, and stop loading. – Hackney

What is the best way to understand which package is unwanted? @Jeniferjeniffer – Adjudicate

@Adjudicate What do you mean by unwanted? Packages only get installed if you install them, or they're dependencies of some other package you install. – Jeniferjeniffer

@Jeniferjeniffer Sorry I meant unnessesary packages. Also can package-list-packages followed by U x run automatically each time emacs daemon starts? – Adjudicate

@Adjudicate U runs package-menu-mark-upgrades and x runs package-menu-execute. You should be able to run those automatically (you'll have to make sure you're in the *Packages* buffer probably with with-current-buffer or similar). I wouldn't recommend it though, upgrades can break things, I'd rather set aside some time to do it deliberately to make sure the upgrade worked. – Jeniferjeniffer

reddit.com/r/emacs/comments/etikbz/… advises against MELPA Stable. I can't say I disagree; I was one of the users who got bitten by false assumptions. – Childe

@Childe Yeah, that's not a bad point. I've switched away from Melpa stable when I updated to Emacs 29. – Jeniferjeniffer

Answer 2

Within Emacs, use M-x list-packages to list all packages which will automatically refresh the archive contents. Afterwards use U to mark all upgradable packages to be upgraded, and x to actually perform the new updates. Emacs will then fetch and install all upgrades, and ask you to whether to remove the old, obsolete versions afterwards.

You may also want to take a look at Carton which provides a more convenient way to manage your packages by declaring them in a dedicated file, and includes a convenient command line client to automatically install and upgrade packages declared in this way.

---

The order of package-archives does not matter. Emacs aggregates the contents of all archives into a single coherent list of available packages and their versions, stored in package-archive-contents.

Upon package-install, Emacs will simply pick the newest version of a package, regardless of the originating archive. For more control about package origin, MELPA provides the melpa package which allows to black- or whitelist packages from specified archives.

Coulisse

That's S-u to be more precise. – Leahleahey

@mcb What's S-u? The package menu doesn't use that binding. – Pulmonic

I mean Shift-u by that. – Leahleahey

@mcb Well, that's why I wrote a capital U 😊 – Pulmonic

It's just for the confused ones like me that think this means <kbd>U</kbd>. – Leahleahey

FWIW, S- means "super key" gnu.org/software/emacs/manual/html_node/emacs/… – Langdon

Can this be done using a bash script? – Adjudicate

Answer 3

In terminal:

emacs

M-x list-packages

this puts you in the *packages* buffer

shift-u x

emacs will ask you (y/n), wait for updates

C-x k <ret>

this will kill the *packages* buffer and return you back to *scratch*

C-x-C-c

this will exit emacs, and let you relaunch via, but you might have to debug :(

emacs

my 2¢

Couchman

I got “no command specified” or something like that when I typed “U”. After running M-x package-refresh-contents, it prompted me to type “U” to mark all for update, and then I was able to type “x” to run the updates. – Shevlo

Could this done automatically when emacs daemon starts? – Adjudicate

@Adjudicate server-mode-hook lets you run arbitrary functions when entering or leaving server-mode. Alternatively, defadvice around server-start. Probably post a separate question if you can't figure it out from these hints. – Childe

Answer 4

This is more of an extended comment on jpkotta's answer.

This is an adjustment I am experimenting with for jpkotta's answer above:

(setq n 0)                                  ; set n as 0
(dolist (pkg pkgs-2b-present)               ; for each pkg in list
  (unless (or                               ; unless
           (package-installed-p pkg)        ; pkg is installed or
           (assoc pkg                       ; pkg is in the archive list
                  package-archive-contents))
    (setq n (+ n 1))))                      ; add one to n
(when (> n 0)                               ; if n > 0, 
  (package-refresh-contents))               ; refresh packages

(replacing (when (not package-archive-contents) (package-refresh-contents))).

The package-list was not refreshing sufficiently often enough for my use-case.

I haven't considered if there is a more efficient solution to my problem; first, I've to see if the problem goes away with this adjustment.

Jalbert

In my answer, package-archive-contents is refreshed only if it doesn't exist. The idea is that you copy your .emacs.d to a new machine and it automatically installs all of your packages, but after that you have to update manually. I try to update about once every two weeks. I'm guessing you added a package to your list, but your package-archive-contents was too old to have it? – Jeniferjeniffer

@jpkotta, I have a dual boot, each using their default ~/.emacs.d/elpa/ for location of packages. However, my version of your package-installing-code is in a "third" partition, (e:/emacs-config or /e/emacs-config/). So I add a package to the list while in one OS, and it should update when I load Emacs in the other OS. For sure there is some element of overkill in my solution, but it seems to be working at least. – Jalbert

@jpkotta, why my package-archive-contents seemed to possibly be so out of wack escapes me. That is an issue I did not narrow down. If I look for and find a "more elegant" solution in the future, I will add it to my post. – Jalbert

(package-refresh-contents) writes to ~/.emacs.d/elpa/archives. It's not clear if you have that directory shared between OSes. I'd recommend against sharing the elpa/ directories, because Emacs bytecode (code in elpa/ is bytecompiled) is not backwards compatible (gnu.org/software/emacs/manual/html_node/elisp/…). You'd have to ensure that both installations have the same version of Emacs. I keep my .emacs.d in version control, and synchronize different installations with that. – Jeniferjeniffer