How to keep dir-local variables when switching major modes?

Question

I'm committing to a project where standard indentations and tabs are 3-chars wide, and it's using a mix of HTML, PHP, and JavaScript. Since I use Emacs for everything, and only want the 3-char indentation for this project, I set up a ".dir-locals.el" file at the root of the project to apply to all files/all modes under it:

; Match projets's default indent of 3 spaces per level- and don't add tabs
(
 (nil .
        (
         (tab-width . 3)
         (c-basic-offset . 3)
         (indent-tabs-mode . nil)
         ))
 )

Which works fine when I first open a file. The problem happens when switching major modes- for example to work on a chunk of literal HTML inside of a PHP file. Then I lose all the dir-local variables.

I've also tried explicitly stating all of the modes I use in ".dir-locals.el", and adding to my .emacs file "dir-locals-set-class-variables / dir-locals-set-directory-class". I'm glad to say they all behave consistently, initially setting the dir-local variables, and then losing them as I switch the major mode.

I'm using GNU Emacs 24.3.1.

What's an elegant way of reloading dir-local variables upon switching a buffer's major-mode?

-- edit -- Thanks for the excellent answers and commentary both Aaron and phils! After posting here, I thought it "smelled" like a bug, so entered a report to GNU- will send them a reference to these discussions.

Landin

In general, you want to hook after-change-major-mode-hook to reread the directory-local variables after the change, e.g. (add-hook 'after-change-major-mode-hook 'reread-dir-locals) -- the trouble is that there is no function reread-dir-locals, and I'm not finding any good way in the Elisp manual to tell Emacs it should reapply the dir-local variables. I haven't time right now to go digging through the Emacs Lisp library sources to find out where and how those are read, but perhaps this will give you a place to start. – Disappointed

Answer 1

As per comments to Aaron Miller's answer, here is an overview of what happens when a mode function is called (with an explanation of derived modes); how calling a mode manually differs from Emacs calling it automatically; and where after-change-major-mode-hook and hack-local-variables fit into this, in the context of the following suggested code:

(add-hook 'after-change-major-mode-hook 'hack-local-variables)

After visiting a file, Emacs calls normal-mode which "establishes the proper major mode and buffer-local variable bindings" for the buffer. It does this by first calling set-auto-mode, and immediately afterwards calling hack-local-variables, which determines all the directory-local and file-local variables for the buffer, and sets their values accordingly.

For details of how set-auto-mode chooses the mode to call, see C-hig (elisp) Auto Major Mode RET. It actually involves some early local-variable interaction (it needs to check for a mode variable, so there's a specific look-up for that which happens before the mode is set), but the 'proper' local variable processing happens afterwards.

When the selected mode function is actually called, there's a clever sequence of events which is worth detailing. This requires us to understand a little about "derived modes" and "delayed mode hooks"...

Derived modes, and mode hooks

The majority of major modes are defined with the macro define-derived-mode. (Of course there's nothing stopping you from simply writing (defun foo-mode ...) and doing whatever you want; but if you want to ensure that your major mode plays nicely with the rest of Emacs, you'll use the standard macros.)

When you define a derived mode, you must specify the parent mode which it derives from. If the mode has no logical parent, you still use this macro to define it (in order to get all the standard benefits), and you simply specify nil for the parent. Alternatively you could specify fundamental-mode as the parent, as the effect is much the same as for nil, as we shall see momentarily.

define-derived-mode then defines the mode function for you using a standard template, and the very first thing that happens when the mode function is called is:

(delay-mode-hooks
  (PARENT-MODE)
  ,@body
  ...)

or if no parent is set:

(delay-mode-hooks
  (kill-all-local-variables)
  ,@body
  ...)

As fundamental-mode itself calls (kill-all-local-variables) and then immediately returns when called in this situation, the effect of specifying it as the parent is equivalent to if the parent were nil.

Note that kill-all-local-variables runs change-major-mode-hook before doing anything else, so that will be the first hook which is run during this whole sequence (and it happens while the previous major mode is still active, before any of the code for the new mode has been evaluated).

So that's the first thing that happens. The very last thing that the mode function does is to call (run-mode-hooks MODE-HOOK) for its own MODE-HOOK variable (this variable name is literally the mode function's symbol name with a -hook suffix).

So if we consider a mode named child-mode which is derived from parent-mode which is derived from grandparent-mode, the whole chain of events when we call (child-mode) looks something like this:

(delay-mode-hooks
  (delay-mode-hooks
    (delay-mode-hooks
      (kill-all-local-variables) ;; runs change-major-mode-hook
      ,@grandparent-body)
    (run-mode-hooks 'grandparent-mode-hook)
    ,@parent-body)
  (run-mode-hooks 'parent-mode-hook)
  ,@child-body)
(run-mode-hooks 'child-mode-hook)

What does delay-mode-hooks do? It simply binds the variable delay-mode-hooks, which is checked by run-mode-hooks. When this variable is non-nil, run-mode-hooks just pushes its argument onto a list of hooks to be run at some future time, and returns immediately.

Only when delay-mode-hooks is nil will run-mode-hooks actually run the hooks. In the above example, this is not until (run-mode-hooks 'child-mode-hook) is called.

For the general case of (run-mode-hooks HOOKS), the following hooks run in sequence:

• change-major-mode-after-body-hook
• delayed-mode-hooks (in the sequence in which they would otherwise have run)
• HOOKS (being the argument to run-mode-hooks)
• after-change-major-mode-hook

So when we call (child-mode), the full sequence is:

(run-hooks 'change-major-mode-hook) ;; actually the first thing done by
(kill-all-local-variables)          ;; <-- this function
,@grandparent-body
,@parent-body
,@child-body
(run-hooks 'change-major-mode-after-body-hook)
(run-hooks 'grandparent-mode-hook)
(run-hooks 'parent-mode-hook)
(run-hooks 'child-mode-hook)
(run-hooks 'after-change-major-mode-hook)

Back to local variables...

Which brings us back to after-change-major-mode-hook and using it to call hack-local-variables:

(add-hook 'after-change-major-mode-hook 'hack-local-variables)

We can now see clearly that if we do this, there are two possible sequences of note:

1. We manually change to foo-mode:

   (foo-mode)
    => (kill-all-local-variables)
    => [...]
    => (run-hooks 'after-change-major-mode-hook)
        => (hack-local-variables)
   

2. We visit a file for which foo-mode is the automatic choice:

   (normal-mode)
    => (set-auto-mode)
        => (foo-mode)
            => (kill-all-local-variables)
            => [...]
            => (run-hooks 'after-change-major-mode-hook)
                => (hack-local-variables)
    => (hack-local-variables)
   

Is it a problem that hack-local-variables runs twice? Maybe, maybe not. At minimum it's slightly inefficient, but that's probably not a significant concern for most people. For me, the main thing is that I wouldn't want to rely upon this arrangement always being fine in all situations, as it's certainly not the expected behaviour.

(Personally I do actually cause this to happen in certain specific cases, and it works just fine; but of course those cases are easily tested -- whereas doing this as standard means that all cases are affected, and testing is impractical.)

So I would propose a small tweak to the technique, so that our additional call to the function does not happen if normal-mode is executing:

(defvar my-hack-local-variables-after-major-mode-change t
  "Whether to process local variables after a major mode change.
Disabled by advice if the mode change is triggered by `normal-mode',
as local variables are processed automatically in that instance.")

(defadvice normal-mode (around my-do-not-hack-local-variables-twice)
  "Prevents `after-change-major-mode-hook' from processing local variables.
See `my-after-change-major-mode-hack-local-variables'."
  (let ((my-hack-local-variables-after-major-mode-change nil))
    ad-do-it))
(ad-activate 'normal-mode)

(add-hook 'after-change-major-mode-hook 
          'my-after-change-major-mode-hack-local-variables)

(defun my-after-change-major-mode-hack-local-variables ()
  "Callback function for `after-change-major-mode-hook'."
  (when my-hack-local-variables-after-major-mode-change
    (hack-local-variables)))

Disadvantages to this?

The major one is that you can no longer change the mode of a buffer which sets its major mode using a local variable. Or rather, it will be changed back immediately as a result of the local variable processing.

That's not impossible to overcome, but I'm going to call it out of scope for the moment :)

Buonomo

Your cogent exegesis is extremely appealing to me, and I wish to subscribe to your newsletter. (I'm also going to be working my way through your SO answers on Emacs, because wow.) – Disappointed

Well thank you for the compliment. This would be one of my more verbose efforts, so you'll be sorely disappointed if you're hoping for the same level of detail from all my answers :) but I do have my moments. I've often learned something new while answering a question, so I hope you do likewise if you do go digging through them. – Buonomo

n.b. In Emacs 26.1 the sequence has changed a bit wrt how and when hack-local-variables is called. See commit 25f455815bfa and commit 26171e02773b for details. – Buonomo

Answer 2

Be warned that I have not tried this, so it may produce undesired results ranging from your dir-local variables not being applied, to Emacs attempting to strangle your cat; by any sensible definition of how Emacs should behave, this is almost certainly cheating. On the other hand, it's all in the standard library, so it can't be that much of a sin. (I hope.)

Evaluate the following:

(add-hook 'after-change-major-mode-hook
          'hack-dir-local-variables-non-file-buffer)

From then on, when you change major modes, dir-local variables should (I think) be reapplied immediately after the change.

If it doesn't work or you don't like it, you can undo it without restarting Emacs by replacing 'add-hook' with 'remove-hook' and evaluating the form again.

Disappointed

Surely you simply want hack-local-variables, rather than the function you're calling? Changing modes kills all local variables, so it's not just dir-locals you need to consider. – Buonomo

FWIW I have one dir-local config which, for a particular mode, evals a different mode and then (hack-local-variables) in order to pick up the config for that other mode. I was slightly bemused when I first tried it, but it works fine. – Buonomo

@Buonomo I probably do want hack-local-variables, but I didn't know about it; I don't actually use dir-local or file-local variables, so this particular cranny of Emacs is new to me. (Short of one fellow who seems recently to have been intrigued by watching me work, I'm the only Emacs user in my organization; configuring my Emacs to match the local conventions is a lot easier than scattering file locals hither and yon, and it keeps vi users from looking at me funny, too. Well, more often than they already do, at least.) – Disappointed

@Buonomo Actually, I'm not so sure about that. Does the file-local variable alist get emptied on major mode change, or does Emacs just not reapply it after the change? In the former case, it'd be necessary to reread dir-local variables, which hack-dir-local-variables-non-file-buffer does and hack-local-variables doesn't seem to. – Disappointed

fundamental-mode (which is the eventual base for any standard major mode) calls kill-all-local-variables. – Buonomo

It's worth noting that this after-change-major-mode-hook approach will cause hack-local-variables to run twice in normal circumstances, as it's invoked by normal-mode right after setting the mode. If that's a problem, you could advise normal-mode to set a flag, and check for that in the custom after-change-major-mode-hook function. – Buonomo

@Buonomo It sounds like you have a much better understanding than mine of how Emacs handles file- and directory-local variables. Why not work up an answer? It'd almost certainly be better than what I've managed. – Disappointed

I suspect if I did it would only be a minor variation on what you've already suggested. I might do so later if no one else posts anything definitive in the interim. I guess a more detailed walk-through of the relevant sequence of events might be useful. – Buonomo

@Buonomo I think it would be, not least for my own edification. – Disappointed

Answer 3

My take on this:

(add-hook 'after-change-major-mode-hook #'hack-local-variables)

and either

(defun my-normal-mode-advice
    (function &rest ...)
  (let ((after-change-major-mode-hook
         (remq #'hack-local-variables after-change-major-mode-hook)))
    (apply function ...)))

if you can live with the annoying

Making after-change-major-mode-hook buffer-local while locally let-bound!

message or

(defun my-normal-mode-advice
    (function &rest ...)
  (remove-hook 'after-change-major-mode-hook #'hack-local-variables)
  (unwind-protect
      (apply function ...)
    (add-hook 'after-change-major-mode-hook #'hack-local-variables)))

otherwise and finally

(advice-add #'normal-mode :around #'my-normal-mode-advice)

Bandeen