Structuring a Statistical Analysis with R Using Emacs/ESS

Question

I am looking for a way to structure my statistical analysis. I currently use Emacs/ESS and the analysis file turned out quite long. I have started to put parts of the code into sourceable functions in a separate file, but still...

I would like to introduce some kind of subheadings/ section titles in the file (Aggregation, Cluster-analysis, Simulations, ...) and write the code beneath it, so I could quickly jump to the sections that I want to work on.

I guess I could just use comments and search for them, but I couldn't get an overview or an index that way. I also thought about using Org-Mode for the Headlines, but that wouldn't be very convenient for collaborators, who work with another Editor.

I know R-Studio implements this with sections, so there will be a solution for emacs, right?

Thank you very much!

PS: something like imenu would work, but that's just for functions, not for sections

Cowbird

Answer 1

Recent (Feb 2013) additions to Orgmode mean that you should now be able to embed org headings in your source code, and then navigate through them using orgstruct-mode. So, upgrade your org mode via git, and then try opening the following example R file. When you are on a comment line that has an org heading embedded, just hit TAB, or shift-TAB, and you should get org-mode headings.

### * Create data
data = list( s1=list(x=1:3, y=3:1), 
     s2=list(x=1:5, y=1:5), s3=list(x=1:4, y=rep(3,4)))

### * Base graphics version


par(mfrow=c(2,2))
lapply(data, plot)

### * Lattice version

nplots <- length(data)
pts.per.plot <- sapply(data, function(l) length(l$x))
df <- data.frame(which=rep(1:nplots, times=pts.per.plot),
                 x=unlist(sapply(data, function(l) l$x)),
                 y=unlist(sapply(data, function(l) l$y)))

xyplot(y~x|which, data=df, layout=c(2,2))

### ** Make the pdf
pdf(file='o.pdf')
xyplot(y~x|which, data=df, layout=c(2,1))                 
dev.off()

### * End of file

### Local Variables:
### eval: (orgstruct-mode 1)
### orgstruct-heading-prefix-regexp: "### "
### End:

Hera

Hm - just tried it, but nothing happens? Version: Org-mode version 7.9.3e (7.9.3e-1123-g33d362 @ /home/rkrug/.emacs.d/org-mode/lisp/). Should I ask on org mailing list? – Decrepit

That's working for me for outlining, and it is very handy, but the headings are not displayed with a particular face. Do you know if there is an option to do that ? – Coextend

I've not seen anything inbuilt to orgstruct that will do extra font-locking, but the other example here with font-lock could be adapted to highlight the headings. – Hera

wow this amazing. is it possible to change the colors according to hierarchy ? – Xl

Answer 2

In general I use org-mode and org-babel, but when I've got to share scripts with others, I've got the following in my .emacs :

(defgroup ess-jb-faces nil
  "Faces used by cutomized ess-mode"
  :group 'faces)

(defface ess-jb-comment-face
  '((t (:background "cornsilk" 
    :foreground "DimGrey"
    :inherit font-lock-comment-face)))
  "Face used to highlight comments."
  :group 'ess-jb-faces)

(defface ess-jb-comment-bold-face
  '((t (:weight bold
    :inherit ess-jb-comment-face)))
  "Face used to highlight bold in comments."
  :group 'ess-jb-faces)

(defface ess-jb-h1-face
  '((t (:height 1.6 
    :weight bold 
    :foreground "MediumBlue"
    :inherit ess-jb-comment-face)))
  "Face used to highlight h1 headers."
  :group 'ess-jb-faces)

(defface ess-jb-h2-face
  '((t (:height 1.2 
    :weight bold 
    :foreground "DarkViolet"
    :inherit ess-jb-comment-face)))
  "Face used to highlight h2 headers."
  :group 'ess-jb-faces)

(defface ess-jb-h3-face
  '((t (:height 1.0 
    :weight bold 
    :foreground "DarkViolet"
    :inherit ess-jb-comment-face)))
  "Face used to highlight h3 headers."
  :group 'ess-jb-faces)

(defface ess-jb-hide-face
  '((t (:foreground "white"
    :background "white"
    :inherit ess-jb-comment-face)))
  "Face used to hide characters."
  :group 'ess-jb-faces)

(font-lock-add-keywords 'ess-mode
   '(("^###\\(#\\)\\([^#].*\\)$" (1 'ess-jb-hide-face t)(2 'ess-jb-h1-face t))
     ("^###\\(##\\)\\([^#].*\\)$" (1 'ess-jb-hide-face t)(2 'ess-jb-h2-face t))
     ("^###\\(###\\)\\([^#].*\\)$" (1 'ess-jb-hide-face t)(2 'ess-jb-h3-face t))
     ("^###\\( .*\\|$\\)" 1 'ess-jb-comment-face t)
     ("^###" "\\*.*?\\*" nil nil (0 'ess-jb-comment-bold-face append))
     ))

With this, any comment with #### at the beginning of the line is formatted as an "header 1". Any comment with ##### is formatted as an "header 2", etc. And lines which begin with ### is seen as a comment with a special font-lock (used for long comments).

This can give something like this :

This is quite hacky, but the advantage is that it only uses standard R comments and as such can be shared without problem with others. In general I use the following for "header 1" : others see it as below, while I enjoy my defined font-lock :

############################################
#### HEADER 1
############################################

With this syntax, you can also use the following to activate outline-minor-mode on the sections defined previously and be able to fold/unfold them :

(add-hook 'ess-mode-hook
      '(lambda ()
         (auto-complete-mode nil)
         (outline-minor-mode 1)
         (setq outline-regexp "\\(^#\\{4,6\\}\\)\\|\\(^[a-zA-Z0-9_\.]+ ?<- ?function(.*{\\)")
         (setq outline-heading-alist
           '(("####" . 1) ("#####" . 2) ("######" . 3)
             ("^[a-zA-Z0-9_\.]+ ?<- ?function(.*{" .4)))))

All this code has not been very well tested, and I'm far from an expert in emacs lisp, so there should be better ways to do it, and don't be surprised in case of bugs !

Coextend

It has taken me a while to actually try your suggestion. But I recently did and the font locking is working. But I use a dark theme ('zenburn') for emacs, and the font locked headlines have a white background now, which is a little hard on the eyes. From looking at your screenshot it seems as if you have fixed this problem somehow. – Cowbird

I managed to change the hideface-background to the color matching my theme. Now I want to edit the Keywords #### to ### *, ##### to ### ** and ###### to ### ***, so that I can used orgstruct-mode easily. Any help with the regular expressions? I tried something along these lines: '(("^###\( *\)\([^#].*\)$" (1 'ess-jb-hide-face t)(2 'ess-jb-h1-face t)) – Cowbird

@Cowbird Maybe something like this ? (not tested) : "^### \\(\\*\\)\\([^*].*\\)$" – Coextend

@Coextend I have been using your above code for a while to organize comments. I just setup a new computer with emacs ess and the am using the same init.el file as my old computer. The comment organization, however, is not working. What am I missing? Do i need to install a package or something? thanks – Tasker

Answer 3

Sounds like you have one big analysis script. That's not very optimal...

Think about redoing it with RMarkdown or Sweave, and using knitr to run it. Then you can use emacs' tools for navigating markdown or TeX and you can use the caching system in knitr to save having to do analyses right from the start when you've changed something near the end.

You also get a formatted report out of the analysis, if you want.

Also, breaking stuff out into separate files for sourcing is a bit sub-optimal - far far better to write functions and put them in a package - and to use the devtools package to make working with it really simple. Just edit your .R files and load_all updates it (none of the complex package build stuff of times past).

Big winnage.

Envoy

One problem with using Rmarkdown is that it is not (yet) really supported in Emacs/ESS. – Depressor

Answer 4

polymode provides R+markdown, R+brew, R+cpp and other stuff. It is still in early stage of development and seems to work reliably only on very recent emacs (24.3).

Sigismond

Answer 5

Similarly to the orgstruct-mode mentioned above you can also use the new outshine package which works together with outline-minor-mode to interpret comments with asterisks as org-mode-like headings. They can be folded/expanded/added/etc using simple shortcuts like TAB, etc.

Consonant

Answer 6

Another option, which I've been happy with over the years is allout-mode. Using per-file local variables:

## Local variables:
## allout-layout: (-1 : 0)
## End:

Attenuator