How to handle citations in Ipython Notebook?

Question

What is the best way to take care of citations in Ipython Notebook? Ideally, I would like to have a bibtex file, and then, as in latex, have a list of shorthands in Ipython markdown cells, with the full references at the end of the notebook.

The relevant material I found is this: http://nbviewer.ipython.org/github/ipython/nbconvert-examples/blob/master/citations/Tutorial.ipynb

But I couldn't follow the documentation very well. Can anyone explain it? Thanks so much!!

Enrapture

What part of the documentation is tripping you up? – Alex

I'm working on a tool to interactively add citations to notebooks, but it's a bit rough so far: github.com/takluyver/cite2c – Illogical

Answer 1

Summary

This solution is largely based on Sylvain Deville's excellent blog post. It allows you to simply write [@citation_key] in markdown cells. The references will be formatted after document conversion. The only requirements are LaTeX and pandoc, which are both widely supported. While there is never a guarantee, this approach should therefore still work in many years time.

Step-by-Step Guide

In addition to a working installation of jupyter you need:

1. LaTeX (installation guide).

2. Pandoc (installation guide).

3. A citation style language. Download a citation style, e.g., APA. Save the .csl file (e.g., apa.csl) into the same folder as your jupyter notebook (or specify the path to the .csl file later).

4. A .bib file with your references. I am using a sample bib file list.bib. Save to the same folder as your jupyter notebook (or specify the path to the .bib file later).

Once you completed these steps, the rest is easy:

1. Use markdown syntax for references in markdown cells in your jupyter notebook. E.g., [@Sh:1] where the syntax works like this: ([@citationkey_in_bib_file]). I much prefer this syntax over other solutions because it is so fast to type [@something].

2. At the end of your ipython notebook, create a code cell with the following syntax to automatically convert your document (note that this is R code, use an equivalent command to system() for python):

    #automatic document conversion to markdown and then to word
    #first convert the ipython notebook paper.ipynb to markdown
    system("jupyter nbconvert --to markdown paper.ipynb")
    #next convert markdown to ms word
    conversion <- paste0("pandoc -s paper.md -t docx -o paper.docx",
                   " --filter pandoc-citeproc",
                   " --bibliography="listb.bib",
                   " --csl="apa.csl")
    system(conversion)
   

   Run this cell (or simply run all cells). Note that the 2nd system call is simply pandoc -s paper.md -t docx -o paper.docx --filter pandoc-citeproc --bibliography=listb.bib --csl=apa.csl. I merely used paste0() to be able to spread this over multiple lines and make it nicer to read.

   The output is a word document. If you prefer another document, check out this guide for alternative syntax.

---

#Extras

1. If you do not like that your converted document includes the syntax for the document conversion, insert a markdown cell above and below the code cell with the syntax for the conversion. In the cell above, enter <!-- and in the cell below enter -->. This is a regular HTML command for a comment, so the syntax will in between these two cells will be evaluated but not printed.

2. You can also include a yaml header in your first cell. E.g.,

    ---
    title: This is a great title. 
    author: Author Name
    abstract: This is a great abstract
    ---
   

Wormy

While converting the .ipynb to .md with pandoc the reference is converted to \[@something\], which doesn't give the desired output when compiling to for example a pdf file. Is this explained by using pandoc instead of nbconvert? – Hollyanne

@Wormy Your code worked for me after tweaking it for Python. Among other small changes, I installed pandoc using brew, I had to replace " --filter pandoc-citeproc" with " --citeproc". Thanks for the explanation of why there was a copy0() function in the code at first. Thanks for sharing! – Paucity

Answer 2

You can use the Document Tools of the Calico suite, which can be installed separately with:

sudo ipython install-nbextension https://bitbucket.org/ipre/calico/downloads/calico-document-tools-1.0.zip

Read the tutorial and watch the YouTube video for more details.

---

Warning: only the cited references are processed. Therefore, if you fail to cite an article, it won't appear in the References section. As a little working example, copy the following in a Markdown cell and press the "book" icon.

<!--bibtex

@Article{PER-GRA:2007,
  Author    = {P\'erez, Fernando and Granger, Brian E.},
  Title     = {{IP}ython: a System for Interactive Scientific Computing},
  Journal   = {Computing in Science and Engineering},
  Volume    = {9},
  Number    = {3},
  Pages     = {21--29},
  month     = may,
  year      = 2007,
  url       = "http://ipython.org",
  ISSN      = "1521-9615",
  doi       = {10.1109/MCSE.2007.53},
  publisher = {IEEE Computer Society},
}

@article{Papa2007,
  author = {Papa, David A. and Markov, Igor L.},
  journal = {Approximation algorithms and metaheuristics},
  pages = {1--38},
  title = {{Hypergraph partitioning and clustering}},
  url = {http://www.podload.org/pubs/book/part\_survey.pdf},
  year = {2007}
}

-->

Examples of citations: [CITE](#cite-PER-GRA:2007) or [CITE](#cite-Papa2007).

This should result in the following added Markdown cell:

References

^{^} Pérez, Fernando and Granger, Brian E.. 2007. IPython: a System for Interactive Scientific Computing. URL

^{^} Papa, David A. and Markov, Igor L.. 2007. Hypergraph partitioning and clustering. URL

Alburg

This looks very useful! But unfortunately under the References session, journals' names do not show up at all... Do you have any suggestions on this? Thanks so much! – Enrapture

@user3698176. Indeed. I have created an issue: bitbucket.org/ipre/calico/issue/218/… – Alburg

The link to the Calico suite is also dead now. Is the project dead too? – Smalt

According to bitbucket.org/ipre/calico/issues/221/… this project is no longer in active development. What's more, the devs have not seen fit to update their own README with a corrected URL. But the documentation is now here: wiki.roboteducation.org/Calico. – Smalt

The code now lives here: github.com/Calysto/notebook-extensions – Renatorenaud

Answer 3

I was able to run it with the following approach:

• Insert the html citation as in the tutorial you mentioned.
• Create ipython.bib in the "standard" bibtex format. It goes into the same file as your *.ipynb notebook file.
• Create the template file as in the tutorial, also in the same directory or else in the (distribution dependent) directory with the other templates. On my system, that's /usr/local/lib/python2.7/dist-packages/IPython/nbconvert/templates/latex.
• The tutorial has the template extend latex_article.tplx. On my distribution, it's article.tplx (without latex_).
• Run nbconvert with --to latex; that generates an .aux file among other things. Latex will complain about missing references.
• Run bibtex yournotebook.aux; this generates yournotebook.bbl. You only need to re-run this if you change references.
• Re-run nbconvert either with --to latex or with --to pdf. This generates a .tex file, or else runs all the way to a .pdf.

If you want html output, you can use pandoc to assemble the references into a tidy citation page. This may require some hand-editing to make an html page you can reference from your main document.

Firsthand

I get an error TemplateNotFound: article.tlpx, when doing nbconvert – Cora

@Vass: There have been some version changes since I wrote the answer. Which version of IPython (or jupyter) are you using? What OS are you using? – Firsthand

@Vass: I'm running Ubuntu 14.04, and ipython version 4.0.0, installed with pip. My article.tplx is in /usr/local/lib/python2.7/dist-packages/nbconvert/templates/latex/article.tplx. Any chance you mis-spelled article.tlpx rather than article.tplx in the template? – Firsthand

On my Ubuntu Focal there is no *.aux file, how can I make it? ipython3 is of version 7.13.0 and jupyter-nbconvert is of version 5.6.1. – Distributee

Answer 4

If you know that you will be converting your notebook to latex anyway, consider simply adding a "Raw" cell (Ctrl+M R) to the end of the document, containing the bibliography just as you would put it in pure LaTeX.

For example, when I need to reference a couple of external links, I would not even care to do a proper BibTeX thing and simply have a "Raw" cell at the end of the notebook like that:

\begin{thebibliography}{1}
\bibitem{post1} 
Holography in Simple Terms. K.Tretyakov (blog post), 2015.\\
\url{http://fouryears.eu/2015/07/24/holography-in-simple-terms/}

\bibtem{book1}
The Importance of Citations. J. Smith. 2010.
\end{thebibliography}

The items can be cited in other Markdown cells using the usual <cite data-cite="post1">(KT, 2015)</cite>

Of course, you can also use proper BibTeX as well. Just add the corresponding Raw cell, e.g:

\bibliographystyle{unsrt}
\bibliography{papers}

This way you do not have to bother editing a separate template file (at the price of cluttering the notebook's HTML export with raw Latex, though).

Arguello

But the references don't appear in the jupyter file. So how do I get that ? – Candiecandied

Answer 5

You should have a look at the latex_envs extension in https://github.com/ipython-contrib/IPython-notebook-extensions (install from this repo, it is the most recent version). This extension contains a way to integrate bibliography using bibtex files and standard latex notation, and generates a bibliography section at the end of the notebook. Style of citations can be (to some extent) customized. Some documentation here https://rawgit.com/jfbercher/latex_envs/master/doc/latex_env_doc.html

Baguio

Answer 6

Here is the Python version of Flo's answer:

# automatic document conversion to markdown and then to word
# first convert the ipython notebook paper.ipynb to markdown

import os

os.system("jupyter nbconvert --to markdown paper.ipynb")

# next convert markdown to MS Word
md_to_word = '''pandoc -s paper.md -t docx -o paper.docx --citeproc --bibliography="references.bib" --csl="frontiers.csl" '''

os.system(md_to_word)

Check that the .csl file is xml and not html.

Justify