Have the same README both in Markdown and reStructuredText

Question

I have a project hosted on GitHub. For this I have written my README using the Markdown syntax in order to have it nicely formatted on GitHub.

As my project is in Python I also plan to upload it to PyPi. The syntax used for READMEs on PyPi is reStructuredText.

I would like to avoid having to handle two READMEs containing roughly the same content; so I searched for a markdown to RST (or the other way around) translator, but couldn't find any.

The other solution I see is to perform a markdown/HTML and then a HTML/RST translation. I found some ressources for this here and here so I guess it should be possible.

Would you have any idea that could fit better with what I want to do?

Copro

Github will render README.rst! – Micky

This is new then :) But good to know, i'll try ! – Copro

If you would like PyPI to support readmes in Markdown, please comment on the feature request at bitbucket.org/pypa/pypi/issue/148/support-markdown-for-readmes – Flemings

Answer 1

I would recommend Pandoc, the "swiss-army knife for converting files from one markup format into another" (check out the diagram of supported conversions at the bottom of the page, it is quite impressive). Pandoc allows markdown to reStructuredText translation directly. There is also an online editor here which lets you try it out, so you could simply use the online editor to convert your README files.

Sesqui

The magical invocation is: pandoc --from=markdown --to=rst --output=README.rst README.md – Unroll

Answer 2

As @Chris suggested, you can use Pandoc to convert Markdown to RST. This can be simply automated using pypandoc module and some magic in setup.py:

from setuptools import setup
try:
    from pypandoc import convert
    read_md = lambda f: convert(f, 'rst')
except ImportError:
    print("warning: pypandoc module not found, could not convert Markdown to RST")
    read_md = lambda f: open(f, 'r').read()

setup(
    # name, version, ...
    long_description=read_md('README.md'),
    install_requires=[]
)

This will automatically convert README.md to RST for the long description using on PyPi. When pypandoc is not available, then it just reads README.md without the conversion – to not force others to install pypandoc when they wanna just build the module, not upload to PyPi.

So you can write in Markdown as usual and don’t care about RST mess anymore.

Absolutism

This doesn't really solve the problem, since if the user doesn't have pypandoc installed (which they likely won't), it'll throw an error, as PyPI expects the long_description field to be RST. If pypandoc is not available, you should set long_description to None or an empty string. – Gemmulation

Nope, it’s needed only when uploading the metadata to PyPi (which is doing only developer of the module, not users). It doesn’t throw any error when user install the module and doesn’t have pypandoc installed. I’ve verified this use case. – Absolutism

This can also throw a runtime error. In order to stay on a safe side I recommend to do try-except in the function. – Atomism

Perfect! Just one thing - I was getting RuntimeError: Missing format! exception until I changed the lambda to read_md = lambda f: convert(f, 'rst', 'md'). Reason being (I'm guessing) that I fed it a string and not a file (so no file extension). – Approximation

@Approximation Your guess is correct. Pandoc is able to autodetect source format from a file extension, but when you fed it a string, you must specify the format explicitly. – Absolutism

Answer 3

2019 Update

The PyPI Warehouse now supports rendering Markdown as well! You just need to update your package configuration and add the long_description_content_type='text/markdown' to it. e.g.:

setup(
    name='an_example_package',
    # other arguments omitted
    long_description=long_description,
    long_description_content_type='text/markdown'
)

Therefore, there is no need to keep the README in two formats any longer.

You can find more information about it in the documentation.

Old answer:

The Markup library used by GitHub supports reStructuredText. This means you can write a README.rst file.

They even support syntax specific color highlighting using the code and code-block directives (Example)

Greatest

Answer 4

PyPI now supports Markdown for long descriptions!

In setup.py, set long_description to a Markdown string, add long_description_content_type="text/markdown" and make sure you're using recent tooling (setuptools 38.6.0+, twine 1.11+).

See Dustin Ingram's blog post for more details.

Abolition

Nice to hear! It's interesting to see how progress is made over time in the python community looking at the history of this issue :). – Copro

Answer 5

You might also be interested in the fact that it is possible to write in a common subset so that your document comes out the same way when rendered as markdown or rendered as reStructuredText: https://gist.github.com/dupuy/1855764 ☺

Defrayal

Answer 6

For my requirements I didn't want to install Pandoc in my computer. I used docverter. Docverter is a document conversion server with an HTTP interface using Pandoc for this.

import requests
r = requests.post(url='http://c.docverter.com/convert',
                  data={'to':'rst','from':'markdown'},
                  files={'input_files[]':open('README.md','rb')})
if r.ok:
    print r.content

Teetotaler

Answer 7

I ran into this problem and solved it with the two following bash scripts.

Note that I have LaTeX bundled into my Markdown.

#!/usr/bin/env bash

if [ $# -lt 1 ]; then
  echo "$0 file.md"
  exit;
fi

filename=$(basename "$1")
extension="${filename##*.}"
filename="${filename%.*}"

if [ "$extension" = "md" ]; then
  rst=".rst"
  pandoc $1 -o $filename$rst
fi

Its also useful to convert to html. md2html:

#!/usr/bin/env bash

if [ $# -lt 1 ]; then
  echo "$0 file.md <style.css>"
  exit;
fi

filename=$(basename "$1")
extension="${filename##*.}"
filename="${filename%.*}"

if [ "$extension" = "md" ]; then
  html=".html"
  if [ -z $2 ]; then
    # if no css
    pandoc -s -S --mathjax --highlight-style pygments $1 -o $filename$html
  else
    pandoc -s -S --mathjax --highlight-style pygments -c $2 $1 -o $filename$html
  fi
fi

I hope that helps

Tarrsus

Answer 8

Using the pandoc tool suggested by others I created a md2rst utility to create the rst files. Even though this solution means you have both an md and an rst it seemed to be the least invasive and would allow for whatever future markdown support is added. I prefer it over altering setup.py and maybe you would as well:

#!/usr/bin/env python

'''
Recursively and destructively creates a .rst file for all Markdown
files in the target directory and below.

Created to deal with PyPa without changing anything in setup based on
the idea that getting proper Markdown support later is worth waiting
for rather than forcing a pandoc dependency in sample packages and such.

Vote for
(https://bitbucket.org/pypa/pypi/issue/148/support-markdown-for-readmes)

'''

import sys, os, re

markdown_sufs = ('.md','.markdown','.mkd')
markdown_regx = '\.(md|markdown|mkd)$'

target = '.'
if len(sys.argv) >= 2: target = sys.argv[1]

md_files = []
for root, dirnames, filenames in os.walk(target):
    for name in filenames:
        if name.endswith(markdown_sufs):
            md_files.append(os.path.join(root, name))

for md in md_files:
    bare = re.sub(markdown_regx,'',md)
    cmd='pandoc --from=markdown --to=rst "{}" -o "{}.rst"'
    print(cmd.format(md,bare))
    os.system(cmd.format(md,bare))

Kinesics