How to make PyPi description Markdown work?

Question

I uploaded a package to PyPi using:

python setup.py register -r pypi
python setup.py sdist upload -r pypi

I'm trying to modify the description, I wrote (please don't edit the formatting of the following piece of code, I made it on purpose to demonstrate my problem):

**my plugin**

This plugin enables you to ... For example:
```python
@attr(section='MySection', id=1)
def test_function(self):
    """
    Bla bla bla
    """
    pass
```

However, the text appears as it is, without the markdown formatting. What am I doing wrong?

Parrie

Answer 1

As of March 16, 2018, PyPI.org aka Warehouse (finally) supports Markdown in long descriptions. Warehouse replaced the old legacy PyPI implementation in April 2018.

You need to:

• Make sure setuptools is upgraded to version 38.6.0 or newer

• Make sure twine is upgraded to version 1.11.0 or newer

• Make sure wheel is upgraded to version 0.31.0 or newer

• Add a new field named long_description_content_type to your setup() call, and set it to 'text/markdown':

  setup(
      long_description="""# Markdown supported!\n\n* Cheer\n* Celebrate\n""",
      long_description_content_type='text/markdown',
      # ....
  )
  

  See PEP 566 - Metadata for Python Software Packages 2.1.

• Use twine to upload your distributions to PyPI:

  $ python setup.py sdist bdist_wheel   # adjust as needed
  $ twine upload dist/*
  

The old legacy PyPI infrastructure would not render Markdown, only the new Warehouse infrastructure does. The legacy infrastructure is now gone (as of 2018-04-30).

Currently, PyPI uses cmarkgfm as the markdown renderer, via the readme_renderer library (using readme_renderer.markdown.render(long_description) to produce HTML output). This means that your markdown documents will render exactly the same as on GitHub; it is essentially the same renderer.

You can validate your package long_description with the twine check command (twine 1.12.0 or newer).

The old < 2018-03-16 answer follows below.

---

Note: this is the old, now outdated answer, as of 2018-03-16 Markdown is supported provided you use the right tools, see above.

PyPI does not support Markdown, so your README will not be rendered into HTML.

If you want a rendered README, stick with reStructuredText; the Sphinx introduction to reStructuredText is a good starting point.

You probably want to install the docutils package so you can test your document locally; you want to run the included rst2html.py script on your README to see what errors are produced, if any. Your specific sample has too many errors:

$ bin/rst2html.py test.rst  > /tmp/test.html
test.rst:7: (ERROR/3) Unexpected indentation.
test.rst:3: (WARNING/2) Inline literal start-string without end-string.
test.rst:3: (WARNING/2) Inline interpreted text or phrase reference start-string without end-string.
test.rst:11: (WARNING/2) Block quote ends without a blank line; unexpected unindent.
test.rst:11: (WARNING/2) Inline literal start-string without end-string.
test.rst:11: (WARNING/2) Inline interpreted text or phrase reference start-string without end-string.

Your code block is using Github's Markdown extensions, which are entirely wrong for reStructuredText. You could use a reST code block (probably, if the PyPI version of docutils is new enough):

.. code-block:: python

    @attr(section='MySection', type='functional+', module='MyModule', id=1)
    def test_function(self):
        """
        This is the original docstring
        """
        pass

To test this locally you'll need to install Pygments as well.

There is a feature request with pull request to add support for Markdown, if you are interested.

Agnosia

Answer 2

As stated in Martijn Pieters' answer, PyPi does not support Markdown.

Instead, convert the Markdown files into RestructuredText and upload those to PyPi.

One way to do this is to install Pandoc and PyPandoc, and then arrange to run them during packaging.

On OS X, Pandoc can be installed using Homebrew (brew install pandoc). Use pip (pip install pypandoc) to install PyPandoc.

Then, modify setup.py like so:

try:
    import pypandoc
    long_description = pypandoc.convert_file('README.md', 'rst')
except(IOError, ImportError):
    long_description = open('README.md').read()

setup(
    name='blah',
    version=find_version('blah.py'),
    description='Short description',
    long_description=long_description,
)

Illusionism

Side question: Why also catch IOError? Is ImportError not the only expected exception in this case? – Rickey

@NickChammas, I might be wrong but the ImportError might catch an exception when pypandoc can't be imported and the IOError might catch the possible case where README.md can't be opened or README.rst can't be written. – Megdal

Thanks for this. One note: you need to include README.md as a data file in setup.py (data_files=[('', ['README.md'])]) otherwise you get an RuntimeError from pypandoc when installing with pip. Alternatively, the exception catch should be more general and feed long_description with a basic version instead. – Pilatus

OSError is raised when pandoc is not installed. I use except (IOError, ImportError, OSError): – Nurture

pypandoc.convert has been depricated. Use pypandoc.convert_file – Orthopter

This answer needs an update: PyPi supports markdown nowadays. – Spode

Answer 3

PyPI supports rst and not markdown as mentioned on other answers. But you don't need pypandoc perse, just pandoc is fine. You can generate the rst file locally first and then run setup.py to upload the package.

upload.sh:

#!/bin/bash
pandoc --from=markdown --to=rst --output=README README.md
python setup.py sdist upload

The generated file named README will be automatically recognized. Be sure to add it to your .gitignore! The setup.py doesn't have to do anything special.

setup.py:

from distutils.core import setup

setup(
    name='mypackage',
    packages=['mypackage'],  # this must be the same as the name above
    version='0.2.8',
    description='short',
    author='Chiel ten Brinke',
    author_email='<email>',
    url='<github url>',  # use the URL to the github repo
    keywords=[],  # arbitrary keywords
    classifiers=[],
)

Then just run bash upload.sh to upload the stuff to PyPI.

Kriskrischer

In my case the README file was not automatically recognized as long_description. One has to add the long_description field to setup.py . Otherwise the README file will not be rendered on pypi. – Priscillaprise

Answer 4

You can set markdown in setup.cfg file too:

[metadata]
...
long_description = file: README.md
long_description_content_type = text/markdown

Checkout my project as an example: on github and on pypi.

Offal

Although setup.py can technically be cleverly jury-rigged to (A) safely open README.md with a context manager, (B) read the contents of that file into a local variable, and (C) set the long_description option to that variable's value outside of that context, no one should actually do that. setup.cfg already implements this correctly, safely, and efficiently on your behalf. Don't reinvent the rusty wheel. Just use setup.cfg for it is good. (This has been a message from your friendly neighbourhood packager.) – Nonproductive

Note: this answer previously mixed the nonstandard description-file key specific to pbr (Python Build Reasonableness) with the standard long-description-content-type key not specific to pbr. I've since edited that erroneous discrepancy away by preferring the standard long_description and long_description_content_type keys. This answer now behaves as expected for everyone – regardless of whether you're using pbr or not. All is now right with the packaging world. – Nonproductive

Works like a charm! Thanks for sharing this. – Astylar

Answer 5

I've had issues with \r characters causing parsing issues where only the first line of the README appears in pypi. The code below fixes the issue, it comes from the pypandoc module repository:

try:
    long_description = pypandoc.convert('README.md', 'rst')
    long_description = long_description.replace("\r","") # Do not forget this line
except OSError:
    print("Pandoc not found. Long_description conversion failure.")
    import io
    # pandoc is not installed, fallback to using raw contents
    with io.open('README.md', encoding="utf-8") as f:
        long_description = f.read()

This way long_description contains a sanitized version of your Readme and you can pass it to the setup() function in your setup.py script.

Nemhauser

Got downvoted, haven't tried this in a while so if this does not work please report it – Nemhauser

Worked for me, thank you so much! I've spent hours troubleshooting. ... Why io.open instead of just open? – Yahweh

Downvote may have been because convert_text requires format=md argument while convert_file does not. I'm not sure what mechanism bare convert uses. Maybe particular situations also require the argument. c.f. github.com/bebraw/pypandoc/blob/master/README.md#usage – Yahweh

Those two functions must be new additions to pypandoc because they did not exist before. Are you saying this code using bare convert does not work ? – Nemhauser

Bare convert didn't work for me on one machine, but did on another. I didn't investigate to see what was different between the two. – Yahweh

Answer 6

I found the other answers here sort of complicated.

To get my github page readme.md file to show on my PyPi page, I followed the following two steps in my setup.py file.

1. Include the following at the top of my setup.py file

from pathlib import Path
this_directory = Path(__file__).parent
long_description = (this_directory / "readme.md").read_text()

These steps directly read in the data from your markdown file into the long_description variable.

2. Input the following parameters to the setup() function

long_description = long_description
long_description_content_type="text/markdown"

This inserts the data from your markdown file into long_description, and then tells setup() what type of data it is (markdown).

As I said, the above two steps worked great on my first try, without any hassles or complexity. Perfect for my little brain. :)

Note this code is adapted from:
https://packaging.python.org/guides/making-a-pypi-friendly-readme/

Witticism

Answer 7

There is a good pip package that worked for me

https://pypi.python.org/pypi/restructuredtext_lint/

I'm using it on my setup now:

https://github.com/pablodav/burp_server_reports/blob/master/setup.py

def check_readme(file='README.rst'):
"""
Checks readme rst file, to ensure it will upload to pypi and be formatted correctly.
:param file:
:return:
"""
errors = rst_lint.lint_file(file)
if errors:
    msg = 'There are errors in {}, errors \n {}'.format(file, errors[0].message)
    raise SystemExit(msg)
else:
    msg = 'No errors in {}'.format(file)
print(msg)

Also I have created a lib to be able to use in py.test later

https://github.com/pablodav/burp_server_reports/blob/master/burp_reports/lib/check_readme.py

Photoelasticity

Answer 8

2023 Update:

No need for a setup.py file. Under your poetry.toml file or simply your .toml file, there is a [tool.poetry] section, under that you put readme = "README.md". Example:

[tool.poetry]
name = "Example Module"
version = "0.1.0"
description = "Module Description"
readme = "README.md"
authors = ["Author Name <[email protected]>"]

Gershwin