Use Sphinx autosummary recursively to generate API documentation

Question

I want to use Sphinx's autosummary extension and templates to generate API docs recursively from docstrings. I want separate pages for each module, class, method, property and function. But it doesn't detect my templates at all. In fact, if I just remove the module.rst file from _templates/autosummary/, it renders the whole file exactly the same way as before. I've followed this SO question to the letter. If you're interested, the full repository is on GitHub.

Edit: It seems it does generate a different file, I had to delete docs/_autosummary for it to read the new template. However, now it generates a file with the sparse header and description header. It doesn't go into the {% if classes %} and {% if functions %} directives.

My directory structure is as follows:

• sparse
• docs
  • conf.py
  • index.rst
  • modules.rst
  • _templates/autosummary/module.rst

Here are the relevant files so far:

index.rst:

.. sparse documentation master file, created by
   sphinx-quickstart on Fri Dec 29 20:58:03 2017.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to sparse's documentation!
==================================

.. toctree::
   :maxdepth: 2
   :caption: Contents:

   modules

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

modules.rst:

API Reference
=============

Modules
-------

.. autosummary::
   :toctree: _autosummary

   sparse

_templates/autosummary/module.rst:

{{ fullname | escape | underline }}

Description
-----------

.. automodule:: {{ fullname | escape }}

{% if classes %}
Classes
-------
.. autosummary:
    :toctree: _autosummary

    {% for class in classes %}
        {{ class }}
    {% endfor %}

{% endif %}

{% if functions %}
Functions
---------
.. autosummary:
    :toctree: _autosummary

    {% for function in functions %}
        {{ function }}
    {% endfor %}

{% endif %}

Kessiah

Why don't you use sphinx-apidoc? – Efrainefram

If sphinx-apidoc can do something similar to what I want (with templates and member summaries), then I shall. Unfortunately I'm not too good with Sphinx and an example would be useful. But as I understand it, sphinx-apidoc is equivalent to using autosummary with autosummary_generate = True. – Kessiah

Answer 1

From Sphinx version 3.1 (June 2020), you can use the new :recursive: option to get sphinx.ext.autosummary to automatically detect every module in your package, however deeply nested, and automatically generate documentation for every attribute, class, function and exception in that module.

See my answer here: https://mcmap.net/q/117428/-automatically-document-all-modules-recursively-with-sphinx-autodoc

Winglet

Answer 2

I ended up needing the following files:

modules.rst:

API Reference
=============

.. rubric:: Modules

.. autosummary::
   :toctree: generated

   sparse

_templates/autosummary/module.rst:

{{ fullname | escape | underline }}

.. rubric:: Description

.. automodule:: {{ fullname }}

.. currentmodule:: {{ fullname }}

{% if classes %}
.. rubric:: Classes

.. autosummary::
    :toctree: .
    {% for class in classes %}
    {{ class }}
    {% endfor %}

{% endif %}

{% if functions %}
.. rubric:: Functions

.. autosummary::
    :toctree: .
    {% for function in functions %}
    {{ function }}
    {% endfor %}

{% endif %}

_templates/autosummary/class.rst:

{{ fullname | escape | underline}}

.. currentmodule:: {{ module }}

.. autoclass:: {{ objname }}

   {% block methods %}
   {% block attributes %}
   {% if attributes %}
   .. HACK -- the point here is that we don't want this to appear in the output, but the autosummary should still generate the pages.
      .. autosummary::
         :toctree:
      {% for item in all_attributes %}
         {%- if not item.startswith('_') %}
         {{ name }}.{{ item }}
         {%- endif -%}
      {%- endfor %}
   {% endif %}
   {% endblock %}

   {% if methods %}
   .. HACK -- the point here is that we don't want this to appear in the output, but the autosummary should still generate the pages.
      .. autosummary::
         :toctree:
      {% for item in all_methods %}
         {%- if not item.startswith('_') or item in ['__call__'] %}
         {{ name }}.{{ item }}
         {%- endif -%}
      {%- endfor %}
   {% endif %}
   {% endblock %}

_templates/autosummary/base.rst:

{{ fullname | escape | underline}}

.. currentmodule:: {{ module }}

.. auto{{ objtype }}:: {{ objname }}

I also needed to go to sphinx/ext/autosummary/generate.py and set imported_members=True in the function generate_autosummary_docs.

If you're not using numpydoc like me, you might need to remove the .. HACK directives.

Kessiah

I cloned your GitHub project and it worked for me too. Setting imported_members=True in the generate_autosummary_docs function seems to be the key. But it is problematic that it should be necessary to edit the Sphinx source code to make it work. – Dunsany

I just saw github.com/sphinx-doc/sphinx/issues/4372. You have __all__ = ["COO", "tensordot", "concatenate", "stack", "dot", "triu", "tril"] in __init__.py and that should be enough IMHO. As far as I can tell, autosummary does not take __all__ into account. – Dunsany

Yes, it should either take __all__ into account or provide a way to include imported members. – Kessiah

Great answer, very helpful! Are you aware of a way to add the documented module attributes (like constant definitions) to the output in modules.rst? – Jinx

@Jinx You would add an attributes section similar to the functions and classes section in _templates/autosummary/module.rst. I haven't tried it myself, but I'm pretty sure it will work. – Kessiah

@HameerAbbasi attributes seems to be empty, but members has content. I can filter out like if item not in all_functions and item not in all_classes ... and get mostly what I want (except for imported module names and symbols, which I did not figure out how to exclude) – Jinx

Answer 3

For those looking for a simple direct solution who do not necessarily require the use of 'autosummary'.

Check out the sphinx-autoapi extension. Which generates code based documentation for all modules/classes/functions incl. methods and attributes without the need for much configuration.

Different