How to pre-process source files while a Sphinx run?

Question

I have set up a Sphinx documentation for my project and would like to extract doc strings for the source files and embed them into the final documentation. Unfortunately, the source file's language (VHDL) is not supported by Sphinx. There seems to be no Sphinx domain for VHDL.

So my ideas is as follows:

• Hook into the Sphinx run and execute some Python code before Sphinx
• The Python codes extracts text blocks from each source file (the top-most multi-line comment block) and assembles one reST file per source file, consisting of this comment block and some other reST markup.
• All source files are listed in an index.rst, to generate the apropriate .. toctree:: directive.
• The text extraction and transformation is done recursively per source code directory.

So the main question is: How to hook into Spinx?

Or should I just import and run my own configuration in conf.py?

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#
from my_preprocessor import my_proc
proc = my_proc()
proc.run()
#
# Test documentation build configuration file, created by
# sphinx-quickstart on Tue May 24 11:28:20 2016.
# ....

I can't modify the build process files: Makefile and make.bat, because the real build process runs on ReadTheDocs.org. RTDs only executes conf.py.

Rear

Any reason you can't create an extension and add it to your project? – Charily

It's very complex to write an extension (parser, object model adapter) for my target language ... pre-processing might be easier. – Rear

Answer 1

As noted in my previous comments and mertyildiran's answer, the official way to hook into Sphinx for a language would be to create an extension to implement a new domain for VHDL.

This has already been done for many other languages - e.g. Erlang, PHP, CoffeeScript - and APIs - e.g. HTTP REST - just to name a few from sphinx-contrib. However, that is going to take a lot of time, which you don't have... You are therefore left with an option to do some quick parsing yourself and then hooking that into your Sphinx build somehow.

Since you are bypassing the official hooks, this question becomes "how do I run my own code inside a Sphinx build?" For which, I would recommend that you simply follow the guidance for a local extension - i.e. put it in a separate directory, add that to your path, then import and invoke it. As noted in the docs:

The configuration file is executed as Python code at build time (using execfile(), and with the current directory set to its containing directory), and therefore can execute arbitrarily complex code. Sphinx then reads simple names from the file’s namespace as its configuration.

As a final though, this opens up the options to use 3rd party packages like pyVhdl2Sch (with a nod again to mertyildiran's answer) to create some schematic and then maybe write your static rst files around it to explain the schematic.

Charily

As an update: Some time has passed since my question was posted here. I published a first example of my VHDL parser on GitHub. Is there a way to get community support from the Sphinx Contrib people to create the adapter? – Rear

I think your best bet is to check out the developer guide. There is a mailing list and an iRC channel for starters... – Charily

Doxygen can parse VHDL, so you could try using Breathe to ingest that into Sphinx. My company uses this exact flow but with C/C++ sources. – Compressor

Answer 2

You are trying to use a sledgehammer to crack a nut.

Sphinx was originally created for the new Python documentation, and it has excellent facilities for the documentation of Python projects, but C/C++ is already supported as well, and it is planned to add special support for other languages as well. http://www.sphinx-doc.org/en/stable/

VHDL is currently not a supported language by Sphinx and because VHDL is a hardware description language its priority for becoming a supported language must be low. You have two options and first one is also my suggestion to you:

1) Use a VHDL specific documentation generator tool instead of Sphinx

VHDocL - http://www.volkerschatz.com/hardware/vhdocl.html

A VHDL documentation utility written in Perl, based on Doxygen.

pyVhdl2Sch - http://laurentcabaret.github.io/pyVhdl2Sch/

pyVhdl2Sch is a documentation generator tool. It takes VHDL files (.vhd) as entry and generates a pdf/svg/ps/png schematic for each input file. Written in pure Python, more community friendly, more up to date.

Sigasi Studio XL Doc - http://www.sigasi.com/products/

High end edition of Sigasi Studio which is a commercial product.

2) Contribute to Sphinx project and add VHDL domain

Follow Sphinx Developer's Guide and become familiar with the project structure. Eventually add vhdl.py to this project directory: https://github.com/sphinx-doc/sphinx/tree/master/sphinx/domains

This second option cannot be explained with a StackOverflow answer. It's up to you if you want to add more functionality to an open source project like Sphinx.

Doane

Non of the mentioned tools work with ReadTheDocs so these are no option for me. – Rear

Perhaps a better analogy would be using a keyboard to crack a paebbel? :) – Cristinecristiona