[Updated: December16, 2022]

As developers, we all know the importance of code documenting: good code is self-explanatory and well-documented. However, we also struggle with keeping documents up to date, especially if we maintain the source code and its documents separately. If we can generate the document based on the source code or based on the code comments, we have a better chance of keeping the document up to date.

Sphinx is a tool that provides the ability to build documents from the code. It supports many programming languages and is widely used in Python projects, including the official Python website. Although Sphinx’s official website provides a lot of useful information, it may be too overwhelming for those who use Sphinx for the first time. At least, that is my experience when I tried to use Sphinx the first time. Therefore, I hope this article can provide a simple and straightforward tutorial for a newbie of Sphinx.

Plot

This tutorial uses a simple Python project (Sample Project) to demonstrate using Sphinx to generate HTML-based documents. The Sample Project is a collection of some types of binary search trees and binary tree traversal. It is well documented by following NumPy docstring style. The Sample Project’s primary purpose is to be a sample code for this Sphinx tutorial using NumPy-style docstrings.

The Sample Project can be downloaded from this GitHub repository.

$ git clone https://github.com/shunsvineyard/python-sample-code.git

The generated documents look like the picture below. It is also available at Read the Docs.

Assumptions and Requirements

This tutorial assumes the following environment:

• Python 3.9
• Sphinx 3.4.3
• Ubuntu 20.04

Note: Sphinx can run on both Linux and Windows. Although this tutorial uses Ubuntu, the steps are the same in the Windows environment.

How to use Sphinx?

Sphinx uses reStructuredText as its markup language. The process of Sphinx generating documents is like this:

Project source code (Python or other supported languages) -> reStructuredText files -> documents (HTML or other supported format)

Sphinx provides two command-line tools: sphinx-quickstart and sphinx-apidoc.

• sphinx-quickstart sets up a source directory and creates a default configuration, conf.py, and a master document, index.rst, which serves as a welcome page of a document.
• sphinx-apidoc generates reStructuredText files to document from all found modules.

In short, we use these two tools to generate Sphinx source code, i.e., reStructuredText files, and we modify these reStructuredText files, and finally use Sphinx to build excellent documents.

Workflow

The same as software needs a developer’s maintenance, writing a software document is not a one-time job. It needs to be updated when the software changes. The picture below demonstrates the basic Sphinx workflow.

The following sections detail each step of the workflow.

Prepare

Before we start using Sphinx, we need to set up our working environment.

user@ubuntu:~$ python3.9 -m venv sphinxvenv
user@ubuntu:~$ source sphinxvenv/bin/activate
(sphinxvenv) user@ubuntu:~$ git clone https://github.com/shunsvineyard/python-sample-code.git
(sphinxvenv) user@ubuntu:~$ cd python-sample-code/
(sphinxvenv) user@ubuntu:~/python-sample-code$ python -m pip install -r requirements.txt

Now, we have the Sample Project and working environment for the Sphinx demo. Because the Sample Project already contains the docs folder, we need to delete it.

The layout of the Sample Project after we delete the docs folder looks like:

├── LICENSE
├── README.rst
├── requirements.txt
├── setup.py
├── tests
└── trees
    ├── __init__.py
    ├── __main__.py
    ├── bin
    │   ├── __init__.py
    │   └── tree_cli.py
    ├── binary_trees
    │   ├── __init__.py
    │   ├── avl_tree.py
    │   ├── binary_search_tree.py
    │   ├── binary_tree.py
    │   ├── red_black_tree.py
    │   ├── threaded_binary_tree.py
    │   └── traversal.py
    └── tree_exceptions.py

Step 1: Use sphinx-quickstart to generate Sphinx source directory with conf.py and index.rst

Assume we want to put all the document related files in the docs directory. So, we begin by creating a Sphinx documentation directory, docs. Then, we go to the docs directory and run sphinx-quickstart.

(sphinxvenv) user@ubuntu:~/python-sample-code$ mkdir docs
(sphinxvenv) user@ubuntu:~/python-sample-code$ cd docs/
(sphinxvenv) user@ubuntu:~/python-sample-code/docs$ sphinx-quickstart

Once we run sphinx-quickstart, it asks a few questions about this project. The followings are the example answers for these questions.

Welcome to the Sphinx 3.4.3 quickstart utility.

Please enter values for the following settings (just press Enter to
accept a default value, if one is given in brackets).

Selected root path: .

You have two options for placing the build directory for Sphinx output.
Either, you use a directory "_build" within the root path, or you separate
"source" and "build" directories within the root path.
> Separate source and build directories (y/n) [n]: y

The project name will occur in several places in the built documentation.
> Project name: Sample Project
> Author name(s): Author
> Project release []: 0.0.1

If the documents are to be written in a language other than English,
you can select a language here by its language code. Sphinx will then
translate text that it generates into that language.

For a list of supported codes, see
https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-language.
> Project language [en]: 

Creating file /home/shunsvineyard/workspace/python-sample-code/docs/source/conf.py.
Creating file /home/shunsvineyard/workspace/python-sample-code/docs/source/index.rst.
Creating file /home/shunsvineyard/workspace/python-sample-code/docs/Makefile.
Creating file /home/shunsvineyard/workspace/python-sample-code/docs/make.bat.

Finished: An initial directory structure has been created.

You should now populate your master file /home/shunsvineyard/workspace/python-sample-code/docs/source/index.rst and create other documentation
source files. Use the Makefile to build the docs, like so:
   make builder
where "builder" is one of the supported builders, e.g. html, latex or linkcheck.

At the end of the sphinx-quickstart, it shows how to build the documents.

You should now populate your master file ./source/index.rst and create other documentation
source files. Use the Makefile to build the docs, like so:
   make builder
where "builder" is one of the supported builders, e.g. html, latex or linkcheck.

If we do make html here, Sphinx will generate the default documents which contain nothing about the Sample Project.

Note: Sphinx is not a tool that offers fully automatic documents generation like Doxygen. sphinx-quickstart only generates some default files such as index.rst and conf.py with basic information answered by a user. Therefore, we need to do some work to make the documents real.

After running sphinx-quickstart, the layout of the docs folder looks like:

docs
├── Makefile
├── build
├── make.bat
└── source
    ├── _static
    ├── _templates
    ├── conf.py
    └── index.rst

Note that Makefile is for Linux, and make.bat is for Windows.

Step 2: Configure the conf.py

sphinx-quickstart generates a few files, and the most important one is conf.py, which is the configuration of the documents. Although conf.py serves as a configuration file, it is a real Python file. The content of conf.py is Python syntax.

Using Sphinx to generate a document is highly configurable. This section demonstrates the most basic configurations: the path to the project source code, the theme for the documents, and extensions.

Set the path to the project

To make Sphinx be able to find the project, we need to uncomment these three lines.

# import os
# import sys
# sys.path.insert(0, os.path.abspath('.')) 

And update the path to the project.

import os
import sys
sys.path.insert(0, os.path.abspath('../..'))

Select the Theme

Sphinx provides many built-in themes. The default is alabaster.

html_theme = 'alabaster'

In this tutorial, we change it to bizstyle.

html_theme = 'bizstyle'

More themes and their configurations can be found at https://www.sphinx-doc.org/en/master/usage/theming.html

Add an extension for NumPy style

The Sample Project uses NumPy style for docstrings. Therefore, we need to add the extension (napoleon) for parsing NumPy style docstrings.

extensions = [
    'sphinx.ext.napoleon'
]

This extension (napoleon) supports NumPy and Google style docstrings and provides several configurable features. For the Sample Project, since we use NumPy style docstrings, we should disable Google style.

napoleon_google_docstring = False

Other settings for napoleon can be found at https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html#module-sphinx.ext.napoleon

The complete conf.py example can be found at https://github.com/burpeesDaily/python-sample-code/blob/master/docs/source/conf.py.

Besides, Sphinx has many built-in extensions and also supports custom extensions. To learn more, please visit https://www.sphinx-doc.org/en/master/usage/extensions/index.html

Now, we have the basic configuration for our project. Next, we use sphinx-apidoc to generate reStructuredText files from the Sample Project source code.

Step 3: Use sphinx-apidoc to generate reStructuredText files from source code

sphinx-apidoc is a tool for automatically generating reStructuredText files from source code, e.g. Python modules. To use it, run

$ sphinx-apidoc -f -o <path-to-output> <path-to-module>

-f means force overwriting of any existing generated files.

-o means the path to place the output files.

For the Sample Project, we run the following command.

(sphinxvenv) user@ubuntu:~/python-sample-code/docs$ sphinx-apidoc -f -o source/ ../trees/

Complete usage of sphinx-apidoc is at https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html.

In the Sample Project, sphinx-apidoc generates a few files, modules.rst, trees.bin.rst, trees.binary_trees.rst, and trees.rst. The layout of the project looks like the following:

docs
├── Makefile
├── build
│   └── doctrees
│       ├── environment.pickle
│       └── index.doctree
├── make.bat
└── source
    ├── _static
    ├── _templates
    ├── conf.py
    ├── index.rst
    ├── modules.rst
    ├── trees.bin.rst
    ├── trees.binary_trees.rst
    └── trees.rst

Step 4: Edit index.rst and the generated reStructuredText files

The other important file that sphinx-quickstart generates is index.rst. index.rst is the master document that serves as a welcome page and contains the root of the ‘’table of contents tree’’ (toctree). The toctree initially is empty when sphinx-quickstart creates index.rst.

.. toctree::
   :maxdepth: 2
   :caption: Contents:

Add the modules to the index.rst

The generated modules.rst contains all the modules. So we need to add the modules.rst to index.rst.

To add document to be listing on the welcome page (index.rst), do

.. toctree::
   :maxdepth: 2
   :caption: Contents:

   modules

Note: when to add another reStructuredText file, use the file name without extension. If there is a hierarchy of the file, use forward slash ‘’/’’ as directory separators.

Add the README.rst to index.rst

Since the Sample Project already has a readme file, README.rst, at the top level of the project, we can add it to the document’s welcome page.

1. Create a readme.rst file under docs/source and add the line .. include:: ../../README.rst. (See https://github.com/burpeesDaily/python-sample-code/blob/master/docs/source/readme.rst)
2. Add the readme to the index.rst, so the welcome page will include it.

After these two steps, the index.rst looks like:

.. Sample Project documentation master file, created by
   sphinx-quickstart on Wed Jan 13 23:24:14 2021.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to Sample Project's documentation!
==========================================

.. toctree::
   :maxdepth: 2
   :caption: Contents:

   readme
   modules

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

See https://raw.githubusercontent.com/burpeesDaily/python-sample-code/master/docs/source/index.rst for the complete example.

Note: when we add a new module, class, API, or any code change that affects the documents, we need to repeat Step 3 and Step 4 to update the documents.

Step 5: Build the documents

The last step to generate the documents is to issue make html (if we want to generate HTML-based documents).

(sphinxvenv) user@ubuntu:~/python-sample-code/docs$ make html

After we run the make html command, a build folder is created under docs. Also, the HTML-based documents are located at build/html. The generated document looks like this:

Conclusion

Although we still need to edit the generated reStructuredText files manually, Sphinx provides an easier way to build a nice document. It also features configurable and extensible abilities via conf.py and extensions. To learn more about Sphinx, check these online resources:

• Sphinx Official Website
• Awesome Sphinx
• reStructuredText Official Website
• reStructuredText Quick Start