$$ \newcommand{\beqa}{\begin{eqnarray}} \newcommand{\eeqa}{\end{eqnarray}} \newcommand{\ep}{\thinspace . } \newcommand{\uvec}{\vec u} \newcommand{\Q}{\pmb{Q}} $$

 

 

 

DocOnce Description

Hans Petter Langtangen [1, 2]
Kristian Gregorius Hustad [3, 2]

[1] Center for Biomedical Computing, Simula Research Laboratory
[2] Department of Informatics, University of Oslo
[3] Centre for Computing in Science Education, University of Oslo

Mar 13, 2017


Warning. Some parts of this manual may be outdated. Please create an issue at https://github.com/hplgit/doconce to report errors.

What Is DocOnce?

DocOnce is a very simple and minimally tagged markup language that looks like ordinary ASCII text, much like what you would use in an email, but the text can be transformed to numerous other formats, including HTML, Sphinx, LaTeX, PDF, reStructuredText (reST), Markdown, MediaWiki, Creole wiki, blogger.com, wordpress.com, Epytext, and also plain (untagged) text for email. From reST or Markdown you can go to XML, OpenOffice, MS Word, HTML, LaTeX, PDF, DocBook, GNU Texinfo, and more.

DocOnce supports a working strategy of never duplicating information. Text is written in a single place and then transformed to a number of different destinations of diverse type: scientific reports, software manuals, books, thesis, software source code, wikis, blog posts, emails, etc. The slogan is: "Document once, include anywhere".

Here are some DocOnce features:

History. The DocOnce development started in 2006 at a time when most popular markup languages used quite some tagging (LaTeX, reStructuredText, HTML). Later, almost untagged markup languages became popular, especially Markdown and its sisters MultiMarkdown, Pandoc-extended Markdown, and Markua. DocOnce looks much like Markdown and is in particular close to the functionality and nature of MultiMarkdown. The advantage of DocOnce, however, is a series of features for supporting both small and large documents (books in particular) with much mathematics and computer code. While Markdown tools are heavily geared toward HTML, DocOnce has strong support for LaTeX since this is the dominating format for books and articles on mathematical subjects. DocOnce can also output Sphinx (not supported by Pandoc or MultiMarkdown), a format that is very attractive for presenting scientific material and software documentation on the web. (DocOnce allows basic Markdown syntax as input, extended with DocOnce syntax as you like.)

Disclaimer. DocOnce applies text transformations, mostly via regular expressions. This is not a fool-proof method of translation compared to real parsing. Moreover, the possibility for tweaking the layout in the DocOnce document is obviously limited (at least compared to LaTeX and HTML) since the text can go to all sorts of markup languages. This disadvantage can be quite easily compensated, however, by clever use of the programmable Mako preprocessor used by DocOnce and by automatic editing of the generated output (e.g., via regular expressions).

DocOnce has been used as writing platform for several books:





Table of contents

What Is DocOnce?
      Demos and Documentation
Markup Based on Special Lines
      Heading with title and author(s)
      Copyright
      Table of contents
      Section headings
      Abstract
      Appendix
      Figures
      Movies
      Copying Computer Code from Source Files
      Inserting the Output from Operating System Commands
      Comments
      Tables
      Lists
Inline Tagging
      Emphasized Words
      Inline Verbatim Text
      Links to Web Addresses
      Links to Mail Addresses
      Links to Local Files
      Quotes
      Non-Breaking Space
      Horizontal rule
      Em-dash
      En-dash
      Ampersand
      Footnotes
      Inline Comments
      Inline Comments for Editing
      Forced Line Breaks
      Inline Mathematics
      Cross-Referencing
      Generalized Cross-Referencing
      Index
      Emojis
Exercises, Problems, Projects, and Examples
      Examples on Exercise Syntax
      Typesetting of Exercises
      List of Exercises, Problems, and Projects
      Numbering of Extra Equations in Solutions
      Typesetting of solutions to exercises
      Extracting Selected Exercises in a Separate Document
      Extracting Exercises as Stand-Alone Documents
      Example on an Exercise
      Exercise 1: Compute integrals
Other Environments
      Blocks of Verbatim Computer Code
      LaTeX Blocks of Mathematical Text
      Macros (Newcommands)
      Writing Guidelines (Especially for LaTeX Users!)
      Typesetting of Algorithms
      Admonitions
      User-Defined Environments
      Example 1: Addition
Bibliography (References)
      Importing your data to the Publish database
      Requirements to input data
      Adding new references to the database
      Exporting the database
      Referring to publications
      Specifying the Publish database
      LaTeX Bibliography Style
Preprocessing and Postprocessing
      The Preprocess and Mako Preprocessors
      Splitting Documents into Smaller Pieces
Writing Slides
      Overview
      Slide Elements
      HTML5 Slides
      LaTeX Beamer Slides
Support for non-English
Misc
      Missing Features
      Git .gitignore File
      Emacs DocOnce Formatter
      Atom Syntax Highlighting for DocOnce
Mako Programming
      The Basics of Mako
      Debugging Python code in Mako
      Example: Nomenclature functionality
      Example: Executing Python and using SymPy Objects in LaTeX
      Example: Extending Tables to Handle Figures
      Example: Defining a Theorem Environment
      Tools for Writing DocOnce Documents
      Debugging
From DocOnce to Other Formats
      Writing a Makefile
      Generating a Makefile
      Spell checking
      Preprocessing
      Removal of Inline Comments
      Notes
      Demo of Different Formats
      Tweaking the DocOnce Output
      Useful Options for doconce format
HTML
      Basic HTML Output
      Typesetting of Code
      Handling of Movies
      HTML Styles
      HTML templates
      Splitting HTML documents
      URL to files hosted on GitHub
      Other HTML options
      Blog Posts
Pandoc and Markdown
      Markdown to HTML conversion
      Strict Markdown
      GitHub-flavored Markdown
      MultiMarkdown
      Strapdown rendering of Markdown text
      Using Pandoc to go from LaTeX to MS Word or HTML
LaTeX
      Overview
      The old ptex2tex step
      LaTeX-PDF: Generate LaTeX (Step 1)
      LaTeX-PDF: Edit the LaTeX File (Step 2, Optional)
      LaTeX-PDF: Generate PDF (Step 3)
      XeLaTeX
      From PDF to e-book formats
      Microsoft Word or LibreOffice
Jupyter (IPython) Notebooks
      Hidden code blocks
      Displaying code as plain text instead of executable cells
      Figures
      Movies
      Admonitions
      References to an External Textbook
      Conversion from Notebook Back to DocOnce
Matlab Notebooks
Plain ASCII Text
reStructuredText
Sphinx
      The Basic Steps
      Links
      Themes
      RunestoneInteractive books
      The manual Sphinx procedure
Wiki Formats
Google Docs
Options for the doconce commands
      doconce format command-line options
Installation of DocOnce and its Dependencies
      DocOnce
      Dependencies
Basic Parsing Ideas
      Typesetting of Function Arguments, Return Values, and Variables
References

Demos and Documentation

This guide is for the experienced DocOnce writer! Do not read the detailed descriptions of DocOnce syntax that follows (this is a manual!) before you have read the tutorial and played a little around with a simple document.

The primary demo for what can be achieved as output from DocOnce documents regards a little scientific report that is compiled to a range of various output documents. A version of that web page contains the specific compilation commands to create each of the output demonstrations.

There is also a demo on the many different ways one can create slides.

Write DocOnce documents in a text editor with monospace font! Some DocOnce constructions are sensitive to whitespace (indentation in lists is a primary example), so you must use a text editor with monospace font (also known as verbatim text). Never use fonts like Arial or Helvetica. (Other popular markup languages such as Sphinx and Markdown are also sensitive to whitespace and require a monospace font in the text editor.)

Tip: Read the FAQ! A lot of tips and problems have over the years been collected in the Troubleshooting and FAQ document, available in Sphinx, HTML, and PDF. The FAQ and this manual are the two key references for how to make use of DocOnce.

The DocOnce tutorial (available in Sphinx, HTML, and PDF formats) has its source code in the GitHub repository for DocOnce, more precisely in the file

doc/src/tutorial/tutorial.do.txt

One can compare this source with the output in HTML, Sphinx, and PDF. The make.sh script in the same directory tells in detail how the various versions were compiled.

The DocOnce source of the current manual is also found at the GitHub repository for DocOnce, in the file

doc/src/manual/manual.do.txt

You can compare this source with the available output in HTML, Sphinx, and PDF.

Markup Based on Special Lines

The DocOnce markup language has a concept called special lines. Such lines starts with a markup at the very beginning of the line and are used to mark document title, authors, date, sections, subsections, paragraphs, figures, movies, lists, etc.

Heading with title and author(s)

Lines starting with TITLE:, AUTHOR:, and DATE: are optional and used to identify a title of the document, the authors, and the date. The title is treated as the rest of the line, so is the date, but the author text consists of the name and associated institution(s) with the syntax

 AUTHOR: name at institution1 and institution2 and institution3

The at with surrounding spaces is essential for adding information about institution(s) to the author name, and the and with surrounding spaces is essential as delimiter between different institutions. An email address can optionally be included, using the syntax

 AUTHOR: name Email: somename@site.net at institution1 and institution2

Multiple authors require multiple AUTHOR: lines. All information associated with TITLE: and AUTHOR: keywords must appear on a single line. Here is an example:

 TITLE: On an Ultimate Markup Language
 AUTHOR: H. P. Langtangen at Center for Biomedical Computing, Simula Research Laboratory & Dept. of Informatics, Univ. of Oslo
 AUTHOR: Kaare Dump Email: dump@cyb.space.com at Segfault, Cyberspace Inc.
 AUTHOR: A. Dummy Author
 DATE: November 9, 2016

Note how one can specify a single institution, multiple institutions (with & as separator between institutions), and no institution. In some formats (including rst and sphinx) only the author names appear. Some formats have "intelligence" in listing authors and institutions, e.g., the plain text format:

Hans Petter Langtangen [1, 2]
Kaare Dump  (dump@cyb.space.com) [3]
A. Dummy Author

[1] Center for Biomedical Computing, Simula Research Laboratory
[2] Department of Informatics, University of Oslo
[3] Segfault, Cyberspace Inc.

Similar typesetting is done for LaTeX and HTML formats.

The current date can be specified as today.

A copyright notice can be given as part of the AUTHOR: command. The syntax is

{copyright,year1-year2|license}

where year1-year2 represents the year(s) and license represents the type of license (e.g., Creative Commons license) if that apply. The year and license parts can be left out. In that case, the current year is used, and no license appears. Note that the year and the license must be identical in all copyright specifications for all authors who claim copyright! (Otherwise, an error message is issued.) The {copyright...} specification can appear after the author's name (and email address) and/or after an organization's name (see examples below).

The syntax of the year1-year part goes as follows:

The syntax of the license part is flexible: How the copyright is typeset depends on the format: Here are some examples on specifying copyright.

 AUTHOR: Joe Doe Email:joe.doe@somemail.com {copyright}
 AUTHOR: Jane Doe {copyright}

Output becomes "Copyright 2015, Joe Doe, Jane Doe" (if the present year is 2015).

 AUTHOR: Joe Doe {copyright,2001-present}

Output becomes "Copyright 2001-2015, Joe Doe" (if the present year is 2015).

 AUTHOR: Joe Doe {copyright,2001-2010|CC BY}
 AUTHOR: Jane Doe Email:jd@kk.org {copyright,2001-2010|CC BY}

Output becomes "Copyright 2001-2010, Joe Doe, Jane Doe. Released under CC Attribution 4.0 license". One can provide the option <linebreak> --CC_license="This work is released under the Creative Commons %s 4.0 license.", and the output becomes "Copyright 2001-2010, Joe Doe, Jane Doe. This work is released under the Creative Commons Attribution 4.0 license."

 AUTHOR: Joe Doe {copyright} at Digital Company {copyright}
 AUTHOR: Jane Doe

In this case, an author and an institution (but not the second author) hold the copyright. The output is typically "Copyright 2015, Joe Doe, Digital Company". Below, two institutions but no authors hold the copyright:

 AUTHOR: Joe Doe at Digital Company {copyright,2015|CC BY}
 AUTHOR: Jane Doe at Analog Company {copyright,2015|CC BY}

The output becomes "Copyright 2015, Digital Company, Analog Company. Released under CC Attribution 4.0 license".

Table of contents

A table of contents can be generated by the line

TOC: on

This line is usually placed after the DATE: line. The value off turns off the table of contents.

The depth of the table of contents is dictated by the command-line option --toc_depth=, which is 2 by default, meaning that sections and subsections are included, but not subsubsections. When making Sphinx documents, toc_depth= is a command-line option for for the doconce sphinx_dir command (and not doconce format).

Section headings

Section headings are recognized by being surrounded by equal signs (=) or underscores before and after the text of the headline. Different section levels are recognized by the associated number of underscores or equal signs (=):

Headings can be surrounded by as many blanks as desired, but the first = must start in column 1 and there must one blank on each side of the heading, between the heading and the = signs. Here are examples on headings:

======= Example on a Section Heading =======

The running text goes here.


===== Example on a Subsection Heading =====

The running text goes here.

=== Example on a Subsubsection Heading ===

The running text goes here.

__A Paragraph.__ The running text goes here.

Abstract

DocOnce also supports abstracts. The syntax is like an ordinary paragraph with heading Abstract, Summary, or *Preface, but the text must be followed by a

Everything up to the first heading, table of contents, or date is taken as the abstract. For articles, the abstract is placed after the date, but before the table of contents or the first ordinary heading. For books one may insert the abstract before the date to make appear on the first page.

Here are examples on an abstract and some type of ending text (section headings, table of contents, or date).

__Abstract.__ This abstract
lasts up to the section heading.

======= Here Is the First Section Heading =======

# or

__Summary.__
This is
a summary.

Even with two paragraphs. It lasts
until the table of contents.

TOC: on

# In books we may place the summary before DATE

TITLE: Some Title
AUTHOR: Some Author

__Summary.__ Here is the backmatter
promotion text for this book, appearing
on the front page...

DATE: today

Appendix

Appendix is supported too: just let the heading start with "Appendix: " This affects only latex output, where the appendix formatting is used - all other formats just leave the heading as it is written.

Figures

Basic syntax

Figures are recognized by the special line syntax

FIGURE:[filename, width=600 frac=0.8] caption

The filename can be without extension, and DocOnce will search for an appropriate file with the right extension. If the extension is wrong, say .pdf when requesting an HTML format, DocOnce tries to find another file, and if not, the given file is converted to a proper format (using ImageMagick's convert utility).

Warning. Note the comma between the filename and the figure size specifications and that there should be no space around the = sign. This syntax must be strictly followed.

Note also that, like for TITLE: and AUTHOR: lines, all information related to a figure line must be written on the same line. Introducing newlines in a long caption will destroy the formatting (only the part of the caption appearing on the same line as FIGURE: will be included in the formatted caption).

The height, width, and frac keywords can be included if desired and may have effect for some formats: the height and width are used for output in the formats html, rst, sphinx, while the frac specification is used for latex and pdflatex to specify the width of the image as a fraction of the text width.

Figure Placement

In web formats (html, sphinx, ipynb, matlabnb, wikis), the FIGURE: command is replaced by an img tag exactly where the FIGURE: appears in the document. LaTeX, however, will normally place the figure at a different location. The generated LaTeX code applies

\begin{figure}[!ht] % my:fig

i..e, we use the "here" option [!ht] to recommend a placement as near the FIGURE: command as possible. One can autoedit the .tex file and modify the figure environment options, e.g.,

Terminal> doconce replace '{figure}[!ht]' '{figure}[t]' mydoc.tex

The above command will change all [!ht] options to [t] (top). A specific figure can also be edited, using the fact that the label is printed at the same line as \begin{figure}:

Terminal> doconce subst '{figure}[!ht] .+my:fig' \
          '{figure}[!h] % my:fig' mydoc.tex

Of greater influence than options like [ht], [h], etc., is the LaTeX code found in the preamble:

\setcounter{topnumber}{2}
\setcounter{bottomnumber}{2}
\setcounter{totalnumber}{4}
\renewcommand{\topfraction}{0.95}
\renewcommand{\bottomfraction}{0.95}
\renewcommand{\textfraction}{0}
\renewcommand{\floatpagefraction}{0.75}
% floatpagefraction must always be less than topfraction!
\usepackage[section]{placeins}  % flush all figs before next section

These values can be manipulated to fine-tune how LaTeX places figures.

Figure References

Suppose we have the DocOnce code

The results are presented in Figure ref{myfig}.

FIGURE:[myfigfile, width=400 frac=0.8] Results for $a=2$. label{myfig}

Different formats will display the figure reference differently. In LaTeX, DocOnce generates the code ... in Figure \ref{myfig}, which reads "... in Figure 5" (article) or "... in Figure 5.2" (book). Requesting the varioref package (with --latex_packages=varioref) makes DocOnce emit \vref references and then the above reference becomes in Figure \vref{myfig}, which reads "... in Figure 5 on page 67". However, if Figure 5 appears on the present page where the reference is done, the page reference is left out, and one can read just "in Figure 5".

Sphinx applies the caption as name of the figure, so the reference reads "... in Figure Results for .", and the caption/name is a link to the figure. Note that Sphinx strips off the mathematics from the caption. In HTML, figures are given numbers, so the reference reads "... in Figure 3", with the figure number as a link to the place in the document where the FIGURE: command was located. The IPython notebook format makes a Markdown link: ... in [Figure](#myfig), where myfig is an anchor such one can click on Figure. The plain text format displays the reference as "... in Figure ref{myfig}.". Wiki formats show "... in Figure myfig.". So to summarize, figure references work best in LaTeX, HTML, and Sphinx. Other formats should avoid figure references with labels.

Inline Figures

The figure caption is optional. If omitted, the figure appears "inline" in the text without any figure environment in LaTeX formats or HTML. An inline figure is handy in LaTeX since it appears exactly where the FIGURE: command appears (figures with captions are encapsulated in the LaTeX figure environment and become floating objects whose placement is up to LaTeX do decide).

Tip: use linebreak to insert space around inline figures. Sometimes inline figures (FIGURE line without caption) get squeezed into the text. You can add vertical space in LaTeX and HTML by inserting several lines with <linebreak>.


Figure 1: A wave.

Choosing the Figure Format

For each output format, there is a preference which type of graphics file that is preferred. That is, for HTML, for instance, we can accept many types of graphics formats. If we specify the extension as part of the filename, DocOnce will try to find that specific format. If not found, it will convert whatever can be used to some format suitable for HTML. It is highly recommended to prepare figures in different formats manually and not rely on automatic conversion by DocOnce – that gives the best quality. For example, plots should always be prepared in both PDF and PNG formats.

If we just specify the filestem of the figure file, DocOonce will pick the most appropriate version of the file for given output format. For HTML there is a preference for .svg, thereafter comes .html (typically Bokeh plots), then .png, then gif, then .jpg. Here is the preference list for the various formats:

Other formats, which can display graphics, will prefer .png files.

Handling Variable Figure Paths

Figure files are usually located in some directory. Sometimes one needs to compile the DocOnce source file(s) from different directories, and then the path to figure files changes. For example, think of a master DocOnce file that includes different sections whose DocOnce source files are located in different directories. If you want to compile a section as stand-alone document, you have to do that from the subdirectory for that section. The path to a common directory for figures may then be like ../fig/myfig.png, while for the master document in the parent directory, the corresponding path is fig/myfig.png.

The simplest way out of this problem is to use the --fig_prefix= command-line option to set a path prefix for the figure filename. When compiling a section in a subdirectory one sets --fig_prefix=../fig while in the parent directory one needs --fig_prefix=fig to compile the master document. In the DocOnce source file one has FIGURE: [myfig, width=...].

A more manual method is to introduce a Mako variable FIGPREFIX that is set on the command line as part of the doconce format command. The FIGPREFIX variable holds a prefix for the path to the figure. In our example one writes

FIGURE: [${FIGPREFIX}/myfig, width=500 frac=0.8] caption label{my:fig}

and set FIGPREFIX=../fig after the doconce format ... command if one compiles a section, while FIGPREFIX=fig is the appropriate value of the path when compiling the master document.

Figures with Subfigures

Combining several image files into one, in a table fashion, can be done by the montage program from the ImageMagick suite:

montage -background white -geometry 100% -tile 2x \
        file1.png file2.png ... file4.png result.png

The option -tile XxY gives X figures in the horizontal direction and Y in the vertical direction (tile 2x means two figures per row and -tile x2 means two rows).

The montage program is only appropriate for bitmap images (PNG, JPEG, GIF, TIFF). Images in the PDF format should be mounted together using pdftk (to combine images to one file), pdfnup (to align them in tabular format), and pdfcrop (to remove surrounding whitespace):

Terminal> pdftk file1.pdf file2.pdf ... file4.pdf output tmp.pdf
Terminal> pdfnup --nup 2x2 tmp.pdf        # output in tmp-nup.pdf
Terminal> pdfcrop tmp-nup.pdf result.png  # output in FE1.png

Instead of using montage, pdftk, etc., one can rely on the convenient command doconce combine_images:

Terminal> doconce combine_images pdf -2 fig1 fig2 fig3 fig4 fig

This command will combine fig1.pdf, fig2.pdf, fig3.pdf, and fig4.pdf with two images per row (-2 option) and place the result in fig.pdf. By just changing the pdf option to png, the same will happen with fig1.png, fig2.png, fig3.png, and fig4.png, resulting in fig.png. The tool employs the technique above for PNG and PDF files to produce ultimate quality of the combined image.

One can also run doconce combine_images with filenames with extension, e.g.,

Terminal> doconce combine_images myfig1.png myfig2.png fig2.png

Here, myfig1,png and myfig2.png are placed next to each other in a new figure file fig2.png.

Sidecaption in LaTeX and HTML

The figure caption can be placed on the (right) side of figures by using the sidecap feature as figure option, e.g., FIGURE: [myfig, width=500 frac=0.5 sidecap=True]. The generated latex and pdflatex output then uses the sidecap package and the SCfigure environment to typeset the figure. Remember to use a quite low frac value for figures with sidecaption (0.5 for instance). A table is used for typesetting a figure with sidecaption in HTML, and a low width value is recommended. The sidecap=True figure option has no impact on other formats.

TikZ Figures

Many LaTeX writers are dependent upon TikZ figures, and these can be used in DocOnce documents, see the demo document.

Plot Files in LaTeX

Users who applies Matplotlib to make figures get plots with fonts that differ from the rest of a LaTeX document. A blog post describes techniques for overcoming this problem. The plotfile is then a .pgf file and one must use the pgf LaTeX package. DocOnce supports .pgf plot files for the pdflatex output format and will make use of such files if they exist. These are included by a simple \input{file.pgf}. If no .pgf file is found, the pdflatex output format will apply .pdf, .png, or .jpg file, in that order of preference.

Interactive Bokeh Plots for HTML

Fancy interactive plots for data exploration can be made with the Bokeh library. Such plots reside in an HTML file. DocOnce will for the HTML output format detect files of this type and use the HTML code in the file to embed the plot(s) in the generated output document.

Below is a complete example on creating a grid of interactive plots where the horizontal axes are coupled to each other. Panning the graph in one plot automatically moves all the other graphs. In this way, one can scroll through a long time series simultaneously for many plots. Grap one graph below, move into the future with the mouse, and watch the other graphs follow!





Such a figure is specified the normal way: if the HTML code for the figure is in myfig.html, write

FIGURE: [myfig] caption

Options like width are ignored for Bokeh plots, unless you have other versions of the figure (myfig.png, for instance, see the box below) where such options may be useful for some formats.

Make alternatives to Bokeh plots. Note that Bokeh plots have only meaning when DocOnce translates the document to HTML. For other formats, one needs to supply figure files that those formats can accept (PNG, PDF, etc.).

Suppose you have made a Bokeh plot in myfig.html. Either you have to embed the FIGURE command inside a preprocessor test that the FORMAT == 'html' or you must provide alternatives like myfig.png. A Bokeh plot will often have a save button that can be used to save the plot to PNG format. This can be used for Sphinx, wikis, and pdfLaTeX (although the latter would appreciate real vector graphics in a PDF plot).

The plot example above is so advanced that there is no natural counterpart in a static PNG or PDF plot.

Tip: reduce the size of Bokeh HTML files. When making Bokeh plots in Python programs, we recommend to use the mode='cdn' option in the call output_file. This argument leads to links to Bokeh tools in the resulting HTML file. Without the argument, Bokeh embeds long HTML code for its tools into the file. DocOnce issues a warning in this case and recommends the mode argument.

Note that with mode='cdn' the HTML code for the plot requires Internet access.

An interactive plot like the one shown above, stored in a file tmp.html, can be made by the code below (download file):

from __future__ import division
from builtins import range
from past.utils import old_div

def bokeh_plot(u, t, legends, u_e, t_e, I, w, t_range, filename):
    """
    Make plots for u vs t using the Bokeh library.
    u and t are lists (several experiments can be compared).
    legends contain legend strings for the various u,t pairs.
    Each plot has u vs t and the exact solution u_e vs t_e.
    """
    import numpy as np
    import bokeh.plotting as plt
    plt.output_file(filename, mode='cdn', title='Comparison')
    # Assume that all t arrays have the same range
    t_fine = np.linspace(0, t[0][-1], 1001)  # fine mesh for u_e
    tools = 'pan,wheel_zoom,box_zoom,reset,'\
            'save,box_select,lasso_select'
    u_range = [-1.2*I, 1.2*I]
    font_size = '8pt'
    p = []  # list of all individual plots
    p_ = plt.figure(
        width=300, plot_height=250, title=legends[0],
        x_axis_label='t', y_axis_label='u',
        x_range=t_range, y_range=u_range, tools=tools,
        title_text_font_size=font_size)
    p_.xaxis.axis_label_text_font_size=font_size
    p_.yaxis.axis_label_text_font_size=font_size
    p_.line(t[0], u[0], line_color='blue')
    p_.line(t_e, u_e, line_color='red', line_dash='4 4')
    p.append(p_)
    for i in range(1, len(t)):
        p_ = plt.figure(
            width=300, plot_height=250, title=legends[i],
            x_axis_label='t', y_axis_label='u',
            x_range=p[0].x_range, y_range=p[0].y_range, tools=tools,
            title_text_font_size=font_size)
        p_.xaxis.axis_label_text_font_size=font_size
        p_.yaxis.axis_label_text_font_size=font_size
        p_.line(t[i], u[i], line_color='blue')
        p_.line(t_e, u_e, line_color='red', line_dash='4 4')
        p.append(p_)

    # Arrange in grid with 3 plots per row
    grid = [[]]
    for i, p_ in enumerate(p):
        grid[-1].append(p_)
        if (i+1) % 3 == 0:
            # New row
            grid.append([])
    plot = plt.gridplot(grid, toolbar_location='left')
    plt.save(plot)
    plt.show(plot)

def demo_bokeh():
    """Plot numerical and exact solution of sinousoidal shape."""
    import numpy as np

    def u_exact(t):
        return I*np.cos(w*t)

    def u_numerical(t):
        w_tilde = (old_div(2.,dt))*np.arcsin(w*dt/2.)
        return I*np.cos(w_tilde*t)

    I = 1               # Amplitude
    w = 1.0             # Angular frequency
    P = 2*np.pi/w       # Period of signal
    num_steps_per_period = [5, 10, 20, 40, 80]
    num_periods = 40
    T = num_periods*P   # End time of signal

    t_e = np.linspace(0, T, 1001)  # Fine mesh for u_exact
    u_e = u_exact(t_e)
    u = []
    t = []
    legends = []

    # Make a series of numerical solutions with different time steps
    for n in num_steps_per_period:
        dt = old_div(P,n)  # Time step length
        t_ = np.linspace(0, T, num_periods*n+1)
        u_ = u_numerical(t_)
        u.append(u_)
        t.append(t_)
        legends.append('# time steps per period: %d' % n)
    bokeh_plot(u, t, legends, u_e, t_e,
               I=1, w=w, t_range=[0, 4*P],
               filename='tmp.html')

demo_bokeh()

Converting Matplotlib Plots to Bokeh

Most Python users apply Matplotlib to create line drawings. Bokeh has a conversion utility from Matplotlib to Bokeh that works well for standard curve plots. The script below demonstrates how to generate a plot in Matplotlib and convert it to a Bokeh tmp.html file.

import numpy as np
import matplotlib.pyplot as plt

x = np.linspace(0, 2*np.pi, 1001)
y1 = np.exp(-x)*np.sin(2*x)
y2 = np.exp(-0.5*x)*np.sin(2*x)

plt.plot(x, y1, 'r-', x, y2, 'b--')
plt.xlabel('x');  plt.ylabel('y')
# legends do not work in Bokeh
#plt.legend([r'$e^{-x}\sin 2x$', r'$e^{-\frac{1}{2}x}\sin 2x$'])
plt.title('Damped sine functions')
plt.savefig('tmp.pdf');  plt.savefig('tmp.png')

# Convert to Bokeh
import bokeh.mpl, bokeh.plotting as bpl
p = bokeh.mpl.to_bokeh(notebook=False, xkcd=False)
#p = bokeh.mpl.to_bokeh()
bpl.output_file('tmp.html', mode='cdn')
bpl.save(p)
#bpl.show(p)

plt.show()


Figure 2: Example on Bokeh figure generated by Matplotlib.

Movies

Movies/videos are inserted using the MOVIE: keyword. This feature works well for the latex, html, rst, and sphinx formats. Other formats try to generate some HTML file and link to that file for showing the movie. If such a link is not appropriate and one wants a figure instead of the movie, use the preprocessor as explained in the box Recommendations below.

The Basic Command

As with FIGURE, the MOVIE command expands just one line and is of the form

MOVIE: [filename, height=xxx width=yyy] possible caption

Note that there must be a blank line after every MOVIE: command. The width and height parameters are not required, but leaving them out may lead to movie sizes you do not want.

Here is a movie in the Ogg format:

A movie in Ogg format.

realized by the command

MOVIE: [mov/wave.igg, width=600] A movie in Ogg format.

A URL works too as movie address:

MOVIE: [http://hplgit.github.io/animate/doc/pub/mov-animate/demo.ogg] Ogg movie in cyberspace.

Ogg movie in cyberspace.

Important. Movies will not work properly in sphinx format unless they are located in a directory (tree) with a name starting with mov. Make it a habit to place figures in fig-X and movies in mov-X directories, where X is a short logical name for the current document (or let the names of the directories be just fig and mov).

MP4, WebM, and Ogg Movies in HTML

If a movie is in Ogg, MP4, or WebM format, and the output format is html, DocOnce will check if the movie file is also in the other formats among Ogg, MP4, and WebM, and include these as well such that the movie has backup formats in case the browser does not support a particular format. Providing a movie in Ogg, MP4, and WebM format is therefore the safest way to ensure that the movie can be played in any browser on any device.

Notice. If you specify a movie in Ogg or WebM format and it also exists in MP4 format, the MP4 format will be loaded first. To avoid having alternative movie formats in HTML, use the --no_mp4_webm_ogg_alternatives command-line option when running doconce format.

Movie Handling in Various Formats

Movies are easiest shown in the HTML format. The reST and Sphinx formats apply the same raw HTML code as the HTML format and therefore have the same capabilities. The LaTeX format supports different methods for embedding movies via the option -latex_movie=.... Proper values are listed below.

  1. href: the \href{run:file}{link} is used for all movies (default), typeset in a one-line quote environment with the movie caption (if present) and a movie counter.
  2. media9: the media9 package is used for Flash and MP4 movies, movie15 for MPEG and AVI files, and a simple \href{run:file}{link} command for other formats. Only Acrobat Reader supports displaying these type of movies.
  3. multimedia: the \movie command (known from LaTeX Beamer) is used for movies.
  4. movie15: the old movie15 package is used.
Where to put the movie file for a PDF document? A major concern when using the default (href) link to a movie in a PDF document, is where to store the movie file. If your document is in a repository at GitHub or Bitbucket, one can use this address for the movie file. However, for the HTML and Sphinx formats, we would like to use some short, local address mov/mymovie.ogg. For LaTeX PDF we can then use the command-line option --prefix_movie= to prefix the local address with the proper address in the repository in the cloud. Suppose the mov directory is found in

https://github.com/n/p/blob/master/doc/mov

We need, however, the raw movie file, which has the address

https://raw.githubusercontent.com/n/p/master/doc/mov/m.ogg

We can then use the line

MOVIE: [mov/m.ogg]

in the document, but compile to LaTeX with the command-line option

--movie_prefix=https://raw.githubusercontent.com/n/p/master/doc/

All links to movies in the PDF file will then be links to the repository file in the cloud while for HTML and Sphinx we link to the local movie files that are stored together with the .html files for the document.

For all other formats, an HTML file that acts as a movie player is generated and linked from the output document. This movie player has essentially the same code as the HTML format would have, except that the video tag is not used, only the embed tag. Some wiki types do have support for videos, e.g., Wikipedia can work with Ogg files, but DocOnce has not yet implemented robust schemes for anything but LaTeX, HTML, and Sphinx output.

YouTube and Vimeo Movies

Many publish their scientific movies on YouTube or Vimeo, and DocOnce recognizes YouTube and Vimeo URLs as movies. When the output from DocOnce is an HTML file, the movie will be embedded, otherwise a URL to the YouTube or Vimeo page is inserted. You should equip the MOVIE: command with the right width and height of embedded YouTube and Vimeo movies. The recipe goes as follows:

  1. click on Share (and on YouTube then Embed)
  2. note the height and width of the embedded movie
A typical MOVIE command with a YouTube movie is then

MOVIE: [http://www.youtube.com/watch?v=sI2uCHH3qIM, width=420 height=315]

MOVIE: [http://vimeo.com/55562330, width=500 height=278] CFD.

Animation Based on Filename Generators

It is possible to define a movie from a set of files, usually plot files, which can be shown in sequence to create an animation. If the files are local on the computer, one can specify them by a simple Unix wildcard notation, as in

MOVIE: [../experiments/frame_*.png]

Output in the HTML, reST, and Sphinx formats will make use of inline JavaScript code to show the frames in sequence. LaTeX employs the animate package for the same purpose. Other formats generates a file (movie_playerX.py, where X is a number) containing the HTML code with JavaScript to show and control the animation. The DocOnce document has a link to this movie viewer.

There is an alternative syntax to the Unix wildcard notation:

MOVIE: [../experiments/frame_%04d.png:0->320]

The filename is specified via printf syntax (typically the same syntax as used to generate the individual frame files). The postfix :0->320 specifies the lower and upper limit of the counter that is used in the printf specification %04d. This latter syntax must be used if the plot files reside on some web server, e.g.,

MOVIE: [http://some.where.net/experiments/frame_%04d.png:0->320]

Here is an example:

   

Recommendations. It is challenging to write robust DocOnce code with movies. The recommended formats in HTML are MP4, WebM, and Ogg. One should preferably make all three. These also works in reST and Sphinx.

The filename generation works very well in LaTeX, while true movie formats pose big challenges. On Linux systems, the media9 does not work well because a proper Flash player for embedding in the PDF file is not always available. The movie15 package also leads to problems because Acrobat Reader depends on an external player to show the files, and the correct plugins to launch players with support for a given format are not trivial to install. Even the plain href{run:file} command relies on an external player and not all formats will be supported on a given computer.

To have really robust code, use filename generators and not movie files.

MOVIE: [../experiments/frame_*.png]

One can write flexible DocOnce code and decide at run time if HTML output should have movie files or filename generators. A relevant snippet using Mako and a user-defined variable HTMLMOVIE is

% FORMAT in ("latex", "pdflatex") or HTMLMOVIE == "files":
MOVIE: [../experiments/frame_*.png]

% else:
MOVIE: [../experiments/movie.ogg]

% endif

With the -DHTMLMOVIE=files flag, animation of individual files will be performed, while any other value than files leads to use of the movie.ogg in all but LaTeX formats. In HTML one will try to load movie.mp4 (if it exists) and then movie.webm (if it exists) and then finally movie.ogg.

There is no way to control the number of frames per second in LaTeX animations based on filename generators such as myframes*.png. However, with a little auto editing in a script one can control the frame rates of the various movies. The rate is specified as 2 in lines on the form

\begin{animateinline}[controls,loop]{2} % frames: f000.png -> f098.png

Setting the rate to 12 instead for this particular movie based on the f%03.png files, the following doconce subst command does the job in a script:

doconce subst ',loop]{2}( .+: f000)' ',loop{12}\g<1>' mydoc.do.txt

Sometimes it is desired to use true movies in web formats and a figure in LaTeX, e.g., a figure with four snapshots from the movie combined into a single figure file with doconce combine_images. A preprocessor test is appropriate for this:

% if FORMAT in ("latex", "pdflatex"):

FIGURE: [myfig, frac=1] caption

% else:

MOVIE: [mymov] caption

% endif

If you encounter a large number of such if-else statements, it is advantageous to write a Mako function in Python:

<%
def figmov(figfile, movfile, caption):
    if FORMAT in ("latex", "pdflatex"):
        return "FIGURE: [%s, frac=1] %s" % (figfile, caption)
% else:
        return "MOVIE: [%s] %s" % (movfile, caption)
% endif
%>

The one can avoid if-else tests in the running code and instead write just

${figmov('myfig', 'mymov', 'caption')}

to insert a movie or figure file, depending on the output format.

Copying Computer Code from Source Files

Another type of special lines starts with @@@CODE and enables copying of computer code from a file directly into a verbatim environment, see the section Blocks of Verbatim Computer Code below.

Inserting the Output from Operating System Commands

When DocOnce is used to document computer program and results from computer code it is important to ensure that the document contains the latest version of the code and the corresponding output. The former is handled by the @@@CODE directive, while the latter has its own directive @@@OSCMD. The syntax reads

@@@OSCMD cmd

where cmd is any text that can be run in the operating system. The output is copied into the DocOnce source. For example,

@@@OSCMD python -c 'print "Hello,\nWorld!"'

results in

Terminal> python -c 'print "Hello,\nWorld!"'
Hello,
World!

There is a command-line option --os_prompt= that can be used to set the (terminal) prompt that prefixes the command:

Comments

Comments intended to be (sometimes) visible in the output document and read by readers are known as inline comments in DocOnce and described in the section Inline Tagging.

Here we address comments in the DocOnce source file that are not intended to be visible in the output document. Basic comment lines start with the hash #:

#
# Here are some comment lines that do not affect any formatting.
# These lines are converted to comments in the output format.
#

Such comment lines may have some side effects in the rst and sphinx formats because following lines are taken as part of the comment if there is not a blank line after the comment.

The Mako preprocessor supports comments that are filtered out before DocOnce starts translating the document. Such comments are very valuable as they will never interfere with the output format and they are only present in the DocOnce source. Mako has two types of comments: lines starting with a double hash ## and multiple lines enclosed by the <%doc> (beginning) and <%doc/> (closing) tags.

If you need a lot of comments in the DocOnce file, consider using Mako comments instead of the single hash, unless you want the comments to be in the source code of the output document.

To comment out or remove large sections, consider using the Preprocess preprocessor and an if-else block with a variable that is undefined (typically something like a test # #ifdef EXTRA in Preprocess, or the Mako equivalent % if EXTRA:).

Tables

Basic Syntax

A table like

time velocity acceleration
0.0 1.4186 -5.01
2.0 1.376512 11.919
4.0 1.1E+1 14.717624

is built up of pipe symbols and dashes:

|--------------------------------|
|time  | velocity | acceleration |
|--r--------r-----------r--------|
| 0.0  | 1.4186   | -5.01        |
| 2.0  | 1.376512 | 11.919       |
| 4.0  | 1.1E+1   | 14.717624    |
|--------------------------------|
Here is an example with centered headings (which is default anyway), and the numbers are left-adjusted in the first column and right-adjusted in the two others.

|--c--------c-----------c--------|
|time  | velocity | acceleration |
|--l--------r-----------r--------|
| 0.0  | 1.4186   | -5.01        |
| 2.0  | 1.376512 | 11.919       |
| 4.0  | 1.1E+1   | 14.717624    |
|--------------------------------|

Typeset result:

time velocity acceleration
0.0 1.4186 -5.01
2.0 1.376512 11.919
4.0 1.1E+1 14.717624

Pipes | can also be inserted to indicate vertical rules in LaTeX tables (they are ignored for other formats):

|--------------------------------|
|time  | velocity | acceleration |
|--l---|----r-----|-----r--------|
| 0.0  | 1.4186   | -5.01        |
| 2.0  | 1.376512 | 11.919       |
| 4.0  | 1.1E+1   | 14.717624    |
|--------------------------------|

Notice.

Quick way of creating tables

It takes some efforts to put the pipes and dashes correctly in the table format. A quick way of generating a DocOnce table is to first put the table entries in a file with comma between the entries. This is essentially a file in the famous CSV format. Data in CSV format can be transformed to DocOnce table format by the doconce csv2table utility:

Terminal> doconce csv2table somefile.csv > table.do.txt

For example, we can write a text file tmp.csv with

time, velocity, acceleration
0.0, 1.4186, -5.01
2.0, 1.376512, 11.919
4.0, 1.1E+1, 14.717624

Running doconce csv2table tmp.csv creates the table

|------c--------------c--------------c-------|
| time         | velocity     | acceleration |
|------c--------------c--------------c-------|
| 0.0          | 1.4186       | -5.01        |
| 2.0          | 1.376512     | 11.919       |
| 4.0          | 1.1E+1       | 14.717624    |
|--------------------------------------------|

If the output from doconce csv2table is redirected to a file:

Terminal> doconce csv2table data.csv > mytable.do.txt

one can easily include this file by # #include "mytable.do.txt" in the DocOnce source file. This is an efficient method for generating DocOnce tables directly from data.

Tables from DocOnce to CSV Data Files

The command-line option --tables2csv (to doconce format) makes DocOnce dump each table to CSV format in a file table_X.csv, where X is a generated table number. This feature makes it easy to load tables from DocOnce documents into spreadsheet programs for further analysis.

Lists

An unordered bullet list makes use of the * as bullet sign and is consistently indented by some chosen spaces as follows

   * item 1
   * item 2
     * subitem 1, if there are more
       lines, each line must
       be exactly intended as shown here
       (i.e., start in the same column)
     * subitem 2,
       also spans two lines
   * item 3

This list gets typeset as

In an ordered list, each item starts with an o (as the first letter in ordered):

   o item 1
   o item 2
     * subitem 1
     * subitem 2
   o item 3

resulting in

  1. item 1
  2. item 2
  3. item 3
Ordered lists cannot have an ordered sublist, i.e., the ordering applies to the outer list only.

In a description list, each item is recognized by a dash followed by a keyword followed by a colon:

   - keyword1: explanation of keyword1

   - keyword2: explanation
     of keyword2 (remember to indent properly
     if there are multiple
     lines)

The result becomes

keyword1:
explanation of keyword1
keyword2:
explanation of keyword2 (remember to indent properly if there are multiple lines)
No indentation - except in lists! DocOnce syntax is sensitive to whitespace! No lines should be indented, only lines belonging to lists. Indented lines may give strange output in some formats. Also note that extra whitespace after "item" indicators (*, o, or -) in lists may give strange behavior.

Inline Tagging

DocOnce supports tags for emphasized phrases, boldface phrases, and verbatim text (also called type writer text, for inline code), colored words, plus LaTeX/TeX inline mathematics, such as \( \nu = \sin(x) \). Links are easy to define, either with a text or just a plain http://google.com. Also a non-breaking space (to avoid linebreak), linebreak,
m-dash (as in m—dash), and horizontal rule can be specified (below).


Limitation of inline tagging. Since DocOnce applies regular expressions to recognize inline tagging, there might be cases where the tags are not correctly interpreted and translated. Fortunately, most such pitfalls are easily circumvented. The troubleshooting document shows some examples.

Emphasized Words

Emphasized text is typeset inside a pair of asterisk, and there should be no spaces between an asterisk and the emphasized text, as in

*emphasized words*

Boldface font is recognized by an underscore instead of an asterisk:

_several words in boldface_ followed by *emphasized text*.

The line above gets typeset as several words in boldface followed by emphasized text. One should only have pure text (no mathematical formulas) between the boldface or emphasize markers, and no leading or trailing blanks (with such blanks, the text will not be recognized as boldface or emphasize).

Colored text is formatted as

some text color{red}{more text in red}

Inline Verbatim Text

Verbatim text, typically used for short inline code, is typeset between backticks:

`call myroutine(a, b)` looks like a Fortran call
while `void myfunc(double *a, double *b)` must be C.

The typesetting result looks like this: call myroutine(a, b) looks like a Fortran call while void myfunc(double *a, double *b) must be C. Note that there must be no leading or trailing spaces inside the backticks.

It is recommended to have inline verbatim text on the same line in the DocOnce file, because some formats (LaTeX in combination with the ptex2tex program (but not doconce pretex)) will have problems with inline verbatim text that is split over two lines.

Notice. Watch out for mixing backticks and asterisk (i.e., verbatim and emphasized code): the DocOnce interpreter is not very smart in detecting such errors. A missing backtick will also quickly create strange output. If you suspect inline code to be the source of problems in the final format, examine the DocOnce source and the output.

Links to Web Addresses

Web addresses with links are typeset as

some URL like "Search Google": "http://google.com".

which appears as some URL like Search Google. The space after colon is optional, but it is important to enclose the link and the URL in double quotes.

To have the URL address itself as link text, put an "URL" or URL before the address enclosed in double quotes:

Click on this link: URL: "https://github.com/hplgit/doconce".

which gets rendered as Click on this link: https://github.com/hplgit/doconce.

(There is also support for lazy writing of URLs: any http or https web address with a leading space and a trailing space, comma, semi-colon, or question mark (but not period!) becomes a link with the web address as link text.)

Links to Mail Addresses

Links that launches a mail to a specified address is written as ordinary URLs, typically as

Send "mail": "mailto:hpl@simula.no"
# Alternative:
to "`hpl@simula.no`": "mailto:hpl@simula.no".

which appears as Send mail to hpl@simula.no.

Links to Local Files

Links to files ending in .txt, .html, .pdf, .py, .f, .f77, .f90, .f95, .sh, .csh, .ksh, .zsh, .c, .cpp, .cxx, .pl, and .java follows the same setup:

see the "DocOnce Manual": "manual.do.txt".

which appears as see the DocOnce Manual. However, linking to local files like this needs caution:

As a consequence, we strongly recommend that one copies the relevant files to a _static or _static-name directory and makes links to files in this directory only (name is the nickname of the DocOnce document, usually the name of the parent directory or main document). Other links to files should use the full URL. If DocOnce is used for HTML output only, then plain links to local files work fine.

If you want a link to a local source code file and have it viewed in the browser rather than being downloaded, we recommend to transform the source code file to HTML format by running pygmentize, e.g.,

Terminal> pygmentize -l bash -f html -O full,style=emacs \
          -o _static/make.sh.html subdir/make.sh

Then you can link to _static/make.sh.html instead of subdir/make.sh. Here is an example where the reader has the file available as src/myprog.py in her software and the document links to _static/myprog.py:

See the code URL:"src/myprog.py" ("view: "_static/myprog.py.html").

Links to files with other extensions are typeset with the filename as link text. The syntax consists of the keyword URL, followed by a colon, and then the filename enclosed in double quotes:

URL: "manual.html"

resulting in the link manual.html.

Quotes

Quotations employ either the emphasized font or double quotation marks. In the latter case, one should not use the character " but rather the (LaTeX-inspired) construction with double backticks and two single quotes:

This is a sentence with ``words to be quoted''.

To candidates for find double quotes that should be transferred to the above type of quotation (which is a common mistake), one can run a regular expression search like

Terminal> find . -name '*.do.txt' -exec grep -E \
          '[^("]"[A-Za-z0-9 ,]+" *[^:`)"]' {} \; -print

This search may give many false hits as double quotes are frequently used in computer code and preprocessor instructions (URLs and hyperlinks should not give hits in the above regular expressions).

Non-Breaking Space

The non-breaking space character is tilde:

Here comes a long line with a specification of a number with unit at the end,
which is an example that requires a "non-breaking space character":
"http://en.wikipedia.org/wiki/Non-breaking_space": 7.4~km is traveled
in~$7.4/5.5\approx 1.345$~s. (Computer code, where the tilde has a
meaning, as in `y = ~x`, is not affected. The non-breaking character only
works between characters, numbers, and the math dollar sign.)

This is rendered as

Here comes a long line with a specification of a number with unit at the end, which is an example that requires a non-breaking space character: 7.4 km is traveled in \( 7.4/5.5\approx 1.345 \) s. (Computer code, where the tilde has a meaning, as in y = ~x, is not affected. The non-breaking character only works between characters, numbers, and the math dollar sign.)

Horizontal rule

A horizontal rule for separating content vertically, like this:


is typeset as four or more hyphens on a single line:

---------

Em-dash

The latex, pdflatex, sphinx, html, pandoc, and ipynb formats support em-dash, indicated by three hyphens: ---. Other formats print just the three hyphens. The em-dash has two applications:

Premature optimization is the root of all evil.—Donald Knuth.
The associated DocOnce source reads

!bquote
*Premature optimization is the root of all evil.*---Donald Knuth.
!equote

En-dash

The en-dash constists of two hyphens, as in --, and can be used instead of a single hyphen to get a slightly longer hyphen (LaTeX writers are especially used to this). Common applications are

The latex, pdflatex, sphinx, html, pandoc, and ipynb formats support en-dash (basically this means that HTML output has &ndash and LaTeX output is not altered). Other formats just show the raw double hyphen.

Ampersand

An ampersand, as in Guns & Roses or Texas A&M, is written as a plain & with space(s) on both sides. Single upper case letters on each side of &, as in Texas A & M, remove the spaces and result in Texas A&M, while words on both sides of &, as in Guns & Roses, preserve the spaces: Guns & Roses. Failing to have spaces before and after & will result in wrong typesetting of the ampersand in the html, latex, and pdflatex formats. If special quoting of the ampersand is undesired, e.g., when one has inserted native LaTeX code for tables, the command-line option --no_ampersand_quote for doconce format turns off the ampersand treatment for all formats.

Footnotes

Typesetting of footnotes employs a common email or Extended Markdown syntax:

Footnotes are typeset according to the output format[^typesetting].
The syntax is optional spaces, opening bracket, hat, a footnote name
without spaces[^remedy-for-name-with-spaces], and closing bracket. The
logical name of the footnote is not used in LaTeX, HTML,
reStructuredText, or Sphinx, because these languages employ numbered
footnotes. Other formats employ the logical name.

[^typesetting]: Typesetting of the footnote depends on the format.
Plain text does nothing, LaTeX removes the definition and inserts the
footnote as part of the LaTeX text.  reStructuredText and Sphinx
employ a similar type of typesetting as Extended Markdown and DocOnce,
and in HTML we keep the same syntax, just displayed properly in HTML.

Footnotes are preferably defined after the paragraph they are used.
The definition is the footnote syntax (some optional space, bracket,
hat, name, bracket) followed by colon and a text.
A new paragraph marks the end of a footnote.

[^remedy-for-name-with-spaces]: Just put in dashes or underscores in
case of spaces.

The text above looks as follows.

Footnotes are typeset according to the output format . The syntax is optional spaces, opening bracket, hat, a footnote name without spaces , and closing bracket. The logical name of the footnote is not used in LaTeX, HTML, reStructuredText, or Sphinx, because these languages employ numbered footnotes. Other formats employ the logical name.

1: Typesetting of the footnote depends on the format. Plain text does nothing, LaTeX removes the definition and inserts the footnote as part of the LaTeX text. reStructuredText and Sphinx employ a similar type of typesetting as Extended Markdown and DocOnce, and in HTML we keep the same syntax, just displayed properly in HTML.

Footnotes are preferably defined after the paragraph they are used. The definition is the footnote syntax (some optional space, bracket, hat, name, bracket) followed by colon and a text. A new paragraph marks the end of a footnote.

2: Just put in dashes or underscores in case of spaces.

Inline Comments

DocOnce also supports inline comments in the text:

[name: comment]

where name is (e.g.) the name of the author of the comment, and comment is a plain text text. Note that there must be a space after the colon, otherwise the comment is not recognized. The name can contain upper and lower case characters, digits, single quote, + and -, as well as space. Next is an example. (hpl's comment 1: Inline comments can span several lines, if desired.)

The name and comment are visible in the output unless doconce format is run with a command-line argument --skip_inline_comments (see the section From DocOnce to Other Formats for an example). Inline comments are helpful during development of a document since different authors and readers can comment on formulations, missing points, etc. All such comments can easily be removed from the .do.txt file by doconce remove_inline_comments (see the section From DocOnce to Other Formats).

Inline comments are typeset in a simple way: boldface name, a numbering of the comment, and then the comment, all in red and in parenthesis. However, with the --latex_todonotes option, LaTeX will apply the todonotespackage to typeset the comments in very visible color boxes.

Inline Comments for Editing

Notice. The inline editing syntax in DocOnce was implemented before the invention of CriticMarkup. Now it would make sense to use the CriticMarkup syntax and associated tools. (DocOnce needs proper rendering of CriticMarkup

Inline comments can also be used to markup editing of the text. The following syntax is supported:

[add: ,]
[add: .]
[add: ;]
[del: ,]
[del: ,]
[del: .]
[del: ;]
[add: some text]
[del: some text]
[edit: some text -> some replacement for text]
[name: some text -> some replacement for text]

That is, one can add, delete, and replace text, and adding or deleting a comma, period, or semicolon leads to special typesetting where such a small edit is highlighted. Below is an example of a text with inline editing.

Originally, we have the text

First consider a quantity $Q$. Without loss of generality, we assume
$Q>0$. There are three, fundamental, basic property of $Q$.

Then, some reader wants to change this text and explicitly demonstrate what is deleted, added, and replaced (as when using track changes in Microsoft Word). The use of the add, del, and replacement construction with -> may look as follows.

First[add: ,] consider [edit: a quantity -> the flux]
[del: $Q$. Without loss of generality,
we assume] $Q>0$. There are three[del: ,] fundamental[del: , basic]
[edit: property -> properties] of $Q$. [add: These are not
important for the following discussion.]

The text gets rendered as

First, (edit 2: add comma) consider (edit 3:) a quantity the flux (edit 4:) \( Q \). Without loss of generality, we assume \( Q>0 \). There are three (edit 5: delete comma) fundamental (edit 6:) , basic (edit 7:) property properties of \( Q \). (edit 8:) These are not important for the following discussion.

Such inline comments with edits are only given special typesetting in the output formats latex, pdflatex, html, and sphinx. Otherwise, just the DocOnce syntax is shown (but that is also quite readable as edit instructions.)

The editing implied by the edit comments can be implemented in the DocOnce file by the command

Terminal> doconce apply_edit_comments mydoc.do.txt

Forced Line Breaks

By ending a line in the DocOnce file with <linebreak> the output format has a forced linebreak at this point. This can be used to typeset poems, songs (if not in a verbatim block), or the origin of quotes. Here is an example:

!bquote
*Program writing is substantially more demanding than book
writing. Why is it so? I think the main reason is that a larger
attention span is needed when working on a large computer program
than when doing other intellectual tasks.* <linebreak>
Donald Knuth cite[p. 18]{Knuth85}, computer scientist, 1938-.
!equote

is rendered as

Program writing is substantially more demanding than book writing. Why is it so? I think the main reason is that a larger attention span is needed when working on a large computer program than when doing other intellectual tasks.
Donald Knuth [1] (p. 18), computer scientist, 1938-.

The <linebreak> is a newline in LaTeX if it has preceding text, otherwise it is a \vspace{3mm}. In HTML, <linebreak> is <br>. Both constructions can be used to either force a linebreak or add vertical space.

Tip on using forced linebreaks. The <linebreak> tag is often useful in slides to avoid overfull lines in bullet lists, portion such lines into separate lines, or to insert vertical space. It can be used in admonitions too to get more space between the title and the text. Remember to have <linebreak> at the end of the line.

Inline Mathematics

Inline mathematics is written as in LaTeX, i.e., inside dollar signs. Many formats leave this syntax as it is (including the two dollar signs), hence nice math formatting is only obtained in LaTeX, HTML, MediaWiki, and Sphinx (Epytext has some inline math support that is utilized).

The following text

Let $a=\sin(x) + \cos(x)$. Then $a^2 = 2\sin(x)\cos(x)$
because $\sin^2x + \cos^2x = 1$.

is rendered as "Let \( a=\sin(x) + \cos(x) \). Then \( a^2 = 2\sin(x)\cos(x) \) because \( \sin^2x + \cos^2x = 1 \)."

Mathematical expressions in LaTeX syntax often contains special formatting commands, which may appear annoying in plain text. DocOnce therefore supports an extended inline math syntax where the writer can provide an alternative syntax suited for formats close to plain ASCII:

Here is an example on a linear system
${\bf A}{\bf x} = {\bf b}$|$Ax=b$,
where $\bf A$|$A$ is an $n\times n$|$nxn$ matrix, and
$\bf x$|$x$ and $\bf b$|$b$ are vectors of length $n$|$n$.

That is, we provide two alternative expressions, both enclosed in dollar signs and separated by a pipe symbol, the expression to the left is used in formats with LaTeX support (latex, pdflatex, html, sphinx, mwiki), while the expression to the right is used for all other formats. The above text is typeset as "Here is an example on a linear system \( {\bf A}{\bf x} = {\bf b} \), where \( \bf A \) is an \( n\times n \) matrix, and \( \bf x \) and \( \bf b \) are vectors of length \( n \)."

Cross-Referencing

References and labels are supported. The syntax is simple:

label{section:verbatim}   # defines a label
For more information we refer to Section ref{section:verbatim}.

The DocOnce label syntax is close that that of labels and cross-references in LaTeX, but note that labels cannot contain whitespace and cannot have a backslash.

When the label is placed after a section or subsection heading, the plain text, epytext, and st formats will simply replace the reference by the title of the (sub)section. All labels will become invisible, except those in math environments. (In the rst and sphinx formats, the end effect is the same, but the label and ref commands are first translated to the proper reST commands by doconce format.) In the html, ipynb, and wiki formats, labels become anchors and references become links, and with LaTeX label and ref are just equipped with backslashes so these commands work as usual in LaTeX.

Since references to sections appear differently in different formats, we provide an example.

..., we refer to Section ref{sec:theory}.

======= Basic Theory =======
label{sec:theory}

A first discovery was that 1+1 is 2.

The reference appears as follows in various output formats:

Labels and references should only be used for (sub)sections, equations, figures, and movies (since DocOnce does not support references to tables and algorithms, for instance). By the way, here is an example on referencing Figure 1. Additional references to the sections LaTeX Blocks of Mathematical Text and Macros (Newcommands) are nice to demonstrate, as well as a reference to equations, say \eqref{myeq1}-\eqref{myeq2}.

References to equations must be in parentheses! LaTeX writers who are used to \eqref{} should observe that there is only one type of reference syntax in DocOnce, ref, without backslash, and that references to an equation with label my:special:eq must feature parentheses and look like

..., see (ref{my:special:eq}).

Hyperlinks to files or web addresses are handled as explained in the section Inline Tagging.

References to equations and sections in other documents can be done by the generalized cross-referencing syntax explained in the next section. However, sometimes one wants in an HTML document or notebook to make references to equations and sections in a LaTeX textbook. This is not well handled by the generalized cross-referencing technique, but DocOnce has a special option for this feature: --replace_ref_by_latex_auxno=../book.aux will read the label and numbering information from ../book.aux and replace all references (ref) by the corresponding number found in the ../book.aux file. Sometimes one wants to use this feature for selected references. In that case, use refaux instead of ref. If there one or more refaux commands in the DocOnce source, only refaux references will be replaced by numbers from the .aux file. Otherwise, all ref commands corresponding to labels in the .aux file will be replaced. In any case, the --replace_ref_by_latex_auxno= option is needed to specify one .aux file.

Generalized Cross-Referencing

Sometimes a series of individual documents may be assembled to one large document, typically a book. In the book one wants to make cross references between chapters and sections, while these become references to external documents when the chapters (or sections) are compiled as stand-alone documents. For example, one can in a DocOnce file file1.do.txt have text like

...as shown in Section ref{sec:eqs}.

with the label sec:eqs defined in another file file2.do.txt. If file1.do.txt and file2.do.txt are combined to a single document, the reference is treated correctly, but if file1.do.txt is compiled as a single document, the label sec:eqs becomes undefined. Then one would instead write

...as shown in the document "Mathematical Equations":
"http://some.net/doc/matheqs.html" cite{math_eqs_2020}.

LaTeX has functionality for referring to labels in external documents. One must use the xr package and list external documents with a command \externaldocument{name} such that LaTeX can extract label information from the name.aux file. We are then able to write the above reference as

\externaldocument{file2}
...
...as shown in Section ref{sec:eqs} in cite{math_eqs_2020}.

and get output like "...as shown in Section 3.4 in [12]." When the numbering of sections in file2.tex changes later, the output from the shown line in file1.tex will automatically be changed if file2.aux is recently compiled (so file2.aux with the mapping from labels to section numbers is updated).

Generalized References

DocOnce mimics a generalization of the LaTeX functionality in the xr package such that one can refer to external documents in other formats than LaTeX (HTML, Sphinx, IPython notebooks, wikis, etc.). This feature is called a generalized reference and involves a reference with three values. The syntax of generalized references reads

ref[internal][cite][external]

If all references in the text internal are to labels defined in the present DocOnce document, the generalized reference becomes the text internal. If one or more references in internal are not labels present in the document and latex or pdflatex is the output format, the generalized reference becomes the text internal followed by cite, while for all other formats the text in external is used.

The example above can now be written as the generalized reference

...as shown in ref[Section ref{sec:eqs}][ in cite{math_eqs_2020}][
the document "Mathematical Equations":
"http://some.net/doc/matheqs.html" cite{math_eqs_2020}].

(Note that there must be no spaces between closing and opening brackets: ][.) When the label sec:eqs is found in the current DocOnce document, this generalized reference becomes

Section ref{sec:eqs}

If not, and latex or pdflatex is the output format, the reference becomes

Section ref{sec:eqs}] in cite{math_eqs_2020}

while in all other cases the reference becomes

the document "Mathematical Equations":
"http://some.net/doc/matheqs.html" cite{math_eqs_2020}

For the reference to a label in an external document to work in the LaTeX case it is required to list this document in the DocOnce file as

# Externaldocuments: file2

Several external documents can be listed with comma as delimiter:

# Externaldocuments: file2, file3, myfile

on a single line. The Externaldocuments comment leads to use of the xr package and insertion of \externaldocument{file2} in the .tex output file. It is a good habit to place the Externaldocument comment after the title, author, and date.

External documents must be recently compiled. When compiling DocOnce documents with generalized references to latex or pdflatex, all documents listed in the Externaldocuments comment must have been recently compiled such that their .aux files are available and updated.

Note that cleaning (doconce clean) of the directory holding an external document will destroy the .aux file, and latex or pdflatex may then complain that a file listed as \externaldocument{} has no .aux file. This is just a warning - the result is question marks in the PDF document.

Generalized References to Chapters

A reference to a chapter in a book becomes just a reference to a complete stand-alone document if chapters are compiled individually. Here is an example:

...as shown in Chapter ref{ch:model}.

This reference works fine if the present document is a book and ch:model is a label of a chapter in the book. However, if the chapter with label ch:model is compiled separately, we would rather write

...as shown in cite{math_eqs_2020}.

where math_eqs_2020 is the citation label for the external document as listed in the Publish database. Or if the output format supports hyperlinks, we would perhaps add such a link:

the document "Mathematical Equations":
"http://some.net/doc/matheqs.html" cite{math_eqs_2020}.

Such references to chapters or complete documents are very much like the previously generalized references, but written with refch instead of ref:

refch[internal][cite][external]

The only difference between refch and ref is that the former, for latex and pdflatex output, just use the cite text and not internal if the labels in the internal text are not found in the document. To be precise, the reference

...as shown in refch[Chapter ref{ch:eqs}][cite{eqs_doc_2008}][
the document "Some Equations": "http://some.net/someeqs/"].

will be

...as shown in Chapter ref{ch:eqs}.

if ch:eqs is a label defined in the present document. It becomes

...as shown in Chapter cite{eqs_doc_2008}.

if ch:eqs is not found in the present document and the output format is latex or pdflatex. In all other cases the result becomes

...as shown in
the document "Some Equations": "http://some.net/someeqs/"].

Generalized References to LaTeX Documents

It is difficult in the [external] part of the generalized reference to refer to equation numbers in an external document. If one wants to refer to a LaTeX document, say a textbook, from some HTML or notebook, then one can use the refaux reference and an .aux file as explained at the end of the previous section on Cross-Referencing. Here is one example:

From ref[(ref{eq1})][ in cite{Langtangen_2045}][
equation (refaux{eq1}) in cite{Langtangen_2045}], we realize that...

For LaTeX output, the reference to eq1 will remain, but for other formats

(refaux{eq1})

will be replaced by (say) (1.5) if we provide the option --replace_ref_by_latex_auxno=mybook.aux and mybook.aux defines label eq1 to have number 1.5. Replacing refaux by ref above will lead to hardcoding of ref{eq1} as 1.5 also in LaTeX output (which is okay, the xr package and giving Externaldocuments: mybook results in the same).

The example above is particularly relevant if one writes exercises that are to be filtered out as notebooks. The notebooks can then refer to a LaTeX book, while in the LaTeX version of the document, the exercises make references to the LaTeX book via the xr package the usual way.

Sometimes one does not want to refer to a LaTeX document in the [external] part of a generalized reference, but to a web document. Then the text must be written in a different way if one has equation or section references. For example,

From ref[(ref{eq1})][ in cite{Langtangen_2045}][
the differential equation for $u(t)$ in the section
"Setting up the model": "http://some.where.net/doc#model"
in cite{Langtangen_2045}], we realize that...

Tool for Generating Generalized References

The doconce ref_external command will read all the labels in the external documents listed in the Externaldocuments: comment and use the Publish database file of the current document (specified by BIBFILE:) to automatically generate substitution commands that translate ordinary LaTeX-style internal references to generalized references in DocOnce syntax. For example, doconce ref_external file1 will find the reference

......as shown in Section ref{sec:eqs}.

as a reference to a label sec:eqs defined in file2, grab the title of file2.do.txt, find the bibliographic data in the Publish file, and make a substitution command

doconce subst "Section\s+ref{sec:eqs}" "..." $files

where "..." is the complete generalized reference for this particular reference. In other words, with doconce ref_external one can automatically generate generalized references between, for example, chapters in a book that exist as stand-alone documents.

References to equations. Generalized references to equations work well in LaTeX, but not in other formats as one cannot resolve the equation number in the external document. It is then better to write different text using the FORMAT variable in Mako:

% if FORMAT in ("pdflatex", "latex"):
By combining ref[\eqref{eqs:g1}-\eqref{eqs:g4}][ in cite{some_doc}][
dummy] we can derive the expression ...
% else:
One can from cite{some_doc} derive the expression
% endif

The doconce ref_external tool generates an external text in case of references to equations that says "reference to specific equations (label eqs:g1 and eqs:g4) in external document "name": "link" is not recommended". One can then search for this text and make a Mako if-else rewrite as shown above.

Limited support. The doconce ref_external tool cannot correctly handle references to a range of sections like

Sections ref{mydoc:sec1}-ref{mydoc:sec2}

The automatically generated generalized references should always be manually checked and edited!

A Worked Example

Here is an example on a specific working generalized reference where the LaTeX output also has a hyperlink:

As explained in
ref[Section ref{subsec:ex}][in "Langtangen, 2012":
"http://hplgit.github.io/doconce/test/demo_testdoc.html#subsec:ex"
cite{DocOnce:test}][a "section":
"http://hplgit.github.io/doconce/test/demo_testdoc.html#subsec:ex" in
the document "A Document for Testing DocOnce":
"http://hplgit.github.io/doconce/test/demo_testdoc.html"
cite{DocOnce:test}], DocOnce documents may include tables.

With latex or pdflatex as output, this translates to

As explained in
Section ref{subsec:ex}, DocOnce documents may include tables.

if the label {subsec:ex} appears in the present DocOnce source, and otherwise

As explained in
Section ref{subsec:ex} in "Langtangen, 2012":
"http://hplgit.github.io/doconce/test/demo_testdoc.html#subsec:ex"
cite{DocOnce:test}, DocOnce documents may include tables.

The latter DocOnce code is translated to the following LaTeX code:

As explained in
Section~\ref{subsec:ex} in
\href{{http://hplgit.github.io/doconce/...}}{Langtangen, 2012}
\cite{DocOnce:test}, DocOnce documents may include tables.

In a format different from latex and pdflatex, the effective DocOnce text becomes

As explained in
a "section":
"http://hplgit.github.io/doconce/test/demo_testdoc.html#subsec:ex" in
the document "A Document for Testing DocOnce":
"http://hplgit.github.io/doconce/test/demo_testdoc.html"
cite{DocOnce:test}, DocOnce documents may include tables.

The rendered text in the current format becomes

As explained in a section in the document A Document for Testing DocOnce [2], DocOnce documents may include tables.

A complete "chapter" reference may look like

As explained in
refch[Chapter ref{ch:testdoc}]["Langtangen, 2012":
"http://hplgit.github.io/doconce/test/demo_testdoc.html"
cite{DocOnce:test}][the document
"A Document for Testing DocOnce":
"http://hplgit.github.io/doconce/test/demo_testdoc.html"
cite{DocOnce:test}], DocOnce documents may include tables.

The output now, if ch:testdoc is not a label in the document, becomes in the latex and pdflatex case

As explained in
"Langtangen, 2012":
"http://hplgit.github.io/doconce/test/demo_testdoc.html"
cite{DocOnce:test}, DocOnce documents may include tables.

That is, the internal reference Chapter ... is omitted since it is not meaningful to refer to an external document as "Chapter". The resulting rendered text in the current format becomes

As explained in the document A Document for Testing DocOnce [2], DocOnce documents may include tables.

Note that LaTeX cannot have links to local files, so a complete URL on the form http://... must be used.

Tip. Use doconce ref_external to get an overview of the external references in a file. Very often you want to rewrite the text to reduce the amount of external referencing. Remember then to compile your document before running doconce ref_external again since the command applies the compiled files to get information (tmp_preprocess_* or tmp_mako_*) if you use any of the Preprocess or Mako preprocessors.

Splitting documents into smaller parts is easy! The generalized references and the doconce ref_external are great tools for splitting an existing document into smaller parts, say one chapter into two or one book into two books. Such a split will normally create a lot of difficulties with cross-document referencing (unless you just write directly in LaTeX with the xr packcage).

Index

An index can be created for the latex, rst, and sphinx formats by the idx keyword, following a LaTeX-inspired syntax:

idx{some index entry}
idx{main entry!subentry}
idx{`verbatim_text` and more}

The exclamation mark divides a main entry and a subentry. Backquotes surround verbatim text, which is correctly transformed in a LaTeX setting to

\index{verbatim\_text@\texttt{\rm\smaller verbatim\_text and more}}

Everything related to the index simply becomes invisible in plain text, Epytext, StructuredText, HTML, and wiki formats. Note: idx commands should be inserted outside paragraphs and admonitions, not in between the text as this may cause some strange behaviour of reST and Sphinx formatting. As a recommended rule, index items are naturally placed right after section headings, before the text begins, while index items related to a paragraph should be placed above the paragraph one a separate line (and not in between the text or between the paragraph heading and the text body, although this works fine if LaTeX is the output format). For paragraphs with === heading, the index keywords should be placed above the heading.

The keywords in the index are automatically placed in a meta tag in html output such that search engines can make use of the them.

SIAM has developed a very useful document on how to create an effective index in a book.

Emojis

Emojis are specified by syntax like :sweat_smile:, followed by whitespace (blank or newline) before and after. The available emoji names are shown at http://www.emoji-cheat-sheet.com. Here is an example:

DocOnce supports emojis for the html, pdflatex, and pandoc formats. All other formats will just print the raw emoji name (like :sweat_smile:). The command-line option --no_emoji removes all emojis from the output so you can get serious in a second.

Exercises, Problems, Projects, and Examples

DocOnce has special support for four types of "exercises", named exercise, problem, project, or example. These are all typeset as special kind of subsections. Such subsections start with a subsection headline surrounded 5 = characters, and last up to the next headline or the end of the file. The headline itself must consists of the word Exercise, Problem, Project, or Example, followed by a colon and a title of the exercise, problem, or project.

Examples on Exercise Syntax

Elements in an Exercise

The next line(s) may contain a label and specification of the name of the result file(s) (if the answer to the exercise is to be handed in) and a solution file. The DocOnce code looks like this:

===== Project: Determine the Distance to the Moon =====
label{proj:moondist}
file=earth2moon.pdf
solution=eart2moon_sol.do.txt

Here goes the running text of the project....

DocOnce will recognize the exercise, problem, project, or example title, the optional label, the optional name of the answer file (file=), the optional name of the solution file (solution=), and the running text. In addition, one can add subexercise environments, starting with !bsubex and ending with !esubex, on the beginning of separate lines. Within the main exercise or a subexercise, three other environments are possible: (full) solution, (short) answer, and hints. The environments have begin-end directives !bans, !eans, !bsol, !esol, !bhint, !ehint, which all must appear on the beginning of a separate line (just as !bc and !ec).

The file= or files= command accepts a comma-separated list of filenames. Sometimes it is appropriate to skip the file extension and just give a filestem, as in file=earth2moon.

Solutions

The solution environment with !bsol and !esol allows inline solution as an alternative to the solution=... directive mentioned above (which requires that the solution is in a separate file). Comment lines are inserted so that the beginning and end of answers and solutions can be identified and removed if desired. It seems that DocOnce authors prefer !bsol and !esol over a separate solution file.

Avoid headings in solutions. Headings in solutions in exercises will appear in table of contents, but if solutions are left out of the document (using the --without_solutions option), the table of contents will be wrong and contain non-existing headings from the solution parts. Therefore, never use headings in solutions, just use paragraph headings (double underscores) if you feel the need for headings.

Avoid numbered equations in solutions. Numbered equations in solutions will influence the global numbering of equations in a document (unless exercises always come at the end of chapters). If you compile versions of the document with and without solutions, the equation numbering may differ between the versions and may cause confusion among readers.

The command line options --without_answers and --without_solutions turn off output of answers and solutions, respectively, except for examples.

Example

A full exercise set-up can be sketched as follows:

===== Exercise: Determine the Distance to the Moon =====
label{exer:moondist}
file=earth2moon.pdf

Here goes main body of text describing the exercise...

!bsubex
Subexercises are numbered a), b), etc.

!bans
Short answer to subexercise a).
!eans

!bhint
First hint to subexercise a).
!ehint

!bhint
Second hint to subexercise a).
!ehint
!esubex

!bsubex
Here goes the text for subexercise b).

!bhint
A hint for this subexercise.
!ehint

!bsol
Here goes the solution of this subexercise.
!esol
!esubex

!bremarks
At the very end of the exercise it may be appropriate to summarize
and give some perspectives. The text inside the !bremarks-!eremarks
directives is always typeset at the end of the exercise.
!eremarks

!bsol
Here goes a full solution of the entire exercise, if solutions to
subexercises are not appropraite.
!esol
!ec

Exercise versus Problem and Project

Recommended rules for using the different "exercise" types goes as follows:

Authors may use Exercise everywhere or make use of the convention in the first three points.

Modified Heading

Sometimes one does not want the heading of an exercise, problem, project, or example to contain the keyword Exercise:, Problem:, Project:, or Example:. By enclosing the keyword in braces, as in

===== {Problem}: Find a solution to a problem =====

the keyword is marked for being left out of the heading, resulting in the heading "Find a solution to a problem". In this case, there is no numbering of exercises, problems, projects, or examples, so --section_numbering=on for numbering all sections in the document may be a desired option.

Data Structure for all Exercises

The various elements of exercises are collected in a special data structure (list of dictionaries) stored in a file .mydoc.exerinfo, if mydoc.do.txt is the name of the DocOnce file. The file contains a list of dictionaries, where keys in the dictionary corresponds to elements in the exercise: filename, solution file, answer, label, list of hints, list of subexercises, closing remarks, and the main body of text.

Typesetting of Exercises

Tailored formatting of exercises in special output formats can make use of the elements in an exercise. For example, one can imagine web formats where the hints are displayed one by one when needed and where the result file can be uploaded. One can also think of mechanisms for downloading the solution file if the result file meets certain criteria. DocOnce does not yet generate such functionality in any output format, but this is an intended future feature to be impelemented.

For now, exercises, problems, projects, examples are typeset as ordinary DocOnce sections (this is the most general approach that will work for many formats). One must therefore refer to an exercise, problem, project, or example by its label, which normally will translate to the section number (in LaTeX, for instance) or a link to the title of the section. The title is typeset without any leading Exercise:, Problem:, or Project: word, so that references like

see Problem ref{...}

works well in all formats (i.e., no double Problem Problem appears).

Remark on Typesettng of Examples

Examples are not typeset similarly to exercises unless one adds the command-line option --examples_as_exercises. That is, without this option, any heading and starting with Example: makes DocOnce treat the forthcoming text as ordinary text without any interpretation of exercise-style instructions. This is usually what you want. With the command-line option --examples_as_exercises, one can use the !bsubex and !bsol commands to indicate a subproblem and a solution in an example. In this way, the typesetting of the example looks like an exercise equipped with a solution. As for exercises, the content of the example runs from the heading until the next subsection.

Examples and exercises share the same counter! If you use the --examples_as_exercises, any subsection starting with Example: will get a number for the example, and this number corresponds to a counter that is shared with Exercise, Problem, and Project. This may work well if you look at Example, Exercise, Problem, and Project as much of the same thing, basically Example is then an Exercise with solution. However, the same numbering of examples and exercises may look quite odd too, so be careful with the --examples_as_exercises option.

List of Exercises, Problems, and Projects

DocOnce also supports listing all exercises, problems, and projects with corresponding page numbers. By default, no such listing is enabled. When running doconce format pdflatex (or latex) there is a command-line option --list_of_exercises that can be set to

There is a special LaTeX environment for the exercises that one can use to further tailor the appearance of exercises (given that one edits the .tex file, preferably by a script after each doconce format command).

There is also a command doconce latex_exercise_toc that makes a special table of contents of the exercises. It truncates too long exercise headings, but by inserting # Short title: Shorter title right after the heading, the Shorter title will be used as title. Usually, this option is used if the automatic truncation of the title of the exercise fails (truncation in the middle of a formula, for instance).

Terminal> doconce latex_exercise_toc mydoc.do.txt "List of Exercises"

Try it out and compare with other methods for listing the exercises.

Numbering of Extra Equations in Solutions

A potential problem arises if you produce two versions of your document, one with solutions and one without solutions (--without_solutions), and there are numbered equations in the solutions. Equations in the text of the document after these exercises sections might then be numbered differently in the different versions you produce of the text. There are two ways out of this problem.

Book

In a book with chapters, collect all exercises at the end in a separate section "Exercises". Equations are numbered chapter-wise, and equations in solutions will not influence the numbering of equations before this exercise section. However, if some of the exercises contain numbered equations in the exercise text, numbered equations in solutions to previous exercises will influence the numbering of equations in forthcoming exercise texts. In such cases, either avoid numbered equations in a solution or in the exercise text.

Use the tag command

Equations can be given explicit numbers, completely governed by the writer. The following equation is given the number 11.5.3 and a label myeq:

!bt
\[ 1 + 1 = 2 \tag{11.5.3}
label{myeq}
!et

Equations in solutions can then be given their own numbers (here, 11.5 could be chapter 11, section 5). One can use a Mako variable to automatically assign appropriate numbers in the tag command.

Typesetting of solutions to exercises

There is an option --exercise_solution= option for controling the typesetting of solutions. Three values are legal: paragraph means that the solution just gets a paragraph heading, admon means that the solution is typeset within a notice admonition environment (!bnotice), while quote means typesetting within a quote environment.

Suggestion for HTML Bootstrap. If your choice is --exercise_solution=admon, and you want the Boostrap HTML output format, use --html_admon=bootstrap_panel. It makes the solutions stand out nicely in the text.

Suggestion for LaTeX. Sometimes the standard admons are not so suitable for typesetting solutions because they may have a background color, or they may be too eye catching. Here is a trick for LaTeX output where we 1) change the admon environment in solutions from notice to question type, 2) change the parameters in the question admon so there is no catchy background color and also a thinner frame, 3) typeset code inside solution environment without any background color. These modifications can be done by editing the mydoc.tex file in the compilation script (if mydoc is the name of the document).

We can identify all begin and end of question admons in solutions because both are tagged with the title (Solution). Looking at the mydoc.tex file, we can therefore accomplish change number 1 by the replacements

doconce replace 'notice_mdfboxadmon}[Solution.]' \
    'question_mdfboxadmon}[Solution.]' mydoc.tex
doconce replace 'end{notice_mdfboxadmon} % title: Solution.' \
    'end{question_mdfboxadmon} % title: Solution.' mydoc.tex

Change number 2 is done with a regular expression edit that replaces the complete definition of question admons:

doconce subst -s '% "question" admon.+?question_mdfboxmdframed\}' \
    '% "question" admon
\colorlet{mdfbox_question_background}{gray!5}
\\newmdenv[        % edited for solution admons in exercises
  skipabove=15pt,
  skipbelow=15pt,
  outerlinewidth=0,
  backgroundcolor=white,
  linecolor=black,
  linewidth=1pt,       % frame thickness
  frametitlebackgroundcolor=blue!5,
  frametitlerule=true,
  frametitlefont=\\normalfont\\bfseries,
  shadow=false,        % frame shadow?
  shadowsize=11pt,
  leftmargin=0,
  rightmargin=0,
  roundcorner=5,
  needspace=0pt,
]{question_mdfboxmdframed}' mydoc.tex

Change number 3 involves mapping pycod and pypro environments inside admons to pycod2 and pypro2 and define these environments by the simpler listing style greenblue, which does not have any background color. Mapping of an environment to another variant inside admons is described in LaTeX code Environments Inside Admonitions in the section Admonitions. For this purpose one employs the --latex_admon_envir_map= option.

Here is a possible command (that employs a blue background for all boxes, except pypro that gets a frame to indicate "program" and sys and dat that get a completely different typesetting):

doconce format pdflatex mydoc \
  "--latex_code_style=default:lst[style=blue1_bluegreen]@
  pypro:lst[style=blue1bar_bluegreen]@
  pypro2:lst[style=greenblue]@
  pycod2:lst[style=greenblue]@
  dat:lst[style=gray]@
  sys:vrb[frame=lines,label=\\fbox{{\tiny Terminal}},
  framesep=2.5mm,framerule=0.7pt,fontsize=\fontsize{9pt}{9pt}]" \
  --exercise_solution=admon --latex_admon_envir_map=2

(Note that we had to split the --latex_code_style= option over several lines – everything should be on one line in the compilation script!)

The --latex_admon_envir_map=2 option renames an environment like pycod inside an admon to pycod2, and the --latex_code_style= option defines the new pycod2 environment to be bluegray syntax highlighting, but no background color. Note that this applies to all question admons, but usually there are few question admons, so very often this will be a specific typesetting for solutions to exercises.

Note that when you use --exercise_solution=admon and --latex_admon=mdfbox (default), you cannot have figures with captions inside the admons, i.e., inside !bsol and !esol in the present problem setting. Only inline figures will work. Write these as FIGURE: the usual way, but drop the caption.

The current example shows how we can tailor DocOnce output to a level far beyond what DocOnce itself can output!

Extracting Selected Exercises in a Separate Document

The command

Terminal> doconce extract_exercises tmp_mako__mydoc.do.txt --filter=key1;key2

extracts all exercises in mydoc.do.txt with keywords key1 or key2 in a separate document mydoc_exer.do.txt. For example, this feature can be used to extract all exercises suitable for being published as IPython/Jupyter notebooks (and perhaps automatically graded by nbgrader). Just attach the keyword ipynb to all exercises suitable for the IPython/Jupyter notebook and run the command with --filter=ipynb. Without --filter=, the extract_exercises utility extracts all exercises such that one can publish this document separately, with or without solutions and/or short answers. By "exercise" we here mean all exercises, problems, and projects.

The numbering of the exercises in the new document follows the numbering in the original document , but you must use the --exercise_numbering=chapter option in addition if you want a chapter-based numbering when extracting exercises from a book.

It is also possible to extract examples along with exercises: just add --examples_as_exercises (to extract the examples only, equip them with a unique keyword that you can use in the --filter= option).

3: If you want a new natural numbering of the extracted exercises only, you must go into the DocOnce source file with the extracted exercises and remove the comment line that contains the labels2numbers dictionary.

Note: Instead of having all filtered exercises in one document, you may (especially for notebooks) want stand-alone documents for each exercise, see the next section.

Extracting Exercises as Stand-Alone Documents

It is sometimes convenient to publish exercises from a larger document as individual documents. With the --exercises_in_zip option, DocOnce will generate a zip file mydoc_exercises.zip (for mydoc.do.txt) with each exercise (problem, project, or example) in a separate .do.txt file. The zip archive also contains a script make.py for translating the .do.txt files to various formats. In university courses it may be attractive to give the students .tex with the exercise text such that the students can fill in the answers and extend the text to a report. Or one may distribute the exercises as IPython/Jupyter Notebook files and let the students fill in answers in the notebooks. This approach can be combined with nbgrader for automatic grading.

The zip archive also contains a file index.do.txt with a list of all the exercise files that can be published on the Internet and used for download of the exercises. The index.do.txt file contains a variable FILE_EXTENSIONS for the type of formats the exercises are available in. The user must edit make.py accordingly so the right set of formats are compiled as desired.

Note: Unzipping the archive packs out the files in a subdirectory standalone_exercises. For figure and movie references to work one needs the --figure_prefix=../ and --movie_prefix=../ options. If the exercise files are distributed to students, make sure figure files are also available (check by compiling the exercises!).

References in stand-alone exercises may not work! Exercises with references to sections in the running text of the original document cause trouble when the exercises are compiled as stand-alone documents. For LaTeX this may work if the original document is compiled in the parent directory of standalone_exercises and the corresponding .aux file is available (the exercise will in such cases make use of the # Externaldocuments: command in the file and use the xr package for cross-referencing between documents). All other formats will face problems with references to the original parent document. When missing references are encountered, a comment about the issue is inserted in the exercise file.

Naming of Exercise Files. The option --exercises_in_zip_filename=X can be used to determine the name of the exercise files. With X=logical, the logical name specified by the file= command in the exercise is used. With X=number, the filename contains the exercise number, either an absolute number (integer) like 132 or a chapter.local_number like 5.2 or B.4 (in case of an appendix), depending on the option --exercise_numbering=X, with X=absolute or X=chapter, respectively.

Example on an Exercise

The next section show the typesetting of the following exercise. For output in HTML with various Bootstrap styles, hints and answers appear as unfolded sections - one must click to open the text.

===== Exercise: Compute integrals =====
label{doconce:manual:exercise:ex}
files=integrals.py, integrals.pdf
keywords=1+1; integrals; SymPy

Use the most appropriate tools to answer the various subexercises.

!bsubex
What is 1+1?

!bhint
Your brain is a perfectly appropriate tool for this task.
!ehint
!esubex

!bans
2
!eans

!bsubex
What is the integral of $e^{-ax}\sin(wx)$?

!bhint
Assume $a$ and $w$ real.
!ehint

!bsol
This is an easy task for SymPy:

!bc pyshell
>>> import sympy as sp
>>> x = sp.symbols('x')
>>> a, w = sp.symbols('a w', real=True,positive=True)
>>> F = sp.integrate(sp.exp(-a*x)*sp.sin(w*x), x)
>>> F
-a*sin(w*x)/(a**2*exp(a*x) + w**2*exp(a*x)) -
w*cos(w*x)/(a**2*exp(a*x) + w**2*exp(a*x))
>>> sp.simplify(F)
-(a*sin(w*x) + w*cos(w*x))*exp(-a*x)/(a**2 + w**2)
!ec
That is, $-\frac{e^{- a x}}{a^{2} + w^{2}}
\left(a \sin{\left (w x \right )}
+ w \cos{\left (w x \right )}\right)$.
!esol
!esubex

!bsubex
Compute $\int_{-\infty}^1 e^{-x^4}dx$.

!bsol
Continuing the last session,

!bc pyshell
>>> F = sp.integrate(sp.exp(-x**4), (x, -sp.oo, 0))
>>> F
gamma(1/4)/4
>>> F.evalf()
0.906402477055477
!ec
!esol
!esubex

!bsubex
!bquiz
Q: What is correct about the integral $\int e^{-t^2}dt$?

Cw: The integral is the error function.
E: Almost correct, but the error function has a slightly different
definition:

!bt
\[ \hbox{erf}(x) = \frac{2}{\sqrt{2}}\int_{0}^x e^{-t^2}dt.\]
!et

Cw: It cannot be computed.
E: That would be correct if computed means ``calculated as
a closed-form formula by hand'', but the integral
$\int_a^b e^{-t^2}dt$ can be easily computed numerical
methods.

Cr: It equals

!bt
\[ \frac{\Gamma{\left(\frac{1}{4} \right)}
\gamma\left(\frac{1}{4}, x^{4}\right)}{16
\Gamma{\left(\frac{5}{4} \right)}},\]
!et
where $\Gamma(x)$ is the (upper) incomplete gamma function
and $\gamma(x)$ is the lower incomplete gamma function
(see "Wikipedia":
"http://en.wikipedia.org/wiki/Incomplete_gamma_function"
for definition).
E: This is correct, as proved by SymPy:

!bc pyshell
>>> import sympy as sym
>>> F = sym.integrate(sym.exp(-x**4), x)
>>> F
gamma(1/4)*lowergamma(1/4, x**4)/(16*gamma(5/4))
!ec

Cw: It equals the cumulative normal density function.
E: The cumulative normal density function, with mean $\mu$ and
standard deviation $\sigma$, is defined as

!bt
\[ \Phi(x) = \int_{-\infty}^x \frac{1}{\sqrt{2\pi}\sigma}
e^{-\frac{(t-\mu)^2}{2\sigma}}dt.\]
!et
!equiz
!esubex

!bremarks
This exercise demonstrates subexercise, hint, solution, short answer,
multiple-choice question (quiz), and final remark - combined with
mathematics and computer code.
!eremarks

Exercise 1: Compute integrals

Use the most appropriate tools to answer the various subexercises.

a) What is 1+1?

Hint.

Your brain is a perfectly appropriate tool for this task.

Answer.

2

b) What is the integral of \( e^{-ax}\sin(wx) \)?

Hint.

Assume \( a \) and \( w \) real.

Solution.

This is an easy task for SymPy:

>>> import sympy as sp
>>> x = sp.symbols('x')
>>> a, w = sp.symbols('a w', real=True,positive=True)
>>> F = sp.integrate(sp.exp(-a*x)*sp.sin(w*x), x)
>>> F
-a*sin(w*x)/(a**2*exp(a*x) + w**2*exp(a*x)) -
w*cos(w*x)/(a**2*exp(a*x) + w**2*exp(a*x))
>>> sp.simplify(F)
-(a*sin(w*x) + w*cos(w*x))*exp(-a*x)/(a**2 + w**2)

That is, \( -\frac{e^{- a x}}{a^{2} + w^{2}} \left(a \sin{\left (w x \right )} + w \cos{\left (w x \right )}\right) \).

c) Compute \( \int_{-\infty}^1 e^{-x^4}dx \).

Solution.

Continuing the last session,

>>> F = sp.integrate(sp.exp(-x**4), (x, -sp.oo, 0))
>>> F
gamma(1/4)/4
>>> F.evalf()
0.906402477055477

d)


Question: What is correct about the integral \( \int e^{-t^2}dt \)?

 Choice A: The integral is the error function.

Almost correct, but the error function has a slightly different definition: $$ \hbox{erf}(x) = \frac{2}{\sqrt{2}}\int_{0}^x e^{-t^2}dt.$$

 Choice B: It cannot be computed.

That would be correct if computed means "calculated as a closed-form formula by hand", but the integral \( \int_a^b e^{-t^2}dt \)? can be easily computed numerical methods.

 Choice C: It equals $$ \frac{\Gamma{\left(\frac{1}{4} \right)} \gamma\left(\frac{1}{4}, x^{4}\right)}{16 \Gamma{\left(\frac{5}{4} \right)}},$$ where \( \Gamma(x) \) is the (upper) incomplete gamma function and \( \gamma(x) \) is the lower incomplete gamma function (see Wikipedia for definition).

This is correct, as proved by SymPy:

>>> F = sp.integrate(sp.exp(-x**4), x)
>>> F
gamma(1/4)*lowergamma(1/4, x**4)/(16*gamma(5/4))

 Choice D: It equals the cumulative normal density function.

The cumulative normal density function, with mean \( \mu \) and standard deviation \( \sigma \), is defined as $$ \Phi(x) = \int_{-\infty}^x \frac{1}{\sqrt{2\pi}\sigma} e^{-\frac{(t-\mu)^2}{2\sigma}}dt.$$

Filenames: integrals.py, integrals.pdf.

Remarks

This exercise demonstrates subexercise, hint, solution, short answer, multiple-choice question (quiz), and final remark - combined with mathematics and computer code.

Other Environments

Blocks of Verbatim Computer Code

Blocks of computer code, to be typeset verbatim, must appear inside a "begin code" !bc keyword and an "end code" !ec keyword. Both keywords must be on a single line and start at the beginning of the line. Before such a code block there must be a plain sentence (at least if successful transformation to reST and ASCII-type formats is desired). For example, a code block cannot come directly after a section/paragraph heading or a table.

Here is a plain code block:

!bc dat
% Could be a comment line in some file
% And some data
1.003 1.025
2.204 1.730
3.001 1.198
!ec

which gets rendered as

% Could be a comment line in some file
% And some data
1.003 1.025
2.204 1.730
3.001 1.198

Tip: let code lines be shorter than 67 characters. Most output formats from DocOnce will get nice-looking code environments if you keep the length of lines in source code below or equal to 67 characters.

Typesetting Styles

There may be an argument after the !bc tag to specify a certain environment (in LaTeX, HTML, or Sphinx) for typesetting the verbatim code. For instance, !bc dat corresponds to the data file environment and !bc cod is typically used for a code snippet. There are some predefined environments explained below. If there is no argument specifying the environment, one assumes some plain verbatim typesetting (usually just the most plain verbatim style in LaTeX or HTML, or python in Sphinx, or if ptex2tex is used, the environment corresponds to ccq which is defined in the config file .ptex2tex.cfg).

By default, pro is used for complete programs, cod is for a code snippet. The syntax Xcod and Xpro imply a program or snippet in computer language X, wehre X can be f for Fortran, c for C, cpp for C++, sh for Unix shells, pl for Perl, m for Matlab, cy for Cython, r for Ruby, js for JavaScript, latex for LaTeX, html for HTML, and py for Python. With no X prefix, cod and pro implies Python code. The argument sys after !bc means a verbatim environment suitable for operating system commands (and output). As mentioned, dat is for a data file or print out, pyshell is for plain interactive Python shell sessions, ipy for interactive IPython sessions, and pysc turns on interactive Python code in executable HTML windows (using a SageMathCell server) while being equivalent to pycod in other formats.

Notebooks and other executable code snippets in HTML may need some initializing code before a snippet works, and if such code is not intended for being shown in the text, one can use the Xhid argument to !bc, where hid indicates a hidden snippet. It becomes executable in live code environments but is otherwise not shown.

All these definitions of the arguments after !bc can be redefined in command-line arguments for the --latex_code_style= option, the .ptex2tex.cfg configuration file for ptex2tex, and in the sphinx code-blocks comments for Sphinx, see below.

idx {-t code envir postfix}

Executable and Non-Executable Code

The ipynb and matlabnb formats create notebooks where computer code can be executed. Sometimes one wants to show code that is not to be executed, but just shown as a text block. Any code environment with a postfix -t indicates that the code is not to be executed, only displayed. For example, !bc pycod-t is a Python snippet not intended for execution. All other formats than the notebook formats ignore the -t postfix.

idx {-h code envir postfix}

Hiding code blocks in HTML

Code blocks in HTML can be visible or not by clicking on an associated button Show/hide code. The behavior is triggered by a postfix -h to the code environment, e.g., !bc pycod-h or !bc pro-h. The -h postfix has no impact on other output formats.

Customizing Code Environments Types for Sphinx

The argument after !bc can in case of Sphinx output be mapped onto any valid Pygments language for typesetting of the verbatim block by Pygments, if you do not want to rely on the defaults. This mapping takes place in an optional comment to be inserted in the DocOnce source file. Here is an example on such a comment line:

# sphinx code-blocks: pycod=python cod=fortran cppcod=c++ sys=console

Three arguments are defined: !bc pycod maps on to the Pygments style python for Python code, !bc cod maps on to the Pygments style fortran for Fortran code, !bc cppcod maps on to the Pygments style c++ for C++ code, and !bc sys maps on to the Pygments console style for terminal sessions. The same arguments would be defined in .ptex2tex.cfg or on the command line for doconce ptex2tex for how to typeset the blocks in LaTeX using various verbatim styles (Pygments can also be used in a LaTeX context).

Examples

Here is a verbatim code block with Python code (pycod style):

!bc pycod
def f(x, y):
    return x + y

# Main program
from math import pi
print 'Testing f:', f(pi, 0)
!ec

The typeset result of this block becomes

def f(x, y):
    return x + y

# Main program
from math import pi
print 'Testing f:', f(pi, 0)

Tip. The enclosing !ec tag of verbatim computer code blocks must be followed by a newline. A common error in list environments is to forget to indent the plain text surrounding the code blocks. In general, we recommend to use paragraph headings instead of list items in combination with code blocks (it usually looks better, and some common errors are naturally avoided).

And here is a C++ code snippet (cppcod style):

!bc cppcod
void myfunc(double* x, const double& myarr) {
    for (int i = 1; i < myarr.size(); i++) {
        myarr[i] = myarr[i] - x[i]*myarr[i-1]
    }
}
!ec

with the rendered result

void myfunc(double* x, const double& myarr) {
    for (int i = 1; i < myarr.size(); i++) {
        myarr[i] = myarr[i] - x[i]*myarr[i-1]
    }
}

Copying Code from Source Files

Computer code can be copied directly from a file, if desired. The syntax is then

@@@CODE myfile.f
@@@CODE myfile.f fromto: subroutine\s+test@^C\s{5}END1

The first line implies that all lines in the file myfile.f are copied into a verbatim block, typset in a !bc Xpro environment, where X is the extension of the filename, here f (i.e., the environment becomes !bc fpro and will typically lead to some Fortran-style formatting). The second line has a fromto: directive, which implies copying code between two lines in the code, typset within a !bc Xcod environment (again, X is the filename extension, implying the type of file). Note that the pro and cod arguments are only used for LaTeX and Sphinx output, all other formats will have the code typeset within a plain !bc environment.) Two regular expressions, separated by the @ sign, define the "from" and "to" lines. The "from" line is included in the verbatim block, while the "to" line is not. In the example above, we copy code from the line matching subroutine test (with as many blanks as desired between the two words) and the line matching C END1 (C followed by 5 blanks and then the text END1). The final line with the "to" text is not included in the verbatim block.

One can also specify the code environment explicitly rather than relying on the file extension:

@@@CODE somefile.py envir=X fromto: def myfunc@def yourfunc

This is the same as writing !bc X and copying parts of the somefile.py text into the DocOnce source file, followed by !ec. For example, if the lines between the myfunc and yourfunc functions actually work as a complete Python program, one could specify envir=pypro to indicate that it is a complete program that can be run as is. By default, copying a part of a .py file will lead to !bc pycod, which indicates a code snippet that normally needs additional code to be run.

Using envir=None results in a pure include of the file, without any surrounding code environment (i.e., no !bc or !ec directives around the contents of the file). The section Example: Nomenclature functionality shows an example.

Let us demonstrate the result of copying a whole file, as specified in the first line above:

C     a comment

      subroutine test()
      integer i
      real*8 r
      r = 0
      do i = 1, i
         r = r + i
      end do
      return
C     END1

      program testme
      call test()
      return

Let us then copy just a piece in the middle as indicated by the fromto: directive above:

      subroutine test()
      integer i
      real*8 r
      r = 0
      do i = 1, i
         r = r + i
      end do
      return

The fromto and from-to directives

Note that the "to" line is never copied into the DocOnce file, but the "from" line is. Sometimes it is convenient to also neglect the "from" line, a feature that is allowed by replacing fromto: by from-to ("from with minus"). This allows for copying very similar code segments throughout a file, while still distinguishing between them. Copying the second set of parameters from the text

# --- Start Example 1 ---
c = -1
A = 2
p0 = 4
simulate_and_plot(c, A, p0)
# --- End Example 1 ---

# --- Start Example 2 ---
c = -1
A = 1
p0 = 0
simulate_and_plot(c, A, p0)
# --- End Example 2 ---

is easy with

from-to: Start Example 2@End Example 2

With only fromto: this would be impossible.

Remark for those familiar with ptex2tex: The from-to syntax is slightly different from that used in ptex2tex. When transforming DocOnce to LaTeX, one first transforms the document to a .p.tex file to be treated by ptex2tex or doconce ptex2tex. However, note that the @@@CODE line is always interpreted by DocOnce first.

Remark for those familiar with the listings package in LaTeX: the listing package can copy code from files, but snippets must be specified through exact line numbers. The @@@CODE directive above works with regular expressions which are much less sensitive to edits of the source code file than the line numbers. Moreover, copy of code from file works in DocOnce across formats (HTML, Sphinx, Markdown, etc.).

The --code_prefix=text option adds a path text to the filename specified in the @@@CODE command. For example

 @@@CODE src/myfile.py

and --code_prefix=http://some.place.net, the file

http://some.place.net/src/myfile.py

will be included. If source files have a header with author, email, etc., one can remove this header by the option '--code_skip_until=# ---'. The lines up to and including (the first) # --- will then be excluded.

The regular expressions in CODE commands are non-greedy! The copying of code from file will search for the first occurrence of the "from" regular expression and then continue up to the first occurrence of the "to" expression. This means a non-greedy regular expression. A to make the expression greedy is to insert a comment statement as the "to" expression to make a special end of the snippet to be copied. For example, you have the code

compute(x)
print x
x = new(x)
compute(x)
print x
x = new(x)
compute(x)

and you want to copy from the first compute and include the second, but not the following print x statement, i.e.,

compute(x)
print x
x = new(x)
compute(x)

fromto: compute@compute will not work. The way to do this is to insert a comment statement after the desired "to" expression:

compute(x)
print x
x = new(x)
compute(x)
# dummy comment
print x
x = new(x)
compute(x)

The regular expressions are then fromto: compute@# dummy comment ("from" and "to" are actually plain text here – no use of special regular expressions).

Removing indentation in copied code

In general, it is not recommended to have a code block where all lines are indented, because in some output formats the indentation is respected (e.g., LaTeX) while others ignore it (e.g., Sphinx). Therefore, at least one line of the code should start in column 1.

Sometimes you want to copy a part of a function where all code is indented. Then you also want to remove the indentation. This is enabled by @@@CODE-X, where X is the number of blanks in the indentation to be removed. For example, @@@CODE-4 will copy the code and remove the first 4 characters from each line.

LaTeX Blocks of Mathematical Text

Blocks of mathematical text are like computer code blocks, but the opening tag is !bt (begin TeX) and the closing tag is !et. It is important that !bt and !et appear on the beginning of the line and followed by a newline.

!bt
\begin{align}
{\partial u\over\partial t} &= \nabla^2 u + f,
label{myeq1}\\
{\partial v\over\partial t} &= \nabla\cdot(q(u)\nabla v) + g.
label{myeq2}
\end{align}
!et

Here is the result: $$ \begin{align} {\partial u\over\partial t} &= \nabla^2 u + f, \label{myeq1}\\ {\partial v\over\partial t} &= \nabla\cdot(q(u)\nabla v) + g. \label{myeq2} \end{align} $$

Support of LaTeX Math in Various Output Formats

The support of LaTeX mathematics varies among the formats:

Important! The main conclusion is that for output beyond LaTeX (latex and pdflatex formats), stick to simple \[ and \] or equation and align or align* environments, and avoid referring to equations in MediaWikis.

Going from DocOnce to MS Word is most easily done by outputting in the latex format and then using the Pandoc or latex2rtf programs to translate from LaTeX to MS Word. Note that only a subset of LaTeX will be translated correctly, and mathematics is notoriously difficult and unpredicatble.

ePub and Mobi

Conversion to the ePub and Mobi formats, popular for reading on Kindle devices, can go via HTML and the Calibre tools. On Debian/GNU Linux, install sudo apt-get install calibre, and run (e.g.)

Terminal> ebook-convert mydoc.html mydoc.epub
Terminal> ebook-convert mydoc.html mydoc.mobi
Terminal> ebook-convert mydoc.pdf  mydoc.epub
Terminal> calibre mydoc.epub  # view ebook

Unfortunately, it seems that HTML and PDF documents with mathematics cannot be converted to ePub by Calibre (PDF looks strange, MathJax is not used to render formulas in HTML). For documents with mathematics, ebook-convert is a fine tool, but blocks of computer code may need reformatting in terms of shorter lines.

Pandoc is another program that can generate ePub (not tested). The ePub3 format supports mathematics via MathML. Some dicussions of this topic appear in http://tex.stackexchange.com/questions/1551/use-latex-to-produce-epub. The LaTeXML program can convert LaTeX to XML and XHTML and can be a good starting point for further conversion to ePub, but the plain latexml fails miserably on LaTeX documents generated from DocOnce (the present manual to be precise).

Sphinx is seemingly good support for compiling the document to the ePub and MOBI formats, but this author has not yet tried it out. One should definitely test Sphinx along with ebookmaker.py (see next paragraph) if the ePub or MOBI is a requested format.

What has been successfully used to convert DocOnce documents with mathematics and code to ePub is the ebookmaker.py script (use this fork by the author for a version of this script that actually works). HTML, in particular with Bootstrap styles, translates well to ePub by this script. The script requires a JSON file describing the content of the book. A typical file for a DocOnce document mydoc.do.txt that is translated to HTML and split into a series of files ._mydoc*.html goes as follows:

{
    "filename" : "mydoc",
    "title" : "Title of the document",
    "authors" : [
        {
            "name" : "Hans Petter Langtangen",
            "sort" : "Langtangen, Hans Petter"
        }
    ],
    "rights" : "Public Domain",
    "language" : "en",
    "publisher": "hpl",
    "subjects" : [ "Science" ],
    "contributors" : [
        {
            "name" : "Hans Petter Langtangen",
            "role" : "author"
        }
    ],
    "identifier" : {
        "scheme" : "URL",
        "value" : "http://somewhere.net"
    },
    "contents" : [
        {
            "type" : "text",
            "source" : "._mydoc*.html"
        }
    ],
    "toc" : {
        "depth" : 2,
        "parse" : [ "text" ],
        "generate" : {
            "title" : "Index"
        }
    }
}

Just edit this file, save it as mydoc.json and run python3 /some/path/to/epubmaker.py mydoc.json to produce mydoc.epub.

If you want to read your DocOnce document on Kindle, sending the PDF file to the email address for the device seems to be a satisfactory option.

Apple iBook Format

A converter to iBooks would be nice. In theory, ePub documents can be imported and converted to iBooks in the iBooks Author application, but ePub files created by ebookmaker.py do not translate well. The .iba files of iBooks documents can be unzipped and the XML code for the book is available in index.xml. However, the XML is undocumented and must be manipulated and filled with the contents of a DocOnce document, e.g., by first translating DocOnce to HTML, and then using BeautifulSoup to get an XHTML version of the HTML that can act as a starting point for filling the XML file for an iBook. See also other ideas.

Dealing with Mathematics in Formats without LaTeX Math Support

If the document targets formats with and without support of LaTeX mathematics, one can use the preprocessor to typeset the mathematics in two versions. After #if FORMAT in ("latex", "pdflatex", "html", "sphinx", "mwiki", "pandoc") one places LaTeX mathematics, and after #else one can write inline mathematics in a way that looks nice in plain text and wiki formats without support for mathematical typesetting. Such branching can be used with mako if-else statements alternatively:

% if FORMAT in ("latex", "pdflatex", "html", "sphinx", "mwiki", "pandoc"):
!bt
\[ \sin^2x + \cos^2x = 1,\]
!et
% else:
!bc
              sin^2(x) + cos^2(x) = 1,
!ec
% endif

Mathematics for PowerPoint/OpenOffice

If you have LaTeX mathematics written in DocOnce, it is fairly easy to generate PNG images of all mathematical formulas and equations for use with PowerPoint or OpenOffice presentations.

  1. Make a Sphinx version of the DocOnce file.
  2. Go to the Sphinx directory and load the conf.py file into a browser.
  3. Search for "math" and comment out the 'sphinx.ext.mathjax' (enabled by default) and 'matplotlib.sphinxext.mathmpl' (disabled by default) lines, and uncomment the 'sphinx.ext.pngmath' package. This is the package that generates small PNG pictures of the mathematics.
  4. Uncomment the line with pngmath_dvipng_args = and set the PNG resolution to -D 200 when the purpose is to generate mathematics pictures for slides.
  5. Run make html.
  6. Look at the HTML source file in the _build/html directory: all mathematics are in img tags with src= pointing to a PNG file and alt= pointing to the LaTeX source for the formula in question. This makes it very easy to find the PNG file that corresponding to a particular mathematical expression.

Macros (Newcommands)

DocOnce supports a type of macros via a LaTeX-style newcommand construction, but only inside mathematical expressions (inline LaTeX math or LaTeX math blocks). Newcommands are not allowed in the running text. Here is an example:

\newcommand{\beqa}{\begin{eqnarray}}
\newcommand{\eeqa}{\end{eqnarray}}
\newcommand{\ep}{\thinspace . }
\newcommand{\uvec}{\vec u}
\newcommand{\Q}{\pmb{Q}}

Notice. If you desire a newcommand for the running text, using a Mako function (written in plain Python) is much more flexible. See the section Mako Programming.

The newcommands must be defined by the standard LaTeX command

\newcommand{name}{definition}

in a separate file with name newcommands*.tex. Use of \def is ignored. Each newcommand definition must appear on a single line.

Newcommands in a file with name newcommand_replace.tex are expanded when DocOnce is translated to other formats, except for LaTeX (since LaTeX performs the expansion itself). Newcommands in files with names newcommands.tex and newcommands_keep.tex are kept unaltered when DocOnce text is translated to other formats, except for the Sphinx format. (Since Sphinx understands LaTeX math, but not newcommands if the Sphinx output is HTML, it makes most sense to expand all newcommands.) Normally, a user will put all newcommands that appear in math blocks surrounded by !bt and !et in newcommands_keep.tex to keep them unchanged, at least if they contribute to make the raw LaTeX math text easier to read in the formats that cannot render LaTeX.

Writing Guidelines (Especially for LaTeX Users!)

LaTeX writers often have their own writing habits with use of their own favorite LaTeX packages. DocOnce is a much simpler format and corresponds to writing in quite plain LaTeX and making the ascii text look nice (be careful with the use of white space!). This means that although DocOnce has borrowed a lot from LaTeX, there are a few points LaTeX writers should pay attention to. Experience shows that these points are so important that we list them before we list typical DocOnce syntax!

Any LaTeX syntax in mathematical formulas is accepted when DocOnce translates the text to LaTeX, but if output in the sphinx, pandoc, mwiki, html, or ipynb formats is also important, one should follow the rules below.