DocOnce Description¶

Demos and Documentation¶

The primary demo for what can be achieved as output from DocOnce documents regards a little scientific report that is compiled to a range of various output documents. A version of that web page contains the specific compilation commands to create each of the output demonstrations.

There is also a demo on the many different ways one can create slides.

The DocOnce tutorial (available in Sphinx, HTML, and PDF formats) has its source code in the GitHub repository for DocOnce, more precisely in the file

doc/src/tutorial/tutorial.do.txt

One can compare this source with the output in HTML, Sphinx, and PDF. The make.sh script in the same directory tells in detail how the various versions were compiled.

The DocOnce source of the current manual is also found at the GitHub repository for DocOnce, in the file

doc/src/manual/manual.do.txt

You can compare this source with the available output in HTML, Sphinx, and PDF.

Heading with title and author(s)¶

Lines starting with TITLE:, AUTHOR:, and DATE: are optional and used to identify a title of the document, the authors, and the date. The title is treated as the rest of the line, so is the date, but the author text consists of the name and associated institution(s) with the syntax

AUTHOR: name at institution1 and institution2 and institution3

The at with surrounding spaces is essential for adding information about institution(s) to the author name, and the and with surrounding spaces is essential as delimiter between different institutions. An email address can optionally be included, using the syntax

AUTHOR: name Email: somename@site.net at institution1 and institution2

Multiple authors require multiple AUTHOR: lines. All information associated with TITLE: and AUTHOR: keywords must appear on a single line. Here is an example:

TITLE: On an Ultimate Markup Language
AUTHOR: H. P. Langtangen at Center for Biomedical Computing, Simula Research Laboratory & Dept. of Informatics, Univ. of Oslo
AUTHOR: Kaare Dump Email: dump@cyb.space.com at Segfault, Cyberspace Inc.
AUTHOR: A. Dummy Author
DATE: November 9, 2016

Note how one can specify a single institution, multiple institutions (with & as separator between institutions), and no institution. In some formats (including rst and sphinx) only the author names appear. Some formats have “intelligence” in listing authors and institutions, e.g., the plain text format:

Hans Petter Langtangen [1, 2]
Kaare Dump  (dump@cyb.space.com) [3]
A. Dummy Author

[1] Center for Biomedical Computing, Simula Research Laboratory
[2] Department of Informatics, University of Oslo
[3] Segfault, Cyberspace Inc.

Similar typesetting is done for LaTeX and HTML formats.

The current date can be specified as today.

Copyright¶

A copyright notice can be given as part of the AUTHOR: command. The syntax is

{copyright,year1-year2|license}

where year1-year2 represents the year(s) and license represents the type of license (e.g., Creative Commons license) if that apply. The year and license parts can be left out. In that case, the current year is used, and no license appears. Note that the year and the license must be identical in all copyright specifications for all authors who claim copyright! (Otherwise, an error message is issued.) The {copyright...} specification can appear after the author’s name (and email address) and/or after an organization’s name (see examples below).

The syntax of the year1-year part goes as follows:

• A single year: 2012.
• A range: 2006-2010. Up to the current year is specified by 2005-present. Any attempt to use an upper limit into the future will be replaced by present, e.g., 2001-2100 becomes 2001-present.
• A single year: date, which fetches the year from the DATE: field in the DocOnce document.

The syntax of the license part is flexible:

• Any formulation can be given. For example,

• Standardized short forms for Creative Commons (CC) licenses are available. For example, CC BY, which means a CC Attribution license. BY can be replaced by BY-NC (Attribution-NonCommercial) and other abbreviations from https://creativecommons.org/licenses/. The command-line option --CC_license= is used to provide a template for embedding the spelled out name of the abbreviated CC license. By default this template reads Released under a CC %s 4.0 license, where %s is to be replaced by the license name. A common value is

How the copyright is typeset depends on the format:

• LaTeX: The copyright appears as a footer on all pages (typeset with the \fancyfoot[C] command from the fancyhdr package). The command-line option --latex_copyright=titlepages leads to a copyright statement in the footer of the titlepage and (for books) the first page of each chapter.
• Sphinx: The copyright is provided as the copyright variable in conf.py and appears in the footer according to the chosen theme.
• HTML: The copyright appears at the bottom of all pages (right before the </body> tag).
• Other formats: A copyright line is inserted after the date.

Here are some examples on specifying copyright.

AUTHOR: Joe Doe Email:joe.doe@somemail.com {copyright}
AUTHOR: Jane Doe {copyright}

Output becomes “Copyright 2015, Joe Doe, Jane Doe” (if the present year is 2015).

AUTHOR: Joe Doe {copyright,2001-present}

Output becomes “Copyright 2001-2015, Joe Doe” (if the present year is 2015).

AUTHOR: Joe Doe {copyright,2001-2010|CC BY}
AUTHOR: Jane Doe Email:jd@kk.org {copyright,2001-2010|CC BY}

Output becomes “Copyright 2001-2010, Joe Doe, Jane Doe. Released under CC Attribution 4.0 license”. One can provide the option ``

becomes “Copyright 2001-2010, Joe Doe, Jane Doe. This work is released under the Creative Commons Attribution 4.0 license.”

AUTHOR: Joe Doe {copyright} at Digital Company {copyright}
AUTHOR: Jane Doe

In this case, an author and an institution (but not the second author) hold the copyright. The output is typically “Copyright 2015, Joe Doe, Digital Company”. Below, two institutions but no authors hold the copyright:

AUTHOR: Joe Doe at Digital Company {copyright,2015|CC BY}
AUTHOR: Jane Doe at Analog Company {copyright,2015|CC BY}

The output becomes “Copyright 2015, Digital Company, Analog Company. Released under CC Attribution 4.0 license”.

Table of contents¶

A table of contents can be generated by the line

TOC: on

This line is usually placed after the DATE: line. The value off turns off the table of contents.

The depth of the table of contents is dictated by the command-line option --toc_depth=, which is 2 by default, meaning that sections and subsections are included, but not subsubsections. When making Sphinx documents, toc_depth= is a command-line option for for the doconce sphinx_dir command (and not doconce format).

Section headings¶

Section headings are recognized by being surrounded by equal signs (=) or underscores before and after the text of the headline. Different section levels are recognized by the associated number of underscores or equal signs (=):

• 9 = characters for chapters
• 7 for sections
• 5 for subsections
• 3 for subsubsections
• 2 underscores (only! - it looks best) for paragraphs (paragraph heading will be inlined)

Headings can be surrounded by as many blanks as desired, but the first = must start in column 1 and there must one blank on each side of the heading, between the heading and the = signs. Here are examples on headings:

======= Example on a Section Heading =======

The running text goes here.


===== Example on a Subsection Heading =====

The running text goes here.

=== Example on a Subsubsection Heading ===

The running text goes here.

__A Paragraph.__ The running text goes here.

Abstract¶

DocOnce also supports abstracts. The syntax is like an ordinary paragraph with heading Abstract, Summary, or Preface, but the text *must be followed by a

• section or paragraph heading
• table of contents (TOC:)
• date (DATE:)

Everything up to the first heading, table of contents, or date is taken as the abstract. For articles, the abstract is placed after the date, but before the table of contents or the first ordinary heading. For books one may insert the abstract before the date to make appear on the first page.

Here are examples on an abstract and some type of ending text (section headings, table of contents, or date).

__Abstract.__ This abstract
lasts up to the section heading.

======= Here Is the First Section Heading =======

# or

__Summary.__
This is
a summary.

Even with two paragraphs. It lasts
until the table of contents.

TOC: on

# In books we may place the summary before DATE

TITLE: Some Title
AUTHOR: Some Author

__Summary.__ Here is the backmatter
promotion text for this book, appearing
on the front page...

DATE: today

Appendix¶

Appendix is supported too: just let the heading start with “Appendix: ” This affects only latex output, where the appendix formatting is used - all other formats just leave the heading as it is written.

Figures¶

FIGURE:[filename, width=600 frac=0.8] caption

\begin{figure}[!ht] % my:fig

Terminal> doconce replace '{figure}[!ht]' '{figure}[t]' mydoc.tex

Terminal> doconce subst '{figure}[!ht] .+my:fig' \
          '{figure}[!h] % my:fig' mydoc.tex

\setcounter{topnumber}{2}
\setcounter{bottomnumber}{2}
\setcounter{totalnumber}{4}
\renewcommand{\topfraction}{0.95}
\renewcommand{\bottomfraction}{0.95}
\renewcommand{\textfraction}{0}
\renewcommand{\floatpagefraction}{0.75}
% floatpagefraction must always be less than topfraction!
\usepackage[section]{placeins}  % flush all figs before next section

The results are presented in Figure ref{myfig}.

FIGURE:[myfigfile, width=400 frac=0.8] Results for $a=2$. label{myfig}

Inline Figures¶

The figure caption is optional. If omitted, the figure appears “inline” in the text without any figure environment in LaTeX formats or HTML.

An inline figure is handy in LaTeX since it appears exactly where the FIGURE: command appears (figures with captions are encapsulated in the LaTeX figure environment and become floating objects whose placement is up to LaTeX do decide).

Tip: use linebreak to insert space around inline figures

Sometimes inline figures (FIGURE line without caption) get squeezed into the text. You can add vertical space in LaTeX and HTML by inserting several lines with <linebreak>.

A wave

FIGURE: [${FIGPREFIX}/myfig, width=500 frac=0.8] caption label{my:fig}

montage -background white -geometry 100% -tile 2x \
        file1.png file2.png ... file4.png result.png

Terminal> pdftk file1.pdf file2.pdf ... file4.pdf output tmp.pdf
Terminal> pdfnup --nup 2x2 tmp.pdf        # output in tmp-nup.pdf
Terminal> pdfcrop tmp-nup.pdf result.png  # output in FE1.png

Terminal> doconce combine_images pdf -2 fig1 fig2 fig3 fig4 fig

Terminal> doconce combine_images myfig1.png myfig2.png fig2.png

Interactive Bokeh Plots for HTML¶

Fancy interactive plots for data exploration can be made with the Bokeh library. Such plots reside in an HTML file. DocOnce will for the HTML output format detect files of this type and use the HTML code in the file to embed the plot(s) in the generated output document.

Below is a complete example on creating a grid of interactive plots where the horizontal axes are coupled to each other. Panning the graph in one plot automatically moves all the other graphs. In this way, one can scroll through a long time series simultaneously for many plots. Our demo looks as follows in a browser:

Such a figure is specified the normal way: if the HTML code for the figure is in myfig.html, write

FIGURE: [myfig] caption

Options like width are ignored for Bokeh plots, unless you have other versions of the figure (myfig.png, for instance, see the box below) where such options may be useful for some formats.

Make alternatives to Bokeh plots

Note that Bokeh plots have only meaning when DocOnce translates the document to HTML. For other formats, one needs to supply figure files that those formats can accept (PNG, PDF, etc.).

Suppose you have made a Bokeh plot in myfig.html. Either you have to embed the FIGURE command inside a preprocessor test that the FORMAT == 'html' or you must provide alternatives like myfig.png. A Bokeh plot will often have a save button that can be used to save the plot to PNG format. This can be used for Sphinx, wikis, and pdfLaTeX (although the latter would appreciate real vector graphics in a PDF plot).

The plot example above is so advanced that there is no natural counterpart in a static PNG or PDF plot.

Tip: reduce the size of Bokeh HTML files

When making Bokeh plots in Python programs, we recommend to use the mode='cdn' option in the call output_file. This argument leads to links to Bokeh tools in the resulting HTML file. Without the argument, Bokeh embeds long HTML code for its tools into the file. DocOnce issues a warning in this case and recommends the mode argument.

Note that with mode='cdn' the HTML code for the plot requires Internet access.

An interactive plot like the one shown above, stored in a file tmp.html, can be made by the code below (download file):

from __future__ import division
from builtins import range
from past.utils import old_div

def bokeh_plot(u, t, legends, u_e, t_e, I, w, t_range, filename):
    """
    Make plots for u vs t using the Bokeh library.
    u and t are lists (several experiments can be compared).
    legends contain legend strings for the various u,t pairs.
    Each plot has u vs t and the exact solution u_e vs t_e.
    """
    import numpy as np
    import bokeh.plotting as plt
    plt.output_file(filename, mode='cdn', title='Comparison')
    # Assume that all t arrays have the same range
    t_fine = np.linspace(0, t[0][-1], 1001)  # fine mesh for u_e
    tools = 'pan,wheel_zoom,box_zoom,reset,'\
            'save,box_select,lasso_select'
    u_range = [-1.2*I, 1.2*I]
    font_size = '8pt'
    p = []  # list of all individual plots
    p_ = plt.figure(
        width=300, plot_height=250, title=legends[0],
        x_axis_label='t', y_axis_label='u',
        x_range=t_range, y_range=u_range, tools=tools,
        title_text_font_size=font_size)
    p_.xaxis.axis_label_text_font_size=font_size
    p_.yaxis.axis_label_text_font_size=font_size
    p_.line(t[0], u[0], line_color='blue')
    p_.line(t_e, u_e, line_color='red', line_dash='4 4')
    p.append(p_)
    for i in range(1, len(t)):
        p_ = plt.figure(
            width=300, plot_height=250, title=legends[i],
            x_axis_label='t', y_axis_label='u',
            x_range=p[0].x_range, y_range=p[0].y_range, tools=tools,
            title_text_font_size=font_size)
        p_.xaxis.axis_label_text_font_size=font_size
        p_.yaxis.axis_label_text_font_size=font_size
        p_.line(t[i], u[i], line_color='blue')
        p_.line(t_e, u_e, line_color='red', line_dash='4 4')
        p.append(p_)

    # Arrange in grid with 3 plots per row
    grid = [[]]
    for i, p_ in enumerate(p):
        grid[-1].append(p_)
        if (i+1) % 3 == 0:
            # New row
            grid.append([])
    plot = plt.gridplot(grid, toolbar_location='left')
    plt.save(plot)
    plt.show(plot)

def demo_bokeh():
    """Plot numerical and exact solution of sinousoidal shape."""
    import numpy as np

    def u_exact(t):
        return I*np.cos(w*t)

    def u_numerical(t):
        w_tilde = (old_div(2.,dt))*np.arcsin(w*dt/2.)
        return I*np.cos(w_tilde*t)

    I = 1               # Amplitude
    w = 1.0             # Angular frequency
    P = 2*np.pi/w       # Period of signal
    num_steps_per_period = [5, 10, 20, 40, 80]
    num_periods = 40
    T = num_periods*P   # End time of signal

    t_e = np.linspace(0, T, 1001)  # Fine mesh for u_exact
    u_e = u_exact(t_e)
    u = []
    t = []
    legends = []

    # Make a series of numerical solutions with different time steps
    for n in num_steps_per_period:
        dt = old_div(P,n)  # Time step length
        t_ = np.linspace(0, T, num_periods*n+1)
        u_ = u_numerical(t_)
        u.append(u_)
        t.append(t_)
        legends.append('# time steps per period: %d' % n)
    bokeh_plot(u, t, legends, u_e, t_e,
               I=1, w=w, t_range=[0, 4*P],
               filename='tmp.html')

demo_bokeh()

import numpy as np
import matplotlib.pyplot as plt

x = np.linspace(0, 2*np.pi, 1001)
y1 = np.exp(-x)*np.sin(2*x)
y2 = np.exp(-0.5*x)*np.sin(2*x)

plt.plot(x, y1, 'r-', x, y2, 'b--')
plt.xlabel('x');  plt.ylabel('y')
# legends do not work in Bokeh
#plt.legend([r'$e^{-x}\sin 2x$', r'$e^{-\frac{1}{2}x}\sin 2x$'])
plt.title('Damped sine functions')
plt.savefig('tmp.pdf');  plt.savefig('tmp.png')

# Convert to Bokeh
import bokeh.mpl, bokeh.plotting as bpl
p = bokeh.mpl.to_bokeh(notebook=False, xkcd=False)
#p = bokeh.mpl.to_bokeh()
bpl.output_file('tmp.html', mode='cdn')
bpl.save(p)
#bpl.show(p)

plt.show()

Movies¶

Movies/videos are inserted using the MOVIE: keyword. This feature works well for the latex, html, rst, and sphinx formats. Other formats try to generate some HTML file and link to that file for showing the movie. If such a link is not appropriate and one wants a figure instead of the movie, use the preprocessor as explained in the box Recommendations below.

MOVIE: [filename, height=xxx width=yyy] possible caption

MOVIE: [mov/wave.igg, width=600] A movie in Ogg format.

MOVIE: [http://hplgit.github.io/animate/doc/pub/mov-animate/demo.ogg] Ogg movie in cyberspace.

https://github.com/n/p/blob/master/doc/mov

https://raw.githubusercontent.com/n/p/master/doc/mov/m.ogg

MOVIE: [mov/m.ogg]

--movie_prefix=https://raw.githubusercontent.com/n/p/master/doc/

MOVIE: [http://www.youtube.com/watch?v=sI2uCHH3qIM, width=420 height=315]

MOVIE: [http://vimeo.com/55562330, width=500 height=278] CFD.

Animation Based on Filename Generators¶

It is possible to define a movie from a set of files, usually plot files, which can be shown in sequence to create an animation. If the files are local on the computer, one can specify them by a simple Unix wildcard notation, as in

MOVIE: [../experiments/frame_*.png]

Output in the HTML, reST, and Sphinx formats will make use of inline JavaScript code to show the frames in sequence. LaTeX employs the animate package for the same purpose. Other formats generates a file (movie_playerX.py, where X is a number) containing the HTML code with JavaScript to show and control the animation. The DocOnce document has a link to this movie viewer.

There is an alternative syntax to the Unix wildcard notation:

MOVIE: [../experiments/frame_%04d.png:0->320]

The filename is specified via printf syntax (typically the same syntax as used to generate the individual frame files). The postfix :0->320 specifies the lower and upper limit of the counter that is used in the printf specification %04d. This latter syntax must be used if the plot files reside on some web server, e.g.,

MOVIE: [http://some.where.net/experiments/frame_%04d.png:0->320]

Here is an example:

   

Recommendations

It is challenging to write robust DocOnce code with movies. The recommended formats in HTML are MP4, WebM, and Ogg. One should preferably make all three. These also works in reST and Sphinx.

The filename generation works very well in LaTeX, while true movie formats pose big challenges. On Linux systems, the media9 does not work well because a proper Flash player for embedding in the PDF file is not always available. The movie15 package also leads to problems because Acrobat Reader depends on an external player to show the files, and the correct plugins to launch players with support for a given format are not trivial to install. Even the plain `h:ref:`run:file” command relies on an external player and not all formats will be supported on a given computer.

To have really robust code, use filename generators and not movie files.

MOVIE: [../experiments/frame_*.png]

One can write flexible DocOnce code and decide at run time if HTML output should have movie files or filename generators. A relevant snippet using Mako and a user-defined variable HTMLMOVIE is

% FORMAT in ("latex", "pdflatex") or HTMLMOVIE == "files":
MOVIE: [../experiments/frame_*.png]

% else:
MOVIE: [../experiments/movie.ogg]

% endif

With the -DHTMLMOVIE=files flag, animation of individual files will be performed, while any other value than files leads to use of the movie.ogg in all but LaTeX formats. In HTML one will try to load movie.mp4 (if it exists) and then movie.webm (if it exists) and then finally movie.ogg.

There is no way to control the number of frames per second in LaTeX animations based on filename generators such as myframes*.png. However, with a little auto editing in a script one can control the frame rates of the various movies. The rate is specified as 2 in lines on the form

\begin{animateinline}[controls,loop]{2} % frames: f000.png -> f098.png

Setting the rate to 12 instead for this particular movie based on the f%03.png files, the following doconce subst command does the job in a script:

doconce subst ',loop]{2}( .+: f000)' ',loop{12}\g<1>' mydoc.do.txt

Sometimes it is desired to use true movies in web formats and a figure in LaTeX, e.g., a figure with four snapshots from the movie combined into a single figure file with doconce combine_images. A preprocessor test is appropriate for this:

% if FORMAT in ("latex", "pdflatex"):

FIGURE: [myfig, frac=1] caption

% else:

MOVIE: [mymov] caption

% endif

If you encounter a large number of such if-else statements, it is advantageous to write a Mako function in Python:

<%
def figmov(figfile, movfile, caption):
    if FORMAT in ("latex", "pdflatex"):
        return "FIGURE: [%s, frac=1] %s" % (figfile, caption)
% else:
        return "MOVIE: [%s] %s" % (movfile, caption)
% endif
%>

The one can avoid if-else tests in the running code and instead write just

${figmov('myfig', 'mymov', 'caption')}

to insert a movie or figure file, depending on the output format.

Copying Computer Code from Source Files¶

Another type of special lines starts with @@@CODE and enables copying of computer code from a file directly into a verbatim environment, see the section Blocks of Verbatim Computer Code below.

Inserting the Output from Operating System Commands¶

When DocOnce is used to document computer program and results from computer code it is important to ensure that the document contains the latest version of the code and the corresponding output. The former is handled by the @@@CODE directive, while the latter has its own directive @@@OSCMD. The syntax reads

@@@OSCMD cmd

where cmd is any text that can be run in the operating system. The output is copied into the DocOnce source. For example,

@@@OSCMD python -c 'print "Hello,\nWorld!"'

results in

Terminal> python -c 'print "Hello,\nWorld!"'
Hello,
World!

There is a command-line option --os_prompt= that can be used to set the (terminal) prompt that prefixes the command:

• --os_prompt=None results in no prompt, just the command.
• --os_prompt=nocmd results in no prompt and no command, just the output.
• --os_prompt=Terminal> is the default setting (as in the example above).

Comments¶

Comments intended to be (sometimes) visible in the output document and read by readers are known as inline comments in DocOnce and described in the section Inline Tagging.

Here we address comments in the DocOnce source file that are not intended to be visible in the output document. Basic comment lines start with the hash #:

#
# Here are some comment lines that do not affect any formatting.
# These lines are converted to comments in the output format.
#

Such comment lines may have some side effects in the rst and sphinx formats because following lines are taken as part of the comment if there is not a blank line after the comment.

The Mako preprocessor supports comments that are filtered out before DocOnce starts translating the document. Such comments are very valuable as they will never interfere with the output format and they are only present in the DocOnce source. Mako has two types of comments: lines starting with a double hash ## and multiple lines enclosed by the <%doc> (beginning) and <%doc/> (closing) tags.

If you need a lot of comments in the DocOnce file, consider using Mako comments instead of the single hash, unless you want the comments to be in the source code of the output document.

To comment out or remove large sections, consider using the Preprocess preprocessor and an if-else block with a variable that is undefined (typically something like a test # #ifdef EXTRA in Preprocess, or the Mako equivalent % if EXTRA:).

Tables¶

   |--------------------------------|
   |time  | velocity | acceleration |
   |--r--------r-----------r--------|
   | 0.0  | 1.4186   | -5.01        |
   | 2.0  | 1.376512 | 11.919       |
   | 4.0  | 1.1E+1   | 14.717624    |
   |--------------------------------|


* The pipes and column values do not need to be aligned (but why write
  the DocOnce source in an ugly way?).

* In the line below the heading,
  one can insert the characters ``c``, ``r``, or ``l`` to specify the
  alignment of the columns (centered, right, or left, respectively).
  One can also use ``X`` for potentially very wide text that must be
  wrapped and left-adjusted (will only affect ``latex`` and ``pdflatex`` where the
  ``tabularx`` package is then used; ``X`` means ``l`` in all other formats).

* Similar character can be inserted in the line above the header to
  align the headings.

* There must be a blank line before and after the table.

* Tables are *inlined* in the text, without numbers or labels for reference.

* Some CSS files used by some HTML styles may overrule the alignment
  characters ``c``, ``r``, and ``l`` and, e.g., center all text.

* For output in LaTeX one can control certain aspects of the typesetting
  of tables: the text size (``--latex_table_format=``), the color of every
  two rows (``--latex_colored_table_rows=``), and the space between
  rows (``--latex_table_row_sep=``).

|--c--------c-----------c--------|
|time  | velocity | acceleration |
|--l--------r-----------r--------|
| 0.0  | 1.4186   | -5.01        |
| 2.0  | 1.376512 | 11.919       |
| 4.0  | 1.1E+1   | 14.717624    |
|--------------------------------|

|--------------------------------|
|time  | velocity | acceleration |
|--l---|----r-----|-----r--------|
| 0.0  | 1.4186   | -5.01        |
| 2.0  | 1.376512 | 11.919       |
| 4.0  | 1.1E+1   | 14.717624    |
|--------------------------------|

Terminal> doconce csv2table somefile.csv > table.do.txt

time, velocity, acceleration
0.0, 1.4186, -5.01
2.0, 1.376512, 11.919
4.0, 1.1E+1, 14.717624

|------c--------------c--------------c-------|
| time         | velocity     | acceleration |
|------c--------------c--------------c-------|
| 0.0          | 1.4186       | -5.01        |
| 2.0          | 1.376512     | 11.919       |
| 4.0          | 1.1E+1       | 14.717624    |
|--------------------------------------------|

Terminal> doconce csv2table data.csv > mytable.do.txt

Lists¶

An unordered bullet list makes use of the * as bullet sign and is consistently indented by some chosen spaces as follows

* item 1
* item 2
  * subitem 1, if there are more
    lines, each line must
    be exactly intended as shown here
    (i.e., start in the same column)
  * subitem 2,
    also spans two lines
* item 3

This list gets typeset as

• item 1
• item 2
  • subitem 1, if there are more lines, each line must be exactly intended as shown here (i.e., start in the same column)
  • subitem 2, also spans two lines
• item 3

In an ordered list, each item starts with an o (as the first letter in ordered):

o item 1
o item 2
  * subitem 1
  * subitem 2
o item 3

resulting in

1. item 1
2. item 2
   • subitem 1
   • subitem 2
3. item 3

Ordered lists cannot have an ordered sublist, i.e., the ordering applies to the outer list only.

In a description list, each item is recognized by a dash followed by a keyword followed by a colon:

- keyword1: explanation of keyword1

- keyword2: explanation
  of keyword2 (remember to indent properly
  if there are multiple
  lines)

The result becomes

keyword1:

explanation of keyword1

keyword2:

explanation of keyword2 (remember to indent properly if there are multiple lines)

Emphasized Words¶

Emphasized text is typeset inside a pair of asterisk, and there should be no spaces between an asterisk and the emphasized text, as in

*emphasized words*

Boldface font is recognized by an underscore instead of an asterisk:

_several words in boldface_ followed by *emphasized text*.

The line above gets typeset as several words in boldface followed by emphasized text. One should only have pure text (no mathematical formulas) between the boldface or emphasize markers, and no leading or trailing blanks (with such blanks, the text will not be recognized as boldface or emphasize).

Colored text is formatted as

some text color{red}{more text in red}

The current output format does not support colors, however.

Inline Verbatim Text¶

Verbatim text, typically used for short inline code, is typeset between backticks:

`call myroutine(a, b)` looks like a Fortran call
while `void myfunc(double *a, double *b)` must be C.

The typesetting result looks like this: call myroutine(a, b) looks like a Fortran call while void myfunc(double *a, double *b) must be C. Note that there must be no leading or trailing spaces inside the backticks.

It is recommended to have inline verbatim text on the same line in the DocOnce file, because some formats (LaTeX in combination with the ptex2tex program (but not doconce pretex)) will have problems with inline verbatim text that is split over two lines.

Links to Web Addresses¶

Web addresses with links are typeset as

some URL like "Search Google": "http://google.com".

which appears as some URL like Search Google. The space after colon is optional, but it is important to enclose the link and the URL in double quotes.

To have the URL address itself as link text, put an “URL” or URL before the address enclosed in double quotes:

Click on this link: URL: "https://github.com/hplgit/doconce".

which gets rendered as Click on this link: https://github.com/hplgit/doconce.

(There is also support for lazy writing of URLs: any http or https web address with a leading space and a trailing space, comma, semi-colon, or question mark (but not period!) becomes a link with the web address as link text.)

Links to Mail Addresses¶

Links that launches a mail to a specified address is written as ordinary URLs, typically as

Send "mail": "mailto:hpl@simula.no"
# Alternative:
to "`hpl@simula.no`": "mailto:hpl@simula.no".

which appears as Send mail to hpl@simula.no.

Links to Local Files¶

Links to files ending in .txt, .html, .pdf, .py, .f, .f77, .f90, .f95, .sh, .csh, .ksh, .zsh, .c, .cpp, .cxx, .pl, and .java follows the same setup:

see the "DocOnce Manual": "manual.do.txt".

which appears as see the DocOnce Manual. However, linking to local files like this needs caution:

• In the html format the links work well if the files are supplied with the .html with the same relative location.
• In the latex and pdflatex formats, such links in PDF files will unless the .tex file has a full URL specified through a \hyperbaseurl command and the linked files are located correctly relative to this URL. Otherwise full URL must be used in links.
• In the sphinx format, links to local files do not work unless the files reside in a _static directory (a warning is issued about this).

As a consequence, we strongly recommend that one copies the relevant files to a _static or _static-name directory and makes links to files in this directory only (name is the nickname of the DocOnce document, usually the name of the parent directory or main document). Other links to files should use the full URL. If DocOnce is used for HTML output only, then plain links to local files work fine.

If you want a link to a local source code file and have it viewed in the browser rather than being downloaded, we recommend to transform the source code file to HTML format by running pygmentize, e.g.,

Terminal> pygmentize -l bash -f html -O full,style=emacs \
          -o _static/make.sh.html subdir/make.sh

Then you can link to _static/make.sh.html instead of subdir/make.sh. Here is an example where the reader has the file available as src/myprog.py in her software and the document links to _static/myprog.py:

See the code URL:"src/myprog.py" ("view: "_static/myprog.py.html").

Links to files with other extensions are typeset with the filename as link text. The syntax consists of the keyword URL, followed by a colon, and then the filename enclosed in double quotes:

URL: "manual.html"

resulting in the link manual.html.

Quotes¶

Quotations employ either the emphasized font or double quotation marks. In the latter case, one should not use the character " but rather the (LaTeX-inspired) construction with double backticks and two single quotes:

This is a sentence with ``words to be quoted''.

To candidates for find double quotes that should be transferred to the above type of quotation (which is a common mistake), one can run a regular expression search like

Terminal> find . -name '*.do.txt' -exec grep -E \
          '[^("]"[A-Za-z0-9 ,]+" *[^:`)"]' {} \; -print

This search may give many false hits as double quotes are frequently used in computer code and preprocessor instructions (URLs and hyperlinks should not give hits in the above regular expressions).

Non-Breaking Space¶

The non-breaking space character is tilde:

Here comes a long line with a specification of a number with unit at the end,
which is an example that requires a "non-breaking space character":
"http://en.wikipedia.org/wiki/Non-breaking_space": 7.4~km is traveled
in~$7.4/5.5\approx 1.345$~s. (Computer code, where the tilde has a
meaning, as in `y = ~x`, is not affected. The non-breaking character only
works between characters, numbers, and the math dollar sign.)

This is rendered as

Here comes a long line with a specification of a number with unit at the end, which is an example that requires a non-breaking space character: 7.4 km is traveled in \(7.4/5.5\approx 1.345\) s. (Computer code, where the tilde has a meaning, as in y = ~x, is not affected. The non-breaking character only works between characters, numbers, and the math dollar sign.)

Horizontal rule¶

A horizontal rule for separating content vertically, like this:

is typeset as four or more hyphens on a single line:

---------

Em-dash¶

The latex, pdflatex, sphinx, html, pandoc, and ipynb formats support em-dash, indicated by three hyphens: ---. Other formats print just the three hyphens. The em-dash has two applications:

• as alternative to an ordinary single hyphen (with space around) in a sentence—except that there are no spaces around the em-dash (this is common, cf. Wikipedia), and
• origin of quotes (where there is no space between the end of the quote and the m-dash):

Premature optimization is the root of all evil.—Donald Knuth.

The associated DocOnce source reads

!bquote
*Premature optimization is the root of all evil.*---Donald Knuth.
!equote

En-dash¶

The en-dash constists of two hyphens, as in --, and can be used instead of a single hyphen to get a slightly longer hyphen (LaTeX writers are especially used to this). Common applications are

• hyphen in compound words as the Navier - Stokes equations (here written as Navier - Stokes)
• hyphen in number ranges like 1 - 9
• hyphen around a sentence in sentence: To be precise - but not to detailed - we should ... (here the hyphen is written as --), as an alternative to em-dash (see section above)

The latex, pdflatex, sphinx, html, pandoc, and ipynb formats support en-dash (basically this means that HTML output has &ndash and LaTeX output is not altered). Other formats just show the raw double hyphen.

Ampersand¶

An ampersand, as in Guns & Roses or Texas A&M, is written as a plain & with space(s) on both sides. Single upper case letters on each side of &, as in Texas A & M, remove the spaces and result in Texas A&M, while words on both sides of &, as in Guns & Roses, preserve the spaces: Guns & Roses. Failing to have spaces before and after & will result in wrong typesetting of the ampersand in the html, latex, and pdflatex formats. If special quoting of the ampersand is undesired, e.g., when one has inserted native LaTeX code for tables, the command-line option --no_ampersand_quote for doconce format turns off the ampersand treatment for all formats.

Footnotes¶

Typesetting of footnotes employs a common email or Extended Markdown syntax:

Footnotes are typeset according to the output format[^typesetting].
The syntax is optional spaces, opening bracket, hat, a footnote name
without spaces[^remedy-for-name-with-spaces], and closing bracket. The
logical name of the footnote is not used in LaTeX, HTML,
reStructuredText, or Sphinx, because these languages employ numbered
footnotes. Other formats employ the logical name.

[^typesetting]: Typesetting of the footnote depends on the format.
Plain text does nothing, LaTeX removes the definition and inserts the
footnote as part of the LaTeX text.  reStructuredText and Sphinx
employ a similar type of typesetting as Extended Markdown and DocOnce,
and in HTML we keep the same syntax, just displayed properly in HTML.

Footnotes are preferably defined after the paragraph they are used.
The definition is the footnote syntax (some optional space, bracket,
hat, name, bracket) followed by colon and a text.
A new paragraph marks the end of a footnote.

[^remedy-for-name-with-spaces]: Just put in dashes or underscores in
case of spaces.

The text above looks as follows.

Footnotes are typeset according to the output format [1]. The syntax is optional spaces, opening bracket, hat, a footnote name without spaces [2], and closing bracket. The logical name of the footnote is not used in LaTeX, HTML, reStructuredText, or Sphinx, because these languages employ numbered footnotes. Other formats employ the logical name.

Footnotes are preferably defined after the paragraph they are used. The definition is the footnote syntax (some optional space, bracket, hat, name, bracket) followed by colon and a text. A new paragraph marks the end of a footnote.

Inline Comments¶

DocOnce also supports inline comments in the text:

[name: comment]

where name is (e.g.) the name of the author of the comment, and comment is a plain text text. Note that there must be a space after the colon, otherwise the comment is not recognized. The name can contain upper and lower case characters, digits, single quote, + and -, as well as space. Next is an example. [hpl’s comment 1: Inline comments can span several lines, if desired.]

The name and comment are visible in the output unless doconce format is run with a command-line argument --skip_inline_comments (see the section From DocOnce to Other Formats for an example). Inline comments are helpful during development of a document since different authors and readers can comment on formulations, missing points, etc. All such comments can easily be removed from the .do.txt file by doconce remove_inline_comments (see the section From DocOnce to Other Formats).

Inline comments are typeset in a simple way: boldface name, a numbering of the comment, and then the comment, all in red and in parenthesis. However, with the --latex_todonotes option, LaTeX will apply the todonotespackage to typeset the comments in very visible color boxes.

Inline Comments for Editing¶

Inline comments can also be used to markup editing of the text. The following syntax is supported:

[add: ,]
[add: .]
[add: ;]
[del: ,]
[del: ,]
[del: .]
[del: ;]
[add: some text]
[del: some text]
[edit: some text -> some replacement for text]
[name: some text -> some replacement for text]

That is, one can add, delete, and replace text, and adding or deleting a comma, period, or semicolon leads to special typesetting where such a small edit is highlighted. Below is an example of a text with inline editing.

Originally, we have the text

First consider a quantity $Q$. Without loss of generality, we assume
$Q>0$. There are three, fundamental, basic property of $Q$.

Then, some reader wants to change this text and explicitly demonstrate what is deleted, added, and replaced (as when using track changes in Microsoft Word). The use of the add, del, and replacement construction with -> may look as follows.

First[add: ,] consider [edit: a quantity -> the flux]
[del: $Q$. Without loss of generality,
we assume] $Q>0$. There are three[del: ,] fundamental[del: , basic]
[edit: property -> properties] of $Q$. [add: These are not
important for the following discussion.]

The text gets rendered as

First, (edit 2: add comma) consider (edit 3: remove a quantity) (insert:)the flux (end insert) (edit 4: delete \(Q\). Without loss of generality, we assume) \(Q>0\). There are three (edit 5: delete comma) fundamental(edit 6: delete , basic) (edit 7: remove property) (insert:)properties (end insert) of \(Q\). (edit 8: add) These are not important for the following discussion. (end add)

Such inline comments with edits are only given special typesetting in the output formats latex, pdflatex, html, and sphinx. Otherwise, just the DocOnce syntax is shown (but that is also quite readable as edit instructions.)

The editing implied by the edit comments can be implemented in the DocOnce file by the command

Terminal> doconce apply_edit_comments mydoc.do.txt

Forced Line Breaks¶

By ending a line in the DocOnce file with <linebreak> the output format has a forced linebreak at this point. This can be used to typeset poems, songs (if not in a verbatim block), or the origin of quotes. Here is an example:

!bquote
*Program writing is substantially more demanding than book
writing. Why is it so? I think the main reason is that a larger
attention span is needed when working on a large computer program
than when doing other intellectual tasks.* <linebreak>
Donald Knuth cite[p. 18]{Knuth85}, computer scientist, 1938-.
!equote

is rendered as

Program writing is substantially more demanding than book writing. Why is it so? I think the main reason is that a larger attention span is needed when working on a large computer program than when doing other intellectual tasks.

Donald Knuth [Ref1] (p. 18), computer scientist, 1938-.

The <linebreak> is a newline in LaTeX if it has preceding text, otherwise it is a \vspace{3mm}. In HTML, <linebreak> is <br>. Both constructions can be used to either force a linebreak or add vertical space.

Inline Mathematics¶

Inline mathematics is written as in LaTeX, i.e., inside dollar signs. Many formats leave this syntax as it is (including the two dollar signs), hence nice math formatting is only obtained in LaTeX, HTML, MediaWiki, and Sphinx (Epytext has some inline math support that is utilized).

The following text

Let $a=\sin(x) + \cos(x)$. Then $a^2 = 2\sin(x)\cos(x)$
because $\sin^2x + \cos^2x = 1$.

is rendered as “Let \(a=\sin(x) + \cos(x)\). Then \(a^2 = 2\sin(x)\cos(x)\) because \(\sin^2x + \cos^2x = 1\).”

Mathematical expressions in LaTeX syntax often contains special formatting commands, which may appear annoying in plain text. DocOnce therefore supports an extended inline math syntax where the writer can provide an alternative syntax suited for formats close to plain ASCII:

Here is an example on a linear system
${\bf A}{\bf x} = {\bf b}$|$Ax=b$,
where $\bf A$|$A$ is an $n\times n$|$nxn$ matrix, and
$\bf x$|$x$ and $\bf b$|$b$ are vectors of length $n$|$n$.

That is, we provide two alternative expressions, both enclosed in dollar signs and separated by a pipe symbol, the expression to the left is used in formats with LaTeX support (latex, pdflatex, html, sphinx, mwiki), while the expression to the right is used for all other formats. The above text is typeset as “Here is an example on a linear system \({\bf A}{\bf x} = {\bf b}\), where \(\bf A\) is an \(n\times n\) matrix, and \(\bf x\) and \(\bf b\) are vectors of length \(n\).”

Cross-Referencing¶

References and labels are supported. The syntax is simple:

label{section:verbatim}   # defines a label
For more information we refer to Section ref{section:verbatim}.

The DocOnce label syntax is close that that of labels and cross-references in LaTeX, but note that labels cannot contain whitespace and cannot have a backslash.

When the label is placed after a section or subsection heading, the plain text, epytext, and st formats will simply replace the reference by the title of the (sub)section. All labels will become invisible, except those in math environments. (In the rst and sphinx formats, the end effect is the same, but the label and ref commands are first translated to the proper reST commands by doconce format.) In the html, ipynb, and wiki formats, labels become anchors and references become links, and with LaTeX label and ref are just equipped with backslashes so these commands work as usual in LaTeX.

Since references to sections appear differently in different formats, we provide an example.

..., we refer to Section ref{sec:theory}.

======= Basic Theory =======
label{sec:theory}

A first discovery was that 1+1 is 2.

The reference appears as follows in various output formats:

• latex and pdflatex: “we refer to Section 2.3” with LaTeX code

• html: ``we refer to the section Basic Theory” with the HTML code

• rst, sphinx: “we refer to the section Basic Theory”
• plain: “we refer to the section “Basic Theory”“
• ipynb: “we refer to the section Basic Theory” with Markdown code

• mwiki: “we refer to the section Basic Theory” with MediaWiki code

Labels and references should only be used for (sub)sections, equations, figures, and movies (since DocOnce does not support references to tables and algorithms, for instance). By the way, here is an example on referencing Figure A wave. Additional references to the sections LaTeX Blocks of Mathematical Text and Macros (Newcommands) are nice to demonstrate, as well as a reference to equations, say (1)-(2).

..., see (ref{my:special:eq}).

Hyperlinks to files or web addresses are handled as explained in the section Inline Tagging.

References to equations and sections in other documents can be done by the generalized cross-referencing syntax explained in the next section. However, sometimes one wants in an HTML document or notebook to make references to equations and sections in a LaTeX textbook. This is not well handled by the generalized cross-referencing technique, but DocOnce has a special option for this feature: --replace_ref_by_latex_auxno=../book.aux will read the label and numbering information from ../book.aux and replace all references (ref) by the corresponding number found in the ../book.aux file. Sometimes one wants to use this feature for selected references. In that case, use refaux instead of ref. If there one or more refaux commands in the DocOnce source, only refaux references will be replaced by numbers from the .aux file. Otherwise, all ref commands corresponding to labels in the .aux file will be replaced. In any case, the --replace_ref_by_latex_auxno= option is needed to specify one .aux file.

Generalized Cross-Referencing¶

Sometimes a series of individual documents may be assembled to one large document, typically a book. In the book one wants to make cross references between chapters and sections, while these become references to external documents when the chapters (or sections) are compiled as stand-alone documents. For example, one can in a DocOnce file file1.do.txt have text like

...as shown in Section ref{sec:eqs}.

with the label sec:eqs defined in another file file2.do.txt. If file1.do.txt and file2.do.txt are combined to a single document, the reference is treated correctly, but if file1.do.txt is compiled as a single document, the label sec:eqs becomes undefined. Then one would instead write

...as shown in the document "Mathematical Equations":
"http://some.net/doc/matheqs.html" cite{math_eqs_2020}.

LaTeX has functionality for referring to labels in external documents. One must use the xr package and list external documents with a command \externaldocument{name} such that LaTeX can extract label information from the name.aux file. We are then able to write the above reference as

\externaldocument{file2}
...
...as shown in Section ref{sec:eqs} in cite{math_eqs_2020}.

and get output like ”...as shown in Section 3.4 in [12].” When the numbering of sections in file2.tex changes later, the output from the shown line in file1.tex will automatically be changed if file2.aux is recently compiled (so file2.aux with the mapping from labels to section numbers is updated).

ref[internal][cite][external]

...as shown in ref[Section ref{sec:eqs}][ in cite{math_eqs_2020}][
the document "Mathematical Equations":
"http://some.net/doc/matheqs.html" cite{math_eqs_2020}].

Section ref{sec:eqs}

Section ref{sec:eqs}] in cite{math_eqs_2020}

the document "Mathematical Equations":
"http://some.net/doc/matheqs.html" cite{math_eqs_2020}

# Externaldocuments: file2

# Externaldocuments: file2, file3, myfile

...as shown in Chapter ref{ch:model}.

...as shown in cite{math_eqs_2020}.

the document "Mathematical Equations":
"http://some.net/doc/matheqs.html" cite{math_eqs_2020}.

refch[internal][cite][external]

...as shown in refch[Chapter ref{ch:eqs}][cite{eqs_doc_2008}][
the document "Some Equations": "http://some.net/someeqs/"].

...as shown in Chapter ref{ch:eqs}.

...as shown in Chapter cite{eqs_doc_2008}.

...as shown in
the document "Some Equations": "http://some.net/someeqs/"].

From ref[(ref{eq1})][ in cite{Langtangen_2045}][
equation (refaux{eq1}) in cite{Langtangen_2045}], we realize that...

(refaux{eq1})

From ref[(ref{eq1})][ in cite{Langtangen_2045}][
the differential equation for $u(t)$ in the section
"Setting up the model": "http://some.where.net/doc#model"
in cite{Langtangen_2045}], we realize that...

......as shown in Section ref{sec:eqs}.

doconce subst "Section\s+ref{sec:eqs}" "..." $files

% if FORMAT in ("pdflatex", "latex"):
By combining ref[(ref{eqs:g1})-(ref{eqs:g4})][ in cite{some_doc}][
dummy] we can derive the expression ...
% else:
One can from cite{some_doc} derive the expression
% endif

Sections ref{mydoc:sec1}-ref{mydoc:sec2}

As explained in
ref[Section ref{subsec:ex}][in "Langtangen, 2012":
"http://hplgit.github.io/doconce/test/demo_testdoc.html#subsec:ex"
cite{DocOnce:test}][a "section":
"http://hplgit.github.io/doconce/test/demo_testdoc.html#subsec:ex" in
the document "A Document for Testing DocOnce":
"http://hplgit.github.io/doconce/test/demo_testdoc.html"
cite{DocOnce:test}], DocOnce documents may include tables.

As explained in
Section ref{subsec:ex}, DocOnce documents may include tables.

As explained in
Section ref{subsec:ex} in "Langtangen, 2012":
"http://hplgit.github.io/doconce/test/demo_testdoc.html#subsec:ex"
cite{DocOnce:test}, DocOnce documents may include tables.

As explained in
Section~\ref{subsec:ex} in
\href{{http://hplgit.github.io/doconce/...}}{Langtangen, 2012}
\cite{DocOnce:test}, DocOnce documents may include tables.

As explained in
a "section":
"http://hplgit.github.io/doconce/test/demo_testdoc.html#subsec:ex" in
the document "A Document for Testing DocOnce":
"http://hplgit.github.io/doconce/test/demo_testdoc.html"
cite{DocOnce:test}, DocOnce documents may include tables.

As explained in
refch[Chapter ref{ch:testdoc}]["Langtangen, 2012":
"http://hplgit.github.io/doconce/test/demo_testdoc.html"
cite{DocOnce:test}][the document
"A Document for Testing DocOnce":
"http://hplgit.github.io/doconce/test/demo_testdoc.html"
cite{DocOnce:test}], DocOnce documents may include tables.

As explained in
"Langtangen, 2012":
"http://hplgit.github.io/doconce/test/demo_testdoc.html"
cite{DocOnce:test}, DocOnce documents may include tables.

Index¶

An index can be created for the latex, rst, and sphinx formats by the idx keyword, following a LaTeX-inspired syntax:

idx{some index entry}
idx{main entry!subentry}
idx{`verbatim_text` and more}

The exclamation mark divides a main entry and a subentry. Backquotes surround verbatim text, which is correctly transformed in a LaTeX setting to

\index{verbatim\_text@\texttt{\rm\smaller verbatim\_text and more}}

Everything related to the index simply becomes invisible in plain text, Epytext, StructuredText, HTML, and wiki formats. Note: idx commands should be inserted outside paragraphs and admonitions, not in between the text as this may cause some strange behaviour of reST and Sphinx formatting. As a recommended rule, index items are naturally placed right after section headings, before the text begins, while index items related to a paragraph should be placed above the paragraph one a separate line (and not in between the text or between the paragraph heading and the text body, although this works fine if LaTeX is the output format). For paragraphs with === heading, the index keywords should be placed above the heading.

The keywords in the index are automatically placed in a meta tag in html output such that search engines can make use of the them.

SIAM has developed a very useful document on how to create an effective index in a book.

Emojis¶

Emojis are specified by syntax like :sweat_smile:, followed by whitespace (blank or newline) before and after. The available emoji names are shown at http://www.emoji-cheat-sheet.com. Here is an example: :sweat_smile:

DocOnce supports emojis for the html, pdflatex, and pandoc formats. All other formats will just print the raw emoji name (like :sweat_smile:). The command-line option --no_emoji removes all emojis from the output so you can get serious in a second.

Examples on Exercise Syntax¶

===== Project: Determine the Distance to the Moon =====
label{proj:moondist}
file=earth2moon.pdf
solution=eart2moon_sol.do.txt

Here goes the running text of the project....

===== Exercise: Determine the Distance to the Moon =====
label{exer:moondist}
file=earth2moon.pdf

Here goes main body of text describing the exercise...

!bsubex
Subexercises are numbered a), b), etc.

!bans
Short answer to subexercise a).
!eans

!bhint
First hint to subexercise a).
!ehint

!bhint
Second hint to subexercise a).
!ehint
!esubex

!bsubex
Here goes the text for subexercise b).

!bhint
A hint for this subexercise.
!ehint

!bsol
Here goes the solution of this subexercise.
!esol
!esubex

!bremarks
At the very end of the exercise it may be appropriate to summarize
and give some perspectives. The text inside the !bremarks-!eremarks
directives is always typeset at the end of the exercise.
!eremarks

!bsol
Here goes a full solution of the entire exercise, if solutions to
subexercises are not appropraite.
!esol
!ec

===== {Problem}: Find a solution to a problem =====

Typesetting of Exercises¶

Tailored formatting of exercises in special output formats can make use of the elements in an exercise. For example, one can imagine web formats where the hints are displayed one by one when needed and where the result file can be uploaded. One can also think of mechanisms for downloading the solution file if the result file meets certain criteria. DocOnce does not yet generate such functionality in any output format, but this is an intended future feature to be impelemented.

For now, exercises, problems, projects, examples are typeset as ordinary DocOnce sections (this is the most general approach that will work for many formats). One must therefore refer to an exercise, problem, project, or example by its label, which normally will translate to the section number (in LaTeX, for instance) or a link to the title of the section. The title is typeset without any leading Exercise:, Problem:, or Project: word, so that references like

see Problem ref{...}

works well in all formats (i.e., no double Problem Problem appears).

List of Exercises, Problems, and Projects¶

DocOnce also supports listing all exercises, problems, and projects with corresponding page numbers. By default, no such listing is enabled. When running doconce format pdflatex (or latex) there is a command-line option --list_of_exercises that can be set to

• toc: include exercises, problems, and projects as part of the table of contents
• loe: make a separate list of exercises, problems, and projects, which appears after the table of contents

There is a special LaTeX environment for the exercises that one can use to further tailor the appearance of exercises (given that one edits the .tex file, preferably by a script after each doconce format command).

There is also a command doconce latex_exercise_toc that makes a special table of contents of the exercises. It truncates too long exercise headings, but by inserting # Short title: Shorter title right after the heading, the Shorter title will be used as title. Usually, this option is used if the automatic truncation of the title of the exercise fails (truncation in the middle of a formula, for instance).

Terminal> doconce latex_exercise_toc mydoc.do.txt "List of Exercises"

Try it out and compare with other methods for listing the exercises.

Numbering of Extra Equations in Solutions¶

A potential problem arises if you produce two versions of your document, one with solutions and one without solutions (--without_solutions), and there are numbered equations in the solutions. Equations in the text of the document after these exercises sections might then be numbered differently in the different versions you produce of the text. There are two ways out of this problem.

!bt
\[ 1 + 1 = 2 \tag{11.5.3}
label{myeq}
!et

Typesetting of solutions to exercises¶

There is an option --exercise_solution= option for controling the typesetting of solutions. Three values are legal: paragraph means that the solution just gets a paragraph heading, admon means that the solution is typeset within a notice admonition environment (!bnotice), while quote means typesetting within a quote environment.

Suggestion for HTML Bootstrap. If your choice is --exercise_solution=admon, and you want the Boostrap HTML output format, use --html_admon=bootstrap_panel. It makes the solutions stand out nicely in the text.

Suggestion for LaTeX. Sometimes the standard admons are not so suitable for typesetting solutions because they may have a background color, or they may be too eye catching. Here is a trick for LaTeX output where we 1) change the admon environment in solutions from notice to question type, 2) change the parameters in the question admon so there is no catchy background color and also a thinner frame, 3) typeset code inside solution environment without any background color. These modifications can be done by editing the mydoc.tex file in the compilation script (if mydoc is the name of the document).

We can identify all begin and end of question admons in solutions because both are tagged with the title (Solution). Looking at the mydoc.tex file, we can therefore accomplish change number 1 by the replacements

doconce replace 'notice_mdfboxadmon}[Solution.]' \
    'question_mdfboxadmon}[Solution.]' mydoc.tex
doconce replace 'end{notice_mdfboxadmon} % title: Solution.' \
    'end{question_mdfboxadmon} % title: Solution.' mydoc.tex

Change number 2 is done with a regular expression edit that replaces the complete definition of question admons:

doconce subst -s '% "question" admon.+?question_mdfboxmdframed\}' \
    '% "question" admon
\colorlet{mdfbox_question_background}{gray!5}
\\newmdenv[        % edited for solution admons in exercises
  skipabove=15pt,
  skipbelow=15pt,
  outerlinewidth=0,
  backgroundcolor=white,
  linecolor=black,
  linewidth=1pt,       % frame thickness
  frametitlebackgroundcolor=blue!5,
  frametitlerule=true,
  frametitlefont=\\normalfont\\bfseries,
  shadow=false,        % frame shadow?
  shadowsize=11pt,
  leftmargin=0,
  rightmargin=0,
  roundcorner=5,
  needspace=0pt,
]{question_mdfboxmdframed}' mydoc.tex

Change number 3 involves mapping pycod and pypro environments inside admons to pycod2 and pypro2 and define these environments by the simpler listing style greenblue, which does not have any background color. Mapping of an environment to another variant inside admons is described in LaTeX code Environments Inside Admonitions in the section Admonitions. For this purpose one employs the --latex_admon_envir_map= option.

Here is a possible command (that employs a blue background for all boxes, except pypro that gets a frame to indicate “program” and sys and dat that get a completely different typesetting):

doconce format pdflatex mydoc \
  "--latex_code_style=default:lst[style=blue1_bluegreen]@
  pypro:lst[style=blue1bar_bluegreen]@
  pypro2:lst[style=greenblue]@
  pycod2:lst[style=greenblue]@
  dat:lst[style=gray]@
  sys:vrb[frame=lines,label=\\fbox{{\tiny Terminal}},
  framesep=2.5mm,framerule=0.7pt,fontsize=\fontsize{9pt}{9pt}]" \
  --exercise_solution=admon --latex_admon_envir_map=2

(Note that we had to split the --latex_code_style= option over several lines - everything should be on one line in the compilation script!)

The --latex_admon_envir_map=2 option renames an environment like pycod inside an admon to pycod2, and the --latex_code_style= option defines the new pycod2 environment to be bluegray syntax highlighting, but no background color. Note that this applies to all question admons, but usually there are few question admons, so very often this will be a specific typesetting for solutions to exercises.

Note that when you use --exercise_solution=admon and --latex_admon=mdfbox (default), you cannot have figures with captions inside the admons, i.e., inside !bsol and !esol in the present problem setting. Only inline figures will work. Write these as FIGURE: the usual way, but drop the caption.

The current example shows how we can tailor DocOnce output to a level far beyond what DocOnce itself can output!

Extracting Selected Exercises in a Separate Document¶

The command

Terminal> doconce extract_exercises tmp_mako__mydoc.do.txt --filter=key1;key2

extracts all exercises in mydoc.do.txt with keywords key1 or key2 in a separate document mydoc_exer.do.txt. For example, this feature can be used to extract all exercises suitable for being published as IPython/Jupyter notebooks (and perhaps automatically graded by nbgrader). Just attach the keyword ipynb to all exercises suitable for the IPython/Jupyter notebook and run the command with --filter=ipynb. Without --filter=, the extract_exercises utility extracts all exercises such that one can publish this document separately, with or without solutions and/or short answers. By “exercise” we here mean all exercises, problems, and projects.

The numbering of the exercises in the new document follows the numbering in the original document [3], but you must use the --exercise_numbering=chapter option in addition if you want a chapter-based numbering when extracting exercises from a book.

It is also possible to extract examples along with exercises: just add --examples_as_exercises (to extract the examples only, equip them with a unique keyword that you can use in the --filter= option).

Note: Instead of having all filtered exercises in one document, you may (especially for notebooks) want stand-alone documents for each exercise, see the next section.

Extracting Exercises as Stand-Alone Documents¶

It is sometimes convenient to publish exercises from a larger document as individual documents. With the --exercises_in_zip option, DocOnce will generate a zip file mydoc_exercises.zip (for mydoc.do.txt) with each exercise (problem, project, or example) in a separate .do.txt file. The zip archive also contains a script make.py for translating the .do.txt files to various formats. In university courses it may be attractive to give the students .tex with the exercise text such that the students can fill in the answers and extend the text to a report. Or one may distribute the exercises as IPython/Jupyter Notebook files and let the students fill in answers in the notebooks. This approach can be combined with nbgrader for automatic grading.

The zip archive also contains a file index.do.txt with a list of all the exercise files that can be published on the Internet and used for download of the exercises. The index.do.txt file contains a variable FILE_EXTENSIONS for the type of formats the exercises are available in. The user must edit make.py accordingly so the right set of formats are compiled as desired.

Note: Unzipping the archive packs out the files in a subdirectory standalone_exercises. For figure and movie references to work one needs the --figure_prefix=../ and --movie_prefix=../ options. If the exercise files are distributed to students, make sure figure files are also available (check by compiling the exercises!).

Naming of Exercise Files. The option --exercises_in_zip_filename=X can be used to determine the name of the exercise files. With X=logical, the logical name specified by the file= command in the exercise is used. With X=number, the filename contains the exercise number, either an absolute number (integer) like 132 or a chapter.local_number like 5.2 or B.4 (in case of an appendix), depending on the option --exercise_numbering=X, with X=absolute or X=chapter, respectively.

Example on an Exercise¶

The next section show the typesetting of the following exercise. For output in HTML with various Bootstrap styles, hints and answers appear as unfolded sections - one must click to open the text.

===== Exercise: Compute integrals =====
label{doconce:manual:exercise:ex}
files=integrals.py, integrals.pdf
keywords=1+1; integrals; SymPy

Use the most appropriate tools to answer the various subexercises.

!bsubex
What is 1+1?

!bhint
Your brain is a perfectly appropriate tool for this task.
!ehint
!esubex

!bans
2
!eans

!bsubex
What is the integral of $e^{-ax}\sin(wx)$?

!bhint
Assume $a$ and $w$ real.
!ehint

!bsol
This is an easy task for SymPy:

!bc pyshell
>>> import sympy as sp
>>> x = sp.symbols('x')
>>> a, w = sp.symbols('a w', real=True,positive=True)
>>> F = sp.integrate(sp.exp(-a*x)*sp.sin(w*x), x)
>>> F
-a*sin(w*x)/(a**2*exp(a*x) + w**2*exp(a*x)) -
w*cos(w*x)/(a**2*exp(a*x) + w**2*exp(a*x))
>>> sp.simplify(F)
-(a*sin(w*x) + w*cos(w*x))*exp(-a*x)/(a**2 + w**2)
!ec
That is, $-\frac{e^{- a x}}{a^{2} + w^{2}}
\left(a \sin{\left (w x \right )}
+ w \cos{\left (w x \right )}\right)$.
!esol
!esubex

!bsubex
Compute $\int_{-\infty}^1 e^{-x^4}dx$.

!bsol
Continuing the last session,

!bc pyshell
>>> F = sp.integrate(sp.exp(-x**4), (x, -sp.oo, 0))
>>> F
gamma(1/4)/4
>>> F.evalf()
0.906402477055477
!ec
!esol
!esubex

!bsubex
!bquiz
Q: What is correct about the integral $\int e^{-t^2}dt$?

Cw: The integral is the error function.
E: Almost correct, but the error function has a slightly different
definition:

!bt
\[ \hbox{erf}(x) = \frac{2}{\sqrt{2}}\int_{0}^x e^{-t^2}dt.\]
!et

Cw: It cannot be computed.
E: That would be correct if computed means ``calculated as
a closed-form formula by hand'', but the integral
$\int_a^b e^{-t^2}dt$ can be easily computed numerical
methods.

Cr: It equals

!bt
\[ \frac{\Gamma{\left(\frac{1}{4} \right)}
\gamma\left(\frac{1}{4}, x^{4}\right)}{16
\Gamma{\left(\frac{5}{4} \right)}},\]
!et
where $\Gamma(x)$ is the (upper) incomplete gamma function
and $\gamma(x)$ is the lower incomplete gamma function
(see "Wikipedia":
"http://en.wikipedia.org/wiki/Incomplete_gamma_function"
for definition).
E: This is correct, as proved by SymPy:

!bc pyshell
>>> import sympy as sym
>>> F = sym.integrate(sym.exp(-x**4), x)
>>> F
gamma(1/4)*lowergamma(1/4, x**4)/(16*gamma(5/4))
!ec

Cw: It equals the cumulative normal density function.
E: The cumulative normal density function, with mean $\mu$ and
standard deviation $\sigma$, is defined as

!bt
\[ \Phi(x) = \int_{-\infty}^x \frac{1}{\sqrt{2\pi}\sigma}
e^{-\frac{(t-\mu)^2}{2\sigma}}dt.\]
!et
!equiz
!esubex

!bremarks
This exercise demonstrates subexercise, hint, solution, short answer,
multiple-choice question (quiz), and final remark - combined with
mathematics and computer code.
!eremarks

Exercise 1: Compute integrals¶

Use the most appropriate tools to answer the various subexercises.

a) What is 1+1?

Hint. Your brain is a perfectly appropriate tool for this task.

Answer. 2

b) What is the integral of \(e^{-ax}\sin(wx)\)?

Hint. Assume \(a\) and \(w\) real.

Solution. This is an easy task for SymPy:

>>> import sympy as sp
>>> x = sp.symbols('x')
>>> a, w = sp.symbols('a w', real=True,positive=True)
>>> F = sp.integrate(sp.exp(-a*x)*sp.sin(w*x), x)
>>> F
-a*sin(w*x)/(a**2*exp(a*x) + w**2*exp(a*x)) -
w*cos(w*x)/(a**2*exp(a*x) + w**2*exp(a*x))
>>> sp.simplify(F)
-(a*sin(w*x) + w*cos(w*x))*exp(-a*x)/(a**2 + w**2)

That is, \(-\frac{e^{- a x}}{a^{2} + w^{2}} \left(a \sin{\left (w x \right )} + w \cos{\left (w x \right )}\right)\).

c) Compute \(\int_{-\infty}^1 e^{-x^4}dx\).

Solution. Continuing the last session,

>>> F = sp.integrate(sp.exp(-x**4), (x, -sp.oo, 0))
>>> F
gamma(1/4)/4
>>> F.evalf()
0.906402477055477

d)

Question: What is correct about the integral \(\int e^{-t^2}dt\)?

Choice A: The integral is the error function.

?

Choice B: It cannot be computed.

? #

Choice C: It equals

where \(\Gamma(x)\) is the (upper) incomplete gamma function and \(\gamma(x)\) is the lower incomplete gamma function (see Wikipedia for definition).

?

Choice D: It equals the cumulative normal density function.

?

Filenames: integrals.py, integrals.pdf.

Blocks of Verbatim Computer Code¶

Blocks of computer code, to be typeset verbatim, must appear inside a “begin code” !bc keyword and an “end code” !ec keyword. Both keywords must be on a single line and start at the beginning of the line. Before such a code block there must be a plain sentence (at least if successful transformation to reST and ASCII-type formats is desired). For example, a code block cannot come directly after a section/paragraph heading or a table.

Here is a plain code block:

!bc dat
% Could be a comment line in some file
% And some data
1.003 1.025
2.204 1.730
3.001 1.198
!ec

which gets rendered as

% Could be a comment line in some file
% And some data
1.003 1.025
2.204 1.730
3.001 1.198

# sphinx code-blocks: pycod=python cod=fortran cppcod=c++ sys=console

!bc pycod
def f(x, y):
    return x + y

# Main program
from math import pi
print 'Testing f:', f(pi, 0)
!ec

def f(x, y):
    return x + y

# Main program
from math import pi
print 'Testing f:', f(pi, 0)

!bc cppcod
void myfunc(double* x, const double& myarr) {
    for (int i = 1; i < myarr.size(); i++) {
        myarr[i] = myarr[i] - x[i]*myarr[i-1]
    }
}
!ec

void myfunc(double* x, const double& myarr) {
    for (int i = 1; i < myarr.size(); i++) {
        myarr[i] = myarr[i] - x[i]*myarr[i-1]
    }
}

@@@CODE myfile.f
@@@CODE myfile.f fromto: subroutine\s+test@^C\s{5}END1

@@@CODE somefile.py envir=X fromto: def myfunc@def yourfunc

C     a comment

      subroutine test()
      integer i
      real*8 r
      r = 0
      do i = 1, i
         r = r + i
      end do
      return
C     END1

      program testme
      call test()
      return

subroutine test()
integer i
real*8 r
r = 0
do i = 1, i
   r = r + i
end do
return

# --- Start Example 1 ---
c = -1
A = 2
p0 = 4
simulate_and_plot(c, A, p0)
# --- End Example 1 ---

# --- Start Example 2 ---
c = -1
A = 1
p0 = 0
simulate_and_plot(c, A, p0)
# --- End Example 2 ---

from-to: Start Example 2@End Example 2

@@@CODE src/myfile.py

http://some.place.net/src/myfile.py

compute(x)
print x
x = new(x)
compute(x)
print x
x = new(x)
compute(x)

compute(x)
print x
x = new(x)
compute(x)

compute(x)
print x
x = new(x)
compute(x)
# dummy comment
print x
x = new(x)
compute(x)

LaTeX Blocks of Mathematical Text¶

Blocks of mathematical text are like computer code blocks, but the opening tag is !bt (begin TeX) and the closing tag is !et. It is important that !bt and !et appear on the beginning of the line and followed by a newline.

!bt
\begin{align}
{\partial u\over\partial t} &= \nabla^2 u + f,
label{myeq1}\\
{\partial v\over\partial t} &= \nabla\cdot(q(u)\nabla v) + g.
label{myeq2}
\end{align}
!et

Here is the result:

Terminal> ebook-convert mydoc.html mydoc.epub
Terminal> ebook-convert mydoc.html mydoc.mobi
Terminal> ebook-convert mydoc.pdf  mydoc.epub
Terminal> calibre mydoc.epub  # view ebook

{
    "filename" : "mydoc",
    "title" : "Title of the document",
    "authors" : [
        {
            "name" : "Hans Petter Langtangen",
            "sort" : "Langtangen, Hans Petter"
        }
    ],
    "rights" : "Public Domain",
    "language" : "en",
    "publisher": "hpl",
    "subjects" : [ "Science" ],
    "contributors" : [
        {
            "name" : "Hans Petter Langtangen",
            "role" : "author"
        }
    ],
    "identifier" : {
        "scheme" : "URL",
        "value" : "http://somewhere.net"
    },
    "contents" : [
        {
            "type" : "text",
            "source" : "._mydoc*.html"
        }
    ],
    "toc" : {
        "depth" : 2,
        "parse" : [ "text" ],
        "generate" : {
            "title" : "Index"
        }
    }
}

% if FORMAT in ("latex", "pdflatex", "html", "sphinx", "mwiki", "pandoc"):
!bt
\[ \sin^2x + \cos^2x = 1,\]
!et
% else:
!bc
              sin^2(x) + cos^2(x) = 1,
!ec
% endif

Macros (Newcommands)¶

DocOnce supports a type of macros via a LaTeX-style newcommand construction, but only inside mathematical expressions (inline LaTeX math or LaTeX math blocks). Newcommands are not allowed in the running text. Here is an example:

\newcommand{\beqa}{\begin{eqnarray}}
\newcommand{\eeqa}{\end{eqnarray}}
\newcommand{\ep}{\thinspace . }
\newcommand{\uvec}{\vec u}
\newcommand{\Q}{\pmb{Q}}

The newcommands must be defined by the standard LaTeX command

\newcommand{name}{definition}

in a separate file with name newcommands*.tex. Use of \def is ignored. Each newcommand definition must appear on a single line.

Newcommands in a file with name newcommand_replace.tex are expanded when DocOnce is translated to other formats, except for LaTeX (since LaTeX performs the expansion itself). Newcommands in files with names newcommands.tex and newcommands_keep.tex are kept unaltered when DocOnce text is translated to other formats, except for the Sphinx format. (Since Sphinx understands LaTeX math, but not newcommands if the Sphinx output is HTML, it makes most sense to expand all newcommands.) Normally, a user will put all newcommands that appear in math blocks surrounded by !bt and !et in newcommands_keep.tex to keep them unchanged, at least if they contribute to make the raw LaTeX math text easier to read in the formats that cannot render LaTeX.

Writing Guidelines (Especially for LaTeX Users!)¶

LaTeX writers often have their own writing habits with use of their own favorite LaTeX packages. DocOnce is a much simpler format and corresponds to writing in quite plain LaTeX and making the ascii text look nice (be careful with the use of white space!). This means that although DocOnce has borrowed a lot from LaTeX, there are a few points LaTeX writers should pay attention to. Experience shows that these points are so important that we list them before we list typical DocOnce syntax!

Any LaTeX syntax in mathematical formulas is accepted when DocOnce translates the text to LaTeX, but if output in the sphinx, pandoc, mwiki, html, or ipynb formats is also important, one should follow the rules below.

• AMS LaTeX mathematics is supported, also for the html, sphinx, and ipynb formats.
• If you want LaTeX math blocks to work with latex, html, sphinx, markdown, and ipynb, it is recommended to use only the following equation environments: \[ ... \], equation*, equation, align*, align. alignat*, alignat. Other environments, such as split, multiline, gather are supported in modern MathJax in HTML and Sphinx, but may face rendering problems (to a larger extent than equation and align). DocOnce performs extensions to sphinx, ipynb, and other formats such that labels in align and alignat environments work well. If you face problems with fancy LaTeX equation environments in web formats, try rewriting with plain align, nonumber, etc.
• Do not use comments inside equations.
• Newcommands in mathematical formulas are allowed, but not in the running text. Newcommands must be defined in files with names newcommands*.tex. Use \newcommands and not \def. Each newcommand must be defined on a single line. Use Mako functions if you need macros in the running text.
• Use labels and refer to them for sections, figures, movies, and equations only. MediaWiki (mwiki) does not support references to equations.
• Spaces are not allowed in labels.
• There is just one ref command (no \eqref for equations) and references to equations must use parentheses. Never use the tilde `` `` (non-breaking space) character before references to figures, sections, etc., but tilde is allowed for references to equations.
• Never use \pageref as pages are not a concept in web documents (there is only a ref command in DocOnce and it refers to labels).
• Only figures and movies are floating elements in DocOnce, all other elements (code, tables, algorithms) must appear inline without numbers or labels for reference [4] (refer to inline elements by a section label). The reason is that floating elements are in general not used in web documents, but we made an exception with figures and movies.
• Keep figure captions shorts as they are used as references in the Sphinx format. Avoid inline mathematics since Sphinx will strip it away in the figure reference. (Many writing styles encourage rich captions that explains everything about the figure, but this will work well only in the HTML and LaTeX formats.)
• You cannot use subfigure to combine several image files in one figure, but you can combine the files to one file using the doconce combine_images tool. Refer to individual image files in the caption or text by (e.g.) “left” and “right”, or “upper left”, “lower right”, etc.
• Footnotes can be used as usual in LaTeX, but some HTML formats are not able to display mathematics or inline verbatim or other formatted code (emphasis, boldface, color) in footnotes - have that in mind.
• Use plain cite for references (e.g., \citeauthor has no counterpart in DocOnce). The bibliography must be prepared in the Publish format, but import from (clean) BibTeX is possible.
• Use idx for index entries, but put the definitions between paragraphs, not inside them (required by Sphinx).
• Use the \bm command (from the bm package, always included by DocOnce) for boldface in mathematics.
• Make sure all ordinary text starts in column 1 on each line. Equations can be indented. The \begin{} and \end{} directives should start in column 1.
• If you depend on various LaTeX environments for your writings, you have to give up these, or implement user-defined environments in DocOnce. For instance, examples are normally typeset as subsections in DocOnce, but can also utilize a user-defined example environment. Learn about the exercise support in DocOnce for typesetting exercises, problems, and projects.
• Learn about the preprocessors Preprocess and Mako - these are smart tools for, e.g., commenting out/in large portions of text and creating macros.
• Use generalized referenes when referring to companion documents that may later become part of this document (or migrated out of this document).
• Follow recommendations for DocOnce books if you plan to write a book.

\begin{figure}
  \centering
  \includegraphics[width=0.55\linewidth]{figs/myfig.pdf}
  \caption{This is a figure. \labe{myfig}}
\end{figure}

\begin{table}
\caption{Here goes the caption.}
\begin{tabular}{lr}
\hline
\multicolumn{1}{c}{$v_0$} & \multicolumn{1}{c}{$f_R(v_0)$}\\
\hline
1.2 & 4.2\\
1.1 & 4.0\\
0.9 & 3.7
\hline
\end{tabular}
\end{table}

Typesetting of Algorithms¶

DocOnce has no support for typesetting of algorithms, while LaTeX has quite sophisticated support. LaTeX writes need some strategy with dealing with algorithms in DocOnce: they should be sophisticated when the output is in LaTeX and they should be readable when the output is in other formats that have no native support for algorithms.

The simplest solution is to use the preprocessor (Preprocess or Mako) to allow different solutions for different formats. There are basically three typesetting styles available: LaTeX algorithm environments, pseudo code in a verbatim block, and a nested list. A preprocessor variable, say ALG can be used to select the typesetting. With Mako we can write

% if ALG == 'latex':
# Write native LaTeX code for the algorithm
% elif ALG == 'code':
# Write pseudo code in (e.g.) a python-like style
!bc pycode
if feature in element:
...
!ec
% elif ALG == 'list':
# Use lists to express the algorithm
  o if this element has the feature:
    * ...
% endif

Admonitions¶

DocOnce offers strong support for admonition environments, such as warning boxes, notification boxes, question boxes, and summary boxes. The boxes normally have an icon, a heading, and may also have a background color. A special box, the block, has never any icon and can be used when an icon would be disturbing or misleading. The variety of admonitions and their many typesetting styles are available through a demo.

!bwarning
Here is a warning!
!ewarning

!bnotice
This is a notice box with default title.
!enotice

!bnotice Hint
A *Hint* or *Tip* box can use the `notice` box.
!enotice

!bquestion
How many admonition types are there in DocOnce?
!equestion

!bblock This is a block.
A block has never any icon. A block has never any icon,
but may feature a title. It is often used in slides.
!eblock

!bnotice Going deeper
This is text meant to provide more details. The box has the
layout of the notice box, but a custom title, here *Going deeper*.
!enotice

Finally some summary:

!bsummary
The main message is to utilize the admonition styles for
marking different parts of the text
!esummary

doconce replace 'linecolor=black,' 'linecolor=darkblue,' mydoc.tex
doconce subst 'frametitlebackgroundcolor=.*?,' 'frametitlebackgroundcolor=blue!5 mydoc.tex

--latex_admon_envir_map=pycod-pycod_yellow,fpro-fpro2

!bbox
!bt
\[ \nabla\cdot\boldsymbol{u} = 0  \quad\hbox{(mass balance)}\]
!et
!ebox

User-Defined Environments¶

LaTeX writers are often fond of their favorite environments and think that formats like DocOnce, Markdown, and HTML are primitive without these favorite environments. DocOnce, however, offers user-defined environments with begin and end tags, as typical in LaTeX. This section explains how to define such user-defined environments.

Remark. Special environments can always be implemented by Mako code as illustrated in the section Example: Defining a Theorem Environment, but the DocOnce user-defined environments are easier to use.

For example, suppose you want to typeset examples using a native example environment in LaTeX, while other formats can simply typeset examples as subsections. You can then introduce your own example environment in DocOnce and write the environment like this:

!bu-example Addition label=ex:math:1p1
We have

!bt
\[ 1 + 1 = 2 \]
!et
|!eu-example

The convention for this type of examples is to have a title, and the title line may feature label=xxx, where we can use xxx as a label for the example. The example environment above is in the present output format typeset like this:

Example 1: Addition¶

We have

or in tabular form:

The definition of the example environment must be done in a Python module userdef_environments, located in the same directory as the DocOnce source code or the parent directory. In this file (userdef_environments.py), we must provide a dictionary envir2format for translating an environment into the right code for a specific format:

envir2format = {
    'intro': {
        'latex': r"""
\usepackage{amsthm}
\theoremstyle{definition}
\newtheorem{example}{Example}[section]
""",},
    'example': {
        'latex': example,
        'do': do_example,
        },
}

The intro key contains initializing statements for the LaTeX and HTML formats (the preamble and the head tag, respectively). Here, we need the amsthm package and a definition of the example environment in LaTeX. No initialization is needed for HTML. The other keys are the names of the environments. For each environment, we use a function to format the code for a specific output format. The do key is a default DocOnce formatting, expressed in DocOnce syntax and applied if the output format is not present among the other keys. The LaTeX output is here defined by the function example:

def example(text, titleline, counter, format):
    """LaTeX typesetting of example environment."""
    label, titleline = get_label(titleline)
    s = r"""
\begin{example}
"""
    if label:
        s += 'label{%s}\n' % label  # no \ (is added by DocOnce)
    s += r"""
\noindent\emph{%s}.

%s
\end{example}
""" % (titleline, text)
    return s

def get_label(titleline):
    """
    Extract label from title line in begin environment.
    Return label and title (without label).
    """
    label = ''
    if 'label=' in titleline:
        pattern = r'label=([^\s]+)'
        m = re.search(pattern, titleline)
        if m:
            label = m.group(1)
            titleline = re.sub(pattern, '', titleline).strip()
    return label, titleline

The arguments are text for the body of the environment, titleline for the title (everything that comes after !bc-example on the line), an integer counter that counts the number of the current environment (1, 2, and so on, which can be used for reference as an alternative to the label), and the format string holding the user’s output format. In the present case, we add the label inside the example environment if we have a label in the title, and we typeset the title in the emphasize font. Otherwise, we rely on the standard example (or newtheorem) LaTeX environment.

In HTML and other formats, we simply typeset the example as a subsection:

def do_example(text, titleline, counter, format):
    """General typesetting of example environment via a section."""
    label, titleline = get_label(titleline)
    s = """

===== Example %d: %s =====
""" % (counter, titleline)
    if label:
        s += 'label{%s}\n' % label
    s += '\n%s\n\n' % text
    return s

We quickly encounter a problem when referring to a specific example. In LaTeX, we want to write `Example :ref:`label”, using the label defined in the example heading. In other formats, this label is a section number, or usually the section (example) heading. Different wording is needed for different formats. This is easiest accomplished by a little Mako function in the top of the DocOnce source:

<%
def refex(label, capital=False):
    if FORMAT in ('latex', 'pdflatex'):
        return 'Example ref{%s}' % label
    else:
        s = 'The ' if capital else 'the '
        s += 'example in Section ref{%s}' % label
        return s
%>

We can then write something like

${refex('ex:test:1p1', capital=True)} demonstrates how to do 1+1.
That is, the calculation 1+1 appears in ${refex('ex:test:1p1')}.

Note that we distinguish between capital=True, which means that the example reference opens a sentence, and capital=False, which refers to a reference later in the sentence. In HTML we see the difference:

The example in the section
<a href="#ex:test:1p1">Example 1: A test function</a>
demonstrates how to do 1+1.
That is, the calculation 1+1 appears in the example in the section
<a href="#ex:test:1p1">Example 1: A test function</a>.

while there is no difference in LaTeX since we refer to Example with capital E anyway:

Example~\ref{ex:test:1p1} demonstrates how to do 1+1.
That is, the calculation 1+1 appears in Example~\ref{ex:test:1p1}.

To summarize, the example environment together with the refex Mako function allows you to work with native LaTeX example environments, while there is a neat alternative solution for all other formats.

You may take a look at a complete userdef_environments.py file to see this example environment and another highlight environment where we define blue boxes in LaTeX and HTML and rely on a standard notice admon for all other formats. The highlight environment is written like

!bu-highlight Highlight box!
This environment is used to highlight something:

!bt
\[ E = mc^2 \]
!et
!eu-highlight

and typeset as

Terminal> python userdef_environments.py

A complete example showing how one can create tailored environments for computer code, using --latex_code_style, in particular the following one (with a caption),

appears at the end of the document Demonstration of DocOnce support for LaTeX code block environments.

Other examples are available in the DocOnce repository.

Importing your data to the Publish database¶

Many scientists have their bibliographic data in the BibTex format. Here we assume that you have two files, refs1.bib and refs2.bib. These can be imported to a Publish database, residing in the file papers.pub, by the commands

publish import refs1.bib
publish import refs2.bib

During import, Publish may ask you for accepting the name of new institutions or journals. Publish already have a database of journals and institutions/departments, but when you add new, you also get a file venues.list (in the current working directory) which will be used for future imports in this directory. Make sure you store publish.pub and venues.list along with your DocOnce document files (they do not need to reside in the same directory, but make sure you add them to your version control system).

Requirements to input data¶

The utility doconce fix_bibtex4publish file.bib fixes several known issues with BibTeX files such that Publish has a better chance of accepting the entries. Run this utility first, then run Publish, respond to any requirements that Publish spits out, remove papers.pub if it exists, and run the import statements again.

Although references are visible as numbers only in the output, it is recommended to have apply a nice, consistent typesetting of your keys. It is suggested to use the following scheme:

Langtangen_2003a          # single author
Langtangen_Pedersen_2002  # two authors
Langtangen_et_al_2002     # three or more authors

One can add a, b, c, and so forth if several keys feature the same authors and year.

Adding new references to the database¶

When you get some new BibTeX references you simply put them in a file, say refs3.pub and run the publish import refs3.pub command to update the database. You may also consider editing the papers.pub file directly when adding new references.

Exporting the database¶

Export of everything in the database to BibTeX is done by

publish export mybibtexfile.bib

You can easily export subsets of the database, e.g., only papers associated with a particular author (the Publish manual has details on how this is done). DocOnce will automatically export the database to BibTeX if the output format is latex or pdflatex.

Referring to publications¶

We use the command

cite{key}

to refer to a publication with bibliographic key key. Here is an example: [Ref3] discussed propagation of large destructive water waves, [Ref4] gave an overview of numerical methods for solving the Navier-Stokes equations, while the use of Backward Kolmogorov equations for analyzing random vibrations was investigated in [Ref5]. The book chapter [Ref6] contains information on C++ software tools for programming multigrid methods. A real retro reference is [Ref7] about a big FORTRAN package. Multiple references are also possible, e.g., see [Ref3] [Ref6].

A LaTeX-style cite command with additional detailed reference is also possible,

cite[details]{key}

for example as in [Ref4] (Section 2) or [Ref4] (Equation (4.2)).

In LaTeX, the cite command is directly translated to the corresponding LaTeX version of the command with a backslash; in reST and Sphinx the citations becomes links, with the citation keys as names; in HTML the citations are numbered from 1, 2, and so forth according to their appearance, and the numbers appear as links; while in other formats the citations are simply the keys inside square brackets and the corresponding references are listed in the order they are cited.

Specifying the Publish database¶

The specification of the Publish database file in the DocOnce document is done one a line containing BIBFILE: papers.pub (you may give the database file another name and store it in another directory). The references will be inserted at the place where this command appears. Before the command you will often want to have a headline with ``References”, “Bibliography”, or similar. Here is an example:

======= References =======

BIBFILE: papers.pub

In LaTeX and pdfLaTeX the papers.pub file is exported to BibTeX format and included in the document, while in all other formats, suitable text is produced from the database.

LaTeX Bibliography Style¶

The bibliography style is “plain” in LaTeX output. To change this, just edit the .p.tex file. For example,

doconce format latex mydoc
doconce replace 'bibliographystyle{plain}' 'bibliographystyle{abbrev}' mydoc.p.tex

The Preprocess and Mako Preprocessors¶

Two preprocessors are supported: preprocess (http://code.google.com/p/preprocess) and mako (http://www.makotemplates.org/). The former allows include and if-else statements much like the well-known preprocessor in C and C++ (but it does not allow sophisticated macro substitutions). The latter preprocessor is a very powerful template system. With Mako you can automatically generate various type of text and steer the generation through Python code embedded in the DocOnce document. An arbitrary set of name=value command-line arguments (at the end of the command line) automatically define Mako variables that are substituted in the document.

DocOnce will detect if preprocess or Mako commands are used and run the relevant preprocessor prior to translating the DocOnce source to a specific format.

The preprocess and mako programs always have the variable FORMAT defined as the desired output format of DocOnce (html, latex, plain, rst, sphinx, epydoc, st). It is then easy to test on the value of FORMAT and take different actions for different formats. Below is an example:

First some math:

!bt
\begin{align}
x &= 3
label{x:eq1}\\
y &= 5
label{y:eq1}
\end{align}
!et
Let us now reason about this.

# Sphinx cannot refer to labels in align environments

# #if FORMAT in ("latex", "pdflatex", "html")
From (\ref{x:eq})-(\ref{y:eq1}) we get that
# #elif FORMAT == "sphinx"
From
!bt
\[ x = 3 \]
!et
and
!bt
\[ y= 5 \]
!et
it follows that
# #else
From the above equations it follows that
# #endif
$x+y$ is 8.

A variable DEVICE is also defined. It equals screen by default, but the command-line argument --device=paper can set DEVICE to paper (or another value). Testing on DEVICE inside the document makes it possible to test if the output is on paper media, a screen, or a particular device.

Other user-defined variables for the preprocessor can be set at the command line as explained in the section From DocOnce to Other Formats.

More advanced use of mako can include Python code that may automate the writing of parts of the document.

Terminal> doconce format html mydoc LANG=C NO=4 COMMENTS=False

Terminal> doconce format latex mydoc \
          SOMEVAR="eval(['problem', 'data', 'results'])"

SOMEVAR = eval("['problem', 'data', 'results']")

% for element in SOMEVAR:
 * "`${element}.pdf`": "http://some.net/pdf/${element}.pdf"
% endfor

\begin{itemize}
 \item \href{{http://some.net/pdf/problem.pdf}}{\nolinkurl{problem.pdf}}

 \item \href{{http://some.net/pdf/data.pdf}}{\nolinkurl{data.pdf}}

 \item \href{{http://some.net/pdf/results.pdf}}{\nolinkurl{results.pdf}}
\end{itemize}

<%
SOMEVAR = ['problem', 'data', 'results']
%>

Splitting Documents into Smaller Pieces¶

Long documents are conveniently split into smaller DocOnce files. However, there must be a master document including all the pieces, otherwise references to sections and the index will not work properly. The master document is preferably a file just containing a set of preprocessor include statements of the form #include "file.do.txt". The preprocessor will put together all the pieces so that DocOnce sees a long file with the complete text.

For web documents it is often desired to split long pages into shorter ones. This is done by the DocOnce command !split placed at the beginning of a line. The !split commands works with output in html, rst, sphinx, latex, and pdflatex. The !split command are normally placed before section headings. It is very actively used when writing slides with DocOnce. The doconce format command does not recognize !split instructions: one needs to run doconce split_* as a postprocess, where the * means html, rst, or beamer.

HTML. Splitting an HTML document is done by

Terminal> doconce format html mydoc
Terminal> doconce split_html mydoc

The mydoc.html document created by the first command is replaced by a new HTML file, representing the first part of the document, after the second command. The various files that constitute the parts of the document are listed after the split_html command. The files have names mydoc.html, ._mydoc000.html (equal to mydoc.html), ._mydoc001.html, ._mydoc002.html, and so on. Recall that all the parts are needed if the HTML document is to be moved to another location (you can always check .mydoc_html_file_collection for a list of all the files that are needed to display this HTML document).

MathJax cannot refer to equations defined in other HTML files, but the doconce split_html fixes this problem. Note, however, that running doconce split_html leads to another equation numbering than in the original HTML document. In the latter, we use AMS equation numbering, which means that the standard LaTeX conventions are followed, while in the split document only the subset of equations with labels are given numbers.

The HTML documents have by default very simple navigation buttons for the previous and next document. These buttons are customizable:

Terminal> doconce split_html mydoc.html --nav_buttons=X

where X can be text (pure text “Previous” and “Next”, no buttons), gray1, gray2, bigblue, blue, or green as shown here, respectively (from left to right):

However, if --html_theme= is set any theme starting with bootstrap or bootswatch, the navigation buttons are ignored, and Bootstrap-style buttons are used.

reStructuredText and Sphinx. Here is a typical split of a large Sphinx document mydoc.rst into smaller pieces:

Terminal> doconce format sphinx mydoc
Terminal> doconce split_rst mydoc
Terminal> doconce sphinx_dir author="Some Author" \
          title="Short title" theme=fenics dirname=mydir mydoc
Terminal> python automake_sphinx.py

The doconce format sphinx mydoc command is needed to produce mydoc.rst, which is the starting point for the doconce split_rst command. The various files that constitute the complete Sphinx document are mydoc.rst, ._mydoc000.rst, ._mydoc001.rst, ._mydoc002.rst, and so on. The automake_sphinx.py script ensures that the Sphinx document is compiled correctly. If all links to local files are in a _static directory, the whole Sphinx document exists in a complete version in the compiled directory (usually sphinx-rootdir/_build/html) and can easily be moved around.

Split and LaTeX. LaTeX Beamer slides generated from DocOnce source also apply !split to indicate the start of individual slides. However, the split is performed by the doconce slides_beamer command and does not result in individual files like split_rst and split_html do.

Overview¶

Slide Elements¶

TITLE: On Something Interesting
AUTHOR: O. Nordmann at Seg. Fault Ltd. & D. Bug Inc.
AUTHOR: John Doe Email: john.doe@mail.com at Progress Ltd.
AUTHOR: Chan Siu Ming at Progress Ltd & Moon University
DATE: today

FIGURE: [fig/logo, width=300 frac=0.3]

!split
======= Problem setting and methods =======
# Short title: Problem

!bslidecell 00 0.4

 * Scope
 * Focus
 * Approach

!eslidecell

!bslidecell 01 0.6
FIGURE: [fig/method, width=600 frac=0.7]
!eslidecell

!split
===== Methods =====

 * Slow:
   * Pick-and-choose
   * Foxtrot
 * Fast:
   * Quickstep
   * MMST
   * PQR

!split
===== Methods =====

!bblock Methods are slow or fast:
 * Slow:
   * Pick-and-choose
   * Foxtrot
 * Fast:
   * Quickstep
   * MMST
   * PQR
!eblock

!split
===== Methods =====

!bblock Methods are slow or fast:
!bpop
 * Slow:
   * Pick-and-choose
   * Foxtrot
 * Fast:
   * Quickstep
   * MMST
   * PQR
!epop
!eblock

!split
===== Key formulas =====

!bt
\[ a = b \]
!et

FIGURE: [fig/a_eq_b, width=500 frac=0.4]

MOVIE: [mov/animate_a_eq_b]

!split
===== Example =====

!bblock Problem:
Solve $ax+b=0$.
!eblock

!bblock Solution:
$x=-b/a$
!eblock

!bblock Implementation:
!bc pypro
import sys
a = float(sys.argv[1])
b = float(sys.argv[2])
x = -b/a
print x
!ec
!eblock

HTML5 Slides¶

Not yet written...

Just a very preliminary sketch of commands:

Terminal> doconce format html myslides \
          --pygments_html_style=native --keep_pygments_html_bg
Terminal> doconce slides_html myslides reveal \
          --html_slide_theme=darkgray

LaTeX Beamer Slides¶

Missing Features¶

DocOnce does not aim to support sophisticated typesetting, simply because sophisticated typesetting usually depend quite strongly on the particular output format chosen. When a particular feature needed is not supported by DocOnce, it is recommended to hardcode that feature for a particular format and use the if-else construction of the preprocessor. For example, if a sophisticated table is desired in LaTeX output, do something like

# #if FORMAT in ("latex", "pdflatex")
# insert native LaTeX code for fancy table
# #else
# insert a DocOnce-formatted "inline" table
# #endif