Programmatic Generators

A programmatic generator is the kind of pipeline extension that owns the whole emit. In a script it writes files through ctx.output, and in C++ it implements Generator::build. Either way it decides which files exist and what goes in them, instead of rendering one page per symbol.

A data-driven generator is the other kind of custom generator: a directory of Handlebars templates and a manifest define it, and code does not. Reach for a programmatic generator when customizing templates is not enough to express the output format.

Registering a generator

A generator is written and loaded like a corpus transform, as Defining Extensions describes. A script registers it with mrdocs.register_generator(id, fn) and a plugin registers it with registerGenerator:

  • JavaScript

  • Lua

  • C++

addons/extensions/hello_world_generator.js
mrdocs.register_generator("hello-world-generator", function(ctx) {
    ctx.output.write("hello.txt", "Hello from a generator\n");
});
addons/extensions/hello_world_generator.lua
mrdocs.register_generator("hello-world-generator", function(ctx)
    ctx.output.write("hello.txt", "Hello from a generator\n")
end)
hello-world-generator/plugin.cpp
#include <mrdocs/Extensions/GeneratorExtensionContext.hpp>
#include <mrdocs/Generator.hpp>
#include <mrdocs/Plugin.hpp>
#include <filesystem>
#include <fstream>

MRDOCS_PLUGIN_MAIN(plugin)
{
    return plugin.registerGenerator("hello-world-generator",
        [](mrdocs::GeneratorExtensionContext const& ctx) -> mrdocs::Expected<void>
        {
            std::filesystem::path const dir(ctx.outputPath);
            std::error_code ec;
            std::filesystem::create_directories(dir, ec);
            std::ofstream os(dir / "hello.txt");
            if (ec || !os)
            {
                return mrdocs::Unexpected(mrdocs::Error(
                    "could not write hello.txt"));
            }
            os << "Hello from a generator\n";
            return {};
        });
}

A generator registered as a function receives a GeneratorExtensionContext. It holds the corpus, the configuration, the output path, the options block and the reporter of the run.

The rules for generator `id`s:

  • Its block of options is generator-options.<id>.

  • You select it on the command line with --generator=<id>.

  • It must differ from the id of every other generator, built-in or not, or Mr.Docs stops the run with generator id="<id>" already exists.

The generator receives the corpus and writes whatever it wants into the output directory. The examples above write one file. A script writes it with the helper ctx.output.write(path, text), a function the script binding adds to the context, as it does ctx.stringify and the functions of ctx.report.

Once a plugin is installed, its generator is selectable like any other, through the generator option:

generator: hello-world-generator

A generator class gets its generator-options.<id> block from getGeneratorParams, which it calls from Generator::build. The block is an empty object when the configuration sets none.

Outputting the corpus

The smallest useful generator hands the corpus straight to JSON.stringify. A complete, runnable example lives at examples/generators/script-driven/json/. Given this input:

simple.cpp
/// A vector in the Euclidean plane.
struct Vector
{
    /// The length (magnitude) of the vector.
    double length() const;
};

In very few lines of code, this extension declares a symbols-json generator that writes every symbol to a single reference.json:

addons/extensions/json.js
mrdocs.register_generator("symbols-json", function(ctx) {
  const indent = ctx.params.indent || 0;
  ctx.output.write(
    "reference.json",
    JSON.stringify(ctx.corpus.symbols, null, indent));
});

Select it with --generator=symbols-json. Because ctx.corpus.symbols is already plain data, the whole generator is one JSON.stringify call, and the shape is yours: map or filter the array, or read ctx.params to add options, before writing it. The id has to be new: a generator whose id is taken, by a built-in generator or by another extension, is refused, so this one cannot be called json, the id of the built-in JSON generator, which is the same idea implemented in C++.

The output is an array of every symbol, each a full DOM object as seen by the templates, with parent/child relationships expressed as id references. Its first entry, the global namespace, begins (the file continues with one object per symbol):

reference.json (excerpt)
[
  {
    "$meta": {
      "type": "NamespaceSymbol",
      "bases": [
        "SymbolCommonBase",
        "Symbol"
      ]
    },
    "anchor": "index",
    "loc": {
      "$meta": {
        "type": "SourceInfo",
        "bases": []
      },
      "loc": []
    },
    "kind": "namespace",
    "id": "4ZrjxJnU1LA5xSyrWMNuXTvSYKwt",
    "extraction": "regular",
    "isCopyFromInherited": false,
    "attributes": [],
    "isInline": false,
    "isAnonymous": false,
    "usingDirectives": [],
    "members": {
      "$meta": {
        "type": "NamespaceTranche",
        "bases": []
      },
      "namespaces": [],
      "namespaceAliases": [],
      "typedefs": [],
      "records": [
        "43E6qXkmkkfeWwCdMLUnhvUC5tmQ"
      ],
      "enums": [],
      "functions": [],
      "variables": [],
      "concepts": [],
      "guides": [],
      "usings": [],
      "macros": []
    }
  },
...

Filtering the corpus

Consider that you need a JSON file representing a search index for your reference documentation, in a format supported by your documentation framework. A complete, runnable example lives at examples/generators/script-driven/search-index/. Given this input:

simple.cpp
/// A vector in the Euclidean plane.
struct Vector
{
    /** The length (magnitude) of the vector.

        @return The Euclidean length.
    */
    double length() const;

    /** Scale the vector componentwise.

        @param sx Factor applied to the x component.
        @param sy Factor applied to the y component.
    */
    void scale(double sx, double sy);
};

the extension declares a search-index generator that emits a single search-index.json aggregating every symbol, an artifact no per-page generator can produce:

addons/extensions/search_index.lua
local url_for

mrdocs.register_generator("search-index", function(ctx)
  local entries = {}
  for _, sym in ipairs(ctx.corpus.symbols) do
    local name = sym.name or ""
    if name ~= "" then
      local entry = {}
      entry.name = name
      entry.url = url_for(ctx.corpus, sym)
      entries[#entries + 1] = entry
    end
  end
  ctx.output.write("search-index.json", ctx.stringify(entries))
end)

--- A symbol's page URL: the chain of anchors from the global namespace down to
--- the symbol, joined by "/", plus ".html". Walk up through `parent` (a symbol
--- id) with `corpus.get`; the global namespace has no parent, so it contributes
--- no path segment.
--- @param corpus table `ctx.corpus`, used to resolve each `parent` id.
--- @param sym table The symbol to build a URL for.
--- @return string url The symbol's page URL.
function url_for(corpus, sym)
  local parts = {}
  local cur = sym
  while cur and cur.parent do
    table.insert(parts, 1, cur.anchor)
    cur = corpus.get(cur.parent)
  end
  return table.concat(parts, "/") .. ".html"
end

Select it with --generator=search-index; it writes search-index.json into the output directory: one entry per named symbol.

search-index.json
[
    {
        "name": "Vector",
        "url": "Vector.html"
    },
    {
        "name": "length",
        "url": "Vector/length.html"
    },
    {
        "name": "scale",
        "url": "Vector/scale.html"
    }
]

Symbol selection

An index does not list every entry of the corpus. The fields that decide what to keep are on Symbol. Its extraction mode (ExtractionMode) says whether it is shown, which regular and see-below symbols are. Its inherited-from id marks a copy of a base class member, and an overload set has its own kind and members.

Table of Contents

A table of contents or a navigation list over the whole API is the same idea as the search index: an artifact aggregated across every symbol, but it emits a nested list instead of JSON objects. Deriving it from the corpus prevents a project from manually maintaining a separate nav partial and re-syncing it whenever the API changes. A complete, runnable example lives at examples/generators/script-driven/toc/:

  • Generator

  • Input

  • Config

addons/extensions/toc.js
mrdocs.register_generator("toc", function(ctx) {
  const roots = buildRoots(ctx.corpus);
  ctx.output.write("toc.md", markdown(roots, 0));
  ctx.output.write("toc.adoc", asciidoc(roots, 0));
  ctx.output.write("toc.html", html(roots));
});

/**
 * Build the top-level navigation nodes for the whole corpus: the page-level
 * children of the global namespace (the one symbol with no `parent`), each
 * expanded into a node tree.
 * @param {object} corpus - `ctx.corpus`.
 * @returns {object[]} The root nav nodes.
 */
function buildRoots(corpus) {
  let root = null;
  for (let i = 0; i < corpus.symbols.length; ++i) {
    if (!corpus.symbols[i].parent) {
      root = corpus.symbols[i];
      break;
    }
  }
  const roots = [];
  const topIds = root ? childIds(root) : [];
  for (let j = 0; j < topIds.length; ++j) {
    const top = corpus.get(topIds[j]);
    if (top && top.name) {
      roots.push(node(corpus, top));
    }
  }
  return roots;
}

// Page-level child categories of a namespace, in the order a nav should list
// them. Record members (methods, fields) live on the record's own page, so the
// walk descends through namespaces only and treats everything else as a leaf.
const CATEGORIES = [
  "namespaces", "records", "enums", "functions",
  "variables", "typedefs", "concepts", "usings", "macros"
];

/**
 * The ids of a namespace's page-level children, across every category in
 * `CATEGORIES`. Anything that is not a namespace is a leaf in the navigation
 * tree, so this returns an empty array for it.
 * @param {object} sym - The symbol whose children to collect.
 * @returns {string[]} The child symbol ids, in category order.
 */
function childIds(sym) {
  const ids = [];
  if (sym.kind !== "namespace" || !sym.members) {
    return ids;
  }
  for (let c = 0; c < CATEGORIES.length; ++c) {
    const list = sym.members[CATEGORIES[c]] || [];
    for (let k = 0; k < list.length; ++k) {
      ids.push(list[k]);
    }
  }
  return ids;
}

/**
 * Build a plain `{ name, url, children }` node for a symbol, recursing through
 * its page-level children. Only namespaces have children; every other symbol
 * is a leaf linking to its own page. Unnamed children are skipped.
 * @param {object} corpus - `ctx.corpus`.
 * @param {object} sym - The symbol to convert.
 * @returns {{name: string, url: string, children: object[]}} The nav node.
 */
function node(corpus, sym) {
  const children = [];
  const ids = childIds(sym);
  for (let i = 0; i < ids.length; ++i) {
    const child = corpus.get(ids[i]);
    if (child && child.name) {
      children.push(node(corpus, child));
    }
  }
  return { name: sym.name, url: urlFor(corpus, sym), children: children };
}

/**
 * The page URL for a symbol: the chain of anchors from the global namespace
 * down to the symbol, joined by "/", plus ".html" (the shape the HTML
 * generator emits).
 * @param {object} corpus - `ctx.corpus`, used to resolve each `parent` id.
 * @param {object} sym - The symbol to build a URL for.
 * @returns {string} The symbol's page URL.
 */
function urlFor(corpus, sym) {
  const parts = [];
  let cur = sym;
  while (cur && cur.parent) {
    parts.unshift(cur.anchor);
    cur = corpus.get(cur.parent);
  }
  return parts.join("/") + ".html";
}

/**
 * Render nav nodes as a nested Markdown bullet list.
 * @param {object[]} nodes - Nav nodes.
 * @param {number} depth - Indentation depth, 0 at the top level.
 * @returns {string} The Markdown list.
 */
function markdown(nodes, depth) {
  let out = "";
  for (let i = 0; i < nodes.length; ++i) {
    const n = nodes[i];
    out += "  ".repeat(depth) + "- [" + n.name + "](" + n.url + ")\n";
    out += markdown(n.children, depth + 1);
  }
  return out;
}

/**
 * Render nav nodes as a nested AsciiDoc list of `xref` links.
 * @param {object[]} nodes - Nav nodes.
 * @param {number} depth - Nesting depth, 0 at the top level (one `*` per level).
 * @returns {string} The AsciiDoc list.
 */
function asciidoc(nodes, depth) {
  let out = "";
  for (let i = 0; i < nodes.length; ++i) {
    const n = nodes[i];
    out += "*".repeat(depth + 1) + " xref:" + n.url + "[" + n.name + "]\n";
    out += asciidoc(n.children, depth + 1);
  }
  return out;
}

/**
 * Render nav nodes as a nested HTML `<ul>` list. Returns an empty string for
 * an empty list, so leaf nodes emit no child `<ul>`.
 * @param {object[]} nodes - Nav nodes.
 * @returns {string} The HTML list.
 */
function html(nodes) {
  if (nodes.length === 0) {
    return "";
  }
  let out = "<ul>\n";
  for (let i = 0; i < nodes.length; ++i) {
    const n = nodes[i];
    out += "<li><a href=\"" + n.url + "\">" + n.name + "</a>" + html(n.children) + "</li>\n";
  }
  out += "</ul>\n";
  return out;
}
simple.cpp
/// Two-dimensional geometry primitives.
namespace geo {

/// A point in the plane.
struct Point
{
    /// The x coordinate.
    double x;

    /// The y coordinate.
    double y;

    /** The distance to another point.

        @param other The point to measure to.
        @return The Euclidean distance between the two points.
    */
    double distance_to(Point other) const;
};

/// How two shapes relate spatially.
enum class Relation
{
    disjoint,    ///< No shared points.
    touching,    ///< A shared boundary only.
    overlapping  ///< Shared interior points.
};

/** The midpoint of two points.

    @param a The first point.
    @param b The second point.
    @return The point halfway between `a` and `b`.
*/
Point midpoint(Point a, Point b);

/// Coordinate-system helpers.
namespace coord {

/** Convert polar coordinates to a Cartesian point.

    @param r The radius.
    @param theta The angle in radians.
    @return The equivalent Cartesian point.
*/
Point from_polar(double r, double theta);

}

}
mrdocs.yml
addons-supplemental:
  - addons
generator: toc
multipage: false
show-namespaces: false
warn-if-undocumented: false
source-root: .
input:
  - .

The generator walks the namespace tree once, then serializes it to three formats from that single walk, so the nav for each output stays in step. The corpus is a tree: the global namespace is the one symbol with no parent, and a namespace lists its children in members, grouped by category (namespaces, records, enums, functions, …​). The walk descends through namespaces and treats every record, enum, and function as a leaf linking to its own page. Select it with --generator=toc; it writes one file per format:

  • Markdown

  • AsciiDoc

  • HTML

toc.md
- [geo](geo.html)
  - [coord](geo/coord.html)
    - [from_polar](geo/coord/from_polar.html)
  - [Point](geo/Point.html)
  - [Relation](geo/Relation.html)
  - [midpoint](geo/midpoint.html)
toc.adoc
* xref:geo.html[geo]
** xref:geo/coord.html[coord]
*** xref:geo/coord/from_polar.html[from_polar]
** xref:geo/Point.html[Point]
** xref:geo/Relation.html[Relation]
** xref:geo/midpoint.html[midpoint]
toc.html
<ul>
<li><a href="geo.html">geo</a><ul>
<li><a href="geo/coord.html">coord</a><ul>
<li><a href="geo/coord/from_polar.html">from_polar</a></li>
</ul>
</li>
<li><a href="geo/Point.html">Point</a></li>
<li><a href="geo/Relation.html">Relation</a></li>
<li><a href="geo/midpoint.html">midpoint</a></li>
</ul>
</li>
</ul>

Reflection Generators

A custom generator does not have to emit documentation. It could be used for any task related to the project’s public API and even write back to the files used to generate the Corpus. For instance, Boost.Describe adds compile-time reflection to C++ types, but its BOOST_DESCRIBE_STRUCT and BOOST_DESCRIBE_ENUM annotations are hand-written. The corpus already knows every type and its members, so the same walk that produces docs can emit those annotations to keep them in sync with the code. A complete, runnable example lives at examples/generators/script-driven/describe/:

  • Generator

  • Input

  • Config

addons/extensions/describe.js
mrdocs.register_generator("describe", function(ctx) {
  const grouped = groupByHeader(ctx.corpus);
  for (let i = 0; i < grouped.order.length; ++i) {
    const header = grouped.order[i];
    const group = grouped.byHeader[header];
    ctx.output.write(sidecarName(header), renderSidecar(header, group.includes, group.order, group.byNs));
  }
});

/**
 * Group the describe annotation of every record and enum in the corpus, first
 * by the header it is defined in and then by its enclosing namespace, keeping
 * first-seen order at every level. Each header group also collects the base
 * sidecars its types depend on, minus its own.
 * @param {object} corpus - `ctx.corpus`.
 * @returns {{order: string[], byHeader: Object<string, {order: string[], byNs: Object<string, string[]>, includes: string[]}>}}
 *   The headers in first-seen order and, for each, its namespaces, their
 *   annotation lines, and the base sidecars to include.
 */
function groupByHeader(corpus) {
  const order = [];
  const byHeader = {};
  for (let i = 0; i < corpus.symbols.length; ++i) {
    const sym = corpus.symbols[i];
    const line = describeMacro(corpus, sym);
    if (line === null) {
      continue;
    }
    const header = sourceHeaderOf(sym);
    if (!header) {
      continue;
    }
    if (!(header in byHeader)) {
      byHeader[header] = { order: [], byNs: {}, includes: [] };
      order.push(header);
    }
    const group = byHeader[header];
    const ns = namespaceOf(corpus, sym);
    if (!(ns in group.byNs)) {
      group.byNs[ns] = [];
      group.order.push(ns);
    }
    group.byNs[ns].push(line);

    const self = sidecarName(header);
    const bases = baseSidecars(corpus, sym);
    for (let s = 0; s < bases.length; ++s) {
      if (bases[s] !== self && group.includes.indexOf(bases[s]) === -1) {
        group.includes.push(bases[s]);
      }
    }
  }
  return { order: order, byHeader: byHeader };
}

/**
 * The Boost.Describe annotation for a symbol: BOOST_DESCRIBE_STRUCT for a
 * record (its base classes and own public members) or BOOST_DESCRIBE_ENUM for
 * an enum (its enumerators). Returns null for any other kind of symbol.
 * @param {object} corpus - `ctx.corpus`.
 * @param {object} sym - The symbol to annotate.
 * @returns {string|null} The macro invocation, or null if `sym` is not a
 *   record or enum.
 */
function describeMacro(corpus, sym) {
  if (sym.kind === "record") {
    const bases = [];
    const baseList = sym.bases || [];
    for (let b = 0; b < baseList.length; ++b) {
      const name = baseList[b].type && baseList[b].type.name && baseList[b].type.name.identifier;
      if (name) {
        bases.push(name);
      }
    }
    const members = ownMemberNames(corpus, sym, "variables").concat(ownMemberNames(corpus, sym, "functions"));
    return "BOOST_DESCRIBE_STRUCT(" + sym.name + ", (" + bases.join(", ") + "), (" + members.join(", ") + "))";
  }
  if (sym.kind === "enum") {
    const values = [];
    const constants = sym.constants || [];
    for (let c = 0; c < constants.length; ++c) {
      const value = corpus.get(constants[c]);
      if (value && value.name) {
        values.push(value.name);
      }
    }
    return "BOOST_DESCRIBE_ENUM(" + sym.name + ", " + values.join(", ") + ")";
  }
  return null;
}

/**
 * The names of a record's own public members in `category`. The interface also
 * carries inherited members; Boost.Describe learns those from the base list, so
 * only members declared on this record (their `parent` is the record) are kept.
 * @param {object} corpus - `ctx.corpus`.
 * @param {object} record - The record symbol.
 * @param {string} category - The interface bucket, "variables" or "functions".
 * @returns {string[]} The own public member names, in declaration order.
 */
function ownMemberNames(corpus, record, category) {
  const names = [];
  const list = record.interface.public[category] || [];
  for (let i = 0; i < list.length; ++i) {
    const member = corpus.get(list[i]);
    if (member && member.name && member.parent === record.id) {
      names.push(member.name);
    }
  }
  return names;
}

/**
 * The fully-qualified enclosing namespace of a symbol ("a::b"), or "" when it
 * sits at global scope. A BOOST_DESCRIBE_STRUCT must appear in the type's own
 * namespace so the unqualified name resolves, so entries are grouped by this.
 * @param {object} corpus - `ctx.corpus`, used to walk `parent` ids.
 * @param {object} sym - The symbol whose namespace to compute.
 * @returns {string} The enclosing namespace, "::"-joined, or "" at global scope.
 */
function namespaceOf(corpus, sym) {
  const parts = [];
  let cur = sym.parent ? corpus.get(sym.parent) : null;
  while (cur && cur.name !== undefined && cur.kind === "namespace") {
    parts.unshift(cur.name);
    cur = cur.parent ? corpus.get(cur.parent) : null;
  }
  return parts.join("::");
}

/**
 * The header a symbol is defined in, taken from its definition location, or ""
 * when the location is unknown. Sidecars are grouped by this so each one can
 * include exactly the header whose types it annotates.
 * @param {object} sym - The symbol whose defining header to read.
 * @returns {string} The header path, or "" if unknown.
 */
function sourceHeaderOf(sym) {
  return (sym.loc && sym.loc.defLoc && sym.loc.defLoc.shortPath) || "";
}

/**
 * The sidecar file name for a header: "foo.hpp" becomes "foo.described.hpp", and
 * a header with any other (or no) extension gets ".described.hpp" appended.
 * Including it gives the described version of that header's types.
 * @param {string} header - The source header path.
 * @returns {string} The sidecar path.
 */
function sidecarName(header) {
  const dot = header.lastIndexOf(".");
  const slash = header.lastIndexOf("/");
  const stem = dot > slash ? header.slice(0, dot) : header;
  return stem + ".described.hpp";
}

/**
 * The sidecars of a record's described base classes: for each base that
 * resolves to a record or enum this generator also describes, the sidecar of
 * the header that base is defined in. Reflecting the record's inherited members
 * needs those base descriptors present.
 * @param {object} corpus - `ctx.corpus`.
 * @param {object} sym - The record whose bases to resolve.
 * @returns {string[]} The base sidecar paths, in base order.
 */
function baseSidecars(corpus, sym) {
  const sidecars = [];
  const baseList = sym.bases || [];
  for (let b = 0; b < baseList.length; ++b) {
    const id = baseList[b].type && baseList[b].type.name && baseList[b].type.name.id;
    const base = id ? corpus.get(id) : null;
    if (base && describeMacro(corpus, base) !== null) {
      const header = sourceHeaderOf(base);
      if (header) {
        sidecars.push(sidecarName(header));
      }
    }
  }
  return sidecars;
}

/**
 * Assemble one sidecar header: an include guard, a do-not-edit banner, an
 * include of the source header and of any base sidecars, the Boost.Describe
 * include, and each namespace's annotations wrapped in a `namespace { ... }`
 * block (a global-scope group, keyed by "", is emitted unwrapped).
 * @param {string} header - The source header this sidecar annotates.
 * @param {string[]} includes - Base sidecar paths to include first.
 * @param {string[]} nsOrder - Namespaces in emission order.
 * @param {Object<string, string[]>} byNs - Annotation lines per namespace.
 * @returns {string} The sidecar header text.
 */
function renderSidecar(header, includes, nsOrder, byNs) {
  const guard = guardName(sidecarName(header));
  let out =
    "#ifndef " + guard + "\n" +
    "#define " + guard + "\n\n" +
    "// Generated by MrDocs from documented types. Do not edit by hand.\n" +
    "//\n" +
    "// Include this to gain Boost.Describe reflection over the types declared\n" +
    "// in " + header + ".\n\n" +
    "#include \"" + header + "\"\n";
  for (let i = 0; i < includes.length; ++i) {
    out += "#include \"" + includes[i] + "\"\n";
  }
  out += "#include <boost/describe.hpp>\n\n";
  for (let g = 0; g < nsOrder.length; ++g) {
    const ns = nsOrder[g];
    if (ns) {
      out += "namespace " + ns + " {\n\n";
    }
    out += byNs[ns].join("\n") + "\n";
    if (ns) {
      out += "\n}  // namespace " + ns + "\n";
    }
    out += "\n";
  }
  out += "#endif  // " + guard + "\n";
  return out;
}

/**
 * An include-guard macro for a sidecar path: uppercased, every run of
 * non-alphanumeric characters collapsed to a single "_", with a leading "_"
 * added if the result would start with a digit.
 * @param {string} sidecar - The sidecar path.
 * @returns {string} The guard macro name.
 */
function guardName(sidecar) {
  let guard = sidecar.toUpperCase().replace(/[^A-Z0-9]+/g, "_");
  if (/^[0-9]/.test(guard)) {
    guard = "_" + guard;
  }
  return guard;
}

  • circle.hpp

  • shape.hpp

  • vec2.hpp

  • level.hpp

#ifndef APP_CIRCLE_HPP
#define APP_CIRCLE_HPP

#include "shape.hpp"

namespace app {

/// A circle.
struct Circle : Shape
{
    /// The radius.
    double radius;

    /// The enclosed area.
    double area() const;
};

} // namespace app

#endif
#ifndef APP_SHAPE_HPP
#define APP_SHAPE_HPP

namespace app {

/// A named shape.
struct Shape
{
    /// The display name.
    const char* name;
};

} // namespace app

#endif
#ifndef APP_VEC2_HPP
#define APP_VEC2_HPP

namespace app {

/// A 2-D vector.
struct Vec2
{
    /// The x component.
    double x;
    /// The y component.
    double y;

    /// The Euclidean length.
    double length() const;
};

} // namespace app

#endif
#ifndef APP_LEVEL_HPP
#define APP_LEVEL_HPP

namespace app {

/// Logging severity.
enum class Level
{
    debug,    ///< Verbose tracing.
    info,     ///< Normal operation.
    warning,  ///< Something looks wrong.
    error     ///< A failure occurred.
};

} // namespace app

#endif
mrdocs.yml
addons-supplemental:
  - addons
generator: describe
multipage: false
show-namespaces: false
warn-if-undocumented: false
source-root: include
input:
  - include

A non-intrusive BOOST_DESCRIBE_STRUCT has to sit at the type’s namespace scope with the type and its bases complete, so the extension emits one sidecar header per input header. In the example, for circle.hpp it writes circle.described.hpp, which includes circle.hpp and then the annotations for the types that header declares, including the annotations for the base class. A project can include the sidecar wherever it wants reflection over that header’s types. Select it with --generator=describe:

Generated sidecars

  • circle.described.hpp

  • shape.described.hpp

  • vec2.described.hpp

  • level.described.hpp

#ifndef CIRCLE_DESCRIBED_HPP
#define CIRCLE_DESCRIBED_HPP

// Generated by MrDocs from documented types. Do not edit by hand.
//
// Include this to gain Boost.Describe reflection over the types declared
// in circle.hpp.

#include "circle.hpp"
#include "shape.described.hpp"
#include <boost/describe.hpp>

namespace app {

BOOST_DESCRIBE_STRUCT(Circle, (Shape), (radius, area))

}  // namespace app

#endif  // CIRCLE_DESCRIBED_HPP
#ifndef SHAPE_DESCRIBED_HPP
#define SHAPE_DESCRIBED_HPP

// Generated by MrDocs from documented types. Do not edit by hand.
//
// Include this to gain Boost.Describe reflection over the types declared
// in shape.hpp.

#include "shape.hpp"
#include <boost/describe.hpp>

namespace app {

BOOST_DESCRIBE_STRUCT(Shape, (), (name))

}  // namespace app

#endif  // SHAPE_DESCRIBED_HPP
#ifndef VEC2_DESCRIBED_HPP
#define VEC2_DESCRIBED_HPP

// Generated by MrDocs from documented types. Do not edit by hand.
//
// Include this to gain Boost.Describe reflection over the types declared
// in vec2.hpp.

#include "vec2.hpp"
#include <boost/describe.hpp>

namespace app {

BOOST_DESCRIBE_STRUCT(Vec2, (), (x, y, length))

}  // namespace app

#endif  // VEC2_DESCRIBED_HPP
#ifndef LEVEL_DESCRIBED_HPP
#define LEVEL_DESCRIBED_HPP

// Generated by MrDocs from documented types. Do not edit by hand.
//
// Include this to gain Boost.Describe reflection over the types declared
// in level.hpp.

#include "level.hpp"
#include <boost/describe.hpp>

namespace app {

BOOST_DESCRIBE_ENUM(Level, debug, info, warning, error)

}  // namespace app

#endif  // LEVEL_DESCRIBED_HPP

Notice the level of customization provided by extensions here. There are so many ways to achieve this that no tool could ever ship it as a built-in feature. A programmatic generator provides you with the corpus, and the project can specify the final customization step.

Round-trip extensions

An extension can register a transform and a generator together, so one addon covers both directions of the same job. For instance, a common task in C++ projects is keeping documentation examples and standalone snippet files in sync:

  • Import example snippets from test or example files into the documentation comments when they are complex enough to be kept in a separate file, typically because they need special setup or teardown code to compile and run.

  • Exporting snippets from documentation comments to testable source files when the snippets are short enough to be written inline.

In the extension:

  • A corpus transform script can read example files and inject their snippets into the matching symbol’s docs.

  • A generator script can write every @code example out to a file the build can compile, so an example in a comment is broken.

An extension that implements both can keep the two in sync: the generator exports snippets from the docs to files if they weren’t t included by the transform function.

  • Extension

  • Headers

  • Example file

  • Config

addons/extensions/snippets.lua
-- Fingerprints of the snippets the transform imported, kept so the generator can
-- skip re-exporting an example that already lives in a compiled file. This is
-- module-level state shared between the two callbacks: the transform fills it,
-- the generator reads it.
local imported = {}

-- Import: for each symbol with no example of its own, append the snippets tagged
-- with its name in `<source>/<header-stem>.cpp` to its documentation, after any
-- description it already carries, and remember them so export skips them.
mrdocs.register_transform("snippets", function(ctx)
  local root = (ctx.params and ctx.params.source) or "example-code"
  local files = {}
  for _, sym in ipairs(ctx.corpus.symbols) do
    if sym.name ~= nil and #code_blocks(sym) == 0 then
      local header = header_of(sym)
      if header then
        local path = root .. "/" .. stem_of(header) .. ".cpp"
        if files[path] == nil then
          files[path] = read_file(path) or false
        end
        local text = files[path]
        if text then
          local snippets = extract_snippets(text, sym.name)
          if #snippets > 0 then
            if not sym.doc then
              sym.doc = {}
            end
            -- A script assigns a fresh block list rather than mutating one in
            -- place, so rebuild the description and append the imported snippets.
            local doc = {}
            for _, b in ipairs(sym.doc.document or {}) do
              doc[#doc + 1] = to_plain(b)
            end
            -- MrDocs heads an inline @code example with an "Example"/"Examples"
            -- heading; add the same before the imported snippets so they read
            -- the same as inline ones.
            local label = (#snippets == 1) and "Example" or "Examples"
            doc[#doc + 1] = {
              kind = "heading",
              children = { { kind = "text", literal = label } },
            }
            for _, snippet in ipairs(snippets) do
              doc[#doc + 1] = { kind = "code", literal = snippet }
              mark_imported(snippet, header)
            end
            sym.doc.document = doc
          end
        end
      end
    end
  end
end)

-- Export: group every symbol's inline @code examples by header and write one
-- runnable test file per header, `<header-stem>.cpp` in the generator's output
-- directory (`ctx.outputPath`, here `generator-options.snippets.output`), one
-- function per symbol plus a `main` that calls them all.
mrdocs.register_generator("snippets", function(ctx)
  local groups = {}
  local order = {}
  for _, sym in ipairs(ctx.corpus.symbols) do
    if sym.name ~= nil then
      local header = header_of(sym)
      if header then
        -- Export only the snippets written inline in the header; the ones the
        -- transform pulled from example files are already tested where they live.
        local blocks = {}
        for _, literal in ipairs(code_blocks(sym)) do
          if not was_imported(literal, header) then
            blocks[#blocks + 1] = literal
          end
        end
        if #blocks > 0 then
          if not groups[header] then
            groups[header] = {}
            order[#order + 1] = header
          end
          local g = groups[header]
          g[#g + 1] = { name = sym.name, line = line_of(sym), literals = blocks }
        end
      end
    end
  end
  for _, header in ipairs(order) do
    local syms = groups[header]
    table.sort(syms, function(a, b) return a.line < b.line end)
    local seen = {}
    local parts = { "#include \"" .. header .. "\"\n#include <cassert>\n" }
    local calls = {}
    for _, sym in ipairs(syms) do
      local fn, def = symbol_function(sym.name, sym.literals, seen)
      parts[#parts + 1] = def
      calls[#calls + 1] = "    " .. fn .. "();"
    end
    parts[#parts + 1] = "int main()\n{\n" .. table.concat(calls, "\n") .. "\n}\n"
    ctx.output.write(stem_of(header) .. ".cpp", table.concat(parts, "\n"))
  end
end)

The helpers that read the example files, extract and dedent the snippets, fingerprint them and convert values are in the full file, snippets.lua.

  • arithmetic.hpp

  • text.hpp

#ifndef APP_ARITHMETIC_HPP
#define APP_ARITHMETIC_HPP

namespace app {

/** Add two integers.

    Returns the arithmetic sum of the two arguments. The example lives in this
    comment and is exported to a file the build compiles, so it never falls out
    of date.

    @par Example

    @code
    int s = app::add(2, 3);
    assert(s == 5);
    @endcode
*/
int add(int a, int b);

/** Multiply two integers.

    Returns the product of the two arguments. Its examples are maintained in a
    separate file.
*/
int multiply(int a, int b);

/** Subtract the second integer from the first.

    Returns `a - b`. Its example is maintained in a separate file.
*/
int subtract(int a, int b);

} // namespace app

#endif
#ifndef APP_TEXT_HPP
#define APP_TEXT_HPP

namespace app {

/** The number of characters before the terminating null.

    Counts the bytes up to, but not including, the null terminator.

    @par Example

    @code
    assert(app::length("abc") == 3);
    @endcode
*/
int length(char const* s);

/** True when the string has no characters.

    Only the empty string is blank; any other string, including whitespace, is
    not.

    @par Examples

    @code
    assert(app::is_empty(""));
    @endcode

    @code
    assert(!app::is_empty("abc"));
    @endcode
*/
bool is_empty(char const* s);

} // namespace app

#endif
example-code/arithmetic.cpp
#include "arithmetic.hpp"
#include <cassert>

void examples()
{
    //[multiply
    int p = app::multiply(4, 5);
    assert(p == 20);
    //]

    //[multiply
    int q = app::multiply(-2, 3);
    assert(q == -6);
    //]

    //[subtract
    int d = app::subtract(9, 4);
    assert(d == 5);
    //]
}
mrdocs.yml
addons-supplemental:
  - addons
generator:
  - adoc
  - snippets
multipage: false
show-namespaces: false
warn-if-undocumented: false
source-root: include
input:
  - include
transform-options:
  snippets:
    source: example-code
generator-options:
  snippets:
    output: exported

In the input, add keeps its example inline in the header comment, while multiply and subtract import theirs from a separate example file. The transform reads its inputs from transform-options.snippets.source, and the generator writes the inline snippets to generator-options.snippets.output. The rendered page then shows every example regardless of where it came from:

Preview
app::add

Add two integers.

Synopsis

Declared in <arithmetic.hpp>

int
add(
    int a,
    int b);
Description

Returns the arithmetic sum of the two arguments. The example lives in this comment and is exported to a file the build compiles, so it never falls out of date.

Example
int s = app::add(2, 3);
assert(s == 5);
app::is_empty

True when the string has no characters.

Synopsis

Declared in <text.hpp>

bool
is_empty(char const* s);
Description

Only the empty string is blank; any other string, including whitespace, is not.

Examples
assert(app::is_empty(""));
assert(!app::is_empty("abc"));
app::length

The number of characters before the terminating null.

Synopsis

Declared in <text.hpp>

int
length(char const* s);
Description

Counts the bytes up to, but not including, the null terminator.

Example
assert(app::length("abc") == 3);
app::multiply

Multiply two integers.

Synopsis

Declared in <arithmetic.hpp>

int
multiply(
    int a,
    int b);
Description

Returns the product of the two arguments. Its examples are maintained in a separate file.

Examples
int p = app::multiply(4, 5);
assert(p == 20);
int q = app::multiply(-2, 3);
assert(q == -6);
app::subtract

Subtract the second integer from the first.

Synopsis

Declared in <arithmetic.hpp>

int
subtract(
    int a,
    int b);
Description

Returns a ‐ b. Its example is maintained in a separate file.

Example
int d = app::subtract(9, 4);
assert(d == 5);

Export runs comments into files, and writes complete translation units, one per header. Each file includes its header and a small prelude, gives every symbol a function that runs its snippets (each in its own scope when there is more than one), and ends with a main that calls them all, so the file both compiles and runs as a test. It exports only the examples written inline in the headers. It leaves out the ones the transform imported, since they already live in a file the build compiles. exported/arithmetic.cpp therefore holds only add, not the imported multiply and subtract:

Exported test files

  • exported/arithmetic.cpp

  • exported/text.cpp

#include "arithmetic.hpp"
#include <cassert>

void add_snippet()
{
    int s = app::add(2, 3);
    assert(s == 5);
}

int main()
{
    add_snippet();
}
#include "text.hpp"
#include <cassert>

void length_snippet()
{
    assert(app::length("abc") == 3);
}

void is_empty_snippet()
{
    {
        assert(app::is_empty(""));
    }
    {
        assert(!app::is_empty("abc"));
    }
}

int main()
{
    length_snippet();
    is_empty_snippet();
}

Knowing which examples to skip is where the two halves of the extension meet. The transform function records a fingerprint of each snippet, and the generator reads that same table to skip them.

This extension can also compose with the ghostwriter transform. A typical problem with technical writers is that their examples are only tested locally, so they can drift and break as the API moves. This extension would ensure these import snippets are also tested as inline scripts.