Python API

nf-docs is a CLI tool first, but it's also a normal Python package. Import it when you need something the CLI doesn't cover: feeding pipeline data into your own tooling, or rendering docs as part of a site build.

pip install nf-docs

The requirements are the same as for the CLI: Java for the Nextflow Language Server, and optionally Nextflow itself for parsing nextflow.config.

Four functions

Four functions cover most of it:

import nf_docs

pipeline = nf_docs.extract("./my_pipeline")
markdown = nf_docs.render(pipeline, "markdown")
pages = nf_docs.render_pages(pipeline, "markdown")
files = nf_docs.generate("./my_pipeline", output_format="html", output="site/")

Unlike the CLI, these never print to the console and never exit the process. They return values and raise exceptions.

Reading pipeline data

extract() gives you a Pipeline object. Use it when you want the data rather than the documentation.

import nf_docs

pipeline = nf_docs.extract("./my_pipeline")

print(pipeline.metadata.name, pipeline.metadata.version)

for process in pipeline.processes:
    print(f"{process.name}: {process.docstring}")

# Required parameters that have no default
for param in pipeline.inputs:
    if param.required and param.default is None:
        print(f"{param.name} ({param.type}) - {param.description}")

Every model has a to_dict(), so handing the whole thing to another tool is one call:

import json

with open("pipeline-api.json", "w") as fh:
    json.dump(pipeline.to_dict(), fh, indent=2)

Rendering into an existing build

render() returns a string. That's usually what you want when another tool owns the output directory, such as a static site generator.

from pathlib import Path
import nf_docs

pipeline = nf_docs.extract("./my_pipeline")

Path("src/content/docs/api.md").write_text(
    "---\ntitle: Pipeline API\n---\n\n" + nf_docs.render(pipeline, "markdown")
)

Renderer-specific options pass straight through:

nf_docs.render(pipeline, "json", indent=4)
nf_docs.render(pipeline, "yaml", default_flow_style=True)
nf_docs.render(pipeline, "html", use_tailwind=False)
nf_docs.render(pipeline, "markdown", title="My Pipeline")

Pass single_file=True for the focused single-document form used for modules, rather than the full pipeline layout.

Rendering pages in memory

render() returns one combined document. Markdown normally spreads across several files, and render_pages() gives you those files as strings without going near the filesystem:

import nf_docs

pipeline = nf_docs.extract("./my_pipeline")

for filename, content in nf_docs.render_pages(pipeline, "markdown").items():
    print(f"{filename}: {len(content)} characters")

The keys are the names generate() would write, relative to the output directory. It's the in-memory equivalent, so a build hook that owns its own output doesn't have to generate into a temporary directory and read the files back:

pages = nf_docs.render_pages(pipeline, "markdown")

for filename, content in pages.items():
    my_site.add_page(f"api/{filename}", content)

Don't assume a fixed set of keys. Markdown always produces index.md and inputs.md, and adds the other four only when the pipeline has something to put in them:

Reproducible output

By default, every format except YAML embeds the nf-docs version and the time of the run, so two builds of an unchanged pipeline differ. Pass include_generation_info=False to turn that off and get byte-identical output:

import nf_docs

pages = nf_docs.render_pages(pipeline, "markdown", include_generation_info=False)
nf_docs.generate("./my_pipeline", output_format="html", include_generation_info=False)

It's a renderer option, so it works anywhere renderer keyword arguments do — render(), render_pages(), generate(), or a renderer you built yourself. What it removes:

HTML keeps its footer either way: the Nextflow attribution and the "generated by nf-docs vX" line are the same on every run, so only the timestamp beside them goes.

Writing files

generate() is the equivalent of nf-docs generate. It returns the paths it wrote:

import nf_docs

for path in nf_docs.generate("./my_pipeline", output_format="html", output="site/"):
    print(f"wrote {path}")

Omit output and it follows the same conventions as the CLI: <pipeline>/docs/ for a whole pipeline, and a file alongside the source when documenting a single module.

Documenting a single module

Pass a path to a .nf file to document just that file. nf-docs also auto-detects a directory holding a module-style main.nf (process definitions, no workflow, no pipeline config), so both of these do the same thing:

nf_docs.generate("modules/mytool/main.nf", output_format="md")
nf_docs.generate("modules/mytool", output_format="md")

Both write modules/mytool/README.md.

To document every module in a repository:

from pathlib import Path
import nf_docs

for main_nf in Path("modules").rglob("main.nf"):
    nf_docs.generate(main_nf, output_format="md")

Progress reporting

Extraction can take a while, because the Language Server has to start and index the workspace. Pass a progress_callback to drive your own progress display. It receives ProgressUpdate objects.

import nf_docs

def show(update: nf_docs.ProgressUpdate) -> None:
    if update.has_progress:
        print(f"{update.message} [{update.current}/{update.total}] {update.detail or ''}")
    else:
        print(update.message)

pipeline = nf_docs.extract("./my_pipeline", progress_callback=show)

rich ships with nf-docs, so a progress bar costs no extra dependency. Passing None for completed or total leaves those values alone, so the same callback handles both the countable and indeterminate phases:

from rich.progress import BarColumn, Progress, SpinnerColumn, TextColumn

import nf_docs

with Progress(
    SpinnerColumn(),
    TextColumn("[progress.description]{task.description}"),
    BarColumn(),
) as progress:
    task = progress.add_task("Starting...", total=None)

    def show(update: nf_docs.ProgressUpdate) -> None:
        progress.update(
            task,
            description=update.message,
            completed=update.current,
            total=update.total,
        )

    pipeline = nf_docs.extract("./my_pipeline", progress_callback=show)

update.phase is an ExtractionPhase member if you want to react to specific stages (LSP_INDEXING, PARSING_SCHEMA, COMPLETE, and so on).

Configuration

The CLI reads ~/.config/nf-docs/config.yaml. The Python API deliberately does not: a library call inside someone else's build script shouldn't change behaviour based on whoever happens to be running it. Library calls use the defaults unless you pass a config explicitly:

import nf_docs

# Defaults - reproducible regardless of the user's config file
nf_docs.extract("./my_pipeline")

# Opt in to the user's config file
nf_docs.extract("./my_pipeline", config=nf_docs.load_config())

# Or set options directly
config = nf_docs.NfDocsConfig(ignore_config_prefixes=["genomes.", "test."])
nf_docs.extract("./my_pipeline", config=config)

The fields, their defaults and what each one does are listed under Configuration file. Two things specific to the API:

• default_format is read only by nf-docs generate. Setting it on a config you pass to extract() or generate() does nothing — the API takes its format as an argument.
• Passing a config that isn't an NfDocsConfig (say, a plain dict) isn't supported. Build one with NfDocsConfig(...), or NfDocsConfig.from_dict(...) if you have a mapping and want the same wrong-type checking load_config() applies.

Caching

nf-docs caches extraction results in ~/.cache/nf-docs/, keyed by pipeline path, nf-docs version, a hash of the pipeline's files, and the configuration used. The config is part of the key so a CLI run and a library call on the same pipeline don't return each other's results. The cache is on by default, matching the CLI:

nf_docs.extract("./my_pipeline", use_cache=False)     # ignore the cache entirely
nf_docs.extract("./my_pipeline", force_refresh=True)  # re-extract, then update the cache

from pathlib import Path

from nf_docs import PipelineCache

PipelineCache().clear(Path("./my_pipeline"))  # one pipeline; takes a Path
PipelineCache().clear()  # everything

Errors

The API raises rather than exiting. The exceptions worth catching:

import nf_docs

try:
    pipeline = nf_docs.extract("./my_pipeline")
except nf_docs.LSPError as e:
    print(f"Language Server problem: {e}")
except nf_docs.ExtractionError as e:
    print(f"Could not extract: {e}")

A missing or unparseable individual source (no schema, a broken nextflow.config) logs a warning and carries on rather than raising. Nothing reaches the console by default. Attach a handler to the nf_docs logger to see these:

import logging

logging.getLogger("nf_docs").addHandler(logging.StreamHandler())
logging.getLogger("nf_docs").setLevel(logging.INFO)

The Pipeline model

Everything is a plain @dataclass, so to_dict(), dataclasses.asdict() and ordinary attribute access all work.

Some helpers:

pipeline.has_content()                  # did we find anything at all?
pipeline.get_entry_workflow()           # the entry workflow, or None
pipeline.get_process_by_name("FASTQC")  # a Process, or None
pipeline.get_input_groups()             # inputs grouped by schema section
pipeline.to_dict()                      # JSON-compatible dict

Progress updates

ProgressUpdate objects carry:

Lower-level pieces

The three functions above wrap classes you can use directly if you need more control:

from nf_docs import PipelineExtractor, get_renderer

extractor = PipelineExtractor(
    workspace_path="./my_pipeline",
    target_file="./my_pipeline/modules/mytool/main.nf",  # single-file mode
    language_server_jar="/path/to/language-server-all.jar",
    nextflow_path="/usr/local/bin/nextflow",
)
pipeline = extractor.extract()

renderer = get_renderer("markdown")(title="My Pipeline")
files = renderer.render_to_directory(pipeline, "docs/")

Subclass BaseRenderer to add your own output format, implementing render() and render_pages(). render_to_directory() is provided: it writes whatever render_pages() returns. Override it only if the result has to depend on what's already in the destination, as the table renderer's does.

What's public

Anything re-exported from the top-level nf_docs namespace, plus the models in nf_docs.models, is the supported API. Everything else may change without notice: underscore-prefixed helpers, the Language Server client internals, the parser modules.

nf-docs is pre-1.0, so the public API may still change between minor releases. Pin a version if that matters to you.