Skip to content

create hello-nf-on-vwb example#24

Open
samhornstein wants to merge 5 commits into
mainfrom
samh/nextflow-workbench-starter
Open

create hello-nf-on-vwb example#24
samhornstein wants to merge 5 commits into
mainfrom
samh/nextflow-workbench-starter

Conversation

@samhornstein

@samhornstein samhornstein commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds nextflow/hello-nf-on-vwb/, a minimal, self-contained Nextflow pipeline that runs unchanged locally or on Verily Workbench via Google Batch. Modeled on the official
Hello Nextflow training: reads greetings from a CSV, uppercases them in parallel, collects into one file. The point is the
Workbench wiring (profiles, params, containers, GCS work directory), not the string processing.

Also links the new example from nextflow/README.md.

What's included

  • Pipelinemain.nf plus modules/ (sayHello, convertToUpper, collectGreetings) and bundled sample data/greetings.csv.
  • Confignextflow.config with three profiles:
    • standard — local, no container
    • docker — local, containerized
    • verily_workbench — Google Batch executor, private-address VMs, Workbench VPC/subnet, project and service account pulled from env vars
  • Paramstest-params.yaml for overriding inputs/outdir.
  • DocsREADME.md covering local runs, Workbench setup (bucket creation), and running from both the Workflows UI and wb nextflow CLI.

Design notes

  • One pipeline, many deploys: same code runs on laptop or Batch; only paths change. Profiles carry the environment differences.
  • Task cpus/memory live on the process block (nf-core conf/base.config pattern), not the executor profile; Google Batch sizes the VM from these. Boot disk stays in the
    profile.
  • Config via params/env vars, not hardcoded values. outdir/work_dir must be full gs:// paths on Workbench — pipeline fails fast otherwise.
  • Public debian:stable-slim image by default; README documents swapping in a private Artifact Registry image.

Commits

  • feat(hello-nf-on-vwb): minimal Nextflow-on-Verily-Workbench example
  • refactor: task cpus/memory to process block, boot disk in profile (nf-core pattern)
  • refactor: rename profile to verily_workbench, params over env vars
  • docs/fix: README simplification, drop logsPath, clarify results vs work dir

Test plan

  • nextflow run main.nf -profile standard produces results/COLLECTED-greetings.txt
  • nextflow run main.nf -profile docker (Docker running)
  • wb nextflow run main.nf -profile verily_workbench with gs:// paths
  • Register + run from Workflows UI

@samhornstein samhornstein changed the title create hello-nf-on-wb example create hello-nf-on-vwb example Jul 20, 2026
@samhornstein
samhornstein marked this pull request as ready for review July 21, 2026 22:54
@samhornstein
samhornstein requested review from PeterSu92 and kirtanav98 and removed request for PeterSu92 July 21, 2026 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant