Skip to content

SIGFPE (divide-by-zero) in GIOI_Lustre_open() opening any file on Lustre when built without HAVE_LUSTRE #234

Description

@whannah1

NOTE - I don't really understand the problem here. I was using a bot to debug something else (MOAB/MBDA) and it stumbled on this. I asked the bot to write up the issue below, so feel free to close this if it doesn't make sense. I have personally confirmed the reproducer, the crash, and the nc_driver=mpiio workaround on the OLCF Andes machine.

Summary

PnetCDF 1.15.0 crashes with SIGFPE (integer divide-by-zero) inside GIOI_Lustre_open() when
opening any existing file on a Lustre filesystem, if the library was built without Lustre
API support (HAVE_LUSTRE undefined). The GIO layer selects the Lustre driver at runtime based
on the statfs() f_type magic (gio/src/gio_fstype.c), regardless of whether the build has any
way to query file striping — and the open path then divides two zero-initialized values.

The crash occurs on the very first ncmpi_open() call, with a single process, for any file
format (CDF-1/2/5) and any striping layout (reproduced with both plain stripe_count=1 layouts
and PFL/DoM composite layouts).

This affects the current conda-forge libpnetcdf 1.15.0 packages (built without liblustreapi
— the shared library contains no llapi_* references), so any conda-forge-based application
(e.g. MOAB tools) crashes out of the box on Lustre systems.

Environment

  • PnetCDF 1.15.0 ("PnetCDF library version 1.15.0 of July 1, 2026"), conda-forge build libpnetcdf 1.15.0 mpi_mpich_hb983576_201
  • MPICH 5.0.1 (conda-forge), x86_64
  • OLCF Andes (login node), RHEL 8.10, kernel 4.18.0-553
  • Lustre client 2.15.3.2_cray (Orion//lustre/orion filesystem)

Reproducer

#include <stdio.h>
#include <mpi.h>
#include <pnetcdf.h>

int main(int argc, char **argv) {
    MPI_Init(&argc, &argv);
    int ncid, err;
    err = ncmpi_open(MPI_COMM_WORLD, argv[1], NC_NOWRITE, MPI_INFO_NULL, &ncid);
    printf("ncmpi_open: err=%d (%s)\n", err, ncmpi_strerror(err));
    MPI_Finalize();
    return 0;
}
$ mpicc pnc_test.c -o pnc_test -lpnetcdf
$ ./pnc_test /lustre/orion/.../any_valid_file.nc     # single process, no mpiexec needed
Floating point exception (core dumped)

The same binary run on a non-Lustre filesystem (or with PNETCDF_HINTS="nc_driver=mpiio")
works fine, confirming the file itself is valid.

Backtrace

Program terminated with signal SIGFPE, Arithmetic exception.
#0  0x... in GIOI_Lustre_open () from libpnetcdf.so.8
#1  0x... in GIO_open ()
#2  0x... in ncmpio_open ()
#3  0x... in ncmpi_open ()
#4  0x... in main ()

=> GIOI_Lustre_open+245:  idiv  %esi
rax  0x0   0
rsi  0x0   0     <- 0 / 0

Root cause

In gio/src/gio_lustre_open.c, GIOI_Lustre_open() (line 1173 in the 1.15.0 release tarball):

  1. stripe_count, numOSTs are initialized to 0 (lines ~1188-1191).
  2. The code that queries actual striping is guarded by #ifdef HAVE_LUSTRE (llapi path) /
    #elif defined(MIMIC_LUSTRE). When neither is defined at build time, both variables
    remain 0.
  3. After the MPI_Bcast of stripin_info[], the open path only checks
    stripin_info[0] != GIO_NOERR before computing (line 1251):
fh->hints->overstriping_ratio = stripin_info[2] / stripin_info[4];   /* 0 / 0 -> SIGFPE */

Note the create path has the missing guard (line ~1132 checks stripin_info[4] == 0 before
the equivalent division at line 1142), but the open path does not.

Additional latent issues even for builds with HAVE_LUSTRE:

  • get_striping() explicitly return 0 when llapi_layout_stripe_count_get() reports
    LLAPI_LAYOUT_DEFAULT / LLAPI_LAYOUT_WIDE / invalid (lines ~380-386). On systems where the
    filesystem default is a PFL/composite layout (e.g. OLCF Orion, which uses progressive layouts
    with a Data-on-MDT first component reporting stripe_count: 0), this returns 0 for ordinary
    files created with default striping, which would feed the same unguarded division in the open
    path.
  • Lustre_set_cb_node_list() divides by striping_factor
    (rounds = num_aggr / striping_factor, line 689), which is also unguarded if
    striping_factor == 0 gets that far.
  • GIOI_Lustre_open_on_demand() may warrant the same check.

Suggested fix

  • In GIOI_Lustre_open(), mirror the create path's guard: check stripin_info[4] == 0 (and
    stripin_info[2] <= 0) after the Bcast, and either fall back to sane defaults (e.g.
    striping_factor=1, overstriping_ratio=1) or fall back to the generic/MPI-IO path instead
    of dividing.
  • Consider not selecting the Lustre GIO driver at all when the library is built without
    HAVE_LUSTRE/MIMIC_LUSTRE, since it has no way to obtain striping information.

Workaround

PNETCDF_HINTS="nc_driver=mpiio" avoids the GIO layer entirely and works correctly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions