Skip to content

MPI Bugs Inside Post Processing #1110

@Cowsreal

Description

@Cowsreal

Describe the bug
Segmentation faults occur during post processing. Not sure why but I've only gotten this to happen in the 1D examples. I've been able to track the issue successfully to the subroutine inside of pre_process/m_data_input.f90

impure subroutine s_read_parallel_data_files(t_step)
It occurs at line 383 at MPI_FILE_READ, when running with 1+ MPI processes. What happens is that after executing rank 0 segmentation faults on MPI_FILE_READ

Look at the following print statements.

Image

All MPI ranks except for 0 are able to print up to "ifile2", except for rank 0. Rank zero does not print "BYE: proc:", so the segmentation fault occurs for rank 0 inside the MPI_FILE_READ call.

Environment

  • OS: MacOs Tahoe 26.0.1

  • Compiler versions GNU 15.2.0

  • OpenMPI 5.0.8

  • Frontier

  • Cray 19.0

I've tried printing size(x_cb_glb) as well as data_size, and it seems correct. I've also tried to print wp and mpI_p and verified that they both are correctly set to double precision.

I think there may be another MPI bug somewhere else in post processing, as repeatly running post processing does not guarantee always finding this bug. Sometimes it crashes somewhere else, and sometimes it fully completes without crashing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working or doesn't seem right

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions