Adding MUSCL Reconstruction

[okBrian]

User description

Description

Added MUSCL Reconstruction with Interface Compression.

Fixes #277

Type of change

Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)

Scope

• This PR comprises a set of related changes with a common goal

If you cannot check the above box, please split your PR into multiple PRs that each have a common goal.

How Has This Been Tested?

• Validated using Shockdroplet, Riemann_test, Rayleigh_taylor, advection, and sodshocktube between WENO, and MUSCL
• Regression Tests.

Test Configuration:
Phoenix, Frontier, Locally using GCC 15.1, and NVHPC 24.5

Checklist

• I have added comments for the new code
• I added Doxygen docstrings to the new code
• I have made corresponding changes to the documentation (docs/)
• I have added regression tests to the test suite so that people can verify in the future that the feature is behaving as expected
• I have added example cases in examples/ that demonstrate my new feature performing as expected.
  They run to completion and demonstrate "interesting physics"
• I ran ./mfc.sh format before committing my code
• New and existing tests pass locally with my changes, including with GPU capability enabled (both NVIDIA hardware with NVHPC compilers and AMD hardware with CRAY compilers) and disabled
• This PR does not introduce any repeated code (it follows the DRY principle)
• I cannot think of a way to condense this code and reduce any introduced additional line count

If your code changes any code source files (anything in src/simulation)

To make sure the code is performing as expected on GPU devices, I have:

• Checked that the code compiles using NVHPC compilers
• Checked that the code compiles using CRAY compilers
• Ran the code on either V100, A100, or H100 GPUs and ensured the new feature performed as expected (the GPU results match the CPU results)
• Ran the code on MI200+ GPUs and ensure the new features performed as expected (the GPU results match the CPU results)
• Enclosed the new feature via nvtx ranges so that they can be identified in profiles
• Ran a Nsight Systems profile using ./mfc.sh run XXXX --gpu -t simulation --nsys, and have attached the output file (.nsys-rep) and plain text results to this PR
• Ran a Rocprof Systems profile using ./mfc.sh run XXXX --gpu -t simulation --rsys --hip-trace, and have attached the output file and plain text results to this PR.
• Ran my code using various numbers of different GPUs (1, 2, and 8, for example) in parallel and made sure that the results scale similarly to what happens if you run without the new code/feature

---

PR Type

Enhancement

---

Description

• Added complete MUSCL (Monotonic Upstream-centered Scheme for Conservation Laws) reconstruction implementation as an alternative to WENO
• Implemented first and second-order MUSCL schemes with multiple slope limiters (Van Albada, MC, Minmod, Superbee, UMIST)
• Added THINC (Tangent of Hyperbola for Interface Capturing) interface compression method for sharp interface preservation
• Integrated MUSCL reconstruction throughout the codebase: viscous module, RHS computation, CBC, surface tension, and MPI support
• Added comprehensive input validation and parameter checking for MUSCL-specific settings
• Created extensive test suite with 6 new example cases: Sod shock tube, shock-droplet interactions, Riemann tests, Rayleigh-Taylor instability, and advection
• Updated documentation with detailed parameter descriptions for reconstruction type selection and MUSCL configuration
• Added GPU acceleration support with OpenACC directives for MUSCL reconstruction
• Extended toolchain to support MUSCL parameter preprocessing and test case generation

---

Diagram Walkthrough

flowchart LR
  A["WENO Reconstruction"] --> B["Generic Reconstruction Interface"]
  C["New MUSCL Module"] --> B
  B --> D["Viscous Module"]
  B --> E["RHS Computation"]
  B --> F["Surface Tension"]
  B --> G["CBC Module"]
  C --> H["THINC Interface Compression"]
  I["New Parameters"] --> C
  I --> J["Input Validation"]
  K["Test Cases"] --> L["Validation Suite"]
  M["Documentation"] --> N["User Guide"]

Loading

File Walkthrough

	Relevant files
Enhancement	20 files m_viscous.fpp Integrate MUSCL reconstruction into viscous module              src/simulation/m_viscous.fpp • Added import for m_muscl module • Replaced hardcoded WENO calls with templated reconstruction scheme selection • Introduced recon_dir variable to replace weno_dir for generic reconstruction direction • Added debug print statement in viscous reconstruction loop +147/-139 m_muscl.fpp Add new MUSCL reconstruction module                                            src/simulation/m_muscl.fpp • New complete MUSCL reconstruction module implementation • Implements first and second-order MUSCL schemes with multiple limiters • Includes interface compression using THINC method • Provides GPU-accelerated reconstruction with OpenACC directives +373/-0  m_cbc.fpp Extend CBC module for MUSCL reconstruction support              src/simulation/m_cbc.fpp • Extended CBC coefficient allocation to support both WENO and MUSCL schemes • Added conditional logic for reconstruction type selection in CBC computations • Modified buffer size calculations to accommodate MUSCL polynomial degrees • Updated deallocation routines for both reconstruction types +41/-29  m_rhs.fpp Integrate MUSCL reconstruction into RHS computation            src/simulation/m_rhs.fpp • Added import for m_muscl module • Replaced hardcoded WENO reconstruction calls with templated scheme selection • Modified reconstruction direction variable naming for genericity • Extended first-order reconstruction to support both WENO and MUSCL types +90/-78  m_global_parameters.fpp Add MUSCL and interface compression global parameters        src/simulation/m_global_parameters.fpp • Added new parameters for MUSCL reconstruction: recon_type, muscl_polyn, muscl_order, muscl_lim • Added interface compression parameters: int_comp, ic_eps, ic_beta • Extended case optimization macros to include MUSCL parameters • Updated GPU memory declarations for new parameters +30/-7    m_surface_tension.fpp Extend surface tension module for MUSCL reconstruction      src/simulation/m_surface_tension.fpp • Added import for m_muscl module • Modified capillary reconstruction to support both WENO and MUSCL schemes • Replaced hardcoded reconstruction calls with templated scheme selection • Updated reconstruction direction variable naming +60/-54  m_mpi_proxy.fpp Add MPI support for MUSCL parameters                                          src/simulation/m_mpi_proxy.fpp • Added MPI broadcast calls for new MUSCL parameters: recon_type, muscl_order, muscl_lim • Extended parameter broadcasting to include interface compression parameters • Updated logical parameter broadcast list for new features +11/-6    m_start_up.fpp Integrate MUSCL module initialization into startup              src/simulation/m_start_up.fpp • Added import for m_muscl module • Extended module initialization to support MUSCL reconstruction • Added conditional initialization and finalization for both WENO and MUSCL modules • Updated parameter declarations for new MUSCL and interface compression features +16/-4    m_checker.fpp Add input validation for MUSCL reconstruction                        src/simulation/m_checker.fpp • Added new input validation subroutine s_check_inputs_muscl • Extended input checking to validate MUSCL parameters and constraints • Added conditional reconstruction type checking in main validation routine +21/-1    m_global_parameters.fpp Add MUSCL parameters to pre-process global parameters        src/pre_process/m_global_parameters.fpp • Added MUSCL reconstruction parameters: recon_type, muscl_polyn, muscl_order • Updated parameter initialization with default values • Modified polynomial degree calculation to support both WENO and MUSCL • Extended coordinate bounds configuration for MUSCL support +14/-5    m_checker_common.fpp Add common MUSCL input validation                                                src/common/m_checker_common.fpp • Added new s_check_inputs_muscl subroutine for MUSCL parameter validation • Extended common input checking to support both WENO and MUSCL reconstruction types • Added constraints checking for MUSCL order and grid size requirements +14/-1    m_mpi_common.fpp Extend MPI domain decomposition for MUSCL support                src/common/m_mpi_common.fpp • Modified MPI domain decomposition to support both WENO and MUSCL reconstruction orders • Updated error messages to include MUSCL in reconstruction order references • Added conditional logic for reconstruction order determination +10/-3    m_helper_basic.fpp Extend coordinate bounds configuration for MUSCL                  src/common/m_helper_basic.fpp • Modified s_configure_coordinate_bounds to accept MUSCL polynomial parameters • Added buffer size calculation logic for MUSCL reconstruction type • Extended coordinate bounds configuration for multiple reconstruction schemes +6/-5      m_constants.fpp Add constants for MUSCL and interface compression                src/common/m_constants.fpp • Added reconstruction type constants: WENO_TYPE and MUSCL_TYPE • Added interface compression default parameters: dflt_ic_eps, dflt_ic_beta, moncon_cutoff • Defined new constants for MUSCL and THINC interface compression features +9/-0      m_mpi_proxy.fpp Add MPI broadcast for MUSCL parameters in pre-process        src/pre_process/m_mpi_proxy.fpp • Added MPI broadcast support for new MUSCL parameters: recon_type and muscl_order • Extended parameter broadcasting list to include reconstruction type selection +1/-2      m_global_parameters.fpp Add MUSCL parameters to post-process global parameters      src/post_process/m_global_parameters.fpp • Added MUSCL reconstruction parameters: recon_type and muscl_order • Updated parameter initialization with default values for post-processing • Extended global parameter declarations for reconstruction type support +4/-0      m_mpi_proxy.fpp Add MPI broadcast for MUSCL in post-process                            src/post_process/m_mpi_proxy.fpp • Added MPI broadcast support for muscl_order parameter • Extended parameter broadcasting for post-processing MUSCL support +1/-1      m_start_up.fpp Add MUSCL parameter reading in pre-process startup              src/pre_process/m_start_up.fpp • Added MUSCL parameters to input file reading: recon_type and muscl_order • Extended parameter declarations for pre-processing MUSCL support +2/-1      m_start_up.f90 Add MUSCL parameter reading in post-process startup            src/post_process/m_start_up.f90 • Added MUSCL parameters to input file reading: recon_type and muscl_order • Extended parameter declarations for post-processing MUSCL support +1/-1      case.py Add MUSCL parameters to case preprocessing                              toolchain/mfc/case.py • Adds MUSCL reconstruction parameters to simulation preprocessing • Introduces recon_type, muscl_order, muscl_polyn, and muscl_lim variables • Fixes bug with WENO order allocation when using MUSCL reconstruction +12/-3
Tests	10 files case.py Add 3D shock-droplet example with MUSCL reconstruction      examples/3D_shockdroplet_muscl/case.py • New 3D shock-droplet interaction example using MUSCL reconstruction • Configures MUSCL order 2 with Van Albada limiter and interface compression • Sets up water droplet in air with shock wave interaction physics +276/-0  case.py Add 1D Sod shock tube example with MUSCL reconstruction    examples/1D_sodshocktube_muscl/case.py • New 1D Sod shock tube example using MUSCL reconstruction • Configures MUSCL order 2 with MC limiter and interface compression • Implements classic shock tube test case for MUSCL validation +71/-0    case.py New 2D shock-droplet example with MUSCL reconstruction      examples/2D_shockdroplet_muscl/case.py • Creates a new 2D shock-droplet test case using MUSCL reconstruction • Configures simulation parameters including domain, patches, and fluid properties • Sets MUSCL-specific parameters: recon_type=2, muscl_order=2, muscl_lim=4, int_comp=T +123/-0  case.py New 2D Riemann test example with MUSCL reconstruction        examples/2D_riemann_test_muscl/case.py • Creates a new 2D Riemann test case using MUSCL reconstruction • Configures four patches with different initial conditions for Riemann problem • Sets MUSCL parameters: recon_type=2, muscl_order=2, muscl_lim=1, int_comp=T +99/-0    case.py New 3D Rayleigh-Taylor example with MUSCL reconstruction  examples/3D_rayleigh_taylor_muscl/case.py • Creates a new 3D Rayleigh-Taylor instability test case using MUSCL reconstruction • Includes viscous effects and body forces for gravity simulation • Sets MUSCL parameters: recon_type=2, muscl_order=2, muscl_lim=4, int_comp=T +111/-0  case.py New 2D advection example with MUSCL reconstruction              examples/2D_advection_muscl/case.py • Creates a new 2D advection test case using MUSCL reconstruction • Configures circular density patch for advection testing • Sets MUSCL parameters: recon_type=2, muscl_order=2, muscl_lim=2, int_comp=T +83/-0    cases.py Add MUSCL reconstruction test case generation                        toolchain/mfc/test/cases.py • Adds alter_muscl() function to generate MUSCL test variations • Creates test cases for different MUSCL orders (1,2) and limiters (1-5) • Integrates MUSCL testing into the test case generation pipeline +17/-0    golden.txt Golden reference data for MUSCL reconstruction tests          tests/86E9A6D4/golden.txt • Provides golden reference data for MUSCL reconstruction validation • Contains numerical results for conservative and primitive variables • Includes time evolution data at different simulation steps +16/-0    golden.txt New MUSCL reconstruction test golden reference data            tests/F33CC4CA/golden.txt • Added new golden test file containing numerical simulation output data • Contains density, momentum, energy, and pressure field data at different time steps • Includes both conservative and primitive variable outputs for validation +16/-0    golden-metadata.txt Test metadata for MUSCL reconstruction validation                tests/86E9A6D4/golden-metadata.txt • Added test metadata file with build configuration details • Contains CMake configuration for pre_process, simulation, and post_process • Includes CPU information and system specifications for test environment • Records test invocation with muscl_order=1 parameter +187/-0
Configuration changes	2 files case_dicts.py Add MUSCL parameter definitions to case dictionaries          toolchain/mfc/run/case_dicts.py • Adds MUSCL-related parameter definitions to case dictionaries • Includes recon_type, muscl_order, muscl_lim, int_comp, ic_eps, ic_beta • Updates case optimization list to include MUSCL parameters +9/-1      case.py Add default reconstruction type to test configuration        toolchain/mfc/test/case.py • Adds default recon_type=1 parameter to test case configuration • Ensures backward compatibility with existing test cases +1/-0
Documentation	1 files case.md Documentation updates for MUSCL reconstruction parameters docs/documentation/case.md • Added recon_type parameter to specify reconstruction method (WENO vs MUSCL) • Added MUSCL-specific parameters: muscl_order, muscl_lim, int_comp, ic_eps, ic_beta • Updated Riemann solver description to include 4 options instead of 3 • Added detailed descriptions for MUSCL slope limiters and interface compression +14/-1

---

[qodo-code-review]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis ✅ 277 - PR Code Verified Compliant requirements: • Add MUSCL reconstruction as an alternative to WENO • Implement complete MUSCL reconstruction functionality Requires further human verification: • Verification that MUSCL reconstruction produces correct numerical results compared to WENO • Performance testing to ensure GPU acceleration works as expected • Validation of the extensive test suite with different physics scenarios

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Debug Code There is a debug print statement that should be removed before production print *, i, l, k, j vL_prim_vf(i)%sf(j, k, l) = vL_x(j, k, l, i) Code Duplication The reconstruction logic is duplicated across multiple subroutines with similar patterns that could potentially be consolidated if (muscl_dir == ${MUSCL_DIR}$) then $:GPU_PARALLEL_LOOP(collapse=4,private='[slopeL,slopeR,slope]') do l = is3_muscl%beg, is3_muscl%end do k = is2_muscl%beg, is2_muscl%end do j = is1_muscl%beg, is1_muscl%end do i = 1, v_size slopeL = v_rs_ws_${XYZ}$ (j + 1, k, l, i) - & v_rs_ws_${XYZ}$ (j, k, l, i) slopeR = v_rs_ws_${XYZ}$ (j, k, l, i) - & v_rs_ws_${XYZ}$ (j - 1, k, l, i) slope = 0._wp if (muscl_lim == 1) then ! minmod if (slopeL*slopeR > 1e-9_wp) then slope = min(abs(slopeL), abs(slopeR)) end if if (slopeL < 0._wp) slope = -slope elseif (muscl_lim == 2) then ! MC if (slopeL*slopeR > 1e-9_wp) then slope = min(2._wp*abs(slopeL), 2._wp*abs(slopeR)) slope = min(slope, 5e-1_wp*(abs(slopeL) + abs(slopeR))) end if if (slopeL < 0._wp) slope = -slope elseif (muscl_lim == 3) then ! Van Albada if (abs(slopeL) > 1e-6_wp .and. abs(slopeR) > 1e-6_wp .and. & abs(slopeL + slopeR) > 1e-6_wp .and. slopeL*slopeR > 1e-6_wp) then slope = ((slopeL + slopeR)*slopeL*slopeR)/(slopeL**2._wp + slopeR**2._wp) end if elseif (muscl_lim == 4) then ! Van Leer if (abs(slopeL + slopeR) > 1.e-6_wp .and. slopeL*slopeR > 1.e-6_wp) then slope = 2._wp*slopeL*slopeR/(slopeL + slopeR) end if elseif (muscl_lim == 5) then ! SUPERBEE if (slopeL*slopeR > 1e-6_wp) then slope = -1._wp*min(-min(2._wp*abs(slopeL), abs(slopeR)), -min(abs(slopeL), 2._wp*abs(slopeR))) end if end if ! reconstruct from left side vL_rs_vf_${XYZ}$ (j, k, l, i) = & v_rs_ws_${XYZ}$ (j, k, l, i) - (5.e-1_wp*slope) ! reconstruct from the right side vR_rs_vf_${XYZ}$ (j, k, l, i) = & v_rs_ws_${XYZ}$ (j, k, l, i) + (5.e-1_wp*slope) end do end do end do end do end if #:endfor Logic Inconsistency The allocation conditions use different logic patterns for WENO vs MUSCL that should be verified for correctness if (recon_type == WENO_TYPE) then idx1%beg = 0 idx1%end = weno_polyn - 1 idx2%beg = 0 idx2%end = weno_order - 3 else if (recon_type == MUSCL_TYPE) then idx1%beg = 0 idx1%end = muscl_polyn idx2%beg = 0 idx2%end = muscl_order - 1 end if

[qodo-code-review]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	✅ Fix MPI datatype mismatch Suggestion Impact: The suggestion was directly implemented - the MPI_BCAST call for recon_type was changed from MPI_LOGICAL to MPI_INTEGER exactly as suggested code diff: - call MPI_BCAST(recon_type, 1, MPI_LOGICAL, 0, MPI_COMM_WORLD, ierr) + call MPI_BCAST(recon_type, 1, MPI_INTEGER, 0, MPI_COMM_WORLD, ierr) The recon_type parameter is declared as an integer but is being broadcast as MPI_LOGICAL. This type mismatch will cause MPI communication errors and potential data corruption. src/simulation/m_mpi_proxy.fpp [185] -call MPI_BCAST(recon_type, 1, MPI_LOGICAL, 0, MPI_COMM_WORLD, ierr) +call MPI_BCAST(recon_type, 1, MPI_INTEGER, 0, MPI_COMM_WORLD, ierr) Suggestion importance[1-10]: 10 __ Why: The suggestion correctly identifies a critical bug where recon_type is broadcast as MPI_LOGICAL instead of MPI_INTEGER, which would cause MPI errors and data corruption.	High
	Fix slope sign assignment logic The sign assignment logic is flawed because it only checks slopeL but should preserve the correct sign based on both slopes. This could lead to incorrect slope reconstruction in MUSCL schemes. src/simulation/m_muscl.fpp [174-177] if (slopeL*slopeR > 1e-9_wp) then - slope = min(abs(slopeL), abs(slopeR)) + slope = sign(min(abs(slopeL), abs(slopeR)), slopeL) end if -if (slopeL < 0._wp) slope = -slope Suggestion importance[1-10]: 6 __ Why: The suggestion correctly identifies that the slope sign assignment can be improved; using the sign intrinsic function is more idiomatic and robust Fortran.	Low
General	✅ Remove debug print statement Suggestion Impact: The debug print statement was removed from the nested loop exactly as suggested code diff: - print *, i, l, k, j Remove the debug print statement as it will generate excessive output during production runs. Debug prints should not be left in production code as they can significantly impact performance and create large log files. src/simulation/m_viscous.fpp [1059] -print *, i, l, k, j +! Debug print removed Suggestion importance[1-10]: 9 __ Why: The print statement is inside a deeply nested loop and will severely degrade performance and generate excessive output, so it must be removed.	High
Update

[wilfonba]

Minor changes requested

[codecov]

Codecov Report

❌ Patch coverage is 35.41147% with 259 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (master@16de11c). Learn more about missing BASE report.
⚠️ Report is 2 commits behind head on master.

Files with missing lines	Patch %	Lines
src/simulation/m_viscous.fpp	12.50%	70 Missing and 7 partials ⚠️
src/simulation/m_muscl.fpp	50.00%	45 Missing and 23 partials ⚠️
src/simulation/m_rhs.fpp	29.62%	36 Missing and 2 partials ⚠️
src/simulation/m_surface_tension.fpp	0.00%	30 Missing and 1 partial ⚠️
src/simulation/m_cbc.fpp	17.24%	21 Missing and 3 partials ⚠️
src/simulation/m_checker.fpp	50.00%	5 Missing and 1 partial ⚠️
src/common/m_checker_common.fpp	50.00%	4 Missing and 1 partial ⚠️
src/common/m_mpi_common.fpp	40.00%	2 Missing and 1 partial ⚠️
src/simulation/m_global_parameters.fpp	84.61%	0 Missing and 2 partials ⚠️
src/simulation/m_start_up.fpp	75.00%	0 Missing and 2 partials ⚠️
... and 2 more

Additional details and impacted files

@@            Coverage Diff            @@
##             master     #966   +/-   ##
=========================================
  Coverage          ?   43.67%           
=========================================
  Files             ?       70           
  Lines             ?    19818           
  Branches          ?     2473           
=========================================
  Hits              ?     8655           
  Misses            ?     9644           
  Partials          ?     1519           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sbryngelson]

well done! passing all tests. some minor corrections from the AI reviewers, looks like @wilfonba already parsed those for you to get rid of the false positives

[sbryngelson]

you removed viscous from m_mpi_proxy I'm not sure if this is intentional or correct

[okBrian]

Wait its still there no?

[sbryngelson]

Code is working! I will look in more detail shortly before merging.