Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(Closes #2716) Add support for module-inlining calls to polymorphic kernels/routines #2732

Open
wants to merge 63 commits into
base: master
Choose a base branch
from

Conversation

arporter
Copy link
Member

@arporter arporter commented Oct 3, 2024

No description provided.

@arporter arporter self-assigned this Oct 3, 2024
@arporter arporter marked this pull request as draft October 3, 2024 09:12
@arporter arporter added in progress NG-ARCH Issues relevant to the GPU parallelisation of LFRic and other models expected to be used in NG-ARCH labels Oct 3, 2024
Copy link

codecov bot commented Oct 3, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.88%. Comparing base (6dbfc59) to head (ce6f6a9).

Additional details and impacted files
@@           Coverage Diff            @@
##           master    #2732    +/-   ##
========================================
  Coverage   99.88%   99.88%            
========================================
  Files         357      357            
  Lines       49652    49763   +111     
========================================
+ Hits        49596    49707   +111     
  Misses         56       56            

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@arporter
Copy link
Member Author

arporter commented Oct 3, 2024

As well as addressing the coverage, I need to extend the LFRic integration tests to use this new functionality.

@arporter
Copy link
Member Author

arporter commented Oct 7, 2024

Added the KernelModuleInlineTrans back into the LFRic transformation script and get a crash when trying to add the interface symbol into the symbol table.

@arporter
Copy link
Member Author

One of the compilation tests fail with the nvidia compiler. This same test passes with gfortran:

image

@arporter
Copy link
Member Author

$ nvfortran -I../dynamo_wrapper0/utilities -I../dynamo_wrapper0/field -I../dynamo_wrapper0/operator -I../dynamo_wrapper0/function_space -I../dynamo_wrapper0/mesh -I../dynamo_wrapper0/scalar -I../dynamo_wrapper0/configuration -c psy.f90
NVFORTRAN-S-0155-Ambiguous interfaces for generic procedure mixed_code (psy.f90: 289)

@arporter
Copy link
Member Author

arporter commented Nov 19, 2024

The problem is that the invoke subroutine is still importing the kernel interface, even though it has been module inlined:

    SUBROUTINE invoke_1(scalar_r_phys, a, istp)
      USE mixed_kernel_mod, ONLY: mixed_code

EDIT: I think this is correct. We've only module-inlined the mixed_kernel for invoke_0, not invoke_1. The latter still uses the imported version of the kernel. Is this compliant with the Fortran standard though?

@arporter
Copy link
Member Author

arporter commented Nov 19, 2024

I'm beginning to think that this is a compiler bug. If I alter the second invoke so that it calls a specific implementation of the kernel (e.g. mixed_code_64) then it compiles fine.
I've made a reproducer and sent it to Lukas and Wayne.

@arporter arporter marked this pull request as ready for review November 22, 2024 10:34
@arporter
Copy link
Member Author

This is ready for a first review. It's quite big but can wait until 3.0 is out of the door. Probably needs to be reviewed by @sergisiso.

@arporter
Copy link
Member Author

In experimenting with this branch, I discovered a bug: the routines that a PUBLIC Interface maps to may themselves be declared PRIVATE in their containing module! Unfortunately, in what I've written I've assumed they must be public.

@arporter
Copy link
Member Author

Trying this updated branch with LFRic (and my enhanced global.py) I now get:

Failed to module-inline routine coordinate_jacobian:
Transformation Error: routine 'coordinate_jacobian' contains accesses to 'jacobian_abr2xyz' which is declared in the same module scope. Cannot inline such a routine.

where jacobian_abr2xyz is actually a routine in the same module so we could inline it...

Also get

File '../lfric/components/science/source/kernel/geometry/chi_transform_mod.F90' does contain module 'chi_transform_mod' but PSyclone is unable to create the PSyIR of it.

This is possibly because that module contains an unsupported form of interface definitions:

  INTERFACE
    SUBROUTINE chi2xyz_interface(chi_1, chi_2, chi_3, panel_id, x, y, z)
      IMPORT :: r_def, i_def
      IMPLICIT NONE
      INTEGER(KIND = i_def), INTENT(IN) :: panel_id
      REAL(KIND = r_def), INTENT(IN) :: chi_1, chi_2, chi_3
      REAL(KIND = r_def), INTENT(OUT) :: x, y, z
    END SUBROUTINE chi2xyz_interface
  END INTERFACE

@arporter
Copy link
Member Author

arporter commented Nov 28, 2024

Attempt to do recursive inlining:

Failed to inline call jacobian_abr2xyz(alpha, beta, radius, panel_id):
Transformation Error: Routine 'jacobian_abr2XYZ_r_single' cannot be inlined because it accesses variable 'r_single' from its parent container.

but r_single is actually named in an import into the parent container => should be easy to handle.

...well, ish. It's a bit messy because there's no clean way to spot all the Symbols a bit of code uses and I don't want to duplicate what validate() does. Maybe we can use VAI?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in progress NG-ARCH Issues relevant to the GPU parallelisation of LFRic and other models expected to be used in NG-ARCH
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants