New Fontend

[sehnem]

@RobertPincus , this is an initial implementation of the frontend concept from issue #31.

It utilizes an xarray accessor that loads the NetCDF file and returns either an SWProblem or an LWProblem, depending on the file type.

Once loaded, the problem allows for modification of certain variables.

The user can also create a standalone Problem if all the variables are provided.

The syntax for the SW case is as follows:

kdist = xr.load_dataset(f"{rte_rrtmgp_dir}/rrtmgp-gas-lw-g256.nc")
lw_problem = kdist.gas_optics.load_atmospheric_conditions(rfmip)
lw_problem.sfc_emis = rfmip["surface_emissivity"].data
solver_flux_up, solver_flux_down = lw_problem.rte_solve()

For the SW case, two parameters define the functions to compute mu0 and toa_flux. Users can provide total_solar_irradiance and solar_zenith_angle values as shown in the example, set the variables directly with a numpy array, or even implement custom functions to compute these values.

sw_problem = kdist.gas_optics.load_atmospheric_conditions(rfmip)

sw_problem.sfc_alb_dir = rfmip["surface_albedo"].data
sw_problem.total_solar_irradiance = rfmip["total_solar_irradiance"].data
sw_problem.solar_zenith_angle = rfmip["solar_zenith_angle"].values

solver_flux_up, solver_flux_down = sw_problem.solve()

This approach may offer flexibility beyond immediate needs, but it provides a more adaptable implementation.

@tcmetzger, I closed the previous draft PR as its changes have been incorporated here, and it hadn’t resolved the issue. This PR also updates the tests, which are running successfully locally. The Conda fix will be required alongside these changes.

[RobertPincus]

@sehnem This looks like a good start. A couple of questions or pieces of feedback:

About the problems:

• You've defined LWProblem and SWProblem. What I'd ideally like is the ability to separate the characterization of the atmosphere (which defines which solver gets used) from the specification of boundary conditions and source functions. Is it possible to have composed problem definitions, i.e. to have "type" problems with either tau, tau, ssa, g, or tau, ssa, P arrays, and then to have LWProblem which matches a type with boundary conditions (sfc_emis) and source functions (lev_source, lay_source, sfc_source)? SWProblem would have boundary conditions (sfc_alb_dir, sfc_alb_dif, toa_flux and mu0). Ideally there wouldn't be a hierarchy i.e. people could refer to LWproblem.tau rather than LWproblem.extinction.tau.
• Note that the gas optics routines will return the problem and the sources but not the boundary conditions.
• Probably I don't understand the xarray accessor methods, but it would be great if the different fields could have attached metadata, which suggests the fields are xr.DataArrays rather than np.ndarrays.
• Probably ignorance again but is there anything which currently supports having the various input and output fields share coordinates and dimensions?
• Is there any introspection, such that people could define a LWproblem and see what the fields are named? (Maybe that's just print(LWProblem)

About the gas optics:

• What you're calling load_atmospheric_conditions() is a transformation from input to a problem definition. In the Fortran this is called gas_optics%gas_optics(). We could choose something different (compute()) but we want a more descriptive name.

[sehnem]

@RobertPincus

You've defined LWProblem and SWProblem. What I'd ideally like is the ability to separate the characterization of the atmosphere (which defines which solver gets used) from the specification of boundary conditions and source functions. Is it possible to have composed problem definitions, i.e. to have "type" problems with either tau, tau, ssa, g, or tau, ssa, P arrays, and then to have LWProblem which matches a type with boundary conditions (sfc_emis) and source functions (lev_source, lay_source, sfc_source)? SWProblem would have boundary conditions (sfc_alb_dir, sfc_alb_dif, toa_flux and mu0). Ideally there wouldn't be a hierarchy i.e. people could refer to LWproblem.tau rather than LWproblem.extinction.tau.

Right now when the gas optics file is loaded in an xarray and the accessor .gas_optics, that extends the netcdf xarray to compute tau, ssa, g. The return type currently is an Problem python class with numpy arrays, and a bunch of checks, setters and the solve method that calls the RTE function.

Note that the gas optics routines will return the problem and the sources but not the boundary conditions.

The problem has default source functions and I created some setters in case the user wants to change it (giving a new function of an array of the source), but I will change it to the gas optics and just return the default.

Probably I don't understand the xarray accessor methods, but it would be great if the different fields could have attached metadata, which suggests the fields are xr.DataArrays rather than np.ndarrays.
Probably ignorance again but is there anything which currently supports having the various input and output fields share coordinates and dimensions?

Using accessors we can have all the metadata and share dimensions, as you mentioned. I can transform the problem into an xarray with other accessor to do that.

Is there any introspection, such that people could define a LWproblem and see what the fields are named? (Maybe that's just print(LWProblem)

This can be created, if we transform the problem into an xarray dataset it will have one by default.

What you're calling load_atmospheric_conditions() is a transformation from input to a problem definition. In the Fortran this is called gas_optics%gas_optics(). We could choose something different (compute()) but we want a more descriptive name.

I will rename it, will have to fix other things in the naming convention.

A question about how the user will usually run the models. Here is my understanding, correct me if I am wrong.

• The gas optics and source functions usually will not change, as they as definitions for the earth atmosphere, so we can consider the default.

• The atmospheric conditions used to compute the variables from gas optics can change, we are loading some netcdf as considering some default names, so the user will have to create a netcdf with the same name for the variables.

• The boundary conditions are a user input the input now it is a numpy array but is should also be a netcdf with custom name for the variables.

• If the user has an external dataset for the boundary conditions that is not in the default format or an array computed from other data sources, it will need to have the variables renamed and saved at the default format, and then loaded to be computed.

The user will be able to change the accessor based on the current code and make it work as needed for any case, but I think that there are some common inputs that may need to be more flexible, what would be those inputs?

I am asking to implement the methods needed to transform the input data.

[RobertPincus]

@brendancol @sehnem and I just had a chat. I explained that the gas optics classes are best viewed as a transformation between input data sets (temperature, pressure, gas concentrations) and problem definitions. The transformations are driven by data but users should have no interaction with that data. For this reason I wonder if the gas optics ought to be a class on its own, perhaps with a (hidden?) xr.Dataset to hold the data

[RobertPincus]

@sehnem had the idea that the gas optics transformations could return xr.Datasets describing complete problems: optical properties (one of three choices)+ orientation flag, radiation sources, boundary conditions. The latter two depend on whether the gas optics instance is for an LW or SW problem. Accessors can be used here to perform common operations and determine which solution path to follow. We could use one set of accessors to describe the LW/SW dichotomy and another to describe the detail involved in the problem description (in Fortran these are known as 1scl, 2str, nstr)