Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

probably don't want to use NF90_CLASSIC_MODEL in only one nf90_create() call, other create questions... #6

Open
edwardhartnett opened this issue Feb 8, 2021 · 9 comments
Assignees
Labels
question Further information is requested

Comments

@edwardhartnett
Copy link
Contributor

In src/module_fv3gfs_ncio.f90:

    ! create netcdf file
    if (dsetin%ishdf5) then
       if (dset%isparallel) then
          if (present(mpicomm)) then
             ncerr = nf90_create(trim(filename), &
                     cmode=IOR(NF90_CLOBBER,NF90_NETCDF4), ncid=dset%ncid, &
                     comm = mpicomm, info = mpi_info_null)
          else
             ncerr = nf90_create(trim(filename), &
                     cmode=IOR(NF90_CLOBBER,NF90_NETCDF4), ncid=dset%ncid, &
                     comm = mpi_comm_world, info = mpi_info_null)
          end if
       else
          ncerr = nf90_create(trim(filename), &
                  cmode=IOR(IOR(NF90_CLOBBER,NF90_NETCDF4),NF90_CLASSIC_MODEL), &
                  !cmode=IOR(NF90_CLOBBER,NF90_NETCDF4), &
                  ncid=dset%ncid)
       end if
       dset%ishdf5 = .true.
    else
       ncerr = nf90_create(trim(filename), &
               cmode=IOR(IOR(NF90_CLOBBER,NF90_64BIT_OFFSET),NF90_SHARE), &
               ncid=dset%ncid)
       dset%ishdf5 = .false.
    endif

Not sure what is going on with NF90_CLASSIC_MODEL. What it does is add a secret attribute in the file, and set a flag within the netcdf-c library, to disallow using any netCDF-4 metadata that does not work in netCDF classic. So, for example, if NF90_CLASSIC_MODEL is used, and we attempt to define an NF90_INT64 variable, we will get an error.

The intent of CLASSIC_MODEL is to allow us to write code is guaranteed to work in any format of netCDF file, so we can easily use the same code for netCDF/HDF5 and classic files.

Perhaps this is what you want, but then probably you would want it everywhere, not just in one nf90_create() call.

NF90_SHARE should probably not be there. Do you have a specific reason for adding it?

NF90_64BIT_OFFSET offers increase to 4 GB var record size. The NF90_CDF5 binary format, based on the work of the pnetcdf team, integrated with netcdf-c, provides a much less restrictive classic format that may be preferrable.

@edwardhartnett edwardhartnett added the question Further information is requested label Feb 8, 2021
@jswhit
Copy link
Contributor

jswhit commented Feb 9, 2021

Ed, I think you are right. I'll make those changes.

@edwardhartnett
Copy link
Contributor Author

Do we want CLASSIC_MODEL?

And do we want CDF5?

The reason I ask, is that CDF5 includes 64-bit and unsigned int types. CLASSIC_MODEL still forbids these types.

@jswhit
Copy link
Contributor

jswhit commented Feb 9, 2021

The module doesn't support things like groups and compound types, so that's probably why I used CLASSIC_MODEL. It would be nice to support 64 bit integers though, so I'll go ahead and just use NF90_NETCDF4 and NF90_CDF5.

@jswhit
Copy link
Contributor

jswhit commented Feb 9, 2021

Note that it doesn't support 64 bit integers now.

@jswhit
Copy link
Contributor

jswhit commented Feb 9, 2021

Looks like the fortran interface doesn't yet support 64 bit and unsigned integers.

jswhit pushed a commit to NOAA-PSL/NCEPLIBS-ncio that referenced this issue Feb 9, 2021
@jswhit
Copy link
Contributor

jswhit commented Feb 9, 2021

#16

@edwardhartnett
Copy link
Contributor Author

Firstly, can you write or wait for test code before making changes?

Secondly, does netcdf-fortran not support CDF5 extra types? Or just the docs? Either way is a bug that should be issued on the netcdf-fortran github site.

@jswhit
Copy link
Contributor

jswhit commented Feb 9, 2021

I don't see any of using 64 bit integers with netcdf-fortran 4.5.3.

@edwardhartnett
Copy link
Contributor Author

Yes, I was afraid of that. I will put in an issue. ;-)

edwardhartnett pushed a commit that referenced this issue Feb 12, 2021
* Create LICENSE.md

* Update LICENSE.md

* don't impose CLASSIC model (issue #6)

* add NF90_CLOBBER back

* fix typo

* update comment

* fix typo

* change name from module_fv3gfs_ncio to module_ncio

* fix typo

* change module name, path to output file

* add template nc file for test

* make sure test finds input data

* don't ignore *.nc files

* change name of template file

* put .nc back in

* remove

* run test

* check output

* fix verbosity

* remove unused variable

* add a couple more tests

* update

* add comment to remind myself to add support for extra integer types

* fix comments

* test reading of a slice

* add missing quantize_data_5d to interface

* test writing of slice

* dump comments from source in here until we have better docs

* update some comments

* fix formatting

* fix typo

* add link to users guide (such as it is)

* update

* update docs for open_dataset, create_dataset

* update docs

Co-authored-by: Hang-Lei-NOAA <44901908+Hang-Lei-NOAA@users.noreply.github.com>
Co-authored-by: jswhit2 <Jeffrey.S.Whitaker@noaa.gov>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

4 participants