grads format (dat+ctl files) support in xarray (or xcdat)?

[jypeter]

Does somebody know if xarray can read the grads files that some models still create?

grads files are a combination of a binary data file (e.g my_data.dat), in a simple enough format, and text control file describing the content of the binary file (e.g my_data.ctl). Details about grads format

Just asking because cdms2 was a very easy and convenient way to read these files, and write back the equivalent .nc file and that somebody at LSCE asked me how to use this cdms2 based script last week to convert his grads files

I have pasted my simple and quite old nc4_grads_to_nc3.py script below. In this case, we would just run it with:

nc4_grads_to_nc3.py my_data.ctl converted_data.nc

Most of the script is just variables and attributes cleaning. What was nice was to be able to open .ctl and .nc files exactly the same way, and then be able to very easily create a decent .nc file

#!/usr/bin/env python

__version__ = "$Id$"

# Convert all the variables in a file of the following format:
#   + NetCDF 4
#   + grads (ctl+dat)
# to the NetCDF 3 format (aka 'NetCDF Classic')
#
# Script based on the old grads2nc.py script
#
# J-Y Peterschmitt, LSCE
#
# To do :
# - Check if input file exists
# - Check write permission for output file
# - deal with include/exclude var list
# - copy file global attributes (if present)
# - add option to disable "squeeze=1" when reading a var from the input file
# - add option to disable autmatic bounds generation
# - ...

# $Log$

import sys, cdms2

#excludevars = ['timelabel']
#excludeatts = ['id']
excludevars = ['bounds_time', 'bounds_latitude', 'bounds_longitude']
excludeatts = ['Conventions']

verbose = False

# use NetCDF3 Classic files
cdms2.setNetcdfShuffleFlag(0)
cdms2.setNetcdfDeflateFlag(0)
cdms2.setNetcdfDeflateLevelFlag(0)

def usage():
    """Display usage information for nc4_grads_to_nc3.py"""

    man = """
Usage : %s (input.nc|input.ctl) output.nc
    Will convert all the variables in a file of the following format
      + NetCDF 4
      + grads (ctl+dat)
    to variables in the NetCDF 3 format in output.nc
    """
    print(man % (sys.argv[0],))

# Check the number of command-line arguments and extract them
if len(sys.argv) != 3:
    usage()
    sys.exit(1)

namefilein = sys.argv[1]
namefileout = sys.argv[2]

# Disable automatic bounds generation by cdms2
#cdms2.setAutoBounds('off')

# Open the input and output file
filein = cdms2.open(namefilein)
fileout = cdms2.open(namefileout, 'w')

# Copy the global attributes
gattlist = filein.attributes.keys()
for attex in excludeatts:
    if attex in gattlist:
        if verbose:
            print('Global attribute [%s] is in exclude list and will not be processed' \
                  % (attex,))
        gattlist.remove(attex)
        
for gatt in gattlist:
    print('Setting', gatt, filein.attributes[gatt])
    setattr(fileout, gatt, filein.attributes[gatt])

# Convert all the variables
varlist = filein.listvariables()

# Exclude some variables, if need be
for varex in excludevars:
    if varex in varlist:
        print('Variable [%s] is in exclude list and will not be processed' \
              % (varex,))
        varlist.remove(varex)
print()

for varconv in varlist:
    print('** Converting %s' % (varconv,))
    var = filein(varconv, squeeze=1)

    # Correct the realtopology attribute of the longitude axis, if the
    # following problem has not been corrected
    #   https://github.com/CDAT/cdms/issues/433
    lon_axis = var.getLongitude()
    if hasattr(lon_axis, 'realtopology') and lon_axis.realtopology != 'circular':
        lon_axis.realtopology = 'circular'
        
    fileout.write(var)

print('\n%i variable(s) was/were converted' % (len(varlist),))

filein.close()
fileout.close()

# The end

[pochedls]

It seems like xGrADS has a lot of the functionality you need – would this work?

[jypeter]

Thanks for the reference!

I had a quick look at the documentation and I should be able to use it for conversion when cdms2 stops working (and I master xarray)

Ideally, someone should compare the output of ncdump -h for a file converted using cdms2, and a file converted with xgrads, in order to check that the CF compliant metadata we expect is there

I have added xgrads to my shared distribution, if somebody needs it. Hopefully, a conda installation will be supported in the future :-)

@pochedls, you can either close this issue, or leave it open in case somebody has time later to do the metadata comparison I mentioned above