Support parsing v34 of PSS/E Data Format

[nickrobinson251]

Since we support v33, i'm hopeful a lot can be reused to add support for v34... and I do have an example v34 file now, but not the v34 manual/specification

Fortunately, the v34 file i have seems to contain comments at the start of each section

• TODO: skip over comment lines with parsing the data! These seem to start @!

Here's how the v34 file looks (with data itself removed):

@!IC, SBASE,REV,XFRRAT,NXFRAT,BASFRQ
0,   100.00, 34,     0,     1, 60.00     / PSS(R)E-34.7    SUN, JAN 30 2022  11:34
JAN 2022 MODEL UPDATE

GENERAL, THRSHZ=0.0001, PQBRAK=0.7, BLOWUP=5.0
GAUSS, ITMX=100, ACCP=1.6, ACCQ=1.6, ACCM=1.0, TOL=0.0001
NEWTON, ITMXN=20, ACCN=1.0, TOLN=0.1, VCTOLQ=0.1, VCTOLV=0.00001, DVLIM=0.99, NDVFCT=0.99
ADJUST, ADJTHR=0.005, ACCTAP=1.0, TAPLIM=0.05, SWVBND=100.0, MXTPSS=99, MXSWIM=10
TYSL, ITMXTY=20, ACCTY=1.0, TOLTY=0.00001
SOLVER, FDNS, ACTAPS=1, AREAIN=2, PHSHFT=1, VARLIM=0
RATING, 1, "RATE1 ", "RATING SET 1                    "
RATING, 2, "RATE2 ", "RATING SET 2                    "
RATING, 3, "RATE3 ", "RATING SET 3                    "
RATING, 4, "RATE4 ", "RATING SET 4                    "
RATING, 5, "RATE5 ", "RATING SET 5                    "
RATING, 6, "RATE6 ", "RATING SET 6                    "
RATING, 7, "RATE7 ", "RATING SET 7                    "
RATING, 8, "RATE8 ", "RATING SET 8                    "
RATING, 9, "RATE9 ", "RATING SET 9                    "
RATING,10, "RATE10", "RATING SET 10                   "
RATING,11, "RATE11", "RATING SET 11                   "
RATING,12, "RATE12", "RATING SET 12                   "
0 / END OF SYSTEM-WIDE DATA, BEGIN BUS DATA
@!   I,'NAME        ', BASKV, IDE,AREA,ZONE,OWNER, VM,        VA,    NVHI,   NVLO,   EVHI,   EVLO
...
0 / END OF BUS DATA, BEGIN LOAD DATA
@!   I,'ID',STAT,AREA,ZONE,      PL,        QL,        IP,        IQ,        YP,        YQ, OWNER,SCALE,INTRPT,  DGENP,     DGENQ, DGENF
...
0 / END OF LOAD DATA, BEGIN FIXED SHUNT DATA
@!   I,'ID',STATUS,  GL,        BL
...
0 / END OF FIXED SHUNT DATA, BEGIN GENERATOR DATA
@!   I,'ID',      PG,        QG,        QT,        QB,     VS,    IREG,     MBASE,     ZR,         ZX,         RT,         XT,     GTAP,STAT, RMPCT,      PT,        PB,    O1,    F1,  O2,    F2,  O3,    F3,  O4,    F4,WMOD, WPF,NREG
...
0 / END OF GENERATOR DATA, BEGIN BRANCH DATA
@!   I,     J,'CKT',     R,          X,         B,                    'N A M E'                 ,   RATE1,   RATE2,   RATE3,   RATE4,   RATE5,   RATE6,   RATE7,   RATE8,   RATE9,  RATE10,  RATE11,  RATE12,    GI,       BI,       GJ,       BJ,STAT,MET,  LEN,  O1,  F1,    O2,  F2,    O3,  F3,    O4,  F4
...
0 / END OF BRANCH DATA, BEGIN SYSTEM SWITCHING DEVICE DATA
@!   I,     J,'CKT',          X,   RATE1,   RATE2,   RATE3,   RATE4,   RATE5,   RATE6,   RATE7,   RATE8,   RATE9,  RATE10,  RATE11,  RATE12, STAT,NSTAT,  MET,STYPE,'NAME' 
...
0 / END OF SYSTEM SWITCHING DEVICE DATA, BEGIN TRANSFORMER DATA
@!   I,     J,     K,'CKT',CW,CZ,CM,     MAG1,        MAG2,NMETR,               'N A M E',               STAT,O1,  F1,    O2,  F2,    O3,  F3,    O4,  F4,     'VECGRP', ZCOD
@!   R1-2,       X1-2,   SBASE1-2,     R2-3,       X2-3,   SBASE2-3,     R3-1,       X3-1,   SBASE3-1, VMSTAR,   ANSTAR
@!WINDV1,  NOMV1,    ANG1,  RATE1-1,  RATE1-2,  RATE1-3,  RATE1-4,  RATE1-5,  RATE1-6,  RATE1-7,  RATE1-8,  RATE1-9, RATE1-10, RATE1-11, RATE1-12,COD1,CONT1,   RMA1,    RMI1,    VMA1,    VMI1, NTP1,TAB1,  CR1,     CX1,   CNXA1,NOD1
@!WINDV2,  NOMV2,    ANG2,  RATE2-1,  RATE2-2,  RATE2-3,  RATE2-4,  RATE2-5,  RATE2-6,  RATE2-7,  RATE2-8,  RATE2-9, RATE2-10, RATE2-11, RATE2-12,COD2,CONT2,   RMA2,    RMI2,    VMA2,    VMI2, NTP2,TAB2,  CR2,     CX2,   CNXA2,NOD2
@!WINDV3,  NOMV3,    ANG3,  RATE3-1,  RATE3-2,  RATE3-3,  RATE3-4,  RATE3-5,  RATE3-6,  RATE3-7,  RATE3-8,  RATE3-9, RATE3-10, RATE3-11, RATE3-12,COD3,CONT3,   RMA3,    RMI3,    VMA3,    VMI3, NTP3,TAB3,  CR3,     CX3,   CNXA3,NOD3
...
0 / END OF TRANSFORMER DATA, BEGIN AREA DATA
@! I,   ISW,     PDES,      PTOL,    'ARNAME'
...
0 / END OF AREA DATA, BEGIN TWO-TERMINAL DC DATA                                                                                                       
@!  'NAME',   MDC,     RDC,      SETVL,     VSCHD,     VCMOD,     RCOMP,    DELTI,METER    DCVMIN,CCCITMX, CCCACC
@! IPR,NBR,ANMXR,ANMNR,   RCR,    XCR,  EBASR,  TRR,    TAPR,   TMXR,   TMNR,   STPR,    ICR,   IFR,   ITR,'IDR', XCAPR,NDR
@! IPI,NBI,ANMXI,ANMNI,   RCI,    XCI,  EBASI,  TRI,    TAPI,   TMXI,   TMNI,   STPI,    ICI,   IFI,   ITI,'IDI', XCAPI,NDI 
...
0 / END OF TWO-TERMINAL DC DATA, BEGIN VSC DC LINE DATA                                                                                          
@!  'NAME',   MDC,     RDC,   O1,  F1,    O2,  F2,    O3,  F3,    O4,  F4
@!IBUS,TYPE,MODE,     DCSET,   ACSET,     ALOSS,     BLOSS,   MINLOSS,     SMAX,      IMAX,   PWF,        MAXQ,      MINQ, VSREG,  RMPCT,NREG
...
0 / END OF VSC DC LINE DATA, BEGIN IMPEDANCE CORRECTION DATA                                                                                          
@!I,   T1,    Re(F1),  Im(F1),    T2,    Re(F2),  Im(F2),    T3,    Re(F3),  Im(F3),    T4,    Re(F4),  Im(F4),    T5,    Re(F5),  Im(F5),    T6,    Re(F6),  Im(F6)
@!     T7,    Re(F7),  Im(F7),    T8,    Re(F8),  Im(F8),    T9,    Re(F9),  Im(F9),    T10,  Re(F10), Im(F10),    T11,  Re(F11), Im(F11),    T12,  Re(F12), Im(F12)
@!      ...
...
0 / END OF IMPEDANCE CORRECTION DATA, BEGIN MULTI-TERMINAL DC DATA                                                                                     @!  'NAME',    NCONV,NDCBS,NDCLN,  MDC, VCONV,     VCMOD, VCONVN
@!  IB, N,ANGMX,ANGMN,    RC,      XC,    EBAS,   TR,    TAP,    TPMX,   TPMN,   TSTP,     SETVL,   DCPF,    MARG,CNVCOD                               @!IDC, IB,AREA,ZONE,   'DCNAME',  IDC2,   RGRND,OWNER
@!IDC,JDC,'DCCKT',MET,   RDC,        LDC
...
0 / END OF MULTI-TERMINAL DC DATA, BEGIN MULTI-SECTION LINE DATA                                                                                       @!   I,     J,'ID',MET,DUM1,  DUM2,  DUM3,  DUM4,  DUM5,  DUM6,  DUM7,  DUM8,  DUM9
...
0 / END OF MULTI-SECTION LINE DATA, BEGIN ZONE DATA
@! I,   'ZONAME'
...
0 / END OF ZONE DATA, BEGIN INTER-AREA TRANSFER DATA                                                                                                   @!ARFROM,ARTO,'TRID', PTRAN
...
0 / END OF INTER-AREA TRANSFER DATA, BEGIN OWNER DATA
@! I,   'OWNAME' 
...
0 / END OF OWNER DATA, BEGIN FACTS DEVICE DATA                                                                                                         
@!  'NAME',         I,     J,MODE,   PDES,      QDES,  VSET,      SHMX,      TRMX,   VTMN,   VTMX,   VSMX,      IMX,     LINX,   RMPCT,OWNER,    SET1,      SET2,VSREF, FCREG,   'MNAME'    ,NREG                                                                                                             
...
0 / END OF FACTS DEVICE DATA, BEGIN SWITCHED SHUNT DATA                                                                                                @!  I,MODSW,ADJM,ST, VSWHI,  VSWLO, SWREG,  RMPCT,   'RMIDNT',      BINIT,N1,     B1, N2,     B2, N3,     B3, N4,     B4, N5,     B5, N6,     B6, N7,     B7, N8,     B8, NREG                                                                                                                                ...
0 / END OF SWITCHED SHUNT DATA, BEGIN GNE DATA                                                                                                         
@!  'NAME',        'MODEL',     NTERM,BUS1...BUSNTERM,NREAL,NINTG,NCHAR                                                                                @!ST,OWNER,NMETR                                                                                                                                       
@! REAL1...REAL(MIN(10,NREAL))                                                                                                                         
@! INTG1...INTG(MIN(10,NINTG))                                                                                                                         
@! CHAR1...CHAR(MIN(10,NCHAR))
...
0 / END OF GNE DATA, BEGIN INDUCTION MACHINE DATA                                                                                                      
@!   I,'ID',ST,SC,DC,AREA,ZONE,OWNER,TC,BC,  MBASE, RATEKV,PC,  PSET,      H,       A,       B,       D,       E,     RA,        XA,        XM,        R1,        X1,        R2,        X2,        X3,       E1,    SE1,   E2,    SE2,   IA1,   IA2, XAMULT                                                   
...
0 / END OF INDUCTION MACHINE DATA, BEGIN SUBSTATION DATA
...
0 / END OF SUBSTATION DATA                                                                                                                             
...
Q

[nickrobinson251]

compared to v33:

• System-wide data is new
• Buses: same
• Loads: 3 new columns on end: DGENP, DGENQ, DGENF
  • add new Union{Float64,Missing} columns
• Fixed Shunts: same
• Generators: 1 new column on end: NREG
  • add new Union{Float64,Missing} column
• Branches: ...very different?! New mid-row column Name+ 12 Rate columns rather than 3 + new mid-row column Met
  • in v33: I, J, CKT, R, X, B, ---- RATE_A, RATE_B, RATE_C, GI, BI, GJ, BJ, ST, ----- LEN, O1, F1, O2, F2, O3, F3, O4, F4
  • in v34: I, J, CKT, R, X, B, NAME, RATE_1, ..., RATE_12, GI, BI, GJ, BJ, STAT, MET, LEN, O1, F1, O2, F2, O3, F3, O4, F4
  • create new Branches34 type
• System switching devices is new
• Transformers:
  • 1st row: 1 new trailing column ZCOD
  • 2nd row: same
  • 3,4,5th row: 12 Rate columns rather than 3 + 1 new trailing column NOD1/2/3
• Area: same
• Two-terminal DC Lines:
  • 1st row: same
  • 2nd/3rd row: 1 new trailing column NDR/NDI (also different column names? ANMXR,ANMNR were ALFMX, ALFMN, and ANMXI,ANMNI were GAMMX, GAMMN)
    • add new Union{Int,Missing} column
• VSC DC Line: 1 new column on end of rows 2+3: NREG1/NREG2
  • add new Union{Float64,Missing} column? Maybe not -- maybe define a VSCDCLine34 type?
• _Impedence Corrections: Not sure... looks like maybe F columns have double / been split into Re/Im?
  • Needs more investigation_
• Multi-Terminal DC Lines: same
  • DC Line ID: same
  • AC converters: same
  • DC Buses: same
  • DC Links: same
• Multi-Section Line Groups: same
• Zones: same
• InterArea Transfer: same
• Owners: same
• FACTS Devices: 1 new column on end: NREG
  • add new Union{Float64,Missing} column
• Swithced Shunts: same
• GNE Devices: same Support parsing GNEDevices from v33 files #50

I don't know what any of these new columns mean, since i've no documentation just some example files...

• NREG maybe (at a complete guess) could be "non-regulating" (i.e. fixed output generators), in which case we'd expect this to be binary 0/1 (so we can use Bool). Might be worth checking if there's any other values in the NREG columns of the v34 files i have.
• MET (Branches` maybe means "metered"
• NDR/NDI i've no idea (the suffizx R is for "Rectifier", and I is for "Inverter")
• Other new ones are DGENP, DGENQ, DGENF (Loads),
• ZCOD (Transformers): generally COD means "code" which means enum i.e. 0, 1, 2, ..., N different modes, so we could check out is there integer (rather than just binary)... but i've no idea what the codes would mean. And per-bus NOD (Transformers) ... no idea (are they ever non-zero in the data we have?)

[nickrobinson251]

comment rows, like @! I,'ID',STAT,AREA,ZONE, ..., can be ignored by Parsers.Options(comment="@!") https://github.com/JuliaData/Parsers.jl/blob/e75d67e78b8e702f2d7e8ab1cc94e50cbe134b5c/src/Parsers.jl#L54