Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to parse MATPOWER data?! #60

Open
nickrobinson251 opened this issue Nov 19, 2021 · 1 comment
Open

Add ability to parse MATPOWER data?! #60

nickrobinson251 opened this issue Nov 19, 2021 · 1 comment
Labels
idea needs some investigation before we decide

Comments

@nickrobinson251
Copy link
Owner

nickrobinson251 commented Nov 19, 2021

Prompted by discussion in https://discourse.julialang.org/t/ann-powerflowdata-jl-a-parser-for-pss-e-format-raw-power-flow-data-files/71722

Turns out power systems research has more than one very silly data format to deal with 😂 😭

We could consider broadening the scope of the package to also parse matpower data...
I think this only makes sense in this package (rather than its own package) if we can re-use a decent amount of what we have here, and broadly take the same approach (parsing the files byte-by-byte, having sections described by their own struct, etc).

Some specific ways in which matpower data is even more gross than PSS/E:

  • it's not even its own file format, it's a matlab file.
  • since it's a matlab file, we have to deal with all types of comments, including end-of-line comments
    • and not sure if this will be harder because whitespace is the delimiter
    • we can probably give up on mid-line comments appearing in the data (is that a thing in matlab?), because something are just too silly.
  • I don't think there's any requirement on the order in which different categories of data appear, meaning we'd have to somehow identify what data
    • possibly the (second part of) names are meaningful? e.g. bus data has to be foo.bus = ...?
    • the first part of this name isn't meaningful. convention seems to be mpc.* but i think that's convention only, and not actually required.
  • the "Generator Cost Data" doesn't not have a fixed number of columns 😒
    • on a quick look i think we could have the last column be a Vector{Vector{Float64}} and push! new columns if/when we encounter them... but it'll mena custom parsing code, not just a custom type

Probably other ways too... I'd never looked at this format before, so likely there's more to discover!

I think worth investigating further

@nickrobinson251 nickrobinson251 added the idea needs some investigation before we decide label Nov 19, 2021
@frapac
Copy link

frapac commented Mar 3, 2022

Hi!
As a follow up to our discussion, we have tried to implement a generic parser, supporting both MATPOWER and PSSE files. The idea is:

  • to use PowerFlowData for the PSSE file
  • to use PowerModels.jl's parser for the MATPOWER file.

Our demo exports the dataset as a PowerFlowData.Network structure. The code can be found here:
https://github.com/exanauts/PowerParsers.jl

The question is: would it make sense to integrate this in PowerFlowData.jl? The motives are:

  1. I don't think it would make sense to spend time implementing a new MATPOWER's parser.
  2. It would be very convenient to have a generic parser for power systems, supported by the community.

Overall, we have found PowerFlowData.jl convenient to use, and well implemented. The only default (for us) is that the structures have a lot of attributes, and sometimes it's hard to figure out the naming without referring to the documentation. For example these attributes. The structures specifying the ownership (Owner) are also hard to understand, with vectors mixing Float64 and missing values. In the end, we get a lot of boilerplate code, just to import the dataset properly in a Network structure... But this may be related to our implementation itself, which is far from being optimized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
idea needs some investigation before we decide
Projects
None yet
Development

No branches or pull requests

2 participants