Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"REST join" - map the rows to data from REST requests #60

Open
OndraZizka opened this issue Nov 8, 2021 · 0 comments
Open

"REST join" - map the rows to data from REST requests #60

OndraZizka opened this issue Nov 8, 2021 · 0 comments

Comments

@OndraZizka
Copy link
Owner

OndraZizka commented Nov 8, 2021

In order to make getting data from a REST API easier,

a result of a SELECT could be mapped to the data from REST API in this way:

-in data.csv -sql "SELECT someId, someValue FROM ..." -map:rest="http://.../{someId}/{someValue}"

Each row of the SELECT result would form an URL using the template from -restJoin,
and the result would be added to a new table.


Alternatively:

This would need quite a few parameters on:

  • how to create the request (HTTP method, url, headers, ...)
  • how to parse the response (format, items location, ...),
  • how to to create the output (name, format, json format, ...)
    and so on.

Some of these parameters are already implemented for -in and -out.
Therefore, it may be smarter to keep CsvCruncher a bit simpler, RISC-like, and instead, implement the REST-mapping as an alternative of -sql:

-in singleImport.csv -out mappedData.csv -map:rest="http://..."

Assuming there would be just a single input.
This would basically be just a limitation on supported use cases.


The mapping could perhaps be done using the Java-based SQL functions, which would

  1. query
  2. store to a table column
  3. parse
  4. extract the values

but this seems quite cumbersome way to do it.

@OndraZizka OndraZizka added this to the Soon-ish milestone Nov 8, 2021
@OndraZizka OndraZizka changed the title "REST join" - map rows to data from REST requests "REST join" - map the rows to data from REST requests Nov 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant