Support writing dags in the starlark language

[lilac]

I had already created a PR for this feature. Here I want to explain some motivations of this design.

1. YMAL is good to start with, but as the DAG's complexity increases, programmers are limited by the expressiveness of it. Then DAG as code style wins. That also explains why Airflow (or other similar solutions) use python for DAG writing.
2. Things where YAML is very short at is condition. We had to rely on some expression DSL to do it, but that is also not flexible enough. What we need is a configuration language.
3. Embedding python in Go is not easy, and we don't actually need a full featured python. So I picked starlark for the job.
4. The starlark language is easy to digest and has a good ecosystem.

In addition, there are existing DAG-like systems that supports starlark, and it works pretty well.

• DroneCI
• Cirrus-CI

[yohamta]

Thank you for your pull request and bringing Starlark to our attention! I wasn't aware of Starlark before, but it seems like a perfect alternative to YAML.

[yohamta]

Hi @lilac!
I just took a look at the code you've been working on and wanted to share some feedback. Firstly, great work on the main method - it's looking really good!

Regarding the metadata like schedule, params, and so on, I'd suggest creating a separate function (maybe something like meta()) instead of including it in the main method. This would make it easier for the web UI and scheduler process to access the schedule configuration for each DAG without having to call main each time, which could potentially be more efficient.

Keep up the fantastic work!