How to handle untracked output generated as a build side-effect

[bobozaur]

Hello,

We encountered this when using sqlx, which can cache compile-time query execution into some files for offline use (without a database).

In a Cargo workspace, people would generally run cargo sqlx prepare with a live database active and the query files would be generated, by default within the crate directory in a .sqlx directory. We wanted to emulate this behavior using Bazel. The sqlx proc-macros do all the heavy lifting and all that's needed for the query files to be generated are some environment variables to be set, specifically DATABASE_URL and SQLX_OFFLINE_DIR.

However, the query files get generated at compile time and they are not declared as output. The query files are named using the convention query-{SQL-HASH}.json so the names are not really deterministic, but we can control the location where they get generated through the SQLX_OFFLINE_DIR environment variable. However, this directory must exist before the target is built, as otherwise the proc-macros will fail.

How could we go about capturing/excavating these query files? Is there some approach to be taken for situations like these? The main problem seems to be that regardless of what gets passed as SQLX_OFFLINE_DIR the query files are untracked and there's no way to reference them.