Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement test_coverage and country_overview in the new pattern #835

Open
hellais opened this issue Apr 10, 2024 · 1 comment
Open

Implement test_coverage and country_overview in the new pattern #835

hellais opened this issue Apr 10, 2024 · 1 comment
Assignees
Labels
ooni/api Issues related to https://github.com/ooni/api ooni/backend Issues related to https://github.com/ooni/backend priority/medium

Comments

@hellais
Copy link
Member

hellais commented Apr 10, 2024

  • /api/_/test_coverage
  • /api/_/country_overview

While we are at it, we only every need to fetch data one year at a time to so we should be able to add constraints about that to it.

@hellais hellais added ooni/api Issues related to https://github.com/ooni/api ooni/backend Issues related to https://github.com/ooni/backend labels Apr 10, 2024
@hellais
Copy link
Member Author

hellais commented Apr 11, 2024

So I was looking into this and thinking about this in the context of also improving how the lookup works in the network and domain pages.

From the looks of it, we basically have 4 bits of information that we need for a particular country, network or domain:

  1. The overall measurement_count
  2. The date of the first measurement
  3. The date of the last measurement
  4. For each calendar year, we need a timeseries of count of measurements per day

The first 3 items, we have to load once upon first render, while for 4. we will have to make a new fetch every time the user clicks on a different calendar year.

At the moment in the network pages we are doing this in a way that's a bit suboptimal, in that we load on first render the whole aggregation result since 12 years before the current date: https://github.com/ooni/explorer/blob/master/pages/as/%5Bprobe_asn%5D.js#L221.

This is problematic for 2 reasons:

  1. Once we reach December 2024, this will no longer be accurate, since we would be missing the first OONI measurements based on this filter: https://github.com/ooni/explorer/blob/master/pages/as/%5Bprobe_asn%5D.js#L226
  2. We are delaying the first render by requesting all the calendar datapoints even for years which we have no plan on rendering

My proposal is therefore that we implement 1 new endpoint and discontinue the existing test_coverage and country_overview endpoints that returns:

  1. The overall measurement_count
  2. The date of the first measurement
  3. The date of the last measurement

We can call it measurement_overview and have it take as parameters probe_cc, probe_asn, test_name, domain and it returns: total_measurements, first_measurement_date, last_measurement_date.

For the counts per year, we can continue using the aggregation endpoint, since it already supports fetching data one year at a time.

@majakomel how does this sound?

@hellais hellais removed their assignment Jul 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ooni/api Issues related to https://github.com/ooni/api ooni/backend Issues related to https://github.com/ooni/backend priority/medium
Projects
None yet
Development

No branches or pull requests

2 participants