Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace overviews by dynamic aggregation #1031

Open
enekid opened this issue Sep 6, 2018 · 2 comments
Open

Replace overviews by dynamic aggregation #1031

enekid opened this issue Sep 6, 2018 · 2 comments
Labels

Comments

@enekid
Copy link

enekid commented Sep 6, 2018

Context: [PROJECT] Deprecate raster overviews

In order to keep current performance when dropping the overviews feature, it's desirable to activate dynamic aggregation also for raster tiles.

@enekid enekid self-assigned this Sep 6, 2018
@enekid enekid added the Node.js label Sep 6, 2018
@enekid
Copy link
Author

enekid commented Sep 11, 2018

Context: Dynamic Aggregations [WIP]

Chain of Adapters

All the dynamic aggregation magic takes place in the Aggregation MapConfig Adapter that is initialized and added to de MapConfig adapters chain in the API router:

adapters = [
                new MapConfigNamedLayersAdapter(templateMaps, pgConnection),
                new MapConfigBufferSizeAdapter(),
                new SqlWrapMapConfigAdapter(),
                new DataviewsWidgetsAdapter(),
                new AnalysisMapConfigAdapter(analysisBackend),
                new VectorMapConfigAdapter(pgConnection),
                new AggregationMapConfigAdapter(pgConnection), // <<<====
                new TurboCartoAdapter()
            ];

Aggregation Adapter

This adapters getMapConfig function checks if the MapConfig should be adapted to use dynamic aggregation. If not, it returns the current adapted MapConfig without adding aggregation:

getMapConfig(user, requestMapConfig, params, context, callback) {
  //...
  if (!this._shouldAdapt(mapConfig, params)) {
      return callback(null, requestMapConfig);
  }
  //...
  this._adaptLayers(connection, mapConfig, requestMapConfig, context, callback);
}

The _shouldAdapt function checks if the param aggregation is true and if the mapConfig is aggregable

_shouldAdapt (mapConfig, params) {
    const { aggregation } = params;

    if (aggregation === 'false') {
        return false;
    }

    if (aggregation === 'true' || mapConfig.isAggregationMapConfig()) {
        return true;
    }

    return false;
}

The isAggregationMapConfig function is defined in the Aggregation Mapconfig model:

class AggregationMapConfig extends MapConfig {
  //...

  isAggregationMapConfig () {
      return this.isVectorOnlyMapConfig() || this.hasAnyLayerAggregation();
  }

  isAggregationLayer (index) {
      return this.isVectorOnlyMapConfig() || this.hasLayerAggregation(index);
  }

  //...
}

The function isVectorOnlyMapConfig is placed in Windshaft and checks that all the MapConfig layers are vector layers (isVectorLayer)

MapConfig.prototype.isVectorLayer = function (index) {
  const layer = this.getLayer(index);
  const type =  getType(layer.type);
  const sql = this.getLayerOption(index, 'sql');
  const cartocss = this.getLayerOption(index, 'cartocss');
  const cartocssVersion = this.getLayerOption(index, 'cartocss_version');

  return type === 'mapnik' && 
             typeof sql === 'string' && 
             cartocss === undefined && 
             cartocssVersion === undefined;
};

The function hasAnyLayerAggregation checks that some of the MapConfig layers are marked for aggregation (hasLayerAggregation)

hasLayerAggregation (index) {
    const layer = this.getLayer(index);
    const { aggregation } = layer.options;

    return aggregation !== undefined && 
               (typeof aggregation === 'object' || typeof aggregation === 'boolean');
}

@dgaubert dgaubert assigned dgaubert and unassigned enekid Sep 14, 2018
@dgaubert
Copy link
Contributor

dgaubert commented Oct 3, 2018

PR: #1038

@dgaubert dgaubert changed the title Use dynamic aggregation also for raster tiles Replace overviews by dynamic aggregation Oct 3, 2018
@dgaubert dgaubert removed their assignment Oct 3, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants