Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce DefaultObservabilitySymbolCollector object memory #41078

Conversation

NipunaMadhushan
Copy link
Contributor

Purpose

Fix OOM issue occurred in choreo-cp-graphql.

Fixes https://github.com/wso2-enterprise/internal-support-ballerina/issues/273

Approach

When observability is enabled, DefaultObservabilitySymbolCollector static object is created which has a memory of ~400MB in the issue mentioned above. It contains all metadata of modules and their documents in the package including syntax trees of each document. It has only been used in Generating Executable stage in compilation to create <module-name>-observability-symbols.jar which contains syntax-tree.json. To create syntax-tree.json, a syntax tree data string has been created which also contains a memory of ~180MB.

In this approach, instead of storing metadata of all modules and documents, those data has been used directly to create syntax tree data string and it has been stored in the PackageHolder where the module data has been stored previously. This method reduces the memory of DefaultObservabilitySymbolCollector static object from ~400MB to ~180MB and additionally syntax tree data string is not needed to be created at the Generating Executable stage.

Samples

Provide high-level details about the samples related to this feature.

Remarks

List any other known issues, related PRs, TODO items, or any other notes related to the PR.

Check List

  • Read the Contributing Guide
  • Updated Change Log
  • Checked Tooling Support (#)
  • Added necessary tests
    • Unit Tests
    • Spec Conformance Tests
    • Integration Tests
    • Ballerina By Example Tests
  • Increased Test Coverage
  • Added necessary documentation
    • API documentation
    • Module documentation in Module.md files
    • Ballerina By Examples

@NipunaMadhushan NipunaMadhushan changed the base branch from reduce-observability-memory to master July 23, 2023 18:19
@NipunaMadhushan NipunaMadhushan changed the title Reduce observability memory Reduce DefaultObservabilitySymbolCollector object memory Jul 23, 2023
@gabilang
Copy link
Contributor

Shall we remove the ModuleHolder and DocumentHolder classes?

@codecov
Copy link

codecov bot commented Jul 25, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (1b14109) 80.79% compared to head (759c0cf) 76.82%.
Report is 1259 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #41078      +/-   ##
============================================
- Coverage     80.79%   76.82%   -3.97%     
+ Complexity    77993    53465   -24528     
============================================
  Files          3414     2881     -533     
  Lines        275030   201542   -73488     
  Branches      49786    26654   -23132     
============================================
- Hits         222210   154837   -67373     
+ Misses        43793    38247    -5546     
+ Partials       9027     8458     -569     

see 1228 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions
Copy link

github-actions bot commented Aug 9, 2023

This PR has been open for more than 15 days with no activity. This will be closed in 3 days unless the stale label is removed or commented.

@github-actions
Copy link

github-actions bot commented Sep 6, 2023

This PR has been open for more than 15 days with no activity. This will be closed in 3 days unless the stale label is removed or commented.

@github-actions github-actions bot added the Stale label Sep 6, 2023
@keizer619 keizer619 removed the Stale label Sep 8, 2023
@NipunaMadhushan
Copy link
Contributor Author

Shall we remove the ModuleHolder and DocumentHolder classes?

Addressed the suggestions

@github-actions
Copy link

github-actions bot commented Oct 6, 2023

This PR has been open for more than 15 days with no activity. This will be closed in 3 days unless the stale label is removed or commented.

@github-actions github-actions bot added the Stale label Oct 6, 2023
@github-actions
Copy link

Closed PR due to inactivity for more than 18 days.

Copy link
Contributor

@gabilang gabilang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@NipunaMadhushan NipunaMadhushan merged commit 8ac9786 into ballerina-platform:master Oct 26, 2023
30 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants