Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: Fix Onnx Connector Memory Problem with Onnx #9716

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

nmoeller
Copy link
Contributor

@nmoeller nmoeller commented Nov 15, 2024

Onnx currently faces memory issues when dividing functionality among multiple methods

Motivation and Context

I was experiencing weird non reproducible memory issues with connector when using phi-3 vision, after tracing the memory it turned out there are some issues when Parameters & Generation are not in the same function. I am already in contact with the PG to adress the issue also in onnx.

There seems to be a memory problem with pybind, because the parameters show a non deterministc behavior, but they should determistic.

To fix the current problem i've decided to merge the Parameter Method and the Generation Method.

Description

Contribution Checklist

Onnx currently faces memory issues when dividing functionality among multiple methods
@markwallace-microsoft markwallace-microsoft added the python Pull requests for the Python Semantic Kernel label Nov 15, 2024
@github-actions github-actions bot changed the title Fix Onnx Memory Problem Python: Fix Onnx Memory Problem Nov 15, 2024
@markwallace-microsoft
Copy link
Member

Python Unit Test Overview

Tests Skipped Failures Errors Time
2869 4 💤 0 ❌ 0 🔥 1m 20s ⏱️

@nmoeller nmoeller marked this pull request as ready for review November 15, 2024 13:28
@nmoeller nmoeller requested a review from a team as a code owner November 15, 2024 13:28
@nmoeller nmoeller changed the title Python: Fix Onnx Memory Problem Python: Fix Onnx Connector Memory Problem with Onnx Nov 15, 2024
@TaoChenOSU TaoChenOSU added this pull request to the merge queue Nov 19, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 19, 2024
@TaoChenOSU TaoChenOSU added this pull request to the merge queue Nov 21, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 21, 2024
@TaoChenOSU TaoChenOSU added this pull request to the merge queue Nov 21, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 21, 2024
@nmoeller
Copy link
Contributor Author

@TaoChenOSU Anything I can do here ? I only see that there is a timeout, do you experience this issue in other pipelines aswell ? Or is it just me ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
python Pull requests for the Python Semantic Kernel
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants