Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

#13332: add ttnn implementation for Bert-Tiny model #13471

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

#13332: add ttnn implementation for Bert-Tiny model #13471

wants to merge 1 commit into from

Conversation

vigneshkeerthivasanx
Copy link
Contributor

Ticket

Link to Github Issue

Problem description

Add ttnn implementation for BERT-Tiny model.

What's changed

ttnn support is enabled for BERT-Tiny model along with demo, e2e device and test perf.

Checklist

  • Post commit CI passes
  • Blackhole Post commit (if applicable)
  • Model regression CI testing passes (if applicable)
  • Device performance regression CI testing passes (if applicable)
  • New/Existing tests provide coverage for changes

@vigneshkeerthivasanx vigneshkeerthivasanx marked this pull request as draft October 4, 2024 12:06
@vigneshkeerthivasanx vigneshkeerthivasanx force-pushed the vignesh/ttnn_bert_tiny branch 3 times, most recently from 0298e2a to 14bb9c3 Compare October 10, 2024 09:01
@vigneshkeerthivasanx vigneshkeerthivasanx mentioned this pull request Oct 15, 2024
Open
@vigneshkeerthivasanx vigneshkeerthivasanx marked this pull request as ready for review October 22, 2024 04:41
@vigneshkeerthivasanx vigneshkeerthivasanx marked this pull request as draft October 22, 2024 05:00
@vigneshkeerthivasanx vigneshkeerthivasanx force-pushed the vignesh/ttnn_bert_tiny branch 2 times, most recently from ad474d6 to 844f09f Compare October 23, 2024 06:04
@vigneshkeerthivasanx vigneshkeerthivasanx marked this pull request as ready for review October 23, 2024 06:05
@saichandax saichandax mentioned this pull request Nov 5, 2024
Open
8 tasks
@vigneshkeerthivasanx vigneshkeerthivasanx force-pushed the vignesh/ttnn_bert_tiny branch 2 times, most recently from 7b6a28c to 12f8b6c Compare November 14, 2024 12:12
@uaydonat uaydonat mentioned this pull request Nov 15, 2024
Open
5 tasks
@uaydonat
Copy link
Contributor

We need to see the Actions that contain the newly added CI tests to be launched and link posted here. The tests have to pass before the PR is approved.

See an example here: #15005 (comment)

@vigneshkeerthivasanx vigneshkeerthivasanx force-pushed the vignesh/ttnn_bert_tiny branch 2 times, most recently from 6b0f86c to 7c6de54 Compare November 18, 2024 09:02
@tt-rkim
Copy link
Collaborator

tt-rkim commented Nov 18, 2024

Please post your passing links.

I believe your integration tests is causing issues with distilbert in nightly FD.

@vigneshkeerthivasanx
Copy link
Contributor Author

Please post your passing links.

I believe your integration tests is causing issues with distilbert in nightly FD.

Will trigger CIs again and post passing links

@vigneshkeerthivasanx vigneshkeerthivasanx mentioned this pull request Nov 19, 2024
Open
5 tasks
uaydonat
uaydonat reviewed Nov 20, 2024
View reviewed changes
models/demos/bert_tiny/tests/test_performance.py Show resolved Hide resolved
models/demos/bert_tiny/demo/demo.py Show resolved Hide resolved
models/demos/bert_tiny/tests/test_performance.py Outdated Show resolved Hide resolved
tests/ttnn/integration_tests/bert_tiny/test_bert_tiny.py
"batch_size",
((8),),
)
def test_bert_output_inference(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this test run anywhere?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a sub-module test for Bert-Tiny model and this is tested in CIs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see test_performance is calling test_bert_tiny.py::test_bert_for_question_answering but I cannot find any where this is called. Maybe I am missing it? Where is call for this test?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_bert_tiny.py::test_bert_for_question_answering is present in the below mentioned path tests/ttnn/integration_tests/bert_tiny/test_bert_tiny.py

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am asking which function is calling test_bert_output_inference. Does this test run anywhere?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this isn't running anywhere; it's simply a test to test that specific sub-module.

@vigneshkeerthivasanx
Copy link
Contributor Author

CI links:
All post-commit tests : Link (All tests are passing)
Nightly fast dispatch tests : n150 , n300, e150 (Bert-Tiny tests are passing)
(Single-card) Demo tests : n150, n300 (Bert-Tiny tests are passing)
(Single-card) Device perf regressions : Link (GS is failing)
(Single-card) Model perf tests : Link (In progress)

@tt-rkim
Copy link
Collaborator

tt-rkim commented Nov 20, 2024

There have been some instability issues with GS device perf. I have rebooted the machine

@vigneshkeerthivasanx
Copy link
Contributor Author

All post-commit tests : Link (All Passed)
Nightly fast dispatch tests : n150 , n300, e150 (Bert-Tiny is passing)
(Single-card) Demo tests : n150 (Bert-Tiny is passing), n300 (Bert-Tiny is passing)
(Single-card) Device perf regressions : Link (GS failing with test hang)
(Single-card) Model perf tests : Link (All passed)

@vigneshkeerthivasanx vigneshkeerthivasanx force-pushed the vignesh/ttnn_bert_tiny branch 2 times, most recently from b30e5d9 to ccd7ec9 Compare November 22, 2024 13:35
@vigneshkeerthivasanx
Copy link
Contributor Author

CI links
All post-commit tests: Link (PASSED)
Nightly fast dispatch tests : Link (Bert_tiny model tests passed)
(Single-card) Demo tests : n150 (PASSED), n300 (PASSED),
(Single-card) Device perf regressions : Link (PASSED)
(Single-card) Model perf tests : Link (Bert_tiny model tests passed)

Can this be merged into the main?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@uaydonat uaydonat uaydonat approved these changes

@cfjchu cfjchu cfjchu approved these changes

@tt-rkim tt-rkim tt-rkim approved these changes

@eyonland eyonland Awaiting requested review from eyonland

@patrickroberts patrickroberts Awaiting requested review from patrickroberts

@yan-zaretskiy yan-zaretskiy Awaiting requested review from yan-zaretskiy

@xanderchin xanderchin Awaiting requested review from xanderchin

@dmakoviichuk-tt dmakoviichuk-tt Awaiting requested review from dmakoviichuk-tt dmakoviichuk-tt is a code owner

@rfurko-tt rfurko-tt Awaiting requested review from rfurko-tt rfurko-tt is a code owner

@TT-BrianLiu TT-BrianLiu Awaiting requested review from TT-BrianLiu TT-BrianLiu is a code owner

@razorback3 razorback3 Awaiting requested review from razorback3 razorback3 is a code owner

@dongjin-na dongjin-na Awaiting requested review from dongjin-na dongjin-na is a code owner

@ayerofieiev-tt ayerofieiev-tt Awaiting requested review from ayerofieiev-tt ayerofieiev-tt is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@vigneshkeerthivasanx @uaydonat @tt-rkim @cfjchu @ayerofieiev-tt