#14690: Cleaned up gtests under tests/tt_metal/tt_metal/ to make th…

…em more readable, organized and maintainable (#14691)

CONTRIBUTING.md

@@ -251,8 +251,8 @@ The new fangled way we run our tests is with Googletest. The way we generally
 structure our tests with this framework is to bundle it into a single
 executable.
 
-You can use `--gtest_filter_test` to filter out the specific test you'd like.
-For example, to build and run the `CommonFixture.DRAMLoopbackSingleCore` on
+You can use `--gtest_filter` to filter out the specific test you'd like.
+For example, to build and run the `DispatchFixture.TensixDRAMLoopbackSingleCore` on
 fast dispatch, you can
 
 1. Build the unit tests:
@@ -261,7 +261,7 @@ fast dispatch, you can
    ```
 2. Run the test:
    ```
-   ./build/test/tt_metal/unit_tests_fast_dispatch --gtest_filter="CommonFixture.DRAMLoopbackSingleCore"
+   ./build/test/tt_metal/unit_tests_api --gtest_filter="DispatchFixture.TensixDRAMLoopbackSingleCore"
    ```
 
 On slow dispatch, to run another specific test, the equivalent would be:
@@ -270,7 +270,7 @@ On slow dispatch, to run another specific test, the equivalent would be:
 2. Run with the slow dispatch mode:
    ```
    export TT_METAL_SLOW_DISPATCH_MODE=1
-   ./build/test/tt_metal/unit_tests/fast_dispatch --gtest_filter_test="BasicFixture.TestL1BuffersAllocatedTopDown"
+   ./build/test/tt_metal/unit_tests/unit_tests_api --gtest_filter="DeviceSingleCardBufferFixture.TestL1BuffersAllocatedTopDown"
    ```
 
 We have split our tests into the two dispatch modes for less pollution of state

tests/scripts/run_cpp_unit_tests.sh

@@ -9,19 +9,25 @@ fi
 
 kernel_path="/tmp/kernels"
 mkdir -p $kernel_path
-TT_METAL_KERNEL_PATH=$kernel_path ./build/test/tt_metal/test_kernel_path_env_var
+TT_METAL_KERNEL_PATH=$kernel_path ./build/test/tt_metal/unit_tests_api --gtest_filter=CompileProgramWithKernelPathEnvVarFixture.*
 rm -rf $kernel_path
 
+./build/test/tt_metal/unit_tests_api
+./build/test/tt_metal/unit_tests_debug_tools
+./build/test/tt_metal/unit_tests_device
+./build/test/tt_metal/unit_tests_dispatch
+./build/test/tt_metal/unit_tests_eth
+./build/test/tt_metal/unit_tests_llk
+./build/test/tt_metal/unit_tests_stl
+
 if [[ ! -z "$TT_METAL_SLOW_DISPATCH_MODE" ]]; then
-    ./build/test/tt_metal/unit_tests
     env python tests/scripts/run_tt_metal.py --dispatch-mode slow
     env python tests/scripts/run_tt_eager.py --dispatch-mode slow
 else
-    ./build/test/tt_metal/unit_tests_fast_dispatch
-    TT_METAL_GTEST_NUM_HW_CQS=2 ./build/test/tt_metal/unit_tests_fast_dispatch_single_chip_multi_queue --gtest_filter=MultiCommandQueueSingleDeviceFixture.*
+    TT_METAL_GTEST_NUM_HW_CQS=2 ./build/test/tt_metal/unit_tests_dispatch --gtest_filter=MultiCommandQueue*Fixture.*
     # Enable this on BH after #14613
     if [[ "$ARCH_NAME" == "wormhole_b0" ]]; then
-        TT_METAL_GTEST_ETH_DISPATCH=1 ./build/test/tt_metal/unit_tests_fast_dispatch
+        TT_METAL_GTEST_ETH_DISPATCH=1 ./build/test/tt_metal/unit_tests_dispatch
     fi
     env python tests/scripts/run_tt_eager.py --dispatch-mode fast
     env python tests/scripts/run_tt_metal.py --dispatch-mode fast

tests/scripts/run_testpoint_perprocess.py

@@ -13,7 +13,7 @@
 
 DEBUG = False
 TT_METAL_HOME = os.environ["TT_METAL_HOME"]
-DEFAULT_GTEST = f"{TT_METAL_HOME}/build/test/tt_metal/unit_tests"
+DEFAULT_GTEST = f"{TT_METAL_HOME}/build/test/tt_metal/unit_tests_api"
 
 
 def extract_list_of_test_points(args: argparse.Namespace):

tests/scripts/run_tests.sh

@@ -76,7 +76,7 @@ run_frequent_api_pipeline_tests() {
     local dispatch_mode=$3
 
     if [[ $dispatch_mode == "slow" ]]; then
-        TT_METAL_SLOW_DISPATCH_MODE=1 ./build/test/tt_metal/unit_tests_frequent
+        TT_METAL_SLOW_DISPATCH_MODE=1 ./build/test/tt_metal/unit_tests_dispatch --gtest_filter=DispatchStress.TensixRunManyTimes
         echo "Running Python API unit tests in SD for frequent..."
         ./tests/scripts/run_python_api_unit_tests.sh
     fi

tests/scripts/run_tools_tests.sh

@@ -12,7 +12,7 @@ if [[ -z "$TT_METAL_SLOW_DISPATCH_MODE" ]] ; then
     echo "Running watcher dump tool tests..."
 
     # Run a test that populates basic fields but not watcher fields
-    ./build/test/tt_metal/unit_tests_fast_dispatch --gtest_filter=*PrintHanging
+    ./build/test/tt_metal/unit_tests_debug_tools --gtest_filter=*PrintHanging
 
     # Run dump tool w/ minimum data - no error expected.
     ./build/tools/watcher_dump -d=0 -w -c
@@ -22,15 +22,15 @@ if [[ -z "$TT_METAL_SLOW_DISPATCH_MODE" ]] ; then
     echo "Watcher dump minimal test - Pass"
 
     # Now run with all watcher features, expect it to throw.
-    ./build/test/tt_metal/unit_tests_fast_dispatch --gtest_filter=*WatcherAssertBrisc
+    ./build/test/tt_metal/unit_tests_debug_tools --gtest_filter=*WatcherAssertBrisc
     ./build/tools/watcher_dump -d=0 -w &> tmp.log || { echo "Above failure is expected."; }
 
     # Verify the error we expect showed up in the program output.
     grep "brisc tripped an assert" tmp.log > /dev/null || { echo "Error: couldn't find expected string in command output:" ; cat tmp.log; exit 1; }
     echo "Watcher dump all data test - Pass"
 
     # Check that stack dumping is working
-    ./build/test/tt_metal/unit_tests_fast_dispatch --gtest_filter=*TestWatcherRingBufferBrisc
+    ./build/test/tt_metal/unit_tests_debug_tools --gtest_filter=*TestWatcherRingBufferBrisc
     ./build/tools/watcher_dump -d=0 -w
     grep "brisc highest stack usage:" generated/watcher/watcher.log > /dev/null || { echo "Error: couldn't find stack usage in watcher log after dump." ; exit 1; }
     echo "Watcher stack usage test - Pass"

tests/scripts/t3000/run_t3000_unit_tests.sh

@@ -8,13 +8,13 @@ run_t3000_ttmetal_tests() {
 
   echo "LOG_METAL: Running run_t3000_ttmetal_tests"
 
-  TT_METAL_SLOW_DISPATCH_MODE=1 ./build/test/tt_metal/unit_tests --gtest_filter="DeviceFixture.EthKernelsDirectSendAllConnectedChips" ; fail+=$?
-  TT_METAL_SLOW_DISPATCH_MODE=1 ./build/test/tt_metal/unit_tests --gtest_filter="DeviceFixture.EthKernelsSendInterleavedBufferAllConnectedChips" ; fail+=$?
-  TT_METAL_SLOW_DISPATCH_MODE=1 ./build/test/tt_metal/unit_tests --gtest_filter="DeviceFixture.EthKernelsDirectRingGatherAllChips" ; fail+=$?
-  TT_METAL_SLOW_DISPATCH_MODE=1 ./build/test/tt_metal/unit_tests --gtest_filter="DeviceFixture.EthKernelsInterleavedRingGatherAllChips" ; fail+=$?
-  TT_METAL_ENABLE_REMOTE_CHIP=1 ./build/test/tt_metal/unit_tests_fast_dispatch --gtest_filter="CommandQueueSingleCardFixture.*" ; fail+=$?
-  ./build/test/tt_metal/unit_tests_fast_dispatch --gtest_filter="CommandQueueMultiDeviceFixture.*" ; fail+=$?
-  ./build/test/tt_metal/unit_tests_fast_dispatch --gtest_filter="DPrintFixture.*:WatcherFixture.*" ; fail+=$?
+  TT_METAL_SLOW_DISPATCH_MODE=1 ./build/test/tt_metal/unit_tests_eth --gtest_filter="DeviceFixture.ActiveEthKernelsDirectSendAllConnectedChips" ; fail+=$?
+  TT_METAL_SLOW_DISPATCH_MODE=1 ./build/test/tt_metal/unit_tests_eth --gtest_filter="DeviceFixture.ActiveEthKernelsSendInterleavedBufferAllConnectedChips" ; fail+=$?
+  TT_METAL_SLOW_DISPATCH_MODE=1 ./build/test/tt_metal/unit_tests_eth --gtest_filter="DeviceFixture.ActiveEthKernelsDirectRingGatherAllChips" ; fail+=$?
+  TT_METAL_SLOW_DISPATCH_MODE=1 ./build/test/tt_metal/unit_tests_eth --gtest_filter="DeviceFixture.ActiveEthKernelsInterleavedRingGatherAllChips" ; fail+=$?
+  TT_METAL_ENABLE_REMOTE_CHIP=1 ./build/test/tt_metal/unit_tests_dispatch --gtest_filter="CommandQueueSingleCard*Fixture.*" ; fail+=$?
+  ./build/test/tt_metal/unit_tests_dispatch --gtest_filter="CommandQueueMultiDevice*Fixture.*" ; fail+=$?
+  ./build/test/tt_metal/unit_tests_debug_tools --gtest_filter="DPrintFixture.*:WatcherFixture.*" ; fail+=$?
 
   # Record the end time
   end_time=$(date +%s)

tests/scripts/tg/run_tg_unit_tests.sh

@@ -5,11 +5,11 @@ run_tg_tests() {
 
   echo "LOG_METAL: running run_tg_unit_tests"
 
-  TT_METAL_ENABLE_REMOTE_CHIP=1 ./build/test/tt_metal/unit_tests_fast_dispatch --gtest_filter="CommandQueueSingleCardFixture.*"
+  TT_METAL_ENABLE_REMOTE_CHIP=1 ./build/test/tt_metal/unit_tests_dispatch --gtest_filter="CommandQueueSingleCard*Fixture.*"
   ./build/test/ttnn/galaxy_unit_tests_ttnn
-  TT_METAL_SLOW_DISPATCH_MODE=1 ./build/test/tt_metal/unit_tests_galaxy --gtest_filter="GalaxyFixture.*:TGFixture.*"
-  ./build/test/tt_metal/unit_tests_galaxy --gtest_filter="GalaxyFixture.*:TGFixture.*"
-  TT_METAL_GTEST_NUM_HW_CQS=2 ./build/test/tt_metal/unit_tests_fast_dispatch_single_chip_multi_queue --gtest_filter="MultiCommandQueueMultiDeviceFixture.*"
+  TT_METAL_SLOW_DISPATCH_MODE=1 ./build/test/tt_metal/unit_tests_device --gtest_filter="GalaxyFixture.*:TGFixture.*"
+  ./build/test/tt_metal/unit_tests_device --gtest_filter="GalaxyFixture.*:TGFixture.*"
+  TT_METAL_GTEST_NUM_HW_CQS=2 ./build/test/tt_metal/unit_tests_dispatch --gtest_filter="MultiCommandQueueMultiDevice*Fixture.*"
 
 }
 

tests/scripts/tgg/run_tgg_unit_tests.sh

@@ -5,10 +5,10 @@ run_tgg_tests() {
 
   echo "LOG_METAL: running run_tgg_unit_tests"
 
-  TT_METAL_ENABLE_REMOTE_CHIP=1 ./build/test/tt_metal/unit_tests_fast_dispatch --gtest_filter="CommandQueueSingleCardFixture.*"
+  TT_METAL_ENABLE_REMOTE_CHIP=1 ./build/test/tt_metal/unit_tests_dispatch --gtest_filter="CommandQueueSingleCard*Fixture.*"
   ./build/test/ttnn/galaxy_unit_tests_ttnn
-  TT_METAL_SLOW_DISPATCH_MODE=1 ./build/test/tt_metal/unit_tests_galaxy --gtest_filter="GalaxyFixture.*:TGGFixture.*"
-  ./build/test/tt_metal/unit_tests_galaxy --gtest_filter="GalaxyFixture.*:TGGFixture.*"
+  TT_METAL_SLOW_DISPATCH_MODE=1 ./build/test/tt_metal/unit_tests_device --gtest_filter="GalaxyFixture.*:TGGFixture.*"
+  ./build/test/tt_metal/unit_tests_device --gtest_filter="GalaxyFixture.*:TGGFixture.*"
   pytest -s tests/ttnn/distributed/test_mesh_device_TGG.py
 }
 

tests/tt_eager/tensors/test_async_tensor_apis.cpp

@@ -14,7 +14,7 @@
 #include "ttnn/tensor/tensor.hpp"
 #include "ttnn/tensor/tensor_impl.hpp"
 #include "ttnn/tensor/types.hpp"
-#include "tests/tt_metal/tt_metal/unit_tests_common/common/common_fixture.hpp"
+#include "tests/tt_metal/tt_metal/common/dispatch_fixture.hpp"
 #include "tt_metal/host_api.hpp"
 #include "ttnn/operations/numpy/functions.hpp"
 
@@ -37,7 +37,7 @@ uint32_t get_device_buffer_address(const Tensor& tensor) {
 }
 }
 
-TEST_F(CommonFixture, TestTensorOwnershipSanity) {
+TEST_F(DispatchFixture, TestTensorOwnershipSanity) {
     // Sanity test tensor read, write and update paths with synchronous
     // Ensure that tensor data is copied and owned as expected
     Device* device = this->devices_[0];
@@ -112,7 +112,7 @@ TEST_F(CommonFixture, TestTensorOwnershipSanity) {
     EXPECT_EQ(readback_tensor.get_shape(), ttnn::Shape(tt::tt_metal::LegacyShape({1, 1, 32, 128})));
 }
 
-TEST_F(CommonFixture, TestAsyncEltwiseBinary) {
+TEST_F(DispatchFixture, TestAsyncEltwiseBinary) {
     Device* device = this->devices_[0];
     device->enable_async(true);
     // Populate these in first loop and verify that deallocation worked - addresses should be identical across loops
@@ -169,7 +169,7 @@ TEST_F(CommonFixture, TestAsyncEltwiseBinary) {
 
 Tensor tensor_identity_copy_function(const Tensor& tensor) { return tensor; }
 
-TEST_F(CommonFixture, TestAsyncRefCountManager) {
+TEST_F(DispatchFixture, TestAsyncRefCountManager) {
     Device* device = this->devices_[0];
     device->enable_async(true);
 
@@ -226,7 +226,7 @@ TEST_F(CommonFixture, TestAsyncRefCountManager) {
     device->enable_async(false);
 }
 
-TEST_F(CommonFixture, TestTensorAsyncDataMovement) {
+TEST_F(DispatchFixture, TestTensorAsyncDataMovement) {
     // Test 2 data paths here (resembles async mode):
     // 1. Main -> Worker: Create a tensor in the main thread. Ensure that it is accessible in the worker thread even
     // after its destroyed

tests/tt_metal/tt_metal/CMakeLists.txt

@@ -35,9 +35,7 @@ set(TT_METAL_TESTS_SRCS
     test_core_range_set.cpp
     test_compile_sets_kernel_binaries.cpp
     test_compile_program.cpp
-    test_kernel_path_env_var.cpp
     test_clean_init.cpp
-    test_create_kernel_from_string.cpp
 )
 
 foreach(TEST_SRC ${TT_METAL_TESTS_SRCS})
@@ -62,21 +60,27 @@ foreach(TEST_SRC ${TT_METAL_TESTS_SRCS})
     list(APPEND METAL_TEST_TARGETS ${TEST})
 endforeach()
 
-add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/unit_tests_common)
-add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/unit_tests)
-add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/unit_tests_fast_dispatch)
-add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/unit_tests_fast_dispatch_single_chip_multi_queue)
-add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/unit_tests_frequent)
+add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/api)
+add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/debug_tools)
+add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/device)
+add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/dispatch)
+add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/eth)
+add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/integration)
+add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/llk)
 add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/perf_microbenchmark)
+add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/stl)
 
 add_custom_target(
     metal_tests
     DEPENDS
         ${METAL_TEST_TARGETS}
-        unit_tests
-        unit_tests_fast_dispatch
-        unit_tests_fast_dispatch_single_chip_multi_queue
-        unit_tests_frequent
         metal_perf_microbenchmark_tests
-        unit_tests_galaxy
+        unit_tests_api
+        unit_tests_debug_tools
+        unit_tests_device
+        unit_tests_dispatch
+        unit_tests_eth
+        unit_tests_integration
+        unit_tests_llk
+        unit_tests_stl
 )

tests/tt_metal/tt_metal/README.md

@@ -0,0 +1,101 @@
+In order to keep our test suite clean, organized and searchable, please follow the guidelines provided below when adding new tests, modifying existing tests or deleting outdated tests.
+
+<!-- toc -->
+
+Table of Contents
+=================
+
+- [Table of Contents](#table-of-contents)
+  - [Test Naming](#test-naming)
+  - [Test Organization](#test-organization)
+  - [Fixture Naming](#fixture-naming)
+  - [Fixture Organization](#fixture-organization)
+  - [File Naming](#file-naming)
+  - [File Organization](#fixture-organization)
+    - [api/](#api)
+    - [debug_tools/](#debug_tools)
+    - [device/](#device)
+    - [dispatch/](#dispatch)
+    - [eth/](#eth)
+    - [integration/](#integration)
+    - [llk/](#llk)
+    - [stl/](#stl)
+    - [test_kernels/](#test_kernels)
+    - [common/](#common)
+
+<!-- Created by https://luciopaiva.com/markdown-toc/ -->
+
+<!-- tocstop -->
+
+## Test Naming
+Prefix test names with the core type(s) that the test is using:
+ - If it's using Tensix cores, prefix it with `Tensix`
+ - If it's using active ethernet cores, prefix it with `ActiveEth`
+ - If it's using idle ethernet cores, prefix it with `IdleEth`
+ - If it's using both active and idle ethernet cores, prefix it with `Eth`
+ - If it's using multiple core types, prefix it with each core type, eg. `TensixActiveEth`, `TensixIdleEth`, `TensixEth`, etc.
+ - If it isn't using any core type, don't prefix it with anything
+
+## Test Organization
+Every test should belong to either a test suite or a test fixture. Use the TEST macro for tests in test suites and the TEST_F or TEST_P macros for tests in test fixtures.
+
+Test suites are ideal for grouping related tests that don’t require shared code. Test fixtures are better suited for related tests that need shared code, which can be defined in the fixture.
+
+Keep related tests grouped together to make it easier to understand the overall test coverage.
+
+## Fixture Naming
+All fixture names should end in `Fixture`.
+
+## Fixture Organization
+Before creating a new fixture, check if an existing fixture meets your needs. If you need to create a new fixture, consider subclassing an existing fixture to avoid duplicating functionality already provided by another fixture.
+
+## File Naming
+File names should include specific prefixes or suffixes based on their content:
+ - Files that contain fixtures should have their names end with `_fixture`
+ - Files that contain helper functions and/or test utilities should have their names end with `_test_utils`
+ - Files that contain tests should have their names start with `test_`
+
+## File Organization
+Place test utility files and fixture files as close as possible to the files that rely on them. For example, if you have a test file `test_A.cpp` in `tests/tt_metal/tt_metal/dispatch/dispatch_buffer/` and another test file `test_B.cpp` in `tests/tt_metal/tt_metal/dispatch/dispatch_program/`, and both need to use a fixture file `C_fixture.hpp`, it is logical to place `C_fixture.hpp` in `tests/tt_metal/tt_metal/dispatch/`. This ensures the fixture is easily accessible to the relevant test files while avoiding unnecessary clutter in a more generic directory like `tests/tt_metal/tt_metal/common/`.
+
+Tests using Google Test should be placed in one of the directories listed below that best aligns with their purpose. If multiple directories seem suitable, use your best judgment to select the most appropriate one.
+
+__Important note: only tests that use Google Test should be placed in the following directories.__
+
+### `api/`
+ - Contains tests that explicitly test `tt-metal`'s API
+ - Contains tests that read from and/or write to the device
+
+### `debug_tools/`
+ - Contains tests for DPrint and Watcher
+
+### `device/`
+ - Contains tests for device initialization and teardown
+ - Contains tests that check device-specific properties
+
+### `dispatch/`
+ - Contains tests that explicitly test for properties relating to dispatch
+ - Contains both slow dispatch and fast dispatch tests
+
+### `eth/`
+ - Contains tests that check ethernet communication between multiple devices
+ - Contains tests that explicitly test ethernet properties on a single device
+
+### `integration/`
+ - Contains tests for real-world use cases, eg. matmul, etc
+
+### `llk/`
+ - Contains tests for compute Low-Level Kernel (LLK) API
+ - Tests don't cover individual compute LLK calls, but cover testing LLK API calls as these are used in compute kernels
+
+### `stl/`
+ - Contains tests which test custom data structures and algorithms used in `tt-metal`
+ - None of the tests in this directory should run on the device
+
+The following directories should be reserved for files that support testing but should not contain actual tests themselves.
+
+### `test_kernels/`
+ - Contains kernels that are used in tests
+
+### `common/`
+ - Contains test fixtures and utilities shared across multiple directories listed above