Skip to content

Commit

Permalink
Results on system MI210
Browse files Browse the repository at this point in the history
  • Loading branch information
zixianwang2022 committed Nov 18, 2024
1 parent be8440c commit 5901e6c
Show file tree
Hide file tree
Showing 34 changed files with 13,334 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
| Model | Scenario | Accuracy | Throughput | Latency (in ms) |
|---------------------|------------|-----------------------|--------------|-------------------|
| stable-diffusion-xl | offline | (16.38183, 236.85707) | 0.208 | - |
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
This experiment is generated using the [MLCommons Collective Mind automation framework (CM)](https://github.com/mlcommons/cm4mlops).

*Check [CM MLPerf docs](https://docs.mlcommons.org/inference) for more details.*

## Host platform

* OS version: Linux-5.14.0-427.42.1.el9_4.x86_64-x86_64-with-glibc2.35
* CPU version: x86_64
* Python version: 3.10.15 (main, Oct 3 2024, 07:27:34) [GCC 11.2.0]
* MLCommons CM version: 3.4.1

## CM Run Command

See [CM installation guide](https://docs.mlcommons.org/inference/install/).

```bash
pip install -U cmind

cm rm cache -f

cm pull repo mlcommons@cm4mlops --checkout=b32ded2a4c3039ad16dadc734bee03dd1a97f228

cm run script \
--tags=run-mlperf,inference,_r4.1-dev,_short,_scc24-base \
--model=sdxl \
--implementation=reference \
--framework=pytorch \
--category=datacenter \
--scenario=Offline \
--execution_mode=test \
--device=rocm \
--quiet \
--precision=float16
```
*Note that if you want to use the [latest automation recipes](https://docs.mlcommons.org/inference) for MLPerf (CM scripts),
you should simply reload mlcommons@cm4mlops without checkout and clean CM cache as follows:*

```bash
cm rm repo mlcommons@cm4mlops
cm pull repo mlcommons@cm4mlops
cm rm cache -f

```

## Results

Platform: aqua-reference-rocm-pytorch-v2.6.0.dev20241118-scc24-base

Model Precision: fp32

### Accuracy Results
`CLIP_SCORE`: `16.38183`, Required accuracy for closed division `>= 31.68632` and `<= 31.81332`
`FID_SCORE`: `236.85707`, Required accuracy for closed division `>= 23.01086` and `<= 23.95008`

### Performance Results
`Samples per second`: `0.20835`

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"starting_weights_filename": "https://github.com/mlcommons/inference/tree/master/text_to_image#download-model",
"retraining": "no",
"input_data_types": "fp32",
"weight_data_types": "fp32",
"weight_transformations": "no"
}
Loading

0 comments on commit 5901e6c

Please sign in to comment.