Merge pull request #81 from zixianwang2022/mlperf-inference-results-s…

…cc24

UCSD Base Mlperf inference results scc24

open/UCSD/code/stable-diffusion-xl/README.md

@@ -0,0 +1 @@
+TBD

open/UCSD/measurements/aqua-reference-rocm-pytorch-v2.6.0.dev20241109-scc24-base/README.md

@@ -0,0 +1,3 @@
+| Model               | Scenario   | Accuracy              |   Throughput | Latency (in ms)   |
+|---------------------|------------|-----------------------|--------------|-------------------|
+| stable-diffusion-xl | offline    | (15.22786, 236.96183) |        0.209 | -                 |

open/UCSD/measurements/aqua-reference-rocm-pytorch-v2.6.0.dev20241109-scc24-base/stable-diffusion-xl/offline/README.md

@@ -0,0 +1,57 @@
+This experiment is generated using the [MLCommons Collective Mind automation framework (CM)](https://github.com/mlcommons/cm4mlops).
+
+*Check [CM MLPerf docs](https://docs.mlcommons.org/inference) for more details.*
+
+## Host platform
+
+* OS version: Linux-5.14.0-427.42.1.el9_4.x86_64-x86_64-with-glibc2.34
+* CPU version: x86_64
+* Python version: 3.11.7 (main, Dec 15 2023, 18:12:31) [GCC 11.2.0]
+* MLCommons CM version: 3.1.0
+
+## CM Run Command
+
+See [CM installation guide](https://docs.mlcommons.org/inference/install/).
+
+```bash
+pip install -U cmind
+
+cm rm cache -f
+
+cm pull repo mlcommons@cm4mlops --checkout=e8235832b1ca225f65ecc8272c597d5c1a112d82
+
+cm run script \
+	--tags=run-mlperf,inference,_r4.1-dev,_short,_scc24-base \
+	--model=sdxl \
+	--implementation=reference \
+	--framework=pytorch \
+	--category=datacenter \
+	--scenario=Offline \
+	--execution_mode=test \
+	--device=rocm \
+	--quiet \
+	--precision=float16 \
+	--env.CM_GET_PLATFORM_DETAILS=no
+```
+*Note that if you want to use the [latest automation recipes](https://docs.mlcommons.org/inference) for MLPerf (CM scripts),
+ you should simply reload mlcommons@cm4mlops without checkout and clean CM cache as follows:*
+
+```bash
+cm rm repo mlcommons@cm4mlops
+cm pull repo mlcommons@cm4mlops
+cm rm cache -f
+
+```
+
+## Results
+
+Platform: aqua-reference-rocm-pytorch-v2.6.0.dev20241109-scc24-base
+
+Model Precision: fp32
+
+### Accuracy Results 
+`CLIP_SCORE`: `15.22786`, Required accuracy for closed division `>= 31.68632` and `<= 31.81332`
+`FID_SCORE`: `236.96183`, Required accuracy for closed division `>= 23.01086` and `<= 23.95008`
+
+### Performance Results 
+`Samples per second`: `0.209132`

open/UCSD/measurements/aqua-reference-rocm-pytorch-v2.6.0.dev20241109-scc24-base/stable-diffusion-xl/offline/aqua-reference-rocm-pytorch-v2.6.0.dev20241109-scc24-base.json

@@ -0,0 +1,7 @@
+{
+  "starting_weights_filename": "https://github.com/mlcommons/inference/tree/master/text_to_image#download-model",
+  "retraining": "no",
+  "input_data_types": "fp32",
+  "weight_data_types": "fp32",
+  "weight_transformations": "no"
+}