Profiling models

qc-ai-test-docs-examples/profile-models/ contains scripts to quickly run multiple models on CPU and NPU on your Dragonwing development board. This is a great place to quickly track model support and model performance. The README of this repository has all the info you'll need.

Example output

# +---------------------------------------------------------------------------+----------------+---------------+----------------+-----------------------+-----------------------+
# | model                                                                     | runtime        | input shape   | variant        | NPU (per inference)   | CPU (per inference)   |
# +===========================================================================+================+===============+================+=======================+=======================+
# | inception_v3-inception-v3-w8a8.tflite                                     | tflite         | (1,224,224,3) | quantized_8bit | 3.377ms.              | 62.04ms.              |
# +---------------------------------------------------------------------------+----------------+---------------+----------------+-----------------------+-----------------------+
# | inception_v3-inception-v3-qualcomm_qcs6490_proxy-w8a8.onnx.zip/model.onnx | onnx           | FAIL          | quantized_8bit | FAIL                  | FAIL                  |
# +---------------------------------------------------------------------------+----------------+---------------+----------------+-----------------------+-----------------------+
# | inception_v3-inception-v3-float.onnx.zip/model.onnx                       | onnx           | (1,3,224,224) | unquantized    | FAIL                  | 161.4ms.              |
# +---------------------------------------------------------------------------+----------------+---------------+----------------+-----------------------+-----------------------+
# | inception_v3-inception-v3-float.tflite                                    | tflite         | (1,224,224,3) | unquantized    | 231.8ms.              | 231.6ms.              |
# +---------------------------------------------------------------------------+----------------+---------------+----------------+-----------------------+-----------------------+
# | inception_v3_w8a8.qcs6490.qnn216.ctx.bin                                  | ai_runtime_sdk | ?             | ?              | 21.62ms.              | -                     |
# +---------------------------------------------------------------------------+----------------+---------------+----------------+-----------------------+-----------------------+