AIBOX-K3 AI Vision Usage

799959745

Step 1:
Download the Bianbu firmware and flash it to the AIBOX-K3.
Firmware flashing tutorial link: https://wiki.t-firefly.com/en/AI ... tmode_spacemit.html

Step 2:
2.1 Obtaining the Code
Install the Git tool

1. sudo apt update
   
2. 
   
3. sudo apt install git

Copy the code

This repository uses Git Submodules to manage the source code of each component; recursive fetching is recommended.

1. git clone --recurse-submodules https://github.com/spacemit-com/model-zoo.git
   
2. # or ssh
   
3. # git clone --recurse-submodules git@github.com:spacemit-com/model-zoo.git

Copy the code

If you have already cloned the main repository but not the sub-repository:

1. cd model-zoo
   
2. git submodule update --init --recursive

Copy the code

If you only want to compile a specific component in the SDK, you can navigate to the corresponding directory and execute `mm` (you also need to source `build/envsetup.sh` first):

1. source build/envsetup.sh
   
2. 
   
3. cd vision && mm
   

Copy the code

The build artifacts are typically installed to output/staging (based on the SDK project).

3. Example Run
Note: The following examples assume you have already completed the one-click compilation as described above (the example executables will be installed in output/staging).

3.1 Computer Vision
Step 1: Download the Model. Download all vision example models (they will be placed in ~/.cache/models/vision/)

1. bash vision/scripts/download_all_models.sh

Copy the code

Step 2: Download resource files (images/videos). Download example image/video resources (they will be placed under ~/.cache/assets/)

1. bash vision/scripts/download_assets.sh

Copy the code

Default directories:
Models: ~/.cache/models/vision/
Resources: ~/.cache/assets/

Step 3: Run the Examples
The following are the commands to run all the examples under vision/examples/ (C++, which can be run directly in the SDK root directory after executing m):

1. # Facial Recognition (Similarity)
   
2. arcface vision/examples/arcface/config/arcface.yaml
   
3. 
   
4. # Face detection
   
5. yolov5-face vision/examples/yolov5-face/config/yolov5-face.yaml
   
6. 
   
7. # Gesture detection
   
8. yolov5_gesture vision/examples/yolov5_gesture/config/yolov5_gesture.yaml
   
9. 
   
10. # Object Detection
    
11. yolov8 vision/examples/yolov8/config/yolov8.yaml
    
12. yolov11 vision/examples/yolov11/config/yolov11.yaml
    
13. 
    
14. # Attitude estimation
    
15. yolov8_pose vision/examples/yolov8_pose/config/yolov8_pose.yaml
    
16. 
    
17. # Instance splitting
    
18. yolov8_seg vision/examples/yolov8_seg/config/yolov8_seg.yaml
    
19. 
    
20. # Image Classification / Emotion Recognition
    
21. resnet vision/examples/resnet/config/resnet50.yaml
    
22. emotion vision/examples/emotion/config/emotion.yaml
    
23. 
    
24. # Multi-target tracking (video/camera)
    
25. # The following two examples demonstrate real-time target tracking and require a screen connection; otherwise, an error will occur.
    
26. bytetrack vision/examples/bytetrack/config/bytetrack.yaml
    
27. ocsort vision/examples/ocsort/config/ocsort.yaml

Copy the code

Example results:
arcface

1. root@firefly:/home/firefly/model-zoo# arcface vision/examples/arcface/config/arcface.yaml
   
2. SpaceMIT EP initialized: /root/.cache/models/vision/arcface/arcface_mobilefacenet_cut.q.onnx
   
3. 相似度: 0.1584
   
4. 判断: 非同一人

Copy the code

yolov5-face

1. root@firefly:/home/firefly/model-zoo# yolov5-face vision/examples/yolov5-face/config/yolov5-face.yaml
   
2. Loading image: /root/.cache/assets/image/006_test.jpg
   
3. SpaceMIT EP initialized: /root/.cache/models/vision/yolov5-face/yolov5n-face_cut.q.onnx
   
4. Detected 3 face(s):
   
5.   face 1 score=0.812 box=[166.036,75.602,196.464,110.691]
   
6.   face 2 score=0.799 box=[44.460,117.240,65.123,145.286]
   
7.   face 3 score=0.748 box=[262.426,125.130,287.910,162.370]
   
8. Result saved to: result_face.jpg
   

Copy the code

yolov5_gesture

1. root@firefly:/home/firefly/model-zoo# yolov5_gesture vision/examples/yolov5_gesture/config/yolov5_gesture.yaml
   
2. Warning: failed to load labels from: assets/labels/gesture.txt
   
3. Loading image: /root/.cache/assets/image/012_gesture.jpg
   
4. SpaceMIT EP initialized: /root/.cache/models/vision/yolov5/yolov5_gesture.q.onnx
   
5. Detected 2 gesture(s):
   
6.   Class 0 (class 0) score=0.900 box=[809.209,762.571,923.803,915.754]
   
7.   Class 5 (class 5) score=0.815 box=[401.791,1386.415,469.781,1524.662]
   
8. Result saved to: result_gesture.jpg

Copy the code

yolov8

1. root@firefly:/home/firefly/model-zoo# yolov8 vision/examples/yolov8/config/yolov8.yaml
   
2. Loading image: /root/.cache/assets/image/006_test.jpg
   
3. SpaceMIT EP initialized: /root/.cache/models/vision/yolov8/yolov8n.q.onnx
   
4. Detected 3 objects:
   
5.   Class 0, Score: 0.9169, Box: [0.5928,114.2840,85.0366,351.8688]
   
6.   Class 0, Score: 0.8943, Box: [230.9659,122.4463,315.6410,371.7448]
   
7.   Class 33, Score: 0.6868, Box: [64.7624,168.9862,247.7783,371.1805]
   
8. Result image saved to: result.jpg
   

Copy the code

yolov11

1. root@firefly:/home/firefly/model-zoo# yolov11 vision/examples/yolov11/config/yolov11.yaml
   
2. Loading image: /root/.cache/assets/image/006_test.jpg
   
3. SpaceMIT EP initialized: /root/.cache/models/vision/yolov11/yolo11n.q.onnx
   
4. Detected 5 objects:
   
5.   1: Class 0, Score: 0.8821, Box: [0.2121,114.1500,84.3134,352.4440]
   
6.   2: Class 0, Score: 0.8821, Box: [230.5667,121.9574,315.4816,372.6761]
   
7.   3: Class 0, Score: 0.5720, Box: [111.6640,48.7848,215.3329,269.6542]
   
8.   4: Class 0, Score: 0.4291, Box: [6.0540,261.3405,14.3792,269.3897]
   
9.   5: Class 33, Score: 0.6063, Box: [67.3937,236.3271,246.2147,372.6607]
   
10. Result image saved to: yolo11_result.jpg

Copy the code

yolov8_pose

1. root@firefly:/home/firefly/model-zoo# yolov8_pose vision/examples/yolov8_pose/config/yolov8_pose.yaml
   
2. Running pose estimation...
   
3. SpaceMIT EP initialized: /root/.cache/models/vision/yolov8_pose/yolov8n-pose.q.onnx
   
4. Detected 1 persons with keypoints
   
5. Result image saved to: yolov8_pose_result.jpg

Copy the code

yolov8_seg

1. root@firefly:/home/firefly/model-zoo# yolov8_seg vision/examples/yolov8_seg/config/yolov8_seg.yaml
   
2. Running instance segmentation...
   
3. SpaceMIT EP initialized: /root/.cache/models/vision/yolov8_seg/yolov8n-seg.q.onnx
   
4. Detected 4 objects with segmentation masks
   
5. Result image saved to: yolov8_seg_result.jpg
   

Copy the code

resnet

1. root@firefly:/home/firefly/model-zoo# resnet vision/examples/resnet/config/resnet50.yaml
   
2. Loading image: /root/.cache/assets/image/005_kitten.jpg
   
3. SpaceMIT EP initialized: /root/.cache/models/vision/resnet/resnet50.q.onnx
   
4. Classification results:
   
5.   1. tabby, tabby cat (confidence: 0.5056)

Copy the code

emotion

1. root@firefly:/home/firefly/model-zoo# emotion vision/examples/emotion/config/emotion.yaml
   
2. Loading image: /root/.cache/assets/image/003_face0.png
   
3. SpaceMIT EP initialized: /root/.cache/models/vision/resnet/emotion_resnet50_final.q.onnx
   
4. Emotion: neutral (class 0, score: 1.7003)
   
5. Result saved to: result_emotion.jpg

Copy the code

For optional parameters for each example (such as --image, --video, --use-camera, --output, threshold, etc.), please refer to the README in the corresponding directory (vision/examples/*/README.md) or see the model-zoo-vision README.