Skip to content

Commit 5e97423

Browse files
yushulxyushulx
committed
benchmark tool
1 parent 9da2b20 commit 5e97423

File tree

7 files changed

+2314
-0
lines changed

7 files changed

+2314
-0
lines changed

‎examples/official/annotation_tool/benchmark/README.md‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
# Barcode Benchmark App
2+
3+
A PySide6 desktop application for benchmarking barcode-reading SDKs against annotated image datasets.
4+
5+
## Features
6+
7+
- **Drag & drop** images or folders onto the image viewer, or use the **Browse Files** button
8+
- **Load annotation JSON** (`barcode-benchmark/1.0` format) to enable ground-truth accuracy measurement
9+
- **Load a custom Dynamsoft template** (optional JSON) to override the default capture settings
10+
- **Select SDKs** — Dynamsoft Barcode Reader and/or ZXing-C++
11+
- **Live progress** and per-file results table updated in real time during the benchmark
12+
- **Image viewer with annotation overlay** — after the benchmark, browse images with:
13+
- **Green polygons** = ground-truth barcode locations (from annotation JSON)
14+
- **Colored rectangles** = barcode locations detected by each SDK (one color per library)
15+
- Barcode text labels drawn next to each box
16+
- **Export HTML report** — self-contained file matching the web-app report layout, with aggregate summary and per-image detail tables
17+
18+
## Folder Structure
19+
20+
```
21+
benchmark_app/
22+
├── gui_benchmark.py # Main application entry point
23+
├── requirements.txt # Python dependencies
24+
├── config/
25+
│ └── benchmark_config.json # SDK configuration (licenses, enabled flags)
26+
├── src/
27+
│ ├── barcode_readers.py # DynamsoftBarcodeReader, ZXingCppReader
28+
│ └── benchmark_framework.py # BarcodeReaderInterface, BenchmarkResult, TestCase
29+
└── README.md
30+
```
31+
32+
## Requirements
33+
34+
See [`requirements.txt`](requirements.txt).
35+
36+
Install:
37+
38+
```bash
39+
pip install -r requirements.txt
40+
```
41+
42+
## Usage
43+
44+
```bash
45+
python benchmark_app/gui_benchmark.py
46+
```
47+
48+
### Workflow
49+
50+
1. **Add images** — drag & drop images / folders onto the viewer, or click **Browse Files**.
51+
Supports: JPG, PNG, BMP, TIFF, WebP.
52+
2. **Load Annotation JSON** *(optional)* — click the toolbar button to load a `barcode-benchmark/1.0` file exported from the annotation tool. The status badge shows `✓ N images, N barcodes`.
53+
3. **Load DBR Template** *(optional)* — load a custom Dynamsoft template JSON to override capture settings.
54+
4. **Select SDKs** — check one or more of: Dynamsoft, ZXing Cpp.
55+
5. **Run Benchmark** — the progress bar tracks each file; the Detailed Results table fills in real time.
56+
6. **Review results in the image viewer**:
57+
- Use **Prev / Next** or click any row in the Detailed Results table.
58+
- **Green polygons** show expected barcode positions from the annotation.
59+
- **Colored boxes** show what each SDK detected; each library uses a distinct color.
60+
- The text strip below the image lists detected values and GT match status.
61+
7. **Export HTML Report** — saves a self-contained HTML file with the same layout as the web-app benchmark report.
62+
63+
## Annotation JSON Format
64+
65+
The annotation JSON must follow the `barcode-benchmark/1.0` schema produced by the annotation tool:
66+
67+
```json
68+
{
69+
"format": "barcode-benchmark/1.0",
70+
"total_images": 5,
71+
"total_barcodes": 12,
72+
"images": [
73+
{
74+
"file": "IMG_001.jpg",
75+
"barcodes": [
76+
{
77+
"text": "1234567890",
78+
"format": "Code128",
79+
"points": [[x1,y1], [x2,y2], [x3,y3], [x4,y4]]
80+
}
81+
]
82+
}
83+
]
84+
}
85+
```
86+
87+
`text` is used for accuracy matching; `points` are used to draw the ground-truth overlay polygon on the image.
88+
89+
## Configuration
90+
91+
Edit `config/benchmark_config.json` to set SDK license keys and enable/disable libraries:
92+
93+
```json
94+
{
95+
"libraries": {
96+
"dynamsoft": { "enabled": true, "license": "YOUR_LICENSE_KEY" },
97+
"zxing_cpp": { "enabled": true, "options": {} }
98+
}
99+
}
100+
```
101+
102+
You can also update the Dynamsoft license through the **SDK Settings** dialog inside the app.
103+

‎examples/official/annotation_tool/benchmark/config/benchmark_config.json‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
{
2+
"benchmark_settings": {
3+
"num_iterations": 100,
4+
"warmup_iterations": 10,
5+
"output_formats": [
6+
"json",
7+
"csv",
8+
"html"
9+
],
10+
"save_intermediate_results": true,
11+
"enable_gpu_acceleration": false
12+
},
13+
"test_image_settings": {
14+
"image_sizes": [
15+
{
16+
"width": 800,
17+
"height": 600
18+
},
19+
{
20+
"width": 1920,
21+
"height": 1080
22+
},
23+
{
24+
"width": 3840,
25+
"height": 2160
26+
}
27+
],
28+
"dpi": 300,
29+
"format": "PNG",
30+
"rotation_angles": [
31+
0,
32+
15,
33+
30,
34+
45,
35+
60,
36+
75,
37+
90
38+
],
39+
"noise_levels": [
40+
0,
41+
5,
42+
10,
43+
15,
44+
20
45+
],
46+
"multiple_barcode_counts": [
47+
2,
48+
5,
49+
10,
50+
20
51+
]
52+
},
53+
"barcode_types": {
54+
"supported_1d": [
55+
"CODE_128",
56+
"CODE_39",
57+
"EAN_13",
58+
"EAN_8",
59+
"UPC_A",
60+
"UPC_E",
61+
"CODABAR",
62+
"ITF"
63+
],
64+
"supported_2d": [
65+
"QR_CODE",
66+
"DATA_MATRIX",
67+
"PDF_417",
68+
"AZTEC"
69+
]
70+
},
71+
"libraries": {
72+
"zxing_cpp": {
73+
"enabled": true,
74+
"options": {
75+
"try_harder": true,
76+
"try_rotate": true
77+
}
78+
},
79+
"pyzbar": {
80+
"enabled": false,
81+
"options": {}
82+
},
83+
"dynamsoft": {
84+
"enabled": true,
85+
"license": "DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==",
86+
"options": {
87+
"try_harder": true,
88+
"enable_inverse_detection": true,
89+
"enable_quiet_zone_detection": true
90+
}
91+
}
92+
},
93+
"metrics": {
94+
"track_performance": true,
95+
"track_accuracy": true,
96+
"track_success_rate": true
97+
},
98+
"output": {
99+
"results_directory": "results/",
100+
"test_images_directory": "test_data/",
101+
"report_filename": "benchmark_report.html",
102+
"detailed_results_filename": "detailed_results.json"
103+
}
104+
}

0 commit comments

Comments
 (0)