
                                                    
2026-05-04 12:01:57 - evalscope - [32mINFO[0m: 
Benchmarking summary:
+---------------------------------------------+----------+
| Key                                         |    Value |
+=============================================+==========+
| Time taken for tests (s)                    | 134.487  |
+---------------------------------------------+----------+
| Number of concurrency                       |   2      |
+---------------------------------------------+----------+
| Request rate (req/s)                        |  -1      |
+---------------------------------------------+----------+
| Total requests                              |  30      |
+---------------------------------------------+----------+
| Succeed requests                            |  30      |
+---------------------------------------------+----------+
| Failed requests                             |   0      |
+---------------------------------------------+----------+
| Request throughput (req/s)                  |   0.2231 |
+---------------------------------------------+----------+
| Average latency (s)                         |   8.8185 |
+---------------------------------------------+----------+
| Average input tokens per request            |  29.4333 |
+---------------------------------------------+----------+
| Output token throughput (tok/s)             | 216.921  |
+---------------------------------------------+----------+
| Total token throughput (tok/s)              | 223.487  |
+---------------------------------------------+----------+
| Average time to first token (s)             |   0.0467 |
+---------------------------------------------+----------+
| Average time per output token (s)           |   0.009  |
+---------------------------------------------+----------+
| Average inter-token latency (s)             |   0.009  |
+---------------------------------------------+----------+
| Average output tokens per request           | 972.433  |
+---------------------------------------------+----------+
| Average decoded tokens per iter (tok/iter)  |   1.002  |
+---------------------------------------------+----------+
| Approx speculative decoding acceptance rate |   0.002  |
+---------------------------------------------+----------+[0m

Running[perf]:   0%|          | 0/1 [02:24<?, ?it/s]
                                                    
2026-05-04 12:01:57 - evalscope - [32mINFO[0m: 
Percentile results:
+-------------+----------+---------+----------+-------------+--------------+---------------+----------------+---------------+
| Percentiles | TTFT (s) | ITL (s) | TPOT (s) | Latency (s) | Input tokens | Output tokens | Output (tok/s) | Total (tok/s) |
+-------------+----------+---------+----------+-------------+--------------+---------------+----------------+---------------+
|     10%     |   0.04   | 0.0089  |  0.0091  |   7.6944    |      22      |     1024      |    109.148     |   111.9665    |
|     25%     |  0.0424  |  0.009  |  0.0091  |   9.3118    |      24      |     1024      |    109.3803    |    112.417    |
|     50%     |  0.0445  | 0.0091  |  0.0091  |   9.3357    |      29      |     1024      |    109.6573    |   112.7629    |
|     66%     |  0.0509  | 0.0091  |  0.0091  |   9.3462    |      32      |     1024      |    109.6984    |   113.1687    |
|     75%     |  0.0536  | 0.0092  |  0.0091  |   9.3506    |      32      |     1024      |    109.7787    |    113.555    |
|     80%     |  0.0547  | 0.0092  |  0.0091  |   9.3597    |      33      |     1024      |    109.8752    |   113.8926    |
|     90%     |  0.0566  | 0.0093  |  0.0091  |   9.3735    |      40      |     1024      |    110.0526    |   115.1791    |
|     95%     |  0.0568  | 0.0094  |  0.0091  |   9.3818    |      42      |     1024      |    110.1566    |    116.637    |
|     98%     |  0.0568  | 0.0095  |  0.0091  |   9.3861    |      47      |     1024      |    133.0837    |   136.2029    |
|     99%     |  0.0568  | 0.0096  |  0.0091  |   9.3861    |      47      |     1024      |    133.0837    |   136.2029    |
+-------------+----------+---------+----------+-------------+--------------+---------------+----------------+---------------+[0m

Running[perf]:   0%|          | 0/1 [02:24<?, ?it/s]
                                                    
2026-05-04 12:01:57 - evalscope - [32mINFO[0m: Save the summary to: outputs/20260504_115932/qwen-3.6-full/parallel_2_number_30[0m

Running[perf]:   0%|          | 0/1 [02:24<?, ?it/s]
Running[perf]: 100%|██████████| 1/1 [02:24<00:00, 144.96s/it]
                                                             
2026-05-04 12:01:57 - evalscope - [32mINFO[0m: Running[perf] 100%| 1/1 [Elapsed: 02:24 < Remaining: 00:00, 144.96s/it][0m

Running[perf]: 100%|██████████| 1/1 [02:24<00:00, 144.96s/it]
Running[perf]: 100%|██████████| 1/1 [02:24<00:00, 144.96s/it]
╭──────────────────────────────────────────────────────────────────────────────╮
│ Performance Test Summary Report                                              │
╰──────────────────────────────────────────────────────────────────────────────╯

Basic Information:
┌───────────────────────┬──────────────────────────────────────────────────────┐
│ Model                 │ qwen-3.6-full                                        │
│ Test Dataset          │ openqa                                               │
│ API Type              │ openai                                               │
│ Total Generated       │ 29,172.999 tokens                                    │
│ Avg Output Rate       │ 216.92 tokens/sec                                    │
│ Total Test Time       │ 134.49 seconds                                       │
│ Output Path           │ outputs/20260504_115932/qwen-3.6-full                │
└───────────────────────┴──────────────────────────────────────────────────────┘


                                       Detailed Performance Metrics                                       
┏━━━━━━┳━━━━━━┳━━━━━┳━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┓
┃      ┃      ┃     ┃      ┃     Avg ┃     P99 ┃     Avg ┃     P99 ┃     Avg ┃     P99 ┃   Gen. ┃ Success┃
┃Conc. ┃ Rate ┃ Num ┃  RPS ┃ Lat.(s) ┃ Lat.(s) ┃ TTFT(s) ┃ TTFT(s) ┃ TPOT(s) ┃ TPOT(s) ┃ toks/s ┃    Rate┃
┡━━━━━━╇━━━━━━╇━━━━━╇━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━━━┩
│    2 │  INF │  30 │ 0.22 │   8.819 │   9.386 │   0.047 │   0.057 │   0.009 │   0.009 │ 216.92 │  100.0%│
└──────┴──────┴─────┴──────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴────────┴────────┘


                              Request Metrics                               
┏━━━━━━┳━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━┓
┃      ┃     ┃ Avg In ┃ P99 In ┃ Avg Out ┃ P99 Out ┃  Decoded ┃       Spec.┃
┃Conc. ┃ Num ┃   Toks ┃   Toks ┃    Toks ┃    Toks ┃ Tok/Iter ┃ Accept Rate┃
┡━━━━━━╇━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━┩
│    2 │  30 │   29.4 │   47.0 │   972.4 │  1024.0 │     1.00 │        0.2%│
└──────┴─────┴────────┴────────┴─────────┴─────────┴──────────┴────────────┘


               Best Performance Configuration               
 Highest RPS         Concurrency 2 (0.22 req/sec)           
 Lowest Latency      Concurrency 2 (8.819 seconds)          

Performance Recommendations:
• The system seems not to have reached its performance bottleneck, try higher concurrency
2026-05-04 12:01:57 - evalscope - [32mINFO[0m: Performance summary saved to: outputs/20260504_115932/qwen-3.6-full/performance_summary.txt[0m
2026-05-04 12:01:57 - evalscope - [32mINFO[0m: HTML report generated: outputs/20260504_115932/qwen-3.6-full/perf_report.html[0m