feat(benchmark): rewrite the public API

docs/.vitepress/config.ts

@@ -904,6 +904,10 @@ export default ({ mode }: { mode: string }) => {
                 text: 'Testing Types',
                 link: '/guide/testing-types',
               },
+              {
+                text: 'Benchmarking',
+                link: '/guide/benchmarking',
+              },
               {
                 text: 'Vitest UI',
                 link: '/guide/ui',

docs/api/advanced/vitest.md

@@ -25,13 +25,13 @@ Vitest 4 added several new APIs (they are marked with a "4.0.0+" badge) and remo
 - `listFile` (use [`getRelevantTestSpecifications`](#getrelevanttestspecifications) instead)
 :::
 
-## mode
+## mode <Deprecated /> {#mode}
 
 ### test
 
 Test mode will only call functions inside `test` or `it`, and throws an error when `bench` is encountered. This mode uses `include` and `exclude` options in the config to find test files.
 
-### benchmark <Badge type="warning">experimental</Badge>
+### benchmark
 
 Benchmark mode calls `bench` functions and throws an error, when it encounters `test` or `it`. This mode uses `benchmark.include` and `benchmark.exclude` options in the config to find benchmark files.
 

docs/config/benchmark.md

@@ -32,39 +32,6 @@ Include globs for in-source benchmark test files. This option is similar to [`in
 
 When defined, Vitest will run all matched files with `import.meta.vitest` inside.
 
-## benchmark.reporters
+## benchmark.retainSamples
 
-- **Type:** `Arrayable<BenchmarkBuiltinReporters | Reporter>`
-- **Default:** `'default'`
-
-Custom reporter for output. Can contain one or more built-in report names, reporter instances, and/or paths to custom reporters.
-
-## benchmark.outputFile
-
-Deprecated in favor of `benchmark.outputJson`.
-
-## benchmark.outputJson {#benchmark-outputJson}
-
-- **Type:** `string | undefined`
-- **Default:** `undefined`
-
-A file path to store the benchmark result, which can be used for `--compare` option later.
-
-For example:
-
-```sh
-# save main branch's result
-git checkout main
-vitest bench --outputJson main.json
-
-# change a branch and compare against main
-git checkout feature
-vitest bench --compare main.json
-```
-
-## benchmark.compare {#benchmark-compare}
-
-- **Type:** `string | undefined`
-- **Default:** `undefined`
-
-A file path to a previous benchmark result to compare against current runs.
+Include `samples` array of benchmark results for API or custom reporter usages.

docs/guide/advanced/index.md

@@ -14,7 +14,6 @@ You can import any method from the `vitest/node` entry-point.
 
 ```ts
 function startVitest(
-  mode: VitestRunMode,
   cliFilters: string[] = [],
   options: CliOptions = {},
   viteOverrides?: ViteUserConfig,
@@ -60,7 +59,6 @@ The ["Running Tests"](/guide/advanced/tests#startvitest) guide has a usage examp
 
 ```ts
 function createVitest(
-  mode: VitestRunMode,
   options: CliOptions,
   viteOverrides: ViteUserConfig = {},
   vitestOptions: VitestOptions = {},