Test sharding runtime unbalance, file/directory weights?

[Arne1303]

Test sharding is great especially on the CI, but we've noticed that there is a large unbalance in runtime between the shards created which diminishes its return.

The unbalance is quite pronounced mainly because there are whole directories which have smaller & faster test files (Like the mode, Middleware, controllerl directories) and others which have files with a lot longer runtime (like the livewire directory). Since Pest always adds the files to the shards in order they are found in the file system all the fast model tests and all the slow livewire tests run in 1 shard each.

Is there any way to reduce the unbalance?

I could imagine 2 ways, 1. add an option for pest to "mix" the files when creating shards, this would probably help, but it would need to be deterministic. Another option would be to add a "shard Weight" option in some way, being able to define that tests in the model directory have a weight of 0.25 compared to the rest would already help a lot.