You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Running the Archive Query Log on a cluster is recommended for large-scale crawls. We provide a Helm chart that automatically starts crawling and parsing jobs for you and stores the results in an Elasticsearch cluster.
405
+
Running the Archive Query Log on a cluster is recommended for large-scale crawls. We provide a [Helm chart](https://github.com/webis-de/archive-query-log/pkgs/container/archive-query-log%2Fcharts%2Farchive-query-log) that automatically starts crawling and parsing jobs for you and stores the results in an Elasticsearch cluster.
406
406
407
407
#### Installation
408
408
409
409
Just install [Helm](https://helm.sh/docs/intro/quickstart/) and configure `kubectl` for your cluster.
410
410
411
411
#### Cluster configuration
412
412
413
-
Crawling the Archive Query Log requires access to an Elasticsearch cluster and some S3 block storage. Configure the Elasticsearch and S3 credentials in a `values.override.yaml` file like this:
413
+
Crawling the Archive Query Log requires access to an Elasticsearch cluster and some S3 block storage. Configure the Elasticsearch and S3 credentials in a `values.yaml` file like this:
414
414
415
415
```yaml
416
416
elasticsearch:
@@ -430,11 +430,18 @@ s3:
430
430
Let us deploy the Helm chart on the cluster (we are testing first with `--dry-run` to see if everything works):
Replace `X.Y.Z` with the latest version of the chart, as found on [GitHub Packages](https://github.com/webis-de/archive-query-log/pkgs/container/archive-query-log%2Fcharts%2Farchive-query-log)
436
437
If everything works and the output looks good, you can remove the `--dry-run` flag to actually deploy the chart.
0 commit comments