Skip to content

Commit 38060dc

Browse files
srozsnyaisrozsnyai
committed
feat: introduce Spanner CDC load test tool with various strategies and Cloud Run deployment.
1 parent b579296 commit 38060dc

File tree

11 files changed

+1160
-0
lines changed

11 files changed

+1160
-0
lines changed

‎spanner-tools/spanner_cdc_loadtest/Dockerfile‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Use a multi-stage build to keep the final image clean
2+
FROM maven:3.9-eclipse-temurin-17 AS builder
3+
WORKDIR /app
4+
COPY pom.xml .
5+
COPY src ./src
6+
# Build the shaded JAR
7+
RUN mvn clean package -DskipTests
8+
9+
# Run stage
10+
FROM eclipse-temurin:17-jre
11+
WORKDIR /app
12+
COPY --from=builder /app/target/spanner-cdc-loadtest-1.0-SNAPSHOT.jar app.jar
13+
14+
# Entry point allows passing arguments directly to the JAR
15+
ENTRYPOINT ["java", "-jar", "app.jar"]

‎spanner-tools/spanner_cdc_loadtest/README.md‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
# Cloud Spanner CDC Load Test
2+
3+
A Java-based load generator for Cloud Spanner, designed to test Change Data Capture (CDC) and general database performance. It supports various strategies to simulate different types of load.
4+
5+
## Strategies
6+
7+
| Strategy | Description |
8+
| :--- | :--- |
9+
| **RANDOM** | Standard Insert/Update/Delete on random IDs. |
10+
| **SEQUENTIAL** | Strict sequence of Insert -> 10 Updates -> Delete on a unique ID. High churn per ID. |
11+
| **HOTSPOT** | High contention on a single shared row (Read-Write Transaction). |
12+
| **ATOMICITY** | Simulates money transfer between two shared rows (Read-Write Transaction). |
13+
| **SATURATION** | Large batch inserts (~1MB per op) to saturate network/CPU. |
14+
| **INTEGRITY** | Rapid Insert/Delete cycles on a fixed pool of keys (Resurrection testing). |
15+
| **READ_HEAVY** | 90% Read (point lookup), 10% Insert. Maintains local cache of 10k recent keys. |
16+
| **MIXED** | Cycles through all the above strategies across workers. |
17+
18+
## Local Usage
19+
20+
### Prerequisites
21+
- Java 17+
22+
- Maven 3.8+
23+
- Google Cloud SDK (`gcloud`)
24+
25+
### Build
26+
```bash
27+
mvn clean package -DskipTests
28+
```
29+
30+
### Run Locally
31+
```bash
32+
java -jar target/spanner-cdc-loadtest-1.0-SNAPSHOT.jar \
33+
-p <PROJECT_ID> \
34+
-i <INSTANCE_ID> \
35+
-d <DATABASE_ID> \
36+
-c 10 \
37+
-s MIXED \
38+
-c 10 \
39+
-s MIXED \
40+
--duration 60 \
41+
--create-schema
42+
```
43+
*Note: Schema creation is disabled by default. Use `--create-schema` to enable it.*
44+
45+
---
46+
47+
## Cloud Run Jobs Deployment
48+
49+
Run massive load tests serverlessly using Cloud Run Jobs.
50+
51+
### 1. Variables
52+
Set your environment variables:
53+
```bash
54+
export PROJECT_ID="your-project-id"
55+
export REGION="us-central1"
56+
# Use Artifact Registry (pkg.dev), NOT gcr.io
57+
export REPO_NAME="spanner-loadtest-repo"
58+
export IMAGE_NAME="${REGION}-docker.pkg.dev/${PROJECT_ID}/${REPO_NAME}/spanner-loadtest:latest"
59+
export SPANNER_INSTANCE="your-instance-id"
60+
export SPANNER_DATABASE="your-database-id"
61+
```
62+
63+
### 2. Initial Setup (One-time)
64+
Create the Artifact Registry repository:
65+
```bash
66+
gcloud artifacts repositories create ${REPO_NAME} \
67+
--repository-format=docker \
68+
--location=${REGION} \
69+
--description="Docker repository for Spanner Load Test"
70+
```
71+
72+
### 3. Build and Push
73+
Build the container image using Cloud Build:
74+
```bash
75+
gcloud builds submit --tag ${IMAGE_NAME}
76+
```
77+
78+
### 4. Create the Job
79+
Create the Cloud Run Job. Note the `--args` syntax requires a single comma-separated string.
80+
81+
```bash
82+
gcloud run jobs create spanner-loadtest \
83+
--image ${IMAGE_NAME} \
84+
--region ${REGION} \
85+
--cpu=4 \
86+
--memory=8Gi \
87+
--memory=8Gi \
88+
--task-timeout=1h \
89+
--args="-p,${PROJECT_ID},-i,${SPANNER_INSTANCE},-d,${SPANNER_DATABASE},-c,5,-s,MIXED,--duration,60"
90+
```
91+
*Note: Ensure `--task-timeout` is greater than your load test `--duration`.*
92+
93+
### 5. Execute the Load Test
94+
Run the job. You can override arguments (e.g., to increase concurrency or change strategy) using the `--args` flag.
95+
96+
**Example: Run with 200 Total Threads**
97+
(50 threads per task * 4 tasks)
98+
99+
```bash
100+
gcloud run jobs execute spanner-loadtest \
101+
--region ${REGION} \
102+
--tasks 4 \
103+
--task-timeout=30m \
104+
--args="-p,${PROJECT_ID},-i,${SPANNER_INSTANCE},-d,${SPANNER_DATABASE},-c,50,-s,HOTSPOT,--duration,300"
105+
```
106+
107+
### 6. Update Configuration (Resources)
108+
To change the CPU or Memory limits for an existing job:
109+
```bash
110+
gcloud run jobs update spanner-loadtest \
111+
--cpu=8 \
112+
--memory=16Gi \
113+
--region=${REGION}
114+
--region=${REGION}
115+
```
116+
**Important:** These resources (8 CPU, 16Gi Memory) are allocated **PER TASK**.
117+
If you execute this job with `--tasks 10`, you will provision **10 separate containers**, each with 8 vCPUs (Total: 80 vCPUs).
118+
Rates are billed per vCPU-second and GB-second for each task.
119+
120+
### Troubleshooting
121+
- **Error: `None of [grpclb] specified`**: The shaded JAR is missing `ServicesResourceTransformer`. Ensure it's in `pom.xml` and rebuild.
122+
- **Error: `denied: gcr.io repo does not exist`**: Use Artifact Registry (`pkg.dev`) as shown above.
123+
124+
125+
# End-to-End
126+
127+
## Variables
128+
```
129+
export PROJECT_ID="your-project-id"
130+
export REGION="us-central1"
131+
export IMAGE_NAME="gcr.io/${PROJECT_ID}/spanner-loadtest:latest"
132+
133+
export SPANNER_INSTANCE="your-instance-id"
134+
export SPANNER_DATABASE="your-database-id"
135+
```
136+
137+
## Initial setup
138+
139+
```
140+
gcloud artifacts repositories create spanner-loadtest-repo \
141+
--repository-format=docker \
142+
--location=${REGION} \
143+
--description="Docker repository for Spanner Load Test"
144+
145+
export IMAGE_NAME="${REGION}-docker.pkg.dev/${PROJECT_ID}/spanner-loadtest-repo/spanner-loadtest:latest"
146+
147+
gcloud builds submit --tag ${IMAGE_NAME}
148+
149+
gcloud run jobs create spanner-loadtest \
150+
--image ${IMAGE_NAME} \
151+
--region ${REGION} \
152+
--cpu=4 \
153+
--memory=8Gi \
154+
--task-timeout=1h \
155+
--args="-p,${PROJECT_ID},-i,${SPANNER_INSTANCE},-d,${SPANNER_DATABASE},-c,4,-s,MIXED,--duration,600"
156+
157+
gcloud run jobs execute spanner-loadtest --region ${REGION} --tasks 50
158+
```
159+
160+
## Rebuild after changes
161+
```
162+
gcloud builds submit --tag ${IMAGE_NAME}
163+
gcloud run jobs execute spanner-loadtest --region ${REGION} --tasks 2
164+
```

‎spanner-tools/spanner_cdc_loadtest/dependency-reduced-pom.xml‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
3+
<modelVersion>4.0.0</modelVersion>
4+
<groupId>com.google.spanner.cdc</groupId>
5+
<artifactId>spanner-cdc-loadtest</artifactId>
6+
<version>1.0-SNAPSHOT</version>
7+
<build>
8+
<plugins>
9+
<plugin>
10+
<artifactId>maven-shade-plugin</artifactId>
11+
<version>3.5.1</version>
12+
<executions>
13+
<execution>
14+
<phase>package</phase>
15+
<goals>
16+
<goal>shade</goal>
17+
</goals>
18+
<configuration>
19+
<filters>
20+
<filter>
21+
<artifact>*:*</artifact>
22+
<excludes>
23+
<exclude>META-INF/*.SF</exclude>
24+
<exclude>META-INF/*.DSA</exclude>
25+
<exclude>META-INF/*.RSA</exclude>
26+
</excludes>
27+
</filter>
28+
</filters>
29+
<transformers>
30+
<transformer>
31+
<mainClass>com.google.spanner.cdc.loadtest.Main</mainClass>
32+
</transformer>
33+
<transformer />
34+
</transformers>
35+
</configuration>
36+
</execution>
37+
</executions>
38+
</plugin>
39+
</plugins>
40+
</build>
41+
<dependencies>
42+
<dependency>
43+
<groupId>junit</groupId>
44+
<artifactId>junit</artifactId>
45+
<version>4.13.2</version>
46+
<scope>test</scope>
47+
<exclusions>
48+
<exclusion>
49+
<artifactId>hamcrest-core</artifactId>
50+
<groupId>org.hamcrest</groupId>
51+
</exclusion>
52+
</exclusions>
53+
</dependency>
54+
</dependencies>
55+
<dependencyManagement>
56+
<dependencies>
57+
<dependency>
58+
<groupId>com.google.cloud</groupId>
59+
<artifactId>libraries-bom</artifactId>
60+
<version>26.32.0</version>
61+
<type>pom</type>
62+
<scope>import</scope>
63+
</dependency>
64+
</dependencies>
65+
</dependencyManagement>
66+
<properties>
67+
<maven.compiler.target>11</maven.compiler.target>
68+
<maven.compiler.source>11</maven.compiler.source>
69+
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
70+
</properties>
71+
</project>

‎spanner-tools/spanner_cdc_loadtest/launch_jobs.sh‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
#!/bin/bash
2+
3+
# Ensure REGION is set; defaults to us-central1 if empty
4+
REGION="${REGION:-us-central1}"
5+
6+
echo "Launching 8 instances of the Spanner load test in ${REGION}..."
7+
8+
for i in {1..80}
9+
do
10+
echo "Starting job execution #$i..."
11+
gcloud run jobs execute spanner-loadtest --region "${REGION}" --tasks 100 &
12+
done
13+
14+
# Wait for all background processes to finish (optional)
15+
wait
16+
17+
echo "All jobs have been triggered."

‎spanner-tools/spanner_cdc_loadtest/pom.xml‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<project xmlns="http://maven.apache.org/POM/4.0.0"
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
5+
<modelVersion>4.0.0</modelVersion>
6+
7+
<groupId>com.google.spanner.cdc</groupId>
8+
<artifactId>spanner-cdc-loadtest</artifactId>
9+
<version>1.0-SNAPSHOT</version>
10+
11+
<properties>
12+
<maven.compiler.source>11</maven.compiler.source>
13+
<maven.compiler.target>11</maven.compiler.target>
14+
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
15+
</properties>
16+
17+
<dependencyManagement>
18+
<dependencies>
19+
<dependency>
20+
<groupId>com.google.cloud</groupId>
21+
<artifactId>libraries-bom</artifactId>
22+
<version>26.32.0</version>
23+
<type>pom</type>
24+
<scope>import</scope>
25+
</dependency>
26+
</dependencies>
27+
</dependencyManagement>
28+
29+
<dependencies>
30+
<!-- Google Cloud Spanner -->
31+
<dependency>
32+
<groupId>com.google.cloud</groupId>
33+
<artifactId>google-cloud-spanner</artifactId>
34+
</dependency>
35+
36+
<!-- Command Line Argument Parsing -->
37+
<dependency>
38+
<groupId>info.picocli</groupId>
39+
<artifactId>picocli</artifactId>
40+
<version>4.7.5</version>
41+
</dependency>
42+
43+
<!-- Utilities -->
44+
<dependency>
45+
<groupId>org.apache.commons</groupId>
46+
<artifactId>commons-lang3</artifactId>
47+
<version>3.14.0</version>
48+
</dependency>
49+
<dependency>
50+
<groupId>com.google.guava</groupId>
51+
<artifactId>guava</artifactId>
52+
</dependency>
53+
54+
<!-- Logging -->
55+
<dependency>
56+
<groupId>org.slf4j</groupId>
57+
<artifactId>slf4j-api</artifactId>
58+
<version>2.0.9</version>
59+
</dependency>
60+
<dependency>
61+
<groupId>org.slf4j</groupId>
62+
<artifactId>slf4j-simple</artifactId>
63+
<version>2.0.9</version>
64+
</dependency>
65+
66+
<!-- Testing -->
67+
<dependency>
68+
<groupId>junit</groupId>
69+
<artifactId>junit</artifactId>
70+
<version>4.13.2</version>
71+
<scope>test</scope>
72+
</dependency>
73+
</dependencies>
74+
75+
<build>
76+
<plugins>
77+
<plugin>
78+
<groupId>org.apache.maven.plugins</groupId>
79+
<artifactId>maven-shade-plugin</artifactId>
80+
<version>3.5.1</version>
81+
<executions>
82+
<execution>
83+
<phase>package</phase>
84+
<goals>
85+
<goal>shade</goal>
86+
</goals>
87+
<configuration>
88+
<filters>
89+
<filter>
90+
<artifact>*:*</artifact>
91+
<excludes>
92+
<exclude>META-INF/*.SF</exclude>
93+
<exclude>META-INF/*.DSA</exclude>
94+
<exclude>META-INF/*.RSA</exclude>
95+
</excludes>
96+
</filter>
97+
</filters>
98+
<transformers>
99+
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
100+
<mainClass>com.google.spanner.cdc.loadtest.Main</mainClass>
101+
</transformer>
102+
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
103+
</transformers>
104+
</configuration>
105+
</execution>
106+
</executions>
107+
</plugin>
108+
</plugins>
109+
</build>
110+
</project>

0 commit comments

Comments
 (0)