http-server

HTTP server

This example demonstrates how to run an HTTP server that accepts and forwards events to OpenMeter. It enables the operation of a low-latency ingestion point near your services, which also handles retries, back pressure, and buffering (if necessary).

The nice thing about this solution is that you can use it as a drop-in replacement for existing integrations since it's compatible with the OpenMeter Ingest API. This means you can use our existing SDKs; the only thing you need to change is the API endpoint. That is, of course, optional: you can use any client library or payload format you prefer and handle the mapping to the CloudEvents format in a Benthos mapping processor.

Table of Contents

• Prerequisites
• Launch the example
• Checking events
• Cleanup
• Advanced configuration
• Production use

Prerequisites

This example uses Docker and Docker Compose, but you are free to run the components in any other way.

Check out this repository if you want to run the example locally:

git clone https://github.com/openmeterio/benthos-openmeter.git
cd benthos-openmeter/examples/http-server

Create a new .env file and add the details of your OpenMeter instance:

cp .env.dist .env
# edit .env and fill in the details

Tip

Tweak other options in the .env file to change the behavior of the example.

Create a meter in OpenMeter with the following details:

• Event type: api-calls
• Aggregation: SUM
• Value property: $.duration_ms
• Group by (optional):
  • method: $.method
  • path: $.path
  • region: $.region
  • zone: $.zone

Tip

Read more about creating a meter in the general examples README.

Launch the example

Launch the event forwarder:

docker compose up -d

Start tailing the logs:

docker compose logs -f forwarder

Open another terminal and send a test event:

curl -vvv http://127.0.0.1:4196/api/v1/events -H "Content-Type: application/cloudevents+json" -d @seed/event.json

Inspect the logs in the other terminal.

Send a batch of events:

curl -vvv http://127.0.0.1:4196/api/v1/events -H "Content-Type: application/cloudevents-batch+json" -d @seed/batch.json

Inspect the logs in the other terminal.

Start the seeder component to continuously send events to the forwarder:

docker compose --profile seed up -d

Warning

The seeder component sends one event per second to the forwarder.

Note

The seeder sends events to the forwarder from 3 days ago up to the current time. For this reason, events may not immediately appear in the event debugger.

You can modify this behavior by editing the seeder configuration file.

Checking events

Read more in the general examples README.

Cleanup

Stop containers:

docker compose --profile seed down -v

Advanced configuration

For this example to work, Benthos needs to be run in streams mode.

(The reason is that the strong delivery guarantees currently do not permit the HTTP server to reply before events are forwarded to OpenMeter, which may lead to timeouts. To learn more, watch this discussion.)

As a result, this example requires two configuration files:

• input.yaml contains the configuration of the HTTP server and validation of incoming events.
• output.yaml contains the configuration of the OpenMeter output.

Run Benthos with these configuration files:

benthos streams input.yaml output.yaml

Check out the configuration files and the Benthos documentation for more details.

Production use

We are actively working on improving the documentation and the examples. In the meantime, feel free to contact us in email or on Discord.

We are more than happy to help you set up OpenMeter in your production environment.