Simple proxy to synchronize a GCS file with an HAProxy .map file

This small Golang server exposes a route that allows synchronizing a GCS (Google Cloud Storage) file with an HAProxy .map file using the Dataplane API.

Use cases

This proxy can be used for various purposes:

1. Implement distributed rate limiting across multiple HAProxy instances without the need for an enterprise license.
2. Apply access rules (allowlisting/denylisting) across all machines in one or more environments.
3. Ensure the reliable updating of Map files.

Demo

https://github.com/matthisholleville/mapSyncProxy/assets/99146727/63d140d9-c7ce-4d55-a460-7fe9dc4d96e0

Local development

1. Requirements

• Docker
• Make
• Python3
• Gsutil
• Go

2. Configuration de HAProxy

To build and run a Docker container with the Dataplane API, execute the following command:

make setup

To verify that the container is running correctly, open a browser and navigate to the following URL: http://127.0.0.1:5555/v2/docs. You should see the Dataplane API documentation.

3. Starting the server

To start the server, execute the following command:

make run

You should see a list of metrics at the following URL: http://localhost:8000/metrics.

4. Pushing the Local File to a GCS Bucket

Before pushing the file to a bucket, ensure that it is accessible from your script execution context. To initiate the file push, execute the following command:

make push

5. Initiating Synchronization

To start synchronization, execute the following command:

make synchronize map_name=rate-limits bucket=$MY_BUCKET_NAME bucket_file_name=gcs.json

You can reconcile all files in a bucket by specifying "*" in the bucket_file_name field. The proxy will take care of downloading all the json files and reconciling (merging) them.

If everything is successful, you should see the following message:

...
{"status":"synchronization success."}

Swagger UI is accessible at http://localhost:8080/swagger/index.html.