ex module - github.com/circleci/ex

README ¶

Go service building blocks

This repository contains a collection of packages designed to aid building reliable, observable, zero-downtime services using Go.

Documentation

Browse the docs at pkg.go.dev. See example / template repo here.

What is in here?

config/o11y Wiring code for the o11y package.
config/secret Don't store your secrets in a string that can be accidentally logged. Use a secret.String instead.
datadog A basic Datadog API client for retrieving metrics back from Datadog.
db Common patterns using when talking to an RDBMS. Only supports PostgreSQL at present.
httpclient A simple HTTP client that adds observability and resilience to the standard Go HTTP client.
httpclient/dnscache A simple DNS cache for use with the HTTP client.
httpserver Starting and stopping the standard Go http server cleanly.
httpserver/ginrouter A common base for configuring a Gin router instance.
httpserver/healthcheck A healthcheck HTTP server that can accept all the checks from a system.
mongoex Experimental Common patterns using when talking to MongoDB.
o11y Observability that is currently backed by Otel. It also supports outputting trace data as JSON and plain or colored text output.
o11y/wrappers/o11ygin o11y middleware for the Gin router.
o11y/wrappers/o11ynethttp o11y middleware for the standard Go HTTP server.
rabbit Experimental RabbitMQ publishing client.
redis Wiring and observability for Redis.
system Manage the startup, running, metrics and shutdown of a Go service.
releases/compiler Compile your Go binaries in a consistent way.
releases/releaser Release your Go binaries in a consistent way.
rundef automatically calculates and sets standard runtime configuration options
termination A handler to aid signal based service termination. (Used internally by the system package).
testing/compiler At CircleCI we aim to acceptance test whole compiled binaries. This package lets us do that.
testing/dbfixture Get a resettable unique database for each test.
testing/download Download releases of binaries for using in end to end service testing.
testing/fakemetrics A fake recording o11y metrics implementation.
testing/fakestatsd A recording statsd server that will listen on a local UDP port.
testing/httprecorder Record HTTP requests inside an HTTP server, and search them.
testing/kongtest If you are using kong for your CLI parsing, this helps in writing golden tests for the CLI definition.
testing/miniofixture Get a temporary bucket to put your objects in.
testing/mongofixture Get an isolated Mongo database for your tests, so they don't interfere.
testing/rabbitfixture Get an isolated RabbitMQ VHost for your tests, so they don't interfere.
testing/redisfixture Get an isolated Redis DB for your tests, so they don't interfere.
testing/releases Helper to determine which binaries to download for end to end tests.
testing/runner Run a binary in an acceptance test (scan output for ports, wait for start).
testing/testcontext Setup a background context that includes o11y.
worker Run a service worker loop with observability and back-off for no work found.

Who is this for?

These packages are intended to be used internally at CircleCI. While this code is licenced permissively with an MIT licence, this is not a true "open source" project, in that there is no active community using it.

No guaranteed API stability

While we do not intentionally aim to break compatibility, we make no promises that we will maintain a stable API if there are good reasons to break it.

Directories ¶

Path	Synopsis
closer Package closer contains a helper function for not losing deferred errors	Package closer contains a helper function for not losing deferred errors
colourise
config
o11y Package o11y is the primary entrypoint for wiring up a standard configuration of the o11y observability system.	Package o11y is the primary entrypoint for wiring up a standard configuration of the o11y observability system.
secret Package secret contains a struct to store secrets in that won't accidentally let them be logged.	Package secret contains a struct to store secrets in that won't accidentally let them be logged.
datadog Package datadog contains a basic Datadog API client for retrieving metrics back from Datadog.	Package datadog contains a basic Datadog API client for retrieving metrics back from Datadog.
db Package db contains a variety of tools for working safely with databases.	Package db contains a variety of tools for working safely with databases.
example module
grpc Package grpchelpers includes helpers to set up grpc connections.	Package grpchelpers includes helpers to set up grpc connections.
internal/testgrpc
httpclient Package httpclient provides an HTTP client instrumented with the ex/o11y package, it includes resiliency behaviour such as configurable timeouts, retries, authentication and connection pooling, with support for backing off when a 429 response code is seen.	Package httpclient provides an HTTP client instrumented with the ex/o11y package, it includes resiliency behaviour such as configurable timeouts, retries, authentication and connection pooling, with support for backing off when a 429 response code is seen.
dnscache Package dnscache contains a simple in-process cache for DNS lookups	Package dnscache contains a simple in-process cache for DNS lookups
metrics
httpserver Package httpserver contains helpers for creating zero-downtime REST APIs.	Package httpserver contains helpers for creating zero-downtime REST APIs.
ginrouter Package ginrouter provides a common base for configuring a Gin router instance, wiring in the standard o11y wrappers.	Package ginrouter provides a common base for configuring a Gin router instance, wiring in the standard o11y wrappers.
healthcheck Package healthcheck contains a simple healthcheck handler.	Package healthcheck contains a simple healthcheck handler.
internal
syncbuffer
kong
vault Package vault provides a Kong CLI Resolver that will provide default flag values from Vault	Package vault provides a Kong CLI Resolver that will provide default flag values from Vault
o11y Package o11y core of the ex observability system, currently backed by Honeycomb's libhoney.	Package o11y core of the ex observability system, currently backed by Honeycomb's libhoney.
httpmetrics
otel Package otel contains an o11y.Provider that emits open telemetry gRPC.	Package otel contains an o11y.Provider that emits open telemetry gRPC.
otel/texttrace Package texttrace is a span exporter for otel that outputs to the ex text console format	Package texttrace is a span exporter for otel that outputs to the ex text console format
semconv
wrappers/baggage Package baggage contains the internal implementation of the o11y baggage.	Package baggage contains the internal implementation of the o11y baggage.
wrappers/o11ygin Package o11ygin contains middleware for the Gin router.	Package o11ygin contains middleware for the Gin router.
wrappers/o11ynethttp Package o11ynethttp contains middleware for the standard Go HTTP server.	Package o11ynethttp contains middleware for the standard Go HTTP server.
plugin
releaser
releaser/internal/cmd command
rabbit Package rabbit contains an experimental RabbitMQ publishing client.	Package rabbit contains an experimental RabbitMQ publishing client.
recontext Package recontext provides means of obtaining a derived context which ignores parent's deadline, timeout, and cancellation.	Package recontext provides means of obtaining a derived context which ignores parent's deadline, timeout, and cancellation.
redis Package redis contains wiring and observability for the go-redis Redis client.	Package redis contains wiring and observability for the go-redis Redis client.
releases
compiler Package compiler aids with compiling your Go binaries efficiently and in a consistent way.	Package compiler aids with compiling your Go binaries efficiently and in a consistent way.
compiler/internal/cmd command
compiler/internal/cmd2 command
compiler/internal/cmd3 command
download Package download helps download releases of binaries.	Package download helps download releases of binaries.
release Package release works with the metadata of execution releases.	Package release works with the metadata of execution releases.
releaser Package releaser aids with publishing your Go binaries efficiently and in a consistent way.	Package releaser aids with publishing your Go binaries efficiently and in a consistent way.
rootcerts Package rootcerts exists to support creating Docker images `FROM scratch`.	Package rootcerts exists to support creating Docker images `FROM scratch`.
rundef Package rundef provides default runtime configurations for services.	Package rundef provides default runtime configurations for services.
system Package system manages the startup, running, metrics and shutdown of a Go service.	Package system manages the startup, running, metrics and shutdown of a Go service.
termination Package termination contains a simple handler for termination signals.	Package termination contains a simple handler for termination signals.
testing Package testing contains various helpers in sub packages for writing high value tests.	Package testing contains various helpers in sub packages for writing high value tests.
cmpextra
compiler Package compiler helps efficiently compile and cleanup your services in acceptance tests.	Package compiler helps efficiently compile and cleanup your services in acceptance tests.
compiler/internal/cmd command
compiler/internal/cmd2 command
dbfixture Package dbfixture will setup a and reset a fresh database for each of your tests.	Package dbfixture will setup a and reset a fresh database for each of your tests.
fakemetricrec
fakemetrics Package fakemetrics provides an implementation of the o11y metrics provider that will record all metrics you send it.	Package fakemetrics provides an implementation of the o11y metrics provider that will record all metrics you send it.
fakestatsd Package fakestatsd provides a recording statsd server that will listen on a local UDP port.	Package fakestatsd provides a recording statsd server that will listen on a local UDP port.
httprecorder Package httprecorder provides a simple way of recording and later retrieving all requests that are sent to an HTTP handler.	Package httprecorder provides a simple way of recording and later retrieving all requests that are sent to an HTTP handler.
httprecorder/ginrecorder Package ginrecorder provides a middleware to wire a httprecorder into Gin routers used in test fakes.	Package ginrecorder provides a middleware to wire a httprecorder into Gin routers used in test fakes.
httprecorder/httpnetrecorder Package httpnetrecorder provides a middleware to wire a httprecorder into test fakes.	Package httpnetrecorder provides a middleware to wire a httprecorder into test fakes.
internal/types Package types contains various interfaces for writing tests.	Package types contains various interfaces for writing tests.
jaeger
kongtest Package kongtest helps write golden tests for your [Kong](https://github.com/alecthomas/kong) CLI parsing.	Package kongtest helps write golden tests for your [Kong](https://github.com/alecthomas/kong) CLI parsing.
miniofixture Package miniofixture is deprecated: use testing/s3fixture instead.	Package miniofixture is deprecated: use testing/s3fixture instead.
rabbitfixture Package rabbitfixture will setup isolated RabbitMQ VHost for your tests, so they don't interfere.	Package rabbitfixture will setup isolated RabbitMQ VHost for your tests, so they don't interfere.
rabbitfixture/internal/rabbit
redisfixture Package redisfixture will setup an isolated Redis DB for your tests, so they don't interfere.	Package redisfixture will setup an isolated Redis DB for your tests, so they don't interfere.
releases Package releases is a helper to determine which binaries to download for end to end tests.	Package releases is a helper to determine which binaries to download for end to end tests.
runner Package runner allows you to run a binary in an acceptance test (scan output for ports, wait for start).	Package runner allows you to run a binary in an acceptance test (scan output for ports, wait for start).
runner/internal/testservice command
s3fixture Package s3fixture provides functions to create a test-scoped (temporary) bucket backed by S3.	Package s3fixture provides functions to create a test-scoped (temporary) bucket backed by S3.
testcontext Package testcontext provides a context with o11y wired in, setup for coloured logging and no-op metrics.	Package testcontext provides a context with o11y wired in, setup for coloured logging and no-op metrics.
testcontextotel Package testcontextotel provides a context with o11y wired in, setup for coloured logging and no-op metrics.	Package testcontextotel provides a context with o11y wired in, setup for coloured logging and no-op metrics.
testrand
worker Package worker runs a service worker loop with observability and back-off for no work found.	Package worker runs a service worker loop with observability and back-off for no work found.