Observability
Prometheus metrics, structured access logging, and OpenTelemetry trace propagation.
Bastion provides comprehensive observability through three pillars: Prometheus-compatible metrics, structured access logging, and OpenTelemetry trace propagation. Together they give full visibility into gateway traffic, upstream health, and request flows.
Prometheus Metrics
Configuration
bastion.WithMetrics(bastion.MetricsConfig{
Enabled: true,
Prefix: "gateway",
})| Field | Type | Default | Description |
|---|---|---|---|
Enabled | bool | true | Enable or disable metrics collection |
Prefix | string | "gateway" | Prefix for all metric names |
Exported Metrics
The gateway exports the following metrics through the Forge metrics system:
Request Metrics
| Metric | Type | Description |
|---|---|---|
gateway.requests_total | Counter | Total number of proxied requests |
gateway.request_duration_seconds | Histogram | Request latency distribution |
gateway.active_connections.<protocol> | Gauge | Active connections by protocol (http, websocket, sse) |
Resilience Metrics
| Metric | Type | Description |
|---|---|---|
gateway.retry_total | Counter | Total retry attempts |
gateway.rate_limited_total | Counter | Total rate-limited requests |
gateway.circuit_breaker_state.<targetID> | Gauge | Circuit breaker state (0=closed, 1=half-open, 2=open) |
Upstream Metrics
| Metric | Type | Description |
|---|---|---|
gateway.upstream_health.<targetID> | Gauge | Target health (1.0=healthy, 0.0=unhealthy) |
gateway.discovery_routes.<source> | Gauge | Number of routes by source (manual, farp, discovery) |
Cache Metrics
| Metric | Type | Description |
|---|---|---|
gateway.cache_hits_total | Counter | Total cache hits |
gateway.cache_misses_total | Counter | Total cache misses |
Custom Prefix
Change the metric prefix to avoid collisions when running multiple gateways:
bastion.WithMetrics(bastion.MetricsConfig{
Enabled: true,
Prefix: "api_gateway",
})
// Metrics: api_gateway.requests_total, api_gateway.request_duration_seconds, etc.Structured Access Logging
Configuration
bastion.WithAccessLog(bastion.AccessLogConfig{
Enabled: true,
RedactHeaders: []string{"Authorization", "Cookie", "Set-Cookie"},
IncludeBody: false,
MaxBodyLogSize: 4096,
})| Field | Type | Default | Description |
|---|---|---|---|
Enabled | bool | true | Enable or disable access logging |
RedactHeaders | []string | ["Authorization", "Cookie", "Set-Cookie"] | Headers to redact in logs |
IncludeBody | bool | false | Include request/response body in logs |
MaxBodyLogSize | int | 4096 | Maximum body bytes to log |
Log Fields
Every proxied request produces a structured log entry with these fields:
Request Fields
| Field | Description |
|---|---|
method | HTTP method (GET, POST, etc.) |
path | Request path |
status | HTTP response status code |
latency_ms | Total request latency in milliseconds |
client_ip | Client IP address (from X-Forwarded-For, X-Real-IP, or RemoteAddr) |
user_agent | Client User-Agent header |
request_id | Value of X-Request-ID header (if present) |
query | Query string (truncated to 200 chars) |
Route Fields
| Field | Description |
|---|---|
route_id | Matched route ID |
route_path | Route pattern |
route_protocol | Route protocol (http, websocket, sse, grpc) |
route_source | Route source (manual, farp, discovery) |
Upstream Fields
| Field | Description |
|---|---|
upstream_url | Selected target URL |
upstream_id | Selected target ID |
Safe Headers
These headers are always included when present:
Accept,Content-Type,Content-Length,Referer,Origin
Sensitive headers listed in RedactHeaders are logged as [REDACTED].
Log Levels
The access logger uses different log levels based on the response status code:
| Status Range | Log Level |
|---|---|
| 200-399 | Info |
| 400-499 | Warn |
| 500+ | Error |
Admin Action Logging
Administrative actions (route creation, config changes, cache invalidation) are logged separately:
{
"msg": "gateway admin action",
"action": "route_created",
"resource": "route-abc123",
"result": "success",
"client_ip": "10.0.0.1",
"user_agent": "curl/7.88.0"
}OpenTelemetry Trace Propagation
Configuration
bastion.WithTracing(bastion.TracingConfig{
Enabled: true,
PropagateFormat: "w3c",
SampleRate: 1.0,
})| Field | Type | Default | Description |
|---|---|---|---|
Enabled | bool | true | Enable or disable trace propagation |
PropagateFormat | string | "w3c" | Trace context format to propagate |
SampleRate | float64 | 1.0 | Sampling rate (0.0 to 1.0) |
Trace Context Propagation
When tracing is enabled, the gateway propagates trace context headers between the client and upstream:
- W3C Trace Context (
"w3c"): Propagatestraceparentandtracestateheaders.
The gateway acts as a pass-through for trace context -- it does not create new spans but ensures that distributed tracing works end-to-end across the gateway boundary.
Sample Rate
The SampleRate controls what fraction of requests participate in tracing:
| Value | Behavior |
|---|---|
1.0 | All requests are traced |
0.5 | 50% of requests are traced |
0.1 | 10% of requests are traced |
0.0 | Tracing is effectively disabled |
Gateway Stats
The proxy engine maintains an aggregated stats structure available via the admin API:
type GatewayStats struct {
TotalRequests int64
TotalErrors int64
ActiveConns int64
ActiveWSConns int64
ActiveSSEConns int64
AvgLatencyMs float64
P99LatencyMs float64
RequestsPerSec float64
CacheHits int64
CacheMisses int64
RateLimited int64
CircuitBreaks int64
RetryAttempts int64
TotalRoutes int
HealthyUpstreams int
TotalUpstreams int
Uptime int64
StartedAt time.Time
}Access via the admin API:
GET /gateway/api/statsPer-Route Stats
Each route tracks its own metrics:
type RouteStats struct {
RouteID string
Path string
TotalRequests int64
TotalErrors int64
AvgLatencyMs float64
P99LatencyMs float64
CacheHits int64
CacheMisses int64
RateLimited int64
}Per-route stats are included in GatewayStats.RouteStats keyed by route ID.
Dashboard Integration
When the dashboard is enabled, all observability data feeds the real-time admin UI:
bastion.WithDashboard(bastion.DashboardConfig{
Enabled: true,
BasePath: "/gateway",
Realtime: true,
})The dashboard displays:
- Live request throughput and error rates
- Latency percentiles (p50, p95, p99)
- Active connections by protocol
- Upstream health status per target
- Circuit breaker states
- Cache hit/miss ratios
- Route-level traffic breakdown
Audit Logging
Administrative actions are recorded as audit events:
type AuditEvent struct {
Action AuditAction
Resource string
Actor string
Timestamp time.Time
Details map[string]any
}Supported audit actions:
| Action | Description |
|---|---|
AuditRouteCreated | A route was created |
AuditRouteUpdated | A route was updated |
AuditRouteDeleted | A route was deleted |
AuditRouteToggled | A route was enabled/disabled |
AuditConfigChanged | Gateway configuration was changed |
AuditDiscoveryForced | Discovery refresh was manually triggered |
AuditCircuitReset | A circuit breaker was manually reset |
AuditCacheCleared | Cache was cleared |
Audit events can be consumed via the AuditSink interface for forwarding to external audit systems.