Comprehensive Monitoring Strategy for Cloud-Native Applications

Observability is crucial for maintaining reliable cloud-native applications. Here's how to implement comprehensive monitoring that provides actionable insights into your system's health and performance.

The Three Pillars of Observability

1. Metrics

Quantitative measurements of system behavior:

var (
    requestDuration = prometheus.NewHistogramVec(
        prometheus.HistogramOpts{
            Name: "http_request_duration_seconds",
            Help: "HTTP request latency distributions.",
            Buckets: prometheus.DefBuckets,
        },
        []string{"method", "endpoint", "status_code"},
    )
    
    requestsTotal = prometheus.NewCounterVec(
        prometheus.CounterOpts{
            Name: "http_requests_total",
            Help: "Total number of HTTP requests.",
        },
        []string{"method", "endpoint", "status_code"},
    )
)

func instrumentHandler(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        
        // Wrap response writer to capture status code
        wrapped := &statusRecorder{ResponseWriter: w, statusCode: 200}
        
        next.ServeHTTP(wrapped, r)
        
        duration := time.Since(start).Seconds()
        labels := []string{r.Method, r.URL.Path, strconv.Itoa(wrapped.statusCode)}
        
        requestDuration.WithLabelValues(labels...).Observe(duration)
        requestsTotal.WithLabelValues(labels...).Inc()
    })
}

2. Distributed Tracing

Track requests across service boundaries:

func TraceHandler(tracer opentracing.Tracer) func(http.Handler) http.Handler {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            span := tracer.StartSpan(r.URL.Path)
            defer span.Finish()
            
            // Add context to request
            ctx := opentracing.ContextWithSpan(r.Context(), span)
            r = r.WithContext(ctx)
            
            // Add tags
            span.SetTag("http.method", r.Method)
            span.SetTag("http.url", r.URL.String())
            
            next.ServeHTTP(w, r)
        })
    }
}

3. Structured Logging

Implement consistent, searchable logging:

func LoggerMiddleware(logger *zap.Logger) func(http.Handler) http.Handler {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            start := time.Now()
            
            next.ServeHTTP(w, r)
            
            logger.Info("HTTP request",
                zap.String("method", r.Method),
                zap.String("path", r.URL.Path),
                zap.Duration("duration", time.Since(start)),
                zap.String("user_agent", r.UserAgent()),
                zap.String("remote_addr", r.RemoteAddr),
            )
        })
    }
}

This monitoring approach has been essential for maintaining high availability and quick issue resolution in production environments.