Lesson 25 / 26

Graceful Shutdown and Observability

Finish in-flight work, then exit.

Signals, Shutdown, structured logs

Services should stop cleanly when the platform sends SIGTERM (for example during a Kubernetes rollout): stop accepting new connections, let in-flight requests finish with a deadline, close resources, then exit. signal.NotifyContext turns signals into a cancelled context, and http.Server.Shutdown(ctx) drains connections. Use structured logging with log/slog, expose metrics, and enable net/http/pprof on an internal port for profiling.

Graceful shutdown (sketch)

Standard library only; not run here.

ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()

go func() {
	if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
		slog.Error("server failed", "err", err)
	}
}()

<-ctx.Done()                                  // wait for SIGTERM / Ctrl+C
shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
srv.Shutdown(shutdownCtx)                     // finish in-flight requests
slog.Info("stopped cleanly")

Use slog for structured logs

log/slog writes key-value or JSON logs that log systems can search and aggregate.

Quick check: What does http.Server.Shutdown do?

  • Restarts the server
  • Kills all connections immediately
  • Stops accepting new connections and waits for in-flight requests to finish
  • Deletes log files
Answer

Stops accepting new connections and waits for in-flight requests to finish — Graceful draining.