When your gRPC client logs rpc error: code = Unavailable desc = ..., the server is not reachable right now — but the error doesn't mean your code is broken. gRPC status 14 (UNAVAILABLE) is a transient condition: the service was up a moment ago or will be soon, and a retry with backoff often succeeds. Understanding what UNAVAILABLE actually means and why it happens is the first step to building reliable clients that don't amplify outages.
In this guide, we'll walk through the causes — from cold starts to name resolution failures — show how to read the full error detail in your logs, and cover the retry strategy that gRPC itself recommends.
What UNAVAILABLE (Code 14) means
According to the official gRPC specification, UNAVAILABLE means the service is currently unavailable. It is transient: the condition is expected to resolve. Clients should retry with exponential backoff. This is different from, say, UNIMPLEMENTED (the method doesn't exist, ever) or INVALID_ARGUMENT (your request is malformed) — both permanent failures that retrying won't fix.
The key contract: UNAVAILABLE is not your fault, and waiting a moment usually solves it.
Real causes and fixes
Server not started or cold start
The problem: The gRPC service simply isn't running yet. In containerized deployments, a pod may have started but the app is still booting; the port isn't listening.
The fix:
- Ensure the gRPC server listener is bound before returning from startup. In Go, don't call
grpc.NewServer()and then lose the reference; keep it alive and serve. - Add health checks so the container orchestrator (Kubernetes, ECS, etc.) doesn't route traffic until the service is ready. Use the gRPC health check protocol:
import "google.golang.org/grpc/health"
import "google.golang.org/grpc/health/grpc_health_v1"
srv := grpc.NewServer()
healthSrv := health.NewServer()
grpc_health_v1.RegisterHealthServer(srv, healthSrv)
healthSrv.SetServingStatus("myservice.MyService", grpc_health_v1.HealthCheckResponse_SERVING)
srv.Serve(lis)
Connection refused: wrong address or port
The problem: The client is pointed at the wrong hostname, port, or IP. The OS returns ECONNREFUSED because nothing is listening on that address.
The fix:
- Double-check
ServiceConfigin your client code or environment variables. Typos are common:localhost:5051vs.localhost:5050, orservice.localvs.service.example.com. - In Kubernetes, verify the Service DNS name:
service-name.namespace.svc.cluster.local. - Test connectivity from the client pod:
grpcurl -plaintext service:5050 listornc -zv service 5050.
Name resolution failure
The problem: The hostname can't be resolved to an IP. This happens when DNS is unreachable, the name doesn't exist, or the pod's DNS configuration is wrong.
The fix:
- Check
/etc/resolv.confinside your pod; it should point to the cluster's DNS (usually10.96.0.10on Kubernetes). - Verify the domain exists:
nslookup service.namespace.svc.cluster.localfrom inside a pod. - If using an external hostname, ensure your proxy or NAT isn't blocking DNS queries.
Load balancer draining or no ready endpoint
The problem: The load balancer (cloud LB, Kubernetes service, or a proxy) has no healthy backend ready. All endpoints are draining, unhealthy, or still starting.
The fix:
- Ensure replicas are running:
kubectl get pods -l app=myservice. - Check pod readiness probes:
kubectl describe pod <pod>and look for "Ready" status. - If you're using circuit breaker or drain patterns, confirm the readiness endpoint is working:
curl http://pod:8080/ready.
TLS handshake failure
The problem: The client and server can't agree on TLS, or the certificate is invalid. The handshake fails before any gRPC frames are sent, causing UNAVAILABLE.
The fix:
// Go example: ensure credentials match between client and server
creds, _ := credentials.NewClientTLSFromFile("cert.pem", "")
conn, _ := grpc.Dial("service:5050", grpc.WithTransportCredentials(creds))
// On the server side:
creds, _ := credentials.NewServerTLSFromFile("cert.pem", "key.pem")
srv := grpc.NewServer(grpc.Creds(creds))
- Verify certificate expiration:
openssl x509 -in cert.pem -text -noout | grep -A2 Validity. - Check that both client and server use the same CA or are both in plaintext mode (
grpc.WithInsecure()on client, no creds on server — but only for development).
Keepalive or idle timeout closing connections
The problem: A persistent connection was idle too long, or the server sent a GOAWAY frame due to a keepalive ping timeout. The client sees UNAVAILABLE because its connection is suddenly broken.
The fix:
- Set keepalive parameters on both client and server. In Go:
// Client
conn, _ := grpc.Dial("service:5050",
grpc.WithKeepaliveParams(keepalive.ClientParameters{
Time: 20 * time.Second, // send ping every 20s
Timeout: 3 * time.Second, // wait 3s for pong
}),
)
// Server
srv := grpc.NewServer(
grpc.KeepaliveParams(keepalive.ServerParameters{
Time: 20 * time.Second,
Timeout: 3 * time.Second,
}),
grpc.KeepaliveEnforcementPolicy(keepalive.EnforcementPolicy{
MinTime: 5 * time.Second,
PermitWithoutStream: true,
}),
)
Server GOAWAY during rolling deploy
The problem: During a canary or rolling restart, the server sends GOAWAY (graceful shutdown) and closes in-flight requests. New requests hit the shutting-down instance and get UNAVAILABLE.
The fix:
- Implement graceful shutdown: stop accepting new requests, wait for in-flight ones to finish, then exit.
- Configure your load balancer to drain connections slowly. On Kubernetes, set
terminationGracePeriodSecondshigh enough for existing streams to finish. - Use connection timeout settings and exponential backoff on the client so it doesn't hammer the draining instance.
Proxy (Envoy, nginx) returning UNAVAILABLE
The problem: An intermediary like Envoy or nginx in the call chain can't reach the backend and returns UNAVAILABLE on behalf of the server.
The fix:
- Check the proxy logs:
kubectl logs -l app=envoy-sidecar | grep UNAVAILABLE. - Verify the upstream cluster in the proxy config points to live endpoints:
curl http://envoy:9901/clusters(Envoy admin API). - Ensure the proxy's health checks are configured correctly and not marking all backends as down.
Retrying on UNAVAILABLE without a backoff is a trap. If the server is unhealthy and can't recover, hammering it with immediate retries only makes things worse. Always pair retries with exponential backoff and jitter; see the next section.
Retry strategy: doing it right
Built-in retry policy via service config
The best way to retry is to let gRPC do it for you, via service configuration. Create a JSON file defining which methods are retryable and the backoff policy:
{
"methodConfig": [
{
"name": [
{
"service": "mypackage.MyService",
"method": "MyMethod"
}
],
"retryPolicy": {
"maxAttempts": 3,
"initialBackoff": "100ms",
"maxBackoff": "1s",
"backoffMultiplier": 2.0,
"retryableStatusCodes": ["UNAVAILABLE", "RESOURCE_EXHAUSTED"]
}
}
]
}
Load it in your client:
import "google.golang.org/grpc"
conn, _ := grpc.Dial("service:5050",
grpc.WithDefaultServiceConfig(serviceConfigJSON),
)
gRPC will automatically retry MyMethod calls that hit UNAVAILABLE, with exponential backoff (100ms, 200ms, 400ms, etc.) capped at 1 second. No manual retry loop needed.
Manual retry with backoff (if service config isn't available)
If you can't use service config, implement retries manually in your client code:
import (
"context"
"time"
)
func callWithRetry(ctx context.Context, client MyServiceClient) (*Response, error) {
var lastErr error
for attempt := 0; attempt < 3; attempt++ {
resp, err := client.MyMethod(ctx, &Request{})
if err == nil {
return resp, nil
}
lastErr = err
if attempt < 2 {
// Exponential backoff with jitter
backoff := time.Duration(math.Pow(2, float64(attempt))) * 100 * time.Millisecond
jitter := time.Duration(rand.Int63n(int64(backoff / 2)))
time.Sleep(backoff + jitter)
}
}
return nil, lastErr
}
Idempotency is critical
Retries only work safely if your method is idempotent: calling it twice with the same input produces the same result as calling it once. If your gRPC method modifies state (e.g., transfers money, creates a record), you must either make it idempotent or ensure the client and server coordinate with an idempotency key.
Reading the error in your logs
Go
import "google.golang.org/grpc/status"
resp, err := client.MyMethod(ctx, &Request{})
if err != nil {
st, _ := status.FromError(err)
fmt.Printf("Code: %v\n", st.Code()) // Code: Unavailable
fmt.Printf("Message: %v\n", st.Message()) // Message: connection refused
// If the server sent rich error details:
for _, detail := range st.Details() {
fmt.Printf("Detail: %v\n", detail)
}
}
Node.js
const { ServiceError } = require("@grpc/grpc-js");
try {
await client.myMethod(request);
} catch (err) {
console.log("Code:", err.code); // Code: 14
console.log("Name:", err.name); // Name: 'Error'
console.log("Message:", err.message); // Message: 14 UNAVAILABLE: connection refused
console.log("Metadata:", err.metadata); // Full gRPC metadata
}
Why this matters for observability
A sea of UNAVAILABLE errors in your logs might be noise (transient blip, pod restart) or a signal of cascading failure. gRPC error monitoring helps you spot the pattern: if UNAVAILABLE spikes when a backend service goes down, you can correlate it with cascading failure across your system. When you drill into a specific error, you can see the stack, the trace ID, and the span waterfall that led to it — telling you whether it's a momentary blip or a sign of deeper trouble.
gRPC status codes cheat sheet: Check out gRPC status codes explained for a quick reference of all 16 status codes and when you'll see each one.
Start tracking errors in minutes
Start tracking gRPC errors and distributed traces across your services. LightTrace captures the full error detail, breadcrumbs, and span waterfalls so you can diagnose UNAVAILABLE in seconds.