Handling Third-Party Dependency Failures Gracefully

When your app talks to third-party APIs—payment processors, email services, map data, analytics—you're trading a small piece of control for convenience. The tradeoff is real: those services go down, rate-limit you, or timeout, and your app either fails visibly or limps along. Handling external dependency failures gracefully is one of the most practical investments in availability you can make, and it starts with understanding what's actually happening when a call fails. LightTrace's error tagging makes it easy to surface whether a problem is yours or theirs—a distinction that dramatically changes how you respond.

Most teams don't plan for this until the first outage. By then, users are already refreshing repeatedly, your error rates spike, and you're scrambling to decide: retry? Fallback? Tell the user to wait? With the right approach, none of that happens. You've already designed a response.

Classify errors: internal vs. external

The first rule of graceful external dependency handling is knowing which failure is which. An error from your own code is something you fix. An error from a third-party is something you work around.

LightTrace tags let you separate these at ingestion time. When you catch a call to an external API, tag the error with metadata that makes it unmissable:

import * as Sentry from "@sentry/node";

async function chargeCard(amount, cardToken) {
  try {
    const result = await stripe.charges.create({ amount, source: cardToken });
    return result;
  } catch (error) {
    Sentry.captureException(error, {
      tags: {
        service: "stripe",
        error_origin: "external",
        dependency_type: "payment_processor"
      },
      contexts: {
        dependency: {
          name: "Stripe",
          region: "us-east-1",
          timeout_ms: 5000
        }
      }
    });
    throw error;
  }
}

When you send that error to LightTrace, you immediately know:

• Is this your bug, or is Stripe down?
• How many calls to Stripe are failing right now?
• Did all payment attempts fail, or just 2%?
• How long has this been happening?

This distinction is why error triage becomes straightforward. Your on-call engineer knows to check Stripe's status page, not your logs.

Implement retry logic with exponential backoff

Not all failures are permanent. A timeout might be a momentary glitch. A 429 (rate limit) will resolve when traffic quiets. The simplest recovery is to try again—but not immediately, and not forever.

Exponential backoff spreads retries over increasing time intervals, reducing pressure on a struggling service:

async function callExternalAPI(url, options = {}) {
  const maxRetries = 3;
  const baseDelayMs = 100;
  let lastError;

  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      const response = await fetch(url, {
        timeout: 5000,
        ...options
      });
      if (!response.ok) {
        // 5xx errors are retryable; 4xx errors are not
        if (response.status >= 500) {
          throw new Error(`Server error: ${response.status}`);
        }
        return response; // Client error; don't retry
      }
      return response;
    } catch (error) {
      lastError = error;
      if (attempt < maxRetries) {
        const delayMs = baseDelayMs * Math.pow(2, attempt);
        await new Promise(resolve => setTimeout(resolve, delayMs));
      }
    }
  }

  throw lastError;
}

The key moves:

• Exponential backoff: delay doubles each attempt (100ms, 200ms, 400ms), so the last attempt waits 400ms.
• Idempotency: only retry if it's safe—POST requests need idempotency keys, GET requests are naturally safe.
• Error classification: only retry 5xx errors, not 4xx. A 400 Bad Request won't get better.

Use circuit breakers and fallback data

After a few failed retries, you've learned something: this service is not recovering soon. Continuing to retry will waste time and frustrate users. A circuit breaker stops calling a failing dependency entirely after a threshold, then occasionally tests whether it's back.

class CircuitBreaker {
  constructor(call, options = {}) {
    this.call = call;
    this.failureThreshold = options.failureThreshold || 5;
    this.successThreshold = options.successThreshold || 2;
    this.timeoutMs = options.timeoutMs || 60000; // 1 minute
    this.state = "CLOSED"; // CLOSED (working), OPEN (failing), HALF_OPEN (testing)
    this.failureCount = 0;
    this.successCount = 0;
    this.nextAttemptTime = Date.now();
  }

  async execute() {
    if (this.state === "OPEN") {
      if (Date.now() < this.nextAttemptTime) {
        throw new Error("Circuit breaker is OPEN");
      }
      this.state = "HALF_OPEN"; // Try once
    }

    try {
      const result = await this.call();
      if (this.state === "HALF_OPEN") {
        this.successCount++;
        if (this.successCount >= this.successThreshold) {
          this.state = "CLOSED";
          this.failureCount = 0;
          this.successCount = 0;
        }
      }
      return result;
    } catch (error) {
      this.failureCount++;
      if (this.failureCount >= this.failureThreshold) {
        this.state = "OPEN";
        this.nextAttemptTime = Date.now() + this.timeoutMs;
      }
      throw error;
    }
  }
}

When a circuit breaker opens, your app needs a fallback. That might be:

• Cached data: if you fetched user preferences from an external service yesterday, use yesterday's copy.
• Degraded mode: show a simplified feature instead of full detail.
• Queue for later: store the request and retry when the service recovers.

async function getUserPreferences(userId) {
  const breaker = new CircuitBreaker(
    () => externalPrefsAPI.fetch(userId),
    { failureThreshold: 5, timeoutMs: 30000 }
  );

  try {
    return await breaker.execute();
  } catch (error) {
    Sentry.captureException(error, {
      tags: { fallback_used: "true" }
    });
    // Fallback to cached or default prefs
    return await cache.get(`prefs:${userId}`) || defaultPreferences;
  }
}

The user still gets a working experience, and LightTrace logs that you're in fallback mode—useful for post-incident analysis.

Track dependency health and SLA

To understand whether a particular external service is reliable enough, you need to measure its actual performance. This is where observability becomes essential.

Log the latency and success rate of each external call:

async function trackedExternalCall(dependencyName, fn) {
  const start = Date.now();
  try {
    const result = await fn();
    const duration = Date.now() - start;
    Sentry.captureMessage(`${dependencyName} call succeeded`, {
      level: "info",
      tags: { dependency: dependencyName },
      contexts: {
        performance: {
          duration_ms: duration,
          success: true
        }
      }
    });
    return result;
  } catch (error) {
    const duration = Date.now() - start;
    Sentry.captureException(error, {
      tags: { dependency: dependencyName },
      contexts: {
        performance: {
          duration_ms: duration,
          success: false
        }
      }
    });
    throw error;
  }
}

// Usage
await trackedExternalCall("email_service", () =>
  mailgun.send({ to: "user@example.com", ... })
);

In LightTrace, you can now build queries: "Show me Stripe's 99th percentile latency this week" or "How often does the email service return 5xx errors?" These metrics let you decide: is this service SLA-compliant? Do I need a backup? Should I negotiate better terms, or switch providers?

Build runbooks and alert rules

When a dependency does fail, your team should know immediately. Set up alert rules in LightTrace that trigger when an external error count exceeds your threshold:

• "Stripe errors > 10/minute" → page on-call.
• "Payment processor down for >5 minutes" → escalate.
• "New type of error from Auth0" → notify Slack (once implemented).

Pair alerts with a runbook: a documented sequence of steps to take when this specific dependency fails. Example:

RUNBOOK: Stripe Payment Processor Down

1. Confirm: Check Stripe's status page (https://status.stripe.com).
2. If Stripe is down: Notify users, switch to fallback (queued processing).
3. If Stripe is up but we're failing: Check firewall rules, API key rotation, rate limits.
4. Retry manual charges once Stripe recovers.
5. Post-incident: Review [error logs in LightTrace](/blog/error-tracking-best-practices/), 
   measure impact (lost revenue, affected users), add test to prevent recurrence.

A runbook turns panic into procedure.

Learn and prevent recurrence

After every dependency outage, review the incident in LightTrace. Look at:

• How many users were affected?
• How long did it take to detect?
• How long did the fallback handle it?
• What logged information would have helped you respond faster?

Use this data to improve. Maybe you need:

• A stricter retry budget (fewer retries to reach fallback faster).
• Better instrumentation (log more context about the external call).
• More aggressive circuit breaker thresholds.
• A different fallback strategy (pre-compute data, not cache it).

This feeds into reducing mean time to recovery (MTTR), which compounds over time. Your first outage costs an hour. Your tenth costs five minutes because you've learned.

The goal isn't to prevent every external failure—you can't. It's to make your app resilient despite them, and to learn faster each time one happens.