loading…
A key client says your SDK is broken. Their integration worked last week, now it does not, and they insist the problem is on your end. You cannot reproduce it locally. What do you do before escalating to Engineering?
Do not argue about ownership. Acknowledge the regression, establish the business impact, and give a concrete next-update time. Ask whether production is blocked, which operations fail, how many users are affected, and whether a safe workaround exists.
Capture the last known success and first known failure with timestamps and timezone. Compare that window against:
“We changed nothing” is a hypothesis. Automatic dependency updates, expiring credentials, certificates, and server-side changes can alter behavior without an application commit.
Request the exact SDK and runtime versions, complete error class and stack trace, sanitized minimal code, endpoint and status, timestamp, request or correlation ID, and one last-known-good request. Never ask the customer to send secrets in logs or chat.
Internally, slice errors by customer, SDK version, endpoint, region, status code, and deployment. Search for correlated support cases and recent changes.
| Experiment | What it tells you |
|---|---|
| Raw HTTP succeeds; SDK fails | SDK serialization, defaults, dependency, or runtime boundary |
| Old SDK succeeds; new SDK fails | Version regression or changed contract |
| Same code succeeds outside customer network | Proxy, DNS, TLS, firewall, or egress policy |
| New credential succeeds | Token, scope, expiry, or account configuration |
| Other customers show the same signature | Shared product or infrastructure regression |
Change one variable at a time and write down the result. Build the smallest reproducible request without customer secrets or unnecessary application code.
Offer only safe, reversible workarounds: pin the last known good version, use the documented raw API temporarily, disable an optional feature, or route around a degraded endpoint. Do not recommend disabling TLS validation, sharing credentials, or retrying unsafe operations indefinitely.
Escalate when you have a reproduction, evidence pointing to your boundary, or customer impact high enough to justify parallel Engineering involvement. Include:
I would not make “I cannot reproduce it” the customer's problem. I would establish impact, reconstruct the change window, collect correlatable evidence, isolate one system boundary at a time, provide a safe workaround, and escalate with an incident packet Engineering can act on immediately.
Write the first customer update and the Engineering escalation ticket. The customer message should communicate ownership and cadence without guessing the cause. The ticket should let an engineer begin investigation without repeating discovery.