I had a few website problems yesterday – not on this website, but on one belonging to a client.
Like my site, this particular client is making use of CloudFlare for content delivery and rapidly-resolved DNS. Unfortunately, this site has had some issues with the service that raised a particularly sticky business issue.
If you’re responding to a customer, you must be aware of context.
The Problem
Due to a few timing issue with the site, we’d configured DNS prematurely within CloudFlare and the actual domain DNS lagged a bit behind. By the time the domain DNS was actually set, CloudFlare had already wiped out our initial settings and reset itself.[ref]I don’t fault CloudFlare one bit for this. It took us two weeks to actually configure DNS on the system, so seeing our settings disappear yesterday, while a major inconvenience, was not entirely a shock.[/ref]
This means I had to set the domain up again. No big deal, except we’d already pointed our nameservers at CloudFlare, so when the system scanned existing DNS records to pull them to the new system, it got stuck in a loop. CloudFlare was scanning nonexistent DNS records, and ported over an entirely blank ruleset.
As a result, the site went down.
I jumped on the issue and immediately tried to reconcile the server’s configuration and get things back up and running. Then CloudFlare itself went down.
Ironically, CloudFlare presented its own “can’t talk to the server” error message – CloudFlare couldn’t talk to CloudFlare. Frustrated, I grabbed a screen capture and posted it to Twitter.
https://twitter.com/EricMann/status/514520685725511680
The Real Problem
My issue here wasn’t just that a client site was down – as the network resolved itself later we were able to port all the necessary DNS records and reestablish a stable system. My issue was the response from customer service.
As CloudFlare’s site was down, I had:
- No way to check on system status. CloudFlare’s status page is hosted on www.cloudflare.com, which was giving me an error.
- No way to submit a support request. I was logged in the administrative dashboard – where I would submit support tickets from in the first place – when the system started kicking out errors.
My screenshot above isn’t of my site being down. It’s of CloudFlare’s administrative panel being down and rendering the service inaccessible to both me and my client.
When CloudFlare representatives attempted to help later in the Twitter conversation, I was beside myself …
@EricMann That doesn't sound right. We can take a deeper look with a support ticket. Can you please create one?
— Cloudflare (@Cloudflare) September 23, 2014
I started the thread on Twitter because the administrative system was down. Our issues managing the website originated from the system being down. I couldn’t submit a support request and turned to Twitter because the system was down.
The response: submit a support request through the system that’s been going down.
If you’re going to respond to customer support requests in any medium, you absolutely must be aware of the context within which you’re responding. If you’re commenting on an existing thread, read all the way to the beginning of the thread to respond. If you’re the second level of support on a call, read back through your predecessors’ notes before asking the customer to explain their situation yet again.
Context is key to being able to understand and resolve any conflict or situation. Without context, you’re wasting both our time.