Imagine lining up before sunrise on a cold November morning the day after thanksgiving, waiting in line to buy up to hundreds of dollars in discounted goods – and then being informed that the retailer couldn’t accept your credit card. Would you shop there again?

This nightmare scenario is what happened to Macy’s shoppers on November 24th, 2017, aka Black Friday. Around noon, overcapacity issues shut down the retail giant’s payment processing systems, turning them into a cash-only enterprise for up to six hours on one of the busiest shopping days of the year. The blow to company revenue may be calculable, but the damage to the company’s reputation is not. How can companies avoid these expensive and embarrassing disasters?

Glitches Are More Common Than You Think

Retailers are plagued by glitches, big and small. Even Amazon, probably the most technologically sophisticated ecommerce platform on Earth, is not immune to the occasional error (Free Echo Dot’s anyone?). So how are smaller and less tech-savvy retailers expected to cope? The glitches they encounter could include:

  • High-volume traffic overloading payment processing servers
  • Orders in online shopping carts failing to complete
  • Incorrect pricing for goods sold online or in stores
  • Mistargeted online advertising
  • Missing opportunities to upsell or cross-sell customers based on faulty analytics

Did you know that customers are up to 3x as likely to post a negative review of your company after a bad experience? And that 80% of potential prospects will desert your company if they read negative reviews of your products and services? Though it’s hard to put a price on the loss of reputation, the lost sales do stack up, averaging $250,000 per incident during Black Friday and Cyber Monday, and $40,000 per incident during the rest of the year.

Smart Error Handling Demands a Proactive Approach

Until recently, retailers had few options when it comes to preventing failures in their ecommerce platforms or their brick-and-mortar stores. The traditional approach was to wait for something to break, and then fix it as quickly as possible. In Macy’s case, “quickly” was about six hours. That’s not an acceptable speed. Another option is to try to track everything that is happening with dashboards and alerts, which quickly grows out of hand – how do you know where to look on your hundreds of beautiful visualizations? Which alert is meaningful when you get hundreds every day?

One of the difficulties in these scenarios is that similar-looking errors might have diverse causes. Your cash registers might not be able to process credit cards, but that could be due to a failure in any number of separate applications. We refer to these errors as “micro-glitches” – multiple, nearly imperceptible failures in multiple locations which accumulate to cause spectacular outages.

Going back to Macy’s, according to news reports they suffered an overcapacity outage of their credit card systems. Even this kind of outage comes with its own subsidiary failures and forewarnings, however. Systems as theoretically robust as the payment processing system for a major retailer don’t fail all at once. There are cumulative warning signs, such as an increasing trickle of transaction failures, or additional latency during card transactions. The ability to detect and interpret these warning signs could have meant averting a system failure at 10:00 AM, as opposed to trying to recover from a crash at noon. Maybe I’m stating the obvious here, but it’s much better and easier to prevent a failure that you identify early than to try to recuperate after the fact.

Where does Anodot fit in? Our AI-powered analytics gives companies the ability to collect and interpret real-time time-series metrics from across their payment processing systems and every other internal and external system, letting them detect and mitigate failures before they begin to drain their revenue and reputations. This can protect companies from the types of disastrous events that Macy’s experienced on Black Friday. To learn more about Anodot, and how our technology can give early warning capabilities to your ecommerce platform, sign up for a free demo today.

Written by Anodot

Anodot leads in Autonomous Business Monitoring, offering real-time incident detection and innovative cloud cost management solutions with a primary focus on partnerships and MSP collaboration. Our machine learning platform not only identifies business incidents promptly but also optimizes cloud resources, reducing waste. By reducing alert noise by up to 95 percent and slashing time to detection by as much as 80 percent, Anodot has helped customers recover millions in time and revenue.

You'll believe it when you see it