Ninety-nine percent of companies surveyed use the cloud somewhere in their business, and 50% of a company’s workload is done in the cloud, according to Flexera 2021 State of the Cloud Report. When your internet connection goes down, it’s taking your company with it.
Internally, your staff depends on email, communication apps like Slack, cloud storage and software, access to information (a.k.a. the web), and VoIP phone systems, not to mention everything on your intranet.
Externally, they depend on the internet for customer and vendor communications, sales channels, product and service delivery, and more. The cost of an internet outage to your business could be between $2,300 to $9,000 a minute, depending on your company’s size and other variables.
A backup connection will help you avoid that by providing continuous internet access for your staff, vendors, and customers.
Internet Outages Are Common
Internet outages happen daily. And because one significant outage can domino into other outages, there are a multitude of reporting sites you can track them with. Downdetector, for example, publishes real-time website outages from some of the internet’s largest sites, including Apple, Walmart, and Bank of America. Fing tracks the top ISP outages and their locations for the past 30 days.
Everyday storms can cause power outages that last a few hours, which is stressful enough. Intense weather events, like Winter Storms Uri and Viola, can take power out for days—and these types of storms are increasing in frequency. Modems, routers, and servers can’t run without power.
Equipment or Software Failure
All equipment has a lifespan—and sooner or later that equipment will fail. Having a service agreement on your internet-related equipment can help get you back up within a day.
Companies that provide a virtual private network (VPN) for their staff for security purposes or access to custom software need a consistent, reliable VPN now more than ever, with so many office staff working from home. Yet when your office’s internet fails, so does your VPN.
Service Provider Failure
Service providers also fail, not only due to weather and equipment failures but also cable and infrastructure damage and more technical problems, such as system misconfigurations.
“While application delivery is dependent on many Internet Service Providers (ISPs), it also increasingly relies on a large and complex ecosystem of Internet-facing services—such as CDN [content delivery networks], DNS [Domain Name System], DDoS [distributed denial-of-service] mitigation, and public cloud,” Network Computing reports. “These services work together to provide exceptional digital experiences to users and even brief disruptions can have a significant impact.”
How significant? In 2019, an ISP in Pennsylvania experienced a routing error that led to problems for Cloudflare, AWS, Facebook, and Discord users, among others, for almost two hours.
Network Traffic and Hackers
Just like a highway traffic jam can prevent you from getting to work on time, network congestion can stop you from moving anywhere online. These internet outages can sometimes be traced to old cables connecting you to your ISP.
But they can also come from a major website becoming overwhelmed with traffic, reducing available pipe for the rest of us. For example, in September 2019, Wikipedia experienced a DDoS attack that not only overwhelmed the site but also caused global traffic jams. Network Computing reported that there was a “significant drop in HTTP server availability from around the world, as well as a dramatic increase in HTTP response times” due to the attack on Wikipedia.
Downtime Costs Add Up Quickly
Your total downtime costs will depend on a variety of factors, including outage severity, company size, business model, and vertical targeting. Whatever the exact cost for your business, it’s likely to be significant.
Costs by Company Size
Atlassian estimates that a 14-hour outage in March 2019 cost Facebook $90 million, and Gremlin estimates that a 2017 outage of Amazon S3 cost the company around $150 million.
Certainly, small to midsized businesses (SMBs) will lose significantly less than the giants, but they often won’t have the resources to weather the crisis. For a small business (less than 100 employees), Carbonite estimates that the hourly cost could be as high as $25,000 or $427 per minute. Midsized (100–999 employees) and large (1,000 employees or more) businesses could see costs of up to $540,000 per hour or $9,000 per minute, according to Atlassian.
To estimate how much downtime costs your business, Gremlin recommends starting by calculating the internet outage losses for every hour your internet connection is out:
Revenue. Multiply your average sale amount by the average number of sales per hour.
Staff productivity. When the internet is down, how many of your employees can’t work? Don’t forget to include the hours IT spends managing the crisis and repairing the damage instead of doing their scheduled tasks.
Money owed to customers. Include not just any compensation but also fees dictated by service-level agreements and possible government fines. To get an hourly rate, divide the cost per outage by the number of hours of the outage.
Hidden costs. Did you need to hire outside consultants to repair and prevent another outage? To complete customer orders or otherwise catch up on work? What does lost data cost you? Again, divide the total hidden costs per outage by the number of hours the internet was out.
Add them together, and you’ll get the per-hour cost for an internet outage. And don’t forget soft costs. Chaos and stress will urge good employees to find work elsewhere, while a frequent inability to order from you will push customers to buy from someone else.
Failover and Redundancy Can Save Your Business
When issues occur, being able to switch to a reliable backup or redundant system is key. Failover is the method of switching between your primary and secondary systems as seamlessly as possible. With redundancy in place, a standby internet service is ready whenever your primary service is unavailable. It can come online manually, with a member of IT making the switch, or automatically, based on the type of failover process you use.
This redundant system ensures that if your internet connection drops out—your business and your customers are still able to access your product or service.
Let’s say you’re a car dealership with a robust website. With social distancing guidelines in place, your dealership depends more than ever on that website to connect you with buyers. Car buyers come to your website to preview the cars on the lot, appraise the value of their trade-in, apply for financing, and schedule a test drive.
Your sales team receives leads from the site and emails car buyers with more information. Your finance team sends paperwork via email, accesses customers’ financial information, and reviews vehicle history. And they use the internet to connect to your state’s registry of motor vehicles to register cars for your customers and to the customer’s insurance company to help set up proper coverage. If their internet connection goes down, car sales grind to a halt.
Internet redundancy prevents all that from happening. When the main service goes down, your backup service kicks in automatically and alerts your IT team of any issues. That frees up their time to focus on resolving the underlying issues that caused the outage without having to spin up a new connection at the same time.
Active-Active vs. Active-Passive Failover
There are two core types of failover redundancy settings: active-active and active-passive connections. Each defines how you initiate or hand off the failover. The type you choose is a direct result of your business and team needs.
In an active-active failover, you have two networked servers connected to your database at all times. If the primary server fails, the secondary server instantaneously manages the connection. This means that you’re essentially running two versions of your application to increase resiliency and balance the server load.
In an active-passive failover, you have a primary network that manages all functionality and a secondary waiting on standby. This means you have a single version of your application running on one node—so when it fails, you have to restart the secondary connection in response.
With active-active failover in place, users see an improvement in connection times because there are increased redundancy and faster failover hand-off times. The connection is more stable, and if one service goes down, the other automatically picks up the slack.
Think about what the best load balance would be between two active internet services. For example, a strategic approach would put all mission-critical apps on one service and the remaining best-effort traffic on the other. You could try a 50–50 split of all traffic. When your backup internet is wireless broadband, which tends to have monthly usage caps, you might find an 80–20 split works better.
Just keep in mind that managing two active networks can require more time and effort from your team. And configuring your networks to have enough diversity could mean additional costs in incoming cables and hookups, equipment, and staff expertise.
The Right Internet Backup Isn’t Just a Duplicate
A backup service won’t do your business any good if it goes down with your primary service. When researching an internet backup, consider the following factors to make the best choice for your business.
Check the route the service takes to get to your business. If it’s the same as your primary internet service, you could lose both with a local outage.
Choosing a different technology for your backup internet can also decrease the chances that both will fail simultaneously. For example, if your primary internet connection is wired, consider cellular for your backup. If you normally connect through fiber optic, use cable internet as your backup. And if both are fiber, ensure they have different points of entry.
If both of your fiber connections enter the building at the same point, that significantly increases the potential for both to fail if issues occur. Each building entry point (BEP) should be physically separate from one another whenever possible.
It’s important to make sure your backup connection can handle your data needs as well. You want to have the same level of service with your internet backup, so your business can keep motoring ahead.
Network diversity adds an additional layer of security by introducing geographical distance for your backup internet connection. This diversity can be the result of physical distance, like that of a colocation data center (Colo) or via the cloud. When you expand your network in this way, it improves the infrastructure—cutting down on the potential for single location events, such as weather or construction, to bring down your network.
Just ensure that when you set up these redundancies, you manage the transport of data security. The transport mode you use will dictate how each network communicates with the other.
Network peering, that is, networks agreeing to exchange local traffic without cost, can also increase connection speeds and stability through their relationships. You’ll want your internet providers to belong to different peering relationships, both with diverse networks. PeeringDB and AS Rank can help you determine your networks’ peering.
How Much Should a Backup Service Cost?
Your backup system shouldn’t cost more than your primary system. It’s just common sense that you wouldn’t pay more for something you won’t use as often.
But telecoms make buying internet services complicated and hide the true cost from customers. Not only does it cost your company more for the service, but it costs you more in time and fees to find and set up service. Purchasing an internet backup connection from your ISP doubles those costs.
Here’s the thing: telecom service is a commodity. Your ISP is transmitting bits of data across a wire, which is measurable and, therefore, easy to price. Instead, telecoms create long buying cycles, allowing them to charge based on what they learn about their customers. In fact, as we’ve written before, “three businesses with the same needs in the same building will likely get different quotes.”
Your only recourse is to do your research on the offerings.
Make a Choice That’s Best for Your Business
With everything a business does online, having an internet backup connection is a necessity for businesses of all sizes. But it’s no small task to bring on a second connection. Determining your needs can be complex. Start at the root: what are your business needs? Consider especially the size of your business and the geographic distribution of its network. Is it in one building? Two? Across the country? Around the world? What are your internet resiliency needs?
Then consider the financial aspects. While comparing quotes can be frustrating, keep in mind your budget. Know what you have already in talent and hardware, how best you can leverage those assets, and where you need to make improvements.
Then fulfill those needs as best you can given your available options.