Bulk Purchasing of Internet / WAN Connectivity: Pros, Cons, and Principles

I was in a quarterly business review with an executive from a Fortune 500 company recently when I was asked about the benefits of “bulk procurement” on dedicated internet circuit pricing, meaning gaining a lower price per Mbps on average when ordering a larger number of services versus the price that you would pay for a single service. 

On the face of it, this seems like a basic and widely accepted concept: with economies of scale from a large telecom service purchase, enterprises can expect lower unit costs. And it is true, that on like-for-like commodity telecom services, you are able to negotiate lower costs when you buy more at once. But how you define “like-for-like” is where the rub is. In the telecommunications services space, particularly with circuits, blindly leveraging bulk pricing as the foundation of an RFP, done incorrectly, can become a suboptimal strategy that leads to higher unit pricing.

In this blog post I will present a few recommended sourcing principles and provide a couple of examples to talk through why suboptimal bulk procurement can lead to higher unit pricing. I will also discuss the bulk site circuit sourcing strategies that Lightyear utilizes to deliver not only the lowest possible price points, but also the highest quality network. 

Before I jump into principles and examples, I need to define two telecom terms used internally at carriers that define how a service is delivered to an end-user customer. 

Type-1 vs. Type-2 Connectivity

Type-1 Connectivity: Type-1 access represents connectivity service that is delivered end-to-end by the same provider. For an internet service, this means that the same company who owns and operates the last-mile fiber that terminates into the service address is also the company that assigns your IP addresses and routes your traffic over their own network / ASN. An example of type-1 service delivery is when AT&T is the last-mile fiber owner, AT&T assigns IP addresses, and AT&T routes your traffic over their own core network, which is ASN-7018. 

Type-2 Connectivity: Type-2 access represents a connectivity service that is delivered by a service provider that is leveraging a third-party for fiber access to the service address. Type-2 service delivery is necessary for a provider to deliver service to a location where they do not have a last-mile fiber network. An example of when type-2 connectivity happens is when you order AT&T internet service in Boston, which is outside of AT&T’s last-mile LEC footprint. To solve this issue, AT&T might lease a layer-2 private line connection from Verizon, (the designated phone company in Boston) to connect the service address to AT&T’s closest point of presence. AT&T then backhauls the traffic from the service address to the nearest AT&T POP over the Verizon fiber connection. Your IPs are assigned by AT&T, and a traceroute will show you connecting to the internet over the AT&T/ASN-7018 network.  

Type-2 connectivity options are mandated by the FCC under the telecommunications act of 1996. Phone companies like Verizon, Lumen and AT&T have no choice than to offer their last-mile fiber to any registered ISP for resale. Additionally, many other service providers like Zayo for example, are willing to support the type-2 model, and have large wholesale distribution channels that focus on selling connectivity to carriers so those carriers can deliver services in a type-2 fashion. Type-2 connectivity is EXTREMELY prevalent, and in many cases it is transparent / not made clear to customers when it is being used, at least at the quoting stage. This is important to keep in mind, as you’ll pay a premium to buy type-2 access from a carrier, as they have to cover the cost of purchasing the underlying service, plus their own mark-up. For more detail on pros and cons of type-1 vs. type-2 access, read our blog post here.

Now that we have defined type-1 and type-2 connectivity, let's move onto sourcing principles.  

Connectivity Sourcing Principles 

When you examine aggregate circuit pricing across a wide range of enterprise customers, there are three sourcing principles that always lead to best pricing. In order to illustrate these principles I am going to again use AT&T as an example carrier, but this logic applies to any facilities-based carrier delivering service over their own last-mile network (Type-1).

1. If you are going to order a service that is being delivered over AT&T fiber, the best price is going to be from AT&T. The explanation for this is simple: AT&T’s wholesale channel (that sells to other carriers) is set up to not cannibalize AT&T’s retail channel. I don’t care what your non-AT&T sales rep tells you about their companies volume discounts or special relationships, we have limitless data that supports this. Why would they let you buy AT&T service for cheaper from someone else?

2. “Highest discount” and “best price” are two entirely different concepts. Getting the highest discount means getting a discounted price that is at the minimum acceptable margin-threshold from a carrier for a specific service. Getting the best price means getting the highest discount from a type-1 provider who has the lowest cost of delivering the service. 

3. If you ask a carrier sales representative to quote a service, you will always get a quote from that carrier, regardless of how cost effective/competitive that carrier may be for that specific service need. The entire industry has been incentivized to be “big deal” chasers, and two services cost more than one. 

Now that we have established those three principles, let’s look at some real world examples.     

Example 1: Bulk Sourcing Within a Single Carrier Footprint

First, let’s consider a simple but often unrealistic situation where unit costs actually will go down when using bulk pricing as the primary RFP strategy. In this example, it is possible to get the best discount as well as the best price.

In this example, an enterprise has 100 sites, all of which fall within AT&T’s 21 state LEC network footprint. The enterprise wants to buy 100, 50 Mbps dedicated internet circuits. They send out an RFP for all 100 sites to multiple carriers, including AT&T, who all return bids. AT&T will win this bid every time (as long as there is not another type-1 provider that is on-net at all 100 sites, which is very uncommon for large enterprises who tend to have diverse geographical footprints).

What is driving the above outcome is principle 1, and the fact that you cannot order an AT&T service, for less money than any provider other than AT&T. Since AT&T is available at all sites, they are the best provider to deliver this service, and since there are many services being ordered you can likely get the highest discount through shrewd negotiation. 

The problem with this example is that it is not realistic, because most enterprise facility footprints fall outside of network boundaries where type-1 connectivity is available from a single provider. 

Example 2: Bulk Sourcing Across Multiple Carrier Footprints

Let’s now consider the same enterprise with 100 locations, but assume (much more realistically) that their sites are distributed evenly across AT&T, Verizon, and Lumen’s footprints. If you assume a lower cost comes with economies of scale, you perform the same workflow as in example 1 and ask multiple providers to all bid on the project. Each provider gets the same list of addresses for all 100 sites. The bids come back, you negotiate, analyze, and choose the lowest bid. This is the status-quo RFP strategy most companies take. 

In this case you may achieve the highest discount due to the fact you are ordering 100 services, but you are not going to get the best possible price. This is because Verizon needs to order access from Lumen and AT&T outside of their own Verizon LEC designated footprint, Lumen needs to order access from AT&T and Verizon outside of their LEC designated footprint, and AT&T needs to order access from Lumen and Verizon outside of their LEC designated footprint. Each carrier’s costs to deliver service increases in this scenario, as every carrier must provide type-2 connectivity.

This example is in violation of principle 1 outlined earlier. However, what makes this even more challenging is principle 3, in that if you ask a carrier sales rep for a quote, you are going to get a quote.

Your sales rep is now chasing a big deal, exactly as they have been incentivized to do. The sales rep is also likely aware that they are not the optimal provider at the sites that fall outside of their respective carrier’s footprint. 

Here is where it gets really tricky!

Due to the carrier sales rep knowing that they are bidding on services outside of where they are most competitive, they take the margin that would normally be available in example 1, where they are all type-1 services, for negotiation, and they apply artificial discounts to the sites in the other carriers’ footprints where they are less competitive. Oftentimes they will even increase the price of the services within their footprint where they are the optimal provider, to present optics that show them to be more competitive in the other carriers’ footprints. Principle 1 allows them to do this with confidence. The problem is that even with the artificial discounts, they are not going to be able to achieve the best type-1 price points.  

What we have now is blended type-1 and type-2 pricing across all sites, from all bidding providers, which is exactly what the pricing teams at most carriers love. You are paying an elevated price in AT&T’s footprint in order to get an artificial discount on the services in Lumen and Verizon’s footprint. The cost per Mbps is above the “best price” at every site, but it is hard to see it because all of the RFP responses incorporate the same flaw, so side-by-side the RFP responses all look consistent. 

To make matters worse, the company in this example is also inadvertently making life much more difficult for themselves, both now and in the future. Here are some examples why:

• The RFP takes forever. It takes a lot of time for AT&T to get wholesale pricing from Lumen and Verizon. Lumen and Verizon also need to get it from each other, as well as AT&T. If you have ever wondered why it takes so long to get pricing back for “just internet access”, this is why.

• Implementation of these services is far more likely to be inconsistent and problematic. Instead of working directly with AT&T, Lumen and Verizon on the installs, the company in example 2 will be forced to communicate indirectly with Lumen and Verizon, through the AT&T project manager. This adds an additional level of complexity to an already very complex implementation workflow. 

• When these services go down (and they all do) it is going to take longer for them to be repaired. For the same reason that implementation is more complex, you will also encounter longer mean-time-to-repair intervals. You will call AT&T to open a trouble ticket, and then need to wait on AT&T to open another ticket with Lumen or Verizon. All transport related troubleshooting (where the majority of repair issues occur) needs to go through this AT&T to Verizon or Lumen, back to AT&T, back to you, game of telephone. 

• You lose fidelity into the makeup of your network. In many cases these type-2 setups are not made clear to end users. This can create situations where an enterprise believes that they have last mile provider diversity, but really don’t. The primary carrier is AT&T type 1, and the secondary is Verizon, being delivered over AT&T transport as type-2. The company doesn’t know this, and a traceroute will show diversity where it doesn’t really exist. 

Feeling the pain yet? We are too, don’t worry. So, how do you do this properly? 

You must move the entire RFP exercise off of the carrier's playing field, which they have purpose-built to put you at a disadvantage on, and onto your own field that optimizes outcomes for your business. This is done with data, in the form of network and pricing intelligence. 

Example 3: Highest Discount and Best Pricing!

When Lightyear has a bulk-site bid request, addresses are run through our network intelligence platform. This allows us to see who all of the carriers are at each address: who is on-net, who is-near net, transport medium, transport capacity, distance to fiber, etc. 

This data allows us to algorithmically request quotes for service addresses from the providers that can deliver as type-1 only, in bulk. Instead of Verizon, Lumen, and AT&T getting an RFP for 100 sites, they get only the request for services that fall within their last mile LEC footprint. When you are dealing with type-1 services, understanding if costs are in-line with “best pricing” becomes much easier. In type-2 scenarios, benchmarking becomes difficult because circuit costs will change based on who the ISPs underlying type-2 provider is, which often varies location to location. 

This solves all of the problems we have outlined in example 2:

• The pricing we receive back will be from the optimal type-1 providers, which will have the highest margins, enabling the biggest discounts. It is reasonable to expect pricing that is 25% less than otherwise would be possible in type-2 scenarios.

• The RFP pricing responses will be returned much faster, because we are avoiding all of the secondary quoting on the wholesale market for type-2 access that must otherwise happen.

• Implementations are far more likely to be consistent and predictable, and mean-time-to-repair will be reduced because we are working directly with the company delivering all elements of the service. 

• We will know exactly how the service is being delivered, and can validate last-mile and ASN diversity with firsthand information.

Network Lifecycle Management in Example 3

The single drawback of leveraging the strategy outlined in example 3 is that you now need to manage multiple carriers throughout the service's lifecycle - more bills, more points of contact, more frustration. 

However, this is where Lightyear’s software steps in for the win. The Lightyear platform will record and report on the entire implementation from credit accept to FOC, and through activation on each service. If there is a problem with an implementation, our professional services team will have it escalated immediately. 

As your services are installed, they are automatically added to our Network Inventory Manager, which acts as your system of record for over 30 unique data points on each service. When you need to know your carrier, cost, circuit ID or IP address at a specific site, all of this information and much more is a click away. Our software will even automatically re-shop your services as the contracts near expiry.

Conclusion

I talk to enterprises every week who leverage the strategy outlined in example 2 in their RFPs. They always believe they have the best price, because they know they did a good job to achieve the best discount, but the reality is they negotiated with a company that cannot achieve the best price and still be profitable. 

This all worked in the days of MPLS, where a single carrier solution was the norm. However, the shift from MPLS to SD-WAN and multi-carrier underlay fabrics renders the “bulk discount” RFP strategy ineffective. Enterprises have been slow to recognize this, and they continue to use their MPLS RFP strategies to procure internet access to support their SD-WAN implementations. They are able to achieve better costs than what they paid for MPLS, so it all looks good on paper, but their costs on average could be reduced by more than 30% by shifting to a different sourcing strategy that leverages a network intelligence centric strategy, over a bulk discount strategy. 

Enterprises must incorporate better data into their workflows and optimize their bidding strategies based on an understanding of how carriers work and a granular understanding of what true market prices are for type-1 service.  

If you want to learn more about how you can do all of this with Lightyear, at no cost, please reach out to us for a meeting. If you have existing services, that suspect may be higher priced type-2, and you want to understand what your cost savings potential is, we can help with that too.