Creating a network architecture that not only scales, but scales with predictable cost models is a challenge that network architects and network managers struggle with every day. Most network architects aren’t necessarily looking for the lowest cost solution, although cost is important, its about creating a framework that scales as certain metrics are met or passed and allowing this scale to be predictive in nature (and thus predictive costs). For example, if I have 2,000 FTTH subscribers today, what will the network look like at 4,000 subscribers and what will the cost to get there be? How about 8,000 subscribers? 20,000? The idea is to have these figures known ahead of time. If the marketing folks are predicting growth from 2,000 to 20,000 subscribers you can create a budget rather quickly and logically.
This article will explore the concept of modular network design, how it looks, how it scales, and why it’s important. No bandwidth or user metrics are defined in this article, as each network and business has their own growth models and capacity, it does however, show the logical progression of scaling the network.
Modular network design is all about creating repeatable and predictable network topologies and sub-topologies. The concept here is that the overall network topology is defined by a series of smaller modules. The common infrastructure modules used are:
- Core – connecting point for other modules. Its primary purposes are to move packets quickly, efficiently, and in most cases transparently (MPLS)
- Access – this module connects directly to the subscriber or business and usually consists of DSLAMs, CMTSes, Ethernet switches, and so on.
- Aggregation – this module aggregates 1 or more access modules and connects it to the core network.
In the case of smaller networks some of these modules will be collapsed. Such as core and aggregation. In either case the functionality and concepts are congruent.
In addition to the infrastructure modules there are service modules. The infrastructure modules in essence provide transport services and connect the subscribers to the services the Service Provider is providing.
Some examples of service modules:
- BRAS/Subscriber Management – provides authentication, NAT, DPI, and traffic control for data subscribers. It then forwards the traffic to the Internet module
- Internet – provides direct Internet access for businesses and other modules. Performs BGP peering, etc.
- Managed Firewall/Security – If a residential or business subscriber signs up for a managed firewall service their connection can be landed in this module for firewall services and other security needs
- Data Center (access to VDI, VPC, NFC, and other DC type services) – a customer VLAN or connection can be connected to this module for access into the SP-hosted DC and the services associated with it.
Putting this all together results in a topology that looks similar to this:
Predictable Costs/Network Planning
Based on this architecture I can start defining what equipment, link speeds, powering, and other design components are required. The goal of this architecture is to be able to add additional modules or expand modules as needed in a predictable way. For example, as I bring on new markets or service areas I know I will add in another access module for each area and connect it to the closest aggregation module (or create a new aggregation module if needed.)
The network architect of this design will need to define metrics for each of the modules. For example perhaps each access module can support 1,000 to 5,000 subscribers and each aggregation module can support 10,000 subscribers. So in this case, the aggregation module could have 10 – 1,000 subscriber access modules connected to it or just 2 – 5,000 subscriber access modules. If we go over that an additional aggregation module can be created to support the next set of subscribers.
So once the modules and architecture are laid out and ideal equipment spec’d metrics can be laid out. These exact metrics will need to be researched and defined based on historical network usage, services offered, take rates, demographics, etc.
Core with single 10G links in ring topology can support up to 30,000 subscribers. Over that an additional set of 10G links will need to be brought up in a LAG configuration.
Aggregation can support 10,000 subscribers. Over that an additional aggregation module will need to be created to support future growth in those areas.
Access can support up to 5,000 subscribers, if we go over that an additional chassis or access ring will need to be added.
For the service modules you will also define how many subscribers and services it can offer before needing an additional module or upgrading the existing module
These metrics should be updated and reevaluated periodically. What it provides is a method for a network architect to predict costs as subscribers and services grow and be able to budget and plan accordingly. You can put these metrics and calculations in a spreadsheet and predict costs and growth pretty accurately. These metrics can then be modified when Internet usage patterns change and when new services are added or removed.
After the network has been planned and metrics defined, this architecture allows simpler deployment methods. Each access layer will connect into the aggregation layer the same way (outside of IP addresses and VLAN information) and when using unified MPLS no aggregation or core configuration is needed to turn-up a new customer. Pseudowires from the aggregation or access (if layer 3/MPLS enabled) are configured to backhaul the traffic across the aggregation/core to the relevant service module(s).
Configurations can be templated and even automated with advanced provisioning tools. The key to making that successful is consistency on network topology and interaction between the different modules/layers. This consistency then correlates into easier troubleshooting. You can narrow down the issue by seeing which access modules are having issues, which aggregation module, or even down to what service module, or what pseudowire is seeing errors.
By creating a modular network architecture, network architects and planners can develop a predictable cost models as well as consistent deployment templates and troubleshooting methods. With the use of end-to-end MPLS the deployment engineer can provision a new or existing customer at the access layer and at the hand-off to the service module without touching any equipment in between in the aggregation or core modules. These advantages combined with consistent troubleshooting can go a long way to reducing OpEx as well as providing a better cost model for budgeting purposes.