Posts Tagged ‘cisco’



Layer 2 – Data Link: Specifications for delivering data across a uniform medium, that provide the functional & procedural means to transfer data between network entities

Data Link Functions:

  • PHYSICAL ADDRESSING: formats signal into data frames, organizing the bits into:
    • Frame Headers: contain the hardware destination & source address
    • Payload: contains the actual data/information being transmitted

    ERROR DETECTION: detection of transmission errors that may occur in the physical layer
    ACCESS ARBITRATION: endeavors to arbitrate between parties contending for access to a medium; in the event of contention specifies how devices detect &recover from such collisions, and may provide mechanisms to reduce or prevent them
    UPPER LAYER PROTOCOL IDENTIFICATION: Data-link frames, do not cross the boundaries of a local network. thus inter-networking and global addressing require higher layer functions. The protocols used to fulfill these functions must be identified.

Data Link Technologies/Protocols

  • L2 Devices – Bridges, Switches, Wireless AP, Network Interface Cards (NIC)
  • L2 LAN Protocols – 802.3 Ethernet, 802.11 Wireless,
  • L2 ISP Protocols – PPP, PPTP, L2TP, Frame Relay, Q.921 (ISDN), ESF (T1)

Data Link Framework

Composed of 2 Sub-Layers

  1. LLC[Logical Layer Control] 802.2; done in software
    • Flow Control & Regulation of data transfer rate
    • Error Detection [via FCS: Frame Check Sequence]
    • Identification Of Layer 3 Protocol [via Protocol Field in Header: DSAP/SNAP]
    • Encapsulation/Decapsulation
  2. MAC[Media Access Control]; done in hardware
  • Hardware Addressing
  • Media Contention
Ethernet Layer 2 Functions

Addressing – defines ID for each network node

Ethernet = MAC address = 48 bits long

Ethernet MAC Addresses – burned in to network devices EEPROM
NOTE: 48 bits represented as 12 digit hex ID
Ex) 0000.0015.E1FF
Decimal: 248 = Hex: F8 – (F is in 16’s column, and 8 is in 1’s column)
Decimal: 17 = Hex: 11

Cisco Notation – 0000.0000.FFFF
Standard Notation – 00:00:00:00:FF:FF

XX: represents 8 bits/1 byte – 0 to 255


(First 24 bits) OUI[Organizationally Unique Identifier] – Manufacturer ID assigned by IEEE
(Last 24 bits)Interface ID – Unique ID for that device

:. 16 million Manufacturers, each with 16 million unique device ID’s

Error Detection – determines if data successfully transmitted across the physical medium

Ethernet = FCS[Frame Check Sequence]

4 byte Data-Link Trailer – essentially algorithm based on frame’s contents is applied before transmission, then compared to results after transmission :. if they are the same -> no errors occurred

/!\NOT Error -Recovery-

Identification Of Encapsulated Data – identifies Layer 3 protocol that encapsulated the data

Ethernet = 802.2 LLC[Logical Layer Control] Sub-headers

i.e. determines if data is an IPX Packet meant for a Novell system, or an IP Packet meant for a Windows system

Arbitration – determines when it is appropriate to use physical medium; how to avoid and/or recover from frame collisions

Ethernet = CSMA/CD[Carrier Sense Multiple Access/Collision Detection]

media access mechanism in which devices ready to transmit first check channel for carrier prior to transmitting; if no carrier is sensed then device can transmit

Collisions: if 2 devices transmit at once -> a collision occurs

  • this collision delays retransmission from those devices for random length of time
  • more systems on network = slower network; 2X Systems -> 10X # of collisions

:. collisions limit the # of systems

[around 40% of bandwidth utilization performance peaks then drops due to collisions]

Collision Detection Process
  1. collision is detected [i.e. voltage is over acceptable range]
  2. jam signal propagates & notifies all devices on network
  3. all devices stop transmitting
  4. set random timers before resending

802.3 Ethernet Frame

Ethernet II = Xerox Ethernet developed by Bob Metcaff

*also called DIX[DEC Intel Xerox] Ethernet

Preamble Destination MAC Source MAC Type Data FCS
8 bytes 6 bytes 6 bytes 2 bytes MTU 4 bytes

IEEE 802.3 Ethernet

Preamble SD Dest MAC Source MAC Length DSAP SSAP Control Data FCS
7 bytes 1 byte 6 bytes 6 bytes 2 bytes 1 byte 1 byte 1-2 bytes MTU 4 bytes

IEEE 802.3 Ethernet w/ SNAP Header

Preamble SD Dest MAC Source MAC Length DSAP SSAP Control SNAP Data FCS
7 bytes 1 byte 6 bytes 6 bytes 2 bytes 1 byte 1 byte 1-2 bytes 5 bytes MTU 4 bytes

MTU[Maximum Transmission Unit] – 64 to 1518 bytes

defines max Layer 3 packet size that can be sent over a specific medium

802.2 LLC Sub-headers

SSAP[Source Service Access Point] – IEEE defined “type” field that identifies the Layer 3 protocol that originated Data

DSAP[Destination Service Access Point] – IEEE defined “type” field that identifies the Layer 3 protocol to send the Data to

SNAP[Sub-Network Access Protocol] – later developed by IEEE to accommodate additional protocols; i.e. extension to DSAP

Control – notifies what type of packet is encapsulated in frame :. reduces cross-protocol broadcasts [i.e. identifies what -fields- will follow]

NOTE: DSAP value of AA & Control value of 03 signifies to use SNAP to identify protocol

/!\TCP/IP requires SNAP

Preamble – a 64-bit (8 byte) field generated by the LAN interface card that contains a synchronization pattern consisting of alternating ones and zeros and ending with two consecutive ones, which allows devices on the network to easily detect a new incoming frame. After synchronization is established, the preamble is used to locate the first bit of the packet.

SFD [Start Frame Delimeter] – the 8-bit (1-byte) value marking the end of the Preamble of an Ethernet frame, which is designed to break this pattern, and signal the start of the actual frame. It has the value 10101011.

Both the Preamble and the SOF assist NIC’s adjustment to slight speed variations between frames

Sigh. You know, these past 5 or so years, I’ve been meandering around so many different aspects of IT…such a huge array of infrastructure technologies, all in order to meet the demands placed on me out in the battlefield. Either being the primary infrastructure architect, designing & operating a data center, and now finally as director of IT at my current job, I’ve needed to be highly competent in multiple disciplines in order to keep my head afloat in the tirade of shit-storms that incessantly barrage every network I’ve managed.

And let me tell you, it has NOT been easy. My list of certs is just ridiculous, but honestly these came out of an effort to simply understand…TRULY understand…all these various, complex technologies.

I was never fortunate enough to have guidance. While I certainly had colleagues in the field, they were in the same position I was– lacking a deep enough, master-level of understanding.

I desperately wanted a mentor, but after numerous failed attempts from IRC and 2600, my impatience finally won out. So throwing my arms up in frustration, I simply resigned myself to the DIY state of mind; left the world of ATI vs Nvidia T&L anisotropy Direct 3D super-mega online PC gaming behind…and took the first real step of my IT career to find those answers on my own.

I certified myself in various disciplines from active directory to information security to service provider networks to virtualization, and yes, it certainly paid off. If I had to, I could run a company’s entire IT department myself (assuming they kept me pumped full of intravenous amphetamines, since nothing interferes with productivity quite like “biology”). I was adaptable enough to work in multiple spaces from collocated data centers, service providers, and of course good old fashioned enterprise.

“Jack of all trades.” Master of some.

And by the way, by “ridiculous” list of certs, I mean freaking RIDICULOUS:

  • TippingPoint Certified Security Expert #2370
  • Information Systems Security (INFOSEC) Professional, NSTISSI 4011
  • VMware Certified Professional vSphere 5
  • Cisco Certified Network Associate (CCNA)
  • Cisco Certified Network Associate: Voice (CCNA: VOICE)
  • Cisco Certified Network Associate: Security (CCNA: SECURITY)
  • Cisco Certified Network Professional (CCNP)
  • Cisco SMB Engineer
  • Cisco SMB Account Manager
  • Microsoft Certified Systems Administrator 2000
  • Microsoft Certified Systems Administrator 2003 / Security
  • Microsoft Certified Technology Specialist: Windows Server 2008 R2, Server Virtualization
  • CompTIA A+ Certification
  • CompTIA Network+ Certification
  • CompTIA iNet+ Certification
  • CompTIA Security+ Certification

Again, all because I was sick and tired of being blind; no experience and certainly no one around who could offer any guidance:

“Hey is a Pentium Pro or Pentium MMX better?” No one knew, so thus came the A+

“Hey how do you control Active Directory replication between forests?” No one new, so thus came the MCSA

“Hey, whats the difference between SRR and WRR queuing on the 3550’s vs the 3560’s?” No one knew, thus came 75% of my CCIP, which I would have completed if the butt-faces @ Cisco Learning hadn’t decommissioned the cert when I was 3/4 of the way through!


You get the idea. But now the time has come to stop spreading myself so wide, and to focus now on my core competency: networking.

The time has come for this jedi to step up and fight for his CCIE.

No more netapp, no more juniper, no more Microsoft: the path I have chosen is the only one that is right for me. I’m sorry Brocade; I’m sorry ISC2. EMC, you’ll just have to wait in line with all your friends.

I want that CCIE and those who oppose me will be cut down into bite-size pieces and fed to my mutant, ill-tempered sea bass.

So, now that I’ve made my decision, how do I proceed? I’ve decided to fork out $500 for a starter study package. It is a lot of money, but it won’t kill me. I was going to do all of this 100% on my own; download PDFs from Cisco, buy a bunch of books off Amazon. But no, that path is too unfocused. I’ve attempted that before, and you end up burning half your energy searching through multiple sources of documentation, and even at times finding the information self-conflicting, if not confusing.

This time, I am going to seek the guidance I’ve been denied in the past; its finally available and certainly within reach. If I am to do this most monstrous of achievements, its time to change old habits and old ways of thinking.

Now then, the first step is the reality check: where am I at in respect to the CCIE requirements?

  • Layer 2 technologies. I’m fairly strong here; certainly in Ethernet, but I also have had a good amount of experience with PPP. I’ve optimized data center cores, so I’ve setup MST a few times, usually though RPVST has been sufficient in the field. Frame Relay on the other hand, not so recent. Last time I configured a frame relay circuit was probably 2006. I’ll give myself 7/10.
  • Implement IPv4. Not too bad here either. Of course I’m sure there is a ridiculous amount of minutia surrounding OSPF and EIGRP, but I’m solid on the fundamentals. Same with BGP; I’ve worked in service provider environments, so I have had practical experience in the real-world. In fact I also took & passed the former CCIP’s BGP exam, so focus on the minutia and more exotic configs. PFR on the other hand, completely ignorant. In fact I had never even heard of this until today. I’ll give myself 8/10.
  • Implement IPv6. Oh god this is the most ANNOYING one. Certainly studied it for my CCNP, but never used in real-world setting, and extremely rusty. Not to mention I personally don’t buy into this “we’re all switching to IPv6 because the world is ending” drama. Maybe if you’re Verizon or AT&T, but I seriously doubt any enterprises or data centers making use of RFC1918 addresses will have a need. Sigh, probably 2/10.
  • Implement MPLS Layer 3 VPN. Did study a little bit of the theory for CCNP, and I can’t speak reasonably intelligently about LDP, LFIB, etc. but unfortunately this was the 1/4 that I didn’t take for the CCIP. I have done a single implementation of this at a data center core to separate customer networks, but I’m sure I have a long way to go in terms of practical implementation. Here I’m a 3/10.
  • Implement IP Multicast. Again, some theory from CCNP, and about 3 real-world implementations for VoIP music on hold & paging. However when I look at a show mroute, I feel the neural synapses in my brain being sucked up into space as I stare dumbfounded at terminal outputs. Because I know the basic steps to setup sparse mode and dense mode, I’d day 5/10.
  • Implement IOS Security. Here I’m ready to rock. I got my CCNA-Security and just loved learning about all the inherent security features on routers & switches. Only because I’m unfamiliar with the new v5 of IOS IPS signatures (back in my day we just had 128MB.SDF and 256MB.SDF) and because I’ve never implemented 802.1x at the switch level, I’m giving myself a 8/10.
  • Implement Network Services. Another strong section. I’ve setup SNMP, NTP, DHCP, and HSRP many times. WCCP not so much, so we’ll go with another 8/10.
  • Implement QoS. I did take the QoS exam, and I have implemented this many times for VoIP implementations. So configuring the policies and working with NBAR, I’m pretty solid. However, the queuing specifics of catalyst switches is something I’m rusty on. Let’s be honest, on 10/100 switches QoS isn’t a life or death thing (certainly not like it is on the WAN side) and usually autoqos voip trust is sufficient. Plus QoS for Frame Relay is a topic here, so I’ll give myself 6/10.
  • Troubleshoot a Network. Oh god. How can I even guess? I’ve certainly been doing this for a while, but this is the CCIE exam. God knows what crazy bullshit they’ll throw in front of me. Plus, when I’m troubleshooting a network, I have the luxury of dual 21” monitors, with all my templates at my disposal on the desktop, as well as all my favorite tools (nmap, tcpdump, etc.) I’ll stay on the conservative side and give myself a 6/10 here.
  • Optimize a Network. I think this section should be renamed to “monitor” a network. Well, I’ve worked with SNMP, FTP/TFTP, HTTP/HTTPS, NetFlow, and syslogging. I’ve setup my share of SPAN/RSPANs for IPS devices. Never really used RMON, and no clue what EEM is. And again, I have dual 21” monitors and historical reports on my solarwinds server to really study & monitor traffic patterns. In the CCIE I’ll have what…90 seconds maybe, to quickly read through several pages of text/CLI outputs on a 15” monitor. I think I’ll give myself another 6/10

So there it is. Now how shall I conquer this mountain?


My tentative battle plan is to focus on Layer 2 and IPv4 routing first, advancing my skills on these as much as possible; after all it is CCIE “Routing & Switching.” After than I’m going to skip over IPv6, which I plan on saving for last, since I feel that is by far my weakest topic, and move onto Multicast. Since Multicast is dependent on existing routing & switching to function, this seems like a natural progression, esp since I’ll need to understand it better in the context of Ethernet and IPv4. Because these topics define the foundation of “Routing & Switching” I will probably focus on these 3 areas exclusively until I can comfortably perform any of their respective “foundation-level” labs.

The concern here is getting “rusty” on these topics if I migrate away to ancillary topics too soon. I’m sure this will happen to a certain extent, but to minimize it as much as possible, I want to reach a state of total comprehension, so any refreshing is just that…refreshing not relearning


Next I will focus on MPLS and QoS. These orbit a little closer than the remain topics, and given their importance to the sister track “Service Provider” I would say this is good focal point for stage 2 of my CCIE journey


Between everything so far, its time to focus on troubleshooting these core milestones, not only as unique technologies, but how they interoperate with each other. And even at this stage, the focus is enormous: OSPF, EIGRP, BGP, Route-Maps, Redistribution, Frame-Relay, Spanning-Tree, PIM-DM, PIM-SM, MPLS, WRED, NBAR, Queuing, Policing, IGMPv2/v3, and THEN troubleshooting the interop between them. This is the core of the CCIE exam. If we were building a person, this is the heart, mind, muscle, and bone of that body


Services, Security & Optimization. In terms of the test, I see these as tasks that will complicate or interfere with a properly working network. So now that I have the intermediate fundamentals down, I can now explore making it work in a more efficient and secure manner


Adding in a single NAT statement, much less IPSec, DAI, DHCP Snooping and ZBFW can bring that happy network to its knees. Clearly troubleshooting complexity increases exponentially. Being able to identify the culprit as a misconfiguration; or is it instead the normal operation of a security mechanism? Understanding the behavior of these competing technologies will be the next major undertaking.


This is where I’ll probably want to drive my car off the freeway and end the misery. Honestly I hate IPv6. Did you have to build it using 128 bit hex addresses IEEE? Really? Did you really have to do that?

“Hey Glen, can you ping the router for me?”

“Sure, what’s the IP?”



I mean really. Why not build off of IPX/SPX? Simply a DECIMAL network number to the card’s MAC address? How much freakin easier would it be?


Well, that is the battle plan for now. Will it change? Maybe…probably. I’m sure I’ll need to adapt. And god forbid Cisco update’s the exam AGAIN while I’m in the middle of studying for it!

Anyway, I’ll keep you updated and document my progress here. Time is a bit tight, but I’m aiming for 1, possibly 2 posts a month.

Til then my padawans, give my best to your wife…and my kid


Ok padawans, this is something that I’ve wanted to cover for quite a while, but with the plethora of obligations monopolizing the jedi’s time, I was out of commission for some months. But fear not! Your jedi is back in the saddle and ready to bring it!


Notice to the audience: this article assumes the reader understands basic networking concepts such as CIDR notation, legacy class A/B/C vs VLSM, and how to subnet both class B and Class C networks. If I say to you, “what is the most efficient subnet mask to support 400 hosts and allowing the most possible networks/subnets” you should know (or a /23 if you prefer) off the top of your head will this requirement. In fact you should know it can scale to 510 hosts, and the moment you get that 511th host, you will need to change masks to (or /22) in which case you can scale up to 1022 hosts

I’ve pasted a chart below to act as a refresher:

CIDR Host Addresses Subnet Mask
/19 8192 (8190 usable)
/20 4096 (4094 usable)
/21 2048 (2046 usable)
/22 1024 (1022 usable)
/23 512 (510 usable)
/24 256 (254 usable)
/25 128 (126 usable)
/26 64 (62 usable)
/27 32 (30 usable)
/28 16 (14 usable)
/29 8 (6 usable)
/30 4 (2 usable)
/31 2 (P2P Only)


/!\ BATTLE TIP: /31 mask can, in fact, be used, as per RFC 3021

This feature has been supported since IOS 12.2T; BUT be aware it is designed to be used on point to point links. Lets think about what you lose going from a /30 to a /31: the network address and the broadcast address. If you’re using a point to point link or non-broadcast media, those addresses are wasted. So /31 will work best on serial links running something like PPP or HDLC, or Frame Relay. They can be used on Ethernet, but since Ethernet is a broadcast-based medium, I don’t recommend it.

So, why do I feel this is important? You’re a Level 5 Network Ninja, CCNA in your hand, burning for the blood of your enemies (or just the AT&T account rep that terminated service due to a small “glitch” in their billing system). You’ve learned every detail of subnetting; you can subdivide a Class C in your sleep, ready to engage…

But, unfortunately, while the various network exams may cover the minute details of protocols and configuration parameters, typically the design aspect…the ~why~ …is left to you to discover through a painful process trial and error (i.e. fix, rinse, repeat). Specifically in this case, the ability to assess an enterprise’s infrastructure and come up with an IP addressing scheme that is easy to manage, easy to route, and consistent across the entire domain.

Think for a minute. You have a college campus or an international retail network, all interconnecting with several global data centers, with multiple classes of traffic, larger sites that contain dozens of IDF’s aggregating via fiber to an MDF with 2 or more service providers, each of which tie into BGP clouds that you control. Oh and let’s not forget…all of this needs to be monitored and secured. How do you tackle this challenge?

Well, first and foremost, you need a consistent method to simplify the administration of the network, and to do that, you need a system that makes all of your network devices as easy as possible to identify, locate, and manage. One of the most critical ways you do this is with your IP addressing scheme:

  • You should be able to look at an IP address and know what it is and where it is.
  • You should use as few lines as possible to control access to and from specific networks
  • You should use as few entries as possible to build a concise, efficient routing table to any destination throughout your enterprise

Sure, if you have a couple branch offices and 100 users …that’s cake. But what about when you have 100 SITES, with voice & video traffic, PCI requirements, multiple 100+TB SAN/NAS devices mirroring across your private WAN that need to be collapsed onto the same core? Or even worse, what if you are the provider with different customer networks that all need to be segregated from each other?

So…rather than repeat the inadequate techniques practiced by all those non-jedi enfeeblings, spewing forth the same generic & over generalized “tips & tricks” …I instead am going to go over a specific case study. One that is based in the real world, and mirrors different aspects of networks I’ve engineered in the past. We will proceed in this exercise making design choices and explaining them as we go.

For that is how you must learn young padawan. You must observe live combat, watching the jedi’s tactics as he battles the forces of darkness, and eventually come to understand the techniques employed, use them, and make them your own. Every thrust and parry; every defensive stance and offensive strike, and above all, to preempt your enemy as he adapts to your fighting style.

To do any less is to overindulge the pedantic at the expense of the practical. And while I strongly believe in knowing your theory, theory alone will not determine the victor in combat.

Balance young Skywalker


The company we will be using is Cisco Jedi, LLC. A US-based retail company, with corporate offices in Los Angeles, Chicago, and New York, as well as 2 co-located data centers (one local to HQ in Los Angeles, and the other, functioning as a DR site in Scottsdale.) Additionally, they have their own retail chain of 400+ stores across US & Canada, with plans to expand another 80 stores, including expanding into EU and South America, by the end of 2015.

Cisco Jedi Network

Cisco Jedi Network


All sites are interconnected by a private MPLS cloud through Verizon, running BGP to redistribute each site’s private networks. They connect over 100mb Ethernet loops that are rate limited down accordingly at each site. Los Angeles branch is using a Cisco 2821 and is rate-limited down to 50mb/s, while the New York and Chicago branch offices are using somewhat newer 1921’s but are rate-limited down to 20mb/s.  Both data centers are running at the full 100mb/s and connect through Cisco 3845.

Corporate HQ in Los Angeles is divided between 2 main sites: Site A and Site B. Both connect to each other by a point to point Cisco 1410 wireless bridge running at 54mb/s over 802.11g. Site B is the warehouse & distribution center which sits approximately 100 meters from Site A, the corporate office containing HR, Operations, Finance, Production, Marketing, and Design departments.

NOTE: Site B (the warehouse) is NOT directly connected to the MPLS network, but rather accesses internal applications & services through its wireless P2P bridge. Contrarily, Site A (HQ) is not directly connected to the internet, but rather connects through the warehouse, traversing the wireless bridge as well.

This is your network.

/!\BATTLE TIP: There are no small amount of considerations as you examine this network. Your mind should look at this topology and attempt to understand design choices & the challenges surrounding them. For instance, the latency between the two main corporate sites over the wireless bridge; esp considering if they have IP phones in the warehouse. You should ask yourself, why was it setup this way? Why not an internet connection and/or MPLS connection at both locations? The two culprits that should immediately come to mind are ISP availability and money. Also, take note the retail sites. These are templatized setups in which the stores internet access all goes through a centralized choke point. Again, analyze why. This being a retail company with PCI requirements, this would allow easy control & restriction of traffic in or out of the retail network. Clearly internet access is needed (otherwise it wouldn’t be provided), more than likely for some type of cloud-hosted application, be it for document collaboration, payroll, or email.

The end goal of this IP scheme is to provide us with a consistent structure that in some way simplifies the massive administrative burden of managing a network. Below I will present the solution, and work backwards to explain these design decisions.


Because of the large number of retail sites that need to be on the network, I’ve elected take our addressing scheme from the supernet addressing space, and will adhere to the following format:

10 . <Site ID> . <VLAN##> . X


10 Los Angeles Corp & Warehouse
12 Chicago Branch
14 New York Branch
200 Los Angeles Data Center
220 Scottsdale Data Center
100-110 Retail Sites**
255 MPLS/BGP Core


Each site can be summarized to a 10.##.0.0/16 address. For example, any device located in the NY branch will be somewhere in the network. Anything in the Scottsdale data center will be in

Store sites require a little more consideration, especially since there are more than 254, we cannot easily summarize the Site ID to just the second octet. Furthermore, each store will need far fewer devices than any of the corporate locations. My recommendation is to assign each store its own /24 class C subnet. However, in doing so, you still need to be able to associate the Store # (assigned by operations) and correlate to a network address. Speaking from experience, it’s highly desirable for all the stores’ subnets to be adjacent to each other to allow for easier route summarization & access control list management. The list below defines the addressing template we will use for the retail environment:

Store 1-199 10.100.(1-199).x
Store 200-399 10.102.(0-199).x
Store 400-599 10.104.(0-199).x
Store 600-799 10.106.(0-199).x
Store 800-999 10.108.(0-199).x
Store 1000-1199 10.110.(0-199).x



Store 22        10.100.22.x/24

Store 122       10.100.122.x/24

Store 222       10.102.22.x/24

Store 522       10.104.122.x/24

Store 1222      10.112.22.x/24


8 Standard Corp Users 10.##.8.0/22 1022
16 Design/Graphics 10.##.16.0/24 254
24 Finance/Credit 10.##.24.0/24 254
32 Voice 10.##.32.0/22 1022
40 Video/Presence 10.##.40.0/22 1022
48 Wireless: Corp 10.##.48.0/22 1022
64 Warehouse User 10.##.64.0/22 1022
72 Wireless: Warehouse 10.##.72.0/22 1022
80 Warehouse  Sorting Systems 10.##.80.0/24 254
88 Guest [Internet Only] 10.##.88.0/22 1022
100 Servers 10.##.100.0/22 1022
104 ESX/vMotion 10.##.104.0/24 254
108 Storage 10.##.108.0/24 254
200 Mgmt/ILO/ Monitoring 10.##.200.0/22 1022
8XX DMZ 172.22.XX.0/16 255 Class C Subnets
999 MPLS/BGP** 10.255.##.0/16 65535 BGP Loopbacks

**See MPLS/Core Section Below


100 Servers 10.##.100.0/22 1022
104 ESX/vMotion 10.##.104.0/24 254
108 Storage 10.##.108.0/24 254
200 Mgmt/ILO/ Monitoring 10.##.200.0/22 1022
8XX DMZ 172.22.XX.0/16 255 Class C Subnets
999 MPLS/BGP** 10.255.##.0/16 65535 BGP Loopbacks

**See MPLS/Core Section Below


8 Standard Corp Users 10.##.8.0/22 1022
16 Design/Graphics 10.##.16.0/24 254
24 Finance/Credit 10.##.24.0/24 254
32 Voice 10.##.32.0/22 1022
40 Video/Presence 10.##.40.0/22 1022
48 Wireless: Corp 10.##.48.0/22 1022
88 Guest [Internet Only] 10.##.88.0/22 1022
100 Servers 10.##.100.0/22 1022
200 Mgmt/ILO/ Monitoring 10.##.200.0/22 1022
999 MPLS/BGP** 10.255.##.0/16 65535 BGP Loopbacks

**See MPLS/Core Section Below


By now it should be apparent that this company operates with two main paradigms: the corporate environment and the retail environment. They both have somewhat similar needs, however each has its own challenges and requirements.


Let’s start with the Retail network. We need something easy, and something that scales—the company is already at 400 stores, and given their expansion plans, you should be prepared to grow to 1000+ over the next 5 years. Furthermore, to control routing updates, ACLs, NAT statements, etc, its best if these addresses are contiguous so the entire retail space can be easily summarized. Again, each store will be given its own /24 subnet for such things as registers, wireless devices, management stations, printers, IP phones, etc.


This is why subnetting is so critical. I had given a similar exercise to one of my employees, and below is the scheme he came up with.

10.100.(1-99).x = Store 1-99 data 10.100.(101-199).x = Store 1-99 voice
10.101.(0-99).x = Store 100-199 data 10.101.(100-199).x = Store 100-199 voice
10.102.(0-99).x = Store 200-299 data 10.102.(100-199).x = Store 200-299 voice
10.103.(0-99).x = Store 300-399 data 10.103.(100-199).x = Store 300-399 voice


Note it’s not necessarily “wrong.” It certainly takes into account separating voice traffic, and overall not a bad solution. However, using TWO class C’s for one retail location; ask yourself the question, do you really think a store will need even 254 devices (much less 510)?

Also consider this is a retail company, whose network must be governed (partially at least) by PCI compliance. Translated: your POS registers need to be on a separate VLAN. Add to that PCI requirements for quarterly wireless scanning, and the fact that the entire earth is using iPads for everything from credit card scans to open heart surgery, you might as well come to grips each retail site will need to be segmented across several VLANs. In light of these considerations, below is template for subnetting each store’s /24

VLAN 10 POS X.X.X.0/26 X.X.X.1 .2 – .62
VLAN 20 WIRELESS X.X.X.64/26 X.X.X.65 .66 – .126
VLAN 30 VOIP X.X.X.128/26 X.X.X.129 .130 – .190
VLAN 40 CORPORATE X.X.X.192/26 X.X.X.193 .194 – .254


This design will accommodate 62 POS registers, 62 wireless devices, 62 devices on the internal corporate network, and 62 devices on the VoIP network, all within a single /24.

So at the end of this setup we’ve set the stage for several things

  1. We can summarize the entire address space with a single ACL statement: 10.100.X.X/12. For instance, if the internet connection at the Scottsdale data center fails, it is a simple matter of modifying a single entry to reroute traffic for all retail locations out of the Los Angeles data center. In fact, a clever admin could have IP SLA setup and/or floating static routes to automatically handle that failover.
  2. We’ve allowed each store to be easily summarized to a /24, and effectively used its space in an easily templatized manner.
  3. Because we have several VLANs at each store, we can treat respective traffic differently. We can prevent the POS subnet from accessing other stores on the MPLS cloud, while allowing the VoIP and Corporate subnets to not suffer this restriction. We can police & prioritize traffic relatively easily based on a device’s VLAN membership.



Next, let’s look at infrastructure VLANs: server, vMotion, storage, and management. Again, consistent and easily recognizable. If you see is syslog or event entry somewhere, you instantly know that this IP belongs to a server located in the Scottsdale data center. The separation of these functions allow you to treat each VLAN differently. Specifically

  • Storage & vMotion traffic should never go across the WAN
  • IP based storage traffic can experience increased performance by using 9KB jumbo frames, esp if 10Gb is not yet installed and you must milk every ounce of speed that you can.
  • The management VLAN needs to function as a privileged network for the purpose of troubleshooting & diagnostics, and as such should not be subject to the same access lists and firewall rules as production networks. Furthermore, since this would also be used for monitoring, heavy amounts of SNMP and NetFlow are required. These can be isolated, and if necessary prioritized down so as not to interfere with network traffic that is business critical.



Lots of subnets here! As with the retail network, separation of these different VLANs allows different prioritizing and security filtering. And again, the management network is given more privileged access for the purpose of troubleshooting.

You should observe a few deliberate design choices right off the bat:

First, there is a correlation between the VLAN # and the third octet of the subnet. While this is not required, when dealing with a network this large, having consistency greatly simplifies the administration and maintenance.

Second, subnets and VLANs are aligned on the mask boundary: 4, 8, 16, etc. This allows for maximum agility. The design department right now is only using a standard class C mask. What for some insane reason they hire another 100 designers? Or decide that every designer needs an additional workstation? Well, we have the space available. Simply go to your DHCP server and update the mask; you have plenty of room to grow. The same is true if you need to insert or subdivide a larger address space into a group of smaller subnets.

/!\SECRET TACTIC: There is a second reason for doing this, however it is far less known. Modern layer 3 devices are moving away from the classic RIB/FIB framework, and now compile their entries into specialized TCAM tables, each with an associated VMR (Value Mask Result). Thus, rather than robbing CPU cycles to perform a sequential lookup, parsing down multiple tables, one entry at a time, these TCAM tables compare a packet to all its entries in parallel! However, like everything else in networking, they live in a binary universe. As a result, entries that can be easily written on binary borders increases the efficiency of how these entries are compiled in the TCAM tables. Much the same way summarization increases efficiency for the routing engine, TCAM tables aggregate entries for multiple scanning engines (routing, ACL, NAT, QoS, etc). Consequently, having fewer entries means a smaller TCAM, which in turn means faster lookups. Thus it behooves the us to utilize prefix alignment as we design our addressing space.


As I’m sure you remember from your CCNA, best practices recommend using a loopback interface as the source/destination for all your routing updates. I’m not going to elaborate on the pros & cons of doing so in this article, but let’s assume for now you’re going to employ this best practice. That being the case, each site’s MPLS router will have a single /32 address in this range, as per below

Store 22
Store 122
Store 222
Store 522



Anything that is publicly accessible will go in this range. However rather than a single dumping ground, it’s probably more prudent to subdivide it as necessary. With this range we can have 255 distinct class C subnets, which should be more than adequate. To keep management as simple as possible, we can correlate the VLAN with the subnet


Email               VLAN 808 – 172.22.8.X/24

eCommerce    VLAN 820 – 172.22.20.X/24


As you can see, having the design laid out before you allows for more thorough comprehension of how to tackle such a daunting task. Not all networks are the same, and there is no one plug & play solution. Take this exercise and experiment with it; modify it to fit your existing network.

The hope is that by understanding the methods you can take this process of analysis, compare it to whatever infrastructure is laid before you, and still meet the objectives that are desirable from all networks:

  • Scalable – a network that can grow with your business without the need to be completely restructured
  • Flexible – a network that is agile enough to adapt to changes in the business, as well as the infrastructure
  • Manageable – a consistent, transparent network that is no more complex than it needs to be

As I said earlier in this article, I’ve wanted to write this topic for some time. I spent a good hour googling for relevant articles; and while I found some decent tutorials on TCP/IP in general, I found nothing that would help an up & coming jr. network administrator to design an address space for large enterprise-class network.

I hope this has been helpful in your fight against the forces of darkness (and incompetence)

Thanks for reading!