Technology Considerations
The selection of a suitable WLAN technology was an easy one. As
the world's leader in the manufacture of enterprise-class WLAN equipment, Cisco
did not have difficulty in choosing the products to deploy. Cisco did, however,
need to define, deploy, and provision a robust end-to-end solution.
Architecture Principles
When considering the architecture of your WLAN, your assessment
must encompass many points. This section examines some of the factors that
affected the enterprise WLAN deployment at Cisco Systems, as follows:
-
Topology
-
802.11 wireless networking standards
-
Client-to-AP ratio
-
Signal strength
-
Roaming
-
Radio cell architecture
-
Global naming standards
-
Cisco Aironet access points
-
Cisco Secure Access Control Server (ACS)
Topology
Early in the planning stage, the Cisco IT WLAN Architecture
team decided that the WLAN would be a secondary network complementing the
existing wired network (that is, a separate "overlay" network). Each large
building would use a single Layer 3 domain within each building to help ensure
session integrity for wireless devices moving within or between floors.
Effectively, each building had a unique wireless subnet, where both the access
points and the wireless devices shared IP addresses from a common Class C
address pool. However, in line with prudent IP address management, smaller
buildings with fewer than 20 or 30 users shared a common VLAN for both wired and
wireless devices.
Additionally, at the time of deployment, the Cisco Aironet
product line was based solely on a distributed, autonomous access point (or
so-called "Intelligent AP") model. Each access point was a unique, managed host
with full intelligence and configurability. As such, the current global WLAN is
a distributed model with over 3000 intelligent IOS access points in production.
Figure 9-1 shows a basic topological diagram of the initial enterprise WLAN. The access points are connected directly to standard
Layer 2 switches, and network management is provided by the Wireless LAN
Solution Engine (WLSE) and the internally developed Enterprise Management (EMAN)
toolset.

In 2000, the architecture standard called for Cisco Aironet 350
Series access points to be connected to the nearest access-layer switch, as
shown in Figure 9-2. A separate cable
provides console access to each access point to mitigate a loss of network
connectivity, a practice that Cisco IT has standardized for all network devices.
The console network is used for out-of-band (OOB) network management,
configuration, and troubleshooting. Figure
9-2 shows how each access point is connected to the production data network
and via a separate cable to the console network.

Because of ongoing developments in WLAN technologies, Cisco
decided to redesign its enterprise wireless network in 2005. This project, known
internally as the NexGen WLAN, will feature a combination of autonomous
(IOS-based) access points and new centrally managed (LWAPP-based) access points,
controlled and managed by WLAN controllers. Further information on the Cisco IT
strategy can be found in the section "What the Future Holds" later in this
chapter.
Note
Lightweight Access Point Protocol (LWAPP) is a protocol used to
allow WLAN controllers to configure, manage, and control access points in the
Cisco Centralized WLAN Solution. LWAPP introduces a split
MAC, which allows real-time frame exchange and certain real-time portions
of MAC management to be accomplished within the access point, while WLAN
controllers handle authentication, security management, and mobility.
More detailed information on LWAPP and the Cisco Centralized
WLAN Solution can be found at http://www.cisco.com/en/US/netsol/ns340/ns394/ns348/ns337/networking_solutions_white_paper0900aecd802c18ee.shtml
or by going to Cisco.com and searching for "Understanding the Lightweight Access
Point Protocol (LWAPP)."
802.11
Wireless Networking Standards
At the time that the architecture team was designing the global
WLAN, the only ratified standard was 802.11b, providing raw data rates of up to
11 Mbps in the 2.4-GHz frequency range. Therefore, this standard was adopted for
the global enterprise wireless network.
Based upon internal Cisco IT policies and procedures, new
products and standards must first undergo prudent and comprehensive testing and
certification before they are used in the production environment. Shortly after
ratification, the 802.11g standard was internally certified for use within Cisco
by the architecture team. Today, therefore, Cisco is deploying 802.11g Cisco
Aironet access points and client devices in its WLANs. 802.11g was selected over
802.11a because it also works in the 2.4-GHz frequency band and therefore offers
seamless backward compatibility with the existing 802.11b network. Apart from
limited lab and showcase sites, the 802.11a standard was not deployed in a
widespread manner, but it will form part of the NexGen WLAN that is currently
being designed by the architecture team; see the "What the Future Holds" section for
more details.
Although 802.11g supports data rates of up to 54 Mbps in the
2.4-GHz band, these higher speeds are only available to 802.11g clients.
Furthermore, 802.11g access points "step down" their speed when older 802.11b
clients are associated to ensure backward compatibility. As such, it is not
uncommon to find many 802.11g access points working at a maximum of 11 Mbps
(effectively in 802.11b, or "legacy" mode). Such circumstances will decrease as
the number of older 802.11b clients diminishes in line with the introduction of
new laptops and replacement of the older devices.
Client-to-AP Ratio
After careful traffic analysis, Cisco IT built its architecture
on a user-to-AP ratio of 25:1 would provide acceptable performance. At that time
(early 2000), it was deemed unlikely that all 25 users would be accessing the
WLAN at the same time and even more unlikely that they would all be
simultaneously sending or receiving large amounts of data. Because the WLAN was
an overlay network, those users who needed to use bandwidth-intensive
applications such as network backups or video streaming were encouraged to use
the wired network and not depend on the wireless network for these
functions.
However, Cisco IT has found that adoption has been extremely
high. Within 12 months of deployment, Cisco IT commissioned an internal "Voice
of the Client" survey, which showed that 92 percent of staff were using the WLAN
on a weekly basis; furthermore, 27 percent of users were relying upon the WLAN
as their "primary or only network access medium." Even with the limitation of
the 802.11b data rate of 11 Mbps (or actual throughput of 6 Mbps), day-to-day
performance has not been adversely affected and is deemed perfectly acceptable
for the vast majority of user activity. Comments from users have been
overwhelmingly positive.
Some Cisco buildings use wireless connectivity almost
exclusively. This includes network backups, software downloads, video unicast,
and Cisco IP Communicator (a software-based IP phone), in addition to standard
web browsing, e-mail, and calendars. Rich Gore, Cisco IT
project manager, says, "With quality of service now supported over wireless,
I've been taking all my phone calls over the wireless network using Cisco IP
Communicator, and it's been working perfectly."
Note
Users always have the option of manually connecting their
laptops to the wired network if they so wish, but this practice is by no means
standard for most users.
Moving forward, a lower user to AP ratio (approximately 12:1)
has been recommended as reliance upon the WLAN increases and adoption has proven
to be widespread. This topic is covered in more detail in the "What the Future Holds" section later
in this chapter.
Signal Strength
Cisco Aironet access points can broadcast up to 100mW
(depending on the regulatory domain). When such high transmission power is used,
it is possible for the WLAN coverage to extend beyond the originally desired
areas, potentially reaching out into parking lots and public areas. After
conducting tests, the architecture team established standards that call for
using the minimum power to reach all areas within buildings, but never exceeding
20mW. That is, the "less is best" approach is taken. Access points are ideally
configured to use 1mW, 2mW, 5mW, and so on, but never more than 20mW.
In some instances, directional antennas have been used to more
narrowly focus the signal, reducing the power required to achieve full coverage.
Where necessary, rather than increasing transmit power to exceed 20mW,
additional access points are installed to cover "dead" spots.
Roaming
To more accurately control roaming, the WLAN client software
(in this case, the Cisco Aironet Client Utility [ACU]) was
configured to roam only under certain circumstancesthat is, when the current signal strength has dropped below a specified threshold or
number of retries. This configuration
reduces the tendency to reassociate to a new access point and helps avoid
flip-flopping.
Each time the user switches from one access point to another,
connectivity is momentarily lost, necessitating reauthentication. Numerous
reauthentication requests can increase load on the authentication server, which
can adversely affect service. This situation can be particularly notable in
wireless voice applications, with clearly discernable "stutter" as the client
reassociates and authenticates.
Radio Cell
Architecture
If cells overlap too much, continual switching
("flip-flopping") is possible. Cisco adopted an overlap of about 15 percent
(roughly 10 feet in most buildings) to minimize this possibility.
As we mentioned in Chapter 5, the 802.11 standard allows for devices to
connect at various data rates depending on the RF environment. To minimize this
effect, the architecture team locked the data rate at 11 Mbps. Thus, the user's
wireless connection will never "step down" but rather will associate to a
different access point when it is far enough away from the original access
point. This solution controls the roaming and avoids flip-flopping between
access points, which in turn greatly assists in troubleshooting and predicting
client behavior.
The policy for 802.11g cells is to permit data speeds as high
as possible, but never less than 802.11b (11 Mbps). This results in the ability
of newer 802.11g clients to associate with the latest model 802.11g access
points at higher than 11-Mbps speeds in some circumstances. However, association
rates at lower than 11 Mbps are never permitted.
Global Naming
Standards
Cisco uses a clear, concise, and consistent naming standard for
all access points. This standard aids greatly in troubleshooting and also
provides users and network engineers with useful information about their current
access point.
The naming standard is as follows:
<site name>-AP<floor><AP
letter>.cisco.com
For example, for the third access point on the second floor of
a New York office, the access point name could be NYC-AP2c.cisco.com
Cisco IT has found that a consistent naming standard allows for
easier management.
Cisco Aironet Access Points
When originally deployed, the Cisco Aironet 350 Series Access
Point was selected as the standard access point. The Cisco Aironet 350 Series
was the most advanced, fully featured wireless access point available. It
supported the 802.11b protocol standard (the most advanced at that time), which
provides data rates of up to 11 Mbps. The Cisco Aironet 350 Series also
supported inline Power over Ethernet (PoE), which greatly simplifies
installation and reduces costs by eliminating the need for separate, dedicated
power cabling to the main supply.
PoE allows the access point to draw power through its Ethernet
cable, from the switch to which it is connected. In some circumstances, where
certain sites did not have switches that supported PoE, Cisco IT used standalone
"power injectors." These devices sit inline between the network switch and the
access point and "inject" DC power into the cable. This allowed Cisco IT to
continue using PoE at all locations, even where they had older switches that did
not provide PoE or that did not have sufficient power capacity to power all the
access points required. Figure 9-3 shows
how a power injector sits in between the access point and the switch.

Today, Cisco IT is expanding and enhancing its initial Cisco
Aironet 350 Series deployments by installing Cisco Aironet 1000, 1100, and 1200
Series access points. These access points support new 802.11 standards and
additional feature enhancements and options for modular and flexible WLAN
deployments, including the centralized, controller-based architecture or the
distributed autonomous access point architecture. At the time of writing,
approximately 25 percent of the access points were the 1200 series. This
percentage will rise to 100 percent with the NexGen WLAN.
Cisco Secure Access Control Server
(ACS)
The Cisco Secure ACS is used as the standard AAA server for the
global WLAN and for other recently introduced services such as 802.1x-based port
authentication for wired Ethernet ports in public areas and Network Access
Control (NAC), part of the Cisco Self-Defending Network security strategy. Pairs
of Cisco Secure ACSs were deployed at strategic locations worldwide.
The value of using a globally distributed AAA architecture
instead of a single AAA server was highlighted by the WLAN deployment. Because
of the greater load that a WLAN creates for AAA, due to authentications and
reauthentications (as the client device roams from AP to
AP), it was important to ensure that all users did not have to rely upon a
single, centralized server. This would have introduced
unacceptable delays for users in geographically remote areas. As such, at 13
different locations around the world, Cisco placed two ACS servers, in a
load-balanced configuration, that served as AAA servers for that local
geographical region.
The ACS servers are fully integrated with the Cisco Active
Directory domain structure, enabling a single sign-on
(SSO) capability. Effectively, AD user credentials are used
not only for access to their laptops and wired network but also to provide
transparent authentication to the wireless network. SSO has greatly reduced the
client impact for users and has helped ensure a common, user-friendly experience
across platforms and transport media. Users need only remember their normal ID
and password for access to their laptop, the wired network, and the wireless
network, and they only have to enter their credentials once each session
regardless of the transport medium they are using.
Network Management
To date, more than 3100 Cisco Aironet access points have been
deployed worldwide, supporting more than 50,000 users. This includes over 37,000
full-time Cisco employees, as well as over 10,000 temporary, contractor, and
vendor staff. A WLAN as widely used as this requires a robust management
capability. Because a dedicated wireless management system was not available in
2000, the Cisco wireless network was managed through EMAN, an internally
developed web-based enterprise-management framework. Today, Cisco IT also uses
the CiscoWorks WLSE, a Cisco appliance for managing WLAN deployments.
Client Management
Client management is a challenging area, and Cisco has
implemented robust business processes to address it. Before 2004, all client
devices were based upon Cisco-manufactured client adaptors, radios, and devices.
However, the Cisco Client Extensions (CCX) is a technology
licensing scheme that allows third-party manufacturers to produce equipment that
supports Cisco value-added capabilities. With CCX, many
third-party client devices and platforms have been introduced within the
production environment.
To address this issue, Cisco made the decision to adopt
third-party wireless software for all platforms. This adoption ensures that a
common software application is used for all operating systems (Windows 2000,
Windows XP, Linux, MacOS, and so on), regardless of the particular adaptor used
in the relevant laptop (Cisco adaptors, Intel Centrino laptops, Macintosh
PowerBooks, and so on).
The third-party supplicant also provides a consistent
management toolset to allow for centralized profile management and
configuration.
A centralized client management solution is also used to
facilitate software distribution and updates.
Service dashboards, which are internal intranet websites, also
provide service information, user communication, software, and self-service
configuration utilities for all users. All Cisco staff can use dashboards for
instructions on how to manually configure or update their systems. Because
dashboards are based on standard HTML pages, they are platform agnostic and
suitable for all platforms and clients that support HTTP.
Service and Support
Network devices, systems, and applications on the Cisco global
network are managed according to levels of impact to the business. Service or
support levels fall into four categories:
-
Priority 1 (P1) Immediate and severe business impact
including revenue loss (actual, not postponed); inability to make or ship
product; inability to develop code or product; inability to meet contractual,
legal, or government-imposed processing deadlines; impact to external Cisco
customers, partners, or supplier processes with negative implications for
relations, market perception, or revenue; or engineering groups unable to work
on a critical customer build or fix other critical account issues.
-
Priority 2 (P2) Adverse
business impact including the inability of an organization (or organizations)
within Cisco to perform daily operations such that it is essentially idle; or
direct and critical impact to executives within the company, or to development,
test, disaster-recovery, or staging environment for a P1 service or system.
-
Priority 3 (P3) Low business
impact including the inability of multiple users to perform their daily tasks
such that they are essentially idle; or impact to a single user under an
approved, documented Service Level Agreement (SLA) requirement, or to a
development, test, disaster-recovery, or staging environment for a P2 service or
system.
-
Priority 4 (P4) Minor or no
business impact to Cisco such as a question or new service request, or a problem
that keeps one employee from performing part of a job function.
Within this support-level structure, Cisco Secure ACSs are
managed as a P1 device because they are critical not only for WLAN access, but
also for NAC, an element of the Cisco Self-Defending Network security strategy.
The wireless network was originally managed as a P4 because it was considered a
secondary network to the wired network. However, because of widespread adoption
and usage within Cisco, support for the WLAN has become equivalent to P2. Cisco
envisions that the NexGen WLAN, based upon more advanced and intelligent
wireless networking technologies, will be formally supported on a P2
basis.
Cisco Support
Team
Cisco has a four-tier support model, as follows:
-
Tier 1: Frontline Global Technical Resource Center
(GTRC) This is equivalent to a standard internal helpdesk. Agents are
familiar with the most common problems and work from prepared scripts and
troubleshooting guides. Each GTRC hub has a nominated wireless LAN expert who is
more familiar with the solution than his colleagues.
Cases that are handled at this level are usually client
configuration issues or the initial reports of service outages. Problems that
cannot be solved by the GTRC are escalated to Tier 2 support.
-
Tier 2: Cisco IT WLAN network
operations team These Cisco IT engineers are responsible for ongoing
network and infrastructure support. The WLAN subteam is made up of engineers who
usually have several years of experience supporting the solution and, in many
cases, were directly involved in the original deployment and design. The IT WLAN
network operations team has access to the access points, switches, routers, AAA
servers, and WLAN controllers that make up the solution. This team also includes
virtual members from the Cisco dedicated security organization and hosting teams
(responsible for the AAA and Active Directory servers).
Cases that are handled at this level are usually AP or
controller configuration issues, service outage problems, requests for enhanced
coverage, and so on. Problems that the IT WLAN network operations team cannot
solve are escalated to Tier 3 support.
-
Tier 3: Cisco IT WLAN architecture
team The IT WLAN architecture team is made up of several senior design
engineers and solutions architects. Members of this team designed the original
solution and have continued their work on evolutionary change and development
over the past five years. This team holds the most technical, business,
security, and program management experience on the Cisco solution.
Cases that are handled at this level are usually fundamental
design or architecture issues, requests for new services or capabilities, and
new product or solution implementation. If a problem cannot be handled at this
level, it is usually a result of a product bug and is escalated to Tier 4
support. This is a rare occurrence because most issues that are escalated this
high relate to solution development rather than bug fixes.
-
Tier 4:
Technical Assistance Center (TAC) and Wireless Networking Business Unit
(WNBU) The TAC is the top level of support within Cisco and for Cisco
customers. Cisco IT can also escalate directly to the WNBU
within Cisco. Only officially noted bugs are escalated to this level.
A team of three and a half full time equivalent (FTE) staff
makes up the Tier 2 IT WLAN network operations staff. Note that this effort is
spread over several people in several countries but that the combined total is
equivalent to 3.5 FTE.
A team of two and a half FTE makes up the Tier 3 IT WLAN
architecture team. This includes the global program manager responsible for
enterprise wireless strategy and architecture.
Cost of
Support
Cisco prices each GTRC support call at US$25 per call. This
results in annualized cost of frontline Tier 1 support of US$318,900.
Cisco budgets US$120,000 per annum as the fully loaded cost of
an FTE. This cost includes salary, assets, workplace costs, business costs, and
so on, and is not indicative of salary alone. This results in annualized cost of
second-line Tier 2 support of US$420,000.
Because of the nature of the Cisco business and the maintenance
of a Tier 3 architecture team, Cisco does not include these costs in the
day-to-day annualized support costs. Cisco believes the maintenance of a
dedicated architecture team is not indicative of a typical enterprise because
not all corporations are based in the networking industry.
This results in a total annualized cost of support as reflected
in Table 9-1.
Table 9-1. Cost of Support
|
Level of Support |
Cost |
|
Frontline support |
$318,900 |
|
Second-/ Third- line support |
$420,000 |
|
Total annual support costs |
$738,900 |
|
Annual support cost per user (50,000 users) |
$14.77 |