The LightReading blog, Open Networking Acronym Soup, covers all the interest groups, communities and standards bodies that are driving this idea of Open Networking, which in itself is a grab bag of topics around SDN, NFV and of course white box/bare metal switches. A recent blog post struck a chord with me at first because the author, Marc Cohn, is a good guy and a friend.

But secondly, and more importantly to everyone else, is to point out his astute observation that “we” (people, users and vendors) try to simplify stuff by using acronyms. I agree. In my past job at Infoblox, people always wanted to know what DDI meant, I would reply in my standard excited way “DNS, DHCP and IPAM’’ and most would agree that DDI was easier to say. So let’s take a look at the acronym soup and examine several key factors that you should know about white boxes. And I will lay them out here and try to keep it simple and break the list into two sections, what you should know now, and what you need to keep an eye on…for now.

OCP – Open Compute Project – This is an organization driven by Facebook. The end game is to foster a community that uses all the same tools and tricks to make any switch operating system (OS) operate with any bare metal switch. While certainly a lofty goal, the last OCP event was the best-attended ever with a host of startups and many key players involved, including Dell, HP and Juniper.  The objective is to create a plug-and-play ecosystem, where you buy an OCP switch, and load on an OCP operating system—and bam—it just works.

ONIE – Open Network Install Environment – ONIE is an open source “install environment” that acts as an enhanced boot loader utilizing facilities in a Linux/BusyBox environment and was officially adopted by OCP in 2014. This small Linux OS enables end users and channel partners to install the target network OS as part of data center provisioning in the fashion that servers are provisioned. Most, if not all, of the white box makers are adopting ONIE. You should make sure you have ONIE on board the bare metal switch you buy if you want to try more than one OS.

ASIC – Application Specific Integrated Circuit – Sure, I bet you all know this one….This is one of the key components that makes a switch a switch and different from a CPU-driven server. Switches have CPUs as well of course. The ASIC has the hardware features that drive functionality at scale. For example, you don’t just want a line-rate Gigabit Ethernet port, you also need a line-rate port with wire-speed access control lists (ACLs) or quality of service (QoS) marking functionality, and that functionality is baked into the ASIC.

ZTP – Zero Touch Provisioning – ZTP has been particularly useful for data center servers, where scale and configuration similarities across systems make automation a necessity. In the server world, the Linux-based OS has revolutionized on-boarding and provisioning. Rather than using command-line interfaces (CLI) to configure these systems individually, administrators can use automation tools to roll out the OS software, patches and packages on new servers with a single command, or the click of a mouse. Now you have ZTP on many switch platforms.

WB or BMX.  Yet more acronyms for white box and bare metal switches.

Developments to Watch over the Next Year

ONL – Open Network Linux – ONL was recently adopted by OCP as a standard Linux distribution for all bare metal switches with apparent support from many white box makers. With the rise of cloud and DevOps methodologies, we’re seeing increased interest in network disaggregation. End users, especially organizations where Linux has been widely adopted, can derive significant operational efficiencies by running Linux on a variety of network hardware. Supporters of ONL ensure that the open networking community can easily leverage bare-metal switches and explore the range of benefits that network disaggregation can offer by standardizing on one Linux distribution. I agree; it keeps it simple. ONL is exactly that idea.

ACPI – Advanced Configuration & Power Interface – Derived from the PC industry, this approach is currently being fostered in the OCP and is widely used in the server/desktop/laptop industry. The idea here is that even if you have the hooks to the CPU and the ASIC, you still need to make sure the fans, sensors and lights on the box are functioning as expected after you port a new OS to your device. So there is considerable action behind the scenes to port to a new “box,” even if the OS works on another box with the same exact ASIC and CPU. Advocates maintain that eventually hardware compatibility lists will go away, and when you put an OCP OS on an OCP bare metal switch it consistently works without much fanfare.

SAI – Switch Abstraction Interface – This is a recently initiated OCP project to drive a consistent API framework across different ASICs. Today each ASIC manufacturer has its own API set, which makes it difficult for end users to adopt them in a meaningful way. Of course you don’t want to be an ASIC engineer or have to build your own entire switch, but you may want enough functionality to adjust aspects, such as counters or the packet processing pipeline for unique traffic patterns that are indicative of your environment.

Ok that is a decent list of key acronyms. Share some in the office with your team mates, or impress your friends at your next cocktail hour!  In the meantime, stand by for more blogs to come on white boxes.


Establishing the Big Data Connection

Establishing the Big Data Connection

Many network vendors will tell you that their network equipment is built for Big Data. However, once deployed, do you have enough Big Data context to effectively monitor, troubleshoot, triage and tune your network? In most cases the answer is no! When designing and deploying a network, administrators must consider whether this network will provide enough Big Data context?

Before we go any further let’s define BIG DATA context.

BIG DATA context is the ability to correlate Big Data events and protocols back to network events and protocols and to be able to classify BIG DATA network flows correctly. To establish the Big Data Connection, we’re going to discuss the requirements to ensure a network is in the class of networks that have Big Data context, how administrators can possibly achieve this, and the role network programmability and agility play in this discussion.

Now let us see how we can build BIG DATA context and act on it.

Building Big Data Context
Network monitoring, tracing, visibility and reporting with Big Data context is accomplished with network equipment that is able to export flow statistics, counters and flow DBs and leverage open systems to classify such flows using Big Data heuristics. Pica8 can easily export flow statistics with sophisticated match rules and since all of its solutions come prepackaged on Open Platforms using Linux-based Broadcom chipsets, those communities can be leveraged for best-of-breed flow classification applications that can be used on flow data, statistics and counters.

Once we have built Big Data context, it becomes easier to tackle network programmability and agility so that network actions can be more proactive and adaptive and less reactive to Big Data events.

Network Programmability
Network programmability is a much used but often misunderstood term, for example programmability is NOT configuring the network with automation tools as some people think. For organizations running Big Data workloads, Network programmability is the ability to recognize Big Data flows and specify policies at different points in the network to enable:

  • Re-routing of flows
  • Rate-limiting or throttling of flows
  • Movement of flows permanently or temporally via access control

Programming these tasks is easy to do with network controllers, such as the OpenDaylight controllers and Pica 8 switches, which can be deployed in different parts of your network and can quickly provision policies in your network to adapt to and react in real-time to Big Data events, such as replication, node addition and node deletion.

Sample Use Case
If you want to monitor the data flow between two vnodes in a RIAK cluster and move them to a less-preferred path if the data amount goes over a certain threshold (~1GByte in this example). I add some sample code here in Python using a RESTful API to implement this use case.

class ControllerClient:

   default_params = {‘format’:'json’, ‘v’:’1.0′}

# connect to server – on localhost for illustration

   server= ‘′

   def __init__(self,server=”):

       if server != “”:

          self.server = server

  def getRequestURL(self,method,params={}):

        requestURL = self.server+’/'+method

        call_params = self.default_params.copy()


        requestURL += urllib.urlencode(call_params)

        return requestURL

  def sendAPIRequest(self,method,params={}):

       data = {‘bytes’:0, ‘flow_id’:0}

       url = self.getRequestURL(method,params)

       f = urllib.urlopen(url)

       data = json.load(f)

def main():

    client = ControllerClient()

    data = client.sendAPIRequest(‘network/analytics/flow_counter_get\

                                                          ?switch_id=sw_id \

                                                          &eth_type = 0×800\

                                                          &src_ip= \

                                                          &dst_ip= \

                                                          &dst_port=5 \


    if data['bytes'] > 1000000000:

      flow_id = data['flow_id']

#reroute flow

     data = client.sendAPIRequest(‘network/router/setflow\


                                                           &eth_type=0×800 \

                                                           &src_ip= \

                                                          &dst_ip= \

                                                          &dst_port=6 \


 if __name__ == “__main__”:


For the curious there are plenty of online resources to programmatically add/edit flows using controllers, such as OpenDaylight, I add one here for reference     https://github.com/fredhsu/odl-scripts/blob/master/python/addflow/odl-addflow.py

and I quote:

def push_path(path, odlEdges, srcIP, dstIP, baseUrl):

  for i, node in enumerate(path[1:-1]):

    flowName = “fromIP” + srcIP[-1:] + “Po” + str(i)

    ingressEdge = find_edge(odlEdges, shortest_path[i], node)

    egressEdge = find_edge(odlEdges, node, shortest_path[i+2])

    newFlow = build_flow_entry(flowName, ingressEdge, egressEdge, node, srcIP, dstIP)

    switchType = newFlow['node']['@type']

    postUrl = build_flow_url(baseUrl, ‘default’, switchType, node, flowName)

   # post the flow to the controller

    resp, content = post_dict(h, postUrl, newFlow)

def build_flow_entry(flowName, ingressEdge, egressEdge, node, srcIP, dstIP):

 # Alternatively I could add a second flow with 0×806 for ARP then 0×800 for IP

  defaultPriority = “500″

  newFlow = {“installInHw”:”false”}

  ingressPort = ingressEdge['edge']['tailNodeConnector']['@id']

  egressPort = egressEdge['edge']['headNodeConnector']['@id']

  switchType = egressEdge['edge']['headNodeConnector']['node']['@type']



  newFlow.update({“ingressPort”:ingressPort, “priority”:defaultPriority})

  newFlow.update({“nwSrc”:srcIP, “nwDst”:dstIP})

  newFlow.update({“actions”:”OUTPUT=” + egressPort})

  return newFlow

def post_dict(h, url, d):

  resp, content = h.request(

                                           uri = url,

                                           method = ‘POST’,

                                           headers={‘Content-Type’ : ‘application/json’},



  return resp, content

Pica8 has open systems designed to ease and accelerate network development, deployment, for a new kind of smart programmable, agile network. Customers can leverage an extensible ecosystem that helps them build a network programmability framework along with sample applications to get started. This includes a Linux platform for network control plane and data plane equipment with well-defined APIs to provision, configure and manage all crucial network elements, such as routing, switching and policy.

Network Agility and Elasticity
Maintaining network agility enables a business to seamlessly adapt and react to dynamic Big Data workloads. To accomplish this, these same businesses will deploy network equipment (such as routers, switches and links) with minimal manual intervention. How? Simple. By using routers, switches that have been virtualized/containerized or with hardware routers and switches that can be easily brought up without manual intervention.

With automated network programmability, once new network paths are created, further activation, port upgrades and remote switch provisioning can enable a topology to be changed on the fly to smoothly react to changes in the Big Data ecosystem. One can use network equipment from pre-provisioned HW, container/VM factories with sizes and form factors for each segment of your network—whether it’s ToR, aggregation/core and access. Conversely, old world network vendors who supply closed systems can take years to develop new features.

Network administrators have for years been running static topologies rather than running network  topologies which are context sensitive and hence dynamic. However with SDN and solutions from vendors like Pica8 it need not be the case any more. By building Big Data Context and infusing network programmability, administrators now have the tools needed to maintain agility and resilience and for once be in charge of their own destiny.


Who doesn’t like automation?  If you’re speaking to somebody in IT, then the short answer is “nobody”.

While the term Zero Touch Provisioning (ZTP) might be increasingly more common to networking, the concept of automation has existed for years in IT.  At its core, ZTP is an automation solution that’s designed to reduce errors and save time when an IT administrator needs to bring new infrastructure online.

This is particularly useful for data center servers, where scale and configuration similarities across systems make automation a necessity.  In the server world, the Linux-based operating system has revolutionized on boarding and provisioning.  Rather than using command-line interfaces (CLI) to configure these systems one at a time, administrators can use automation tools to roll out the operating system software, patches, and packages on new servers with a single command, or the click of a mouse.

Advanced scripting capabilities also allow administrators to tailor the boot configuration of these systems with profiles for specific applications.  So for example, if you need ten servers for a new Hadoop cluster, you can load this with one profile, but if you need six new servers for a new web application, you can roll that out using a different profile.

Essentially, automation drastically reduces the amount of time when you take a server out of the box to when it’s functioning in a production environment – all while minimizing the risks of manual configuration errors and missed keystrokes, or the additional challenge of knowing which driver or library is the correct one.

What about the network world?

The basic question here is why should it be any different?  Much like servers, network devices have traditionally been managed via the CLI.  What’s more, network administrators need to do this manually on each individual device.

Consider the typical on boarding and provisioning process of a network switch.  A network switch has traditionally been coupled with a pre-loaded proprietary network operating system.  Technicians must use CLI or the manufacturers own tools to provision a switch.  This can be broken down into three basic steps:

  1. When the new device arrives, it already has an OS to help bootstrap the device.  It is removed from the box and goes to a staging area. Here the administrator checks the operating system version, and makes any updates – for patches, bug fixes, or any new feature updates as necessary.
  2. An initial configuration is made to establish basic network connectivity.  This includes parameters such as administrator and user authentication information, the management IP address and default gateway, basic network services (DHCP, NTP, etc) and enabling the right L2 and L3 network protocols are all examples of the bootstrap process.
  3. Once the initial OS and configuration has been verified, the device can be installed into the environment (racked and cabled), where further customized configuration can be made (either locally via the console or through a remote access protocol) that is specific to the application and location within the network.

On Boarding a New Switch

The details may vary slightly for each environment, but the basics remain the same.  This can be a verytime consuming process.  Now extrapolate this model to ten network switches.  Or twenty.  Or one hundred.  And when you consider that for each individual switch, there’s an opportunity for a configuration error that can bring down the network or create exposure and a security risk, the conclusion is obvious: there has to be a better way.

How does ZTP help with this process for the network?  Remove all the manual configuration and steps listed above, and what you have left is ZTP.  In this model, the network administrator receives the new hardware and the first thing they do is to physically install the device – rack and cable the switch.   Once these physical connections are made, the technician no longer has to touch the box – hence the name, “zero touch”.

With the ZTP system in place, once the switch is powered on, it uses standard network protocols to fetch everything it needs for provisioning.  It can send a DHCP query to get the proper IP address for connectivity and management.  It can then use BootP/TFTP to get the right operating system image.  And then another TFTP request to get the right configuration file based on the application profile.

In this model, once the network administrator sets up the IP address scheme via the DHCP server, and the OS and configuration files on the TFTP server, they can effectively roll out tens, hundreds, and thousands of switches in this way – all fully customizable and without the time consuming and error prone manual configuration process.

Sounds like a no brainer right?  Now juxtapose this with some mega trends that are happening in the data center today.

The first of these is how more and more, the data center is becoming an application-driven economy that is fueling data center growth and virtualization.  Bringing applications to market faster are the key to gaining a competitive advantage.  Therefore, the faster IT teams are able to bring infrastructure online to support these applications, the better.  With ZTP and server virtualization prevalent in the server world, it’s become extremely important to automate the network processes as well.  Ask any network administrator, and they clearly don’t want to be viewed as the long pole in the tent.

The second is bare-metal switching.  If the applications are driving the top line, then it’s the hardware going to help with the bottom line.  Commoditization of network hardware is the next logical evolution, with the rapid adoption of merchant silicon.  More and more customers are seeing less differentiation in the hardware, and more differentiation in the speed, features, and operational simplicity that the software can provide. Today, three manufacturers (Big Switch, Cumulus, and Pica8) are offering Linux-based OSs for bare-metal switches – effectively bringing the efficiency and familiarity of Linux to the network world.

In the context of these trends, it’s even more important to implement ZTP and automation practices into the network.  As more applications come online, IT teams are being taxed to keep the infrastructure up to date – including provisioning, scaling, troubleshooting, and maintenance.  This is not sustainable in any manual based process.

And as hardware and software continues to be decoupled, it’s critical to find a way to automate the new operational model.  If I can purchase hundreds of switches from an OEM or ODM and rack these devices – would you rather install the OS and configure each of these individually, or do this through an efficient methodology using well known, reliable network protocols.

Much like the server world before it, the network world is seeing some significant technology shifts.  Automation, software defined devices, and bare metal switches are all contributing to a fast-paced and dynamic environment in the data center.  With ZTP, the network is leveraging best practices from the server world to drive greater speed and operational efficiency.

In short, it’s become an essential way to automate the network.  Now who wouldn’t like that?


Pica8 Says ‘Yes’ and Challenges the FUD

Up to this point, OpenFlow has mostly been deployed in research and higher-education environments.  These early trials have shed some light on interesting use cases, what OpenFlow is good for, and of course, what OpenFlow might not be so good for.

This is important because OpenFlow and SDN adoption is only going to grow.  It’s imperative that we understand these limitations – specifically, what’s real and what’s FUD.

One of these is scale.

If you’ve kicked the tires on OpenFlow, one question you may have heard is “How many flows does that switch support?”  However, this question is only part of the story.  It’s like asking only about a car’s top speed when what you should be thinking other things too – such as fuel efficiency and maintenance.  So to figure out the right questions, we first need to go over a bit of background.

In its most basic terms, any network traffic, whether it’s Layer 2, Layer 3, or something else, is governed by a of forwarding rules as defined by a series of protocols.  If it’s this MAC, do this.  If it’s that IP, go there.  Each of these “rules” is stored on the switch, in memory, in something called the Forwarding Information Base (FIB) or Router Information Base (RIB).

OpenFlow is a little different.  It allows for more point-to-point connections that you can base on business logic “rules”..  These rules, or flows, are implemented in a different way.  If I see packet type x, perform action y.  It doesn’t have to follow the OSI networking model, and as such, gives users added flexibility to govern how traffic behaves. This works great for policy-based networking and driving business logic into the network.  But at its heart, it’s another networking protocol and the basic concept is the same.  The key difference with this approach is that these flows are not storied in say the FIB, but in the switch Ternary Content Addressable Memory (TCAM).

Now here is where things get interesting.

Let’s look at the switch with the Broadcom Trident II ASIC. Pretty much every major switch vendor has a switch with this ASIC including us with both pre-loaded and bare metal switch options through our Hardware Ecosystem (link).  Trident II enables you to store up to two thousand entries in the TCAM.  Most every other switch vendor has used this data point to drive the perception that OpenFlow will not scale.

Well, we at Pica8 agree – to an extent.  Two thousand flows are not enough – if you have 400 network nodes and each of those speaks five other nodes, that already maxes out your TCAM.  So what did Pica8 do to solve this?  Two things:

  1. First, we made the TCAM table much more efficient.  Instead of treating this as a hard limit of two thousand flow entries, we chose to slice and dice that memory into smaller chunks.  Each flow entry is designed for a packet header, but in many cases, you don’t need to inspect the entire header to determine the right action.  Having 3 smaller tables can go a long way – one table for port, another for MAC, and another for signature can increase the total number of rules.  In some cases you might just need to match the port? Or the destination IP?  Or the MAC?  Or a combination of the above?  Think of this as a 2000 page book – with each page having just enough room for one packet header.  But if your flow doesn’t need to match the entire header, you’ll have lots of whitespace.  We’ve filled up every page to the margin.   In addition to that, we implemented the usage of wildcards, aggregation, de-duplication and other techniques to optimize the table.  With all these enhancements, we’ve managed to effectively double the capacity of flows in the TCAM.  But that’s still not enough to make an appreciable difference right?  So…
  2. Second, we attached the FIB table to the TCAM.  This is a capability that we have leveraged on the Trident II ASIC.  In this way, we’ve vastly expanded the number of entries that use a standard IP longest prefix match algorithm, while also freeing up even more space in the TCAM by eliminating the need for IP lookups.

Both of these innovations contribute to an OpenFlow implementation that supports over two hundred THOUSAND flows – all on the exact same hardware that the other guys use.  And this number makes a lot more sense as we expect more customers to roll out larger OpenFlow networks into production.

So, when you want to give OpenFlow a try – make sure you ask the right questions about the limitations you’ve been presented with.  You might be surprised at the answer.  To learn more about SDN scaling and Pica8, send us a note at sales@pica8.com. We would love to hear your comments.


CrossFlow Networking

When Worlds Colliding is Not Such a Bad Thing

If you’re a fan of the 90’s sitcom Seinfeld, you’re undoubtedly familiar with more than a few Seinfeld-isms – terms originated from the show that have made their way into our daily vernacular.  One such term, “worlds colliding” describes a theory in which it’s best to keep your different worlds (as defined by social spheres, e.g. friends, family, colleagues, etc.) separate.

How does this relate to networking you ask?

Well let’s look at one world – Layer-2/Layer-3 networks.  These are the networks that people have been building for decades.  They consist of switches and routers and leverage protocols and technologies that networking gurus are familiar with such as Ethernet, VLANs, trunking, BGP, OSPF and more.   These protocols govern how traffic is forwarded and are built upon the 7-layer OSI model.  And because this model is (relatively) mature, there’s an inherent reliability, and a clear understanding of how these networks are built, how they work, and how they are maintained.

Then there’s the second world – the world of SDN and in this example, OpenFlow.  With OpenFlow, you can do some interesting things, such as using a centralized controller to create rules and policies that dictate where traffic needs to go.  In theory, this approach is more flexible and dynamic, and enables users the ability to drive business logic into the network.  If you want to trigger some traffic monitoring, network tapping, or bandwidth calendar based upon users, times, or geographies, you can do that with OpenFlow.

The problem today is that these worlds remain separate.  And this creates added costs and complexity for users because of the necessity to build, operate, maintain, and troubleshoot separate networks.  Wouldn’t it be better if you could have the flexibility of OpenFlow for policy-based networking with the efficiency of Layer-2/Layer-3 for traffic forwarding?  Enter CrossFlow Networking.

CrossFlow Networking is a unique capability delivered on PicOS 2.4.  It allows these worlds to “collide” (in a good way).  With CrossFlow, users can selectively integrate OpenFlow into certain parts of their network for specific applications, while maintaining the efficiency and performance of the tried and true Layer-2/Layer-3 protocols.

How does this work?  OpenFlow allows users to stitch in a unique path for a specific application.  We do this by allowing OpenFlow to fine tune or override the switching (FIB) or router (RIB) tables in the switch.  These tables are “wired” by how Layer-2 and Layer-3 protocols converge to a best path for the traffic.  In some cases, that path may not be ideal for the application (for example, you may want a specific application to access data or a network service that resides somewhere else in the network).  One possible solution would be to adjust the switching and routing topology to get the desired behavior, but that takes time and is disruptive. CrossFlow Networking solves this by allowing an OpenFlow rule to trigger specific behavior, and then modifying the packet appropriately to use the existing FIB and RIB tables.  This gives users granular control to allow a specific policy to change behavior, without disrupting the topology of the existing network.

Ultimately, CrossFlow Networking simplifies the process of integrating SDN into today’s networks.  It bridges the operational gap between traditional networking and SDN, while also reducing CapEx for customers.

To borrow one more Seinfeld-ism, with CrossFlow, network operators can achieve a little bit more “Serenity Now”.

To learn more about CrossFlow Networking and Pica8, send us a note at sales@pica8.com. We would love to hear your comments.




The History of Open

Everybody is talking about “open” this or that – from Cisco making claims to new companies embracing open source code as a means of developing or accelerating their go-to-market strategies. But what does “open” really mean?

One challenge in using a broad and you might say amorphous term like open is that it can lead to confusion or a negative first impression that “this is just marketing.” To get some perspective, let’s look back a bit and see how we got to this point of open and what the original intent was.

Open systems are computer systems that provide some combination of interoperability, portability, and open software standards. (“Open” can also refer to specific installations that are configured to allow unrestricted access by people and/or other computers; this article does not discuss that meaning.)

The term “open” was popularized in the early 1980s, mainly to describe systems based on Unix, especially in contrast to the more entrenched mainframes, minicomputers, and engineering workstations in use at that time. Unlike older legacy systems, the newer generation of Unix systems featured standardized programming interfaces and peripheral interconnects. Third party development of hardware and software was encouraged, which was a significant departure from the norm of the time. We saw companies such as Amdahl and Hitachi going to court for the right to sell systems and peripherals that were compatible with IBM’s mainframes.

The definition of “open system” became more formalized in the 1990s with the emergence of independently administered software standards such as The Open Group‘s Single UNIX Specification. As client/server networking took hold in the late 80s and early 90s, switching vendors followed this tightly-coupled design rationale. Every aspect of a vendor’s solution was designed around tight integration of the OS with components and subsystems, from memory allocation, to managing CPU utilization, to the forwarding ASICs. Differentiation was driven up from the system architecture designed around custom components.

In the late 90s, the component industry and “white box” or ODM (original device manufacturers) started to take more ownership of subsystem and system design. This started us back onto some degree of abstraction. Switches were being built that could have the CPU easily replaced; different types of memory components were another example.

Related to the above history, the mainframe to the PC transition, we discussed how the PC brought forth the idea of hardware and software abstraction. That brought us to the idea of the OS as something that could also be open, with a set of tools that fostered application development.

And then the server opened up.  Over the last 15 years, much has changed.

On the server side, we have seen the transition from Microsoft to Linux and new business models evolving from companies like Red Hat. Then we saw abstraction re-emerge through server virtualization, and the idea of white box servers to drive the hardware agnostic thinking once again, similar to the PC

Now we are looking at a similar evolution on the network side. Some say SDN drives hardware-agnostic thinking.  Having said that, many vendors still hold on to that mainframe idea that “my apps will only run best on my metal.”

So to summarize our first idea, if the network follows this seemingly well-traveled path, just like we saw with early Unix systems, third party development of hardware and software was encouraged, which was a significant departure from the norm of the time.

Here’s what hardware agnostic thinking can bring to networks. First, like PCs and servers, hardware abstraction creates operational consistency. That drives down costs over time.  The second thing it brings is transparency – you can look inside to see not only Intel, but gain better visibility to truly control your traffic. The idea of external programmability opens that Cisco Pandora’s box but in a good way.  Now you can decide how and when to forward traffic that might need that level of granular control.

So to a large extent, the idea of open network hardware delivers freedom to choose the capacity, port configuration and even color of your “box.”

Now with those early Unix systems, there was another attribute: standardized programming interfaces. So let’s extend the idea of open to the network to the idea of programmability, and that takes us to the ability to tune the system, which is the goal of open-source projects like OpenStack.

So how do we tune an OS or a “stack?”

One means for all vendors to help here is to offer a variety of interfaces for programmability — so called application programming interfaces or APIs. So called Open APIs (often referred to as OpenAPI new technology) is a phrase used to describe sets of technologies that enable devices, websites or applications to interact with each other by using REST, SOAP, JavaScript and other web technologies. Linux is also becoming a popular means to “program,” largely driven by the DevOps thinking we all see in cloud environments.

In the case of OpenStack, plugins for your network OS would ensure an OpenStack command is accepted and acted upon – the network can be programmed to conform to an application need.  From the stack perspective, APIs at each level would help shape the degree of tuning you did to better suit your needs.

So to summarize this aspect of open, APIs can help support open source projects, where the standardization is the common set of APIs, with the result being a stack tuned to better meet your specific needs.

And from the network OS point of view, just like you tune a server OS to meet your needs, the idea of tuning the OS for specific application environments is something to consider.

If history repeats itself, we will see more hardware abstraction on the networking side, and we will see more and more agreement amongst vendors on a common set of APIs.


Where to Start with SDN

For the 3rd installment on my three part SDN series, building on A Business Case for SDN, and the SDN Ecosystem,  the most practical way to start exploring an SDN deployment is with a proof of concept (POC). But even if you have the approval to go ahead with an SDN POC, it can be difficult to know where to start. Let’s cut through the uncertainty and lay out what it takes to do a successful SDN POC.

Identify a pain point

Start by identifying a key pain point in networking that you’d like to address with SDN. For example, you might want to improve campus security, or improve the performance of collaborative tools, or streamline your data center. Specific tasks in these areas include adding a network tap, increasing the speed of a LAN link, or reassigning VLANs.

We’ll assume you have surveyed business unit leaders, ranked overall IT strategies and come back with one SDN application to start your evolution. Similar to a cloud or BYOD initiative, giving visibility for SDN can help you bring the company together, and can also build support for improving how IT can drive the business. If you understand the pain points and how SDN can improve operations, you can evangelize how SDN can be a competitive advantage for each department. You can then rank departmental projects in order of priority and use each group as an example of how SDN can drive economic as well as employee benefits.

For example, on the campus, you may rank security as your top concern and therefore focus the SDN discussion around onboarding devices into the network. SDN could augment your BYOD strategy and help with the onboarding process – you could look at the behavior of recently-added devices and potentially shut those ports down or isolate a device.

Thinking through the metrics for success

The POC involves setting up an SDN test bed and determining if the SDN solution can deliver the benefits you’re expecting. For sake of argument, let’s pick an SDN network tap as the application. The idea here is to be able to turn on tapping functionality on any SDN-enabled port, thereby not having to use parallel fixed infrastructure providing tapping functionality.

The choice here is either fixed CapEx / OpEx for purpose-built monitoring and tapping tools, or leveraging an SDN network tap that can be bolted onto your network and used as a dynamic probe. Here the POC could explore the overall cost of the gear (CapEx) and the cost for training in both non-SDN and SDN paradigms (OpEx). You could also look at whether SDN gives you all the functionality or just a subset.  Once you have the technical details, you can make an informed decision.

Building an SDN POC using a network tap

To lay out the whole solution, we need:

  • An SDN-capable network switch and OpenFlow-enabled switch operating system, allowing external control from the OpenFlow driven controller
  • SDN controller software such as Ryu, NOX or OpenDaylight
  • A network application (in our case, a network tap)
  • Conventional network tap gear for functionality and usability comparisons.

You can assemble these components by buying from different vendors, buy a starter or developers kit that includes all of these components, or go with an all-in-one solution from a mainstream vendor such as Cisco or HP if you’re willing to pay a higher price.

Think about the skills that you and your team have.  If you are Python warriors, for example, then Ryu (built by NTT Laboratories) might be a good sandbox for you.  If you know your team needs GUIs and your scripting bench is already overtasked, then think about something more GUI-based, like OpenDaylight. Today not all controllers are alike. Some are single threaded; some are built for high availability environments. Mapping your team’s skills to the development needs of your SDN stack will go a long way in ensuring that your adoption of SDN technology matches your needs and abilities.

Expanding the POC

Once the POC is carried out, it can be expanded into a trial by extending the POC to a functional area of network operations.

The list of SDN ideas is long and focused on the idea of IT agility. Here’s a more complete list to leave you with beyond the one we have walked through.

  • Establishing virtual Ethernet networks without VLANs
  • Enabling applications to dynamically request services from the network
  • Reducing CAPEX by using bare-metal switches instead of name-brand switches
  • Evolving network functionality more rapidly based on a software development lifecycle
  • More easily implementing QoS
  • Implementing more effective security functionality

An SDN POC will familiarize you with the benefits of SDN and help you to build a strong business case for SDN within your organization. By assembling a few SDN components you can demonstrate the value of SDN compared with the traditional way of doing things.


The SDN Ecosystem

As a follow on to my blog about building a business case for an SDN deployment, there are now dozens of companies offering SDN-related products – so many that you might find it difficult to separate the hype from the meat. Let’s look at some categories of SDN products and how each of them fits into an overall SDN solution.

The key components of an SDN solution are ASICs, switches, a controller, and the applications or services that run over the network.


ASICs have a long history in networking by driving scale and performance. In a clock cycle, very complex tasks can be accomplished. Without the ASIC, the central CPU would be overwhelmed performing those same tasks (remember those so called “one arm routers”). The need for ASICs created a new set of suppliers such as Broadcom, Marvell and Mellanox, and most recently Intel through its acquisition of Fulcrum. We can expect more and more specialization in ASICs as the industry pivots on the SDN theme. Over the last decade, the merchant silicon vendors have diversified and specialized products for vertical markets. For example, an ASIC optimized for the data center might have VxLAN support, while another tuned for a carrier application might have rich features for MPLS support.


Switches handle the work of directing network traffic to various endpoints. Switch vendors implement protocols that enable their switches to communicate with an SDN controller, which tells the switch how to direct traffic. Most of the following vendors are traditional switch vendors, with the exception of Accton and Quanta Computer, which provide bare-metal switches without an OS. Pica8, Big Switch and Broadcom are also exceptions, as they provide a network operating system that is loaded onto a commodity bare-metal switch. Players in this space include traditional infrastructure vendors like Cisco, HP, and IBM and startups like Pica8.

SDN Controllers

SDN controllers are the brains of the SDN solution, the place where the user interacts with the network. SDN controllers tell switches how to direct traffic based on policies the user sets. While OpenFlow is not the only SDN controller protocol out there, it’s the most visible one. It’s worth noting that in a recent survey by Dr. Jim Metzler of Webtorials Analyst Division, 43 percent of IT managers surveyed said that their SDN solutions will likely or definitely include OpenFlow, while only 3 percent said they would not include OpenFlow. Market participants playing in this space include big companies like Cisco, HP, and NEC and startups like Big Switch and PLUMGrid.

Network Applications and Services

The members of this category provide network services and applications such as security and optimization that are part of an overall SDN solution. Rather than simply directing traffic, these applications process traffic with firewall, load balancing, or other applications. Application vendors include Big Switch, Cisco (through ACI/Insieme), HP, NEC, PLUMgrid and Riverbed.

Open Versus Proprietary Solutions

Another way to look at the SDN ecosystem is to separate vendors into open versus proprietary. Legacy switch vendors have large revenue streams and customer bases to protect, so they tend to offer proprietary solutions that lock the user in. Proprietary vendors include Cisco, HP, and IBM, while open vendors include Big Switch and Pica8.

Traditional networking vendors have tightly coupled software and hardware, which imposes unique operational frameworks for each vendor and creates a “one vendor, end-to-end topology” sales mantra. The SDN movement is challenging this idea. The idea of software leading and hardware following also raises the idea of hardware commoditization. The best approach is to leverage the degree to which any technology commoditizes, use that as a competitive advantage and then add specific differentiation where customers see value.

SDN has the promise for customers to achieve a more consistent operational framework regardless of the network vendor (and hardware). Sure, SDN leaves the door open for vendors to specialize as well through custom programmability, but let’s consider what the data center world looks like once we have one “control” standard and therefore one common way of operating multi-vendor networks.

In this data center world, the idea that one vendor provides an end-to-end network will go the way of VHS players and fax machines. In an end-to-end environment, you can buy off on the idea that that your operational environment will be simpler ostensibly due to lower training and operational costs, but on many levels, end-to-end has not delivered consistency. Feature differences emerge between product families, driven by ASIC differences; there are OS and CLI differences across product families; and there are architectures across product families that are not consistent.  Clearly, the more we can abstract the operational details from the hardware, the more we will truly deliver on open SDN—and deliver the benefits of a common operational environment end to end.

This change is being backed up by survey data. In the Webtorials survey, over 48 percent of respondents cited openness of the solution as either extremely or very important, while just 7.4 percent cited it as not important.

It’s encouraging to see growing industry support around the idea of being more open, as indicated by more and more support of OpenFlow. The OpenFlow protocol is gaining momentum. Members that represent the customer voice include Deutsche Telecom, Facebook, Google, Microsoft, Yahoo! and Verizon. On the vendor side, IBM, HP and Cisco all talk about supporting OpenFlow on portions of their switching portfolios.

The continuing emergence of OpenFlow and SDN promises to change the meaning and the benefit of “end to end.” Vendors will no longer dictate terms through their respective


As a new trend begins to emerge and gain traction, making a compelling business case is one of the key milestones toward achieving the Holy Grail: Worldwide adoption. Once that adoption takes place, then flood gates will open. Let’s face it—if everyone could make a compelling business case, then everyone would create them. So what’s the path to a good business case and where do you start? Here are a few suggestions. Consider this a template and a starting point for building a business case for SDN.

What do you want out of SDN?
I have met many customers that have started a short list of ideas for SDN in an effort to first decide what they wanted out of SDN and how it can potentially benefit their company. In this video from this year’s Open Networking Summit, it was apparent that most people believed that SDN would increase network agility by:

  • Simplifying configuration and provisioning, thereby reducing OpEx by minimizing or eliminating manual configuration
  • Performing traffic engineering with an end-to-end view of the network
  • Supporting the dynamic movement, replication and allocation of virtual resources
  • Establishing virtual Ethernet networks without VLANs
  • Enabling applications to dynamically request services from the network
  • Evolving network functionality more rapidly based on a software development lifecycle
  • Implementing more effective security functionality

What are the pain points?
I have found that IT teams survey their line of business or departments to learn about their technology needs. With that information, the more savvy teams examine whether the needs can be met by having more nimble IT services that can react quickly and seamlessly to changing conditions. If you perform a similar type of analysis, you can then map the pain points back to IT processes needed and then consider how SDN will help.

In the data center, for example, a common pain point is the desire to make virtual machines (VMs) mobile, or to help manage workloads or for HA. The challenge is how to move a VM without disrupting the underlying network. Here the goal is to have a means to move the VMs and seamlessly “orchestrate” changes in the physical network.

Set up a Proof of Concept
Also at ONS this year, many proclaimed that 2014 is the year of the SDN Proof of Concept (POC). It means setting up an SDN test bed and determining if the SDN solution can deliver the benefits you’re expecting. Note that the SDN setup time should be included in any evaluation of the approach, along with the time it takes to achieve the benefits you are seeking.   

In a data center project, the SDN POC would ideally mock up two racks: one with SDN and one without SDN. This POC would look at the operational aspects of reassigning VLANs and creating virtual domains within a fixed IP fabric. Thinking in terms of service metrics, such as time-to-new application or service availability, or time-to-fault resolution, will help create tangible benefits and metrics for judging the value of SDN.

Build consensus
Similar to what many IT leaders found with cloud or BYOD initiatives, giving visibility for SDN can help you bring the company together and build support for improving how IT can drive the business. If you understand the pain points and how SDN can improve operations for the data center, you can evangelize how SDN can be a competitive advantage for each department.

Determine SDN cost savings
Based on an understanding of the improvement in operational efficiency between using an SDN solution and managing networks more traditionally, you can come up with a comparison of CapEx and OpEx costs for the two different approaches. You should prepare a multi-year financial analysis of the costs and benefits that will sell the solution over the long term.

Create a presentation
Finally, create a presentation that summarizes your findings. It does not have to be flashy, but instead use it as a tool to crystallize your thinking and help the broader audience see the big picture, and that you have defined a path and metrics for success. It’s important to tie the benefits of SDN back to direct business value, so you should include some examples of how SDN eliminates pain points in business units and enables the IT department to perform more efficiently and cost effectively. The presentation should also address what your IT organization will do to mitigate the risk associated with implementing the SDN solution.

Garnering resources for any project can be an arduous initiative. Building a successful business case for SDN in particular can contribute to that difficulty. As noted at the beginning of this blog, use this as a template to get started. By putting yourself in a position to focus on the POCs, you will discover that the results will speak for themselves and go a long way to help strip away the SDN hype—and more importantly—help your business perform better.


Over Layer Versus Under Layer

Do you feel like you are in data center acronym soup these days?   I sure feel it, and I think sometimes tech-speak can help mask the real driver for change. In the data center, we are striving for a new model.  The idea of real time resource allocation and reallocation, the ideal organism that responds perfectly to every request and oh, did I mention resiliency in the whole stack for instant recovery from any fault. Wow, that would be great!  I think we have a ways to go. For now, the latest craze is to add the word virtualization to each topic.

Why is that?  I think it is because virtualization has helped us learn that you can decouple the hardware and software and create layers of abstraction that lead to better systems.  And here “better” could be lower power / cooling and space utilization, or it could the idea that a virtual machine (VM) can be your 18 wheeler, or container ship, and move the application or data anywhere you want, to help in that resource allocation / re allocation or resiliency story I mentioned above.

Now if we look on the network side, the thinking has been extended.  We’re hoping we bring this same degree of abstraction and agility to the network side.

The networking acronym soup you’ll see prevalent today are network virtualization (NV), Network Functions Virtualization (NFV) and software-defined networks (SDN) mentioned in the same breath numerous times, and this can be confusing for those who wonder what the difference is. But there’s a simple way to differentiate NV, NFV, and SDN if we consider them as “over layer” and “under layer” technologies.

The over layer/virtual network infrastructure approach is focused on working with an existing network, and building in two more degrees of abstraction.  It addresses the aspect of how to define a logical element or domain, and the idea of being able to create a service on the fly.  Network Virtualization (NV) is about building tunnels (also called overlays) through the existing network, and Network Functions Virtualization is applying a service (such as a firewall or load balancer) to one of those tunnels.

For example, you may want to connect two isolated domains without changing the underlay, and that would require NV technology. If you want to spin up a firewall on that tunnel, you use NFV. NV and NFV solutions are delivered on an x86 server platform.

NV is valuable because it saves administrators from having to physically wire up each new domain connection, especially for virtual machines that get created. This is useful because administrators don’t have to change what they have already done. They get a new way to virtualize their infrastructure and make changes on top of an existing infrastructure.

Once virtual tunnels were enabled, that begged a question: If administrators can set up a VM by pointing and clicking, why can’t they turn up a firewall or IDS/IPS in the same way? This is what NFV enables. NFV uses best practices as base policies and configurations for different network elements. If you have a specific tunnel you’re punching through the infrastructure, you can add a firewall or IDS/IPS to just that tunnel. The popular functions for this are firewalls and IDS/IPS systems from companies like PLUMgrid or Embrane.

NFV runs on high-performance x86 platforms, and it enables users to turn up functions on selected tunnels in the network. The goal is to allow people to create a service profile for a VM, or flow, and leverage x86 server muscle to build an abstraction on top of the network (the tunnel) and then build virtual services on that specific logical environment.  Once in place, NFV saves a lot of time on manual provisioning and training.

So in summary, NFV and NV create a service abstraction (NFV) and a logical abstraction (NV) to help manage VMs and critical flows in real time and with more control. You can see why service providers are keen to be able to spin up customer services on the fly, adding incremental revenue for premium customers, or more stringent SLAs. For the enterprise, the idea is to have point-and-click orchestration for streamlining cloud behavior for internal users, or the company’s e-commerce portal.

In contrast, the under layer or physical network infrastructure approach is focused solely on the network fabric itself. This is where many believe SDN comes in. With SDN, you care about routing/switching/management and provisioning the network itself. You might support OpenFlow. SDN solutions are delivered on a switch, and the ASIC in the switch is an integral part of the story.

SDN uses canned processes to provision the network. For example, instead of building a network tap using an appliance, administrators could use SDN to program the network when they want to build a tap. SDN makes the network programmable by separating the control plane (telling the network what goes where) from the data plane (sending packets to specific destinations). It relies on switches that can be programmed through an SDN controller using an industry standard control protocol, such as OpenFlow.

So to tie it all together, NV and NFV add virtual tunnels and functions to the physical network, while SDN changes the physical network, and therefore is really a new externally driven means to provision and manage the network. A use case for SDN may involve moving a large “elephant flow” from a 1G port to a 10G port, or aggregation of lot of “mice flows” to one 1G port. Big Switch and Pica8 are examples of companies selling SDN-related products.

Another simple way to tell the difference is the underlying infrastructure required. Anything running on a server is either NV or NFV, while anything running on a switch and controller is SDN.  I am sure some readers will find an exception to this, but having it, this simple distinction will be right more times than not.  And I would also point out that they are all software-led or -driven functions, so some pundits want to put all these new ideas under the SDN umbrella.

Another area for discussion is that today, NFV and NV advocates position the overlay as truly separate and agnostic of the under layer below. My personal view is this will be the case for very localized environments, say in a rack deployment.  Having said that, as soon as you start moving machines across data centers or between data centers, I think I have a harder time believing the under layer and over layer can be completely agnostic. Time will tell.

The net net for you is that long term, I do believe we are on the right path for the over layer and under layer technologies to work together to provide a programmable network. By understanding the differences between them, we can move toward a programmable network model that works – no matter what we call it.