Archive for the ‘Pica8 Deep Dive’ Category


As a new trend begins to emerge and gain traction, making a compelling business case is one of the key milestones toward achieving the Holy Grail: Worldwide adoption. Once that adoption takes place, then flood gates will open. Let’s face it—if everyone could make a compelling business case, then everyone would create them. So what’s the path to a good business case and where do you start? Here are a few suggestions. Consider this a template and a starting point for building a business case for SDN.

What do you want out of SDN?
I have met many customers that have started a short list of ideas for SDN in an effort to first decide what they wanted out of SDN and how it can potentially benefit their company. In this video from this year’s Open Networking Summit, it was apparent that most people believed that SDN would increase network agility by:

  • Simplifying configuration and provisioning, thereby reducing OpEx by minimizing or eliminating manual configuration
  • Performing traffic engineering with an end-to-end view of the network
  • Supporting the dynamic movement, replication and allocation of virtual resources
  • Establishing virtual Ethernet networks without VLANs
  • Enabling applications to dynamically request services from the network
  • Evolving network functionality more rapidly based on a software development lifecycle
  • Implementing more effective security functionality

What are the pain points?
I have found that IT teams survey their line of business or departments to learn about their technology needs. With that information, the more savvy teams examine whether the needs can be met by having more nimble IT services that can react quickly and seamlessly to changing conditions. If you perform a similar type of analysis, you can then map the pain points back to IT processes needed and then consider how SDN will help.

In the data center, for example, a common pain point is the desire to make virtual machines (VMs) mobile, or to help manage workloads or for HA. The challenge is how to move a VM without disrupting the underlying network. Here the goal is to have a means to move the VMs and seamlessly “orchestrate” changes in the physical network.

Set up a Proof of Concept
Also at ONS this year, many proclaimed that 2014 is the year of the SDN Proof of Concept (POC). It means setting up an SDN test bed and determining if the SDN solution can deliver the benefits you’re expecting. Note that the SDN setup time should be included in any evaluation of the approach, along with the time it takes to achieve the benefits you are seeking.   

In a data center project, the SDN POC would ideally mock up two racks: one with SDN and one without SDN. This POC would look at the operational aspects of reassigning VLANs and creating virtual domains within a fixed IP fabric. Thinking in terms of service metrics, such as time-to-new application or service availability, or time-to-fault resolution, will help create tangible benefits and metrics for judging the value of SDN.

Build consensus
Similar to what many IT leaders found with cloud or BYOD initiatives, giving visibility for SDN can help you bring the company together and build support for improving how IT can drive the business. If you understand the pain points and how SDN can improve operations for the data center, you can evangelize how SDN can be a competitive advantage for each department.

Determine SDN cost savings
Based on an understanding of the improvement in operational efficiency between using an SDN solution and managing networks more traditionally, you can come up with a comparison of CapEx and OpEx costs for the two different approaches. You should prepare a multi-year financial analysis of the costs and benefits that will sell the solution over the long term.

Create a presentation
Finally, create a presentation that summarizes your findings. It does not have to be flashy, but instead use it as a tool to crystallize your thinking and help the broader audience see the big picture, and that you have defined a path and metrics for success. It’s important to tie the benefits of SDN back to direct business value, so you should include some examples of how SDN eliminates pain points in business units and enables the IT department to perform more efficiently and cost effectively. The presentation should also address what your IT organization will do to mitigate the risk associated with implementing the SDN solution.

Garnering resources for any project can be an arduous initiative. Building a successful business case for SDN in particular can contribute to that difficulty. As noted at the beginning of this blog, use this as a template to get started. By putting yourself in a position to focus on the POCs, you will discover that the results will speak for themselves and go a long way to help strip away the SDN hype—and more importantly—help your business perform better.

, , ,


Do you feel like you are in data center acronym soup these days?   I sure feel it, and I think sometimes tech-speak can help mask the real driver for change. In the data center, we are striving for a new model.  The idea of real time resource allocation and reallocation, the ideal organism that responds perfectly to every request and oh, did I mention resiliency in the whole stack for instant recovery from any fault. Wow, that would be great!  I think we have a ways to go. For now, the latest craze is to add the word virtualization to each topic.

Why is that?  I think it is because virtualization has helped us learn that you can decouple the hardware and software and create layers of abstraction that lead to better systems.  And here “better” could be lower power / cooling and space utilization, or it could the idea that a virtual machine (VM) can be your 18 wheeler, or container ship, and move the application or data anywhere you want, to help in that resource allocation / re allocation or resiliency story I mentioned above.

Now if we look on the network side, the thinking has been extended.  We’re hoping we bring this same degree of abstraction and agility to the network side.

The networking acronym soup you’ll see prevalent today are network virtualization (NV), Network Functions Virtualization (NFV) and software-defined networks (SDN) mentioned in the same breath numerous times, and this can be confusing for those who wonder what the difference is. But there’s a simple way to differentiate NV, NFV, and SDN if we consider them as “over layer” and “under layer” technologies.

The over layer/virtual network infrastructure approach is focused on working with an existing network, and building in two more degrees of abstraction.  It addresses the aspect of how to define a logical element or domain, and the idea of being able to create a service on the fly.  Network Virtualization (NV) is about building tunnels (also called overlays) through the existing network, and Network Functions Virtualization is applying a service (such as a firewall or load balancer) to one of those tunnels.

For example, you may want to connect two isolated domains without changing the underlay, and that would require NV technology. If you want to spin up a firewall on that tunnel, you use NFV. NV and NFV solutions are delivered on an x86 server platform.

NV is valuable because it saves administrators from having to physically wire up each new domain connection, especially for virtual machines that get created. This is useful because administrators don’t have to change what they have already done. They get a new way to virtualize their infrastructure and make changes on top of an existing infrastructure.

Once virtual tunnels were enabled, that begged a question: If administrators can set up a VM by pointing and clicking, why can’t they turn up a firewall or IDS/IPS in the same way? This is what NFV enables. NFV uses best practices as base policies and configurations for different network elements. If you have a specific tunnel you’re punching through the infrastructure, you can add a firewall or IDS/IPS to just that tunnel. The popular functions for this are firewalls and IDS/IPS systems from companies like PLUMgrid or Embrane.

NFV runs on high-performance x86 platforms, and it enables users to turn up functions on selected tunnels in the network. The goal is to allow people to create a service profile for a VM, or flow, and leverage x86 server muscle to build an abstraction on top of the network (the tunnel) and then build virtual services on that specific logical environment.  Once in place, NFV saves a lot of time on manual provisioning and training.

So in summary, NFV and NV create a service abstraction (NFV) and a logical abstraction (NV) to help manage VMs and critical flows in real time and with more control. You can see why service providers are keen to be able to spin up customer services on the fly, adding incremental revenue for premium customers, or more stringent SLAs. For the enterprise, the idea is to have point-and-click orchestration for streamlining cloud behavior for internal users, or the company’s e-commerce portal.

In contrast, the under layer or physical network infrastructure approach is focused solely on the network fabric itself. This is where many believe SDN comes in. With SDN, you care about routing/switching/management and provisioning the network itself. You might support OpenFlow. SDN solutions are delivered on a switch, and the ASIC in the switch is an integral part of the story.

SDN uses canned processes to provision the network. For example, instead of building a network tap using an appliance, administrators could use SDN to program the network when they want to build a tap. SDN makes the network programmable by separating the control plane (telling the network what goes where) from the data plane (sending packets to specific destinations). It relies on switches that can be programmed through an SDN controller using an industry standard control protocol, such as OpenFlow.

So to tie it all together, NV and NFV add virtual tunnels and functions to the physical network, while SDN changes the physical network, and therefore is really a new externally driven means to provision and manage the network. A use case for SDN may involve moving a large “elephant flow” from a 1G port to a 10G port, or aggregation of lot of “mice flows” to one 1G port. Big Switch and Pica8 are examples of companies selling SDN-related products.

Another simple way to tell the difference is the underlying infrastructure required. Anything running on a server is either NV or NFV, while anything running on a switch and controller is SDN.  I am sure some readers will find an exception to this, but having it, this simple distinction will be right more times than not.  And I would also point out that they are all software-led or -driven functions, so some pundits want to put all these new ideas under the SDN umbrella.

Another area for discussion is that today, NFV and NV advocates position the overlay as truly separate and agnostic of the under layer below. My personal view is this will be the case for very localized environments, say in a rack deployment.  Having said that, as soon as you start moving machines across data centers or between data centers, I think I have a harder time believing the under layer and over layer can be completely agnostic. Time will tell.

The net net for you is that long term, I do believe we are on the right path for the over layer and under layer technologies to work together to provide a programmable network. By understanding the differences between them, we can move toward a programmable network model that works – no matter what we call it.

, ,



SDN Lifecycle Management

Recently, we met with a friend who has done an amazing job of understanding the lifecycle management of virtual machines (VMs). As the CTO of a very large cloud provider, he explained in deep detail how he took advantage of Moore’s Law and doubled the amount of VMs in each rack each year, while maintaining or shrinking the cost per rack. As a result, he has doubled the amount of earning potential in each data center while driving cost down, even as his staff is ripping out servers long before their traditional three- to four-year lifecycle and purchasing new ones. He is buying servers at a 3-to-1 ratio over a three-year period when compared with a typical server lifecycle, yet his cost to operate the data center is going down and his productivity is going up by 2x every year.  Amazing!

While we enjoyed learning of his success, when we hear these stories, we think “Could this have the same type of impact somewhere in the network?” It got us to ask why customers traditionally hang on to their top-of-rack switches for four or five years and sometimes longer.

What is different about the network versus servers?

Obviously, development cycles of server processors differ significantly from those of network switching ASICs. While you may get double the processing power in a server every year, it may take five years to see a tenfold change in port speed on a switch and even longer for the price on the ports and NICs to become even close to consumable.

Secondly, the server market is much more commoditized and the ecosystem is much more open. Servers can easily be customized with processors, RAM, disk, and NICs. With Linux, the operating system (OS) is open and not tied specifically to either the hardware or the applications. And further, the tools do not change when the server changes.

In the data center network, in terms of devices, switches are mostly fixed-configuration. The OS is not open and is tied to the switch. When the hardware is changed — especially if you are bold enough to change vendors — many of the tools and applications need to be painstakingly modified, accommodating new APIs and MIBs or other proprietary interfaces. Because this environment is so closed, proprietary, and inflexible, changing any layer of the network ecosystem is a costly exercise. If you want to change your management tools, it impacts the OS below. If you want to modify the OS and if the vendor allows it, you may be required to change the switch hardware or the tools used to deploy and manage the switch.

What if the OS was open and truly separated from the hardware?  What if you could swap out the hardware without having to change the OS or any of the changes you made to this open OS?  What if you could change the management tools or provisioning system without having to worry about the complexity it may create with your proprietary or closed programming interfaces?

It’s time to see which vendors are going to take the needed steps to simply ignore conventional thinking and bring forward a new type of solution that enables their customers to realize the benefits of open networking and enjoy greater efficencies in their data centers.

, , ,


Mind the Gap

One of my pleasures of traveling is listening to the way people speak both with their dialects and their phrases. For those of you that have been to London and ridden “The Tube,” you know that familiar recording, “Mind the Gap.” After talking with several people at this year’s Open Networking Summit (ONS) this past week, I heard that same phrase in my head.


In this case, the “gap” is the chasm that early software defined networking (SDN) adopters have to cross to get started.  Because SDN is a new idea, crossing the gap represents being prepared to challenge old ideas about networking and even your own experiences.

If you really think about it, you don’t want to just mind the gap—you want to be careful not to fall into the old ways of thinking—but you want to cross that gap and keep moving forward. To do that from an open networking perspective, you have to create an opportunity and dig in, grab a controller and an SDN-ready switch and start hacking.

I had a fantastic discussion with a customer at the ONS week who had safely crossed the gap.  Let’s call him Joe. Joe is part of a team that operates a mid-sized data center.  His goal was to leverage bare-metal white box switches to eventually remove chassis from his network and use SDN to more efficiently steer traffic. He was keen to tell us how he downloaded Ryu (from NTT Labs) and realized that he could reprogram MAC addresses and therefore engineer his network with a completely new variable. You can think of the MACs as more extensible than IP addressing.  In non-SDN networks, MACs are generally static and fixed. This is just another new degree of freedom with SDN. The key take away is that SDN enables you to re-think pretty much every aspect of networking. Think of some restrictions you had to design around in the past; now see if SDN unlocks new potential.

Let’s look a bit closer at what is different.  While everyone was focusing on northbound APIs and open source controllers over the last 12 months, people missed the most critical change as a result of embracing SDN – you have much more freedom to control the traffic. When the control plane is completely separated from the data plane, we are liberating the network forwarding decisions from the restrictions of Layer-2 / Layer-3 / Layer-4 boundaries. Users can push the innovation to create more flexible and programmable ways to handle traffic.  For example, instead of following the good old OSI model of putting Layer-2 under Layer-3, we could create a model where Layer-2 is on top of Layer-3. This simple example inspired tunneling technology that is still a big discussion today (VXLAN, GRE, etc).

You may have different reactions to this thinking. Having said that, this is the year of SDN proof of concepts. Jim Metzler of Ashton-Metzler has put out a video about how he sees the market utilizing SDN over the next year or so. For those who attended the SDN Idol Award presentations on March 3, there are use cases now from many vendors that show the business case, the solution, and the metrics of improvement. That is huge progress.

Don’t mind the gap, jump over the gap and accelerate your SDN vision


As many of you know, or newcomers to IT see, we love our acronyms.  For whatever reason, IT is littered with two, three or four letter acronyms.   SDN seems to have accelerated this phenomenon.   As this title suggests I will describe SDN, NV and NFV in this blog.  All of them in our opinion (at Pica8) are software driven schemes that will forever change the way we think about service and application delivery.  Each is a different approach to network programmability. Let’s look into the latest acronyms.

Network Virtualization (NV)

NV is for anybody who’s using virtual machine technology. One data center challenge is to move VMs across different logical domains. NV attacks this problem. NV creates logical segments in an existing network by dividing the network at the flow level (similar to partitioning a hard drive). The goal is to allow people to move VMs independently of their existing infrastructure and not have to reconfigure the network.

NV is an overlay. Rather than physically connecting two domains in a network, NV creates a tunnel through the existing network to connect two domains. NV saves administrators from having to physically wire up each new domain connection, especially for virtual machines. With NV, administrators don’t have to change what they have already done: they get a new way to virtualize their infrastructure and make changes on top of an existing infrastructure.

NV runs on high-performance x86 platforms.

Network Functions Virtualization (NFV)

If NV offers the capability to create tunnels through a network and use per-flow service thinking, NFV puts network services on those tunnels. NFV virtualizes Layer 4-7 functions such as firewall or IDPS, or even load balancing (application delivery controllers) and applies them to specific tunnels created by NV.

The question NFV answers is this: If administrators can set up a VM by pointing and clicking, why can’t they turn up a firewall or IDS/IPS in the same way? If you have a specific tunnel you’re punching through the infrastructure, you can add a firewall or IDS/IPS to just that tunnel. The goal is to allow people to create a service profile for a VM, or flow, and leverage x86 muscle to build an abstraction on top of the network (the tunnel) and then build virtual services on that specific logical environment. Once in place, NFV saves a lot of time on manual provisioning and training.

NFV also reduces the need to overprovision: rather than buying big firewall or IDS/IPS boxes that can handle a whole network, the customer can buy functions for the specific tunnels that need them. This reduces initial CapEx, but the operational gains are the real advantage. NFV can be thought of as a parallel to VMware, with a few boxes running a lot of virtual servers, and a point and click provisioning system.

NFV runs on high-performance x86 platforms.

Software Defined Networking (SDN)

Rather than virtualizing existing connections and running services on top of them, SDN uses canned processes to provision the network. For example, instead of building a network tap using an appliance, users can use SDN to program the network when they want to build a tap.

SDN makes the network programmable by separating the control plane (telling the network what goes where) from the data plane (sending packets to specific destinations). It relies on switches (rather than on X86 servers) that can be programmed through an SDN controller using an industry standard control protocol like OpenFlow.

While NV and NFV add virtual tunnels and functions to the existing physical network, SDN changes the physical network, and therefore is really a new externally driven means to provision and manage the network. A use case may involve moving a large “elephant flow” from a 1G port to a 10G port, or aggregation of lot of “mice flows” to one 1G port.

To make it short and sweet, NV and NFV are for moving VMs and implementing network services on existing networks, while SDN requires a new network construct where the data and control planes are separate.

, , ,


Conversations are swirling throughout the tech industry about whether white box switches are disrupting the networking industry, similarly to how white box manufacturers helped commoditize the server industry. If this recent InfoWorld article, is not enough to persuade you, consider that even John Chambers himself has recently chimed in on the threat of white boxes eroding Cisco’s margins.                                                                                         

The idea of white box switching from a Pica8 perspective is to help create an operating abstraction between the “metal” (in our case white box switches from original device manufacturers, or ODMs) and the network operating system (OS) itself.  When that’s created, you have a degree of OS portability.

In a typical first meeting with a prospect, we frequently get asked if they can port a version of our OS on their existing Cisco switches. At first blush, it makes sense but let’s examine the three key issues that need to be addressed to truly make an OS compatible with a “bare metal” switch.

The network OS must comply with three key component areas within the device: the CPU and the motherboard (or the BSP), “U-Boot” (or universal boot loader), and the ASIC’s APIs. All three collectively need to be matched to the network OS in order for you to port your OS onto that same piece of metal. This is why all the so-called white box, software defined networking (SDN) vendors either sell the complete switch and OS solution or provide a list of pre-qualified devices so you can directly buy the hardware. Both paths deliver the same solution. I will mainly focus on these three component areas so you can assess the level of portability of your network OS.

The CPU and the motherboard – Thanks to PCs and the server market, making your OS work with any CPU is the easiest part of the puzzle to solve. Any modern Linux OS will have the needed drivers and kernel support for all common CPUs. You can assume this is not a problem, but the motherboard consists of other chips than the CPU. You need to make sure you have full access to the driver code of all chips to ensure that complete functionality can be accessed through the new OS. This need is new and particularly topical for Ethernet switches because servers don’t have U-Boot functionality–they have BIOS.

The universal boot loader (U-Boot) As part of a modern Linux OS, the U-Boot is essential in both white box servers (think Pixie Boot) and white box switches. This Linux OS that holds all the details on the shell of the box itself like the fans, lights and USB ports as examples that map directly to the shell or actual box. As part of Facebook’s push toward an open switch specification augmenting the Open Compute Project (OCP) for servers, a common U-Boot loader for future switching hardware is being discussed. Other than this initiative, U-Boot loaders are hidden from users and are one of the reasons you must have pre-qualified white boxes. Every manufacturer has a different U-Boot loader; and even within product families of the same vendor, the U-Boot code could be different.

The ASIC – Every white box or legacy switch has an ASIC in it. The ASIC provides performance scaling and is key to how switching helps build a high-bandwidth network. Whether you leverage merchant silicon (Broadcom, Intel, Marvell, Mellanox), or your own ASICs, you need to have a means to connect the chip to your software stack. That is through the ASIC programming interface or software development kit (SDK). For merchant silicon, you sign a license with that vendor, while in-house ASICs are internally developed. Either way, you can see the barrier here. As a user, you need to have access to this information, either through the OS vendor telling you they are compatible with the ASIC, or in our example above, the existing vendor has to allow you to see this API in order for the OS to sit on a legacy switch.

So in summary, leveraging a modern Linux distribution will take care of the CPU; however, you have to confirm your OS works with the U-Boot code on your switch and the ASICs. This is why porting even the most portable network OS to Cisco, for example, is not easy (putting aside any warranty or legal issues). Cisco would also have to tell you what their ASIC API looks like and give you access to their U-Boot code.

So can portable network operating systems “retread” all those old switches out there? It could happen. I think you will see more opening up of the boundary between the network OS and the box. Universal portability, not so much, but selective hardware compatibility, yes!


, ,


When is enough, enough?

I recently came across this article from John Dix – who made the point that throughout the year, SDN events have helped monitor and inform the community on where the adoption is really occurring. Many articles like this suggest to me that the market understands the idea of SDN opening up a “stack” as in the entire solution – from the metal, to the OS, to the applications. Yet today, there is not enough understanding to necessarily pull the ideal stack together. Articles such as this ask a common question we are all trying to answer: How much SDN is enough to see the value of SDN?

Customers help us see the value in “de-laminating the stack” and moving toward a horizontal model instead of the traditional, fully integrated and closed system that legacy networking vendors now provide. This makes sense because a more open SDN was designed in part to enable innovation and help break some of the vendor lock-in that a closed system fosters. Conversely, for even visionary customers, it can be daunting to try to pull the stack back together and add value to their specific environments.

Do customers see SDN as part of their collective visions for the future? Sure, however, sharing that vision does not mean they have the resources, skills or tools today to start.

If you are a follower of “chasm crossing” (see for example Geoffrey Moore), then SDN will first follow a pre-market signal from innovators and early adopters, then we will sit in the chasm until the mass market catches up and finally has the pain that we can alleviate.

So what can we do to get past this initial phase? Or ensure we don’t get stuck in the chasm?

Counter to our Pica8 messaging, sometimes you need to combine elements or components to deliver a more complete solution. Over the last year, we have added more than 220 customers to our roster and we see three common customer groups:

The brave that have time, resources and enthusiasm – This group is dominated by research labs and universities with graduate students willing to learn. From this group, we have seen considerable innovation and they have helped us understand what combinations of controllers work best for certain application types.

The second group is similar to the classic “early adopters” – They see the future, but need more than a sliver of the stack. They need a platform to develop on. This group is more from industry than academia with operational experience and best practices they can leverage from the server side and they have enough scripting talent to take a base system and build on top.

The last group is the traditional, more pragmatic “early majority” – They have not moved yet.  They are watching and learning and they help us acknowledge that we still have that classic chasm to cross.

When will the pragmatists jump in? I don’t have a crystal ball, but having said that, our customers have told Pica8 that we have “enough” of a solution for them to extract business benefits based on solid economics. Over the last year, I have seen us make true progress in being able to capture the leading edge of those pragmatists.


We Gotta Flavor You’ll Like

Pica8 recently changed our Linux distribution to Debian with our release of PicOS™ 2.0 over the summer.

If you are on the network side and have not explored Linux just yet, this Linux distribution timeline chart from Wikipedia is not only fascinating, it tells a story. As you can see, there are many of choices in this space! More flavors than Baskin-Robins can ever imagine!

So why did we pick Debian?  Was it because Google is using it?  Perhaps it is because we came to the same conclusion they did.  From’s own site, there are at least 17 reasons to use Debian. From our open networking perspective, let me indulge you on Pica8’s opinion of the top three reasons:

  • Debian GNU/Linux is one of the most popular Linux distributions for personal and Internet server machines. Debian is characterized as a solid Linux; subsequently, it has been deployed as a base for other Linux distributions.  Distrowatch lists 144 active Debian derivatives. Debian ranked second only to Ubuntu, which is derived from Debian, as the Most Used Linux Distribution for both personal and organizational use in a 2007 survey by We wanted to leverage a distribution that has a following, help build a bridge between traditional networking using Command Line Interface (CLI) thinking and the Linux-based operational environment on the DevOps /server side.
  • Debian supports multiple architectures and kernels.  Currently, Debian supports an impressive number of CPU architectures. To be truly hardware agnostic, Debian helps ensure we have extensibility and choice.
  • Many free tools from GCC to native support of Python. This is key because we want to ensure that we give customers options, and one of our Open Networking tenets is to promote the idea that a network device is more of a development platform than a closed box.

So why not something else like CentOS?  In fact, we like CentOS as much as Debian, but we had to choose one. We see cloud adoption of CentOS because it has a wide variety of driver support. However, CentOS in these environments is sitting on a server, so those drivers are important. At Pica8, Debian resides on a white box bare metal switch, and with this design model, we found Debian a better fit.

Enjoy our new Linux flavor – we believe it gives you the ideal developer’s platform environment for exploring open networking.



, , , ,


Know Your ASIC Inside!

We all tend to want to position a new idea and compare it to an old or familiar reference point. As the industry learns how to internalize SDN, we frequently hear comments like, “… deliver your software on a white box, right?”  And for many SDN companies the answer is yes.  And in some situations, the next question is “can you put your operating system (OS) on another vendor’s switch?”

The main difference between a white box switch and white box server is the ASIC.  Servers and switches both have CPUs, just like your PC.  Both have memory, just like your PC.  The main difference between a switch and these other devices is the ASIC.

ASICs have a long history in networking by driving scale and performance. In a cone clock cycle, very complex tasks can be accomplished. Without the ASIC, the central CPU would be overwhelmed performing those same tasks. They also created a new set of suppliers, such as Broadcom, Marvell and Mellanox, and most recently Intel through their acquisition of Fulcrum. We expect more and more specialization as the industry pivots on this SDN theme. Over the last decade, the merchant Silicon vendors have diversified and specialized products for vertical markets. For example, an ASIC optimized for the data center might have VxLAN support, while another tuned for a carrier application might possess rich features for MPLS support.

The ASIC, however, just like any other component in a system, adds complexity while simultaneously solving the issue of scale and performance.  Traditional networking vendors build embedded systems, where their merchant or homegrown ASICs are part of a tightly coupled stack including the OS.  If you are a believer in a hardware-agnostic SDN vision, then the idea of moving an operating system randomly between different ASICs has some challenges.

To be able to have true hardware agnostic software, some degree of pre-qualification is needed to ensure all the timing, memory allocation and subsystems still talk to each other as they did in that old reference design now within your chosen stack.  This is why porting one vendor’s OS to another vendor’s hardware is not viable without some mutual sharing of APIs between the ASIC and the OS.

In the new world of white box switches, we deal with this by conducting compatibility testing and agreements with merchant ASIC vendors to tune to their specific software development kit (SDK).  A white box switch from Accton may have a Broadcom ASIC while another version of the same box may include Intel and both need to be pre-qualified with the OS. This is the only real stumbling block to true software portability.  And I can’t think of an SDN vendor that has not worked this out for you. So not to worry

As we delaminate the traditional stack and enable you to either build your own stack or work with a system integrator to build a customer stack for you, you should be aware that these pre-qualification issues have been hidden from you in the past. Now you have a choice to either dive in and personalize your stack or let sleeping dogs lie.

The majority of SDN early adopters we are working with are more focused on what they do with the stack, and assume that we know how to qualify white boxes.  Over time we believe that all customers can build their own stacks based on their application environment and skill sets and truly have that parallel we see on the server side.  The real pull for this transition is the use cases that drive SDN.  So for now, it still helps to know your ASICs.  If you have specific features to implement, it is likely that not all the ASIC vendors have the same story.


, ,


While OpenFlow serves to optimize the path between source and destination for a specific flow, its challenge today is the limited number of flows that the current ASIC can support: approximately 1,000 to 4,000 flows. Some view this as a short-term limitation, certain that ASICs will soon scale up remarkably; others claim that the problem can only be solved with x86 or NPU-based solutions (Network Processing Unit), providing flexible table size—and at the penalty of performance. Some disapprove the OpenFlow design altogether, dubious a practical or economic solution will ever emerge.

Yet do we truly need half a million or a million flows to solve a significant networking problem? Today’s ASIC implementation of flow tables relies on TCAM (Ternary Content Addressable Memory) inside the chip to perform flow matching, an expensive and power-hungry technology. Considering that it is highly costly to expand TCAM’s flow capacity by even just 5,000 or 10,000 flows (resulting in a veritably expanded selling price), waiting for commercial chips to scale to millions of flows seems like wishful thinking.

At Pica8, we believe the solution lies in separating logical flows from physical flows, software from hardware: in improving the design of flow distribution rules. Given that a typical top-of-rack switch has around 50 ports, each port ends up with, at most, a few hundred flows. This is not very useful for any data center or enterprise, and far from enough for aggregation switches. But if you can classify traffic with VLAN IDs and MAC addresses—in a web hosting data center, for example—then you might need fewer than a hundred flow entries to handle the traffic. In fact, there are plenty of network applications, even at the aggregation layer, that require only a small number of flows to distribute the traffic.

Think back to the early days of the Personal Computer, when the 256KB DRAM limitation of the PC-XT forced programmers to cope with limited memory space and write only programs that used minimal physical memory. But as technology advanced, new operating systems were able to separate the logical memory from the physical memory: paging technology enabled the memory manager to swap out unused pages from the physical memory and into a paging file, while swapping in referenced pages from the logical memory. The limit of a system’s logical memory was essentially removed, dramatically improving the PC’s programmability. Meanwhile, hardware vendors gradually developed technology to expand the DRAM from 256KB to today’s hundreds of GB: an improvement factor of over 1 million times in 30 years.

We believe that same idea can be applied to the future of flow distribution. Just as how paging technology is leveraged to separate logical memory from physical memory, caching can be used to separate logical flows from physical flows. While research on the hardware component is ongoing—Stanford professor Nick McKeown is leading researchers to design new chips that can scale to half a million flows with just a 15% increase in chip space, for one—it is crucial to focus on developing SDN software and applications that free the limitations of logical flows.

So what can you do, in place of waiting, while the new ASIC-replacing chips take years to be designed and implemented? We say play with what you’ve got: save your money by avoiding those expanded-TCAM ASICs, and invest in new software instead. Redesigning flow distribution is key. SDN’s physical counterpart will eventually advance, but as history has shown: software can lead the game long before the hardware arrives. Start playing now.