Python Authors: Pat Romanski, Matt Davis, AppDynamics Blog, Donald Meyer, Liz McMillan

Blog Feed Post

Networking integration with DevOps tools: switches are not just servers

OpscodeLogo_Tag_FINALA couple of years ago at the OpenFlow Symposium in San Jose, I talked to Jeremy Stretch (@packetlife) as the event was warming up. I was not really that knowledgeable of tools like Puppet and Opscode at the time. He told me I needed to look at Puppet and integrate it with Junos. And so began my march towards these types of tools and DevOps more generally.

Since that time, I have come to appreciate a couple of things: first, the majority of network folks don't know much about these tools. While they are in use at a lot of companies, they tend to be used on the server side more than the networking side. We have a relatively small echo chamber within the networking blogosphere and Twitterverse, but none of us should fool ourselves into thinking that mainstream understanding is high. Second, applying these tools to networking in the same way they are applied to servers is horribly missing the point.

So given the first thing I learned, I feel like I need to do a little bit of education on how tools like Opscode's Enterprise Chef or PuppetLabs's equivalent work. Since we announced what our integration with Opscode looks like, I will start there.

The basic premise is that when teams set up servers, they go through a bunch of steps that include putting an OS on the box, adding applications, doing initial setup and configuration, and so on. When you are adding servers by the hundreds, these tasks eat up time that could otherwise be used to do other stuff. So Opscode has built a product that lets teams use Ruby or Python to automate the tasks.

Opscode’s flagship product Enterprise Chef is designed with this problem in mind. Nodes–be they physical or virtual servers–are meaningless until they are assigned a role in the infrastructure. What function they serve determines how they need to be set up: operating system, provisioning, connectivity, security, and application installation. Chef operates by installing client software on each node. That client software is then directed to server software that runs either within the data center or in the cloud for hosted solutions.

When a new server is provisioned, users register it on the Chef Server through a Chef Workstation-initiated client install using the hostname and IP address. Then users assign a role to the server. A role is a user-defined definition; it might be a webserver, a Hadoop node, or any other useful functional entity.

The steps that would normally be taken to configure this node are captured as Ruby code in what Chef calls recipes and cookbooks. These recipes are included in role definitions, so that whenever a new node of a certain type is initiated, the configuration steps are automatically inherited and executed. For instance, all compute servers for a specific enterprise application would require the same setup.

Recipes, cookbooks, and attributes are stored and managed on the Chef server. When a new node is brought online, the Chef server pushes the recipe contents down to the Chef client for local execution. If the setup for a particular role type changes, the recipes on the Chef server are modified, and updates are sent to all configured clients of that role type. Through Chef, Opscode has essentially automated server setup in the datacenter. But can the same principles be applied to the network that supports these servers?

So when networking companies started to see that these tools were valuable, the first thing they did was put the client software on the device and then create the abstractions required to handle provisioning. Essentially, they were making routers and switches look like servers. Users would set up recipes (or their other-product equivalents) and then initially provision networking gear the same way that they had set up new servers.

This is actually a useful thing to do. But the reality is that networking gear is not deployed in the same volumes that compute and storage servers are. And the issue with networking is more tied to edge policy than it is to basic config. And to make things more difficult, edge policy needs to change whenever applications change. This is why people have been bemoaning the network contribution to the time to make changes in the data center.

So the real goal shouldn't be to treat switches like servers but rather to tie the network (all of the network, not just the new device) to the servers that are driving its traffic.

This requires more than just putting the client software on the switch. But I didn't get that nuance for a really long time because I was thinking more about "integrating the tool" than I was about doing the right thing. Customers were asking for integration, but what they really want is subtly different.

What customers really want is for the network to be provisioned correctly when they add (or change) something on the application side. When there is a new server added or a new application turned on, the network should just come along. That means that existing network devices need to have configuration changes whenever something new is added. This goes well beyond initial router or switch setup.

When we did our integration with Opscode, the first thing we did was enable one-touch provisioning. This isn't the same thing as the "zero-touch provisioning" you see elsewhere. The one touch we are talking about is the server touch. You set up a new web server, and the network gets provisioned along with it. We can automatically establish relationships, optimize paths for that traffic, and push config to the switch the server is connected to.

But it doesn't end there. If there is a problem with the web server, you need to troubleshoot. Troubleshooting information might reside all over the network. If you know the roles of individual servers and you know what ports they are attached to, you can do troubleshooting commands that show all the web-server-related information in the network. 

The point here is that integrating tools like Chef well should result in something way more useful than just using the same tool to do initial provisioning.

I know I have been vague here, but we are doing a show and tell with this stuff on Friday: http://www.sdncentral.com/education/devops-sdn-plexxi-opscode-chef-demo/2013/09/

And there is a solution brief that provides a bit more detail here: http://www.plexxi.com/resource/whitepapers-use-cases-solution-briefs/solution-brief-affinities-in-action-plexxi-and-opscode/#sthash.7CSgEGY8.dpbs

The post Networking integration with DevOps tools: switches are not just servers appeared first on Plexxi.

Read the original blog entry...

More Stories By Michael Bushong

The best marketing efforts leverage deep technology understanding with a highly-approachable means of communicating. Plexxi's Vice President of Marketing Michael Bushong has acquired these skills having spent 12 years at Juniper Networks where he led product management, product strategy and product marketing organizations for Juniper's flagship operating system, Junos. Michael spent the last several years at Juniper leading their SDN efforts across both service provider and enterprise markets. Prior to Juniper, Michael spent time at database supplier Sybase, and ASIC design tool companies Synopsis and Magma Design Automation. Michael's undergraduate work at the University of California Berkeley in advanced fluid mechanics and heat transfer lend new meaning to the marketing phrase "This isn't rocket science."

IoT & Smart Cities Stories
The challenges of aggregating data from consumer-oriented devices, such as wearable technologies and smart thermostats, are fairly well-understood. However, there are a new set of challenges for IoT devices that generate megabytes or gigabytes of data per second. Certainly, the infrastructure will have to change, as those volumes of data will likely overwhelm the available bandwidth for aggregating the data into a central repository. Ochandarena discusses a whole new way to think about your next...
CloudEXPO | DevOpsSUMMIT | DXWorldEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
All in Mobile is a place where we continually maximize their impact by fostering understanding, empathy, insights, creativity and joy. They believe that a truly useful and desirable mobile app doesn't need the brightest idea or the most advanced technology. A great product begins with understanding people. It's easy to think that customers will love your app, but can you justify it? They make sure your final app is something that users truly want and need. The only way to do this is by ...
Digital Transformation and Disruption, Amazon Style - What You Can Learn. Chris Kocher is a co-founder of Grey Heron, a management and strategic marketing consulting firm. He has 25+ years in both strategic and hands-on operating experience helping executives and investors build revenues and shareholder value. He has consulted with over 130 companies on innovating with new business models, product strategies and monetization. Chris has held management positions at HP and Symantec in addition to ...
Dynatrace is an application performance management software company with products for the information technology departments and digital business owners of medium and large businesses. Building the Future of Monitoring with Artificial Intelligence. Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more busine...
DXWorldEXPO LLC announced today that Big Data Federation to Exhibit at the 22nd International CloudEXPO, colocated with DevOpsSUMMIT and DXWorldEXPO, November 12-13, 2018 in New York City. Big Data Federation, Inc. develops and applies artificial intelligence to predict financial and economic events that matter. The company uncovers patterns and precise drivers of performance and outcomes with the aid of machine-learning algorithms, big data, and fundamental analysis. Their products are deployed...
Cell networks have the advantage of long-range communications, reaching an estimated 90% of the world. But cell networks such as 2G, 3G and LTE consume lots of power and were designed for connecting people. They are not optimized for low- or battery-powered devices or for IoT applications with infrequently transmitted data. Cell IoT modules that support narrow-band IoT and 4G cell networks will enable cell connectivity, device management, and app enablement for low-power wide-area network IoT. B...
The hierarchical architecture that distributes "compute" within the network specially at the edge can enable new services by harnessing emerging technologies. But Edge-Compute comes at increased cost that needs to be managed and potentially augmented by creative architecture solutions as there will always a catching-up with the capacity demands. Processing power in smartphones has enhanced YoY and there is increasingly spare compute capacity that can be potentially pooled. Uber has successfully ...
SYS-CON Events announced today that CrowdReviews.com has been named “Media Sponsor” of SYS-CON's 22nd International Cloud Expo, which will take place on June 5–7, 2018, at the Javits Center in New York City, NY. CrowdReviews.com is a transparent online platform for determining which products and services are the best based on the opinion of the crowd. The crowd consists of Internet users that have experienced products and services first-hand and have an interest in letting other potential buye...
When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things'). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing. IoT is not about the devices, its about the data consumed and generated. The devices are tools, mechanisms, conduits. This paper discusses the considerations when dealing with the...