We don’t need no manifesto…

“If you don’t eat your meat, you can’t have any pudding. How can you have any pudding if you don’t eat your meat?” –Voltaire

3229643937_13bddcbf8e_oRecently there was a post on O’Reilly Radar that cited an “identity crisis” in the world of DevOps. The post called out a number of issues: namely that there are cliques in DevOps, there is nothing new in DevOps, and there is no product messaging and positioning in the world of DevOps. It ended with the call for DevOps to define itself through a manifesto, similar to the Agile Manifesto or maybe even The Cathedral and the Bazaar.

There has long been a desire to “define DevOps”. Every few months a blog post pops up that says, “Define It,” and goes on to slam the “Knights Templar of DevOps” for holding on so close to the lack of a definition. What many of these “revolutionaries” are not aware of is that the need to define DevOps has already been addressed. The de facto definition exists on Wikipedia, and it was agreed upon by the “Knights” that Wikipedia would hold the definition. This allows the definition to evolve and change over time as the industry evolves and changes. (At this point I should mention that there are no “Knights Templar of DevOps.”)

This also prevents one person or company from being the “definer of DevOps.” It gives ownership of the definition to the community. Compare that with something like ITIL, where the definitions are owned by a company and haven’t evolved much since 2007.

The definition on Wikipedia also gives guide rails for individuals and companies to work inside. If a manifesto is needed, the manifesto is up to the individual organizations that wish to transform themselves. A one-size-fits-all manifesto, as some have asked for, really isn’t effective. Each organization is different, and the guide rails give you the guidance needed to define what works best for your organization.

The DevOps Community has seen tremendous growth over the last several years. 2013 and 2014 were defining years for the community. Looking at the growth of DevOps Days, there is a clear inflection point in 2013 that shows tremendous growth in the conference series.

Screen Shot 2015-01-15 at 7.38.18 PM

Having participated in 9 DevOps Days events myself, I can attest to the fact that the growth of the community is nothing short of phenomenal. At many of these conferences I have attended, over 70% of the attendees were new to the DevOps community. Other DevOps conferences have also popped up, such as Camp DevOps and DevOps Enterprise Summit. Couple this with many conferences now having DevOps specific tracks, and I ask, “What cliquey echo chamber?”

And speaking of the DevOps Enterprise Summit, if DevOps is so poorly defined, ambiguous, and amorphous, how can there be an entire conference focusing solely on success stories of large Enterprises? The last thing enterprises want to implement is something that is untested and unproven. Yet somehow many have begun a DevOps journey and have been successful at it.

DevOps is a cultural and professional movement. It is not a product, solution, or something you can buy off the shelf. As such, it’s rather obvious that there wouldn’t a product messaging or marketing positioning. Product and marketing positioning is obviously important when you are  tryingto sell something. It is irrelevant in the context of a community oriented movement focused on transformation.

One of the most  fascinating changes I have seen over the years in technology is the interest in other areas outside of technology. When you look at transformation, it’s important to look outside of the bubble that you live in for ideas of how to change. Yes, DevOps is often focused on ideas that have been discussed at length in other circles. That is a refreshing change in some circles of tech that think that they are the end-all be-all of significant ideas. “Nobody can teach me how to do this better,” was a common refrain when I entered the IT industry, and it still is today. When I did my MBA, the most amazing experience was learning something outside of technology, and thinking about how I could take this learning and make our industry better. Looking outside, learning, and bringing that knowledge back into the community is a defining feature of DevOps.

While the original post makes some interesting points, it seems to be proposing solutions to problems that don’t exist. In some cases, these supposed problems are defining features of the movement (lack of a firm manifesto; no product definition; old wine, new skin.) which has built a vibrant and effective community that is fundamentally transforming the way companies do IT, in spite of any challenges it faces as a growing movement.

Every synthesis of old ideas and new approaches makes some people uncomfortable and others skeptical. Results are what drive adoption, and the results from a DevOps practice are well-documented at this point. Instead of seeing challenges and hoping for a central authority to solve them, let’s continue on the path of sharing our best ideas with one another and constantly improving.

Thanks to Pete Cheslock, Bridget Kromhout, Stathy Touloumis, and George Miranda for the feedback on this post.

The Goat Farm

goats

There’s been a lot of focus on DevOps in the Enterprise as of late. The DevOps Enterprise Summit showed that DevOps in the Enterprise is a hot topic, and that large enterprises can be successful with DevOps.

One thing that critics say we need more of is prescriptive guidance on implementing DevOps in the Enterprise. This has created this pseudo-movement known as “Enterprise DevOps” to try and fill the perceived gap.  But what the DevOps Enterprise Summit has shown us is that there is sufficient guidance out there to transform enterprises with the ideas of DevOps.

However, a recent podcast highlighted to me that it is often hard for some people to find the information they need to get started with DevOps. That is why the DevOps Enterprise Summit was great. It consolidated Enterprise success stories into one place, and is easier to find.

In continuing this vein, I am happy to announce that I’ll be starting a new podcast on DevOps in the Enterprise. I am also excited to be joined by Ross Clanton on this journey. Ross is an Ops leader at Target, brings years of experience in Enterprise IT, and has been a champion of DevOps transformation. Ross and I will focus on real world DevOps transformations in the Enterprise by interviewing leaders in the space.

We will launch our podcast in mid December, so watch this space for our first few episodes of “The Goat Farm”.

DevOps in the Duck Production Industry

This is a collaborative blog post written by attendees at DevOps Days Belgium 2014. As such, feel free to repost under a Creative Commons Attribution-NoDerivatives 4.0 license. Please retain this header as well.

release-the-quackin-memeIn my 25 years of experience in DevOps, there has never been such a defining movement as DevOps for Ducks. Devops for Ducks is showing up in Gartner’s magic quadrant for the first time this year.  Traditionally, the problem with Ducks are that they are fowl and are not goats. DevOps makes everything better. Even food.

But as we look at the production of quality Duck in the world we begin to understand that it is fundamentally broken. That’s where DevOps comes in. DevOps, has the power to change how we produce ducks in several environments. It allows people to collaborate on the Ducks. Zhang Chi Woo of China, creator of the Duck Production System – or DPS for short — has revolutionized not only Duck production, but the entire way the world thinks about Ducks.

Metrics and Duck Logs

When initially planning the pipeline for Duck production, you must take the “Duckly Metrics Factor” into consideration. As the “DMF” is defined, it will provide insight into the duck and allow you to make sure it is DevOps.

Most importantly, no one should forget that if it walks like a duck, and quacks like a Duck, then it is certainly a Duck, so we should define a metric for how our production Ducks are walking and quacking. Try Graphite. Ducks love it. There is even a collectd plugin available for duck egg incubation systems (well the only problem is all the native dependencies it pulls in).

The recently created DuckCare system closely monitors all the Ducks and can notify based on key log entries. If a Duck needs attention care workers are dispatched right away through DuckDuty.

Deployment Pipeline

Whereas before we often had to wait for eggs to hatch, a process which takes valuable productive time away from the duck producing units, we now foster a culture of continuous duck production. This way of working will inevitably lead to things that break. But that’s ok. We don’t point the finger at somebody who made a mess of dropping duck eggs in the kitchen. In the DevOps culture, we do a standup, avoiding the sticky part of the floor, and then we all get a mop, clean up and go release some more ducks in the water.

An important part of the DPS, is constant exercise of Duck functional testing at any stage of the producing unit, because we all value quality. With visual duck analytics we see that people in the industry are finally getting all their ducks in a row. As in many things, visualizing your work is important, and visual duck analytics allows this.

Cross Duck Collaboration

At “Duckly”, we’ve been using Flowduck to speak to Ducks on a daily basis. It replaced the previously used HangDucks (which is just mean), Hipduck (which was nice at the beginning), IRDucks (which eventually made the ducks radioactive), Duckfire (OI!) and Duckmail (which is way faster than snail mail of course).

Collaboration becomes hard when trying to control more than 2 or 3 teams producing Ducks on a daily/decadely basis. That’s why we wrote “Duckme”. Duckme (http://duckhub.com/duckly/duckmemore). is an open-source DevOps-is-everything tool. It provides an easy and accessible way to create “Duck Correspondants”. Those DC’s, are in charge of making sure DuckDevs and DuckOps talk to eachother (and to ducks). Duckme provides a DUCKLESS API for sending eDucks and altering Ducks.

Duck Culture

You don’t need culture to produce ducks (especially DevOps). A toxic culture will also work just fine. Make sure you put high pressure on the Ducks at all times.

Ducker

As part of these worldwide changes to the flow of Ducks, new tooling for Ducks has evolved. Ducker provides the containerization of Ducks through lightweight methods known as Rubber Duckies. While the technology for Rubber Duckies has been around for many years, Ducker makes Rubber Duckies easier to consume through technologies such as DuckerHub. Additionally, Ducker overcomes the limitations of traditional Duck Virtualization Platforms (DVP), and Duck Configuration Management (DCM) systems.

Ducks For Developers

To make duck development accessible to all (DevOps), we’ve developed Duckrant. Duckrant lets you spin up virtual ducks, provision them and destroy them once you’re done (The virtual ducks, not the ducks themselves, mind you. WE LOVE DUCKS! is our moto.)

http://duckhub.com/duckly/ducks/ contains an unimaginable amount of base ducks for you to experiment on (e.g. Duck with Chef, Duck with Ducker, Basic Ducks and the notorious Duck Quickstart.)

Duck Future

With the growing movement of DevOps for Ducks (initially started by Patrick Duckois), it’s important that we begin to think about the worldwide implications of the free flowing of ducks. With the great ducks come great responsibility, and that is the great challenge as we begin to think more about what we call DuckOps.

Wax On, Wax Off

“Wow, you’ve run an ultramarathon. That must have been hard.”

“Kinda, I guess.”

That’s often how the conversation goes when someone finds out I’ve run an ultramarathon (any race over 26.2 miles). It’s an interesting conversation because people are often fixated on the execution of a single event in my life. An event that lasts 7.5 hours over the course of one day. What people don’t say is, “Wow, you trained for six months, five days a week, in the cold and snow, spending 7-8 hours a week running, cross training a fewtimes a week in order to run that far at one time?”

People are often focused on the event, and not what it takes to get to the event. Society focuses on the next great shortcut to achieve something better, cheaper, faster. The thing many of us don’t want to recognize is that the tried and true method of achieving something is hard work and perseverance. You can’t effectively run a marathon or ultra without putting in the hard work of training. You must spend hours a week training. You must slowly build up your miles. You must not over train or else you will get hurt. Sure, there are different methods of training, such as Crossfit Endurance which focuses less on running and more on building endurance, but you still have to train. Anything else will result in failure.

I’m reminded of The Karate Kid (the 1984 original). Daniel was bullied and felt he needed to learn karate. He found Mr. Miyagi, who was willing to teach him. Mr. Miyagi sets Daniel to work painting his house, waxing his car, and staining his deck. Daniel doesn’t want to be a servant; he wants to learn karate! In his overanxiousness to learn, he confronts Mr Miyagi about doing all these chores. We end up finding out that Daniel was learning karate all along.

And now, like with everything, people want the shortcut to DevOps. They want the secret sauce that they need in order to transform their business and IT. They want the certification they can earn in a few hours of work to prove they can DevOps. They get frustrated when we tell them there is no easy path. They get frustrated when we tell them that they need to just start doing something, anything to start improving their work. As with the runner, you just have to start running to start training for a marathon.

We now have over five years of history in the DevOps movement. There is plenty of information available for “how to get started”, but the single most important thing a newcomer needs to realize is that it takes a curiosity, a desire to learn, and a willingness to listen to other people that might be different than you. As Aneel Lakhani said in his below ignite talk, “…the single strongest signal that we have something to learn is the fact that a difference exists.”

And there’s the rub with the “classification of DevOps” that people are undertaking. This classification seeks to alienate groups because differences exist, not unite them because they have something to learn from each other. After reading the latest post on the subject, a friend tweeted about going through those exact same things in a 100 person company right now. If enterprises solve problems in their own little world of DevOps, how do we ensure that this knowledge flows down to the “non-enterprise” DevOps practitioner? This stratification divides us rather than unites us.

When we willfully disregard core tenets of the movement, such as Collaboration and Sharing, we are essentially back to where we’ve always been, and no one moves forward. That is why hearing the stories coming out of the DevOps Enterprise Summit is encouraging. Large companies are opening up and telling their stories of transformation. Large companies are proving that the ideas of DevOps work, no matter your size.

Veteran of the Process Wars

Tokyo. I’m still in Tokyo. I wake up, rub the sleep from my eyes, and roll out of bed. As I rise I take a quick look at my watch for any new emails. Nothing. There’s not much email anymore. Not since the process wars of 2016.

Many people won’t talk about the process wars. They changed the way most of the InfoTech industry works and how we are allowed to think of our jobs. In the past, I might have been responsible for running hundreds of servers. Now, the machines are in charge. I’m only allowed to feed the machines code.

We are grouped together in small teams that write code as a unit. We are “kept in line” by dogs that are trained to attack us. They say it’s for our own good. That it’s for the betterment of industry. If we try to touch the servers, if we try to do anything other than write code and feed it to the machines, the dogs will bite us.

We are kept in line by process and controls, but not like we used to be kept in check. Before, we used to have weekly meetings reviewing the work we wanted to perform. The meetings were always a joke amongst my peers. Typically they were run by a VP that had no clue what we wanted to change on the machines. If we wanted something bad enough, we could social engineer our way to what we wanted.

No more today. When we write the code to feed the machines, we have to write tests. They tell us these tests are for our own good. The tests confirm that the machines are set up exactly how they want them to be. We actually write the tests first, and they always fail the first time through. Then we write code to bring the tests into compliance.

If the tests weren’t bad enough, we also have automated tests to make sure our code conforms to “Style Guidelines”. They tell us these guidelines are to ensure conformity and consistency. I say they are there to hold us down and control us. They also require us to “lint” our code before we feed it to the machines. This once again is to ensure “conformity”.

Even with all of this verifying for conformity, we still aren’t allowed to directly feed the code directly to the machines. We must check the code into a repository where more machines take over to verify that we have successful tests and we meet the conformity guidelines. The machines also verify that our code works with code written by others, in my group and other groups.

Then, a Conformity Checker reviews my changes to make sure they are compliant with the policy. We are no longer independent; we are no longer allowed to game the review board. We feed the machines compliant code or else we end up on the streets. Three strikes and we’re out.

Which is why I’m in Tokyo. I’ve had two strikes at my company in the United States. They shipped me over to the Tokyo division for reeducation. Japan culture is heavily based on order and process. During the process wars they helped lead the revolution. Many of the compliance tools were written by or enabled by Japanese technology leaders. Yukihiro Matsumoto (codenamed Matz) was responsible for designing the programming language I now feed to the machines. Gosuke Miyashita wrote the tools that I am required to use to test my code before feeding it to the machines. And then there was Kohsuke Kawaguchi, the creator of the master machine that ensures all the code is compliant, automatically with no humans to game the process.

It’s all very neat and orderly now. I take requests for code from the owners of the machines, I write compliant code, and the machines automatically verify my work. Eventually the machines apply my code, and the owners get exactly what they want. I’ll wake up the next however many hundreds of mornings and do just this. No more, no less. It’s all very neat and orderly now.

Labeling DevOps Hurts the Movement

If you label me, you negate me.” –Kierkegaard or Dick Van Patten

INo Labels posted a rather inflammatory tweet last week (shocking, I know) that attacked the concept of “Enterprise DevOps.” For those of you that don’t know there has been a movement, led mainly by traditional enterprise software vendors, to define a unique branch of DevOps that focuses solely on specific challenges that Enterprise IT shops face in their journey to a DevOps operating model. The theory being that traditional DevOps ideas won’t work in Enterprise IT, and there must be Enterprise specific ideas for DevOps.

This tweet gained a few favorites and retweets, but it also ruffled feathers. The most boisterous response was from a Benjamin Wotton. Now I have never met Benjamin in person that I can recall, nor had we ever before interacted online. Benjamin’s problem with my tweet was that he uses the term “Enterprise DevOps” all the time, and that he is proud of it. He also pointed out that his definition encapsulates the fact that DevOps in the enterprise is more complex & multifaceted. That I can agree with, and I pointed out Rob Cummings’ idea, that DevOps in the enterprise just requires more patience, which in my mind takes into account that Enterprises are more complex and multifaceted.

But that is not the tweet exchange that interests me the most. Patrick Debois responded in the middle of this exchange and made a compelling point by questioning whether I was being inclusive. To me that equaled the Godfather of DevOps putting a horse head in my bed.

After I thought about this, the whole idea of “Enterprise DevOps” actually creates an exclusive environment that the DevOps community fights hard against. “Enterprise DevOps” creates an environment that says Enterprises can’t learn from “Standard DevOps”, and reinforces this idea that enterprises can’t learn anything from non-enterprises (web only shops, startups, “webscale” companies, etc).

But this is completely wrong. Netflix was often accused of being a unicorn; this mythical creature that no organization could become. As the industry has progressed, Netflix has instead become more and more a model of success that Enterprises try to emulate. I have worked with major banks, retailers, software shops, and more that all seek to emulate some aspect of the Netflix model. While they all don’t seek to become a unicorn, some seek to have a horn, some seek to have magical blood, and some seek to crap rainbows.

The fact of the matter is there is no one true DevOps. There are operating models that we might seek to emulate, but every organization’s needs are different and approaches require tailoring to the current needs and maturity of the organization. This is as true for the 1 person shop as it is for the 10,000 person shop. Cookie-cutter Enterprise IT operations has always been a pipe dream being sold by those who sell really expensive software and professional services contracts. It won’t happen with ITIL, Business Service Management, DevOps, or any other buzzword of the day.

The idea of DevOps is about listening. It’s about listening to those that are smaller than us; it’s about listening to those that are larger than us. It’s about having an open mind that someone (internal or external) might actually have an idea that can help us, and having the flexibility in our organization to rapidly experiment with that idea.

As Andrew Shafer is fond of saying, “We get the DevOps we deserve.” If you choose to ignore the advice of others, if you think you can learn nothing from the smaller shops and only rely on “Enterprise DevOps™” advice, you miss out on a world of opportunity and ideas that could help you improve. In the end you’ll have your DevOps: the one you deserve.

 

Get Your Head Out of Your aaS

3815168722_faee10cf62_bI’ve been floating between the worlds of Cloud and DevOps for a while now and it is interesting to see the Cloud world finally start to realize the real value is in DevOps. It’s great that more people are starting to pay attention to this cultural and professional movement. What is not great is how the Cloud experts tend to get wrapped up in some debates that are trivial and meaningless, in the larger scheme of things. Take for instance two persistent debates I am seeing over IaaS vs. PaaS, and then which PaaS is better. I hate to be the one to break it to these camps, but it doesn’t matter; at the end of the day you are selling plumbing fixtures that crap flows through.

To understand what I mean, lets take a step back. In 2008, I started pursuing my MBA at The Ohio State University. One of the core requirements of the degree was Operations Management. In Operations Management, you learn manufacturing optimization through ideas such as Lean and Six Sigma. The book “Learning to See” was part of the course material and it focused on optimization of manufacturing processes through visualization, also known as Value Stream Mapping. As the course progressed, I had a personal epiphany. As we kept walking through manufacturing processes, and Value Streams, what I quickly realized was that the work we did in IT was all about manufacturing a good or service someone would be consuming. Automation in the IT world is about (or should be about) optimizing these Value Streams and (hopefully) eliminating waste from the system. My Operations Management course really taught me to see (pun intended) and to think differently about how we worked in IT.

I took this new found knowledge back to my work where it was summarily ignored by my boss and coworkers, and lacking support I shelved my ideas. Little did I know many of the Lean principals I had learned would be at the forefront of how IT is changing today, and was already being changed at that time in 2008, I just didn’t know it.

When somebody asks me what DevOps is, I often respond with the simple idea that “DevOps is about increasing the flow of work through IT.” I borrow this idea heavily from “The Phoenix Project“, but I find it is the most simplest way to capture the essence of this cultural and Imageprofessional movement. And that is where Value Stream Mapping and the ideas of Lean come into the conversation. Books like the “The Phoenix Project“, and notable DevOps contributors such as John Willis expound the values of these techniques to optimize the IT Manufacturing chain, be it Development work or Operations work.

Value Stream Maps are relatively simple. They identify the flow of a raw material through Screen Shot 2014-04-03 at 11.07.33 PMvarious processes that add value. They also identify areas of waste in the system, and they help in building the Future State Map, or the Value Stream that you want to achieve in the future after optimizing the system. The most basic and valuable thing about Value Stream Maps is how they allow you to easily visualize your work, and once it is visualized it is easy to understand and optimize.

If you look at the first current state map, you can easily see how relabeling the boxes to reflect common IT tasks, say in a server build process, makes this a powerful tool for IT. Replace the box names with another process – maybe code build, testing, and release – and you see once again how Value Stream Mapping is a key tool in fixing our broken IT.

Now that we’ve established a method for the optimization of our IT processes, let’s go back to thinking about Cloud and the debates around Iaas, PaaS, and the PaaS vendors. Take the second Value Stream Map. Say this diagram more accurately reflected server builds and the time it took to install an OS was one hour. We optimize this process through our IaaS based Cloud, public or private, and get the time down to 5 minutes. That is awesome, we’ve saved 55 minutes and really optimized that process. Go team!

If “premature optimization is the root of all evil”, then local optimization is the Devil’s half brother. In the above example we saved 55 minutes, but the total time of work flowing through the system is still 67 days, 23 hours. And that is where we come back to Cloud. IaaS is a local optimization. It is great, it is awesome, but it is a very small piece of the puzzle. PaaS is another local optimization, but instead of optimizing one process it optimizes three or four. Which is great, but many IT organizations are going to “adopt Cloud for business agility and speed, then be sadly surprised when their local optimization does little to fix their screwed up system. Cloud can be a great enabler, but it is only a small piece of the larger system. It is high time more of us focus on the larger system.