The Goat Farm – Episode 11 – DevOps at Asurion

Asurion is definitely one of those companies that you didn’t know existed until you needed to use them. Like that time you dropped your phone in the toilet by accident. In this episode, we speak to Jon Klein of Asurion to hear how they’ve started adopting DevOps principles in their work at Asurion.

Jon shares with us how they got started, how they are continuing their success, and how they’ve even gotten the attention of their CIO. Jon gives a good picture of what it takes for Operations teams to refine their work internally, to make them more effective for their internal customers.

Download MP3 – iTunesStitcherRSS

Show notes:


 

jonkleinJon Klein – TwitterLinkedIn

Jon started his career as “the” IT guy at his family’s construction supply and equipment dealership back in 2002, handling everything from desktop support to network administration. He fell in love with Open Source, playing with Linux in a friend’s basement and freelancing on the side. He eventually took a job with a contracting firm as a Linux Admin for ServiceBench, Inc in 2006. Now nearly a decade and 2 acquisitions later, Jon works for the parent company, Asurion and has been on the forefront of the DevOps movement, building cross-functional teams and breaking down org silos. He currently runs a cross-functional team of infrastructure engineers and developers dedicated to the rapid delivery of platforms and infrastructure.

DevOps at IBM – The Goat Farm – Episode 9

How does IBM manage to run web sites for some the World’s largest sporting and television events? With the practices of DevOps of course! In this episode, Ross and Michael talk to Brian O’Connell of IBM.

Brian tells us of his journey to DevOps practices through stumbling onto the ideas of Chef and Infrastructure as Code. We talk about the cultural shift required when it comes to who owns delivery of changes and ownership of those changes. Brian also tells us how they leverage the “build, measure, learn” product development loop.

The sites Brian and team help run are some of the more high profile, and highly visited sites in the world. Brian talks about the challenges when trying to introduce DevOps to such high profile sites, and mistakes that were made along the way. We also talk about some of the tooling Brian and team use, and how they effectively deploy enterprise software packages.

Download MP3 – iTunesStitcherRSS

Show notes:


Brian O’Connell – TwitterLinkedInBrian O'Connell

Brian O’Connell is a Senior Technical Staff Member at IBM that leads a team focused on DevOps, predictive analytics, big data, and cloud technologies.

Brian joined IBM in 2001, starting as a software engineer. He built many software systems to support the continuous availability and events infrastructure.  His expertise includes architecting and developing scalable server applications, concurrency, advanced visualizations, and big data.

From 2007 until 2011 Brian was the lead infrastructure technology advocate and designer for the World Wide Sponsorship Marketing (WWSM) client. His role included strategic technical direction, evaluating technology pilots and the end to end delivery of highly visible web events. In that role, he successfully delivered all IBM sponsorship web sites including The Masters, Wimbledon, Roland Garros (French Open), US Open Tennis, US Open Golf, Australian Open, and The Tony Awards. Brian designed systems to manage the infrastructure and applications used by the client including a focus on defining plans, strategies and architectures for the installation, operation, migration and management of complex information systems.
Brian has had more than 250 patents issued, is an IBM designated Master Inventor and a Franz Edelman laureate.

Adrian Cockcroft of Battery Ventures – The Goat Farm – Episode 8

In this episode we talk to the famous (or infamous) Adrian Cockcroft of Battery Ventures. Adrian is known for his work at Netflix and his work to migrate them to a Cloud first strategy, then before that for his book on Sun performance tuning.

Adrian has been doing a lot of work talking to CIOs of large enterprises and helping them understand where ideas such as DevOps, microservices, Cloud are taking the industry. He allows tells us how he is helping CIOs realize how their IT organizations must transform to adopt these new ideas. This episode is all about how the horses are growing horns to become the unicorns.

(Editor’s note: We are really sorry about the audio on this episode. Adrian was in Portland, Michael was in Amsterdam, and Ross was in Minneapolis. While we could have cut a bunch of the bad audio, the content was so good we didn’t want to drop anything. Apologies.)

Download MP3 – iTunesStitcherRSS


Show Notes


Adrian Cockcroft – LinkedInTwitter

Adrian Cockcroft has had a long career working at the leading edge of technology. He’s always been fascinated by what comes next, and he writes and speaks extensively on a range of subjects. At Battery, he advises the firm and its portfolio companies about technology issues and also assists with deal sourcing and due diligence.

Before joining Battery, Adrian helped lead Netflix’s migration to a large scale, highly available public-cloud architecture and the open sourcing of the cloud-native NetflixOSS platform. Prior to that at Netflix he managed a team working on personalization algorithms and service-oriented refactoring.

Adrian was a founding member of eBay Research Labs, developing advanced mobile applications and even building his own homebrew phone, years before iPhone and Android launched. As a distinguished engineer at Sun Microsystems he wrote the best-selling “Sun Performance and Tuning” book and was chief architect for High Performance Technical Computing.

DevOps at Standard Bank – The Goat Farm – Episode 6

It’s been awhile since the last episode, but we are back with a bang! In this episode we talk to Standard Bank, the largest bank in Africa, about the challenges they faced in taking a DevOps approach in their organization.

Compliance at Velocity was one of the tracks at this year’s ChefConf. Our guest Josef Langerman discusses corporate compliance and the scale of how broad and wide regulations can affect an enterprise’s approach to DevOps, leveraging Agile, and delivering the right solutions for customers/guests.
Listen to Josef’s recount of Standard Bank’s journey – including discovery of change
champions, driving a new, DevOps culture, and establishing a set of themes to
continuously improve and advocate for new ways to satisfy the company’s needs.

We recorded this episode at ChefConf 2015 and we were happy to have Jason Walker of Target as our guest host. If you want to find out more about the topics we discussed, check out the links below.

Download MP3 – iTunesStitcherRSS

Guest Info:

Josef Langerman – LinkedIn – Twitter

Information Technology executive with experience across the Airline, Retail and Investment Banking Industries. My focus is on maximizing development throughput and large scale software development using DevOps and Agile approaches. I am also passionate about higher education and IT research. My teaching and research focus is on Project Management and Software Development.

 

Show Notes:

Jason Walker at ChefConf:

Rachel Chalmers at ChefConf:

 

Taylorism, Hating Agile, and DevOps at CSG – The Goat Farm – Epsiode 4

It’s time for another episode of The Goat Farm. On this episode Ross and I talk about: Why do Managers Hate Agile?Taylorism, hiring and the problems with specialization, and we chat with Scott Prugh about his experiences with DevOps at CSG.

We also touch on the idea of running internal conferences to help spread the word of new technologies within the company, and Ross shares his excitement over the wildly successful DevOps Days at Target. Stay tuned for an upcoming episode where we will talk more about running your own events.

Download MP3 – iTunesStitcherRSS

Guest Info:

Scott Prugh, Chief Architect – TwitterLinkedIn

Scott is responsible for the overall Platform Architecture and Development in North American Cable. Scott led the architecture teams in developing many of CSG’s next generation assets including Product Configurator and Order Capture.  Scott has been instrumental in leading change at CSG implementing Lean, Agile and DevOps practices.  These changes have resulted in 90% reductions in release impact. Scott is a Lean Practitioner and certified as a Scaled Agile Framework Program Consultant. Previously, Scott was CTO of Telution and built the core runtime and billing architecture for the COMx product suite.

Here is Scott’s talk at the DevOps Enterprise Summit 2014

 

What if Everything We’ve Been Doing is Wrong?

60-wrong-way

After I wrote my last post, I was talking with Donnie Berkholz as we traveled to FOSDEM. Donnie commented on how powerful of a post it was, yet it left the reader hanging. He, and other readers, wanted more. So I’ve taken the liberty of breaking down more of the reasons Enterprise IT needs a “special kind of DevOps” as posted by Andi Mann. I don’t want anyone to think I am picking on Andi personally. Rather, his post reminds me of all the excuses Enterprises give as to why “We can’t change”. As Mick told Rocky, “There ain’t no can’ts!”

  • They cannot achieve the same levels of agility and personal responsibility as a smaller or less complex organization.

Why Not? Principles that teach agility and speed have long been used at large companies such as Microsoft. (Yes, feel free to say Microsoft is a bad example, they are still one of the world’s largest software companies.) Additionally, if one doesn’t want to take personal responsibility for what they produce for a company, maybe they are in the wrong job for the wrong company?

  • They cannot stream new code into production and just shut down for a couple of hours to fallback if it fails.

This is fool-hardy to begin with. The goal of methods such as Continuous Integration is to be constantly building releases and testing them to catch problems before they are released to production. Also, the idea is to test small changes, so you know exactly what breaks, rather than large chunks of code. Large enterprises “cannot stream new code” because they haven’t built the necessary flows in front of production releases to effectively and efficiently test and verify code changes. This requires IT organizations to fully automate their processes all the way down to server builds, a process they often are incapable of doing because of an attachment to the “old way of doing things”.

  • They rarely ever have ‘two pizza teams’ for development or operations (indeed, they are lucky if they have ‘two Pizza Hut teams’).

The size of the team is nearly always irrelevant. Within each Pizza Hut there are tables, and each table consumes the pizza buffet. The goal of DevOps is to increase the flow of the work through those tables so the teams can eat their pizza and leave quicker. As I’ve said before, focusing on the Silos is the wrong way to solve the problem. Rather focus on the grain elevators that move the grain to produce something meaningful.

  • They cannot sign up for cloud services with a credit card without exceeding their monthly limit and/or being fired.

Get an MSA/PO with the cloud vendor or build a Private Cloud. Cloud or no cloud, building strong automation on top of existing VM or server infrastructure can help alleviate many problems in service delivery.

  • They cannot allow developers to access raw production data, let alone copy it to their laptop for development or testing.

Scrub the data. DevOps or not, this is a problem that we’ve solved years ago. When I worked at a major e-commerce site, real data was often required for testing, but that data was always cleaned of any sensitive PII. This is not an issue that is unique to DevOps.

  • They cannot choose to stream new code into production in violation of a change freeze, or even without the prior approval of a CAB.

Once again, one assumes that DevOps is all about willy nilly pushing of code to production. One aspect of DevOps is about increasing the flow of the work through the system by optimizing the centers where value is added. As I’ve discussed before, principles and practices of DevOps actually help things like Change Control.

  • They cannot just tell developers to carry pagers ‘until their software is bedded in’ (not least because their developers have always carried pagers, and on a full-time basis).

If Devs already carry pagers, then they’ve already been told to carry pagers, hence, “they” can indeed tell their Devs to carry pagers. Additionally, bedding in of the software should happen in the lower environments as discussed previously. If you’ve done things right before production, pagers become a tool that is used when things go really badly. It’s a form of monitoring and incident response that becomes meaningful again because you aren’t being paged for endless break fix work.

  • They cannot put developers and operators together because one team works 24×7 shifts in 7data centers while the other works 16-hour days in 12 different locations.

Well, good, they at least have 16 hours a day together. Highly distributed remote teams are becoming more and more common. Technology is evolving to help bring this concept of remote work and people are finding creative ways to work around it. I’m also against the idea that DevOps is all about merging dev and ops onto one team, because that is not the point. The idea, as already stated, is to increase the flow of work between Dev and Ops and build a culture of continuous improvement between the two groups (three groups if you include the business). Dev, Ops, Business, who gives a shit. The point is working towards a common goal, no matter where you sit.

What large IT shops cannot do is be satisfied anymore with the status-quo. They cannot accept the ways of the past any longer, and they have to start thinking about blowing up their way of doing things. They cannot let the castles and fiefdoms of the past get in the way any longer.

I think the single most powerful question any IT shop can ask themselves is, “What if everything we’ve been doing over the last X years is completely wrong?” Start there, and reevaluate everything you’ve been doing to achieve (or not achieve) the results your customers require.