Project Razor: EMC and Puppet Labs

Today EMC and Puppet Labs are releasing via open source community contribution, a key module that serves to ground the last mile of the DevOps process. Project Razor is a tool that I’ve personally been looking for, for years. Razor is a service oriented (RESTful), model based, and policy driven “Operating System provisioning tool” which use their native installers for hardware compatibility.

During the project design meetings, we determined that we wanted a tool that could be:RazorBlade.jpg

  • a programmable power controller, exploiting BMC/IPMI or an Intelligent powerstrip
  • a capability discovery tool, using a downloadable kernel to take and maintain system inventory
  • OS delivery processor, that is state machine driven, using native OS installers (ESX, Ubuntu, SLES, RedHat, even Windows), delivering what we might call a targeted “bare minimum” OS (with just enough package to join the MCollective)
  • a consistent handoff, of the explicitly provisioned computer, to the DevOps environmentm for configuration control and customization
  • and have a pluggable, easily extensible state machine that could act as a framework for downstream work (since no one solution fits IT).

Having architected large scale clouds like Sun Cloud and developed for Cloud Foundry based platform environments, I found myself spending a huge amount of time provisioning bare metal boxes with an ever changing portfolio of hardware and Operating Environments. Then, once I put the box in service, I had another set of scripts (Post Install) that would consistently provision the application stacks. For me, there were really 2 things that were inefficient, one was the fact that the installation models (usually scripts) were brittle because of machine differences, software builds/updates, and configurations, and the second was that I kept cobbling together all the different pieces: ipmi, tftp, httpd, dhcpd, syslog, + a ton of scripts. The second failure was the fact that each OS distributor employed their own provisioning systems isolated to their OS; but my environment needed 3 linux variants, VMware vSphere 5, Windows 8 and even Solaris.

Constantly looking for a better way, we evaluated a variety of DevOps tools to replace the “post OS” scripts, and settled on Puppet from Puppet Labs. For me the ability to finally get back to my programming roots within an traditional “System Administration” context was just the flexibility that we needed. We could build a first class MVC, exploit re-use, modern programming languages and an amazing flexibility of deployment.

201205090727.jpg In my view, this is an environment designed, not for the traditional system administrator, but for the hyper-productive, administrative developer. Not catering to the lowest common denominator, the flexibility afforded by the PL approach can help to tear down some of the old walls between development and deployment in a valuable and empowering way.

We built Razor to integrate with the DHCP and network boot services, to support the cycling of power, and the taking inventory information.  Interestingly we used Facter here to deliver, dynamically the “inventory taking modules” so that we could keep the inventory kernel small, but support the dynamic delivery of key sensors just in time.  The inventory information is registered with Razor, and the system is put in an wait state until Razor decides what to do with this node.

Razor takes the inventory and provides a set of matching expressions to help one “tag” key features in order to add this machine to the appropriate sets.  Razor then enables the administrator to define a model = what they want done, and a policy that maps the model against the tags, so that when a tag is matched a policy is triggered and a model delivered.  The net result, if the box fits a known profile, the model (OS + configurations) is delivered and Razor then deals with the boot / iPXE delivered OS images, and HTTP delivered customized package sets to that node (including the Puppet Agend), and when all is done (and the second reboot happens), hands this node off to Puppet for subsequent customization.

What’s cool about this solution? Razor with Puppet now support the end-to-end configuration control capability for a brand new “bare metal” node, into a working, deployed, managed, configuration with repeatability, auditability, and flexibility.

Why open source? For EMC, we see that the cloud market place is emerging quickly, and further recognize that the transparency and open-ness of frameworks will be pre-requisite to broad adoption for innovation and safety, respectively.  This just the first of many contributions that EMC technology team will make to the open source communities in support of massive scale cloud computing, scale out architectures, and the improved automation and operational excellence. Our architectural patterns for these environments are strongly influenced by DevOps strategies and SideBand controllers as typified by the OpenStack architecture. In areas like Quantum (Software Defined Networks), Cinder (Storage/Volume Controllers), and Nova (Compute Controllers). To add to this list, I believe that Puppet + Razor become a “Configuration Controller, which, can serve as mature model based policy driven glue-ware for cross controller customized and configuration managed integration.

Not to sell our VMware partners short! We believe that there is an exceptionally strong role for Razor + Puppet = top/bottom DevOps in the ability to stand up both VMware clusters in the vCloud model, but also to scale out the delivery of integrated applications on top of that environment.

We’ll be showing a really neat demonstration of Puppet and Razor at EMC World today, so please do tune in to “Chad’s World“. And, I do know that some now want more detail, but I want to call out the two key EMC developers who made Razor possible, namely, Nick Weaver and Tom McSweeney. Both of them worked tirelessly first on trying to get the other industry configuration controllers (Baracus from SuSE and Crowbar from Dell), and then to architect the pluggable dynamic state machine for Razor detailed here in Nick’s Blog. And of course the Puppet Labs team including Nan, Dan, Teyo, Nigel, Jose, Scott and Luke who integrated with our team seamlessly and made this possible.

Lastly, I need to call out Chuck Hollis, who is constantly looking for the cool factor in the industry and in EMC, who has detailed Razor in his blog.

8 thoughts on “Project Razor: EMC and Puppet Labs

    1. hmmm, grammar police? 3 linux variants [ubuntu, sles & rhel], VMware … Windows.

      And I do agree Windows != linux, shame vCenter still requires it for now.


  1. hi,
    I am facing some issues installing razor. please help me with the problems.
    1. My proxies are disabled to download from the site directly using the command

    #puppet module install puppetlabs/razor –version 0.3.0

    2. I downloaded two razor images…
    — from this site I downloaded the available razor master
    — from this site I downloaded the another puppetlabs/razor (above command only points to this site )

    Now, I am not able to install any razor module downloaded. I am not able to find any excutable file? Can you please brief me with the steps how to build razor environment and install it? What all packages I need to download? Once I will install it I will be thruogh withe configuration and access of the tool.

  2. “In my view, this is an environment designed, not for the traditional system administrator, but for the hyper-productive, administrative developer”

    I my opinion system admins were doing fine on their own with perl and shell scripting and irrespective of all the hype puppet have not made deploymnet less time consuming because most IT shops already had other automation tools. Puppet is just another tool to automate configuration but there are many other and in fact simpler ways in to do the same in traditional unix based systems.

    1. Loks, I cannot disagree, I didn’t mean to cast any disparaging remarks on administrators. In the face of ever larger scale environments, clouds and the like, we are seeing a massive expansion of OS’s (as deployed in VM’s or web scale) that can no longer afford personal attention. One presenter I recently saw said “name your pets, number your livestock.” In these conditions, we do need to scale the administration role through automation to deal with the “too busy for other things”, or “allow me to test and then version at scale” models as afforded by puppet labs or Bosh/Foundry. These allow scripts to become “teamable” and allow for community based innovation for further acceleration.

      HTH, dan

Leave a Reply