Dave Jilk

Subscribe to Dave Jilk: eMailAlertsEmail Alerts
Get Dave Jilk via: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn

Related Topics: Virtualization Magazine, Desktop Virtualization Journal, DevOps Journal

DevOpsJournal: Article

The Transition from Virtual Appliances to Model-Driven Deployment

Model-driven deployment is the way of the future and one of the hallmarks of a true devops shop

Virtual appliances are binary files representing the complete and fully configured state of a server. They are a popular approach to the deployment of application software in virtualized data centers as well as in public or private clouds, because once the "golden master" is created, deployment of that server state is fast, easy, and repeatable. However, there are significant disadvantages to the use of virtual appliances. Model-driven deployment is a new approach that largely eliminates these issues, better matches today's fast-moving, agile environments, and takes full advantage of the flexibility of virtualization and cloud computing.

The Downside of Virtual Appliances
Deployment of software using virtual appliances is widely considered a standard practice. It is not, however, a best practice, and many shortcomings have become apparent as virtualized data centers and public clouds have become widespread:

  • Image proliferation and management is a cumulatively increasing burden. New versions of the applications are released. Optional features and configurations create a combinatoric effect. Because boot images typically need to be co-located with the compute resources, multiple copies of the library need to be managed if there are multiple data centers.
  • Images also create a commitment to a particular operating system distribution, and the binaries are not usually portable between different IaaS environments. Sometimes the operating system needs to conform to corporate or cloud provider standards, so making these options available is once again an exercise in combinatorics and painful image library management.
  • To mitigate this burden, some IT departments and public cloud services maintain only a limited set of images, and system administrators in the department or at the customer perform manual post-boot upgrading or customization. This approach progressively reduces the benefit of using the virtual appliance in the first place, because technical resources are required and repeatability is degraded or eliminated.
  • A virtual appliance generally only offers time savings and repeatability for the initial deployment. Subsequent changes, such as application version upgrades, operating system upgrades, transition to a larger (or smaller) system or to a different data center, all have to be handled either manually or via a different approach entirely. This means there is one approach used for deployment and another for management.
  • Worse, the details of a virtual appliance are typically opaque: the configuration is implicit in the binary and is typically not documented. Because the staff handling later system and application management did not produce this original configuration, before changing anything they effectively must reverse-engineer it - a time-consuming and error-prone process. And with each generation of the image (where it is booted and modified, and a new image is created), these details become increasingly distant from memory.
  • Because they have to be created as a separate step before they are tested or otherwise made available, virtual appliances were better suited to an era of waterfall software development with distinct endpoints, in contrast to today's agile, deploy-frequently model of operations.

Model-Driven Deployment
Model-driven deployment is an approach that encapsulates the deployment and management of a system into a series of procedural operations. It provides a mechanism by which the problems listed above can be either eliminated or handled with a repeatable and efficient process. Model-driven deployment is to virtual appliances as dynamic websites are to printed newspapers. It is closely related to the emerging field of devops, in which the roles of software developers and system administrators, and the line between pre- and post-production, begins to blur. In its simplest form, a model like this performs the same series of steps that a system administrator would otherwise do manually.

Important, a model still must include a boot image -  the physical or virtual machine has to boot from something, after all. But the boot image for a model-driven deployment is normally a generic, stripped-down operating system distribution with no application software installed on it. This means that, at most, a handful of images is required so as to offer different operating system distributions.

The real power of models is in two techniques from the world of software development: abstraction and modularization, and they are often combined. Abstraction means that the script uses input parameters (either provided directly when it is called, or obtained by querying the system on which it resides) to vary the flow and the details of how the deployment is to proceed. Abstraction is the means by which a model-driven deployment avoids the combinatorial explosion described above for virtual appliances.

Modularization means that the model scripts are separated into reusable components that can be maintained separately and combined as needed. For example, the standard installation of a MySQL database might be a module that is used by many different application models. Modularization makes management of scripts much easier and more reliable, because the module can be tested once and used in many places; further, any fixes or changes need only be made in one place.

As mentioned above, the initial deployment is only the first operation of many over the application's life cycle. A complete model also includes scripts associated with application or system version upgrades, backups, and other management functions. These management scripts also need to use abstraction and modularization for the benefits of the model to accrue.

Benefits and Shortcomings of Models
Model-driven deployment retains almost all of the advantages of virtual appliances while eliminating or mitigating their shortcomings:

  • Combinatoric proliferation of images and the associated management burden is eliminated. While it is necessary to manage the model scripts, this effort is proportional to the sum of the number of components and variants rather than their multiplicative product.
  • Consequently, there need be no trade-off between portability or flexibility and maintenance.
  • The details of the deployment are documented explicitly in the model script. There is no need for reverse-engineering or straining to remember the procedure.
  • Equally important, adding new configurations (e.g., a different operating system distribution) is merely a matter of creating a variation of the abstraction already in the script. In a virtual appliance approach, the configuration would typically be performed from scratch.
  • Scripts are small files that can be updated and deployed quickly in an Agile context.
  • A consistent, unified approach handles both deployments and application life cycle management functions. The various scripts plus the vanilla boot images that comprise the model contain all of the information about the deployment - there is no prior knowledge or history of which one must be aware.
  • With the right tools, model-driven deployment can be incorporated into larger automated business processes by taking advantage of the fact that server provisioning can be automated.
  • Model testing can be more extensively automated using unit tests for modules and integration tests for entire systems. In very forward-looking environments, these automated tests can become an element of the model.

There are two key disadvantages of model-driven deployment in comparison with virtual appliances:

  • Many system administration professionals are accustomed to using virtual appliances as a primary means of deployment. "Moving the cheese" will undoubtedly cause some disruption, but the transition is inherent in the devops movement and it is largely inevitable.
  • Model-driven deployment adds latency. When a virtual appliance boots up, it is ready to go, whereas a model will have to download files and install and configure them. This latency is primarily an issue in an environment where rapid scaling is required (such as burst handling). In such environments, generally only one or two system roles scale in that fashion, and it may make sense to create images for those roles, as long as those images are created via a model.

The holy grail of model-driven deployment would be the inclusion of an image-burning capability within the cloud infrastructure orchestration systems. For those roles that must be rapidly deployable when scaling, the system could automatically burn the image whenever anything in the model changed, and automation tools could then make run-time decisions whether to use an image or a model to deploy the system.

Model-driven deployment is the way of the future and one of the hallmarks of a true devops shop. Organizations need to plan to transition from using virtual appliances as their primary means of deploying systems to an approach that uses flexible, maintainable scripts - not only for initial deployment but also for ongoing management of the system. System administrators and their management will need to do some organizational retooling to effect this transition. The end result will be more efficient and reliable systems management.

More Stories By Dave Jilk

Dave Jilk has an extensive business and technical background in both the software industry and the Internet. He currently serves as CEO of Standing Cloud, Inc., a Boulder-based provider of cloud-based application management solutions that he cofounded in 2009.

Dave is a serial software entrepreneur who also founded Wideforce Systems, a service similar to and pre-dating Amazon Mechanical Turk; and eCortex, a University of Colorado licensee that builds neural network brain models for defense and intelligence research programs. He was also CEO of Xaffire, Inc., a developer of web application management software; an Associate Partner at SOFTBANK Venture Capital (now Mobius); and CEO of GO Software, Inc.

Dave earned a Bachelor of Science degree in Computer Science from the Massachusetts Institute of Technology.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.