Managing the Healthcare.gov Cloud Migration

The Challenge: Moving Complex Federal Systems to the Cloud

Migrating the back-end system components to the cloud was a massive endeavor full of technical, communication, and business operations challenges. Operating around the clock 365 days a year, Healthcare.gov manages more than two petabytes of data used to serve health insurance customers on the front end, along with back-end systems that provide data management and services to the consumer-facing applications.

On average, about 10,000 users access the site daily. But during the system’s peak operating period of open enrollment in November and December, site traffic increases by as much as 10 to 15 times. To provide for this business capability, CMS leveraged three data centers and their associated physical hardware assets. Exchange systems were scaled to support peak levels of service year-round yet were only needed at peak scale during the limited business cycle windows (e.g., consumer open enrollment, issuer plan submission). It was clear that the costs associated with the existing solution were not sustainable for the long term.

The Approach: Integrating Across an Ecosystem of Stakeholders

Working across 13 different workstreams, Booz Allen was deeply involved in the planning, implementation, and execution of this highly visible domestic program. Over a 5-year period starting in July 2015, our integrated team provided a wide array of capabilities and expertise ranging from strategic services, project management, communication outreach, and training to engineering services, technical reviews, and 24/7/365 monitoring.

As part of this support, Booz Allen is the Marketplace System Integrator for Healthcare.gov—the technical and operational arm for CMS. Our team operated across a complex multi-vendor, multi-functional ecosystem of more than 2,000 people, working with developers, IT program managers, data source partners, vendor teams, federal agencies, and other important stakeholders. For more than 25 years, we have been an essential partner for CMS, bringing to this high-profile project a strong mix of health policy and mission domain experts, combined with cloud engineers, architects, and security experts.

In developing a methodology for migrating all back-end Healthcare.gov systems to the cloud, our team used the firm’s cloud migration blueprint as a starting point to tailor to the needs of the CMS business. Recognizing the short time frame, CMS opted to leverage a course of action to lift, improve, shift, and optimize:

  • Lift systems out of the data centers primarily “as-is”
  • Improve only in areas that produce a tangible benefit, without affecting the core architecture or posing significant risks to cost or schedules
  • Shift the full scope of the application and its data to the cloud, without negative impact to normal operational tempos, consumer experience, performance, or security
  • Optimize post-migration to fully realize the potential of the cloud

This framework provided a common language for teams to discuss the technical, process, and service model changes that would be made during the migration. In addition, this helped to scope technical changes to validate activities that were best left until after the migration was complete.

The Solution: Optimizing Open Enrollment for Healthcare Consumers

To successfully complete this endeavor, both education and communication were critical. Hundreds of stakeholders had widely varying understanding of what CCIIO provided, let alone how cloud computing would impact it. In consultation with CCIIO leadership, Booz Allen’s experts quickly realized they needed to enhance collaboration and mature the collective understanding of cloud and cloud architecture and build buy-in for the initiative.

The Booz Allen team created a multifaceted plan comprised of a technical transformation to embrace DevOps and automation in addition to a strategic communications plan with consistent messaging tailored for different audiences. To commit to consistency across system operations, we developed a concept of operations for shared services to be leveraged by all systems in addition to workstreams to increase cloud adoption through data migration efficiencies and architectural workshops with teams. A key message that CCIIO leadership established early on: “The cloud migration will do no harm to open enrollment.” Because open enrollment is consumer-facing, any possibility of disruption was a major stress point for CCIIO’s business owners and teams. In building the migration plan, Booz Allen worked with CCIIO to ensure business would be affected as little as possible and that consumers would have a reliable user experience.

On the first business day of operations in the cloud, processes were already running faster. For example, 8-hour database backups decreased to 3 hours with AWS’s data processing and storage at scale. During the ramp up to open enrollment, testing environment scale up and scale out activities took hours instead of weeks. With back-end systems smoothly operating in their new secure cloud home, teams have transitioned to the optimization phase to fully implement automation and autoscaling. In addition, with the major transformational change that happened behind the scenes, it was a seamless front-end transition for the thousands of consumers who access Healthcare.gov every day.