July 25th: IBM announces version 2.2.7 of IBM Spectrum Copy Data Management.

In the new version, the licensing model has changed from ‘USABLE’ to ‘MANAGED’ capacity, which enables more cost effective licensing of IBM Spectrum Copy Data Management. Instead of licensing the full capacity of the array, only the managed source capacity (volumes containing the source data) has to be licensed. Capacity for copies does not have to be licensed. This version is only available via Fix Central. The announcement letter can be found here:



Don’t Settle for Less

Don’t Settle for Less

The Need for Automation
Data is the core of any business, and securing access to that data is the primary role of IT. If there was a summer blockbuster made based on an IT administrator, the trailer might sound something like this:  “In a world where data is growing at an exponential rate, there is one IT administrator who will stand up against the odds and solve all problems… with scripts!”

Some IT administrators resemble ‘MacGyver’ in their writing of scripts to keep up with the ever-changing demands of the role. But unlike the movies, scripting alone often doesn’t solve the problem. They need the right tool set that is reliable, auditable and requires minimal or no effort to manage. Common tasks for an IT admin include:

  • Managing servers: installing, patching and updating software
  • Storage: provisioning new LUNs, ensuring connectivity, managing capacity
  • Backup: maintaining data protection SLAs
  • Test/Dev: ensuring that the right people get the right data promptly and securely, keeping it refreshed, etc.

In any organization, development teams and business critical application admins get the most attention, and budgets are mostly allocated to those teams. The IT teams are often almost an afterthought. As a result, IT professionals are forced to work with whatever tools are available. In addition, oftentimes there is a myriad of point products that introduces complexity and silos of management.  These and other problems make it almost essential for these admins to come up with creative ways to make their life easier with some basic form of automation. All too often, automation has become synonymous with scripts, but scripts often are not the best answer.

The Problem
Let’s take a look at three common use cases where admins tend to use scripts, and for which a simple and easy solution is available.

Backup and Recovery

This is the number one pain in many administrators’ jobs.  A typical day in the life of a backup admin may involve monitoring and redriving backup jobs, validating protection of key applications, configuring snapshots of application data, tracking down wasted storage, ensuring copies of data are available offsite for Disaster Recovery, and performing capacity management for backup.

Test and Dev

This is another set of challenges for admins.  Development and QA teams are always looking for infrastructure and data to build and test their products. To ensure bugs are caught at the right time, they want to test their code on real data and systems that match the characteristics of the production environment. Provisioning infrastructure and giving access to production data is not trivial. So these requests go in a ticketing system and stay in the queue for a while, and guess who takes the blame for the slow response?


Real-time analytics lead to better business decisions. This means copies of latest production data needs to made available to the analytics team instantly.  It’s a similar problem as the test/dev use case and it’s most likely in the same ticketing queue.

The Solution
At this point, you may be thinking, “I know all of this already. I have written scripts for these use cases. How can you help?” Perhaps these problems are best solved by a simple and easy-to-use tool called IBM Spectrum Copy Data Management. It’s packaged as a single .ova file that can be deployed in your environment in less than 15 minutes. It works with your existing infrastructure and leverages the native data services in-place and agentlessly.  IBM Spectrum CDM has the ability to take snapshots instantly, ensure application awareness and instantly provision those snapshots for recovery, Test/Dev and analytics workloads. To get more details on how it works check out https://www.ibm.com/us-en/marketplace/spectrum-copy-data-management.

Here’s why IBM Spectrum Copy Data Management is much better than scripts.

Application awareness

Storage vendors have advanced snapshotting capabilities and usually include it in the storage suite. These snapshots are typically crash-consistent. Spectrum CDM is application-aware and invokes the appropriate APIs to put the app in a consistent state. This enables granular database-level recovery. CDM supports hypervisors, like VMware, and applications, like Oracle, Microsoft SQL, SAP HANA, EPIC EHR and InterSystems Cache. CDM also has the ability to drive application-aware replication between storage arrays and can instantly mount the database from the snapshots.

A script-driven solution has to be customized for individual databases and usually doesn’t have the ability to map the data to appropriate storage volumes. This is hard-coded and cannot scale to hundreds of databases. VMware-based scripts don’t usually support Virtual Machine (VM) snapshots and are just crash-consistent. It’s very difficult to implement application awareness for apps hosted within a VM guest.


At the heart of Spectrum CDM is a powerful metadata catalog that keeps track of the lineage and location of all data copies. Within CDM, a user can choose a particular version of the snapshot and also pick a secondary storage controller where the data has been replicated. This catalog also allows users to search for individual objects like VMs or databases and report on protection compliance, RPOs and RTOs.

Scripts don’t have a catalog, and without a catalog, backup and restore functionality is restricted to simple backups and simple recoveries, like revert to original location.


CDM provides instant recovery for VMs and individual databases. CDM provisions a full copy of the database in minutes to any location, be it to recover from a data loss due to disaster, mount copies for test or development purposes, or to run near real-time analytics. CDM also supports database masking tools and can provision a masked version of the database to developers.

Here are some of the limitations we see with scripting for a virtualized environment

  • Can’t restore VMs that are already deleted. The VM must exist in order to be reverted to the old version.
  • Can’t restore VMs that were moved from one datastore to another. You lose context of all snapshots created before the VM was moved.
  • Can’t restore VMs that are renamed. You lose context of all snapshots created before the VM was renamed.
  • Can’t restore from replicated snapshots, only local primary snapshots.
  • Restores one VM/Datastore/RDM at a time and while it is running, it is blocked and waiting for the operation to complete.
  • Can only restore/clone one VM at a time – each operation creates its own snapshot clones. Several operations running in sequence to restore a subset of VMs could lead to several recovery clone datastores.
  • If you use FC LUNs, all restore operations that led to an HBA rescan can hang the UI for several minutes. Multi-pathing setups could be a major challenge.
  • No snapshot/restore for VMs across multiple datastores
  • Requires restore target to be a cluster. A vCenter with multiple standalone ESXs is no good.
  • Cannot clone to a new VM onto the same cluster, likely because it doesn’t handle disk signaturing and conflict resolution
  • Can’t restore to a specific folder
  • Can’t restore within a specific network – potential conflicts with production.


CDM offers granular Role-Based Access Control (RBAC), which ensures that the right people get access to the right data at the right time. It also provides audit logs to keep track of who performed each operation. With cyber threats growing in number, it’s important to keep tight control on operations and limit access to users, but still provide them with enough access to continue day-to-day operations.

Scripts offer no RBAC and usually require highest privileges. This is a big security risk, as anyone can use them to take control of all the data and leave no trace behind.


There are checks built in that maintain proper operation of policies. It’s assumed that there will be connection issues, occasional time-outs, and CDM factors all that in the product design. If these occur, CDM has retry logic that tracks the successful completion of a policy. There are also detailed log messages with more information about the operation.

Scripts are brittle. They don’t factor in a lot of error conditions. They work great if everything in the environment is in order. Any change in the servers, storage or the network could break down the operations and a good script should consider all the conditions and error codes from different sources. Scripts can have retry logic and log messages included, but they need to be customized for each operation, are tedious to build and maintain, and are not worth the effort.


CDM is a fraction of the cost of a storage array. Due to its ‘in-place’ nature, CDM is the most storage-efficient data management software in the market. It does not need a separate appliance and significantly beats the cost of keeping and managing an additional storage system.

‘Scripting is free’ is a myth. Scripts are not written in anyone’s free time. They take up several admin-hours, depending on the scope and complexity involved. It’s even worse if the scripts are handed out for free by vendors as a stopgap. Now the admin has inherited all of the limitations described above, which will surely require some debug and managing of scripts that they did not write and need to learn from scratch. Complex database environments may require hundreds of scripts for different tasks.


CDM has a dedicated support team that assists with any customer issues. CDM has some of the highest rated support professionals in the industry and offers 24x7x365 support to customers. CDM also provides constant updates and patches to keep up with the version changes from application and storage vendors.

Scripting is usually done by one or two individuals in an organization. They are responsible for all issues with those scripts, whether they like it or not. They also require constant updating. For example, if Microsoft SQL upgrades their version or issues a patch, the script needs to be changed, whereas CDM works with those vendors to stay a step ahead.  And it is important to remember that the scripting expert in an organization may decide to retire or change jobs, in which case you may have no one on hand who understands how the scripts work.


CDM follows an agile methodology and releases software at least once a quarter. Typical releases include support for new storage platforms and/or new applications. We have exciting new features to address cloud native applications in the upcoming releases.

Scripts are written for specific use cases and don’t really address future roadmap items.

Choosing when to use scripts
This not a blog against scripters or their ability to automate operations. This is simply an attempt to point out that before choosing to write scripts, make sure you have analyzed the scale of effort and factored in all the risks.

Scripting is helpful when a product like CDM is not available. The only other acceptable use of scripts is when you have automated everything, like the guy mentioned in this blogà http://www.escapistmagazine.com/forums/read/7.885383-Programmer-Writes-Scripts-To-Automate-His-Job-Email-Wife-And-Make-Lattes , and don’t have to even show up in the office. Until then, please give IBM Spectrum CDM a try.

May 9th: IBM announces version 2.2.6 of IBM Spectrum Copy Data Management.

IBM Spectrum Copy Data Management transforms IT Operations with Software-Defined Data Services. Additional application and database support for SAP HANA, EPIC and MS SQL Server are introduced. Improvements in the user experience are achieved by reducing clicks for common use cases and simplified Storage and Application workflows in conjunction with the ability to search and recover databases and storage objects.

Adding support for SAP HANA
IBM Spectrum Copy Data Management introduces support for SAP HANA on xLinux and pLinux to extend Copy Data Management to SAP HANA applications.

Adding support for Epic/Cache
IBM Spectrum Copy Data Management introduces support for EPIC/Cache databases to extend Copy Data Management to EPIC applications.

Extending Support for MS SQL Servers to Physical Servers
IBM Spectrum Copy Data Management introduces support for MS SQL Server on physical servers to extend Copy Data Management on MS SQL Server.

Long distance vMotion support
IBM Spectrum Copy Data Management adds support for long distance VMotion during Instant Recoveries to enable recoveries across vCenters and Datacenters. This enables the Cloud use case of seamless Disaster Recovery across vCenters and Datacenters.

User Interface improvements
The IBM Spectrum Copy Data Management interface consistency and theme are improved. Customers benefit from an improved overall user experience.
– Reduced number of clicks for common use cases
– Simplified Storage and Application workflows
– Ability to Search and Recover Databases and Storage objects

Summary: Supported Applications and Databases
2017-05-30 12_47_03-Clipboard

IBM announces IBM Spectrum Copy Data Management

IBM has announced a solution for Copy Data Management on October 25th 2016:
By modernizing many common IT processes, IBM Spectrum Copy Data Management simplifies copy data management, enabling automation and orchestration of data copies. The copies can then be leveraged for value-creating use cases, whether on-premises or in the cloud. IBM Spectrum Copy Data Management enables the following use cases:

Automated copy management
IT teams can leverage the core policy engine, catalog and reporting of IBM Spectrum Copy Data Management to dramatically improve IT operations that rely on copies of data, including disaster recovery, testing and development, business analytics, and local recovery. IBM Spectrum Copy Data Management improves operations by using automated, servicelevel based copy policies that are consistent, reliable and easily repeatable. This can provide huge savings in operational expenditures. Without IBM Spectrum Copy Data Management, functions that rely heavily on copies are typically managed using a complex mix of scripts, tools or other products not optimized for copy management. With IBM Spectrum Copy Data Management, you have a holistic, simplified approach that greatly reduces cycle time and frees staff to manage more productive projects.

Development and operations (DevOps) organizations are increasingly moving toward modernized DevOps processes for faster delivery of new applications to market. IBM Spectrum Copy Data Management allows the IT team to use their existing storage infrastructure to enable DevOps and to meet the needs of the development teams for rapid deployment of infrastructure. IBM Spectrum Copy Data Management templates define the policies for infrastructure deployment, and the whole system is accessible via the REST API, which is a fundamental requirement for DevOps. With IBM Spectrum Copy Data Management, DevOps teams get “infrastructure as code.” Rather than following legacy processes to requisition IT resources, developers include the infrastructure deployment commands directly within their development systems, such as Chef, Puppet or IBM Bluemix®. Pre-defined scripts and plug-ins for popular DevOps tools simplify implementation.

Next-generation data protection and disaster recovery
Through its template-based management and orchestration of application-aware copies, IBM Spectrum Copy Data Management becomes a powerful solution for next-generation data protection and recovery. Compared to the slow restores of traditional backup, IBM Spectrum Copy Data Management enables IT to mount and instantly access copies that are already in the production storage environment. IBM Spectrum Copy Data Management catalogs all snapshots, replicas, and the solution alerts you if a snap or replication job was missed or failed. Disaster recovery can be fully automated and tested non-disruptively. In addition, IBM Spectrum Copy Data Management can coordinate sending data to Amazon Web Services (AWS) S3. This provides an easy-to-use, low-cost option for longer term or archival storage of protection copies.

Automated test and development
The speed and effectiveness of test and development processes are most often limited by the time it takes to provision IT infrastructure. Typical organizations take weeks to deploy infrastructure; even the most efficient can take several days. With IBM Spectrum Copy Data Management, test and development infrastructure can be spun up in minutes, either on an automated, scheduled basis or with push-button ease on demand.

Leverage hybrid cloud compute
IBM Spectrum Copy Data Management is a powerful enabler of the hybrid cloud that empowers IT to take advantage of cloud compute resources. IBM Spectrum Copy Data Management not only helps you move data to the cloud, it enables you to bring up live application environments that can leverage the less expensive, elastic compute infrastructure in the cloud. You can spin up workloads and then spin them back down reliably. This maximizes the economic benefit of the cloud by using and paying for only the infrastructure you need. IBM Spectrum Copy Data Management is an easy-to-deploy software platform that is designed to leverage the existing infrastructure in the IT environment. It works directly with hypervisor and enterprise storage APIs to provide the overall orchestration layer that leverages the copy services of the underlying infrastructure resources. IBM Spectrum Copy Data Management also integrates with AWS S3 for cloud-based data retention, as well as Puppet, Bluemix and other platforms.