Don’t Settle for Less

Don’t Settle for Less

The Need for Automation
Data is the core of any business, and securing access to that data is the primary role of IT. If there was a summer blockbuster made based on an IT administrator, the trailer might sound something like this:  “In a world where data is growing at an exponential rate, there is one IT administrator who will stand up against the odds and solve all problems… with scripts!”

Some IT administrators resemble ‘MacGyver’ in their writing of scripts to keep up with the ever-changing demands of the role. But unlike the movies, scripting alone often doesn’t solve the problem. They need the right tool set that is reliable, auditable and requires minimal or no effort to manage. Common tasks for an IT admin include:

• Managing servers: installing, patching and updating software
• Storage: provisioning new LUNs, ensuring connectivity, managing capacity
• Backup: maintaining data protection SLAs
• Test/Dev: ensuring that the right people get the right data promptly and securely, keeping it refreshed, etc.

In any organization, development teams and business critical application admins get the most attention, and budgets are mostly allocated to those teams. The IT teams are often almost an afterthought. As a result, IT professionals are forced to work with whatever tools are available. In addition, oftentimes there is a myriad of point products that introduces complexity and silos of management.  These and other problems make it almost essential for these admins to come up with creative ways to make their life easier with some basic form of automation. All too often, automation has become synonymous with scripts, but scripts often are not the best answer.

The Problem
Let’s take a look at three common use cases where admins tend to use scripts, and for which a simple and easy solution is available.

Backup and Recovery

This is the number one pain in many administrators’ jobs.  A typical day in the life of a backup admin may involve monitoring and redriving backup jobs, validating protection of key applications, configuring snapshots of application data, tracking down wasted storage, ensuring copies of data are available offsite for Disaster Recovery, and performing capacity management for backup.

Test and Dev

This is another set of challenges for admins.  Development and QA teams are always looking for infrastructure and data to build and test their products. To ensure bugs are caught at the right time, they want to test their code on real data and systems that match the characteristics of the production environment. Provisioning infrastructure and giving access to production data is not trivial. So these requests go in a ticketing system and stay in the queue for a while, and guess who takes the blame for the slow response?

Analytics

Real-time analytics lead to better business decisions. This means copies of latest production data needs to made available to the analytics team instantly.  It’s a similar problem as the test/dev use case and it’s most likely in the same ticketing queue.

The Solution
At this point, you may be thinking, “I know all of this already. I have written scripts for these use cases. How can you help?” Perhaps these problems are best solved by a simple and easy-to-use tool called IBM Spectrum Copy Data Management. It’s packaged as a single .ova file that can be deployed in your environment in less than 15 minutes. It works with your existing infrastructure and leverages the native data services in-place and agentlessly.  IBM Spectrum CDM has the ability to take snapshots instantly, ensure application awareness and instantly provision those snapshots for recovery, Test/Dev and analytics workloads. To get more details on how it works check out https://www.ibm.com/us-en/marketplace/spectrum-copy-data-management.

Here’s why IBM Spectrum Copy Data Management is much better than scripts.

Application awareness

CDM
Storage vendors have advanced snapshotting capabilities and usually include it in the storage suite. These snapshots are typically crash-consistent. Spectrum CDM is application-aware and invokes the appropriate APIs to put the app in a consistent state. This enables granular database-level recovery. CDM supports hypervisors, like VMware, and applications, like Oracle, Microsoft SQL, SAP HANA, EPIC EHR and InterSystems Cache. CDM also has the ability to drive application-aware replication between storage arrays and can instantly mount the database from the snapshots.

Scripts
A script-driven solution has to be customized for individual databases and usually doesn’t have the ability to map the data to appropriate storage volumes. This is hard-coded and cannot scale to hundreds of databases. VMware-based scripts don’t usually support Virtual Machine (VM) snapshots and are just crash-consistent. It’s very difficult to implement application awareness for apps hosted within a VM guest.

Catalog

CDM
At the heart of Spectrum CDM is a powerful metadata catalog that keeps track of the lineage and location of all data copies. Within CDM, a user can choose a particular version of the snapshot and also pick a secondary storage controller where the data has been replicated. This catalog also allows users to search for individual objects like VMs or databases and report on protection compliance, RPOs and RTOs.

Scripts
Scripts don’t have a catalog, and without a catalog, backup and restore functionality is restricted to simple backups and simple recoveries, like revert to original location.

Recovery

CDM
CDM provides instant recovery for VMs and individual databases. CDM provisions a full copy of the database in minutes to any location, be it to recover from a data loss due to disaster, mount copies for test or development purposes, or to run near real-time analytics. CDM also supports database masking tools and can provision a masked version of the database to developers.

Scripts
Here are some of the limitations we see with scripting for a virtualized environment

• Can’t restore VMs that are already deleted. The VM must exist in order to be reverted to the old version.
• Can’t restore VMs that were moved from one datastore to another. You lose context of all snapshots created before the VM was moved.
• Can’t restore VMs that are renamed. You lose context of all snapshots created before the VM was renamed.
• Can’t restore from replicated snapshots, only local primary snapshots.
• Restores one VM/Datastore/RDM at a time and while it is running, it is blocked and waiting for the operation to complete.
• Can only restore/clone one VM at a time – each operation creates its own snapshot clones. Several operations running in sequence to restore a subset of VMs could lead to several recovery clone datastores.
• If you use FC LUNs, all restore operations that led to an HBA rescan can hang the UI for several minutes. Multi-pathing setups could be a major challenge.
• No snapshot/restore for VMs across multiple datastores
• Requires restore target to be a cluster. A vCenter with multiple standalone ESXs is no good.
• Cannot clone to a new VM onto the same cluster, likely because it doesn’t handle disk signaturing and conflict resolution
• Can’t restore to a specific folder
• Can’t restore within a specific network – potential conflicts with production.

RBAC

CDM
CDM offers granular Role-Based Access Control (RBAC), which ensures that the right people get access to the right data at the right time. It also provides audit logs to keep track of who performed each operation. With cyber threats growing in number, it’s important to keep tight control on operations and limit access to users, but still provide them with enough access to continue day-to-day operations.

Scripts
Scripts offer no RBAC and usually require highest privileges. This is a big security risk, as anyone can use them to take control of all the data and leave no trace behind.

Reliability

CDM
There are checks built in that maintain proper operation of policies. It’s assumed that there will be connection issues, occasional time-outs, and CDM factors all that in the product design. If these occur, CDM has retry logic that tracks the successful completion of a policy. There are also detailed log messages with more information about the operation.

Scripts
Scripts are brittle. They don’t factor in a lot of error conditions. They work great if everything in the environment is in order. Any change in the servers, storage or the network could break down the operations and a good script should consider all the conditions and error codes from different sources. Scripts can have retry logic and log messages included, but they need to be customized for each operation, are tedious to build and maintain, and are not worth the effort.

Cost

CDM
CDM is a fraction of the cost of a storage array. Due to its ‘in-place’ nature, CDM is the most storage-efficient data management software in the market. It does not need a separate appliance and significantly beats the cost of keeping and managing an additional storage system.

Script
‘Scripting is free’ is a myth. Scripts are not written in anyone’s free time. They take up several admin-hours, depending on the scope and complexity involved. It’s even worse if the scripts are handed out for free by vendors as a stopgap. Now the admin has inherited all of the limitations described above, which will surely require some debug and managing of scripts that they did not write and need to learn from scratch. Complex database environments may require hundreds of scripts for different tasks.

Support

CDM
CDM has a dedicated support team that assists with any customer issues. CDM has some of the highest rated support professionals in the industry and offers 24x7x365 support to customers. CDM also provides constant updates and patches to keep up with the version changes from application and storage vendors.

Scripts
Scripting is usually done by one or two individuals in an organization. They are responsible for all issues with those scripts, whether they like it or not. They also require constant updating. For example, if Microsoft SQL upgrades their version or issues a patch, the script needs to be changed, whereas CDM works with those vendors to stay a step ahead.  And it is important to remember that the scripting expert in an organization may decide to retire or change jobs, in which case you may have no one on hand who understands how the scripts work.

Future-Proof

CDM
CDM follows an agile methodology and releases software at least once a quarter. Typical releases include support for new storage platforms and/or new applications. We have exciting new features to address cloud native applications in the upcoming releases.

Scripts
Scripts are written for specific use cases and don’t really address future roadmap items.

Choosing when to use scripts
This not a blog against scripters or their ability to automate operations. This is simply an attempt to point out that before choosing to write scripts, make sure you have analyzed the scale of effort and factored in all the risks.

Scripting is helpful when a product like CDM is not available. The only other acceptable use of scripts is when you have automated everything, like the guy mentioned in this blogà http://www.escapistmagazine.com/forums/read/7.885383-Programmer-Writes-Scripts-To-Automate-His-Job-Email-Wife-And-Make-Lattes , and don’t have to even show up in the office. Until then, please give IBM Spectrum CDM a try.