«WHITE PAPER Driving Smart Data Management in the 3rd Platform Era — An Integrated Approach to Managing, Accessing and Protecting Critical Data ...»
Driving Smart Data Management in the 3rd Platform Era —
An Integrated Approach to Managing, Accessing and
Protecting Critical Data
Sponsored by: CommVault
Organizations are facing a massive and disruptive shift to what IDC calls the "3rd Platform". This is
built on the four technology pillars of Big Data, cloud, mobile and social, something that presents
significant opportunities to transform how businesses are run, driving future growth and innovation.
The goal of this new platform is to accelerate new capabilities, lower costs, and reduce the need for major capital investments in IT infrastructure. However, this requires organizations to rethink how datacenters should be built and designed, the types of systems and applications that are needed, and how they should be operated. Changes of this magnitude are not common.
Recent years have seen a tremendous growth in the amount of data that is being generated from transactions and interactions. Servers, networks, machines, sensors, cameras, and countless other devices are continuously capturing and generating data — something that is driving increased spending on storage. In fact, IDC forecasts that Asia/Pacific excluding Japan (APEJ) spending on storage for Big Data will continue to grow at a compound annual growth rate (CAGR) of 42% between 2012 and 2017.
Much of this growth is driven by risk mitigation initiatives to comply with regulations, and the need to service unforeseeable requirements and future analysis. However, the cost of collecting and analyzing all of the data, in its entirety, is becoming economically unfeasible for many organizations.
At the same time, data today resides in multiple locations — not only on-premises but also in various silos and third-party datacenters, in highly virtualized environments. All this adds yet more complexity, especially when determining the most efficient and reliable way to ingest, protect, organize, access,
preserve, and also delete all this data when needed. In light of this organizations need to:
Develop an effective data management strategy. Organizations should evaluate current and future business plans and define a framework including milestones, deadlines, and policies around data management and data quality models. Understanding data size, predicted growth, variety, and location is critical.
Optimize the storage layer. Data proliferation requires organizations to revisit their storage architectures. Organizations looking at reducing their storage footprint should increase focus on the use of archiving, data deduplication, thin provisioning and storage tiering.
March 2014, IDC #AP14988X Ensure data protection. Organizations should consider using a flexible and scalable data management solution that automates data protection, recovery, and archive processes while enabling policy-based management across internal and external IT assets.
METHODOLOGYThis IDC White Paper is based on primary and secondary research across the buy and sell side for the business analytics market in the APEJ region. IDC conducts regular CIO and executive decision making focused surveys across APEJ, and also obtains insights from analysts' interactions with CIOs of Asian organizations. A combination of the following surveys and studies were used to draw some of the insights in this White Paper.
IDC AP Smart Data Management Survey (IDC SDM Survey) sponsored by CommVault. In September/October 2013, IDC conducted this survey across vertical industries in Asia/Pacific in order to understand the top data management drivers and challenges in the context of major trends like Big Data and cloud. The results are based on insights gathered from over 500 IT decision makers across Australia, New Zealand, China, Korea, Singapore, Malaysia and Thailand.
IDC APEJ C-Suite Barometer. The C-Suite Barometer provides intelligence and insights into how 1,000 organizations in APEJ rated their business and IT priorities, pressures and preferences in relation to achieving business goals.
IDC AP Business Analytics Pulse Survey. This survey provides intelligence and insights into how over 776 CIOs and IT decision makers across APEJ rate and rank their Business Analytics priorities for both the short and the long term. This survey includes country, vertical industry and organization size specific insights.
IN THIS WHITE PAPER
This IDC White Paper explores the customer challenges associated with data management in the context of the 3rd Platform, particularly around Big Data and cloud. The paper also highlights the top priorities, business and IT challenges, and trends impacting data management for AP organizations. In addition, the paper takes a look at how the adoption of CommVault Simpana can help organizations in AP address these data management challenges, and provides guidance for end-user organizations.
©2014 IDC #AP14988X 2
SITUATION OVERVIEWThe State of the Market in Asia/Pacific While the shift toward a less predictable market means that uncertainty has become the norm, the Asia/Pacific region remains as a reliable engine for growth, and multinational corporations and Asian companies alike will continue to relentlessly look to Asia for future opportunities. This in turn is raising the pressure on business executives in the region to drive the expected results for the year ahead, while concurrently being challenged by the increasing levels of risk, limited budgets and unpredictable customers. As a result, this has turned the region into a hyper-competitive place, and organizations that fail to adapt to the required business flexibility while achieving the necessary speed to market will find it hard to compete.
The top business priorities for 2014, based on IDC's APEJ C-Suite Barometer Study, are finding new customers or customer segments, handling escalating cost of operations (especially labor costs) and finding new innovations to compete in the market. As a result, a renewed focus on customer centricity will be top of the business agenda, together with a focus on technologies that can drive innovation and help companies improve customer engagement, and the knowledge of which customers contribute the most to the business. This will be centered on gaining a holistic customer view to obtain real insight into the value and life cycle of each customer. In order to do this, organizations will not only look at transactional data, but will increasingly capture and analyze wider sources of data within and outside their organizations.
However, in order to drive these initiatives organizations will need to increase the collaboration between the business and IT. One of the key inhibitors for this is the fact that IT is overburdened with the growing complexity of business requirements, as well as with regular end-user and technical support issues — something that is hampering their ability to be an innovation engine (as required by the business).
Also, regional organizations' lack of IT technology skills — especially the advanced skills needed to support 3rd Platform projects — is another facet of the problems facing CIOs and project managers. For example, Big Data and analytics projects require the right skill sets, with data integration as a foundation and then forecasting, modeling, and simulation, in order to explore patterns that can guide future action. From a cloud perspective, while on-premises infrastructure is certain to remain in all APEJ markets, the technology skills of the average IT professional need a revamp to include, by default, the ability to plan and manage complex virtualized server and storage environments.
Finally, based on the same IDC's APEJ C-Suite Barometer Study, IT organizations are claiming that their limited budget is not adequate in supporting business growth/needs. Therefore, IT will increase focus on return on investment (ROI) led initiatives, as well as look at any opportunity that can help drive cost savings. For example, a growing number of organizations feel that the cost of collecting and analyzing the increasingly large data volumes is becoming economically unfeasible. In fact, based on the IDC SDM survey, 87% of companies in Asia/Pacific stated that they are willing to reduce their storage costs.
©2014 IDC #AP14988X 3 This willingness, which is a result of the aforementioned focus on cost saving, is consistent across countries in the region (see Figure 1). Nevertheless, IDC expects risk mitigation initiatives to comply with regulations and the need to service unforeseeable requirements and future analysis will continue to drive increase in storage spending. In fact, IDC forecasts that APEJ spending on storage for Big Data will grow at a CAGR of 42% between 2012 and 2017.
FIGURE 1 Willingness to Reduce Storage Costs across AP AP - 75% of Organizations say “yes”
Source: IDC Smart Data Management Survey, 2013 Note: All data provided in the following sections of this document is derived from the IDC SDM Survey, unless stated otherwise.
Big Data The intelligent economy produces a constant stream of data that is being monitored and analyzed.
Social interactions, mobile devices, facilities, equipment, research and development (R&D), simulations, and physical infrastructure all contribute to the flow. In aggregate, this is what is called Big Data. IDC's definition of Big Data technologies describes a new generation of technologies and architectures designed to economically extract value from very large volumes of a wide variety of data by enabling high-velocity capture, discovery, and/or analysis.
Irrespective of size and industry, organizations in Asia/Pacific are seeing their data growing faster today than it used to a few years ago. 39% of organizations in AP claim that their data is growing by 20–50% year-over-year (YoY), while 13% say that this growth is over 50%. Of all industries, financial services, retail and wholesale, and media and telecommunications have the fastest data growth. The
split by subregion is as follows:
ANZ (Australia and New Zealand) — 48% of organizations in ANZ stated that their data is growing by 20–50% YoY. 17% said that this growth is over 50%.
ASEAN (Association of Southeast Asian Nations) — 39% of organizations in ASEAN claimed that their data is growing by 20-50% YoY. 17% said that this growth is over 50%.
NA (North Asia) — 32% of organizations stated that their data is growing by 20-50% YoY. 7% said that this growth is over 50%.
Admittedly, the challenge of handling ever-larger data volumes is not new for most storage administrators. However, Big Data can drive some companies to the limits of their current architectures faster. The existing data volumes and their future growth are relative to a company's ability to manage its data assets. The lack of skill sets, optimal tools, and gaps in its processes can further increase this challenge. Furthermore, size is not the only data dimension that matters.
Even the smallest organizations are now facing the challenge of managing new data types that are being produced and captured. Firstly, because most of this data is captured outside the organization, and secondly, because most of the growth comes from semi-structured and unstructured data types such as machine to machine (M2M), GPS location data, video or social media feeds. This is adding more complexity from an information governance perspective. At the same time, a significant proportion of these data types being captured are not yet analyzed (see Figure 2).
N=505 Source: IDC Smart Data Management Survey, 2013 As depicted in Figure 2, 79% of organizations in AP capture user generated text today, but only 62% of those analyze that data. But more significantly is video, which is captured by 46% of organizations and only analyzed by 29%, or audio (35% captured vs. 21% analyzed). Looking at this from a subregion
perspective, the difference between data captured and analyzed becomes more apparent: