Realize the value of Hadoop

---

Viewpoint

Realize the value of Hadoop Evaluation, adoption, and value of data and analytics

Viewpoint

Table of contents

1 Hadoop realities 3 A logical adoption cycle 3 Methods and tools 5 Deriving Business Value 5 Partner considerations 6 Seek real value 6 Author

Viewpoint

Page 1

As organizations strive to identify and realize the value in Big Data, many now seek more agile and capable analytic systems. Apache Hadoop is a maturing technology framework that promises measurable savings and value, and is enjoying significant uptake in the enterprise environment. But very real challenges remain for this emerging data and analytic model. Companies and public institutions may struggle to acquire the skills, tools, and capabilities needed to successfully implement Hadoop projects. Others are working to clarify a logical pathto-value for data- and analytic-oriented deployments. In this document, Hewlett Packard Enterprise (HPE) describes a proven strategy to extract value from Hadoop investments. This paper examines the opportunities and obstacles many organizations face and explores the processes, tools, and best practice methods needed to reach those objectives.

Hadoop realities As organizations of all kinds embrace the use of Big Data and analytics, most are now moving beyond traditional business intelligence (BI) to a more advanced and comprehensive analytic environment. Forward-looking executives now view data and analytics as vital tools needed to improve customer relationships, to accelerate speed-to-market, to survive in an increasingly dynamic marketplace, and to drive sustainable value. The value of data The very nature of the data revolution—driven by the four V’s: volume, variety, velocity, and vulnerability—now poses unique challenges to many organizations. Informed observers expect the global volume of data to reach 35 zettabytes by 2020. Unstructured data now comprises 85 percent of information, with growing volumes of data flowing from increasingly ubiquitous sensors, mobile devices, video streams, and social networks. Given the speed and reach of this data revolution, it is perhaps not surprising that many organizations are less than prepared to meet these challenges. A survey of business and IT executives show that 56 percent are unsure of how to derive value from Big Data, 41 percent say Big Data is a key strategic challenge, and 34 percent report at least one failed Big Data initiative.1 In early 2015, Forrester Consulting surveyed 166 global IT and business managers about how their organizations were using, or expected to use, the emerging generation of agile analytics. Commissioned by HPE, the study found that CIOs can better leverage advanced analytics by making those systems more intuitive, easier to use, and more widely available across the organization.2

1

 artner Big Data Buying Behavior G Workshop Report for HPE, 2013

2

“ Business Success Through Pervasive Analytics, How Top Business Performers Are Using BI Technologies And Analytics To Become DataDriven And Agile”, A Forrester Consulting Thought Leadership Paper Commissioned By HPE, May 2015.

Viewpoint

Page 2

Business outcomes

Data-driven products and services

Value themes

Customer experience

Modernize/leverage

Process & tools

Corporate and financial performance

Managing company risks

Operationalize analytics

Discovery

Development integration

Implementation

Tools

Best practices

Processes

Data governance

Platform

Data

Operations & IT optimization

Hadoop CRM, SCM, ERP

Documents

Social media

Texts

Video

Open data

GPS

Email

Audio

Transactional data

Images

Mobile

Sensor data

Weather data

Machine data

Figure 1. A framework for advanced data and analytics

Hadoop emerges As businesses, public agencies, and other organizations struggle with these challenges, many now view the emerging Hadoop ecosystem as a logical way to process and analyze Big Data. Apache Hadoop is an open-source software framework for distributed storage and processing of extensive data sets, using simple program modules across clusters of commodity-level computing hardware. Reliable and scalable, Hadoop is designed to run on anything from a single server to thousands of machines. The primary Hadoop project incorporates common utilities, a distributed file system, frameworks for resource management and job scheduling, and technology for parallel processing of large data volumes. Most enterprises have deployed, or are considering the deployment of, Hadoop environments. Business and IT leaders expect Hadoop to help them extract value from their data, and to reduce their total cost of ownership (TCO) for BI and analytics. The Hadoop stack continues to rapidly evolve, and now incorporates solution features that allow organizations to build and deploy IT and business solutions. Challenges remain While Hadoop has quickly gained traction as a viable open source technology in the Big Data and analytics marketplace, as seen with the broader data revolution, a number of significant challenges have emerged. A Hadoop implementation presents very complex planning, deployment, and long-term management challenges. There is currently a general lack of Hadoop skills in the marketplace. Although the Hadoop technology stack continues to evolve, it is still maturing, and thus poses a higher degree of difficulty and uncertainty. The business-oriented drivers for many data and analytic projects are often unclear, or less than precise. Discovery projects can be somewhat lacking in focus. Not surprisingly, many organizations are struggling to identify a clear path-to-value for their current Hadoop and other data, analytics, and BI investments.

Viewpoint

Page 3

A logical adoption cycle

For many, identifying end-state value is the core challenge for data- and analytics-oriented initiatives. Hewlett Packard Enterprise believes the correct response is a robust and field-proven approach to the evaluation and adoption of Hadoop and similar efforts.

A carefully planned, phased, and cost/benefit-balanced adoption environment is crucial to the successful implementation of Hadoop, or any other complex technology system. HPE recognizes a proven, three-step approach to the implementation of sophisticated data or analytic systems—Discovery, Development and Integration, and Implementation. • Discovery—In this initial exploratory phase, organizations consider potential project candidates, build and evaluate the business case for potential Hadoop initiatives, and reject, when appropriate, poor project candidates. A majority of Hadoop projects fall into the discovery category. • Development and Integration—Once a project has proven its value, it must be built and integrated with existing applications, and with the larger BI and analytics landscape. • Implementation—In this final critical phase, organizations may roll out multiple, industrialstrength Hadoop applications. Depending on the nature of the organization, those deployments may occur across a complex IT landscape, in multiple clusters, and on a global scale.

Methods and tools The good news is that a growing range of techniques and technologies are now available to support the Hadoop framework environment. The following outlines some of the major tools and resources organizations can use to ensure more successful data and analytic project outcomes. Expert guidance Managing information as a strategic asset requires unique organizational and governance structures. Detailed planning and the use of consistent methods are needed to reduce the cost and risk of data and analytic projects. Not surprisingly, forward-looking organizations increasingly seek expert advice and counsel in the use of Hadoop and other advanced data and analytic technologies. An experienced partner can offer end-to-end guidance for BI modernization—from initial assessments, to the scope, planning and strategy, to proof-of-value, and actionable roadmaps. Competent advisory services may include preliminary designs, realistic cost estimates, and prioritized phased planning for building a sustainable, Hadoop-enabled BI architecture. Organizations can use this input to make better decisions about technical architectures, required skills and competencies, governance models, and delivery platforms. Discovering value-driven Hadoop uses Exploring the business value and possible avenues for data and analytic advances can be difficult. Few organizations possess a structured methodology, specialized data visualization, and sharing tools as part of an integrated platform, skilled resources, collaboration methods, and best practices to support experimentation that is both ambitious and cost-effective. To solve this challenge, HPE recommends a formal and structured approach to data and analytic discovery. This collaborative model provides a number of experimental options, and allows organizations to explore, test, and learn about Hadoop, as well as other data and analytic opportunities, in a safe, cost-efficient environment. This phased, expert-based discovery model allows organizations to deploy analytic solutions more quickly, and with lower and more predictable investments. It reduces disruptions to existing processes and data, while increasing productivity and long-term revenue. A robust discovery environment opens a clear path to business value—identifying weak projects and quickly proving the value of strong data and analytic opportunities.

Viewpoint

Page 4

Leveraging the right platform Given the complex, rapidly changing nature of data and analytic technologies, organizations are naturally reluctant to invest heavily in capital-intensive systems that may too quickly become obsolete. Fortunately, a new generation of consumption-based services now allows enterprises of all kinds to quickly pursue data- and analytic-driven opportunities, without the high cost and obsolescence risks of traditional infrastructure. As-a-service solutions are now available for Hadoop enterprise Big Data analytics, for realtime BI, and for cloud-based data and analytic capabilities. Those cost-effective models allow organizations to more swiftly derive valuable insights—again, without investing in hardware, software licenses, procurement and installation, as well as ongoing management of what is often a one-off test and deployment environment. This model lets companies consume and pay only for what they use, and eliminates much of the concern over technology refreshes. It invites teams to more quickly discover insights, and to develop and deploy winning analytic applications in a production-ready environment. Data management considerations One of the most daunting issues facing any business intelligence strategy is how to fully harness all available data, whether well-known structured systems, the burgeoning universe of unstructured information, and the data from virtually any internal and external source. The answer is what some might call a truly modern analytics platform—one that incorporates hybrid data management and workload optimization capabilities, seamlessly bridging traditional and more advanced data and business intelligence technologies. HPE envisions a hybrid approach that optimizes existing BI technologies, and that integrates analytics into business systems and processes across data centers, secure value chains, as well as public and private cloud environments. Deployment and integration Implementation of the data lake is a vital step in an analytic initiative. This is where the different types of data are stored to enable the analytics. Strong integration to and from the data lake requires the development and execution of the technical, process, and organizational architectures needed to capture, manage, retain, and deliver information from the data lake across the enterprise. Robust content management allows organizations to give employees, customers, and value chain partners secure-butconvenient access to information and processes. Data governance HPE also believes that to fully leverage data and analytics as strategic assets, organizations must have a formal and effective structure for information governance of the data lake. As in other areas of a business, strong governance can reduce costs and risk, while ensuring greater efficiency from the governed activities. Any good governance approach should serve to classify, archive, and manage physical and electronic data in a reliable and cost-effective way. That requires careful planning, and should encompass data ownership, storage and flows, visibility, confidentiality, and security. A robust governance structure addresses policies, processes, and systems—and should meet dataoriented regulatory requirements and business objectives.

Viewpoint

Page 5

Deployment alternatives In their efforts to fully exploit data and analytics, businesses and public agencies are often limited by talent deficiencies, the constraints of available resources, and financial pressures to limit capital spending. For these and other related reasons, growing numbers of organizations are now accessing consumption-based, managed analytic and BI services. While new generation analytics are gaining ground, traditional BI will be viable for the foreseeable future. By working with a competent provider for managed services, companies can reduce data warehouse and BI costs, while improving service levels. Managed analytic services can eliminate the need to build internal statistical bureaus, as well as the need to hire and pay on-staff data scientists. This approach enables companies to acquire expertise and capabilities, without adding soonto-be-obsolete infrastructure. It supports service levels that adjust quickly and easily, allowing organizations to better meet changing business and technology demands. It can also be an add-on option to more traditional on-premise-based approaches to handle peaks in demand, leading to hybrid models.

Deriving business value Based on decades of experience in enterprise-class data and analytic environments, HPE sees the Hadoop stack driving business value along two broad themes: • Modernizing existing BI environments. Hadoop can be implemented to allow existing BI systems to better handle the volume, variety, velocity, and vulnerabilities of business intelligence systems. Certainly, traditional BI tools are reaching the upper limits of their capabilities. By introducing Hadoop, organizations can lower their analytic TCO and run costs, while at the same time deploying modernized versions of Oracle, SAP, and other BI stacks. • Operationalizing analytics. Hadoop supports the execution of use cases for a vast array of data emanating from numerous internal and external sources. Consumer-oriented firms could, in just one example, better target buyers based on their historical preferences and social media activities. By operationalizing those applications into existing business processes, organizations can drive business value and competitive advantages.

Partner considerations Given the complexity of the still-emerging Hadoop environment, it comes as no surprise that many organizations now recognize the value of working with partners that are experienced in enterprise-grade data and analytic systems. CIOs and others may consider several qualities when assessing potential Hadoop allies. A helpful partner should, at the very least, offer demonstrated experience and expertise in the current Apache Hadoop framework, tools, experience with multiple deployment models, and best practice methodologies. Look for teams with extensive integration skills, and teams who understand not only how to deploy around existing BI infrastructure, but to extend its useful life. For organizations across the industrial and public spectrum, HPE can be that partner. The company has emerged as a leading source of guidance, expert skills, analytic platforms, systems, and services to support Hadoop implementations. HPE Analytics and Data Management services bring the right people, processes, and technologies to enterprise-class challenges. HPE specializes in evaluating and aligning analytic investments with business objectives, allowing organizations to explore Big Data opportunities, and to glean insights and drive value from information. HPE has invested globally to enable the delivery and management of the information ecosystem—and stands ready to discuss Hadoop and other analytic needs.

Viewpoint

Seek real value The crucial importance of data is undeniable in today’s enterprise environment, and forwardlooking organizations are correctly seeking more practical and agile analytic systems. A Hadoop framework presents both challenges and opportunities—it is important to chart the right path for discovery, through development and implementation, and in long-term governance and management of the data lakes. HPE has developed this point of view describing how organizations can map a clear journey to more productive, cost-effective outcomes. By adopting these proven methods, leaders can better identify, leverage, and monetize the inherent value in enterprise information.

Author Ashim Bose PhD Sr. Director, Technology & Platforms, Hewlett Packard Enterprise - Analytics and Data Management; Consumption Based Platform Services Practice Leader Ashim Bose leads the Analytics and Data Management Technology Platform Services. With over 20 years of industry experience in automotive, industrial, airlines, and space exploration, Bose also has a PhD in computer science and a master’s degree in mechanical engineering, both with a specialization in artificial intelligence.

Sign up for updates Rate this document © Copyright 2015 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein. 4AA6-2993ENW, November 2015