IBM Open Data Analytics for zOS – Highly Optimized Analytics Environment
IBM Open Data Analytics for zOS (IzODA) provides a comprehensive ability to leverage two of the most popular open source toolsets currently in use, by data scientists and application developers, for analytics. IzODA brings key proprietary and open source technologies to analyze, evaluate, visualize and leverage large volumes of data that reside on IBM z Hardware.
A Robust Platform for the Consolidation of Analytics
IzODA, which includes both Spark and Anaconda, provides a robust platform for the consolidation of analytics, both access to enterprise data sources and scalability, which makes IBM Z look like any other data source—extending the possibilities for analytics. With Spark and Anaconda, organizations can leverage existing analytics skills, and reduce time to value for delivery of new analytics visualizations.
Security, data access, and the ability to bring the compute nodes nearer to the data sources will improve responsiveness, accuracy, reduces data latency, and they help to eliminate unnecessary Extract Transform & Load (ETL) processes used to migrate data to other platforms. IzODA’s Optimized Data Layer provides unique data abstraction capabilities that enable data scientists to access key data sources, whether they are on or off the mainframe, through a common interface.
What’s included in IzODA
IzODA leverages zIIP processors to offload a significant amount of compute requirements and, as an offering, provides the following components to simplify the ordering, installation and maintenance of the solution:
z/OS IzODA Spark
- z/OS IzODA Spark (z/OS Spark) is built on Apache Spark, a high-performance, general execution engine for large-scale data processing. One of its key features is the capability to perform in-memory computing. Unlike traditional large data processing technologies, Spark allows caching of intermediate results in memory rather than writing them to disk, thereby dramatically improving the performance of iterative processing.
z/OS IzODA Mainframe Data Service
- z/OS IzODA Mainframe Data Service (Data Service or MDS) provides integration facilities for both IBM Z data sources and other off-platform data sources. The Data Service provides your Apache Spark application with optimized, virtualized, and parallelized access to a wide variety of data.
z/OS IzODA Anaconda
- z/OS IzODA Anaconda includes Python and Anaconda Python packages for data science, which provide data scientists with a comprehensive solution for integrating computations to the data.
z/OS IzODA Livy – included with Spark
- z/OS IzODA Livy is built on Apache Livy. It is a REST service used in conjunction with Spark, which enables users to submit Spark jobs without having the Spark client installed. This enables developers to harness the data analytics power that Spark is capable of providing from within a web or mobile application. Jobs are submitted to the Livy server through REST API that contains information about the Spark application to be run. An interactive Scala or Python session with Livy can also be started.
How does the IBM Db2 Analytics Accelerator (IDAA) fit into all of this?
The IDAA leverages the high Quality of Service capabilities of the System z hardware, while also streamlining and enabling near real-time data analytics and query optimization, via the appliance. Query optimization is seamlessly identified by the Db2 Optimizer, and queries that benefit from the accelerator are transparently executed on the appliance—offloading query executions, while maintaining complete transparency of execution location to the requesting end user. The IDAA becomes an extension to the existing Db2 on zOS infrastructure.
IDAA provides a seamless offloading of queries to enable substantial benefit to the organization by delivering data driven insights:
- Transparent to applications and reporting tools that access Db2 data
- Inherits Db2 for z/OS data security attributes
- Simplified database design – no need for indexes
- Eliminates need for tuning of analytic queries
- Fast deployment and time-to-value
- Eliminates duplication of data and data storage
- Eliminates substantial CPU requirements of Extract/Transform/Load (ETL) batch processes
- Eliminates excessive/unnecessary table views and indexes, further reducing administrative overheads
When both IzODA and IDAA are present, SparkSQL calls that qualify will be transparently accelerated on the IDAA. Combining IzODA and IDAA can provide unprecedented performance and analytics compute power on massive quantities of data, without requiring massive data movement.
With today’s corporate demands to have near real-time analytics of customer behaviors, customer purchase trends and real world sales trends, the ability to provide a simplified, integrated solution for cohesive analytics is essential to maintain business responsiveness, acuity and timely analytics insight.
Next Generation Analytics
The hybridization of System z multiprocessing and qualities of service technologies, along with the Open Data Analytics and IDAA architecture of massively parallel processing of data, provides fit-for-purpose and mixed-workload capabilities within a single computing ecosystem. This computing model provides exceptional capabilities for both transaction-oriented applications (OLTP) and seamless business analytics (OLAP), while also meeting the needs of data scientists’ predictive and cognitive analytics requirements. The addition of Open Data Analytics (IzODA) to an existing System z is the next logical step in creating a highly optimized environment, with secure management of data and centralized analytics, as well as outstanding price, performance and administrative features that can continue to address the range of analytics needs for today’s business community.
More Information
As a Platinum IBM business partner, Mainline has extensive experience with IBM mainframe systems, and we can help you extract more value out of your IBM z investment. For more information, please contact your Mainline Account Executive directly or click here to contact us with any questions.
Related Blogs/Videos:
IDAA and IIAS – Enterprise Analytics and Acceleration Options