Tuesday 27 August 2013

510. Big Data - real-time analytics

real-time analytics

E-Mail

Essential Guide

From data gathering to competitive strategy: The evolution of big data

· This article is part of an Essential Guide, our editor-selected collection of our best articles, videos and other content on this topic. Explore more in this guide:

4. - Big data: Some terms to know: Read more in this section

Read more in this section

· Explore other sections in this guide:

View all sections in this guide

Real-time analytics is the use of, or the capacity to use, all available enterprise data and resources when they are needed. It consists of dynamic analysis and reporting, based on data entered into a system less than one minute before the actual time of use. Real-time analytics is also known as real-time data analytics, real-time data integration, and real-time intelligence.
Technologies that support real-time analytics include:

Processing in memory (PIM) -- a chip architecture in which the processor is integrated into a memory chip to reduce latency.
In-database analytics -- a technology that allows data processing to be conducted within the database by building analytic logic into the database itself.
Data warehouse appliances -- combination hardware and software products designed specifically for analytical processing. An appliance allows the purchaser to deploy a high-performance data warehouse right out of the box.
In-memory analytics -- an approach to querying data when it resides in random access memory (RAM), as opposed to querying data that is stored on physical disks.
Massively parallel programming (MPP) -- the coordinated processing of a program by multiple processors that work on different parts of the program, with each processor using its own operating system and memory.

Applications of real-time analytics
In CRM (customer relations management), real-time analytics can provide up-to-the-minute information about an enterprise's customers and present it so that better and quicker business decisions can be made -- perhaps even within the time span of a customer interaction. Real-time analytics can support instant refreshes to corporate dashboards to reflect business changes throughout the day. In a data warehouse context, real-time analytics supports unpredictable, ad hoc queries against large data sets. Another application is in scientific analysis such as the tracking of a hurricane's path, intensity, and wind field, with the intent of predicting these parameters hours or days in advance.
The adjective real-time refers to a level of computer responsiveness that a user senses as immediate or nearly immediate, or that enables a computer to keep up with some external process (for example, to present visualizations of Web site activity as it constantly changes).
See also:

1. real-time business intelligence,

2. operational business intelligence,

3. data mining,

4. business analytics,

5. ad-hoc analysis

509. Big Data - Six disruptive possibilities from big data

Strata

Six disruptive possibilities from big data

Specific ways big data will inundate vendors and customers.

My new book, Disruptive Possibilities: How Big Data Changes Everything, is derived directly from my experience as a performance and platform architect in the old enterprise world and the new, Internet-scale world.

I pre-date the Hadoop crew at Yahoo!, but I intimately understood the grid engineering that made Hadoop possible.

For years, the working title of this book was The Art and Craft of Platform Engineering, and when I started working on Hadoop after a stint in the Red Hat kernel group, many of the ideas that were jammed into my head, going back to my experience with early supercomputers, all seem to make perfect sense for Hadoop.

This is why I frequently refer to big data as “commercial supercomputing.”

In Disruptive Possibilities, I discuss the implications of the big data ecosystem over the next few years.

These implications will inundate vendors and customers in a number of ways, including:

1. The disruption to the silo mentality, both in IT organizations and the industry that serves them, will be the big story of big data.

2. The IT industry will be battered by the new technology of big data because many of the products that pre-date Hadoop are laughably unaffordable at scale.

Big data hardware and software is hundreds of times faster than existing enterprise-scale products and often thousands of times cheaper.

3. Technology as new and disruptive as big data is often resisted by IT organizations because their corporate mandate requires them to obsess about minimizing OPEX and not tolerate innovation, forcing IT to be the big bad wolf of big data.

4. IT organizations will be affected by the generation that replaces those who invested their careers in Oracle, Microsoft, and EMC.

The old adage “no one ever gets fired for buying (historically) IBM” only applies to mature, established technology, not to immature and disruptive technology.

Big data is the most disruptive force this industry has seen since the introduction of the relational database.

5. Big data requires data scientists and programmers to develop a better understanding of how the data flows underneath them, including an introduction (or reintroduction) to the computing platform that makes it possible.

This may be outside of their comfort zones if they are similarly entrenched within silos.

Professionals willing to learn new ways of collaborating, working, and thinking will prosper.

That prosperity will be as much about highly efficient and small teams of people as it is about highly efficient and large groups of servers.

6. Civil liberties and privacy will be compromised as technology improvements make it affordable for any organization (private, public or clandestine) to analyze the patterns of data and behavior of anyone who uses a mobile phone.

tags: Big Data, business, customers, disruption, Disruptive Possibilities, ecosystem, enterprise, vendors

508. Big Data - The Real World of Big Data

The Real World of Big Data

The benefits of Big Data are often spoken of in the future tense. As in, “Big Data will someday provide enterprises of all types critical insights that allow for increased profitability, improved efficiency and other untold riches.”

Same goes for the technology. Hadoop, some say, will be the foundation of data storage and analytics in the enterprise once it’s proven enterprise-ready.

The reality, however, is that Big Data is today – here and now – delivering on its many promises. We at Wikibon have been documenting Big Data in the Real World for the last two years, including publishing a series of vertical-specific research notes highlighting how enterprises in retail, banking, media, utilities and pharma are leveraging Big Data analytics to drive performance.

We were also the first analyst firm, in February of 2012, to publish a comprehensive Big Data market size report. Today, Wikibon’s Big Data Vendor Revenue and Market Forecast 2012-2017 provides figures on real revenue that vendors, large and small, are today deriving from Big Data products and services.

Today, we’re proud to debut the latest offering in our ongoing efforts to chronicle the reality of Big Data, this Real World of Big Data infographic. Our goal with the infographic is to communicate the real world applications of and revenue created by Big Data in the enterprise in a visually exciting way. You’ll notice a handful of Big Data uses cases, including dynamic price optimization at Sears, real-time marketing campaign analysis by Lyris and analytics to support drug discovery at Bristol Myers Squibb.

There’s no question we have a long way to go before Big Data is ubiquitous in the enterprise. But, as our infographic makes clear, there are plenty of Big Data success stories to talk about in the here and now. Have an interesting use case or success story with Big Data? Share it in the comments or Tweet us at @Wikibon and @jeffreyfkelly.

jignasa