BI AND THE PATH TO PREDICTIVE ANALYTICS

BI AND THE PATH
ANALYTICS
TO
PREDICTIVE
A Review of the Perceivant BI Platform for MidMarket Organizations
Robin Bloor, Ph.D.
Rebecca Jozwiak
WHITE PAPER
BI AND THE PATH TO PREDICTIVE ANALYTICS
Executive Summary
Perceivant’s Platform and Services deliver a comprehensive Business Intelligence (BI) solution
that focuses primarily on mid-sized businesses. In this paper we discuss its capability in the
context of the business use of BI. Our conclusions are summarized in the following bullet
points:
• Mid-sized companies grow at a faster rate than large enterprise organizations and
experience the same need as large organizations to exploit BI. In recent years, the
investment in BI has been increasing at a greater rate than almost all other investments
in IT.
• Nevertheless, most mid-market organizations have yet to properly exploit the
opportunity that BI provides, most of them having gone no further than to implement
regular reporting. The investment in dashboards and, to a larger degree advanced
analytics, has lagged.
• Conceptually, BI can be categorized as providing four distinct capabilities, all of which
can be provided by Perceivant:
!
– Hindsight: Consisting primarily of regular reporting
!
– Oversight: Consisting primarily of dashboards and associated capabilities
!
– Insight: Consisting primarily of drilldown capabilities and data mining
!
– Foresight: Consisting primarily of predictive analytics
• The Perceivant Platform targets the mid-market by offering a relatively low-cost, lowrisk software stack to provide BI to entry level users. It can be tailored for customization
or used out of the box. This can be regarded as a BI foundation for the business.
• Perceivant Services enhances this foundation through consultation both on the IT side to
specialize and enhance the capabilities it delivers, and in the area of analytics where it
can assist customers by providing data scientist expertise and consulting to exploit its
platform
• As a consequence mid-market businesses are equipped to compete head-to-head with
larger organizations reaping the many benefits of a comprehensive BI capability, as
illustrated by the brief use cases described in this paper. In our view, mid-market
companies would be wise to consider the solutions and services Perceivant provides.
1
BI AND THE PATH TO PREDICTIVE ANALYTICS
What is a Mid-Market Company to Do?
Definitions vary, but a mid-market company can generally be defined as having between 50
and 1,000 employees, with revenues in the $2 million to $100 million range. Research suggests
that while mid-sized companies account for just 3.2% of all companies in the U.S., they
provide 31% of all revenue and grow – in terms of both revenue and jobs – at a faster rate
than larger companies. Although organizations at this level are in the expansion phase, they
have established a niche and often have a long-term plan for growth. As such, a recent survey
showed that 60% of mid-market executives ranked technology as having the greatest
potential to increase business productivity.
Large businesses often have extensive BI capabilities established over years, involving a
variety of capabilities. In mid-market organizations an extensive focus on BI is less common.
Most likely BI functionalities will include regular reporting coupled with the ability to extract
data from various sources to be analyzed in MS Excel. The prevalence of media stories about
Big Data can sound daunting to a mid-market organization, and while some may think they
will never have the need to gather large heaps of data and subject it to any kind of analysis,
Big Data can be a relative term. Even if they do recognize the need, it is unlikely that these
companies have the resources in the form of IT staff or “data scientists” to explore and
leverage it.
Nevertheless they gather and process data, and the volume of their data increases year after
year. There can be little doubt that such businesses could profitably employ BI technologies
that, at the moment, they tend not to use, and perhaps know little about. The rapid adoption
of BI technology has been an established trend in the IT market for well over a decade.
According to analyst reports, BI spending has exhibited double-digit growth in most years,
growing at a faster rate than most other areas of IT spending. Clearly large organizations have
found good reason to invest in this technology, but few businesses in the mid-market have
capitalized on the business opportunity that BI technology presents.
Categories of BI
There is a wide variety of BI software that can be deployed in almost any business. For the
sake of clarity, we can classify all BI products and services as belonging to one of four
categories, according to what they deliver: hindsight, oversight, insight and foresight.
•
Hindsight: To this category belongs the reporting software that almost all businesses
use: regular reports, aggregations, decision support summaries, trend reports and so
on, often including graphs and bar charts for clarity. Such reports are historic to some
degree, since they are compiled from information that is days, weeks or months old.
•
Oversight: There are a variety of BI capabilities that help to provide oversight in the
sense of providing fairly up-to-date information, often showing thresholds that may
directly trigger alerts to some staff when an encouraging or discouraging trend
appears. These include dashboards and business process oversight capabilities. They
are normally fed by fairly recent or even real-time information.
•
Insight: BI tools that provide a business with insight range from online analytical
processing (OLAP) to data mining. OLAP provides drill-down capabilities that allow
the user to investigate the fine details of aggregations of data: sales figures,
2
BI AND THE PATH TO PREDICTIVE ANALYTICS
manufacturing defects, price trends and so on. Data mining embodies a variety of
sophisticated statistical techniques that reveal important trends and correlations. Such
BI capabilities are most effective when they harmonize with business processes to
ensure that their results lead to action.
•
Foresight: In terms of delivering business benefit, predictive analytics can be the most
effective aspect of BI. It can transform a business from being reactive to proactive.
Rather than simply responding to business events and trends, predictive analytics
enables a company to anticipate and exploit such events and trends.
There is a rough logical order to
the implementation of these BI
categories, from the perspectives
of both IT and business. From the
IT perspective, many reporting
capabilities (hindsight) can be fed
directly by data from source
systems. Where this is not feasible,
a data warehouse that organizes
company data can be built and
used to feed such capabilities.
FORESIGHT
(Predictive analytics)
Business
Benefit
Also IT
complexity
INSIGHT
(OLAP, data mining,
etc.)
OVERSIGHT
(dashboards, alerts,
etc.)
HINDSIGHT
(reporting)
Oversight can be more challenging
depending upon how current the
data needs to be. Dashboards, for
Time
example, require data feeds to be
more up-to-date than is sometimes
Figure 1. The Evolution of BI
feasible to achieve with a data
warehouse. Nevertheless it is usually possible to establish data feeds that are as current as
such BI applications need them to be.
Insight is a far more specialized area, since it involves extracting specific subsets of data and
applying a whole series of analytical techniques until useful knowledge is discovered within
the data. This is an ongoing data exploration activity which can fan out in many different
directions.
Finally, the foresight of predictive analytics usually requires the integration of data from
many sources including sources external to the business. From the IT perspective, it can be a
significant challenge, but from the business perspective, it can be essential for growth.
As we illustrate in Figure 1, while the implementation of the different categories of BI
becomes increasingly challenging, the business benefits escalate with each new capability. It is
important to note that these business benefits will not be delivered unless the business itself
adapts its processes to accommodate the intelligence that they provide. This is particularly
important when it comes to insight and foresight. Mid-sized businesses are unlikely to be able
to attract fully experienced data scientists, since data analytics is a highly skilled activity that
requires a knowledge of statistics, and predictive analytics tools need to be presided over by
such a skilled individual. In addition, it is important that the business adjusts its activities to
swiftly and surely leverage the knowledge that such activities deliver. This is not so easy to
achieve.
3
BI AND THE PATH TO PREDICTIVE ANALYTICS
Savvy mid-market companies want to expand, and their challenges are as complex and
relevant as those of a large enterprise, if not more so. They have the advantage, however, of
being nimble enough to adjust business processes more quickly than their monolithic
counterparts. Adopting advanced BI tools that cater to mid-market operations and resources
can undoubtedly put a mid-sized business at the leading edge of the competition.
One company that specializes in delivering and implementing advanced BI capabilities to
mid-range businesses is Perceivant. We now describe the solutions the company offers.
4
BI AND THE PATH TO PREDICTIVE ANALYTICS
The Perceivant Platform and Services
A frequent barrier to mid-market adoption of BI technology is cost, both in terms of hard and
soft resources. Growing companies simply cannot justify the risk and expense of the
hardware, software, training and licensing that usually accompany a BI implementation.
Perceivant offers a scalable software-as-a-service (SaaS) platform which eliminates the need to
purchase additional machines and reduces the human capital required to maintain the
environment, in turn providing enterprise-grade software at a fraction of the cost. What’s
more, Perceivant lets customers begin with a low-risk, short-term license which can be
extended after adoption.
As data grows, it tends to disperse in ways that make it difficult, if not impossible, to access.
This is especially true for organizations that don’t have individuals whose specific job is to
manage the data. Perceivant built its platform to negate this disparity, keeping the business
user in mind. It collects and joins all the organization’s data and stores it in a custom,
purpose-built data warehouse in the cloud. This gives mid-sized businesses the opportunity
to perform complex analysis on much larger data sets than they ever had before. Users
immediately have access to real-time data and the advanced analytical tools Perceivant
provides, and if desired, they can export the results to MS Excel or the interface to which they
are accustomed. It is worth noting that this takes place without any requests to IT.
Perceivant does this by helping customers leverage the power of familiar technologies:
Hadoop, Elastic Search and NoSQL databases. It also integrates with other public services
such as Google Prediction and Google BigQuery. But it has developed its own proprietary
and unique integration tools to make access to data and analytics both painless and easy to
use. In a recent performance benchmark using web traffic data, Perceivant returned a query
5x faster than Hadoop’s Hive, and it did so with only one server, compared to Hive’s six. It
has properly architected a platform solution that merges the power of commodity software
with a specialized, purpose-designed user interface, which can lead to faster insights and
better business process. As with any hosted service, users do not have to learn the inner
workings of several different applications, but instead can operate within their existing
environment.
Out of the box, the Perceivant Platform is natively suitable for the typical needs of the
customer. Additionally, from the initial implementation to the end-user experience, Perceivant
Services can provide custom solutions where required. Mid-market businesses thrive because
they know their bread and butter: the customer. Perceivant follows this model and offers
tailored solutions to meet the needs of the organization through direct consultancy and a
deep knowledge of analytics. Through its software and services, Perceivant can guide new-toBI users from basic reporting (hindsight) to the proactive realm of predictive analytics
(foresight).
The Perceivant message is clear: get the data in and give access, fast. Mid-range businesses do
not have the luxury of waiting six months or more for a BI installation. Time-to-value is
critical. The Perceivant Platform provides customers with an affordable entry point to BI via
its integrated software stack and can lead them down the path to achieve hindsight,
oversight, insight and foresight. A mid-sized business can transform its operations and
processes literally overnight, gaining advantage over competitors and closing the distance
between itself and better armed companies who use these tools every day.
5
BI AND THE PATH TO PREDICTIVE ANALYTICS
In the corporate environment, those who analyze, win. It is our view that Perceivant delivers
exactly the kind of business value that end users need: a unified view of information assets,
the ability to perform complex analytics on those assets, and an easy-to-use interface. Taken
together, the solutions that it provides can catapult a mid-sized company into unprecedented
growth. Perceivant has removed the barriers that often hinder BI adoption, and because of
this, mid-sized companies now have the opportunity to leverage an enterprise-class solution
with enterprise-class results. Organizations that fall into the mid-market category and wish to
reap the benefits of BI would do well to consider Perceivant.
6
BI AND THE PATH TO PREDICTIVE ANALYTICS
How Customers Use Perceivant
College Correlates Data Points to Increase Retention and Plan for the Future
Student recruitment and retention are the primary business goals for any college and are
ultimately driven by many factors: quality of teachers, availability of courses, classroom size
and demographic measures outside of the college’s control.
One of the largest community colleges in the United States had all of its data about teachers,
courses, and full and part time students in a traditional ERP system. Its data analytics
platform could provide baseline reporting on these areas but lacked the ability to correlate
data points between the silos of information. In addition, these reports took over 20 hours to
render, at a cost of $250,000 per year for the software license alone, with additional support
and hardware costs on top of that. The system also provided no historical view of data.
The college chose to move to the Perceivant Platform for its knowledge of predictive analytics
and its ability to offer services that would help leverage data at a market-competitive price.
The real-time nature of the reporting, the ability for these reports to be accessed by
individuals across the organization (rather than only by the IT staff) and Perceivant’s native
ability to cleanse data were also deciding factors.
Using the Perceivant Platform, the college will be able to connect data points around
enrollment, attrition, teachers and class assignments, student prospects and facility
management. They will use the data to predict future enrollment, manage class sizes, decide
course offerings and more accurately plan classroom usage in an effort to improve retention
for future semesters. They’ll utilize historical data to decipher what contributed to drop-out
rates and combine these with external economic data to create demographic-specific retention
programs.
In short, the Perceivant Platform is aiding the college in a path to cost savings, better
utilization of resources and increased student achievement.
Software Company Increases Efficiencies and Real-Time Response to Customer
Requests
A software company serving some of the world’s largest online retailers was tasked with
analyzing 5TB of website traffic and millions of product SKUs per client. With 50 employees
and $6 million in revenue, human and capital resources were scarce.
Often asked to find correlating product and shopper attributes in the midst of these enormous
amounts of traffic and SKUs, the company’s existing solution was a combination of Hadoop/
Hive and MongoDB, with updates entering the system nightly via batch. The limitations of
this solution for the company were two-fold.
First, many internal, cross-team and technical resources were needed to prepare and export
the data, with even more resources needed to analyze it. Productivity suffered with the
inefficiencies of creating a customer support ticket, handing it off to another department,
waiting for the data query to occur, then waiting for the results. Additionally, if insufficient
data was available, it would not be known until the entire query/send process was complete,
7
BI AND THE PATH TO PREDICTIVE ANALYTICS
forcing analysts to restart the process. Attaining a full and complete data set would often take
an entire day, something the company could neither afford nor manage to meet the real-time
demands of its clients. The second challenge was cost versus performance. Trying to run
analytics against Hive was still too complex, and it was slow, even for basic queries.
After attempting many changes within its own architecture, the company chose the
Perceivant Platform. Users found the interface familiar, resembling an Excel PivotTable, and
on-boarding new users to the platform was a simple task. Rather than data access and
analysis being a cross-functional process within the organization, data analysts had quick
access to the actual data and reporting functions, making them more efficient and responsive
to customer requests. The company has found Perceivant to be considerably cheaper and
faster. Queries that took 36 seconds in the Hive setup are taking 7 seconds on the Perceivant
Platform, at a lower cost.
Healthcare Company Opens New Revenue Streams
On average, companies pay almost $3.00 an hour to provide health benefits to their workers.
Decreasing these costs can therefore have significant bottom-line impact. A healthcare
company serving self-insured businesses received 7 million health insurance claims a day for
processing. The company wanted the capability to turn these claims into usable information
their clients could leverage to better know their employees’ health challenges (in aggregate)
and reduce healthcare costs.
While the company investigated developing its own in-house solution, the Perceivant
Platform’s competitive pricing and hosted solution, with no required internal IT resources,
changed its mind. The company used the Perceivant Platform to integrate its BI tools with its
client portal to provide high-level access to claims data. This initial implementation came
with few set-up costs and launched quickly, with very little risk. In Phase Two of the project,
the company will utilize Perceivant to provide additional BI features, creating a new revenue
stream by providing predictive analytical data based on claim information.
About The Bloor Group
The Bloor Group is a consulting, research and technology analysis firm that focuses on open research and the use
of modern media to gather knowledge and disseminate it to IT users. Visit both www.TheBloorGroup.com and
www.InsideAnalysis.com for more information. The Bloor Group is the sole copyright holder of this publication.
❏ PO Box 200638 ❏ Austin, T X 78 7 20 ❏ Tel: 512–524–3689 ❏
www.InsideAnalysis.com
www.BloorGroup.com
8