The Future of
Business Intelligence
&
Data Warehousing

Created by Matthew Elphick

Business Intelligence

What is Business Intelligence?



Set of techniques & tools to transform data into useful information for analysis.

Business Intelligence Qualities & Benefits


A decision aiding tool

Transforms data into something useful that can be analysed.

Front end, relies on data.

Data Warehousing

What is Data Warehousing?



A repository of different sources of data structured for query and analysis.

Data Warehouse Qualities & Benefits


Back end, behind-the-scenes.

Data in one location to simplify access.
This means that the sources and warehouse do not affect each other.

Simplification and centralisation of query logic.

Tracks change via historical snapshots of when data changes, useful as transaction systems may not.

Data structure optimised for reading large amounts of data on demand.

How are BI & DW used?

The Business Intelligence applications act as the front end to access and analyse the data held within a Data Warehouse.

A BI Portal acts as the interface for the Data Warehouse and Business Intelligence applications.


Usually this is as a browser application.

This portal should be -
  • Usable - it should be easy to find things.
  • Content Rich - it should contain reports and analytical functionality as well as documentation, support etc.
  • Consistent - it should ideally look and feel similar to other applications the organisation uses.
  • Current - it should be updated regularly.
  • Value Oriented - it must make the user feel that it is worth using.

Current Trends

Back in 2009 there was a report predicting the top BI trends -

  • Green Computing
  • Social Networking Services
  • Data Visualisation
  • Mobile BI
  • Predictive Analysis
  • Composite Applications
  • Cloud Computing
  • Multitouch

As of today, what are the state of these areas?

Green Computing

Ideas on topics of data compression and general processing acceleration for hardware and software generally lead to lower power consumption, contributing to a greener approach.

Green Computing

Data Warehouses may exist in Data Centers, which have had several best practices guides proposed, such as one by the National Renewable Energy Laboratory for the U.S. Department of Energy in 2011.

These guides suggest approaches such as waste heat recycling, on-site generation, cooling systems, and air management.

Social Networking Services

In 2009, Facebook had 360 million monthly active users.

By 2014, they had 1,393 million monthly active users.

Social Networking Services

There have been issues with bringing together the traditional quantitative data and the qualitative social media data.

This issue is lessened by BI systems that can manage both structured and unstructured data.

Social Networking Services

There are still issues with privacy in this area that have yet to be fully considered.

Privacy


In 2000, Latanya Sweeney found that 87% of all Americans could be identified with only 3 pieces of information - (ZIP code, birthdate, sex).

In 2012, Target figured out a girl was pregnant before her father knew.

...deferential privacy, a way of letting people make statistical queries on some kind of records without seeing the records themselves.
Kira Radinsky (Co-founder of SalesPredict) in 2014

Social Business Intelligence

At the moment this is a growing area that's branching in two different directions.

  • The first uses social tools to collaborate on business intelligence.
  • The second uses business intelligence tools on the social media itself, internally and externally.

A product of the second approach is the Social Business Index, launched in 2011, that provides an insight into how 'social' a company is.

Data Visualisation

This has become an overwhelmingly important factor in first-time BI buyers. (40% mention is as their top priority.)

It is closely linked to dashboards.
(Which ~100% of buyers cite as their top requested tool.)

With the growing use and complexity of BI, it is more important to make the information easier to consume.

This is area is impacting on Mobile BI.

Used with the Storytelling approach as an effective way to convey data.

Storytelling

Some may struggle with data-driven decision making because they don’t know the story behind the data.

The idea is that the visualisation is something that is created from the story.

AIm to influence and enagage on an emotional and logical level.

Can allow interactive exploration or perhaps acts as graphical elements with an intended interface.

Mobile BI

There remains issues with this, and Mobile BI is still in the 'early adopter' phase, and has yet to fully mature.

Some issues include Security Considerations at different levels.

Despite this, in a June 2012 CFO Magazine Survey, 89% ranked Mobility as the #1 technology of importance to their company's success over the next three years.

Mobile BI

MicroStrategy released MicroStrategy Mobile for iPhone in 2010, and an Android version followed shortly in 2011.

JasperSoft (acquired in 2014 by TIBCO) also provided an open source SDK to help build mobile BI apps alongside their own native iPhone, and non-native iPad apps.

There are other Mobile BI products, such as Oracle Business Intelligence Mobile, IBM's Cognos Business Intelligence, and Tableau Server.

Looking to the Future

  • General all around improvements to response time, scaling and concurrency.
  • Real-time, or close to it.
  • Open source replacing vendor. (JasperReports, Pentaho)
  • Expanding to use more unstructured data.
  • Even more social media and external data - this will require intermediaries who rate data quality.
  • An increase in storytelling use in data visualisation.

Real-time, expanding to unstructured data


Recent entrants into the BI space include Zoomdata (founded 2012), which focuses on real-time and unstructured data, via data models in Hadoop, MongoDB or Cassandra.

Storytelling

In 2014 storytelling was a popular subject of dicussion, including by the likes of Google and Harvard Business Review.

Google in particular suggests that a hybrid author-reader approach based on research by the Standford Visualisation Group, with them demonstrating the approach via their The Customer Journey to Online Purchase Tool.

This approach is similar to the Data Journalism trend that is occuring in media, and the fact that some BI systems are now being used in this area as well may impact the path they will take, or may create new, specialised, systems.

Companies Future Outlooks

SAP's Outlook


In 2014 research indicates that employees are more likely to use cloud-based BI tools compared to traditional tools.

Focusing on the Cloud, with recent acquisitions and the launch of Hana Cloud, which can be used to deploy Business Intelligence. In such a case Hana can be used as a platform and as a data source.

Microsoft


In 2014 Microsoft released PowerBI, their move into the Business Intelligence sector.

A key feature of PowerBI is that it provides access to natural language queries.

They are also looking to incorporate the newly launched Azure Machine Learning Service.

The inclusion of Machine Learning will allow applications to not only provide current status results and responses to queries, but to also begin to make predictions, play out hypothetical scenarios and also to learn to make suggestions.

Q & A's