BI Best Practices

Adopting best practices in business intelligence give you a better chance of success with your own BI deployment. There are certain imperatives that apply regardless of what type of business you have, what type of BI solution you select or how big your BI implementation is.

Prepare Well Ahead

Customer-facing programs and monitoring of critical business processes constantly demands fresh data. Regardless of what BI tools you deploy, once users recognise the value of reliable, consistent data they make demands for data latency to be reduced even further.

Reaching real-time BI requires implemention of real-time technologies. The key is to plan ahead of demand and stay ahead. Making the right BI technology choices up front of demand is the only way you will keep ahead of business user requests.

Planning a 'services' approach for real-time BI infrastructure up front means that data loads can be automatically managed by continuously monitoring all warehouse processes and respond rapidly to issues - such as data loading failures and long query response times. Bureau automates all processes all the time, regardless of how often they run.

Know Where Real Time Data Is Really Needed.

It is easy for business users to demand real time data, but often the value given does not validate the additional investment to deliver data beyond daily or 2 hourly updates.

• Real-time data feeds require more infrastructure and are more difficult to manage.
• Real-time processes must be constantly monitored. Issues arise within business hours, rather than during batch laods done overnight. This requires fast response support teams, putting further pressure on support staff.
• Additional hardware and capacity is required to store the data. Generally, each real-time feed requires two servers, one to run the load and a second to back it up.
• Real-time data feeds from some source systems can be expensive and even impossible to implement.

Justification for real time data is a critical part of your BI solution planning.

Educate the Business

Many business users either don't have the insight or the time to explore what is possible with real-time BI. A common mistake is for business users to want new systems to replicate what the old ones did - just without the bugs or with a few additional cells.

By developing prototypes of potential new BI systems that would manage hub operations better, users realise that technology can help them improve the way they manage their business. The biggest challenge becomes finding the resources to support the ideas that users have.

Align Decision-making and Business Processes

There are three sources of latency in real-time BI:

1. Time required to extract data from source systems
2. Tme required to analyze the data
3. Tme required to act upon the data

The first two are easily dealt with using real-time technologies. The third, the most problematic is getting people and processes to change. If you fail to ensure that downstream decision-making and business processes efficiently utilize real-time data, the value of having real time data decreases.

Co-existing Strategic and Tactical Decision Support

Traditionally, data warehouses have focused on supporting strategic decisions. Operational decision making was supported by operational data stores.

Real-time technologies moves decision making to the tactical level, in an 'operational BI' model. In a BI environment, strategic and tactical decision support co-exist in the same warehouse environment. However, the performance requirements for each differ:

• Strategic decision support - analysis of large amounts of data at less frequent periods, but requiring high processing capacity
• Tactical decision support - repeated access to only a limited amount of data, in real time or very frequent intervals. Queries are small, requiring less processing.

These differing requirements must be scoped to set priorities - for example, a data-mining query should have a lower priority than a tactical query. Each class of query must be sized for capacity planning and priority in the queue. Continuous monitoring alerts support IT if the queue gets out of control.

The Right People For The Right Jobs

Too often, BI resources are drawn from traditional IT pools. Most of these people have experience with transactional systems and databases supporting transactional systems. The design and operation of a data warehouse to support BI is fundamentally different and requires not only a different skill set, but a different mind set. Some can appreciate the difference and are able to apply their experience with transaction-oriented, real-time systems to the BI environment. Many cannot.

BI requires more understanding of how the business works. Building Standard Business Value libraries and Master Data sets needs a contextual basis. Good technical skills does not necessarily translate into good BI skills.

Eventually, the data warehouse personnel, who work most closely with business users, develop considerable business knowledge. In the interim, IT resources and business users must work closely together to ensure this divide is bridged.

Automation of ETL Processes

Extracting data from transactional systems and feeding it in real time to a data warehouse is a finely tuned process. A process that needs to be as automated as possible. Human intervention should only be required when an alert is triggered from the monitoring system.

The process must also be be a 'services' model - flexible and reusable.

Manage Storage Retention Periods

With real-time data warehousing, data changes more frequently than with traditional warehousing. To store all changed records over extended periods results in massive data volumes and significantly impacts computing resources and data access.

Just as query requirements are carefully analysed and scoped, data availability must also be carefully assessed. The business must carefully decide which changes to store in the warehouse and which changes should be overwritten. For example, an airline constantly overwrites a flights ETA, as there is no business value in tracking changes. However, all data about customers, bookings, and seat inventory are preserved at every point in time, as this data supports a a wide variety of business uses.

To reduce complexity further, views can be created that only include active records, shielding users who need current data from complex query statements.

Using 3NL

Storing the enterprise warehouse data in 3rd normal form supports and encourages enterprise-wide use. Data is easily maintained, redundancies are eliminated, shared data definitions are agreed and an enterprise-wide view of the data is provided. This the the most likely model to provide data that meets the requirement of 'a single version of the truth'.

Business users who are confident in enterprise wide data are less protective about wanting to retain their local data silos.

Next: BI Excellence Survey

Back To Top

BI Strategy Index | BI Vision | PM Objectives | Drivers Of BI | Barriers To BI | BI Lifecycle | BI Strategy Document | BI Long Term | BI Evolution | BI Best Practices | BI Excellence Survey | BI Scorecard | BI & ERP | BI & SMBs | BI and Cloud Computing