Evaluation BI Alternatives

Once the brainstorming process is completed, and the BI Blueprint prepared, evaluation of BI opportunties is undertaken. This is done by a smaller group of individuals; at a minimum this should include a business analyst and an IT or systems expert.

The aim of this phase is to convert the information from the BI blueprint into a list of BI opportunity areas for more detailed evaluation.

There are four steps in the process of evaluating BI opportunities:

• Group requirements into opportunity areas.
• Score opportunities by importance.
• Grade opportunities by difficulty.
• Rank opportunities (including possible return on investment)

Group Requirements into Opportunity Areas

The BI blueprint is expanded and reorganized into BI categories or BI groups of measures and dimensions. The opportunity areas can then be individually discussed and evaluated.

These opportunity areas can be assessed both in business terms and technical terms

Business - a consistent set of requirements for a group of users that can be supported by the same system structures or solution. For instance in the sales function, opportunity areas include: sales management, prospect management, and demand forecasting. . Typical cross-functional opportunity areas are product contribution analysis and customer/channel profitability.

Technical - grouping into opportunities areas means that measure requirements can be assessed across all dimensions at the same lowest level of detail, and also viewed in terms of data source.

The following table provides an example of opportunity areas across industries.

Gap & Conflict Analysis

By arranging the BI Blueprint informatin into this opportunity area structure, you can then perform an analysis to identify gaps and conflicts between information needs of different functional areas, and the data available.

For instance, using our previous sales example, a sales BI opportunity is shown to ne driven by the need for information by sales representative. This need is in somewhat different from the product margin and customer support measures needed at a higher role level in sales, plus it differs in terms of the time dimension, by needing weekly timetable.

Likewise, a customer support opportunity is driven by measures - # calls and call length. Neither of thes measures have any alignment with sales, orders, and other measures. Thus we see a gap in alignment between measures used both within the opportunity areas as well as between two opportunity areas.

Scoring Opportunities by Importance

Comprehensive BI Opportunity analysis will likely reveal may opportunity areas in most functional areas of the business, plus a number of potential cross functional areas.

We now need to rank the importance of each opportunity, based on a scoring using three criteria:

1. Tactical vs. Strategic
2. Actionability - can action be taken based on the BI information
3. Impact - the impact of that action on overall operational [tactical] and corporate [strategic] performance

For each of these criteria, a score of high, medium, and low is assigned to each opportunity area.

Tactical vs. Strategic

Consider each opportunity area in terms of its tactical vs. strategic impact on the business. This includes the BI impact on short-term objectives and operating results [Tactical] vs. long-term goals or significant competitive advantages [strategic].

The most important impact of business intelligence is often strategic in nature, from the compunding effect of improved decision making by managers and executives. Unfortunately the quality of decision making is difficult to measure, requiring a high level of faith in purely strategic BI initiatives. For this reason, strategic BI initiatives, are at higher risk without a strong commitment by executives and high level managers to use the BI system.

While this criterion overlaps the materiality issue

Tactical vs. strategic opportunities can be characterized by:

Strategic opportunites include those where:

• There is greater interest in BI solutions and higher potential for use at higher levels of the organization, thereby affording a greater the impact on the organisation.
• There is a high degree of cross-functional requirements and user groups.
• The BI solution has the potential to align disparate organisational functions such as customer support/channel profitability or financial planning/forecasting.

Strategic opportunties should not be automatically scored low priority, just as Tactical opportunities should not be automatically scored low priority, especially if systems are already in place to support BI implementation. However it can be challenging and time wasting to gain management support on strategic BI opportunities if there are critical tactical issues that BI can resolve without the political expense.

Larger projects can often be both tactical as well as strategic. These arise particularly when information needs span many organizational levels. In these instances, the addtional effort to resolve potential political issues are worthwhile. Such political issues are often difficult to overcome, but organisations that make the leap benefit in many collateral ways, other than just direct impacts of BI. In this way, a BI program can be a significant change agent.

Actionability of Information

Although the BI system may reveal where and why a particular result in being recorded, impact on corporate performance is only possible if action can be taken to remedy the issue. For instance:

• Sales volumes can be boosted by reducing pricing, bundling or further promotional activity.
• Production quality can be remedied through reprimand, reassignment of resources, heightened QA or more training.

BI opportunities that score low on actionability of information automatically make them low priority on overall importance of the information.

Impact Of Action Taken Based on BI Information

For each area, determine the benefits in terms of long term savings or revenue potential. Is this result material to the success of the business in meeting its objectives.

In a production environment, there are both strategic and tactical impacts of production efficiencies, inventory levels, supplier relations, and customer satisfaction are critical success areas, and therefore any impact in these areas is material to overall success.

BI opportunities that are not financially or strategically material regardless of actionability should be scored low priority. This typically includes all 'noncore activities'.

Applying the Importance Criteria

Using our example opportunity areas we apply an overall grade of high, medium, or low to indicate the overall importance rating.

It is important to remember that 'importance' does not refer to the importance of the functional area to the business, rather it refers to the impact of the specific BI opportunities being proposed.

Grade Opportunities by Difficulty

Especially for the first few BI projects, it is important to choose BI opportunity areas that are easier to implement—regardless of how the potential projects rank in importance.

Understanding the difficulty of each opportunity requires both an understanding of the current business culture, and its propensity to change, the political difficulties and the IT challenges and the BI skill available.

A quick test for difficulty is based on three criteria:

1. Cross-functionality of design - more difficult to design and implement due to variances in measures, data hierarchies and terminology, and more politically complex. BI design across multiple functional groups, generally have more measures, more dimensions, lower levels of detail, and more complex hierarchies. Cross-functional opportunities are given a score of hard. Functional opportunities are given a score of easy.
2. Existence and accessibility of data - scoring depends upon whether data is available to support both current and future measures, dimensions and metrics, and how easy it is to access the data, and process it within the timeframes required. This requires estimation of data volume for both storage and volume that must be processed with each refresh cycle.
3. Complexity of calculations - depends greatly on source of data. While relational databases, particularly OLAP databases have extensive capabilities for calculating measures across multiple dimensions, the more complex the calculated measures required, the more difficult the implementation will be. Projects that assimilate data for base measures from the OLTP systems and roll up the data into hierarchies across each dimension without many calculated measures are the easiest to calculate. Sourcing information from multiple OLTP systems can also be especially challenging. Whist these issues can be resolved, they do add more complexity to the system design and implementation.

Applying these three criteria to assign an overall easy, medium, or hard difficulty grade to each opportunity area.

Applying the Difficulty Criteria

At this stage of the program, assessment of difficulty, based on the above three criteria is largely a quick assessment. During implementation of the BI solution, these difficulty ratings will be adjusted, as the business requirements are refined and the source data to support the dimensions and measures is located.

Ranking Opportunities

The final step of evaluating BI opportunities are:

1. Creating a BI Scorecard - to allow quick visualisation of how how different BI opportunities compare
2. Cost Benefits Analysis - assessing the costs, benefits, and financial returns of specific opportunity areas.

Creating a BI Opportunity Scorecard

The BI Opportunity Scorecard is a pictorial representation of the opportunity areas evaluated using the criteria of importance and difficulty.

To build the scorecard:

1. Draw a quadrant - labelling clockwise from top left; High importance/Low difficulty; High importance/High difficulty; Low importance/Low difficulty; Low importance/high difficulty
2. Number the various opportunity areas and assign a brief description
3. Place each opportunity number into the appropriate quadrant.

The quadrant is purely a relative gauge for assessing the importance and feasibility of information requirements.

From the scorecard, a sense of the relative feasibility of BI opportunities is achieved. You can then develop a short list for further analysis in terms of development cost, benefits, and financial return.

High/Easy Opportunities - good candidates to evaluate more and start soon.

Medium/Easy and Low/Easy Opportunities - good quick wins. Weigh the relative value of the opportunities against their difficulty of implementation.

High/Medium and High/Hard Opportunities - consider doing a pilot project to limit the resource commitment until you determine how difficult the project will be.

Medium/Medium and Low/Medium Opportunities - again consider pilots, but after funding for the above pilots.

Medium/Hard and Low/Hard Opportunities - little justification at this time. Place on hold.

Costs, Benefits Analysis

BI opportunities are typically more difficult to evaluate than other IT projects using OLTP systems, where the traditional return on investment, payback, and discounted cash flow techniques, are more commonly evaluated, and cost of process failures known.

Business intelligence benefits are not easily quantifiable in advance. The impact of having information sooner, the improved quality of decisions, the new marketplace insights and tactics, and potential shifts in competitive strategy are long term outcomes that result from a culmination of tactical efforts.

Assessment is even more complex for cross-functional opportunities such as product margin analysis and pricing [to support product profitability and positioning decisions], and customer-oriented projects such as customer/channel profitability and customer satisfaction.

The impact of having good information about products or customers has a large impact on decisions around pricing strategies, product offerings, and customer segmentation, but are difficult to quantify in financial terms.

The best effort will be attained by documenting quantitative items first - hard project costs and cost savings, revenue benefits. Then add the intangible benefits long term and short term.

The real return on investment from business intelligence increases proportional to its use within the organisation and the adoption of a BI attitude in decision making.

Hard Project Costs

Project costs typical include:

• Cost of new hardware or the opportunity cost of using current hardware
• Cost of software, including ETL tools, relational and OLAP databases, application packages, and front-end software for users
• Internal development costs—staff and expenses for designing and implementing the BI opportunity
• External development costs—service professionals experienced in BI development (This expense is particularly important when dealing with first-time or complex opportunity areas.)
• Internal training (both business and technical IT staff need to get up to speed on the BI technologies to be used and user training on front-end tools)
• Ongoing maintenance after implementation

Tangible Benefits

Tangible, quantifiable benefits may include a return on investment or payback calculation such as:

• Time saved in producing reports
• Operating efficiencies from specific information, for example, the Audi assembly line economics
• Lower investment levels—that is, cost of capital savings—resulting from process improvements such as better inventory or accounts receivable
• Improved customer service and satisfaction and therefore higher revenues from add-on products, services, warranties, replacement business, and so forth

Intangible Benefits

Intangible benefits are more difficult to quantify, but offer the greatest and fastest paybacks, such as:

• Improved operational and strategic decisions from better and more timely information
• Improved employee communications and job satisfaction resulting from a greater sense of empowerment
• Improved knowledge sharing

Back To: Identifying BI Opportunties

Back To Top