Agile Business Intelligence

This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these messages) This article contains wording that promotes the subject in a subjective manner without imparting real information. Please remove or replace such wording and instead of making proclamations about a subject's importance, use facts and attribution to demonstrate that importance. (April 2024) (Learn how and when to remove this message) This article contains promotional content. Please help improve it by removing promotional language and inappropriate external links, and by adding encyclopedic text written from a neutral point of view. (April 2024) (Learn how and when to remove this message) This article may need to be rewritten to comply with Wikipedia's quality standards. You can help. The talk page may contain suggestions. (March 2013) This article needs attention from an expert in Business. The specific problem is: Need to review and possibly re-write Agile Business Intelligence. WikiProject Business may be able to help recruit an expert. (March 2013) (Learn how and when to remove this message)

Agile business intelligence (ABI) refers to the use of agile software development for business intelligence (BI) projects. ABI attempts to enable the BI team, business people, analysts and/ or stakeholders to make business decisions more quickly.^[1][2]

There are multiple approaches for improving or increasing BI agility. Factors considered important for the success of ABI projects include: a holistic approach to BI architectures, organizational forms, and technologies; and the use of agile process models adapted to BI.

Agile methodology works on an iterative principle. This involves providing new features to end users sooner than with a traditional waterfall processes, which deliver only the final product. With this technique, the requirements and design phases overlap with development, thus reducing the development cycles to achieve faster delivery. It promotes adaptive planning, evolutionary development and delivery, a time-boxed iterative approach, and encourages rapid and flexible responses to change.^[3] ABI encourages business users and IT professionals to think about their data differently, and aims to achieve a low total cost of change (TCC).^[2] With ABI, the focus is not on solving every BI problem at once but rather on delivering pieces of BI functionality in manageable chunks via shorter development cycles and documenting each cycle as it happens.^[4][5]

Agile business intelligence is a continual process, enabling managers to access quick and accurate product data for informed decision-making. ABI enables rapid development using the agile methodology. Agile techniques are a way to promote the development of BI applications, such as dashboards, balanced scorecards, reports, and analytic applications.^[6]

According to the research by the Aberdeen Group, organizations with the most established ABI implementations are more likely to have processes in place for ensuring that business needs are being met.^[7] The success of ABI implementation also heavily depends on end user participation and "frequent collaboration between IT and the business."^[7]

Agile business intelligence (ABI) is a methodology that integrates processes, tools, and organizational structures to enable decision-makers to adapt more effectively to dynamic business and regulatory environments.^[7]

Aberdeen's Maturity Class Framework^[5] uses three key performance criteria:

1. Availability of timely management information: IT should be able to provide the right and accurate information in a timely manner to the business managers to make sound business decisions. “This performance metric captures the frequency with which business users receive the information they need in the time-frame they need it.”^[5]
2. Average time required to add a column to an existing report: Sometimes new columns need to be added to an existing report to see the required information. "If that information cannot be obtained within the time required to support the decision at hand, the information has no material value. This metric measures the total elapsed time required to modify an existing report by adding a column."^[5]
3. Average time required to create a new dashboard: This metric considers the time required to access any new or updated information and it measures the total elapsed time required to create a new dashboard. ^[5]

Agile SDLC Iterative Process

Bruni^[8] in her article 5 Steps to Agile BI, outlines the five elements that promote an ABI enterprise environment.

1. Agile Development Methodology: “Need for an agile, iterative process that speeds the time to market of BI requests by shortening development cycles.”^[8]
2. Agile Project Management Methodology: Continuous planning and execution. Planning is done at the beginning of each cycle, rather than one time at the beginning of the project as in traditional projects. In Agile project, scope can be changed any time during the development phase.
3. Agile Infrastructure: The system should have virtualization and horizontal scaling capability. This gives flexibility to easily modify the infrastructure and could also maintain near-real-time BI more easily than the standard Extract, transform, load (ETL) model.^[8]
4. Cloud & Agile BI: Many organizations are implementing cloud technology now as it is the cheaper alternative to store and transfer data. Companies that are in their initial stages of implementing Agile BI should consider cloud technology, as cloud services can now support BI and ETL software to be provisioned in the cloud.^[8]
5. IT Organization & Agile BI: To achieve agility and maximum effectiveness, the IT team should interact with the business but also address the business problems and should have a strong and cohesive team.^[8]

Wayne Kernochan of Infostructure Associates conducted a two-year study of a number of BI processes across a handful of businesses, and came up with the below model and its goals:^[9]

1. Data entry — accuracy
2. Data consolidation — consistency
3. Data aggregation — scope
4. Information targeting — fit
5. Information delivery — timeliness
6. Information analysis —analytic ability

Kernochan also discovered these common issues with the current BI processes:^[9]

• 20% of data contains errors (accuracy).
• 50% of data is inconsistent (consistency)
• It typically takes 7 days to get data to the end user (timeliness)
• It isn't possible to do a cross-database query on 70% of company data (scope)
• Executives don't receive the data they need 65% of the time (fit)
• 60% of the time, users can't do immediate online analysis of data they receive (analyse ability)
• 75% of new key information sources that surface on the Web are not passed on to users within the year (agility)

The result concluded that adding agility to existing business intelligence will minimize problems. Organizations are slowly transitioning their processes to agile methodology and development. ABI will play a big part in the company's success as it "emphasizes integration with agile development and innovation."

There are several factors that influence the success of ABI.

20% of data is inaccurate and about 50% is inconsistent and these numbers increase with new type of data. Processes need to be re-evaluated and corrected to minimize data entry errors.^[9]

Often, companies have multiple data stores, and data is scattered across multiple data stores. "Agility theory emphasizes auto-discovery of each new data source and automated upgrade of metadata repositories to automatically accommodate the new information."^[9]

Data aggregation is a process in which information from many data stores is pulled and displayed in a summary report. Online analytical processing (OLAP) is a simple type of data aggregation tools which is commonly used.

One of the key principles of ABI is to deliver the right data at the right time to the right individual. Historical data should also be maintained for comparing the current performance with the past.^[9]

One of the largest benefits of ABI is in improving the decision-making of its users. Real ABI should focus on analysis tools that make an operational process or new product development better.^[9] The ABI approach will save companies money, time, and resources that would otherwise be needed to build a traditional data warehouse using the Waterfall methodology.

• A team of developers and business representatives should be assembled to work together.
• Select either a business stakeholder or technical liaisons to represent the business.
• Identify and prioritize appropriate user stories or requirements to address during an initial project.^[1]
• Assess various ABI delivery tools that can integrate with your existing data warehouse and BI environment.^[1]
• Initiate iterative development process

ABI drives its users to self-serve BI. It offers organizations flexibility in terms of delivery, user adoption, and ROI.

Using Agile methodology, the product is delivered in shorter development cycles with multiple iterations.^[10] Each iteration is working software and can be deployed to production.

In an Agile development environment, IT and business personnel work together (often in the same room) refining the business needs in each iteration.^[10] "This increases user adoption by focusing on the frequently changing needs of the non-technical business user, leading to high end-user engagement and resulting in higher user adoption rates."^[10]

Organizations can achieve increased rate-of-return (ROI) due to shorter development cycles. This minimizes the IT resources and time while delivering working, relevant reports to end-users.^[10]

1. A program charter should be created, which will set the stakeholder expectations on how ABI system will work.^[11]
2. Start with the business information that needs to provide context for scope.^[11]
3. Iterations should be timed.^[11]
4. Stress on data discovery through the requirements and design phases.^[11]
5. Use the Agile process of incremental and iterative development and deployment.^[11]
6. Validate the BI architecture and get approval on the proof of concept.^[11]
7. Data validation and verification should be completed for each development iteration.^[11]
8. Use flow charts or diagrams to explain the BI process along with some documentation.^[11]
9. Any change that will be deployed to production should be thoroughly tested in a regression environment.^[11]
10. Have a formal change control; this will minimize the risk as all changes have to be approved before it goes into production.^[11]