The Science of Analysing Data

Data analytics is the science of analysing data in order to make inferences. At BI builders we offer a broad range of analytical techniques, each suitable for its own distinctive problem. 

  • Descriptive analytics helps answer questions about the current status-quo. It usually includes dashboard, reports and different type of KPI’s. 
  • Diagnostic analytics helps answer questions about why things happened. This may include analytics to discover anomalies, or unknown relations among data elements. 
  • Predictive analytics helps answer questions about what will happen in the future. Techniques to answer such questions falls in the field of machine learning 
  • Prescriptive analytics helps answer questions about what should be done. Prescriptive analytics is what we often call artificial intelligence. It is a kind of machine learning that takes in new data all the time to produce more accurate predictions and more well-defined decisions. 

Process of a project 

Descriptive, diagnostic, predictive and prescriptive analytics supports and build on each other. Therefore, we advise our clients to start with descriptive analytics and move to predictive and prescriptive analytics as they gain more insight. 

While descriptive analysis is quite straightforward to understand, Diagnostic, Predictive and Prescriptive analytics requires more explanation. In the following we will describe three main steps of dealing with these types of analytics: 

  • Problem definition 
  • Data exploration 
  • Modeling 
  • Validation, implementation and visualization 

Problem definition 

When defining the problem, it is important to discuss the business reasons for an analytics project. Together with all the stakeholders, we will try to envision what the solution will look like and what it should be capable of. This is usually a mix between technical challenges and analytical challenges as it’s a key step of the analytical process. 

Data exploration. 

The quality of the data used in the model will decide the quality of the result. Here are some important elements to keep in mind: 

  • Variable identification
    Understanding the business process is a key element in identifying the variables to be used in the analytics process. Still, this is a trade-off, as reducing too much the set of variables taken in consideration may improve the model performance but may also hide some “unknown” – but important – relations among fields not obviously related. 
  • Univariate analysis
    The relevant variables are analysed individually by using mathematical methods to identify their distributionThe result of this step is typically visualized by means of boxplots or distributions (in the case of numeric variables) or with bar charts (in the case of categorical variables). Outlier identification is an important aspect of this analysis. 
  • Bivariate analysis
    Bivariate analysis explores possible relationships between variables. Bivariate analysis is perhaps the simplest way to determine to what extent we can predict a value of one variable from another. Uni- and bivariate analysis (often called descriptive or diagnostic statistics) can help to both discover “unknown” relations between data fields and also to reduce the number of variables to be used in the model by identifying “variables that carry out the same information”. 
  • Missing value treatment 
    We sometimes need to deal with missing values in the data to avoid a biased model. Imputation techniques is a typical way to handle missing data. 
  • Feature engineering (variable creation and transformation)
    Feature engineering is the science (and art) of extracting more information from existing data. Which variables needs to create and transformed is strictly correlated to the problem definition. 


Depending on the problem definition and the results from the data exploration phase we may choose mathematical (often statistical) models with variant levels of complexity; from simple linear regression to machine learning by means of clustering or decision trees 

Validation, implementation and visualization  

When the model is refined enough to be used, it needs to be validated on new data to assess it’s ‘usefulness and accuracy’. 

In most cases, feature engineering, modeling and validation are an iterative process, where new knowledge gained by the exploration and modeling processes leads to new ideas worth exploring.  


Curious to know more on what we have done in the advanced analytics field? Contact us at:  

A look back at 2019

A look back at 2019


As Christmas is upon us, we take a moment to appreciate the highlights of 2019

BI Builders

2019 has been a really exciting year. We can definitely say that we are a growth company. The last year we have had a 40% increase in revenue, and it looks like we will continue to grow with even higher numbers in 2020. We really have exciting days ahead, with a lot of opportunities for the future.


Visibility is important, and in the last months, we have seen a lot about BI Builders at Linkedin and Facebook. Customer videos form Brynild Gruppen, Petoro and Norsea Group all make me proud of our company and our services.

New Colleagues

As the demand from new and existing customers continues to grow, so does our need for new talent. This year we have strengthened our team with 16 new employees.

Our Oslo office has grown from two to nine highly motivated colleagues, our technical resource pool has increased and we have secured key management roles to assist further growth for BI Builders.

Even with this increase in manpower, we are still looking for more to join the team.

Teambuilding trip to Italy

This autumn we went on a teambuilding trip to Italy. It is for most employees one of the highlights of the year. This year it was even together with our better halves.

As a CEO it is important to show appreciation, not only to the employees but also to their partners. It is thanks to their support that we can continue to deliver great results and meet our customers’ expectations.

We had a great weekend in the beautiful city of Florence. Enjoying biking, wine-tasting, team development, sightseeing, and fantastic food.  We are already looking forward to our next trip.


There has been many great events in 2019. I think we have participated in more than 20 events this year. Among these both Gartner in London and Barcelona together with Pass Summit in Seattle have been most fruitful.

Gartner London

Gartner Data and Analytics Summit in March. Quite a busy week with a lot of visitors and interesting sessions. A busy London week gave a lot of homework for the following weeks.

Gartner Barcelona

Gartner IT Symposium in Barcelona in November. The week was packed with meetings, lectures, and visits to our stand in the Expo section. My highlight was Regional Director, Terje Vatle’s presentation on “Why your data platform in the cloud is too expensive and doesn’t deliver on time”.

PASS Summit

We also attended PASS Summit 2019 in Seattle. This was a technical event, and the team got valuable insight that they brought back and shared with the rest of the team. SQL Server 2019 was released and looks very promising with respect to future options.

Breakfast Seminar Oslo

As part of our growing presence in Oslo, we invited IT managers, CTO’s and tech-interested contacts to a Breakfast Seminar in Oslo.

The seminar was held at MESH, and we welcomed over 50 eager guests. We are now looking forward to our next breakfast seminar in Stavanger in February 2020.

New York Office

We attended the Innovation Norway’s Entrepreneurial Marketing Programme this spring. A really exciting program put together by Innovation Norway. We learned a lot and gained valuable insight into the American way of running business. If you ever get the opportunity to attend you should all accept. This is really useful.

This summer we announced that BI Builders was opening an office in New York. This is a new chapter for us as a company, and we are excited to take on this challenge. Our Business Development Manager, Henning Hummervoll will lead the process from New York, with close collaboration with the management team and technical support team in Norway

Gaselle Company 2018

For the fourth year, we made the Gaselle list. I am very proud of everyone in BI Builders who has contributed to this success. Not only did we make the list, but we were on the 20th place out of 511 companies!

These were just some of the highlights from 2019. I am proud of everything that we have achieved and excited to make 2020 even more eventful.

– Jarle Soland, CEO

“They just don’t want to change” – About technology adoption and a data-driven culture.

“They just don’t want to change” – About technology adoption and a data-driven culture.

Eva Van Schijndel

The transition towards a data driven organization is one with many technical, organizational and cultural challenges. It requires courage from employees and there is a theory behind it. Integrating a technology challenges the social rules of a business. Those rules must be addressed and sometimes even rewritten alongside the desire to innovate.

What is a data-driven culture? Being data-driven means that data is the center from which the operating environment executes its tasks with the purpose to enhance the efficiency, effectiveness and to give it a  competitive edge. Besides the obvious technical and organizational challenges that are part of a transition towards a data-driven organization, fostering a culture that legitimizes the change seems to be the most elusive.

In my work as a data scientist, I often hear management expressing their frustration about the willingness of colleagues to adopt new technologies. Resisting employees are judged for their ‘lack of knowledge’ of the technology and called ‘laggards’. This term comes from the innovation-diffusion graph of Everett Rogers which is originally used to explain how, why and at what rate new ideas and technology spread (Rogers, 2003).

I notice that this graph is used a lot to classify and judge people, but adopting a technology is not just the result of identities like the ‘early adopter’ or ‘the laggard’. Sometimes the right supporting system for change is simply not in place. In my experience, one of these supporting systems that are often lacking is ‘coaching’.


Eva Van Schijndel

The theory is as follows: People, of any age and any group, have an inherent need to know how they can obtain rewards and avoid punishment. In a working environment, this means we like to know the recipe for a successful career. We look at other colleagues to discover the ‘rules of the game’ and use it to orient our behavior. Knowing ‘what to do’ reduces feelings of uncertainty and helps to create stability when we do our work (Geels, 2004.). The rules are very stubborn because we tend to enforce, endorse and reproduce them to protect ourselves from the possibility of chaos and from feeling unsure about ourselves.

There are three kinds of rules in organizations:

  1. Regulative rules (laws, sanctions and cost structures)
  2. Normative rules (norms, values and authority systems)
  3. Cognitive rules.

These cognitive rules are very difficult to track down as they are taken for granted. They describe things you often do not think about like: what you should know, what you should prioritize or what you should believe.

While all rules can be responsible for a slow adoption rate of a technology, I believe that constraining cognitive rules remain under the radar, because they are often not recognized nor acted upon with a clear plan. Instead of investigating possible constraining rules, integration of technology is often ‘forced’. Management takes it for granted (which is a belief and therefore a cognitive rule) that employees resist change. They gamble that by forcing the change with an executive decision (which is a normative rule) employees will turn around eventually. This strategy may seriously limit the potential impact in terms of adoption of innovation because without supporting rules the decision lacks legitimacy. This is sometimes referred to as ‘system failure’ due to institutional constraints (Klein Woolthuis, Lankhuizen, & Gilsing, 2005).


So how can management better understand these elusive cognitive rules and act upon them strategically? Let me elaborate with some examples. A ‘lack of time’ is a well-known argument against change. You can ask yourself which cognitive rules are preventing colleagues from adopting a technology. The ‘lack of time’- the argument is possibly raised in defense; co-workers have their projects and deadlines from before a technology is introduced. The old rules still demand these projects to be finished in time. They will have to be rewritten to allow co-workers the time to learn and get used to the new technology for example by down prioritizing other projects.

The argument that colleagues just ‘hold on to their old methods’ might have something to do with avoiding feelings of uncertainty. Co-workers have a body of knowledge that has enabled them to do their work and deliver a certain quality. A new technology still needs to be learned, making him or her unsure if they can deliver the same quality as they did before. The rules need to be redefined to assure a colleague that it’s okay that their work might differ from the old standard and that as a team they can write new a quality standard together.

The argument that co-workers just ‘lack knowledge’ assumes that if one has the same knowledge about a technology like you have, they will see the world your way. This is also a cognitive rule, a belief, set by the person in favor of the technology. They are incapable to hear the arguments of colleagues that are against the new technology because of their own bias.

There might be more than one way to solve the problem. Redefining cognitive rules is difficult because they are taken for granted. However, by reflecting on your own thoughts they may reveal themselves. Coaches can help with this reflection. To redefine cognitive rules that deal with performance coaching may be done by managers, who are in the position to endorse and enforce the new rules once they are redefined which generates a strong basis for a legitimate change. To redefine cognitive rules that deal with beliefs coaching may be done by independent coaches who are trained to ask reflective questions allowing someone to explore their goals and the possible biases that drive these goals.

These examples are just a small part of the process of innovation and technology adoption. This study is much larger and very complex with many more interacting variables. Even though it is a complex process, hiring coaches or allowing co-workers to train to become coaches is a very easy method to discover some of the more elusive drivers of resistance to change. On an institutional level, this will increase the impact of the adoption of technology and smoothen the process. And a smoother process means that a business is more flexible enabling it to develop a more competitive edge.


Works Cited

Geels, F. (2004.). From sectoral system of innovation to socio-technical system; Insights about dynamics and change from sociology and institutional theory. . Research policy, 33, 887-920.

Klein Woolthuis, R., Lankhuizen, M., & Gilsing, V. (2005). A systems failure framework for innovation policy design. Technovation, 609-619.

Rogers, E. (2003). Diffusion of Innovation, 5th Edition. Simon and Schuster.


Meet Eva van Schijndel at our Free Breakfast Seminar

Analytics Consultant, Eva will present at BI Builders free breakfast seminar on November 28th. The seminar is for anyone who is interested in the topic “How to use data as a strategic resource”

Sign up today to guarantee your spot.

DWH Automation – Progression Over Time

DWH Automation – Progression Over Time

By Gaurav Sood.

In all the recorded history one common thing amongst any successful enterprise has been the presence of Data. It can well be argued that computer only came into existence some 70 years ago, but data was always present much like air and water. Data must be at the center of every business decision we take as business owners. In 2017 Economist published a report titled “The world’s most valuable resource is no longer oil, but data”. I feel data is much more powerful than oil. First, it’s never going to cease to exist, secondly, with time we are only going to produce more data. But in its unrefined form data is not of much use, so like oil, it must be refined and turned into insights that drive business decisions to make it a profitable entity.

Some of the biggest companies globally have used data towards targeted marketing to propel an idea or spread propaganda. The widespread disruption happening today is a result of all businesses moving towards a digital era. Leading companies in Norway today have digital and data mentioned in their strategies and yearly report to shareholders (for most industries and public sector using data better is a strategic focus area).

Keeping these factors in mind, it becomes essential to work on procuring the right data, transforming it into meaningful information and eventually deciphering the information into a business-related action. The usual way is to set up a Data warehouse (DWH) which will help in data storage, integration and feeding transformed data to decision-makers. However, this is easier said than done. Setting up a DWH comes with its own set of challenges like

  • Figuring out the technology to use keeping in mind the competence available and size of the DWH
  • Mapping all the sources and targets.
  • Gathering all the business logic in one place.
  • Data quality.
  • Business dependency on the DWH determining the importance to keep it as updated as possible.
  • Need for considerable Time, Effort & Cost.

and possibly many more.

An interesting way to put the DWH discussion in 1 phrase is  “The DWH is dead, long live the DWH”. All major global digital enterprises(FB, Google, Netflix, Tesla, etc.) have a DWH of some sort but they need to be realistic, adjust governance levels and be more agile in their ways of working. DWH is usually associated with cumbersome and endless projects, long time to market and the endless need to try and create one model for the whole enterprise which is inherently almost impossible (thus not delivering on the promises).

A typical DWH lifecycle looks something like this:
Up until 2010, there were not many automation tools available in the market to accelerate the process of setting up a DWH. Businesses were heavily dependent on programmers to automate whatever part of the flow they could. Setting up a mid-sized DWH with data from 20 – 30  different sources and creating 15-20 reports could easily take anywhere between 4 – 10 months depending on the number and experience of the resources implementing it. This can be a long time for a business to start getting some return on investments.

Forrester defines Datawarehouse Automation (DWA) as  “DWA is not a data warehouse appliance, nor data-warehouse-as-a-service (DWaaS) – it’s software that automatically generates a data warehouse by analyzing the data itself and applying best practices for DW design embedded in the technology. Another name for this type of technology is “metadata-generated analytics”.

The automation scenario has changed over the last decade or so with a multitude of DWA tools coming into the market. Some of the more established technology players like Microsoft, Google have launched their own DWA tools. There has been a spurt in the availability of automation tools from small startups. This has obviously led to fierce competition which is good for the end consumer as he will get the best product. DWA is just a collection of DWH best practices bundled into software to provide businesses with faster access to insights and their data.

Features of a DWA tool

  1. Simplified capture of the Data Warehouse Design.
  2. Automated Build (i.e. Generate Code and metadata)
  3. Automated Deployment of code to the Server
  4. Automated Batch execution of the ETL code on the Server.
  5. Automated Monitoring and Reporting of the Batch execution.
  6. Automated optimization of data loads (Parallel Vs Series).
  7. Metadata based active governance and control of your data.
  8. Agility in responding faster to the changing business needs.

In the past Data warehousing has taken too long and the results haven’t been too flexible. A small change or improvement could take up to weeks or months to be implemented. Amid this progression towards DWA, a lot of other options were tried like Big Data, Self Service BI etc. But a Data warehouse provides additional benefits like

  1. The ability to store history.
  2. Reduced risk of reliance on key individuals.
  3. Data augmentation.

Automation does not mean throwing out the concepts of Data Warehousing, in fact, it reinforces the same concepts with more focus on the execution of the Data Warehouse development.

DWA is often confused with Self-serve Data Preparation (SSDP). This is not entirely correct. SSDP is primarily meant for data scientists/data engineers working on specific use-cases. It is not meant to be used for Enterprise level DWH deployments. DWA and SSDP offer different features. Holistic/enterprise metadata control is not the same as building one simple (SSDP) pipeline in a cloud-based solution.

The main aim of these automation tools is to create solutions which make it possible for business users to access data, integrated from multiple sources and to prepare that data using drag and drop features and a simple, intuitive interface. They should be able to perform

  • Data Preparation
  • Test theories and hypotheses.
  • Prototype test price points.

Most of the DWA tools available in the market are GUI based and you can set up a DWH with just a few clicks. Many of the existing DWA tools offer lineage functionalities as well as automated regression and quality tests, efficient loading routines, simplified deployments between environments and extensive generation of documentation.

A simple illustration of different functions covered by a DWA tool is in the image below.

With all the digital disruption happening around the globe, it is more important than ever to make sense of the abundant pools of data. Businesses need the ability to make smarter decisions at a click of a button. Traditional DWH methodologies and best practices must come together in building a data solution which can support the ever-changing business needs, hence the need for automation.

DWH Automation: Transforming the world of DWH & BI

Back in 2007 when I first started working with ETL processes and building Data warehouses, it was a steep learning path. Setting up a Data Warehouse (DWH) required plenty of planning and project management before the actual development could start. There were various steps that lead to a functional DWH capable of servicing the business needs.

To name a few

  • Create Source to Target Mappings including business rules and logic.
  • Create a naming convention to be used throughout the DWH.
  • Document the business logic as you go on building the DWH.
  • Generate and store Surrogate Keys.
  • Decide on what goes in SCD 1 and SCD 2.
  • Design the architecture, whether it would be Star or Snowflake or just some form of denormalized data.
  • Deciding on the load frequency.
  • Finally loading it into target data model.

It is estimated that in a normal Business Intelligence (BI) project, close to 80% of the time and budget is spent on setting up the DWH. It is important to understand here that a solid DWH architecture and design sets up your entire BI project for success. Any critical failures or misunderstandings in the design and architecture of DWH can have serious business consequences. Considering these factors automating your DWH implementation is a step that every company would like to invest in.

At BI Builders we have just the right product for your DWH automation – Xpert BI. It’s a culmination of best practices in DWH implementation gathered over the years by developing DWH solutions on premise and in cloud. Xpert BI integrates all your data, from local files, complex systems to cloud applications, in a central information platform, thereby empowering the team to produce actionable insights at a quicker pace than before. You can choose your own infrastructure, Xpert BI supports both on premise and cloud instances. It is certified for Microsoft Azure and is also available in the marketplace. It generates standard SQL code in the backend, so it’s easy to debug.

Here is how Xpert BI delivers faster implementation of a DWH.

Xpert BI is a one stop solution for implementing

  • New DWH from scratch.
  • Creating a Data Mart on top of/alongside an existing DWH to support a specific business area.
  • Exporting on premises data to the cloud, with Export Groups (Exporting data from an entire source system to Azure Data Lake can be done in a few clicks)
  • Documenting SSAS models and exposing them to business users.

We are working round the clock at BI Builders to improve the product and come up with connectors to make your data integration as smooth as possible.

Check out our website to know more about the product, available data connectors and read customer success stories.

Around the world in 14 days

Around the world in 14 days

Being in the software business, one of our most important tasks is to let our customers know about our product. One of the ways we do that is to attend various conferences around the world. For two consecutive weeks in November we will be spend a lot of time on the conference carpet, in sessions, and in our hotel rooms.

Pass Summit 2017

First out is the PASS Summit in Seattle. This is the Microsoft SQL Server user based conference with a lot of great topics on both traditional SQL Server and data warehousing, but probably more topics on the new architecture and the new possibilities in Azure.

We are living in exiting times with regards to data strategy, data architecture and technology.

Since we are a Microsoft partner we need to both have an opinion and a strategy with respect to Azure. It is going to be very exciting to talk to the best SQL Server people in the world about both our current product and to lift the veil on the future of Xpert BI. So, if you are going to PASS, drop by booth K4 at the launchpad area and talk to us, we might have some nice swag to give away. And I promise you will have a great data strategy or data warehouse talk and of course a demo of the best DWA tool on the marked.

While I am writing this, I am trying to come up with a topic for a ten-minute speed talk on a Norwegian conference coming up in October. I think my topic will be something like, if everyone is a data scientist, who is going to do the ETL? It is still is a mystery to me that people are skeptical of doing DWA on their ETL so that the road to the data scientist role gets shorter. I guess one of our goals on the PASS Summit is to convert some of the manual ETL developers to see that DWA can be a good thing, and not only yet another costly software we have to learn.

After we say goodbye to the space needle we fly directly to Barcelona to attend the Gartner Symposium.

The Gartner Symposium is a bit different from the PASS Summit, where the PASS Summit gather the SQL Server nerds from all over the world, the Gartner Symposium is more a C level Gartner Symposium ITXPO Barcelona 2017conference. Our focus here will be to show the great benefits of investing in our software to enable not only your data warehouse but also your digitalization strategies.

Anja, our head of project implementation and co-founder of BI Builders, is going to talk about how your “old” technology can co-exist with the more “modern” ways of modelling or storing your data (Please, pay attention to the quotation marks).


BI & Analytics – What will be your Fit for Purpose solution?

Does the introduction of new technologies mean your current toolsets are obsolete, and will they be able to co-exist? 

BI Builders will discuss the impact of the changing size and content of data in organizations regarding reporting, analytics and fact-based decision-making.

I am confident this is a hot topic for most of both the BI and analytic leaders and the CIO and CTO’s attending the conference.

So, as much as we hope this will be a great way to both meet new customers and partners from around the globe, we also hope we get to learn something from all the attendees and other conference partners’ as well.

It is you that make our product and our advising better by letting us learn from what you do. At the same time, we hope that we will be able to inspire the attendees into doing things smarter, cheaper and faster.