DWH Automation – Progression Over Time

By Gaurav Sood. In all the recorded history one common thing amongst any successful enterprise has been the presence of Data. It can well be argued that computer only came into existence some 70 years ago, but data was always present much like air and water. Data must be at the center of every business decision we take as business owners. In 2017 Economist published a report titled “The world’s most valuable resource is no longer oil, but data”. I feel data is much more powerful than oil. First, it’s never going to cease to exist, secondly, with time we are only going to produce more data. But in its unrefined form data is not of much...

DWH Automation: Transforming the world of DWH & BI

Back in 2007 when I first started working with ETL processes and building Data warehouses, it was a steep learning path. Setting up a Data Warehouse (DWH) required plenty of planning and project management before the actual development could start. There were various steps that lead to a functional DWH capable of servicing the business needs. To name a few Create Source to Target Mappings including business rules and logic.Create a naming convention to be used throughout the DWH.Document the business logic as you go on building the DWH.Generate and store Surrogate Keys.Decide on what goes in SCD 1 and SCD 2.Design the architecture, whether...

Last Christmas I gave you Hadoop, but the very next day, you gave it away

I have spent some of the quiet Christmas nights in front of a burning fireplace reading articles about the future of the data warehouse. There are many opinions and arguments on how the future of data warehousing will be. Two types of professionals are argumenting the pros and the cons in regards of technical architecture surrounding your data warehouse solution. The technocrats make strong arguments in regards of specific technologies that will solve your challenges in the new data area. On the other hand, you have professionals that are more of the old school and are skeptical to let technology drive the type of challenge you are going to...

Around the world in 14 days

Being in the software business, one of our most important tasks is to let our customers know about our product. One of the ways we do that is to attend various conferences around the world. For two consecutive weeks in November we will be spend a lot of time on the conference carpet, in sessions, and in our hotel rooms. First out is the PASS Summit in Seattle. This is the Microsoft SQL Server user based conference with a lot of great topics on both traditional SQL Server and data warehousing, but probably more topics on the new architecture and the new possibilities in Azure. We are living in exiting times with regards to data strategy,...

My thoughts on «IT i Praksis»

On August 10th IKT-Norge, Visma and Rambøll presented their 10th Annual report called “IT i Praksis”. It’s a survey meant to look at the maturity of practical IT use mainly in the public sector of Norway. It’s a huge report and I won’t cover all the topics here but there were a couple of things that stood out. The survey really boils down to the word digitalization. How can the public sector move faster so their users get the best digital experience possible? This isn’t just a challenge for the public sector. It’s the same challenges if you are a retailer, a bank or really any other business where you need to connect with users in a digital...

My take on Data Lake

(I’m a poet and I know it) I have been spending a lot of time warning people about the pitfalls, or rather deep ends, of the Data Lakes and Big Data initiatives. First, I want to state that I think the Data Lake as a concept is a great idea, but it needs to co-exist with your more traditional data architecture. So, what I’m discussing in this post is my take on the total data architecture as I see it in today’s data age. We all know the data warehouse mantra “One single version of the truth” or the variance “One single source of the truth”. The single source statement will be even less relevant with the coming of Big Data. But the one...

Self-Service Data Preparation, why would you do that?

In my previous posts, I have talked a lot about preparing your data and not forgetting your structured data in this time of Big Data, IOT and other cool stuff. Today I thought I’d take a step back and try to explain what our company do when it comes to preparing and structuring data and discuss how this can be a viable option for you. So, what is Xpert BI? We call it Self-Service data preparation or data warehouse automation. So, what does that mean and why should you do this instead of doing traditional ETL or ELT?  Our CTO and the brain behind Xpert BI, Erik Frafjord, had an idea that the process of getting to your data prepped and ready...

9 days and 8 nights in London

I just got back from a long and exiting trip to London, where BI Builders has been sponsoring two big events. First Big Data World, where I was a speaker and then the Gartner Analytics Summit, where my colleague Anja was a speaker. And I would like to share my thoughts and key takeaways from this inspiring trip. So firstly, Big Data World, a huge conference that was co-located with four other conferences; Smart IOT, Cloud Security Expo, Cloud Expo and Data Centre World. As you probably can imagine the buzz words flew high here, Big Data vendors in all shapes and sizes had a booth and it was amazing to both hear about and get demonstrations...

Are we heading towards chaos? Or am I just getting old?

Last night I was attending an after work meet up where the topic was “Clash Of The Titans”. Microsoft, IBM, SAP and Oracle was presenting their BI and analytic solutions, both what they can offer today and how their future releases will be. Remembering back to the year 2000 when I started my first job as a data warehouse developer, we programmed SAS code without any intellisense on a dark blue background with a white font. After writing 500 lines of code we said a small prayer and then we pressed F8. Usually we got lucky, other times we didn’t, and had to use the rest of the day finding that small typo or looking for that breach in logic...

Why are we only talking about lakes, reservoirs, swamps and clouds?

The buzzwords and the endless possibilities that can be made in the cloud seems to be on everybody’s minds these days. Did we forget our core business? At the breakfast table at the TDWI conference in February 2016 I met two gentlemen and we started talking about Big Data, cloud BI and the other buzzwords. They told me a story from their company. “One day my boss comes into my office and asked, how are we on big data? I had to ask him back, what do you mean? My boss asked again, how are we on big data? Do we have Hadoop? And I said No? Well, we need that, my boss replied and walked out the door. So now we have a Hadoop cluster in the cloud...

©Copyright | BI Builders