The discussion continued for some time, until, this xkcd cartoon was pulled up. The issue as to why GroupB was correct was then apparent to all, even the least technical, and the discussion was over. 

BI, as an idea, is a highly abstracted concept that permeates into every fissure of an organization. Everybody makes decisions, and every decision is based off information. Albeit, sometime’s is pretty horrible information (and thus is bad BI), but it’s information nonetheless. BI Ranges all the way from the Data Warehousing to the Decision-Making.

When it comes time to implement a new BI system at your workplace, how do you draw the lines of jurisdiction of your new team or department? You can’t simply have an all encompassing ‘God Team’ that collects information, analyzes data, and makes everyone’s decisions for them. The exact implementation will, of course, vary greatly from organization to organization. Various people are better suited for different parts of the BI machine. Database Admins and Data Architects are more suited for being in control of the Data Warehousing and Data Governing. Senior Management, making the driving decisions, often is a primary customer of BI systems. So how is the best way to determine what’s the role of the ‘Business Intelligence Team’ and what is not?

This is something that I am currently struggling with. I’d like to create the said ‘God Team’, but it’s just not realistic. The objective is to maximize information use while minimizing overhead associated with the BI Team.

This is an open ended post, and am hoping to hear thoughts from others. I may also add follow-up posts to this one.

That is, would it ever be conceivable to want an hourly or daily report on a historical event? I’m going to argue yes, in some circumstances this might give you the upper hand.  The historical data may change, but because it already happened and is in the past, some might be willing to take a one time observation and move on. Below is an actual e-mail thread from a few days ago at work that prompted the thought. 

From: Dave
Sent: Thursday, February 26, 2009 10:06 AM
To: Henry VIII
Subject: Spanish Armada

Hey, History Buff -

British defeat of the Spanish Armada interest you at all?

From: Henry VIII
Sent: Thursday, February 26, 2009 10:07 AM
To: David Kennedy
Subject: RE: Spanish Armada

Heck YA! I LOVE Tudor England! Why??

From: David Kennedy 
Sent: Thursday, February 26, 2009 10:09 AM
To: Henry VIII
Subject: RE: Spanish Armada

Apparently, new evidence shows this may have been attributable to the English having Big-Ass guns.

http://tech.slashdot.org/article.pl?sid=09/02/24/2337202&from=rss

From: Henry VIII
Sent: Thursday, February 26, 2009 10:15 AM
To: David Kennedy
Subject: RE: Spanish Armada

??? As opposed to a Big-Ass NAVY, made from uber lightweight wood? I MUST read about this!!

From: David Kennedy 
Sent: Thursday, February 26, 2009 10:18 AM
To: Henry VIII
Subject: RE: Spanish Armada

I know it! I was always like ‘Big Ass Navy! Yeah!’. But now I’m like ‘Big-Ass Navy and Bad-Ass Guns! HELL YEAH! History just keeps getting’ better.’

From: Henry VIII
Sent: Thursday, February 26, 2009 10:23 AM
To: David Kennedy
Subject: RE: Spanish Armada

I need you to automate for me the number of Spanish deaths caused by big ass guns as opposed to Spanish deaths caused by stealthiness of big ass navy.

I also need to to pull stats on to what extent the weight of gold and booty caused Spanish ships to sink off the coast of Britain as opposed to what sinkage was caused by pure wrath of God.

Please leave out any outliers like scurvy or senseless drunken duels in your deaths report.

Thank you.

From: David Kennedy 
Sent: Thursday, February 26, 2009 10:31 AM
To: Henry VIII
Subject: RE: Spanish Armada

Automate?!? You mean this is a recurring report?

Otherwise, your request looks fine.

From: Henry VIII
Sent: Thursday, February 26, 2009 10:33 AM
To: David Kennedy
Subject: RE: Spanish Armada

Clearly it is recurring, it is recurring as you just presented new findings this morning.

Shall I request a ticket?

From: David Kennedy 
Sent: Thursday, February 26, 2009 10:36 AM
To: Henry VIII
Subject: RE: Spanish Armada

Actually, in all seriousness, that brings up a good point…. Just because something happened once does not imply that it would necessarily be a one time report – as new information may be uncovered or revealed.

And there was some hidden humor in my comment “History just keeps getting’ better.”; History doesn’t change – just our understanding of it. Somehow though, I giggled like a school-girl to myself when I made the comment.

But if our understanding of it changes – then in some situations, there may be the need for continuous (or at least periodic) visibility.

Oh, and the Analytics team is very busy with projectX items. Can’t be bothered, so don’t put in a ticket. I’ll just pull the report as a personal favor.

I consequently  felt the need to see what all the talk was about. 

R is a Statically Scoped, Weakly Typed programming language. It is an open source project available for most platforms from the R Projects website.

I installed R and went through a few basic tutorials on-line – just to get the feel for it.

To me, it feels more like a computational environment than an traditional programming language. It definitely reminds me of my MATLAB, Maple, and Octave * days. However, according to the R Project’s page on R, “For computationally-intensive tasks, C, C++ and Fortran code can be linked and called at run time. Advanced users can write C code to manipulate R objects directly. ” Although that does sound more like a programming language to me, the environment still allows for quick commands. For example, you can quickly fire up R and type ’1+2′. This is a power that you don’t get with a language like C. Some people would consider R like ‘Excel with a command prompt’. 

In Windows, the R GUI Environment consists of a command prompt, a script editor, a plot window, and a data editor. You can  do anything you want through the command prompt. Plots appear in the plow window. You can use the data editor to edit data sets in spreadsheet fashion.  The script editor allows you to type up commands and save them for later.  In Linux, fire up R by typing ‘R’ at the command prompt. From there on you’re in R-mode.  

 

R Windows GUI Environment

I am, however, struggling with how to write an actual ‘program’ in R. So far the best that I’ve been able to manage is a collection of commands that I can either run individually or as a group.  

After playing with R for a little while, it definitely feels like a language only worth learning if you have a need or interest in stats/data analysis **

* Not that I’m implying that MATLAB, Maple, and Octave aren’t real languages! 
** I hate to say ‘only worth learning if [any condition], but you get the idea .

A Data Cube is a data structure used in Business Intelligence and Analytics to allow for fast analysis of data.

Okay, sounds great, right? But what does that mean? A Data Cube is a data structure specifically designed to provide large amounts of information quickly, especially for visual display.

But isn’t that what my database if for? No, not exactly. A [relational] database is great at recording very granular data in no particular order. A data cube contains data that has been collected and is ready for immediate display in a table, chart, or graph. 

How is that really any superior to a database, I still don’t get it. I mean, it sounds like we’re just pulling data from the database and putting it in a temporary location before it’s rendered as a graph. Why not exclude the middle man and go straight from database to graph? This is the beauty of the data cube. A data cube is called a cube because it is multidimensional. A cube is 3D, but a ‘Data Cube’ may be as many dimensions as you want.  Three example dimensions of a cube might be { [Products], [Cities], [Time] }.  You may quickly view data as Product Sales per Quarter, Product Popularity by City,  or Sales per City per Quarter. You may also clump data together, or break it down: view data by country instead of by city, or view by month instead of by quarter. 

Viewing the data from different angles like this would still be possible from a database – but queries would take longer to run, databases could suffer performance drops, and changing your point-of-view of data would be a much more painful experience. 

Wow, sounds great. In summary, a data cube is a data structure storing data in multiple dimensions, allowing for quick analysis of the data from different angles. The data cube may be used as a source of data for reports, graphs, charts, or casual curiosity.