Big Change #3 – New Job

Recently, in May 2008, I took a position with Stratagem. Though the summer I worked for 2 places as a consultant, KHS in Waukesha, and The Dept. Of Regulation and Licensing (DRL) in Madison.

A little history. I was full time for W3i around 2 years, and then went independent for about a year, then as a consultant with Stratagem. All the different types (full, indie, and consultant) have the pros and cons, and it all depends on what you are doing, where you are, and things you are working on.

I have done .NET, Team Lead, Database Stuff, C++, Data Warehouse/BI (Business Intelligence), ERP Stuff, more .NET and everything in between. Over the past 2 years I have come to love the database stuff more and more, especially BI and Data Warehousing. It is funny, because pretty much 99% of people have no clue what “BI” is. I was just a High Tech Happy Hour at Pooleys and everyone I talked to, “What is BI?” – and this is a tech event!! Anyways, I really do like BI and wanted to move my career forward doing BI. Stratagem hired me to do BI and work on their BI stuff, but there just wasn’t work out there, and I really wasn’t going down the path I wanted to. Stratagem is a great place, great people, I just wasn’t enjoying what I have been working on. (Although I did work on a small part time project for a couple weeks at night doing some SSIS stuff, which was exactly what I wanted to be doing full time!)

Now everyone probably knows I have an iPhone and use it extensively. A couple months ago, I installed the “Career Builder” app from the app store to check it out. It uses your location based on GPS, which I thought was cool. Just for kix, I typed in “Data Warehouse” and there was a result near Madison! Sweet, but what about details? Yeah everything that hit my buttons. Microsoft, SQL Server 2005, Analysis Services, Integration Services, some .NET, etc. Awesome!

I sent in my resume, and figured I would either hear back, and get the job, or wouldn’t hear back at all, not knowing how long the position had been out there, etc. I waited and finally heard back! Sweet. A phone interview, in person interview, informal interview, and another in person interview, and I got the job.

But who is this job with you ask? Well I am proud to say that this upcoming Tuesday I will be the newest “BI Architect” at Trek Bicycle Corp. You know, “Trek”. “Trek Bikes”. The awesome bike company. The HUGE bike company. The type of bikes Lance Armstrong rides. The USA Teams ride. The bikes that have won a ton of Tour De France’s. Sweet, sweet bikes.

Trek is located in Waterloo, WI, which is about 15 miles from where I live now in Madison, and about 14 from where I am moving tomorrow in Sun Prairie, WI. It is about 4 miles from where our band practices, in Marshall, WI. It is out in countryside, on the edge of a small town. Weird how the world HQ for this company is located in such a small town.

In any event, it is a new adventure that I am very excited for. I am really looking forward to get back into BI stuff head first and work back in SQL Server!! (I have been working Oracle for the last 2 months – ugh!)

I will even start riding bikes I’m sure, and I’m betting I am the envy of all my bike nerd friends out in PDX!!!

So this is Big Change #3 out of 3. #1 was moving, #2 is the baby, and #3 is the job. Another whirlwind couple of weeks here, and then the next couple of months, but I am excited and don’t worry, you will see my blogging to continue here. Although I am twittering more ( but I still want to blog a couple of times a week. And hopefully I dive into BI/Data Warehousing and have more cool stuff to blog about in that realm.

One thing, this is the first time since I graduated college where my title isn’t some kind of “Software Dev” title. When I was indie, I really didn’t have a title, even though I was doing BI, so that is different. I don’t see myself going back to being a full time dev, even though I can do .NET. I will still use programming and development as a tool with my BI work, and also just for myself or helping friends.

Go buy a Trek! Save Gas! 😉

Business Intelligence Geeky/Programming SQLServerPedia Syndication

Business Intelligence and Analysis Services in the "Real World"

A reader sent me an email this weekend:

I wonder if I could as your advice as a BI / Data warehousing specialist.I have been studying Analysis Services recently having read a couple of books Which step me through the process of building the cubes etc but as I don’t come From a DB background one thing is not clear to me is how does one determine that They need to use BI / Analysis Services etc in the real world? As you,  I am a .NET developer with a background of building thick client apps and am  Familiar with creating layered architectures etc building on frameworks like NHibernate  Etc to abstract out the DB stuff into my more familiar object world.   My question how does one Generally interface with this Data Warehousing / Analysis Services stuff in the real world? I am looking for answers from people who have used these technologies in anger and not Like me from canned text book examples scenarios etc. Thanks for your time it is would be appreciated.


And I wrote an response, but after reading the response I figured I could post it up here as it is pretty general.

Basically what I have seen is this. You make your application (either web or windows) that saves transactional data, or logs, or something like that. End users usually want reports off of that data. At first, developers will report directly off that data (OLTP database). It becomes slow, and unusable after tons of data is there, so then developers tweak timeouts, things like that. Then the next step is custom summarizations into other tables, and then archiving off transactional data. Usually, because developers don’t know about data warehousing/BI stuff, all this stuff is custom up to this point.


Eventually, someone realizes that, “hey, there is a way to do this better”, which involves a structured ETL either using stored procs or SSIS or something similar. Also, instead of just ad-hoc or custom summarization tables, a well designed data warehouse (OLAP) database is key.
Now, from there, you can write your reports off your OLAP database, which is OK because the data is summarized, scrubbed, etc. But you really give it an adrenaline boost when you create a cube off that data warehouse OLAP db, it takes care of the summarization, the relationships, all that. You put the reporting in the hands of your end users (excel 2007 for example) – let them pivot and slice and dice the data, its all ready and set for them to do it, with really nothing you have to do on your end except make sure the cube is processed regularly off the DW.

You are basically abstracting your OLTP data up two levels… In all reality you should be able to query your OLTP data for a time frame and get say revenue, and then query the OLAP and the CUBE and get the same results. Now, with your cube, you can still access data from it in your .NET apps using AMO, which is cool as well, or you can write Reporting Services reports directly off the cube as well, makes it a lot easier than writing custom .NET reports.
So, interfacing with your Data Warehouse, the best options to get data in is usually SSIS packages or stored procedures, no .NET coding really. To get data out, you would probably want to use Reporting Services, or you can query it with SqlClient in .NET apps like you would a regular OLTP database.
The cube, you can get data out using AMO objects in .NET, you can query using MDX, XMLA, etc in management studio, or you can write reporting services reports, but the best client is Excel 2007, its built to be tied to SSAS 2005, so it really works out well.

One thing about all the technologies (SSIS, SSRS, SSAS, AMO, etc) is that there really isn’t a lot online as far as examples and documentation, or it is scattered. It is getting better though. Most of the stuff I do I just have to figure out on my own, through trial and error, but it is OK.

Maybe not the cover all response but it kind of covers what I have seen in the past, from my own experiences, and others.

Technorati tags: , , , , , , , , , , , , , , , , , ,