Agile Business Intelligence

Business Intelligence – 3 Years of Agile

Last year at the PASS summit, I ran into someone and was discussing project management and Business Intelligence. They were so adamant that agile couldn’t work. And I had to correct them, as I have been doing agile now for three years at Trek in our BI group.

Yes, some things don’t exactly parallel to software development, but many things do. Sprints, Standups, estimations, stories, points, scrum master, releasing/delivering value on your iterations. And now even more with process like unit testing of SQL code and more, things are getting closer to software development in that regard.

I have an entire blog category dedicated to agile – more concepts but also talking about Business Intelligence teams in some of them.

Just remember, you can do BI and Agile, it works, and you can deliver a ton of value to the organization. Someone who might argue with you, well, they don’t know what they are doing in BI or in Agile, or the organization isn’t willing to change, which of course then it wouldn’t work.

Advice. Make sure you hold retrospectives, and make sure you make adjustments from whatever comes out of them.


Agile: Group Technical Review Worth It?

Every since I started practicing agile development and processes, I have lobbied for some type of group tech review before release. At the end of the sprint, the tech team (devs) get together and everyone can go over what they did, give some insight to what they did and why and everyone else can get a glimpse into the other work that was done. Tech reviews aren’t meant to beat anyone down, but they shouldn’t waste everyones time either. I think developers need to get that interaction as well as learning how to show what they did to their peers. Thoughts?

For the most part this idea of a sprintly tech review has worked well, but in others it breaks down.

Why? For one it takes time. If you are so crunched at the end of a sprint, then who has time to do a tech review? In some teams, this hasn’t been an issue, maybe it was better planning or estimating, or maybe just dumb luck, but doing the review made it into the sprint.

Maybe on some teams doing mini reviews with the group throughout the sprint instead of all at the end is the way to go. Maybe not doing any review at all.

My head is still with doing a group review at the end. I think the estimation and wrapping up at the end of a sprint probably would need to get looked at from a process perspective. What can be automated? I’m sure more than maybe you are doing now, etc.

I for one welcome new ways of doing things, but more efficient and still get the points across that need to be. Bring on the ideas – let me know your thoughts in the comments.


Agile: Epics

So most people are familiar with “projects”. Something you do, have an end goal, usually a budget, resources, plan, etc. Sometimes you have projects within projects. Like a good example may be thinking of the next version of Microsoft Windows as the main project and revamping solitaire for the next version as the “epic”. Your end goal is to keep your main project moving forward, and do stories that apply to that project, but you also might run into mini projects along the way that could cross over numerous sprints, and you want to “group” like/similar stories together to form an “epic”.

What is nice about epics is they still let you do your sprints, and stories as normal,but you have a more higher reaching goal to get a set of stories complete to complete your epic. It also lets you look back and then see how many stories and points you did toward that epic and understand the effort,etc that went to completing it.

Use epics to group stories together to form a common goal for a similar “thing”. It will let you say “let’s focus on the blah epic” and you can make sure you get that piece of functionality done and track it and see all those stories together in the end.

P.S. Blogged from my iPad


Agile: Roles – Scrum Master

Funny title, of course. Everyone with no idea about Agile/Scrum always laughs when they hear the title. Scrum Master (or is it Scrummaster?) – What is that? What a goofy title?

Yeah, but it is what it is. You are probably trying to run some kind of Agile/Scrum process, and you need then a Scrum Master 🙂

But what does a Scrum Master do? Well a lot of it might depend on your process and team but here are a few things (there are probably a hundred more)..

  • Facilitate the Daily Standup/Scrum
    Each day when you meet for your 15 minute standup, the Scrum Master should make things start on time, and keep flowing, and make sure people are answering the 3 questions (What I do today? yesterday? What is in my way?). The Scrum Master should cut off people from going long, make sure things get tabled or moved to a hallway discussion instead of taking of everyones time.
  • Make sure the Burndown/Velocity is being tracked
    The charts! Of course, someone needs to make sure the metrics are being tracked and also displayed. Now depending on your team, it might just mean .. well, nothing but making sure do’ers are updating their burndown correctly. You might have virtual charts. But in some teams, you might need to print the charts for the board, or you might even need to add the burndown yourself depending on your process/system. Keeping track of these metrics is key. You want to keep people informed of your progress every day (at least for the burndown – I like to track velocity as a bullet chart for the sprint and update each day as well).
  • Get all the Stories Ready for Planning
    In some teams, the scrum master might be the only person doing this, in others there may be a group of analysts, etc. But the Scrum Master should be sure to have the stories ready for planning, to score and discuss. Your team probably has a backlog of bugs, features, enhancements, technical debt, and it should be prioritized, but the Scrum Master should be where the buck stops to make sure everything is in order for planning.
  • Facilitate Sprint Planning
    Just like facilitating the daily scrums, the Scrum Master should facilitate the “Sprintly” Sprint Planning meetings. Review, Retrospective, Scoring. Keeping things moving, being an observer but not really a decision maker – that is for the team to do.
  • Be the “Blocker” From Outside Distractions
    Once you start doing Agile, things change, sometimes they have to change dramatically. If your team of do’ers is 3rd level support, or even 1st level support, they are going to need a buffer or blocker from outside distractions. People wanting things now! and support calls and walk ups, and everything else you can think of. The Scrum Master should and needs to learn how to say “No” to everyone when needed. “No, we can’t do that now, but we can look at doing it next Sprint” or.. “No, we can’t look at this right away, but if we get ahead, we can pull it in.. maybe”, or “No, we (just flat out) aren’t doing this”
  • Work with Product Owner on Story Prioritization
    The Scrum Master needs to work with the Product Owner (another Role for another blog post) to get the stories in order, and prioritized. This should happen sometime a few days or week before the Sprint Planning so things can get organized without being rushed.
  • Drive the Process, Improve the Process
    As with everyone involved, the Scrum Master should try to improve the process along the way. There are always way to do things differently which might lead to things running smoother. Change the day of the planning? Print cards? Auto burndown? etc, etc. The Scrum Master should work with the team and Product Owner to help continuously improve the entire process, and the Scrum Master should be the driver of the current process. People have questions on the process? They should ask the Scrum Master.

As you can see, those are just a few of the things the Scrum Master needs to do to help the Agile/Scrum process along. Many times, teams don’t have someone that will take the bull by the horns and drive the process, step up to the role. Other times it is just apparent who the person should be, and in others, it is a well defined role.

The Scrum Master’s role is very important in the process, as much as the do’ers and the Product Owner and others..But they should be the ones that *own* the Agile process and help move things forward.


Agile: Scoring

Once you have your stories defined and you are ready to “score” them, there are some things you need to keep in mind.

What scale are you going to use for scoring?

I use 1,2,3,5,8,11, and 13. Why? Well, that is just what we used when I started doing Agile, and it seems to work 🙂 Fibonacci + 11. I know I probably should change it so it kind of goes along with the more mainstream numbers that some 3rd party tools and other people recommend. Maybe, someday 🙂

1 is the lowest, 13 the highest.

At first, the team has no idea what a 1 or 2 or 8 or anything is. It takes time. Over time the team gets a gut feel for what a 1 is, or a 3, or whatever.

Don’t equate points to time! No, a 1 isn’t an hour, or a day, or X. It is a 1.

What are points?

Well, I try to explain it as time, effort, complexity, outside factorness, craziness, etc, etc. You kind of have to bundle all those things up and say, yea, that is a 2. It is more than a 1, less than a 3. Each team is going to have a different meaning for what a 3 is.

During your sprint planning, what you want to do is give everybody a pack of “planning poker” cards so they can score as you go through stories. (There are also web apps for virtual teams, iPhone apps instead of using planning poker cards). As you get done reading the story, and answering any questions, the team should “score” – show their cards. If they are all the same – great! You have a winner.

If some are different, then you discuss or come to an agreement on why some do’ers think it should be higher or lower, and get a score at the end. Example? 6 devs, 4 say it is a 2, 1 says a 3 and 1 says 1, well, most say it is 2, do the other 2 devs agree? if so, you can mark it a 2. If not, you might discuss why. The one dev saying it is a 3 might say, “what about this an that” and then everyone realizes, oh yeah, it is more than 2, and you go with a 3.

What happens if you see 13’s?

What I try to do is say – if you score it a 13, then the story is too big, you should break it up into smaller stories. With that logic, you should never have a 13 on your board.

The biggest thing to remember is that the team is going to have to take a few sprints to get used to the scoring, and understand how stories fit into points. You ARE going to over and under score – it is just reality. But don’t adjust the scores during the sprint. Things just tend to work themselves out over time. What you can do is capture actuals at your retrospective and compare to what the original scores were. The team might get a better idea on what they should be scoring things then. Another thing to try is a scoring guide. After some time, you can say “every story we do that involves making a new button on a form is usually a 2” so then if you have a story like that, and the team scores it a 8, or they don’t know what to score it, you can look back at your guide and say -hey, we usually score these 2’s, so is a 2 ok?

Who scores?

This is a good question, who should score? My opinion? Just the do’ers. The scrum master shouldn’t score, and product owner/analysts shouldn’t score. Just the do’ers.

What do you do with the points/scores?

You track your daily burndown, and your “velocity” per sprint. More on that later.

Just remember, think of stories as being scored by points. Or bananas. Or stars. Just not hours 🙂


Agile: Daily Standup, Daily Scrum

I hate meetings. Really, I do. They are toxic.

But one of the things your team needs to do is meet daily. With agile, it’s called the daily standup or daily scrum meeting. But what is it?

  • Every Day, 15 minutes
    The meeting needs to be every day. Every day (Monday-Friday). No Exceptions. No days off, no skipping, no excuses. Schedule it every day. 15 minutes. Try to get it in the same place
  • Stand up
    The reason it is called a “standup” meeting is that you need to actually stand up. Why? Because sitting down makes people take their time. Standing up makes people move faster, because they don’t want to stand for 60 minutes.
  • Three Questions
    What I did yesterday? What am I doing today? What’s in my way? Simple enough
  • Not Everyone
    You want your do’ers there, you want the scrum master and product owner, maybe some others, but not everyone needs to talk. Go around the room and the do’ers should give an update, others should just listen.
  • Details, but not too much
    Do’ers, when giving updates, should let team know what stories they are working on. If you use a virtual+physical board, you will have story #’s. Don’t just spout the numbers. Spout the numbers and the story title, and where you are at, maybe even burndown if you track that manually.
  • Keep on Track
    easy to get off track. Even with standing and whatever. Scrum master should make sure things keep moving. In, update, out, go. 15 minutes? Try to get done in 10.

Should you meet on your sprint planning day? I say yes.

What else does the daily standup bring us? Communication. Everyone should know what everyone else is working on, where things are at. People can’t hide. They need to give updates, they probably should have a good reason or be working on something else off sprint if they have no update.

What else does it bring? In my eyes: Discipline. Making sure you know what you did, what you are doing, and always meeting at the same time, and doing the same thing every day brings a little discipline to the team. Keeping things on track, 15 minutes, cruising through updates, instead of meeting speak and 60 minutes of babbling. It does good for everyone. It lets your do’ers get on with their day so they can keep burning down points!


Agile: The Story Board

The story board…

If you have used Agile for Project Management or Software Development, and used Scrum, you probably have heard of the “Story Board”.

Trek BI Agile Story Board

Funny thing though, I have talked to a few people that have came through and they saw the boards we have and it was “oohh I have never seen one in real life”.. but yet they have run or been involved with scrum before.

So what is the deal with the “board”?

  • The Board is a focal/gathering point
    Every day at your standup/scrums, you should bring the board in and use it as a tool. Also at the sprint planning. Think of it as another team member. Talk to it. Talk in front of it, analyze it, adjust it, show it off, explain things on it to people
  • The Board shows tangible evidence of results
    You should hang your velocity and burndown charts on the board, you should have a nice full current and next sprint that changes every day as people do things and new items get prioritized. Your backlog should be filling up. Communicate your iteration dates and maybe other key info, but most important, if someone walks up, they can see your burndown.
  • The Board is a talking point
    People around the office start asking “What is that thing?” People in other teams want to know what is up. Other people might come up and ask where things are at, or where their “story” might be. You can use the board to start discussions on agile and also about your project
  • The Board let’s your focus
    As a developer or “do’er” for your project, you can focus on just the current sprint. You can focus on the cards on the board. You see slightly what might be coming in next sprint, but you are focused. As a product owner or team lead, you can see progress and focus on the current 2 weeks.
  • The Board makes it tough for distributed teams
    If your team has some members off site part of whole of the time, then it is harder for them to actually “use” the board. They need a virtual board as well. They need virtual stories or some way to update progress.
  • The Board adds a little overhead for Scrum Masters
    With a online system or virtual system, the Scrum master can update things there, and so can the developers. With a board, and especially with a board + virtual system, the scrum master has to make sure things are in sync. What should they focus on? The board or the virtual system? Both. But since the board is visible to EVERYONE who might walk by, they need to make sure it is up to date and in working order.

My thoughts: The board *should* exist, you *should* use a board.

But, what if you run distributed teams? Then.. if majority of them are not on site, then use a virtual board. If it is a mix, use a physical and virtual board.

But what if we don’t have room for a board? Then make room.

But a board costs money. Not really. I made one out of things lying around the office. Minimal cost. You can expense story cards, etc.

But xyz.. well, I am sure there is an answer.

If anything, start with a board, and cards (another blog post on that) for at least one year, and then look at going virtual, *if it makes sense*. I still think the use of the physical board is a key part of the process, but like with anything Agile, the process always differs by team and project.

You can always roll the board somewhere to demo. You can show the board to anyone. Any worker, any executive, a group of many people, or small. You might not be able to do that with a virtual system: you might be limited by number of accounts or other system limitations.

The board might be a “relic” of agile, but I think it lends itself to running the process in a way many can understand, without having to learn any software, and also lets your team communicate progress and results easily. Use a board? Just do it.

Business Intelligence Work

Agile in Business Intelligence? Of Course!

About 3.5 years ago I was introduced to Agile at the Agile 2006 Conference. After that, and implementing it in a software dev environment, I found that it just works. Sprints, Scrums, Stories, Backlog, Velocity, all the pieces fit and work.

Now that I am managing a Business Intelligence group, which when I started wasn’t doing *anything* as far as a method, I had to ask myself if doing Agile would work (I knew it could, but it is different than software dev in many ways, similar in others).

Back in October, my group went Agile. We set up a board, got some index cards, and just started Agile. A big paradigm shift at work for IT, but we needed to do something.

With Business Intelligence, we really don’t have *code* to work on, but more “objects” (Cubes, Dimensions, Reports, etc). As a team we needed to figure out – what is a story? What is a feature/enhancement/task. What is an epic? How are we going to score things, etc.

The first few sprints (2 week sprints – Wednesday’s to Tuesdays) our velocity was lower and/or we just scored things a little weird. But since then we have learned our “zone” of scoring stories and we got into a groove of releasing our BIG cube every 2 weeks, and releasing the smaller changes when completed.

We do the daily *scrum* for 15 minutes, and track burndown on stories, which lets us make some cool burndown charts that we tack up on our board, and we have some other cool bullet charts to track velocity by sprint, to our original, and final goal, and more.

What has Agile brought our group? Confidence, Stability, Ability to Meet Expectations. Agility. Results. and more..

Do we run into issues yet? Of course. Can we adjust and handle them. You bet! Are we continuously learning and changing our process to make it better? Yep. Always room for improvements.

What else does Agile bring us? Visibility to our customers, and to our peers in IT. Eventually the “BI” stuff should just run, over and over, iteratively.

Trek BI Agile Story Board

Agile isn’t a silver bullet though. It isn’t easy. You still need to work to keep things organized and on track. You have to fight that organizational gravity that sucks teams back in and people in as well, and throws that scope creep back onto stories and projects. You also have to fight to get rid of your technical debt, which depending on how long things have been running before you started even thinking about Agile, might take you a while.

This post is more of a high level “Yes We Can” type post about Agile in BI. I haven’t decided yet, but my guess is I might have some more detailed posts on how I like to run an Agile project, and what we are doing as a team to handle situations that come up, and just how we do things.

In the end though, just remember, have fun!