Applying Agile to Large Programmes of integrated Projects

steve palmer's picture

Hi

I'm working with an organization at the moment that is trying to apply agile software development principles across large programmes of projects. They are mainly using Scrum with a few ideas borrowed from XP and, on the whole, it is working well for them when applied to individual teams working on a particular application.

The big business win for this organization is not the development of individual applications but the ability to integrate these applications to deliver new products to the market place quickly. Service-oriented architecture and web services are seen as the technical solution to this, and where each application that needs to participates in the solution already exposes suitable services, this works reasonably well.

The challenge comes in coordinating development when a number of applications need enhancements to provide suitable services. The applications themselves can vary from legacy systems, to commercial packages such as Siebel, to bespoke software written in J2EE or on .Net. Each application has its own team that is largely co-located but the teams themselves are distributed across the country and in some cases the team is off-shore on another continent.

The organization would like to reap the benefits of agile development for these types of integration projects. A Scrum of Scrums is the only technique that has been suggested.

I was wondering whether anyone reading this has any experiences they could share of applying FDD style processes and principles at this level.

Thanks and have fun

Steve

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Jeff De Luca's picture

App Delivery Framework

Hi Steve.

The big business win for this organisation is not the development of individual applications but the ability to integrate these applications to deliver new products to the market place quickly. Service-oriented architecture and web services are seen as the technical solution to this, and where each application that needs to participates in the solution already exposes suitable services, this works reasonably well.

That's a good outcome for them. This way of thinking is the real value proposition of SOA - it makes integration a first-class concern.

The challenge comes in coordinating development when a number of applications need enhancements to provide suitable services. The applications themselves can vary from legacy systems, to commercial packages such as Siebel, to bespoke software written in J2EE or on .Net. Each application has its own team that is largely co-located but the teams themselves are distributed across the country and in some cases the team is off-shore on another continent.

The organization would like to reap the benefits of agile development for these types of integration projects. A Scrum of Scrums is the only technique that has been suggested.

I was wondering whether anyone reading this has any experiences they could share of applying FDD style processes and principles at this level.

It sounds like what you want to know about is how to co-ordinate multiple projects (whether they're related or not)? This is a big topic and it brings in many other concerns and I'm not sure from your brief question which concerns you are interested in. So, let's get a conversation going and I'll be glad to give you my thoughts and practices.

There is an organisational/structural framework for the projects across an organisation. This framework then also extends to incorporate things such as Enterprise Architecture, such as strategic alignment of IT and the business (a hot topic these days) and also IT governance (another hot topic these days).

Let's just look at the organisational structure for projects first - this will bring in program management and a piece of the business alignment discussion.

I like to use Lewis' Results Hierarchy for this. There are four levels in the hierarchy and I like to think about them in two pieces - two levels each. The top of the hierarchy is the program. Program management is not new, but like most things in I.T., it's not well understood (translation: often misunderstood and misapplied). Most think of a program as simply a collection of projects. This is blurred even further these days with the addition of the phrase portfolio management (also a collection of projects). Most also try and manage programs like projects - this is a fundamental mistake.

The Lewis Results Hierarchy

So, the top of the hierarchy is the program. A program is a co-ordinated set of actions brought about to achieve a strategic business change. A strategic business change might be improved customer retention, or doing more business with each customer, or faster time to market with new products, or...

A strategic business change doesn't become a program until you've established a set of business outcomes that collectively, will achieve the strategic business change that is the goal of the program. A business outcome is a well-defined change that is evident to people (customers, suppliers, employees, channels, and so on). You could say "users" here, but it is a bit broader than the usual definition of users.

An example of a business outcome might be moving from paper/fax/phone to web services for purchase orders and other supply-chain documents.

So, when you've identified a strategic program the next step is to determine what business outcomes will be needed to achieve the goals of that program so that you can establish an initiative that will achieve each one. Initiatives consist of several co-ordinated projects and they are the next level in the hierarchy.

A project is a collection of tasks that deliver tangible results. A tangible result might be a recommendation for a system vendor, an installed, customised, integrated system, a new process design, and so on.

Projects are then broken up into tasks and sub-tasks that lead to the achievement of milestones where a milestone is simply the occurrence of a recognisable event.

At this level things get a bit interesting. For software construction projects, FDD already gets away from tasks as an activity as opposed to milestones as a product and blends the two things together through the FBS. This is a real breakthrough and why so many other methods are also talking about features and an FBS.

In fact, you could say that FDD adds other levels to the hierarchy. That is a project contains subject areas that contain business activities that contain features. Note though that here, FDD has emphasised the Result column (client valued feature(s)) and de-emphasised the tasks to produce them. This is the fundamental mind-shift from a WBS to an FBS.

Another common mis-application is to not understand the concept of a project that defines the next project, or a project that does not deliver a piece of software (such as a purchase recommendation, or a process improvement, etc.). These are all projects too and should have the same sorts of controls and governance that software projects do.

However, programs are not managed the same way as projects and that is a common mistake. e.g. programs don't have milestones,projects do. Programs don't manage dollars and people the same way projects do.

The project is the batch in an I.T. shop. Projects are the things through which I.T. delivers value.

With this sort of framework in place you can look at any task that is being performed and know exactly how it relates right back to the business. i.e. it's for this project as part of this initiative which delivers this business outcome which is a part of this strategic business change.

I've skipped over some details here so you'll probably already have questions. On top of this you need to add governance and PMOs and etc. i.e. how prioritisation and resource allocation is done.

All I'll add at this point, as an aside, is that other governance structures come into play here, such as steering committees, tech councils, etc. and also groups such as enterprise architecture can participate much more effectively as they are making decisions based on known business capabilities rather than just technology influences.

Also, any attempt to do anything "enterprise" (e.g. integration) requires I.T. be involved up front with the business planning (think of it as the feed to the top level of the results hierarchy). Interestingly the major resistance to change here comes from the business itself; they're just not used to I.T. engaging with them at this stage.

Jeff

steve palmer's picture

Thanks for the in-depth

Thanks for the in-depth reply ... and sorry for delay in reading it thoroughly ... pretty busy at Borland these days.

Most of the above applies.

Immediate question is thankfully a bit lower level.

Integration projects within a particular programme want to reuse several services exposed by other projects within the programme. However, some of these services need modifiying to accommodate the precise requirements of the integration project.

So we have the classic code-ownership issue of team A needs changes to team B and team C's code. Team B and Team C also have other work to do. The teams want to apply agile principles but are finding it hard to scale those up to this level of cross project coordination. Obviously you cannot go collective ownership because team B may be Java and team C stored procedures exposed as web services. Traditional 'submit prioritised change requests and wait until teams b and c to get around to implementing them' feels inadequate some how.

FDD solves the 'implement feature by feature' + 'code ownership' by using dynamically formed feature teams. Is there an equivalent that can be applied at this higher level?

Any other advice on applying agile principles to programmes would also be welcome, of course Smiling

Have fun

Steve

Jeff De Luca's picture

Treat it like any other external dependency

Hi Steve,

I don't see this as a problem specific to Agile but perhaps I'm missing some of the detail.

To me this is just like any normal external project dependency and you would treat it accordingly.

Perhaps SI is an example. Let's say in building a Lending system at a Bank we have a requirement to synch customer data to a centralised customer system (such as a CRM). We have our own work to do, that is UI and PD features for the customer functions in this Lending app, and then we also have SI features to deal with talking to/from this external CRM system. Finally, we have a dependency on this CRM system itself to make some mods to itself to support us.

That's a standard (and typical) project issue. There needs to be agreement reached with the CRM system team on the changes, the design of the interfaces, etc. and the work has to be scheduled in a way that satisfies your projects needs. Where there's contention or disagreement, that's also a standard and typical cross-team issue that is resolved by any number of organisational structures that are organisation specific. But, basically, you as the Lending project PM will escalate the issue for resolution. You have to - after all, your project is dependent on it.

You wouldn't normally bring that external team inside your project (Lending in this example) but you would (or should) track their progress as a part of your projects progress reporting. You will often have a rep from that team logically become part of your project team. It depends on the size and scope of the external work and its impact to yours.

Jeff