Archive for the ‘agility’ Category

A Dashboard for Pipeline Integrity

December 20, 2010 Leave a comment

A dashboard for pipeline integrity to aid in compliance with US rules for integrity management

“I want to log-on to my computer and be told the status of the system. Today, I have to ask a specific engineer that looks after the area I want to know about who says he will get back to me. Two days later I get an answer, usually with caveats because he doesn’t have all the up to date information.”

anonymous pipeline manager


  • Problem: a need for situational awareness and process control
  • Integrated data for situational awareness and process control
  • A pipeline information management system (PIMS)
  • Building an integrity dashboard on your existing IT infrastructure


Problem: a need for situational awareness

On March 31, 2009 the PODS organization hosted a meeting in Sugar Land, Texas called Mapping Our Pipeline Future.  Seventeen pipeline operating companies participated.  The intent of this meeting was to identify and prioritize operational needs that are not being met by the PODS committee or by vendors.

At this meeting four pipeline operators made short presentations describing their integration efforts to date.  Afterwards the meeting participants were divided into five groups for general discussion followed by specific discussion groups after lunch.

A prevalent thread at this meeting concerned leveraging integrated data.  Some of the wish list items generated along this vein include:

  • “Improved situational awareness and decision making” – Anadarko Midstream, operator presentation
  • “Data should be made to work for the [user] through more empowered processes” – general discussion groups, room 2
  • “Missing is a way to tie/connect business processes to the tasks” – specific discussion groups, key issues facing operators

In general these requests to leverage integrated data fell into one of two categories, requests for improved visibility and requests for better processes.  These two categories can be summarized as follows:

  1. Situational awareness – the need to know what is happening, what has happened, or what will happen
  2. Better processes – the need to orchestrate individual tasks once situational awareness has been achieved

This paper will present a lightweight approach for building pipeline dashboards for key players in the integrity process.  This approach leverages your data, regardless of where it resides, to achieve situational awareness.  This situational awareness in turn allows for better processes.

This approach is independent of your integration platform, PODS or other.  It leverages existing investments in IT infrastructure and gets stronger with each additional  investment.  What is needed to make this solution available to the industry is a set of vendor-neutral standards describing interfaces for data sensors and process actuators.

Integrated data for situational awareness and process control

The illustration below was taken from one of the operator presentations at the PODS mapping our pipeline future meeting.

This image serves to illustrate the hope that many operators have placed in integrated data.  Integrated data is the hub around which the various departments involved in integrity management interact.  With PODS in place, everything should go smoothly.

Ironically, integrated data can actually make situational awareness and control worse, not better.  The need for more people and more procedures means that there are more opportunities for things to go wrong.  Critical conditions can hide within a flood of “normal” data.  Instead of seeing an improvement, many operators are finding that their data is bad, their processes are broken, and integrated data is expensive to maintain.

Integrated data alone can never provide the awareness and control that is needed.  PODS and other data standards simply describe how to store and exchange information.  Data standards are concerned with data, not people or processes.  They do not describe how data is gathered, verified, and inserted into the database.  They only describe where to put it and what format to use.

While integrated data is necessary for better situational awareness and control, integrated data is just a first step.  The illustration above would be more accurate if the central puzzle piece showed a software application instead of a database like PODS.

Software which orchestrates the data, people, processes, and events of integrity management is the next step.  A dashboard which lets you see a summary of the system and to drill into and examine details from a single location would be a big help.  If this dashboard enabled you to respond to this summary information and to alter the process you would have the situational awareness and process control you seek.

A pipeline information management system (PIMS)

Both ASME B31.8S and the integrity rules hint at a global information system which integrates the people, processes, and data associated with maintaining a pipeline.  The intent of this theoretical system is to provide the integrity manager with the situational awareness and control required to orchestrate the integrity process.

These software systems are called pipeline information management systems (PIMS). In a PIMS the integrity manager…

  • has global visibility into past, present, and future activity on every asset.
  • is able to quickly define, deploy, and modify procedures for compliance.
  • sets up alerts which trigger action when warranted.

On its face a PIMS is a dashboard application for the integrity process. In the business world, “the dashboard is the CEO’s killer app, making the gritty details of a business that are often buried deep within a large organization accessible at a glance to senior executives. So powerful are the programs that they’re beginning to change the nature of management, from an intuitive art into more of a science.”

These same benefits apply to asset management and to integrity compliance.

A dashboard application is a console into which gauges and controls that are of interest to an individual user are chosen from a library of prebuilt tools.  These tools are added to individual dashboards and are configured based on individual responsibility.  We’ll call these tools “widgets”.

Each widget needs software to make it work.  To build a widget, a software developer combines the capabilities of software “sensors” and “actuators”.  Sensors monitor data, dates, and events.  Actuators enable changes to a procedure or to data.  This simple concept can provide tremendous value for relatively low cost.  As you improve your underlying IT infrastructure you are able to build better and more useful widgets.

A dashboard doesn’t replace existing IT investments, it leverages them.  The better your underlying IT infrastructure the stronger your dashboard system can be.  If your data is integrated, you can create sensors which monitor this central location.  If you install messaging between your asset data and your work management system, you can take advantage of this new infrastructure to track, or even alter work once it has been scheduled.  If you install a workflow system, you can use your dashboard as an access point to adjust individual task steps as needed.

There is a non-linear return on your investment.  Invest in one widget and you realize a benefit of 1.  Two widgets give you a benefit of 4.  Invest in exactly the right number, and the benefits are immeasurable.  By putting the right information into the right hands a dashboard can have a tremendous impact on reducing operating costs.

Building an integrity dashboard for domestic use

While dashboards for pipeline integrity exist, they are usually custom affairs.  The few commercial solutions available are targeted at international operators who are not affected by the PHMSA rules.  What is needed is an affordable, off the shelf system that has been designed specifically for compliance with the US integrity rules.  What this looks like is a set of dashboard widgets which are generally useful to US pipeline operators and which can be deployed into common IT environments.

Off the shelf solutions are available from other industries and can be adapted to meet our needs. Since the 1970s banks and other information-intensive industries have been integrating their computer systems and processes in a quest for better situational awareness and process control.  They have tried and studied workflow systems, business process re-engineering, and dozens of other techniques.  It is not a simple problem, but recent efforts have been very successful.

Gartner Research says that the most recent round of developments, business process management systems (BPMS), “gives companies visibility into processes that are key to cost management.  BPM is a lifeline in this troubled economy.  It helps companies find and avoid hidden costs – to keep companies in business.”

While this sounds like a major rework of your IT infrastructure, it isn’t.  BPMS encompasses most of the common solutions that get your IT folks excited – integrated data, workflow management, SOA, web 2.0 and many, many more.  The better your IT infrastructure, the better your dashboard can be.

What your IT team needs is some guidance regarding where to make their next major investment.  To do this it would be helpful if you could tell them the kinds of widgets that would provide you with the most benefit in terms of your own situational awareness and control.  Some examples to consider:

  • on-the-fly alignment sheets that follow map view
  • Live flowcharts which enable you to intervene in tasks like data scrubbing
  • project builder/selector or a project budgeter/scheduler

Sure you need lots of help, but where can you get the most help for the least money? To explore this problem, I would like to set up some formal research. After some initial comparison research between pipeline processes and business processes we would develop a series of test software installations with the objective of determining:

  • a protocol for measuring the benefits of dashboard widgets
  • a series of benchmark measurements for return on investment
  • a baseline set of widgets arranged from largest to smallest benefit and from most cost to least cost


BPM – The Adoption In The Financial Industry Versus Early Expectations

December 16, 2009 Leave a comment

Very nice article which describes adoption of technology in banking sector.  It suggests the road to nirvana is bumpy.  Very well written.  No sources.

Bpm – The Adoption In The Financial Industry Versus Early Expectations

Categories: agility Tags: , ,

Selling an agile approach…

December 1, 2009 Leave a comment


I came across this question at Discussion: Agile | LinkedIn (account required).  In the original posting Myroslava Trotsyuk referenced an article (How Agile Practices Reduce Requirements Risks) by Ellen Gottesdiener which describes 6 project risks that are mitigated by agile practices.  In response Lanis Brett Ossman raised the question:

Why is it difficult to sell an agile approach when delivering software solutions?

In explaining the question, Lanis Brett created his own list of 6 objections to adopting an agile approach.  While he labeled them as “risks”, I have encountered these same objections when trying to answer the question “why agile?”

– Risk 1: For the “major” cost, customers expect developers to figure out the details. They expect developers to know their business, for the money they are paying.

– Risk 2: Added cost for paying employees to review progress rather than do their regular job. Again, why can’t the developer nail down the details up front?

– Risk 3: Developers typically eat the added cost of poor impact analysis. Not the customer’s fault.

– Risk 4: Many developers just allow scope creep, so the customer does it expecting no impact to them. Creep is often small changes, but they add up fast.

– Risk 5: Defective requirements pretty much involve the same customer issues as Risk 1.

– Risk 6: Why should I, the customer, pay for your learning curve implementing new tools and techniques?

Nice list of risks/objections.  The original article was compelling, but outside of project risk it doesn’t address other objections to an agile approach.  When the customer comes back with “purchasing insists we know all the costs up front”  or “It is the rules, sorry” you often have no choice but comply – taking the cost-overrun bullet yourself instead of letting it hit the customer.  At this point, you have to overcome the purchasing or rules objection.  There is no point in pleading project risk anymore.

As the original article stated, software is different than other types of products.  In the physical world project estimation and sales are relatively straightforward.  Not so in the software world where our deliverables are unbounded.  We are not even limited by our materials or tools.  We can create tools as needed.  If we need a new fundamental data structure, or other “materials” we can create those too.

Until it is fully delivered, software only exists in the imagination of those requesting it and those building it.  There is plenty of room for misunderstanding and error.  This is the problem Agility was created to solve.  For it to work, though, you need to sell something that fundamentally looks agile.

Are you selling an agile project or a traditional one in which you hope to apply agile techniques?

Case study:
If I am an oil company ordering a drilling platform, I expect to specify its dimensions, drilling depth, and other requirements for the finished unit.  I expect the contractor to take my finished specifications, then working from a wireframe model, I expect the contractor to come up with a list of materials and standard times for attaching those materials.  The sum of the parts rolls up to a final cost estimate.  This is monolithic.

Similarly, as an oil company asking for “asset management software” I expect the vendor to pull pre-built elements off the shelf and attach them, like welding, to produce a final product that meets my specifications.  When I hear that I need to participate in discussions about every rivet, bolt, and plate it doesn’t make sense.  Why can’t you just pull 5 compressors and some 2″ steel plate off the shelf and put something together?

Points of failure:
Physical world –
Despite its enormous complexity, a drilling platform is made up of only a few hundred basic elements – steel plate, bolts, pumps, steel tube.  During construction these basic parts are used repeatedly. By modifying or “fabricating” these elements into the proper shape, they can be attached to an adjacent part. The complexity of this originates from the volume of repeating elements.

Software world –
Software is different. If a basic element is repeated, it is a sign of poor programming. If a software program has 100 different parts, every one of them should be unique and independently useful in shaping the final result.  Our raw materials – software classes – need to each function flawlessly to receive information, act on that information, and pass that information onto another class.  If a single element malfunctions it can affect the entire application.  Complexity arises from not only the number of assembled parts, but also from the fact that each one of these parts is individually unique.

Bringing it back around:
Our customers are accustomed to purchasing monolithic items in which the number of fundamental parts is small.  In theory if we could break our offering into elements, each of which is no more complicated than a drilling platform, we would reduce our software risks to the (embarrassingly) relatively low risk of building a drilling platform.

To achieve this we need to get our offerings (what we sell) small enough where the functionality is represented by a few hundred classes at the most.  By keeping what we sell small, we lower everyone’s risk – ours and our customers’.  We can build up complex solutions from these smaller elements.

I came across the following article Still playing planning poker? Stop gambling with your project budget. – About Agility which addresses the question of estimating an agile project.  In it Robert Neri describes an approach to using user stories to estimate the project.  Admittedly a user story is too fine grained to be useful in a sales situation.  We need something a bit bigger, but not too big.

What does this look like?
Try breaking your offering into smaller lego sized chunks of value.  Instead of selling “asset management” try selling solutions to individual problems within this domain.  The trick is finding a level of usefulness that we can sell.  I can’t sell a login screen, but I can sell a “remaining life monitor”.  I can’t take the risk of selling a “control panel”, but I can handle selling a “panel shell” and some “snap-ins”.

In the way that a user story breaks functionality into something we can build, we need a way to break a project into valuable chunks we can sell.  Right now I am working with a concept I call “critical paths to value”.  Instead of proposing to build the entire offshore rig, I’ll start with proposing we build a boat that could hold a drilling platform.  A floating platform is the first critical path to value.  The next would be the drilling rig, followed by a control room, and next maybe a transfer point for offloading oil.

So where are the critical paths to value in my own application?  I am working on it.  I’ll post an update as things unfold.