The Siren Song of Integration

Integration is one of those elusive things one can hear a lot in our line of work. Many many meetings go something like this:

Person1: “And we are building this simple web site where people register and then you can send them newsletters.”

Person2: “Oh, we already have this Poodle system for online training and then we could integrate users’ profiles with that.”

Person3: “Yes, that’s a great idea! Then we can manage them from one place… and by the way, while we’re at it, let’s integrate this with our network logins, so we can see and manage all information in one place.” 
…

This was a really simple paraphrase of a conversation I heard recently; in many cases, the subject of conversation includes much more voluminous and complex data.

Sounds great from the perspective of someone owning all these different properties or a computer geek dreaming about the Hollywood-style scenarios of “push a button and see it all come out.”

In practice though, integration of this kind is premature optimization and in majority of cases leads either to implementation failure or maximum frustration of users of such services, or both. The situation is especially frustrating in edge scenarios, where part of this integrated whole needs to interface with users outside of the integrated system.

Integration of multiple systems brings additional complexity to manage the integrated parts, and this complexity gets passed on to the user as well, in many cases without the real benefits: different users of an integrated systems only have touch points with one small part of the integrated whole. The likelihood of someone interacting with the whole integrated system rapidly falls as more components are integrated. Component integration is an emergent property: rarely one gets it right in advance as things tend to be used in unintended ways.

We could easily apply this thinking to building an organization’s web site. Usually the group in charge approaches this from the integrated point of view, imposing a common look and feel and some organization that either reflects the organizational structure or some topical structure. Of course, there are always departments who would like to have a different position within the web site, leading to a lot of background political play, frustration, and a general slowdown.

More and more I’m convinced web sites or any multi-stakeholder software applications should be built from very simple independent components and then let the integration opportunities emerge through time. Many times it takes more time to develop and manage integration features than it costs in human effort to manage a little of the overlap separately. (Not to mention that loose coupling is sexy and so web2.0).

Redundancy is not always a bad thing, nor are serendipity and opportunism, all of which are often forgotten in the logical world of computer people. Used properly, all three and can lead to new learning and stronger opportunities for evolution and integration.