Formal UAT thoughts

If you have spent any time with enterprise clients you know there are a number of things that are inherent in the process of rolling out a 3rd party hosted technology solution. These can range from the relatively minor challenges like the numerous approvals required for seemingly mundane items that add to the project calendar to more fundamental challenges, like formal user acceptance testing.

The requirement for formal UAT is one that puts enterprise clients directly at odds with SaaS. If a client insists on running formal UAT for every new release before it goes live this puts a significant burden on the vendor to support this requirement.

A formal UAT process involves creating test plans and test cases and standing up a test environment to perform the actual tests. Even if a client has a vetted set of test cases for regression testing, there will be new features that require creating new test cases.

In theory the 3rd party vendor can simply send over the test cases that have been developed in house to the client, and in fact many clients may ask for such cases. I am personally skeptical of the value here. If the vendor is sending over the test cases and the goal of UAT is to provide a legitimate independent certification that the application is functioning correctly then those two things seems to be at odds with each other, no?

The truth of the matter is that in many instances internal resources at large organizations are overwhelmed with the amount of work they need to accomplish. Having the vendor send over the test cases saves them a significant amount of work.

Creation of test cases aside the process to execute a formal UAT cycle with an enterprise client can easily take 4 - 12 weeks, depending on the size of the project.

If a SaaS organization has a release cycle of once a quarter the UAT effort is one that will never end, and will add to the overhead of any SaaS organization. This means resources in terms of relationship management staff, project management staff as well as possibly other technical resources, and this is just the effort to handle the UAT portion of a migration to a new code base.

This is a potentially significant hidden cost to the client. It is one that is rarely discussed when an organization is considering the total cost of ownership with any SaaS solution.

None of this should be taken as saying that formal UAT is not valid, or has no place for an enterprise organization with a SaaS vendor, but carefully spelling out what the impact of such requirements can / will be moving forward is crucial. The right spot to address this is pre-sales of course, but that rarely happens as this conversation is too detailed for a sales cycle.

Setting some threshold at which a formal UAT would need to occur could also help. For example no formal UAT required for small incremental releases, but for all major releases some version of UAT.

Finally there has to be some level of trust between the enterprise client and vendor. This trust can only come with a history of successful migrations to the latest version of the code that occur without incident. This can only come with time.

SaaS market to collapse in 2 years

I am always willing to read ideas that contradict my own.

Here is an interesting interview with Harry Debes, CEO of Lawson software. I don't think I agree with much of what he is saying, but worth reading.

Time will tell if he is right, and two years go by really quickly.

Chrome and the enterprise

I have run across a number of posts lately that are touting how Chrome will change the browser landscape. This may very well be true, however I just cannot see how any enterprise will really start allowing their employees to this browser officially.

Just a point of reference: most of the clients we deal with on a regular basis are still officially using IE 6 as their corporate browser. Enterprise IT approvals move slowly. As frustrating as this may seem it is for good reason. These departments are responsible for the stability and safety of the corporation's infrastructure (and all of the assest that represents). While moving slowly cuts into the ability for an organization to react quickly it also means that things are stable. If I am in charge of the infrastructure at a large enterprise I will value stability at the cost of agility in most instances.

There a number of posts I have seen lately that are excited about Chrome as it relates to SaaS, try this one for instance. I just can't see how any privacy / compliance manager at any enterprise organization will allow their employees to use Chrome, if for no other reason than all the instances where Chrome is sending browsing information back to Google. Read the privacy policy and the EULA carefully.

Let's not even mention the potential security holes.

I just cannot see how any enterprise will embrace Chrome. I am not the only one who sees things this way. Dennis Howlett at ZDNet has a good post that aligns with our exeperience in the real world here.

upgrades and contracts

I have been spending a fair amount of time lately thinking about contracts.  If you are delivering a SaaS solution to the enterprise there are a number of challenges that exist that aren't there when you are delivering a consumer service. That statement seems obvious enough, right?

The essential challenge is around how to provide protection / assurances to normally risk averse organizations (the Fortune 100 set) around upgrades and yet ensure your organization is not stuck supporting old code bases indefinately.

I started writing a post about some of the nuances, but it quickly became too long for even me to remain intersted.

I think ulitimately the best answer to ensure your client base wants to upgrade. This can only be accomplished by continuing to provide the right features, the right enhancements so that clients are excited about upgrading.

Ultimately if you can't make that happen and you are reliant upon a contract clause to force a client to accept the latest version of the code you have a much larger relationship issue. Those issues aren't going to be fixed by any contract clause.