If you have spent any time with enterprise clients you know there are a number of things that are inherent in the process of rolling out a 3rd party hosted technology solution. These can range from the relatively minor challenges like the numerous approvals required for seemingly mundane items that add to the project calendar to more fundamental challenges, like formal user acceptance testing.
The requirement for formal UAT is one that puts enterprise clients directly at odds with SaaS. If a client insists on running formal UAT for every new release before it goes live this puts a significant burden on the vendor to support this requirement.
A formal UAT process involves creating test plans and test cases and standing up a test environment to perform the actual tests. Even if a client has a vetted set of test cases for regression testing, there will be new features that require creating new test cases.
In theory the 3rd party vendor can simply send over the test cases that have been developed in house to the client, and in fact many clients may ask for such cases. I am personally skeptical of the value here. If the vendor is sending over the test cases and the goal of UAT is to provide a legitimate independent certification that the application is functioning correctly then those two things seems to be at odds with each other, no?
The truth of the matter is that in many instances internal resources at large organizations are overwhelmed with the amount of work they need to accomplish. Having the vendor send over the test cases saves them a significant amount of work.
Creation of test cases aside the process to execute a formal UAT cycle with an enterprise client can easily take 4 - 12 weeks, depending on the size of the project.
If a SaaS organization has a release cycle of once a quarter the UAT effort is one that will never end, and will add to the overhead of any SaaS organization. This means resources in terms of relationship management staff, project management staff as well as possibly other technical resources, and this is just the effort to handle the UAT portion of a migration to a new code base.
This is a potentially significant hidden cost to the client. It is one that is rarely discussed when an organization is considering the total cost of ownership with any SaaS solution.
None of this should be taken as saying that formal UAT is not valid, or has no place for an enterprise organization with a SaaS vendor, but carefully spelling out what the impact of such requirements can / will be moving forward is crucial. The right spot to address this is pre-sales of course, but that rarely happens as this conversation is too detailed for a sales cycle.
Setting some threshold at which a formal UAT would need to occur could also help. For example no formal UAT required for small incremental releases, but for all major releases some version of UAT.
Finally there has to be some level of trust between the enterprise client and vendor. This trust can only come with a history of successful migrations to the latest version of the code that occur without incident. This can only come with time.
Subscribe to:
Post Comments (Atom)
1 comment:
UAT sometimes also suggests that the feedback will be able to influence product design, as it does for off-the-shelf software. That isn't in play at all for a SaaS firm, and has to be clearly addressed at the onset of any engagement.
Post a Comment