preload

Alternative Facts. “Happy path” and “edge cases”.

Posted by Albert Gareev on May 02, 2017 | Categories: DiscussionsNotes

Continuing my thoughts on alternative facts in testing and quality assurance.

Happy Path?

Guilty as charged. I, too, used to say “happy path testing” and “happy path coverage”. What I meant was something like.. eh..

RatMaze2
“a typical user entering typical data in a typical way”.

Unfortunately (or fortunately, depends on how to look at it) I did not realize what it actually means.

“Happy Path” User..

  • Perfectly understands the domain and functionality
  • Is perfectly familiar with the UI
  • Operates in the way accounted for by the software
  • Perfectly remembers everything
  • Perfectly understand prompts and responses made by the product (even when there are none)
  • Never makes mistakes
  • Never gets distracted
  • Only enters the data that were proven to be correct

..plus “Happy Path” Product..

  • Is used on the very particular infrastructure and device, guaranteed to be working in the very particular way (which was proven to be working)
  • Does not have any variation or fluctuation
  • Can be understood only in one, perfectly correct way

..created by “Happy Path” Team..

  • Perfectly understood business requirements and the purpose
  • Designed and implemented them in that one, the only correct way

..equals One Big Edge Case

The point is, until we sufficiently learned about the users, the product, and the environment, we have no idea what usage pattern is mainstream and what would be the edge cases. So let me give you a few examples.

Story One

Training instructor points at “Save” button and says “click Save”. Twenty users click “Save” on their laptops. Twenty users get a timeout error and locked sessions. Pre-sales training goes sour.

Load test session involving simultaneous HTTP requests from twenty users would be a happy path scenario in this context, even though the product didn’t need to support more than a hundred of users working concurrently.

Story Two

In a system recording legal documents – court decisions – every letter and character, including punctuation and extra spaces, must be preserved as in the original, or else the document can be successfully challenged in court. To shock of the product owner, it was discovered that some electronic records were missing certain data. After an extensive and expensive investigation certain cases were isolated. It appeared, that “less than” sometimes was shortened to “<“. Unfortunately, the code treated “<” as opened and not properly closed xml tags, truncating rightmost part entirely.

Testing for special characters and html injections in this context would be a happy path scenario, even though operators of this non-public product were skilled legal clerks who were never expected to intentionally misuse the product.

Story Three

A husband and wife order a checkbook. Their names and address are supposed to be printed on the checks. As in “Mr. & Mrs. Smith.” Except that when they receive the checkbooks they see this.

Testing for [at least some] special characters in html and pdf in this context would be a happy path scenario, because “&” is a common notation nowadays, but it’s automatically replaced by “&amp” unless this case is properly handled.

Story Four

A data conversion from a 64-bit floating point number to a 16-bit signed integer value caused overflow and a hardware exception. As a result, the spacecraft destabilized and disintegrated.

As explained in Wikipedia, Ariane 5 design was based on previously successful Ariane 4. “In Ariane 4 flights using the same type of inertial reference system there had been no such failure because the trajectory during the first 40 seconds of flight is such that the particular variable related to horizontal velocity cannot reach, with an adequate operational margin, a value beyond the limit present in the software.”

Redefining what is a “happy path” and adequate testing would be the strategy to discover the problem before it so spectacularly occurred in production.

Conclusion?

Metaphors might be useful. But they may also induce confusion, miscommunication, and reification fallacy. People may and will use different terminology, and it’s not for the testers to police the use of expressions.

When it comes to “happy path”, always explore what that means in the context of your product and users.

When it comes to “edge cases”, consider both the likelihood and impact. High potential damage means high risk even with a low likelihood.

Your real users may not follow your imaginary happy path!

RatMaze1
Science: experiments only work if your subjects follow the rules


Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported
This work by Albert Gareev is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported.