preload

Programs don’t care about quality, companies should

Posted by Albert Gareev on Dec 09, 2010 | Categories: Notes

Quality is value to some person.
Gerald Weinberg

The story began when I received a new chequebook.

An image of a cheque with a bug

Every bank, especially any of top 5 banks [in Canada], has a lot of claims about how much quality matters to them, and how much they care about their customers. Since I was not satisfied by a quality of service I decided to discuss it with my bank.

Let me digress a little here. Quite often, when spoken about an abstract subject, every side has its own meaning for it, and that becomes a ground for misunderstanding. When one side is a person and another is an organization, that makes it even more complicated. I suppose, sides at least should carefully listen to each other, understand each others position, and then work towards a mutually comfortable definition.

So I wanted to state the problem and explain why it’s important to me.
With a telephone banking, that means you better be ready to tell your story a few times while they transfer your call from one person to another. Also be ready to get annoyed by promotions / info-gathering questions got thrown upon you.

Finally, I reached the person who was comfortable to take on the talk about the problem (i.e. “customer complaint”, as they called it).

What I heard.

1) “That’s just a typo”. (No-o, it’s a bug in your program!)
2) Cheques /should be/ acceptable. (Note the claim that obviously can’t be assured)
3) I may order another cheque book with a discount. (Now, bad service means you should buy more of it?)

So, I didn’t like the answers. I explained that as following.

1) To me, typo in anything related to my name is not just a typo. Furthermore, I know it’s a defect in software. And to me, as a customer, it’s not good quality of service.
2) Maybe cheques should be accepted. But there’s a chance they may be not. And I don’t want to guess about it every time I make a payment. Your opinion that they should be accepted won’t help me resolving issues if it didn’t happen. And I don’t want to give cheques while saying “sorry, it’s messed up, please accept it”.
3) I’ll for sure order another cheque book but with another bank. I need and I want service quality without a discount on quality.

In the second iteration.

1) They do a good job on their part. That was the problem with software.
2) I’ll get another cheque book free of charge.

That was pretty much it in the scope of the problem. But after the talk I still felt some kind of resentment because the quality problem was only covered, but not resolved.

And here are my points on that.

As a customer

  • If any object, or a mechanism, or software, or even a particular person as a part of a service has flaws in quality of work then the entire service has flaws in quality of work. If a program that is used to provide service makes mistakes – for a service receiver – that’s the same as the service provider making mistakes.
  • If I received a service of a bad quality, I don’t care how good your other services are.
  • That was another reminder of importance having a few options of choice. If you are locked-in you will have to eat what you’re given.

 

As a tester

  • Testing against possible threats to a value (for customers) is not the same as testing that a program works. If they would use Test Framing approach, they’d probably discover the defect.
    Maybe a draft of a cheque book, displayed on the screen, was looking correct, because web browser handled the special characters. But no one actually tried to print the draft to see how the product would look like. Is it because they didn’t have such a test case? – Then the test case based approach (scripted approach) was not good enough.
  • Defect severity is a commonly used metric. Severity level is defined based upon a possible outage of a program. Can’t login? – ShowStopper severity. However, on a client side, severity may or may not be related to the impact on quality. So, even if a bug report does not contain a field “impact”, providing details about it is very important.

 

As an automation engineer

  • Since I’ve got to know about this particular defect, I can think of ways to implement checking  for it. But once this problem is fixed it’s unlikely to happen again. Anyways, checking a particular parsing functionality is much better to perform at the function level, i.e. with a unit test.
  • I could employ automation to feed a variety of data into the application to investigate what other special characters are not handled properly. That means, automation must be created quickly and still provide reliable results – otherwise it’s worthless investment of time.
  • One will never catch such problems with GUI Record/Playback automation approach (which is neither testing nor programming). To be able to, you have to be skilled in both.

  • 2 responses to "Programs don’t care about quality, companies should"

  • shilpa
    9th December 2010 at 8:44

    Good post. I like the way you broke down the issues and thought of it from the org and customer point of view. I think it’s a good lesson for testers.. where they have to step outside what is given to them and think of the end user.

    [ Albert’s reply. Thanks. Though we can’t really know: maybe such defects were reported but rejected as a low priority? Thus, maybe it’s a lesson not only for testers… ]

  • Darren McMillan
    9th December 2010 at 11:13

    Hi Albert,

    Very nice post & I’m glad that you’ve followed up on your original post about this, with this excellent article, which in my eyes doesn’t just communicate frustrations on your part, and the poor way in which you as a customer had your complaint handled. It also provides excellent feedback for how testers should consider other aspects of testing, other than simply “Does it work?”

    [ Albert’s reply.
    It’s not that customer complaint was so poorly handled. After all, they apologized, and they promised a replacement at no cost.
    But my bug report was rejected. “Application’s fault is not our fault”, that’s what they said.
    My feeling is that I could probably do better articulating it. If I were a tester on that project I’d book a face-to-face meeting to follow up on the phone conversation. ]

    Certainly it shows people how to communicate defects correctly with a developer, team lead, manager and so on, to not be neglected or placed at the bottom of a pile.

    Considering the impact out with simply “does it work?” is very important!

    Very nice post, I’ll share this with my team, thanks.

    Cheers,
    Darren.

    [ Albert’s reply.
    Yes, to me this example is worth sharing as another proof that Risk-Based Testing approach is way more powerful than conformance testing approach.
    Thanks! ]

Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported
This work by Albert Gareev is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported.