preload

“Is manual testing dying?”

Posted by Albert Gareev on Dec 06, 2016 | Categories: DiscussionsNotesQuora

“Is manual testing dying?”, “Is manual testing dead?”, “When manual testing will be completely replaced by automation?”,..

These questions come up very frequently. I see them on LinkedIn. I see them on Quora. I see them on Twitter.

I’d categorize responses I’ve seen loosely in 3 types. Samples below.

  • “NO! It’s not manual!!! It’s all skilled and thinking process! What an idiot!”
  • “Yes. All testing can be replaced with automation and tools.”
  • “There’s much less manual testing needed for a number of reasons but it will never go away completely.”

When I think of type (1) responses I see an emotional charge. I learned that from Gerald Weinberg. The source of the emotion might be feelings of fear, or insecurity, or even superiority. I’m not completely surprised. The same feelings express, for example, recruiters and accountants, – their professions also undergo a disruption, and much of previously manual labor is taken up by the tools.

Personally, I lean more towards type (3) answer with my own perspective. Eventually I took it to Quora. Cross-posting my answer from there.


Yes, manual testing is dying. And here’s why.

Convergence of roles, evolution of skill-sets

See how different species have very similar looks? – They were evolving to fit similar environment.

Also notice that one of these species is no longer present.

convergence2

(Image source: Pinterest * Artis Natura Magistra *)

Software engineering industry goes through disruption. There are less and less compartmentalized roles where people perform one function.

In testing specifically, the paradigm of manual execution of test case scripts is dying off. Disruption is happening in many ways.

  • Test design and execution happen in parallel (exploratory testing)
  • Programmers test at the product level, testers review code and write unit tests
  • Product owners and business analysts perform testing and deal directly with the programmers.
  • Approaches like Agile, DevOps, Lean put an emphasis on the quality throughout the process. The whole team assumes responsibility for the quality at every step of the process. The whole team tests.

And while “pure-black-box-QA-functional-testing” roles may go extinct like those Icthyosauruses, testingcritical thinking and problem solvingwill remain and thrive being a common ancestor of all engineering professions.


And here’s ANOTHER thing I would like to die.

– This odd classification of testing as “manual” and “automated”.

If a tester operates the product by hand – i.e. manually – and through the GUI – some people call it “manual” testing. Well, then a whole lot of stuff is “manual”: manual driving, manual calling, manual cooking, manual eating.. even manual recruitment and manual management.

And on the contrast, if code execution or some tools are involved in the process, some people call it “automated” testing. Well, name one activity where tools or programs are not involved to some extent! Nowadays, cars and phones, kitchen mixers and sex toys, TVs and washing machines have programs running in them. Everything is “automated”!

And whenever we use a computer we always use programs and tools.

Now, there’s a specific subset of understanding, related to creation of programs to aid our testing.

  • if I use some tool to drive the GUI actions and check some values I might appear “automating” those “manual” test cases.
  • if I code API calls and verification of data I might appear doing “automated” testing, in contrast with “manual” operation through the GUI.

I consider this a silly misunderstanding. We may not operate by hand, per se, by we always engage our brains. We may perform GUI operations manually or run a script, but we always exercise our wetware to evaluate the results, and we always investigate “by hand” whenever we need additional information.

There are plenty of products that accept input by other means than GUI – files, database, API. Testing them requires use of certain tools. Some tools are readily available, some need to be coded.

There are also products that accept input in a form of gestures, so operating by hand is implied.

There are also aspects that are too hard or too expensive or too unreliable to formalize with automation, namely usability and accessibility, so doing it the same way as customers will do is the optimal approach.

But any kind of testing always requires using this thing between your ears. There’s no “manual” or “automated” testing. There’s testing; by far, an inherently human activity.

Roles and titles might be different, and there might be no dedicated tester role, what’s important is keeping up the skilled testing function for all people in the team.


  • 2 responses to "“Is manual testing dying?”"

  • Alex
    8th December 2016 at 3:54

    Thank you for the post.
    I am totally agree with the idea, that testing should not be split into only manual and only automated.
    QA Engineer should perform a full set of testing activities to ensure the quality of the product. And these activities include writing / debugging automated tests if it is necessary.
    I want also to highlight that test automation activities (in the form of creating test solutions, tools and infrastructure) – is the job for dedicated engineers. It is a separate career path and it requires a solid knowledge of programming.

    [ Albert’s reply.
    Thanks Alex.
    I agree that “test automation activities” require dedication and solid programming skills.
    I’m not sure about the “separate career path” thing. Imagine a car repairman who never drives himself. No matter how well he knows car mechanics I’d prefer someone else. ]

  • Nilanjan
    9th December 2016 at 3:57

    [Albert’s comments inline.]

    I don’t think that is a accurate read of the industry.
    -And hi to you, Nilanjan :)
    Thanks for adding your perspective.

    People who use automation have a very light weight interpretation of testing. Check the Rails tutorial as an example:
    https://www.railstutorial.org/book/static_pages#sec-our_first_test
    -As a matter of fact I went and checked. That chapter says “Adding a first test”. It is simple and lightweight because people learn from starting small. What the authors do say next, that testing is a challenging thing and requires extensive practice. So what was your point, again?
    (In any case, one example from one book is never a sufficient representation.)

    Of course, that doesn’t prevent people from using their brains. However, on the other hand, there is no obligation to use their brains.
    -You do realize that you sound incredibly insulting, don’t you? I’d caution you from stereotyping developers like you do.

    Manual testing as followed by the AST and CDT is much richer, resulting in conferences dedicated to testing.
    -I do not know of any evidence that the Association for Software Testing or Context-Driven Testing principles follow or recommend to follow “manual testing”. In fact, “manual testing” is not even mentioned in the given Principles or the AST’s Mission Statement.
    By the way, I’m a member of the AST since 2011, CDT practitioner since circa 2006.

    Compare the one line in the Rails guide to a conference.
    -Excuse me, are you serious?

    It might also be useful to post links of how people think when creating automation.
    -This is why I linked to the original Quora question where people expressed their opinions. Whether you think they’re “wrong”, it’s their genuine thoughts. People hardly change their experience-based opinion if they were simply told that they are “wrong”.

    In their books, Brian Marick and Ron Jeffries demonstrate their thinking. However, it is very code focused.
    -Because they focus on code testing, not product testing. That is normal. Engine maintenance manual is also engine-focused and won’t have much about transmission or tires.

    Without looking at code, there are very few examples of developers, who can explain what to test. Without looking at requirements there are even fewer.
    -I can code well in some programming languages and I can read code of other. That only makes me a better tester. And yes, many programmers have testing skills underdeveloped. Guess what? Their trade also undergoes a disruption. Those unable to keep up are also at risk of going extinct.
    I wouldn’t absolve testers here, you know. How many would say that it’s impossible to test without requirements? Unfortunately, still too many.

    Tell them to not write code when testing and you won’t find any developer standing.
    -I have a suspicion that you have never been at a professional code review session. Testing is not about writing code or banging keys, for that matter. May I suggest you to read Gerald Weinberg’s “Perfect Software and Other Illusions About Testing” before we continue our discussion?

    Thanks for taking time to comment, anyway.

  • Leave a Reply

    * Required
    ** Your Email is never shared

Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported
This work by Albert Gareev is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported.