Shadow Work In Testing. Automation

Posted by Albert Gareev on Nov 06, 2015 | Categories: NotesNotes

This writing is a cross-reference to the post  with regards to the concept of Shadow Work.

A quick reminder.

In economics, shadow work refers to unpaid labor in the form of self service. Shadow work has one or few of the following attributes.

  • Transferring part of the service from company to customer
  • Takes away personal time
  • Because of automation, people actually work more, not less

So, what’s about automation?

“Because of automation, people actually work more, not less”. This seemed to me odd, at first. Don’t we automate tasks to accomplish more by doing something else? After all, I (sometimes) do a lot of task automation in testing (what I used to call “test automation” before). But then it made sense. Automation performs something; yes. But it is for people, must be triggered (sometimes, also guided) by people, and people have to review it and make decisions. Automation is inflexible – so it forces people to adapt for it’s hard-coded process and struggle when it gets stuck. Automation is irresponsible, so it doesn’t have to worry, but people do, and they now have to worry for the parts they don’t control.

In the previous post I highlighted typical negative patterns associated with the shadow work.


  • Don’t see it (or perceive it as a “norm”)
  • Don’t know about it (not aware anyway)
  • Don’t understand (because it’s technical aspect lying beneath)
  • Don’t account for it in planning and estimation
  • Don’t give recognition

Top negative attributes

  • Unexpected
  • Unscheduled
  • Unsupported
  • Unrecognized

Main emotions

  • Disappointment
  • Frustration
  • Distrust

Let’s see how it applies with regards to the shadow work caused by the automation.

In the context of testing, often a misleading term is used – “automated testing tools”. First of all, automation is merely capable of checking. Then, automation tools are rather, “automation development tools”.

No automation tool, whether purchased or downloaded for free, will go ahead and start checking. They must be configured. The code must be written and debugged. They must be run on right environment and with the right parameters. Ultimately, it’s still humans who must review and investigate the results.

And this is where the shadow work creeps in.

Even with a proper implementation of Robustness the automation will have various problems not related to the application bugs. API and SOAP scripts are more stable. GUI scripts tend to be the most brittle. Performance scripts may fluctuate.

As the result, the scripts do require some “massaging” to get the results. A good framework may help to reduce the expenses but only to some extent. All this prep work is passed on back to testers as a shadow work. There’s an exception – if a company wants to pay for these costs it may hire a dedicated automation support specialist. Sometimes, big corporations end up with an army of automation folks outnumbering testers but underskilled in testing.

But let’s say the automation scripts did run. If the framework has good variety of comparison rules, and the logic was coded well, we may not get too many false positives. Otherwise, a huge amount of time (that could be dedicated to testing!) is wasted on investigation of errors. But even passing results are not that comforting! How do you know that they are not false negatives? Let alone the question whether the scripts are checking the right things. Again, human skill and time are required to validate the automation. These are all the overheads, the unaccounted for shadow work because of the automation.

Emotions follow pretty much the same pattern as before. At first, it’s a disappointment. Then stress and frustration. And then distrust and rejection of automation.

How to handle these problems?

I found useful setting the right expectations. TERMS, if you like.

Being conscious that the automation has no use by itself is critical. It’s only as good as people who use it. It’s their diligence and testing skills that get the job done. Endorse those, and make sure that the trade off between testing time and automation time, including the shadow work, is balanced by the value.


Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported
This work by Albert Gareev is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported.