WTA05: A Programmer’s Hat
I continue with the first part of a mission from Weekend Testing Americas session #5.
This was an educational exercise.
What we had?
A simplistic environment
It’s a square. It has finite size. Objects, contained in it, are static – they never change their behavior.
5 commands to use
5 simple, non-abstract commands. Go Forward, Turn Left, Turn Right, Jump Forward,Activate.
A simple hierarchy
Main function can use 2 functions, and the functions can call each other. Function calls go toward instructions count.
Code Boundaries
Number of instructions is limited. Up to 12 in the main function, 8 per subfunction.
Visual Feedback
Execution of instructions is animated. Everything is transparent.
– In other words, we had a quite perfect programming sandbox.
How did it go?
I cite participants’ responses, see chat transcript here.
I’d have to take a lot more time into designing the runs if I couldnt have a trial run. Hmmm, wonder if I can relate this to TDD and agile….
I guess it takes time to understand the system and commands but even at expert level you cannot ignore trial runs.
Trial and error benefits were similar to those already mentioned. Eg. Confirming use of functions, which direction the arrows turned, finding out if man would walk off board or walk of ledges, etc.
I found at some levels I deliberately did more trial and error plugging in mistakes to see what would happen.
I don’t think I would be able to build from first attempt. Very few times I would succeed.
Even if I tried to plan it to the N’th degree I would benefit more from some trial and error.
[Trial and error] much better than spending a lot of time designing something that fails at the last step – or turns the wrong way on step 2 out of 15
I think the process moves more quickly through the adaptation rather than attempting to preplan all the steps.
..so basically find repetitive paths in “method” and put in “function”. Then call the “function”
…at the moment, I think I need to go and do some refactoring…
My point being, how many levels did we go through and test the light “method” before we realized, “oh wait, that doesn’t behave the way I thought it would”.
To sum up, the provided playground was sufficient enough
to discover through practice many of fundamental programming concepts:
algorithm, structuring code, reusable functions, refactoring, and recursion.
But wait…
…What’s about “attempts”? What about “trial and error”? What about “exploring behavior”?
Here’s the “Big Secret” revealed…
…programming is a trial-and-error process!
- Programmers do not create bug-free code in the first attempt
- Programmers can not create accurate (vague criteria) code if they don’t know enough (another vague criteria) of the environment and goals
- Programmers may use (and do use) functions without complete knowledge (yet another vague criteria) about their behavior
- Programmers explore and learn about their programs, including their own code
- Programmers do refactoring, which may either improve code, or introduce defects, or both
Appearance of bugs in code is a natural process
but experienced programmers reduce the occurrence of certain types of bugs.
Having feedback allows programmers to learn and improve
but they need to do that continuosly as the context changes.
What we can learn as testers from this exercise?
Be careful with prototypes
Same solution that works (solves a problem) in one context might not be suitable at all in another context. To implement a trial solution (prototype) requirements and context are simplified. Some risks are ignored. Testing, carried within this artificial box, may not identify any of true risks that threaten to the value of the product if it’s implemented in the real scale.
Maybe, it’s time to revisit our test environment?
Drill down the layers
Development evolves towards more and more layered approach. It makes high-level programming languages more abstract, and programmers more productive. On the other hand, a programmer may not even realize how many layers are deep down under the functions used, and do they behave as expected without doing something unwanted.
Working code does not mean solved problem
Test-driven development approach allows programmers obtaining feedback faster. But what kind of feedback it is? It’s a set of confirmations that tiny pieces of code return the “right” result. Based on that feedback, programmers may only learn not to make certain types of mistakes.
Service functionalities
Each “business functionality” is wrapped up in many service functionalities. Here goes the complexity. And side effects. Per one variable that stores a “business value” there are tens (or hundreds) of variables used in interim purposes. Here go endless boundary problems and software “glitches”. Of course, rules, what and how these “service variables” are used are not documented in requirements. Therefore, testers need to use an approach, completely different from a “test case, conformance based” to reveal the problems that threaten to the value of the product.
One response to "WTA05: A Programmer’s Hat"
I will be the first to admit that I have stumbled through programming through the years, and in many ways, the more tools and layers that appear that “help” me with the process, the more frustrated I tend to get. I compare this to the days when I had just a text editor (using vi, no less) and two Xterm windows, one to edit my file and another to run my commands and see what happens. using gcc to compile native C code or just making shell scripts to do various things in UNIX.
[ Albert’s reply. I kind of miss those days too because I had a product level vision. Nowadays, in large-scale development, programmers code “functionalities”, testers “verify” “functionalities”, all with no idea what customer really needs. This is where division of labour plays a cruel joke… ]
The beauty of working with the UNIX and text only model was that there was little that I had to remember and include, and a lot of it was specific to a small tool or utility. What you didn’t have in one tool you could pipe or redirect to another. the Windows approach totally stymied and frustrated me, and thus, I rarely spent any more time with it than I had to (it used to irritate me that I had to write all this code to draw a window and set it up… I wanted to parse files and get info, why do I have to deal with all this extra stuff (sorry, end rant).
[ Albert’s reply. Again, felt similar. Scripting languages to help, though. For example, if you have MS Excel installed – you’ve got VBA. I wrote a lot of “quick-and-dirty” VB scripts for data transformation, generation, etc. James Bach has many references to Perl on his web-site; Michael Bolton uses Ruby. ]
The challenge I see with wearing the programmers hat is that it’s more than just programming. It’s gotten to the point where knowing an IDE front and back is essential, having dozens of libraries on auto call is needed (and you better know how all those libraries work)… it made doing quick and dirty programming tedious. I admit I much preferred the older UNIX model, but I’m willing to work more towards getting up to speed with the newer IDE approach (not like I really have a choice ;) ).
[ Albert’s reply. As a tester, the concern I have on all these libraries-APIs-Frameworks pyramids is that quantity of code has grown hugely, but the quality went down – as well as quality of programming. Programmers became users too, and they use libraries provided without a complete understanding about its behavior. Applications are created with visual design or record/playback tools, with thousands of default settings. To business it looks like “just 20 lines of code”, while testing needs to cover the application consisting of 20,000,000 of code – or more. ]
[ Overall, I deeply appreciate your support, Michael, WTA 05 was a great session, and I look forward to having more! Thanks, Albert ]