Replies to This Discussion

Permalink Reply by Jake Brake on January 9, 2009 at 12:56

Phil,

My work spans that colo(u)r spectrum with an emphasis on gray. Most of my tests target architecture. However, I am transitioning to green as decpicted here. ;)

Permalink Reply by Jim Hazen on January 9, 2009 at 17:55

Black. I test at the 'system' level. I mainly deal with things from the User interaction and business workflow levels. I also work with the Performance aspects as well. I don't delve into code & data much, unless it is backend verification of the data in the database.

Permalink Reply by Mohammad Shahwaiz on January 9, 2009 at 21:48

I am also comes in gray category 'coz most of the time i test the modules for database and scripts. I did projects till date those are related to data flow between one program to another. I did white box testing as well for Java project using Junit and Security testing using Appscan tool.

Permalink Reply by Alan Page on January 9, 2009 at 22:28

Bright Gray (or grey if you prefer)

Permalink Reply by Glenn Halstead on January 9, 2009 at 22:30

Mostly Black for me. It can be a fairly long trail from end to end for the testing we do, touching many apps within the overall system / soution.

Permalink Reply by Michael Bolton on January 10, 2009 at 3:52

I deny the existence of the box.

---Michael B.

Permalink Reply by Michele Smith on January 10, 2009 at 11:33

Hi Michael,

I am quite curious about your answer. Could you explain further?

Permalink Reply by phil kirkham on January 10, 2009 at 12:07

I was almost expecting that sort of answer from Mr B ;)

I'm sure he'll be happy to expand further

Permalink Reply by Michael Bolton on January 12, 2009 at 4:27

Correct on both counts, Phil. :)

(Thank you for asking a question that generated a good thought for a blog post. It'll look a lot like this:)

I'm not sure the distinction between shades of boxes is terribly helpful. The box metaphor is helpful in one way: it reminds me to think about constraints by which I don't want to be bound. I can almost always do something about some of those constraints. Meanwhile, there are some constraints that I can only ever circumvent to some degree, not completely.

The metaphor of the black box is intended to represent a system for which we have no knowledge of the internals. As a tester, I'm rarely in that position unless I want to be, and I rarely want to be Simply by asking for it, I can usually pop the lid off the box entirely by asking for the source code itself, or for other useful forms of information about the internals. If I can't see the code, at least I can ask lots of questions (I us... Heuristic Test Strategy Model as a point of departure for them). Even without that, by using tools of various kinds I can always peer inside the box to some extent—listing out the imported and exported functions, inspecting the resource table, or examining the compiled binary in a text editor for clues. By using knowledge and experience, I can make reasonable inferences about the internals. But those determinations don't tell me what's going on inside, and that might be crucially important. Thinking solely in terms of the black box limits me.

(The metaphor of the white box, to me, is silly, since a white box is just as opaque as a black one. Note that this is an example of matching bias; the opposite of black is white for most purposes, so people talk about white-box testing and forget what the metaphor is supposed to represent. Glass-box testing covers the intended meaning better for me.)

So let's look at the glass box, then. The glass box is clear, allowing us to see everything in its innards. That's certainly a nice idea, but it prompts some questions. What does "everything" mean? Is it really important to see everything? Could we see everything even if we looked?

Consider: we generally consider the glass box to be the source code for an application, consisting of instructions that our developers have written. Yet our developers make calls to application frameworks, third-party libraries, operating system functions, code that other developers wrote, hardware interfaces, and so on. Our code may provide services and functions for other code. So we're not really seeing everything; we're seeing what there is to see, and how it might refer to other black or glass boxes. We might be able to take advantage of our knowledge of the the internals of our code to use its interfaces or to instrument it in some way. But even if we can read the source code, we can't necessarily determine the significance of the function for some task, nor can we anticipate completely how it might go wrong. We might be able to determine if the code has problems in it, but we can't determine if the code was written in an optimal way, since we're residents of our own state of the art, as Billy Vaughan Koen might say (see his book Discussions of the Method). We can't completely recognize the programmer's intentions, her mental shortcuts, her brilliance, or her blind spots; the code might suggest things about that, but it doesn't tell us conclusively. Even if we can anticipate the platforms on which the code is supported, we can't represent them all in the lab, and we can't determine if a change to those platforms will render our code useless. Visibility into the glass box might be helpful, but it doesn't necessarily and on its own tell us about what might be important to someone who matters. Indeed, thinking too much in terms of the glass box may dazzle or distract us into thinking that the box itself is the important thing, rather than the value that the box provides with respect to the rest of the system and its human users.

The box that I'm currently looking, black or glass, at is always connected to other boxes. No box ever really stands alone; its behaviour is ultimately interesting only with respect to other boxes. Some of the more important black boxes here, boxes into which I cannot truly see, are the minds of the people who wrote the code and the minds of the people who use it. Thinking solely in terms of the glass box limits me, just as surely as thinking solely in terms of the black box does.

So: am I a black box tester? If I feel that I'm in that mode, one of my first impulses is to consider whether it's important, for the current question I have, to figure out what's going on inside. Am I a glass box tester? If I'm in that mode, one of my first impulses is to consider what I'm not noticing and how this box interacts with other boxes, some of which may be glass, some of which may be black, and some of which may be frosted, transparent, translucent, made of Lexan, made of cellophane, or made of diamond, all in every colour of the rainbow.

So whatever the colour of the box, whatever its visibility, the value of my examination is limited if I'm only looking at its insides or its outsides. As much as I can, I want to comprehend them both. So whatever the colour of the box, whatever its visibility, the value of my examination is limited if I'm only looking at its insides or its outsides. As much as I can, I want to comprehend them both. In particular, if I'm trapped in a box, I want to escape before the food and oxygen runs out.

Hope this helps,

---Michael B.

Permalink Reply by phil kirkham on January 12, 2009 at 8:41

Thanks for the full explanation of your thinking - glad my question was able to provide inspiration for a blog post

and now we could expand this further and wonder why you deny the box but others are happy to accept they are black, white, grey ( or purple )...

Permalink Reply by Michele Smith on January 14, 2009 at 9:31

Thanks Michael,

I like the explanation because once the application/software is installed, it is within the system. This makes staying within the confines of the box nearly impossible.

One of the examples that you used, "developers make calls to application frameworks, third-party libraries, operating system functions, code that other developers wrote, hardware interfaces, and so on", is probably the chief reason that I have to ask questions that are outside of the box.

Permalink Reply by Shrini on January 14, 2009 at 10:04

>>>I have to ask questions that are outside of the box.

Hi Michele,

It would be helpful to think of many boxes. When you say "out of the box", I presume that you are referring to "only" one box that is either glass box(where you can see the stuff inside hence enables you to test those components that are /otherwise/ not visible) or a black box where you can only see what you can see from your eyes from outside.

In reality there are many boxes ... (that is the prime reason why some of us have problem with box analogy) or as I mentioned in the reply to this post ....boxes are what our models are made up of. If there is a box, it mostly has boundaries or edges ... since boxes themselves are imaginary (models) so are the boundaries of those boxes. Everything is like fluid.

Phrases like "outside the(not "a") box" or "inside the box" - indicate that there is only one box ... that is a narrow model, in my opinion. So what is outside a box ...many boxes probably ... It is like asking what is there in the solar system other than planets,satellites, and other celestial stuff ... note the solar system is a model of how we understand but that is not all... There is a natural problem that we often confuse the model to the real thing.

When people happily accept the notion of black, grey, white - I think they apply the model of one box. A black box tester will think that she has a black box (at times /pretending/ as though she can not see the stuff inside) and interacts with the stuff that is supposed to behave in certain way (regardless of the /structure/ stuff inside). While a whit box tester has a glass box or a transparent white box to work with .. she sees the stuff and structure inside and tests with it.

Hence in "one box" (color) analogy - all the software/hardware (application under test, operating system, all third party software etc) components are abstracted as one cohesive unit that is put inside a huge box which at times behaves clear box or opaque black box or any other color depending upon the observer (the tester) who is out side the box - an inside-out model ... (what would happen if the observer is inside the box and looks at the walls and structure of the box - boundaries etc ? how will that change the model ?)

So you can see ... this a very primitive model ...

Shrini Kulkarni

• ‹ Previous
• 1
• 2
• 3
• 4
• Next ›
• Page