QA StepsLog in

Exploratory testing with a charter

Exploratory testing means designing and running tests at the same time, while you learn the product: each next test comes from what the last one showed you. It still has a plan. Before the first click you write a test charter, one sentence that says what to explore, with what, and what to look for. You test inside a fixed time box, often 20 minutes, and take short notes as you go. When the time is up you stop and debrief: what you covered, what you found, and what you left out. This way of working is called session-based exploratory testing, and it helps most when specs are thin or time is short.

3 parts
of a charter: what, with what, for what
20 minutes
one time box, then stop
4 kinds of notes
tried, found, question, for later
Arti, the QA Steps tutor, in a scene for the article "Exploratory testing with a charter"

How to run an exploratory session, step by step

  1. Step 1

    Write the charter

    One sentence: the target (one page or feature), the resources (data, tools, ideas) and the kind of problem to look for. For example: "Explore the cart with coupon codes and quantities to discover wrong totals."

  2. Step 2

    Set a timer

    Twenty minutes is a good first time box. Put the app in one window and your notes in another.

  3. Step 3

    Test and take notes

    One line per note, each starting with its word: Tried, Found, Question or For later. For a Found line, add what you expected and where the expectation comes from.

  4. Step 4

    Stop when the time is up

    Even in the middle of an idea. The idea goes into the notes as For later.

  5. Step 5

    Debrief

    Answer three questions in writing, or with your lead: what did I cover, what did I find, what did I leave out.

  6. Step 6

    Plan the next session

    Each Found line becomes a bug report, each question goes to the product owner, and what you left out becomes the next charter.

What exploratory testing is, and what it is not

You choose the next test from what the last one showed you. Switch the currency, and the price looks odd; so you switch again, on another page, and compare. A written test case fixes its steps in advance. An exploratory session lets you follow the product.

In interviews you may hear that exploratory testing is clicking around with no plan. Clicking with no plan has its own name: ad hoc testing, informal testing with no analysis or design behind it. Without a charter, you see something odd about a price and cannot find it again. "How is ad hoc different from exploratory?" is a favorite interview question, and the charter is your answer.

Exploratory testing is useful when there are few specs or little time, and it can use other techniques inside a session: if a field has a range, partitions and boundaries tell you which few values to type in the minutes you have.

Diagram: What exploratory testing is, and what it is not

A test charter example, with notes

Shelf is the practice shop the author's courses use. The page under test: Hearthline Classic Kettle 1.7 l, USD 49.00. Compare two charters for the same 20 minutes. "Test the product page" names only a target. The charter below names the target, the resources and the kind of problem to look for, so every next idea has to serve one mission.

The example also shows notes from the first five minutes. Four kinds of notes, because each one leads somewhere different. Tried shows what is covered. Found becomes a bug report. A question goes to the product owner. For later becomes the next charter. A text file or a sheet of paper next to the browser is enough.

Charter and notes from one session
Charter: Explore the product page with currency switching
and long product names to discover problems with how
prices and ratings are shown.
Time box: 20 minutes

Tried: switched USD to EUR and back to USD. The price
  returns to USD 49.00, with the code shown each time.
Tried: the rating under the name reads 4.6, from 212 reviews.
Found: the same kettle shows 4.6 on its page and 4.5 in
  the search results. Expected: the same rating wherever
  the shop shows it.
Question: should the cart follow the currency chosen on a
  product page? The spec doesn't say.
For later: photo zoom on a phone, another session.

Staying on the charter

Twelve minutes into the session on prices and ratings, a new line goes into the notes: "the review form takes 2,000 characters." Is there a limit? What happens at 5,000? Every one of those is a fair test idea, and none of them is in the charter. Follow the review form now and the time runs out with the currency switch half-tested. Write it in one line under For later and go back. A parked idea is kept: after the session it becomes a charter of its own, or a bug report if the spec sets a limit and the form ignores it.

The second judgment call: odd is not always a bug. Say the reviews are listed newest first. Look for the source of the expected result. If the spec says reviews are sorted by rating, you have a finding. If the spec says nothing, you have a question for the product owner. Write it down with what you saw and move on. You do not decide the shop's rules alone, and you do not stop the session to argue about one.

The debrief, and why written checks still count

Below is the debrief of a 20-minute session on Shelf's cart. The charter: explore the cart with quantities, the coupon SAVE25 and the shipping line to discover wrong totals. The shop promises "Shipping: USD 4.99, free from USD 50.00 or when every item ships free", counted on the subtotal before the coupon, and a coupon that takes a fixed amount off the subtotal, never more than the subtotal.

The last answer, what was left out, is the one people skip, and it is the most useful. It tells the team what this session does not promise. A session that found nothing and reports "the page is fine" reads "no defects found" as "no defects".

Exploratory sessions add to written checks; they never replace them. A written check repeats the same steps release after release: add 2 Steel kettles, apply SAVE25, expect a total of USD 23.00. It catches what used to work and has stopped. A session finds what nobody thought to write down. Teams need both.

Debrief of a cart session, on a build with defects
Covered: shipping just below, at and above USD 50.00;
  SAVE25 in lower case, on a small cart and on a cart
  over USD 50.00; an unknown code; a negative quantity;
  the limit of 10.
Found (one bug report each):
  - At exactly USD 50.00, shipping is still USD 4.99:
    "Add USD 0.00 more for free shipping."
  - USD 9.99 book with SAVE25: −USD 25.00, Shipping
    USD 4.99, Total USD 0.00.
  - Quantity -2 gives a negative total.
Question: SAVE25 is valid for 1 to 10 items. What if
  the cart grows to 11 after the coupon is applied?
Left out: quantities 0 and 11 typed in the cart, pressing
  Apply twice, products that ship free.
  -> the next charter

Where to learn this

Questions and answers

What is exploratory testing?

Designing and running tests at the same time, while you learn the product. A good session has a plan: a one-sentence charter, a time box, notes, and a short debrief at the end.

What is a test charter? Can you give an example?

A short mission for one exploratory session: what to explore, with what, and what to look for. For example: "Explore the cart with coupon codes and quantities to discover wrong totals. Time box: 20 minutes."

What is session-based exploratory testing?

Exploratory testing done in time-boxed sessions, each guided by a charter, with notes taken during the session and a debrief after it: what was covered, what was found, what was left out.

How do you do exploratory testing?

Write a charter, set a timer, test and take one-line notes (tried, found, question, for later), stop when the time is up, debrief, and turn what you left out into the next charter.

Is exploratory testing the same as ad hoc testing?

No. Ad hoc testing is informal testing with no analysis or design behind it. Exploratory testing has a charter written before the first click, a time box and notes, so its results can be named and repeated.

Does exploratory testing replace test cases?

No. Written checks repeat the same steps every release and catch what has stopped working. Exploratory sessions find what nobody thought to write down. Teams need both.

More guides

See the first course

The full outline of every course is available: you can see each lesson before you start.

All courses

Cookies keep you signed in. With your OK, one more remembers which link brought you here. Learn more