Judging the judges: tasting and theory, WSET-style

Posted By Chris Kaplonski on Mar 12, 2016

Earlier this week, I sat in a hall in the rather decrepit-feeling Royal National Hotel in London to take the WSET Diploma Unit 6 (Fortified Wines) exam. There were somewhere between 130-140 of us, all Diploma students. Did I pass? I don’t know. It will take 3-4 months to find out. I correctly identified two out of the three wines (an Oloroso and a Rutherglen Muscat) but in the tasting section, correct identification is only about 10 percent of the mark. So it helps, but not a whole lot. As for the ‘written’ or ‘theory’ section, well, I’ll come back to that.

The exam was a Big Deal for several reasons, some of which I’ll touch on here. Others I’ll leave to future posts, since the exam helped, among other things, concretise some things I’d been thinking about in terms of anthropology and the senses. So, why was this exam – my second of six for the diploma – a Big Deal? Let me list a few things.

My colleagues and friends are doubtless quite relieved I won’t be complaining about fortified wine and WSET exams, at least until I find out if I have to resit the exam. Last year, the exam had a pass rate of 53%, so it is a distinct possibility. In general, the pass rate seems to hover around 60%.

It also means, at least until the results come out, that I can put away all those books on fortified wines, some of which make academic writing almost a pleasure to read in comparison.

In many ways, I think of this as the first ‘real’ WSET exam for the Diploma. Our first exam was multiple-choice, and those I tend to do well on. This was the first of four exams where we have to write tasting notes, evaluation and also answer other questions. So it was a good idea of what to expect in later parts of the course.

As I already noted, it helped clarify some things I’d been thinking about, which is usually a good thing in research.

On that aspect, the exam was also an interesting piece of participant-observation. Perhaps I’ll write up my fieldnotes as a blog post sooner or later.

But now, let me give some unsolicited advice to WSET. This may be inadvisable – after all, my exam still has to be marked – but I’m banking on it being done by student number, not name. Nonetheless, I’m a little hesitant to put these comments out in public. But I will, since I think they need to be stated.

First – I realise it is hard, if not impossible, to provide proper (or anything like) wine-tasting facilities for 140 people. Fair enough. But no facilities to wash glasses, etc? Again – I do recognise the issue with finding a place, but surely you could find a place with a few more restrooms close by, for people who do want to rinse out their glasses. Or maybe some water coolers or something, with buckets near them? It was also appreciated – if common-sense – that you provided buckets to dump the samples in after the exam. I know there were some at the front of the room. But really, one five-litre bucket by the door everyone has to exit through? Surely you could spring for at least two?

But more importantly, and perhaps controversially, let’s talk about examiner’s reports and what they suggest about the test questions. I’ll preface all of this by saying, yes, I know people get Distinctions and Merits as well as Passes on the exams. I realise this could be taken as pre-emptive sour grapes on my part. But from chatting with others, I think at least some of these points are shared more widely.

I read the various examiner’s reports you (WSET) have posted on the website. They were useful in giving some idea of what you expected us to answer on the written part of the exam. But they also highlight what I see as several shortcomings. First, the vagueness of the questions. You don’t seem to want a theoretical argument, but rather a data dump. Tell us everything you know about X, in effect. The questions could also be clearer. The three questions were prefaced with ‘In relation to fortified wines, write about each of the following.’  One of them was ‘c) Pedro Ximénez.’ Fine, but since this relates to both a grape and a type of sherry, how do we answer? Will you give full credit to someone who writes well on the grape, but doesn’t cover the sherry, since that’s not how they understood the question? It isn’t clear, and direction to the students would have been helpful. (I covered both, just to be safe, but who knows?) Similarly, one of the examiners’ reports complains about ‘not impressive’ results, when one of the topics was ‘very mainstream’: ‘oxidation.’ Mainstream, perhaps, but also maddeningly vague in what is expected. The process? The impact on taste? Impact on longevity? All of it in ten minutes? In what depth? Students are not mind-readers.

More importantly, I would call attention to recurring remarks in the reports. Pretty much every year of the 10 or 12 years of reports on file has, in its section on the written part of the Unit 6 exam a passage that runs: ‘Results for this question were worse than the tasting, resulting in a poor result for this unit as a whole. The large number of candidates achieving very low marks on this paper, suggests that many of them either did not prepare sufficiently for this exam, or simply are not yet ready for a qualification of this level.’  

Here is where the unsolicited advice comes in, based on over two decades of teaching at the University level. To me, the repeated appearance of this sort of comment in the report suggests four things, not all of which are mutually exclusive:

  1. The report is correct, and the students haven’t prepared and / or are capable enough.
  2. WSET is ultimately failing in its approach to test preparation. Yes, there’s a large amount of self-directed work required. That’s fine. But when you cover a topic four or five months before the exam, with no intervening chance to ask questions about revising for the exam, or to recalibrate your senses? Not so fine.
  3. You are happy with the pass rate, in which case, tell your examiners not to complain so much.
  4. You are not happy with the pass rate. In which case, given it is a complaint repeated year after year after year, perhaps it is time to consider that the problem lies not with the students, but with the exam? In my experience, if one test is particularly out of line with the general trend of results, we may put it down to the particular group of students, but we’d also take another look at the exam. And if year after year we got results we weren’t happy with, we’d take a long hard look at ourselves and the questions we ask.

Ok, unasked-for advice done with, for now. In light of what I’ve written, you may wonder (if you’ve read this far) if I think the WSET Diploma is worth doing. That’s a question that has a rather complicated answer, but I’ll end this post with the short form: Yes, I do. And I’ll be back next year to complete it.

Submit a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Share This