On the Perils of Research Audit Tools: Reimagining the PBRF in New Zealand, by Catherine Trundle

The grades will soon be in for New Zealand academics. 2018 marks the end of the current cycle of the Performance-Based Research Fund (PBRF). Like the REF in the UK, this academic audit tool is designed to measure the 'quality' of our research 'outputs'. Every six years it assigns each academic an individual grade (A, B, C or R*), and these grades are in turn financialized into funding units that our universities receive.** 

New Zealand universities have become deeply invested in the monetary benefits conferred by this system, and perhaps more so in the standing they gain or lose within the research excellence rankings that it generates. 

This is a system designed to slowly get under the skin of daily academic practice, and indeed to change academic practice. Many academics now say yes or no to invitations and requests based on how prestigious they will look in their PBRF portfolios, or how much they will impede the work of producing recognized 'outputs' by the cut-off date. We come to see 'gaps' in our academic work based on the categories deemed valuable by the PBRF. It encourages a newly competitive scholarly ethos by conceiving of the academic Commons as in fact disaggregated into a market of comparable, entrepreneurial, and individualized researchers. 

The PBRF is an auditing tool that creates de-radicalizing incentives in academia by making us think that the criteria for becoming a successful academic or a prestigious academy have already been settled. Rather, such criteria should always remain emergent, up for debate, and inherently unstable as we grapple with different ideas about how to make and remake the university into a powerful space for critique and an important public good.

The PBRF aligns us evermore with free market values. It encourages us to publish in high-ranking journals, most of which are for-profit businesses that rely upon the unpaid labour of academics to generate significant private wealth, and which lock our research away from the public behind paywalls. It de-incentivizes us to write or research in risky, experimental ways, or publish in emerging, non-profit and open access outlets not yet conferred with prestige by for-profit citation measures and university ranking systems.

Critique offers us one avenue to resist the PBRF, but we also have to think beyond it. What would academic life be like without it? What would a radical, liberating version of the PBRF look like? These questions would generate a thousand different answers among academics in Aotearoa New Zealand.

People sometimes respond to critiques of audit tools by replying, ‘So, you think you’re answerable to no one?’ This misreads our concerns. To be answerable means to feel beholden to particular questions and particular relationships, to be made to reflect upon one's practice through a specific lens. This is not inherently bad; it all depends on the questions. 

So, to which questions do we want to be beholden? Which ones might best align with our ethics and our obligations, and also make us uncomfortable in productive, rather than atomizing and disheartening ways? These are the questions we need to be discussing in New Zealand academia right now. And we shouldn't all come up with the same answers.

In the spirit of this speculative critique, here are the 'performance' questions that matter to me right now. Like real life, taken together they are not a seamless whole but a balancing act, and the motivations of some tug against the realities of others. Nor are they easily measurable.

I’m not claiming to be great at all of these; they are certainly an aspirational list. Some days, in fact, I’d give myself a lousy score. But rather than these criteria being about whether I get a shiny prize to wave about every 6 years, I think they’d encourage reflection, discourage complacency, and shore up a more socially embedded, critical academic project.

So here goes, my own PBRF, the Personally Bravest Research Framework:

Performance Based Research Fund.jpg
  1. In terms of 'outputs', which human beings did you bring into the world, love, cherish, or exhaust yourself caring for, and how did this role of care remake your academic self, your teaching and your writing? 
  2. In what ways did you practice forms of good self-care in an increasingly competitive academic environment that is designed to make you feel you should be working all the time, ever smarter and ever harder?
  3. In what ways did you generously discuss ideas with friends, colleagues, and students, furthering the scholarship and intellectual curiosity of others, for which you will likely receive no 'outputs' or citations?
  4. Did you manage to keep hold of a good idea a little longer to let it percolate and mature, rather than rushing it out so you'd meet an annual quota for publications? 
  5. How hard did you fight to improve the rights of those colleagues working in precarious conditions, who can't even imagine having the time, resources or security to write or to let an idea or project percolate at its own pace? 
  6. How often did you put aside the label of 'data' and find other ways to connect with your research objects, participants or partners? 
  7. How effective have you been at calling out an all-male panel, a mansplainer, the gendering of service work, the low number of women who get grants and research awards, or reach the rank of professor in New Zealand, and the widespread exclusion of POC in academic life here, even if there was a personal cost to you in doing so? 
  8. Who and how do you cite? How do you give due recognition to all the actors that inspire, support and encourage your thinking, rather than focusing on those scholars and networks whose prestige and power within your field demand deference?
  9. How many times did those of us with privileges give up an opportunity to shine or garner prestige in order to facilitate a person participating or gaining their due recognition who academia has marginalized or underappreciated for too long? 
  10. Did you find ways to talk openly about the failures, impasses, vulnerabilities and uncertainties within your research, not just your successes and accolades?
  11. Did you allow yourself and your colleagues to say all of the following without worrying about the contradictions: ‘this is just a job’, ‘this is not just a job’, ‘I’m terribly smart,’, ‘I’m such an idiot’, ‘my students frustrate me’, ‘my students amaze me’, ‘this is the best thing I’ve ever written’, ‘I can’t write’, ‘my research might change things in my field’, ‘my research is totally meaningless’, ‘I’m done here’, ‘I’m just getting started.

So, what would your Personally Bravest Research Framework look like?


*  A is the top grade and only a small number of academics gain this grade. R means ‘research inactive’.

** With an A grade being worth substantially more money than a C grade.

Catherine Trundle is an anthropologist at Victoria University of Wellington, New Zealand, who researches and teaches about inequality and wellbeing, environmental health, the politics of proof, and gender and kinship. She is also a mother to an energetic toddler, and writes a blog about pies in New Zealand.