There’s an increasing volume of research being published, driven in large part by the ‘publish or perish’ culture and the associated regular national assessments of research quality, and also university league tables. Because research is presented in print form, this research takes on a status and an authority that may sometimes not reflect its real value or worth. Knowing how to interpret and make sense of this outpouring of research is a key skill for new researchers and also a skill that more experienced researchers need to practice and reinforce on a regular basis.
Two academics from the Australian National Centre for the Public Awareness of Science at the Australian National University have just published in The Conversation an easy-to-read but very effective guide to the ‘10 stuff-ups we all make when interpreting research‘. It cautions against relying on just one study, misinterpreting significance, misinterpretation of effect, and the quality-quantity fallacy (among others).
Its well worth a read and is also worth distributing more widely amongst both commencing and experienced researchers. I commend it to the congregation…
You’ve collected your data, coded it, got your license for SPSS, inputted (is inputted a word?) it, and finally run correlations for all variables against all other variables and you’ve got an interesting selection of significant correlations. If they mean anything, you could have an or even a whole thesis! But how do you decide if they mean anything? That is, how do you move from correlation to causation?
Today’s Conversation includes a well-written, easy to understand guide to the issues and pitfalls in moving from correlation to causation.
It will be helpful to those who are venturing into this area in their research for the first time, and will also be useful if you’ve got undergraduate or taught Masters students who’re unsure about stats and how to interpret data.
To illustrate the sort of issues involved, consider these two correlations taken from a great site described by one of my colleagues as ‘possibly the best site I’ve seen all year..’
The first shows the very high correlation between the consumption of Mozzarella cheese and the award of Civil Engineering PhDs in the US:
The second, a similarly high correlation between the award of Computer Science PhDs in the US and amusement arcade revenues.
Food for thought!
After a break for a holiday to catch up with family in the UK and a short stop over in Hong Kong, a city I’ve never previously visited, I’m promoted to reinvigorate Doctoral Education To Go…by an excellent short piece by Susan Carter on the Doctoral Writing SIG.
I’m interested in the issue of metaphor and doctoral education and Susan’s blog post is a good example of how a well-chosen mental picture or metaphor can stimulate the research writer’s thinking about and then executing an article or, indeed, a full thesis.
Using the metaphors of bare bones, skeletons and X-rays shines a light on the importance of the underlying structure to a piece of writing which may be hidden by the flesh of the argument but, without which, the flesh (that is your data and argument) just doesn’t stand up. Without that skeleton, it is highly probable your article or thesis will become something akin to a jellyfish.
Well worth a quick read and a think about.
Have you ever thought how much easier it would be to get published if you could review you own papers for journals? Well, here’s the story of one, initially successful but ultimately unsuccessful, attempt to do just that. As one of my colleagues said, a story of ‘too much vibration and not enough control’!! (Thanks Judy.) Its a mute lesson in how not to be an academic.
The targeted journal was the respected Journal of Vibration and Control (JVC), which focuses on the area of mechanical engineering. The journal (published by Sage) has now retracted 60 (yes sixty) articles after finding evidence that over a hundred fake email addresses for potential reviewers had been set up in an attempt to get favorable reviews. The SAGE announcement says that ‘on at least one occasion… Peter Chen (of the National Pingtung University of Education in Taiwan, on whom the enquiry focused) reviewed his own paper under one of the aliases he created.’
Hopefully the fact that, on this occasion at least, the fraud has been uncovered will help deter others from following suit. However, the pressure to publish and the pressure on academics to deliver grants awarded at least in part on their publication record may make some others think about the benefits of finding ways of reviewing their own papers. These pressures make even more important the teaching of the ethics of the academic profession to students across the full spectrum of disciplines.
The last couple of decades or so have seen significant change in the higher education systems of both Australia and England. From systems that were essentially free to undergraduate students with no charges for fees and which also provided non-repayable grants to support living costs (for domestic students at least), both countries have seen escalating costs imposed on students and a general opening up of competition between universities. It is hard to overstate the extent to which higher education has changed during this period. From being regarded as a national investment in the country’s future, an individual’s higher education experience is now firmly positioned as being an individual’s investment in their own future. Any benefit to the country is seen as a consequence of that primary purpose rather than being the primary purpose itself.
A short history of these fundamental changes (and the way in which each country has borrowed from the other in a classic instance of policy transfer) is shown in a simple form in The Guardian, If you’re interested in higher education policy and where things might be headed, its worth spending a few minutes on it. It can be found here.
An EU-funded project, UNIKE (Universities in the Knowledge Economy), has just published a report containing its first set of notes. The report looks at European policy on doctoral education over the last 15 years or so and at US policy in the period since World War 2. In addition to the policy discussion, there is also a short essay on the contribution of EURODOC to the European debates.
The report comes out of a workshop being held as part of the project, and is the first of six which will cover:
a) History of policy debates about doctoral education (the topic of this report)
b) Governance narratives and the reshaping of doctoral education
c) Specificity of social science doctorates
d) Partners’ own practices of doctoral education
e) Working for/researching in other organisations
f) Academic freedom
The report is simply written (a good trait which doctoral students would do well to develop) and lays out the ground in an easy-on-the-eye manner. For anyone seeking to understand or work in higher education, a knowledge of where the sector/system has come from will be increasingly important as the pace of change continues to increase. This report offers a good primer on the topic and also offer a series of questions for further consideration (and thereby a possible research agenda for those interested in doctoral education as a research area).
If I have one criticism it is this. The report assumes that there is only a single European doctoral system and fails to acknowledge that each European country has its own traditions, practices and ethos of doctoral education forming strong sub-systems within the overarching system that is sustained by the Bologna process and the work undertaken under the Salzburg banner. For example, the UK system is very different from the French, German, Italian and other systems (which are different from each other) and readers new to the policy debates (at whom this report is aimed) need to be reminded of this. The current system is much more nuanced than this paper sometimes suggests. Nevertheless, this report offers a useful addition to any research student, early career researcher or academic’s reading list.
Have you ever had a paper rejected by a journal? Was it a lower impact journal that you chose because you really wanted to get something published? Well, fear not. All is not lost!
In a blog that is well worth a read, Vincent Calcagno writes about a large-scale research project he has undertaken on paper rejections by journals and subsequent article publication.
He writes, ‘As an ecologist, I like to see the jungle of scientific publishing as an intricate food-web in which journals are dreary predators competing for the valuable food items that research findings and manuscripts are. They sometimes catch the food directly from the source, and sometimes recycle food-items previously rejected by another predator. With our data, we thus reconstructed the “food web” of scientific journals, where trophic links really are publication of an article previously rejected by another journal.’
Amongst the more important findings were:
- about 75 per cent of all articles had been published by the first journal to which they were submitted
- ‘high impact journals tended to publish fewer first-intent submissions (and thus more articles that had been previously sent elsewhere).’
The implication of this is that there may well be what he terms ‘a ‘benefit of rejection’ by which he means that the article is improved as a consequence of peer review (or possibly through the author(s) being prompted to be more careful and select a more appropriate journal) and thereby achieves publication by a better quality journal (measured by impact factor) than would otherwise have been the case.
Something to hold onto the next time you receive a rejection email!