Repeat, repeat, repeat

Today while I was reading up articles on solar cells I came across something that touched a real chord with me:

In the September 2014 issue of Nature Photonics, Zimmermann et al. had a commentary piece titled “Erroneous efficiency reports harm organic solar cell research” on page 669.

The authors commented that mischaracterization or solar cell power conversion efficiencies and inconsistent data being published in scientific journals (in the field of solar cells) was particularly harmful for the area. The race for getting the best results and publishing them in the journals with highest impact factor, has in part led to people being less careful about incorrect measurements and poor reporting.

The danger when such articles multiply and proliferate is that the data being reported is unreliable and one doesn’t know which data/papers to trust. The progress of the field as a whole is hampered.

Having data and results that can be trusted, repeated and verified is a must for scientific research. In some cases, the methods to be used for characterization are clearly laid out and researchers can follow these, and/or conduct standardized tests/measurements to show the veracity of their results. This instills confidence in readers about the work and should positively impact the citation of the work too.

Obviously such issues are not confined to one field alone. For numerical modeling as I have said in previous posts, benchmarking results of a new technique against existing test cases/analytical solutions is a must!

The sheer number of the papers that were reporting results which overestimated performance though was quite a shock!

I think from now on I am going to be even more rigorous about my results as well as those of the papers I review/edit!

Publishing a paper with simulation results

Dealing with tons of papers on simulation (or containing some simulation) results, as reviewer, editor (of IEEE Photonics Journal and JEOS RP) and conference chair of OWTNM 2015, I thought I’d point out a few (hopefully) useful things that I’ve picked up.

Some pointers are general and applicable to any paper, and others more specific to simulation related papers.

General:

– when suggesting potential reviewers for your manuscript, make sure the email addresses of these people are correctly entered! I cannot tell you how frustrating it is as an editor to find this information is inaccurate and preferred reviewers cannot be contacted.

– do not suggest reviewers from your own research group or institution, to avoid a potential conflict of interest

– make sure your own inbox doesn’t bounce back emails from the editor!

Simulation related:

– for papers that present new algorithms and methods, it is imperative to convince  reviewers that the method works. The best way to do so is to show test results for the method against problems for which analytical or well known solutions exist. The more standardised and well accepted the test problem, the more credibility it lends your method. Benchmarking against results from other well cited results can also be quite helpful. Do this before applying the method to a new structure and presenting those results.

–  show tolerance and stability behaviour: since the method will have a number of parameters it is a good idea to show how error/accuracy varies as a function of these variables. The stability behaviour of the method against parameters can indicate the robustness of the method.

– if there are limitations to the method, it may be worth discussing these sometimes. It can help to define the applicability of the method.

These all constitute useful information for reviewers and editors when making decision to accept a manuscript: how sound the manuscript is technically, how useful will it be to readers, how widely it will be read and how well presented it is?

A view from the other side: journal publishing

For a long time I have been on one side of the table: writing and submitting papers to various academic and technical journals. So I have like most scientists my views on the process.

Now that I am also sitting on the other side of the table: part of the journal editorial team, it is an entirely new experience!
I recently joined the IEEE Photonics Journal as Associate Editor and in early 2014 I took over as Section Editor for the Journal of European Optical Society: Rapid Publications.

As an author, almost inevitably I felt the review process took too long. Why cant reviewers send their reviews on time – I would fume.
Occasionaly I felt the reviewer/s may not be from the exact technical area and hence not fully able to see the nuances in my paper! They are missing the whole point/don’t appreciate the fine technical details of this area- I would fume!
Very often I felt that changing 1 sentence in the manuscript did not merit a “major revision required” judgement!
Changing reviewers where revisions were needed also came across as disruptive to the process to me as an author. After all, the new reviewer seems to have a completely different perspective from the older one!

And now?

Well as a section/associate editor dealing with each submission assigned to me raises many difficulties:
– to find appropriately qualified reviewers who are willing to perform the review within reasonable time. This can be fiendishly difficult!
– when the reviews are late, how the devil do you get people to do this voluntary work on time and respond to your reminder emails!
– how do I find appropriate reviewers if the manuscript is in an area that I am not a specialist in?
– make sure I dont send too many manuscripts to the same reviewers- avoid reviewer fatigue!
– ensure that there is some diversity in the reviewers (gender balance for example)
– check the reviews and see their quality, relevance, fairness, confidence (not satisfied means starting all over with new reviewers!)
– make decisions where the reviewers do not agree

So when I submitted a manuscript recently to Optics Express and began tracking its progress, I found that I was guessing the internal workings of the process as much as I wanted it to be speedy and positive!

It sure made me see publishing in a different light. How do you feel about it?

Looking for deep impact

In the last couple of months I have come across articles (describing studies) on scientific publishing, more specifically publishing scientific papers. It’s got me scratching my head a bit!

One study reported in Physics Review Letters (vol. 74, 208) discusses how papers that combine (and get the balance right) unusual work/findings/knowledge and more conventional understanding tend to have a higher impact.

When most of us write papers based on our results or proposed ideas, the concerns foremost in our minds often are on how best to present the material, engage the audience and highlight the novelty, the breakthrough, convince readers of the claims and so on. There is also the usual set of decisions on which journal to choose, the kind of paper (letter/full length article), include media files to supplement or not and so on.

I doubt that anyone would want a low impact/poor citations. Yet, the writing and paper planning process for me has never till now included a strategy of the sort intuited by the study I mention above. Would it be possible to adopt such a reasoning in my approach to planning and writing papers? Should I even try and can it be successful? Can it be applied to research grants as well?

So that’s genuinely surprised me and what you think and your experiences about this would be really interesting to know.

Review for my book

I feel a bit like a kid whose story has been read by the teacher and who is going to give her verdict!

My publisher sent across this review from the Midwest Book Review:
Finite Element Modeling Methods for Photonics
B.M. Azizur Rahman and Arti Agrawal
“Finite Element Modeling Methods for Photonics provides a powerful resource describing the applications of FEM in photonics devices and covers everything from problem-solving applications to real-world examples and mathematical concepts. Engineers involved in developing photonic components will find this a powerful guide to the simulation process as a whole, with chapters including formulas, structure analysis, discussions of different methods and approaches, and investigations applying different methods to problems. The result is a powerful technical reference highly recommended for any engineering library.”
– The Midwest Book Review
November 2013

Needless to say I am pleased (blush). Hope the readers find the book useful.

Hey! Have you read…

Harry Potter, Game of Thrones, To Kill a Mockingbird…

I could probably make a list of the books that I’ve loved and asked my friends cartoon on science booksand family if they’ve read them too. Not only that, I could make a list of books that people have asked me about, those that hit the bestseller lists and become all the rage. You probably have your own mental lists ready at this stage too!

To me a good book pulls me into its universe, making me feel what its characters experience, think like them and about them. It makes me care about the outcomes and the implications of the story. It’s magic of a sort.

So what really intrigues me are the books on science written for the average reader by the technical expert! Think ‘The Selfish Gene’ by Dawkins or ‘A Brief History of Time’ by Hawking (and many others..).

We’re used to thinking of “A Song of Ice and Fire” and “Harry Potter” as magical universes. But the best non-fiction books can show you the magic world of real Science as well.

These fantastic books present to us some incredibly complex and challenging scientific concepts/arguments in a way that we can understand and appreciate. They make us think and question, they rouse a sense of wonder and curiosity about Science. They can inspire.

The book I enormously enjoyed recently was Philip Ball’s ‘Nature’s Patterns‘ in three parts which discussed patterns we see in nature and how they form. It’s brilliant!

A typical scientific career (an academic one, at any rate) primarily involves writing for technical audiences within the discipline through journals and text/research books. The rewards in this profession too are mostly connected to citations of journal articles by other scientists, impact factors and so on. So what drives people to write pop Science books?

I am speculating, and think it could be a combination of few things:

– fame (if you can write something hugely successful)

– success that may come in part because of that fame

– money (again if the book is sufficiently successful)

– love of writing

– desire to share with other people the science you find so captivating

– influence people, policy

– to win awards (such as Royal Society Winton Prize  for books, Wellcome Trust Book Prize, AAAS/Subaru SB&F Prize for Excellence in Science Books among others )

Yet writing about Science for a lay audience is a daunting task and even for those who do undertake it, success is hardly guaranteed. On the Guardian site 5 very famous writers (James Gleick, Lone Frank, Steven Pinker, Joshua Foer and Brian Greene) talk about writing successful Science books (Science writing: how do you make complex issues accessible and readable?)

So do you think you’ll have a go at your own book?

I hope you will and that I will get to read it!

Meanwhile I will work on a follow up blogpost on general Science communication – keep a look out and happy writing!

Paid Access journals: some thoughts

In a recent post (Open Access: needs more work!) I wrote on how Article Publishing Charges (APC) can hinder equality in the gold route of OA publishing.

Today, I want to discuss the paid access traditional journals and publishers.

The boycott of Elsevier highlighted like nothing else the downsides of the business model employed by many professional publishers. I list only a few below:

–          Using Government/taxpayer funded research to generate profit without contributing to research funding

–          No sharing of revenues with authors

–          Not paying the reviewers members of editorial work for their labour

–          Bundling of several journals and selling journal access at very high prices

Yet I argue that these journals and publishers have an important role and place in academic publishing in the future, provided they make some key changes (given later).

Why?

Well, here are some reasons:

–          Some journals have been established for decades and are well recognized. There is benefit to authors by publishing (citation, impact, recognition, prestige) and to readers as they feel the quality of work is good.

–          It takes a long time and a lot of resource to establish a new journal, and so should we neglect what has been built over the years, if it can be adapted?

–          OA (gold route) shifts the cost burden entirely onto authors via APCs- pricing out some authors, see Open Access: needs more work! while traditional subscription-based journals put the entire cost burden onto readers, allowing authors (even those that are resource-strapped) to publish in good journals. (Caveat: these may increasingly have lower citation rates and impact factors than the top OA journals)

–          Membership of editorial boards of such journals is a factor in success with promotions, jobs and grant applications; it is a marker of seniority in the profession.

So clearly the gold OA (authors pay all) and traditional journal publishing model (reader pay all) are at opposite ends of the spectrum. Each has its benefits and problems. I think we need journals from both business models to meet the diverse needs of a global research community.

However, the ‘reader pay all’ or traditional model needs changes to survive and thrive in the advent of OA.

What changes would I suggest?

–          Charge for individual journals rather than bundles, with a sensible pricing model

–          Pay editors and reviewers fairly for their work

–          Share revenue with authors: for example, after each year work out the profit a journal has made and share a percentage of the profit with authors (profit/number of articles)

In addition, by revenue sharing these publishers will incentivize authors and the larger academic community to continue reading, publishing and reviewing articles for their journals.

While this will obviously reduce the profit to publishers and their shareholders, it will perhaps make them sustainable. Sustainability and survival, in my view, make a sufficiently strong business case for change.

All of the above will no doubt be hideously complicated and perhaps not implementable in form suggested. I hope though, that it leads people to propose improvements that are indeed feasible.

Open Access: needs more work!

Open Access (OA) journals have made a massive impact in the world of academic publishing. A handful of such journals has grown into a massive range.open access

Much has been written about OA, the gold(authors pay the publication charges for their articles which are free to download) and green (authors can place a copy of their articles in pre-print or post-print format in online repositories operated by their universities or databases such as Arxiv, see more on the Romeo project) routes (see also Open access funding models), its power and the boycott of established publishers such as Elsevier, the Finch report  and much more, see (OASIS, We neet to talk about kevin , er, open access etc.

Clearly OA is here to stay and has a lot of power- on average open-access articles are cited 176% more often than print/paid-subscription articles (see for example, bibliography on studies on impact of open access, Swan 2010). Is it then the path to information for all, the true access to knowledge for all?

At this point it is also worth mentioning that there are some very big discipline specific factors that affect OA journals. There are reports that compare Humanities and Social Sciences and Science (HSS) with Science, Technology and Medicine (STM) journals see for example this link.

I believe that open access to research is part of the story, not all of it.

The debate on OA has focused a lot (especially with regards to STM journals) on the benefits to readers. Implicit in this is the assumption that all authors have equal access to publication in OA journals[1].

I want to discuss this assumption.

As the number of open access journals, the frequency of issues and number of articles increase, the net effect is a massive information deluge that a reader cannot cope with. The result is that readers apply selection criteria to filter out and read some journals more regularly than others. Quality of articles, impact factors and citation rates influence the selection among other factors. Thus, not all quality work and journals get the same readership.

The knock-on effect is that authors increasingly want to publish in the journals that give their work the most impact. This means journals with higher citation rates and large impact factors, large readership, and strategic readers. The competition is for eyeballs and for article space in some of these highly prized open access journals.

This is where the issue of publication charges becomes crucial.

I take the example of my research area of Optics, in which the OA journals that figure in the top 20 (ranked by 5 year Impact Factor (IF)) are Optics Express (ranked 4 with 5 year IF=3.666), IEEE Photonics Journal (ranked 13 with IF=2.320), see Journal Citation reports. All the OA journals have publication charges. The minimum charges for Optics Express and IEEE Photonics Journal are over $1000 (see links to Optics Express publication fees, IEEE Photonics Journal charges).

Publication charges can price out researchers from

1. Individuals/small groups/universities with relatively modest funding

2. Researchers from poorer countries, currency conversion rates further exacerbate the pain

Though authors can still publish in the traditional journals that do not charge a publication fee and then make their work freely available through online repositories (the green route to OA), this has problems:

1. Difference in impact factors will still affect these authors when applying for grants, jobs, promotions etc., i.e. anything competitive where journal prestige and citations are important.

2. Its not completely certain their work will get the same attention (?)  (if published in a non OA journal).

Consider for example: Optics Express has a higher number of total cites (54094, ranked 2nd by number of cites for all Optics journals and 5th by 5year IF) than its much older and also very prestigious sister journal Optics Letters (ranked 3rd by total number of cited-45759, and 7th by 5 year IF), even though Optics Express is only in its 20th year, while Optics Letters is in its 37th year of publication. Source: Journal Citation reports from Web of Science, figures taken on 5th October 2012.

3. Not all institutions worldwide have online repositories, or all disciplines suitable online archives

Thus the gold route to OA where the burden of publication charges is borne solely by the authors can also lead to inequality: some can afford to publish in the best OA journals, while some cant!

Can we afford this sort of inequality in research? Do we want it?

If we cant read the interesting, novel, unique work done by talented people from diverse backgrounds the entire research community would be poorer. We would all suffer in that if the quality of what we read is not the best possible, our own output cannot be the best possible.

To me this suggests that OA needs more work for it to truly serve the research community. We need to find a way to distribute the publishing costs so that deserving authors are not priced out of publishing and readers are not (as a result priced out) of reading such work.


[1] On his website librarian Micah VanderGrift  quotes from a study by Suber and Sutton that 83% of OA journals by learned society publishers do not charge publication fees. This figure is an overview of OA journals across disciplines, hence the scenario could be very different for individual disciplines/areas.