I'm doing a bunch of reviewing now. People have decided I'm a good person to send philosophy papers submitted to AI meetings. Maybe. But the below is about engineering and science as well as philosophy writing.
At the meta level, the most important thing a paper needs to demonstrate is that it is useful. This is done first and foremost by showing that something similar (or better) hasn't been written before. In engineering, this is done by reviewing the state of the art, then demonstrating a new system that goes beyond that, whether quantitatively or qualitatively. "Better" can be better performance, which can usually be measured directly, but can also be greater reliability, greater transparency, greater ease of build, greater simplicity. Statistics are often used to give evidence that something is better or at least equivalent to the state of the art in its performance, but if there's a qualitative advance that's not necessary.
My understanding is that for philosophy, the demonstration of novelty is done by recounting individual authors' currently vying for the leading perspective on the issue. This recounting must be careful enough to demonstrate not only knowledge of the argument but respect for it. Then the main contribution of the paper is shown either to complement, extend, or dispute each of the salient existing positions. It is perfectly fine to fully align with an existing author's stated position, so long as that position is not universally accepted and there is a novel contribution in undermining one of the competitor positions.
So to be clear, any kind of paper should start with an introduction to the topic area and the novel contribution, then review the state of the art, then present the new contribution. The difference is that the "results" section of an engineering or science paper is replaced by the discussion section just described.
I didn't really mention science earlier. Science is just the subset of philosophy for which you can demonstrate evidence rather than only argument. So the structure of the paper is that the contribution is described and demonstrated much as for an engineering paper, but then there is also a discussion section that follows showing the consequences for other contending theories. I suppose engineering papers might also be benefited by discussing the competing approaches and which should now be extended or abandoned, but to be honest I don't see very many of those papers.
In theory, peer review involves an expert with perfect knowledge of a field who knows whether the paper being reviewed is making a contribution by whether the reviewer has learned something novel. But in practice, no one can know everything, so a convincing review then discussion of the state of the art is absolutely essential. A lot of expertise is actually recognising how convincing that review and discussion are. Because human knowledge is vast, reviewer is likely to know at least a few papers that any author won't have known, but that in itself doesn't show a paper shouldn't have been written. If enough papers presently considered to be state-of-the-art are addressed and improved by the present paper, then that paper deserves to be published.
Update: this is part of a series; thanks CJ for reminding me to link these. See also
At the meta level, the most important thing a paper needs to demonstrate is that it is useful. This is done first and foremost by showing that something similar (or better) hasn't been written before. In engineering, this is done by reviewing the state of the art, then demonstrating a new system that goes beyond that, whether quantitatively or qualitatively. "Better" can be better performance, which can usually be measured directly, but can also be greater reliability, greater transparency, greater ease of build, greater simplicity. Statistics are often used to give evidence that something is better or at least equivalent to the state of the art in its performance, but if there's a qualitative advance that's not necessary.
My understanding is that for philosophy, the demonstration of novelty is done by recounting individual authors' currently vying for the leading perspective on the issue. This recounting must be careful enough to demonstrate not only knowledge of the argument but respect for it. Then the main contribution of the paper is shown either to complement, extend, or dispute each of the salient existing positions. It is perfectly fine to fully align with an existing author's stated position, so long as that position is not universally accepted and there is a novel contribution in undermining one of the competitor positions.
So to be clear, any kind of paper should start with an introduction to the topic area and the novel contribution, then review the state of the art, then present the new contribution. The difference is that the "results" section of an engineering or science paper is replaced by the discussion section just described.
I didn't really mention science earlier. Science is just the subset of philosophy for which you can demonstrate evidence rather than only argument. So the structure of the paper is that the contribution is described and demonstrated much as for an engineering paper, but then there is also a discussion section that follows showing the consequences for other contending theories. I suppose engineering papers might also be benefited by discussing the competing approaches and which should now be extended or abandoned, but to be honest I don't see very many of those papers.
In theory, peer review involves an expert with perfect knowledge of a field who knows whether the paper being reviewed is making a contribution by whether the reviewer has learned something novel. But in practice, no one can know everything, so a convincing review then discussion of the state of the art is absolutely essential. A lot of expertise is actually recognising how convincing that review and discussion are. Because human knowledge is vast, reviewer is likely to know at least a few papers that any author won't have known, but that in itself doesn't show a paper shouldn't have been written. If enough papers presently considered to be state-of-the-art are addressed and improved by the present paper, then that paper deserves to be published.
Update: this is part of a series; thanks CJ for reminding me to link these. See also
Comments
Personally, I think it is also important to replicate previous work, which is very tricky if you want a paper that is publishable also.
It is also important to think about Type 1 Errors and Type 2 Errors.
Here is an important paper that was not judged to be publishable.
http://www.wired.com/wiredscience/2009/09/fmrisalmon/
... research lab put a dead salmon into an MRI machine, showed the dead fish pictures, and then asked it what it thought ... paper was turned down by several publications ...
Scanning Dead Salmon in fMRI Machine Highlights Risk of Red Herrings
By Alexis Madrigal | September 18, 2009 | 5:37 pm | Categories: Brains and Behavior