|Call for Abstracts||Notice prepared and disseminated to the stormwater and related sectors calling for interested parties to submit abstracts on papers they intend to prepare for the Conference.|
|Non-Refereed||A paper submitted to the Conference subject to informal review and selection by the Technical Committee.|
|Refereed||A Paper submitted to the Conference subject to formal peer review by the Scientific Review Committee.|
|Reviewer||Persons listed on the Scientific Review Committee register or Technical Committee who undertake a review of abstracts or a peer review of papers submitted for potential inclusion in the Conference program.|
|Review process||The process of abstract / paper review for potential selection for Conference. Depending on context, it may apply to Non-refereed or Refereed process, or both.|
|Technical Committee||A sub-committee appointed by the National Stormwater Association Committee made up of members from a State or National Stormwater association. By default, this is the Conference Sub-committee members, though others outside this sub-committee may be recruited to assist.|
|Scientific Review Committee||A group of appropriately qualified and / or experienced persons recruited by the Conference Program Chair to conduct formal peer reviews of Papers submitted for the conference that have been identified as “to be Refereed”.|
Stormwater Australia and its member organisations hold conferences from time to time, and generally distribute a public Call for Abstracts for consideration in line with the conference theme and program streams.
Conferences may request either abstracts for Refereed or Non-Refereed submissions, or a mixture of both.
These polices have been developed to assist in making determinations to accept or reject submissions. They are intended to outline a robust process that is procedurally fair and ensures relevant conferences presentations are of a high standard.
Call for Abstracts and Papers
A Call for Abstracts will preface any conference. It should be prepared and circulated amongst target communities of practice well ahead of the conference date. To ensure a wide diversity of contributions and to allow for new ideas and perspectives, it should also be circulated more widely (i.e. through the use of different distribution channels) and made publicly available (i.e. through websites) and should not be exclusive to the Stormwater membership.
The call for abstracts and papers should clearly
- Outline key dates for submission, review, and presentation
- Outline conference themes and requested streams for presentation
- Provide key contact details for the conference, including Conference Chair and Program Chair, and relevant conference organising personnel
- Outline submission and selection process for different submission types (e.g. refereed and non refereed submission)
- Provide evaluation criteria
- Provide specifications for final paper (e.g. formatting, referencing)
- Outline any final review process and basis for requesting modifications, caveats, clarifications or restrictions to final content and
- Outline any process for managing disputes (i.e. in the event of an adverse review).
Recommended Evaluation Criteria
The following criteria are recommended for evaluating both Refereed and Non-Refereed papers, and criteria used should be scored to allow comparative assessment:
- Consistency with the conference topics or streams: The paper must address one or more of the conference topics or streams.
- Originality: The paper is an original piece of work, be it research, experience or case studies of projects undertaken.
- Significance of the subject/project: The paper adds to the knowledge, understanding and practical application of stormwater management.
- Technical content and Methods/ Logical development, clarity and completeness: The paper is structured to clearly outline the methodology, aims, results, discussions and conclusions, and described in clear, and accessible language.
- Appropriate use of citations.
- Likely degree of interest created.
Recommended Evaluation Process
Papers marked as “To Be Refereed” should be assigned to relevant persons listed on the Scientific Review Committee Register. Assignment should be based on the match between reviewer and paper content. Papers not so marked should be assigned to committee members listed on the Technical Committee similarly.
Papers sent out for review should be marked against each of the chosen evaluation criteria using a suitably graduated scale to allow an overall score to be calculated. The evaluation process should involve multiple reviewers who are selected to represent a diversity of views, and relevant technical understanding.
The evaluation should be undertaken in effect to ‘screen’ papers to allow a balanced program to be established. It is intended to obtain sufficient information to allow this screening to occur without requiring excessive effort on the part of the submitter.
It is recognised that additional time and effort may be involved in preparing and managing refereed requests and that timelines may require a more developed abstract to be submitted for papers which are requesting this service. Under these circumstances there should be a similar process to ensure that the intent of the evaluation criteria are achieved, albeit with greater involvement of the Scientific Review Committee members.
Under all circumstances the evaluation results should be retained.
Accepting Papers after evaluation
Papers are able to be accepted after the evaluation process is complete based on the following set of decision criteria:
- Papers meet a minimum standard in each evaluation criteria
- The program should seek to achieve a balance across all advertised streams
- There are sufficient program spaces to include them
- Papers achieving a higher overall evaluation score should receive preference in each stream.
For a number of reasons the Conference Program Committee should consider accepting additional papers on a provisional or ‘wait listed’ basis. These reasons could include the following:
- It is a condition of acceptance that presenters will register to attend the conference. Some presenters may not be able to meet this commitment resulting in a paper being withdrawn
- Some presenters may not be able to complete a paper in time, and it will be withdrawn
- If, as a result of more detailed reviews that occur post acceptance the paper is withdrawn
- If, as a result of final drafting the paper does not meet acceptable standards in relation to consistency with abstract, avoidance of defamatory statements or is deemed to be overly commercially biased, the paper may be later rejected from the conference.
Outcomes of the Refereed review process should be communicated to the lead author. Specific evaluation scores should not be revealed.
It is acceptable to provide a summary of review comments, although the identities of and attributions to specific reviewers should not be identified (blind review).
The decision regarding whether to communicate specific comments lies with the Conference Program Chair and in making this they should be mindful of whether comments are likely to be taken as constructive. If they have the potential to cause offence they may be provided in a modified format (while still communicating the general intent) or not disclosed. Generally, this decision process should be consistent across all papers reviewed.
The reviewers will evaluate the paper, noting weaknesses or problems along with suggestions for improvement. Reviewer’s evaluations usually include an explicit recommendation of what to do with the paper from options provided by the journal or funding agency. Most recommendations are along the lines of the following:
- to unconditionally accept the manuscript or proposal,
- to accept it in the event that its authors improve it in certain ways,
- to reject it, but encourage revision and invite resubmission,
- to reject it outright.
Where a formal review has been requested (refereed paper) and modifications are required, the paper will be returned to the Author for update based on the recommendation of the reviewer. The revised paper is then returned to the reviewer for final approval before it will be accepted for presentation and publication.
Non-peer review papers are subject to a less formal review than refereed papers. After acceptance the authors can expect a final check of the paper prior to it being approved for publication in the program. This process will be jointly managed by the Conference Program Chair and general Conference Committee. All papers that are presenting case studies should be submitted as a Non-refereed paper.
Refereed Paper Process
Authors opting for the Refereed option can expect to undergo a rigorous and fair process consistent with international academic standards.
The review process will be managed by the Conference Program Chair, who will be responsible for the overall standard of review, and will determine any course of action should a dispute arise.
The Conference Program Chair will be supported by the Scientific Review Committee and the Conference Manager.
Conference Program Chair
With the support of the conference organisers or secretariat, the Conference Program Chair will:
- Identify suitably credentialed people to assist in the refereed paper review process
- Oversee the paper review process
- Where a potential conflict of interest is raised, work to identify suitable alternate reviewers
- Where review panels differ in their individual assessments of papers, work with the relevant reviewers to achieve a consensus result. If this is not able to be achieved, to ensure that another, independent review is undertaken.
- Where potential issues of unethical conduct, impropriety or misconduct are identified as part of the refereed paper review process, to ensure these are noted and forwarded to Conference Chair with any recommendations on how they should be investigated or handled.
- Assist the conference organisers to ensure a balanced program is put together.
Where review panel members are unable to agree on the outcomes of review it is desirable to undertake an independent, separate review however this may not always be possible. Under these circumstances the Conference Program Chair may decide to undertake their own review. A recommended process to undertake this is set out below.
- Convene a meeting with the reviewers in question to resolve the issue
- Determine if the differences in opinion of the reviewers can be overcome by including additional qualification or contextual information to the manuscript
- If the opinion of one reviewer over another is believed justified the rationale should be documented and provided to the author as part of feedback given
- Consider seeking additional clarification from the author regarding points of contention, and in doing so provide responses to the reviewers for their response.
- Consider offering to accept the paper as a non-refereed submission.
At the conclusion of any review by the Conference Program Chair their determination on outcome is final. The process and rationale taken in coming to any final determination should be noted in the event a disputed outcome is registered.
Conference Organiser/ Secretariat
The conference organisers or secretariat shall be responsible for:
- Managing the call for abstracts
- Receiving abstracts and papers
- Distributing papers and abstracts to the Scientific Review Committee on behalf of the Program Chair
- Receiving and collating feedback from the Scientific Review Committee to assist the Program Chair
- Communicating outcomes of the review process on behalf of the Program Chair.
It is not the role of the conference organisers or secretariat to make decisions on the veracity of the refereed paper process.
Scientific Review Committee
Note: these have been adapted from http://publicationethics.org/files/Ethical_guidelines_for_peer_reviewers_0.pdf
Members of the Scientific Review Committee should:
- only agree to review manuscripts for which they have the subject expertise required to carry out a proper assessment and which they can assess in a timely manner
- respect the confidentiality of peer review and not reveal any details of a manuscript or its review, during or after the peer-review process, beyond those that are released as part of conference proceedings
- not use information obtained during the peer-review process for their own or any other person’s or organization’s advantage, or to disadvantage or discredit others
- declare all potential conflicting interests, seeking advice from the Conference Program chair if they are unsure whether something constitutes a relevant interest
- not allow their reviews to be influenced by the origins of a manuscript, by the nationality, religious or political beliefs, gender or other characteristics of the authors, or by commercial considerations
- be objective and constructive in their reviews, refraining from being hostile or inflammatory and from making libellous or derogatory personal comments
- acknowledge that peer review is largely a reciprocal endeavour and undertake to carry out their fair share of reviewing and in a timely manner
- provide the Conference Program Chair with personal and professional information that is accurate and a true representation of their expertise and
- recognize that impersonation of another individual during the review process is considered serious misconduct.
- respond in a reasonable time-frame, especially if they cannot do the review, and without intentional delay
- declare if they do not have the subject expertise required to carry out the review or if they are able to assess only part of the manuscript, outlining clearly the areas for which they have the relevant expertise
- only agree to review a manuscript if they are fairly confident they can return a review within the proposed or mutually agreed time-frame, informing the Conference Program Chair promptly if they require an extension
- declare any potentially conflicting or competing interests (which may, for example, be personal, financial, intellectual, professional, political or religious), and seeking advice from the Conference Program Chair if they are unsure whether something constitutes a relevant interest.
On being approached to review any particular work, potential reviewers should inform the Conference Program Chair if:
- they work at the same institution as any of the authors (or will be joining that institution or are applying for a job there)
- they are or have been recent (e.g. within the past 3 years) mentors, mentees, close collaborators or joint grant holders with authors
- they have a close personal relationship with any of the authors
- agree to review afresh any manuscript they have previously reviewed for another Conference Program Chair as it may have changed between the two submissions and the Conference Program Chairs’ criteria for evaluation and acceptance may be different
- ensure suggestions for alternative reviewers are based on suitability and not influenced by personal considerations or made with the intention of the manuscript receiving a specific outcome (either positive or negative)
- not agree to review a manuscript just to gain sight of it with no intention of submitting a review
- decline to review if they feel unable to provide a fair and unbiased review
- decline to review if they have been involved with any of the work in the manuscript or its reporting
- decline to review if asked to review a manuscript that is very similar to one they have in preparation or under consideration at another conference
During review process reviewers should:
- notify the Conference Program Chair immediately and seek advice if they discover either a conflicting interest that wasn’t apparent when they agreed to the review or anything that might prevent them providing a fair and unbiased review
- read the manuscript, ancillary material (e.g. reviewer instructions, required ethics and policy statements, supplemental data files) and any specific instructions thoroughly, getting back to the Conference Program Chair if anything is not clear and requesting any missing or incomplete items they need to carry out a full review
- notify the Conference Program Chair as soon as possible if they find they do not have the expertise to assess all aspects of the manuscript waiting until submitting their review to do this may will unduly delay the review process
- not involve anyone else in the review of a manuscript, including junior researchers they are mentoring, without first obtaining permission from the Conference Program Chair
- where assistance is provided, the names of any individuals who have helped them with the review should be included with the returned review so that they are associated with the manuscript in the Conference Program Chair’s records and can also receive due credit for their efforts
- keep all manuscript and review details confidential
- contact the Conference Program Chair if circumstances arise that will prevent them from submitting a timely review, providing an accurate estimate of the time they will need to do a review if still asked to do so
- in the case of double-blind review, if a reviewer suspects the identity of the author(s) notify the Conference Program Chair if this knowledge raises any potential conflict of interest
- notify the Conference Program Chair immediately if they come across any irregularities, have concerns about ethical aspects of the work, are aware of substantial similarity between the manuscript and a concurrent submission to the conference or suspect that misconduct may have occurred during either the research or the writing and submission of the manuscript. Reviewers should, however, keep their concerns confidential and not personally investigate further unless the Conference Program Chair asks for further information or advice
- not intentionally prolong the review process, either by delaying the submission of their review or by requesting unnecessary additional information from the Conference Program Chair or author(s).
Final Vetting of Papers
Final papers selected for conference proceedings can expect to be subject to a ‘final vetting.’
In the case of Refereed papers, the Conference Program Chair must be satisfied that the paper satisfactorily addresses all issues raised in the review process, and, in doing so, may rely on the Scientific Review Committee to assist in making this determination.
In the case of Non-refereed papers the conference programmers must be satisfied that the final paper:
- meets the general promise of the submitted abstract, or is likely to provide equal or greater interest in the chosen stream
- does not expose the conference organisers to excessive risk of libel or defamation by unfavourably identifying any product, practice, process, company or person in specific terms that would allow them to be identified
- is of sufficient general interest and avoids oblique references to products, companies or proprietorial know how that could be considered by a reasonable person as advertorial in nature.
In consultation with the Conference Program Chair the Conference Committee may convene a separate panel of industry practitioners to assist in making these determinations.
If these conditions are not able to be met, consideration should be given to requesting modifications to the paper or rejecting it.
Where an author feels that the outcomes of the review process is not fair or reasonable they should have a right of appeal to the Conference Chair.
In this instance the complaint should be put in writing clearly stating the grounds on which a review is being requested.
In determining whether to entertain a review the Conference Chair (in consultation with the Conference Program Chair) should use the following as a guide when considering the subject matter of the manuscript:
- Where the complaint relates to the qualifications of reviewers, they should be satisfied that the reviewers were suitably qualified. If not, the use of an independent reviewer should be considered.
- Where the complaint relates to bias or conflict of interest amongst reviewers, the rationale behind this should be examined. A conservative approach should be taken, and if there is a prospect of bias then an independent review should be considered.
Where the complaint relates to the actions of the Conference Program Chair, before deciding to take action, the Conference Chair (in consultation with the Conference Committee) would need to be satisfied that:
- the Conference Program Chair hadn’t largely adhered to the requirements of this policy and
- the complaint was broadly representative of the process and not an isolated incident.
Where action was deemed necessary the Conference Chair would need to consider options in consultation with the Conference Committee. Actions would need to balance the integrity of the program, while remaining respectful of the effort exerted by both authors and reviewers.