California Department of Transportation

This site is produced by the Caltrans Division of Environmental Analysis (DEA) and Division of Local Assistance (DLA).

Adobe Acrobat Reader is required to view and print PDF files. Adobe Acrobat is required to edit online forms and save them to your computer.

acrobat reader

logo

Last Updated: Friday, February 3, 2012 11:30 AM


Exhibit 2.14:GUIDELINES FOR Peer Review of CULTURAL RESOURCES Reports

Exhibit 2.14 PDF version here.

Review of cultural resources documents (studies, reports, findings of effect, and other types of documentation) for quality control is an essential part of the environmental process. The requirement for quality control review of environmental documents under CEQA/NEPA is described in the Division of Environmental Analysis (DEA) Chief’s January 13, 2003 memo to District Environmental Directors, entitled “Written Certification of Environmental Document Quality Control Reviews.” Under the Section 106 Programmatic Agreement (Section 106 PA), Caltrans as assigned by FHWA is responsible for quality control of cultural resources documents. Caltrans PQS who meet the appropriate requirements of Section 106 PA Attachment 1 must peer review all documents prepared under the Section 106 PA.

In accordance with Section 106 PA Stipulation XV, Caltrans will not transmit documentation prepared under the Section 106 PA to FHWA or SHPO until it has been reviewed and approved by Caltrans PQS.

Benefits of Conducting Peer Reviews

Conducting peer reviews of cultural resources studies, reports and documentation has been standard Caltrans policy for many years. Benefits include improving the quality of all levels of documentation, sharing information between authors and reviewers, promoting relative consistency in style and content, and avoiding delays caused by FHWA or SHPO raising questions or rejecting findings.

While no one enjoys receiving criticism, however constructive, few people can objectively critique their own work. Peer reviewers provide authors with a valuable service, offering suggestions to assist in producing a professionally sound, technically correct, clearly written final document. Reviews can identify problems at a stage when the author can make changes and avoid later, more public criticism. Peer reviews clearly benefit both the process and the individuals involved.

Who Conducts Peer Reviews

Any appropriately qualified Caltrans PQS, either Headquarters or District staff, may conduct peer reviews of cultural studies prepared by coworkers or consultants. A professional in the same discipline as the author, at the same or higher PQS level or equivalent, should conduct the primary peer review. Additionally, work completed by staff certified at the Co-Principal Investigator level must be reviewed by staff certified at the Principal Investigator level for the appropriate discipline, in accordance with Section 106 PA Attachment 1. See Chapter 2 Section 2-5.5 for more specific information on PQS who are certified to do peer reviews and reviews for approval.

Either District PQS, or upon request, Cultural and Community Studies Office (CCSO) PQS, may peer review documents. When CCSO PQS are requested to conduct peer reviews, they will give them the highest priority in work assignments, completing reviews within 15 working days, preferably much less, of receiving the request. Districts may arrange routine reviews through the CCSO Section 106/PA Coordination Branch Chief.

Submit requests for review through supervisors, not directly to peers, as reviews constitute work assignments.

In general, it is beneficial to have a variety of seasoned staff review the work of newer staff, to help them learn standard Caltrans procedures. It is also worthwhile for newer staff to seek reviews from staff in different offices for broader perspectives and to become familiar with the range of staff expertise available.

Authors sometimes are more comfortable requesting reviews by close colleagues who might be reluctant to criticize, who feel obliged to "support" them, or whose perspectives closely mirror their own, but such reviews may have limited usefulness. Reviewers who offer honest comments and different perspectives provide authors more of a service and a learning experience leading to better documents. Sometimes it is useful to request that professionals in other disciplines also review a work for potential pitfalls or red flags that could be a concern for review agencies.

Reviewers' Responsibilities

Reviewers must maintain professional objectivity and not allow personal feelings about the author, the project, or the resources to influence the tone or content of the review. At the same time, reviewers should not suspend professional standards to avoid offending someone. They are required to assert their professional judgment on any issue that may be critical to the acceptability of the document. Any criticism should be presented in a firm but helpful and respectful manner.

Reviewers should examine documents to determine:

  • Does the document fulfill its intended purpose?
  • Are the findings reasonable, backed by logic and supporting evidence, and presented clearly?
  • Is the document adequate for review agency concurrence?

Peer reviewers should give these reviews a high priority and provide a prompt turnaround, 15 working days or less, unless requested otherwise.

Guiding Principles

Response memos should provide all important comments, both positive and negative, relating to the acceptability of the document, accuracy of content, and agreement with findings. Make a serious effort to recognize good work and offer genuine compliments that recognize positive aspects of the document. Comments will be more readily accepted when they are presented in a balanced review.

Present any suggestions for improvement in a friendly and constructive manner. Be extremely careful in the tone of the response memo, in the wording of any criticism and the context in which it is expressed. Avoid sarcasm, officiousness, personal criticisms, nitpicking, or imposition of personal style. It is never appropriate to challenge a peer's professionalism, intelligence, or standards. In general, phrase all comments courteously, with sensitivity and awareness as to how authors will receive them. Consider how you might react to the same comments. A peer antagonized may someday be in the position to return the favor.

Level of Comments

Carefully consider the level of comments, how detailed they need to be, taking into account the type of document, the resources or issues involved, the experience of the author, and whether prepared in-house or by a consultant.

Concentrate primarily on issues of substantial concern. Omit discussion of professional differences of opinion unless critical to the document's acceptability. Note any factual errors or loose ends and offer suggestions for improvement if needed. Mark minor comments, such as misspellings, typos, or grammatical errors, in the text only; the response memo might simply identify a need for more careful proofreading. Calling out repeated occurrences of minor errors page by page in the comment memo comes across as nagging.

On staff-authored documents, it can be useful to comment on clarity, format, and presentation, including correcting grammatical or spelling errors, knowing that these documents reflect on Caltrans as a whole. Constructive comments can help encourage staff toward long-term improvement.

Review consultant-prepared documents primarily to determine their adequacy for the purpose intended. Judge the findings for acceptability, whether they will accomplish the goal of compliance with the appropriate laws and regulations. The primary concern should be for identifying what have been called "fatal flaws,” that is, elements that could cause reviewing agencies to reject the documents. Concentrate on such substantive comments, and keep minor, non-substantive comments separate, noting that they are simply suggestions for improvement. Imperfect consultant-prepared documents may be accepted without requiring changes to be made if document inadequacies can be addressed in the transmittal memo.

What to Look for as a Reviewer

In general, reviewers should keep the following questions in mind:

  • Is the undertaking clearly described?
  • Is the Area of Potential Effects (APE) adequately described, mapped, and justified?
  • If a Study Area was used, is it distinguished from the APE?
  • Are all ground-disturbing activities, including utility relocation, staging areas, etc., included in the Direct APE?
  • Are all areas subject to indirect effects included in the Indirect APE?
  • Are identification and survey efforts adequate?
  • Are historic contexts adequately developed for evaluations?
  • Are all evaluated properties shown on APE maps?
  • For work done by consultants, are their qualifications provided?
  • For work done by Caltrans staff, are their PQS levels specified and appropriate for actions taken under the Section 106 PA?

For each National Register eligible historic property, check to make sure the following elements are included:

  • Criteria under which found eligible.
  • Justification for eligibility.
  • Level of significance.
  • Period of significance.
  • Contributing and non-contributing elements.
  • National Register boundaries, both described in the text and shown on maps, including the APE map and the DPR 523 map.

For effect findings, be sure the following are included:

  • Historic properties adequately described for understanding effects.
  • Project effects described for each historic property.
  • One effect finding given for the undertaking as a whole.
  • Any special conditions such as ESAs adequately described to justify the effect finding.

Exhibit 2.17 provides guidance on describing National Register eligibility and effect findings.

Format

Peer review comments should be written, to document the review findings, but the format can vary, depending on circumstances. If the original request was informal, handwritten notes or an informal memo may suffice; e-mail requests can usually be answered by e-mail; formal memos usually receive formal responses in memos signed by a branch or office chief.

Authors' Responsibilities

It is a mark of professionalism to be able to receive criticism graciously as much as it is to give it tactfully. Try to approach comments with an open mind, without getting defensive, recognizing that reviewers expend valuable time and effort in order to help the author. Their assistance is intended to lead to improvements; therefore, reviewers' suggestions should be taken seriously, and relevant comments should be incorporated.

In considering comments, it often can be helpful to meet with a reviewer if clarification is needed or in order to gain a better understanding of issues that were raised. Even a comment that misses the mark can reveal an unclear area needing correction, perhaps identifying a problem if not a solution. In any case, authors must strive to maintain objectivity and not allow personal feelings to influence reaction to comments or assessment of comments' validity.

Remember: peer review comments are advisory only. Authors bear the responsibility for evaluating comments honestly and for determining where changes need to be made. If a major disagreement regarding a document's ultimate acceptability cannot be resolved at the staff level, refer the impasse to the respective supervisors. In the end, however, final responsibility remains with the authors and the authors’ management.

Peer reviewers’ names should be kept on record and comments retained in the project files. Transmittal memos also may name the peer reviewers. As peer reviewers offer advice only, however, and have no say over the final document, they should not be asked to sign the title page.

Conclusion

The purpose of cultural resource studies is Caltrans' compliance with federal and state laws and regulations. Reports and other documents are written to communicate findings on cultural resources in the project area clearly and compellingly to managers and review agencies. These efforts are not conducted to impress one’s peers or to achieve personal or academic goals. Instead, they are intended to produce competent, professional documents that will be adequate for fulfilling compliance responsibilities. Peer reviews are an essential part of that process.