Effects of Training for Inexperienced Surveyors on Data Quality of Tree Diameter and Height Measurements

Due to the large number of sample plots and variables to be measured, inexperienced surveyors are expected to take field measurements in National Forest Inventories (NFIs). However, very little information exists on the data quality that can be expected from inexperienced surveyors given different levels of training. We evaluated the quality of data produced by inexperienced undergraduate students when measuring the most fundamental variables: tree diameter using a diameter tape and height using an ultrasonic Vertex III hypsometer. We found that a single training session on how to use the instruments and how to reduce measurement errors was insufficient for inexperienced surveyors to achieve measurement quality objectives (MQOs). Providing a single feedback of control team measurements significantly improved data quality, except in the measurements of tree height of broad-leaved trees, but additional feedback did not contribute to further improvement. We propose that field training courses for inexperienced surveyors incorporate a one-day exercise with feedback instruction.


Introduction
National Forest Inventories (NFIs) have been carried out globally in many countries.The number of variables to be measured in NFIs has steadily increased, incorporating not only living trees but also lesser life forms and dead woody materials to meet national and international requirements for carbon and biodiversity reporting (Corona and Marchetti 2007, Westfall and Woodall 2007, Magnussen et al. 2007).Like other large-scale and long-term environmental monitoring programs, NFIs need to use several field teams in any given year, and teams are likely to change over time because of the sample sizes (Ghosh et al. 1995) and number of variables involved.Thus, field teams cannot be composed purely of experienced surveyors with extensive training; new, inexperienced surveyors will inevitably need to be recruited to enable field measurements to be completed (Westfall and Woodall 2007).In such cases, initial training of inexperienced surveyors is essential before they can begin taking field measurements within an NFI program.However, there is limited information on the level of expertise needed for surveyors to achieve measurement quality objectives (MQO) (Pollard et al. 2006).
There are three types of variables that can be obtained in the field; measured, identified or visually estimated.A 'measured' variable is obtained by an instrument such as calipers or a diameter tape to assess diameter at breast height (DBH).An 'identified' variable is determined by visual inspection (e.g., of bark, flowers and leaves to identify species).A 'visually estimated' variable is an ocular estimation (e.g., percent cover of the ground vegetation, forest type or vertical structure) (Winter et al. 2008).It is known that a high level of expertise is needed for accurate identification of species (Scott and Hallam 2002) and it is difficult even for experienced surveyors to obtain precise values of visually estimated variables (e.g., Ghosh et al. 1995, Ferretti et al. 1998).It can be assumed that inexperienced surveyors will achieve higher accuracy for 'measured' variables than 'identified' and 'visually estimated' ones and, therefore, it would be logical to assign them to field measurements of 'measured' variables.However, the level of precision that can be achieved by inexperienced surveyors and the optimum level of training needed to achieve adequate data quality of 'measured' variables are unknown.
Of the many variables recorded in NFIs, the most fundamental are DBH and tree height.They are measured not only in NFIs (Winter et al. 2008) but also in research projects on forest ecology and management for estimating stand volume, biomass and size structure.These variables are commonly measured using instruments, so a certain level of measurement accuracy can be expected, even from inexperienced surveyors.There have been experimental studies on measurement errors of these variables (Barker et al. 2002, Elzinga et al. 2005, Johnson and Haag 1985, Myers 1961, Olume 1980).The quality of data generated by operational field measurement teams has been evaluated using blind re-measurements by a control team in some NFI programs: the Change Monitoring Inventory (CMI) of British Columbia, Canada (Resources Inventory Committee 2007), the Forest Inventory and Analysis program (FIA) of the USDA Forest Service (Pollard et al. 2006, Westfall andWoodall 2007) and the Swiss and Japanese NFIs (Kaufmann andSchwyzer 2001, Kitahara et al. 2009).However, none of these studies provided information on data quality from inexperienced surveyors or the effects of training on the performance of such surveyors.In addition, a relatively new hypsometer, the ultrasonic Vertex III, has recently become widely used for research (e.g., Lindgren 2000, Monty 2008, Watt et al. 2005) and NFI measurements (Kitahara et al. 2009) and is regarded as being quicker to use and as or more accurate than traditional instruments such as a Blume-Leise or relascope (Bozic et al. 2005).However, very few studies have quantified measurement errors from the Vertex III (Kitahara et al. 2009).
The aim of this study was to evaluate data quality of DBH and tree height measurements by inexperienced surveyors under different levels of training.We used data from an undergraduate student training course in which a diameter tape and Vertex III hypsometer were used.During the field exercise, three different levels of instruction were given to the inexperienced surveyors, and their data quality was determined from independent re-measurements made by a control team.We finally propose a training procedure for inexpe-rienced surveyors engaged in NFIs to achieve common MQOs and suggest a measure for further improvement of data quality.

Field Training Course
The field training course for undergraduate students of the Department of Forest and Forest Products Sciences, Kyushu University, Japan was carried out in August 2006 in Shiiba Research Forest of Kyushu University (32°22´N, 131°09´E).In total, 15 trainees were divided into three field teams (Team_A, Team_B, Team_C); none of the trainees had any previous experience in forest inventory.Trainers were staff of the Laboratory of Forest Management, Kyushu University, all of whom were highly experienced in taking field measurements for research and education purposes, including serving as the control measurement team to check the quality of data from operational field teams in the Japanese National Forest Inventory in 2005 and 2006 (Kitahara et al. 2009).
For this training course we established 24 plots randomly on a 50-m by 50-m grid.Fifteen plots were composed of even-aged conifer plantations (nine of Cryptomeria japonica and six of Chamaecyparis obtusa), eight of natural broadleaved forest with main species of Fagus crenata, Quercus crispula, Sapium japonicum and Pterocarya rhoifolia, and one contained a mixture of Cryptomeria japonica and broad-leaved trees.For each plot, the sample trees to be measured were selected based on the Bitterlich point sampling method using the Spiegel relascope (Avery and Burkhart 2002) and were then numbered.The total sample number of trees measured was 235 conifers and 56 broad-leaved trees.The minimum units of the measurement were 1 mm for DBH and 0.1m for the height.The ranges (mean) of DBH and tree height were 4.9-47.3(20.1) cm and 6.2-17.5 (12.9) m for conifer and 3.6-65.5(25.1) cm and 4.2-22.7 (13.2) m for broad-leaved, respectively.
To evaluate the data quality of measurements made by the field teams, all trees were independ-ently re-measured by a control team composed of the trainers.The control team performed careful and unhurried measurements to reduce measurement errors.For DBH measurements, the 1.2-m height from the ground for the tape position was carefully determined for each tree using a 2-m pole, as per the field manual of the Japanese NFI.When one surveyor measured DBH using a diameter tape, another person carefully checked whether the tape was set perpendicular to the stem axis and was tight and not twisted.The Vertex III hypsometer uses ultrasonic pulses together with a transponder fixed to the target tree.The instrument measures the distance, angle and horizontal distance to the transponder and displays tree height after taking a reading by pointing at the tip and base of the tree (Husch et al. 2003, Bozic et al. 2005).In this study, the transponder was positioned at the breast height measured using a pole.The viewing points to look at the tree tip were carefully selected to achieve an appropriate distance from the measured tree; tree height measurements were repeated three or four times for each tree and only the average of these measurements was recorded and used as tree height.
We carried out three levels of training for the three field teams over three days.The first level of training involved instructing the inexperienced surveyors in how to use the instruments, the diameter tape for DBH and Vertex III for tree height.All surveyors practiced on one or two trees in the Shiiba Research Forest office yard.During this initial instruction, the trainer explained to the surveyors how to reduce measurement errors.For DBH measurement, instructions were given to the trainees to 1) place the tape at a height of 1.2 m from the ground on the upper slope side of the measured tree, 2) place the tape on a plane perpendicular to the tree axis and 3) apply adequate medium tension to the tape.For tree height measurement, the instructions were given to 1) calibrate the Vertex III, 2) place the transponder at a height of 1.2 m and 3) ensure there is adequate distance from the tree to view the tree tip.To decide the 1.2-m height for DBH and tree height measurements, the field manual of the Japanese NFI requires that a measurement tape or pole has been used for each tree.However, the operational field teams of the NFI were unlikely to follow this rule, choosing instead to determine the 1.2-m height from a position on their body primarily measured but then based on experience (Kitahara et al. 2009).Therefore, this study followed this custom; the trainees were asked to remember the 1.2-m height position on their bodies using a measurement pole before starting the field measurements.After the initial instruction, the field teams carried out field measurements for almost one third of the total 24 plots, with different plots being measured by different field teams.Just after the completion of field measurements, the discrepancy between the field teams and the control team was evaluated.
The second level of training consisted of informing each field team of the data quality obtained from the first training.A histogram of the discrepancy between the field teams and the control team along with basic statistics on the average differences and standard deviations of the differences were shown to the field teams.The reasons for the discrepancy were then discussed so that the field teams might improve their measurement techniques and data quality.For DBH measurement, the teams were informed that placing the tape above the specified breast height (1.2 m) results in underestimation of DBH, and that overestimation results from fixing the tape on the stem or trunk on a plane non-perpendicular to the tree axis, with weak tension and/or in a position lower than the correct position.For tree height measurement, it was reiterated that there should be ample distance to view the top of the measurements and that the transponder should be positioned at a height of exactly 1.2 m.After this feedback instruction, the field teams carried out field measurements on about half of the remaining plots (one third of the total).The measurement errors were calculated to evaluate the effects of this second level of training.
The third level of training involved repeating the second level of training.That is, feedback instruction of the results from the second level training and field measurement of the rest of the plots.

Data Analysis
We used the paired data between the control team and each of the field teams to evaluate data quality of the inexperienced field teams under different levels of training.We calculated difference (Dif), defined as the field team measurement minus the control measurement, percentage difference (PDif), defined as absolute difference divided by the control team measurement, and standard deviation (SD) of the differences, a measure of random measurement errors (Solberg andStrand 1999, Kaufmann andSchwyzer 2001).The statistical significance of differences between Dif and PDif of the first and second levels of training and Dif and PDif of the second and third levels of training was determined by the Wilcoxon sign test (Dobbertin et al. 2004, Bussotti et al. 2003) and for SD by the F-test.
We compared our data to the measurement quality objective (MQO) standards adopted in other monitoring programs and to the measurement errors obtained from the previous experimental studies.The MQO tolerance in the FIA program is defined as the maximum acceptable difference between independent measurements of individual trees (± 0.25 cm for DBH and ± 10% for tree height), and the MQO standard is the minimum acceptable proportion of measurements that are within the required tolerance (95% for DBH and 90% for tree height) (Pollard et al. 2006, Westfall andWoodall 2007), which we refer to as the compliance rate.In the case of the CMI, the tolerances themselves are used as MQOs, i.e., average values of PDif of < 2% for DBH and < 3% for tree height (Resource Inventory Committee 2007).

Results
Table 1 and Fig. 1 show the deviations of the field team measurements from the control team measurements for DBH for each of the three levels of training.The measurements after the first training tended to involve overestimation of DBH for both conifers and broad-leaved trees.This overestimation was significantly reduced from the first to the second measures, except for the broadleaved trees of Team_A, but not from the second to third measurements for all teams and tree types.The PDif was also reduced significantly from the first to second measurements except for broadleaved trees of Team_C but not from the second Table 1.Deviations of the field team measurements from the control team measurements of DBH undertaken with three levels of training, denoted as 1st, 2nd, and 3rd.Conif = coniferous trees, Broad = broadleaf trees, n = sample number of trees, Dif = difference, SD = standard deviation, PDif = percentage difference, Compliance rate = compliance rate with the MQO tolerance (± 0.25 cm error) adopted in the FIA program (Pollard et al. 2006).Asterisks on values for the 2nd-or 3rd-level training indicate significant differences (p < 0.05) between the indicated value and the value for the preceding level of training using the Wilcoxon sign test for Dif and PDif and F-test for SD.
Table 2. Deviations of field team measurements from the control team measurements of tree height undertaken with three levels of training, denoted as 1st, 2nd, and 3rd.Abbreviations are the same as in Table 1.Compliance rate = compliance rate with the MQO tolerance (± 10% error) adopted in the FIA program (Pollard et al. 2006).
to third measurements for all the cases.The average PDif for DBH in the first measurement was larger than 2% for each team and sub-group of trees; 2% is the measurement quality objective (MQO) standard adopted in the CMI in British Columbia, Canada (Resources Inventory Committee 2007).This MQO was achieved in the second and third measurements except by the broad-leaved trees of Team_C (Table 1, second measurement: 2.3%).The compliance rate of the MQO tolerance (± 0.25 cm) adopted in the FIA program (Pollard et al. 2006) increased from the first to second measurements for all groups and tree types (Table 1).From the second to third measures, however, the compliance rate was relatively unchanged for conifers and decreased for the broad-leaved trees of two teams.The compliance rate of pooled data for all teams was around 60% even after the second and third measurements (Table 1), much lower than the MQO standard of 95% and the average compliance of 89% achieved in the control survey of the FIA program between 2000 and 2003 (Pollard et al. 2006).
The values of SD for DBH measurements tended to fall significantly from the first to second measures, except for broad-leaved trees of two of the teams (Table 1).On the other hand, the changes from the second to third measurements were relatively small for conifers and differed among the teams for broad-leaved trees.The SD values pooled for all teams changed from 0.6 in the first measurement to 0.4 in the second and third measurements for conifers and from 0.8 to around 0.5 for broad-leaved trees.These values achieved following the second and third training exercises are similar to or even smaller than the values previously reported for experienced surveyors, e.g., 0.20-0.80(Barker et al. 2002), 0.47 (Elzinga et al. 2005) and 0.81 (Kaufmann and Schwyzer 2001).Significant changes in Dif and PDif of the tree heights were found for only a few cases (Table 2, Fig. 2): Dif for broad-leaved trees of Team_A and Team_B from the first to second measurements and of Team_B from the second to third measures; and PDif for conifers of Team_A from the first to second measurements and of Team_C from the second to third measures.The average PDif values were generally much higher for broad-leaved trees than for conifers, with the pooled values being 3.6% for conifers and 8.7% for broad-leaved trees.The average PDif in the first measurement was larger than 3%, the MQO standard adopted in the CMI (Resources Inventory Committee 2007), for all cases, but the results after the second and third training sessions were closer to this standard of 3% for conifers.The rate of compliance with the MQO tolerance (± 10%) adopted in the FIA program (Pollard et al. 2006) slightly increased over the successive training levels for conifers, with the FIA standard of 90% compliance rate being achieved after the first level training by Team_C and after the second level of training by the other teams.On the other hand, the compliance rates for broad-leaved trees did not increase consistently over the training levels and were much lower for conifers than the 90% standard, except for the third level measurements of Team_C.
For conifers, the SD for tree height decreased significantly from the first to the second measurements consistently for all teams, but only slightly decreased from the second to third measurements in Team_A and even increased in Team_B (Table 2).For broad-leaved trees, there were no consistent changes with training level.The SD values for conifers pooled for all teams decreased from 1.0 in the first measurement to 0.7 in the second and third measurements, which is similar to the levels achieved by experienced surveyors.From five experimental studies in Japan using various types of hypsometers, excluding the Vertex III, an average SD of 0.80 within a range of 0.46-1.4was achieved (Ishikawa and Ito 2002, Miyamoto et al. 1998, Naito 1990, Shibata et al. 1999, Yoshida 1991).However the pooled SD value for broad-leaved trees remained high, around 2.0, even for the second and third levels of training, though this compares favorably with the value of 2.3 obtained from a control survey of the Swiss NFI (Kaufmann and Schwyzer 2001).

Discussion
This study quantified data quality from DBH and tree height measurements made by inexperienced surveyors, who could be expected to contribute to the field measurements of large scale forest inventory programs such as NFIs.The training of field surveyors is an important task in a quality assurance program needed for any long-term and large-scale environmental monitoring program (Ferretti et al. 1998), especially when a new, inexperienced surveyor is involved.We found the effects of training levels on data quality differed between DBH and tree height and between conifers and broad-leaved trees.
For DBH, providing only initial training in how to use the instrument (a diameter tape) was not enough to produce data that achieves the MQO standard of < 2% for average PDif.On the other hand, the second level of training, feedback instruction using re-measurement data from the control team, significantly improved both PDif and SD, with the MQO of 2% being achieved except for broad-leaved trees of one team.However, we found that additional feedback instruction, a third level of training, contributed little to data quality, indicating that only one iteration of feedback instruction is enough to improve DBH measurements.The overestimation found in the first level of training in this study was not consistent with the results from operational field teams of the Japanese NFI, who tended to underestimate DBH compared with control measurements made by the same control surveyors used in the current study (Kitahara et al. 2009).The reason for this discrepancy is unknown, but the underestimation by experienced surveyors in the NFI can result from positioning the tape above the breast height of 1.2 m, while the overestimation by inexperienced surveyors may come from placing the tape non-perpendicular to the tree axis, weak tape tension and/or in a lower position on the trunk (Kitahara et al. 2009).
The MQO standard for DBH adopted in the FIA program, 95% compliance rate within ± 0.25 cm of control team measurements, was not achieved even after the second level of training in this study, implying that this standard may be too stringent for inexperienced surveyors under the measurement protocol of the Japanese NFI used in this study.We believe that marking the breast height or ground level in the Japanese NFI could further improve the DBH measurement, as we already suggested in the previous study (Kitahara et al. 2009).This would be in line with the FIA standard, as done by the US FIA, Swiss NFI and Canadian CMI program.
The second and third training sessions did not improve PDif and the rate of compliance with the FIA standard as much for tree height as they did for DBH.For conifers, however, a 90% compliance rate within 10% tolerance adopted in the FIA program was achieved by all teams after the second level of training, and the average PDif was close to the CMI standard of 2% after the second level.In addition, the second level of training significantly reduced the SD values, a measure of random error, to the level achieved by experienced observers in previous studies, but the third level did not consistently improve the errors any further.These results indicate that, for conifers, one feedback session is enough to improve data quality, including for tree height measurements, and that initial instrument-use training alone is not enough.
We found that the major problem is in tree height measurements of broad-leaved trees; no consistent improvements were found with successive levels of training, and the inexperienced surveyors could not achieve the MQO standards of the FIA program and CMI or the SD values of the experienced surveyors from previous experimental studies.This problem may relate to the inherent difficulty of measuring tree height of broad-leaved trees (Kitahara et al. 2009).The main stems of broad-leaved trees are often not standing vertically but leaning.Since all hypsometers assume that trees are vertical, the height of trees leaning away from an observer will be underestimated while that of trees leaning toward an observer will be overestimated (Husch et al. 2003, Philip 1994).This error can be reduced by applying an adjustment if the angle of lean is measured (Husch et al. 2003, Philip 1994).However, the field guide of the Japanese NFI does not address the issue of leaning trees, so neither the field teams nor the control team measured the angle of lean in this study.Further research is needed on the effect of correcting measurement errors of leaning trees.
This study focused on the initial training required for inexperienced surveyors before they start taking field measurements for long running NFIs.However, it is unknown whether inexperienced surveyors who receive initial training and a feedback session can maintain levels of data quality over the long term.Indeed, this study found a few cases in which the random error (SD) of DBH measurements increased from the second to third levels of training (Table 1).In addition, studies reveal that even experienced surveyors cannot necessarily always achieve MQOs during NFI measurements (Kitahara et al. 2009, Pollard et al. 2006, Westfall and Woodall 2007).Therefore, we conclude that regular checking of data quality through blind re-measurements is crucial, even for trained surveyors, if MQOs of field measurements are to be consistently met in the long run.

Conclusion
In large-scale, long-term forest inventory programs, new and inexperienced surveyors are expected to contribute to field measurements using instruments involving a large number of sample plots and measured variables.We found that providing only initial training in how to use the instruments was insufficient for inexperienced surveyors to achieve measurement quality objectives for tree diameter and height.Adding a oneday exercise in which re-measured data were fed back to the surveyors significantly improved data quality, except for tree height measurements of broad-leaved trees.Additional feedback instruction did not contribute further improvement.We propose that field training courses for NFIs incorporate a one-day exercise with feedback instruction so that inexperienced surveyors can achieve MQOs of i) < 2% average percentage difference (PDif) for DBH of both conifers and broad-leaved trees and ii) < 3% average PDif or 90% compliance rate for measurements to be within ± 10% error for tree height of conifers.Further improvement of data quality, might be achieved by marking the breast height or ground level, and by correcting measurements of tree height for lean, a common condition of broadleaved trees in other NFI's.

Fig. 1 .
Fig. 1.Difference (Dif) and percentage difference (PDif) between DBH measurements made by field teams and a control team for three measurements corresponding to three levels of training, denoted as 1st, 2nd, and 3rd.

Fig. 2 .
Fig. 2. Difference (Dif) and percentage difference (PDif) of tree height measurements between the field teams and a control team under three levels of training, denoted as 1st, 2nd, and 3rd.