The Accuracy of Manually Recorded Time Study Data for Harvester Operation Shown via Simulator Screen

The aim of the study was to investigate the effect of work experience on the accuracy and variation of observers recording the operation time of a harvester. A simulated thinning operation using a harvester, shown as video via a television screen in laboratory conditions, was observed by 20 inexperienced students and 10 experienced work study researchers. All the observers timed the different work elements of the harvester work with special fieldwork timers. The duration of different work elements measured by the human observers were compared to the corresponding recordings by the harvester’s automated data collector. Although the inexperienced students made more measurement mistakes than the experienced researchers, the differences in measurement error averages were not statistically significant between the groups. However, the variances of tree specific errors were significantly higher in the measurements done by the students. As inexperienced recorders, the students were not able to properly record short work elements, which lasted a maximum of 4 seconds. Due to systematic measurement errors, there was a large variation in the timing structures of the work elements among all observers. Observers’ skills and experience seems to affect measurement accuracy and thus the derived results, especially in intensive time studies. Therefore, the recorder should receive detailed training and practical experience in timing of different work elements of forest operations. In the future, with the use of automated data collectors time studies with large, detailed and accurate data will be implemented. However, due to the varying timing conditions in the forest, manual data collection is still required because of its greater flexibility.


Introduction
Generally speaking, time studies are the most common work measurement method (Vöry 1954, Haarlaa et al. 1984, Björheden 1991).One important aim of work studies is to measure the working time and the amount of work done; thus time study is a tool of work study (Pukkila 1959).As Harstela (1991) has defined, work studies in general could include all kinds of studies concerning human beings at work, other factors of production or working conditions.The purpose of work studies is to conduct the investigation of all factors which affect the efficiency and economy of the situation being reviewed, in order to facilitate improvement (ILO 1979).In forestry works, time studies have been used to determine piece rates and to rationalize production (Makkonen 1954, Björheden 1991, Harstela 1993).
During the last 20 years, timing techniques in forestry operations have developed from decimal watches to 2000-millenium's automated data recorders of forest machines (e.g.Peltola 2003, Kariniemi 2003) (Fig. 1).When decimal watch and paper-forms were replaced by field computers during the 1980s, in time studies in forest works, the possibility to measure more detailed and accurate work elements increased.
The newest time study technique for machine works − automated data collectors attached to forest machines' computers and CAN-bus channels − enables the collection of more accurate and detailed data (Kariniemi 2003, Peltola 2003, Väätäinen et al. 2003).Highly detailed projection of machine work enabling the recording of remarkably short and overlapping work elements and functions has taken the investigation of forest work to a new level.However, new methods can not automatically register unforeseen situations and the change of conditions, therefore, the presence of a researcher in the study site is often essential (Peltola 2003, Väätäinen et al. 2003).
In order to make reliable and detailed time studies, it is important to comprehensively determine the actual steps of the study itself.According to Harstela (1991) the following steps have to be taken into account: -select the work to be studied.
-plan measurement procedure and divide the job into elements.-choose the measurement techniques.
-choose the workers and train them.
-record all the relevant data relating to conditions, methods and elements of the activity.-examine the recorded data to ensure that the most suitable methods and working techniques are used and foreign elements are separated from the relevant ones.
In addition to the steps mentioned above, and their influences on the final recorded data and measurement errors, there is also the effect of the individual observer's abilities and technique if timing is conducted manually.How the observer interprets the work elements, and the changing moment of each element and how meticulously they observe the studied object and handle the time recording device.Moreover, how significant is the impact of these measurement errors in the timing on time consumption distributions of measured time elements?The effect of researchers' subjective observation in the measurement accuracy and overall result in time studies has not been thoroughly examined previously.
A few studies and surveys concentrated on observers measuring errors of wrongly or nonrecorded time elements and capabilities to record varying length of time elements (Pukkila 1959, Pehkonen 1973, 1978, Kärkkäinen 1975), have been made in the era of decimal watches.According to Pukkila (1959) the most accurate recordable time-element, with a manual decimal watch, was about two seconds.Pehkonen (1973Pehkonen ( , 1978) ) found, in laboratory research, that there was a significant correlation between the duration of work elements and unrecorded work elements.Also the researcher's fatigue increases the ratio of missing time elements of a short duration (Kärkkäinen 1975).For example, Pehkonen (1973) found that measuring accuracy got worse after two hours of time study.
Some recent work studies have compared the differences between manual timing and recordings of automated data collectors of harvesters (Norden andGranlund 2003, Väätäinen et al. 2003).Systematic differences and biased measurement errors in results were explained mostly by the abilities of the researcher to visually detect and record the change of time elements compared to automated recording of machine movements.Also an important issue did arise, that it is essential to accurately determine the moment of change of time elements.Pehkonen (1978) and Peltola (2003) found that the data collector (observer) has a certain impact on how certain actions are interpreted, although the definitions of time element changes are accurate.Manual data collection, therefore brings subjectivity to the work study.
The techniques in time studies in forest operations have significantly changed recently, and as a result there is a need to adapt the current recommendations for time concepts and work elements to these new techniques (Kariniemi 2003, 2006, Peltola 2003).However, there is a need for manual time studies and time observers, when measuring the whole work process.This is especially in shorter studies, with quite limited data, and in fairly varying circumstances, where the requirements of the researcher observing also the change of the conditions and other factors which affects work performance.Currently there is no clear understanding of the variations in time consumptions and timing errors among time study observers and how these affect the eventual results of the studies.Furthermore, the impact of the experience and the habits on the results is also unclear.This leads to the question of whether the researcher's recorded time consumptions are accurate and reliable enough to truly reflect the often intensive forest work?
The main objective of this study was to investigate the effect of work experience on the accuracy and variation of observers recording harvester's operation time.A supplementary aim was also to clarify whether measurement errors and differences between the observers affect the structure and ratio of timings of work elements within time studies.

Research Material and Practical Method
The time study was conducted in a TV-studio, where each researcher studied 40 minutes of identical video-material of simulator harvester logging (Fig. 2).The video material of the thinning showed the felling of 81 trees and also included the sound of the harvester operation.All the observers chosen for this study made a time study based on uniform instructions.
The pool of time study observers consisted of 20 novices and 10 experienced researchers.The observers were divided into three groups (10 observers/group) according to their training and experience level.Two groups consisted of students divided according to their level of practice before the time study: 15 minutes (students 15 min) or 30 minutes (students 30 min).None of the individuals in these groups had any previous time study experience.Forestry researchers, who had previously conducted time studies in the field, were in the third group (researchers).They also were given training for 15 minutes before the experiment.Before the introductory training all the time study observers had the same familiarization of the work elements and the work element definitions and recording codes were distributed to observers a few days before the study.They recorded the work elements using Rufco-900 field computers (Fig. 1) applying different number codes for the various work elements.The timing accuracy of Rufco-900 is 0.6 seconds (1 cmin).
In this study, the harvesting stages with a singlegrip harvester were divided into more detailed elements: 1) moving forward, 2) steer out the boom and grab, 3) felling, 4) processing (delimbing and cross-cutting), 5) reversing, 6) steer the boom front and 7) pause time.Driving forward and reversing started when the harvester started to move and ended when the harvester stopped to perform another task.Steer out the boom and grab started when the boom started to swing toward a tree and ended when the harvester head rested on a tree and the felling cut begun.
Felling started when the felling cut began and ended when the feeding and delimbing of the stem (processing) started.Processing consisted of delimbing and crosscutting.Processing ended when the operator lifted the harvester head to an upright position immediately after the final crosscut of the stem.Steer the boom front occurred when the operator steered the harvester head to the front of the machine before moving forward.Pause times were short time elements when no machine movements occurred.Pause time consisted mainly of work planning.In this simulation environment, of a first-thinning operation, there were no other work elements such as clearing saplings, piling logs, moving tops and branches, which occur in real harvesting.
In addition, to further analyse the observers' recorded material a division of main and complementary work elements were conducted (see Björheden 1991).Work elements 2, 3 and 4 were the main work elements repeated for each tree.While elements 1, 5, 6 and 7 were defined as complementary work elements.Generally, the complementary work elements are more difficult to identify and record compared to the main work elements, furthermore the complementary elements where not conducted on each tree.
Time consumption data comprising of two main work elements (felling and processing) -recorded using an automated data logger (PlusCan from Plustech Ltd.) (Fig. 1) -was used as reference data in this study.The definitions of starting and ending points of the felling work element and processing work element were identical to the respective definitions of the manual time study.The timing accuracy of the PlusCan-device is thousandth part of a second.

Analysis of the Research Material
A comparison of all the observers was conducted based on average time consumption for the distribution of work elements in order to compare the differences in the work element timings among the observers and their experience category.All the time consumptions of each time element where a code was missing or an incorrect code had been entered were examined and defined as "recording with error code".In addition the measuring errors in time consumptions for all the observers were examined for felling and processing work elements for each stem.The measuring error was counted per trunk by subtracting the value (a reference value) of the automated data logger from the time value of observer.Standard deviations and trends of measuring errors (box plots) were also counted for each observer.
The average measuring error in each experience category was statistically tested with a mixed effects model with stem size as a covariant and experience level of the observer as the random factor.The equality of the measuring errors' variances between the experience groups were pairwise tested using Levene's test (Milligen and Jonsson 1984).Also the researchers' fatigue during the time study was determined using Levene's test for each experience group.
For the testing the level of fatigue the time study was broken down into four sections of 10 minutes.The time sections were set as independent factors in the Levene's test for fatigue.

The Frequency of Recorded Work Elements
The total number of recorded main work elements per observer did not clearly differ between each experience group; students 15 min recorded an average of 241 main work elements, students 30 min 241 elements and researchers 243 elements.
In other words, the students failed to record two elements whereas the researchers managed to record all the main work elements during the recording period, on average.Furthermore, the number of main work elements in different time intervals for each group was similar.However, there were clear differences between the experience groups when recording the complementary work elements.The total number of recorded complementary work elements per observer averaged 98 elements for the researchers, 93 elements for students 30 min and only 82 elements for students 15 min.Students 15 min recorded complementary work elements of 2 seconds and shorter (average 7.1 elements per observer) which was 44% less than the value of researchers (12.7).With elements of 4 seconds and shorter students 15 min averaged 35.3 complementary elements and researchers 53.9 which equalled a difference of 35% (Fig. 3).
The major differences for complementary work elements were in reversing and steer the boom front.These differences were mainly in the short timings; in the reversing work element of 4 seconds and shorter timings students 15 min recorded 42% less than researchers and, in steer the boom front work element 53% less.Straight after the timing study the observers were asked which work element they felt to be the most difficult to record, the responses were, in order of difficulty, steer the boom front and reversing.
The observers made few error recordings by incorrect coding during the time study.On average, students 15 min made about 5 error recordings each, students 30 min 3 and researchers 1 error recording each during the study.

The Differences in Structuring of Work Elements of Observers
The average work element timings between each group did not differ significantly (Fig. 4).The biggest difference in the group averages was found in the timing of pause time (0.5 seconds/stem).However, the recordings of individual observers differed remarkably in many work elements.For example in the main work elements, the range of observers' average timings were 7.6-11.6seconds in steer out the boom and grab, 4.8-6.3seconds in felling and 7.7-10.0seconds in processing, and the maximum time difference was in its highest in steer out the boom and grab work element (34% difference).The 95% confidence level of the individual observers' average recordings revealed significant variation between individual timings (Fig. 4.).It was notable that the confidence level decreased when experience level increased in most of the work elements.Furthermore, the confidence levels were surprisingly high in all the experience groups, especially in the steer out the boom and grab work element.

Measuring Accuracy within Experience Groups
The comparison of the observers' recorded data and PlusCan-data logger's reference data revealed the observers' actual timing errors for both felling and processing work elements for each measured stem.The reference value per stem, on average, was 5.75 seconds for felling and 9.10 seconds for processing.In the processing work element 62% of the researchers' timing errors were within the error interval of ±0.5 seconds, while the value for students 15 min was 33% and for students 30 min 47%.The largest error average for an observer in the felling work element was found in the group of students 15 min, where the difference was 17% less than the reference value.
In the processing work element the largest difference for an observer was 15% (in the group of students 15 min).
The experience level had no statistically significant influence on the measuring error averages of the observers in either the felling or processing work elements, when analysing the influence with the mixed effects model (Table 1).The significance values of the test for the effect of experience was 0.526 in felling and 0.215 in processing.
There were significant differences in the variances of observers' measurement errors between the groups of students and researchers (Table 3).
In the case of felling, there was no significant difference in error variances between students 15 min vs. students 30 min, whereas the difference of error variance was significant for students 15 min vs. researchers and students 30 min vs. researchers.In the processing work element the experience level had a statistically significant influence on error variances for all experience groups (Table 3).
Fig. 5 presents the box-plot charts of the distribution of measurement errors for all experience groups in the felling and processing work elements.Unlike felling, in processing the reduction in the measurement error deviation when increasing the experience level is clear.In the felling  work element the average measurement error was very close to zero for all experience groups.In processing, the average measurement error for the students 15 min was -0.63 seconds, for the students 30 min -0.41 seconds and for the researchers -0.24 seconds.In processing the standard deviations were 2.01 seconds for the students 15 min and 0.81 seconds for the researchers.
The observers' fatigue during the time study did not have any effect on the measurement error on the basis of the analysis of the research data and also the recorded error codes by the observers had a minor effect on the measurement accuracy (Nuutinen 2005).Furthermore, the effects of age, sex, playing of computer games and use of computer on measurement error was examined.The results found that these factors also did not influence the results (Nuutinen 2005).

Discussion
The number of observers in the study was sufficient in order to explore the influence of observer's work experience on measurement accuracy.The number of cut trees was adequate to investigate the actual objectives of this study.However the length of the measured video material could have been longer, at least 2 hours, to reveal how the observer's fatigue affects timing accuracy.The accuracy and reliability of the PlusCan-data logger was tested and confirmed with video technique using timing accuracy of hundredth part of a second.
This study stressed the teaching and demonstrating of work element definitions and divisions before the study started.Additionally, the harvester simulator environment made it easier to detect the transition moments of time elements during the time study.Thus, in the stable and unique study environment of the studio, the use of identical study material for all observers, and controlling other factors that may influence the timing was easier than if the study was conducted in the field.
Though the study was conducted in a studio environment, the results can be generalised for real time study practices to some extent.However, before making deeper generalisations based on this study further research should be made.For example, a group of time study researchers conducting a timing of harvester work at the same time in the forest.Timing should last long enough to obtain adequate repetitions of the time elements.This will allow the influence of researcher's fatigue to be included.Furthermore, other influencing factors on measurement accuracy, such as climatic conditions, visual obstructions should be included.
The frequency of observed short time elements revealed the differences between the students and researchers.Short complementary work elements, which are not easy to observe and detect, are especially problematic for beginners.Additionally, errors in coding occur more often for beginners than experienced researchers.Several studies made in the era of decimal watches have also shown that there is a higher risk of recording errors when the work elements are shorter (Pukkila 1959, Pehkonen 1973and 1978, Kärkkäinen 1975).Automatic data collection devices attached to the forest machines are able to record detailed data of the different work elements in an accurate and economical fashion (Peltola 2003, Kariniemi 2003, 2006).This could be a reasonable argument to record short work elements by means of automatic data collection.
The study results revealed that time studies made by researchers were more controlled and reliable.The variance of measurement error was smaller for researchers than students.However, statistically significant differences did not occur in the measuring error averages between the experience groups.In addition, not only among students but also among researchers, observers had systematic differences in the actual moments of recording.Pehkonen (1978) found that the most important factors affecting the measuring accuracy in time study were time study methods and equipment, facility to observe break points, time to be measured, skill of the observer and human factors at the moment of measuring.Peltola (2003) also concluded that depending on the individuals there can be significant differences in determining and interpreting structures of work elements despite accurate definitions.
The important finding of the study was that there was a significant variation both in timing accuracy and in recorded work time distributions between observers in all experience levels.When comparing the shortest average timings of the observers to the longest ones, maximum of a 34% time difference occurred.These systematic differences can be mainly explained by the different interpretation of the break points of time elements.Some of these differences may also derive as a result of confusion during timing when a large number of elements occurred during a short period of time.In this study differences in the recording times of the break points of time elements among the observers can be explained e.g. with the reaction time, interpretation of time elements and level of accuracy which all are personal characteristics and therefore create differences between individuals.
If differences can be revealed in such restricted and uniform conditions used in this study, it can be assumed that these variations are more significant out in the field.Väätäinen et al. (2003) for example compared manual time study data recorded by fieldcomputer to automated data logger recordings in a time study of a single grip harvester in the forest.The average manual measurement error per stem was in the felling work element 8 times (0.599 sec) and in the processing work element 2.5 times (0.367 sec) bigger than in this study.Furthermore, Nordén and Granlund (2003) have found 8-38% errors from manual time study collection compared to the automated data collection of a harvester.
In intensive time studies of harvester operations it is vital to consider the skills and experience of the observer.Since it has a straight influence on the accuracy of timing and therefore on the results.Time studies are often used as a basis for conducting important conclusions or using certain technology of working methods in forest operations.Therefore, the recorder must have a high level of professional skill and also should receive detailed training and practical experience in timing of forest operations before the field measurements.In the future, education and qualification of time study researchers should be considered to ensure the adequate ability of time study researchers.Moreover, automated and manual time studies alike will be needed in the future and their integrated use should be advanced by creating a new collective standard of time study work elements.

Fig. 1 .
Fig. 1.Measuring equipment used in time studies.Left to right -decimal watch, field computer, automatic data collector of forest machines (PlusCan by Plustech Ltd.).

Fig. 2 .
Fig. 2. Time study laboratory and a sample picture of cutting in a harvester simulator environment from TVscreen.

Fig. 3 .
Fig. 3.The frequency of recorded complementary work elements per observer between experience groups in time consumption classes during the 40 min time study.

Fig. 4 .
Fig. 4. Average time consumption structure of work elements of observers among experience groups in 40 minutes time study.Line segments identify the 95% confidence levels of observers' average timing values in each experience group.

Table 1 .
The mixed effects model table for testing the influence of experience level on the average measuring errors of observers in the felling work element.Dependent variable: measuring error of felling.

Table 2 .
The mixed effects model table for testing the influence of experience level on the average measuring errors of observers in the processing work element.Dependent variable: measuring error of processing.

Table 3 .
The significance for the equality of the measuring error variances between experience groups.Workelements felling and processing.Compared experience groups: 1 = students 15 min, 2 = students 30 min, 3 = researchers.