Machine activity recognition is important for accurately esti- mating machine productivity and machine maintenance needs. In this paper, we present ongoing work on how to recognize activities of forklift trucks from on-board data streaming on the controller area network. We show that such recognition works across different sites. We first demon- strate the baseline classification performance of a Random Forest that uses 14 signals over 20 time steps, for a 280-dimensional input. Next, we show how a deep neural network can learn low-dimensional representa- tions that, with fine-tuning, achieve comparable accuracy. The proposed representation achieves machine activity recognition. Also, it visualizes the forklift operation over time and illustrates the relationships across different activities. © Springer Nature Switzerland AG 2020

Machine Activity Recognition (MAR) can be used to monitor manufacturing processes and find bottlenecks and potential for improvement in production. Several interesting results on MAR techniques have been produced in the last decade, but mostly on construction equipment. Forklift trucks, which are ubiquitous and highly important industrial machines, have been missing from the MAR research. This paper presents a data-driven method for forklift activity recognition that uses Controller Area Network (CAN) signals and semi-supervised learning (SSL). The SSL enables the utilization of large quantities of unlabeled operation data to build better classifiers; after a two-step post-processing, the recognition results achieve balanced accuracy of 88% for driving activities and 95% for load-handling activities on a hold-out data set. In terms of the Matthews correlation coefficient for five activity classes, the final score is 0.82, which is equal to the recognition results of two non-domain experts who use videos of the activities. A particular success is that context can be used to capture the transport of small weight loads that are not detected by the forklift’s built-in weight sensor. © 2022 by the authors. Licensee MDPI, Basel, Switzerland.

Research on machine activity recognition (MAR) is drawing more attention because MAR can provide productivity monitoring for efficiency optimization, better maintenance scheduling, product design improvement, and potential material savings. A particular challenge of MAR for human-operated machines is the overlap when transiting from one activity to another: during transitions, operators often perform two activities simultaneously, e.g., lifting the fork already while approaching a rack, so the exact time when one activity ends and another begins is uncertain. Machine learning models are often uncertain during such activity transitions, and we propose a novel ensemble-based method adapted to fuzzy transitions in a forklift MAR problem. Unlike traditional ensembles, where models in the ensemble are trained on different subsets of data, or with costs that force them to be diverse in their responses, our approach is to train a single model that predicts several activity labels, each under a different context. These individual predictions are not made by independent networks but are made using a structure that allows for sharing important features, i.e., a context ensemble. The results show that the gated recurrent unit network can provide medium or strong confident context ensembles for 95% of the cases in the test set, and the final forklift MAR result achieves accuracies of 97% for driving and 90% for load-handling activities. This study is the first to highlight the overlapping activity issue in MAR problems and to demonstrate that the recognition results can be significantly improved by designing a machine learning framework that addresses this issue. © 2023 The Author(s)