Rtx 2070

Training and testing of all the experimental models were done using a Ryzen-5 2600x (6-core) processor, 16GB system RAM and NVIDIA RTX 2070 GPU with 8GB VRAM (unless specified otherwise). We also utilized the open-source object detection and instance segmentation framework MMDetection (Chen et al., 2019 (link)) based on the PyTorch deep learning library to implement our model architecture of choice, as it offers an easy-to-use modular codebase. After each epoch of training, we evaluated the model on the validation set. So, after the whole training period, we saved the checkpoint of the best performing model.

We use the Tensorflow 1.15 and NVIDIA RTX 2070 to train the network, which required 6 h, 30,000 iterations, and 10 epochs.

The training and validating processes were carried out using the epoch datasets of the four subjects on a PC. For this process, a total of 892,839 training epochs and 224,209 validating epochs were used. For the training process, a PC with an Nvidia RTX 2070 GPU of 8 Gb memory was used with a learning rate of 0.0003 and a batch size of 64 for each model. All the models were written using Python 3.8 with TensorFlow and Keras. To evaluate the performance of the implemented five deep learning models, two conventional criterion metrics were used: accuracy and inference time. To calculate the accuracy, Equation (1) was used, where $T_p, F_n, F_p$ and $T_n$ represents the sample number of true positives, false negatives, and true negatives, respectively. On the other hand, the inference time $t_{inference}\left(ms\right)$ represents the time needed for the model to output a classification label. Meanwhile, the inference time is given as Equation (2), where $t_{inp}$ is the time value when the data is input to the model. $t_{out}$ is the time value when the result classification label is obtained.
$$Accuracy (\%)=\frac{T_p+T_n}{T_p+T_n+F_p+F_n}x 100$$
$$t_{inference}\left(\mathrm{ms}\right)=t_{out}-t_{inp}$$

Before training the denoising autoencoder (DAE) and the classifier via the extracted motion sequences, $\mathit{K}$ , each sequence was downsampled to 1 FPS to reduce training time²⁰ . Moreover, the sequences were normalized using min–max normalization. Lastly, the performance scores were pre-processed via z-normalization, and one hot encoding was used for the class labels. The same pre-processing pipeline as the PC datasets was used for the JIGSAWS dataset kinematics.
The batch size was one during training because each input has a different sequential length. The training was regulated using early-stopping based-on validation loss with the patience of 4 and 20 epochs for DAE and classifier training, respectively, for the PC datasets. These values were 40 and 200 for the JIGSAWS dataset¹³ . Finally, we incorporated class weights into the training to account for imbalance. (For hyperparameter selection, see Supplementary Information / Hyperparameter selection).
Notably, when developing the VBA-Net on the PC datasets, we repeated the training for ten sessions, ensuring robust hyperparameter selection. The training was conducted on a workstation with AMD Ryzen 7 2700X and NVIDIA GeForce RTX 2070.