Threshold-Based Inference Optimization

Example 34

The system of any of Examples 23 to 33, wherein the operation further comprises: performing inferences for a subset of the one or more subsequent inferences using the machine learning model and the received weight information; determining that results of the subset of the one or more subsequent inferences using the machine learning model and the quantized weight information are outside the threshold performance level relative to results of the subset of the one or more subsequent inferences using the machine learning model and the received weight information; and based on the determining, performing additional inferences using the machine learning model and the received weight information.