Synthetic datasets were designed to mimic the real datasets as closely as possible. We first fit a specified model to the real data to identify parameters and q-values for each predictor-target pair. We then planted predictor-target pairs for each significant (q≤0.2) predictor-target pair identified from the real data. Specifically, we generated a synthetic target amino acid for each consensus amino acid in the sequence, such that (1) if the amino acid had no significant (q≤0.2) associations, then the amino acid was generated according to the parameters of the independent evolution model (the null model from the univariate case), and (2) if the amino acid had M>0 associations, then the amino acid was generated according to the given multivariate model with the predictor parameters s1,…, sM, taken from the real data. When an observation was missing in the real data, the corresponding observation in the synthetic data was also made to be missing. We treated amino acid insertions/deletions and mixtures as missing data.
Our goal was to generate data that is as realistic as possible, both in the values of the parameters used and the number of predictors deemed correlated with the target. Because our recall rate is less than 100% (see section on synthetic results), planting only those associations that are found in the real data would result in a smaller proportion of synthetic predictor-target pairs called significant than real predictor-target pairs called significant. We therefore planted two associations for every observed significant association in the real data and reduced the number of independently evolving codons accordingly. For the Noisy Add model, this procedure planted 72 HLA-codon and 612 codon-codon associations in the HOMER cohort and 114 HLA-codon and 952 codon-codon associations in the combined HOMER-Durban cohort. In hindsight, doubling the number of planted associations was an overcompensation, as experiments on this synthetic data yielded a 75% recall rate. Nonetheless, the doubling produced a reasonable result, as Noisy Add declared 0.56% of all synthetic predictor-target pairs significant at q≤0.2 compared to 0.65% of all predictor-target pairs in the real data for the combined HOMER-Durban cohort.