Cell Communication

We compare the performance of CellChat with three other tools, including SingleCellSignalR⁹ , iTALK¹⁰ , and CellPhoneDB^{16 (link)} . We compare our database CellChatDB with other existing analogous databases, including CellTalkDB⁷¹ , CellPhoneDB^{16 (link)}, iTALK¹⁰ , SingleCellSignalR⁹ , Ramilowski2015^{72 (link)}, NicheNet^{13 (link)} and ICELLNET⁷³ . SingleCellSignalR scores a given ligand-receptor interaction between two cell populations using a regularized product score approach based on average expression levels of a ligand and its receptor and an ad hoc approach for estimating an appropriate score threshold. iTALK identifies differentially expressed ligands and receptors among different cell populations and accounts for the matched ligand-receptor pairs as significant interactions. CellPhoneDB v2.0 predicts enriched signaling interactions between two cell populations by considering the minimum average expression of the members of the heteromeric complex and performing empirical shuffling to calculate which ligand–receptor pairs display significant cell-state specificity. The detailed description of how these methods were performed is available in Supplementary Note 3.
Both CellChat and CellPhoneDB, but not SingleCellSignalR, and iTALK, consider multi-subunit structure of ligands and receptors to represent heteromeric complexes accurately. To evaluate the effect of neglecting multi-subunit structure of ligands and receptors, we compute false positive rates for the tools that use only one ligand and one receptor gene pairs. The false positive interactions are defined by the interactions with multi-subunits that are partially identified by iTALK and SingleCellSignalR. The ground truth of the interactions with multi-subunits is based on our curated CellChatDB database. For example, for Tgfb1 ligand and its heteromeric receptor Tgfbr1/Tgfbr2 curated in CellChatDB, if the method only identifies one of the two pairs (Tgfb1–Tgfbr1 and Tgfb1–Tgfbr2), then we consider this prediction as one false positive interaction.
We performed subsampling of scRNA-seq datasets using a ‘geometric sketching’ approach, which maintains the transcriptomic heterogeneity within a dataset with a smaller subset of cells^{96 (link)}. We evaluated the robustness of inferred interactions from subsampled datasets using three measures, including TPR, FPR, and ACC, which were defined in Supplementary Note 3. Note that such subsampling analysis was used to evaluate the consistency rather than accuracy.

Six pairs of Kaplan-Meier curves were used in the validation exercise. These were drawn from a subset of publications [12 (link),27 (link)-29 (link)] that formed part of a look-back review of survival time analysis methods used in economic evaluations [13 (link)]. We carried out a reconstruction of twenty-two survival probabilities, seven median survival times, six hazard ratios and four standard errors of the log hazard ratios that were reported in these four publications. Each was reconstructed on two occasions by the same three observers. Two of the three observers were not involved in the development of the algorithm.
Reproducibility and accuracy of the method was evaluated for each of the 4 different levels of information ('all information', 'no numbers at risk', 'no total events' and 'neither'). To assess the differences between the reconstructed statistics and the original ones, the natural scale was used for the survival probabilities, while the log scale was used for medians, HRs and their uncertainties. Kaplan Meier curves and Cox HRs based on reconstructed data were estimated using the R routines survfit and coxph.
We fitted a standard two-way ANOVA with repeated measures to the differences between the reconstructed outcomes and the original outcomes, either on the natural or the log scale depending on the statistic considered. The components of variance were exemplar, observer, exemplar × observer interaction, and within-cell error. Because the p-value from the F-ratio test for the interaction was in all cases above 10%, we pooled the interaction term with the within-cell error term. The approach chosen is similar to what is referred to in engineering applications as 'gauge repeatability and reproducibility' [30 (link),31 (link)].
The reproducibility represents the error if a single observer does a single reconstruction for a specified statistic. This was estimated as the sum of the within-observer and between-observer error. Monte Carlo simulation from the fitted ANOVA model was used to obtain the 95% confidence intervals around the standard deviations. The degrees of freedom for the within, the between and the outcome variations were assumed to follow chi-square distributions. To ensure robust inference, 150 000 samples of degrees of freedom were drawn from each of these distributions, i.e. for each source of variation. Then, the mean squares estimates were calculated, using the sum of squares obtained by the ANOVA and the sample obtained by the simulation, for each of the 150 000 samples and for each of the sources of variation. The corresponding 150 000 within, between and outcome standard deviations were subsequently estimated and we finally extracted the 2.5 and 97.5 percentiles to obtain the confidence intervals estimates.
To assess accuracy we examined the mean difference between the reconstructed statistics and the original ones. The resulting mean bias, or mean error (ME) reflects systematic over- or underestimation. The 95% confidence intervals are obtained directly from the estimation of the standard deviations given by the ANOVA. We also recorded absolute bias or mean absolute error (MAE). This ignores the direction of the errors and measures their magnitude, giving a measure of the absolute accuracy of the reconstructed outcomes. A simulation method was again used to obtain the 95% confidence intervals, which assumed that MEs were normally distributed. For each statistic, to ensure robust inference, 150 000 samples were drawn from the normal distribution with the observed mean and variance, as given by the ANOVA. We then calculated the corresponding 150 000 absolute values of these numbers and we finally extracted the 2.5 and 97.5 percentiles to obtain the confidence intervals estimates.
Finally we recorded the variation in the difference between reconstructed and original statistics that was due to the choice of exemplars, i.e. to the 22 survival probabilities, 7 medians, 6 HRs and 4 standard errors of the log HRs. This gives a further indication of the accuracy of the method.

To construct a database of ligand-receptor interactions that comprehensively represents the current state of knowledge, we manually reviewed other publicly available signaling pathway databases, as well as peer-reviewed literature and developed CellChatDB. CellChatDB is a database of literature-supported ligand-receptor interactions in both mouse and human. The majority of ligand–receptor interactions in CellChatDB were manually curated on the basis of KEGG (Kyoto Encyclopedia of Genes and Genomes) signaling pathway database (https://www.genome.jp/kegg/pathway.html). Additional signaling molecular interactions were gathered from recent peer-reviewed experimental studies. We took into account not only the structural composition of ligand-receptor interactions, that often involve multimeric receptors, but also cofactor molecules, including soluble agonists and antagonists, as well as co-stimulatory and co-inhibitory membrane-bound receptors that can prominently modulate ligand-receptor mediated signaling events. The detailed steps for how CellChatDB was built and how to update CellChatDB by adding user-defined ligand-receptor pairs were provided in Supplementary Note 1. To further analyze cell–cell communication in a more biologically meaningful way, we grouped all of the interactions into 229 signaling pathway families, such as WNT, ncWNT, TGFβ, BMP, Nodal, Activin, EGF, NRG, TGFα, FGF, PDGF, VEGF, IGF, chemokine and cytokine signaling pathways (CCL, CXCL, CX3C, XC, IL, IFN), Notch and TNF. The supportive evidences for each signaling interaction is included within the database.

Clonally derived D1 mouse mesenchymal stromal cells (MSCs; American Type Cell Culture, CRL-12424) were used since they were used by a number of studies on cell-material interactions in the context of tissue regeneration [15 (link), 16 (link)]. Cells were cultured in complete high glucose Dulbecco’s Modified Eagle Media (DMEM; Thermo Fisher Scientific) supplemented with 10% fetal bovine serum (FBS; S11550, Atlanta Biologicals), 100 units/mL penicillin and 100 μg/mL streptomycin, and 2 mM GlutaMAX (Thermo Fisher Scientific). Cells were passaged when they reached 80% confluence in a 175 cm² flask by detaching with trypsin-EDTA (Thermo Fisher Scientific). Passage numbers less than 15 were used for this study.