- Journal List
- J Assist Reprod Genet
- v.40(4); 2023 Apr
- PMC10224881
As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsem*nt of, or agreement with, the contents by NLM or the National Institutes of Health.
Learn more: PMC Disclaimer | PMC Copyright Notice
J Assist Reprod Genet. 2023 Apr; 40(4): 837–844.
Published online 2023 Jan 28. doi:10.1007/s10815-023-02719-w
PMCID: PMC10224881
PMID: 36708430
Kassie J. Bollig,
1 Suneeta Senapati,1 Mary D. Sammel,2 Peter Takacs,3 Jared C. Robins,4 Daniel J. Haisenleder,5 and Kurt T. Barnhart1
Author information Article notes Copyright and License information PMC Disclaimer
Associated Data
- Supplementary Materials
Abstract
Purpose
To validate the use of a multiple biomarker test panel for predicting first trimester pregnancy outcome in a multi-center cohort.
Methods
A case-control study of women presenting with pain and bleeding in early pregnancy at 5–10 weeks gestational age was performed at three academic centers. Sera from women with ectopic pregnancy (EP), viable intrauterine pregnancy (IUP), and miscarriage (SAB) were analyzed via immunoassay for Activin A (AA), Progesterone (P4), A Disintegrin And Metalloprotease-12 (ADAM12), pregnancy-associated plasma protein A (PAPP-A), glycodelin (Glyc), and human chorionic gonadotropin (hCG). Biomarkers were assessed for reproducibility using medians, ranges, standard deviations, and area under receiver-operating characteristic curve (AUC) and accuracy in early pregnancy outcome classification compared to a previous derivation population.
Results
In 192 pregnancies, the biomarkers demonstrated good reproducibility with similar medians, ranges, and AUCs when compared to the derivation population except glycodelin. Pregnancy location was conclusively classified in 53% (n = 94) of the whole study sample with 78% accuracy. Pregnancy viability was conclusively classified in 58% (n = 112) of the new sample with 89% accuracy. Results were similar with subsequent model revisions where glycodelin was excluded and in the subgroups of subjects with a hCG below 2000 mIU/mL and a gestational age less than 6 weeks.
Conclusion
The use of a panel of biomarkers to maximize test accuracy of a prediction of pregnancy location and prediction of pregnancy viability was reproducible and validated in an external population from which it was derived, but clinical utility is limited based on the test characteristics obtained.
Supplementary Information
The online version contains supplementary material available at 10.1007/s10815-023-02719-w.
Keywords: Ectopic pregnancy, Biomarker, Validation
Introduction
In a field of consistently improving techniques and technologies, the distinction between a normal versus abnormal early pregnancy remains a vexing clinical challenge for health care providers and their female patients. The importance of being able to accurately make the diagnosis of either a viable intrauterine pregnancy (IUP), pregnancy loss or spontaneous abortion (SAB), or ectopic pregnancy (EP) cannot be overstated, with each of these outcomes necessitating unique treatment pathways as well as contributing strikingly different rates of maternal morbidity and mortality. As the most immediately dangerous of these outcomes, ectopic pregnancy is purported to account for approximately 2% of all pregnancies and remains the most common cause of pregnancy-related death in the first trimester as well as the leading cause of hemorrhage-related pregnancy mortality [1]. Faced with this reality, it is imperative that efforts should be made to improve the accuracy of this diagnosis as early in the clinical course as possible.
The initial triage of women presenting with early pregnancy currently includes the diagnostic integration of patient symptoms, laboratory serum human chorionic gonadotropin (hCG) levels, and pelvic ultrasound [2] with no single entity being diagnostic for any specific pregnancy outcome. Despite improving technical skill and technology, ultrasound at first evaluation of early pregnancy may be inconclusive in up to 40% of women, particularly in those with pregnancies less than 7 weeks of gestation and hCG values below the discriminatory zone [3]. Serial serum “hCG curves” or models of trend have been defined for ongoing IUP [4], SAB [5], and EP [6], and when deviations from normal hCG curves occur, providers have initiated investigations to identify and treat potential EPs as part of the standard of care [2, 7, 8]. However, up to 16.8% of EPs, 7.7% of IUPs, and 39.0% of SABs may be misclassified by the use of these serial trends alone and result in devastating consequences including disruption of a viable pregnancy and adverse effects from inappropriate utilization of methotrexate or surgery [9].
The use of biomarkers, or molecules produced by affected individuals that signal specific exposures or disease states, has been increasingly studied to aid in the diagnosis of EP as a companion diagnostic test to strategies listed above. If successful, earlier and accurate diagnosis of these pregnancy states will not only promote early, less-invasive intervention that protects against the catastrophic outcome of tubal rupture and maternal mortality, but will also decrease the inappropriate administration of medication or surgery when a viable pregnancy is misclassified. Early investigations of biomarkers for pregnancy diagnosis have shown that the use of multiple markers together, in place of single biomarkers alone, can improve EP identifications with reproducible accuracy. Biomarker selection has been based on potential biologic processes that may be overactive or abnormal in EPs such as trophoblast function [10–16], corpus luteum function, [10, 16–18] inflammation [15, 18], decidual cell function [19], and angiogenesis [14, 16, 17, 20]. Taken together, these previous studies have led to conflicting conclusions on which combination of biomarkers may prove most useful. Studies differ by patient population, biomarkers tested, population size evaluated, biomarker detection technique, and differing priorities of maximizing sensitivity, specificity, and accuracy. Our previous work has focused on combining putative biomarkers in classification trees to maximize sensitivity and specificity for pregnancy location and viability to maximize accuracy of diagnosis [21]. When we evaluated 230 pregnancies (81 SAB, 72 EP, 77 IUP), we developed a classification tree for viability with Activin A, glycodelin, and A Disintegrin And Metalloprotease-12 (ADAM12) definitively classified pregnancy location in 29% of our sample with 100% accuracy for EP and Progesterone (P4), and pregnancy-associated plasma protein A (PAPP-A) classified pregnancy viability in 61% of the sample with 94% accuracy.
Despite the stark increase in investigations evaluating potential biomarkers, the majority of these efforts have failed to perform the subsequent step of validating the set of biomarkers in a temporally distinct population [22]. The objective of this study, therefore, is to externally validate our previously derived companion diagnostic test [21] in a new population of women distinct form its derivation. Our aims were to (1) assess if the assays for each marker were reproducible and (2) evaluate the accuracy of classification in our distinct population.
Materials and methods
Study design
Participants for the study were selected from an ongoing prospective cohort [21]. This nested case-control design consisted of 192 women with symptomatic (pain and/or bleeding) early pregnancy who presented for immediate evaluation to one of three academic centers (University of Pennsylvania, Eastern Virginia Medical school, Northwestern University) (65 IUPs, 66 EPs, and 61 SABs). Inclusion for the initial prospective cohort consisted of the following criteria: (1) complaints of abdominal pain, vagin*l bleeding, or both; (2) serum hCG of 100–60,000 mIU/mL; (3) 5–10 weeks of gestation by last menstrual period; and (4) agreement to participate in data and serum collection for the Ectopic Pregnancy Biomarkers Bank after informed consent. Specimens were selected from the bank to be representative of each three outcomes (not to be similar across outcomes). The hCG range inclusion criteria was purposefully expanded from that of the original study to evaluate if results would validate in a more pragmatic population. Institutional Review Board approval was obtained at all study sites. Participants were excluded if (1) they had received any treatment during the current pregnancy prior to enrollment; (2) they had evidence of gestational trophoblastic disease; (3) they were diagnosed with a non-tubal ectopic pregnancy; or (4) there was evidence of multiple gestation.
Specimens were collected from 2014 to 2017. We have previously published the protocols for patient data collection, biomarker selection, serum collection, and sample assays in our derivation study [21]. Briefly, maternal age, gestational age, race, ethnicity, study site, and initial hCG were collected for each subject upon entry, and final pregnancy outcome was obtained through chart abstraction. Each participant was followed prospectively until their final pregnancy outcome, as defined by international consensus [23]. A viable intrauterine pregnancy (IUP) was defined as ultrasound evidence of an intrauterine gestational sac, yolk sac, and fetal pole with cardiac activity. Spontaneous abortion (SAB) was categorized as an embryonic loss (fetal pole > 4 mm with no cardiac activity) [24, 25] or an anembryonic gestation (gestational sac > 16 mm with no identified yoke sac or fetal pole) or with no change in size of fetal pole of gestational sac one week apart with evidence of products of conception on histopathology. Ectopic pregnancy (EP) was defined as laparoscopic evidence or an extrauterine gestation or ultrasound demonstrating an adnexal mass without evidence of an intrauterine pregnancy or an increase in hCG level after uterine evacuation.
Data collection and biomarker assays
Serum was collected at the point of initial presentation, centrifuged at 1500 rpm for 5 min, split into 0.5-mL aliquots, and stored at – 80 °C. Selected samples were sent to the University of Virginia Ligand Assay and Analysis Core and immunoassays performed. Based on both a putative and agnostic approach through prior investigations, the markers for this study included ADAM-12, Progesterone (P4), Activin A, PAPP-A, glycodelin, and beta human chorionic gonadotropin (hCG). Assay characteristics are described in Supplemental Table 1, available online. All biomarkers were assessed on the same specimen for each subject.
Sample size
We assumed for the purposes of these calculations that our current markers have 95% or better accuracy among those classified (i.e., proportion of those classified that were true positives and true negatives). To obtain a two-sided 95% confidence interval for an observed accuracy rate of 95% with a half-width of 3.1% (95% confidence interval 91.9 to 98.1%), we would need 63 cases and 120 controls.
Because EP was the rarest outcome, SAB and IUP subject selection was frequency-balanced with representative samples from each outcome. Based on the three clinical outcomes, we assigned case and control status based on the outcome of pregnancy location or viability in order to validate our previous findings. For the outcome of pregnancy location, EPs were classified as cases and IUPs and SABs were controls. For the outcome of pregnancy viability, EPs and SABs were classified as cases and IUPs were controls. Assays for each biomarker were run after study sample selection; thus, selection was blind to biomarker profile.
Statistical analysis
Baseline characteristics of subjects were evaluated by using the Kruskal-Wallis test for continuous measures, and Pearson Chi-square or Fisher-exact tests for categorical variables.
We first assessed the reproducibility of the assays for each marker individually. We evaluated the medians, ranges, and standard deviations as well as area under receiver-operating characteristic curves (AUCs) that determined discrimination for each maker stratified by outcome. The Kruskal-Wallis test was used to compare differences among biomarker distribution by pregnancy outcome and the Wilcoxon Rank Sum test used to assess biomarker differences by source population (derivation study vs. validation [current] study).
We next applied the previously derived decision trees for using the markers simultaneously to predict (a) location of the pregnancy (EP vs. IUP +SAB) and (b) viability of the pregnancy (IUP vs. EP + SAB). Serum concentrations of each biomarker, for each subject, were used to create a prediction of outcome using the algorithm reported in our previous paper (Fig. (Fig.1).1). The predicted outcome was compared to actual outcome in order to assess the number of subjects definitively classified (and those who were inconclusive), as well as the accuracy of the prediction. Accuracy is a summary statistic defined as the proportion of true classifications out of all classifications (TN+TP)/(TN+TP+FP+FN). We assess prediction of each test for (a) location and (b) viability, separately.
Fig. 1
Final diagnosis of each individual was only made if classification of the subject was in agreement from both trees. If both trees classified the subject as a case or a control, definitive classification was determined. If the classification did not agree (i.e., one tree classified the subject as case and the other a control), the subjects were not definitively diagnosed and were labeled “indeterminate.” a Derived decision trees to predict pregnancy location. b Derived decision trees to predict pregnancy viability
Validation of our model was assessed in a series of planned analyses and compared to results obtained in the derivation study [21]. Then, subsequent analyses were performed to determine if the model could be optimized. Analyses included (1) assessment of prediction based on the exact algorithm of biomarkers including the threshold for each marker derived from our initial study, (2) assessment of prediction of the model using the same biomarkers but allowing revision of the thresholds of the individual markers based on the concentrations found in the new population, and (3) assessment of prediction of the model if markers were added or removed from the original model. Similar to the derivation study, we applied the decision trees to clinically important subgroups of patients including those less than 6 weeks gestational age and those with an hCG less than 2000 mIU/mL.
Results
A total of 192 women composed of 65 IUPs, 66 EPs, and 61 SABs were included in the final analysis. Baseline characteristics are reported in Table Table1.1. There were no differences between groups of participants in regard to race or ethnicity. Subjects with an IUP were more likely to be younger than those with an EP or SAB (p = 0.0002). Women with IUPs presented with higher initial hCG values (p < 0.0001). Women with EPs and SABs presented earlier than those with an IUP (p = 0.0002).
Table 1
Baseline characteristics of the study population
| EP (n = 66) | IUP (n = 65) | SAB (n = 61) | p value | |
|---|---|---|---|---|
| Maternal age median (years, IQR) | 28 [24, 32] | 26 [21, 30] | 30 [27, 35] | 0.0002 |
| Hispanic | 4 (6%) | 2 (3%) | 2 (3%) | 0.64 |
| Caucasian | 8 (12%) | 10 (15%) | 13 (21%) | 0.29 |
| Black | 55 (83%) | 47 (72%) | 42 (69%) | |
| Other | 3 (5%) | 8 (12%) | 6 (10%) | |
| hCG, median (mIU/mL, IQR) | 2391 [902, 6965] | 27991 [10816, 56149] | 8315 [4060, 18330] | < 0.0001 |
| Gestational age median (days, IQR) | 42 [37, 49] | 50 [43, 58] | 45 [42, 53] | 0.0002 |
Open in a separate window
Kruskal-Wallis test was used for continuous measures, and Pearson Chi-square or Fisher-exact tests were used for categorical variables
Assessing reproducibility of individual biomarkers
The medians and ranges of all six markers produced for subject by outcome are presented in (Table (Table2)2) and compared to values obtained in our deviation study. Markers that were found to be statistically different across outcomes in the derivation population were also found to be statistically different across outcomes in this new validation population. Exceptions include glycodelin, in which levels were shown to have significantly different medians among each pregnancy outcome in the derivation study, but values were similar in this validation population.
Table 2
Biomarker distributions by pregnancy outcome and source
| Biomarker | EP | IUP | SAB | p value+ | p value* | |
|---|---|---|---|---|---|---|
| ADAM12 | D | 0.43 [0.26–0.76] | 0.53 [0.36–0.82] | 0.59 [0.39–0.87] | 0.02 | 0.053 |
| V | 0.42 [0.18–0.85] | 0.77 [0.43–1.47] | 0.88 [0.42–1.23] | < 0.001 | ||
| Progesterone | D | 5.0 [2.9–13.5] | 21.0 [15.0–24.7] | 2.6 [1.5–6.4] | < 0.001 | 0.016 |
| V | 4.6 [2.4–12.2] | 19.6 [14.18–27.9] | 9.1 [5.17–14.4] | < 0.001 | ||
| Activin A | D | 206.8 [161.4–254.1] | 289.3 [225.4–365.8] | 325.9 [265.6–460.3] | < 0.001 | 0.015 |
| V | 231.1 [175.3–290.6] | 334.3 [255.9–514.8] | 402.5 [273.7–602.4] | < 0.001 | ||
| PAPP-A | D | 0.18 [0.13–0.24] | 0.20 [0.15–0.27] | 0.26 [0.17–0.34] | < 0.001 | 0.819 |
| V | 0.05 [0.00–0.09] | 0.55 [0.06–1.44] | 0.75 [0.29–1.67] | < 0.001 | ||
| Glycodelin | D | 0.32 [0.22–0.60] | 0.2 [0.1–0.41] | 0.26 [0.1–0.86] | 0.008 | 0.053 |
| V | 0.00 [0.00–3.87] | 0.00 [0.00–12.88] | 0.00 [0.00–6.90] | 0.671 | ||
| hCG | D | 823 [120–18733] | 4271 [124–19895] | 632 [102–17714] | < 0.001 | < 0.001 |
| V | 2391 [902, 6965] | 27991 [10816, 56149] | 8315 [4060, 18330] | < 0.001 | ||
Open in a separate window
Values of presented in median [interquartile range]. D, derivation study; V, validation study
+Kruskal-Wallis test for differences in median by outcome (EP vs. IUP vs. SAB)
*Wilcoxon rank sum test for differences in median by source (validation vs. derivation study)
We also evaluated biomarker differences between the derivation versus validation population (Table (Table2).2). The distributions of Progesterone, Activin A, and beta human chorionic gonadotropin were statistically different among the derivation and validation populations, although with overlapping interquartile ranges.
As a final assessment of reproducibility, we compared each biomarker’s individual performance to determine discrimination among pregnancy outcomes by assessing area under the curve for the prediction of outcome for each marker individually (Table (Table3).3). The biomarkers that contributed to the pregnancy location algorithm (Activin A, glycodelin, and ADAM12) and pregnancy viability algorithm (PAPP-A and Progesterone) all showed similar AUCs to the derivation study subject population.
Table 3
Individual biomarker performance for pregnancy location and viability
| Biomarker | Pregnancy location: EP vs. IUP + SAB | Pregnancy viability: IUP vs. EP + SAB | ||
|---|---|---|---|---|
| Derivation data AUC [95% CI] | Validation data AUC [95% CI] | Derivation data AUC [95% CI] | Validation data AUC [95% CI] | |
| Activin A | 0.79*++ [0.73–0.85] | 0.77*++ [0.70, 0.83] | 0.56 [0.49–0.64] | 0.58 [0.49, 0.66] |
| PAPP-A | 0.61 [0.54–0.69] | 0.80 [0.734, 0.87] | 0.55++ [0.47–0.62] | 0.61++ [0.52, 0.69] |
| Glycodelin | 0.61* [0.54–0.68] | 0.54* [0.45, 0.62] | 0.60 [0.52–0.68] | 0.53 [0.43, 0.63] |
| ADAM12 | 0.60++ [0.52–0.69] | 0.67++ [0.59, 0.75] | 0.51 [0.43–0.58] | 0.59 [0.51, 0.68] |
| Progesterone | 0.58 [0.50–0.65] | 0.77 [0.69, 0.85] | 0.88*++ [0.83–0.93] | 0.84*++ [0.79, 0.90] |
| hCG | 0.61 [0.53–0.68] | 0.80 [0.73, 0.87] | 0.76 [0.70–0.83] | 0.81 [0.74, 0.89] |
Open in a separate window
Values in bold indicate biomarkers that contribute to the sensitivity and/or specificity trees as shown in Fig. Fig.11
*Contribute to the sensitivity tree; ++contribute to the specificity tree
Assessing number classified and accuracy of prediction, refining algorithms, and subgroup comparisons
We then evaluated the number of pregnancies classified and accuracy of the decision algorithms using the original decision trees (Fig. (Fig.1).1). Pregnancy location was conclusively classified in 53% (n = 94) of the whole study sample with 78% accuracy (compared to conclusive classification of 29% (n = 67) with 100% accuracy in the derivation population). Pregnancy viability was conclusively classified in 58% (n = 112) of the new sample with 89% accuracy (compared to conclusive classification of 61% (n = 140) of patients with 97% accuracy) (Table (Table4).4). Results were similar in the model where the threshold for each biomarker was revised using CART on the new dataset which had modest differences in the mean and range of each biomarker, and if we removed glycodelin and added hCG. The results were also similar in both the derivation and the validation sample in the subgroups of subject with an hCG level below 2000 mIU/mL and a gestational age less than 6 weeks (Table (Table44).
Table 4
Comparison of performance of original algorithm, revisions, and important subgroups
| Original | Revision of thresholds | Model revision: drop glycodelin and add hCG | hCG < 2000 | Gestational age < 6 weeks | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Conclusive classification | Accuracy among classified [95% CI] | Conclusive classification | Accuracy among classified [95% CI] | Conclusive classification | Accuracy among classified [95% CI] | Conclusive classification | Accuracy among classified [95% CI] | Conclusive classification | Accuracy among classified [95% CI] | ||
| Location | 67/228 (29%) | 100% [95–100] | 67/228 (29%) | 100% [95–100] | 75/228 (33%) | 99% [93–100] | 27/112 (24%) | 100% [87–100] | 30/80 (38%) | 100% [88–100] | Derivation |
| 94/175 (54%) | 78% [69–87] | 69/176 (39%) | 78% [67–87] | 87/185 (47%) | 85% [76–92] | 9/32 (28%) | 89% [52–100] | 87/185 (47%) | 85% [76–92] | Validation | |
| Viability | 140/230 (61%) | 97% [93–99] | 140/230 (61%) | 97% [93–99] | 143/230 (62%) | 97% [92–99] | 82/112 (73%) | 99% [93–100] | 38/80 (48%) | 89% [76–97] | Derivation |
| 112/192 (58%) | 89% [81–94] | 112/192 (58%) | 89% [81–94] | 108/175 (64%) | 87% [79–93] | 29/34 (85%) | 97% [82–100] | 35/56 (63%) | 91% [77–98] | Validation | |
Open in a separate window
Summarizes the results of our validation study compared to the original derivation study, subsequent revisions, and important subgroup comparisons
Discussion
A companion diagnostic for pregnancy location and/or non-viability of an early gestation will be useful in multiple ways. The utility of a predictive test, or panel of tests will likely be greatly informed by its validity and accuracy. For a prediction model to be useful, it should be valid in a separate population from which it was derived [26, 27]. External validation includes both obtaining similar concentrations for each biomarker in a new population (assessing the heterogeneity of the predictor) and reproducing the test characteristics of a proposed companion diagnostic using a population distinct from its derivation. External validation is a critical step in biomarker development and a stage when many biomarkers fail [22]. These data validate the possibility of prediction based on a model of multiplexed biomarkers in an external population from which they were derived.
The evaluated model to detect viability and location of pregnancy is based on iterative analysis [17, 21]. We have demonstrated that a possible companion diagnostic is best derived from multiple markers. Because there are three outcomes (IUP, SAB, and EP), it is preferable to dichotomize the outcome into two tests based on different biomarkers or thresholds (one for location and one for viability). Additionally, we have demonstrated that because the error of missing an EP or interruption of an IUP are both grave, it is optimal to develop a test that maximize accuracy of prediction, rather than maximizing sensitivity or specificity (at the expense of the other). If a woman received a prediction that is indeterminate, care can proceed using the current standard of serial hCG concentration and ultrasound [28].
For the purposes of external validation, it is important to assess utility of biomarkers in the intended population of use and not to artificially make the characteristics of the population similar. Examination of our baseline characteristics of each pregnancy subgroup demonstrates that, in general, women diagnosed with an IUP present with higher hCG, women with an EP present with a lower hCG, and women with a miscarriage present with an intermediate value. Additionally, those with EPs and SABs presented at earlier gestational ages. Given the typical manner in which women ultimately diagnosed with each of these entities present to care, these differences are expected and representative of clinical practice.
To assess if a biomarker reproduced similar values and prognostic results in a separate population, we assessed the median values of each biomarker, in each outcome, in both the derivation and validation population. We also assessed the reproducibility of concentration of each biomarker (stratified by outcome) comparing median values in the original and validation population. Finally, the ability of each biomarker to discriminate among pregnancy outcome was assessed with AUC. Our findings suggest that the concentration of most of biomarkers were reproducible in the separate external population. Exceptions included the median concentration of glycodelin, where there were statistically different outcomes in one, but not both, of the two populations (Table (Table22).
Assessment of the median value of each marker (stratified by outcome) across populations was also similar with a few exceptions. The median values of the biomarkers Progesterone, Activin A, and hCG were statistically different among the derivation and validation populations but may still be useful marker due to overlapping interquartile ranges in the two populations (Table (Table2).2). Assessment of each marker to discriminate outcome demonstrated that AUC for each biomarker (individually) was also very similar in both populations (Table (Table3).3). Based on these findings, we concluded that apart from glycodelin, the markers in our original multiple marker panel (Activin A, ADAM12, Progesterone, and PAPP-A) demonstrated good initial individual validation as potential markers for pregnancy viability and location.
The next step of validation was the use of the markers in combination to assess the prediction of our models. External validation is the action of testing the original prediction model in a set of new patients to determine whether the model works to a satisfactory degree. We noted that overall, there was a slight increase in the number of subjects classified and a slight decrease in accuracy of prediction. Prediction was not dramatically improved with revision of the thresholds of the cut points for markers to better reflect the actual values obtained in the new sample (Table (Table44).
There was poor reproducibility of the results obtained with glycodelin. However, the number of subjects classified and the accuracy of prediction was similar when glycodelin was removed from the model and replaced with hCG (Table (Table4).4). The original and revised model noted similar test characteristics in the clinically important subgroups of women with pregnancies < 6 weeks of gestation and pregnancies with hCG < 2000 mIU/mL (Table (Table44).
Overall, these data validate the concept of using artificial intelligence (classification tree analysis) of multiple putative biomarkers and a twostep logic to optimize accuracy, as it demonstrated similar results in a new external population. However, mathematical validation of a concept does not mean the model is clinically useful. Overall test characteristics were only moderately predictive. The clinical utility of a predictor of the presence or absence of an ectopic pregnancy that can provide an answer in 54% of the population with accuracy of 78–89% is debatable. Indeed, this accuracy is largely similar to the rate of inconclusive results seen using the current standard of care, transvagin*l ultrasound. The prediction of a nonviable gestation was modestly better, with definitive prediction in about 60% of subjects with accuracy ranging from 87 to 97%. There is no demarcated threshold in a diagnostic test that defines when a test is clinically useful. Moreover, there is no consensus if a test for women at risk for early pregnancy loss should maximize sensitivity, specificity, or accuracy.
This study represents early development and validation of the use of biomarkers to aid in prediction of women at risk for a nonviable or ectopic pregnancy using well-phenotyped pregnancy outcomes. This is an important and necessary step in development of possible clinical tests, and results should be disseminated even when negative. When optimal biomarkers are validated, further steps will include a population-based cohort to allow calculation of predictive value specifically in women with more clinically challenging presentations such as a pregnancy of unknown location. Additionally, a final set of validated biomarkers should ultimately be assessed in combination with current diagnostic methods such as transvagin*l ultrasound and perhaps serial assessment of hCG and progesterone.
Based on these results, it is possible that prediction of outcome in the complex clinical situation of early pregnancy loss is not achievable due to biological overlap of the outcomes. It is also possible that the markers in the model can be improved with better candidates, and discovery of more predictive novel markers may improve discrimination. Such work is ongoing.
In conclusion, these results demonstrate that the utilization of a panel of biomarkers using logic to maximize test accuracy of a prediction of pregnancy location as well as a prediction of pregnancy viability was reproducible and validated in an external population from which it was derived. However, due to a large number of subjects receiving an indeterminate result and moderate accuracy, the clinical utility of these potential companion diagnostic tests is limited. To better predict outcome for women at risk for early pregnancy failure, further research is needed to discover and validate new markers that may be used alone or in combination with the markers assessed in this study.
Supplementary information
ESM 1(23K, docx)
Supplemental Table 1
Authors’ contributions
All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by Kurt T. Barnhart, Suneeta Senapati, Mary D. Sammel, Peter Takacs, Jared C Robins, and Daniel J. Haisenleder. The first draft of the manuscript was written by Kassie Bollig, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Funding
This study was funded by NIH R01HD076279.
Declarations
Ethics approval
Approval was obtained from the ethics committees of the University of Pennsylvania, Eastern Virginia Medical School, and Northwestern University. The procedures used in this study adhere to the tenets of the Declaration of Helsinki.
Consent to participate
Informed consent was obtained from all individual participants included in the study.
Competing interests
The authors declare no competing interests.
Footnotes
Data regarding any of the subjects in the study has not been previously published.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
1. Creanga AA, Syverson C, Seed K, Callaghan WM. Pregnancy-related mortality in the United States, 2011-2013. Obstetrics and gynecology. 2017;130:366–373. doi:10.1097/AOG.0000000000002114. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
2. American College of Obstetrics & Gynecologists ACOG Practice Bulletin No. 191: tubal ectopic pregnancy. Obstetrics and gynecology. 2018;131:e65–e77. doi:10.1097/AOG.0000000000002464. [PubMed] [CrossRef] [Google Scholar]
3. Barnhart KT, Simhan H, Kamelle SA. Diagnostic accuracy of ultrasound above and below the beta-hCG discriminatory zone. Obstetrics & Gynecology. 1999;94:583–587. [PubMed] [Google Scholar]
4. Barnhart KT, Sammel MD, Rinaudo PF, Zhou L, Hummel AC, Guo W. Symptomatic patients with an early viable intrauterine pregnancy: hCG curves redefined. Obstetrics & Gynecology. 2004;104:50–55. doi:10.1097/01.AOG.0000128174.48843.12. [PubMed] [CrossRef] [Google Scholar]
5. Barnhart K, Sammel MD, Chung K, Zhou L, Hummel AC, Guo W. Decline of serum hCG and spontaneous complete abortion: defining the normal curve. Obstetrics & Gynecology. 2004;104:975–981. doi:10.1097/01.AOG.0000142712.80407.fd. [PubMed] [CrossRef] [Google Scholar]
6. Silva C, Sammel MD, Zhou L, Gracia C, Guo W, Hummel AC, et al. hCG profile for women with an ectopic pregnancy. Obstetrics & Gynecology. 2006;107:605–610. doi:10.1097/01.AOG.0000198635.25135.e7. [PubMed] [CrossRef] [Google Scholar]
7. Barnhart KT. Ectopic pregnancy. New England Journal of Medicine. 2009;361:379–387. doi:10.1056/NEJMcp0810384. [PubMed] [CrossRef] [Google Scholar]
8. Seeber BE, Barnhart KT. Suspected ectopic pregnancy. Obstetrics & Gynecology. 2006;107(2 pt 1):399–413. doi:10.1097/01.AOG.0000198632.15229.be. [PubMed] [CrossRef] [Google Scholar]
9. Morse CB, Sammel MD, Shaunik A, Allen-Taylor L, Oberfoell NL, Takacs P, et al. Performance of human chorionic gonadotropin curves in women at risk for ectopic pregnancy: exceptions to the rules. Fertility and Sterility. 2012;97:101–106. doi:10.1016/j.fertnstert.2011.10.037. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
10. Mueller MD, Raio L, Spoerri S, Ghezzi F, Dreher E, Bersinger NA. Novel placental and nonplacental serum markers in ectopic versus normal intrauterine pregnancy. Fertility and Sterility. 2004;81:1106–1111. doi:10.1016/j.fertnstert.2003.08.049. [PubMed] [CrossRef] [Google Scholar]
11. Beer LA, Tang HY, Sriswasdi S, Barnhart KT, Speicher DW. Systematic discovery of ectopic pregnancy serum biomarkers using 3-D protein profiling coupled with label-free quantitation. Journal of Proteome Research. 2011;10:1126–1138. doi:10.1021/pr1008866. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
12. Rausch ME, Beer L, Sammel MD, Takacs P, Chung K, Shaunik A, et al. A disintegrin and metalloprotease protein-12 as a novel marker for the diagnosis of ectopic pregnancy. Fertility and Sterility. 2011;95:1373–1378. doi:10.1016/j.fertnstert.2010.12.040. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
13. Horne AW, Shaw JL, Murdoch A, McDonald SE, Williams AR, Jabbour HN, et al. Placental growth factor: a promising diagnostic biomarker for tubal ectopic pregnancy. The Journal of Clinical Endocrinology & Metabolism. 2011;96:E104–E108. doi:10.1210/jc.2010-1403. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
14. Daponte A, Pournaras S, Polyzos NP, Tsezou A, Skentou H, Anastasiadou F, et al. Soluble FMS-like tyrosine kinase-1 (sFlt-1) and serum placental growth factor (PlGF) as biomarkers for ectopic pregnancy and missed abortion. The Journal of Clinical Endocrinology & Metabolism. 2011;96:E1444–E1451. doi:10.1210/jc.2011-0037. [PubMed] [CrossRef] [Google Scholar]
15. Zhang X, Wang C. Predictive value of PAPP-A for ectopic pregnancy and analysis of related factors. Experimental and Therapeutic Medicine. 2021;22(2):801. doi:10.3892/etm.2021.10233. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
16. Ugurlu EN, Ozaksit G, Karaer A, Zulfikaroglu E, Atalay A, Ugur M. The value of vascular endothelial growth factor, pregnancy-associated plasma protein-A, and progesterone for early differentiation of ectopic pregnancies, normal intrauterine pregnancies, and spontaneous miscarriages. Fertility and Sterility. 2009;91:1657–1661. doi:10.1016/j.fertnstert.2008.02.002. [PubMed] [CrossRef] [Google Scholar]
17. Rausch ME, Sammel MD, Takacs P, Chung K, Shaunik A, Barnhart KT. Development of a multiple marker test for ectopic pregnancy. Obstetrics & Gynecology. 2011;117:573–582. doi:10.1097/AOG.0b013e31820b3c61. [PubMed] [CrossRef] [Google Scholar]
18. Witt BR, Wolf GC, Wainwright CJ, Johnston PD, Thorneycroft IH. Relaxin, CA-125, progesterone, estradiol, Schwangerschaft protein, and human chorionic gonadotropin as predictors of outcome in threatened and nonthreatened pregnancies. Fertility and Sterility. 1990;53:1029–1036. doi:10.1016/S0015-0282(16)53580-8. [PubMed] [CrossRef] [Google Scholar]
19. Skubisz MM, Brown JK, Tong S, Kaitu’u-Lino T, Horne AW. Maternal serum macrophage inhibitory cytokine-1 as a biomarker for ectopic pregnancy in women with a pregnancy of unknown location. PLoS One. 2013;8:e66339. doi:10.1371/journal.pone.0066339. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
20. Martínez-Ruiz A, Sarabia-Meseguer MD, Pérez-Fornieles J, Vílchez JA, Tovar-Zapata I, Noguera-Velasco JA. Placental growth factor, soluble fms-like tyrosine kinase 1 and progesterone as diagnostic biomarkers for ectopic pregnancy and missed abortion. Clinical biochemistry. 2014;47:844–847. doi:10.1016/j.clinbiochem.2014.03.013. [PubMed] [CrossRef] [Google Scholar]
21. Senapati S, Sammel MD, Butts SF, Takacs P, Chung K, Barnhart KT. Predicting first trimester pregnancy outcome: derivation of a multiple marker test. Fertility and Sterility. 2016;106:1725–1732. doi:10.1016/j.fertnstert.2016.08.044. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
22. Palmer SS, Barnhart KT. Biomarkers in reproductive medicine: the promise and can it be fulfilled? Fertility and Sterility. 2013;99:954–962. doi:10.1016/j.fertnstert.2012.11.019. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
23. Barnhart K, van Mello NM, Bourne T, Kirk E, Van Calster B, Bottomley C, et al. Pregnancy of unknown location: a consensus statement of nomenclature, definitions, and outcome. Fertility and Sterility. 2011;95:857–866. doi:10.1016/j.fertnstert.2010.09.006. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
24. Levi CS, Lyons EA, Zheng XH, Lindsay DJ, Holt SC. Endovagin*l ultrasound: demonstration of cardiac activity in embryos of less than 5.0 mm in crown-rump length. Radiology. 1990;176:71–74. doi:10.1148/radiology.176.1.2191372. [PubMed] [CrossRef] [Google Scholar]
25. Goldstein SR. Significance of cardiac activity on endovagin*l ultrasound in very early embryos. Obstetrics & Gynecology. 1992;80:670–672. [PubMed] [Google Scholar]
26. Altman DG, Royston P. What do we mean by validating a prognostic model? Statistics in medicine. 2000;19:453–473. doi:10.1002/(SICI)1097-0258(20000229)19:4<453::AID-SIM350>3.0.CO;2-5. [PubMed] [CrossRef] [Google Scholar]
27. Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model. BMJ. 2009;338:b605. doi:10.1136/bmj.b605. [PubMed] [CrossRef] [Google Scholar]
28. Senapati S, Barnhart KT. Biomarkers for ectopic pregnancy and pregnancy of unknown location. Fertility and Sterility. 2013;99:1107–1116. doi:10.1016/j.fertnstert.2012.11.038. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
Articles from Journal of Assisted Reproduction and Genetics are provided here courtesy of Springer Science+Business Media, LLC
