What is the best allocation of a fixed experimental budget…replicates or additional experimental design points?
Series article 7. #metabolomics #lipidomics #massspectrometry #cancer #cancerresearch #analyticalchemistry
Looking for ways to clean up your metabolomic and other omic data? Check out these tips. 42 Factors Series – #7
Welcome back to Article 7 in the 42 Factors Series. In the wake of recent, high-profile articles (Wasserstein et al.; Ruberg et al.) addressing the role of statistics in scientific decision-making, today I’ll attempt to contribute to the debate by addressing a question that I’m frequently asked: “How many replicates should I use in my metabolomics experiment?” If you use omic technologies to identify biomarkers, drug targets, or biological mechanisms, you’ll want to keep reading.
Factor – Number of Replicates
Throughout the wide world of experimental science, we include statistical replicates in experiments to compute the error associated with measurements and to assign “significance” (or lack thereof) to differences between groups. In the clinical trial setting, statistics play a major role, for example, in determining whether a treatment was effective. In clinical diagnostics, statistics play a major role in establishing thresholds that define positive and negative test results.
By contrast, there are many contexts in which statistics are unnecessary (I can hear the statisticians groaning, but stay with me), and here I’ll attempt to convince you that exploratory metabolomics is one such context. When omic experiments are exploratory in nature, the use of replicates to create statistical significance often comes at a cost of eliminating broader testing of experimental factors. Deciding the number of replicates to analyze is frequently driven by cost, so investigators should think carefully about the balancing act between replicate sampling vs. coverage of experimental parameters that might influence the observed effect. In short, false positives (avoided by having sufficient replicates) are a nuisance, but false negatives (avoided by having sufficient coverage of key experimental parameters) can be even more costly.
To support my claim, let’s first consider an experimental design in which we have two treatment groups (vehicle, drug), two time points (0, 24 h after treatment), and n=3 biological replicates. (As discussed in the previous article on fasting vs. fed states, we’ll model the fed state at t=0.)
A nutrient metabolite like glutamine may yield the following results:
The lack of difference between the black and red bars at each time point would prompt us to conclude that glutamine is not modulated by the drug treatment. Unfortunately, that conclusion could be completely wrong; consider the scenario in which this drug actually causes acceleration of glutamine degradation:
This example illustrates a case where a statistics-prioritized experimental design yielded a false negative conclusion, failing to identify a real biological difference between the two treatment groups. Regretfully, this situation is not uncommon in metabolomics.
For statistical analysis to deliver accurate conclusions in any experiment, replicates should capture the largest source of variation in the experiment. In drug trials, the largest source of variation is almost invariably the patient, so data are acquired from lots of patients. In metabolomics we’ve identified more than 42 Factors that introduce variation, and, as the example above illustrates, time is usually a primary source of variation. Since metabolism is a dynamic process, this makes sense.
The selection of time points is clearly important in metabolomics, potentially causing the investigator to draw the incorrect conclusion of “no effect” or “no difference between groups.” Accordingly, I recommend that investigators not worry about statistical significance in exploratory, range-finding studies. Particularly when dealing with expensive technologies like metabolomics, you’ll increase your odds of success by allocating your experimental budget toward additional time points, with just one (n=1) sample per time point. After identifying time points that exhibit maximal differences between groups, we encourage researchers to conduct a validation experiment that includes replicates for the purpose of statistically substantiating the conclusions.
Time is also a major source of variation in preclinical and clinical studies. I often quip that if I left n=10 freshly drawn blood samples on the laboratory bench overnight, the results may be statistically significant but could also be completely inaccurate, since many metabolites would degrade. I maintain the position that statistics do not need to be a priority in exploratory studies, and service providers should guide investigators responsibly with an emphasis on considering major sources of variation in experimental workflows.
Take-home messages: 1) Statistical design in experiments should focus on capturing the major source(s) of variation; 2) in metabolomics, the major source of variation is usually time; 3) we recommend preparing samples at multiple time points without replicates for exploratory metabolomics; 4) incorporate replicates in validation studies.
*After writing this article I found a publication supporting the assertion that investigators should emphasize selection of time points over replicates. This philosophy is gaining traction.
Thanks to present and former members of my team for their efforts toward studying this factor.
Do you have any pressing metabolomics questions? Leave a comment.
0 comments