Lesson 7: Rules and Exceptions
Please go through the lesson, making sure to click the green ‘completed’ button once you have read through the content.
Statistics: Tools, Not Rigid Rules
Statistical tools are intended to provide a critical analysis of events, to promote Evidence-Based Medicine rather than personal opinions, beliefs or definitive statements. Therefore it would be paradoxical and naive to impose the same methods to all situations, as a new scientific ideology (paradigm). The tools must be adapted to the biological, medical and ethical situations, not the opposite. So, even if the normal distribution is widely encountered, and randomised controlled trials are often the best experimental design, statisticians and methodologists continuously develop new tools for uncommon situations.
In some situations, the distribution of values shows an asymmetrical shape as illustrated below.
The number of children per family.
- the most frequent families include two children,
- this number cannot be less than 0 (even for statisticians!),
- some families include 4, 6 or even (rarely) up to 10 children.
The multiplication of bacteria evolves in a geometric manner, e.g. doubling every 3 hours in a culture.
So the standard values for diagnosis of urinary infections ranged as:
<1 000 CFU/ml : sterile
< 10 000 CFU/ml: no infection
From 10 000 to 100 000 CFU/ml: possible infection
> 100 000 CFU/ml: infection
Briefly, the number of 0 of the value (its logarithm, log10) is considered rather the value itself; as illustrated in the writing of values: 10 to the power of 3 (for 1000), 10 to the power of 5 (for 100,000), etc. In such distributions, the use of the logarithm of the values render the distribution symmetrical, therefore the distribution is called Log-normal distribution. In these cases, the usual Standard Deviation, ± SD, does not make sense due to the asymmetry. The Confidence Intervals depict the variability better.
Most often, biological parameters depend on numerous factors, resulting in a normal (or log-normal) distribution.
Nevertheless, some distributions may include two or more peaks, especially when one given factor deeply influences the parameter. When the factor is identified, e.g. gender for hormone levels, genetic factor for metabolism, the different populations are analysed separately. In other cases, the multi-modal distributions makes the usual tools for description (mean, SD) unsuitable and the Gaussian toolbox cannot be used.
Fortunately, the statisticians developed tests requiring no conditions concerning the distribution of the compared parameters. For example, rank-tests are not based on the values but on the rank of individuals in a common sorting. Despite of the restriction in the amount of information (rank may be determined on the basis of individual values, not the contrary), such tests, like the Mann Whitney U test are efficient especially in the case of very dispersed distributions. A statistical method is called robust when it has little dependency on a given statistical model. So, rank-tests, which are part of non-parametric statistics, are robust tests.
“RCT”: Gold Standard Not Universal Duty
In Randomised Controlled Trials (RCT), the number of participants is usually pre-defined. The comparison of treatments is only conducted after the last patient included has completed the study. In a sequential analysis, the number of participants is not fixed in advance. Instead data are evaluated as they are collected, the study is stopped in accordance with a pre-defined stopping rule as soon as significant results are observed. This scheme may be interesting when the knowledge about expected effect or variability is poor, (double risk of lack of conclusion or excessive number of participants). A conclusion may sometimes be reached at an earlier stage than with non-sequential schemes, at consequently lower financial and/or human cost. Because a new group of participants is only included if, after analysis of the results of the former group, the question remains unsolved, the sequential analysis is only usable with endpoints with a short response-to-treatment time.
The design of an adaptive clinical trial prospectively plans opportunities for modification of some aspects of the study, according to the data still to be acquired.
The nature, time and procedures of these modifications are strictly defined in the protocol. The adaptation may concern:
- The number of participants after reassessment of the difference between treatments (based on the early termination of the study if the required number seems unrealistic (avoiding a non-conclusive study)
- The criteria of inclusion, narrowing for reasons of feasibility or focusing on the population of special interest
- The randomisation procedure especially by modifying the randomisation rate in favour of the most efficient treatment at the time of reassessment.
- The treatment regimens
In a study aiming to define the minimum effective dose of a new drug, dosages of 12, 25, 50 and 100 mg are tested in 12 participants. Following the first planned assessment, the 12 mg dosage appears inefficient. Subsequently, the reassessed range will be 25, 50, 75 and 100 mg, in 12 other participants. The second reassessment shows that results with 100mg are not better than with 75 mg. The third phase tests only 3 doses: 25, 50 and 75 mg in 12 participants. Dosage reassessment resulted in focusing the tested dosages around the goal which is both methodologically (fine-tuning around the dosage of interest) and ethically (more participants with therapeutic dosages) interesting.
Basic vocabulary: Words you should know
Adaptive clinical trials – Bayesian approaches – Sequential analysis – Interim analysis
Technical Vocabulary: Important words to know
ANOVA – Confounding variable – Factorial design – Frequentist methods – Logistic regression – Odds ratio – Dose ranging study
Advanced Vocabulary: Useful words to know
Bayes’ theorem – Cohort study – Frequentist methods – Historical control – Interaction (qualitative and quantative) – Meta-analysis – Multiple logistic regression – Retrospective study – Population Pharmacokinetics
Abstract Only is Risky
In order to develop a range of oestrogen patches, with adapted doses, a pharmacist needs to know the current body weight of the female population.
In the free abstract of a recent epidemiological paper he finds:
63.5 Â± 14.5 kg.
Based on classical models, he expects body weights to be distributed as indicated in the graph.
Fortunately, he orders the full-text paper and discovers another pattern…
The shape of the observed distribution is very different from a Gaussian distribution.
The peak of the distribution is in the 50-55kg range i.e. lower than the calculated mean, and the profile is notably asymmetric, narrowed on the left part, enlarged on the right one. This kind of distribution is not so rare in medicine, especially when there is a biological barrier for the lower values and not for the higher ones. For example here, a body weight 70 kg higher than the mean is extremely rare (135 kg), but possible. Conversely a body weight of 65-70 kg, i.e. 5 kg does not make sense.
Mean ± SD Only Makes Sense in Symmetric Distributions
Even if the calculation of mean and standard deviation is always possible, their use is irrelevant for the description of such asymmetrically distributed populations.
Median & Percentiles
Because mean and standard deviation are not convenient in asymmetric distributions, other markers were developed. The peak of the distribution is the more frequent value (here 52.1 kg). This peak is also called mode. Based only on the rank of the observed values, the percentiles describe the distribution of the individuals.
By sweeping from the low to the high values the individuals are counted.
- When 10% of the population has been swept, the value is considered as the 10th percentile,
- When 25% of the population has been swept, the value is considered as the 25th percentile¦
The 50% percentile which divides the population in 2 equal subgroups is also called median. Percentiles describe the distribution based only on observed values, without any hypothesis on a given statistics model.
Statistics tests are based on mathematical models. The most frequent test used to compare 2 means, called student t test is based on a Gaussian (normal) distribution, and therefore is not convenient for atypical distributions. In these situations alternative testings are available.
In some cases, the asymmetry of the distribution may be corrected using a mathematical transformation, often a logarithmic transformation. The transformed variable, resulting in a distribution close to a Gaussian distribution, allows the use of the classical statistics tests. In other situations, no transformation is able to make the observed distribution close to a Gaussian one. As with the percentiles for description, alternative statistics tests are based on the rank of the individual rather than on the value itself. Such tests are called non parametric tests.
Documents for further reading
Typical Histogram Shapes and What They Mean American Association for Quality (ASQ) Quality Tools
Dose-Finding & Reassessment
Experimental designs for phase I and phase I/II dose-finding studies J O’Quigley and S Zohar British Journal of Cancer (2006) 94, 609-613.
A Sequential Test in Practice
Triangular test applied to the clinical trial of azithromycin against relapses in Plasmodium vivax infections Stéphane Ranque, Sékéné Badiaga, Jean Delmont and Philippe Brouqui Malaria Journal 2002, 1:13
Choosing a Statistical Test
A review of usable statistical tests according to the distribution Harvey Motulsky Intuitive Biostatistics, Oxford University Press Inc.