Indicator “Unexpected location”
Loading Tree…
Definition
Observed location parameters differ from the expected location parameter.
Explanation
This indicator applies to continuous variables and targets differences between expected and observed location parameters (e.g. mean or median).
Example
Example 1
In an adult European general population sample a plausible mean body weight in kilograms is expected to lie within 75-90 kg. Yet the observed number is almost 178. At inspection, it is revealed that the unit of measurement was mistakenly converted to pounds, resulting in a weight of approx. 178 pounds instead of 81kg.
Example 2
An assessment of hip circumference in a study is conducted by five examiners. The assignment of participants to examiners is approximately random. This motivates the assumption that means should be approximately the same across examiners. The number of cases and mean hip circumference per examiner is displayed below:
| Examiner ID | N observations | Mean circumference |
|---|---|---|
| 342 | 320 | 102 cm |
| 333 | 180 | 103 cm |
| 231 | 270 | 109 cm |
| 123 | 255 | 102 cm |
| 345 | 23 | 94 cm |
The overall mean is 104cm with a standard deviation of 10cm. Key findings are:
Observer 342, 333 and 123 have highly comparable results implicating a high degree of standardization between them.
Observer 231 has a substantially higher mean . The number of examinations is high, therefore it is concluded that some quality issue exists
Observer 345 has a much lower mean of 94cm. However, the number of examinations is low. This introduces uncertainty as to the interpretation of this deviation. it may be possible that a minor series of subjects with a lower circumference has occurred.
Note: Without further information it cannot be told which examiners perform better should some unexpected discrepancy be observed.
Guidance
Checks for unexpected location parameters of measured variables should be conducted in any study.
Deviations of observed from expected location parameters may indicate a wide range of issues such as examiner effects, device effects but also sampling issues. For example, an unexpected location parameter may reflect sampling bias rather than information bias.
In a designed study, little effects of study design factors, such as devices or examiners, should be exerted on measurements. Finding associations of relevance between these factors and measurements are commonly indicative of measurement error.
For any interpretation it is important to take the number of cases into account. Low numbers may introduce a considerable amount of uncertainty.
Interpretation
Within variables:
The larger the deviation of expected and observed location parameters the larger the probability of a lower data quality.
Across variables:
The higher the number or percentage of variables affected by unexpected location parameter related issues, the higher the probability of a low data quality.
Descriptors
Implementations
Literature
Nonnemacher M, Nasseh D, Stausberg J. Datenqualität in der medizinischen Forschung: Leitlinie zum Adaptiven Datenmanagement in Kohortenstudien und Registern. Berlin: TMF e.V..; 2014.
Stausberg J, Bauer U, Nasseh D, et al. Indicators of data quality: review and requirements from the perspective of networked medical research MIBE 2019;15(1):1-8.
