Background: Missing data is a common nuisance in eHealth research: it is hard to prevent and may invalidate research findings.
Objective: In this paper several statistical approaches to data "missingness" are discussed and tested in a simulation study.
Basic approaches (complete case analysis, mean imputation, and last observation carried forward) and advanced methods (expectation
maximization, regression imputation, and multiple imputation) are included in this analysis, and strengths and weaknesses
are discussed. Methods: The dataset used for the simulation was obtained from a prospective cohort study following participants
in an online self-help program for problem drinkers. It contained 124 nonnormally distributed endpoints, that is, daily alcohol
consumption counts of the study respondents. Missingness at random (MAR) was induced in a selected variable for 50% of the
cases. Validity, reliability, and coverage of the estimates obtained using the different imputation methods were calculated
by performing a bootstrapping simulation study. Results: In the performed simulation study, the use of multiple imputation
techniques led to accurate results. Differences were found between the 4 tested multiple imputation programs: NORM, MICE,
Amelia II, and SPSS MI. Among the tested approaches, Amelia II outperformed the others, led to the smallest deviation from
the reference value (Cohen's d = 0.06), and had the largest coverage percentage of the reference confidence interval (96%).
Conclusions: The use of multiple imputation improves the validity of the results when analyzing datasets with missing observations.
Some of the often-used approaches (LOCF, complete cases analysis) did not perform well, and, hence, we recommend not using
these. Accumulating support for the analysis of multiple imputed datasets is seen in more recent versions of some of the widely
used statistical software programs making the use of multiple imputation more readily available to less mathematically inclined