Evaluating the effectiveness of a clinical weight loss program
In this study, I analyze a dataset of a 12-month clinical weight loss program to find out:
what factors affect the 12 month weight loss results;
whether an intervention was effective in increasing weight loss.
The dataset contains 234 participants of the weight loss study, and for each participant, we have information on his/her age, race, type of treatment, and the weight change 12 month after the treatment. All relevant biological information has been condensed into one variable called biomarker. The data has been processed to ensure privacies of the participants.
This project was a final project for the statistical modeling class (graduate-level) at UT Austin, Dec 2014.
I plotted every parameter against each other to have a quick look at the relationships between different parameters. The age, race, biomarker variables are self-explanatory, and the other two variables are
- wtch - 12 month weight loss in kg
- treatment - 1= control, 2 or 3 = intervention
We can see the biomarker seems to linearly correlate with the age of the subject. The total numbers of samples are very small in race 2 and 4, and the race 3 seems to have a younger population than race 1.
A quick look
First, let’s contruct a full model with all data points and all avaliable variables as covariants.
￼￼This model does not fit the data very well. The R-squared value is 0.099, which means only about 10 percent of the variations in the data is explained by the model. However, the model is still significant. With a p value of 0.001, we can safely reject the null hypothesis of no relationship all all between the weightloss and all the variables. Race 3 is the most significant covariant with a p value of 5.64e-5, and age is the second most important covariance with a p value of 0.113.
Considering the median of the weight loss is -9.89, the residuals are very large for this model, but we don’t see any strucutre in the residual vs. fittet plot. 10 points show very large leverage. This could be a problem because these 10 points can have too big influence on our accessing the model fit. We may want to exclude those data point, lower their weights, or use a different cost function.
Adding Interaction Terms
To see if including interaction terms between different variables will imporve the model fit, I construct a series of expanded models:
|1-1||Full model with race and treatment as factor variables|
|1-2||Same as 1-1 but without the treatment as a covariant|
|2-1||Add dummy variables for race to include interaction terms between age and race|
|2-2||Same as 2-1 but without the treatment as a covariant|
|3-1||Include dummy variables for both race and treatment|
The BICs and R-squared of the full model and models with interaction terms are shown below. Models with interaction terms have larger BICs and smaller R-squared compared to the full model, which means they fit the data worse than the full model. So I decided not to include the interaction terms.
As seen in exploritory analysis, biomarker has the largest P value, and it seems to correlate with age. We did a simple linear regression model for biomarker ~ age, and found R squared to be 0.4651. This means age predicts the biomarker reasonably well, but the two are not exactly the same. However, since those two variables are not independent of each other, including biomarker will greatly inflate the variance. I decided to examine a few reduced models.
|4-1||Reduced model without biomarker as a covariant|
|4-2||Same as 4-1 but without the treatment as a covariant|
|4-1c||Same as 4-1 but without the two outliers|
|5-1||Reduced model with only age and treatement as covariants|
|5-2||Reduced model with only age as the covariant|
We can see models without race doe not fit the data well (5-1 and 5-2, with larger BIC and much smaller R- squared compared to other models). On the other hand, excluding the two outliers can significantly improve our model fit (with BIC 27 smaller than the model including the outliers). So we pick model 4-1 c, the reduced model without biomarker as a covariant and without the two outliers for our analysis.
Judging from the expectaion and standard error, we are 68 percent confident that the treatment increases weightloss (a smaller wtch value). Treatment 3 has slightly larger effect than Treatment 2, with a expect (1.722 - 1.348 = 0.374) increase in weightloss.
We added dummy variables for the factor variable race and treatment, and constructed a series of models in JAGS. The general form of the model is: wtch ~ N(mu, tau) mu ~ BX
Since we don’t have any prior knowlege associate with the weightloss, we simply choose diffuse normal priors for all the betas in all of our models. We use a diffuse gamma prior for all the taus.
The covariances and DIC for each model is shown in the table below. We are using penalized deviance from dic.samples as our measurement.
|1||age + bio + race + treat||1616|
|2||age + race + treat||1614|
|3||age + race3 only + treat||1613|
|4||age + treat||1635|
|5||age + race||1619|
In model 1, the 95% interval of all the covariance includes zero, therefore we can’t constrain the importance of parameters with great confidence. In model 2, we are 95% confident that the age is a significant factor. Since we have very few samples in race 2 and 4, we decided to try only including race 3 as a covariance. As expected, excluding biomarker imporves the variance in the model parameters. In model 3, both age and race 3 are proved to be significant at 95% confident level. We tried models with no treatment, with no race as covariants, and the model only with age as the covariants. All of them have larger DIC then model 3, and they do not provide any further insight of the data. Therefore we chose model 3 for our inference.
We should note that in the frequentist approach, we included all the races in the model. However, in the Bayesian approach, we only include one dummy variable for race 3. Since we eliminated two variables that are closely dependent on existing variables (race 2, 3, and 4 are not independent of each other), the model significance is improved.
After a few experiments, we decided to thin the data by 100 to improve the mixing. We run 2 chains with 100000 sample each, so we have 2000 samples after thinning. The thinning does not change the modeled coefficients very much. The output parameters of model 3 are (in the order of ‘Inter’, ‘race3’, ‘age’, ‘treat2’, ‘treat3’):
Judging from the coefficients, we are 68% confident that treatment 3 can increase weightloss (with am expected value of -1.69). Treatment 2 is likely to increase the weightloss, but we can’t say it with 68% confidence. On the other hand, race 3 have significantly larger weighloss compared to other races, and the weighloss decreases as age increases. 10 years of increase in age leads to a decrease of 2.2 in weightloss.
Samples for betas and the PDFs of betas are shown below. We can see the mixing is generally well, and we have tighter constrain on treat 2, and treat 3 compared to other variables.