Evaluating the effectiveness of a clinical weight loss program
Introduction
In this study, I analyze a dataset of a 12month clinical weight loss program to find out:

what factors affect the 12 month weight loss results;

whether an intervention was effective in increasing weight loss.
The dataset contains 234 participants of the weight loss study, and for each participant, we have information on his/her age, race, type of treatment, and the weight change 12 month after the treatment. All relevant biological information has been condensed into one variable called biomarker. The data has been processed to ensure privacies of the participants.
This project was a final project for the statistical modeling class (graduatelevel) at UT Austin, Dec 2014.
The data
I plotted every parameter against each other to have a quick look at the relationships between different parameters. The age, race, biomarker variables are selfexplanatory, and the other two variables are
 wtch  12 month weight loss in kg
 treatment  1= control, 2 or 3 = intervention
We can see the biomarker seems to linearly correlate with the age of the subject. The total numbers of samples are very small in race 2 and 4, and the race 3 seems to have a younger population than race 1.
Frequentist Analysis
A quick look
First, let’s contruct a full model with all data points and all avaliable variables as covariants.
￼￼This model does not fit the data very well. The Rsquared value is 0.099, which means only about 10 percent of the variations in the data is explained by the model. However, the model is still significant. With a p value of 0.001, we can safely reject the null hypothesis of no relationship all all between the weightloss and all the variables. Race 3 is the most significant covariant with a p value of 5.64e5, and age is the second most important covariance with a p value of 0.113.
Considering the median of the weight loss is 9.89, the residuals are very large for this model, but we don’t see any strucutre in the residual vs. fittet plot. 10 points show very large leverage. This could be a problem because these 10 points can have too big influence on our accessing the model fit. We may want to exclude those data point, lower their weights, or use a different cost function.
Adding Interaction Terms
To see if including interaction terms between different variables will imporve the model fit, I construct a series of expanded models:
Model  Specification 
11  Full model with race and treatment as factor variables 
12  Same as 11 but without the treatment as a covariant 
21  Add dummy variables for race to include interaction terms between age and race 
22  Same as 21 but without the treatment as a covariant 
31  Include dummy variables for both race and treatment 
The BICs and Rsquared of the full model and models with interaction terms are shown below. Models with interaction terms have larger BICs and smaller Rsquared compared to the full model, which means they fit the data worse than the full model. So I decided not to include the interaction terms.
Reduced models
As seen in exploritory analysis, biomarker has the largest P value, and it seems to correlate with age. We did a simple linear regression model for biomarker ~ age, and found R squared to be 0.4651. This means age predicts the biomarker reasonably well, but the two are not exactly the same. However, since those two variables are not independent of each other, including biomarker will greatly inflate the variance. I decided to examine a few reduced models.
Model Specification  
41  Reduced model without biomarker as a covariant 
42  Same as 41 but without the treatment as a covariant 
41c  Same as 41 but without the two outliers 
51  Reduced model with only age and treatement as covariants 
52  Reduced model with only age as the covariant 
We can see models without race doe not fit the data well (51 and 52, with larger BIC and much smaller R squared compared to other models). On the other hand, excluding the two outliers can significantly improve our model fit (with BIC 27 smaller than the model including the outliers). So we pick model 41 c, the reduced model without biomarker as a covariant and without the two outliers for our analysis.
Judging from the expectaion and standard error, we are 68 percent confident that the treatment increases weightloss (a smaller wtch value). Treatment 3 has slightly larger effect than Treatment 2, with a expect (1.722  1.348 = 0.374) increase in weightloss.
Bayesian Approach
Model Comparison
We added dummy variables for the factor variable race and treatment, and constructed a series of models in JAGS. The general form of the model is: wtch ~ N(mu, tau) mu ~ BX
Since we don’t have any prior knowlege associate with the weightloss, we simply choose diffuse normal priors for all the betas in all of our models. We use a diffuse gamma prior for all the taus.
The covariances and DIC for each model is shown in the table below. We are using penalized deviance from dic.samples as our measurement.
Model  Covariants  DIC 
1  age + bio + race + treat  1616 
2  age + race + treat  1614 
3  age + race3 only + treat  1613 
4  age + treat  1635 
5  age + race  1619 
6  age  1636 
In model 1, the 95% interval of all the covariance includes zero, therefore we can’t constrain the importance of parameters with great confidence. In model 2, we are 95% confident that the age is a significant factor. Since we have very few samples in race 2 and 4, we decided to try only including race 3 as a covariance. As expected, excluding biomarker imporves the variance in the model parameters. In model 3, both age and race 3 are proved to be significant at 95% confident level. We tried models with no treatment, with no race as covariants, and the model only with age as the covariants. All of them have larger DIC then model 3, and they do not provide any further insight of the data. Therefore we chose model 3 for our inference.
We should note that in the frequentist approach, we included all the races in the model. However, in the Bayesian approach, we only include one dummy variable for race 3. Since we eliminated two variables that are closely dependent on existing variables (race 2, 3, and 4 are not independent of each other), the model significance is improved.
Inference
After a few experiments, we decided to thin the data by 100 to improve the mixing. We run 2 chains with 100000 sample each, so we have 2000 samples after thinning. The thinning does not change the modeled coefficients very much. The output parameters of model 3 are (in the order of ‘Inter’, ‘race3’, ‘age’, ‘treat2’, ‘treat3’):
Judging from the coefficients, we are 68% confident that treatment 3 can increase weightloss (with am expected value of 1.69). Treatment 2 is likely to increase the weightloss, but we can’t say it with 68% confidence. On the other hand, race 3 have significantly larger weighloss compared to other races, and the weighloss decreases as age increases. 10 years of increase in age leads to a decrease of 2.2 in weightloss.
Samples for betas and the PDFs of betas are shown below. We can see the mixing is generally well, and we have tighter constrain on treat 2, and treat 3 compared to other variables.