Commit 4358b39e authored by Rubin, Paul's avatar Rubin, Paul

Added defaults for alpha parameters and forward/backward demo code.

parent 79e1516b
......@@ -2,7 +2,7 @@
title: "Stepwise Regression Demonstration"
output: html_notebook
author: Paul A. Rubin (rubin@msu.edu)
date: August 20, 2017
date: November 9, 2018
---
# Preamble
......@@ -22,6 +22,16 @@ One other note: since the code uses R's `drop1` and `add1` functions, it respect
Consider this to be "beta" code (and feel free to improve it). I've done somewhat limited testing on it, beyond what you see in this notebook.
### Update (November 2018)
Besides correcting a few spelling errors (oops), I have made a few minor modifications to the function, and added demonstration code below for forward and backward stepwise regression.
The changes to the function are as follows:
* The `alpha.to.enter` parameter now has a default value of 0.0 (which will prevent any variables from entering the model if you fail to specify a value for the parameter).
* The `alpha.to.leave` parameter now has a default value of 1.0 (which will prevent any variables from leaving the model if you fail to specify a value for the parameter).
# Function definition
The following defines the stepwise function.
......@@ -39,7 +49,7 @@ The following defines the stepwise function.
#' @return the final model
#'
stepwise <-
function(full.model, initial.model, alpha.to.enter, alpha.to.leave, data = NULL) {
function(full.model, initial.model, alpha.to.enter = 0.0, alpha.to.leave = 1.0, data = NULL) {
# Sanity check: alpha.to.enter should not be greater than alpha.to.leave.
if (alpha.to.enter > alpha.to.leave) {
warning("Your alpha-to-enter is greater than your alpha-to-leave, which could throw the function into an infinite loop.\n")
......@@ -214,6 +224,24 @@ stepwise(Fertility ~ 0 + Agriculture + Examination + Education + Catholic + Infa
Every model has an intercept.
Next, we run "forward" stepwise regression (in which variables may enter but may not leave the model under construction) and "backward" stepwise regression (in which variables may leave but may not enter).
To demonstrate forward regression, we begin with a model containing only a constant term and the Examination variable, and observe that Examination (which has been consistently dropped above) remains in the final model. The demonstration code omits the alpha.to.leave value, but setting it explicitly to something sufficiently large would produce the same result.
```{r}
stepwise(Fertility ~ 1 + Agriculture + Examination + Education + Catholic + Infant.Mortality, Fertility ~ 1 + Examination, alpha.to.enter = aToEnter)
```
Notice that Examination ends up with a pretty high p-value (~0.3) but remains in the model.
To demonstrate backward regression, we start with a model containing all variables except Education (which in previous models tended to be the strongest predictor in terms of p-value), and use the default value for alpha.to.enter. The demonstration code omits the alpha.to.enter value, but setting it explicitly to something sufficiently small would produce the same result.
```{r}
stepwise(Fertility ~ 1 + Agriculture + Examination + Education + Catholic + Infant.Mortality, Fertility ~ 1 + Agriculture + Examination + Catholic + Infant.Mortality, alpha.to.leave = aToLeave)
```
Notice this time that Education never made it into the model.
Before we abandon this data set, there is one other thing worth noting. Your alpha-to-enter *must not be larger* than your alpha-to-leave. Although you can get away with that sometimes, it carries the potential to put stepwise regression into an infinite loop (adding and dropping the same variable repeatedly), so the function disallows it.
```{r}
......
This diff is collapsed.
......@@ -22,7 +22,7 @@ The following defines the stepwise function.
#' @return the final model
#'
stepwise <-
function(full.model, initial.model, alpha.to.enter, alpha.to.leave, data = NULL) {
function(full.model, initial.model, alpha.to.enter = 0.0, alpha.to.leave = 1.0, data = NULL) {
# Sanity check: alpha.to.enter should not be greater than alpha.to.leave.
if (alpha.to.enter > alpha.to.leave) {
warning("Your alpha-to-enter is greater than your alpha-to-leave, which could throw the function into an infinite loop.\n")
......@@ -237,6 +237,26 @@ stepwise(Fertility ~ 1 + Agriculture + Examination + Education + Catholic + Infa
stepwise(Fertility ~ 1 + Agriculture + Examination + Education + Catholic + Infant.Mortality + X, Fertility ~ 1 + Agriculture + Examination + Education + Catholic + Infant.Mortality + X, aToEnter, aToLeave, data = swiss2)
```
# Test forward and backward stepwise regression
Test forward stepwise (leave alpha.to.leave at its default).
```{r}
stepwise(Fertility ~ 1 + Agriculture + Examination + Education + Catholic + Infant.Mortality, Fertility ~ 1, alpha.to.enter = aToEnter)
```
Start with Examination in the initial model and make sure it does not leave.
```{r}
stepwise(Fertility ~ 1 + Agriculture + Examination + Education + Catholic + Infant.Mortality, Fertility ~ 1 + Examination, alpha.to.enter = aToEnter)
```
Now try backward regression from a model containing Examination but not Education to make sure that Education does not leave.
```{r}
stepwise(Fertility ~ 1 + Agriculture + Examination + Education + Catholic + Infant.Mortality, Fertility ~ 1 + Agriculture + Examination + Catholic + Infant.Mortality, alpha.to.leave = aToLeave)
```
```{r}
rm(swiss, swiss2)
```
......
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment