PSY 503: Foundations of Statistical Methods in Psychological Science

class: center, middle, inverse, title-slide

.title[
# PSY 503: Foundations of Statistical Methods in Psychological Science
]
.subtitle[
## More LM: Transformations
]
.author[
### Jason Geller, Ph.D. (he/him/his)
]
.institute[
### Princeton University
]
.date[
### Updated:2022-11-07
]

---

<div style = "position:fixed; visibility: hidden">
`$$\require{color}\definecolor{red}{rgb}{1, 0, 0}$$`
`$$\require{color}\definecolor{green}{rgb}{0, 1, 0}$$`
`$$\require{color}\definecolor{blue}{rgb}{0, 0, 1}$$`
</div>

# Outline

- Check-In Q&A

- Transformations

- Linear Transformations

- Centering
     - Standardization
    
  - Nonlinear Transformations
  
      - Logarithms
      
- Polynomial regression

---
# Check-In Q&A

1. Can a model be too good?

2. What is a grand mean?

3. Intercept of the linear model

4. In effects coding/sum coding, why is centering necessary?

5. `$\eta^2$` vs `$\eta_p^2$`

6. Would you ever use dummy coding with multiple levels? 
---
# Check-in Questions

- How many predictors is too many?

.pull-left[

- General rule of thumb is 10-20 participants for every predictor variable in your model

- Use cross validation

- Use other methods like lasso or ridge regression

- Only include predictors that are theoretically meaningful
]

.pull-right[
<img src="Overfitting-1.png" width="100%" style="display: block; margin: auto;" />
]
---
- What is grand mean?

- It is the mean of the means

`$$\frac{Group1_{mean} + Group2_{mean}}{2}$$`

```r
senses %>%
  group_by(Modality) %>% # get each group
  dplyr::summarise(meanval=mean(Val))%>% # get mean
 dplyr::summarise(meanofmeans=mean(meanval)) # mean of means
```

---
- Sum of Squares

<img src="aov_table.bmp" width="70%" style="display: block; margin: auto;" />
---
# Restricted and Full Model

`$$F = \frac{SS_{R}-SS_{F}/{df_{R}-df_{F}} (p-1)}{SS_{F}/df_F(N-p)} = \frac{MS_{model}}{MS_{error}}$$`

```r
restricted <- lm(Val~ 1, data=senses) # intercept-only model

full <- lm(Val~Modality, data=senses) # full model 
```
---
# Weekly Check-In Questions

- Significance of the intercept value

-  The *p*-value for the intercept tells us if the intercept is different from 0
  
    - Most often this is not of importance

`$$H_0:\beta_0=0$$`
---
# Weekly Check-In Questions

- In effects coding/sum coding, why is centering necessary/why would we want to use grand mean?

- Centering is what occurs as a function of taking the mean of dummy codes 
  
  - Might not care about reference level

- When dealing with one factor with 2 levels dummy coding vs. sum coding does not matter
  
- Sum coding is what ANOVA is doing to test main effects and interactions (more on this later)

---
# Weekly Check-In Questions

- Eta-squared vs partial eta-squared 
`$$\eta^2$$`

- % of total variance explained by IV

`$$\eta_p^2$$`

- % of variance explained by IV partailing out other variables in the model

---
# Weekly Check-In Questions

-  Would you ever use dummy coding with multiple levels?

---
# Dummy Codes (Seidman, Wade, & Geller, 2022)

-  Waitlist (do not complete a group, just measures)
  -  Group-only (attend a group + measures)
  
  - Self-affirmation + group (complete an individual intervention, then attend group, + measures)

- GOvsSA = Waitlist (0), Group only (0), Self-affirmation (1)

- WLvsGO = Waitlist (0), Group only (1) Self-affirmation (1)

- GroupvsNo = Waitlist (0), Group (1), Self-Affirmation (1)

---
# `emmeans`

- Get means and pairwise comparisons

```r
# get pairwise tests between all groups

sen <- lm(Val~Modality, data=senses)

as.data.frame(emmeans::emmeans(sen, specs = "Modality")) %>%
flextable()
```

<template id="6d7d013b-6c02-4732-90a3-7027c1a0b805"><style>
.tabwid table{
 border-spacing:0px !important;
 border-collapse:collapse;
 line-height:1;
 margin-left:auto;
 margin-right:auto;
 border-width: 0;
 display: table;
 margin-top: 1.275em;
 margin-bottom: 1.275em;
 border-color: transparent;
}
.tabwid_left table{
 margin-left:0;
}
.tabwid_right table{
 margin-right:0;
}
.tabwid td {
 padding: 0;
}
.tabwid a {
 text-decoration: none;
}
.tabwid thead {
 background-color: transparent;
}
.tabwid tfoot {
 background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
.katex-display {
 margin: 0 0 !important;
}
</style><div class="tabwid"><style>.cl-221dcbc0{}.cl-221965c6{font-family:'Helvetica';font-size:11pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-221977f0{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-221977fa{margin:0;text-align:right;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-2219a6c6{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-2219a6d0{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-2219a6d1{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-2219a6da{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-2219a6db{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-2219a6e4{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-221dcbc0'><thead><tr style="overflow-wrap:break-word;"><td class="cl-2219a6e4">Modality</td><td class="cl-2219a6db">emmean</td><td class="cl-2219a6db">SE</td><td class="cl-2219a6db">df</td><td class="cl-2219a6db">lower.CL</td><td class="cl-2219a6db">upper.CL</td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-2219a6d0">Sight</td><td class="cl-2219a6c6">5.579663</td><td class="cl-2219a6c6">0.01889440</td><td class="cl-2219a6c6">400</td><td class="cl-2219a6c6">5.542518</td><td class="cl-2219a6c6">5.616808</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-2219a6d0">Smell</td><td class="cl-2219a6c6">5.471012</td><td class="cl-2219a6c6">0.05317357</td><td class="cl-2219a6c6">400</td><td class="cl-2219a6c6">5.366477</td><td class="cl-2219a6c6">5.575546</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-2219a6d0">Sound</td><td class="cl-2219a6c6">5.405193</td><td class="cl-2219a6c6">0.03248092</td><td class="cl-2219a6c6">400</td><td class="cl-2219a6c6">5.341338</td><td class="cl-2219a6c6">5.469047</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-2219a6d0">Taste</td><td class="cl-2219a6c6">5.808124</td><td class="cl-2219a6c6">0.03878081</td><td class="cl-2219a6c6">400</td><td class="cl-2219a6c6">5.731884</td><td class="cl-2219a6c6">5.884364</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-2219a6da">Touch</td><td class="cl-2219a6d1">5.534435</td><td class="cl-2219a6d1">0.03224121</td><td class="cl-2219a6d1">400</td><td class="cl-2219a6d1">5.471052</td><td class="cl-2219a6d1">5.597818</td></tr></tbody></table></div></template>
<div class="flextable-shadow-host" id="bd702a58-383d-4d02-9225-5d72dbcec6a3"></div>
<script>
var dest = document.getElementById("bd702a58-383d-4d02-9225-5d72dbcec6a3");
var template = document.getElementById("6d7d013b-6c02-4732-90a3-7027c1a0b805");
var caption = template.content.querySelector("caption");
if(caption) {
 caption.style.cssText = "display:block;text-align:center;";
 var newcapt = document.createElement("p");
 newcapt.appendChild(caption)
 dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

---
# Pairwise Comparisons

```r
library(flextable)
# get pairwise tests between all groups
means1 = emmeans::emmeans(sen, specs = "Modality")
# use pairs
flextable(as.data.frame(pairs(means1)))
```

<template id="8079e1e4-2b13-4cfa-b95e-234a67c00985"><style>
.tabwid table{
 border-spacing:0px !important;
 border-collapse:collapse;
 line-height:1;
 margin-left:auto;
 margin-right:auto;
 border-width: 0;
 display: table;
 margin-top: 1.275em;
 margin-bottom: 1.275em;
 border-color: transparent;
}
.tabwid_left table{
 margin-left:0;
}
.tabwid_right table{
 margin-right:0;
}
.tabwid td {
 padding: 0;
}
.tabwid a {
 text-decoration: none;
}
.tabwid thead {
 background-color: transparent;
}
.tabwid tfoot {
 background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
.katex-display {
 margin: 0 0 !important;
}
</style><div class="tabwid"><style>.cl-22312b8e{}.cl-222d5202{font-family:'Helvetica';font-size:11pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-222d5bda{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-222d5bdb{margin:0;text-align:right;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-222d8808{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-222d8812{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-222d881c{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-222d881d{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-222d881e{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-222d8826{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-22312b8e'><thead><tr style="overflow-wrap:break-word;"><td class="cl-222d881e">contrast</td><td class="cl-222d8826">estimate</td><td class="cl-222d8826">SE</td><td class="cl-222d8826">df</td><td class="cl-222d8826">t.ratio</td><td class="cl-222d8826">p.value</td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-222d8812">Sight - Smell</td><td class="cl-222d8808">0.10865148</td><td class="cl-222d8808">0.05643072</td><td class="cl-222d8808">400</td><td class="cl-222d8808">1.925396</td><td class="cl-222d8808">0.3055011645361</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-222d8812">Sight - Sound</td><td class="cl-222d8808">0.17447036</td><td class="cl-222d8808">0.03757671</td><td class="cl-222d8808">400</td><td class="cl-222d8808">4.643046</td><td class="cl-222d8808">0.0000456942086</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-222d8812">Sight - Taste</td><td class="cl-222d8808">-0.22846083</td><td class="cl-222d8808">0.04313872</td><td class="cl-222d8808">400</td><td class="cl-222d8808">-5.295957</td><td class="cl-222d8808">0.0000019440745</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-222d8812">Sight - Touch</td><td class="cl-222d8808">0.04522812</td><td class="cl-222d8808">0.03736969</td><td class="cl-222d8808">400</td><td class="cl-222d8808">1.210289</td><td class="cl-222d8808">0.7454337311622</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-222d8812">Smell - Sound</td><td class="cl-222d8808">0.06581888</td><td class="cl-222d8808">0.06230922</td><td class="cl-222d8808">400</td><td class="cl-222d8808">1.056327</td><td class="cl-222d8808">0.8286561493660</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-222d8812">Smell - Taste</td><td class="cl-222d8808">-0.33711231</td><td class="cl-222d8808">0.06581321</td><td class="cl-222d8808">400</td><td class="cl-222d8808">-5.122259</td><td class="cl-222d8808">0.0000046594174</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-222d8812">Smell - Touch</td><td class="cl-222d8808">-0.06342336</td><td class="cl-222d8808">0.06218459</td><td class="cl-222d8808">400</td><td class="cl-222d8808">-1.019921</td><td class="cl-222d8808">0.8461335445944</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-222d8812">Sound - Taste</td><td class="cl-222d8808">-0.40293120</td><td class="cl-222d8808">0.05058618</td><td class="cl-222d8808">400</td><td class="cl-222d8808">-7.965243</td><td class="cl-222d8808">0.0000000000000</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-222d8812">Sound - Touch</td><td class="cl-222d8808">-0.12924225</td><td class="cl-222d8808">0.04576577</td><td class="cl-222d8808">400</td><td class="cl-222d8808">-2.823993</td><td class="cl-222d8808">0.0397092275599</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-222d881d">Taste - Touch</td><td class="cl-222d881c">0.27368895</td><td class="cl-222d881c">0.05043260</td><td class="cl-222d881c">400</td><td class="cl-222d881c">5.426827</td><td class="cl-222d881c">0.0000009903166</td></tr></tbody></table></div></template>
<div class="flextable-shadow-host" id="9004802c-b81b-4be3-9df2-814c9691738f"></div>
<script>
var dest = document.getElementById("9004802c-b81b-4be3-9df2-814c9691738f");
var template = document.getElementById("8079e1e4-2b13-4cfa-b95e-234a67c00985");
var caption = template.content.querySelector("caption");
if(caption) {
 caption.style.cssText = "display:block;text-align:center;";
 var newcapt = document.createElement("p");
 newcapt.appendChild(caption)
 dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>
---
# Today's Datasets

- Pitch and Age

```r
data = read_csv("https://raw.githubusercontent.com/jgeller112/psy503-psych_stats/master/static/slides/10-linear_modeling/data/age_pitch.csv")
```
---
# Today's Datasets

- Memory and Time

- 13 subjects were asked to memorize a list of disconnected items
 
 - The subjects were then asked to recall the items at various times up to a week later

```r
log_df <- tibble::tribble(
 ~time, ~prop,
 1L, 0.84,
 5L, 0.71,
 15L, 0.61,
 30L, 0.56,
 60L, 0.54,
 120L, 0.47,
 240L, 0.45,
 480L, 0.38,
 720L, 0.36,
 1440L, 0.26,
 2880L, 0.2,
 5760L, 0.16,
 10080L, 0.08
 )
```
---
# Transformations

- When should we transform our data?

- To make our data more *interpretable*
  
--

- When our data is *non-linear*

- When our data is *skewed* 
  
---
# Linear Transformations

- Linear transformations

- Adding, subtracting, dividing by, or multiplying a variable with a constant
    
  - Does not change the relationships in a genuine way (“same model”)

---
# Linear Transformations

- Linear transformations

- Adding, subtracting, dividing by, or multiplying a variable with a constant
  
  - Does not change the relationships in a genuine way (“same model”)
  
    - Common: centering and standardizing
---
# Transformations

- Nonlinear transformations

- Transformation that affects different data points differently
  
  - Changes the relationships (“different model”)
  
      - Common: Logarthims 
    
  - Makes models more in line with assumptions
---
# Linear Transformations

<img src="More_LM_transformations_centering_files/figure-html/unnamed-chunk-13-1.png" width="100%" style="display: block; margin: auto;" />
---
# Linear Transformations

<img src="More_LM_transformations_centering_files/figure-html/unnamed-chunk-14-1.png" width="100%" style="display: block; margin: auto;" />
---
# Centering

> Sometimes our data is *nonsensical* (e.g., Intercept when Age = 0)

- Centering changes the model so that 0 is mean/average of X

<img src="More_LM_transformations_centering_files/figure-html/unnamed-chunk-15-1.png" width="100%" style="display: block; margin: auto;" />
---
# Centering - How?

```r
df <- mutate(data,
 age_c = age - mean(age, na.rm = TRUE)) # center
```

```r
library(datawizard) # package to center and standardize

df_wiz <- data %>% 
 mutate(age_c = datawizard::center(age))

head(df_wiz)
```

---
# Standardize Predictors

- Dividing centered mean by `$\sigma$`

- Puts variables on same metrics

<img src="More_LM_transformations_centering_files/figure-html/unnamed-chunk-18-1.png" width="100%" style="display: block; margin: auto;" />
---
# Standardizing - How?

```r
df_stand <- data %>%
 mutate(age_z = age_c / sd(age, na.rm = TRUE))# standardize z score
```

---
# Output

```r
lm(pitch~age, data=data)
```

```
## 
## Call:
## lm(formula = pitch ~ age, data = data)
## 
## Coefficients:
## (Intercept)          age  
##    215.9581      -0.6936
```

.pull-left[

```r
lm(pitch~age_c, data=df)
```

```
## 
## Call:
## lm(formula = pitch ~ age_c, data = df)
## 
## Coefficients:
## (Intercept)        age_c  
##    175.2105      -0.6936
```
]

.pull-right[

```r
lm(pitch~age_z, data=df_wiz)
```

```
## 
## Call:
## lm(formula = pitch ~ age_z, data = df_wiz)
## 
## Coefficients:
## (Intercept)        age_z  
##     175.211       -9.192
```
]
---
# Centering Around Other Values

- We could also make 0 correspond to some other sensible/useful value

- Smallest logically possible value?

---
# Centering

- Good if zero is *not meaningful*
  
  - Do not center if zero is meaningful
  
---
class: middle

# When in doubt, center!

---
class: middle

# Nonlinear Transformations

---
# Logathimthic Transformations

<img src="log.bmp" width="90%" style="display: block; margin: auto;" />
---
# Logathimthic Transformations

<img src="base.PNG" width="90%" style="display: block; margin: auto;" />
---
# Logathimthic Transformations

- Exponentiation

- Takes small numbers and grows them

`$$10^1 = 10$$`
`$$10^2 = 100$$`
`$$10^3 = 1000$$`
`$$10^4 = 10000$$`
`$$10^5 = 100000$$`
---
# Log Transformations

- Logarithmic

- How many times must one “base” number be multiplied by itself to get some other particular number?

- Takes large numbers and shrinks them

`$$1 = log_{10}(10)$$`
`$$2 = log_{10}(100)$$`
`$$3 = log_{10}(1000)$$`
`$$4 = log_{10}(10000)$$`
`$$5 = log_{10}(100000)$$`
- R uses natural log base = 2.7182

---
# Log Transfomrations

- Tracks the order of magnitude

- Large numbers shrink more than smaller numbers

- Compression effect (larger values are closer to the other)

```r
# time in ms
RTs <- c(600, 650, 700, 1000, 4000)

logRTs <- log(RTs) # base = 2.718282

logRTs # log rts 
```

```
## [1] 6.396930 6.476972 6.551080 6.907755 8.294050
```

```r
exp(logRTs) # get ms numbers back 
```

```
## [1]  600  650  700 1000 4000
```
---
# Predictor `$X$` Transformations

- Makes `$X$` more linear

- Makes `$X$` more Normal

- Does not fix heteroscedasticity

- Lets plot our memory data
 
---
<img src="More_LM_transformations_centering_files/figure-html/unnamed-chunk-27-1.png" width="100%" style="display: block; margin: auto;" />
---
# Other Checks

```r
lm(prop~time, data=log_df)%>%
check_model()
```

---
# Log Transformation

```r
log_df <- log_df %>% 
 mutate(log_time=log(time))

head(log_df)
```

---
# Check Again

- Much better!

<img src="More_LM_transformations_centering_files/figure-html/unnamed-chunk-30-1.png" width="100%" style="display: block; margin: auto;" />
---
# Let's Analyze

```r
lm(prop~log_time, data=log_df)
```

```
## 
## Call:
## lm(formula = prop ~ log_time, data = log_df)
## 
## Coefficients:
## (Intercept)     log_time  
##     0.84642     -0.07923
```
---
# Interpretation

- When using logarithms, you model percentage increase or decrease instead of absolute differences

- For log transformed predictors

- Divide the coefficient by 100
  
    - 1% increase in the independent variable increases (or decreases) the dependent variable by (coefficient/100) units
  
  -  For x percent increase, multiply the coefficient by log(1.x)
 
---
# Check Assumptions Again

```r
lm(prop~log_time, data=log_df)%>%
check_model()
```

<img src="More_LM_transformations_centering_files/figure-html/unnamed-chunk-32-1.png" width="100%" style="display: block; margin: auto;" />
---
# Other `$X$` Transformations

- Square root `$\sqrt(y)$`

- Inverse transformations (1/y)

- Makes interpretation hard!

---
# Outcome `$Y$` Transformations

- Makes `$Y$` more linear

- Makes `$Y$` normal

- Helps correct heteroscadacity

---
# Outcome `$Y$` Transformations

- logarithmic

- Power transformations (Box-Cox)

- Square root `$\sqrt(y)$`

- Inverse transformations (1/y)

- Makes interpretation hard! 
---
# Log Y Interpretation

- Dependent/response variable

- Exponentiate the coefficient, subtract one from this number, and multiply by 100
  
  - For every one-unit increase in the independent variable, our dependent variable increases by about X%

---
# X and Y Log Transforamtion Interpretation

- When dependent/response variable and independent/predictor variable(s) are log-transformed

- Interpret the coefficient as the percent increase in the dependent variable for every 1% increase in the independent variable
  
    - Example: the coefficient is 0.198. For every 1% increase in the independent variable, our dependent variable increases by about 0.20%. For x percent increase, calculate 1.x to the power of the coefficient, subtract 1, and multiply by 100
    
    - Example: For every 20% increase in the independent variable, our dependent variable increases by about (1.20 0.198 – 1) * 100 = 3.7 percent

---
# When Should You Log Transform?

- Ideally: When it’s theoretically motivated

- Common in linguistics and psychology:
    
      - Word frequency
      - Response times
      - Perceptual magnitudes

- After you look at the relationship to DV

- *If you want/need to center, apply log transform after!*
---
# Data Transformations

1. If the primary problem with your model is non-linearity, look at a scatter plot of the data to suggest transformations that might help

2. If the variances are unequal and/or error terms are not normal, try a "power transformation"

3. Be transparent!

---
# Power Ladder

<img src="power.jpg" width="50%" style="display: block; margin: auto;" />
---
class: middle

# Polynomial Models

---
# Polynomial Models

.pull-left[

-  A nonlinear regression method that models the relationship between X and Y using polynomials

- Polynomial is mathematical expression
of operators and non-negative powers

]

.pull-right[
<img src="orthogonal-curves-1.png" width="100%" style="display: block; margin: auto;" />
]
---
# Memory Example

<img src="More_LM_transformations_centering_files/figure-html/unnamed-chunk-35-1.png" width="100%" style="display: block; margin: auto;" />
---
# Polynomial Regression

`$$\begin{equation}
Y_i = \beta_0 + \beta_1 x_i + \beta_2 x_i^2 + \epsilon_i.
\end{equation}$$`

```r
log_df_quad <- log_df%>%
 mutate(time2=time^-2) # add in quadratic
```

---
# Testing Polynomial Models

- Analyze all the terms in model

- Model comparisons

- Testing some of the `$X$` terms, starting with the lowest order term and including the next higher-order terms from there

- Include all lower terms of that variable

---
# Polynomial Regression

```r
log_df_quad <- log_df%>%
 mutate(time2=time^-2) # add in quadratic
 
lm(prop ~ time + time2, data=log_df_quad) %>%
 model_parameters()
```

<table class="huxtable" style="border-collapse: collapse; border: 0px; margin-bottom: 2em; margin-top: 2em; ; margin-left: auto; margin-right: auto; " id="tab:unnamed-chunk-37">
<col><col><col><col><col><col><col><col><col><tr>
<th style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0.4pt; padding: 6pt 6pt 6pt 6pt; font-weight: bold;">Parameter</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: bold;">Coefficient</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: bold;">SE</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: bold;">CI</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: bold;">CI_low</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: bold;">CI_high</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: bold;">t</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: bold;">df_error</th><th style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: bold;">p</th></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0.4pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">(Intercept)</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.487   </td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.04    </td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.95</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.398   </td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.576   </td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">12.2 </td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0pt 0pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">10</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0.4pt 0.4pt 0pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">2.57e-07</td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0pt 0.4pt; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">time</td><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">-4.99e-05</td><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">1.14e-05</td><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.95</td><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">-7.54e-05</td><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">-2.44e-05</td><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">-4.36</td><td style="vertical-align: top; text-align: right; white-space: normal; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">10</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0pt 0pt; padding: 6pt 6pt 6pt 6pt; font-weight: normal;">0.00141 </td></tr>
<tr>
<td style="vertical-align: top; text-align: left; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0.4pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">time2</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.362   </td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.125   </td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.95</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.084   </td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.64    </td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">2.9 </td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">10</td><td style="vertical-align: top; text-align: right; white-space: normal; border-style: solid solid solid solid; border-width: 0pt 0.4pt 0.4pt 0pt; padding: 6pt 6pt 6pt 6pt; background-color: rgb(242, 242, 242); font-weight: normal;">0.0158  </td></tr>
</table>

---
# Model Fit Quadratic

```r
anim.1<- ggplot(log_df, aes(time,prop))+
 geom_point(size=5)+
 theme_bw(base_size=18)+ 
 stat_smooth(method = "lm", formula = y ~ x + I(x^2), size = 1)

anim.1
```

<img src="More_LM_transformations_centering_files/figure-html/unnamed-chunk-38-1.png" width="100%" style="display: block; margin: auto;" />
---
# Model Fit Comparison

```r
# all

lm_quad <- lm(prop ~ time + time2 + I(time^3), data=log_df_quad)

# forward

lm_lin <- lm(prop ~ time, data=log_df_quad)

lm_quad <- lm(prop ~ time + time2, data=log_df_quad)

anova(lm_lin, lm_quad)
```

---
# In-Class Activity

- English Lexicon Project

- High frequency words responded to faster than low frequency words

```r
# Load the frequency data

ELP <- read_csv("https://raw.githubusercontent.com/jgeller112/psy503-psych_stats/master/static/slides/12-Transformations_Centering/data/ELP_frequency.csv")
```

---
# Log Transformation

```r
# Log10-transform frequency, log-transform RTs:

ELP <- mutate(ELP,
 Log10Freq = log10(Freq),
 LogRT = log(RT))

head(ELP) %>%
  flextable()
```

<template id="c81898bc-a788-4656-b259-d899fbfaa9a0"><style>
.tabwid table{
 border-spacing:0px !important;
 border-collapse:collapse;
 line-height:1;
 margin-left:auto;
 margin-right:auto;
 border-width: 0;
 display: table;
 margin-top: 1.275em;
 margin-bottom: 1.275em;
 border-color: transparent;
}
.tabwid_left table{
 margin-left:0;
}
.tabwid_right table{
 margin-right:0;
}
.tabwid td {
 padding: 0;
}
.tabwid a {
 text-decoration: none;
}
.tabwid thead {
 background-color: transparent;
}
.tabwid tfoot {
 background-color: transparent;
}
.tabwid table tr {
background-color: transparent;
}
.katex-display {
 margin: 0 0 !important;
}
</style><div class="tabwid"><style>.cl-24d4fc44{}.cl-24d13b90{font-family:'Helvetica';font-size:11pt;font-weight:normal;font-style:normal;text-decoration:none;color:rgba(0, 0, 0, 1.00);background-color:transparent;}.cl-24d14626{margin:0;text-align:left;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-24d14627{margin:0;text-align:right;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);padding-bottom:5pt;padding-top:5pt;padding-left:5pt;padding-right:5pt;line-height: 1;background-color:transparent;}.cl-24d16be2{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-24d16bec{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 0 solid rgba(0, 0, 0, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-24d16bf6{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-24d16bf7{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 0 solid rgba(0, 0, 0, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-24d16c00{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}.cl-24d16c01{width:54pt;background-color:transparent;vertical-align: middle;border-bottom: 2pt solid rgba(102, 102, 102, 1.00);border-top: 2pt solid rgba(102, 102, 102, 1.00);border-left: 0 solid rgba(0, 0, 0, 1.00);border-right: 0 solid rgba(0, 0, 0, 1.00);margin-bottom:0;margin-top:0;margin-left:0;margin-right:0;}</style><table class='cl-24d4fc44'><thead><tr style="overflow-wrap:break-word;"><td class="cl-24d16c01">Word</td><td class="cl-24d16c00">Freq</td><td class="cl-24d16c00">RT</td><td class="cl-24d16c00">Log10Freq</td><td class="cl-24d16c00">LogRT</td></tr></thead><tbody><tr style="overflow-wrap:break-word;"><td class="cl-24d16bec">thing</td><td class="cl-24d16be2">55,522</td><td class="cl-24d16be2">621.77</td><td class="cl-24d16be2">4.744465</td><td class="cl-24d16be2">6.432570</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-24d16bec">life</td><td class="cl-24d16be2">40,629</td><td class="cl-24d16be2">519.56</td><td class="cl-24d16be2">4.608836</td><td class="cl-24d16be2">6.252982</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-24d16bec">door</td><td class="cl-24d16be2">14,895</td><td class="cl-24d16be2">507.38</td><td class="cl-24d16be2">4.173041</td><td class="cl-24d16be2">6.229260</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-24d16bec">angel</td><td class="cl-24d16be2">3,992</td><td class="cl-24d16be2">636.56</td><td class="cl-24d16be2">3.601191</td><td class="cl-24d16be2">6.456079</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-24d16bec">beer</td><td class="cl-24d16be2">3,850</td><td class="cl-24d16be2">587.18</td><td class="cl-24d16be2">3.585461</td><td class="cl-24d16be2">6.375331</td></tr><tr style="overflow-wrap:break-word;"><td class="cl-24d16bf6">disgrace</td><td class="cl-24d16bf7">409</td><td class="cl-24d16bf7">705.00</td><td class="cl-24d16bf7">2.611723</td><td class="cl-24d16bf7">6.558198</td></tr></tbody></table></div></template>
<div class="flextable-shadow-host" id="2223c87e-ff48-4a50-9bdb-2ff2f7c6e73c"></div>
<script>
var dest = document.getElementById("2223c87e-ff48-4a50-9bdb-2ff2f7c6e73c");
var template = document.getElementById("c81898bc-a788-4656-b259-d899fbfaa9a0");
var caption = template.content.querySelector("caption");
if(caption) {
 caption.style.cssText = "display:block;text-align:center;";
 var newcapt = document.createElement("p");
 newcapt.appendChild(caption)
 dest.parentNode.insertBefore(newcapt, dest.previousSibling);
}
var fantome = dest.attachShadow({mode: 'open'});
var templateContent = template.content;
fantome.appendChild(templateContent);
</script>

---
# Plot

.pull-left[
<img src="More_LM_transformations_centering_files/figure-html/unnamed-chunk-42-1.png" width="100%" style="display: block; margin: auto;" />

]

.pull-left[
<img src="More_LM_transformations_centering_files/figure-html/unnamed-chunk-43-1.png" width="100%" style="display: block; margin: auto;" />
]
---
# Regression

```r
# Fit a regression model:
ELP_mdl <- lm(LogRT ~ Log10Freq, data = ELP) %>%
model_parameters()
```

---
# Centering & Standardizing

```r
# Center and standardize in one go:

ELP <- mutate(ELP,
 Log10Freq_c = Log10Freq - mean(Log10Freq),
 Log10Freq_z = Log10Freq_c / sd(Log10Freq_c))

# Select the frequency columns to compare:

ELP %>% 
  dplyr::select(Freq, Log10Freq, Log10Freq_c, Log10Freq_z)
```

---
# Centering & Standardizing

```r
# Same as before, but this time using datawizard:

ELP <- mutate(ELP,
 Log10Freq_c = datawizard::center(Log10Freq),
 Log10Freq_z =datawizard::standardise(Log10Freq))
```

---
# Regression

```r
# Fit raw, centered and unc
ELP_mdl_c <- lm(LogRT ~ Log10Freq_c, ELP) %>%
 model_parameters()

ELP_mdl_z <- lm(LogRT ~ Log10Freq_z, ELP) %>%
 model_parameters()
```