3 Day 3 (June 11)

3.1 Announcements

• If office hours times don’t work for you let me know

• Recommended reading

  • Chapters 1 and 2 (pgs 1 - 27) in Linear Models with R

3.2 Matrix algebra

• Column vectors
  • \(\mathbf{y}\equiv(y_{1},y_{2},\ldots,y_{n})^{'}\)
  • \(\mathbf{x}\equiv(x_{1},x_{2},\ldots,x_{n})^{'}\)
  • \(\boldsymbol{\beta}\equiv(\beta_{1},\beta_{2},\ldots,\beta_{p})^{'}\)
  • \(\boldsymbol{1}\equiv(1,1,\ldots,1)^{'}\)
  • In R

  y <- matrix(c(1,2,3),nrow=3,ncol=1)
  y

  ##      [,1]
  ## [1,]    1
  ## [2,]    2
  ## [3,]    3

• Matrices
  • \(\mathbf{X}\equiv(\mathbf{x}_{1},\mathbf{x}_{2},\ldots,\mathbf{x}_{p})\)
  • In R

  X <- matrix(c(1,2,3,4,5,6),nrow=3,ncol=2,byrow=FALSE)
  X

  ##      [,1] [,2]
  ## [1,]    1    4
  ## [2,]    2    5
  ## [3,]    3    6

• Vector multiplication
  • \(\mathbf{y}^{'}\mathbf{y}\)
  • \(\mathbf{1}^{'}\mathbf{1}\)
  • \(\mathbf{1}\mathbf{1}^{'}\)
  • In R

  t(y)%*%y    

  ##      [,1]
  ## [1,]   14

• Matrix by vector multiplication
  • \(\mathbf{X}^{'}\mathbf{y}\)
  • In R

  t(X)%*%y

  ##      [,1]
  ## [1,]   14
  ## [2,]   32

• Matrix by matrix multiplication
  • \(\mathbf{X}^{'}\mathbf{X}\)
  • In R

  t(X)%*%X

  ##      [,1] [,2]
  ## [1,]   14   32
  ## [2,]   32   77

• Matrix inversion
  • \((\mathbf{X}^{'}\mathbf{X})^{-1}\)
  • In R

  solve(t(X)%*%X)

  ##            [,1]       [,2]
  ## [1,]  1.4259259 -0.5925926
  ## [2,] -0.5925926  0.2592593

• Determinant of a matrix
  • \(|\mathbf{I}|\)
    
  • In R

  I <- diag(1,3)
  I

  ##      [,1] [,2] [,3]
  ## [1,]    1    0    0
  ## [2,]    0    1    0
  ## [3,]    0    0    1

  det(I)

  ## [1] 1

• Quadratic form
  • \(\mathbf{y}^{'}\mathbf{S}\mathbf{y}\)
• Derivative of a quadratic form (Note \(\mathbf{S}\) is a symmetric matrix; e.g., \(\mathbf{X}^{'}\mathbf{X}\))
  • \(\frac{\partial}{\partial\mathbf{y}}\mathbf{y^{'}\mathbf{S}\mathbf{y}}=2\mathbf{S}\mathbf{y}\)
• Other useful derivatives
  • \(\frac{\partial}{\partial\mathbf{y}}\mathbf{\mathbf{x^{'}}\mathbf{y}}=\mathbf{x}\)
  • \(\frac{\partial}{\partial\mathbf{y}}\mathbf{\mathbf{X^{'}}\mathbf{y}}=\mathbf{X}\)