In reply to Killian Miller.

You are correct that the matrix A = X`X - x_i`x_i might be singular. FOr example, if X has is a 2x2 matrix, then A is singular.

You ask if there is any reason to expect A to be nonsingular. Yes. You can make geometric arguments (by counting dimensions) that show that A will be nonsingular almost always. Another proof comes from realizing that A is the same matrix that you would get deleting the i_th row of X and forming the crossproduct matrix of the result. If you can remove each row of X and still obtain a full column-rank matrix, the SMW formula will be true.