About Linear Algebra for Computational Linguistics

Shape Bookkeeping and Diagnosing a Dimension Mismatch Principal Component Analysis for Dimension Reduction word2vec as Implicit PMI Matrix Factorization Positive Semi-Definiteness and the Covariance Matrix tf-idf as Diagonal Reweighting of a Count Matrix Softmax Function Change of Basis for Vectors in Linear Algebra Truncated SVD and the Eckart-Young Optimality Claim Document-Term Count Matrices in Text Mining The Unembedding Projection from Hidden State to Vocabulary Logits Johnson-Lindenstrauss Lemma for Dimensionality Reduction Systematic Linear Combination Concepts in Vector Spaces Definition of the Logarithm as Inverse of Exponentiation in Mathematics Compressed Sparse Row Storage for Term-Document Matrices Linear Independence of Vectors Self Attention Mechanism in Transformers Matrix-Vector Multiplication via Row-Column Dot Products and the Column Space Laws of Exponents for Integer Exponents in Algebra The Cartesian Coordinate Plane Eigenvalues and Eigenvectors Definitions in Linear Algebra Multi-Head Attention Singular Value Decomposition via Eigenvalue Decomposition of A^TA and AA^T Gram-Schmidt Orthogonalization Algorithm Vector-Arithmetic Analogies as Linear Structure in Embedding Space Length as Square Root of Self-Dot-Product Solving Inconsistent Linear Systems Using Least Squares Square Roots and the Pythagorean Theorem Eigenvalues and Eigenvectors for Symmetric Matrices Nonnegative Matrix Factorization in Machine Learning The Transpose and Why K^T Appears in Attention Choosing Cosine over Euclidean Distance in High Dimensions Vector Angle and Orthogonality via the Dot Product Vector Addition and Scalar Multiplication Unit Circle Definition of Sine and Cosine Dot Product Operation for Vectors in Linear Algebra Function Notation and Evaluation in Algebra Dot Product Projection of One Vector onto Another Nearest-Neighbour Retrieval over an Embedding Matrix Arithmetic Mean and Variance of a Data Set Vectors as Geometric Arrows and Ordered Number Lists Summation Notation for Finite Series in Mathematics Cosine Similarity in High-Dimensional Vector Spaces Latent Semantic Analysis in Natural Language Processing Rank and Column Space as Information Capacity L1 and L2 Norms and Unit Normalization Matrix Multiplication as Composition of Linear Transformations Layer Normalization as a Per-Vector Operation One-Hot Encoding