Tensor product - Citizendia

In mathematics, the tensor product, denoted by $\otimes$ , may be applied in different contexts to vectors, matrices, tensors, vector spaces, algebras, topological vector spaces, and modules. Mathematics is the body of Knowledge and Academic discipline that studies such concepts as Quantity, Structure, Space and In Mathematics, a vector space (or linear space) is a collection of objects (called vectors) that informally speaking may be scaled and added In Mathematics, a matrix (plural matrices) is a rectangular table of elements (or entries) which may be Numbers or more generally History The word tensor was introduced in 1846 by William Rowan Hamilton to describe the norm operation in a certain type of algebraic system (eventually In Mathematics, a vector space (or linear space) is a collection of objects (called vectors) that informally speaking may be scaled and added In Mathematics, an algebra over a field K, or a K -algebra, is a Vector space A over K equipped with In Mathematics, a topological vector space is one of the basic structures investigated in Functional analysis. In Abstract algebra, the concept of a module over a ring is a generalization of the notion of Vector space, where instead of requiring the scalars In each case the significance of the symbol is the same: the most general bilinear operation. In Mathematics, a bilinear map is a function of two arguments that is linear in each In some contexts, this product is also referred to as outer product. In Linear algebra, the outer product typically refers to the tensor product of two vectors. The term "tensor product" is also used in relation to monoidal categories. In Mathematics, a monoidal category (or tensor category) is a category C equipped with a Bifunctor &otimes: C

Example:

$\begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \\ \end{bmatrix} \otimes \begin{bmatrix} b_{11} & b_{12} \\ b_{21} & b_{22} \\ \end{bmatrix} = \begin{bmatrix} a_{11}B & a_{12}B \\ a_{21}B & a_{22}B \\ \end{bmatrix} = \begin{bmatrix} a_{11} b_{11} & a_{11} b_{12} & a_{12} b_{11} & a_{12} b_{12} \\ a_{11} b_{21} & a_{11} b_{22} & a_{12} b_{21} & a_{12} b_{22} \\ a_{21} b_{11} & a_{21} b_{12} & a_{22} b_{11} & a_{22} b_{12} \\ a_{21} b_{21} & a_{21} b_{22} & a_{22} b_{21} & a_{22} b_{22} \\ \end{bmatrix}$

Resultant rank = 4, resultant dimension = 4×4.

Here rank denotes the tensor rank (number of requisite indices), while dimension counts the number of degrees of freedom in the resulting array; the matrix rank is 1. In Mathematics, the modern Component-free approach to the theory of Tensors views tensors initially as Abstract objects expressing some definite type of The column rank of a matrix A is the maximal number of Linearly independent columns of A.

A representative case is the Kronecker product of any two rectangular arrays, considered as matrices. In mathematics the Kronecker product, denoted by \otimes is an operation on two matrices of arbitrary size resulting in a Block matrix. A dyadic product is the special case of the tensor product between two vectors of the same dimension. In Mathematics, in particular Multilinear algebra, the dyadic product \mathbb{P} = \mathbf{u}\otimes\mathbf{v} of two

1 Tensor product of two tensors
2 Tensor product of vector spaces
3 Universal property of the tensor product
4 Tensor product of Hilbert spaces
5 Relation with the dual space
6 Types of tensors, e.g., alternating
7 Over more general rings
8 Tensor product for computer programmers
- 8.1 Array programming languages
9 Notes
10 See also

Tensor product of two tensors

There is a general formula for the components of a product of two (or more) tensors. History The word tensor was introduced in 1846 by William Rowan Hamilton to describe the norm operation in a certain type of algebraic system (eventually For example, if U and V are two covariant tensors of rank m and n (respectively), then the components of their tensor product are given by

$(U\otimes V)_{i_1i_2...i_{m+n}} = U_{i_{1}i_{2}...i_{m}}V_{i_{m+1}i_{m+2}i_{m+3}...i_{m+n}}$ . For other uses of "covariant" or "contravariant" see Covariance and contravariance. ^[1]

Thus, the components of the tensor product of two tensors are the ordinary product of the components of each tensor.

Note that in the tensor product, the factor U consumes the first rank(U) indices, and the factor V consumes the next rank(V) indices, so

$\mathrm{rank}( U \otimes V )=\mathrm{rank}(U)+\mathrm{rank}(V)$

Example

Let U be a tensor of type (1,1) with components U^α_β, and let V be a tensor of type (1,0) with components V^γ. Then

$U^\alpha {}_\beta V^\gamma = (U \otimes V)^\alpha {}_\beta {}^\gamma$

and

$V^\mu U^\nu {}_\sigma = (V \otimes U)^{\mu \nu} {}_\sigma$ .

The tensor product inherits all the indices of its factors.

Kronecker product of two matrices

Main article: Kronecker product

With matrices this operation is usually called the Kronecker product, a term used to make clear that the result has a particular block structure imposed upon it, in which each element of the first matrix is replaced by the second matrix, scaled by that element. Contravariant and covariant tensors A contravariant tensor of order 1(T^i is defined as \bar{T}^i = T^r\frac{\partial \bar{x}^i}{\partial x^r} In mathematics the Kronecker product, denoted by \otimes is an operation on two matrices of arbitrary size resulting in a Block matrix. In the mathematical discipline of Matrix theory, a block matrix or a partitioned matrix is a partition of a matrix into rectangular smaller matrices For matrices $U$ and $V$ this is:

$U \otimes V = \begin{bmatrix} u_{11}V & u_{12}V & \cdots \\ u_{21}V & u_{22}V \\ \vdots & & \ddots \end{bmatrix} = \begin{bmatrix} u_{11}v_{11} & u_{11}v_{12} & \cdots & u_{12}v_{11} & u_{12}v_{12} & \cdots \\ u_{11}v_{21} & u_{11}v_{22} & & u_{12}v_{21} & u_{12}v_{22} \\ \vdots & & \ddots \\ u_{21}v_{11} & u_{21}v_{12} \\ u_{21}v_{21} & u_{21}v_{22} \\ \vdots \end{bmatrix}$ .

Tensor product of multilinear maps

Given multilinear maps $f (x 1,. In Linear algebra, a multilinear map is a Mathematical function of several vector variables that is linear in each variable . . x k)$ and $g (x 1,. . . x m)$ their tensor product is the multilinear function

$(f \otimes g) (x_1,...,x_{k+m})=f(x_1,...,x_k)g(x_{k+1},...,x_{k+m})$

Tensor product of vector spaces

The tensor product $V \otimes W$ of two vector spaces V and W over a field $K$ has a formal definition by the method of generators and relations. In Mathematics, a vector space (or linear space) is a collection of objects (called vectors) that informally speaking may be scaled and added

To construct $V \otimes W$ , one begins with the set of ordered pairs in the Cartesian product V×W. Cartesian square redirects here For Cartesian squares in Category theory, see Cartesian square (category theory. For the purposes of this construction, regard this Cartesian product as a set rather than a vector space. The free vector space on V×W is defined by taking the vector space in which the elements of V×W are a basis. In Mathematics, a free module is a Free object in the category of modules Given a set S, a free module on S is a (particular construction Symbolically,

$F(V\times W) = \left\{\sum_{i=1}^n \alpha_i e_{(v_i\times w_i)}\mid n\in\mathbb{N}, \alpha_i\in K, (v_i\times w_i)\in V\times W \right\},$

where we have used the symbol e_{(v × w)} to emphasize that these are taken to be linearly independent for distinct $(v\times w)\in V\times W$ .

The tensor product arises by defining the following three equivalence relations in F(V×W):

$e_{(v_1+v_2)\times w} \sim e_{v_1\times w}+e_{v_2\times w}\!$
$e_{v\times (w_1+w_2)} \sim e_{v\times w_1}+e_{v\times w_2}\!$
$ce_{v\times w}\sim e_{(cv)\times w} \sim e_{v\times (cw)} \!$

where v,v_i,w,w_i are vectors from V and W (respectively), and c is from the underlying field K. In Mathematics, an equivalence relation is a Binary relation between two elements of a set which groups them together as being "equivalent" Denoting by $\sim$ the space generated by these three equivalence relations, the definition of the operator $\otimes$ is then the quotient space

$V \otimes W = F(V \times W) / \sim$

The equivalence class of (v×w) is called the tensor product of v and w, denoted $v \otimes w$ . In Linear algebra, the quotient of a Vector space V by a subspace N is a vector space obtained by "collapsing" N In Mathematics, given a set X and an Equivalence relation ~ on X, the equivalence class of an element a in X The space $\sim$ is mapped to the kernel, so that the above three equivalence relations become

$(v_1+v_2)\otimes w - (v_1\otimes w+v_2\otimes w) = 0$
$v\otimes (w_1+w_2) - (v\otimes w_1+v\otimes w_2) = 0$
$0v\otimes w=v\otimes 0w=0(v\otimes w)=0.$

The resulting space $V \otimes W$ is a vector space, which can be verified by directly checking the vector space axioms. It is called the tensor product space of V and W. Given bases ${v i}$ and ${w i}$ for V and W respectively, the tensors of the form $v_i \otimes w_j$ forms a basis for $V \otimes W$ . The dimension of the tensor product therefore is the product of dimensions of the original spaces; for instance $\mathbb{R}^m \otimes \mathbb{R}^n$ will have dimension $m n$ .

Universal property of the tensor product

The tensor product is characterized by a universal property. In various branches of Mathematics, certain constructions are frequently defined or characterised by an abstract property which requires the existence of a unique Morphism Consider the problem of embedding the Cartesian product V × W into a vector space X via a bilinear map φ. The tensor product construction V ⊗ W, together with the natural embedding map φ : V × W → V ⊗ W given by

$\varphi (u,w)= u \otimes w ,$

is the "universal" solution to this problem in the following sense. For any other such pair (X, ψ), where X is a vector space, and ψ a bilinear mapping V × W → X, there exists a unique linear map

$T : V \otimes W \rightarrow X$

such that

$\psi = T \circ \varphi.$

Assuming this universal property, it can be readily verified that the tensor product is unique up to isomorphism.

An immediate consequence is the identification of

$B(V \times W, X),$

the bilinear maps from V × W to X and the linear maps

$L(V \otimes W, X).$

The natural isomorphism maps ψ to T. In Category theory, a branch of Mathematics, a natural transformation provides a way of transforming one Functor into another while respecting the internal

Tensor product of Hilbert spaces

Main article: Tensor product of Hilbert spaces

Further information: Positive definite kernel#Direct sum and tensor product

The tensor product of two Hilbert spaces is another Hilbert space, which is defined as described below. In Mathematics, there are usually many different ways to construct a topological tensor product of two Topological vector spaces For Hilbert spaces or In Operator theory, a positive definite kernel is a generalization of a positive matrix. This article assumes some familiarity with Analytic geometry and the concept of a limit.

Definition

The discussion so far has been purely algebraic. In light of the extra structure on Hilbert spaces, one would like to introduce an inner product, and therefore a topology, on the tensor product that arise naturally from those of the factors. Let H₁ and H₂ be two Hilbert spaces with inner products $\langle \cdot,\cdot\rangle_1$ and $\langle \cdot,\cdot\rangle_2$ , respectively. Construct the tensor product of H₁ and H₂ as vector spaces as explained above. We can turn this vector space tensor product into an inner product space by defining

$\langle\phi_1\otimes\phi_2,\psi_1\otimes\psi_2\rangle = \langle\phi_1,\psi_1\rangle_1 \, \langle\phi_2,\psi_2\rangle_2 \quad \mbox{for all } \phi_1,\psi_1 \in H_1 \mbox{ and } \phi_2,\psi_2 \in H_2$

and extending by linearity. In Mathematics, an inner product space is a Vector space with the additional Structure of inner product. That this inner product is the natural one is justified by the identification of scalar-valued bilinear maps on H₁ × H₂ and linear functionals on their vector space tensor product. Finally, take the completion under this inner product. In Mathematical analysis, a Metric space M is said to be complete (or Cauchy) if every Cauchy sequence of points in M has The resulting Hilbert space is the tensor product of H₁ and H₂.

Properties

If H₁ and H₂ have orthonormal bases {φ_k} and {ψ_l}, respectively, then {φ_k ⊗ ψ_l} is an orthonormal basis for H₁ ⊗ H₂. In Mathematics, an orthonormal basis of an Inner product space V (i

Examples and applications

The following examples show how tensor products arise naturally.

Given two measure spaces X and Y, with measures μ and ν respectively, one may look at L²(X × Y), the space of functions on X × Y that are square integrable with respect to the product measure μ × ν. In Mathematics the concept of a measure generalizes notions such as "length" "area" and "volume" (but not all of its applications have to do with In Mathematics, the Lp and ℓp spaces are spaces of p-power integrable functions, and corresponding If f is a square integrable function on X, and g is a square integrable function on Y, then we can define a function h on X × Y by h(x,y) = f(x) g(y). The definition of the product measure ensures that all functions of this form are square integrable, so this defines a bilinear mapping L²(X) × L²(Y) → L²(X × Y). Linear combinations of functions of the form f(x) g(y) are also in L²(X × Y). In Mathematics, linear combinations are a concept central to Linear algebra and related fields of mathematics It turns out that the set of linear combinations is in fact dense in L²(X × Y), if L²(X) and L²(Y) are separable. This shows that L²(X) ⊗ L²(Y) is isomorphic to L²(X × Y), and it also explains why we need to take the completion in the construction of the Hilbert space tensor product. In Abstract algebra, an isomorphism ( Greek: ἴσος isos "equal" and μορφή morphe "shape" is a bijective

Similarly, we can show that L²(X; H), denoting the space of square integrable functions X → H, is isomorphic to L²(X) ⊗ H if this space is separable. The isomorphism maps f(x) ⊗ φ ∈ L²(X) ⊗ H to f(x)φ ∈ L²(X; H). We can combine this with the previous example and conclude that L²(X) ⊗ L²(Y) and L²(X × Y) are both isomorphic to L²(X; L²(Y)).

Tensor products of Hilbert spaces arise often in quantum mechanics. Quantum mechanics is the study of mechanical systems whose dimensions are close to the Atomic scale such as Molecules Atoms Electrons If some particle is described by the Hilbert space H₁, and another particle is described by H₂, then the system consisting of both particles is described by the tensor product of H₁ and H₂. For example, the state space of a quantum harmonic oscillator is L²(R), so the state space of two oscillators is L²(R) ⊗ L²(R), which is isomorphic to L²(R²). The quantum harmonic oscillator is the quantum mechanical analogue of the classical harmonic oscillator. Therefore, the two-particle system is described by wave functions of the form φ(x₁, x₂). A more intricate example is provided by the Fock spaces, which describe a variable number of particles. The Fock space is an Algebraic system ( Hilbert space) used in Quantum mechanics to describe Quantum states with a variable or unknown number of

Relation with the dual space

In the discussion on the universal property, replacing X by the underlying scalar field of V and W yields that the space $(V \otimes W)^\star$ (the dual space of $V \otimes W$ , containing all linear functionals on that space) is naturally identified with the space of all bilinear functionals on $V \times W$ . In Mathematics, any Vector space V has a corresponding dual vector space (or just dual space for short consisting of all Linear functionals In Mathematics, a functional is traditionally a map from a Vector space to the field underlying the vector space which is usually the Real In other words, every bilinear functional is a functional on the tensor product, and vice versa.

Whenever $V$ and $W$ are finite dimensional, there is a natural isomorphism between $V^\star \otimes W^\star$ and $(V \otimes W)^\star$ , whereas for vector spaces of arbitrary dimension we only have an inclusion $V^\star \otimes W^\star\subset (V \otimes W)^\star$ . In Abstract algebra, an isomorphism ( Greek: ἴσος isos "equal" and μορφή morphe "shape" is a bijective So, the tensors of the linear functionals are bilinear functionals. This gives us a new way to look at the space of bilinear functionals, as a tensor product itself.

Types of tensors, e. g. , alternating

Linear subspaces of the bilinear operators (or in general, multilinear operators) determine natural quotient spaces of the tensor space, which are frequently useful. In Topology and related areas of Mathematics, a quotient space (also called an identification space) is intuitively speaking the result of identifying See wedge product for the first major example. Another would be the treatment of algebraic forms as symmetric tensors. In Mathematics, a homogeneous polynomial is a Polynomial whose terms are Monomials all having the same total degree; or are elements of the same

Over more general rings

The notation $\otimes_R$ refers to a tensor product of modules over a ring R. In Mathematics, the tensor product of modules is a construction that allows arguments about bilinear maps (roughly speaking "multiplication" to be carried out in In Mathematics, a ring is an Algebraic structure which generalizes the algebraic properties of the Integers though the rational, real

Tensor product for computer programmers

Array programming languages

Array programming languages may have this pattern built in. In Computer science, array programming languages (also known as vector or multidimensional languages generalize operations on scalars to apply For example, in APL the tensor product is expressed as $\circ . \times$ (for example $A \circ . \times B$ or $A \circ . \times B \circ . \times C$ ). In J the tensor product is the dyadic form of */ (for example a */ b or a */ b */ c). Not to be confused with the J++ or J# programming languages The J programming language, developed in the early 1990s by

Note that J's treatment also allows the representation of some tensor fields (as a and b may be functions instead of constants -- the result is then a derived function, and if a and b are differentiable, then a*/b is differentiable). In Calculus, a branch of mathematics the derivative is a measurement of how a function changes when the values of its inputs change

However, these kinds of notation are not universally present in array languages. Other array languages may require explicit treatment of indices (for example, Matlab), and/or may not support higher-order functions such as the Jacobian derivative (for example, Fortran/APL). MATLAB is a numerical computing environment and Programming language. In Mathematics and Computer science, higher-order functions or '''functionals''' are functions which do at least one of the following In Vector calculus, the Jacobian is shorthand for either the Jacobian matrix or its Determinant, the Jacobian determinant. Fortran (previously FORTRAN) is a general-purpose, procedural, imperative Programming language that is especially suited to

Notes

^ Analogous formulas also hold for contravariant tensors, as well as tensors of mixed variance. Although in many cases such as when there is an inner product defined, the distinction is irrelevant. In Mathematics, an inner product space is a Vector space with the additional Structure of inner product.

Contents