Banach Spaces, Hilbert Space

Hilbert Space

A hilbert space (biography) is a banach space with a dot product. So - if you were brought here by a search engine, and you want to understand hilbert spaces, you need to be familiar with banach spaces, and the dot product in n space. The definition of dot product, given below, is consistent with the euclidean definition in Rⁿ.

The dot product is a binary operator whose operands are vectors in a banach space S. The result is a real number. If x and y are vectors in S, the dot product is indicated by a literal dot, as in x.y, hence the name dot product. This is also called an inner product.

The dot product respects scaling, so that c×(x.y) = cx.y = x.cy for any real constant c. Also, the dot product respects addition in either component. As a corollary, x.0 = x.y-x.y = 0, and similarly, 0.x = 0.

Symmetry is another requirement: x.y = y.x.

Finally, x.x = |x|², where |x| is the norm of x in the banach space S.

Thus far, a banach space could have any norm, but in a hilbert space, the norm is tied to the dot product.

Notice that the traditional dot product in euclidean space satisfies all these properties, thus Rⁿ is a hilbert space.

A finite dimensional banach space is homeomorphic to Rⁿ, as demonstrated in the previous section. Hence a finite dimensional banach space can be viewed as a hilbert space, by applying the euclidean norm and dot product.

Complex Space

Although we have skirted this topic thus far, a complex banach space is no different than a real banach space. If S is a complex vector space it is certainly a real vector space, and the distance metric is unchanged, thus S becomes a real vector space that is also a complete metric space, i.e. a real banach space.

The dot product changes slightly when S is viewed as a complex banach space. A constant is conjugated when applied to the second operant, so that cx.y is the conjugate of x.cy. Also, x.y and y.x are conjugates. This is consistent with the definition of dot product in n dimensional complex space, where the second vector is conjugated, then corresponding components are multiplied, and the pairwise products are added together. This seems like a trick, but it facilitates x.x = |x|². Concentrate on one component in an n dimensional vector, say a+bi. This is multiplied by a-bi, giving a²+b², which contributes to the norm of the vector, just as it would in real space, where a and b are separate (real) components. Therefore the traditional dot product in n dimensional complex space satisfies our properties, and Cⁿ is a complex hilbert space.

The Space of Continuous Functions

Let M be the space of continuous real valued functions on [0,1]. (You can use complex functions if you like; it's not much different.) This is a real vector space, and the norm |f| = sqrt(∫ f²) makes it a normed vector space. Actually we should stop and prove this is a norm. Because f is continuous, it cannot stray from 0 at a single point; it must leave 0 over a subinterval, giving a nonzero integral. Thus f = 0 iff |f| = 0. Properties of linearity are straightforward. Finally, |f+g| ≤ |f|+|g| because the same is true of the step functions that approach f and g. These step functions are represented as vectors in n space, where the triangular inequality always holds.

Complete this metric space to build a banach space S. The completion includes every function that is approached by continuous functions. This includes piecewise continuous functions. Let f_n = 0 from 0 to ½-1/n, then slope up to 1 at x = ½+1/n, then remain at 1 across the rest of the unit interval. The limit is the discontinuous function that is 0 on [0,½), ½ at ½, and 1 on (½,1].

Note that S may include functions that are not integrable. In fact the elements of S may not be functions at all. Tweak the above example, so that when n is odd the sloping line segment runs from ½-1/n,0 up to ½,1, and when n is even the segment runs from ½,0 up to ½+1/n,1. The limit function is 0 on [0,½) and 1 on (½,1], but is not defined at x = ½. Even more bizarre examples are possible. So - you might wonder about the distance metric in S, where functions, and their integrals, are not well defined. Remember, distance in S ultimately comes from distance in M, which is always well defined. If f and g are functions in S, |f,g| is the limit of the distances between the functions that approach f and the functions that approach g. In any metric space, this limit exists, hence distance is well defined in S, and makes S a metric space. In fact S is a complete metric space, since the completion is always complete. If these concepts are unfamiliar to you, you can review metric spaces here.

With |f| defined on M, and on S, S becomes a complete normed vector space, or a banach space. Let's turn it into a hilbert space.

Let f.g be the integral of f×g in M. Since f×g is continuous, this is well defined, and when f = g, the result is the square of the norm. Verify the properties of linearity and symmetry, and ∫ f×g becomes a dot product for M.

We need to extend this to all of S. Let f_n and g_n be cauchy sequences in M, defining elements of S. By ignoring leading terms, we can assume that all the terms in f_n are within 1 of each other. (This is the property of being cauchy, with ε set to 1.) Similarly, assume the terms of g are within 1 of each other. Now if |f_n| exceeds |f₁|+1, the triangular inequality is violated. Therefore all the terms of f, and all the terms of g, have norms below some constant w.

For a small ε, smaller than 1, go out in both sequences so that functions beyond f_n, or beyond g_n, are within ε of each other. Consider pairs of functions f_i and f_j, and g_i and g_j. The difference between the two dot products is the integral of f_ig_i-f_jg_j. Can this be bounded by some constant times ε?

For notational convenience let u = f_i and let u+a = f_j. Let v = g_i and let v+b = g_j. The integrand becomes ub+va+ab. Consider the last term first. The integral of ab, squared, is no larger than the integral of a² times the integral of b². How do we know? This is the cauchy schwarz inequality. Since a is the gap between f_i and f_j, a.a is ε². Similarly, b.b is ε². The square of a.b is no larger than ε⁴, hence a.b ≤ ε².

Apply cauchy schwarz to the integral of ub, and bound it below the square root of the integral of u² times the integral of b². This is |u|×|b|, or wε. Similarly, v.a is bounded below wε. Put this all together and the difference between the dot products in positions i and j is bounded by (2w+1)ε. The sequence of dot products is cauchy, and converges to a real number. This is the dot product of the two sequences f_n and g_n in S.

Is this well defined? Let another sequence of continuous functions h_n represent the same element in S as g_n. In other words, their difference, e_n, converges to 0. Consider the limit of f_n.e_n as n approaches infinity. Each term is bounded below |f_n|×|e_n|, and |e_n| approaches 0, while |f_n| is bounded below w. The dot products f_n.g_n and f_n.h_n converge to the same real number, and dot product is well defined in S.

The properties of linearity and symmetry are straight forward, so consider the last property, the dot product of a sequence f_n with itself. Replace the terms with |f_n|². Since |f_n| approaches the norm of the entire sequence, by definition, and the limit of the squares is the square of the limit, the dot product approaches the norm of the sequence, squared. That completes the proof.

In summary, the completion of the continuous functions on [0,1] is a banach space, and a hilbert space, using integration and limits to define the norm and the dot product.