We shall take the notion of set and element as primitive notions. Let 0 denote the empty set (distinguished from zero by context). Write "I" for "intersection" and "U" for "union" which are defined as follows: If A and B are subsets of S, then A I B = {s in S| s is in A and B} and A U B = {s in S| s is in A or B}. Intersections and unions of finite collections follow by induction. More generally, for any set J, we define the union of a J-indexed family of subsets of S by: U {V_j | j in J} = {s in S| there exists j in J with s in V_j} and I {V_j | j in J} = {s in S| for all j in J, s in V_j}. _Complement_ is the anaologue of reversing 0 and 1 for a binary sequence. If A is a subset of S, then S-A = {s in S| s not in A} is the complement of A.
For any set S, the _power set_ of S, denoted 2^S, is the collection of all subsets of S. Equivalently, 2^S can be identified with the set of all functions from S to the set {0,1} by regarding each subset T of S as the function which takes value 1 on the elements of T and 0 on the elements in its complement S-T.
A _topological space_ is an ordered pair (X,t), where X is a nonempty set and t is any subset of 2^X which satisfies the following three axioms:
If X is any nonempty set, the _discrete topology_ for X is 2^X, and the _codiscrete topology_ for X is {0,X}. (The reader should check that the three axioms hold for each of these toplogies, which are, respectively, the largest and smallest possible topologies for X.)
Suppose that A is any subset of X and t is a topology for X. Then t_A = {A I V | V in t} is a topology for A (Exercise 1). That is, a subset of A is open if and only if it is the intersection of A with an open subset of X. For instance, if X is the real line and A is the closed interval [0,1], then the interval [0,1/4) is an open subset of A since it is the intersection of A with (-1/4,1/4) which is open in X. We call t_A the subspace on A induced by the topology t on X.
A _partition_ of a set S is a pair of subsets A,B such that S is the union of A and B and A and B have empty intersection. A topological space X is called _disconnected_ if it has a nontrivial partition by two open sets. Nontrivial means that neither of the two open sets is 0 or X; that is, X is disconnected if there exist nontrivial open sets V, W with V U W = X and V I W = 0. X is _connected_ if it is not disconnected.
Let X = {a,b}. The family t = {0,X,{a}} is a topology for X. Is (X,t) connected or disconnected? (Exercise 2.) Also, show that for any X with more than one element, (X,2^X) is disconnected (Exercise 3).
A function f:X \to Y is called a _continuous map_ between topological spaces (X,t) and (Y,s) if for every V in s, f^-1(V) is in t (i.e., the inverse image of an open set must be open).
This is a generalization of the notion of continuous function given in calculus as we now show. We define the standard topology, denoted m, for the real line (-oo,oo) as follows: A subset V is open if whenever s is in V, then there exists a positive number h (which depends on s and V) such that all points within distance less than h of s are also in V; that is, V contains (s-h,s+h). If a is less than b, then the interval (a,b) is open in this sense because if c is in (a,b), then for h = min {|a-c|,|b-c|}, (c-h,c+h) is contained in (a,b). The reader should now check that continuity in the sense of calculus of a function from R to R is equivalent to continuity as a map of topological spaces, with respect to the topology m. (The reader can check that all of this works if instead of the reals, we used any metric space - i.e., a set with a notion of distance satisfying the usual properties of positive definiteness, symmetry and the triangle inequality. The resulting topology is called the _metric topology_.)
A subset A of X is called connected (wrt t) if t_A is connected.
Theorem: If A is a connected subset of X wrt t and f:X \to Y is a continuous map of (X,t) to (Y,s), then f(A) is a connected subset of Y wrt s. (Exercise 4: Prove this!)
A _covering_ of a subset A of X is a family F = {A_j | j in J} of subsets A_j of X such that A is contained in U {A_j | j in J}. If X is a topological space and the A_j are open, then F is called an _open covering_. (Note that we are now surpressing explicit reference to the topology t.) The term "covering" is often shortened to "cover".
The topological space (X,t) is defined to be _compact_ if for every open cover F of X, there is a FINITE subset F' of F which is also a cover of X. A subset of X is compact if the induced (subspace) topology is compact. If X is compact and A is any closed subset, then A is compact.
Theorem: If A is a compact subset of X wrt t and f:X \to Y is a continuous map of (X,t) to (Y,s), then f(A) is a compact subset of Y wrt s. (Exercise 5: Prove this!)
Congratulations. You've just proved that a continuous function fromm R to R maps a closed interval onto a closed interval, which gives both the extreme and intermediate value theorems from calculus.
The algebra of sets tells us that the intersection of any family of closed sets is closed. Hence, if A is any nonempty subset of X, the intersection of all closed subsets of X which contain A must also be closed, and we call this set the _closure_ of A, denoted cl(A), or cl(A,t) if it is necessary to indicate the topology t on X with respect to which closure is taken.
Call a topology t _stronger_ than the topology t' (both for the same set X) if t is contained in t'. Hence, closed in t implies closed in t'. Thus, if t is stronger than t', then cl(A,t) contains cl(A,t').
For the set C(Q) of all continuous functions defined on the cube in d-dimensional space to the real line the topology t_sup induced by the sup norm (i.e., the metric topology where the metric is norm of difference) is stronger than the topologies t_p induced by the L_p norm, for p strictly between 1 and oo. (Recall that the L_p norm for an element f of C(Q) is the p-th root of the integral of |f|^p over Q.) Indeed, the L_p norm of f cannot exceed the product of the sup norm by the p-th root of the volume of Q and WLOG the latter factor is 1. Geometrically, this actually means that the unit ball of the sup-norm is contained in the unit ball of any L_p-norm unit ball.
If X is a topological space and A is a subset, we call A _dense_ in X (with respect to a particular topology t) if cl(A,t) = X. The reader should check that if A is dense with respect to t and t is stronger than t', then A is dense wrt t' as well. It follows that the density theorem for neural network functions wrt sup norm topology implies that it holds for any L_p-topology, for p strictly between 1 and oo.
Back to the classroom page or neural page . Jan. 16, 2001; pck