This is intended to be a research seminar. It is assumed that students have ``mathematical maturity'' but there are no course prerequisites other than sufficient interest in this strange blend of math, science and technology.
A nice general definition is that ``neural network'' refers to any sort of graph-theoretic structure used for computing. That is, one has a (presumably large) set of processors which are interconnected in a network like the neuronal cells of the animal brain. Indeed, the early versions of neural nets were based quite literally on an over-simplified model of neurons and the ``wetware'' still provides a role model and metaphor. Today, we know that neuronal cells are much more complex but the guiding principles remain applicable. (One of the exciting uses of neural networks is to model psychological and neurophysiological processes.)
Neural networks have arisen independently in computer science, cognitive science and mathematics. Much of the recent work in quantum computing can be rephrased in terms of neural networks and amounts to yet another independent rediscovery. Physicists had previously utilized the neural network metaphor in modeling ``spin glass'' and phase changes. Neuroscience, philosophy and psychology also have studied the concept.
Neural networks are well-suited to problems of pattern recognition which is a significant part of the AI puzzle. For example, one might want to have the ability to recognize impending collisions in a transportation system or to predict power usage in an urban electrical grid.
Neural network theory includes work by Hilbert, Arnold and Kolmogorov, arguably among the greatest mathematicians of the previous century. Mathematical research in this area involves a broad range of subdisciplines - notably, analysis, geometry, graph theory and statistics. Further, neural networks provide a new perspective on nonlinear approximation and will likely have applications to nonlinear optimization. Indeed, the combination of disciplines involved has already led to novel insights.
Therefore, in addition to the rising practical interest in this topic, neural networks are an appropriate subject for theoretical mathematics. Conversely, those from other disciplines who wish to investigate neural networks will be well served by having knowledge of the foundations. Finally, it is to be hoped that the ``glamour'' of the subject will motivate a new group of people to study math.
The following is a tentative syllabus for the course.
1. Overview of neural networks: History, status and prospects
2. Feedforward neural networks and mathematical physics
For each of the following, we use the name of one well-known practitioner stand for the study of a particular approach:
3. Boltzmann machines
4. Hopfield networks
5. Grossberg ART
6. Amari information geometry
7. Vapnik
8. Sejnowski
9. Hecht-Nielsen
10. Kohonnen
11. Kurkova
12. Nonlinear approximation
1. What is a neural network? (the basic concept and some of its history; behavior of the elementary units; feedforward vs. fully interconnected topologies; connectionism and computation; functional approximation and representation; finding the weights)
2. Analysis and neural networks (polynomial approximation; Weierstrass and Chebyshev contributions; normed linear spaces; dense subsets as universal approximation; connections with integral transforms)
3. Geometry and neural networks (affine geometry and uniqueness of parameterization; geometry of balls, cubes and hyperoctahedra - implications for approximation; geometry of the unit ball and best approximation; convexity)
4. Statistics and neural networks (probabilistic neurons; adding noise to the weights; adding noise to the activation function; pattern recognition; quasi-orthogonality; signal processing; quantum computing)
5. Graph theory and neural networks (implementation constraints and bounded vertex degree; graph invariants relevant to neural network design; random graphs and their properties; topology of graphs and implications for implementation)
There are papers available on these topics by various researchers, including the lecturer, with his colleagues Andrew Vogt, at Georgetown, and Vera Kurkova, at the Czech Academy of Sciences Institute of Computer Sciences in Prague. Students with an interest in any of these mathematical aspects of neural networks are encouraged to attend.
However, I intend to keep the seminar sufficiently ``grounded'' that the relevance of various mathematical formulations will be plausible enough that students with ideas regarding implementations and applications of neural methods will similarly have a chance to distinguish themselves through their research projects.
Paul C. Kainen, Department of Mathematics, Georgetown University; 202-687-2703; e-mail
Back to my
classroom page or home page /html