Don H. Johnson

J.S. Abercrombie Professor Emeritus of Electrical & Computer Engineering
Duncan Hall 2041               713.348.4956 (office)
ECE Department, MS #380        713.348.5686 (FAX)
Rice University 
Houston, Texas 77005-1892      dhj@rice.edu

Publications

Library

Course and Activity Web Pages

 

ELEC 241: Fundamentals of Electrical Engineering I (MWF 11-12, DH1064)

 

Errata to Johnson & Dudgeon, Array Signal Processing: Concepts and Techniques

Auditory Neuroscience: How is Sound Processed by the Brain?

Biography of Edward L. Norton, co-originator of the Mayer-Norton equivalent circuit

Historical outline of electrical and computer engineering

Brief thoughts on teaching

Don Johnson received the S.B. and S.M. degrees in 1970, the E.E. degree in 1971, and the Ph.D. degree in 1974, all in electrical engineering from the Massachusetts Institute of Technology. He joined M.I.T. Lincoln Laboratory as a staff member in 1974 to work on digital speech systems. In 1977, he joined the faculty of the Electrical and Computer Engineering Department at Rice University, where he is currently the J.S. Abercrombie Professor in that department and Professor in the Statistics Department. At MIT and at Rice, he received several institution-wide teaching awards, including Rice's George R. Brown Award for Excellence in Teaching and the George R. Brown award for Superior Teaching four times. He was a cofounder of Modulus Technologies, Inc. He was President of the IEEE's Signal Processing Society, and he received the Signal Processing Society's Meritorious Service Award for 2000 and was one of the IEEE Signal Processing Society's Distinguished Lecturers. Professor Johnson is a Fellow of the IEEE.

Professor Johnson's present research activities focus on issues in statistical signal processing. Particular areas of interest are non-Gaussian signal processing, development of a theory of information processing (as opposed to signal processing) , and the transmission and extraction of information by neural signals (particularly in sensory systems). A new area—art forensics—is a new focus. A curriculum vita (PDF) is available as well as a list of recent publications, some of which have not appeared in print.

Current students and their projects:

• Ilan Goodman, a doctoral student investigating information theoretic performance limits for neural prosthetics
• Eva Dyer, a master's student developing a model for feature extraction in V1

Non-Gaussian Statistical Signal Processing

All signal processing techniques exploit signal structure; when the signals are random, we want to understand the probabilistic structure of irregular, ill-formed signals. Such signals can be either be bothersome (noise) or information-bearing (discharges of single neurons). Our research is predicated on the notion that a deep understanding of a signal's structure will result in signal processing algorithms that can either suppress bothersome signals or enhance information-bearing ones. Current research ranges from fundamental studies of non-Gaussian signals and how systems extract and represent information to applying these theories to the analysis of neural data and modeling of how neural structures process information.

	Stationary non-Gaussian signals occur frequently in practical situations. For example, the amplitude distributions of ambient underwater sounds and of background electromagnetic signals have been found to deviate strongly from a Gaussian characterization.
	Neural discharges are modeled as stochastic point processes, which have no waveform, thereby disallowing Gaussian models.
	Combined discharges of neural populations and DNA sequences represent examples of symbolic data, which have amplitudes selected from a finite set: the signal takes on values drawn from the alphabet representing base pairs {A, C, G, T}. These signals are particularly interesting since amplitudes have no mathematical operations defined for them: No field or group can be meaningfully defined for them. We have found ways of computing the Fourier and wavelet transforms of symbolic signals.

A Theory of Information Processing

Electrical signals can carry two "things:" power and information. Signal processing theory concerns the structure of the signals (it does not describe what they represent nor how well they represent it) and frames how systems affect signal components (not the how the system's actions affect what it conveys). Understanding power formed the basis of electrical engineering, and is well understood; understanding information representation and processing is far from complete. Shannon's work describes how to efficiently represent signals (whether they represent information or not) and how communication channels can reliably communicate digital signals. Our work considers the information rather than the signal, and we have quantified how well a signal codes information and how systems affect the information conveyed by their inputs. The fundamental aspects of our theory are:

• Information can have any form and need not be stochastic.
  In general, we make no assumption about what form information takes. We take it to be a member of a set (either countable or uncountable). This symbol represents the meaning of the information that signals encode. Information
  uninteresting to the information sink may also be encoded into X.
• Signals represent information.
  Information does not exist in a tangible form; rather information is always encoded into a signal. The signal representing some information can take on very different forms. For example, the text you are reading now could also have been
  read to you. Thus, the text's alphabetic/punctuation symbol sequence, the image of a page and the air pressure at your ears each represent the same information as well as superfluous. In the first case, the signal is a sequence drawn from a discrete, finite alphabet; in the latter two, the signal is analog. Any viable information processing theory must place a variety of signals on the same footing and allow various representation modes to be compared.
• What may or not be information is determined by the information sink.
  The recipient of the information—the information sink—determines what about a signal it receives is informative.
  Returning to the example describing the various ways in which the information in this paper could be represented, extraneous information to the page's central theme is also contained in the signal. The text comes from a Roman alphabet expressing English prose. The page image reveals page numbers, fonts and details of the mathematical notation. The spoken text encodes the sex of the reader and dialect, which provides clues as to his/her region of origin. Such information may
  or may not be considered extraneous by some succeeding processing. For example, speaker identification systems are presumably designed to ignore what is being said. It is for the recipient to determine what information is. An immediate consequence of this assumption is that no single objective measure can quantify the information contained in a signal. "Objective" here means analysis of a signal out of context and without regard to the recipient.
• When systems act on their input signal(s) and produce output signal(s), they indirectly perform information processing.
  Systems map signals in their input space onto their output space. Because information does not exist as an entity, systems affect the information encoded in their inputs by their input-output map. We can assess the effect a system has on information-bearing signals by examining the input and output signals from the viewpoint of the ultimate information sink.
• The result of processing information is an action.
  An action is a quantifiable behavior influenced by the information. We assume that every action is a signal. For example, the influence of a voice command can be measured by observing the recipient's consequent behavior.

We apply our theory to issues in the area of neural information processing.

Neural Information Processing

In the nervous system, sensory information is represented in single neurons by sequences of action potentials—brief, isolated pulses having identical waveforms—occurring randomly in time. These signals are usually modeled as point processes; however, these point processes have a dependence structure and, because of the presence of a stimulus, are non-stationary. Thus, sophisticated non-Gaussian signal processing techniques are needed to analyze data recorded from sensory neurons to determine what aspects of the stimulus are being emphasized and how emphatic that representation might be. A recent paper analyzes well-established data analysis techniques for single-neuron discharge patterns. Another recent paper describes how we applied our theory of information processing to neural coding. Another paper describes information theoretic (capacity) results for neural populations and hints at how they can be applied to neural prosthetics.

Art Forensics

Signal processing aids the area of art forensics by developing algorithms for assessing paintings in various ways. As part of the Thread Counting Project centered at Cornell, Professor Johnson developed the best performing algorithm for determining canvas weave densities in x-ray images made of van Gogh paintings (provided to the Project by the Van Gogh Museum, Amsterdam). An example x-ray from a portion of a van Gogh painting and the 2D spectrum shows how the weave of the canvas corresponds distinctive spectral points. A conference paper summarizes the algorithm.

Fundamentals of Spectral Approach to Thread Counting

Original x-ray swatch