The perception of speech and other auditory signals of comparable complexity requires that we process the signal in a manner that permits both the determination of local temporal order (e.g. pest versus pets) and the analysis of subtle intonation changes (e.g. lunch? lunch!). Successful lexical access demands that we resolve elements changing in time on the order of 20-50ms. Successful analysis of prosodic phenomena demands that we resolve frequency changes of less than 10Hz. Models that rely on a single time scale for the analysis of complex auditory signals have a difficult time capturing both types of phenomena. I argue that there are at least two time scales (temporal integration windows) that are critical for the analysis and representation of speech, 25-50ms and 200-300ms. Evidence for temporal integration on this order is provided by psychophysical and neurobiological data. A temporal multiresolution model, asymmetric sampling in time (AST), is discussed as one possible approach.