The existence of problems in DTIME(2^O(n)) which require exponential-size circuits to compute implies PRGs(which is the assumption in IW) seems plausible since otherwise we would have non-uniformity giving a speedup on EVERY computational problem -- which imply P=RP=BPP, by Impagliazzogoes completely against the current thinking that does not see a "too significant" gap between uniform and non-Wigderson:uniform complexity for "normal" problems. This thinking comes from the fact that there are very few examples where a "non-uniform" algorithm is known that is significantly better than the known uniform one http://www.math.ias.edu/~avi/PUBLICATIONS/MYPAPERS/IW97/proc.pdf(again except for derandomization).
Another piece of "evidence" is that relative to a random oracle we do have P=BPP.