There is an old joke about the smallest non-interesting number being interesting in itself (I have heard it attributed to Richard Hamming). This is then used to justify the argument that every number must be interesting. I have always found this induction unsatisfactory.
Here is an attempt at a more precise argument. Consider a set of rules, represented as sentences of some logic. The interesting numbers are then the smallest numbers that are models of some set of sentences. In first-order logic each natural number can be distinguished from every other one, so the sentences true for any natural number will differ from those true for any other, and every number will be interesting.
Now suppose our rules are given in terms of a bounded model of computation, so that only finite sets of properties can be associated with each number, and so that every property can be checked with a bounded amount of some resource (for instance, time or space).
Is every number still interesting with resource-bounded properties?
Edit: to clarify, resource bounds is my focus here. With unbounded resources this devolves into a question about definability and large ordinals, as discussed by Aaron and Kaveh. The idea is that each interesting number should be definable by a sentence that can be checked given a bounded amount of resources. (It might also make sense to allow some finite set of sentences as separate building blocks instead of making them into single sentences via conjunction.)