The other extreme is to say two programs are equivalent iff they compute the same function (or show the same observable behavior in similar environments). But these are not good: not all programs checking primality are the same. We can add a line of code with no effect on the result and we would still consider it the same program.
This is not an extreme: program equivalence must be defined relative to a notion of observation.
The most common definition in PL research is contextual equivalence. In contextual equivalence, the idea is that we observe programs by using them as components of larger programs (the context). So if two programs compute the same final value for all contexts, then they are judged to be equal. Since this definition quantifies over all possible program contexts, it is difficult to work with directly. So a typical research program in PL is to find compositional reasoning principles which imply contextual equivalence.
However, this is not the only possible notion of observation. For example, we can easily say that the memory, time, or power behavior of a program is observable. In this case, fewer program equivalences hold, since we can distinguish more programs (eg, mergesort is now distinguishable from quicksort). If you want to (say) design languages immune to timing channel attacks, or to design space-bounded programming languages, then this is the sort of thing you have to do.
Also, we may choose to judge some of the intermediate states of a computation as observable. This always happens for concurrent languages, due to the possibility of interference. But you might want to take this view even for sequential languages --- for example, if you want to ensure that no computations store unencrypted data in main memory, then you have to regard writes to main memory as observable.
Basically, there is no single notion of program equivalence; it is always relative to the notion of observation you pick, and that depends on the application you have in mind.