The complexity class AM are those problems which can be solved by a two-round interactive proof system between a prover "Merlin" and a verifier "Arthur". A problem — which tests some property of an object X — is in AM if:
For YES instances, for a random "challenge" message (of polynomial length) Arthur generates, with high probability Merlin can formulate a (polynomial length) reply which Arthur can use as evidence that X has the property;
For NO instances, for a random challenge message Arthur generates, with high probability Merlin cannot formulate any reply which can be used as evidence for the property being tested for on X.
— The class described doesn't change if we require Merlin to give a useful answer not just with high probability, but for any challenge that Arthur may issue; we might say in this case that we require Merlin's answer always to be valid for YES instances, and what Arthur tests is the validity of the answer. So if Merlin ever produces an invalid response, Arthur knows that the problem instance is a NO instance. This is the setting I'd prefer to consider.
An example is Graph Non-Isomorphism: given graphs G and H with the same set of vertex labels, Arthur can randomly select one of the graphs and produce a "scrambled" version F by permuting its vertex labels, sending a presentation of it to Merlin. If the two graphs are non-isomorphic, Merlin can identify which of G or H Arthur chose by determining whether F ≅ G or F ≅ H, and can respond by identifying which of the two F is isomorphic to. If the two graphs G and H are isomorphic, however, Merlin cannot distinguish which graph F came from, and any answer he gives can only be correct by chance. Thus, for YES instances Merlin can always send a valid response to any challenge; for NO instances any response which Merlin might send will be with high probability invalid.
In the above problem, not only does there exist a valid response that Merlin can issue to Arthur for each challenge, but in fact there is a unique valid response: i.e. indicate which of G or H Arthur chose, given that this can be determined by identifying which is isomorphic to F.
Does imposing a constraint along these lines — that for YES instances, for any challenge Arthur might send, there is exactly one valid response for Merlin — yield a more restrictive class, in the sense of yielding a class which is not known to equal AM?