Is there any analog of the computational classes $FP$ and $FNP$ with probabilistic or quantum Turing machines? If so, what are the relation with other computational classes?
I hope the following partially answers your question. I've never seen this observation published anywhere (please correct me if I'm wrong).
First of all, FQMA - a quantum analog of FNP - was defined: https://complexityzoo.uwaterloo.ca/Complexity_Zoo:F#fqma
One of the compelling reasons to define the complexity class NP is that there is a decision to search reduction: polynomial access to an oracle that solves NP-Complete problems allows you to search for a solution of any problem in FNP in polynomial time.
A decision to search reduction is not known between QMA and FQMA . For example, access to an oracle for the Local Hamiltonian problem is not known to be sufficient to efficiently construct a ground state.
Another related (although, not directly related to your question) reduction which is known for many classical NP problems, but not for QMA problems is downward self reducibility: A language L is downward self reducible if there is a cook reduction from L to itself such that the queries are shorter than the original input. For example, given a Local Hamiltonian H with m terms, decide whether its minimal energy is below a or above b (where b-a>1/poly) by asking questions regarding Hamiltonians which have m-1 terms or less. Many decision to search reductions have this additional property.