Theorem 1. The problem has an $O(m\log m)$-time algorithm, where $m=|M|$.
Proof sketch.
Fix an instance of the problem.
Assume without loss of generality that $M\subseteq (0, 1)$.
(Otherwise an optimal solution $a_1,\ldots, a_n$ for $M$
can be obtained by computing an optimal solution $a'_1, a'_2, \ldots, a'_{n'}$ for $M'=M\cap (0, 1)$,
then prepending the elements in $M\cap [1, \infty)$ in decreasing order.)
By rewriting, the problem is equivalent to finding
$p=(p_1, p_2, \ldots, p_n) \subseteq M$ maximizing
$$\text{value}(p) = \sum_{i=1}^n \big(\textstyle\prod_{j=1}^{i-1} p_j\big) (1-p_i) g(p_i),~~~~~~~~~(1)$$
where $g(z) = f(z)z/(1-z)$.
Remark for intuition. This interpretation is nice because then $\text{value}(p)$ is the expectation of the value returned by the following random process:
- for $i \gets 1, 2, \ldots, n$:
- $~~~$ with probability $1-p_i$, halt and return $g(p_i)$ (otherwise (with probability $p_i$) continue)
- return 0
With this interpretation, it is not too hard to see that we should use all elements of $M$ (in some order), and further that we should use them greedily---in order of decreasing $g(p_i)$.
The proof below gives the details.
Here is the algorithm:
- let $q=(q_1, q_2, \ldots, g_m)$ be $M$ sorted by decreasing $g(q_i)$ (breaking ties arbitrarily).
- return $q$
Clearly this can be done in $O(m\log m)$ time.
To finish we show that $q$ is an optimal solution.
By inspection of (1), there is an optimal solution
that uses all elements of $M$ (as appending unused elements
to any solution cannot decrease the cost).
Let $p=(p_1, p_2, \ldots, p_m)$ be any such optimal solution.
Define an inversion in $p$ to be a pair $i, j\in [m]$
such that $i < j$ and $r(i) > r(j)$,
where $r(i)$ is the rank of $p_i$ in $q$
(so $q_{r(i)} = p_i$).
If there are no inversions, then $p=q$ so $q$ is optimal, and we are done.
Otherwise, there is an inversion $(i, i+1)$.
Consider modifying $p$ by swapping $p_i$ with $p_{i+1}$.
By calculation, this swap
increases the solution value by $\prod_{j=1}^{i-1} p_j$ times
$$
\begin{aligned}
&(1-p_{i+1}) g(p_{i+1})+ p_{i+1} (1-p_i) g(p_i) \\
{}-{} & p_i (1-p_{i+1}) g(p_{i+1}) - (1-p_i) g(p_i) \\
{}={} & [g(p_{i+1}) - g(p_i)] (1-p_i)(1-p_{i+1}) \\
{}\ge{} & 0 & (\text{using } r(i) > r(i+1), \text{ so } g(p_i) \le g(p_{i+1})).
\end{aligned}
$$
So this swap reduces the number of inversions by one
and yields another optimal solution.
So repeatedly swapping yields an optimal solution $p$ with no inversions, i.e., with $p=q$, so $q$ is optimal.$~~~~\Box$