# Reference for automatically deriving dynamic programming algorithms from recursive algorithms?

This is what I'm looking for. Take a recursive algorithm:

def fib(n):
if n == 0 or n == 1:
return n
else:
return fib(n-1) + fib(n-2)


and turn it into this:

def fib(n):
a, b = 0, 1
for _ in range(n):
a, b = b, a + b
return a


...in a way that's fully automated.

My motivation is a hypothesis about automatic differentiation. The independent researcher Sandro Magi has had the following idea:

• The dual numbers can be used to express forward mode automatic differentiation in very little code. This fact is well known.

• Magi has shown that the dual numbers have a mathematically similar counterpart which he calls the codual numbers. On some level, there is no difference between the dual numbers and codual numbers; they are simply two different ways of representing the same mathematical object on a computer.

• The codual numbers naturally express something similar to reverse-mode autodiff, which is of major interest to applications.

Unfortunately, Magi's simple algorithm exhibits exponential time behaviour on some instances. A good implementation of autodiff shouldn't behave that way. Simple experiments show that all known examples of exponential time behaviour can be solved using memoisation (as I've done here). The introduction of memoisation makes the time complexity of the resulting algorithm difficult to gauge.

Whenever exponential time behaviour gets fixed using memoisation, it indicates a connection to dynamic programming. In fact, this can be considered an instance of top-down dynamic programming, instead of the usually preferable bottom-up dynamic programming. I conjecture therefore that tape-based autodiff is the result of applying a transformation on Magi's codual numbers, turning it into an instance of bottom-up dynamic programming. This makes the time complexity legible.

• Such an automatic system seems to be unlikely to exist... Specific toy problems (like the above) can of course be handled. Automatic memoization (which is already enough to reduce the running time to polynomial) are supported by some languages like lisp and scheme. A nice description of this is in en.wikipedia.org/wiki/… . A more "standard" description how to do this transformation is in Jeff Erickson algorithms book. Maybe ask in compilation/programming-languages forum about this... Jan 13 at 22:29
• There are many posts on eliminating recursion, such as: softwareengineering.stackexchange.com/questions/347582/… Jan 14 at 2:20
• Related are compiler optimizations like "fusion" (stackoverflow.com/questions/38905369/…), and supercompilation (stackoverflow.com/questions/9067545/what-is-supercompilation), although those are pretty advanced compared to basic recursion elimination. I mention them because it's not always possible to eliminate recursion without adding allocation, but this can sometimes be done via fusion/supercompilation passes. Jan 14 at 2:25
• @SarielHar-Peled I've already mentioned memoisation in my question. In Python, it's even easier than you said: docs.python.org/3/library/functools.html - This results in a "top-down" style of dynamic programming instead of a (loosely defined) "bottom-up" style. I think the tape-based approach to autodiff is an instance of bottom-up style, and so memoisation alone is not enough.
Jan 14 at 8:33