I ended up getting a hint on how to accomplish this by reading the leo semantics post from Jeffrey Kegler's deprecated Marpa algorithm page. https://metacpan.org/pod/release/JKEGL/Marpa-PP-0.005_006/pod/Advanced/Algorithm.pod#Leo-Semantics:-Indirect-and-Lazy
Snippit that really helped:
Leo Semantics: Indirect and Lazy
Leo's hints about semantic processing, while brief, were insightful.
The first decision to make with the Leo semantics was direct versus
indirect. The direct approach does the semantic processing with the
Leo items themselves, without converting them. This has the advantage
that costs are incurred only for the Leo items that are actually used
by the semantics. It has the very serious disadvantage of intertwining
the Leo logic with the semantics. Dealing directly with Leo items
would more than double the complexity of the logic for Marpa's
semantics. Because of this, Marpa rejected the direct approach.
Leo 1991 suggests an indirect approach. The indirect approach is to
expand the Leo completions into the stacks of Earley items for which
Leo completions are a shorthand. However, in a naive implementation,
the indirect approach eliminates every advantage of using the Leo
speedups -- it merely moves processing from the recognizer into the
Marpa optimizes by using a lazy implementation of the indirect
approach. Only those Leo completions useful for the semantics are
For a lazy and indirect implementation, in all cases, time complexity
is the same as with a direct implementation. But for some ambiguous
grammars, the space required grows from linear to quadratic with any
indirect approach. This does not change the worst case time-complexity
-- that was already quadratic. Fortunately, this worst case -- highly ambiguous parses where all the parse results are demanded -- is of
limited practical interest.
I ended up using the lazy approach as well and did just in time evaluation for only items needed to complete the parse tree. This avoids the intermediate items that the normal algorithm adds.