MaxEnt IRL Run-time optimization

[AdamGleave]

It's pretty slow right now, I suspect the value iteration is slow. Good to do some profiling to pin it down. Possible that moving things to GPU would speed things up (although I suspect environments we're using so far are small enough it's not worth the overhead).

[rohinmshah]

I have an implementation of value iteration in Numpy that's about 50-100x faster than my Python implementation in FastOptimalAgent here. (But my Python implementation is likely a lot slower than yours, because it is very generic, so you'll probably see something like 20x speedup.)

[AdamGleave]

Thanks for the link! I'm confused how that implementation works: it doesn't seem to look at transition probabilities. What assumptions is it making? (A deterministic gridworld, perhaps?)

[rohinmshah]

Oh, yes, it's assuming a deterministic MDP. I forgot that your gridworlds are slippery. That said, you should only need to change the lines that add discounted_values to the qvalues to include all possible transitions -- it should be a simple change, if you're willing to hardcore the transition function (and the actions are already hardcoded so why not the transitions too).

…

On Tue, Mar 27, 2018, 11:28 AM Adam Gleave ***@***.***> wrote: Thanks for the link! I'm confused how that implementation works: it doesn't seem to look at transition probabilities. What assumptions is it making? (A deterministic gridworld, perhaps?) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#10 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABOMU8sUW-PVfQTCc4xu0PW2fa59d2Ayks5tioTKgaJpZM4S8NPJ> .