Skip to content

Speed up import time #18

@AdamGleave

Description

@AdamGleave

It currently takes around 2 seconds to load the pirl module. Most of the time this isn't a big deal, but due to a combination of:

  1. We restart each Ray worker after a single call to work around TensorFlow/CUDA brokenness;
  2. A complete experiment can consist of 100s of calls;
    this adds up to a significant overhead when the results are mostly cached. (The overhead is still there for non-cached calls but is negligible compared to the cost of e.g. running an RL/IRL algorithm.)

I'm not sure how much we can realistically shave off. Just importing Ray takes ~0.5s, and we can't avoid that. Importing TensorFlow seems to be a big culprit; we actually can delay that, but we'd need to make imports in pirl.agents and pirl.irl be on-demand. (Currently they all get loaded by the config.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions