rewards always zero

[nuomizai]

Hi, @kaymen99 , thanks for your code. Is there anything wrong with the her_augmentation function under HER.py file where the re-computed reward is always zero?

reward = agent.env.compute_reward(future_achgoal, future_achgoal, 1.0)

And why should we take the future observation as the augmented observation, shouldn't we keep the observation in the current timestep, i.e., obs, _, _ = obs_array[index].values() as the augmented observation?