HistoryEncoder may not be persisting rolling history

[DavidLXu]

I noticed a possible issue in HistoryEncoder.put_history() that may prevent the module from actually accumulating multi-step history across forward passes.

In dywa/src/models/modules.py, the method does this:

buffer = getattr(self, 'history')
buffer = th.concat([buffer[:, 1:], tensor.detach().unsqueeze(1)], dim=1)
return buffer

This creates a new local tensor and returns it, but it does not appear to write the updated tensor back to self.history.

As a result, even with history_len=5, the model may not be accumulating history as intended. Each forward pass may effectively see:

[0, 0, 0, 0, current_obs]

instead of:

[obs_{t-4}, obs_{t-3}, obs_{t-2}, obs_{t-1}, obs_t]

If so, the dynamics adaptation through historical interaction may be weaker than intended. The FiLM conditioning path is present architecturally, but it may be conditioned mostly on the current frame plus zero padding rather than on a real rolling interaction history.

A possible fix would be to update the registered buffer in-place, e.g.:

buffer = self.history
buffer[:, :-1].copy_(buffer[:, 1:].clone())
buffer[:, -1].copy_(tensor.detach())
return buffer

The currently commented-out lines below this code seem to suggest that this may have been the intended behavior.

It would be great to confirm whether this is intentional or whether self.history should be updated in-place here.

[SteveOUO]

I noticed a possible issue in HistoryEncoder.put_history() that may prevent the module from actually accumulating multi-step history across forward passes.

In dywa/src/models/modules.py, the method does this:

buffer = getattr(self, 'history')
buffer = th.concat([buffer[:, 1:], tensor.detach().unsqueeze(1)], dim=1)
return buffer
This creates a new local tensor and returns it, but it does not appear to write the updated tensor back to self.history.

As a result, even with history_len=5, the model may not be accumulating history as intended. Each forward pass may effectively see:

[0, 0, 0, 0, current_obs]

instead of:

[obs_{t-4}, obs_{t-3}, obs_{t-2}, obs_{t-1}, obs_t]

If so, the dynamics adaptation through historical interaction may be weaker than intended. The FiLM conditioning path is present architecturally, but it may be conditioned mostly on the current frame plus zero padding rather than on a real rolling interaction history.

A possible fix would be to update the registered buffer in-place, e.g.:

buffer = self.history
buffer[:, :-1].copy_(buffer[:, 1:].clone())
buffer[:, -1].copy_(tensor.detach())
return buffer
The currently commented-out lines below this code seem to suggest that this may have been the intended behavior.

It would be great to confirm whether this is intentional or whether self.history should be updated in-place here.

Thanks for pointing this out, @DavidLXu. You are right — this was not intentional.

The rolling history buffer should be updated in-place, but we accidentally uploaded a version where put_history() returns a newly concatenated tensor without persisting it back to self.history. This means the history was not accumulated across forward passes as intended.

We have fixed this issue by updating self.history in-place so that the buffer correctly keeps the rolling history.

Thanks again for catching this!