Conversation
|
@kit1980 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
Thanks! Looks like torchfix is a very useful tool! Note that the upstream code also has this bug: https://github.com/codertimo/BERT-pytorch/blob/master/bert_pytorch/model/embedding/position.py#L13 Can you help submit a PR to the upstream code, and reference it in this PR? Thanks! |
|
I'll submit upstream. |
Fix require_grad typos (should be requires_grad). Before the fix, the code doesn't cause any errors but doesn't do what it's supposed to do. Also see pytorch/benchmark#1771
|
@kit1980 Thanks! Can you please also add the upstream PR link (codertimo/BERT-pytorch#104) in this PR? |
| # Compute the positional encodings once in log space. | ||
| pe = torch.zeros(max_len, d_model).float() | ||
| pe.require_grad = False | ||
| pe.requires_grad = False |
There was a problem hiding this comment.
Could you please add https://github.com/codertimo/BERT-pytorch/pull/104 here so that we can track the upstream code change?
|
@kit1980 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
3 participants
Fix require_grad typos (should be requires_grad).
Before the fix, the code doesn't cause any errors but doesn't do what it's supposed to do.
Fixed with TorchFix https://github.com/pytorch/test-infra/tree/main/tools/torchfix
Upstream PR: codertimo/BERT-pytorch#104