Skip to content

Commit 8563609

Browse files
committed
Update notes in ScaledStdConv impl
1 parent 678ba4e commit 8563609

File tree

1 file changed

+4
-3
lines changed

1 file changed

+4
-3
lines changed

timm/models/layers/std_conv.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,8 @@ class ScaledStdConv2d(nn.Conv2d):
7070
7171
Paper: `Characterizing signal propagation to close the performance gap in unnormalized ResNets` -
7272
https://arxiv.org/abs/2101.08692
73+
74+
NOTE: the operations used in this impl differ slightly from the DeepMind Haiku impl. The impact is minor.
7375
"""
7476

7577
def __init__(
@@ -100,11 +102,10 @@ def forward(self, x):
100102
class ScaledStdConv2dSame(nn.Conv2d):
101103
"""Conv2d layer with Scaled Weight Standardization and Tensorflow-like SAME padding support
102104
103-
NOTE: operations and default eps slightly changed from non-SAME impl to closer match Deepmind Haiku impl.
104-
Fore the sake of completeness, numeric differences are minor with arprox .005 top-1 difference.
105-
106105
Paper: `Characterizing signal propagation to close the performance gap in unnormalized ResNets` -
107106
https://arxiv.org/abs/2101.08692
107+
108+
NOTE: the operations used in this impl differ slightly from the DeepMind Haiku impl. The impact is minor.
108109
"""
109110

110111
def __init__(

0 commit comments

Comments
 (0)