Cannot repeat the results of Mask2Former+DiNAT-Large on ADE20K

Thank you for your interesting work! I am attempting to replicate the results of Mask2Former+DiNAT-Large on ADE20K. I have not made any changes to the settings, but the current training results show a relatively large gap compared to the reported results. I suspect that the pretrained weights have not been processed correctly.
I downloaded the weights from https://shi-labs.com/projects/dinat/checkpoints/imagenet22k/dinat_large_in22k_224_11x11interp.pth and used the following script for pretrained weight conversion:
import pickle as pkl
import sys

import torch

if __name__ == "__main__":
    input = sys.argv[1]

    obj = torch.load(input, map_location="cpu")
    if "model" in obj.keys():
        obj = obj["model"]
    if "state_dict" in obj.keys():
        obj = obj["state_dict"]

    res = {"model": obj, "__author__": "third_party", "matching_heuristics": True}

    with open(sys.argv[2], "wb") as f:
        # https://discuss.pytorch.org/t/runtime-error-when-loading-pytorch-model-from-pkl/38330
        # pkl.dump(res, f)
        torch.save(res, f)

Is there anything wrong with this approach? I am looking forward to hearing from you soon.

Here are more details:
`[10/06 17:01:42] detectron2 INFO: Full config saved to /data4/chenlinwei/code/ResolutionDet/detectron2exp/ADE20K_mask2former_NAT/NAT_160k/config.yaml
[10/06 17:01:42] d2.utils.env INFO: Using a generated random seed 42688149
[10/06 17:01:49] d2.engine.defaults INFO: Model:
MaskFormer(
(backbone): D2DiNAT(
(patch_embed): ConvTokenizer(
(proj): Sequential(
(0): Conv2d(3, 96, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
(1): Conv2d(96, 192, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
)
(norm): LayerNorm((192,), eps=1e-05, elementwise_affine=True)
)
(levels): ModuleList(
(0): BasicLayer(
(blocks): ModuleList(
(0): NATBlock(
(norm1): LayerNorm((192,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=6, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=192, out_features=576, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=192, out_features=192, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): Identity()
(norm2): LayerNorm((192,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=192, out_features=384, bias=True)
(act): GELU()
(fc2): Linear(in_features=384, out_features=192, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(1): NATBlock(
(norm1): LayerNorm((192,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=6, kernel_size=11, dilation=14, rel_pos_bias=True
(qkv): Linear(in_features=192, out_features=576, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=192, out_features=192, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.010)
(norm2): LayerNorm((192,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=192, out_features=384, bias=True)
(act): GELU()
(fc2): Linear(in_features=384, out_features=192, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(2): NATBlock(
(norm1): LayerNorm((192,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=6, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=192, out_features=576, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=192, out_features=192, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.021)
(norm2): LayerNorm((192,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=192, out_features=384, bias=True)
(act): GELU()
(fc2): Linear(in_features=384, out_features=192, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
)
(downsample): ConvDownsampler(
(reduction): Conv2d(192, 384, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(norm): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
)
)
(1): BasicLayer(
(blocks): ModuleList(
(0): NATBlock(
(norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=12, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=384, out_features=1152, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=384, out_features=384, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.031)
(norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=384, out_features=768, bias=True)
(act): GELU()
(fc2): Linear(in_features=768, out_features=384, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(1): NATBlock(
(norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=12, kernel_size=11, dilation=4, rel_pos_bias=True
(qkv): Linear(in_features=384, out_features=1152, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=384, out_features=384, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.041)
(norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=384, out_features=768, bias=True)
(act): GELU()
(fc2): Linear(in_features=768, out_features=384, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(2): NATBlock(
(norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=12, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=384, out_features=1152, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=384, out_features=384, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.052)
(norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=384, out_features=768, bias=True)
(act): GELU()
(fc2): Linear(in_features=768, out_features=384, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(3): NATBlock(
(norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=12, kernel_size=11, dilation=7, rel_pos_bias=True
(qkv): Linear(in_features=384, out_features=1152, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=384, out_features=384, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.062)
(norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=384, out_features=768, bias=True)
(act): GELU()
(fc2): Linear(in_features=768, out_features=384, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
)
(downsample): ConvDownsampler(
(reduction): Conv2d(384, 768, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
)
)
(2): BasicLayer(
(blocks): ModuleList(
(0): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.072)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(1): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=2, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.083)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(2): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.093)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(3): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=3, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.103)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(4): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.114)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(5): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=2, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.124)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(6): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.134)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(7): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=3, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.145)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(8): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.155)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(9): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=2, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.166)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(10): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.176)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(11): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=3, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.186)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(12): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.197)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(13): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=2, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.207)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(14): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.217)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(15): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=3, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.228)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(16): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.238)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(17): NATBlock(
(norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=24, kernel_size=11, dilation=3, rel_pos_bias=True
(qkv): Linear(in_features=768, out_features=2304, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=768, out_features=768, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.248)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=768, out_features=1536, bias=True)
(act): GELU()
(fc2): Linear(in_features=1536, out_features=768, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
)
(downsample): ConvDownsampler(
(reduction): Conv2d(768, 1536, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(norm): LayerNorm((1536,), eps=1e-05, elementwise_affine=True)
)
)
(3): BasicLayer(
(blocks): ModuleList(
(0): NATBlock(
(norm1): LayerNorm((1536,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=48, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=1536, out_features=4608, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=1536, out_features=1536, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.259)
(norm2): LayerNorm((1536,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=1536, out_features=3072, bias=True)
(act): GELU()
(fc2): Linear(in_features=3072, out_features=1536, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(1): NATBlock(
(norm1): LayerNorm((1536,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=48, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=1536, out_features=4608, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=1536, out_features=1536, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.269)
(norm2): LayerNorm((1536,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=1536, out_features=3072, bias=True)
(act): GELU()
(fc2): Linear(in_features=3072, out_features=1536, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(2): NATBlock(
(norm1): LayerNorm((1536,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=48, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=1536, out_features=4608, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=1536, out_features=1536, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.279)
(norm2): LayerNorm((1536,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=1536, out_features=3072, bias=True)
(act): GELU()
(fc2): Linear(in_features=3072, out_features=1536, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(3): NATBlock(
(norm1): LayerNorm((1536,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=48, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=1536, out_features=4608, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=1536, out_features=1536, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.290)
(norm2): LayerNorm((1536,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=1536, out_features=3072, bias=True)
(act): GELU()
(fc2): Linear(in_features=3072, out_features=1536, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
(4): NATBlock(
(norm1): LayerNorm((1536,), eps=1e-05, elementwise_affine=True)
(attn): NeighborhoodAttention2D(
head_dim=32, num_heads=48, kernel_size=11, dilation=1, rel_pos_bias=True
(qkv): Linear(in_features=1536, out_features=4608, bias=True)
(attn_drop): Dropout(p=0.0, inplace=False)
(proj): Linear(in_features=1536, out_features=1536, bias=True)
(proj_drop): Dropout(p=0.0, inplace=False)
)
(drop_path): DropPath(drop_prob=0.300)
(norm2): LayerNorm((1536,), eps=1e-05, elementwise_affine=True)
(mlp): Mlp(
(fc1): Linear(in_features=1536, out_features=3072, bias=True)
(act): GELU()
(fc2): Linear(in_features=3072, out_features=1536, bias=True)
(drop): Dropout(p=0.0, inplace=False)
)
)
)
)
)
(norm0): LayerNorm((192,), eps=1e-05, elementwise_affine=True)
(norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
(norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
(norm3): LayerNorm((1536,), eps=1e-05, elementwise_affine=True)
)
(sem_seg_head): MaskFormerHead(
(pixel_decoder): MSDeformAttnPixelDecoder(
(input_proj): ModuleList(
(0): Sequential(
(0): Conv2d(1536, 256, kernel_size=(1, 1), stride=(1, 1))
(1): GroupNorm(32, 256, eps=1e-05, affine=True)
)
(1): Sequential(
(0): Conv2d(768, 256, kernel_size=(1, 1), stride=(1, 1))
(1): GroupNorm(32, 256, eps=1e-05, affine=True)
)
(2): Sequential(
(0): Conv2d(384, 256, kernel_size=(1, 1), stride=(1, 1))
(1): GroupNorm(32, 256, eps=1e-05, affine=True)
)
)
(transformer): MSDeformAttnTransformerEncoderOnly(
(encoder): MSDeformAttnTransformerEncoder(
(layers): ModuleList(
(0): MSDeformAttnTransformerEncoderLayer(
(self_attn): MSDeformAttn(
(sampling_offsets): Linear(in_features=256, out_features=192, bias=True)
(attention_weights): Linear(in_features=256, out_features=96, bias=True)
(value_proj): Linear(in_features=256, out_features=256, bias=True)
(output_proj): Linear(in_features=256, out_features=256, bias=True)
)
(dropout1): Dropout(p=0.0, inplace=False)
(norm1): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(linear1): Linear(in_features=256, out_features=1024, bias=True)
(dropout2): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=1024, out_features=256, bias=True)
(dropout3): Dropout(p=0.0, inplace=False)
(norm2): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
)
(1): MSDeformAttnTransformerEncoderLayer(
(self_attn): MSDeformAttn(
(sampling_offsets): Linear(in_features=256, out_features=192, bias=True)
(attention_weights): Linear(in_features=256, out_features=96, bias=True)
(value_proj): Linear(in_features=256, out_features=256, bias=True)
(output_proj): Linear(in_features=256, out_features=256, bias=True)
)
(dropout1): Dropout(p=0.0, inplace=False)
(norm1): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(linear1): Linear(in_features=256, out_features=1024, bias=True)
(dropout2): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=1024, out_features=256, bias=True)
(dropout3): Dropout(p=0.0, inplace=False)
(norm2): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
)
(2): MSDeformAttnTransformerEncoderLayer(
(self_attn): MSDeformAttn(
(sampling_offsets): Linear(in_features=256, out_features=192, bias=True)
(attention_weights): Linear(in_features=256, out_features=96, bias=True)
(value_proj): Linear(in_features=256, out_features=256, bias=True)
(output_proj): Linear(in_features=256, out_features=256, bias=True)
)
(dropout1): Dropout(p=0.0, inplace=False)
(norm1): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(linear1): Linear(in_features=256, out_features=1024, bias=True)
(dropout2): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=1024, out_features=256, bias=True)
(dropout3): Dropout(p=0.0, inplace=False)
(norm2): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
)
(3): MSDeformAttnTransformerEncoderLayer(
(self_attn): MSDeformAttn(
(sampling_offsets): Linear(in_features=256, out_features=192, bias=True)
(attention_weights): Linear(in_features=256, out_features=96, bias=True)
(value_proj): Linear(in_features=256, out_features=256, bias=True)
(output_proj): Linear(in_features=256, out_features=256, bias=True)
)
(dropout1): Dropout(p=0.0, inplace=False)
(norm1): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(linear1): Linear(in_features=256, out_features=1024, bias=True)
(dropout2): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=1024, out_features=256, bias=True)
(dropout3): Dropout(p=0.0, inplace=False)
(norm2): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
)
(4): MSDeformAttnTransformerEncoderLayer(
(self_attn): MSDeformAttn(
(sampling_offsets): Linear(in_features=256, out_features=192, bias=True)
(attention_weights): Linear(in_features=256, out_features=96, bias=True)
(value_proj): Linear(in_features=256, out_features=256, bias=True)
(output_proj): Linear(in_features=256, out_features=256, bias=True)
)
(dropout1): Dropout(p=0.0, inplace=False)
(norm1): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(linear1): Linear(in_features=256, out_features=1024, bias=True)
(dropout2): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=1024, out_features=256, bias=True)
(dropout3): Dropout(p=0.0, inplace=False)
(norm2): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
)
(5): MSDeformAttnTransformerEncoderLayer(
(self_attn): MSDeformAttn(
(sampling_offsets): Linear(in_features=256, out_features=192, bias=True)
(attention_weights): Linear(in_features=256, out_features=96, bias=True)
(value_proj): Linear(in_features=256, out_features=256, bias=True)
(output_proj): Linear(in_features=256, out_features=256, bias=True)
)
(dropout1): Dropout(p=0.0, inplace=False)
(norm1): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(linear1): Linear(in_features=256, out_features=1024, bias=True)
(dropout2): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=1024, out_features=256, bias=True)
(dropout3): Dropout(p=0.0, inplace=False)
(norm2): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
)
)
)
)
(pe_layer): Positional encoding PositionEmbeddingSine
num_pos_feats: 128
temperature: 10000
normalize: True
scale: 6.283185307179586
(mask_features): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1))
(adapter_1): Conv2d(
192, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
(norm): GroupNorm(32, 256, eps=1e-05, affine=True)
)
(layer_1): Conv2d(
256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
(norm): GroupNorm(32, 256, eps=1e-05, affine=True)
)
)
(predictor): MultiScaleMaskedTransformerDecoder(
(pe_layer): Positional encoding PositionEmbeddingSine
num_pos_feats: 128
temperature: 10000
normalize: True
scale: 6.283185307179586
(transformer_self_attention_layers): ModuleList(
(0): SelfAttentionLayer(
(self_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(1): SelfAttentionLayer(
(self_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(2): SelfAttentionLayer(
(self_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(3): SelfAttentionLayer(
(self_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(4): SelfAttentionLayer(
(self_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(5): SelfAttentionLayer(
(self_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(6): SelfAttentionLayer(
(self_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(7): SelfAttentionLayer(
(self_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(8): SelfAttentionLayer(
(self_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
)
(transformer_cross_attention_layers): ModuleList(
(0): CrossAttentionLayer(
(multihead_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(1): CrossAttentionLayer(
(multihead_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(2): CrossAttentionLayer(
(multihead_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(3): CrossAttentionLayer(
(multihead_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(4): CrossAttentionLayer(
(multihead_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(5): CrossAttentionLayer(
(multihead_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(6): CrossAttentionLayer(
(multihead_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(7): CrossAttentionLayer(
(multihead_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
(8): CrossAttentionLayer(
(multihead_attn): MultiheadAttention(
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(dropout): Dropout(p=0.0, inplace=False)
)
)
(transformer_ffn_layers): ModuleList(
(0): FFNLayer(
(linear1): Linear(in_features=256, out_features=2048, bias=True)
(dropout): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=2048, out_features=256, bias=True)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
)
(1): FFNLayer(
(linear1): Linear(in_features=256, out_features=2048, bias=True)
(dropout): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=2048, out_features=256, bias=True)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
)
(2): FFNLayer(
(linear1): Linear(in_features=256, out_features=2048, bias=True)
(dropout): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=2048, out_features=256, bias=True)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
)
(3): FFNLayer(
(linear1): Linear(in_features=256, out_features=2048, bias=True)
(dropout): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=2048, out_features=256, bias=True)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
)
(4): FFNLayer(
(linear1): Linear(in_features=256, out_features=2048, bias=True)
(dropout): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=2048, out_features=256, bias=True)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
)
(5): FFNLayer(
(linear1): Linear(in_features=256, out_features=2048, bias=True)
(dropout): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=2048, out_features=256, bias=True)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
)
(6): FFNLayer(
(linear1): Linear(in_features=256, out_features=2048, bias=True)
(dropout): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=2048, out_features=256, bias=True)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
)
(7): FFNLayer(
(linear1): Linear(in_features=256, out_features=2048, bias=True)
(dropout): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=2048, out_features=256, bias=True)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
)
(8): FFNLayer(
(linear1): Linear(in_features=256, out_features=2048, bias=True)
(dropout): Dropout(p=0.0, inplace=False)
(linear2): Linear(in_features=2048, out_features=256, bias=True)
(norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
)
)
(decoder_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
(query_feat): Embedding(100, 256)
(query_embed): Embedding(100, 256)
(level_embed): Embedding(3, 256)
(input_proj): ModuleList(
(0): Sequential()
(1): Sequential()
(2): Sequential()
)
(class_embed): Linear(in_features=256, out_features=151, bias=True)
(mask_embed): MLP(
(layers): ModuleList(
(0): Linear(in_features=256, out_features=256, bias=True)
(1): Linear(in_features=256, out_features=256, bias=True)
(2): Linear(in_features=256, out_features=256, bias=True)
)
)
)
)
(criterion): Criterion SetCriterion
matcher: Matcher HungarianMatcher
cost_class: 2.0
cost_mask: 5.0
cost_dice: 5.0
losses: ['labels', 'masks']
weight_dict: {'loss_ce': 2.0, 'loss_mask': 5.0, 'loss_dice': 5.0, 'loss_ce_0': 2.0, 'loss_mask_0': 5.0, 'loss_dice_0': 5.0, 'loss_ce_1': 2.0, 'loss_mask_1': 5.0, 'loss_dice_1': 5.0, 'loss_ce_2': 2.0, 'loss_mask_2': 5.0, 'loss_dice_2': 5.0, 'loss_ce_3': 2.0, 'loss_mask_3': 5.0, 'loss_dice_3': 5.0, 'loss_ce_4': 2.0, 'loss_mask_4': 5.0, 'loss_dice_4': 5.0, 'loss_ce_5': 2.0, 'loss_mask_5': 5.0, 'loss_dice_5': 5.0, 'loss_ce_6': 2.0, 'loss_mask_6': 5.0, 'loss_dice_6': 5.0, 'loss_ce_7': 2.0, 'loss_mask_7': 5.0, 'loss_dice_7': 5.0, 'loss_ce_8': 2.0, 'loss_mask_8': 5.0, 'loss_dice_8': 5.0}
num_classes: 150
eos_coef: 0.1
num_points: 12544
oversample_ratio: 3.0
importance_sample_ratio: 0.75
)
[10/06 17:01:50] mask2former.data.dataset_mappers.mask_former_semantic_dataset_mapper INFO: [MaskFormerSemanticDatasetMapper] Augmentations used in training: [ResizeShortestEdge(short_edge_length=..., max_size=2560, sample_style='choice'), RandomCrop_CategoryAreaConstraint(crop_type='absolute', crop_size=[640, 640], single_category_max_area=1.0, ignored_category=255), <detectron2.projects.point_rend.color_augmentation.ColorAugSSDTransform object at 0x7fe32ad50850>, RandomFlip()]
[10/06 17:01:51] d2.data.datasets.coco INFO: Loaded 20210 images with semantic segmentation from /data3/chenlinwei/dataset/ade/ADEChallengeData2016/images/training
[10/06 17:01:52] d2.data.build INFO: Using training sampler TrainingSampler
[10/06 17:01:52] d2.data.common INFO: Serializing 20210 elements to byte tensors and concatenating them all ...
[10/06 17:01:52] d2.data.common INFO: Serialized dataset takes 4.78 MiB
[10/06 17:01:53] fvcore.common.checkpoint INFO: [Checkpointer] Loading from /data3/chenlinwei/code/ResolutionDet/Neighborhood-Attention-Transformer/dinat_large_in22k_224_11x11interp_backbone.pth ...
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.pixel_decoder.adapter_1.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.pixel_decoder.adapter_1.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.pixel_decoder.layer_1.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.pixel_decoder.layer_1.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.0.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.0.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.1.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.1.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.2.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.2.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.3.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.3.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.4.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.4.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.5.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.5.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.6.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.6.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.7.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.7.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.8.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_cross_attention_layers.8.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.0.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.0.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.1.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.1.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.2.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.2.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.3.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.3.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.4.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.4.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.5.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.5.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.6.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.6.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.7.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.7.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.8.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_ffn_layers.8.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.0.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.0.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.1.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.1.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.2.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.2.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.3.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.3.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.4.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.4.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.5.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.5.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.6.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.6.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.7.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.7.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.bias in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.8.norm.bias in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.bias will not be loaded. Please double check and see if this is desired.
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: Shape of norm.weight in checkpoint is torch.Size([1536]), while shape of sem_seg_head.predictor.transformer_self_attention_layers.8.norm.weight in model is torch.Size([256]).
[10/06 17:01:54] d2.checkpoint.c2_model_loading WARNING: norm.weight will not be loaded.
To be clear, are you trying to train the model on ADE20k, or are you running inference?
P.S. could you kindly remove your last message, and send logs, particularly ones this long as files so they're easily searchable and don't make it difficult to interact in the issue?

I am trying to train the model on ADE20K. Perhaps I used the wrong pretraining weights. Could you tell me which one is the correct Img22K weight for Mask2Former?
I downloaded the weights from , which cannot directly loaded for Mask2Former training, because it's prefix has no 'backbone.'.

I see, you're right; Mask2Former requires converting the standard pth pickle file into one compatible with detectron2, and we provide that under mask2former/tools, which I think is what you're using.
However, I'd note that following Swin, we used the checkpoint that was fine-tuned on ImageNet at 384x384 (with the extended kernel size), and not the plain pre-trained checkpoint with lerped RPB.
You can find that checkpoint (in the DiNAT table under classification)[https://github.com/SHI-Labs/Neighborhood-Attention-Transformer/blob/main/classification/DiNAT.md], or directly through (this link)[https://shi-labs.com/projects/dinat/checkpoints/imagenet1k/dinat_large_in22k_in1k_384_11x11.pth].

Thanks a lot! Just one more detail: The paper (Table 15) states that all backbones were pre-trained on ImageNet-22K. However, the checkpoint seems to be pre-trained on ImageNet-22K and fine-tuned on ImageNet-1k. Is this the correct weight to achieve 58.1 mIoU on ADE20K?

Yes it is.

I have cited your work in may new paper, thank you for your excellent work~

Thank you; good luck with your paper!

Hello! After replacing the pretrained weights as per your suggestion, the training results still do not seem to be good enough. It achieved ~54.0% accuracy at iteration 75,000, but it appears to be very difficult to reach 57.1% even after 160,000 iterations, according to my experiences with training Mask2Former.

The frozen stages are set to -1, but config.yaml shows FREEZE_AT = 1. Is this correct? There is a discussion in  stating that the frozen stages should be 5. I am confused.
Could you provide the training log.txt? It could be very helpful.
BTW, I only changed the code to use torch.checkpoint to save memory, but it should not affect the results.

class BasicLayer(nn.Module):
    ....
        self.use_checkpoint = use_checkpoint

    def forward(self, x):
        for blk in self.blocks:
            if self.use_checkpoint:
                x = checkpoint.checkpoint(blk, x)
            else:
                x = blk(x)
        if self.downsample is not None:
            x_down = self.downsample(x)
            return x, x_down
        else:
            return x, x

Would you mind sharing how many GPUs you're training with?
In our experience, ADE20K is overall very difficult to reproduce with even identical settings, as opposed to COCO and Cityscapes, and one key parameter to look out for is the number of GPUs, because it heavily affects batch statistics.
For your reference, I've attached a log from one of our last ADE20K runs, which finishes at 57.5 single scale mIoU, and evals at 58.2 multi scale mIoU.
dinat-l-ade20k-semseg.log
I'm not sure what config config you're referring to; freeze at is 0 in the base config and not overridden.

Thanks for your help! Now, I am training with 4 GPUs (4 imgs per GPU). Maybe I should increase it to 8?
Regarding frozen stages, the default value in this link is -1, and the default for FREEZE_AT is 0. I have verified that changing FREEZE_AT does not affect the frozen stages (I printed the self.frozen stages at this link). It seems like FREEZE_AT does not work as intended. But I think frozen_stages=1 is correct, am I right?

Closing this due to inactivity.


Add a tag to the cachedir

Since there are now two files to make when a cachedir is created, use the temporary-dir -> rename technique.
CacheDir tests no longer pre-create the cache directory, they should be verifying it's created properly upon request (one unit test still makes sure passing an existing empty directory works, too).
There are no externally visible changes (except the new tag file if someone goes and explicitly looks, but that's an "implementation detail", not part of the behavior).
Contributor Checklist:

[X] I have created a new test or updated the unit tests to cover the new/changed functionality.
[X] I have updated CHANGES.txt and RELEASE.txt (and read the README.rst).
[X] I have updated the appropriate documentation

Hmmm, I had made the updates to CHANGES and RELEASE, but it looks like they never got committed. I'll fix that when resolving the conflict.


Add styling to teaser pattern

Styled Teaser title

There are some problems with the teasers on the home page example: (http://localhost:8889/patterns/04-pages-homepage-homepage-en/04-pages-homepage-homepage-en.html):


Usage on Heroku

Hello! Thanks so much for writing this gem.
The examples mention localhost and following them I got this set up for my local dev environment quite easily. However, I'm not sure what to use a config for when I've deployed the app (we're using Heroku). I'd be happy to write up and make a PR for the README for how to do this if you could point me in the right direction.
Thanks!

I'd be interested in this as well.


Correct a typo in the parameter name

thank you for fixing it, we also need to update the readme: https://github.com/openstack-k8s-operators/ci-framework/blame/main/ci_framework/roles/rhol_crc/README.md#L15

@raukadah oh, right, thanks!

/lgtm


Dynamically changing val_mapping on existing parameters does not produce expected behaviour.

Dynamically changing val_mapping on existing parameters does not produce expected behaviour.
Steps to reproduce

create a parameter on an instrument instance (here named: 'vna') (or have an existing one)

vna.add_parameter(
    name="sweep_type",
    get_cmd='SWPT?',
    set_cmd='SWPT {}',
    val_mapping={
        "Linear frequency": "LINF",
        "Power": "POWE"
    }
)

update val_mapping

vna.sweep_type.val_mapping = {'Linear frequency': 'LINF', 'Logarithmic frequency': 'LOGF'}

set parameter to a: removed or b: added value or c: get value from instrument (instrument replying with: 'LOGF')
a: vna.sweep_type('Power')
b: vna.sweep_type('Logarithmic fequency')
c: vna.sweep_type()

Expected behaviour
a: ValueError: ("'Power' is not in {'Logarithmic frequency', 'Linear frequency'}; ...)
b: parameter set to: 'Logarithmic frequency' without error
c: return 'Logarithmic frequency'
Actual behaviour
a: KeyError: ('Power', 'setting vna_sweep_type to Power')
b: ValueError: ("'Logarithmic fequency' is not in {'Power', 'Linear frequency'}; ...
c: KeyError: ("'LOGF' not in val_mapping", 'getting vna_sweep_type')
System
It would be helpful to provide such information:
Windows 10
If you are using a released version of qcodes (recommended):
0.34.1 (from ...\qcodes\_version.py)
Remarks
I'm trying to wirte an instrument driver and can change vals or unit without issue.
But maybe I am using the wrong approach, and Qcodes actully works as intended?

@DavidMerges Thanks for the report. I agree this is not great.
I don't think val_mappings were ever designed to be replaced but then at least we should prevent the user from doing it
if that is the case.
Specifically the issue is likely that setting val_mapping in __init__ will also update vals and inverse_valmapping
overwriting the val_mapping will do non of this. See the code here
I can see two options:

Make val_mapping a property that cannot be replaced but raises an error if you try to do this
Make it a property that allows the user to change it and updated other attributes.

To determine which solution is best I would like to better understand your use case for replacing the val_mapping. Can you say something about that? I have always expected the val_mapping to be static but there may be good reasons it should not be.

@jenshnielsen Thank you for your reply and your help.
As I stated, I'm trying to write an instrument driver.
I'm new to Qcodes, so maybe the approach I'm taking at the moment is not the best.
The Instrument is a "Network/Spectrum/Impedance Analyzer"
Depending on the mode I'm in the parameters of the instruments change.
For the sweep type you will have:

'Linear freqency', 'Log frequency', 'Frequency list' and 'Power' when in Network Analyzer mode
'Linear freqency', and 'Power' when in Spectrum Analyzer mode

It's a complex instruments, and there are quite a few interdepended parameters.
For example the sweep start parameter will change dependend on the Analyzer mode and the sweep type
After changing or requesting the current Analyzer mode I would like to update these paramters in the software-modell accordingly. So that when in Spectrum Analyzer mode you cannot choose 'Log frequency' as sweep type
Maybe it is better to delete the parameter instances and create new when the Analyzer mode changes? There are a few parameters that are not available dependend on the Analyzer mode so I might have to delete/create these ones any way.
I provided a few entries from the programming manual:

I thought more on this problem, and I'm still not sure if I'am maybe just using the wrong approach - maybe there is another way to structure interdependent parameters?
I would like to be able to take a snapshot (with update=True) and also allow the users to use the help() function - but a lot of the parameters are interdependent. (valid Ranges, Units, value maps etc. are subject to change)
When the 'Analyzer mode' attribute changes the 'start'(STAR) attribute will change - but it will also be dependent on the 'sweep_type' (SWPT) which in tun will also be dependent on the 'Analyzer mode'
Updating parameters (even delete/create them) accordingly seems the logical route for me, this is why i tried to change the val_mapping.

Sorry, Yes I think updating the val_mapping is reasonable. I would be happy to review / help with a pr that made val_mapping a property and added a setter such that the inverse_val_mapping and validator is updated when val_mapping is changed.
I think that will also need some logic to ensure that the user either updates val_mapping or validator but not both
For other instruments we have been able to split the functionality into multiple modules and toggle between the modules/channels such that one module has code for one mode and the other for another mode but I don't think that makes sense here.

Hello,
thank you for your help.
I'm still learning a lot about qcodes and I'm happy to submit a PR regarding this issue.
I (hopefully) finished the driver for our network analyzer today and plan to submit it to Qcodes_contrib_drivers.
I ended up with creating the parameter instances with val_mapping of all possible values and then changed parameter.vals according to instrument state.
The only time a val_mapping would need to be changed is when you have an instrument where you ended up having the duplicates of key or values in the val_mapping dict using this method. I'm not sure if this is likely to happen, so making val_mapping read only semms to be fine, too.
I'm happy to go either way with the implementation, what do you think?

That makes sense. If I understand it correctly you suggest

setting the val mapping with all possible values.
when changing mode replace the vals validator according to the mode but leave the val_mapping (and inverse_val_mapping) alone.

That sounds like a good way of doing it. We can always extend later if there is a compelling reason for over writiing the val_mapping


Optimize Privatelink DNS Endpoint Table

In order to fill the Azure policies with the appropriate group IDs the table should be optimized to determine which group id belongs to what.
It is currently unclear if a service lacks a group ID such as Azure SQL Managed Instance (Microsoft.Sql/managedInstances) where there is no group_id/sub resource type mentioned.
Also it is unclear if for example the and mentioned in Azure Migrate (Microsoft.Migrate) / migrate projects, assessment project and discovery site is part of the actual group id or is a separator.
So there is a lot of guessing involved when creating the policies and hope that they are correct and will work.

Service
Private link resource type
Subresource
Private DNS zone name
Public DNS zone forwarders

Azure SQL Managed Instance
Microsoft.Sql/managedInstances
null or none or N/A
privatelink.{dnsPrefix}.database.windows.net
{instanceName}.{dnsPrefix}.database.windows.net

or:

Service
Private link resource type
Subresource
Private DNS zone name
Public DNS zone forwarders

Azure Migrate
Microsoft.Migrate
migrate projects,  assessment project and discovery site
privatelink.prod.migration.windowsazure.com
prod.migration.windowsazure.com

Azure Migrate
Microsoft.Migrate
migrate projects,  assessment project and discovery site
privatelink.prod.migration.windowsazure.com
prod.migration.windowsazure.com

https://learn.microsoft.com/en-us/azure/private-link/private-endpoint-dns#azure-services-dns-zone-configuration
https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ready/azure-best-practices/private-link-and-dns-integration-at-scale
I am aware that the table will get wider with more information, another option would be to provide this information in machine readable format via a Json, to make it easier to consume it.
At least the usage of inline code to differentiate between text and actual group_ids would help.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: a88af182-32aa-a8f3-359d-b92ae1cfb8c7
Version Independent ID: 01ce07fc-4bc2-def5-9ecd-a80330ad9488
Content: Azure Private Endpoint DNS configuration
Content Source: articles/private-link/private-endpoint-dns.md
Service: private-link
GitHub Login: @asudbring
Microsoft Alias: allensu

@AndreasMWalter , thank you for your feedback. We have assigned this to the content owner/author for review.
@asudbring , for your review.

Also it is very confusing, when one needs two dns zones or just one: For example
This indicates that two DNS Zones are required

However only one zone is required, since one is a subzone of the other

Why this has not been done here is inconclusive, I assume that some subdomains of azurehealthcareapis.com may not be overridden and therefore only specific subdomains where chosen.

The whole thing feels a lot inconsequential.
And while I am at it: The table should warn of simply attaching the privatelink DNS zone for Azure Monitor https://learn.microsoft.com/en-us/azure/azure-monitor/logs/private-link-security
because as soon as someone simply attaches the DNS Zone privatelink.monitor.azure.com to their hub, it will break access to Azure Monitor.
(For that matter deploying privatelink.blob.core.windows.net also breaks access to the global agents' solution packs storage account if I remember correctly)

Assigning to PM for review.
#reassign:@ivapplyr

@AndreasMWalter
Thank you for the insightful feedback. Though I will need to sit down to see how we'll talk things moving forward with the relevant doc updates, here are some things to note.

Every resource (except Private Link Service) should have a Group ID and Membername. If that is missing, It is a gap in our documentation that we will need to close with that respective service.
'and' or spaces in general shouldn't be a part of the groupid. It should be one name (i.e. blob, table, queue, etc). Are you seeing instances where that is not the case?
You are right that the DNS Zone mappings can definitely be a bit cumbersome :p Each service has particular logic that makes it difficult to restrict to 1 Private DNS Zone for 1 public FQDN. The thought is that the Private DNS zone will override the public FQDN, so any label to the left of the 'Privatelink' would be any additional cname, but the "privatelink.serviceName.com" would be the "main" DNS zone that is configured.

@asudbring let's sync internally on how we can identify actionable improvements here.

Lemme just paste my YAML Code which I use for my Terraform Deployment:

- privateLinkServiceId: Microsoft.HDInsight
  privateDnsZoneName: privatelink.azurehdinsight.net
  privateEndpointGroupId: ""
  location: global
- privateLinkServiceId: Microsoft.Kusto
  privateDnsZoneName: privatelink.westeurope.kusto.windows.net
  privateEndpointGroupId: ""
  location: westeurope
- privateLinkServiceId: Microsoft.PowerBI/privateLinkServicesForPowerBI
  privateDnsZoneName: privatelink.analysis.windows.net
  privateEndpointGroupId: ""
  location: global
- privateLinkServiceId: Microsoft.PowerBI/privateLinkServicesForPowerBI
  privateDnsZoneName: privatelink.pbidedicated.windows.net
  privateEndpointGroupId: ""
  location: global
- privateLinkServiceId: Microsoft.PowerBI/privateLinkServicesForPowerBI
  privateDnsZoneName: privatelink.tip1.powerquery.microsoft.com
  privateEndpointGroupId: ""
  location: global

Naturally the list may have been updated since when I opened the issue in December, a reason why I would love  to be able to consume a machine readable file for this, so I don't need to scan the table in the docs and add in additional lines of code.

s.o. same applies, this is my yaml from early December issues in the docs may have been fixed... or not
- privateLinkServiceId: Microsoft.KeyVault/managedHSMs
  privateDnsZoneName: privatelink.managedhsm.azure.net
  privateEndpointGroupId: Managed HSMs #????
  location: global
- privateLinkServiceId: Microsoft.Migrate
  privateDnsZoneName: privatelink.prod.migration.windowsazure.com
  privateEndpointGroupId: migrate projects
  location: global
- privateLinkServiceId: Microsoft.Migrate
  privateDnsZoneName: privatelink.prod.migration.windowsazure.com
  privateEndpointGroupId: assessment project
  location: global
- privateLinkServiceId: Microsoft.Migrate
  privateDnsZoneName: privatelink.prod.migration.windowsazure.com
  privateEndpointGroupId: discovery site
  location: global

I understand that it is difficult due to the fact that the Services are handling it differently, that's why I am making the point that the documentation kinda needs to be spot on, if we want to be able to automate the stuff.
Which at least I want to be able...
No hard feelings :)

Thank you for you dedication to our documentation.
Unfortunately, we have been unable to review this issue in a timely manner. We sincerely apologize for the delayed response. We will create an internal tracking work item to address this issue. For now, we are closing this issue. If you have any additional information you would like to provide, please respond to this issue with any additional details.
Please continue to provide feedback about the documentation. We appreciate your contributions to our community.
#please-close


Make sure that a cert with extensions gets version number 2 (v3) [1.0.2]

Fixes #4419

Merged.  ed0245e08fdf374cd6351a1ae8117d7382115a21


Bump Mono.TextTemplating and xunit.runner.visualstudio

Pull Request Test Coverage Report for Build<PHONE_NUMBER>6
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 92.925%

Totals

Change from base Build<PHONE_NUMBER>9:
0.0%

Covered Lines:
3419

Relevant Lines:
3644

💛  - Coveralls


changeEditorInterface is not a valid property name for a content type.

When I run the following example migration
contentful-migration --space-id SPACEID 16-change-editor-interface.js
I get the this error
Errors in migration-cli-master/examples/16-change-editor-interface.js
Line 12: "changeEditorInterface" is not a valid property name for a content type.
10:     required: true
11:   });
12:   blogPost.changeEditorInterface('slug', 'slugEditor');
13: };

Hey @coffeepunk,
the new version of the Migration CLI hasn't been released yet, it's just merged to master 🙂
Did you use the codebase from this repository to run the example migration, or is contentful-migration the binary from NPM?
Best,
Stephan

Hi @zcei I used the binary from the NPM package. I just ran the global install command this morning to get the latest from NPM.

Okay then this explains it. The example will only work once the new version is released that includes the code for changeEditorInterface.
It will probably be released in the upcoming week.

Alright, I tried the master binary and then the example migration runs just fine. Looking forward the release.


chore: Don't query audio devices on linux

Fixes #2299

Didn't think it could be that easy, lol
Tested, works completely fine, no issues in dashboard as far as i can tell as well

I'm late on this, but looks fine. It doesn't try to rewrite the whole server-dashboard stack and just fixes the warnings.


[Enhancement] Simplify non-training installs

This removes the deepspeed install dependency and moves it to additional installation steps similar to flash attention and nina so that anyone needing just the inference steps can ignore the added dependencies needed by deepspeed.

Thank you very much for the contribution. We have moved these dependencies to pip install -e ".[training]" in the main repo in another commit.

Awesome!


Shrink images to reduce size of the homepage (from 1.51 MB to 468.63 kB)

Homepage size went from 1.51 MB to 468.63 kB (~70% lighter)
support.png from 317.6 KB to 72.7 KB
learn-about-monero.png 282.5 KB to 64.9 KB
create-wallet.png 236.9 KB to 60.0 KB
monero-accepted.png 210.8 KB to 59.4 KB
exchange.png 324.26 KB to 33.80 KB

Trimmed ~5% more space from each image after running zopflipng on them as suggested in #1096.
Results:
support.png from 72.7 KB to ~67 KB
learn-about-monero.png from 64.9 KB to ~60KB
create-wallet.png from 60.0 KB to ~56KB
monero-accepted.png from 59.4 KB to ~54KB
exchange.png from 33.80 KB to ~28KB


wrong droplet image id

packer build .
==> digitalocean.supabase: Error creating droplet: POST https://api.digitalocean.com/v2/droplets: 422  You specified an invalid image for Droplet creation.

https://github.com/digitalocean/supabase-on-do/blob/main/packer/supabase.pkr.hcl#L28
No "ubuntu-22-10-x64". maybe "ubuntu-23-10-x64"?

needs to be 22.04, docker-ce is not available on 23.10


Question re SPFx 1.12.1 and authentication (specifically, ADAL to MSAL)

What type of issue is this?
Question
What SharePoint development model, framework, SDK or API is this about?
💥 SharePoint Framework
Target SharePoint environment
SharePoint Online
What browser(s) / client(s) have you tested

[ ] 💥 Internet Explorer
[ ] 💥 Microsoft Edge
[ ] 💥 Google Chrome
[ ] 💥 FireFox
[ ] 💥 Safari
[ ] mobile (iOS/iPadOS)
[ ] mobile (Android)
[X] not applicable
[ ] other (enter in the "Additional environment details" area below)

Additional environment details
SPFx 1.12.1
node 14.17.0
Issue description
I have a question re SPFx and authentication. I am currently in the middle of upgrading an SPFx solution from 1.10.0 to 1.12.1. The primary reason for doing this SPFx upgrade is of course because it is overdue and good practice. The other is prompted by the upcoming retirement/deprecation of ADAL in favor of MSAL. I will be honest up-front and state that authentication is not my strongest point. Looking under the hood of the AADHttpClient (which we use) in our current 1.10.0 project I see references to it using ADAL but a future upgrade to MSAL. Is this something implemented in the sp-http package required by 1.12.1? Or am I misunderstanding the nature of this migration from ADAL to MSAL (at least insofar as it affects SPFx solutions in SPO) and is it something handled in the "plumbing" by SPO and so entirely independent of the SPFx version used?

@tcattermole - sorry for the really long delay.
Authentication part is implemented by the service. So we're in charge of what auth library is being used behind the scenes.
It does not directly depend on the version of SPFx you're using in the project.


add autodiff prototype

What changes were proposed in this pull request?
Adds automatic differentiation.
(Please fill in changes proposed in this fix)
How was this patch tested?
WIP: Needs some functions yet.
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
Please review
https://github.com/deeplearning4j/deeplearning4j/blob/master/CONTRIBUTING.md before opening a pull request.

Got rid of demo

@agibsonccc What's the connection of this request with this project https://github.com/uniker9/JAutoDiff ?

@andresmasegosa I adapted the core MIT licensed jautodiff to work with our ndarrays and severely modified it to add a graph structure. At this point it's a fork where I left the headers in there for the license.
Basically the jautodiff primitives are a hook for us to build an operation computation graph.

@andresmasegosa not quite sure..other things to work on, should have a draft out sometime in june.

Hey folks - for those watching this PR. Consider this VERY alpha. The basics are there but I intend on doing some follow ups on python testing among other things. We need access to the computation graph component among other things in this pull request right now.


Cloned your website

Hello skiedude. As you suggested on your website, I cloned your repo to have our own structure notifications. They work, yet I do have a minor issue. When I click the "Test" button for webhooks, I land on an "Whoops. Something went wrong" page. However, the actual notifcations come thru. Any suggestions?
I'd also like to know if you intend to take the project up again or if you would ok with me hosting it publicly.
You can contact me either here or ingame as "Jesco".
Cheers.

Whats in the storage/logs/*.log (date) file when you hit test?
I'd guess file permissions

laravel-2019-12-28.log
[2019-12-28 16:15:03] prod.DEBUG: Ending FuelCheck for Junsoraert - Deep Holes Drilling for Jesco_Sklor, not unanchoring
[2019-12-28 16:15:03] prod.DEBUG: Starting UnanchorCheck for Junsoraert - Rockmonster Pleasure Hub owned by Jesco_Sklor
[2019-12-28 16:15:03] prod.DEBUG: Ending FuelCheck for Junsoraert - Rockmonster Pleasure Hub for Jesco_Sklor, not unanchoring
[2019-12-28 16:15:03] prod.DEBUG: Starting UnanchorCheck for Scuelazyns - Greasy Pit owned by Jesco_Sklor
[2019-12-28 16:15:03] prod.DEBUG: Ending FuelCheck for Scuelazyns - Greasy Pit for Jesco_Sklor, not unanchoring
[2019-12-28 16:30:04] prod.DEBUG: Starting FuelCheck for J144956 - Utopia Planitia owned by Jesco_Sklor
[2019-12-28 16:30:04] prod.DEBUG: Ending FuelCheck for J144956 - Utopia Planitia for Jesco_Sklor
[2019-12-28 16:30:04] prod.DEBUG: Starting FuelCheck for J144956 - Ishimura owned by Jesco_Sklor
[2019-12-28 16:30:04] prod.DEBUG: Ending FuelCheck for J144956 - Ishimura for Jesco_Sklor
[2019-12-28 16:30:04] prod.DEBUG: Starting FuelCheck for Scuelazyns - Deep Holes Miners Bar & Cafe owned by Jesco_Sklor
[2019-12-28 16:30:04] prod.DEBUG: Ending FuelCheck for Scuelazyns - Deep Holes Miners Bar & Cafe for Jesco_Sklor
[2019-12-28 16:30:04] prod.DEBUG: Starting FuelCheck for Junsoraert - Lemmings Mining Emporium owned by Jesco_Sklor
[2019-12-28 16:30:04] prod.DEBUG: Ending FuelCheck for Junsoraert - Lemmings Mining Emporium for Jesco_Sklor
[2019-12-28 16:30:04] prod.DEBUG: Starting FuelCheck for J144956 - Fortuna owned by Jesco_Sklor
[2019-12-28 16:30:04] prod.DEBUG: Ending FuelCheck for J144956 - Fortuna for Jesco_Sklor
[2019-12-28 16:30:04] prod.DEBUG: Starting FuelCheck for Junsoraert - Deep Holes Drilling owned by Jesco_Sklor
[2019-12-28 16:30:04] prod.DEBUG: Ending FuelCheck for Junsoraert - Deep Holes Drilling for Jesco_Sklor
[2019-12-28 16:30:04] prod.DEBUG: Starting FuelCheck for Junsoraert - Rockmonster Pleasure Hub owned by Jesco_Sklor
[2019-12-28 16:30:04] prod.DEBUG: Ending FuelCheck for Junsoraert - Rockmonster Pleasure Hub for Jesco_Sklor
[2019-12-28 16:30:04] prod.DEBUG: Starting FuelCheck for Scuelazyns - Greasy Pit owned by Jesco_Sklor
[2019-12-28 16:30:04] prod.DEBUG: Ending FuelCheck for Scuelazyns - Greasy Pit for Jesco_Sklor
[2019-12-28 16:45:03] prod.DEBUG: Refreshed token for Jesco_Sklor
[2019-12-28 16:45:03] prod.DEBUG: Checking Extractions for Jesco_Sklor
[2019-12-28 17:00:04] prod.NOTICE: Token for Jesco_Sklor was not expired
[2019-12-28 17:00:04] prod.DEBUG: Starting Structure pull for character Jesco_Sklor
[2019-12-28 17:00:05] prod.DEBUG: Finished Structure pull for character Jesco_Sklor
[2019-12-28 17:15:02] prod.DEBUG: Starting UnanchorCheck for J144956 - Utopia Planitia owned by Jesco_Sklor
[2019-12-28 17:15:02] prod.DEBUG: Ending FuelCheck for J144956 - Utopia Planitia for Jesco_Sklor, not unanchoring
[2019-12-28 17:15:02] prod.DEBUG: Starting UnanchorCheck for J144956 - Ishimura owned by Jesco_Sklor
[2019-12-28 17:15:02] prod.DEBUG: Ending FuelCheck for J144956 - Ishimura for Jesco_Sklor, not unanchoring
[2019-12-28 17:15:02] prod.DEBUG: Starting UnanchorCheck for Scuelazyns - Deep Holes Miners Bar & Cafe owned by Jesco_Sklor
[2019-12-28 17:15:02] prod.DEBUG: Ending FuelCheck for Scuelazyns - Deep Holes Miners Bar & Cafe for Jesco_Sklor, not unanchoring
[2019-12-28 17:15:02] prod.DEBUG: Starting UnanchorCheck for Junsoraert - Lemmings Mining Emporium owned by Jesco_Sklor
[2019-12-28 17:15:02] prod.DEBUG: Ending FuelCheck for Junsoraert - Lemmings Mining Emporium for Jesco_Sklor, not unanchoring
[2019-12-28 17:15:02] prod.DEBUG: Starting UnanchorCheck for J144956 - Fortuna owned by Jesco_Sklor
[2019-12-28 17:15:02] prod.DEBUG: Ending FuelCheck for J144956 - Fortuna for Jesco_Sklor, not unanchoring
[2019-12-28 17:15:02] prod.DEBUG: Starting UnanchorCheck for Junsoraert - Deep Holes Drilling owned by Jesco_Sklor
[2019-12-28 17:15:02] prod.DEBUG: Ending FuelCheck for Junsoraert - Deep Holes Drilling for Jesco_Sklor, not unanchoring
[2019-12-28 17:15:02] prod.DEBUG: Starting UnanchorCheck for Junsoraert - Rockmonster Pleasure Hub owned by Jesco_Sklor
[2019-12-28 17:15:02] prod.DEBUG: Ending FuelCheck for Junsoraert - Rockmonster Pleasure Hub for Jesco_Sklor, not unanchoring
[2019-12-28 17:15:02] prod.DEBUG: Starting UnanchorCheck for Scuelazyns - Greasy Pit owned by Jesco_Sklor
[2019-12-28 17:15:02] prod.DEBUG: Ending FuelCheck for Scuelazyns - Greasy Pit for Jesco_Sklor, not unanchoring
[2019-12-28 17:30:03] prod.DEBUG: Starting FuelCheck for J144956 - Utopia Planitia owned by Jesco_Sklor
[2019-12-28 17:30:03] prod.DEBUG: Ending FuelCheck for J144956 - Utopia Planitia for Jesco_Sklor
[2019-12-28 17:30:03] prod.DEBUG: Starting FuelCheck for J144956 - Ishimura owned by Jesco_Sklor
[2019-12-28 17:30:03] prod.DEBUG: Ending FuelCheck for J144956 - Ishimura for Jesco_Sklor
[2019-12-28 17:30:03] prod.DEBUG: Starting FuelCheck for Scuelazyns - Deep Holes Miners Bar & Cafe owned by Jesco_Sklor
[2019-12-28 17:30:03] prod.DEBUG: Ending FuelCheck for Scuelazyns - Deep Holes Miners Bar & Cafe for Jesco_Sklor
[2019-12-28 17:30:03] prod.DEBUG: Starting FuelCheck for Junsoraert - Lemmings Mining Emporium owned by Jesco_Sklor
[2019-12-28 17:30:03] prod.DEBUG: Ending FuelCheck for Junsoraert - Lemmings Mining Emporium for Jesco_Sklor
[2019-12-28 17:30:03] prod.DEBUG: Starting FuelCheck for J144956 - Fortuna owned by Jesco_Sklor
[2019-12-28 17:30:03] prod.DEBUG: Ending FuelCheck for J144956 - Fortuna for Jesco_Sklor
[2019-12-28 17:30:03] prod.DEBUG: Starting FuelCheck for Junsoraert - Deep Holes Drilling owned by Jesco_Sklor
[2019-12-28 17:30:03] prod.DEBUG: Ending FuelCheck for Junsoraert - Deep Holes Drilling for Jesco_Sklor
[2019-12-28 17:30:03] prod.DEBUG: Starting FuelCheck for Junsoraert - Rockmonster Pleasure Hub owned by Jesco_Sklor
[2019-12-28 17:30:03] prod.DEBUG: Ending FuelCheck for Junsoraert - Rockmonster Pleasure Hub for Jesco_Sklor
[2019-12-28 17:30:03] prod.DEBUG: Starting FuelCheck for Scuelazyns - Greasy Pit owned by Jesco_Sklor
[2019-12-28 17:30:03] prod.DEBUG: Ending FuelCheck for Scuelazyns - Greasy Pit for Jesco_Sklor
[2019-12-28 17:45:03] prod.DEBUG: Refreshed token for Jesco_Sklor
[2019-12-28 17:45:03] prod.DEBUG: Checking Extractions for Jesco_Sklor
[2019-12-28 18:00:04] prod.NOTICE: Token for Jesco_Sklor was not expired
[2019-12-28 18:00:04] prod.DEBUG: Starting Structure pull for character Jesco_Sklor
[2019-12-28 18:00:06] prod.DEBUG: Finished Structure pull for character Jesco_Sklor
[2019-12-28 18:15:02] prod.DEBUG: Starting UnanchorCheck for J144956 - Utopia Planitia owned by Jesco_Sklor
[2019-12-28 18:15:02] prod.DEBUG: Ending FuelCheck for J144956 - Utopia Planitia for Jesco_Sklor, not unanchoring
[2019-12-28 18:15:02] prod.DEBUG: Starting UnanchorCheck for J144956 - Ishimura owned by Jesco_Sklor
[2019-12-28 18:15:02] prod.DEBUG: Ending FuelCheck for J144956 - Ishimura for Jesco_Sklor, not unanchoring
[2019-12-28 18:15:02] prod.DEBUG: Starting UnanchorCheck for Scuelazyns - Deep Holes Miners Bar & Cafe owned by Jesco_Sklor
[2019-12-28 18:15:02] prod.DEBUG: Ending FuelCheck for Scuelazyns - Deep Holes Miners Bar & Cafe for Jesco_Sklor, not unanchoring
[2019-12-28 18:15:02] prod.DEBUG: Starting UnanchorCheck for Junsoraert - Lemmings Mining Emporium owned by Jesco_Sklor
[2019-12-28 18:15:02] prod.DEBUG: Ending FuelCheck for Junsoraert - Lemmings Mining Emporium for Jesco_Sklor, not unanchoring
[2019-12-28 18:15:02] prod.DEBUG: Starting UnanchorCheck for J144956 - Fortuna owned by Jesco_Sklor
[2019-12-28 18:15:02] prod.DEBUG: Ending FuelCheck for J144956 - Fortuna for Jesco_Sklor, not unanchoring
[2019-12-28 18:15:02] prod.DEBUG: Starting UnanchorCheck for Junsoraert - Deep Holes Drilling owned by Jesco_Sklor
[2019-12-28 18:15:02] prod.DEBUG: Ending FuelCheck for Junsoraert - Deep Holes Drilling for Jesco_Sklor, not unanchoring
[2019-12-28 18:15:02] prod.DEBUG: Starting UnanchorCheck for Junsoraert - Rockmonster Pleasure Hub owned by Jesco_Sklor
[2019-12-28 18:15:02] prod.DEBUG: Ending FuelCheck for Junsoraert - Rockmonster Pleasure Hub for Jesco_Sklor, not unanchoring
[2019-12-28 18:15:02] prod.DEBUG: Starting UnanchorCheck for Scuelazyns - Greasy Pit owned by Jesco_Sklor
[2019-12-28 18:15:02] prod.DEBUG: Ending FuelCheck for Scuelazyns - Greasy Pit for Jesco_Sklor, not unanchoring
[2019-12-28 18:30:03] prod.DEBUG: Starting FuelCheck for J144956 - Utopia Planitia owned by Jesco_Sklor
[2019-12-28 18:30:03] prod.DEBUG: Ending FuelCheck for J144956 - Utopia Planitia for Jesco_Sklor
[2019-12-28 18:30:03] prod.DEBUG: Starting FuelCheck for J144956 - Ishimura owned by Jesco_Sklor
[2019-12-28 18:30:03] prod.DEBUG: Ending FuelCheck for J144956 - Ishimura for Jesco_Sklor
[2019-12-28 18:30:03] prod.DEBUG: Starting FuelCheck for Scuelazyns - Deep Holes Miners Bar & Cafe owned by Jesco_Sklor
[2019-12-28 18:30:03] prod.DEBUG: Ending FuelCheck for Scuelazyns - Deep Holes Miners Bar & Cafe for Jesco_Sklor
[2019-12-28 18:30:03] prod.DEBUG: Starting FuelCheck for Junsoraert - Lemmings Mining Emporium owned by Jesco_Sklor
[2019-12-28 18:30:03] prod.DEBUG: Ending FuelCheck for Junsoraert - Lemmings Mining Emporium for Jesco_Sklor
[2019-12-28 18:30:03] prod.DEBUG: Starting FuelCheck for J144956 - Fortuna owned by Jesco_Sklor
[2019-12-28 18:30:03] prod.DEBUG: Ending FuelCheck for J144956 - Fortuna for Jesco_Sklor
[2019-12-28 18:30:03] prod.DEBUG: Starting FuelCheck for Junsoraert - Deep Holes Drilling owned by Jesco_Sklor
[2019-12-28 18:30:03] prod.DEBUG: Ending FuelCheck for Junsoraert - Deep Holes Drilling for Jesco_Sklor
[2019-12-28 18:30:03] prod.DEBUG: Starting FuelCheck for Junsoraert - Rockmonster Pleasure Hub owned by Jesco_Sklor
[2019-12-28 18:30:03] prod.DEBUG: Ending FuelCheck for Junsoraert - Rockmonster Pleasure Hub for Jesco_Sklor
[2019-12-28 18:30:03] prod.DEBUG: Starting FuelCheck for Scuelazyns - Greasy Pit owned by Jesco_Sklor
[2019-12-28 18:30:03] prod.DEBUG: Ending FuelCheck for Scuelazyns - Greasy Pit for Jesco_Sklor
[2019-12-28 18:45:02] prod.DEBUG: Refreshed token for Jesco_Sklor
[2019-12-28 18:45:02] prod.DEBUG: Checking Extractions for Jesco_Sklor
[2019-12-28 19:00:03] prod.NOTICE: Token for Jesco_Sklor was not expired
[2019-12-28 19:00:03] prod.DEBUG: Starting Structure pull for character Jesco_Sklor
[2019-12-28 19:00:05] prod.DEBUG: Finished Structure pull for character Jesco_Sklor

Quick update.  I forgot to change the pwning group to psacln (Plesk Client). Works perfectly now.
So only one question remains. Would you be ok with me hosting it for the public?

Glad you got it sorted out. You are more than welcome to host a public version, just no assumed endorsements from me on your trustworthyness :)


[Bug] Can't open viewer while running GPU parallelized rendering

Hi!
I created a custom task on top of PickCube that loads a 3D scene from the Matterport3D dataset to add visual distractions during model evaluation. The function to load the scene looks like this:
def load_Matterport(builder):
    paths = sorted(list(glob.glob(os.path.join("./data/matterport3d/*.glb"))))
    path = random.choice(paths)
    pose = Pose(q=[0, 0, 0, 1])  # y-axis up for Matterport scenes
    # Add offset to workspace
    offset = np.array([0, 0, 0.8])
    builder.initial_pose = Pose(-offset)
    builder.add_visual_from_file(path, pose)
    actor = builder.build_static(name='background')
    # actor.set_pose(Pose(-offset))
    return actor

and is called inside a custom subclass of TableSceneBuilder like this:
class RandomBackgroundTableSceneBuilder(TableSceneBuilder):
    def build(self):
      ...
      load_Matterport(builder)
      builder.add_visual_from_file(
          filename=table_model_file, scale=[scale] * 3, pose=table_pose
      )
      ...

which then gets used inside the _load_scene function.
However, I run into the following error when I try to render a rollout from this env with render_mode=human:
RuntimeError: Modifying a scene (add/remove object/camera) is not allowed after creating the batched render system.
This is easily avoidable by setting the sim_backed to CPU, however if I train the model with sim_backend=gpu (which I am to leverage parallel env training), then evaluating the model with sim_backend=cpu results in significantly worse performance. Not sure if this is a known consequence of using different rendering backends or a bug, but just thought I'd bring it up here in case it's a solvable problem.
Thanks!

I think there may be some issue at the moment with using the viewer in addition to running GPU parallelized rendering. Someone raised the same issue on discord (which I am not sure how to link to on github). I'll try and get around to investigating this, this issue has been around for months now.
A temporary workaround is to train on the GPU backend, evaluate on the GPU backend and save trajectories with the RecordEpisode wrapper. Now if you want to use the viewer (done with render_mode=human), you can replay collected trajectories on the GPU onto the CPU using environment states by doing a for loop like
for i in range(100):
  env.set_state_dict(state_dicts[i])
  env.render_human() # or env.render() if render_mode=="human"

This is in fact how all of the nice videos/photos are generated usually on our documentation since high quality ray-tracing only works with the single CPU sim at the moment. Let me know if you need any help on the specifics of how to replay trajectories with environment state data, it might be a little bit under-documented at the moment.


Add Client Feature (list-client)

Linked the appropriate PR, as the same functionality is effectively achieved.


Consider increasing darkness of shadow mask around Alaska?

The idea here would be to darken the shadow mask by 10-20% so that Alaska "pops" more.  This should be an easy adjustment in Geoserver and can be done in production.

@brucecrevensten did we do it?

Done!