Regarding the effectiveness of IRPE in the original Transformer

Thank you very much for your contributions to the IRPE module. It has been very helpful to me. My question is whether the IRPE module is equally effective as the original Transformer module (i.e., the one provided in torch.nn). The reason is that the code framework of DETR is not suitable for my image classification task, and I need to use the original Transformer code. This means that I haven't implemented any positional encoding calculations in my backbone model. Instead, I have incorporated your IRPE module into the Transformer of my model. In the initial experiments using the same seed, I observed that the evaluation metrics such as mAP and F1-score did not change significantly in the first few epochs, but there was a 0.1% improvement in accuracy. I'm unsure if this improvement is due to the successful integration of the module or if I have not properly added the IRPE module. Additionally, I have calculated the number of parameters and found that it hasn't changed much. Is this normal and reasonable? Once again, I would like to express my gratitude for your work, and I hope to receive your response.

Hi @Zhong1015 ,
Thanks for your attention to our work!
Here is an example to integrate iRPE into the original Transformer model. (https://github.com/microsoft/Cream/blob/main/iRPE/DeiT-with-iRPE/rpe_vision_transformer.py#L79).
In general, using rpe_k only is enough. The smaller the model, the more improvement iRPE brings.
You can check whether rpe_k is not None.

Thank you very much for your response. Let me try the suggestions you mentioned.

Hello, @wkcn , I have carefully reviewed your code and found that based on the code you provided, the code of DETR is closer to what I am currently using. However, in my Transformer code, there is no definition for the 'pos' class, which causes some dimension mismatches after adding IRPE (confirming that 'rpe_k' has been assigned). This mismatch mainly manifests in the 'hw' dimensions, and the error points to a return of NoneType value in the multi-head attention section.
I believe the issue lies in DETR, where it handles certain 'pos' parameter information that is not addressed in my Transformer code. I'm unsure whether I should slowly re-implement IRPE in my code following the method used in DEIT or modify the DETR code to apply IRPE. As I'm still a beginner in computer vision, I would greatly appreciate your guidance on this matter.
This is my Transformer code.
class Transformer(Module):
r"""A transformer model. User is able to modify the attributes as needed. The architecture
is based on the paper "Attention Is All You Need". Ashish Vaswani, Noam Shazeer,
Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and
Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information
Processing Systems, pages 6000-6010. Users can build the BERT(https://arxiv.org/abs/1810.04805)
model with corresponding parameters.
Args:
    d_model: the number of expected features in the encoder/decoder inputs (default=512).
    nhead: the number of heads in the multiheadattention models (default=8).
    num_encoder_layers: the number of sub-encoder-layers in the encoder (default=6).
    num_decoder_layers: the number of sub-decoder-layers in the decoder (default=6).
    dim_feedforward: the dimension of the feedforward network model (default=2048).
    dropout: the dropout value (default=0.1).
    activation: the activation function of encoder/decoder intermediate layer, relu or gelu (default=relu).
    custom_encoder: custom encoder (default=None).
    custom_decoder: custom decoder (default=None).

Examples::
    >>> transformer_model = nn.Transformer(nhead=16, num_encoder_layers=12)
    >>> src = torch.rand((10, 32, 512))
    >>> tgt = torch.rand((20, 32, 512))
    >>> out = transformer_model(src, tgt)

Note: A full example to apply nn.Transformer module for the word language model is available in
https://github.com/pytorch/examples/tree/master/word_language_model
"""

def __init__(self, d_model: int = 512, nhead: int = 8, num_encoder_layers: int = 6,
             num_decoder_layers: int = 6, dim_feedforward: int = 2048, dropout: float = 0.1,
             activation: str = "relu", custom_encoder: Optional[Any] = None, custom_decoder: Optional[Any] = None) -> None:
    super(Transformer, self).__init__()

    if custom_encoder is not None:
        self.encoder = custom_encoder
    else:
        encoder_layer = TransformerEncoderLayer(d_model, nhead, dim_feedforward, dropout, activation)
        encoder_norm = LayerNorm(d_model)
        self.encoder = TransformerEncoder(encoder_layer, num_encoder_layers, encoder_norm)

    if custom_decoder is not None:
        self.decoder = custom_decoder
    else:
        decoder_layer = TransformerDecoderLayer(d_model, nhead, dim_feedforward, dropout, activation)
        decoder_norm = LayerNorm(d_model)
        self.decoder = TransformerDecoder(decoder_layer, num_decoder_layers, decoder_norm)

    self._reset_parameters()

    self.d_model = d_model
    self.nhead = nhead

def forward(self, src: Tensor, tgt: Tensor, src_mask: Optional[Tensor] = None, tgt_mask: Optional[Tensor] = None,
            memory_mask: Optional[Tensor] = None, src_key_padding_mask: Optional[Tensor] = None,
            tgt_key_padding_mask: Optional[Tensor] = None, memory_key_padding_mask: Optional[Tensor] = None) -> Tensor:
    r"""Take in and process masked source/target sequences.

    Args:
        src: the sequence to the encoder (required).
        tgt: the sequence to the decoder (required).
        src_mask: the additive mask for the src sequence (optional).
        tgt_mask: the additive mask for the tgt sequence (optional).
        memory_mask: the additive mask for the encoder output (optional).
        src_key_padding_mask: the ByteTensor mask for src keys per batch (optional).
        tgt_key_padding_mask: the ByteTensor mask for tgt keys per batch (optional).
        memory_key_padding_mask: the ByteTensor mask for memory keys per batch (optional).

    Shape:
        - src: :math:`(S, N, E)`.
        - tgt: :math:`(T, N, E)`.
        - src_mask: :math:`(S, S)`.
        - tgt_mask: :math:`(T, T)`.
        - memory_mask: :math:`(T, S)`.
        - src_key_padding_mask: :math:`(N, S)`.
        - tgt_key_padding_mask: :math:`(N, T)`.
        - memory_key_padding_mask: :math:`(N, S)`.

        Note: [src/tgt/memory]_mask ensures that position i is allowed to attend the unmasked
        positions. If a ByteTensor is provided, the non-zero positions are not allowed to attend
        while the zero positions will be unchanged. If a BoolTensor is provided, positions with ``True``
        are not allowed to attend while ``False`` values will be unchanged. If a FloatTensor
        is provided, it will be added to the attention weight.
        [src/tgt/memory]_key_padding_mask provides specified elements in the key to be ignored by
        the attention. If a ByteTensor is provided, the non-zero positions will be ignored while the zero
        positions will be unchanged. If a BoolTensor is provided, the positions with the
        value of ``True`` will be ignored while the position with the value of ``False`` will be unchanged.

        - output: :math:`(T, N, E)`.

        Note: Due to the multi-head attention architecture in the transformer model,
        the output sequence length of a transformer is same as the input sequence
        (i.e. target) length of the decode.

        where S is the source sequence length, T is the target sequence length, N is the
        batch size, E is the feature number

    Examples:
        >>> output = transformer_model(src, tgt, src_mask=src_mask, tgt_mask=tgt_mask)
    """

    if src.size(1) != tgt.size(1):
        raise RuntimeError("the batch number of src and tgt must be equal")

    if src.size(2) != self.d_model or tgt.size(2) != self.d_model:
        raise RuntimeError("the feature number of src and tgt must be equal to d_model")

    memory = self.encoder(src, mask=src_mask, src_key_padding_mask=src_key_padding_mask)
    output = self.decoder(tgt, memory, tgt_mask=tgt_mask, memory_mask=memory_mask,
                          tgt_key_padding_mask=tgt_key_padding_mask,
                          memory_key_padding_mask=memory_key_padding_mask)
    return output

def generate_square_subsequent_mask(self, sz: int) -> Tensor:
    r"""Generate a square mask for the sequence. The masked positions are filled with float('-inf').
        Unmasked positions are filled with float(0.0).
    """
    mask = (torch.triu(torch.ones(sz, sz)) == 1).transpose(0, 1)
    mask = mask.float().masked_fill(mask == 0, float('-inf')).masked_fill(mask == 1, float(0.0))
    return mask

def _reset_parameters(self):
    r"""Initiate parameters in the transformer model."""

    for p in self.parameters():
        if p.dim() > 1:
            xavier_uniform_(p)

class TransformerEncoder(Module):
r"""TransformerEncoder is a stack of N encoder layers
Args:
    encoder_layer: an instance of the TransformerEncoderLayer() class (required).
    num_layers: the number of sub-encoder-layers in the encoder (required).
    norm: the layer normalization component (optional).

Examples::
    >>> encoder_layer = nn.TransformerEncoderLayer(d_model=512, nhead=8)
    >>> transformer_encoder = nn.TransformerEncoder(encoder_layer, num_layers=6)
    >>> src = torch.rand(10, 32, 512)
    >>> out = transformer_encoder(src)
"""
__constants__ = ['norm']

def __init__(self, encoder_layer, num_layers, norm=None):
    super(TransformerEncoder, self).__init__()
    self.layers = _get_clones(encoder_layer, num_layers)
    self.num_layers = num_layers
    self.norm = norm

def forward(self, src: Tensor, mask: Optional[Tensor] = None, src_key_padding_mask: Optional[Tensor] = None) -> Tensor:
    r"""Pass the input through the encoder layers in turn.

    Args:
        src: the sequence to the encoder (required).
        mask: the mask for the src sequence (optional).
        src_key_padding_mask: the mask for the src keys per batch (optional).

    Shape:
        see the docs in Transformer class.
    """
    output = src

    for mod in self.layers:
        output = mod(output, src_mask=mask, src_key_padding_mask=src_key_padding_mask)

    if self.norm is not None:
        output = self.norm(output)

    return output

class TransformerDecoder(Module):
r"""TransformerDecoder is a stack of N decoder layers
Args:
    decoder_layer: an instance of the TransformerDecoderLayer() class (required).
    num_layers: the number of sub-decoder-layers in the decoder (required).
    norm: the layer normalization component (optional).

Examples::
    >>> decoder_layer = nn.TransformerDecoderLayer(d_model=512, nhead=8)
    >>> transformer_decoder = nn.TransformerDecoder(decoder_layer, num_layers=6)
    >>> memory = torch.rand(10, 32, 512)
    >>> tgt = torch.rand(20, 32, 512)
    >>> out = transformer_decoder(tgt, memory)
"""
__constants__ = ['norm']

def __init__(self, decoder_layer, num_layers, norm=None):
    super(TransformerDecoder, self).__init__()
    self.layers = _get_clones(decoder_layer, num_layers)
    self.num_layers = num_layers
    self.norm = norm

def forward(self, tgt: Tensor, memory: Tensor, tgt_mask: Optional[Tensor] = None,
            memory_mask: Optional[Tensor] = None, tgt_key_padding_mask: Optional[Tensor] = None,
            memory_key_padding_mask: Optional[Tensor] = None) -> Tensor:
    r"""Pass the inputs (and mask) through the decoder layer in turn.

    Args:
        tgt: the sequence to the decoder (required).
        memory: the sequence from the last layer of the encoder (required).
        tgt_mask: the mask for the tgt sequence (optional).
        memory_mask: the mask for the memory sequence (optional).
        tgt_key_padding_mask: the mask for the tgt keys per batch (optional).
        memory_key_padding_mask: the mask for the memory keys per batch (optional).

    Shape:
        see the docs in Transformer class.
    """
    output = tgt

    for mod in self.layers:
        output = mod(output, memory, tgt_mask=tgt_mask,
                     memory_mask=memory_mask,
                     tgt_key_padding_mask=tgt_key_padding_mask,
                     memory_key_padding_mask=memory_key_padding_mask)

    if self.norm is not None:
        output = self.norm(output)

    return output

class TransformerEncoderLayer(Module):
r"""TransformerEncoderLayer is made up of self-attn and feedforward network.
This standard encoder layer is based on the paper "Attention Is All You Need".
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez,
Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in
Neural Information Processing Systems, pages 6000-6010. Users may modify or implement
in a different way during application.
Args:
    d_model: the number of expected features in the input (required).
    nhead: the number of heads in the multiheadattention models (required).
    dim_feedforward: the dimension of the feedforward network model (default=2048).
    dropout: the dropout value (default=0.1).
    activation: the activation function of intermediate layer, relu or gelu (default=relu).

Examples::
    >>> encoder_layer = nn.TransformerEncoderLayer(d_model=512, nhead=8)
    >>> src = torch.rand(10, 32, 512)
    >>> out = encoder_layer(src)
"""

def __init__(self, d_model, nhead, dim_feedforward=2048, dropout=0.1, activation="relu"):
    super(TransformerEncoderLayer, self).__init__()
    self.self_attn = MultiheadAttention(d_model, nhead, dropout=dropout)
    # Implementation of Feedforward model
    self.linear1 = Linear(d_model, dim_feedforward)
    self.dropout = Dropout(dropout)
    self.linear2 = Linear(dim_feedforward, d_model)

    self.norm1 = LayerNorm(d_model)
    self.norm2 = LayerNorm(d_model)
    self.dropout1 = Dropout(dropout)
    self.dropout2 = Dropout(dropout)

    self.activation = _get_activation_fn(activation)

def __setstate__(self, state):
    if 'activation' not in state:
        state['activation'] = F.relu
    super(TransformerEncoderLayer, self).__setstate__(state)

def forward(self, src: Tensor, src_mask: Optional[Tensor] = None, src_key_padding_mask: Optional[Tensor] = None) -> Tensor:
    r"""Pass the input through the encoder layer.

    Args:
        src: the sequence to the encoder layer (required).
        src_mask: the mask for the src sequence (optional).
        src_key_padding_mask: the mask for the src keys per batch (optional).

    Shape:
        see the docs in Transformer class.
    """
    src2 = self.self_attn(src, src, src, attn_mask=src_mask,
                          key_padding_mask=src_key_padding_mask)[0]
    src = src + self.dropout1(src2)
    src = self.norm1(src)
    src2 = self.linear2(self.dropout(self.activation(self.linear1(src))))
    src = src + self.dropout2(src2)
    src = self.norm2(src)
    return src

class TransformerDecoderLayer(Module):
r"""TransformerDecoderLayer is made up of self-attn, multi-head-attn and feedforward network.
This standard decoder layer is based on the paper "Attention Is All You Need".
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez,
Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in
Neural Information Processing Systems, pages 6000-6010. Users may modify or implement
in a different way during application.
Args:
    d_model: the number of expected features in the input (required).
    nhead: the number of heads in the multiheadattention models (required).
    dim_feedforward: the dimension of the feedforward network model (default=2048).
    dropout: the dropout value (default=0.1).
    activation: the activation function of intermediate layer, relu or gelu (default=relu).

Examples::
    >>> decoder_layer = nn.TransformerDecoderLayer(d_model=512, nhead=8)
    >>> memory = torch.rand(10, 32, 512)
    >>> tgt = torch.rand(20, 32, 512)
    >>> out = decoder_layer(tgt, memory)
"""

def __init__(self, d_model, nhead, dim_feedforward=2048, dropout=0.1, activation="relu"):
    super(TransformerDecoderLayer, self).__init__()
    self.self_attn = MultiheadAttention(d_model, nhead, dropout=dropout)
    self.multihead_attn = MultiheadAttention(d_model, nhead, dropout=dropout)
    # Implementation of Feedforward model
    self.linear1 = Linear(d_model, dim_feedforward)
    self.dropout = Dropout(dropout)
    self.linear2 = Linear(dim_feedforward, d_model)

    self.norm1 = LayerNorm(d_model)
    self.norm2 = LayerNorm(d_model)
    self.norm3 = LayerNorm(d_model)
    self.dropout1 = Dropout(dropout)
    self.dropout2 = Dropout(dropout)
    self.dropout3 = Dropout(dropout)

    self.activation = _get_activation_fn(activation)

def __setstate__(self, state):
    if 'activation' not in state:
        state['activation'] = F.relu
    super(TransformerDecoderLayer, self).__setstate__(state)

def forward(self, tgt: Tensor, memory: Tensor, tgt_mask: Optional[Tensor] = None, memory_mask: Optional[Tensor] = None,
            tgt_key_padding_mask: Optional[Tensor] = None, memory_key_padding_mask: Optional[Tensor] = None) -> Tensor:
    r"""Pass the inputs (and mask) through the decoder layer.

    Args:
        tgt: the sequence to the decoder layer (required).
        memory: the sequence from the last layer of the encoder (required).
        tgt_mask: the mask for the tgt sequence (optional).
        memory_mask: the mask for the memory sequence (optional).
        tgt_key_padding_mask: the mask for the tgt keys per batch (optional).
        memory_key_padding_mask: the mask for the memory keys per batch (optional).

    Shape:
        see the docs in Transformer class.
    """
    tgt2 = self.self_attn(tgt, tgt, tgt, attn_mask=tgt_mask,
                          key_padding_mask=tgt_key_padding_mask)[0]
    tgt = tgt + self.dropout1(tgt2)
    tgt = self.norm1(tgt)
    tgt2 = self.multihead_attn(tgt, memory, memory, attn_mask=memory_mask,
                               key_padding_mask=memory_key_padding_mask)[0]
    tgt = tgt + self.dropout2(tgt2)
    tgt = self.norm2(tgt)
    tgt2 = self.linear2(self.dropout(self.activation(self.linear1(tgt))))
    tgt = tgt + self.dropout3(tgt2)
    tgt = self.norm3(tgt)
    return tgt

def _get_clones(module, N):
return ModuleList([copy.deepcopy(module) for i in range(N)])
def _get_activation_fn(activation):
if activation == "relu":
return F.relu
elif activation == "gelu":
return F.gelu
raise RuntimeError("activation should be relu/gelu, not {}".format(activation))

Hello, @wkcn , I have carefully reviewed your code and found that based on the code you provided, the code of DETR is closer to what I am currently using. However, in my Transformer code, there is no definition for the 'pos' class, which causes some dimension mismatches after adding IRPE (confirming that 'rpe_k' has been assigned). This mismatch mainly manifests in the 'hw' dimensions, and the error points to a return of NoneType value in the multi-head attention section.
I believe the issue lies in DETR, where it handles certain 'pos' parameter information that is not addressed in my Transformer code. I'm unsure whether I should slowly re-implement IRPE in my code following the method used in DEIT or modify the DETR code to apply IRPE. As I'm still a beginner in computer vision, I would greatly appreciate your guidance on this matter.
This is my transformer code:
`import copy
from typing import Optional, Any
import torch
from torch import Tensor
from .. import functional as F
from .module import Module
from .activation import MultiheadAttention
from .container import ModuleList
from ..init import xavier_uniform_
from .dropout import Dropout
from .linear import Linear
from .normalization import LayerNorm
class Transformer(Module):
r"""A transformer model. User is able to modify the attributes as needed. The architecture
is based on the paper "Attention Is All You Need". Ashish Vaswani, Noam Shazeer,
Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and
Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information
Processing Systems, pages 6000-6010. Users can build the BERT(https://arxiv.org/abs/1810.04805)
model with corresponding parameters.
Args:
    d_model: the number of expected features in the encoder/decoder inputs (default=512).
    nhead: the number of heads in the multiheadattention models (default=8).
    num_encoder_layers: the number of sub-encoder-layers in the encoder (default=6).
    num_decoder_layers: the number of sub-decoder-layers in the decoder (default=6).
    dim_feedforward: the dimension of the feedforward network model (default=2048).
    dropout: the dropout value (default=0.1).
    activation: the activation function of encoder/decoder intermediate layer, relu or gelu (default=relu).
    custom_encoder: custom encoder (default=None).
    custom_decoder: custom decoder (default=None).

Examples::
    >>> transformer_model = nn.Transformer(nhead=16, num_encoder_layers=12)
    >>> src = torch.rand((10, 32, 512))
    >>> tgt = torch.rand((20, 32, 512))
    >>> out = transformer_model(src, tgt)

Note: A full example to apply nn.Transformer module for the word language model is available in
https://github.com/pytorch/examples/tree/master/word_language_model
"""

def __init__(self, d_model: int = 512, nhead: int = 8, num_encoder_layers: int = 6,
             num_decoder_layers: int = 6, dim_feedforward: int = 2048, dropout: float = 0.1,
             activation: str = "relu", custom_encoder: Optional[Any] = None, custom_decoder: Optional[Any] = None) -> None:
    super(Transformer, self).__init__()

    if custom_encoder is not None:
        self.encoder = custom_encoder
    else:
        encoder_layer = TransformerEncoderLayer(d_model, nhead, dim_feedforward, dropout, activation)
        encoder_norm = LayerNorm(d_model)
        self.encoder = TransformerEncoder(encoder_layer, num_encoder_layers, encoder_norm)

    if custom_decoder is not None:
        self.decoder = custom_decoder
    else:
        decoder_layer = TransformerDecoderLayer(d_model, nhead, dim_feedforward, dropout, activation)
        decoder_norm = LayerNorm(d_model)
        self.decoder = TransformerDecoder(decoder_layer, num_decoder_layers, decoder_norm)

    self._reset_parameters()

    self.d_model = d_model
    self.nhead = nhead

def forward(self, src: Tensor, tgt: Tensor, src_mask: Optional[Tensor] = None, tgt_mask: Optional[Tensor] = None,
            memory_mask: Optional[Tensor] = None, src_key_padding_mask: Optional[Tensor] = None,
            tgt_key_padding_mask: Optional[Tensor] = None, memory_key_padding_mask: Optional[Tensor] = None) -> Tensor:
    r"""Take in and process masked source/target sequences.

    Args:
        src: the sequence to the encoder (required).
        tgt: the sequence to the decoder (required).
        src_mask: the additive mask for the src sequence (optional).
        tgt_mask: the additive mask for the tgt sequence (optional).
        memory_mask: the additive mask for the encoder output (optional).
        src_key_padding_mask: the ByteTensor mask for src keys per batch (optional).
        tgt_key_padding_mask: the ByteTensor mask for tgt keys per batch (optional).
        memory_key_padding_mask: the ByteTensor mask for memory keys per batch (optional).

    Shape:
        - src: :math:`(S, N, E)`.
        - tgt: :math:`(T, N, E)`.
        - src_mask: :math:`(S, S)`.
        - tgt_mask: :math:`(T, T)`.
        - memory_mask: :math:`(T, S)`.
        - src_key_padding_mask: :math:`(N, S)`.
        - tgt_key_padding_mask: :math:`(N, T)`.
        - memory_key_padding_mask: :math:`(N, S)`.

        Note: [src/tgt/memory]_mask ensures that position i is allowed to attend the unmasked
        positions. If a ByteTensor is provided, the non-zero positions are not allowed to attend
        while the zero positions will be unchanged. If a BoolTensor is provided, positions with ``True``
        are not allowed to attend while ``False`` values will be unchanged. If a FloatTensor
        is provided, it will be added to the attention weight.
        [src/tgt/memory]_key_padding_mask provides specified elements in the key to be ignored by
        the attention. If a ByteTensor is provided, the non-zero positions will be ignored while the zero
        positions will be unchanged. If a BoolTensor is provided, the positions with the
        value of ``True`` will be ignored while the position with the value of ``False`` will be unchanged.

        - output: :math:`(T, N, E)`.

        Note: Due to the multi-head attention architecture in the transformer model,
        the output sequence length of a transformer is same as the input sequence
        (i.e. target) length of the decode.

        where S is the source sequence length, T is the target sequence length, N is the
        batch size, E is the feature number

    Examples:
        >>> output = transformer_model(src, tgt, src_mask=src_mask, tgt_mask=tgt_mask)
    """

    if src.size(1) != tgt.size(1):
        raise RuntimeError("the batch number of src and tgt must be equal")

    if src.size(2) != self.d_model or tgt.size(2) != self.d_model:
        raise RuntimeError("the feature number of src and tgt must be equal to d_model")

    memory = self.encoder(src, mask=src_mask, src_key_padding_mask=src_key_padding_mask)
    output = self.decoder(tgt, memory, tgt_mask=tgt_mask, memory_mask=memory_mask,
                          tgt_key_padding_mask=tgt_key_padding_mask,
                          memory_key_padding_mask=memory_key_padding_mask)
    return output

def generate_square_subsequent_mask(self, sz: int) -> Tensor:
    r"""Generate a square mask for the sequence. The masked positions are filled with float('-inf').
        Unmasked positions are filled with float(0.0).
    """
    mask = (torch.triu(torch.ones(sz, sz)) == 1).transpose(0, 1)
    mask = mask.float().masked_fill(mask == 0, float('-inf')).masked_fill(mask == 1, float(0.0))
    return mask

def _reset_parameters(self):
    r"""Initiate parameters in the transformer model."""

    for p in self.parameters():
        if p.dim() > 1:
            xavier_uniform_(p)

class TransformerEncoder(Module):
r"""TransformerEncoder is a stack of N encoder layers
Args:
    encoder_layer: an instance of the TransformerEncoderLayer() class (required).
    num_layers: the number of sub-encoder-layers in the encoder (required).
    norm: the layer normalization component (optional).

Examples::
    >>> encoder_layer = nn.TransformerEncoderLayer(d_model=512, nhead=8)
    >>> transformer_encoder = nn.TransformerEncoder(encoder_layer, num_layers=6)
    >>> src = torch.rand(10, 32, 512)
    >>> out = transformer_encoder(src)
"""
__constants__ = ['norm']

def __init__(self, encoder_layer, num_layers, norm=None):
    super(TransformerEncoder, self).__init__()
    self.layers = _get_clones(encoder_layer, num_layers)
    self.num_layers = num_layers
    self.norm = norm

def forward(self, src: Tensor, mask: Optional[Tensor] = None, src_key_padding_mask: Optional[Tensor] = None) -> Tensor:
    r"""Pass the input through the encoder layers in turn.

    Args:
        src: the sequence to the encoder (required).
        mask: the mask for the src sequence (optional).
        src_key_padding_mask: the mask for the src keys per batch (optional).

    Shape:
        see the docs in Transformer class.
    """
    output = src

    for mod in self.layers:
        output = mod(output, src_mask=mask, src_key_padding_mask=src_key_padding_mask)

    if self.norm is not None:
        output = self.norm(output)

    return output

class TransformerDecoder(Module):
r"""TransformerDecoder is a stack of N decoder layers
Args:
    decoder_layer: an instance of the TransformerDecoderLayer() class (required).
    num_layers: the number of sub-decoder-layers in the decoder (required).
    norm: the layer normalization component (optional).

Examples::
    >>> decoder_layer = nn.TransformerDecoderLayer(d_model=512, nhead=8)
    >>> transformer_decoder = nn.TransformerDecoder(decoder_layer, num_layers=6)
    >>> memory = torch.rand(10, 32, 512)
    >>> tgt = torch.rand(20, 32, 512)
    >>> out = transformer_decoder(tgt, memory)
"""
__constants__ = ['norm']

def __init__(self, decoder_layer, num_layers, norm=None):
    super(TransformerDecoder, self).__init__()
    self.layers = _get_clones(decoder_layer, num_layers)
    self.num_layers = num_layers
    self.norm = norm

def forward(self, tgt: Tensor, memory: Tensor, tgt_mask: Optional[Tensor] = None,
            memory_mask: Optional[Tensor] = None, tgt_key_padding_mask: Optional[Tensor] = None,
            memory_key_padding_mask: Optional[Tensor] = None) -> Tensor:
    r"""Pass the inputs (and mask) through the decoder layer in turn.

    Args:
        tgt: the sequence to the decoder (required).
        memory: the sequence from the last layer of the encoder (required).
        tgt_mask: the mask for the tgt sequence (optional).
        memory_mask: the mask for the memory sequence (optional).
        tgt_key_padding_mask: the mask for the tgt keys per batch (optional).
        memory_key_padding_mask: the mask for the memory keys per batch (optional).

    Shape:
        see the docs in Transformer class.
    """
    output = tgt

    for mod in self.layers:
        output = mod(output, memory, tgt_mask=tgt_mask,
                     memory_mask=memory_mask,
                     tgt_key_padding_mask=tgt_key_padding_mask,
                     memory_key_padding_mask=memory_key_padding_mask)

    if self.norm is not None:
        output = self.norm(output)

    return output

class TransformerEncoderLayer(Module):
r"""TransformerEncoderLayer is made up of self-attn and feedforward network.
This standard encoder layer is based on the paper "Attention Is All You Need".
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez,
Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in
Neural Information Processing Systems, pages 6000-6010. Users may modify or implement
in a different way during application.
Args:
    d_model: the number of expected features in the input (required).
    nhead: the number of heads in the multiheadattention models (required).
    dim_feedforward: the dimension of the feedforward network model (default=2048).
    dropout: the dropout value (default=0.1).
    activation: the activation function of intermediate layer, relu or gelu (default=relu).

Examples::
    >>> encoder_layer = nn.TransformerEncoderLayer(d_model=512, nhead=8)
    >>> src = torch.rand(10, 32, 512)
    >>> out = encoder_layer(src)
"""

def __init__(self, d_model, nhead, dim_feedforward=2048, dropout=0.1, activation="relu"):
    super(TransformerEncoderLayer, self).__init__()
    self.self_attn = MultiheadAttention(d_model, nhead, dropout=dropout)
    # Implementation of Feedforward model
    self.linear1 = Linear(d_model, dim_feedforward)
    self.dropout = Dropout(dropout)
    self.linear2 = Linear(dim_feedforward, d_model)

    self.norm1 = LayerNorm(d_model)
    self.norm2 = LayerNorm(d_model)
    self.dropout1 = Dropout(dropout)
    self.dropout2 = Dropout(dropout)

    self.activation = _get_activation_fn(activation)

def __setstate__(self, state):
    if 'activation' not in state:
        state['activation'] = F.relu
    super(TransformerEncoderLayer, self).__setstate__(state)

def forward(self, src: Tensor, src_mask: Optional[Tensor] = None, src_key_padding_mask: Optional[Tensor] = None) -> Tensor:
    r"""Pass the input through the encoder layer.

    Args:
        src: the sequence to the encoder layer (required).
        src_mask: the mask for the src sequence (optional).
        src_key_padding_mask: the mask for the src keys per batch (optional).

    Shape:
        see the docs in Transformer class.
    """
    src2 = self.self_attn(src, src, src, attn_mask=src_mask,
                          key_padding_mask=src_key_padding_mask)[0]
    src = src + self.dropout1(src2)
    src = self.norm1(src)
    src2 = self.linear2(self.dropout(self.activation(self.linear1(src))))
    src = src + self.dropout2(src2)
    src = self.norm2(src)
    return src

class TransformerDecoderLayer(Module):
r"""TransformerDecoderLayer is made up of self-attn, multi-head-attn and feedforward network.
This standard decoder layer is based on the paper "Attention Is All You Need".
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez,
Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in
Neural Information Processing Systems, pages 6000-6010. Users may modify or implement
in a different way during application.
Args:
    d_model: the number of expected features in the input (required).
    nhead: the number of heads in the multiheadattention models (required).
    dim_feedforward: the dimension of the feedforward network model (default=2048).
    dropout: the dropout value (default=0.1).
    activation: the activation function of intermediate layer, relu or gelu (default=relu).

Examples::
    >>> decoder_layer = nn.TransformerDecoderLayer(d_model=512, nhead=8)
    >>> memory = torch.rand(10, 32, 512)
    >>> tgt = torch.rand(20, 32, 512)
    >>> out = decoder_layer(tgt, memory)
"""

def __init__(self, d_model, nhead, dim_feedforward=2048, dropout=0.1, activation="relu"):
    super(TransformerDecoderLayer, self).__init__()
    self.self_attn = MultiheadAttention(d_model, nhead, dropout=dropout)
    self.multihead_attn = MultiheadAttention(d_model, nhead, dropout=dropout)
    # Implementation of Feedforward model
    self.linear1 = Linear(d_model, dim_feedforward)
    self.dropout = Dropout(dropout)
    self.linear2 = Linear(dim_feedforward, d_model)

    self.norm1 = LayerNorm(d_model)
    self.norm2 = LayerNorm(d_model)
    self.norm3 = LayerNorm(d_model)
    self.dropout1 = Dropout(dropout)
    self.dropout2 = Dropout(dropout)
    self.dropout3 = Dropout(dropout)

    self.activation = _get_activation_fn(activation)

def __setstate__(self, state):
    if 'activation' not in state:
        state['activation'] = F.relu
    super(TransformerDecoderLayer, self).__setstate__(state)

def forward(self, tgt: Tensor, memory: Tensor, tgt_mask: Optional[Tensor] = None, memory_mask: Optional[Tensor] = None,
            tgt_key_padding_mask: Optional[Tensor] = None, memory_key_padding_mask: Optional[Tensor] = None) -> Tensor:
    r"""Pass the inputs (and mask) through the decoder layer.

    Args:
        tgt: the sequence to the decoder layer (required).
        memory: the sequence from the last layer of the encoder (required).
        tgt_mask: the mask for the tgt sequence (optional).
        memory_mask: the mask for the memory sequence (optional).
        tgt_key_padding_mask: the mask for the tgt keys per batch (optional).
        memory_key_padding_mask: the mask for the memory keys per batch (optional).

    Shape:
        see the docs in Transformer class.
    """
    tgt2 = self.self_attn(tgt, tgt, tgt, attn_mask=tgt_mask,
                          key_padding_mask=tgt_key_padding_mask)[0]
    tgt = tgt + self.dropout1(tgt2)
    tgt = self.norm1(tgt)
    tgt2 = self.multihead_attn(tgt, memory, memory, attn_mask=memory_mask,
                               key_padding_mask=memory_key_padding_mask)[0]
    tgt = tgt + self.dropout2(tgt2)
    tgt = self.norm2(tgt)
    tgt2 = self.linear2(self.dropout(self.activation(self.linear1(tgt))))
    tgt = tgt + self.dropout3(tgt2)
    tgt = self.norm3(tgt)
    return tgt

def _get_clones(module, N):
return ModuleList([copy.deepcopy(module) for i in range(N)])
def _get_activation_fn(activation):
if activation == "relu":
return F.relu
elif activation == "gelu":
return F.gelu
raise RuntimeError("activation should be relu/gelu, not {}".format(activation))

`

Hi @Zhong1015 , you can obtain the hw variables in https://github.com/microsoft/Cream/blob/73afa00ae492928e836bfbe2f249ce08a655cae9/iRPE/DETR-with-iRPE/models/transformer.py#L108
I integrated iRPE in TransformerEncoderLayer only (https://github.com/microsoft/Cream/blob/main/iRPE/DETR-with-iRPE/models/transformer.py#L189).

Thank you very much for your reply, which has been very helpful to me. Lastly, I would like to know if your IRPE module remains effective without the two parameters, query_embed and pos_embed, present in DETR?

Hi@wkcn，Thank you very much for your reply, which has been very helpful to me. Lastly, I would like to know if your IRPE module remains effective without the two parameters, query_embed and pos_embed, present in DETR?

@Zhong1015
It remains effective without query and pos_embed.
As shown in our paper, context relative position encoding (w/ query_embed) is better than bias relative position one (w/o query_embed). The model with bias relative position encoding is better that the one without any relative position encoding.
The absolute position encoding (pos_embed) is required for object detection, but not required for image classificiation if using relative position encoding.

Your response has been very beneficial to me. I greatly appreciate you taking the time to answer my questions amidst your busy schedule. Thank you once again!

Hello,@wkcn I have another question that requires your response. According to your code, regarding the 'hw' parameter used in the forward pass of the Transformer, is it solely for obtaining the height and width of the image after extracting the main features, facilitating the calculation of the 'rpe_q, k, v' parameters? This implies that when using the original Transformer (where parameters H and W are extracted outside the Transformer), providing the corresponding height and width values of the main features in the forward pass of the Transformer should suffice for IRPE to function properly.Because I'm not handling the shape operations of image features within the Transformer itself, I consistently encounter quite troublesome issues with the calls to 'h' and 'w'.If you could reply to my question, I would be extremely grateful.

<EMAIL_ADDRESS>you again for your response. Previously, in order to pass the corresponding h, w parameters, I directly printed the shape of the feature map extracted from the backbone model and assigned the values of h and w in the attention function. Moreover, my feature map is indeed square, with dimensions of 12x12. After experimenting, I obtained some new results. Is it also feasible and reasonable for me to do it this way?

@Zhong1015 Did you mean the two experiments (w/ and w/o passing hw) got different results?

<EMAIL_ADDRESS>I mean is the situation with and without IRPE. I believe I successfully added IRPE and obtained some new results under the same seed. Specifically, I followed your DETR code and added the relevant IRPE code to the corresponding part of my code, using a parameter num_bucket of 49. For the resolution (h, w) part, I directly assigned the values of (h, w) extracted from the backbone to the corresponding (h, w) values in the part of your code where IRPE is added to DETR. My (h, w) values are both 12, which is what you referred to as "square." I want to know if this approach is correct.

<EMAIL_ADDRESS>I mean is the situation with and without IRPE. I believe I successfully added IRPE and obtained some new results under the same seed. Specifically, I followed your DETR code and added the relevant IRPE code to the corresponding part of my code, using a parameter num_bucket of 49. For the resolution (h, w) part, I directly assigned the values of (h, w) extracted from the backbone to the corresponding (h, w) values in the part of your code where IRPE is added to DETR. My (h, w) values are both 12, which is what you referred to as "square." I want to know if this approach is correct.

Correct.


ELK stack Logspout make fails

Have forked
https://github.com/oracle/docker-images/tree/master/ContainerCloud
to
https://github.com/mikarinneoracle/docker-images
and I’m running make. While Logstash and Kibana are built fine into my Docker-hub, the Logspout gives an error:
..clip..
git clone https://github.com/gorilla/context /go/src/github.com/gorilla/context
cd /go/src/github.com/gorilla/context
git submodule update --init --recursive
cd /go/src/github.com/gorilla/context
git show-ref
cd /go/src/github.com/gorilla/context
git submodule update --init --recursive
Removing intermediate container f491cd640c6d
The command '/bin/sh -c /build.sh' returned a non-zero code: 1
make[1]: *** [image] Error 1
I’m attaching the log for logspout make. Does this work for you? I’m running OSX.
Any idea what’s going wrong ?
The same result if running make under elk or under logspout dir.

Error log for make under logspout dir
error.log.txt

@mikarinneoracle sorry about that! Pushing a fix for it today.

Many thanks, the make worked now.
Mika
From: Karthik Gaekwad<EMAIL_ADDRESS>Reply-To: oracle/docker-images<EMAIL_ADDRESS>Date: Saturday 11 March 2017 at 06:07
To: oracle/docker-images<EMAIL_ADDRESS>Cc: Mika Rinne<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [oracle/docker-images] ELK stack Logspout make fails (#292)
@mikarinneoracle sorry about that! Pushing a fix for it today.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Thanks for verifying @mikarinneoracle! Also, really cool container work from you!!


Support for rpath instead of DYLD_LIBRARY_PATH?

I don't know if this is actually a cargo issue rather than rustfmt issue, but here goes...
Supposedly (https://github.com/rust-lang/cargo/issues/4421#issuecomment-326110997) DYLD_LIBRARY_PATH is no longer the approved way to link libraries, as it requires turning off System Integrity Protection, and @rpath is the mandated way to accomplish such dynamic linkage going forward.
Is there a way that rustfmt can adapt to such a world? E.g. by invoking install_name_tool to point to the installed version of libsyntax during installation?

(Full disclosure: I did not read the tips in the readme when I first tried to use rustfmt after doing cargo install rustfmt; the error I subsequently got is what led me to discover install_name_tool which I successfully used to get rustfmt working on my own system. but of course that was manual hacking, not a general purpose patch to rustfmt and/or cargo to automate such a process for all users...)

The plan for rustfmt is to install with rustup, which I think addresses this issue. That is tracked elsewhere. It would be nice if we could work when build from source too.

rustfmt does not use rustc_private anymore, so this should not be an issue anymore.


near ")": syntax error in "SELECT * FROM blocks

11|counter | [2024-04-10 11:41:09][INFO] Block: 838599 (60.49, hashes: L:78e87 / TX:48f18 / M:639b4)
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR
11|counter | ERROR:root:SQLITE_LOG: near ")": syntax error in "SELECT * FROM blocks WHERE block_index IN (()) ORDER BY block_index ASC" (1) SQLITE_ERROR

another, related problem?
12|counterparty  | ERROR:root:SQLITE_LOG: row value misused in "SELECT * FROM blocks WHERE block_index IN ((?,?,?,?,?)) ORDER BY block_index ASC" (1) SQLITE_ERROR

should be fixed with https://github.com/CounterpartyXCP/counterparty-core/commit/65992ec13c3d81fd608460f8f1f8f10483f9425c and https://github.com/CounterpartyXCP/counterparty-core/pull/1630


Support Collection

Add support Illuminate\Support\Collection type

Hi @niladam and @batFormat
First up apologies for the delay - haven't had much time for OS stuff the past couple of months.
@batFormat - your PR looks fine but could you just give me a quick overview of why it's necessary?  Just want to understand what problem is solves so I can update the docs if necessary

From the looks of it -- it allows you to pass a collection. This way, you can basically use some models' collection as the options :)

Sadly, i'm having issues with packagist but i'm planning on testing this today.

Actually I hadn't thought about this but this PR is actually a little behind the latest version of Nova Checkboxes.  In v 0.1.0 the way data is passed to and from the vue component was changed.  See: https://github.com/fourstacks/nova-checkboxes/releases/tag/v0.1.0
@batFormat if you still feel like this PR would be useful, could you update to explain what problem is solves specifically and then update your PR by basing it on v 0.1.0
@niladam - Please check out v 0.1.0 to see if that does what you need.  If you want to use some attribute of a model that returns a collection as the options for the checkboxes then you can already do this doing something like this:
Checkboxes::make('Something')
    ->options($model->attribute_that_returns_collection->all())

@fourstacks Thank you. I didn't see that. Anyways, i'm having some issues where i cannot check the boxes. I'm unsure what's causing it.
If i use it on a separate model i'm not having this issue. Are there any known compatibility issues ?

@niladam - would you mind opening a separate issue with some more details for the issue you're describing above - would prefer not to derail this PR with other issues. Thanks

In my case, $value has collection type.
Therefore, if $value is a collection there is a reason to get a simple array from the collection, I think this does not violate the logic of the package,

@batFormat - thanks for updating the PR and sorry for such a long delay in getting onto this.  The update will be in the next release.


[Feature Request] Add module settings for set on client side Metric and Imperial explicitily

I have a group where half use the interface with metric language (e.g. Germany) and the other half use imperial distance (e.g. USA), for all of them I would like to show both the option in meters and feet, with the current settings modules this is not possible.
Can you add client side settings modules to explicitly enable metric or imperial distance for the second label ?
Something like:
"Add metric distance as secondary measurement"
"Add imperial distance as secondary measurement"
If both are selected just check what the first measurement use and choice the other ?

Hmm i am not sure if i understand you completely.
What measurement do your scenes have? If they are feet (which would be imperial) and the metric conversions are activated, then everyone should see both imperial and metric (if you dont hide the foundry labels)

Sorry completely forgot about this.  The module already do that, i just need to sync the settigns between players.


Deprecate BUILDKITE_SECRETS_KEY

Related to #174, this deprecates the use of BUILDKITE_SECRETS_KEY

@lox can you review? I personally have used neither the KMS support, nor BUILDKITE_SECRETS_KEY


Missing Enable/Disable

Hi,
Nice plugin ;)
But missing important functionality of enabled/disabled.
Best Regards.

Hi @demonslord, thanks!
Its next point in plugin's roadmap
I will add this feature in a week.

I have implemented disabled items functionality and updated docs.
Write here If you found a bug or some missing functionality related to this issue.
Thanks!

@styopdev Thanks ;)


[Feature request] Jump to the previous VD

Now that we can jump to an arbitrary VD by number, it would help to have a way to jump back to where we came from :) Thx!

The function to switch to the last active desktop could perhaps be implement similar to this and that commit...

@Konfekt
what's the use case ? do you want alt+tab behavior but for virtual desktop ?
or do you want to store it in a history ? (instead of storing only the last active desktop)
 #Include %A_LineFile%\..\..\VD.ahk
 VD.RegisterDesktopNotifications()
 VD.CurrentVirtualDesktopChanged:=Func("CurrentVirtualDesktopChanged")
 CurrentVirtualDesktopChanged(desktopNum_Old, desktopNum_New) {
     ToolTip % desktopNum_Old ", " desktopNum_New
 }

I will implement it using this code
then you will just do:
 VD.goToDesktopNum(VD.lastDesktopNum)

Exactly, Single Alt+Tab for Desktops. Thank you very much, storing the last desktop in a variable on a desktop change and accessing it by a function bound to a keyboard shortcut, say Alt+Tab, achieves this.


[python-package] init_score causes the predictions different

Description

I want to use some prior predictions from other method as init_score and speed up training, but when setting this argument, predictions are quite different. I think I misunderstood this feature. How does it work?
Reproducible example

import pandas as pd
import lightgbm as lgb
from sklearn import datasets
from sklearn.linear_model import LinearRegression

diabetes = datasets.load_diabetes()
n, p = diabetes["data"].shape

def lgb_model(init_score):
    train_set = lgb.Dataset(
        data=diabetes["data"],
        label=diabetes["target"],
        feature_name=diabetes["feature_names"],
        init_score=init_score,
    )
    params = {
        "objective": "l2",
        "verbosity": -1,
        "seed": 0,
    }
    model = lgb.train(params, train_set)
    predictions = model.predict(diabetes["data"])
    return predictions

init_model = LinearRegression().fit(diabetes["data"], diabetes["target"])
results = pd.DataFrame(
    {
        "target": diabetes["target"],
        "init - None": lgb_model(None),
        "init - mean": lgb_model([diabetes["target"].mean()] * n),
        "init - linear": lgb_model(init_model.predict(diabetes["data"])),
    }
)

print(results.head())

And the results are:
   target  init - None  init - mean  init - linear
0   151.0   162.653224    10.519740     -39.368330
1    75.0    75.949063   -76.184421      13.700046
2   141.0   141.158077   -10.975407     -22.048918
3   206.0   217.419851    65.286367      56.010205
4   135.0   131.487873   -20.645611     -17.713920

After setting init_score from mean model or linear model, the results will change.
Environment info
LightGBM version or commit hash: 4.3.0
Command(s) you used to install LightGBM
pip install lightgbm

macOS Sonoma 14.1
Python 3.10.13
Conda 24.1.2
Additional Comments

Should I add the init_score to lgb prediction? It seems useful for this "l2" objective, but still weird for e.g. "tweedie".
import pandas as pd
import lightgbm as lgb
from sklearn import datasets
from sklearn.linear_model import LinearRegression

diabetes = datasets.load_diabetes()
n, p = diabetes["data"].shape

def lgb_model(init_score):
    train_set = lgb.Dataset(
        data=diabetes["data"],
        label=diabetes["target"],
        feature_name=diabetes["feature_names"],
        init_score=init_score,
    )
    params = {
        "objective": "tweedie",  # change to tweedie
        "verbosity": -1,
        "seed": 0,
    }
    model = lgb.train(params, train_set)
    predictions = model.predict(diabetes["data"])
    if init_score is not None:
        predictions += init_score. # add init_score
    return predictions

init_model = LinearRegression().fit(diabetes["data"], diabetes["target"])
results = pd.DataFrame(
    {
        "target": diabetes["target"],
        "init - None": lgb_model(None),
        "init - mean": lgb_model([diabetes["target"].mean()] * n),
        "init - linear": lgb_model(init_model.predict(diabetes["data"])),
    }
)

print(results.head())

   target  init - None  init - mean  init - linear
0   151.0   157.811490   152.952215     207.116677
1    75.0    75.164031   152.952215      69.071033
2   141.0   140.120412   152.952215     177.882790
3   206.0   223.799875   152.952215     167.914458
4   135.0   119.464777   152.952215     129.462258

Hey @oooo26, thanks for using LightGBM.

Should I add the init_score to lgb prediction?

Yes

It seems useful for this "l2" objective, but still weird for e.g. "tweedie"

You should be able to match the outputs of init_score=None if you set it to the same value that LightGBM starts the boosting from, which is different for each objective. For l2 is the mean of the target, but for tweedie is the log of that
https://github.com/microsoft/LightGBM/blob/28536a0a5d0c46598fbc930a63cec72399a969e4/src/objective/regression_objective.hpp#L468-L470

Thank you for helping!
Let me double check, for tweedie, I should set init_score as the log scale of initial predictions and, after fitting, still add the normal scale of them?

Oh I see. it should be like
model = lgb.train(params, train_set)
predictions = model.predict(diabetes["data"])
predictions = np.exp(np.log(predictions) + train_set.init_score)

It is a bit complex i think...but anyway, thank you!

Yes, some objectives have a ConvertOutput method, which is used to translate the raw scores (which the model is trained to predict) to the actual scores (the ones we're interested in). In the case of Tweedie it's the exp (as you've correctly done) https://github.com/microsoft/LightGBM/blob/5dfe7168d42898b66da3513eb8cab68ef2b23eeb/src/objective/regression_objective.hpp#L460-L462
So another alternative would be to always predict the raw score, add the init score (which will be on the same "scale"), and then apply that transformation. So the following would be equivalent but maybe easier since for each objective you'd only need to apply the ConvertOutput method.
    convert_output = np.exp  # for tweedie and poisson
    predictions = model.predict(diabetes["data"], raw_score=True)
    if init_score is not None:
        predictions += init_score
    return convert_output(predictions)

Closing since I believe the issue has been solved, feel free to reopen if you run into any more problems.


@sendgrid/mail doesnt support cc.

@sendgrid/mail doesnt support cc.
In below code, cc is not supported and showing following error "UnhandledPromiseRejectionWarning: Error: Bad Request".
                    const msg = {
                    from<EMAIL_ADDRESS>                    to: ['nithyakr@in.ibm.com',distContactEmail],
                    cc<EMAIL_ADDRESS>                   subject: 'Reminder: ' +doc.User_Programs+ ' Nomination'
                    html: message + nominationLink ,
                    };
                    sgMail.send(msg);

Recommend catching the rejection for more details why it was a bad request:
sgMail.send(msg).catch(function (error) {
     console.error(error.response.body);
 });


File Type Converter

Description of the new feature / enhancement
A context menu option for quickly converting (or copy/coverting) file types such as bmp to png or jpg, or mp3 to wav or ogg. Video files would likely be tricky and time consuming so I'd doubt that would be practicle. But for images and audio it would be amazing.
Scenario when this would be used?
As a game dev I'm constently converting files to different types for different reasons. I also know that my video editor program doesn't support certain audio types I find online and it'd be a very handy way to save time.
Supporting information
I do not any supporting information. But file conversion is something I've searched for many times online (esp for bulk conversions), and I'd doubt I'm the only one. That software exists for a reason.

Can it be added into PowerToys?

/dup #13276


Things blow up when you mix supplied and inferred keys

I believe this is the root cause of a bug in Greenruhm.
@warrenv

#23 related?

PR #24 seems to fix this.

Let's write a test that verifies this. =)


ERROR in ./src/polyfills.ts Module not found: Error: Can't resolve 'core-js/es6/reflect' in '/home/ec2-user/environment/aws-serverless-workshops/MultiRegion/2_UI/src' ERROR in ./src/polyfills.ts Module not found: Error: Can't resolve 'core-js/es7/reflect' in '/home/ec2-user/environment/aws-serverless-workshops/MultiRegion/2_UI/src'

Error while running npm run build command. Kindly help.

+1

that could help:
index 01e1361..c04cd0e 100644
--- a/MultiRegion/2_UI/package.json
+++ b/MultiRegion/2_UI/package.json
@@ -1,6 +1,6 @@
 {
   "name": "sxr-ui",
-  "version": "0.0.0",
+  "version": "0.1.0",
   "license": "MIT",
   "scripts": {
     "ng": "ng",
@@ -24,7 +24,7 @@
     "@swimlane/ngx-datatable": "^13.1.0",
     "aws-amplify": "^1.0.11",
     "aws-amplify-angular": "^2.0.3",
-    "core-js": "*",
+    "core-js": "2.6.0",
     "eslint": "^5.5.0",
     "ng-bootstrap": "*",
     "ngx-facebook": "*",```


Model hangs when warming up for tensorrt optimization

Description
ONNX with tensorrt optimization hangs/times out when launched with warmup.
Triton Information
What version of Triton are you using? r21.12
Are you using the Triton container or did you build it yourself? Using Triton container nvcr.io/nvidia/tritonserver:21.12-py3
To Reproduce
We have an image embedding network that we compiled from CLIP with ViT-b32 model.
We have a model config:
name: "clip_embedding_gpu"
platform: "onnxruntime_onnx"
max_batch_size: 128
input [{
  name: "IMAGE_PREPROCESSED"
  data_type: TYPE_FP32
  dims: [3, 224, 224]
}]
output [{
  name: "IMAGE_EMBEDDING"
  data_type: TYPE_FP16
  dims: [512]
}]

instance_group {
  kind: KIND_GPU
}
dynamic_batching {
  max_queue_delay_microseconds: 25
}
optimization {
  execution_accelerators {
    gpu_execution_accelerator: [
      {
        name: "tensorrt"
        parameters [
          {
            key: "precision_mode"
            value: "FP16"
          }
        ]
      }
    ]
  }
}

When starting up, we noticed that models were timing out for about 6~7 runs of perf_analyzer then it starts to produce results. We saw about 25% speedup.
Now, when we enable warm up:
model_warmup [{
  name: "warmup"
  batch_size: 128
  inputs: [{
    key: "IMAGE_PREPROCESSED"
    value: {
      data_type: TYPE_FP32
      dims: [3, 224, 224]
      zero_data: true
    }
  }]
}]

We saw a little delay in triton server startup, which is expected. However, perf_analyzer run now times out without every recovering.
Expected behavior
Once tritonserver started, perf_analyzer should show performance at an optimal level.

@chajath : Can you share your model so that we can try and repro this issue on our end? If you cannot share the exact model, a similar network with randomized weights also works provided it reproduces the issue.

Actually, it was just that model warmup was taking very very long, I think. In the new release I've managed to warm the model up. Closing this issue.


Document ACME account mismatch

We should add how to troubleshoot the error ACME server URL host and ACME private key registration host differ to the documentation.
Probably under https://cert-manager.io/docs/faq/acme/

/priority backlog


Multiple parameters causing API to generate no image and break in production

Hi everyone, I'm finding some errors with this API in production, particularly when I add a size parameter to the following URL:
This runs fine before I add the sizing:
https://ui-avatars.com/api/?name=Bob+Alice&background=0D8ABC&color=fff&rounded=true
This breaks when I add sizing:
https://ui-avatars.com/api/?name=Bob+Alice&background=0D8ABC&color=fff&rounded=true&size=256
This used to work before, but suddenly it's failing.
Here's the reverse:
This works with the sizing:
https://ui-avatars.com/api/?name=Bob+Alice&size=256&rounded=true
This breaks when I add the colors:
https://ui-avatars.com/api/?name=Bob+Alice&size=256&rounded=true&&background=0D8ABC&color=fff
I'm not entirely sure what's going on, but it's definitely strange.

Having same issues. Storage/cache issue?

Yeah, all my old ones seem to be running alright. Anything that had been built before is returning, it's new images that are breaking. I pulled the repo, but can't seem to find anything. My suspicion is that it's a cache issue, but will wait for @LasseRafn to chime in!

Happening to me as well. Noticed that it only breaks when color and/or background is added.
Works:
https://ui-avatars.com/api/?name=John+Doe&rounded=true&bold=true
Breaks:
https://ui-avatars.com/api/?name=John+Doe&rounded=true&bold=true&color=0D8ABC
https://ui-avatars.com/api/?name=John+Doe&rounded=true&bold=true&background=0D8ABC

@Valkyrurr - that's super interesting, because check this out:
https://ui-avatars.com/api/?name=John+Doe&background=0D8ABC&color=fff&rounded=true
If I move your rounded to the end, it's suddenly passing.

Woah. I'm not even using the rounded parameter in my current implementation. Some hex values just do not work though... e2430a for example. I'm pretty sure these values worked before, for atleast months on using this API.

Likewise, I'm feeling like @karlerss is probably on the money here, but hard to debug without being able to look at the logs. I'm sure @LasseRafn will know what's up (and thank you, of course for hosting this incredibly useful API).

Thats super strange! let me dig into it — thanks for doing all the debugging

Okay so as Issue #45 points out, its simply related to server storage. Every image is cached for a period of time — however as of now (5 minutes ago) we had 214.007.808 images stored (causing the storage to be full 😅
I'll be implementing a better cleanup job and in the meantime simply clean images more frequently!
Sorry

https://ui-avatars.com/api/Michele Vodaro/400/333941/FFFFFF stopped suddently for me and giving The image <> can not be displayed because it contain errors message. Image seems to be started loading again. Any inputs?

I'll be implementing a better cleanup job and in the meantime simply clean images more frequently!

Seems like that fixes it. Thank you, @LasseRafn !

Follow-up: I setup alerts for disk-space (and other key metrics) so this will be caught before it happens (if it does) in the future

Next up is:

cleaning job fixes
error monitoring
get git repo in sync with live

Thanks a ton! Looks great!

Hey @LasseRafn ! Can it be that issue generating new images is back?
This url works:
https://eu.ui-avatars.com/api/?name=JA&rounded=true&background=ffffff&color=1590E3&bold=true&font-size=0.65
But this ones doesn't:
https://eu.ui-avatars.com/api/?name=LA&rounded=true&background=ffffff&color=1590E3&bold=true&font-size=0.65


Smoke test when testing again relay node deployed by solo

Description
I was able to run smoke test pass when running again local launched hedera-local-node,
but could not get smoke test run when running again relay deployed by solo
Solo is using hedera-json-rpc-relay chart version 0.47.0
https://github.com/hashgraph/hedera-json-rpc-relay/tree/main/charts/hedera-json-rpc-relay
Steps to reproduce
These are steps to launch hedera network locally with solo
./test/e2e/setup-e2e.sh

solo node keys --gossip-keys --tls-keys --key-format pfx

solo node setup

solo node start
solo mirror-node deploy
solo relay deploy

kubectl port-forward svc/relay-node0-node1-node2-hedera-json-rpc-relay -n solo-e2e 7546:7546 &
kubectl port-forward svc/haproxy-node0-svc -n solo-e2e 50211:50211 &
kubectl port-forward svc/fullstack-deployment-hedera-explorer -n solo-e2e 8080:80 &

From terminal output as well as K9s I can tell the network /hedera nodes are running, mirror node are running

******************************* Solo *********************************************
Version			: 0.24.0
Kubernetes Context	: kind-solo-e2e
Kubernetes Cluster	: kind-solo-e2e
Kubernetes Namespace	: solo-e2e
**********************************************************************************
✔ Initialize [0.1s]
✔ Enable mirror-node [6s]
  ✔ Prepare address book [0.4s]
  ✔ Deploy mirror-node [5s]
✔ Check pods are ready [1m10s]
  ✔ Check Postgres DB [14s]
  ✔ Check REST API [40s]
  ✔ Check GRPC [52s]
  ✔ Check Monitor [1m10s]
  ✔ Check Importer [1m10s]
  ✔ Check Hedera Explorer [10s]

******************************* Solo *********************************************
Version			: 0.24.0
Kubernetes Context	: kind-solo-e2e
Kubernetes Cluster	: kind-solo-e2e
Kubernetes Namespace	: solo-e2e
**********************************************************************************
✔ Initialize [0.3s]
✔ Prepare chart values
✔ Deploy JSON RPC Relay [17s]
✔ Check relay is ready [26s]

When launched smoke test now, I got this output
  Smoke Tests
    Relay
      1) Should perform a crypto transfer transaction
    Websocket server
      2) "before all" hook for "Should receive events for subscribed contract"

  0 passing (3s)
  2 failing

  1) Smoke Tests
       Relay
         Should perform a crypto transfer transaction:
     ProviderError: [Request ID: 95f6bc18-0a17-40e0-818c-5b2fb33315c3] Error invoking RPC: queryA and queryAaaa ENODATA fullstack-deployment-rest
      at HttpProvider.request (/Users/jeffrey/hedera-local-node/test/smoke/node_modules/hardhat/src/internal/core/providers/http.ts:90:21)
      at processTicksAndRejections (node:internal/process/task_queues:95:5)
      at HardhatEthersProvider.getBalance (/Users/jeffrey/hedera-local-node/test/smoke/node_modules/@nomicfoundation/hardhat-ethers/src/internal/hardhat-ethers-provider.ts:171:21)
      at Context.<anonymous> (/Users/jeffrey/hedera-local-node/test/smoke/test/Smoke.js:17:39)

  2) Smoke Tests
       Websocket server
         "before all" hook for "Should receive events for subscribed contract":
     ProviderError: [Request ID: fd9de4af-46a1-43da-a47a-df4696f138ed] Unknown error invoking RPC
      at HttpProvider.request (/Users/jeffrey/hedera-local-node/test/smoke/node_modules/hardhat/src/internal/core/providers/http.ts:90:21)
      at processTicksAndRejections (node:internal/process/task_queues:95:5)
      at HardhatEthersProvider.getBlockNumber (/Users/jeffrey/hedera-local-node/test/smoke/node_modules/@nomicfoundation/hardhat-ethers/src/internal/hardhat-ethers-provider.ts:134:25)
      at HardhatEthersSigner.sendTransaction (/Users/jeffrey/hedera-local-node/test/smoke/node_modules/@nomicfoundation/hardhat-ethers/src/signers.ts:122:25)
      at ContractFactory.deploy (/Users/jeffrey/hedera-local-node/test/smoke/node_modules/ethers/src.ts/contract/factory.ts:111:24)
      at Context.<anonymous> (/Users/jeffrey/hedera-local-node/test/smoke/test/Smoke.js:39:16)

And in addition, here is log from relay
[2024-05-08 14:42:21.592 +0000] INFO (rpc-server/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96): [GET]: /health/readiness 200 0 ms                                                                   │
│ [2024-05-08 14:42:31.594 +0000] TRACE (relay-eth/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96): undefined chainId()                                                                                │
│ [2024-05-08 14:42:31.594 +0000] INFO (rpc-server/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96): [GET]: /health/readiness 200 0 ms                                                                   │
│ [2024-05-08 14:42:31.594 +0000] INFO (rpc-server/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96): [GET]: /health/liveness 200 0 ms                                                                    │
│ [2024-05-08 14:42:41.593 +0000] INFO (rpc-server/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96): [GET]: /health/liveness 200 0 ms                                                                    │
│ [2024-05-08 14:42:41.593 +0000] TRACE (relay-eth/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96): undefined chainId()                                                                                │
│ [2024-05-08 14:42:41.593 +0000] INFO (rpc-server/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96): [GET]: /health/readiness 200 0 ms                                                                   │
│ [2024-05-08 14:42:49.340 +0000] TRACE (relay-eth/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96): [Request ID: cec16d1c-bbb0-43b2-89db-c4f557f83fa1] chainId()                                        │
│ [2024-05-08 14:42:49.342 +0000] INFO (rpc-server/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96): [Request ID: cec16d1c-bbb0-43b2-89db-c4f557f83fa1] [POST]: eth_chainId 200 24 ms                    │
│ [2024-05-08 14:42:49.359 +0000] TRACE (relay-eth/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96): [Request ID: 86b8ba6d-d7fe-4216-8ef7-377a42280f48] getBalance(account=0xbF8F135840EaE2a4A3710b8F66c │
│ [2024-05-08 14:42:49.486 +0000] TRACE (mirror-node/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96):  Retry delay 250 ms on 'undefined'                                                                │
│ [2024-05-08 14:42:49.738 +0000] TRACE (mirror-node/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96):  Retry delay 500 ms on 'undefined'                                                                │
│ [2024-05-08 14:42:50.240 +0000] TRACE (mirror-node/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96):  Retry delay 750 ms on 'undefined'                                                                │
│ [2024-05-08 14:42:50.994 +0000] ERROR (mirror-node/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96): [Request ID: 86b8ba6d-d7fe-4216-8ef7-377a42280f48] [GET] accounts/0xbF8F135840EaE2a4A3710b8F66c98 │
│     err: {                                                                                │
│       "type": "Error",                                                                                │
│       "message": "queryA and queryAaaa ENODATA fullstack-deployment-rest",                                                                                │
│       "stack":                                                                                │
│           Error: queryA and queryAaaa ENODATA fullstack-deployment-rest                                                                                │
│               at MirrorNodeClient.handleError (/home/node/app/packages/relay/dist/lib/clients/mirrorNodeClient.js:235:31)                                                                                │
│               at MirrorNodeClient.<anonymous> (/home/node/app/packages/relay/dist/lib/clients/mirrorNodeClient.js:206:22)                                                                                │
│               at Generator.throw (<anonymous>)                                                                                │
│               at rejected (/home/node/app/packages/relay/dist/lib/clients/mirrorNodeClient.js:25:65)                                                                                │
│               at process.processTicksAndRejections (node:internal/process/task_queues:95:5)                                                                                │
│     }                                                                                │
│ [2024-05-08 14:42:50.994 +0000] ERROR (relay-eth/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96): [Request ID: 86b8ba6d-d7fe-4216-8ef7-377a42280f48] Error raised during getBalance for account 0xbF8 │
│     err: {                                                                                │
│       "type": "MirrorNodeClientError",                                                                                │
│       "message": "queryA and queryAaaa ENODATA fullstack-deployment-rest",                                                                                │
│       "stack":                                                                                │
│           Error: queryA and queryAaaa ENODATA fullstack-deployment-rest                                                                                │
│               at MirrorNodeClient.handleError (/home/node/app/packages/relay/dist/lib/clients/mirrorNodeClient.js:224:29)                                                                                │
│               at MirrorNodeClient.<anonymous> (/home/node/app/packages/relay/dist/lib/clients/mirrorNodeClient.js:206:22)                                                                                │
│               at Generator.throw (<anonymous>)                                                                                │
│               at rejected (/home/node/app/packages/relay/dist/lib/clients/mirrorNodeClient.js:25:65)                                                                                │
│               at process.processTicksAndRejections (node:internal/process/task_queues:95:5)                                                                                │
│       "statusCode": 567                                                                                │
│     }                                                                                │
│ [2024-05-08 14:42:50.994 +0000] ERROR (rpc-server/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96): [Request ID: 86b8ba6d-d7fe-4216-8ef7-377a42280f48] Error invoking RPC: queryA and queryAaaa ENODAT │
│ [2024-05-08 14:42:50.994 +0000] INFO (rpc-server/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96): [Request ID: 86b8ba6d-d7fe-4216-8ef7-377a42280f48] [POST]: eth_getBalance 500 1636 ms               │
│ [2024-05-08 14:42:51.009 +0000] TRACE (relay-eth/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96): [Request ID: 30396f04-d099-4e4c-9f7a-bad581aa1817] blockNumber()                                    │
│ [2024-05-08 14:42:51.011 +0000] TRACE (mirror-node/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96):  Retry delay 250 ms on 'undefined'                                                                │
│ [2024-05-08 14:42:51.263 +0000] TRACE (mirror-node/63 on relay-node0-node1-node2-hedera-json-rpc-relay-6c9c4768d9-tbv96):  Retry delay 500 ms on 'undefined'

Log from mirror node REST
│ (node:1) ExperimentalWarning: The Node.js specifier resolution flag is experimental. It could change or be removed at any time.                                                                                │
│ (Use `node --trace-warnings ...` to show where the warning was created)                                                                                │
│ 2024-05-08T17:51:48.763Z INFO Startup Loaded configuration source: /home/node/app/config/application.yml                                                                                │
│ 2024-05-08T17:51:48.767Z INFO Startup Loaded configuration source: /usr/etc/hedera/application.yaml                                                                                │
│ 2024-05-08T17:51:48.769Z INFO Startup Override config with environment variable HEDERA_MIRROR_REST_DB_USERNAME=mirror_rest                                                                                │
│ 2024-05-08T17:51:48.769Z INFO Startup Override config with environment variable HEDERA_MIRROR_REST_DB_HOST=fullstack-deployment-postgres-postgresql                                                                             │
│ 2024-05-08T17:51:48.769Z INFO Startup Override config with environment variable HEDERA_MIRROR_REST_DB_PASSWORD=******                                                                                │
│ 2024-05-08T17:51:48.769Z INFO Startup Override config with environment variable HEDERA_MIRROR_REST_DB_NAME=mirror_node                                                                                │
│ bigint: Failed to load bindings, pure JS will be used (try npm run rebuild?)                                                                                │
│ 2024-05-08T17:51:50.254Z INFO Startup Response compression is enabled                                                                                │
│ 2024-05-08T17:51:50.259Z INFO Startup stateproof REST API is disabled                                                                                │
│ 2024-05-08T17:51:50.261Z INFO Startup Server running on port: 5551

Additional context
No response
Hedera network
other
Version
latest
Operating system
macOS

Another question I have is that after started hedera-local-node,
mirror node log was flooded with following messages:
2024-05-08 15:33:34 2024-05-08T20:33:34.821Z INFO 7d400abf ::ffff:<IP_ADDRESS> GET /api/v1/transactions/0.0.2-1715200403-213000000 in 2 ms: 200
2024-05-08 15:33:34 2024-05-08T20:33:34.833Z INFO dc435999 ::ffff:<IP_ADDRESS> GET /api/v1/transactions/0.0.2-1715200403-113000000 in 2 ms: 200
2024-05-08 15:33:34 2024-05-08T20:33:34.871Z WARN daa708fb ::ffff:<IP_ADDRESS> GET /api/v1/transactions/0.0.2-1715200404-613000000 in 2 ms: 404 Not found NotFoundError Not found
2024-05-08 15:33:34 2024-05-08T20:33:34.884Z WARN fcd6b5e9 ::ffff:<IP_ADDRESS> GET /api/v1/transactions/0.0.2-1715200404-113000000 in 1 ms: 404 Not found NotFoundError Not found
2024-05-08 15:33:34 2024-05-08T20:33:34.916Z INFO 109ee28e ::ffff:<IP_ADDRESS> GET /api/v1/transactions/0.0.2-1715200403-513000000 in 2 ms: 200
2024-05-08 15:33:34 2024-05-08T20:33:34.925Z INFO e8d65c2e ::ffff:<IP_ADDRESS> GET /api/v1/transactions/0.0.2-1715200403-13000000 in 2 ms: 200
2024-05-08 15:33:34 2024-05-08T20:33:34.971Z WARN 5932c37b ::ffff:<IP_ADDRESS> GET /api/v1/transactions/0.0.2-1715200404-713000000 in 1 ms: 404 Not found NotFoundError Not found
2024-05-08 15:33:35 2024-05-08T20:33:35.071Z WARN f6bf32b4 ::ffff:<IP_ADDRESS> GET /api/v1/transactions/0.0.2-1715200404-813000000 in 1 ms: 404 Not found NotFoundError Not found
2024-05-08 15:33:35 2024-05-08T20:33:35.074Z INFO 289d8414 ::ffff:<IP_ADDRESS> GET /api/v1/transactions/0.0.2-1715200403-413000000 in 2 ms: 200
2024-05-08 15:33:35 2024-05-08T20:33:35.140Z WARN 34067086 ::ffff:<IP_ADDRESS> GET /api/v1/transactions/0.0.2-1715200404-213000000 in 2 ms: 404 Not found NotFoundError Not found
2024-05-08 15:33:35 2024-05-08T20:33:35.171Z WARN 36635584 ::ffff:<IP_ADDRESS> GET /api/v1/transactions/0.0.2-1715200404-913000000 in 2 ms: 404

Who sent out such requests and how to disabled, since some useful information maybe
buried by such flood of request.


Why Haven't I Met You? by Cameron Dallas unsupported.

Check if issue with swaglyrics or whether song lyrics unavailable on Genius.

Fixed.


New validation rule for Post operationIds

@dsgouda,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

Associated with https://github.com/Azure/autorest/issues/1959


Issue when reload the tile when coming from another page

Hi, I have an issue with my project. Actually, I have a map based on ngx-leaflet and ngx-leaflet-markercluster, and all works fine. My issue (or bug, I don't know why this happening) is this:  in my application I have two pages (login and map pages). If I log in, my app redirects me to map page and all works. However, if I logout and then start it again (or simply switch to a different page than the map page), the markers and clusters load well, but the tile remains grey. Only if I choose another tile in layers control the tile is loaded again. Any ideas?

[X] Tell us what version of Leaflet, Angular, and ngx-leaflet you're using:
Angular CLI: 1.7.1
Angular: 5.2.6
ngx-leaflet: 3.0.2
ngx-leaflet-markercluster: 1.0.0
leaflet: 1.3.1
[X] Include code that reproduces your issue

map.component.ts

import { Component, OnInit, NgZone } from '@angular/core';
import { MapService } from './services';
import * as L from 'leaflet';
import 'leaflet.markercluster';
import 'leaflet-extra-markers';
import * as icono from './iconos';
import { LeafletMarkerClusterModule } from '@asymmetrik/ngx-leaflet-markercluster';
import { Alteracion } from './dto/alteracion';
import { DialogModule } from 'primeng/dialog';

@Component({
  selector: 'map',
  templateUrl: './map.component.html',
  styleUrls: ['./map.component.scss']
})
export class MapComponent implements OnInit {

  public alteraciones: any[];
  public alteracion: any = {};
  display: boolean = false;

  markerClusterGroup: L.MarkerClusterGroup;
  markerClusterData: any[] = [];
  markerClusterOptions: L.MarkerClusterGroupOptions;

  leafletOptions: L.MapOptions = {
    zoom: 6,
    center: L.latLng(40.4166395, -3.7046087)
  };

  baseLayers: { [layerName: string]: L.Layer } = {
    'OSM': this.mapService.baseMaps.OSM,
    'Catastro': this.mapService.baseMaps.Catastro
  };
  layersControlOptions: L.ControlOptions = { position: 'topright' };

  constructor(private mapService: MapService, private zone: NgZone) { }

  ngOnInit() { }

  onMapReady(map: L.Map) {
    L.Icon.Default.imagePath = 'assets/img/theme/vendor/leaflet/';
    setTimeout(() => {
      map.invalidateSize();
    }, 0);

    this.mapService.getListAlteraciones(50, 297).subscribe(
      result => {
        this.alteraciones = result;
        this.generateData(map);
      },
      err => console.log(err)
    );
  }

  markerClusterReady(group: L.MarkerClusterGroup) {
    this.markerClusterGroup = group;
  }

  getInfoAlteracion(id_alteracion: string) {
    this.mapService.getInfoAlteracion(id_alteracion).subscribe(
      result => {
        this.alteracion = result;
        this.display = true;

      },
      err => console.log(err)
    );
  }
  generateData(map: L.map) {
    const data: any[] = [];
    let marker: any;

    L.geoJson(this.alteraciones, {
      pointToLayer: (feature, latlng) => {
        marker = L.marker(latlng, {
          icon: this.getIconMarker(feature.properties.tipo_alteracion)
        });

        marker.on('click', (e) => {
          this.zone.run(() => {
            this.getInfoAlteracion(feature.properties.id_alteracion);
          });
        });
        data.push(marker);
      }
    });

    map.on('zoomend', (e) => {
      if ( map.getZoom() > 16) {
        map.addLayer(this.mapService.baseMaps.Catastro);
      } else if ( map.getZoom() < 16) {
        map.addLayer(this.mapService.baseMaps.OSM);
        this.mapService.baseMaps.Catastro.remove();
      }
    });
    this.markerClusterData = data;
  }
    getIconMarker(t) {
      return (t === '01') ? icono.registradores : (t === '10') ? icono.alteracion_fisica : (t === '11')
      ? icono.contrato_traslativo : (t === '12') ? icono.donacion_transmision : (t === '13') ?
        icono.acto_urbanistico : (t === '14') ? icono.herencia : icono.otros;
    }
}
```

> map.component.html

```
<!-- Map -->
<div leaflet
     [leafletOptions]="leafletOptions"
     [leafletBaseLayers]="baseLayers"
     [leafletLayersControlOptions]="layersControlOptions"
     (leafletMapReady)="onMapReady($event)"
     [leafletMarkerCluster]="markerClusterData"
     [leafletMarkerClusterOptions]="markerClusterOptions"
     (leafletMarkerClusterReady)="markerClusterReady($event)">
</div>
```

If your map is being created and destroyed (through routing, or an ngIf, etc.), you need to recreate your baselayers each time the map is created. You can't reuse baselayers between different map instances. Once the layers are created, they are tied to that map instance.
I suspect that's what's happening here. They work the first time cause they go from never being tied to a map to being tied to a map. But, once you route away and then back, you're trying to reuse already created layers that have been bound to the original map.
The solution would be to store the baselayer configs in the map service, but recreate the baselayers each time the map is created (either in onMapReady or in ngOnInit depending on your use case).

Issue solved! Only adding this code now works despite the routing
    ngOnDestroy() {
      this.map.remove();
    }

@Aw3same I know this issue is closed but I'm having the exact same problem.
in my onMapReady i grab the instance of the map like so:
public onMapReady(map: L.Map) {
    // map is now loaded, add custom controls, access the map directly here
    this._map = map;
  }

however when I try to call .remove() in my ngOnDestroy like so:
ngOnDestroy() {
    this._map.remove();
  }

I get the following error in the console:
core.js:12301 ERROR Error: Uncaught (in promise): TypeError: Cannot read property 'remove' of undefined
TypeError: Cannot read property 'remove' of undefined
    at LeafletMapComponent.push../src/app/gis/components/leaflet-map/leaflet-map.component.ts.LeafletMapComponent.ngOnDestroy (leaflet-map.component.ts:25)
    at callProviderLifecycles (core.js:18639)
    at callElementProvidersLifecycles (core.js:18607)
    at callLifecycleHooksChildrenFirst (core.js:18597)
    at destroyView (core.js:19659)

Any ideas?

Issue solved! Only adding this code now works despite the routing
    ngOnDestroy() {
      this.map.remove();
    }

Hi, what you have in this.map? In the code above, I do not see such variable?

Hi vitomir! Sorry for the late answer.
In my this. map, I have :
map: L.map;

onMapReady(map: L.Map) {
	this.map = map;
       /*...*/
 }

Hope that helps!

Hi Angel,
Yes, it helps. Now I understand where this.map comes from. However, this does not fix my problem. My case is as follows:
I have a component with 2 tabs, map and list. In the map tab, I have filters and the map, where you can see all the vehicles on the map. When you click on a vehicle, it goes to details page and from the details page, you can go back. This works perfect, no grey parts on the map and it loads perfect. However, on the list page, I have filters and a list with the vehicle. When you click on a vehicle in the list, it will go again to the details page. When you go back, it will return you to the list tab of course, but when you go again to the map tab, then the map is not loaded correctly and almost all of it is in grey.
Do you have any ideas ? I tried everything what came up to my mind (and your solution) and nothing works ..


Fix types file

The change is replacing line 51 in index.d.ts
export function assert(assertion: Assertion<T>): <T>(assertion: Assertion<T>) => void
with
export type assert = <T>(assertion: Assertion<T>) => void

Released to npm as v7.1.0


[tls] add the fqdn containing the cluster.local to the service certs

Jira: OSPRH-3268

/test openstack-operator-build-deploy-kuttl

/test openstack-operator-build-deploy-kuttl

/test openstack-operator-build-deploy-kuttl


[BUG] MessageToPocoConverter overrides any custom ServiceBusReceivedMessage open converters

Library name and version
Microsoft.Azure.WebJobs.Extensions.ServiceBus 5.7.0
Describe the bug
Hey,
As in title, I'm trying to add my custom open converter for ServiceBusReceivedMessage:
...
context.AddOpenConverter<ServiceBusReceivedMessage, MessageEnvelope<OpenType>>(typeof(GenericMessageEnvelopeConverter<>));
...

also tried with OpenType.Poco
...
context.AddOpenConverter<ServiceBusReceivedMessage, MessageEnvelope<OpenType.Poco>>(typeof(GenericMessageEnvelopeConverter<>));
...

Here is my open converter:
internal class GenericMessageEnvelopeConverter<T> : IConverter<ServiceBusReceivedMessage, MessageEnvelope<T>>

Expected behavior
My custom open converter is called instead of MessageToPocoConverter.
Actual behavior
Every time when queue triggers:
 [FunctionName(nameof(ServiceBusStart))]
        public async Task ServiceBusStart(
            [ServiceBusTrigger("%queueName%", Connection = "SbConnString")] MessageEnvelope<MyCustomType> msg,
            [DurableClient] IDurableOrchestrationClient starter)

it calls MessageToPocoConverter - which is added in ServiceBusExtensionProvider  - well I am not 100% it calls MessageToPocoConverter, but I get this error  when using MessageEnvelope without parameterless constructor .
I believe that I've wired up everything correctly since when I change my custom converter from open to regular one:
...
context.AddConverter<ServiceBusReceivedMessage, int>(new GenericMessageEnvelopeConverter());
...

the converter is called as expected
Reproduction Steps
Create custom ServiceBusReceivedMessage open converter and trigger it with message.
Environment
local machine Windows 10, netcoreapp3.1, VS 2022

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

Hi @pposps,
I imagine that both your open converter and the library open converter would match, but the library converter is used because it is added first. Would it be possible to not use generics in your message type so that you can use a regular converter?
If that is not feasible, we may need to submit this as a feature request to the WebJobs repo to add more intelligent matching when using open converters.

Fair enough, in that case I will move this over to the WebJobs SDK for consideration.

Issue moved to Azure/azure-webjobs-sdk #2925 via Zenhub


Idea: combining with free nginx

Just some idea / brainstorm. Since free nginx was also recently created by another core developer.
Would it not be a good idea to combine the time & effort? I mean, I see a lot of overlap.
I personally don't know how good the relationship is between these two projects. But please consider it since I believe we more or less want to achieve the same together: a free and good nginx replacement.

Hi. We are definitely looking for cooperation. As soon there will be valuable changes in freenginx we'll be happy to integrate them in Angie as well.
But there actually not so many overlap. Having a lot of experience with Maxim in the past, I'd rather say that we have quite different views of nginx future. And I dare say, under his control there wouldn't be a half of features that Angie currently provides over nginx. Note that for the last 10+ years nginx was under almost full control of Maxim, and the current shape of nginx, like it or not, mostly obligates to his views.

Would it not be cool if you list some of those most important extensions that Angie delivers which isn't part of nginx/free and nginx on your Readme?

Thanks for giving me more insides and answering my questions. I will close it for now.


v2.4 Beta 1 Launching Issue

Hi, I downloaded the beta version on my iPad Air (iOS 9.3.5) but after trusting the device, it does not launch at all. Should I upgrade the software to iOS 10? Or is it working only on iPhones? Thanks!

Not working on iPhone either.

Not sure why it's not working for you   It is working on my iphone6 iOS 10.02.

Not working. Iphone 6s IOS 10.

Not working. Iphone SE IOS 10

I think there is a problem in the IPA package...
Someone can test compiling and run the app with Xcode ?

I can confirm this problem

PokeGO V2.4 is now available on the AppStore :+1:
https://itunes.apple.com/us/app/ipokego-for-pokemon-go/id1138277066?l=fr&ls=1&mt=8


Require TabMon Workbook 10.0v

Hello,
This is regarding case # 02693058.
We have captured Performance metrics in a CSV format for Tableau server. We are facing issues while trying to open the result CSV file in TabMon workbook.
As discussed in our previous call, one among the solutions is to check the version of TabMon workbook. Version needs to be same for Tableau desktop and TabMon. The version of TabMon that we are using here is 9.3 and We have Tableau desktop 10 installed . Below is the version line mentioned in the screenshot for your reference.
We have downloaded the latest version of TabMon from the  link provided (https://github.com/tableau/TabMon/blob/master/TabMon/Resources/Sample Workbooks/9.3 - 10.0/TabMon.twb) but we could not find 10.0 version in Git hub. Kindly provide us the link/url for 10.0v file.

Hello,
The 10.0 workbook is included in the TabMon install. To find it please replace the root place holder and go to the directory below.
\TabMon\Resources\Sample Workbooks\9.3 - 10.0
This directory will contain the 9.3-10.0 workbook.

The workbook in \TabMon\Resources\Sample Workbooks\9.3 - 10.0 is of version 9.3. We could not find version 10. Kindly help us on this

The workbook in "\TabMon\Resources\Sample Workbooks\9.3 - 10.0" is for version 9.3 to version 10
If you have the workbook from this location, you have the correct one for version 10

Hey kmohiddin,
Naiana-Tableau is correct. The workbook in that directory spans both versions of Tableau. If there are any further issues let me know.
Until then, I am closing this out.


Enforce Integer only value for Max-Age Cookie

Description
As per the HTTP standard guidelines, Max-Age for a Set-Cookie can only be an integer.

Max-Age= Optional
Number of seconds until the cookie expires. A zero or negative number will expire the cookie
immediately. Older browsers (ie6, ie7, and ie8) do not support max-age. For other browsers, if both > (Expires and Max-Age) are set, Max-Age will have precedence.

Current Cookie class provided by sanic has a fallback catch block that uses any value that you provide to max-age as a valid item and sets it.
Reference

encode method in Cookie class
Community Discussion Thread

Expected Behavior
If you pass a valid interger value to the max-age parameter, it needs to be set. Otherwise, it needs to be defaulted to -1 or 0 causing it to auto expire.

Related test:
https://github.com/huge-success/sanic/blob/cea1547e08230b6ad49eb7777fd8db5335382b7a/tests/test_cookies.py#L141-L156