Synthyra
/

ESMplusplus_small

Model card Files Files and versions

Bug in attention map computation

#3

by gionii - opened Mar 20, 2025

In the following line: https://huggingface.co/Synthyra/ESMplusplus_small/blob/main/modeling_esm_plusplus.py#L324, you are updating attention_mask rather than attn_bias which is actually used to mask attention values.

I am assuming you followed this template https://pytorch.org/docs/stable/generated/torch.nn.functional.scaled_dot_product_attention.html

Synthyra org Mar 20, 2025

Thanks for pointing this out! We have fixed the typo.

If you have any other questions or comments please let me know.
Best,
Logan

lhallee changed discussion status to closed Jun 11, 2025

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment