MTU recommendation about Azure VMs

[Enter feedback here]
In this article, it is mentioned Azure platform MTU is 1400 and out of order fragment will be droped due to Linux OS.
Should we recommend to set Azure VM MTU to 1400?
Any difference about TCP and UDP fragmentation processing in azure platform?
Azure and VM MTU
The default MTU for Azure VMs is 1,500 bytes. The Azure Virtual Network stack will attempt to fragment a packet at 1,400 bytes.
Note that the Virtual Network stack isn't inherently inefficient because it fragments packets at 1,400 bytes even though VMs have an MTU of 1,500. A large percentage of network packets are much smaller than 1,400 or 1,500 bytes.
Azure and fragmentation
Virtual Network stack is set up to drop "out of order fragments," that is, fragmented packets that don't arrive in their original fragmented order. These packets are dropped mainly because of a network security vulnerability announced in November 2018 called FragmentSmack.
FragmentSmack is a defect in the way the Linux kernel handled reassembly of fragmented IPv4 and IPv6 packets. A remote attacker could use this flaw to trigger expensive fragment reassembly operations, which could lead to increased CPU and a denial of service on the target system.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d9e17ce9-1b86-b744-f5c8-c86db7267376
Version Independent ID: 915a5706-f1ba-525e-a8dc-338574e3c3f7
Content: TCP/IP performance tuning for Azure VMs
Content Source: articles/virtual-network/virtual-network-tcpip-performance-tuning.md
Service: virtual-network
GitHub Login: @rimayber
Microsoft Alias: rimayber

I understand your point – but “probably not”. For a couple reasons.

We can’t control the internet path, so, unless you stay on microsoft network, then you’re likely to get fragmented somewhere along the way
Most packets are much smaller than 1400 or 1500 – so there’s no risk of fragmentation

It’s probably best to take a position that “fragmentation isn’t bad”. And, I think there’s been a fix recently where Azure will correctly re-assemble out of order fragments – but customers have to opt-in to it.
Thanks
From: Alex_Yin<EMAIL_ADDRESS>Sent: Tuesday, January 26, 2021 12:28 PM
To: MicrosoftDocs/azure-docs<EMAIL_ADDRESS>Cc: JR Mayberry<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: [MicrosoftDocs/azure-docs] MTU recommendation about Azure VMs (#69477)
[Enter feedback here]
In this article, it is mentioned Azure platform MTU is 1400 and out of order fragment will be droped due to Linux OS.
Should we recommend to set Azure VM MTU to 1400?
Any difference about TCP and UDP fragmentation processing in azure platform?
Azure and VM MTU
The default MTU for Azure VMs is 1,500 bytes. The Azure Virtual Network stack will attempt to fragment a packet at 1,400 bytes.
Note that the Virtual Network stack isn't inherently inefficient because it fragments packets at 1,400 bytes even though VMs have an MTU of 1,500. A large percentage of network packets are much smaller than 1,400 or 1,500 bytes.
Azure and fragmentation
Virtual Network stack is set up to drop "out of order fragments," that is, fragmented packets that don't arrive in their original fragmented order. These packets are dropped mainly because of a network security vulnerability announced in November 2018 called FragmentSmack.
FragmentSmack is a defect in the way the Linux kernel handled reassembly of fragmented IPv4 and IPv6 packets. A remote attacker could use this flaw to trigger expensive fragment reassembly operations, which could lead to increased CPU and a denial of service on the target system.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d9e17ce9-1b86-b744-f5c8-c86db7267376
Version Independent ID: 915a5706-f1ba-525e-a8dc-338574e3c3f7
Content: TCP/IP performance tuning for Azure VMshttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fvirtual-network%2Fvirtual-network-tcpip-performance-tuning%23azure-and-fragmentation&data=04|01|rimayber%40microsoft.com|7f1248fa823f4a261ab708d8c1bb1dd6|72f988bf86f141af91ab2d7cd011db47|1|0|637472356728791665|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=fybUeRJ5wzI3FOdpuIr7GJ6sdgNKhBOBOBXkA9llZW4%3D&reserved=0
Content Source: articles/virtual-network/virtual-network-tcpip-performance-tuning.mdhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fblob%2Fmaster%2Farticles%2Fvirtual-network%2Fvirtual-network-tcpip-performance-tuning.md&data=04|01|rimayber%40microsoft.com|7f1248fa823f4a261ab708d8c1bb1dd6|72f988bf86f141af91ab2d7cd011db47|1|0|637472356728801657|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=SZMPUakjIHKrfew7%2BTK8uST9GdmWCs0uwkGqzi3C0%2FY%3D&reserved=0
Service: virtual-network
GitHub Login: @rimayberhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frimayber&data=04|01|rimayber%40microsoft.com|7f1248fa823f4a261ab708d8c1bb1dd6|72f988bf86f141af91ab2d7cd011db47|1|0|637472356728806642|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=%2FOdzK%2FeHqRbzXBg78eZfYcI9Awc02ykO6HuZdvqRpSw%3D&reserved=0
Microsoft Alias: rimayber

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fissues%2F69477&data=04|01|rimayber%40microsoft.com|7f1248fa823f4a261ab708d8c1bb1dd6|72f988bf86f141af91ab2d7cd011db47|1|0|637472356728811633|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=SrSQHB4B7nXqvRMV5CXiM%2BtQvwXMo8mJPqBD129M8ms%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALVZVKPUDLGBFW6AGLSWOLDS3ZHFHANCNFSM4WS3XGAA&data=04|01|rimayber%40microsoft.com|7f1248fa823f4a261ab708d8c1bb1dd6|72f988bf86f141af91ab2d7cd011db47|1|0|637472356728816624|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=di%2BWjkNcCYycjRbda4640AMPgcQqQxWdgQp7BHgPvjg%3D&reserved=0.

I understand your point – but “probably not”. For a couple reasons.

We can’t control the internet path, so, unless you stay on microsoft network, then you’re likely to get fragmented somewhere along the way
Most packets are much smaller than 1400 or 1500 – so there’s no risk of fragmentation

It’s probably best to take a position that “fragmentation isn’t bad”. And, I think there’s been a fix recently where Azure will correctly re-assemble out of order fragments – but customers have to opt-in to it.
Thanks
From: Alex_Yin<EMAIL_ADDRESS>Sent: Tuesday, January 26, 2021 12:28 PM
To: MicrosoftDocs/azure-docs<EMAIL_ADDRESS>Cc: JR Mayberry<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: [MicrosoftDocs/azure-docs] MTU recommendation about Azure VMs (#69477)
[Enter feedback here]
In this article, it is mentioned Azure platform MTU is 1400 and out of order fragment will be droped due to Linux OS.
Should we recommend to set Azure VM MTU to 1400?
Any difference about TCP and UDP fragmentation processing in azure platform?
Azure and VM MTU
The default MTU for Azure VMs is 1,500 bytes. The Azure Virtual Network stack will attempt to fragment a packet at 1,400 bytes.
Note that the Virtual Network stack isn't inherently inefficient because it fragments packets at 1,400 bytes even though VMs have an MTU of 1,500. A large percentage of network packets are much smaller than 1,400 or 1,500 bytes.
Azure and fragmentation
Virtual Network stack is set up to drop "out of order fragments," that is, fragmented packets that don't arrive in their original fragmented order. These packets are dropped mainly because of a network security vulnerability announced in November 2018 called FragmentSmack.
FragmentSmack is a defect in the way the Linux kernel handled reassembly of fragmented IPv4 and IPv6 packets. A remote attacker could use this flaw to trigger expensive fragment reassembly operations, which could lead to increased CPU and a denial of service on the target system.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d9e17ce9-1b86-b744-f5c8-c86db7267376
Version Independent ID: 915a5706-f1ba-525e-a8dc-338574e3c3f7
Content: TCP/IP performance tuning for Azure VMshttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fvirtual-network%2Fvirtual-network-tcpip-performance-tuning%23azure-and-fragmentation&data=04|01|rimayber%40microsoft.com|7f1248fa823f4a261ab708d8c1bb1dd6|72f988bf86f141af91ab2d7cd011db47|1|0|637472356728791665|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=fybUeRJ5wzI3FOdpuIr7GJ6sdgNKhBOBOBXkA9llZW4%3D&reserved=0
Content Source: articles/virtual-network/virtual-network-tcpip-performance-tuning.mdhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fblob%2Fmaster%2Farticles%2Fvirtual-network%2Fvirtual-network-tcpip-performance-tuning.md&data=04|01|rimayber%40microsoft.com|7f1248fa823f4a261ab708d8c1bb1dd6|72f988bf86f141af91ab2d7cd011db47|1|0|637472356728801657|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=SZMPUakjIHKrfew7%2BTK8uST9GdmWCs0uwkGqzi3C0%2FY%3D&reserved=0
Service: virtual-network
GitHub Login: @rimayberhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frimayber&data=04|01|rimayber%40microsoft.com|7f1248fa823f4a261ab708d8c1bb1dd6|72f988bf86f141af91ab2d7cd011db47|1|0|637472356728806642|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=%2FOdzK%2FeHqRbzXBg78eZfYcI9Awc02ykO6HuZdvqRpSw%3D&reserved=0
Microsoft Alias: rimayber

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fissues%2F69477&data=04|01|rimayber%40microsoft.com|7f1248fa823f4a261ab708d8c1bb1dd6|72f988bf86f141af91ab2d7cd011db47|1|0|637472356728811633|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=SrSQHB4B7nXqvRMV5CXiM%2BtQvwXMo8mJPqBD129M8ms%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALVZVKPUDLGBFW6AGLSWOLDS3ZHFHANCNFSM4WS3XGAA&data=04|01|rimayber%40microsoft.com|7f1248fa823f4a261ab708d8c1bb1dd6|72f988bf86f141af91ab2d7cd011db47|1|0|637472356728816624|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=di%2BWjkNcCYycjRbda4640AMPgcQqQxWdgQp7BHgPvjg%3D&reserved=0.

thanks @rimayber.
I agree with "fragmentation is not bad", but still confusing with why Azure Virtual Network stack will attempt to fragment a packet at 1,400 bytes and then it might be dropped due to "out of order".
Given a more straightforward example:  If there is an TCP/IP packet from internet to Azure, it is 1460 bytes (payload length) + 20 bytes (TCP header) + 20 Bytes (IP header) = 1500 Bytes ,

so azure platform will fragment it to two pieces 1400 + 60 ?
If those two fragmented packets reach to destinaiton in azure  by "out of oder", it is possible to be dropped due to (Virtual Network stack is set up to drop "out of order fragments,") ?
if those "out of order" fragmented packets are coming from internet , I mean when they reach to azure destination, they will be dropped by azure due to the same reason?

thanks @rimayber.
I agree with "fragmentation is not bad", but still confusing with why Azure Virtual Network stack will attempt to fragment a packet at 1,400 bytes and then it might be dropped due to "out of order".
Given a more straightforward example:  If there is an TCP/IP packet from internet to Azure, it is 1460 bytes (payload length) + 20 bytes (TCP header) + 20 Bytes (IP header) = 1500 Bytes ,

so azure platform will fragment it to two pieces 1400 + 60 ?
If those two fragmented packets reach to destinaiton in azure  by "out of oder", it is possible to be dropped due to (Virtual Network stack is set up to drop "out of order fragments,") ?
if those "out of order" fragmented packets are coming from internet , I mean when they reach to azure destination, they will be dropped by azure due to the same reason?

Well, the math about 1400 vs 1500 has to consider virtual network tunneling protocols adding encapsulation from the host. Adds more to the packet size.
So, that is likely the reason for 1400.
But, I think your assumptions are correct - although if you're trying to get exact technical detail, then we should ask expand audience.
I believe this problem is specific to UDP.
RNM TSG (eng.ms)https://eng.ms/docs/cloud-ai-platform/azure/azure-core-networking/sdn/regional-network-manager/rnm-tsg/troubleshooting/enable-udp-fragment-reordering
From: Alex_Yin<EMAIL_ADDRESS>Sent: Tuesday, January 26, 2021 5:55 PM
To: MicrosoftDocs/azure-docs<EMAIL_ADDRESS>Cc: JR Mayberry<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [MicrosoftDocs/azure-docs] MTU recommendation about Azure VMs (#69477)
thanks @rimayberhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frimayber&data=04|01|rimayber%40microsoft.com|7b26682fc2d144e1eee908d8c1e8d9c8|72f988bf86f141af91ab2d7cd011db47|1|0|637472553128055439|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=i3WxXxKb9yHbMksG3e8v4ZmXQaJnsgjY1vXOxdg0bn0%3D&reserved=0.
I agree with "fragmentation is not bad", but still confusing with why Azure Virtual Network stack will attempt to fragment a packet at 1,400 bytes and then it might be dropped due to "out of order".
Given a more straightforward example: If there is an TCP/IP packet from internet to Azure, it is 1460 bytes (payload length) + 20 bytes (TCP header) + 20 Bytes (IP header) = 1500 Bytes ,

so azure platform will fragment it to two pieces 1400 + 60 ?
If those two fragmented packets reach to destinaiton in azure by "out of oder", it is possible to be dropped due to (Virtual Network stack is set up to drop "out of order fragments,") ?
if those "out of order" fragmented packets are coming from internet , I mean when they reach to azure destination, they will be dropped by azure due to the same reason?

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fissues%2F69477%23issuecomment-767464923&data=04|01|rimayber%40microsoft.com|7b26682fc2d144e1eee908d8c1e8d9c8|72f988bf86f141af91ab2d7cd011db47|1|0|637472553128060430|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=6eu5T8b1y2NFFOmCAGF%2FaxwnhatArs9W5rIqdLeZQrE%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALVZVKNQVSCNNN6TVDZRE6DS32NQ3ANCNFSM4WS3XGAA&data=04|01|rimayber%40microsoft.com|7b26682fc2d144e1eee908d8c1e8d9c8|72f988bf86f141af91ab2d7cd011db47|1|0|637472553128065419|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=FDwJAGHkAoJD7%2BbN5vEIm3kK1cwArDjeEk7xiJyV%2FzA%3D&reserved=0.

Well, the math about 1400 vs 1500 has to consider virtual network tunneling protocols adding encapsulation from the host. Adds more to the packet size.
So, that is likely the reason for 1400.
But, I think your assumptions are correct - although if you're trying to get exact technical detail, then we should ask expand audience.
I believe this problem is specific to UDP.
RNM TSG (eng.ms)https://eng.ms/docs/cloud-ai-platform/azure/azure-core-networking/sdn/regional-network-manager/rnm-tsg/troubleshooting/enable-udp-fragment-reordering
From: Alex_Yin<EMAIL_ADDRESS>Sent: Tuesday, January 26, 2021 5:55 PM
To: MicrosoftDocs/azure-docs<EMAIL_ADDRESS>Cc: JR Mayberry<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [MicrosoftDocs/azure-docs] MTU recommendation about Azure VMs (#69477)
thanks @rimayberhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frimayber&data=04|01|rimayber%40microsoft.com|7b26682fc2d144e1eee908d8c1e8d9c8|72f988bf86f141af91ab2d7cd011db47|1|0|637472553128055439|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=i3WxXxKb9yHbMksG3e8v4ZmXQaJnsgjY1vXOxdg0bn0%3D&reserved=0.
I agree with "fragmentation is not bad", but still confusing with why Azure Virtual Network stack will attempt to fragment a packet at 1,400 bytes and then it might be dropped due to "out of order".
Given a more straightforward example: If there is an TCP/IP packet from internet to Azure, it is 1460 bytes (payload length) + 20 bytes (TCP header) + 20 Bytes (IP header) = 1500 Bytes ,

so azure platform will fragment it to two pieces 1400 + 60 ?
If those two fragmented packets reach to destinaiton in azure by "out of oder", it is possible to be dropped due to (Virtual Network stack is set up to drop "out of order fragments,") ?
if those "out of order" fragmented packets are coming from internet , I mean when they reach to azure destination, they will be dropped by azure due to the same reason?

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fissues%2F69477%23issuecomment-767464923&data=04|01|rimayber%40microsoft.com|7b26682fc2d144e1eee908d8c1e8d9c8|72f988bf86f141af91ab2d7cd011db47|1|0|637472553128060430|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=6eu5T8b1y2NFFOmCAGF%2FaxwnhatArs9W5rIqdLeZQrE%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALVZVKNQVSCNNN6TVDZRE6DS32NQ3ANCNFSM4WS3XGAA&data=04|01|rimayber%40microsoft.com|7b26682fc2d144e1eee908d8c1e8d9c8|72f988bf86f141af91ab2d7cd011db47|1|0|637472553128065419|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=FDwJAGHkAoJD7%2BbN5vEIm3kK1cwArDjeEk7xiJyV%2FzA%3D&reserved=0.

Yes,  the out of order fragmentation could occur in azure cloud due to multi-path.  About TCP out of  fragmented packets, it seems we have reassable capability azure destination and rare issue is reported for that.
But for UDP out of oder fragmentated packets, we need to enable this UDP re-ordering flag for specific subscrition per region.
Just want to confirm if we have such difference for TCP and UDP, if yes, should we add more detailed description about azure drop "out of order" fragmentation packets?
Azure and fragmentation
Virtual Network stack is set up to drop "out of order fragments," that is, fragmented packets that don't arrive in their original fragmented order. These packets are dropped mainly because of a network security vulnerability announced in November 2018 called FragmentSmack.

Yes,  the out of order fragmentation could occur in azure cloud due to multi-path.  About TCP out of  fragmented packets, it seems we have reassable capability azure destination and rare issue is reported for that.
But for UDP out of oder fragmentated packets, we need to enable this UDP re-ordering flag for specific subscrition per region.
Just want to confirm if we have such difference for TCP and UDP, if yes, should we add more detailed description about azure drop "out of order" fragmentation packets?
Azure and fragmentation
Virtual Network stack is set up to drop "out of order fragments," that is, fragmented packets that don't arrive in their original fragmented order. These packets are dropped mainly because of a network security vulnerability announced in November 2018 called FragmentSmack.

Can you propose changes or make the updates?
From: Alex_Yin<EMAIL_ADDRESS>Sent: Tuesday, January 26, 2021 8:15 PM
To: MicrosoftDocs/azure-docs<EMAIL_ADDRESS>Cc: JR Mayberry<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [MicrosoftDocs/azure-docs] MTU recommendation about Azure VMs (#69477)
Yes, the out of order fragmentation could occur in azure cloud due to multi-path. About TCP out of fragmented packets, it seems we have reassable capability azure destination and rare issue is reported for that.
But for UDP out of oder fragmentated packets, we need to enable this UDP re-ordering flag for specific subscrition per region.
Just want to confirm if we have such difference for TCP and UDP, if yes, should we add more detailed description about azure drop "out of order" fragmentation packets?
Azure and fragmentation
Virtual Network stack is set up to drop "out of order fragments," that is, fragmented packets that don't arrive in their original fragmented order. These packets are dropped mainly because of a network security vulnerability announced in November 2018 called FragmentSmack.

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fissues%2F69477%23issuecomment-767533419&data=04|01|rimayber%40microsoft.com|532c41f049ac4c9b6fe608d8c1fc6534|72f988bf86f141af91ab2d7cd011db47|1|0|637472637062453624|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=91OF7unVs5h3GCSwVFGTL2MxGnEouhN9v6JCKiYqmwE%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALVZVKLEUCNJPS55PLVURHDS3255RANCNFSM4WS3XGAA&data=04|01|rimayber%40microsoft.com|532c41f049ac4c9b6fe608d8c1fc6534|72f988bf86f141af91ab2d7cd011db47|1|0|637472637062458621|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=sAoAy4OIlqXiv1PAKgkAEIJslZQULOLSUoqcQs5PAEU%3D&reserved=0.

Can you propose changes or make the updates?
From: Alex_Yin<EMAIL_ADDRESS>Sent: Tuesday, January 26, 2021 8:15 PM
To: MicrosoftDocs/azure-docs<EMAIL_ADDRESS>Cc: JR Mayberry<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [MicrosoftDocs/azure-docs] MTU recommendation about Azure VMs (#69477)
Yes, the out of order fragmentation could occur in azure cloud due to multi-path. About TCP out of fragmented packets, it seems we have reassable capability azure destination and rare issue is reported for that.
But for UDP out of oder fragmentated packets, we need to enable this UDP re-ordering flag for specific subscrition per region.
Just want to confirm if we have such difference for TCP and UDP, if yes, should we add more detailed description about azure drop "out of order" fragmentation packets?
Azure and fragmentation
Virtual Network stack is set up to drop "out of order fragments," that is, fragmented packets that don't arrive in their original fragmented order. These packets are dropped mainly because of a network security vulnerability announced in November 2018 called FragmentSmack.

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fissues%2F69477%23issuecomment-767533419&data=04|01|rimayber%40microsoft.com|532c41f049ac4c9b6fe608d8c1fc6534|72f988bf86f141af91ab2d7cd011db47|1|0|637472637062453624|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=91OF7unVs5h3GCSwVFGTL2MxGnEouhN9v6JCKiYqmwE%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALVZVKLEUCNJPS55PLVURHDS3255RANCNFSM4WS3XGAA&data=04|01|rimayber%40microsoft.com|532c41f049ac4c9b6fe608d8c1fc6534|72f988bf86f141af91ab2d7cd011db47|1|0|637472637062458621|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=sAoAy4OIlqXiv1PAKgkAEIJslZQULOLSUoqcQs5PAEU%3D&reserved=0.

According to the description of  "Azure and fragmentation" part, the "out of order" fragments will be dropped.
Is it partially untrue?
If it is TCP "out of order" fragmented packets, azure destination can reassemable them and will not drop them.
If it is UDP "out of order" fragmented packets,  it is by design azure destination will drop them. But we have option to enable "enable-udp-fragment-reordering" to reassemable those packets to avoid dropping.

According to the description of  "Azure and fragmentation" part, the "out of order" fragments will be dropped.
Is it partially untrue?
If it is TCP "out of order" fragmented packets, azure destination can reassemable them and will not drop them.
If it is UDP "out of order" fragmented packets,  it is by design azure destination will drop them. But we have option to enable "enable-udp-fragment-reordering" to reassemable those packets to avoid dropping.

Does this enable-udp-fragment-reordering option really exist? The only place I have found mention of this option is indeed in this thread? but nowhere else.

Does this enable-udp-fragment-reordering option really exist? The only place I have found mention of this option is indeed in this thread? but nowhere else.

It does, I had to ask for it to resolve an issue with a UDP app. The support person wasn't sure about it until I linked this thread and was able to get rimayber to help.
It is weird, but I don't think MS wants to accept that fragmenting UDP results in tons of out of order packets on the other end, screwing with some apps. I wish they didn't bother fragmenting anything below 1400, seems like a great way to create weird problems.

wish azure has exposed this enable-udp-fragment-reordering option via azure interface by default at subscription level such that folks are aware of it's consequences, we have been facing quite a lot of stability issues because of this issue where UDP encapsulated fragmented packets are getting dropped and we stumbled upon this github issue and it helped us resolve our issue
sharing the exact issue details here for others reference,

@hotgore  could you please provide a bit information how to set this "enable-udp-fragment-reordering" option ? Troubleshooting a Radius EAP-TLS Authentication issue and seems it's related with IPSec to Azure and fragmented UDP datagram lost.

Enable UDP Fragment Reordering on a Subscription - Overview (visualstudio.com)https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsupportability.visualstudio.com%2FAzureNetworking%2F_wiki%2Fwikis%2FANPTA%2520Wiki%2F444499%2FEnable-UDP-Fragment-Reordering-on-a-Subscription&data=04|01|rimayber%40microsoft.com|8ae92e13d49b4634e2f408d9efc92af4|72f988bf86f141af91ab2d7cd011db47|1|0|637804469675622068|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=0HdBWLlNsEtn6m3ScuyjWOmJBQMeyfgwezKCLpUMnWM%3D&reserved=0
Sent from Outlookhttp://aka.ms/weboutlook
From: Jack Chen @.>
Sent: Friday, March 11, 2022 7:47 AM
To: MicrosoftDocs/azure-docs @.>
Cc: JR Mayberry @.>; Mention @.>
Subject: Re: [MicrosoftDocs/azure-docs] MTU recommendation about Azure VMs (#69477)
@hotgorehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fhotgore&data=04|01|rimayber%40microsoft.com|45d5cf23d5d842a8855c08da02f8a7da|72f988bf86f141af91ab2d7cd011db47|1|0|637825564240951413|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=T2PyKMEBiRca%2B4uewifBJ5ujgmA%2B5pkUM8WBRg9pMdM%3D&reserved=0 could you please provide a bit information how to set this "enable-udp-fragment-reordering" option ? Troubleshooting a Radius EAP-TLS Authentication issue and seems it's related with IPSec to Azure and fragmented UDP datagram lost.

Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fissues%2F69477%23issuecomment-1064659272&data=04|01|rimayber%40microsoft.com|45d5cf23d5d842a8855c08da02f8a7da|72f988bf86f141af91ab2d7cd011db47|1|0|637825564240951413|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=BSan9hI%2BRh6%2Flcm6fzEfZpi1k3f1NenxOcN9yJWiz6g%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALVZVKLCVMQJ3JWKKPUAW33U7KJYLANCNFSM4WS3XGAA&data=04|01|rimayber%40microsoft.com|45d5cf23d5d842a8855c08da02f8a7da|72f988bf86f141af91ab2d7cd011db47|1|0|637825564240951413|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=iR3je9UjxLHyrelWiqXHxs3vHiM5nmOANvIuhascI5Q%3D&reserved=0.
Triage notifications on the go with GitHub Mobile for iOShttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04|01|rimayber%40microsoft.com|45d5cf23d5d842a8855c08da02f8a7da|72f988bf86f141af91ab2d7cd011db47|1|0|637825564240951413|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=br82G71yCGvJvMXd7lRzbVAgJ8SSO1lYXEB1pF5rX0o%3D&reserved=0 or Androidhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04|01|rimayber%40microsoft.com|45d5cf23d5d842a8855c08da02f8a7da|72f988bf86f141af91ab2d7cd011db47|1|0|637825564240951413|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=fv1yXMfjAdAYEp%2BI4Yj78u4QWsZ%2FUZYlUMuhjm38pYM%3D&reserved=0.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

Thanks @rimayber , I tried https://supportability.visualstudio.com/AzureNetworking/_wiki/wikis/ANPTA Wiki/444499/Enable-UDP-Fragment-Reordering-on-a-Subscription but I don't have access.

Basically you need a support ticket to request it. The doc link is for internal support employees
Sent from Outlookhttp://aka.ms/weboutlook
From: Jack Chen @.>
Sent: Friday, March 11, 2022 10:09 AM
To: MicrosoftDocs/azure-docs @.>
Cc: JR Mayberry @.>; Mention @.>
Subject: Re: [MicrosoftDocs/azure-docs] MTU recommendation about Azure VMs (#69477)
Thanks @rimayberhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frimayber&data=04|01|rimayber%40microsoft.com|5e459c0376584e6eef7b08da030c76d7|72f988bf86f141af91ab2d7cd011db47|1|0|637825649315882223|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=IcwvTr%2BVja4TIvxNn5hvyLRb%2FZyeYjz0zX%2FliU%2FdDFY%3D&reserved=0 , I tried https://supportability.visualstudio.com/AzureNetworking/_wiki/wikis/ANPTA Wiki/444499/Enable-UDP-Fragment-Reordering-on-a-Subscriptionhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsupportability.visualstudio.com%2FAzureNetworking%2F_wiki%2Fwikis%2FANPTA%2520Wiki%2F444499%2FEnable-UDP-Fragment-Reordering-on-a-Subscription&data=04|01|rimayber%40microsoft.com|5e459c0376584e6eef7b08da030c76d7|72f988bf86f141af91ab2d7cd011db47|1|0|637825649315882223|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=J5%2BLduVOxDMCGcTHdrHSxxotaBdqeKQDNWxQlNlxzoY%3D&reserved=0 but I don't have access.

Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fissues%2F69477%23issuecomment-1064727597&data=04|01|rimayber%40microsoft.com|5e459c0376584e6eef7b08da030c76d7|72f988bf86f141af91ab2d7cd011db47|1|0|637825649315882223|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=PuemR7NNFjqfEzCoKof2CdkHuQVmvOthoLIyAUMVeiE%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALVZVKO6OQNXU2QSO3IM4K3U7K2MDANCNFSM4WS3XGAA&data=04|01|rimayber%40microsoft.com|5e459c0376584e6eef7b08da030c76d7|72f988bf86f141af91ab2d7cd011db47|1|0|637825649315882223|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=05FlDF1gVP9RxLJRzgF%2FIyxES7ytMgTmV7hiOaXv0ww%3D&reserved=0.
Triage notifications on the go with GitHub Mobile for iOShttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04|01|rimayber%40microsoft.com|5e459c0376584e6eef7b08da030c76d7|72f988bf86f141af91ab2d7cd011db47|1|0|637825649315882223|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=ZAjyNgUhgcqPYttLXwPbwt712OGr2IorKy8imkMpg1o%3D&reserved=0 or Androidhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04|01|rimayber%40microsoft.com|5e459c0376584e6eef7b08da030c76d7|72f988bf86f141af91ab2d7cd011db47|1|0|637825649315882223|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000&sdata=u33Lvv2SorLsepLH6N5JPQ4TA6kbzJSq95mQ0agUm90%3D&reserved=0.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

Basically you need a support ticket to request it. The doc link is for internal support employees

I open a support ticket for "enable-udp-fragment-reordering".
So far support allowed me to "enable-udp-fragment-reordering" only on the VM Public IP inbound Traffic. When using an Azure Load Balancer or a VPN via a Virtual Network Gateway, the fragment are still dropping.

I was just informed of this discussion, and I would tend to disagree with "fragmentation is not bad." It causes a whole host of issues, for which the entire QUIC protocol (built on UDP and the really the next protocol for the internet) has taken the stance to set the "Don't Fragment" bit to make sure fragmentation doesn't happen.
But to that end, does the Azure network stack fully respect this bit? Or is there some virtualization layer that does some kind of fragmentation regardless? If so, that's really bad.
BTW, I don't understand the reasoning for this claim: "Most packets are much smaller than 1400 or 1500 – so there’s no risk of fragmentation". Generally, smart protocols will dynamically discover the largest MTUs that "work". But if the network fragments the larger MTUs it ends up causing more problems...

@nibanks I agree. These statements do not make sense:

"fragmentation is not bad."
"Most packets are much smaller than 1400 or 1500 – so there’s no risk of fragmentation"

In the same sense have commented an issue which was closed without any sound reasoning:
https://github.com/MicrosoftDocs/azure-docs/issues/65888#issuecomment-1296894484

...the entire QUIC protocol ... has taken the stance to set the "Don't Fragment" bit to make sure fragmentation doesn't happen.
Are you sure? The "Don't Fragment" bit is normally being set by the operating system as part of the Path MTU Discovery algorithm.

Are you sure? The "Don't Fragment" bit is normally being set by the operating system as part of the Path MTU Discovery algorithm.

About what part? The OS is involved in TCP based communication and may use the Don't Fragment bit when doing PMTUD, yes. When it's done in QUIC, the OS isn't necessarily involved in anything more than providing a UDP socket. Then, it's the app layer that's telling the UDP socket to set the Don't Fragment bit.

@fedda-no I had same problem using FortiAuthenticator as Radius server, Fortigate as site2site VPN server in Azure. EAP-TLS cert based authentication some time work, sometime not, reason is fragmented UDP datagram got lost. Fortigate had a "set ip-fragmentation pre-encapsulation" option and that ( plus reduce MTU) fixed the problem for me.

UDP Fragmentation is a pain in Azure, I have special a problem with EAP-TLS in Radius UDP packet:
1. Standard Azure VNet drops all EAP TLS where UDP packet get fragmented (NAS to Radius) when the NAS encapsulate the client certificate in the UDP packet (it needs to be fragmented), Azure drops this (NAS logs it with: Radius Server Dead, Radius server: Not Response from NAS).

* This is for ALL radius traffic traversing Azure, example if you use Azure VWAN, Site to Azure, and Azure Express to Onprem DC (where Radius is hosted), UDP packet get "dropped".

2. Initial fix for this is to:

   * Get Azure to enable enable-udp-fragment-reordering.

3. However I still have problem if I host example Cisco ISE 3.2 in Azure, where I experience that Azure Vnet drops the "server certificate response", again, Cisco ISE 3.2 deployed from marked space are setup with standard MTU of 1500, where we see randomly that Azure still dropping the radius response from Cisco ISE. NAS reports: Radius Server dead jump to next radius server in list, ISE Response to the next request: Authentication process already in process and drops the request.  But this problem are random, not consistent as [From private repo #1](https://github.com/MicrosoftDocs/azure-docs/pull/1).

* Fix in progress, trying to change the MTU on the Cisco ISE server in Azure to 1400, to prevent any form for fragmentation to occur on the Radius response (special when Radius has to encapsulate the server certificate in response = needs to fragment the packet, not 100% sure if the resolves the problem.

So a final rule of thumb, running 802.1x EAP traffic, Radius server in Azure or sending 802.1x radius traffic through Azure you have to be very, very careful, due to the UDP fragmentation issue / rules in Azure breaking the RFC.
Microsoft should make sure that Azure vNet follows the RFC, and not blame it on some "security reason here and there" breaking the entire UDP stack. If they have a problem with the RFC in the UDP protocol, then work on changing the RFC.

Did changing the MTU fix the issue?

How is this issue resolved?
On Fri, Mar 17, 2023 at 7:57 PM Allen Sudbring @.***>
wrote:

Thank you for you dedication to our documentation.
Unfortunately, we have been unable to review this issue in a timely
manner. We sincerely apologize for the delayed response. We are closing
this issue. If you feel that the problem persists, please respond to this
issue with additional information.
Please continue to provide feedback about the documentation. We appreciate
your contributions to our community.
#please-close
—
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/azure-docs/issues/69477#issuecomment-1474277318,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA74HIPEBNRDAGTYQX46K2LW4SX3BANCNFSM4WS3XGAA
.
You are receiving this because you commented.Message ID:
@.***>

Can you enlighten me as to how exactly this issue is "resolved"?
On Fri, Mar 17, 2023 at 7:57 PM Allen Sudbring @.***>
wrote:

Thank you for you dedication to our documentation.
Unfortunately, we have been unable to review this issue in a timely
manner. We sincerely apologize for the delayed response. We are closing
this issue. If you feel that the problem persists, please respond to this
issue with additional information.
Please continue to provide feedback about the documentation. We appreciate
your contributions to our community.
#please-close
—
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/azure-docs/issues/69477#issuecomment-1474277318,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA74HIPEBNRDAGTYQX46K2LW4SX3BANCNFSM4WS3XGAA
.
You are receiving this because you commented.Message ID:
@.***>

In my case, you have to triple check that Azure Support actually have enable in the vnet / vwan setup. My problem was that Azure support said it was, when in reality it wasn’t, and after getting another SME they finally got it enable correctly. Get Azure Support to enable enable-udp-fragment-reorderingThen you do not need to play around with mtu size etc…Regards,Fredrik E. Jensen17. mar. 2023 kl. 20:01 skrev gideonoost @.***>:
Can you enlighten me as to how exactly this issue is "resolved"?
On Fri, Mar 17, 2023 at 7:57 PM Allen Sudbring @.***>
wrote:

Thank you for you dedication to our documentation.
Unfortunately, we have been unable to review this issue in a timely
manner. We sincerely apologize for the delayed response. We are closing
this issue. If you feel that the problem persists, please respond to this
issue with additional information.
Please continue to provide feedback about the documentation. We appreciate
your contributions to our community.
#please-close
—
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/azure-docs/issues/69477#issuecomment-1474277318,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA74HIPEBNRDAGTYQX46K2LW4SX3BANCNFSM4WS3XGAA
.
You are receiving this because you commented.Message ID:
@.***>

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

Issue is not resolved.
Funny that Microsoft requires customers to ask for a secret flag to fix their broken network design. Yeah, build the network on GRE in a way so customers can't use GRE themselves, keep the MTU default so you have to fragment, and then take a stateless protocol, fragment it and deliver the fragments out of order. No one in the world would think that fragmenting UDP packets and delivering them out of order is a good idea. When customers complain their only option is the secret code word whereby the customer can't actually verify it is enabled. This idiocy will only get worse as more and more protocols use UDP (QUIC).

I'm having a problem with an RTSP server, which emits RTP payload at maximum 1472 bytes. This renders to max 1514 bytes on ETH, surely to big for MTU 1500. I traced the output of the Azure network interface I have access to and found, that all UDP packages left with Don't fragment bit set. In result the RTSP client couldn't successfully fetch RTSP via UDP, while TCP/TLS worked fine.
I managed to convince the nice developer to provide me a means to reduce the RTP payload size. And already the first attempt was a success (payload 1400). I think the max RTP payload is somewhat like 1458, wich in turn would render to ETH 1500. Don't fragment is still not set, but the packages "come through" now.
Full story https://github.com/aler9/rtsp-simple-server/discussions/1588
Not sure, why everybody says Azure fragments at 1400...

UDP Fragmentation is a pain in Azure, I have special a problem with EAP-TLS in Radius UDP packet:

Standard Azure VNet drops all EAP TLS where UDP packet get fragmented (NAS to Radius) when the NAS encapsulate the client certificate in the UDP packet (it needs to be fragmented), Azure drops this (NAS logs it with: Radius Server Dead, Radius server: Not Response from NAS).

This is for ALL radius traffic traversing Azure, example if you use Azure VWAN, Site to Azure, and Azure Express to Onprem DC (where Radius is hosted), UDP packet get "dropped".

Initial fix for this is to:

Get Azure to enable enable-udp-fragment-reordering.

However I still have problem if I host example Cisco ISE 3.2 in Azure, where I experience that Azure Vnet drops the "server certificate response", again, Cisco ISE 3.2 deployed from marked space are setup with standard MTU of 1500, where we see randomly that Azure still dropping the radius response from Cisco ISE. NAS reports: Radius Server dead jump to next radius server in list, ISE Response to the next request: Authentication process already in process and drops the request.  But this problem are random, not consistent as From private repo #1.

Fix in progress, trying to change the MTU on the Cisco ISE server in Azure to 1400, to prevent any form for fragmentation to occur on the Radius response (special when Radius has to encapsulate the server certificate in response = needs to fragment the packet, not 100% sure if the resolves the problem.

So a final rule of thumb, running 802.1x EAP traffic, Radius server in Azure or sending 802.1x radius traffic through Azure you have to be very, very careful, due to the UDP fragmentation issue / rules in Azure breaking the RFC.
Microsoft should make sure that Azure vNet follows the RFC, and not blame it on some "security reason here and there" breaking the entire UDP stack. If they have a problem with the RFC in the UDP protocol, then work on changing the RFC.

for me... your 'fix in progress' is what resolve our auth issues. Setting the MTU on the ISE node to 1300 did the trick.
initial testing is good and as long as nothign breaks with the mtu at 1300 then we will leave it as that. We only use ISE for 802.1x.


DatabricksSubmitRunOperatorAsync fails with KeyError: 'state'

Describe the bug
While trying to use DatabricksSubmitRunOperatorAsync on an existing Databricks Cluster, encountered this error:
{taskinstance.py:1641} ERROR - Trigger failed:
Traceback (most recent call last):

  File "/usr/local/lib/python3.9/site-packages/airflow/jobs/triggerer_job.py", line 296, in cleanup_finished_triggers
    result = details["task"].result()

  File "/usr/local/lib/python3.9/site-packages/airflow/jobs/triggerer_job.py", line 360, in run_trigger
    async for event in trigger.run():

  File "/usr/local/lib/python3.9/site-packages/astronomer/providers/databricks/triggers/databricks.py", line 72, in run
    run_state = await hook.get_run_state_async(self.run_id)

  File "/usr/local/lib/python3.9/site-packages/astronomer/providers/databricks/hooks/databricks.py", line 44, in get_run_state_async
    state = response["state"]

KeyError: 'state'

If I try to use the regular non-async DatabricksSubmitRunOperator, it works just fine.
Environment Details:

Local Airflow on astro-runtime:5.0.8
Field Engineering Databricks Spark Cluster
OpenLineage Enabled

To Reproduce
Steps to reproduce the behavior:

Spin-up local Airflow Environment using astro with runtime 5.0.8
Copy the following DAG:

from airflow import DAG
from airflow.decorators import task
from datetime import datetime

from astronomer.providers.databricks.operators.databricks import DatabricksSubmitRunOperatorAsync
from airflow.providers.databricks.operators.databricks import DatabricksSubmitRunOperator

default_args = {
    'owner': 'Cosmic Energy DE',
}

with DAG('test_aws'
        , start_date=datetime(2022,8,18)
        , catchup=False
        , max_active_runs=1
        , schedule_interval='@daily'
        , default_args=default_args
        , tags = ['transform', 'daily', 's3', 'snowflake', 'lineage'],
    ) as dag:

    task_databricks_process=DatabricksSubmitRunOperatorAsync(
        task_id="databricks_process",
        json={
            "existing_cluster_id" : '0810-130606-yjnn4pqp',
            "notebook_task": {
                "notebook_path" : "/Shared/ghcnd-stations",
                "base_parameters": {
                    "access_key": "{{ var.value.fe_access_key }}",
                    "secret_key": "{{ var.value.fe_secret_key }}",
                    "execution_date": "{{ ds_nodash }}",
                    "file_name": "ghcnd-stations"
                }
            },
            "timeout_seconds": 600
        }
    )

   task_databricks_process

Go to localhost:8080
Enable the DAG and it will fail
Replace DatabricksSubmitRunOperatorAsync with DatabricksSubmitRunOperator and it will run

Expected behavior
Failed Databricks Task when using the Async Operator
Screenshots
N/A
Desktop (please complete the following information):

OS: MacOS
Browser Chrome
12.5.1

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Logs for failed and successful run attached herewith:
SUCCESS_DatabricksSubmitRunOperator.log
FAILED_DatabricksSubmitRunOperatorAsync.log

We should release 1.8.1 that contains this fix -- cc @phanikumv @rajaths010494

I tried to debug it but was unable to reproduce it.
What version of databricks sync provider you are using and what is the value wait_for_termination param?
Also, would it be possible for you to share the trigger log?
For now, I have made the changes in DatabricksSubmitRunOperatorAsync to propagate the trigger log to the worker.  https://github.com/astronomer/astronomer-providers/pull/612
Also, we run https://github.com/astronomer/astronomer-providers/blob/main/astronomer/providers/databricks/example_dags/example_databricks.py in our integration test it runs fine would you like to try it in your env?

@manmeetkaur did you get a chance to try the example DAG given above in your env?

@manmeetkaur please confirm if you got a chance to try the example DAG in your env.

Closing this as we haven't received any reply. Please feel free to re-open in case of any more issues.


Fix unsupported ScalarType BFloat16

Fix/ workaround to error:
TypeError: Got unsupported ScalarType BFloat16

Yes, you will need at least 32GB of memory for the 7B model.

For reference there is an issue created in PyTorch https://github.com/pytorch/pytorch/issues/90574

The bfloat16 uses 7 bits for precision and 8 bits for range; the float16 uses 10 bits for precision and 5 bits for range, meaning that it is actually more precise than the bfloat16. Fortunately, the llama2 weights fall within the float16 range, so it is a safe conversion for inference—in fact, huggingface uses float16 for Llama model inference IIRC. On the other hand, if you want to train the model, you should avoid using the float16 since it can result in nan errors.


feat(almin): add BeginTransaction/EndTransaction Payload

Changes

Add BeginTransactionPayload.ts and EndTransactionPayload.ts
EventHub receive these event via onBeginTransaction and onEndTransaction
This event needed logging library like almin-logger

TODO
Should we make Context#on~UseCase deprecated?
The user can use context.events.on~UseCase insteadof it.
Pros

Reduce Context interface size

Cons

The user not have merits

I think that is transaction correct term?
Current implementaion is not transaction, It create new UnitOfWork and run it.
This is similar with https://nodejs.org/api/vm.html#vm_vm_runinnewcontext_code_sandbox_options
We rename the term to more match implementation.

Most data transaction use sub-transaction insteadof multiple commit

Almin need End of the transaction.
Because, no commit is needed.
DataBase often not support no commit in a transaction.
But, Application does side-effect in a transaction.


Remove query parameters from the callback_url

As in the subject line & perhaps a little easier to manage than https://github.com/omniauth/omniauth-oauth2/pull/142.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 1 (0.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 84.615%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

lib/omniauth/strategies/oauth2.rb
0
1
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
77

Relevant Lines:
91

💛  - Coveralls


Remove AspNetCore.Mvc dependency in App.Metrics.AspNetCore.Mvc referencing AspNetCore.Mvc.Core instead

@WhatFreshHellIsThis re api projects and not being forced to take a dependency on razor for example ^


Basic instructions didn't work with previous global install

Describe the bug
I followed the directions on "Getting Started".
They mentioned:
npm uninstall -g create-react-app

Which I did. When I then resumed the tutorial, I got an error.
Did you try recovering your dependencies?
N/A
Which terms did you search for in User Guide?
I was at the start page.
Environment
[C:\c\React]node --version
v12.5.0
[C:\c\React]npx --version
6.14.6
C:\c\React]npx create-react-app --info
Command failed: C:"Program Files"\nodejs\node.exe "C:\Users\JohnQPublic\AppData\Roaming\npm\node_modules\npm\bin\npm-cli.js" config get cache --parseable
internal/modules/cjs/loader.js:628
throw err;
^
Error: Cannot find module 'C:\c\temp"C:\Users\JohnDGwinner\AppData\Roaming\npm\node_modules\npm\bin\npm-cli.js"'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:625:15)
at Function.Module._load (internal/modules/cjs/loader.js:527:27)
at Function.Module.runMain (internal/modules/cjs/loader.js:839:10)
at internal/main/run_main_module.js:17:11 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}
Steps to reproduce

Go to the "Getting Started Page"
Read the recommendation.
Follow recommendation:  npm uninstall -g create-react-app
Continue the getting started steps:
npx create-react-app cra-test

Error - I gather, as I'd just uninstalled it of course.
Expected behavior
The "getting Started" should actually work, and not require prior knowledge.
Actual behavior
C:\c\React]npm uninstall -g create-react-app
removed 91 packages in 2.156s

[C:\c\React]npx create-react-app cra-test --template typescript
Command failed: C:\"Program Files"\nodejs\node.exe "C:\Users\JohnQPublic\AppData\Roaming\npm\node_modules\npm\bin\npm-cli.js" config get cache --parseable
internal/modules/cjs/loader.js:628
    throw err;
    ^

Error: Cannot find module 'C:\c\React\"C:\Users\JohnQPublic\AppData\Roaming\npm\node_modules\npm\bin\npm-cli.js"'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:625:15)
    at Function.Module._load (internal/modules/cjs/loader.js:527:27)
    at Function.Module.runMain (internal/modules/cjs/loader.js:839:10)
    at internal/main/run_main_module.js:17:11 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

Reproducible demo
As Given Above
A Fix
If you've previously installed create-react-app globally via npm install -g create-react-app, we recommend you uninstall the package using npm uninstall -g create-react-app to ensure that npx always uses the latest version; then npm install create-react-app.
Emphasis mine. If you do this, then it works; of course, it dumps node_modules all over your parent directory. I'd far rather just do an update to the global objects, so I'm not sure about the recommendation.
For beginners though - it should just work as typed exactly.
Working version:
C:\c\React]npm install create-react-app
npm WARN saveError ENOENT: no such file or directory, open 'C:\c\React\package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open 'C:\c\React\package.json'
npm WARN React No description
npm WARN React No repository field.
npm WARN React No README data
npm WARN React No license field.

+<EMAIL_ADDRESS>added 97 packages from 45 contributors and audited 97 packages in 7.062s

6 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

[C:\c\React]npx create-react-app cra-test --template typescript

Creating a new React app in C:\c\React\

Installing packages. This might take a couple of minutes.
Installing react, react-dom, and react-scripts with cra-template-typescript...

.... etc

Well yeah it was stale, it was totally ignored. I posted a workaround as well as the point that for beginners, it should work as typed.
Someone needs to QA the docs and releases from a clean machine.
I could help FaceBook set that up via GitHub actions :)

Sorry for the long delay in commenting.


chore: Fix AzDriverNode registration error message

What type of PR is this?
/kind cleanup

What this PR does / why we need it:
Fixes the error message when AzDriverNode registration fails.
Which issue(s) this PR fixes:

Requirements:

[x] uses conventional commit messages

[ ] includes documentation
[ ] adds unit tests
[ ] tested upgrade from previous version

Special notes for your reviewer:
Release note:
none

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 42.721%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
179

Relevant Lines:
419

💛  - Coveralls


Issue #590: clarify usage of configuration macros

Minor update to README to clarify the usage of configuration macros #590
This is a

[ ] Breaking change
[ ] New feature
[ ] Bugfix

I have

[ ] Merged in the latest upstream changes
[ ] Updated CHANGELOG.md
[x] Updated README.md
[ ] Run the tests

The command "sudo cmake -DBUILD_SHARED_LIBS=ON ." failed and exited with 1 during .

Don't related to my changes. Looks like develop branch is not stable.


Switch from ruimarinho to lnzap bitcoind docker image

Switch from ruimarinho/docker-bitcoin-core to LN-Zap/docker-bitcoind.
Since the lnd docker image is also from lnzap, this might be more convenient in the future.

No. ruimarinho seems better maintained.


v0.15.1 cannot display memo visibility

Describe the bug
After modifying the memo visibility, the current memo visibility icon cannot be displayed on the homepage
Steps to reproduce
Add or modify a memo, and specify visibility
Screenshots or additional context
current:

expected:

It will be released in v0.16.0.


mixed precision training

Hey,
I've worked on reimplementing RIN based on the authors repo and this repo but I cannot manage to make it work with mixed precision and I see you do make use mixed precision here.
When naivly switching to bfloat16 or float 16, my model get stuck in a weird state:

left: bfloat16; right float32
Did you encounter such issues in your implementation? If so do you have some pointers to make it work?
Thanks!

@nicolas-dufour Hey Nicolas, there was an issue with the way I set up mixed precision in accelerate
in case you were using the same logic, it should be fixed in 0.7.6

@nicolas-dufour thanks for sharing those float32 results! 😄

Hey @lucidrains i'm using PL instead of accelerate but behaviour should be the same.
After further investigation, it seems there is a real difficulty to leverage mixed precision for this network.
If setting f32 for all the embeddings calculation and for the qkv linear projections of the cross attention layers, I do manage to get better convergence, but the training remains very unstable and blows-up mid training. Also, performance is really subpar with respect to fp32 throughout the training.
For now I'm trying to stabilize it without making architecture changes, but not sure if it's possible without forcing to many fp32 in the arch.
Will update here if i make any progress

@nicolas-dufour yea, if you figure out the cause do submit a PR

@nicolas-dufour decided to offer a way to turn off the linear attention, in case that is the source of the instability you are experiencing

@lucidrains thanks ! Sadly it's still pretty unstable. The authors said on their official repo that they didn't used mixed precision, so maybe this architecture cannot be trained with mixed precision and need major redesign to be stable in mixed precision =(

@nicolas-dufour yes indeed 😢 i'll also share that i tried a similar architecture for some contract work a while back and experienced the same

@lucidrains no thanks for the pointer, will try it out.

@lucidrains  Tried the qk_norm but it didn't change a thing. Also tried to add a LN for the from tokens of the cross attention but training still is subpar and doesn't converge

@nicolas-dufour bummer! this may be caveat to this architecture then
thank you for running the experiments and sharing this!

This is indeed the only way I found. It's a bug where parameters that are not set to store gradients in mixed precision never store gradients again.
This issue is discussed here https://github.com/pytorch/pytorch/issues/112583
To make mixed precision work one key component was also to catch Nans and Infs on the gradients and skip those batches


Prevent missclicking on the game over screen

After dying, the Game Over screen is shown, which then awaits for the user to press a key to continue the game. Since the screen is shown immediately after death, the user could still press a key to move the snake, and not to continue the game. To fix that I tried putting a call to pygame.time.wait there, but that does not seem to work.
Remove the current solution and add another one, that really prevents missclicking after snake's death.

Hey, may I try this one?

Sure, go ahead @Vicfs


Update RedirectPlugin.php

Fix Icon

@RichardFrontwise thanks, but Craft 3 has changed the way of loading icons in the sideNav. I have chosen a new one for now, feel free to suggest an other!


Support http and https for Kubernetes Service appProtocol

What does this PR do?
Currently if a Kubernetes Service species the appProtocol field as http or https, despite Traefik having support for those protocols, Traefik will raise the error
Cannot load HTTPBackendRef <namespace>/<service>: unsupported application protocol http

Documentation for the appProtocol field: https://kubernetes.io/docs/concepts/services-networking/service/#application-protocol. Notably, it may be any IANA standard service names, such as http or https.
Motivation
I was trying to expose the RabbitMQ management UI, using a RabbitMQ cluster controlled by https://github.com/rabbitmq/cluster-operator. The service this operator creates sets the appProtocol field to http, which causes Traefik to error. As it stands, Traefik only works properly with http/https when the field is not set.

Hello @WillDaSilva and thanks for opening this pull request,
We also think that the https appProtocol must be supported for GRPCRoute (to be able to do gRPC over https). The server protocol should be adapted accordingly here.
Would you mind doing the changes, or is it ok for you if we push review commits as we want to cut a new RC soon?

@kevinpollet please feel free to push changes to my branch. You'd know much better than I what needs to be changed.


Fix CI on ghc-head

This unbreaks CI on ghc-head. The idea follows #859. There's a couple reasons for doing this:

We'll always be up to date on GHC changes. Worse case scenario, we wait a day and manually restart CI, but CI should never be red for more than a day.
It'll be trivial to point to a custom GHC artifact, so Haddock branches which need custom GHC branches (like wip/hi-haddock) can continue to run CI

Making CI green also forced me to fix some breakage that's snuck in since stuff was last green.

You could simplify the .travis script by using language:nix instead I think if that's a path you want to go down.

You could simplify the .travis script by using language:nix instead I think if that's a path you want to go down.

Is there a polished way in Nix to install the validate artifacts and add the GHC HEAD overlay?  That sort of thing would be quite nice indeed.  Could you expand more on what you had in mind?

I guess you would just want nix to install ghcHEAD and cabal but leave cabal to manage all haskell dependencies.
Setting up that environment can be managed in one command:
nix run -f https://github.com/mpickering/ghc-artefact-nix/archive/master.tar.gz ghcHEAD cabal-install

Perhaps you could also keep most of the current script the same and install the dependencies globally to save yourself mucking around unpacking them.
nix-env --dry-run -i -f https://github.com/mpickering/ghc-artefact-nix/archive/master.tar.gz ghcHEAD cabal-install

for the record: I would prefer some clean nix solution here (especially if it worked uniformly for GHC HEAD and non GHC HEAD), but there are other yaks to shave right now.


Connecting data from Teams PowerBI App not working

Hello,
I tried to connect the data to our Tennant using OAuth2 via the Teams PowerBI App but every time it opens the web browser (Edge or Chrome) with a popup disabled it completely closes the preview.powerbi.com link. I tried adding both powerbi.com and microsoftonline.com still doesn't work.
Performing the data connection on a browser directly on the powerbi website works.
rmbuan

Thanks for your comment, @rmbuan. This issue has been assigned to the doc author for review.

Thanks for your comment, @rmbuan.
This looks like a support request or product question rather than documentation feedback.
You might find a solution in the Power BI Community. If not, try our support page.


Fix image with frame alignment

This PR utilizes Primer.css classes to address https://github.com/gollum/gollum/issues/838.  The tests are still failing due to changes in the class names, but I would like input regarding the expected functionality before I fix those class names in the tests. The current functionality in this PR is as follows:

Adding float will create a floating image, with text wrapping around the image. Floats will be aligned left by default, unless align=right is specified. Left and right are the only two valid options for floating images.
Adding frame will add a border to the image. If alt=alt text is set, the image will be accompanied by a caption below the image, with text alignment set to the alignment of the image (left, center, or right).
Images with and without frames can be aligned left, center, or right (e.g.,align=center). If alignment is set to left, center, or right, the image will be "containered".
Images without float or alignment {left, center, right} will not be "containered". For these images, the img attribute align may be set to one of top texttop middle absmiddle bottom absbottom baseline. I'm not sure what this does or is supposed to do, but that followed from the previous implementation.

@penguinpowernz Would appreciate your input on this PR as well!

Nice! That was quite a puzzle with the new primer classes.

If alignment is set to left, center, or right, the image will be "containered".

Not sure what the reason for that is, but as it appears to be the current behavior, I am fine with it. 👍

All the other behavior makes sense to me!

Thanks. Would you mind giving it a spin locally to see if everything works as you would expect?

align left|center|right seems to be working, as is text-wrap with float. But frame and alt only seem to work when align is also set, and not on their own:

Was that already the case in the previous implementation? Would it be difficult to change?

But frame and alt only seem to work when align is also set, and not on their own

Fix in https://github.com/gollum/gollum-lib/pull/347/commits/2b0712d4efbef9a67fff7e591cef3480f8b29c5d. It is now possible again to give an image a frame / border without setting align explicitly (it defaults to left alignment).

Sweet!


[Bug] Consolidate validation for Card.href and Page.path

Description

Fix https://github.com/mckinsey/vizro/issues/285 by consolidating and  validation for Card.href and Page.path
Replace dcc.Link with dbc.NavLink (this seemed to be broken for external links before - we might have to add more tests here @l0uden)
The following are valid inputs for the Card.href now:

https://google.com
http://google.com
/page-title
page-title  (didn't work previously)
Page Title (didn't work previously)

Change doc and demo examples to use the more intuitive way of linking to pages via the Card.href

Screenshot
Notice

[x] I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":

I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.
I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorized to submit this contribution on behalf of the original creator(s) or their licensees.
I certify that the use of this contribution as authorized by the Apache 2.0 license does not violate the intellectual property rights of anyone else.
I have not referenced individuals, products or companies in any commits, directly or indirectly.
I have not added data or restricted code in any commits, directly or indirectly.

Closing as we decided to not enable this feature.

In more detail...
The problem with the proposed solution for Card.href is that really we should be referring to the page id and not the page title. But the page id is not necessarily the same as the page path since (a) there’s discrepancy of whether / is scrubbed from the path depending on whether you set the page.path manually or whether it's taken from the id automatically (b) the annoying homepage case where id/title and path don’t match up (maybe changed in #366) and (c) we let users set page.path manually (which in hindsight was a mistake I think since I’m pretty sure no one does it now and it makes this sort of thing a bit awkward :grimacing:). So that means that page.path could be completely different from page.id, in which case the solution proposed in this PR won't work at all.
So really the way we should be doing this is:

user supplies page id
look it up in mode manager to find relevant page
get relative path out of that page and use that for the link

https://github.com/mckinsey/vizro/pull/348/commits/09305d69638dd80a1369691bbd9c7a9edaa196ea shows how we might do this. While it's not that hard to do, it does add some complexity and at some point in the future I'd hope that the current href behaviour of Card might be removed at all in favour of #226.
If we do add this in future to lookup by page.id we should also consider whether to remove the functionality that providing a relative path would work so that the only options are to provide page.id or a complete href. This might be a bit tidier but would be a breaking change.


linenos=inline: not enough space between numbers and code lines

I'm using Pygments via Sphinx, but I've seen that this also happens with Pygments alone ...
Observed behavior:
pygmentize -f html -O full=True,linenos=inline,style=solarized-light hello.py

Expected behavior:
pygmentize -f html -O full=True,linenos=table,style=solarized-light hello.py

hello.py:
#!/usr/bin/env python3

a = int(input('gimme a number: '))
if a < 42:
    print("that's quite small, innit?")
else:
    print("wow, that's big!")

A bit off topic ... both generated HTML files have an empty level-2 header before the code block:

However, somehow the spacing (based on the margin CSS values, I guess?) at the top is quite different.


the GPU memory is too large

@zsc Thank you for your sharing! As your paper said, you can train with batch size 16 on 8 2080TI GPUs when you use the pytorch framework. But when I want to train your network, the GPU memory is large as 8.5G with batch size 1. So what is the problem?

@zsc Thank you for your sharing! As your paper said, you can train with batch size 16 on 8 2080TI GPUs when you use the pytorch framework. But when I want to train your network, the GPU memory is large as 8.5G with batch size 1. So what is the problem?

I have the same question.


add four seasons oolong

https://www.curioustea.com/tea/oolong-tea/four-seasons-oolong/

https://www.teafromtaiwan.com/loose-oolong-tea/four-season


Histogram & Shared Plot State Hook

New

Plot.ly based histogram plot with a variety of UI options.
A default control panel for histogram plots.
A variety of UI widget to compose control panels from.
A custom hook which is used to share/manage state between control panel(s) and plot(s).

Updated

Rolled in latest updates to Storybook.

Future Work

Improve the integration of Storybook controls and histogram stories.
Add a de-bounce feature to the binWidth slider to allow for (1) immediately responsive UX for user, but (2) avoid spamming backend servers with API requests.
Create default control panels for all other plot types.
Create additional control panel widgets as necessary.

Overall, this is a really good start. I see that you have deviated form the props type specified in https://redmine.apidb.org/issues/41311. I think we should keep to that, as much as possible. I realize that there may be some things missing. We can address those as needed.

I see that we left out count from props.data[number].series[number]. This should definitely be added.
The grouped layout doesn't make sense for histograms, so should not be an option.
I like the various color options you added.
Adding the title props was good. We might also props for axis labels: dependentAxisLabel and independentAxisLabel.

I accidentally promoted this from draft to a real PR, but that's probably not a bad thing.
I have been reviewing the code, in the knowledge that there are things still to be committed/pushed.
My first conclusion/opinion is that we should not address the "axis coordinate space" w.r.t. Plotly box-select.  At the moment the x-range (or y-range if in vertical mode) emitted in the Action panel is in "bin space", which means the first bin starts at x=0, second bin at x=1 etc.  I think our client code should convert this to "data variable space" (e.g. some numbers in the thousands, in story "Single with Controls"). The client knows the bin starts/ends so I think this should work.  If we instead tried to get Plotly to plot in "data variable space" (e.g. with box select producing numbers in the thousands in that example) then we may have to manually specify bar width... That will be a pain when dealing with monthly date-based data.  We don't really want February bars to be narrower than the others.

I accidentally promoted this from draft to a real PR, but that's probably not a bad thing.
I have been reviewing the code, in the knowledge that there are things still to be committed/pushed.
My first conclusion/opinion is that we should not address the "axis coordinate space" w.r.t. Plotly box-select.  At the moment the x-range (or y-range if in vertical mode) emitted in the Action panel is in "bin space", which means the first bin starts at x=0, second bin at x=1 etc.  I think our client code should convert this to "data variable space" (e.g. some numbers in the thousands, in story "Single with Controls"). The client knows the bin starts/ends so I think this should work.  If we instead tried to get Plotly to plot in "data variable space" (e.g. with box select producing numbers in the thousands in that example) then we may have to manually specify bar width... That will be a pain when dealing with monthly date-based data.  We don't really want February bars to be narrower than the others.

A possible issue is that the PlotControls might not work with date-based data.
This, in usePlotControls.tsx (does this have any JSX in it?) could fail?  Apologies if that's already on the TO DO list.
          if (typeof bin.binStart === 'string') {
            console.error('String Bin Start Values are not yet supported.');

A possible issue is that the PlotControls might not work with date-based data.
This, in usePlotControls.tsx (does this have any JSX in it?) could fail?  Apologies if that's already on the TO DO list.
          if (typeof bin.binStart === 'string') {
            console.error('String Bin Start Values are not yet supported.');

Final comment from me...!
@eikonomega, do you have a plan for adding controls to another plot type, or would you prefer to leave that as an exercise for another developer after this PR is merged?  I'm in favour of either.

Final comment from me...!
@eikonomega, do you have a plan for adding controls to another plot type, or would you prefer to leave that as an exercise for another developer after this PR is merged?  I'm in favour of either.

I'm planning on adding default controls for all plot types.
At least, that's what I'm hoping to work on as soon as this is merged!

Michael Dunn (a.k.a. Big Fat Panda)
Ave Crux, Spes Unica
Research Programmer
Center for Research Computing
Email<EMAIL_ADDRESS>Twitter: @eikonomega
Returning hate for hate multiplies hate, adding deeper darkness to a night
already devoid of stars. Darkness cannot drive out darkness; only light can
do that. Hate cannot drive out hate; only love can do that. - MLK
On Mon, Jan 25, 2021 at 2:57 PM bobular<EMAIL_ADDRESS>wrote:

Final comment from me...!
@eikonomega https://github.com/eikonomega, do you have a plan for
adding controls to another plot type, or would you prefer to leave that as
an exercise for another developer after this PR is merged? I'm in favour of
either.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/VEuPathDB/web-components/pull/29#issuecomment-767074833,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AADFRQZ5EPXO3NEP7KWL6TTS3XEJJANCNFSM4UBLWWQQ
.

I'm planning on adding default controls for all plot types.
At least, that's what I'm hoping to work on as soon as this is merged!

Michael Dunn (a.k.a. Big Fat Panda)
Ave Crux, Spes Unica
Research Programmer
Center for Research Computing
Email<EMAIL_ADDRESS>Twitter: @eikonomega
Returning hate for hate multiplies hate, adding deeper darkness to a night
already devoid of stars. Darkness cannot drive out darkness; only light can
do that. Hate cannot drive out hate; only love can do that. - MLK
On Mon, Jan 25, 2021 at 2:57 PM bobular<EMAIL_ADDRESS>wrote:

Final comment from me...!
@eikonomega https://github.com/eikonomega, do you have a plan for
adding controls to another plot type, or would you prefer to leave that as
an exercise for another developer after this PR is merged? I'm in favour of
either.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/VEuPathDB/web-components/pull/29#issuecomment-767074833,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AADFRQZ5EPXO3NEP7KWL6TTS3XEJJANCNFSM4UBLWWQQ
.

Should this be considered merge-ready? If so, I can do a final review and
merge of all looks good. Or Bob can, if he would like 🙂. Let me know!
On Mon, Jan 25, 2021 at 2:59 PM Michael Dunn<EMAIL_ADDRESS>wrote:

I'm planning on adding default controls for all plot types.
At least, that's what I'm hoping to work on as soon as this is merged!

Michael Dunn (a.k.a. Big Fat Panda)
Ave Crux, Spes Unica
Research Programmer
Center for Research Computing
Email<EMAIL_ADDRESS>Twitter: @eikonomega
Returning hate for hate multiplies hate, adding deeper darkness to a night
already devoid of stars. Darkness cannot drive out darkness; only light can
do that. Hate cannot drive out hate; only love can do that. - MLK
On Mon, Jan 25, 2021 at 2:57 PM bobular<EMAIL_ADDRESS>wrote:

Final comment from me...!
@eikonomega https://github.com/eikonomega, do you have a plan for
adding controls to another plot type, or would you prefer to leave that
as
an exercise for another developer after this PR is merged? I'm in favour
of
either.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/VEuPathDB/web-components/pull/29#issuecomment-767074833
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AADFRQZ5EPXO3NEP7KWL6TTS3XEJJANCNFSM4UBLWWQQ
.

—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/VEuPathDB/web-components/pull/29#issuecomment-767075999,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACZEU6EAGZLJVZBMOD45FDS3XERLANCNFSM4UBLWWQQ
.

Should this be considered merge-ready? If so, I can do a final review and
merge of all looks good. Or Bob can, if he would like 🙂. Let me know!
On Mon, Jan 25, 2021 at 2:59 PM Michael Dunn<EMAIL_ADDRESS>wrote:

I'm planning on adding default controls for all plot types.
At least, that's what I'm hoping to work on as soon as this is merged!

Michael Dunn (a.k.a. Big Fat Panda)
Ave Crux, Spes Unica
Research Programmer
Center for Research Computing
Email<EMAIL_ADDRESS>Twitter: @eikonomega
Returning hate for hate multiplies hate, adding deeper darkness to a night
already devoid of stars. Darkness cannot drive out darkness; only light can
do that. Hate cannot drive out hate; only love can do that. - MLK
On Mon, Jan 25, 2021 at 2:57 PM bobular<EMAIL_ADDRESS>wrote:

Final comment from me...!
@eikonomega https://github.com/eikonomega, do you have a plan for
adding controls to another plot type, or would you prefer to leave that
as
an exercise for another developer after this PR is merged? I'm in favour
of
either.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/VEuPathDB/web-components/pull/29#issuecomment-767074833
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AADFRQZ5EPXO3NEP7KWL6TTS3XEJJANCNFSM4UBLWWQQ
.

—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/VEuPathDB/web-components/pull/29#issuecomment-767075999,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACZEU6EAGZLJVZBMOD45FDS3XERLANCNFSM4UBLWWQQ
.

Should this be considered merge-ready? If so, I can do a final review and merge of all looks good. Or Bob can, if he would like slightly_smiling_face. Let me know!

Not quite. Mike's got a few more things to do, and yes, I'll look after the merge :smiley:

Should this be considered merge-ready? If so, I can do a final review and merge of all looks good. Or Bob can, if he would like slightly_smiling_face. Let me know!

Not quite. Mike's got a few more things to do, and yes, I'll look after the merge :smiley:

Just the streamlining of the props passing in the stories file still remaining to commit/push, right @eikonomega ?

Just the streamlining of the props passing in the stories file still remaining to commit/push, right @eikonomega ?

I just pushed some changes to src/stories/plotControls/HistogramControls.stories.tsx
My main aim was to understand the usePlotControls better by trying to fix the TS errors that were causing a build check fail.  I think I succeeded there (TS errors are gone) but I am not sure about the following:

Setting the initial/default binWidth in the story code
Was this the correct way to allow availableUnits in the story.args?

-export const AdditionalOptions: Story<usePlotControlsParams<HistogramData>> = (
+export const AdditionalOptions: Story<usePlotControlsParams<HistogramData>
+                     & {
+                       availableUnits: string[],
+                       selectedUnit: string,
+                     }> = (

the units widget doesn't actually work...


Mimikatz load_pe error

I saw https://github.com/Thoxy67/rspe used in rust for pe load. I tested locally and found that Mimikatz nc cve-2024-26229 would fail. For Mimikatz, I tried to use donut to convert it into shellcode and it was successful. The remaining two still failed. Some details nc is compiled using pascal language. I want to know how you solved it.

Did you read the comments in the code? ;-)

https://github.com/rtecCyberSec/Packer_Development/blob/master/Rust/peload/peload.rs
reflective_loader(data.clone());
There doesn't seem to be any difference...

I‘ll not give you a full solution for it because this whole repository is meant to be used for learning purposes.
You’ll need to fix a bug in the rspe library code itself. So first step instead of just „importing and using“ the library, get the code and troubleshoot where it’s crashing with mimikatz.
This way you will find out:

how a reflective PE loader works
what’s wrong. 🙃

Providing the full solution will not lead to learning effects here.
Greetings

By opening an issue in another repo you will not learn ;-) I’m going to close this for now as there is no todo for this repo. 👍


Coverity scan

We should set up a branch to point Coverity Scan at, for another source of static analysis.
I signed us up already thinking I could get it all done in 5 minutes but it turns out it's not quite as trivial as I thought. I'll leave this issue here to let everyone know it's being thought about at least!
https://scan.coverity.com/projects/recastnavigation-recastnavigation

Here are the scripts I use on Ubuntu x64 in case you might need them:
covupdate.sh used to download latest Coverity Scan version and configure it for gcc. You can find the correct wget command clicking on "Submit Build" on coverity
wget https://scan.coverity.com/download/linux-64 --post-data "token=<your secret project token>&project=<Your Project Name>" -O coverity_tool.tgz
tar -zxvf coverity_tool.tgz
rm -f -r cov-analysis
mv cov-analysis-linux64-* cov-analysis
rm -f -r coverity_tool.tgz
cd cov-analysis
cd bin
./cov-configure --gcc

cov.sh used to build the project (recast in your case) with Coverity Scan tool creating the data for analysis. it has to be executed in the folder with project files. It will create a subfolder "cov-int" with everything needed by coverity and zip it for the upload.
make clean
/home/user/cov-analysis/bin/cov-build --dir cov-int make -k
/home/user/cov-analysis/bin/cov-import-scm --dir cov-int --scm git --log cov-int/scm_log.txt 2>&1
tar czvf tc.tgz cov-int

covup.sh used to upload the analysis data. You can find the correct curl command clicking on "Submit Build" on coverity
curl --form project=<project name> --form token=<secret token> --form email=<email>  --form<EMAIL_ADDRESS>--form version=none --form description=none https://scan.coverity.com/builds?project=<project name>

Fantastic, thanks @jackpoz!

I've done a build and uploaded to Coverity Scan - there are some good issues reported! This is definitely a useful service.
I'll have a look at automating this through Travis and setting up a coverity_scan branch so as not to run it all the time, as Coverity recommends.

I browsed through your build history on TravisCI, the frequency in which they run is well within the bounds of how many times you can allow CoverityScan to run. I suggest having it automate the main branch and sticking the Coverity badge in the readme as this is a place many developers go to start improving projects. It's easy to forget pushing to the coverity_scan branch and I was sad to see the project hasn't been scanned in almost a year!

We can setup a github cron job that does this automatically. I can either help walk someone through it or do it myself if I had access. OpenMW makes extensive use of Recast and have found some issues in it (by accident) that we now exclude along with other libraries.
Our idea was to just fork the libraries we use on our github repo, run the cronjobs there on https://github.com/OpenMW in our own Coverity account then share the reports here in issues and possibly send PRs as well. If we can cut out the middle-man (us) then that would be better. :)

Migrated to #642 to revive or replace automatic coverity scanning.


Using DotNetZip 1.16.0

Hi,
I think eXpandIO/24.1.701 is using DotNetZip 1.16.0 and there is no newer Version.
DotNetZip Directory Traversal vulnerability.
https://github.com/advisories/GHSA-xhg6-9j5j-w4vf
Can you change it?

I did not checked yet how many resources needed to this task, however I want to give u my perspective about those vulnerability warnings. In short they are endless and can lead to spent a great number of resources to constantly upgrade all these packages we use, not to say that each upgrade may cost us another problem.
So, my strategy is not to change anything unless there is a reason and at least a basic research. Because yes there is a vunerability there but does it affect us? does it affect your projects? in what level?
Please provide details and then happy to put some time to see what's going on there.
Note this package is only used from the IO module (Windows) and to upgrade it to the System.IO.Compression it requires the following steps

Create sample
Upgrade the code
Test the code

they may look simple but timing wise it can take from a few hrs to a full day depending on what u find in your way.

In principle I agree with you.
That´s why we try to avoid too many nugets, espacially if the function can be used by standard system. We use Devexpress as base system. Devexpress is maintened. eXpandFramework is based on Devexpress. Can ZIP/Compression functions be used by Devexpress, so that DotNetZip is not needed?

eXpandFramework is based on Devexpress

not entirely true, eXpand uses packages from all nugget ecosystem. As I said there is no clarification where this vunerability affects u and making changes has a cost. When the IO module was authored there was no build in support so I chose this external lib


ValidateEntity not Exist any more in DBContext

in the EF6 i have the following code in DbContext and i want to upgrade to EF Core 1.1 but i don't find the method ValidateEntity , so is there is any way to do the same code but in different method or what the solution will be !
        protected override DbEntityValidationResult ValidateEntity(DbEntityEntry entityEntry, IDictionary<object, object> items)
        {
            if (entityEntry.Entity is BaseGoTechEntity)
            {
                var entity = (BaseGoTechEntity)entityEntry.Entity;
                if (entityEntry.State == EntityState.Added)
                {
                    entity.DateCreated = DateTime.Now;
                    entity.DateUpdated = DateTime.Now;
                }
                else if (entityEntry.State == EntityState.Modified)
                {
                    entity.DateUpdated = DateTime.Now;
                    Entry(entity).State = EntityState.Modified;
                }
                else if (entityEntry.State == EntityState.Deleted)
                {

                }
                //entity.DateCreated
            }
            return base.ValidateEntity(entityEntry, items);
        }
    }

@abuzuhri It doesn't look like that code is actually doing validation. I would try moving this kind of processing to SaveChanges.


Add BookmarkList example

Add example for #1346.

I was expecting a proper example in https://github.com/gtk-rs/gtk4-rs/tree/master/examples instead

Oh yeah, that is a good idea too!
I was thinking of having a example  section in the docblock to sit shows in the documentation and when using the "Go to definition" in the IDE.
Does this project accept examples in the docblock too?

Please test locally first....


[Words / What I feel] UI. General view of the exercises. The word matrix is broken.

Precondition:
Log in with an existing user. Open: Groups / Speech Exercises / Words / What I feel.
Steps:

Select exercise 7 (for example).
Actual result: The word matrix is shifted.
Expected Result: The word matrix is not shifted.

Exercise 7: https://brainup.site/groups/2/series/1/subgroup/20/exercise/608/task/608
Exercise 10: https://brainup.site/groups/2/series/1/subgroup/20/exercise/611/task/611
Exercise 13: https://brainup.site/groups/2/series/1/subgroup/20/exercise/614/task/614
Exercise 18: https://brainup.site/groups/2/series/1/subgroup/20/exercise/619/task/619
Exercise 21: https://brainup.site/groups/2/series/1/subgroup/20/exercise/622/task/622
Exercise 24: https://brainup.site/groups/2/series/1/subgroup/20/exercise/625/task/625
Exercise 29: https://brainup.site/groups/2/series/1/subgroup/20/exercise/630/task/630
Exercise 32: https://brainup.site/groups/2/series/1/subgroup/20/exercise/633/task/633
Exercise 35: https://brainup.site/groups/2/series/1/subgroup/20/exercise/636/task/636
Exercise 41: https://brainup.site/groups/2/series/1/subgroup/20/exercise/641/task/641
Exercise 43: https://brainup.site/groups/2/series/1/subgroup/20/exercise/644/task/644
Exercise 46: https://brainup.site/groups/2/series/1/subgroup/20/exercise/647/task/647

fixed, should be retested

fixed


ig.directThread.getByParticipants returns empty items

Form
Put an [x] if you meet the condition, else leave [ ].
Requirements

[x] I've searched the Issues
[x] I've read the basic concepts
[x] I'm using the latest version
[x] I've debugged my code using the DEBUG variable.

Platform

[x] I'm using Node.js 10.23.0

Description
A specific description of your bug, so it's understandable to anyone.
If you have pictures or videos to upload, please add a link like this: [title](url).
When I try to retrieve messages sent from specific users by using directThread.getByParticipants, I correctly receive the thread but the items property is empty. I checked in my instagram app and there is a text message. I tried with different accounts, different proxies and so on. I don't have this problem using ig.feed.directInbox or ig.feed.directPending.
Code
Add a meaningful section of your code here. If you are using TypeScript replace js with typescript.
router.post('/user', async function(req, res, next) {
  const { pk } = req.body;

  try {
    const result = await ig.directThread.getByParticipants([pk]);
    if (result.thread) {
      res.json({ messages: result.thread });
    } else {
      res.json({ messages: [] });
    }
  } catch(e) {
    const message = e.message;
    res.errorMessage = message;
  } finally {
    next();
  }
});

Thanks for your help

Is the items property an empty array ([]), undefined or null?
Regardless of the value, you still have the thread's thread_id. You can use this to get the thread-feed.


Add extraPodLabels to oapInit so Job can have additional labels

Issue:  oap-init job is unable to be completed when istio-injection: true is set on a namespace.  The container specific to the job works, but the job does not mark as complete due to the lack of istio-init completing.
Solution: Add additional labels to spec.template.metadata.labels (e.g. sidecar.istio.io/inject: false), to the job-generated pod, which enables the job to complete properly when Istio is installed..
Unblock oap-init job from being blocked when Istio injection is enabled for a namespace

@eric-futureself Any update?


Week 9: Documentation Updates

Added Autocomplete component to developer guide.

Codecov Report
Base: 66.87% // Head: 66.87% // No change to project coverage :thumbsup:

Coverage data is based on head (dfe1b2e) compared to base (9d0194f).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff            @@
##             master      #71   +/-   ##
=========================================
  Coverage     66.87%   66.87%
  Complexity      575      575
=========================================
  Files            88       88
  Lines          1941     1941
  Branches        237      237
=========================================
  Hits           1298     1298
  Misses          556      556
  Partials         87       87

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.