Support snippet-less completion

Some clients don't support completions snippets. It can be argued that clients generally should be improved to support snippets, but

I can imagine embedding rust-analyzer in the contexts where full snippet support does not make sense (for example, completion in the eval field of a debugger)
I can imagine some users might prefer simple identifier completions

So, we should support this use-case. Specifically, we should only send snippet completions if supports them.
A simplest fix here (and a good initial implemtation) is change code around here to just strip $i snippet markers from text edit, if the client doesn't support snippets. A similar "fixup" approach is used for folding ranges.
However in general it seems like the core completion engine should just provide different completions depending on whether snippets are enable or not. This should be done via feature flags (example), and requires some design to make sure that we don't twice as much code, but still provide best possible completions with and without snippets.

Thanks for opening this issue. As mentioned in #1705, not providing plain text completion is a protocol violation, so "clients should be improved" is not a valid statement.

Some clients don't support completions snippets. It can be argued that clients generally should be improved to support snippets

As a maintainer of such a client and someone with a strong opinion that goes against snippets, here's my .02c:
I'm a vim user. Snippets in vim are expanded on something called CompleteDone event, which is triggered whenever something closes the completion menu. That can be ctrl-y, which works fine with snippets and simply closes the menu. However this completely breaks my workflow. I'm used to just continuing to type after selecting a completion. If I hit enter instead, the new line is entered before the snippet is expanded, which messes up a lot of things and to work around this is incredibly tricky (speaking from the client maintainer's point of view). So I have two options if I want to use snippets:

Break my workflow in favour of a less efficient one.
Put much more work than I think it's worth, for a feature that I would still find bad.

That's because with plain text completion, I know ahead of time what text will be inserted. This is again, very important for my workflow.

So the instead, ycmd maintainers, unanimously, settled for a third option - not implementing snippets.

However in general it seems like the core completion engine should just provide different completions depending on whether snippets are enable or not.

So, I haven't looked at the rust-analyzer code, but a client like ycmd would expect foo insertion text for a foo(/* any amount of args */).

Now I didn't mean to sound harsh, in case I did. It's just that a lot of servers assume snippets are supported, even though it can be argued that they provide a pretty horrible user experience.
On a side note, my workflow is basically the workflow of all ycmd users, so drastic changes like snippets could very much be more harm than good, considering that ycmd is much older than LSP.

Would that make insertion text just foo()?

Yes. As I've said, that would be a good initial implementation, which would primarily focus on plumbing the required flags.
For the final implementation, a corresponding flag should be added to the internal CompletionConfig, and than this setting need to be honored everywhere where we use snippets.
After that, we either should assert here that we don't see a compleiton with a snippet, or we should just not add such completions.

After that, we either should assert here that we don't see a compleiton with a snippet, or we should just not add such completions.

I've actually figured a much neater way to do this: introduce an unforgable SnippetCap token, which proves that this capability is available, and add it as an argument to snippet producing functions.
I've implemented this idea in https://github.com/rust-analyzer/rust-analyzer/pull/4116
No the only thing left is the actual plumbing to fetch the cap from the client....

Plumbing implemented in #4117
Here's example of completion with a snippet:

And here's the same completion without the snippet:

In your example I would still argue that the completion is probably completing too much. It’s unexpected that selecting ‘size’ competes a whole line of text including a function declaration.
I kind of get it. It’s boilerplate, so let’s write it for the user.
My concern is that sort of thing is divisive and opinionated. Perhaps that’s intentional (if so, so be it), but certainly my users are unlikely to expect that and would likely be model-breaking for completion systems and users accustomed to more natural word expansion.
The original point put across by Boris is that completion systems doing too much breach’s workflow because when stuff happens that you don’t expect you end up having to rewind. Let’s say you develop in N languages and N-1 behave one way, eg like clangd or jdt.ls or gopls) and the Nth one behaves like this. The challenge for the user in that case is to switch gears. We would like to avoid having the user switch gears and offer consistent powerful experience across languages.
Anyway just my 2p. Tl:dr there are a large quantity of users that don’t expect the completion system to complete more than a word at a time. It’s probably worth considering that group.

As usual, the user is (almost) always right, and, if some users complain about this feature, we'll introduce a feature flag,  like, for example, we did  for automatic () insertion. However, the reasoning should be "I've tried this feature and I hate this", rather "this is wrong because completion isn't supposed to work this way".
With my "unsolicited advice giver" hat on ( :-) ), I think for ide-like tools it's generally good to:

provide opinionated defaults,
but give the user power to choose.

I did add support for features which I, personally, don't use and find counter-productive, like argument snippets (yes, I hate stupid snippets myself), which, I believe, only exists because parameter hint UI is universally horrible (IntelliJ's version is kind of ok, actually, but the API used to implement it is very crufty).
From re-reading original comment here, I get the impression that the primary reason for not supporting snippets is not that "users expect a single word", but that vim expects a single word, and that it's a pain to work around this limitation. So, to me, the path that maximizes user happiness (without taking work needed into account), is:

fix Vim's core to make snippet-based completion ergonomic for the user. I think this might entail some interesting UX design questions around combining completion and modal editing. I think the best example to learn from would be IDEA Vim. It is maintained by a person who is both a JetBrainser and a Vimmer, so it probably solves the problem taking both perspective into account.
add an option to ycmd to take advantage of snippets, to give the user the choice
keep this option off by default, to nudge users towards more productive (according to opinionated views of ycmd maintainers, which are totally valid) workflows.

I get the impression that the primary reason for not supporting snippets is not that "users expect a single word", but that vim expects a single word, and that it's a pain to work around this limitation

No, vim plugins can adequately support snippets.  I have even implemented it for YCM. But, as you say "provide opinionated defaults" - we tried it, we evaluated it, and we decided we don't like it as it doesn't fit well with our philosophy and the mental model our users have of what we provide.
Add to that the complexity of doing it well, the fact there's only 2 of us and neither of us really want it, and we decided against it.
Similarly to you, if we have 1000s of users asking for it, we might rethink, but so far not so many. The "option" we offer is to use my fork. Mileage variance notwithstanding.
Our point was really that we made this decision and we declared that decision to the Language Server using capabilities. We expect that decision to be honoured. We could just as well have merged my snippets branch and offered the user the ability to globally disable it by, again, declaring in the capabilities that we don't support them (this prevents the user having to configure this for each and every project/language combination in unique, marvellous and usually undocumented ways).
Anyway, as you say, that's water under the bridge. The capability is now honoured (thanks!).
I've tested rust-analyzer today with YCM and updated instructions for using it: https://github.com/ycm-core/lsp-examples#rust-rust-analyzer. (For the record i looked in the log and it'f full of errors for other parts of the capabilities that are not being honoured, but i know you have a GitHub issue for the general case, and everyone has a backlog/priorities).
But getting back to it, my suggestion about the function-declaration was largely orthogonal to that, and comes back to the original philosophy of largely getting out of the way and assuming users know what to type (it's mentioned here from 2013 http://val.markovic.io/articles/youcompleteme-a-fast-as-you-type-fuzzy-search-code-completion-engine-for-vim), but more that the consistency of experience across languages is a truly powerful thing. YCM has been providing consistent experience across languages for years, and i think our users appreciate that, so we're generally in favour of minimising surprising behaviours. Like i said it was really just a take-it-or-leave-it thought. In this case, opinionated default = complete full declaration, feature option=don't.

On a side note, i'd like to test the "full function declaration" completion because it may hit edge cases in the textEdit expansion - would you be willing to let me know a simple test case for it ? I don't rust, so i'd be grateful if there's a place i can trigger it (e.g. in rust-analyzer source code).

YCM has been providing consistent experience across languages for years, and i think our users appreciate that, so we're generally in favour of minimising surprising behaviours.

Obviously we care more about providing a great experience for Rust than about providing a consistent experience across languages. But I want to push back a bit on the "surprising" part -- the completion is labeled fn size_hint(..). It seems to me pretty obvious that it's not going to insert just one word (though I actually think it should even be labeled fn size_hint(..) {} to make that even clearer). Maybe this is also about how exactly the UI for the completions works.

On a side note, i'd like to test the "full function declaration" completion because it may hit edge cases in the textEdit expansion - would you be willing to let me know a simple test case for it ?

trait Foo {
    fn bar(x: u64) -> f32;
}
struct S;
impl Foo for S {
    // (type bar here)
}

I think that should be enough to trigger it (if you put it into main.rs in a new Cargo project), though I haven't tested it.

actually that does work quite well

did notice something odd though (you can briefly see it in the demo). when selecting a macro, while the label contains the !, the insertion_text (or textEdit) does not .
You can see here that the 'newText' does not contain the !
"filterText":"debug_assert!","insertTextFormat":1,"kind":2,"label":"debug_assert!","textEdit":{"newText":"debug_assert","range":{"end":{"character":8,"line":7},"start":{"character":4,"line":7}}}},

Seems wrong to me? Expected?

Kind of expected: ! is the part of !($0) snippet, that's why it is left out of the edit (so that only identifier is completed). It is present in the label because that's a natural way to indicate that this is macro. However, that last bit seems to work by accident -- the label for snippeted completion should be debug_assert!(...) and it isn't

but the label and what's inserted don't match....i mean the label looks like the right thing to insert to me (and that's what rls would insert here):

Note: rls is also giving a "not all trails im0plemented" warning with a fixit (code action) to insert it. can/does rust-analyzer do the same ?
Sorry for the comparison to rls. I realise apples/pairs, but it's in relation to the suggestion that rust-analyzer be the "official" LS for rust, which i'm fine with, just YCM uses rls by default (as the "supported" LS) and should that decision go ahead, we'd want to switch to rust-analyzer too.

but the label and what's inserted don't match....i

In general, label and insert text don't need to match. For example, when importing a macro, like in
use std::debug_assert;

the ! is not needed in the insert text (because it is for syntactically disambiguate macro invocations), but i'd still expect to see it in a label, because ! is perceived as part of macro names (for example, docs refer to macros with !).
In this case, I didn't add ! without snippets as that's close to "complete only identifier" model and requires one less case to handle, but, as you find that adding ! is not surprising, I'll add it: #4121

can/does rust-analyzer do the same ?

We do, but only the file is saved. For compiler provided fixits, we just run compiler once in a while, like the old-school flymake/flycheck/ale(?). In the end, we hope to provide the same fixits from in-memory model, but we are not there yet.

(We do provide the "implement missing trait items" fixit though even without saving, we just don't show a diagnostic!)

We do, but only the file is saved

I see, is it based on didSave ?  If so that would explain it (we don't support the didSave notification)....

Client Capability:
property name (optional): textDocument.synchronization.didSave
property type: boolean
The capability indicates that the client supports textDocument/didSave notifications.

(We do provide the "implement missing trait items" fixit though even without saving, we just don't show a diagnostic!)

Oh right. so a code-action in the broad sense. I'll try it.

Well, i just get an error when running that. Apologies, that's off-topic for this conversation, so posting in case you're interested. Feel free to hide this comment (and the previous 2)

Ah yeah, most (if not all) of our code actions require support from the client (i.e. that it implements the rust-analyzer.applySourceChange command). I wonder whether we actually need that in most cases.

most (if not all) of our code actions require support from the client

:(

@flodiebold I apologize if this is off topic, but can you say more about that (or point me to more info about that)?  I've just set up rust-analyzer with vim using vim-lsp, and I'm trying to test with the "fill match arms" code action.  It fails with this error:
{'response': {'id': 39, 'jsonrpc': '2.0', 'error': {'code': -32601, 'message': 'unknown request'}}, 'request': {'id': 39, 'jsonrpc': '2.0', 'method': 'workspace/executeCommand', 'params': {'arguments': [{'label': 'Fill match arms', 'cursorPosition':  ...

In the full error output I see the expected text of the match arms coming back, so this does seem to be a client issue as you alluded to, but I can't find more information about how this is supposed to work.

@davepacheco
RA responds to textDocument/codeAction requests with a CodeAction containing a Command named rust-analyzer.applySourceChange. After this point is where the error comes from.
Your client is trying to handle that Command according to the specification:

Make an ExecuteCommand.
Catch all ApplyEdit server-to-client requests and apply them.

That requires the server to implement ExecuteCommand for each of the named Commands that the server produces.
Instead, rust-analyzer expects clients to know about rust-analyzer.applySourceChange and implement a special case. To make it work in my rust-analyzer branch, I've done this:
    if code_action_command[ 'command' ][ 'command' ] == 'rust-analyzer.applySourceChange':
      code_action_command[ 'edit' ] = code_action_command.pop( 'command' )[ 'arguments' ][ 0 ][ 'workspaceEdit' ]
      return self.CodeActionLiteralToFixIt( request_data, code_action_command )

Except for the name, this is the exact same thing I had to do for java.apply.workspaceEdit. However there's one more thing. rust-analyzer.applySourceChange also contains the "place cursor at X:Y position" extension to WorkspaceEdit object, which I have completely ignored.


[WIP] Issues in the code snippets in Interfaces and Indexers C# programming guide

Interfaces( C# programming guide) code snippets were missing but were linked to the Indexers at docs.microsoft.com
Indexers code snippets containing code for Interfaces.

Further inspection revealed that this might be because of issues in migrating from msdn.microsoft to docs.microsft.
I will resolve this issues at the earliest.

Closed as it was fixed in PR #1598


docs(opendataset): update module docstring of data loaders

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 86.557%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
7598

Relevant Lines:
8778

💛  - Coveralls


Implement some functions (Stencil, Uniform2fv, PixelStorei) to OpenGL…

… and WebGL.

LGTM.
Thank you for doing this!


Support API version v20170131 in apiloader

This change is

@JiangtianLi,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

@acs-bot test this please


sample_azure_iot_embedded_sdk_with_retry.c returns MQTT CONNECT FAIL status:65541

NETXDUO 6.1.7 - Tied to latest NETX distributed by STM32Cube IDE.
Using sample_azure_iot_embedded_sdk_with_retry.c I am consistently getting MQTT CONNECT FAIL status:65541 (0x10005) when trying to connect to my IOT Hub. The IOT hub is available as other clients (Python Azure SDK) can connect to the Hub).  Is there a way to turn on more debugging or some idea of where to start debugging this.
Nx_SNTP_Client application started..
STM32 IpAddress: XXX.XXX.XXX.XXX
SNTP client connected to NTP server : < ca.pool.ntp.org >
SNTP update :
Mar 23, 2022 22:8:18.859 UTC
23-03-2022 / 22:08:18
[INFO] Azure IoT Security Module has been enabled, status=0
IoTHub Host Name: Myazurehub.azure-devices.net; Device ID: mydevice.
[ERROR] IoTHub client connect fail: MQTT CONNECT FAIL status: 65541
Disconnected from IoTHub!: error code = 0x00010005
Failed on nx_azure_iot_hub_client_connect!
reconnecting iothub, after backoff
[ERROR] IoTHub client connect fail: MQTT CONNECT FAIL status: 65541
Disconnected from IoTHub!: error code = 0x00010005
reconnecting iothub, after backoff
[ERROR] IoTHub client connect fail: MQTT CONNECT FAIL status: 65541
Disconnected from IoTHub!: error code = 0x00010005
reconnecting iothub, after backoff

I have also  tried the secondary SAS key and the results are the same.

I have attached my app_netxduo.c file which sets up the IP instance and SNTP client before calling the sample_entry.
`
/* USER CODE BEGIN Header /
/*

@file    app_netxduo.c
@author  MCD Application Team
@brief   NetXDuo applicative file

@attention

Copyright (c) 2021 STMicroelectronics.
All rights reserved.

This software is licensed under terms that can be found in the LICENSE file
in the root directory of this software component.
If no LICENSE file comes with this software, it is provided AS-IS.

/
/ USER CODE END Header */
/* Includes ------------------------------------------------------------------*/
#include "app_netxduo.h"
/* Private includes ----------------------------------------------------------/
/ USER CODE BEGIN Includes */
#include "nx_ip.h"
#include "stm32f4xx_hal_rtc.h"
#include "nx_secure_tls_api.h"
#include <time.h>
/* USER CODE END Includes */
/* Private typedef -----------------------------------------------------------/
/ USER CODE BEGIN PTD */
TX_THREAD AppMainThread;
TX_THREAD AppSNTPThread;
TX_THREAD AppIoTClientThread;
TX_SEMAPHORE Semaphore;
NX_PACKET_POOL           AppPool;
NX_IP                    IpInstance;
NX_DHCP                  DhcpClient;
NX_DNS                   DnsClient;
NX_SNTP_CLIENT           SntpClient;
TX_EVENT_FLAGS_GROUP     SntpFlags;
ULONG                    IpAddress;
ULONG                    NetMask;
CHAR                     buffer[64];
UCHAR                    *pointer;
struct tm timeInfos;
/* RTC handler declaration /
RTC_HandleTypeDef RtcHandle;
/ USER CODE END PTD */
/* Private define ------------------------------------------------------------/
/ USER CODE BEGIN PD */
static UINT kiss_of_death_handler(NX_SNTP_CLIENT *client_ptr, UINT KOD_code);
static void display_rtc_time(RTC_HandleTypeDef *hrtc);
static void rtc_time_update(NX_SNTP_CLIENT *client_ptr);
static UINT unix_time_get(ULONG unix_time);
/ USER CODE END PD */
/* Private macro -------------------------------------------------------------/
/ USER CODE BEGIN PM */
/* USER CODE END PM */
/* Private variables ---------------------------------------------------------/
/ USER CODE BEGIN PV /
/ set the SNTP network interface to the primary interface. /
UINT  iface_index =0;
/ USER CODE END PV */
/* Private function prototypes -----------------------------------------------/
/ USER CODE BEGIN PFP */
static VOID App_Main_Thread_Entry(ULONG thread_input);
static VOID App_SNTP_Thread_Entry(ULONG thread_input);
static VOID App_IoT_Client_Thread_Entry(ULONG thread_input);
static VOID ip_address_change_notify_callback(NX_IP *ip_instance, VOID *ptr);
static VOID time_update_callback(NX_SNTP_TIME_MESSAGE *time_update_ptr, NX_SNTP_TIME *local_time);
VOID sample_entry(NX_IP ip_ptr, NX_PACKET_POOL pool_ptr, NX_DNS dns_ptr, UINT (unix_time_callback)(ULONG unix_time));
//UINT azure_iot_nx_client_entry(
//NX_IP ip_ptr, NX_PACKET_POOL pool_ptr, NX_DNS dns_ptr, UINT (unix_time_callback)(ULONG unix_time));
/ USER CODE END PFP /
/

@brief  Application NetXDuo Initialization.
@param memory_ptr: memory pointer
@retval int
*/
UINT MX_NetXDuo_Init(VOID *memory_ptr)
{
UINT ret = NX_SUCCESS;
TX_BYTE_POOL byte_pool = (TX_BYTE_POOL)memory_ptr;

/* USER CODE BEGIN MX_NetXDuo_MEM_POOL /
/ USER CODE END MX_NetXDuo_MEM_POOL */
/* USER CODE BEGIN MX_NetXDuo_Init */
printf("Nx_SNTP_Client application started..\r\n");
CHAR *pointer;
/* Allocate the memory for packet_pool.  */
if (tx_byte_allocate(byte_pool, (VOID **) &pointer,  NX_PACKET_POOL_SIZE, TX_NO_WAIT) != TX_SUCCESS)
{
return TX_POOL_ERROR;
}
/* Create the Packet pool to be used for packet allocation */
ret = nx_packet_pool_create(&AppPool, "Main Packet Pool", PAYLOAD_SIZE, pointer, NX_PACKET_POOL_SIZE);
if (ret != NX_SUCCESS)
{
return NX_NOT_ENABLED;
}
/* Allocate the memory for Ip_Instance */
if (tx_byte_allocate(byte_pool, (VOID **) &pointer, 4 * DEFAULT_MEMORY_SIZE, TX_NO_WAIT) != TX_SUCCESS)
{
return TX_POOL_ERROR;
}
/* Create the main NX_IP instance */
ret = nx_ip_create(&IpInstance, "Main Ip instance", NULL_ADDRESS, NULL_ADDRESS, &AppPool, nx_stm32_eth_driver,
pointer, 4 * DEFAULT_MEMORY_SIZE, DEFAULT_MAIN_PRIORITY);
if (ret != NX_SUCCESS)
{
return NX_NOT_ENABLED;
}
/* create the DHCP client */
ret = nx_dhcp_create(&DhcpClient, &IpInstance, "DHCP Client");
if (ret != NX_SUCCESS)
{
return NX_NOT_ENABLED;
}
/* Allocate the memory for ARP */
if (tx_byte_allocate(byte_pool, (VOID **) &pointer, ARP_MEMORY_SIZE, TX_NO_WAIT) != TX_SUCCESS)
{
return TX_POOL_ERROR;
}
/* Enable the ARP protocol and provide the ARP cache size for the IP instance */
ret = nx_arp_enable(&IpInstance, (VOID *)pointer, ARP_MEMORY_SIZE);
if (ret != NX_SUCCESS)
{
return NX_NOT_ENABLED;
}
/* Enable the ICMP */
ret = nx_icmp_enable(&IpInstance);
if (ret != NX_SUCCESS)
{
return NX_NOT_ENABLED;
}
/* Enable the UDP protocol required for  DHCP communication */
ret = nx_udp_enable(&IpInstance);
if (ret != NX_SUCCESS)
{
return NX_NOT_ENABLED;
}
/* Enable the TCP protocol */

ret = nx_tcp_enable(&IpInstance);
if (ret != NX_SUCCESS)
{
return NX_NOT_ENABLED;
}
/* Allocate the memory for main thread   */
if (tx_byte_allocate(byte_pool, (VOID **) &pointer, MAIN_THREAD_MEMORY, TX_NO_WAIT) != TX_SUCCESS)
{
return TX_POOL_ERROR;
}
/* Create the main thread */
ret = tx_thread_create(&AppMainThread, "App Main thread", App_Main_Thread_Entry, 0, pointer, MAIN_THREAD_MEMORY,
DEFAULT_MAIN_PRIORITY, DEFAULT_MAIN_PRIORITY, TX_NO_TIME_SLICE, TX_AUTO_START);
if (ret != TX_SUCCESS)
{
return NX_NOT_ENABLED;
}
/* Allocate the memory for SNTP client thread   */
if (tx_byte_allocate(byte_pool, (VOID **) &pointer, SNTP_CLIENT_THREAD_MEMORY, TX_NO_WAIT) != TX_SUCCESS)
{
return TX_POOL_ERROR;
}
/* Initialize TLS.  */
nx_secure_tls_initialize();
/* create the SNTP client thread */
ret = tx_thread_create(&AppSNTPThread, "App SNTP Thread", App_SNTP_Thread_Entry, 0, pointer, SNTP_CLIENT_THREAD_MEMORY,
DEFAULT_PRIORITY, DEFAULT_PRIORITY, TX_NO_TIME_SLICE, TX_DONT_START);
if (ret != TX_SUCCESS)
{
return NX_NOT_ENABLED;
}
/* Create the event flags. */
ret = tx_event_flags_create(&SntpFlags, "SNTP event flags");
/* Check for errors */
if (ret != NX_SUCCESS)
{
return NX_NOT_ENABLED;
}
/* set DHCP notification callback  */
tx_semaphore_create(&Semaphore, "DHCP Semaphore", 0);
#if 0
/* Allocate the memory for IoT client thread   */
if (tx_byte_allocate(byte_pool, (VOID **) &pointer, IOT_CLIENT_THREAD_MEMORY, TX_NO_WAIT) != TX_SUCCESS)
{
return TX_POOL_ERROR;
}
/* create the SNTP client thread */
ret = tx_thread_create(&AppIoTClientThread, "App IoT Client Thread", App_IoT_Client_Thread_Entry, 0, pointer, IOT_CLIENT_THREAD_MEMORY,
DEFAULT_PRIORITY, DEFAULT_PRIORITY, TX_NO_TIME_SLICE, TX_DONT_START);
if (ret != TX_SUCCESS)
{
return NX_NOT_ENABLED;
}
#endif
/* USER CODE END MX_NetXDuo_Init */
return ret;
}
/* USER CODE BEGIN 1 */
/**

@brief  ip address change callback.
@param ip_instance: NX_IP instance
@param ptr: user data
@retval none
*/
static VOID ip_address_change_notify_callback(NX_IP *ip_instance, VOID ptr)
{
/ release the semaphore as soon as an IP address is available */
tx_semaphore_put(&Semaphore);
}

/**

@brief  Main thread entry.

@param thread_input: ULONG user argument used by the thread entry

@retval none
*/
static VOID App_Main_Thread_Entry(ULONG thread_input)
{
UINT ret;
ret = nx_ip_address_change_notify(&IpInstance, ip_address_change_notify_callback, NULL);
if (ret != NX_SUCCESS)
{
Error_Handler();
}
ret = nx_dhcp_start(&DhcpClient);
if (ret != NX_SUCCESS)
{
Error_Handler();
}
/* wait until an IP address is ready */
if(tx_semaphore_get(&Semaphore, TX_WAIT_FOREVER) != TX_SUCCESS)
{
Error_Handler();
}
ret = nx_ip_address_get(&IpInstance, &IpAddress, &NetMask);
if (ret != TX_SUCCESS)
{
Error_Handler();
}
PRINT_IP_ADDRESS(IpAddress);
/* the network is correctly initialized, start the TCP server thread */
tx_thread_resume(&AppSNTPThread);
/* this thread is not needed any more, we relinquish it */
tx_thread_relinquish();
return;
}

/**

@brief  DNS Create Function.

@param dns_ptr

@retval ret
*/
UINT dns_create(NX_DNS *dns_ptr)
{
UINT ret = NX_SUCCESS;
/* Create a DNS instance for the Client */
ret = nx_dns_create(dns_ptr, &IpInstance, (UCHAR *)"DNS Client");
if (ret)
{
Error_Handler();
}
/* Initialize DNS instance with the DNS server Address */
ret = nx_dns_server_add(dns_ptr, DNS_SERVER_ADDRESS);
if (ret)
{
Error_Handler();
}
return ret;
}

/**

@brief  SNTP thread entry.

@param thread_input: ULONG user argument used by the thread entry

@retval none
/
/ Define the client thread.  */
static void App_SNTP_Thread_Entry(ULONG info)
{
UINT ret;
RtcHandle.Instance = RTC;
ULONG  seconds, fraction;
ULONG  events = 0;
UINT   server_status;
NXD_ADDRESS sntp_server_ip;
NX_PARAMETER_NOT_USED(info);
sntp_server_ip.nxd_ip_version = 4;
/* Create a DNS client */
ret = dns_create(&DnsClient);
if (ret != NX_SUCCESS)
{
Error_Handler();
}
/* Look up SNTP Server address. */
ret = nx_dns_host_by_name_get(&DnsClient, (UCHAR *)SNTP_SERVER_NAME,
&sntp_server_ip.nxd_ip_address.v4, DEFAULT_TIMEOUT);
/* Check for error. */
if (ret != NX_SUCCESS)
{
Error_Handler();
}
/* Create the SNTP Client */
ret =  nx_sntp_client_create(&SntpClient, &IpInstance, iface_index, &AppPool, NULL, kiss_of_death_handler, NULL);
/* Check for error. */
if (ret != NX_SUCCESS)
{
Error_Handler();
}
/* Setup time update callback function. */
nx_sntp_client_set_time_update_notify(&SntpClient, time_update_callback);
/* Use the IPv4 service to set up the Client and set the IPv4 SNTP server. */
ret = nx_sntp_client_initialize_unicast(&SntpClient, sntp_server_ip.nxd_ip_address.v4);
if (ret != NX_SUCCESS)
{
Error_Handler();
}
/* Run whichever service the client is configured for. */
ret = nx_sntp_client_run_unicast(&SntpClient);
if (ret != NX_SUCCESS)
{
Error_Handler();
}
else
{
PRINT_CNX_SUCC();
}
/* Wait for a server update event. */
tx_event_flags_get(&SntpFlags, SNTP_UPDATE_EVENT, TX_OR_CLEAR, &events, PERIODIC_CHECK_INTERVAL);
if (events == SNTP_UPDATE_EVENT)
{
/* Check for valid SNTP server status. */
ret = nx_sntp_client_receiving_updates(&SntpClient, &server_status);
if ((ret != NX_SUCCESS) || (server_status == NX_FALSE))
{
/* We do not have a valid update. /
Error_Handler();
}
/ We have a valid update.  Get the SNTP Client time.  */
ret = nx_sntp_client_get_local_time_extended(&SntpClient, &seconds, &fraction, NX_NULL, 0);
ret = nx_sntp_client_utility_display_date_time(&SntpClient,buffer,64);
if (ret != NX_SUCCESS)
{
printf("Internal error with getting local time 0x%x\r\n", ret);
Error_Handler();
}
else
{
printf("\r\nSNTP update :\r\n");
printf("%s\r\n\n",buffer);
}
}
else
{
Error_Handler();
}
/* Set Current time from SNTP TO RTC */
rtc_time_update(&SntpClient);
/* We can stop the SNTP service if for example we think the SNTP server has stopped sending updates */
ret = nx_sntp_client_stop(&SntpClient);
if (ret != NX_SUCCESS)
{
Error_Handler();
}
/* When done with the SNTP Client, we delete it */
ret = nx_sntp_client_delete(&SntpClient);
if (ret != NX_SUCCESS)
{
Error_Handler();
}
/* Toggling LED after a success Time update /
//while(1)
//{
/ Display RTC time each second  /
//display_rtc_time(&RtcHandle);
//BSP_LED_Toggle(LED1);
/ Delay for 1s */
//tx_thread_sleep(100);
//}
display_rtc_time(&RtcHandle);
BSP_LED_Toggle(LED1);
/* the network is correctly initialized, start the TCP server thread */
//tx_thread_resume(&AppIoTClientThread);
/* this thread is not needed any more, we relinquish it */
//tx_thread_relinquish();
sample_entry(&IpInstance, &AppPool, &DnsClient, unix_time_get);
}

/**

@brief  IoT client thread entry.

@param thread_input: ULONG user argument used by the thread entry

@retval none
/
/ Define the IoT client thread.  */
static VOID App_IoT_Client_Thread_Entry(ULONG thread_input)
{
UNUSED(thread_input);
/* Initialize TLS.  */
//nx_secure_tls_initialize();
//sample_entry(&IpInstance, &AppPool, &DnsClient, unix_time_get);
}

/* This application defined handler for handling a Kiss of Death packet is not
required by the SNTP Client. A KOD handler should determine
if the Client task should continue vs. abort sending/receiving time data
from its current time server, and if aborting if it should remove
the server from its active server list.
Note that the KOD list of codes is subject to change. The list
below is current at the time of this software release. */
static UINT kiss_of_death_handler(NX_SNTP_CLIENT *client_ptr, UINT KOD_code)
{
UINT    remove_server_from_list = NX_FALSE;
UINT    status = NX_SUCCESS;
NX_PARAMETER_NOT_USED(client_ptr);
/* Handle kiss of death by code group. */
switch (KOD_code)
{
case NX_SNTP_KOD_RATE:
case NX_SNTP_KOD_NOT_INIT:
case NX_SNTP_KOD_STEP:
/* Find another server while this one is temporarily out of service.  */
status =  NX_SNTP_KOD_SERVER_NOT_AVAILABLE;

break;

case NX_SNTP_KOD_AUTH_FAIL:
case NX_SNTP_KOD_NO_KEY:
case NX_SNTP_KOD_CRYP_FAIL:
/* These indicate the server will not service client with time updates
without successful authentication. */

remove_server_from_list =  NX_TRUE;

break;

default:
/* All other codes. Remove server before resuming time updates. */

remove_server_from_list =  NX_TRUE;
break;

}
/* Removing the server from the active server list? */
if (remove_server_from_list)
{
/* Let the caller know it has to bail on this server before resuming service. */
status = NX_SNTP_KOD_REMOVE_SERVER;

}
return status;
}
/* This application defined handler for notifying SNTP time update event.  */
static VOID time_update_callback(NX_SNTP_TIME_MESSAGE *time_update_ptr, NX_SNTP_TIME *local_time)
{
NX_PARAMETER_NOT_USED(time_update_ptr);
NX_PARAMETER_NOT_USED(local_time);
tx_event_flags_set(&SntpFlags, SNTP_UPDATE_EVENT, TX_OR);
}
/* This application updates Time from SNTP to STM32 RTC */
static void rtc_time_update(NX_SNTP_CLIENT *client_ptr)
{
RTC_DateTypeDef sdatestructure;
RTC_TimeTypeDef stimestructure;
struct tm ts;
CHAR  temp[2] = {0, 0};
/* convert SNTP time (seconds since 01-01-1900 to 01-01-1970)
EPOCH_TIME_DIFF is equivalent to 70 years in sec
calculated with www.epochconverter.com/date-difference
This constant is used to delete difference between :
Epoch converter (referenced to 1970) and SNTP (referenced to 1900) */
time_t timestamp = client_ptr->nx_sntp_current_server_time_message.receive_time.seconds - EPOCH_TIME_DIFF;
/* convert time in yy/mm/dd hh:mm:sec */
ts = *localtime(&timestamp);
/* convert date composants to hex format /
sprintf(temp, "%ld",  (unsigned long)(ts.tm_year - 100));
sdatestructure.Year = strtol(temp, NULL, 16);
sprintf(temp, "%ld", (unsigned long) (ts.tm_mon + 1));
sdatestructure.Month = strtol(temp, NULL, 16);
sprintf(temp, "%d", ts.tm_mday);
sdatestructure.Date = strtol(temp, NULL, 16);
/ dummy weekday */
sdatestructure.WeekDay =0x00;
if (HAL_RTC_SetDate(&RtcHandle, &sdatestructure, RTC_FORMAT_BCD) != HAL_OK)
{
Error_Handler();
}
/* convert time composants to hex format */
sprintf(temp,"%d", ts.tm_hour);
stimestructure.Hours = strtol(temp, NULL, 16);
sprintf(temp,"%d", ts.tm_min);
stimestructure.Minutes = strtol(temp, NULL, 16);
sprintf(temp, "%d", ts.tm_sec);
stimestructure.Seconds = strtol(temp, NULL, 16);
if (HAL_RTC_SetTime(&RtcHandle, &stimestructure, RTC_FORMAT_BCD) != HAL_OK)
{
Error_Handler();
}
}
/* this application displays time from RTC */
static void display_rtc_time(RTC_HandleTypeDef *hrtc)
{
RTC_TimeTypeDef RTC_Time;
RTC_DateTypeDef RTC_Date;
HAL_RTC_GetTime(&RtcHandle,&RTC_Time,RTC_FORMAT_BCD);
HAL_RTC_GetDate(&RtcHandle,&RTC_Date,RTC_FORMAT_BCD);
printf("%02x-%02x-20%02x / %02x:%02x:%02x\r\n",
RTC_Date.Date, RTC_Date.Month, RTC_Date.Year,RTC_Time.Hours,RTC_Time.Minutes,RTC_Time.Seconds);
}
static UINT unix_time_get(ULONG *unix_time)
{
RTC_TimeTypeDef RTC_Time;
RTC_DateTypeDef RTC_Date;
HAL_RTC_GetTime(&RtcHandle,&RTC_Time,RTC_FORMAT_BIN);
HAL_RTC_GetDate(&RtcHandle,&RTC_Date,RTC_FORMAT_BIN);
uint8_t hh = RTC_Time.Hours;
uint8_t mm = RTC_Time.Minutes;
uint8_t ss = RTC_Time.Seconds;
uint8_t d = RTC_Date.Date;
uint8_t m = RTC_Date.Month;
uint16_t y = RTC_Date.Year;
uint16_t yr = (uint16_t)(y+2000-1900);
time_t currentTime = {0};
struct tm tim = {0};
tim.tm_year = yr;
tim.tm_mon = m - 1;
tim.tm_mday = d;
tim.tm_hour = hh;
tim.tm_min = mm;
tim.tm_sec = ss;
currentTime = mktime(&tim);
/* Return number of seconds since Unix Epoch (1/1/1970 00:00:00).  */
*unix_time =  (ULONG)currentTime;
return(NX_SUCCESS);
}
/* USER CODE END 1 */
`

Forgot to attach the console output
`
Nx_SNTP_Client application started..
STM32 IpAddress: XXX.XXX.X.XXX
SNTP client connected to NTP server : < fr.pool.ntp.org >
SNTP update :
Mar 24, 2022 15:29:10.374 UTC
24-03-2022 / 15:29:10
24-03-2022 / 15:29:10
24-03-2022 / 15:29:10
24-03-2022 / 15:29:10
24-03-2022 / 15:29:10
24-03-2022 / 15:29:10
24-03-2022 / 15:29:10
24-03-2022 / 15:29:10
[INFO] Azure IoT Security Module has been enabled, status=0
IoTHub Host Name: myhub.azure-devices.net; Device ID: stm32F429.
24-03-2022 / 15:29:10
24-03-2022 / 15:29:10
24-03-2022 / 15:29:10
24-03-2022 / 15:29:12
24-03-2022 / 15:29:12
24-03-2022 / 15:29:12
24-03-2022 / 15:29:13
24-03-2022 / 15:29:13
24-03-2022 / 15:29:13
24-03-2022 / 15:29:14
24-03-2022 / 15:29:15
24-03-2022 / 15:29:16
24-03-2022 / 15:29:16
24-03-2022 / 15:29:16
24-03-2022 / 15:29:17
24-03-2022 / 15:29:18
24-03-2022 / 15:29:19
24-03-2022 / 15:29:19
24-03-2022 / 15:29:19
24-03-2022 / 15:29:19
24-03-2022 / 15:29:20
24-03-2022 / 15:29:21
[ERROR] IoTHub client connect fail: MQTT CONNECT FAIL status: 65541
Disconnected from IoTHub!: error code = 0x00010005
Failed on nx_azure_iot_hub_client_connect!
reconnecting iothub, after backoff
24-03-2022 / 15:29:22
24-03-2022 / 15:29:22
24-03-2022 / 15:29:22
24-03-2022 / 15:29:23
24-03-2022 / 15:29:24
24-03-2022 / 15:29:25
24-03-2022 / 15:29:25
24-03-2022 / 15:29:25
24-03-2022 / 15:29:25
24-03-2022 / 15:29:25
24-03-2022 / 15:29:26
24-03-2022 / 15:29:28
24-03-2022 / 15:29:28
24-03-2022 / 15:29:28
24-03-2022 / 15:29:28
24-03-2022 / 15:29:29
24-03-2022 / 15:29:30
24-03-2022 / 15:29:31
24-03-2022 / 15:29:31
24-03-2022 / 15:29:31
24-03-2022 / 15:29:32
24-03-2022 / 15:29:33
24-03-2022 / 15:29:33
24-03-2022 / 15:29:34
24-03-2022 / 15:29:34
24-03-2022 / 15:29:34
24-03-2022 / 15:29:35
`

Hi @dsilva-vd , I saw there are only 10 packets in the pool. What is the value of RX BD? One simple test you can try is to increase packet pool count from 10 to 30 and see if the problem is solved.

Hi @TiejunMS I increased NX_APP_MEM_POOL_SIZE to 70kBytes and increased the packet pool count to 30. This didn't change the behavior.
Just to confirm these are the changes I made:
#define PAYLOAD_SIZE                 1536 #define NX_PACKET_POOL_NUM_PACKETS   30 #define NX_PACKET_POOL_SIZE          (( PAYLOAD_SIZE + sizeof(NX_PACKET)) * NX_PACKET_POOL_NUM_PACKETS)
PACKET_POOL_SIZE is used to allocate the packet pool
What is RX BD? Can you point me to a file or name of define I can search for?
Thanks

If you are using official driver from ST, then the value is defined here. Since the problem is not solved after increasing the packet pool, it is not due to lack of packet. Are you able to capture packet trace by wireshark and share with us? It is helpful to understand what happens during the connection.

Hi I will try and capture a Wireshark but as the device is a wired device I will try and find a way to do man in the middle listening.
The defines are as follows
#define ETH_MAX_PACKET_SIZE       1524U    /*!< ETH_HEADER + ETH_EXTRA + ETH_VLAN_TAG + ETH_MAX_ETH_PAYLOAD + ETH_CRC */ #define ETH_RX_BUF_SIZE                ETH_MAX_PACKET_SIZE /* buffer size for receive               */ #define ETH_TX_BUF_SIZE                ETH_MAX_PACKET_SIZE /* buffer size for transmit              */ #define ETH_RXBUFNB                    4U       /* 4 Rx buffers of size ETH_RX_BUF_SIZE  */ #define ETH_TXBUFNB                    4U       /* 4 Tx buffers of size ETH_TX_BUF_SIZE  */

ST32 Ethernet - bridge mode adapter.zip
Hi @TiejunMS Here is a wireshark capture. The ST32 device has address <IP_ADDRESS>. The network connectivity is bridged through <IP_ADDRESS> (which was running Wireshark).
I don't see any packets from the the ST32 device indicating it is starting the MQTT connection to the Azure IoT Hub. I await your thoughts on reviewing the wireshark capture.
Thanks

@dsilva-vd , it does not look like the bridge is working. Could you double check the bridge configurations?
Btw, if you would like to share your sample_config.h with us for checking, feel free to send it to<EMAIL_ADDRESS>

@TiejunMS Here are the files for my iot client. I believe I renamed the sample_config.h to azure_iot_client_config.h
IoTClient.zip
I will look into the bridging configuration again tomorrow. Thanks

@dsilva-vd , could you fill in the valid hostname, device_id and symmetric key in the config.h? Just want to double check the configurations are all correct. You can delete the device once it is verified. To avoid credential leak publicly, you can send it through support email listed above.

@TiejunMS I sent in the config file with credentials to the support email today. I'll keep working on the wireshark capture.

@dsilva-vd , I have verified the credentials you shared are valid in our official sample. If you are using the same sample, then the issue might be related to Ethernet driver. Did you port the driver or it is provided by ST? Anyway, a packet trace will be helpful to show us what happened during the connection.

@TiejunMS The Ethernet driver is provided by ST in an example on setting up a simple SNTP client (using Ethernet). I took that example and after successfully connecting to SNTP server and getting a time update I start the Azure IoT embedded client thread.

@TiejunMS I'm closing this issue. STM has released an AZure RTOS version of their cellular FW package and my end goal is azure connection with cellular PPP interface. The ethernet example was for temporary development. I will re-integrate the azure iot embedded client with the cellular FW package and open a new ticket if I encounter any issues.
Thanks

Thanks for letting me know. Feel free to reopen the issue if needed.

@TiejunMS I need to reopen this issue.
I have been able to integrate the Azure IoT client  (sample_azure_iot_embedded_skd_with_retry, Netxduo 6.1.7 release) with the Azure RTOS STM32 Cellular package.
The IoT client behavior is exactly the same as when I was running the client over Ethernet.  The same function call returns the error value. The last time you helped troubleshoot this issue you wanted to see a Wireshark capture. That won't be simple with the cellular modem so is there another option to troubleshoot this?
I could purchase the same reference kit used in the Netx example code i.e. B-L475E-IOT01A2 (B-L475E-IOT01A1) and run the same example to capture Wireshark but as that example works I'm not sure what it would tell us.
if this is helpful I've attached a screenshot of the ThreadX thread list the moment the mqtt connect call is made. Perhaps there is a thread priority issue that you can see

Forgot to reopen issue

@dsilva-vd , since you were able to reproduce the issue on both Ethernet and Cellular interfaces, I think the issue is related to application. So could you capture the packets through Ethernet which should be much simpler?

Hi @TiejunMS I captured a wireshark capture using an passive network tap. I'm not sure I have the correct wireshark settings but here is the captured log. The device's IP is <IP_ADDRESS>
Ethernet Azure IoT Hub capture.zip

Updated capture. The first one was not captured properly and I deleted it.  I can see from the logs there is a TLS failure (Fatal Alert)
Ethernet Azure IoT Hub capture.zip

Hi @TiejunMS ,
I had removed the 1st cert as I saw the comment that MS was going to transition to Digicert and removed it. After adding the cert back in the Ethernet example worked.  I also found on the ethernet sample that I needed a larger packet pool for Net X.
I made the same changes (adding back in root cert 1) on my cellular example and successfully connected there as well.
Thank you for your help!

Good to know it is working!

Hi @dsilva-vd, the start date of the Azure IoT root certificate migration from June1st 2022, to no earlier than Feb 15th, 2023. you may check here: https://techcommunity.microsoft.com/t5/internet-of-things-blog/azure-iot-tls-critical-changes-are-almost-here-and-why-you/ba-p/2393169


Added pre- / post-transcribe hooks to facilitate media essence embedding

Link the Issue(s) this Pull Request is related to.
Requires change to OTIO core (for tests to pass):
https://github.com/AcademySoftwareFoundation/OpenTimelineIO/pull/1711
Summarize your change.
This PR adds AAF adapter specific hooks that can be used to implement just-in-time transcoding of media for embedding purposes. It also adds the embed_essence argument to the write functionality, which will attempt to import media references. The new create_edgecode argument allows to control if an edge code should be applied to the AAF. Only .wav, .dnx and .aaf can be embedded. Thanks a lot @markreidvfx for your help with the embedding code.
New hooks introduced:

otio_aaf_pre_write_transcribe: useful for transcoding media for embedding
otio_aaf_post_write_transcribe: useful for cleaning up in-memory timeline / reverting to original media
otio_aaf_pre_read_transcribe: useful to modify AAF structure in-memory before read
otio_aaf_post_read_transcribe: useful to extract media from AAF file post timeline transcription

Reference associated tests.
Draft PR since I'm missing a test for .aaf embedding still
tests/test_aaf_adapter.py

@markreidvfx I confirmed all but one of the tests producing valid results in Avid MC. However I can't get the audio import test to produce results that MC can load. It errors out on import. Any chance you could have a look at this? Otherwise this PR is good for review now. :)

@reinecke @jminor After our discussion I reworked the code a little and it now offers both read and write hooks to facilitate embedding & extracting of media. I also added test coverage for all supported media types using snippets from Picchu.


Error unmarshaling campaign list -- wrong type for 'settings' field

Calling the client.Campaign.List method results in the following error:
json: cannot unmarshal array into Go struct field Campaign.data.settings of type mailerlite.CampaignSettings
The API version that the client specifies (2022-12-01) causes the server to return an array for the 'settings' field of a campaign, while the client expects an object. (It looks like the server started returning an object instead of an array at version 2023-01-11).

Yes, that seems to have worked.


Flushing most events

I'm curious why you're explicitly flushing on most calls? Seems like that's going to consume a fair bit of bandwidth for common uses

I second this. HTTP request for every track and trackWithProperties is a lot.
https://github.com/mixpanel/mixpanel-iphone/blob/84f116804a48312f397f5ae09abea76d3cf5083f/Mixpanel/Mixpanel.h#L570
mixpanel docs say it'll flush every minute by default.
or you can init with flushInterval to override (0 is never):
https://github.com/mixpanel/mixpanel-iphone/blob/84f116804a48312f397f5ae09abea76d3cf5083f/Mixpanel/Mixpanel.h#L322
They also use app lifecycle to flush as well, when it enters the background it'll flush then too which is probably just enough on its own regardless of the minute timer.

The reason behind this is that not knowing the type of project this library will be used in, I'd prefer to be on the safe side. Most apps fail to get an average session time of 1 minute anyway.
What would be nice would be to create a separate call for flushing, or flushing when the app goes in background.
PR anybody? :)

It should already be flushing when you go in the background. That's what
mix panel does by default, on my mobile but I thought I already linked to
that? Maybe not
On Mon, Feb 8, 2016 at 11:48 PM Davide Scalzo<EMAIL_ADDRESS>wrote:

The reason behind this is that not knowing the type of project this
library will be used in, I'd prefer to be on the safe side. Most apps fail
to get an average session time of 1 minute anyway.
What would be nice would be to create a separate call for flushing, or
flushing when the app goes in background.
PR anybody? :)
—
Reply to this email directly or view it on GitHub
https://github.com/davodesign84/react-native-mixpanel/issues/10#issuecomment-181749522
.

What would be nice would be to create a separate call for flushing, or flushing when the app goes in background.

Mixpanel's library does this already


Response performance

Hi,
Version 5.6.1 (no issue with 5.5.1)
JSON response with ~10MB size is not processed neither via Chrome neither via FF browsers - both pops up question "Stop or Wait"
Not related to performance but we host same API on syst, acc and prod envs:
On some envs FF generates view:

This one is preffered.
On others:

There's no way Swashbuckle can be affecting the actual performance of your endpoints. So, I'm guessing this is likely an issue with the updated version of swagger-ui that comes with Swashbuckle 5.6.1.
To confirm this, you could try using the previous version for the SwaggerUI package only. For example, something like the following in your Startup.cs:
  <ItemGroup>
    <PackageReference Include="Swashbuckle.AspNetCore.Swagger" Version="5.6.1" />
    <PackageReference Include="Swashbuckle.AspNetCore.SwaggerGen" Version="5.6.1" />
    <PackageReference Include="Swashbuckle.AspNetCore.SwaggerUI" Version="5.5.1" />
  </ItemGroup>

If the error goes away with this set of packages then that indicates an issue with the latest version of swagger-ui, specifically 3.32.5. With that being the case, I would recommend posting an issue to that repo instead


[GUIDE] Add bulk guide

Description
Add bulk guide.
Issues Resolved
Closes [#278, #279].
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

You're missing DCO, same as other PRs.

fixed


-b, --bssid arg

Hi! Great work on hcxtools!
I'm wondering if it's possible to get similar functionality to aircrack-ng's --bssid arg to specify a specific AP, and without having to create a file and pass it in with --filterlist_ap, etc. Would be a lot quiter and less tedious.
Hopefully this feature isn't already here and I'm just oblivious...
Thank you!

How should the BSSID option work:
as display filter?
as capture filter (in that case, useful packets will get lost)?
as attack filter?
as protect filter?
By the current  hcxdumptool options you can set, what ever you want, from totally silent to aggressive transmission, from display filter to  to capture filter, from attack (target) filter to protect (own devices) filter. In addition to that, you can reload the filter list while hcxdumptool is running.
Therefore we need at least two option switches: target filter list and kind of filter.
hcxdumptool's filtering is totally different to aircrack-ng (or better explained: the whole engine is totally different to aircrack-ng). On aircrack-ng suite you have different tools:
airodump-ng to capture (receive only)
aireplay-ng to attack (transmit only)
Here is an example: filtered airodump-ng cap file:
$ hcxpcapngtool 6.2.0-42-gcb7bf91 reading from 5a2ce49f548c0c44f532859236232f01.cap...

summary capture file
--------------------
file name................................: 5a2ce49f548c0c44f532859236232f01.cap
version (pcap/cap).......................: 2.4 (very basic format without any additional information)
timestamp minimum (GMT)..................: 03.08.2021 19:50:48
timestamp maximum (GMT)..................: 03.08.2021 21:03:05
used capture interfaces..................: 1
link layer header type...................: DLT_IEEE802_11 (105)
endianess (capture system)...............: little endian
packets inside...........................: 17976
BEACON (total)...........................: 1
ACTION (total)...........................: 41
PROBERESPONSE............................: 241
DEAUTHENTICATION (total).................: 12879
AUTHENTICATION (total)...................: 20
AUTHENTICATION (OPEN SYSTEM).............: 20
ASSOCIATIONREQUEST (total)...............: 3
ASSOCIATIONREQUEST (PSK).................: 3
WPA encrypted............................: 353
EAPOL messages (total)...................: 25
EAPOL RSN messages.......................: 25
ESSID (total unique).....................: 1
EAPOLTIME gap (measured maximum usec)....: 9752
EAPOL ANONCE error corrections (NC)......: working
REPLAYCOUNT gap (recommended NC).........: 8
EAPOL M1 messages (total)................: 14
EAPOL M2 messages (total)................: 1
EAPOL M3 messages (total)................: 10
EAPOL pairs (total)......................: 1
EAPOL pairs (best).......................: 1
EAPOL M12E2 (challenge)..................: 1
PMKID (useless)..........................: 14

Warning: out of sequence timestamps!
This dump file contains frames with out of sequence timestamps.
That is a bug of the capturing tool.

Warning: excessive number of deauthentication/disassociation frames detected!
That can cause that an ACCESS POINT change channel, reset EAPOL TIMER,
renew ANONCE and set PMKID to zero.
This could prevent to calculate a valid EAPOL MESSAGE PAIR
or to get a valid PMKID.

Warning: missing frames!
This dump file does not contain undirected proberequest frames.
An undirected proberequest may contain information about the PSK.
It always happens if the capture file was cleaned or
it could happen if filter options are used during capturing.
That makes it hard to recover the PSK.

The last two warnings will explain what happened while doing the aircrack-ng way:
Warning: missing frames!
Especially if the CLIENT use MAC randomization, all information of it will get lost if you set a BSSID filter like the aircrack-ng way.
Warning: excessive number of deauthentication/disassociation frames detected!
aireplay-ng and airodump-ng don't talk with each other and the deauthentication went on and on and on, even if an EAPOL 4way handshake was received.

compared to a hcxdumptool/hcxlabtool pcapng file:
hcxpcapngtool 6.2.0-42-gcb7bf91 reading from 389797df3a6041522dd7d321faf33c59.pcapng...

summary capture file
--------------------
file name................................: 389797df3a6041522dd7d321faf33c59.pcapng
version (pcapng).........................: 1.0
operating system.........................: Linux 5.10.36-2-ARCH
application..............................: hcxlabtool 0.1.0
interface name...........................: wlan0
interface vendor.........................: 74da38
openSSL version..........................: 1.1
weak candidate...........................: 12345678
MAC ACCESS POINT.........................: a468bc027b0d (incremented on every new client)
MAC CLIENT...............................: fcc233e9f145
REPLAYCOUNT..............................: 64550
ANONCE...................................: 389797df3a6041522dd7d321faf33c59e20246fae16baada9653747eac25e55c
SNONCE...................................: ee8e5600006bcf37d77bc1933736f3f370f9e100daa2bd58b2e975eef1687e48
timestamp minimum (GMT)..................: 19.05.2021 03:29:13
timestamp maximum (GMT)..................: 19.05.2021 09:24:19
used capture interfaces..................: 1
link layer header type...................: DLT_IEEE802_11_RADIO (127)
endianess (capture system)...............: little endian
packets inside...........................: 99338
WIRELESS DISTRIBUTION SYSTEM.............: 92
BEACON (total)...........................: 3211
PROBEREQUEST.............................: 4357
PROBEREQUEST (directed)..................: 18
PROBERESPONSE............................: 1839
AUTHENTICATION (total)...................: 2146
AUTHENTICATION (OPEN SYSTEM).............: 2146
ASSOCIATIONREQUEST (total)...............: 1248
ASSOCIATIONREQUEST (PSK).................: 1155
REASSOCIATIONREQUEST (total).............: 74
REASSOCIATIONREQUEST (PSK)...............: 40
IDENTITIES...............................: 9
EAP (total)..............................: 1118
EAP CODE request.........................: 764
EAP CODE response........................: 24
EAP ID...................................: 409
EAP-SIM..................................: 34
EAP-AKA..................................: 345
EAPOL messages (total)...................: 84728
EAPOL RSN messages.......................: 84666
EAPOL WPA messages.......................: 62
ESSID (total unique).....................: 5574
ESSID changes (detected maximum).........: 1 (information: option --max-essids=2 and --all recommended)
EAPOLTIME gap (measured maximum usec)....: 78780448
EAPOL ANONCE error corrections (NC)......: working
REPLAYCOUNT gap (suggested NC)...........: 14
EAPOL M1 messages (total)................: 26850
EAPOL M2 messages (total)................: 57012
EAPOL M3 messages (total)................: 727
EAPOL M4 messages (total)................: 139
EAPOL pairs (total)......................: 403107
EAPOL pairs (best).......................: 951
EAPOL ROGUE pairs........................: 893
EAPOL M12E2 (challenge)..................: 918
EAPOL M32E2 (authorized).................: 32
EAPOL M34E4 (authorized).................: 1
PMKID (useless)..........................: 205
PMKID (total)............................: 510
PMKID (best).............................: 75
PMKID ROGUE..............................: 53
malformed packets (total)................: 2
ESSID error (malformed packets)..........: 2

session summary
---------------
processed pcapng files................: 1

...big difference.

Please also notice:
target of aircrack-ng = AP (mostly hardened)
target of hcxdumptool/hcxlabtool = CLIENT (mostly weak)

"Would be a lot quieter and less tedious"
No.
From --help:
--filtermode=<digit> : user space filter mode for filter list
                                 ...
                                  notice: this filter option will let hcxdumptool protect or attack a target - it is neither a capture nor a display filter

What you see in real time display is not what hcxdumptool transmit

I think I found a solution.
By this commit:
https://github.com/ZerBea/hcxdumptool/commit/e9099ce43dcde9d75b228f992817e4789d932d1a
you can either use a filter list or a single MAC
example using a filter list:
$ sudo hcxdumptool -i wlp39s0f3u1u5 --filterlist_ap=filterlistap --filtermode=1
initialization of hcxdumptool 6.2.0-25-g8f86712...

start capturing (stop with ctrl+c)
NMEA 0183 SENTENCE........: N/A
INTERFACE NAME............: wlp39s0f3u1u5
INTERFACE PROTOCOL........: IEEE 802.11
INTERFACE HARDWARE MAC....: 1cbfce3b5bdf (not used for the attack)
INTERFACE VIRTUAL MAC.....: 1cbfce3b5bdf (not used for the attack)
DRIVER....................: rt2800usb
DRIVER VERSION............: 5.13.10-arch1-1
DRIVER FIRMWARE VERSION...: 0.36
openSSL version...........: 1.1
ERRORMAX..................: 100 errors
BPF code blocks...........: 128
FILTERLIST ACCESS POINT...: 4 entries
FILTERLIST CLIENT.........: 0 entries
FILTERMODE................: protect
WEAK CANDIDATE............: 12345678
ESSID list................: 0 entries
ACCESS POINT (ROGUE)......: 0022f160ec99 (BROADCAST HIDDEN used for the attack)
ACCESS POINT (ROGUE)......: 0022f160ec9a (BROADCAST OPEN used for the attack)
ACCESS POINT (ROGUE)......: 0022f160ec9b (used for the attack and incremented on every new client)
CLIENT (ROGUE)............: e0cb1dc49825
EAPOLTIMEOUT..............: 20000 usec
EAPOLEAPTIMEOUT...........: 2500000 usec
REPLAYCOUNT...............: 63530
ANONCE....................: 6d452ea4854a0d5e586fc78acee42b0cd1a83248f725e341e78cec3b42ecb908
SNONCE....................: b78fe1c7a96ed0bce4d0bc0b2cb01a99d07f4d37120583bb3030963c9f6e780b

example using a single MAC instead of a filter list:
$ sudo hcxdumptool -i wlp39s0f3u1u5 --filterlist_ap=11:22:33:44:55:66 --filtermode=1
initialization of hcxdumptool 6.2.0-25-g8f86712...

start capturing (stop with ctrl+c)
NMEA 0183 SENTENCE........: N/A
INTERFACE NAME............: wlp39s0f3u1u5
INTERFACE PROTOCOL........: IEEE 802.11
INTERFACE HARDWARE MAC....: 1cbfce3b5bdf (not used for the attack)
INTERFACE VIRTUAL MAC.....: 1cbfce3b5bdf (not used for the attack)
DRIVER....................: rt2800usb
DRIVER VERSION............: 5.13.10-arch1-1
DRIVER FIRMWARE VERSION...: 0.36
openSSL version...........: 1.1
ERRORMAX..................: 100 errors
BPF code blocks...........: 128
FILTERLIST ACCESS POINT...: 1 entries
FILTERLIST CLIENT.........: 0 entries
FILTERMODE................: protect
WEAK CANDIDATE............: 12345678
ESSID list................: 0 entries
ACCESS POINT (ROGUE)......: 1000aa5c9b49 (BROADCAST HIDDEN used for the attack)
ACCESS POINT (ROGUE)......: 1000aa5c9b4a (BROADCAST OPEN used for the attack)
ACCESS POINT (ROGUE)......: 1000aa5c9b4b (used for the attack and incremented on every new client)
CLIENT (ROGUE)............: e00db913fb69
EAPOLTIMEOUT..............: 20000 usec
EAPOLEAPTIMEOUT...........: 2500000 usec
REPLAYCOUNT...............: 64013
ANONCE....................: 456300291ca7689ee9d583cea5983188aa3b10b9f102714f22456379336d0759
SNONCE....................: 3545a38e8ed68f7481e9bd531f65f5f56c2be3abe0d9a0daa426148891e3f531

or

$ sudo hcxdumptool -i wlp39s0f3u1u5 --filterlist_ap=112233445566 --filtermode=1
initialization of hcxdumptool 6.2.0-25-g8f86712...

start capturing (stop with ctrl+c)
NMEA 0183 SENTENCE........: N/A
INTERFACE NAME............: wlp39s0f3u1u5
INTERFACE PROTOCOL........: IEEE 802.11
INTERFACE HARDWARE MAC....: 1cbfce3b5bdf (not used for the attack)
INTERFACE VIRTUAL MAC.....: 1cbfce3b5bdf (not used for the attack)
DRIVER....................: rt2800usb
DRIVER VERSION............: 5.13.10-arch1-1
DRIVER FIRMWARE VERSION...: 0.36
openSSL version...........: 1.1
ERRORMAX..................: 100 errors
BPF code blocks...........: 128
FILTERLIST ACCESS POINT...: 1 entries
FILTERLIST CLIENT.........: 0 entries
FILTERMODE................: protect
WEAK CANDIDATE............: 12345678
ESSID list................: 0 entries
ACCESS POINT (ROGUE)......: 205b2ac978eb (BROADCAST HIDDEN used for the attack)
ACCESS POINT (ROGUE)......: 205b2ac978ec (BROADCAST OPEN used for the attack)
ACCESS POINT (ROGUE)......: 205b2ac978ed (used for the attack and incremented on every new client)
CLIENT (ROGUE)............: b025aa8496a1
EAPOLTIMEOUT..............: 20000 usec
EAPOLEAPTIMEOUT...........: 2500000 usec
REPLAYCOUNT...............: 62841
ANONCE....................: 061bd252325382f439f440b15542656a57626bbeb6e76821835962ecee6f9cc0
SNONCE....................: e977e41581cad5672638cba863b7ed6b862ab51d9239dda195c40c37ae55d9f4

Please notice:
This is a filter within the transmit branch. It is neither a capture filter, nor a display filter. You can't compare this filtering with the method used by aircrack-ng.

I think this is what I was looking for. Not necessarily a packet filter, but in the sense of "okay, my org has access points with SSID 'X', I found one access point and have the MAC address, let's target that quickly, aka do not need to make a separate file with the MAC. Just pass it in as an arg.
Thank you!

It's a fine improvement and I'm glad that I could help.
Cheers
Mike


Converter appears to be unhappy with some file uploads

Error message from cloudrun console/logs:
2022/01/18 19:55:42 http: panic serving <IP_ADDRESS>:58343: runtime error: index out of range [5] with length 4
goroutine 77 [running]:
net/http.(*conn).serve.func1()
/usr/local/go/src/net/http/server.go:1802 +0xb9
panic({0xaaeae0, 0xc00a0f2828})
/usr/local/go/src/runtime/panic.go:1047 +0x266
github.com/osrg/gobgp/pkg/packet/bgp.(*IPAddrPrefixDefault).decodePrefix(0xc009de3f80, {0xc003ed83f5, 0x0, 0x12}, 0x2a, 0xb3)
/go/pkg/mod/github.com/osrg/gobgp@v2.0.0+incompatible/pkg/packet/bgp/bgp.go:1178 +0x189
github.com/osrg/gobgp/pkg/packet/bgp.(*IPAddrPrefix).DecodeFromBytes(0xc009de3f80, {0xc003ed83f4, 0x13, 0x13}, {0x0, 0x0, 0x0})
/go/pkg/mod/github.com/osrg/gobgp@v2.0.0+incompatible/pkg/packet/bgp/bgp.go:1230 +0x134
github.com/osrg/gobgp/pkg/packet/bgp.(*BGPUpdate).DecodeFromBytes(0xc00a384e40, {0xc003ed836b, 0x203002, 0x203002}, {0x0, 0x0, 0x0})
/go/pkg/mod/github.com/osrg/gobgp@v2.0.0+incompatible/pkg/packet/bgp/bgp.go:9118 +0xcad
github.com/osrg/gobgp/pkg/packet/bgp.parseBody(0xc00047cf20, {0xc003ed836b, 0x98, 0x9c}, {0x0, 0x0, 0x0})
/go/pkg/mod/github.com/osrg/gobgp@v2.0.0+incompatible/pkg/packet/bgp/bgp.go:9382 +0x278
github.com/osrg/gobgp/pkg/packet/bgp.ParseBGPMessage({0xc003ed8358, 0xc003ed8344, 0x40ee07}, {0x0, 0x0, 0x0})
/go/pkg/mod/github.com/osrg/gobgp@v2.0.0+incompatible/pkg/packet/bgp/bgp.go:9397 +0xfc
github.com/osrg/gobgp/pkg/packet/mrt.(*BGP4MPMessage).DecodeFromBytes(0xc009de3e00, {0xc003ed8344, 0xc00047cfb8, 0x979a86})
/go/pkg/mod/github.com/osrg/gobgp@v2.0.0+incompatible/pkg/packet/mrt/mrt.go:810 +0x45
github.com/osrg/gobgp/pkg/packet/mrt.ParseMRTBody(0xc00047d13c, {0xc003ed8344, 0xc00047d030, 0xc3})
/go/pkg/mod/github.com/osrg/gobgp@v2.0.0+incompatible/pkg/packet/mrt/mrt.go:1001 +0x612
github.com/routeviews/google-cloud-storage/pkg/mrt_converter.parseUpdate({0xc0056d5721, 0x10}, 0xc00047d13c, {0xc003ed8340, 0xc7, 0xc7})
/app/pkg/mrt_converter/mrt_converter.go:159 +0x111
github.com/routeviews/google-cloud-storage/pkg/mrt_converter.convert({0xc0056d5721, 0x10}, {0xc01522a000, 0x1d00bb, 0x244000}, {0xbc2660, 0xc0022ce930}, 0xb27d28)
/app/pkg/mrt_converter/mrt_converter.go:213 +0x469
github.com/routeviews/google-cloud-storage/pkg/mrt_converter.processMRTArchive({0xbd5bf8, 0xc000446a80}, 0xc000346140, 0xc00047d648, 0xc000314f00)
/app/pkg/mrt_converter/mrt_converter.go:270 +0x24c
github.com/routeviews/google-cloud-storage/pkg/mrt_converter.ProcessMRTArchive(...)
/app/pkg/mrt_converter/mrt_converter.go:249
main.(*server).archiveUploadHandler(0xc00030e120, {0x0, 0x4f516d}, 0xc00045a600)
/app/cmd/converter/main.go:76 +0x4a5
net/http.HandlerFunc.ServeHTTP(0x9e4280, {0xbd3918, 0xc00035e620}, 0xc00035e620)
/usr/local/go/src/net/http/server.go:2047 +0x2f
net/http.(*ServeMux).ServeHTTP(0x0, {0xbd3918, 0xc00035e620}, 0xc00045a600)
/usr/local/go/src/net/http/server.go:2425 +0x149
net/http.serverHandler.ServeHTTP({0xbd2428}, {0xbd3918, 0xc00035e620}, 0xc00045a600)
/usr/local/go/src/net/http/server.go:2879 +0x43b
net/http.(*conn).serve(0xc000318dc0, {0xbd5ca0, 0xc0003029f0})
/usr/local/go/src/net/http/server.go:1930 +0xb08

Looks like the errors are caused by eqix specifically - I didn't see errors from other collectors (yet). I'm trying to figure out why it doesn't work with eqix archives.
A few issues in the code:

I only log a warning when an update can't be parsed, which led to this out-of-bound issue when reaching the end of a file.
We are seeing 50k+ errors because the pub/sub subscription is default to retry unack'ed messages immediately. I've set it to exponential backoff to reduce some noise.

On Wed, 19 Jan 2022 00:55:40 +0000,
yuhao-he @.***> wrote:

[1  <text/plain; UTF-8 (7bit)>]
Looks like the errors are caused by eqix specifically - I didn't see errors from other collectors (yet). I'm trying to figure out why it doesn't work with eqix archives.

where did you dig out the eqix only part? (maybe I'm just seeing what is 'http logs' ?)

A few issues in the code:

I only log a warning when an update can't be parsed, which led to this out-of-bound issue when reaching the end of a file.
We are seeing 50k+ errors because the pub/sub subscription is default to retry unack'ed messages immediately. I've set it to exponential backoff to reduce some noise.

ah-ha! ok, cool. that sounds like a better plan :)

--
Reply to this email directly or view it on GitHub:
https://github.com/routeviews/google-cloud-storage/issues/17#issuecomment-1015973708
You are receiving this because you authored the thread.
Message ID: @.***>
[2  <text/html; UTF-8 (7bit)>]

where did you dig out the eqix only part? (maybe I'm just seeing what is 'http logs' ?)

It's probably not eqix only - but I did see a ton of logs when "start conversion" messages for eqix but zero success messages (I will display error messages later). This is probably due to the immediate retries.
Btw, it looks like only around 10 collectors have been uploaded today. Is there any way to look at the progress of uploads?

The uploader is seeming to have problems getting out of a single directory in the Walk() function.
I'd spent time yesterday attempting to debug it, for instance:
https://gist.github.com/morrowc/188800b5ee4af025e037cc3f1d1de453
That code will walk down like is shown in the 1st comment.
The mass_uploader code, though, reading logs output:
https://gist.github.com/morrowc/795ea8dfa77bb363695360282f3f9d1b
seems to never get back out of the hole it dug on the first pass through a directory.

Oh, 'how to see how far its gotten'... I suppose either: "read logs" (which is hard when it's not on your workstation) or the metrics collection could be exposed better/externally? suggestions?


fix for Math.asinh, Math.acos, Math.atanh....

Please add tests that would fail without all of these changes.

@ljharb , for test/math.js it is possible to use code between line 1427 and line 1459,
almostEqual at https://github.com/paulmillr/es6-shim/blob/master/test/math.js#L4 should be changed to use same code as isNear, so it will measure relative error, not absolute, and it will work better for small values

:'(
I do not know how to work with git.

I've done the var x changes in master, so you'll want to rebase onto the latest master (and also reduce the 5 commits down).
As for isNear - I'd be happy to accept a separate PR to change the almostEqual expect implementation to be more precise, and update the associated tests.

@ljharb
continuation in https://github.com/paulmillr/es6-shim/pull/338

@Yaffle Please don't do this in the future - creating extra pull requests clutters up the repo, and they can't ever be deleted. Stick with one branch name per pull request.


Tray icon doesn't work in production

Hi Team,
Thanks for the fantastic starter project, it's saved a lot of time, but unfortunately I've lost probably more time trying to get the tray icon to work :(
I'm guessing it has something to do with way Webpack is setup, but I'm now on week 3 trying to get this to work.
The tray icon will show fine in development, but no matter what I do it will not show in production. I'm using a .ico file in the static folder, and the static folder is being copied into the packages/win-unpacked/resources folder. All the other images used in the app work, but the tray icon just won't.
Any thoughts on what the issue might be?

I had a similar issue with Tray and I ended up just doing a base64 encode of the image and putting that directly into the file where tray was invoked, rather than trying to get the path to the image correct. . .not a great solution, but it worked.


fix: Add a verity_cleanup function to test helpers.bash

The device may be busy (race with udev) so we should retry the removal

What type of PR is this?
bug
Which issue does this PR fix:
#541
What does this PR do / Why do we need it:
Adds a retry with backoff to the call into the device mapper removal library
If an issue # is not available please add repro steps and logs showing the issue:
Testing done on this change:
Automation added to e2e:
Will this break upgrades or downgrades?
Does this PR introduce any user-facing change?:
no

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Do you have evidence or experience that this actually fixes the issue?

Do you have evidence or experience that this actually fixes the issue?

Not yet, I didn't have a setup to run the CI locally (going to set that up now);

Also, you should 'Fixes #541' in your commit message.

Sure, will add.

ok 6 --no-squashfs-verity works in 2sec
ok 7 mount + umount works in 2sec
ok 8 mount + umount + mount a tree of images works in 3sec
ok 9 bad existing verity device is rejected in 12sec

\o/


Twitter Follow

Proposal:
Proposal written by Paquimex
Is it a proposal for the website or for the bot?: bot
What is the title of your proposal?: Twitter Follow
Explain your proposal more accurately (It's best to give as much information as possible, so that we can implement the proposal better): I've seen bots being able to post tweets from selected accounts on the server, I'd like to see such function in this bot
Why should we add this feature?: For different kinds of announcements and following different Twitter accounts on the server, taking that feature to an already multifunctional bot will make severs cleaner
ReportID: 185

➤ LenoxBot commented:
Approve:
good idea - then monkey mussn’t‘ post any tweets in <#497326422117842948> 😉
Denarioyt#3352

➤ LenoxBot commented:
Approve:
agree with Denario
Profxy#5396

➤ LenoxBot commented:
every server should be able to set these accounts
Profxy#5396

➤ LenoxBot commented:
Approve:
twitter bot --> love it!
Dadi#7808

Could be abused. For the work that we have to invest to program this feature would not be worth


Fabric vs CFT doc

Convert existing document to markdown. Needs editing.
Fixes #862

This lgtm as as first version. I'd merge it

Lemme have a final look after lunch


gradle_4: remove

Description of changes
It was last updated in 2018.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

The openjfx11 changes can be superseded by https://github.com/NixOS/nixpkgs/pull/205169.

Sweet, https://github.com/NixOS/nixpkgs/pull/205169 has been merged onto staging. I'm planning to wait for it to reach master in a few weeks, because I don't see any value in stacking this on top of it. By that time, https://github.com/NixOS/nixpkgs/pull/205841 might be in as well, and then we can repurpose this PR to remove both Gradle 4 and Gradle 5.


Make sure everything is up to date with the new plugin and make a release

I am creating this issue to foster a discussion about @Darkyenus updates to the plugin and the best way to go about releases. As a member of the RoboVM team, I want to make sure sbt-robovm gets the attention it deserves.
I have been investigating using https://jitpack.io/ as an alternative way to distribute sbt-robovm so that releases can be faster and easier.
Feel free to discuss anything about how we can manage the plugin better moving forward.

Jitpack certainly looks interesting. One thing though: with publishing to default sbt plugin repo, users didn't have to add a repo for sbt to search in. If I understand correctly, with Jitpack they will have to? It probably isn't a big problem, but still something worth considering I think. What about releases (when RoboVM updates) to sbt repo and Jitpack for snapshots?
Anyway, before the release, we should probably rewrite or at least change the README a bit. It started getting a bit messy when I was incrementally adding new info, so it is not as clear as it could be. And more importantly, there are still links referencing my forks, which is no longer necessary.
Also, somebody who has RoboVM license should check if the debugger works. I tried to add it during the time it was released, everything should be there, but I have never seen it working. And if it works, there should be instructions in README.
In the future, it would be worth writing a more comprehensive documentation, probably on the wiki. I can contribute to that.
And as a reminder, this is a good time to do any refactoring/changes which are breaking. So if somebody sees any architectural problem with the code, speak up now.
Btw. Congratulations on the marriage n' stuff and welcome back, thanks for coming

These are all great ideas. I will add issues for each of the remaining things to do, and I can try the issues that need a RoboVM license.

I believe we have addressed each of these issues in separate issues, so closing for now.


Move the Class Data Type facets above the Categorized As facet

Problem Description:
Currently each Class Data Type facet can be found at the very end of it's facet list. This placement does not make sense when looking at the other facet placements.
Expected Behavior/Solution:
Move the Class Data Types to be above Categorized As, as categorized as is a subclass of Class Data Types.
Requirements:

For all Objects, Works, People & Groups, Concepts and Events tab in Simple or Advanced Search the Class Data Type will be above the Categorized As facet.
Ex: Object Class
Categorized As

Needed for promotion:
If an item on the list is not needed, it should be crossed off but not removed.

[x] Wireframe/Mockup - Amy
[ ] Committee discussions - Sarah
[ ] Feasibility/Team discussion - Sarah
[ ] ~Backend requirements - TBD~
[x] Frontend requirements- TBD
[ ] Are new regression tests required for QA - Amy
[ ] ~Questions~
~List of questions for discussions. Answers should be documented within the issue.~

UAT/LUX Examples:

https://lux.collections.yale.edu/view/results/objects?q={"AND"%3A[{"text"%3A"andy"%2C"_lang"%3A"en"}%2C{"text"%3A"warhol"%2C"_lang"%3A"en"}]}&sq=andy+warhol
TST: https://lux-front-tst.collections.yale.edu/view/results/objects?q={"AND"%3A[{"text"%3A"andy"%2C"_lang"%3A"en"}%2C{"text"%3A"warhol"%2C"_lang"%3A"en"}]}&sq=andy+warhol

Dependencies/Blocks:
N/A
Related Github Issues:
N/A
Related links:

Bugherd: https://www.bugherd.com/projects/284041/tasks/2418

Wireframe/Mockup:
Current: *Image is similar for all other tabs Works, People & Groups, Concepts and Events

Updated: *Image update will be the same for all other tabs Works, People & Groups, Concepts and Events

Approved by UAT 9/12

Looks good in PROD. Closing.


Can't connect to DB after integer exceeds 32 bits

Our server is using mongoerl in master branch. We found a bug which causes our server down about every month in mongoerl. The trace is as follows:
["web request failed",
{path,"l"},
{type,error},
{what,{badmatch,{-2146148680,
{reply,false,false,true,0,0,
[{setName,<<"rs0">>,ismaster,true,secondary,false,
hosts,
[<<"ip-10-121-13-170:27017">>,
<<"ip-10-121-12-150:27017">>,
<<"ip-10-121-12-108:27017">>],
primary,<<"ip-10-121-13-170:27017">>,me,
<<"ip-10-121-13-170:27017">>,
maxBsonObjectSize,16777216,
maxMessageSizeBytes,48000000,localTime,
{1371,74346,818000},
ok,1.0}]},
<<>>}}},
After looking into the source of mongodb-erlang, I think this is mainly because that after request_id was requested more than<PHONE_NUMBER> times(every time it will use ets:update_counter to increase by 1), it will be larger than 32 bit int, then this error will occur. We are using this in production environment so we need a urgent fix. I checked refactor branch, it seems not many documents on that new branch with new interfaces and so it's impossible for us to risk upgrading it to new. Could anyone please tell me what to do to avoid this problem, probably a quick fix for this in mongoerl?
Thanks very much

I've updated pull request to match upstream.


Doc issue (Mac Fortran support)

I was looking through the documentation to try it out. :-)
"Clang Fortran compiler" was mentioned here: http://www.qmlcode.org/installation.html#note-on-apple-mac-support
I think you mean that the "GCC Fortran compiler, gfortran" through homebrew does not include OpenMP support.
AFAIK, there is no Clang Fortran compiler.
You can get a gfortran from here, which I think includes the OpenMP support: http://hpc.sourceforge.net

Thanks, Geoff! Yeah, there seems to be some confusion here. I have some minor updates coming next week, I'll make sure to update the text.
I am not sure how I should go about updating setup.py, since the default  Fortran compiler on mac does not support OpenMP, so enabling this would break "pip install qml" for most mac users.
I am also working on adding build/testing on osx to Travis ci, although it has been a bit of a tedious task.

Updated installation instructions.


Add documentation to use OpenZeppelin Contracts Upgradeable fork

Add documentation on using OpenZeppelin Contracts Upgrade Safe fork to Upgrades Plugins documentation.
Community feedback is that this would be helpful and avoid community from using OpenZeppelin Contracts when they need to use the Upgrade Safe fork:
https://forum.openzeppelin.com/t/how-to-use-ownable-with-upgradeable-contract/3336/10?u=abcoathup

This is currently documented in Writing Upgradeable Contracts.


move linter to webpack base build since linting check should also be done for prod build

and check other projects where this has to be updated
check if eslint-plugin-promise package is required because on another project prod build which uses linter plugin also required this lib while it is ont in linter plugin dependencies

eslint-plugin-promise was not requried most probably that another project was using something in addition that required this module


Tracker compatible with Python 3.9

Update deprecated Thread.isAlive to Thread.is_alive to be compatible with Python 3.9
REF: https://github.com/python/cpython/pull/11454

Thank you!


Is it possible to load view controller from storyboard(not XIB)?

Is it possible to load view controller from storyboard(not XIB)? This line of code only for XIB
var controller : UIViewController = UIViewController(nibName: "SnapNShareViewController", bundle: nil)

Please advice. Thank you.

    controller = [self.storyboard instantiateViewControllerWithIdentifier:@"Your Controller ID"]

yes it is. You have to change the Storyboard ID. Here is my code to load from storyboard and not nib:
   let storyboard = UIStoryboard(name: "Main", bundle: NSBundle.mainBundle())

    let controller2 = storyboard.instantiateViewControllerWithIdentifier("second") as! secondViewController

where "second" is the name and storyboard ID of my view controller that I was loading from, and secondViewController is its corresponding UIViewController class.

after doing like this not working properly


installation is not working, write /proc/sys/fs/binfmt_misc/register: invalid argument

i am trying to build arm64 image on amd64 host.  I have installed qemu, but running the command docker run --rm --privileged tonistiigi/binfmt:latest --install arm64 gives i am trying to build arm64 image on amd64 host.  I have installed qemu, but running the command docker run --rm --privileged tonistiigi/binfmt:latest --install arm64 gives
installing: arm64 cannot write to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: invalid argument
{
  "supported": [
    "linux/amd64",
    "linux/386"
  ],
  "emulators": [
    "qemu-aarch64",
    "qemu-aarch64_be",
    "qemu-alpha",
    "qemu-arm",
    "qemu-armeb",
    "qemu-hppa",
    "qemu-m68k",
    "qemu-microblaze",
    "qemu-microblazeel",
    "qemu-mips",
    "qemu-mips64",
    "qemu-mips64el",
    "qemu-mipsel",
    "qemu-mipsn32",
    "qemu-mipsn32el",
    "qemu-or1k",
    "qemu-ppc",
    "qemu-ppc64",
    "qemu-ppc64le",
    "qemu-riscv32",
    "qemu-riscv64",
    "qemu-s390x",
    "qemu-sh4",
    "qemu-sh4eb",
    "qemu-sparc",
    "qemu-sparc32plus",
    "qemu-sparc64",
    "qemu-xtensa",
    "qemu-xtensaeb"
  ]
}

Docker version : 20.10.7
OS: Ubuntu 16.04.5
any help will be appreciated. :)

Looks like you have some emulators already installed. Can you run the following commands and give me the output?:
docker pull tonistiigi/binfmt
docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-*
docker run --privileged --rm tonistiigi/binfmt --install all
docker run --rm arm64v8/alpine uname -a

Hey @crazy-max , sure.. please find the outputs below:

docker pull tonistiigi/binfmt

it was able to pull successfully

docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-*

uninstalling: qemu-aarch64 OK
uninstalling: qemu-aarch64_be OK
uninstalling: qemu-alpha OK
uninstalling: qemu-arm OK
uninstalling: qemu-armeb OK
uninstalling: qemu-hexagon OK
uninstalling: qemu-hppa OK
uninstalling: qemu-m68k OK
uninstalling: qemu-microblaze OK
uninstalling: qemu-microblazeel OK
uninstalling: qemu-mips OK
uninstalling: qemu-mips64 OK
uninstalling: qemu-mips64el OK
uninstalling: qemu-mipsel OK
uninstalling: qemu-mipsn32 OK
uninstalling: qemu-mipsn32el OK
uninstalling: qemu-or1k OK
uninstalling: qemu-ppc OK
uninstalling: qemu-ppc64 OK
uninstalling: qemu-ppc64le OK
uninstalling: qemu-riscv32 OK
uninstalling: qemu-riscv64 OK
uninstalling: qemu-s390x OK
uninstalling: qemu-sh4 OK
uninstalling: qemu-sh4eb OK
uninstalling: qemu-sparc OK
uninstalling: qemu-sparc32plus OK
uninstalling: qemu-sparc64 OK
uninstalling: qemu-xtensa OK
uninstalling: qemu-xtensaeb OK
{
  "supported": [
    "linux/amd64",
    "linux/386"
  ],
  "emulators": null
}

docker run --privileged --rm tonistiigi/binfmt --install all

installing: arm cannot write to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: invalid argument
installing: ppc64le cannot write to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: invalid argument
installing: riscv64 cannot write to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: invalid argument
installing: mips64 cannot write to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: invalid argument
installing: arm64 cannot write to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: invalid argument
installing: s390x cannot write to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: invalid argument
installing: mips64le cannot write to /proc/sys/fs/binfmt_misc/register: write /proc/sys/fs/binfmt_misc/register: invalid argument
{
  "supported": [
    "linux/amd64",
    "linux/386"
  ],
  "emulators": null
}

docker run --rm arm64v8/alpine uname -a

 The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64) and no specific platform was requested
standard_init_linux.go:228: exec user process caused: exec format error

@Deepak275 You might have a pretty old kernel. What's the output of uname -a?

@crazy-max yes, it was kernel issue. I had 4.4, upgraded it to 4.15 and it worked. Thanks a lot for the help :)


fix pylint and flake8 errors on main branch

there are a few to go, I should also look into these

All errors fixed. We now finally have again a main branch with no failing tests.


Eliminate ability for ringpop and partial affinity to thrash

r @raynos @rf

:ok:

story checks out.


Translation: Missing strings

When I first translated BoxBilling (3.6.11) 2 or 3 years ago. I found that there were some missing translation strings in language file and I made my own translation file adding those missing strings.
Recently I setup 4.19.1 and cross checked it with my previous translation (3.6.11) and Transifex translation. After merging, admin panel menus and messages seems ok mostly except some pages. (I didn't make comprehensive check of client interface yet. But it seems better from admin interface.)
There are two types of missing translations.

Missing strings in .po file. If I locate and put the string into .po file, I can see translation.
So I attach those missing strings below. Updating .pot file and syncing with Transifex will be enough for them.

Un implemented translations.
I made some intense work and got screen shots of translation issues. I will add these issues one by one to git. So developers could create new translation strings.

"#" Remarks needed for my works and can be removed
Missing Strings
# Page:Orders | Overview | Pending setup
msgid "Pending setup"
msgstr ""
# Admin Menu: Customers, Orders, Invoices | Overview
msgid "Overview"
msgstr ""
# Admin Menu: Customers, Orders, Invoices | Advanced search
msgid "Advanced search"
msgstr ""
# Admin Menu: Invoices | Transactions overview
msgid "Transactions overview"
msgstr ""
# Admin Menu: Invoices | Transactions search
msgid "Transactions search"
msgstr ""
# Admin Menu: Products | Product / Services
msgid "Products / Services"
msgstr ""
# Admin Menu: Products | Product addons
msgid "Product addons"
msgstr ""
# Admin Menu: Support | Client tickets
msgid "Client tickets"
msgstr ""
# Admin Menu: Support | Advanced ticket search
msgid "Advanced ticket search"
msgstr ""
# Admin Menu: Support | Announcements
msgid "Announcements"
msgstr ""
# Admin Menu: Configuration | Domain registration
msgid "Domain registration"
msgstr ""
# Admin Menu: Activity
msgid "Activity"
msgstr ""
# Admin Menu: Activity | Events history
msgid "Events history"
msgstr ""
# Admin Menu: Activity | Staff logins history | Line 3477 (members)
msgid "Staff logins history"
msgstr ""
# Admin Menu: Activity | Client logins history | Line 4329 Client(s)
msgid "Client logins history"
msgstr ""
# Admin Menu: Addons | Languages
msgid "Languages"
msgstr ""
# Admin Menu: Addons | File editor
msgid "File editor"
msgstr ""
# Button: Continue
msgid "Continue"
msgstr ""
# Form: Order search
msgid "Filter orders"
msgstr ""
# Form: Order search
msgid "Creation date"
msgstr ""
# Form: Invoices Summary
msgid "Unpaid invoices"
msgstr ""
# Form: Invoices Summary
msgid "Refunded invoices"
msgstr ""
# Form: Invoices Summary
msgid "Paid invoices"
msgstr ""
# Form: Invoices Summary
msgid "New Invoice"
msgstr ""
# Form: Invoices Summary
msgid "Prepare"
msgstr ""
# Form: Invoice Detail
msgid "Invoice items"
msgstr ""
# Form: Invoice Detail
msgid "Issued at"
msgstr ""
# Form: Invoice Detail
msgid "Tax"
msgstr ""
# Form: Invoice Detail
msgid "Payment method"
msgstr ""
# Form: Invoice Search
msgid "Filter invoices"
msgstr ""
# Form: Invoice Search
msgid "All statuses"
msgstr ""
# Form: Invoice Search
msgid "Issue date"
msgstr ""
# Form: Client Tickets
msgid "New email"
msgstr ""
# Form: Client Tickets
msgid "Receivers name"
msgstr ""
# Form: Tickets Filter
msgid "Filter support tickets"
msgstr ""
# Form: Tickets Filter
msgid "Order ID"
msgstr ""
# Form: Tickets Filter
msgid "Ticket subject"
msgstr ""
# Form: Tickets Filter
msgid "Ticket messages"
msgstr ""
# Form: Tickets Filter
msgid "Helpdesk"
msgstr ""
# Breadcrumbs: Staff members login history
msgid "Staff members login history"
msgstr ""
# Breadcrumbs: Clients login history
msgid "Clients login history"
msgstr ""
# Panel: Metric
msgid "Metric"
msgstr ""
# Panel: Today
msgid "Today"
msgstr ""
# Panel: Yesterday
msgid "Yesterday"
msgstr ""
# Panel: This month so far
msgid "This month so far"
msgstr ""
# Panel: Last month
msgid "Last month"
msgstr ""
# Panel: Define date interval for graphs
msgid "Define date interval for graphs"
msgstr ""
# Panel Button: Update graphs
msgid "Update graphs"
msgstr ""
# Panel: Income
msgid "Income"
msgstr ""
# Panel: Recent clients activity
msgid "Recent clients activity"
msgstr ""
# Panel: Recent staff activity
msgid "Recent staff activity"
msgstr ""

Sorry. I guess this issue should be under extensions repo instead of boxbilling repo. Is it possible to move there?


adjusted disk size for HANA LOG disks

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: MSwbratton sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Alloy implementation

Quick question.
I am trying to implement in Alloy, everything seems to work fine but images are not showing up.  Using the stock example, I am guessing my paths are incorrect, specifically in:
var imgSample1 = Ti.Filesystem.resourcesDirectory + 'images\image1.jpg';
The images are located in app\assets\images
Any ideas that could guide me in the right direction...?
Btw, thanks for this fantastic contribution.

@ktucc Have you been able to resolve the issue with Alloy? I tried it to put in the images across various folders but still the images were not getting.


User ops example "Fact" can't work.

I build tensorflow from sources(master). When i try to use the user_ops example "Fact", I got the follow errors. I try to change to branch r0.10, the error does't exists. It may be a bug in master branch?
In [11]: tf.user_ops.my_fact()

AttributeError                            Traceback (most recent call last)
<ipython-input-11-d6bdebf36140> in <module>()
----> 1 tf.user_ops.my_fact()

/path/to/tensorflow/_python_build/tensorflow/python/user_ops/user_ops.py in my_fact()
     26 def my_fact():
     27   """Example of overriding the generated code for an Op."""
---> 28   return gen_user_ops._fact()

AttributeError: 'module' object has no attribute '_fact'

a redundant BUILD in core/user_ops, removed it and it works.


l2 connection - supported aside service token

Added support to create ecx/v3 l2 connections using a Fabric Service Token .

Request field is primaryServiceToken
Value is returned in the get connection by uuid in vendorToken
Value is exposed in the client as ServiceToken
This attribute is not supported in redundant connections API calls (secondary attributes), not even from the portal. To create a redundant connection, it will be required two connection requests, one for each service token

fix https://github.com/equinix/ecx-go/issues/2

It would build reviewer confidence if this field were included in the spec (described at https://developer.equinix.com/catalog/buyerv3#operation/createConnectionUsingPOST).


Bug: wrong merge conflicts detection

Maybe it's upstream related.
https://github.com/go-gitea/gitea/pull/26271#issuecomment-1664879177

Probably webhook racing


[Task]: Optimize blob conversion during execution for file picker inputs

Is there an existing issue for this?

[X] I have searched the existing issues

SubTasks
Currently we read blob urls into string text while executing, and then convert the text into a plain text blob to ensure correct encoding while passing the data over the network. There is a possibility of skipping these conversions in some conditions that we should investigate.

Analysis :
There is a function inside app/client/src/sagas/ActionExecution/PluginActionSaga.ts file which has a function evaluateActionParams. The above task is referenced to this particular functionality.


Event Handler

Event Handler is a massive issue,
Automatic Event Handling should be added so then when tmi.js gets updated the project doesnt need to be manually done. This would also allow projects to be more scalable

Agreed, I think I know a way to improve it

Listening to all the events and checking which ones the user want to recieve is a bad idea since it creates a number of useless listeners

@leonardobagi If you could possibly check my fix for Branch 2.0 that would be great

Yea thats exactly what I did in my PC lol. I will make a small refactor on loading events/commands and push it to 2.0 branch


SNOW-1830511, SNOW-1830542 Decoder logic for DataFrame.agg, DataFrame.group_By, and RelationalGroupedDataFrame.agg

Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fixes SNOW-1830511 and SNOW-1830542

Fill out the following pre-review checklist:

[x] I am adding a new automated test(s) to verify correctness of my new code

[ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing

[ ] I am adding new logging messages
[ ] I am adding a new telemetry message
[ ] I am adding new credentials
[ ] I am adding a new dependency
[ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes.
[x] I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: Thread-safe Developer Guidelines

Please describe how your code solves the related issue.

Added decoder logic for DataFrame.agg, DataFrame.group_By, and RelationalGroupedDataFrame.agg. The corresponding tests are: DataFrame.agg.test, RelationalGroupedDataFrame.test, and RelationalGroupedDataFrame.agg.test.

Switched to draft because I missed some logic


Template validator (null checking) causes issues when accessing keys with special characters

Handlebars allow us to access keys/variables with spaces or special characters like {} in their names using []. For example:
{{[str with a space]}}

But our validator is not properly working when this syntax is used. For example:
> ERMrest.renderHandlebarsTemplate('{{{[col]}}}', {"col": 'a'}, null, {avoidValidation: true})

'a'

> ERMrest.renderHandlebarsTemplate('{{{[col]}}}', {"col": 'a'}, null, {avoidValidation: false})

null

> ERMrest.renderHandlebarsTemplate('{{{[#_Released]}}}', {"#_Released": 'a'}, null, {avoidValidation: true})

'a'

> ERMrest.renderHandlebarsTemplate('{{{[#_Released]}}}', {"#_Released": 'a'}, null, {avoidValidation: false})

null

We found this issue while working on new features on plot app. The #_ Released is the name of one of the columns in the table created for this plot. And we wanted to change the “tooltip” that shows on hover of one of the bars and use multiple columns in that tooltip (including one with a special character).

While we should fix this, for now I documented this as one of the limitation of our automatic null detection in here.
As I described there, the workaround for this bug is usage of the block sytnax. So either a dummy {{#if true}} or [# Released]}}.


Wrap SkBitmap::setAlphaType

Please wrap the native method SkBitmap::setAlphaType()
There's currently no simple and fast way to add an alpha channel to a bitmap (e.g. after loading it from a file). Only solution I could think of was to copy the bitmap pixel by pixel, which takes about a second for larger ones.
Thanks.

Before I add this, I wanted to check that you are aware of the fact that you can tell the decoder what image configuration that you want.
Instead of SKBitmap.Decode(string path), you can use the overload: SKBitmap.Decode(string path, SKImageInfo info)
https://developer.xamarin.com/api/member/SkiaSharp.SKBitmap.Decode/p/System.String/SkiaSharp.SKImageInfo/
This will result in the image being loaded and converted into the desired color type and alpha type in a single decode operation.
The SkBitmap::setAlphaType(...) function does not actually do anything except switch the alpha type field, leaving the pixel data the same. This means that by changing the alpha type, the pixel data may be "different". For example, if you switch the alpha type from premultiplied to unpremultiplied, the color has effectively changed, since the color is related to the alpha value.

Ah, thanks for the info, I should have looked at the implementation of setAlphaType, in that case it won't do the job.
OK, I'll try Decode with ImageInfo.

I had missed your comment about changing the alpha type by using CopyTo. I had already tried that, it changes the alpha type of the target bitmap to match the type of the source, so it doesn't work.
Decode with a specified SKImageInfo does work. The only small inconvenience is that I need to open the stream twice, once to read the bounds and another time to read the image (since SKImageInfo wants to be given the image bounds).

Ah, in that case, you can use the SKCodec directly with an extra step:
// open the codec
var codec = SKCodec.Create(stream);

// get the dimensions from the codec
var info = codec.Info;

// create the bitmap (use the dimensions and optionally any desired color/alpha type)
var bitmap = new SKBitmap(info.Width, info.Height);

// decode the bitmap
codec.GetPixels(bitmap.Info, bitmap.GetPixels());

Thanks, this works and is indeed better.

I'll close this as this solves your problem, and I am not really wanting to wrap SkBitmap::setAlphaType.
Feel free to reopen if this is needed.