[BLE Mesh] How to send a message using UART and app in a mesh network ? (IDFGH-1874)

Environment

Development Kit:    [ESP32-DevKitC]
Kit version (DevKitC): [v4]
Module or chip used:  [ESP32-WROOM-32D]
IDF version (ran git describe --tags to find it):
// v4.0-dev-301-gc7c419da4
Build System:      ESP-IDF Tools  // [Make|CMake]
Compiler version (ran gcc --version to find it):
// gcc (MinGW.org GCC-6.3.0-1) 6.3.0
Operating System:     [Windows]
Power Supply:         [USB]

Problem Description
I have no idea what message and in which format can be sent through UART 1 to a certain ble mesh node for turning on/off an LED.
//Detailed problem description goes here.
Expected Behavior
A message through the UART1 is sent to the device1 (ble_mesh_client_model) the same message is resent/relayed to the device2 (ble_mesh_node) and turns ON/OFF an LED belongs to the device2.
Actual Behavior
Steps to repropducer
-Device1 (ESP32-DevKitC) that runs the ble_mesh_client_model project.
-Device2 (ESP32-DevKitC) runs the ble_mesh_node project.
-nRF Mesh app is used for provisioning these two nodes
-The pin numbers 16 and 17 (on the (ESP32-DevKitC) are used to send a message (UAR1)
with a terminal emulator (termite)

I have opened a serial session and can send messages successfully to the micro that is running ble_mesh_client_model (device1 ). At this point, if I monitor the device1 (ble_mesh_client_model ) and send a message like "0001" to it., on the command line I see:
I (153032) BOARD: board_uart_task: input 0x00000001, remote_addr 0x0001

My questions is:
Which message and in which format must be sent to device1 (ble_mesh_client_model ) that turns an LED on/off in device2 (ble_mesh_node)?
Secondly, these two nodes are indeed proxy nodes, because I can send ON/OFF messages to them through nRF Mesh app, Is that right?
// It helps if you attach a picture of your setup/wiring here.

Code to reproduce this issue
Debug Logs
Other items if possible

The UART is only for setting the node address, there is no command to set the state through the UART.  The code that forwards the message to the node is here:
https://github.com/espressif/esp-idf/blob/9f145ff165099f32b92b7ee022a1debc5f0a583d/examples/bluetooth/esp_ble_mesh/ble_mesh_client_model/main/ble_mesh_demo_main.c#L338-L349

Hi John-nx,
 We recommend that you read the documentation related to the project, under the same path.

 Detailed steps in the documentation.

Thanks for the response.
The problem is the documentation/tutorial is really vague.
Could you please shed some lights on the followings:

What is the usage of setting an address using UART1 when I can change the state of any led/element using the nRF Mesh app anyways? (I mean without even using UART)

I can connect to ANY node (even my device2 which is flashed with ble_mesh_node), and change the state of an element in the other nodes (like node flashed with ble_mesh_client_model), always using nRF mesh app, what does this suppose to mean?
It must work vice versa or not?  Do both of the nodes have the relay role?

Again, since I can connect to both nodes (ble_mesh_client_mode and ble_mesh_node) with nRF Mesh app and send commands, this means that these BOTH nodes are indeed proxy nodes ?

Thank you so much

Hi John-nx，
environment： nrf Mesh + Ble_mesh_client_mode  + ble_mesh_node

Ble_mesh_client_mode is based on ble_mesh_node.

Ble_mesh_client_mode can control the ble_mesh_node node, but needs to know the address of the ble_mesh_node node as the destination address of the message.
You need to enter the address of another ble_mesh_node node through the serial port and store it in the ble_mesh_client_mode node in this way.
You can see the processing behavior of the message received by the ble_mesh_client_mode node, which has sent a new message to control the ble_mesh_node node.

Proxy and relay You can configure in menuconfig in coexistence, which is currently enabled by default.

Hi John-nx，
environment： nrf Mesh + Ble_mesh_client_mode + ble_mesh_node

Ble_mesh_client_mode is based on ble_mesh_node.

Ble_mesh_client_mode can control the ble_mesh_node node, but needs to know the address of the ble_mesh_node node as the destination address of the message.
You need to enter the address of another ble_mesh_node node through the serial port and store it in the ble_mesh_client_mode node in this way.
You can see the processing behavior of the message received by the ble_mesh_client_mode node, which has sent a new message to control the ble_mesh_node node.

Proxy and relay You can configure in menuconfig in coexistence, which is currently enabled by default.

Thanks for your fast response.
I understood your response which is the same as what is written in tutorial.
However, my point was something else, let me put in in another way:
ble_mesh_node node cal ALSO control Ble_mesh_client_mode ! and there is NO need to enter the address of another node to control its LEDs.
Does this make sense to you?
P.S. about Proxy and relay we are agreed that they are active then.

Hi John-nx，
 You can do this by using the group address, as long as you know the destination address of the sent message.

Yes exactly I do use group address, have subscribed each element to a group,
Then again, I don't know any other way to send a message!
Is there any tutorial/documentation on how to send a message using any APP to a certain node? and monitor the relayed message ?
Moreover, where in the "menuconfig" we can config the proxy and relay, can you give me the exact path?

Hi John-nx，

Componet config-->BLE Mesh Support --> Relay support
Componet config-->BLE Mesh Support --> BLE Mesh GATT Proxy Service

Is there any tutorial/documentation on how to send a message using any APP to a certain node? and monitor the relayed message?

Maybe you can refer to ble_mesh_provisioner demo,Can be solved(how to send a message to a certain node  )
Maybe you can refer to ble_mesh_fast_provisioner,Used custom message and used EspBleMesh app.

Hi Again,
I really appreciate your responses, I have a better vision on the whole thing now, however, my question is still there!
I took a look at the examples that you mentioned, they are not addressing about sending a command to a node (using unicast address). In my opinion, there is a lack of documentation here!
Honestly, I don't know why should I use a micro as a provisioner! while I can use a smartphone or computer device as it is stated in the Bluetooth website!
Let me go one step backward and state the issue as clear as I can:
In practice when a mesh network is created, how one is supposed to control the nodes?

either by a smartphone or another node which is used as a remote control, do you agree on that?
At this point,
if you use nrf mesh app or any other app to simulate sending a message, there must be a guide on how to send messages to different types of addresses! (This a configuration thing at the app level)
-and if a node is sopposed to be used plz refer me to the functions that do the same thing

Thanks!

@WCCWCC @negativekelvin

Hi @John-nx
Sorry for the misunderstanding. Let's go back to the original questions.

Which message and in which format must be sent to device1 (ble_mesh_client_model ) that turns an LED on/off in device2 (ble_mesh_node)?
Answer:
Our idea here is to use the Generic OnOff Client Model in device1 (ble_mesh_client_model) to control the Generic OnOff Server Model in device2 (ble_mesh_node). So the message sent to device1 is the Generic OnOff Set message which is sent by the nRF Mesh App for example. And the format of the message is opcode (2 octets) + onoff (1 octet) + TID (1 octet). You can refer to the section 3.2.1 in Mesh Model Spec for more information about the message format.

Secondly, these two nodes are indeed proxy nodes, because I can send ON/OFF messages to them through nRF Mesh app, Is that right?
Answer:
The Proxy feature of the two nodes are both enabled. But in this situation, only device1 works as a Proxy Node which is directly connected to the phone. You can use the nRF Mesh App  to control the device1 (send ON/OFF messages through GATT bearer), then device1 will foward the received messages (Generic OnOff Set) to device2 using the advertising bearer. The relationship is listed below.
nRF Mesh App ---------GATT---------> device1 ---------ADV----------> device2

Note: UART in the ble_mesh_client_model example is just used to input the unicast address of device2 (ble_mesh_node).

Thanks for you patience

Hi @Campou,
Thank you so much for your response, now things are clear.
I managed to use the UART1 to choose which address (unicast) or addresses (group) are to be controlled.
However, this whole thing raises some even more important questions, would you please also shed some light on them?
-1. So, It seems that it is not possible to use the Publish field in the nRF app to set target addresses, is this right? and if yes this is going to be added or not?
I mean it is expected to configure the Publication Address for each client/switch in the Generic OnOff model, using the provisioner which in this case is nRF app.
-2. In the  ble_mesh_client_model example, something is fishy!
Consider the communication process which you mentioned:
nRF Mesh App ---------GATT---------> device1 ---------ADV----------> device2
I expect to configure only "Geberic ON OFF Client" in the device1 (ble_mesh_client_model), however, to be able to control the device2
nRF Mesh App ---------GATT---------> device1** ---------ADV----------> device2**
I needed to configure the device 1 like below:
Generic ON OFF Client: Bound App key
Generic ON OFF Server: Bound App key, subscribe to the created group named MyGroup
Now using the MyGroup I can control device 2 (elements that are set by the UART)
Thing whole thing does not really make sense because, I am controlling LEDs in the device 2 (ble_mesh_node ) by another LED in device 1 (ble_mesh_client_model ), and not by using a switch!
The strange thing is in the nRF app I do see a switch icon alongside the MyGroup icon, but it just does not have any functionality or at least I do not know a way to configure it in order to control LEDs in the device 2.
My whole point is using the ble_mesh_client_model I expect to have/configure a physical switch or a software switch using nRF app, in order to publish a message. Is this right and can I do this?
Thank you in advance.

Hi @John-nx

I think it's possible to set the target address using the Publish field. And I had a try using the nRF Mesh App with your idea, it works fine. The only difference in ble_mesh_client_model example is that we use esp_ble_mesh_model_publish API to publish messages to ble_mesh_node. And in the nRF Mesh App, we should set the Publication address of the Generic OnOff Client Model to the unicast address of the ble_mesh_node.
Yes, the whole idea here is using Generic OnOff Client Model(in device1) to control the Generic Server Model(in device2). And you can use a physical switch in device1 to publish control messages to device2 with the publication address of device1 set to the unicast address of device2.

About the nRF Mesh App, I can only subscribe a group address to a model, and haven't find any icons to send messages to the group address.

Thanks.

Yes, the whole idea here is using Generic OnOff Client Model(in device1) to control the Generic Server Model(in device2). And you can use a physical switch in device1 to publish control messages to device2 with the publication address of device1 set to the unicast address of device2.

That's what's I'm saying ! This is supposed to be the idea!
Have you actually used a physical switch in device1 to publish control messages to device2? Or do you know somebody who did that?
If yes, which pins did youuse and how did you do that? (the boot button obviously is not set for this job)
There is nothing about a switch in the ble_mesh_client_model example code!
On the other hand, the function switch_init() is defined in the ble_mesh_node code!!!!!
About using the Publish field, it is NOT possible to use it in for the ble_mesh_client_model example I'll try esp_ble_mesh_model_publish though!
Thanks

Hi @Campou

Yes, the whole idea here is using Generic OnOff Client Model(in device1) to control the Generic Server Model(in device2). And you can use a physical switch in device1 to publish control messages to device2 with the publication address of device1 set to the unicast address of device2.

That's what's I'm saying ! This is supposed to be the idea!
Have you actually used a physical switch in device1 to publish control messages to device2? Or do you know somebody who did that?
If yes, which pins did youuse and how did you do that? (the boot button obviously is not set for this job)
There is nothing about a switch in the ble_mesh_client_model example code!
On the other hand, the function switch_init() is defined in the ble_mesh_node code!!!!!
About using the Publish field, it is NOT possible to use it in for the ble_mesh_client_model example I'll try esp_ble_mesh_model_publish though!
Thanks

Hi @John-nx
The Boot button in ESP32-DevKitC can be used to send messages. And for the guidance of using Boot button, you can refer to the esp-iot-solution for some help, which includes a component for the button.
And we can modify the example properly in the future based on your idea.
Thanks & Regards

Hi @Campou,
By the way, I wanted to try the publish thing that you mentioned:

I think it's possible to set the target address using the Publish field. And I had a try using the nRF Mesh App with your idea, it works fine. The only difference in ble_mesh_client_model example is that we use esp_ble_mesh_model_publish API to publish messages to ble_mesh_node. And in the nRF Mesh App, we should set the Publication address of the Generic OnOff Client Model to the unicast address of the ble_mesh_node.

What do you mean by "esp_ble_mesh_model_publish" ? there is no such thing, where can I find it?
Thanks

Hi @John-nx
esp_ble_mesh_model_publish() (in the header file esp_ble_mesh_networking_api.h) is an API which can be used to publish messages. And before using this API to publish messages, please make sure the corresponding Model Publication state has been configured properly.
Thanks.

Hi @Campou
Yes, I understood that, however, I actually have not started to change the example codes as there are some major issues and first I need to have a basic functionality. Please let me know about the  https://github.com/espressif/esp-idf/issues/4074 .
By the way, this is something minor, but now that we talked long about this example, did you know that the UART used in this example is not the one connect to the USB? Meaning that you cannot send the UART msg through USB.
Thanks

Hi @John-nx
Yes, the UART is just used to input the remote device address. And we have changed the client model example in the branch release/v4.0, will be released later.
Thanks.

Hi @John-nx I have the same problem, Did you manage to solve it? I can easily turn on/off led using my phone and app nRF Mesh, but I have to do this without phone, just by using device1 and device2.

Hi @peakymember and @John-nx,
I modified the ble_mesh_node example so you can publish the button state. With the nRF App as a provisioner you can subscribe the message to an unicast or group adress.
First I added a publisher model
static esp_ble_mesh_client_t onoff_client;
ESP_BLE_MESH_MODEL_PUB_DEFINE(onoff_cli_pub, 2 + 2, ROLE_NODE); //Two bytes for the message op code and two for the message
                                                                //the two message bytes are mandatory because otherwise the message is to short

static esp_ble_mesh_model_t pub_model[] = {
	ESP_BLE_MESH_MODEL_GEN_ONOFF_CLI(&onoff_cli_pub, &onoff_client),
};

//add the pub model into the element array
static esp_ble_mesh_elem_t elements[] = {
    ESP_BLE_MESH_ELEMENT(0, root_models, ESP_BLE_MESH_MODEL_NONE),
    ESP_BLE_MESH_ELEMENT(0, extend_model_0, ESP_BLE_MESH_MODEL_NONE),
    ESP_BLE_MESH_ELEMENT(0, extend_model_1, ESP_BLE_MESH_MODEL_NONE),
    ESP_BLE_MESH_ELEMENT(0, pub_model, ESP_BLE_MESH_MODEL_NONE),
};

When you call this function the state of the button gets published and you can toggle yout Led's
static void gen_onoff_publisher(uint8_t data)
{
    esp_err_t err;
    onoff_cli_pub.msg->data[2]=data;

    ESP_LOGI(TAG, "%s, Publish 0x%02x from addr 0x%02x",__func__, data,onoff_client.model->element->element_addr);
    err = esp_ble_mesh_model_publish(pub_model, ESP_BLE_MESH_MODEL_OP_GEN_ONOFF_SET_UNACK,
                                     2,(uint8_t*) onoff_cli_pub.msg->data+2, ROLE_NODE);
    if (err != ESP_OK) {
        ESP_LOGE(TAG, "%s: Publish Generic OnOff Status failed", __func__);
        return;
    }
}

I hope this helps you for a first test!

Hi @trapplab
In which branch and commit you solve it?
HAVE YOU PUBLISHED A MESSAGE USING THE BUTTON ON THE MICRO?
For the client example one expects:
Firstly, Setting a publish address to a switch, for example using the nRF mesh (which in this example is implemented poorly by UART)
Secondly, and more importantly publish the message set in the previous step using a physical switch for example boot button on the micro (which in this example is not implemented at all )
@peakymember if you want to use this client example, the code needs to change.
best

Hi @John-nx
you can find the modified node example in my forked repository esp-idf in the branch release/v4.0.
I added a client model to the nodes model list to publish the button state.
I use a MH-ET LIVE MiniKit with a ESP32-WROOM-32 module on it. I connected an external button to the GPIO port number 18 and use the onboard led connected to GPIO port 2.
After flashing every setting is done with the nRF App.

To use the example I provision the ESP32 with the nRF App.
Then you can exchange the Application Key with the Generic On Off Client and publish its state to a unicast adress or group adress.
After that you exchange the App. Key for the Generic On Off Server and subscribe to a group adress
The led should now toggle if you press the button

With this example you can't send the adress via UART. And i think it's unnecessary because the adress selection could be made with the nRF App as a provisioner.
best regards

Hi @trapplab
Thanks, I see what you did.
I tried to build the example that you modified but I am having a problem building it! There are errors
I use idf tools and windows 10 cmd as the build environment, and I tried to use both remotes/origin/release/v4.0 and ble_mesh_release/esp-ble-mesh-v0.6.1 branches as the IDF path.
Can you plz tell me if you have done pull request or not?
I think either your forked version should be synchronized with the original upstream repo or they should merge your modifications.
Honestly, solving these problems is just a huge overhead to have a simple functionally! And it seems they won't finish !! Having said that let me know plz about your build environment and possibly other modifications that you did apart from the ble_mesh_node example folder.
Thanks
Regards

With this example you can't send the adress via UART. And i think it's unnecessary because the adress selection could be made with the nRF App as a provisioner.

Of course, the UART is useless! That's even not the UART 1 is not even the one connected to the USB!!!!!!!!!!!!!!!!!

Thanks for reporting, the onoff_client example has been updated at here, feel free to reopen if the issue still happens, or please help file a new ticket for new issue. Thanks.


WIP - Framerange support merge and refactor test

Testing through CI

CI gfxreconstruct build queued with queue ID 14176.

CI gfxreconstruct build # 1355 running.

CI gfxreconstruct build # 1355 failed.

CI gfxreconstruct build queued with queue ID 14191.

CI gfxreconstruct build # 1356 running.

CI gfxreconstruct build # 1356 passed.

CI gfxreconstruct build queued with queue ID 9734.

CI gfxreconstruct build # 1590 running.

CI gfxreconstruct build queued with queue ID 11593.

CI gfxreconstruct build # 1597 running.

CI gfxreconstruct build # 1597 failed.

CI gfxreconstruct build queued with queue ID 11659.

CI gfxreconstruct build # 1599 running.

CI gfxreconstruct build queued with queue ID 11670.

CI gfxreconstruct build # 1600 running.

CI gfxreconstruct build # 1600 failed.

CI gfxreconstruct build queued with queue ID 13144.

CI gfxreconstruct build # 1601 running.

CI gfxreconstruct build queued with queue ID 13293.

CI gfxreconstruct build # 1602 running.

CI gfxreconstruct build queued with queue ID 13325.

CI gfxreconstruct build queued with queue ID 13331.

CI gfxreconstruct build # 1604 running.

CI gfxreconstruct build # 1604 passed.

CI gfxreconstruct build queued with queue ID 12980.

CI gfxreconstruct build # 1634 running.

CI gfxreconstruct build # 1634 passed.

CI gfxreconstruct build queued with queue ID 14761.

CI gfxreconstruct build # 1646 running.

CI gfxreconstruct build # 1646 failed.

CI gfxreconstruct build queued with queue ID 14848.

CI gfxreconstruct build queued with queue ID 14908.

CI gfxreconstruct build queued with queue ID 14969.

CI gfxreconstruct build # 1650 running.

CI gfxreconstruct build # 1650 passed.


rare: 1.8.9 -> 1.9.0

Description of changes
https://github.com/Dummerle/Rare/releases/tag/1.9.0
Unfortunately, games do not launch in the current state (unless you launch them via CLI).
I suspect that the reason why is this change:

Run Games in a second rare process to keep them running, if rare closes

And I guess that this line and #150841 the cause of the problem.
I'm not sure how to continue from here, so I'm waiting for your input.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

@SuperSamus could you please specify why the tagged PR is to blame? This is to understand the reason and see what can be done

The linked issue is about the fact that wrapped executables have a different name.
Rare, as a new feature, creates a new instance of itself when launching a game. In order to communicate with the new instance (for example, to know when it's closed), the app uses its name ("rare_{self.app_name}"), which is changed due to the issue.
(I linked the wrong line (app_name refers to the game, whoops), updated the main comment)

Btw @SuperSamus, I am not sure on how to tackle the issue, to be honest. It still works if running through the CLI, so at least we know it works, and for me that would be enough to get merged. Now, I saw that there was another issue regarding the same PR you indicated, so maybe it would be the best to call someone who can handle that better and see if a solution could be provided.

It still works if running through the CLI, and for me that would be enough to get merged

The entire point of Rare is being a GUI for Legendary (which is what a user would use if a CLI was enough), so no.

The entire point of Rare is being a GUI for Legendary (which is what a user would use if a CLI was enough), so no.

Then someone must prioritize how to solve the issue then. Like said before, I don't know how to tackle it.

The issue seems to be that rare is starting itself again using python, and the wrapper in a shell script, leading to this error
[GameUtils] INFO: Start new Process: (/nix/store/sz0j8k8ljh7y8qgyfxgqb3ws11bcy4gs-python3-3.10.6/bin/python3.10 /nix/store/imzm7mhdjrqqzblbm538d98mh7ghz525-rare-1.9.1/bin/rare start 01f1e8046e9e433db190a611a0411579)
  File "/nix/store/imzm7mhdjrqqzblbm538d98mh7ghz525-rare-1.9.1/bin/rare", line 2
    PATH=${PATH:+':'$PATH':'}
         ^
SyntaxError: invalid syntax

with this patch, this remove the python part in the command line and games are running. Tested on rare 1.9.1.
diff --git a/rare/utils/misc.py b/rare/utils/misc.py
index 4492074..5352dac 100644
--- a/rare/utils/misc.py
+++ b/rare/utils/misc.py
@@ -190,6 +190,7 @@ def get_rare_executable() -> List[str]:
         executable = [sys.executable]

     executable[0] = os.path.abspath(executable[0])
+    executable.pop(0)
     return executable

I can't run a game with it

Thanks @rapenne-s!

I never tried rare before, but I had to run it in a nix-shell with wine to be able to have some results. However, the results are mixed... I tried ~ 8 games, only "among us" and "a short hike" did run, many just error out in the console without message, and some games display a critical error at start. Is this expected or am I doing it wrong?

Axiom Verge
Celeste
Hyper Light Drifter
The Escapists
World Of Goo
Rogue Legacy

I used wine package and tried later with wineWoWFull.staging something in the shell before running Rare, they gave similar results.

Tried the PR now and got:
error: hash mismatch in fixed-output derivation '/nix/store/29kbhq107icdgycx918s03jf425d33rc-source.drv':
         specified: sha256-9dY8Krni+WI2zLo3qFAQ+0xoGgOVjPRWHTymlYInoQA=
            got:    sha256-2l90fs4/RTdcezf0dj8uNDJFvDw9nW9oxlskWPuG4rI=

Anything left for this to be merged?

Is there anything left to do for this to be merged?

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/1304


do not include iostream in the public header

The problem is that iostream will inject static variable for each translation unit, that bloats binary size.
Use iosfwd instead.

Oh I was unaware of this, thanks for the suggestion!

I notice that after this change I see compilation errors complaining about cerr not existing in std. This is on Ubuntu 18.04.
.../wasmtime.hh:185:10: error: ‘cerr’ is not a member of ‘std’

I notice that after this change I see compilation errors complaining about cerr not existing in std. This is on Ubuntu 18.04.
.../wasmtime.hh:185:10: error: ‘cerr’ is not a member of ‘std’
Also stream operator<< errors like:
wasmtime.hh:489:8: error: no match for ‘operator<<’ (operand types are ‘std::ostream {aka std::basic_ostream<char>}’ a
nd ‘const char [5]’)
     os << "v128";
     ~~~^~~~~~~~~

then #include, not

Thanks! Yes, sorry I meant to follow up my comment with your suggestion. I made this change and it works fine - though I found I still need to use iostream since cerr is defined in iostream and is used in wasmtime::Result::abort() which is giving me these issues.
Does it mean though that everywhere wasmtime is included one would end up being required to include  anyway? Maybe I missing something?

Thanks! Yes, sorry I meant to follow up my comment with your suggestion. I made this change and it works fine - though I found I still need to use iostream since cerr is instantiated in iostream and is used in wasmtime::Result::abort() which is giving me these issues.
Does it mean though that everywhere wasmtime is included one would end up being required to include ostream anyway? Maybe I missing something?

then use stdio.h

Thanks! Yes, sorry I meant to follow up my comment with your suggestion. I made this change and it works fine - though I found I still need to use iostream since cerr is instantiated in iostream and is used in wasmtime::Result::abort() which is giving me these issues.
Does it mean though that everywhere wasmtime is included one would end up being required to include ostream anyway? Maybe I missing something?

first. do not use abort() as name since some libcs might define it as macros.
then you
#include<cstdio>

fprintf(stderr,"error: %s\n",this->err().message());

std::terminate() or std::abort() or __builtin_trap();

BTW, i do not think unwrap() is a good design tbh.

Does https://github.com/bytecodealliance/wasmtime-cpp/pull/21 solve the issues recently described here?

Yes, #21 is a definite improvement! I now can include  instead of . It looks like ostream is still required by ValKind for example: https://github.com/bytecodealliance/wasmtime-cpp/blob/main/include/wasmtime.hh#L468
Would it makes sense to include  in the wasmtime.hh? I'm not sure what the standard practices are for public headers like this?

Ah yeah sorry I'm not super experienced in writing C++ libraries so I also don't know what to do here in that regard. I'm also surprised I haven't seen any of this in testing. Do you know how I would be able to reproduce these errors myself?

Ah yeah sorry I'm not super experienced in writing C++ libraries so I also don't know what to do here in that regard. I'm also surprised I haven't seen any of this in testing. Do you know how I would be able to reproduce these errors myself?

Actually not including iostream is extremely important in wasm since just include iostream bloats 1MB of binary size (if you are using libstdc++) or 400 kb for libc++. Which is extremely large.
WASM is designed for small binary size, iostream is something you should never ever use tbh.

Yeah, @alexcrichton - I had the same thought. I think maybe the gtest.h header is including ostream which is why it doesn't show in the testsuite?


 I thought it was only a problem of iOS. On my Mac and Safari it works, so there must be a version of Safari or combination Safari version- MacOS version that does not work?
Do you have of tested systems/browsers?

Can you describe the exact browser string?

Unfortunately, not. The teacher has reported this, said what a student reported her about his problems.

My PR #344 will at least help on the white screen on unsupported versions of safari.

Great, thanks! You know I have now on my 3 instances about 300 users in total, so I get A LOT of strange reports of bugs.

I can't help you more on this...

I know.... thanks.


[otbn,rtl] Drop pending RND prefetch at end of operation

The interesting commit here is the second one, with the following commit message:

Consider a situation where we have two operations, A and B, that run
back-to-back. Suppose operation A writes to RND_PREFETCH and then
finishes before the EDN replies. Now suppose that operation B also
writes to RND_PREFETCH. In this case, the RTL uses a signal called
rnd_req_queued_q to track the fact that it should send out another RND
request once the first (ignored) one comes back. But what if operation
B also finishes before the EDN has replied to the first request?
Once the result comes in, there was a slight mismatch between how the
ISS and RTL handled things (depending on the timing of the next
operation, the ISS might or might not have sent out another request).
But this whole situation is silly! We know we don't care about the
result of the second prefetch, so we don't need to do it at all.
This patch tweaks the RTL to drop the pending request in this
situation and adds a "forget" function in the ISS to do the same.

However, making this change exposes a second (simpler) problem: that we weren't updating rnd_prefetch_pending_q correctly if we ignored the result of the prefetch. The first commit fixes that by getting rid of the signal entirely.

No worries: this sort of thing is always much easier on the second time around. (And, honestly, this took quite a few tries at my end before everything worked!)


use cache to speed up travis build

Codecov Report

Merging #44 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #44   +/-   ##
=======================================
  Coverage   68.72%   68.72%
=======================================
  Files           9        9
  Lines         422      422
  Branches       60       60
=======================================
  Hits          290      290
  Misses        132      132

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ebebaa2...7e5086b. Read the comment docs.

Closed cause it does not really speed up...


Misc: SuperBlock, WAL, Replica, VOPR

Apologies for the enormous PR; this is everything that was waiting for dj-vopr-workload to merge. Many of these changes are entangled with storage verification improvements. Others just would have had conflicts otherwise.
Misc

[ ] CI: Run reliably-passing fuzzers.
[ ] build.zig:

[ ] Clean up benchmark and fuzzer "namespaces".
[ ] git_commit() is used to build the git commit hash into tigerbeetle version's output at compile time.

[ ] Document linked_event_chain_open and overflows_timeout.
[ ] Add tigerbeetle version subcommand.
[ ] BlockType for grids. (This is going to be reworked in a future PR to give grid blocks a distinct "magic number" from ordinary messages.)

Cleanup

[ ] Remove benchmark_array_search.zig — I think this file was duplicated somehow when LSM was merged. It is also in src/lsm/eytzinger_benchmark.zig.
[ ] Remove perf.zig — this was a backport of an 0.9 feature to 0.8, which we don't still need.

StateMachine

[ ] StateMachine: add overflows_timeout transfer error to catch when timestamp + timeout would overflow.

Utils

[ ] util.zig: copy_disjoint, copy_left, copy_right helpers (std.mem.copy()/copyBackwards() but with extra assertion). Use it everywhere instead of mem.copy{,Backwards}.
[ ] In NodePool.deinit()/MessagePool.deinit(), assert that all nodes/messages have been restored to the pool.
[ ] RingBuffer: Export count_max declaration.

Tests

[ ] Fuzz the ManifestLog.
[ ] Fuzz the SuperBlock
[ ] Unit test SuperBlock quorums.
[ ] Unit/fuzz test SuperBlock repair.

VOPR

[ ] Storage (src/test/storage.zig)

[ ] Document the per-zone fault injection patterns.
[ ] Add superblock zone fault injection. (This is used by both the superblock fuzzer and the VOPR).
[ ] Coalesce extra arguments into Options with better defaults, since so many tests/fuzzers are using the Storage now (not just the VOPR).
[ ] Add "crash faults": when we simulate a crash of a replica, fault the targets of in-flight writes.
[ ] generate_faulty_areas is now generate_faulty_wal_areas. Other zones will use different fault injection patterns, based on what their repair methods can tolerate.
[ ] Disable "crash faults" when replica_count == 1 — we need to implement redundant WAL header repair for that to work (it is an existing TODO).
[ ] Add Storage.copy(), which is used by the superblock and manifestlog fuzzers to clone/"fork" a Storage so they can try to open + repair from it, before we use the old storage to continue on our way.
[ ] Add back Jamie's size_used() function. (I can't find the commit it was in, but hopefully this is right).
[ ] Add verify_write_wal_header: when a redundant header is written, verify that we didn't leak it before the prepare.
[ ] Add verify_write_superblock: when a superblock sector is written, verify its trailers are already on-disk. Also checks that superblock writes are properly serialized (to guard against correlated errors).
[ ] Add some helper functions for reading from the storage, e.g. storage.grid_block(address). These can be used by StorageChecker and fuzzers.

[ ] MessagePool/Network/PacketSimulator: Use a separate message pool for the network so that we don't have to allocate an extra 20 messages in each replica/client pool. (As an unexpected bonus, this dramatically simplifies the message-leak check in Cluster.crash_replica().)
[ ] StorageChecker: This works, but too well :wink:. It is disabled until compaction's concurrent block allocation issue is fixed, so that the VOPR can still be useful and not crash right away.

WAL

[ ] Documentation!
[ ] Don't leak redundant headers before a prepare is on disk.
[ ] Remove the TODO: When participating in a view change, each replica should only send the headers from its working set that it knows it prepared. — this turns out to be safe as-is.
[ ] Split the Zone.wal into Zone.wal_headers and Zone.wal_prepares. This enables more fine-grained fault injection and verification in the Storage.
[ ] Change recover case @D when replica_count=1 from vsr to fix.
[ ] When we know the size of the message we are reading, use a smaller read buffer.
[ ] Improve torn repair recovery

[ ] This also checks the "only one concurrent write to the WAL" invariant for replica_count=1.
[ ] Add a TODO: For very small WALs (not usually a concern) we must be careful not to write to all redundant header sectors simultaneously, since a crash could leave us without redundant headers (which is very bad for a single-replica cluster).

[ ] Change case @D for replica_count=1 from vsr to fix to allow a torn header to be recovered.
[ ] Add Journal.remove_entry() — This is used in one (tricky) case in repair_prepares to discard an op that we don't need and can't repair.
[ ] Make Journal.writing() more conservative.

Replica

[ ] Document additional invariants.
[ ] Add on_compact() and on_checkpoint() handlers (like on_change_state); used by the StorageChecker.
[ ] Use SuperBlock.vsr_state.commit_min_checksum to anchor the hash chain — since vsr_state.commit_min is checkpointed, we don't want to rely on it being intact in our WAL in order to commit, and we just need the checksum to hash-chain the next commit.

[ ] Use this in valid_hash_chain_between.

[ ] Additional assertions on outgoing messages.

SuperBlock

[ ] Document VSRState invariants.
[ ] Remove Magic, didn't do anything.
[ ] Add SuperBlock.vsr_state.commit_min_checksum (see Replica).
[ ] Remove the SuperBlock.staging sector — this isn't needed since both view_change() and checkpoint() now store their update (vsr_state) on the Context, which is queued.
[ ] Remove the extra copyset; only 4 copies now.

view_change() no longer bumps the sequence twice.
Only check the parent hashchain opportunistically.

[ ] Repair after open. See: RepairIterator in superblock_quorums.zig.
[ ] Move Quorums into its own file.

tigerbeetle version
Sample non-verbose output (the "experimental" is because we aren't tagging versions yet, but eventually that would be semver):
TigerBeetle version experimental

Sample verbose output:
TigerBeetle version experimental

git_commit="552d1ca5bda2ee533d203e5c3a5f0795fbfe2053"
deployment_environment=Environment.development
log_level=2
replicas_max=6
clients_max=8
quorum_replication_max=3
port=3001
address="<IP_ADDRESS>"
memory_size_max_default=1073741824
cache_accounts_max=8192
cache_transfers_max=65536
cache_transfers_pending_max=65536
journal_slot_count=1024
journal_size_max=1073872896
connections_max=14
message_size_max=1048576
message_body_size_max=1048448
pipeline_max=8
connection_delay_min_ms=50
connection_delay_max_ms=1000
connection_send_queue_max_replica=4
connection_send_queue_max_client=2
client_request_queue_max=32
tcp_backlog=64
tcp_rcvbuf=4194304
tcp_sndbuf_replica=4194304
tcp_sndbuf_client=2097152
tcp_keepalive=true
tcp_keepidle=5
tcp_keepintvl=4
tcp_keepcnt=3
tcp_user_timeout_ms=17000
tcp_nodelay=true
cache_line_size=64
sector_size=4096
direct_io=true
io_depth_read=8
io_depth_write=8
superblock_copies=4
size_max=17592186044416
block_size=65536
block_count_max=268435456
lsm_trees=30
lsm_levels=7
lsm_growth_factor=8
lsm_key_size_max=32
lsm_table_size_max=67108864
lsm_manifest_node_size=16384
lsm_batch_multiple=4
lsm_snapshots_max=32
lsm_value_to_key_layout_ratio_min=16
tick_ms=10
rtt_ticks=30
rtt_multiple=2
backoff_min_ticks=10
backoff_max_ticks=1000
clock_offset_tolerance_max_ms=10000
clock_epoch_max_ms=60000
clock_synchronization_window_min_ms=2000
clock_synchronization_window_max_ms=20000
verify=true
journal_size_headers=131072
journal_size_prepares=1073741824
is_32_bit=false

Known problems (I'll address tomorrow):

Message leak in my PacketSimulator/Network rework.
CI trouble with the ManifestLog workflow.
CI trouble with the Simulator Canary workflow (I'm going to leave it in even though it often fails right now; I'll just remove the test workflow's dependency on it).


&DataType::Struct(_) error when using features=["dtype-struct"]

As soon as I try to add features=["dtype-struct"] to my Cargo.toml as described in https://marcogorelli.github.io/polars-plugins-tutorial/struct/ I get the following error during maturin develop
error[E0004]: non-exhaustive patterns: `&DataType::Struct(_)` not covered
   --> ~/.cargo/git/checkouts/pyo3-polars-c52492f603fd8974/7a353c3/pyo3-polars/src/types.rs:357:15
    |
357 |         match &self.0 {
    |               ^^^^^^^ pattern `&DataType::Struct(_)` not covered
    |
note: `DataType` defined here
   --> ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/polars-core-0.43.1/src/datatypes/dtype.rs:44:1
    |
44  | pub enum DataType {
    | ^^^^^^^^^^^^^^^^^
...
92  |     Struct(Vec<Field>),
    |     ------ not covered
    = note: the matched value is of type `&DataType`
help: ensure that all possible cases are being handled by adding a match arm with a wildcard pattern or an explicit pattern as shown
    |
492 ~             },
493 +             &DataType::Struct(_) => todo!()
    |

The dependencies in my Cargo.toml are as follows:
[dependencies]
pyo3 = { version = "0.22.2", features = ["extension-module", "abi3-py38"] }
pyo3-polars = { version = "0.17.0", features = ["derive"] }
serde = { version = "1", features = ["derive"] }
polars = { version = "0.43.1", features=["dtype-struct"], default-features = false }
faer = { version = "0.18" }

# hopefully temporary until
# https://github.com/pola-rs/pyo3-polars/pull/103 is included in
# a release
[patch.crates-io]
pyo3-polars = { git = "https://github.com/pola-rs/pyo3-polars.git" }

and my rust version is rustc 1.83.0-nightly (1d68e6dd1 2024-09-21). Is this a known issue? Any ideas how I can fix it?

thanks @wirhabenzeit - can you try to add this feature to pyo3-polars as well?

@MarcoGorelli thanks -- this solved it for me!


config: making this bad boy build for python3.7

What this PR does / why we need it:
Downgrades NumPy and Pandas to maintain. support for Python3.7.
Which issue(s) this PR fixes:

Fixes #

Exciting! Is it enough to just make this change or is there more to come?

What this PR does / why we need it: Downgrades NumPy and Pandas to maintain. support for Python3.7.
Which issue(s) this PR fixes:
Fixes #

I have this branch ready: https://github.com/Affirm/feast/tree/0.23-affirm-0.1
for master we have to keep it clean to be able to sync with feast main repo

What this PR does / why we need it: Downgrades NumPy and Pandas to maintain. support for Python3.7.
Which issue(s) this PR fixes:
Fixes #

I have this branch ready: https://github.com/Affirm/feast/tree/0.23-affirm-0.1 for master we have to keep it clean to be able to sync with feast main repo

makes sense, i'll change the default branch

What this PR does / why we need it: Downgrades NumPy and Pandas to maintain. support for Python3.7.
Which issue(s) this PR fixes:
Fixes #

I have this branch ready: https://github.com/Affirm/feast/tree/0.23-affirm-0.1 for master we have to keep it clean to be able to sync with feast main repo

    "google-api-core",
    "googleapis-common-protos",
    "grpcio-reflection",
    "numpy<1.22,<2",
    "pandas>=1,<2",
    "protobuf",
    "proto-plus",
    "PyYAML",
    "uvicorn[standard]",
    "tensorflow-metadata",
    "dask",

those are all conflicting dependencies :(

@hao-affirm which changes should i incorporate into this then? Just the setup.py?

@hao-affirm which changes should i incorporate into this then? Just the setup.py?

I'll get that work soon. the dependency is comlex so i'll take care of it as i already figured out the correct versions. doing some testing in ofs environment now

@hao-affirm should I close this PR then? are you going to open a different one? or will you take this one over?

closing in favor of: https://github.com/Affirm/feast/pull/2


feat(push): push notifications

Summary
Part 1 of a 3 part proposal for Push Notifications support in Tauri; this includes support in Tao for AppDelegate methods needed for APNS registration on macOS and iOS.
Part of tauri-apps/tauri#11651.
Please see the PR to Tauri for a full description:

tauri-apps/tauri#11652

@amrbashir could i get a workflow run? i'm curious how this does in ci. i think it should pass


Remove all unnecessary comments.

Go through all files and remove code sections that are commented out unless they are vital for further development.

Closing this one since there are no unnecessary comments left in the code.


VariablesView offers no visual indication as to when it stops showing possible values due to large number of variables

Version and Platform (required):

Binary Ninja Version: 3.2 dev

Bug Description:
VariablesView will query and display PossibleValueSet information for each variable. This can take too much time when the variables list is long so currently we don't query if there are more than some threshold number of variables. There are two solutions possible solutions.
A. Be more performant and perhaps add some caching. We can probably cache things as long as the variable list doesn't change, and we only refresh the list if the variable list has changed. Additionally we could probably only query for the variables that are on screen as another way to improve performance.
B. Add some visual indicator that variable values aren't being displayed as there are too many variables.

It's not the number of variables, but the "complexity" of the function. Users can change the complexity cutoff with ui.view.variables.pvsComplexityLimit. Though there should absolutely be some indication that this is happening

There is a third option here I'd prefer that's kinda a combination of the two -- a button similar to the way you can force analysis on large functions. This would also serve as a visual indication that analysis hasnt happened. If caching is added on top of this, all the better.

Fixed in 3.6.4626-dev


pytest test_models failed hypothesis health check

self = <selfdrive.car.tests.test_models.TestCarModel_39_GENESIS_GV70_1ST_GEN testMethod=test_panda_safety_carstate_fuzzy>

    @pytest.mark.nocapture
>   @settings(max_examples=MAX_EXAMPLES, deadline=None,
              phases=(Phase.reuse, Phase.generate, Phase.shrink))
E             hypothesis.errors.FailedHealthCheck: The method TestCarModelBase.test_panda_safety_carstate_fuzzy was called from multiple different executors. This may lead to flaky tests and nonreproducible errors when replaying from database.
E             See https://hypothesis.readthedocs.io/en/latest/healthchecks.html for more information about this. If you want to disable just this health check, add HealthCheck.differing_executors to the suppress_health_check settings for this test.

test_models.py:325: FailedHealthCheck

Oh, this is only a new warning in 6.102.6 and not a real bug since it's expected to run the base test class multiple times


undef CLAMP_COLOR_VALUE

undef CLAMP_COLOR_VALUE

It's not necessary...

I just saw you used it here.
https://github.com/ibireme/YYKit/blob/master/YYKit/Text/String/NSAttributedString%2BYYText.m#L437


Legalize [su]extend.i64 to iconst/sshr_imm + iconcat

This was already done for [su]extend.i128, and is necessary for codegen for 32-bit x86 (#1089).

Do I need to do something about the CI?

Yes, I don't know why this would change the test output though.

I don't know why this would change the test output though.

Hm, I'm not sure either unfortunately.

The test change is consistent with this patch, the uextend.i64 in the test case legalizes differently:
; over in filetests/filetests/isa/riscv/legalize-abi.clif:

function %split_call_arg(i32) {
    fn1 = %foo(i64)
    fn2 = %foo(i32, i64)
block0(v0: i32):
    v1 = uextend.i64 v0
    call fn1(v1)
    ; check: $(v1l=$V), $(v1h=$V) = isplit v1
    ; check: call fn1($v1l, $v1h)
    call fn2(v0, v1)
    ; check: call fn2(v0, $V, $V)
    return
}

The check that fails is $(v1l=$V), $(v1h=$V) = isplit v1, looking to see that the extended v1 is split back into two i32 arguments. Since this legalizes into an iconst.i32 0 the two i32 values are passed directly instead of going through a concat/split. I'm a little surprised to see it that smart, but that looks like what's happening.
Since the spirit of this test looks to be that the i64 argument of fn1 is split into two i32 that the platform can use, just changing the check will do:
function %split_call_arg(i32) {
    fn1 = %foo(i64)
    fn2 = %foo(i32, i64)
block0(v0: i32):
    v1 = uextend.i64 v0
    call fn1(v1)
    ; check: $(v1h=$V) = iconst.i32 0
    ; check: call fn1(v0, $v1h)
    call fn2(v0, v1)
    ; check: call fn2(v0, $V, $V)
    return
}

I'm a little surprised to see it that smart, but that looks like what's happening.

isplit and iconcat inside legalizations are immediately applied when possible without creating an instruction and then legalizing them afterwards.

Thanks @iximeow! I've fixed the test.

I noticed on Sunday or so that macos CI for 2d6c977 failed with what looked like a transient error (downloading the rust toolchain timed out or something along those lines), and tried rerunning it then, only to forget to revisit shortly thereafter. It looks like that rerun failed, believing that commit no longer exists?
git checkout --progress --force 2d6c977a4b9061c7a94d324dbaa0a9a93ba383cd
##[error]fatal: reference is not a tree: 2d6c977a4b9061c7a94d324dbaa0a9a93ba383cd
##[error]Git checkout failed with exit code: 128
##[error]Exit code 1 returned from process: file name '/home/runner/runners/2.169.1/bin/Runner.PluginHost', arguments 'action "GitHub.Runner.Plugins.Repository.v1_0.CheckoutTask, Runner.Plugins"'.

I just reran it again to the same failure. The last time I'd seen this category of failure was last week when GitHub was having issues a week or two ago, but that doesn't seem to be the case here.

I fiddled with git some, hope this helps.


MAINT: Avoid using np.compat.integer_types

This was removed in 2.0, since it is just int anyway since Python 2 is not a thing anymore.
A better pattern is probably:
        try:
            axes = (operator.index(axes),)
        except TypeError:
            pass

(broadening this to non Python integers).  But it doens't matter much and this is the more trivial change.

Thanks @seberg!
/test mini

Test failures are irrelevant.


Question, advice

Is there an offline manual?
Exporting a cube with holes bool'd from a pair of cylinders... obj looks good in blender, but not daz3d - normals whacked
ty

We apologize, but we do not have an offline manual available. Regarding daz3d, we do not have information about it.

We apologize, but we do not have an offline manual available. Regarding daz3d, we do not have information about it.

hello. as 'cad for artists' which art programs do you support for materials and rendering?


Improve build

Fix Unix script Shebang, see issue #21
Fix Gradle 6.5 Warnings
Use Java 8, see issue #19

Hi @gquintana Thank you for these corrections


design: Not create a new instance of setting tab

Fixes: #418
The functionality works as expected, need review on approach.
Approach - For Settings functional tab, it is not appended to the sidebar and to the windows menu. The settings icon at bottom is given an active tab which is removed when some other tab is activated.
You have tested this PR on:

[ ] Windows
[x] Linux/Ubuntu
[ ] macOS

This works out great, and the one thing i like about this PR is that the settings web view is not destroyable so the the time it takes to reopen the webview is instant which IMO is great. Although i wish we could somehow move the setting btn/icon on top (i think we just need to show the btn on top since we can't move it).

@abhigyank can you resolve the merge conflicts? Also, is this ready for review?

@akashnimare Done! Yes it is ready for review.

Works great. Just noticed a bug. When you add or remove a server the setting page should not be active.

Looks good. The other bug I have noticed is when you click on setting page, Window menu shows the first server as checked but at this time it shouldn't.
Repro -

Add two Zulip servers
Open the second server
Click on settings
Check Window menu

@abhigyank merged, thanks 🚀


Allow v1 Icons to accept uppercase first letter in icon name to facilitate v2 migration

Allow v1 Icons to accept uppercase first letter in icon name to facilitate v2 migration, since v2 expects Capitalized icon names, and v1 does not recognize this format.

Correct


Cloud execution fails after first rule execution. Output files are generated but can't be seen by next rule.

Snakemake version
7.12.1
When using --kubernetes option with Google Kubernetes Engine, rule output files that are stored on Google Cloud Storage (or S3) are not recognized by the next rule in the workflow.
Minimal example
from snakemake.remote.GS import RemoteProvider as GSRemoteProvider
GS = GSRemoteProvider()

prefix='mybucket/test1/'

rule all:
  input:
    GS.remote(f'{prefix}test_1.txt')

rule example:
  output:
    GS.remote(f'{prefix}test_1.txt', mkdir_remote=True)
  shell: 'touch {output}'

When this snakefile is run locally with the command snakemake --cores 1, all rules run, and the workflow completes without issues.
However, when this snakefile is run in the cloud with the following command:
snakemake --kubernetes --use-conda --default-remote-provider GS --default-remote-prefix mybucket/test1 -j 1
the rule example job runs runs, but the rule all job fails with the following error:
OSError: Missing files after 5 seconds.
even though the file does exist in the remote storage. I did try increasing --latency-wait to no avail. It is not able to see the output from the previous rule. If you now rerun the same command, rule all recognizes the file test_1.txt exists in cloud storage and reports:
Nothing to be done (all requested files are present and up to date).
So it appears that a cloud workflow can both read and write from the remote storage for the first rule that is executed, but files written to remote storage by any rule are "missing" for subsequent rules that require them as input. Upon manual check, the output file does indeed exist in google cloud storage. You can resubmit the job, where it will now see that file, but again, only a single rule will run. The output of that rule will not be seen by the next rule in the workflow.
Again, the workflow completes without issue when executed locally with remote storage (both GCS and s3). However, when using cloud execution with --kubernetes, rule output files are generated correctly, but cannot be recognized as input for the next rule. I have reproduced the error with both Google Cloud Storage and S3.
error.log
Thanks for your attention to this issue.

Hm, I've been using GS remote recently without issue. In your example you prepend the input and output files with prefix, with the same prefix you provide in the CLI option --default-remote-prefix. Perhaps this is causing an issue?


Restrict pytest to < 6.0

With the release of pytest 6.0.0 a previous warning seems to now be an error. See here who have also noted this issue. Latest pytest issues here.

Geez, MPI tests hanging. Will investigate either a bit later or more likely in the morning now.


Finalize CRD & APIService after 1h

What this PR does / why we need it:
Increase the time after which CRDs and APIService are being finalized to 1h.
Special notes for your reviewer:
cc @vlerenc
Release note:

When deleting a Shoot, CRDs and APIServices registered in it now have 1h to cleanup before being forcefully finalized.

Can you elaborate on the background of this change?


TypeScript d.ts files are missing in the latest (2.0.6) release

Bug
TypeScript d.ts files are missing in the latest (2.0.6) release

@seavan thanks looking into it now


Cant figure out how to configure this extension to find/lint *.bats files

Hi, thanks for this extension!
I am trying to figure out how to use this for linting *.bats-files. (https://github.com/bats-core/bats-core)
Support in shellcheck was added here: https://github.com/koalaman/shellcheck/pull/1469
Any help would be appreciated!

Simply mark the files as Shell Script files, did you try that?

Ahh! That works a charm! Thanks for the prompt reply and keep up the good work 🍾

And for anyone else getting lost on this:
Adding something like this to .vscode/settings.json will make the "marking" permanent
{
    "files.associations": {
        "*.bats": "shellscript"
    }
}


Support for text selection on the JSON API reference site.

The Spec website https://ethereum.github.io/execution-apis/api-documentation/ does not support selecting the method name and description, so there's no easy way to select and copy them. (tested chrome + safari)
Maybe this has something to do with the click-drop-down syle.
It would be useful if this was supported.

@mattsse copied over to https://github.com/open-rpc/docs-react/issues/502

duplicate of https://github.com/ethereum/execution-apis/issues/315


[afreecatv] Unable to extract station

Checklist

[X] I'm reporting a broken site
[X] I've verified that I'm running yt-dlp version 2022.07.18 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
Australia
Provide a description that is worded well enough to be understood
VODs that are age restricted do not download correctly, showing the message "Unable to extract station".
Previously, the --cookies argument had to be used to download age restricted videos but the above message is returned with and without --cookies on 2022.07.18.
yt-dlp -vU --cookies "C:\scripts\cookies.txt" https://vod.afreecatv.com/player/70395877

Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
C:\Windows>yt-dlp -vU --cookies "C:\scripts\cookies.txt" https://vod.afreecatv.com/player/70395877
[debug] Command-line config: ['-vU', '--cookies', 'C:\\scripts\\cookies.txt', 'https://vod.afreecatv.com/player/70395877']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.07.18 [135f05e] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 4.4-full_build-www.gyan.dev (setts), ffprobe 4.4-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.07.18, Current version: 2022.07.18
yt-dlp is up to date (2022.07.18)
[debug] [afreecatv] Extracting URL: https://vod.afreecatv.com/player/70395877
[afreecatv] 70395877: Downloading webpage
ERROR: [afreecatv] 70395877: Unable to extract station; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 644, in extract
  File "yt_dlp\extractor\afreecatv.py", line 232, in _real_extract
  File "yt_dlp\extractor\common.py", line 1194, in _search_regex

C:\Windows>

Still occurring on 2022.11.11
C:\Scripts>yt-dlp -vU --cookies "C:\scripts\cookies.txt" https://vod.afreecatv.com/player/70395877
[debug] Command-line config: ['-vU', '--cookies', 'C:\\scripts\\cookies.txt', 'https://vod.afreecatv.com/player/70395877']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19043-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.4-full_build-www.gyan.dev (setts), ffprobe 4.4-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [afreecatv] Extracting URL: https://vod.afreecatv.com/player/70395877
[afreecatv] 70395877: Downloading webpage
ERROR: [afreecatv] 70395877: Unable to extract station; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 674, in extract
  File "yt_dlp\extractor\afreecatv.py", line 232, in _real_extract
  File "yt_dlp\extractor\common.py", line 1228, in _search_regex

C:\Scripts>

I'm not all that knowledgeable about python but all the information seems to be there so i'm not quite sure why this isn't working?
Here's what I've found out...
this code
        station_id = self._search_regex(
            r'nStationNo\s*=\s*(\d+)', webpage, 'station')

is looking at the page source for "nStationNo", which is there?

and the regex (not a master of regex, although who is right?) pulls out the station_id using the regex that worked on regex101.com (unlike python regex works differently?)

but for reasons that still elude me, the regex fails (fatally), which gets to this code and displays the message shown in the console:
        elif fatal:
            raise RegexNotFoundError('Unable to extract %s' % _name)

I exported my cookies from Chrome and could download the video.
Could you please check your cookie file?
# Netscape HTTP Cookie File
# This file is generated by yt-dlp.  Do not edit.

.afreecatv.com	TRUE	/	FALSE	0	AbroadChk	OK
.afreecatv.com	TRUE	/	FALSE<PHONE_NUMBER>	AbroadVod	OK
.afreecatv.com	TRUE	/	FALSE	0	PdboxBbs	[...]
.afreecatv.com	TRUE	/	FALSE	0	PdboxTicket	[...]
.afreecatv.com	TRUE	/	FALSE	0	PdboxUser	[...]
.afreecatv.com	TRUE	/	FALSE	0	RDB	[...]
.afreecatv.com	TRUE	/	FALSE<PHONE_NUMBER>	TempCook	[...]
.afreecatv.com	TRUE	/	FALSE<PHONE_NUMBER>	_au	[...]
.afreecatv.com	TRUE	/	FALSE<PHONE_NUMBER>	_ausa	0xdeadbeef
.afreecatv.com	TRUE	/	FALSE<PHONE_NUMBER>	_ausb	0xdeadbeef
.afreecatv.com	TRUE	/	FALSE	0	isBbs	1
vod.afreecatv.com	FALSE	/	FALSE<PHONE_NUMBER>	ChatArea	off

The Pdbox session cookies with expire=0 are mandatory. Though they will not be used with --cookies-from-browser.


Automator: update istio/client-go@master dependency in istio/istio@master

Generated by Automator - 2024-03-13T23:22:49+00:00

@istio-testing: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-cni_istio
f9cb64a6ad24aaf9be756beaf6310063796a1f73
link
true
/test integ-cni

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


Improve parallelism of Target parsing

Still slow for two reasons:

We are waiting for dealloc of the project for some reason
Merging takes a long time, and we really just need an append

Still slight improvements here, so merging.


Added uk gov regions

Added uk gov regions

Datasource works

Customer test pending

this has been tested in oc4 tenancy
it is working fine for all oc4 regions


Merge pull request#1837from bgoonz/master

Browser metadata
Path:      /
Browser:   Chrome 99.0.4844.51 on Windows 10
Viewport:  1424 x 662 @1.350000023841858x
Language:  en-US
Cookies:   Enabled

Open in BrowserStack

Open Deploy Preview · Mark as Resolved


(chore) fix CSS in the dev tool to be cross-browser once again

Related to #3133
Fixes #3290

Changes
Replace CSS that only worked on Gecko browsers with more generic CSS
Checklist

[ ] Added markup tests, or they don't apply here because...
[ ] Updated the changelog at CHANGES.md

Snap? :) Does it look equally snazzy or did we revert to the older style?

Just added a screenshot, I think it looks just as snazzy :smile: It preserves the same behavior as far as I can tell.

Could we get maybe 2px of padding between the double dashed lines so it's easier to see the bounding rects? (ie bottom padding on the spans I think)  I'll merge either way but I think that would be  small improvement.

I added a padding of 5px. 3px seemed like too little for me.

Thanks.


re-logo the things

for @syleeeee
we need to update
./zipkin-lens/src/img/zipkin-sm-logo.svg
./zipkin-lens/src/img/zipkin-logo.svg
Also the banner, which was originally made by @making in #1612
./zipkin-server/src/main/resources/banner.txt
you can verify afterwards by starting the server using the instructions here:
https://github.com/openzipkin/zipkin/blob/master/zipkin-server/README.md#building-locally
maybe during this or after, the color scheme should change for lens. cc @openzipkin/ui

I'll contribute banner.txt ;)

Things were re-logo'd, thanks !

Did we relogoed for attic and contrib too?
On Thu, 9 Apr 2020, 19:41 Jorg Heymans<EMAIL_ADDRESS>wrote:

Things were re-logo'd, thanks !
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/openzipkin/zipkin/issues/2737#issuecomment-611651605,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAXOYAQ6KGJXW4F3VBJFCNLRLYCENANCNFSM4IJ33JJQ
.

Did we relogoed for attic and contrib too?

we could use the old logo for attic, for the sake of irony :)

Attic still has the old logo +1 to keep it, i wanted to re-logo contrib but i think it requires more rights than I currently have. After that we can close this issue.

heh.. try now?

Yep, new logo applied to contrib thanks,

https://twitter.com/zipkinproject is still using old one 🤔

unfortunately we have no access to the twitter account to change that, and
have failed for years getting someone to help us do that. I'll try again..
On Sat, May 9, 2020 at 3:50 PM Toshiaki Maki<EMAIL_ADDRESS>wrote:

https://twitter.com/zipkinproject is still using old one 🤔
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/openzipkin/zipkin/issues/2737#issuecomment-626124217,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAPVVZ4CPMZOAOA5ULYOALRQUDNXANCNFSM4IJ33JJQ
.


Add GOVERNANCE and MAINTAINERS

please review again @nathanielcopeland @Meakk

period could be removed I agree.
The ** are just bold, not headers.

The ** are just bold, not headers.

Sure but aren't they semantically (sub-)headings? in which case they might as well be proper headings in the markup.

pros: less hacky, can be anchor linked (eg section 3)
cons: corresponding paragraphs are very short so they'd look better as run-in headings? (which is what the MVG authors were trying to emulate I assume)

yeah I think that it looks better

I removed the periods.

looks like @Meakk and @snoyer agree that it should be subsection, ill make the change.


Using CSV to backup entire domain

So, I'm trying to backup emails from my entire domain (~700 users) -- I can create a CSV from the google admin panel, but I don't know how to draw from the CSV in order to backup all of the emails.  With GAM, there is a csv command, which works beautifully, but that doesn't exist for GYB.  Here is what I have tried:
for email in company.csv do gyb --email %email --service-account --batch-size 20
But it just hangs, with no error outputted (this was with a trial CSV with only 2 emails).
Am I getting the formatting wrong?  I have tried a few different ways but to no avail.  Any help is appreciated!

This isn't really a GYB issue (bug). Please open a thread on the discussion group.
Jay

I recently had to help a buddy with this, and wrote this quick script.. adding it here in case someone runs into this in the future.
#!/bin/bash
# Purpose: Read Users Report CSV File from G-Suite and perform backup to local mailbox
# based on predefined search query
# ------------------------------------------
GYB_BIN=/home/backupusr/bin/gyb/gyb
BACKUP_FOLDER=/mnt/backup/gsuite
LOGFILE_NAME=$BACKUP_FOLDER/gyb_logs/$(basename $0)-`date +"%Y-%m-%d-%H-%M"`.gyb.log
USERS_CSV_FILE="/mnt/backup/gsuite/all_users.csv"
SEARCH_REGEX="older_than:3y"
[ ! -f $GYB_BIN ] && { echo "$GYB_BIN file not found"; exit 99; }
[ ! -d $BACKUP_FOLDER ] && { echo "$BACKUP_FOLDER folder not found"; exit 99; }
[ ! -f $USERS_CSV_FILE ] && { echo "$USERS_CSV_FILE file not found"; exit 99; }
cd $BACKUP_FOLDER
while IFS="," read -r User skip
do
    echo "Backing up $SEARCH_REGEX of mails from mbox of : $User"
    $GYB_BIN --email $User --action backup --search $SEARCH_REGEX --batch-size 10 --service-account | tee -a $LOGFILE_NAME
    fi
done < $USERS_CSV_FILE


Using ServiceTransportType.Persistent in Management SDK breaks enum serialization

I'm using Management SDK in Azure Function to send messages back to our SignalR clients. Recently we switched to ServiceTransportType.Persistent (websocket) in the SDK to improve performance, but then messages arriving on the client side had Enums serialized as integers rather than strings, despite our Newtonsoft.Json settings. When using ServiceTransportType.Transient transport everything was working fine and our JSON settings were properly respected.
Please note since there is no way to configure Newtonsoft.Json in ServiceManagerBuilder currently, we are configuring it globally in function Startup class in the following manner:
JsonConvert.DefaultSettings = () => new JsonSerializerSettings
{
   ContractResolver = new CamelCasePropertyNamesContractResolver(),
   Converters = { new StringEnumConverter() }
};

This has worked perfectly find with 'transient' transport, so for example the below enum:
public enum Error
{
   SignalLost
}

was serialized as "SignalLost", but after switching to 'persistent' transport it's serialized as 0.

Could you provide the version of Management SDK and the function app project?
The .csproj file is better.

I cannot share the entire project file but here's an excerpt. We're running on .NET Core 3.1.
    <PackageReference Include="Microsoft.Azure.Functions.Extensions" Version="1.1.0" />
    <PackageReference Include="Microsoft.Azure.SignalR.Management" Version="1.7.1" />
    <PackageReference Include="Microsoft.Azure.WebJobs.Extensions.EventHubs" Version="4.0.1" />
    <PackageReference Include="Microsoft.Extensions.Http" Version="3.1.12" />
    <PackageReference Include="Microsoft.Extensions.Logging.ApplicationInsights" Version="2.16.0" />
    <PackageReference Include="Microsoft.NET.Sdk.Functions" Version="3.0.11" />
    <PackageReference Include="Newtonsoft.Json" Version="12.0.3" />
    <PackageReference Include="StyleCop.Analyzers" Version="1.1.118">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
    </PackageReference>

the json serializer is changed to System.Text.Json in function v3 and .Net Core 3.1, please use:
using System.Text.Json.Serialization;
and then add JsonStringEnumConverter like this:
    [JsonConverter(typeof(JsonStringEnumConverter))]
    public enum Error
    {
        SignalLost
    }

I'm not using output binding but Microsoft.Azure.SignalR.Management nuget. The reason is actually that I don't want to use System.Text.Json.Serialization, but rather Newtonsoft.Json. I can see in the code below that in the Transient transport mode the nuget is actually using Newtonsoft.Json through RestHubLifetimeManager and RestClientclasses:
https://github.com/Azure/azure-signalr/blob/9e9062797001c937e92808dbb92c594bd3c59aa6/src/Microsoft.Azure.SignalR.Common/Utilities/RestClient.cs#L139
But for the Persistent transport there is some more low-level SignalR stuff happening in the SDK so it's not clear how to control serialization for this case, because I don't see any JSON serialization there.

Persistent transport reuse Microsoft.Azure.SignalR package, it uses System.Text.Json by default.
Plan to expose the configuration of JSON serialization in the future, then you can configure the System.Text.Json or Newtonsoft.Json.
For now, Management SDK doesn't support it. How about keep using Transient mode for now? I will update here when the JSON configuration is exposed.

Yeah, for now we switched back to Transient mode. It'd be great to get JSON configuration exposed moving forward. Looking forward to it!

Hi @wanlwanl any updates on progress of message serialization settings?

@SebastianBienert We have added a method to customize the message serialization for both two modes. But the method is not public now. Will make it public for the next release.
https://github.com/Azure/azure-signalr/blob/dev/src/Microsoft.Azure.SignalR.Management/ServiceManagerBuilder.cs#L55


[BUG] PostgreSQL meant to upgrade (9.2 -> 10) but is failed and unstartable post-ELevate

Describe the bug
When checking if ELevate is safe to run, it reports that PostgreSQL 9.2 exists and will be one-way upgraded to v10:
* 2024-07-27 21:02:18 [WARN] You have postgresql-server version 9.2 installed.
This will be upgraded irreversibly to version 10.0 when you switch to CloudLinux 8

However when the ELevate job is complete, it leaves PostgreSQL broken. Automatic restart-fix does not work.
Manual attempt:
Jul 29 21:42:35 SQL systemd[1]: Starting PostgreSQL database server...
Jul 29 21:42:35 SQL postgresql-check-db-dir[135060]: An old version of the database format was found.
Jul 29 21:42:35 SQL postgresql-check-db-dir[135060]: Use 'postgresql-setup --upgrade' to upgrade to version '10'
Jul 29 21:42:35 SQL postgresql-check-db-dir[135060]: See /usr/share/doc/postgresql/README.rpm-dist for more information.
Jul 29 21:42:35 SQL systemd[1]: postgresql.service: Control process exited, code=exited status=1
Jul 29 21:42:35 SQL systemd[1]: postgresql.service: Failed with result 'exit-code'.
Jul 29 21:42:35 SQL systemd[1]: Failed to start PostgreSQL database server.

So we try that, have to install a package, try again, then it fails too:
$ postgresql-setup --upgrade
ERROR: Please install the postgresql-upgrade package.
$ yum install postgresql-upgrade -y
$ postgresql-setup --upgrade
 * Upgrading database.
ERROR: pg_upgrade tool failed
ERROR: Upgrade failed.
 * See /var/lib/pgsql/upgrade_postgresql.log for details.

That log file contains:
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok

*failure*
Consult the last few lines of "pg_upgrade_server.log" for
the probable cause of the failure.

connection to database failed: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/lib/pgsql/.s.PGSQL.5432"?

could not connect to source postmaster started with the command:
"/usr/lib64/pgsql/postgresql-9.2/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D "/var/lib/pgsql/data-old" -o "-p 5432 -b  -c listen_addresses='' -c unix_socket_permissions=0700 -c unix_socket_directory='/var/lib/pgsql'" start
Failure, exiting

To Reproduce
Steps to reproduce the behaviour:

Have PostgreSQL 9.x installed on machine to-be-elevated.
Upgrade via ELevate to RHEL v8 derivative as normal.
PostgreSQL cannot start nor be upgraded.

Expected behavior
It should do as mentioned, and successfully upgrade PostgreSQL to v10.

We currently block elevate upgrades if Postgresql is installed. What version of elevate did you use? /scripts/elevate-cpanel --version

Version 49 reported.

However, we only block it if you have customer databases installed. Which active databases did you have in the 9.x version?

Which active databases did you have in the 9.x version?

None that I can see. Safe to remove per normal then?

yep.

A note after the fact:
The reason postgresql-setup --upgrade fails is due to a bug in the postgresql-upgrade package AlmaLinux 8 inherited from RHEL. Specifically, the EL7 package used by CentOS 7 applies a custom patch that backports from 9.3 into 9.2 the ability to specify multiple Unix socket locations as unix_socket_directories instead of a single location with unix_socket_directory. In the EL8 version of the package used by AlmaLinux 8, the minimal version of PostgreSQL 9.2 built to allow postgresql-setup --upgrade to function does not apply this additional patch. Therefore, PostgreSQL 9.2 does not recognize the directive in its configuration file, which it considers to be a fatal error, preventing the upgrade from working.
The workaround is to change unix_socket_directories to unix_socket_directory just before running postgresql-setup --upgrade. This is reflected in the example set of instructions we provide in the Blockers page of our documentation.


Decrease line height on headers for improved readability

Current line height across all typography is 1.6 (golden ratio) x font-size. This works for body text but is too large for headings (h1-h5) and should be reduced (tightened up) for readability.
To do

[x] Work on optimal line height ratios for h1-h5
[ ] Review and approve the change with Konstantin
[ ] Update Documentation
[ ] Update Figma UI Kit
[ ] Update DS Typography Page

Unfortunately, we have this ugly accessibility rule that requires us to do line height (line spacing) to at least 1.5 times the font size;  So, the closest we can bring it is 1.5 :(
Also, if we make it less than 1.5 the accessibility tools will mark it as an error and our score will go down.
So, making it 1.5 is not a big difference from 1.6, But at lest we can make that.

Thank you for your input @kkoryaka! Let's go with 1.5 for headers then if that's the minimum. I don't want to affect the scores negatively.
I'll update the documentation, Figma UI kit, and website mockup to reflect 1.5 for headings.
FYI, I will probably create a separate ticket for myself to think about this a bit more at a later date. I want to have a convo around accessibility beyond the technical requirements at some point. USWDS and Accessibility.digital.gov allow for less than 1.5 on headers, so I'd like to look into how they've justified being able to use smaller line height (maybe trade-offs, other research, etc.)

Actually, i found pretty interesting take on headings: https://ux.stackexchange.com/questions/140142/accessible-line-height-for-headings-h1-h2-h3-etc
So, since headings aren't the main paragraph content, the line-height doesn't have to be 150% to be accessible. And we can fool accessibility checkers by adding line-hight to the html tag (the way accessibility.digital.gov does it)

. So, I think we can pull it off.

@kmbrlygl let me know when you have something ready

Here's what I'm proposing for the line heights for headings (in Figma)
I initially had different ratios for each heading but I thought that was too many different variables, so I landed on 1.4 ratio looking the best without being too tight or too loose for all headings. Let me know what you think!
FYI - I made a note re: why I used Vietnamese. I pulled random text from the Cannabis site (Google translated).
cc @adam-little

That looks good to me @kmbrlygl. Just please be sure to update the text styles in our shared Figma library. Thanks!

@kkoryaka I changed my mind about using only one ratio for headers. I'd decided to decrease the ratio by 1/8 step for each ascending level. Essentially as font-size increases, the ratio decreases. Here's it is in the UI Kit Library. The updated spacing is published. I am working on updating the documentation on the Visual Design style guide.
Let me know if you have questions.

@kmbrlygl  can you confirm that the headers are per your spec in the release branch: https://1-0-2-a11y-design.pr.designsystem.webstandards.ca.gov/

Yes, this is good to go.


[7.x] Add withQueryString method to Paginator

This PR adds a new method withQueryString to the Paginator class, which basically allows you to append all query strings to the paginator links, so instead of doing this:
{{ $users->appends(request()->query())->links() }}

You can now do:
{{ $users->withQueryString()->links() }}

I believe this is more readable and actually describes what you are doing better than appends(request()->query()).

We can't really use the global foundation request() helper from within the Paginate component.

Check out the PaginationServiceProvider and see how we inject callbacks that can resolve this kind of thing into the paginator for another approach you could take to this PR.


[AE] Add category for Clothing Sewing Materials

It has been flagged [issues/3443] that our current taxonomy lacks a dedicated category specifically for clothing sewing materials. This gap could potentially affect merchants who need to apply state-specific tax exemptions for items like fabric, thread, buttons, and zippers used in clothing construction, as required by New Jersey and Minnesota laws.
Existing categories under Arts & Entertainment > Hobbies & Creative Arts > Arts & Crafts can be a suitable umbrella for these products, but they do not clearly distinguish between general crafting materials and those intended for clothing (see image below). We propose creating a new category: Arts & Entertainment > Hobbies & Creative Arts > Arts & Crafts > Clothing Sewing Materials. This should more accurately denote clothing sewing-related materials and could be expanded in the future as needed or serve as a parent to applicable subcategories such as Craft Fasteners & Closures, Crafting Fibers, or Textiles.

fyi @FredyPla, as per this issue: https://github.com/Shopify/custom-data-issues/issues/3443


Add svelte-writable-derived and svelte-webext-storage-adapter

I make two utilities that make Svelte stores more powerful, and I hope you'll agree they should be in the list.

Good stuff, thank you. Do you have any thoughts on including these in their own top level or sub heading? I imagine there may be more store-oriented libraries in the future.
Also, the current formatting has the author included after the repo link - I just fixed it in this commit.

aaaaargh I just noticed I typoed "transforming"; could you fix that, too?

Yep fixed - I missed it too.


Dynamic shapes

Hi,
Is it so that interference helper do not support currently dynamic shapes e.g. in onnx model which would be used by tensorRT? Do you have plans to add support for this?
-Mikko

I don't have a plan for it


Adding event listeners on navigator.mediadevices throws error

What is the current behavior?
undefined is not an object (evaluating 't.addEventListener.apply')
1
File "../node_modules/react-use/esm/util.js" line 8 col 16 in on
return obj.addEventListener.apply(obj, args);
2
File "../node_modules/react-use/esm/useMediaDevices.js" line 23 col 9 in useMediaDevices
on(navigator.mediaDevices, 'devicechange', onChange);

This is Rollbar's minified error. It happens on iOS only. Although useMediaDevices check for
navigator && !!navigator.mediaDevices but I think this condition is true in this case and the navigator.mediaDevices.addEventListener is undefined.
Steps to reproduce it and if possible a minimal demo of the problem. Your bug will get fixed much faster if we can run your code and it doesn't have extra dependencies other than react-use. Paste the link to your JSFiddle or CodeSandbox example below:

Try it on ios browser. I haven't reproduced it, but I am getting the stats from Rollbar dashboard
Edit: I tried the app on the iPhone with Firefox. I can confirm its happening.

What is the expected behavior?
These errors should not happen
A little about versions:

OS: iOS
Browser (vendor and version): Chrome for iOS 83 and Mobile Safari 13
React: 15.1.0
react-use: 15.1.0
Did this worked in the previous package version? Don't know

I am willing to work on it. My idea is to verify that in on =>  obj.addEventListener is a function. Let me know what you think

What can be done based on workaround I did:

Safari on mobile does not support ondevicechange so I would suggest adding a check for it before calling on and off. Its available in the documentation
Add a check if enumerateDevices exist before calling it. This returns undefined in some ios devices


Minimum requirement

Added an option to put a minimum coverage constraint.
This resolves issue #68

By default minimum coverage constraint is zero
Number in config file represents the required minimum percentage of total coverage
Process exits by an error but still outputs the test report

Just noticed that using this feature with the one implemented in either pull requests #72 or #71 will cause the reporting to always fail, will work on that to make it work only on the tested files.

Just noticed that using this feature with the one implemented in either pull requests #72 or #71 will cause the reporting to always fail, will work on that to make it work only on the tested files.

#71 is now merged, could you adapt this PR to be based on the current master code instead?
Also, I think users would expect the minimum requirement % calculation to include the untested files when includeuntestedfiles is set.

Can I open another PR with the change?

Is this still in progress. Would love to get this feature landed

@allbarbos Yes, please!
(I'm very low on bandwidth for maintaining LuaCov — in fact, the previous maintainer passed away and I'm carrying the torch, but I'd love to get some maintainership help on this! If any of you wants to help out, please update this PR and let me know and I'll grant commit access)

@hishamhm I fixed up the PR. But it has issues beyond the simple feature;

the message and error value is generated in the default-reporter. Using another reporter will not behave the same, depending on how it was created.
moving it into the generic machinery is tricky, since you do not want an early exit because of this if the report has to be uploaded a coverage tool (eg. coveralls)

A simple option would be to leave this as it is today, and have a CI user run 2 reports. First the one they want for display/reporting. And then the default one, which will display the error.
(though that means one cannot configure the stats files to be cleared, since the second run would need those as well)

I'm leaning towards a separate reporter that doesn't really report anything, but just generates the error message.


Achieve action needs to calculate eligibility when two or more of the same value achievement are available

Basically, if the player already has 1 age 1 achievement, they would need 2 * 5 * 1 for a second age 1 achievement.  This achieve option should be aware of this before displaying the achieve option during the player's turn.

The eligibility logic is on the backend, but there is some frontend work to do here as well. We need to adjust the tooltips when hovering over claimable achievements, and we also need to think about which achieve buttons we want to show. For example, if there are two Echo 2s and one Base 2, and all 3 are eligible to be achieved, do we actually want to show three buttons for that? Or do we just show one button and clicking it will achieve an arbitrary achievement (forcing them to click on a specific card if they care which achievement they will get). My guess is that most times players won't care which achievement they claim if they are all the same number. Unless they are tracking the movement of cards, they won't know which are which.

Also, Laser does too if I recall correctly.

I'd say one button for the achieve 2 action (which would lead to a choice).  If they are still eligible for the next 2, then that option would pop back up again.

I was thinking they'd have a choice of "Achieve 2", "Achieve 3", etc. and then after they select it, then they'd have an option to select which one they'd want (like how the artifacts choose achievements already).  If, after the first selection, they are still eligible for another achievement of the same value, the option would stay there and be a valid selection until they lose eligibility.

Okay, but what's the purpose of getting them to click on "Achieve X" if they still need to click on a specific card afterwards? They already have the alternative to click on the achievement directly:

Personally, I think the best user experience would be as followed: If they don't care which specific achievement they want then they can click on "Achieve X" and we will give them an achievement arbitrarily. If they do care, then they can click on a specific card.
The added bonus of this proposed approach is that it's less work for us. This is already the existing behavior.

Oh yeah, that's totally better.  I didn't realize you could click on achievements already.

Oh great! Glad we are on the same page now!

I took a look at this and isn't clear how to add in the bonuses to the score.  It looks like the score is tracked by how it changes from one move to the next (i.e. adding/removing cards to/from the score pile).  The bonuses are not moved into a score pile...they become visible by dogma or melding cards so calculating the "delta" would require knowing how many bonuses (and the max value) before and after the dogma.

I took a look at this and isn't clear how to add in the bonuses to the score. It looks like the score is tracked by how it changes from one move to the next (i.e. adding/removing cards to/from the score pile). The bonuses are not moved into a score pile...they become visible by dogma or melding cards so calculating the "delta" would require knowing how many bonuses (and the max value) before and after the dogma.

I think you posted that comment on the wrong issue. This has nothing to do with bonuses.

This work should be done.


Fix typo Slides instead of Sheets in PowerPoint docs

how-to-retrieve-the-number-of-slides-in-a-presentation-document.md
Changes Sheets to Slides (structure was probably copied from Excel documentation.)

Docs Build status updates of commit 09b1b71:
:warning: Validation status: warnings

File
Status
Preview URL
Details

docs/how-to-retrieve-the-number-of-slides-in-a-presentation-document.md
:warning:Warning

Details

docs/how-to-retrieve-the-number-of-slides-in-a-presentation-document.md

Line 26: [Warning-insecure-link] Link 'http://www.microsoft.com/en-us/download/details.aspx?id=30425' is insecure. Links to Microsoft sites must use 'https' instead of 'http'.
Line 90: [Warning-hard-coded-locale] Link 'https://msdn.microsoft.com/en-us/library/office/documentformat.openxml.packaging.presentationdocument.open.aspx' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites.
Line 93: [Warning-hard-coded-locale] Link 'https://msdn.microsoft.com/en-us/library/office/documentformat.openxml.packaging.presentationdocument.presentationpart.aspx' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites.
Line 123: [Warning-hard-coded-locale] Link 'https://msdn.microsoft.com/en-us/library/office/documentformat.openxml.packaging.presentationpart.slideparts.aspx' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites.
Line 151: [Warning-hard-coded-locale] Link 'https://msdn.microsoft.com/en-us/library/office/documentformat.openxml.presentation.slide.show.aspx' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites.
Line 158: [Warning-insecure-link] Link 'http://msdn2.microsoft.com/EN-US/library/bb301979' is insecure. Links to Microsoft sites must use 'https' instead of 'http'.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 8148b1a:
:warning: Validation status: warnings

File
Status
Preview URL
Details

docs/how-to-retrieve-the-number-of-slides-in-a-presentation-document.md
:warning:Warning

Details

docs/how-to-retrieve-the-number-of-slides-in-a-presentation-document.md

Line 26: [Warning-insecure-link] Link 'http://www.microsoft.com/download/details.aspx?id=30425' is insecure. Links to Microsoft sites must use 'https' instead of 'http'.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 1422520:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/how-to-retrieve-the-number-of-slides-in-a-presentation-document.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


Enable server-side encryption for all S3 objects

While investigating https://github.com/DataBiosphere/azul/issues/4715 it was discovered that the following buckets contain unencrypted objects:
HCA dev:

data-browser.dev.lungmap.net (753 unencrypted objects)

HCA prod:

archive-preview.humancellatlas.org (501 unencrypted objects)
data-browser.lungmap.net (753 unencrypted objects)

Please enable server-side encryption for all the objects in these buckets.

Server-side encryption has now been enabled for all buckets but it only affects object written after it was enabled on a bucket. The unencrypted objects need to be deleted or overwritten.
@bvizzier-ucsc, can anything be done to get CleverCanary to respond to this? It would be helpful to have a timeline for resolution.

Hi @dsotirho-ucsc can let us know the urgency of this request? Are there any timelines associated with this we should be aware of? This will help us prioritize this work.  cc @bvizzier-ucsc

Also, @dsotirho-ucsc I just did a dev and prod deployment for lungmap.
Can you let me know if this fixed the issue for:

data-browser.dev.lungmap.net and
data-browser.lungmap.net

Cheers and Thanks,
D

@NoopDog, please direct priority and scheduling questions at me and @bvizzier-ucsc. Thank you!
To answer your question, this is a medium-severity finding so it has a 90 day remediation target. The finding is from last year so we're already way past the deadline, luckily not in a currently compliant deployment. If you could fix this ASAP that would help limit the damage to our track record wrt timely mediation in the eyes of the AO and the yearly 3PAO.
It just requires reading and writing the objects again so it should be real easy to fix (< 10min) with aws s3 sync.

@hannes-ucsc Scheduling questions I will direct exclusively to @bvizzier-ucsc. I would prefer if any tickets assigned to us already contained the information required for Ben and me to prioritize against our other tasks.
The data-browser.dev.lungmap.net and data-browser.lungmap.net buckets should already be fixed. If you or @dsotirho-ucsc can verify that would be great.
Also if you can fix archive-preview.humancellatlas.org in 10 minutes that would be great. Woud you be able to claim this ticket?
Cheers and Thanks,
D

@bvizzier-ucsc has a lot on his plate and I would like to help where I can so if you could also include me in scheduling questions, that would be helpful. Thank you!

Also if you can fix archive-preview.humancellatlas.org in 10 minutes that would be great. Woud you be able to claim this ticket?

Sorry, not at this moment. If it takes you considerably longer than 10min, let us know and we'll see what we can do to help.

Triaging to discuss the expected timeline with @bvizzier-ucsc.

This is a bucket owned and populated by the front end team.  The proper function needs to be validated by them once the change is made.

Not deleted just encrypted
On Wed, Oct 4, 2023 at 3:25 PM Ben Vizzier @.***> wrote:

This is a bucket owned and populated by the front end team. The proper
function needs to be validated by them once the change is made.
—
Reply to this email directly, view it on GitHub
https://github.com/DataBiosphere/data-browser/issues/3647#issuecomment-1747731539,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAYW6ENQLGQBPENYOWQHEMTX5XO3ZAVCNFSM6AAAAAA4PRZBPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBXG4ZTCNJTHE
.
You are receiving this because you were mentioned.Message ID:
@.***>

I just did it myself. It really just entailed aws s3 sync <bucket_uri> . && aws s3 sync <bucket_uri> .. Thanks anyway.


No way to print asset URLs without protocol.

There seems to be no way of printing an asset url such that it leaves off the protocol.
I want to print an image tag like so:
<img src="//mydomain.com/Path/to/images/myimage.jpg"/>

There is no way to print a protocol-less URL automatically.  You can either display an absolute URL or a relative one.  If you want protocol-less you will need to build the URL yourself by appending the relative path to the domain.

Thank you for confirming this... can this be a feature?