Plugin architecture for modules

This is my initial draft at using a plugin architecture for loading modules with optional shared library dependencies. This is to address issue #1406.
There are a few major things here:

I've switched from using GLOB_RECURSE in src/CMakeLists.txt to instead having a separate CMakeLists.txt that explicitly lists sources within each subdirectory that compiles down to a static library for that subdirectory. I can backtrack here if needed, but I think the end result is a bit cleaner and it's apparently against CMake best practices to use GLOB_RECURSE for defining the source files for a target anyway, so I think this should be OK.

All modules are built as shared library files. All of the modules that don't have external dependencies are currently all built together as one big shared library that's linked in at compile time. The others are all built as their own standalone shared libraries.
Because of how the module factory registration function works, this is currently mandatory, but I don't think it is that big of a limitation.

Currently the list of plugins is hard-coded in utils/plugin.hpp; this is non-ideal but probably OK for now. In the future, we might want to explore how to define a plugin path (or list of paths) to search for modules, but I didn't bother here because there's apparently some goofiness with dlopen() with paths that have any slashes in them (where it assumes the path to be absolute).

I'm almost certain to have screwed something up somewhere, so I appreciate any feedback on this (and if people could run it through its paces to root out any remaining bugs).

Just a quick input, haven't reviewed it thoroughly yet, only looked at the basic approach.
If we are going to separate out the modules as libraries, would it be a good idea to move all the source files belonging to a module into a single directory?
So for example for alsa the adapters/alsa/control.cpp and adapters/alsa/mixer.cpp files would move into the src/modules/alsa folder.
To the points you've made:

I'm not really a fan of building a library for each subdirectory. What would be the advantage over having each subdirectory add all its files onto a list of source files and pass that list up to their parent? But I agree that explicitly listing files could save some trouble.

"Because of how the module factory registration function works, this is currently mandatory, but I don't think it is that big of a limitation." What exactly is mandatory? That the modules without external dependencies are built as one library?

I agree. But this can probably wait until we officially support third party compiled modules in this plugin architecture.

I won't be able to properly review this for the next two weeks because I'm going on vacation, but I think getting this merged will be a longer process anyways.

Also, for the next time, please create the branch for a PR in your own fork, if you push directly to this repo it spams the RSS feed.

If we are going to separate out the modules as libraries, would it be a good idea to move all the source files belonging to a module into a single directory?
So for example for alsa the adapters/alsa/control.cpp and adapters/alsa/mixer.cpp files would move into the src/modules/alsa folder.

Yeah, this is a good idea.

I'm not really a fan of building a library for each subdirectory. What would be the advantage over having each subdirectory add all its files onto a list of source files and pass that list up to their parent? But I agree that explicitly listing files could save some trouble.

In hindsight, I agree with you and I've switched to using this approach as it's cleaner.

"Because of how the module factory registration function works, this is currently mandatory, but I don't think it is that big of a limitation." What exactly is mandatory? That the modules without external dependencies are built as one library?

Building the modules as a shared library is currently mandatory. (They don't need to be together in one shared lib necessarily, but they need to be in a shared lib.) Doing so gets CMake to add the library path in the build directory to the rpath so that the plugins can be found when running polybar directly from the build folder. If we don't link against any shared library, the rpath is not modified and it won't search the build/lib dir when we try to dlopen() the plugins. There might be another way to get around this, but the easiest thing for now is to just build them as a .so and let CMake take care of the rest.

I won't be able to properly review this for the next two weeks because I'm going on vacation, but I think getting this merged will be a longer process anyways.

NBD. This is a big change so we should make sure we do it right.

Codecov Report

Merging #1409 into master will increase coverage by 3.66%.
The diff coverage is 17.64%.

@@            Coverage Diff            @@
##           master   #1409      +/-   ##
=========================================
+ Coverage    5.84%    9.5%   +3.66%
=========================================
  Files         161      91      -70
  Lines        8918    5283    -3635
=========================================
- Hits          521     502      -19
+ Misses       8397    4781    -3616

Flag
Coverage Δ

#unittests
9.5% <17.64%> (+3.66%)
:arrow_up:

Impacted Files
Coverage Δ

include/cairo/context.hpp
0% <0%> (ø)
:arrow_up:

src/components/controller.cpp
0.27% <0%> (-0.01%)
:arrow_down:

src/components/bar.cpp
0.2% <0%> (ø)
:arrow_up:

src/utils/plugin.cpp
11.11% <11.11%> (ø)

src/utils/bspwm.cpp
3.12% <14.28%> (+1.37%)
:arrow_up:

src/utils/restack.cpp
40% <40%> (ø)

include/utils/restack.hpp
66.66% <66.66%> (ø)

include/components/config.hpp
0% <0%> (ø)
:arrow_up:

include/x11/extensions/xkb.hpp
0% <0%> (ø)
:arrow_up:

include/events/signal.hpp
0% <0%> (ø)
:arrow_up:

... and 53 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 68ae7a8...e9ca55a. Read the comment docs.

Any update on this?


Alternative solution to db crash

Another potential solution to database unresponsiveness. Discuss and merge if prefered.

Closing this pull request due to non-complete solution and unwanted behavior when no db connection available.


gRPC 0.13.0 compile expects protobuf to use pkgconfig, but with cmake it does not

There are platforms (like MinGW64 / MSYS2) where currently the cmake-build of protobuf works better than the autotools build. But when using cmake with protobuf, then gRPC will not find protobuf due to cmake's lack of pkgconfig-support. It would be good if gRPC would recover from this lack, by falling back to testing for protobuf. The test exists already in the Makefile, but is not used if pkgconfig is available.
See also https://github.com/google/protobuf/issues/1384

You can disable pkg-config tests by doing:
HAS_PKG_CONFIG=false make


"Stack smashing error" when trying to run the demo

Error with G++ 5.3, Ubuntu 14.04

Thanks for reporting. I am slightly busy with other projects right now. I'll look into it asap!

Thank you, looking forward it :+1:

I can't reproduce this here on a vanilla Ubuntu 14.04 64bit. Compiles fine.
1.) Standard g++ is
gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.1)
Any specific requirement on why you use g++ 5.3? How did you install g++ 5.3? Could you try with the default gcc and look if the bug still occurrs?
2.) If you are on a 64bit system: Did you try to build the pattern data? The supplied pattern data are only for 32bit.
3.) If you are on a 32bit system: Can you try to build the pattern data and use those instead of the supplied ones?

Make gave out following error:

So i manually compile and run

I have just reinstalled Lubuntu 15.10 32bit, so GCC is newer (5.2.1)
I will try again with with old version soon.

well the error message apparently says that the linker can't find the lib.
And looking at your screenshot, you have no
/ctegaki-lib/demo/lib/libctegaki.a
so, try to create that lib subdirectory within demo and copy the lib to it, or change the makefile accordingly....

it seems your issue has been solved. Otherwise, please open a new one...


Esphome feature request?

Hi @buxtronix
first of all - thanks a lot for your work with these blind engines - it is a nice addition to our smarthome;) btw - have you ever concidered to move / port the project over to esphome ?

That would be really good.

This is already in progress, I've got an open PR to enable ble client support:
https://github.com/esphome/esphome/pull/1177
With a follow-up one that then enables am43 support. You can check out my fork here:
https://github.com/buxtronix/esphome/tree/am43

Wow that sounds great! Is it something I can try out or is it not ready yet?

Yes the fork works very well. It's been controlling a pair of my blinds daily for many months.

@buxtronix great 🤘 Sorry for asking a lot, but how do I use your fork ? Can I copy the sm43 libraries over in my existing esphome under HA?

You would need the entire repo as some of the other parts of the esphome codebase are changed.
If you are using Hass.io, I think you can set the config options "esphome_version" to "buxtronix:am43" though I havent been able to try this myself.

Sounds like the Way to go @buxtronix ! I Will try - though om afraid it Will mess up my other configurations :/ maybe I will wait til your pr goes live in a new release . Any clue how long it will go before merging it int the base code ?

If you are using Hass.io, I think you can set the config options "esphome_version" to "buxtronix:am43" though

Thanks, it works. Do you have got a configuration example for the am43 gateway ??

Please re-fetch and try again. (though i'm not sure how to force a re-fetch, perhaps flip 'esphome_version' to the previous, restart, then back to my fork.)

still have an issue((
Compiling /data/am43_gateway/.pioenvs/am43_gateway/src/esphome/components/am43_cover/am43_base.cpp.o
Compiling /data/am43_gateway/.pioenvs/am43_gateway/src/esphome/components/am43_cover/am43_cover.cpp.o
Compiling /data/am43_gateway/.pioenvs/am43_gateway/src/esphome/components/api/api_connection.cpp.o
Compiling /data/am43_gateway/.pioenvs/am43_gateway/src/esphome/components/api/api_pb2.cpp.o
src/esphome/components/am43_cover/am43_cover.cpp: In member function 'virtual void esphome::am43_cover::Am43Cover::loop()':
src/esphome/components/am43_cover/am43_cover.cpp:24:13: error: 'class esphome::am43_cover::Am43Cover' has no member named 'node_state_'
   if (this->node_state_ == espbt::ClientState::Established && !this->logged_in_) {
             ^
src/esphome/components/am43_cover/am43_cover.cpp:26:59: error: 'class esphome::ble_client::BLEClient' has no member named 'gattc_if_'
     auto status = esp_ble_gattc_write_char(this->parent_->gattc_if_, this->parent_->conn_id_,
                                                           ^
src/esphome/components/am43_cover/am43_cover.cpp:26:85: error: 'class esphome::ble_client::BLEClient' has no member named 'conn_id_'
     auto status = esp_ble_gattc_write_char(this->parent_->gattc_if_, this->parent_->conn_id_,
                                                                                ^
src/esphome/components/am43_cover/am43_cover.cpp: In member function 'virtual void esphome::am43_cover::Am43Cover::control(const esphome::cover::CoverCall&)':
src/esphome/components/am43_cover/am43_cover.cpp:46:13: error: 'class esphome::am43_cover::Am43Cover' has no member named 'node_state_'
   if (this->node_state_ != espbt::ClientState::Established) {
             ^
src/esphome/components/am43_cover/am43_cover.cpp:52:59: error: 'class esphome::ble_client::BLEClient' has no member named 'gattc_if_'
     auto status = esp_ble_gattc_write_char(this->parent_->gattc_if_, this->parent_->conn_id_,
                                                           ^
src/esphome/components/am43_cover/am43_cover.cpp:52:85: error: 'class esphome::ble_client::BLEClient' has no member named 'conn_id_'
     auto status = esp_ble_gattc_write_char(this->parent_->gattc_if_, this->parent_->conn_id_,
                                                                                ^
src/esphome/components/am43_cover/am43_cover.cpp:61:59: error: 'class esphome::ble_client::BLEClient' has no member named 'gattc_if_'
     auto status = esp_ble_gattc_write_char(this->parent_->gattc_if_, this->parent_->conn_id_,
                                                           ^
src/esphome/components/am43_cover/am43_cover.cpp:61:85: error: 'class esphome::ble_client::BLEClient' has no member named 'conn_id_'
     auto status = esp_ble_gattc_write_char(this->parent_->gattc_if_, this->parent_->conn_id_,
                                                                                ^
src/esphome/components/am43_cover/am43_cover.cpp: In member function 'virtual void esphome::am43_cover::Am43Cover::gattc_event_handler(esp_gattc_cb_event_t, esp_gatt_if_t, esp_ble_gattc_cb_param_t*)':
src/esphome/components/am43_cover/am43_cover.cpp:82:33: error: 'class esphome::ble_client::BLECharacteristic' has no member named 'handle_'
       this->char_handle_ = chr->handle_;
                                 ^
src/esphome/components/am43_cover/am43_cover.cpp:84:70: error: 'class esphome::ble_client::BLEClient' has no member named 'gattc_if_'
       auto status = esp_ble_gattc_register_for_notify(this->parent_->gattc_if_, this->parent_->remote_bda_,
                                                                      ^
src/esphome/components/am43_cover/am43_cover.cpp:84:96: error: 'class esphome::ble_client::BLEClient' has no member named 'remote_bda_'
       auto status = esp_ble_gattc_register_for_notify(this->parent_->gattc_if_, this->parent_->remote_bda_,
                                                                                ^
src/esphome/components/am43_cover/am43_cover.cpp:85:60: error: 'class esphome::ble_client::BLECharacteristic' has no member named 'handle_'
                                                       chr->handle_);
                                                            ^
src/esphome/components/am43_cover/am43_cover.cpp:92:13: error: 'class esphome::am43_cover::Am43Cover' has no member named 'node_state_'
       this->node_state_ = espbt::ClientState::Established;
             ^
src/esphome/components/am43_cover/am43_cover.cpp:110:65: error: 'class esphome::ble_client::BLEClient' has no member named 'gattc_if_'
           auto status = esp_ble_gattc_write_char(this->parent_->gattc_if_, this->parent_->conn_id_,
                                                                 ^
src/esphome/components/am43_cover/am43_cover.cpp:110:91: error: 'class esphome::ble_client::BLEClient' has no member named 'conn_id_'
           auto status = esp_ble_gattc_write_char(this->parent_->gattc_if_, this->parent_->conn_id_,
                                                                                ^
*** [/data/am43_gateway/.pioenvs/am43_gateway/src/esphome/components/am43_cover/am43_cover.cpp.o] Error 1
========================= [FAILED] Took 10.58 seconds =========================

Ah yes, this is a bit of a mess right now after some updates to the parent branch. I'll get it sorted and update today.

It should be good now.

It should be good now.

After the flashing FW I have received an error -
The firmware has been successfully compiled, but for some reason, I can't flash it via ESPHome, so I just tried to burn firmware.bin via nodeMCU Flasher.
st:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371
ets Jun  8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371

This is my config:
esp32_ble_tracker:

ble_client:
  - mac_address: 02:4D:FB:F0:2B:2E
    id: art_left
  - mac_address: 02:69:32:F0:A5:1D
    id: art_right

cover:
  - platform: am43_cover
    name: "ArtLeft Cover"
    id: art_left_cover
    ble_client_id: art_left
  - platform: am43_cover
    name: "ArtRight Cover"
    id: art_right_cover
    ble_client_id: art_right

sensor:
  - platform: am43
    ble_client_id: art_left
    battery_level:
      name: "AM43 Left battery"
    illuminance:
      name: "AM43 Left light"
    update_interval: 120s
  - platform: am43
    ble_client_id: art_right
    battery_level:
      name: "AM43 Right battery"
    illuminance:
      name: "AM43 Right light"
    update_interval: 120s
  - platform: uptime
    name: Uptime Sensor

switch:
  - platform: restart
    name: "Device Restart"
  - platform: ble_client
    name: "Left AM43 On"
    ble_client_id: art_left
  - platform: ble_client
    name: "Right AM43 On"
    ble_client_id: art_right
    id: right_on

Hi
First of all - Thanks for your great work.
I try your config with my am43.  I have two - left and right :) but only one works at a time
if I turn on both curtains, only one will work.
if I turn off any of them, then the second will work correctly.
the one that does not work gives an error - [am43_cover:079]: [ArtLeft Cover] No control service found at device, not an AM43..?
below is the log after reboot. I use your config, the only other is mac
https://pastebin.com/0LwmKjtt

Hi
First of all - Thanks for your great work.
I try your config with my am43. I have two - left and right :) but only one works at a time
if I turn on both curtains, only one will work.
if I turn off any of them, then the second will work correctly.
the one that does not work gives an error - [am43_cover:079]: [ArtLeft Cover] No control service found at device, not an AM43..?�[0m
below is the log after reboot. I use your config, the only other is mac
https://pastebin.com/0LwmKjtt

I have the same problem with my 2 motors :(

Hi,
I also want to thank you.
I get the firmware on the device and it is running. But I not can connect to the AM43. At the app it is possible. I of course also delete the connection to the app before I try to access over the ESP32. Do you saw that issue before?
[14:04:36][D][esp32_ble_tracker:180]: Starting scan...
[14:04:41][D][ble_client:045]: Found device at MAC address [02:81:B1:08:0C:E7]
[14:04:41][I][ble_client:083]: Attempting BLE connection to 02:81:b1:08:0c:e7
[14:04:42][I][ble_client:140]: [02:81:b1:08:0c:e7] ESP_GATTC_DISCONNECT_EVT
[14:04:42][D][sensor:092]: 'AM43 Blind1 battery': Sending state nan % with 0 decimals of accuracy
[14:04:42][D][sensor:092]: 'AM43 Blind1 light': Sending state nan % with 0 decimals of accuracy
[14:04:42][I][ble_client:113]: [02:81:b1:08:0c:e7] ESP_GATTC_OPEN_EVT
[14:04:42][W][ble_client:115]: connect to 02:81:b1:08:0c:e7 failed, status=133
[14:04:42][D][esp32_ble_tracker:180]: Starting scan...
[14:04:52][W][am43:090]: [02:81:b1:08:0c:e7] Cannot poll, not connected
[14:05:10][D][sensor:092]: 'Uptime Sensor': Sending state 1184.43005 s with 0 decimals of accuracy
Thanks

Anybody have an idea on post above?
Thanks

@JochenKlenk I have the same problem, I did manage to connect one of the device, after uploading via usb-cable and not OTA.
But noticed with two am43-devices only one gets connected, the first gets connected (I see the pin-message) and the second the pin-message is missing. I also did a reset and everytime I upload via usb-cable or ota another AM43 gets connected first. The first connected device is perfectly working and the second not.

Еhanks for the blind controller
I use your config:
sensor:
  - platform: am43
    ble_client_id: art_left
    battery_level:
      name: "Kitchen_left_blind battery"
    illuminance:
      name: "Kitchen_left_blind light"
    update_interval: 120s

But ESPHome gives error:
Platform not found: sensor.am43
Does anyone know how to solve?
At the same time the am43_cover and ble_client components is connected and works great.

Thanks for the blind controller
I use your config:
sensor:
  - platform: am43
    ble_client_id: art_left
    battery_level:
      name: "Kitchen_left_blind battery"
    illuminance:
      name: "Kitchen_left_blind light"
    update_interval: 120s

But ESPHome gives error: Platform not found: sensor.am43
Does anyone know how to solve?
At the same time the am43_cover and ble_client components is connected and works great.

I got the same problem.
But other than that it works great! Much easier than the original way of this repo.
Thanks man!

Hi,
same issue for me.
esphome requires Python '>=3.7,<4.0' but the running Python is 3.6.9
System: Home Assistant OS 5.13  (aarch64 / raspberrypi4-64)
Home Assistant Core: 2021.5.0b6
Home Assistant Supervisor: 2021.04.3
Thanks

Is there a write up on how to use / setup the AM43 on a ESP32 using ESPHOME as my version of ESPHOME 1.17 (updated today) does recognize AM43 as a platform?
Regards Dave

Cool! It looks like https://github.com/esphome/esphome/pull/1177 is merged.
What is now the easiest way to use your library? Vanilla esphome with this code in a local directory?

When does it come in official ESPhome release? Is it stable now - I tried it before, with two blind engines - only one did I get to work - they did not work together:(

Спасибо за слепой контроллер,
использую ваш конфиг:
sensor:
  - platform: am43
    ble_client_id: art_left
    battery_level:
      name: "Kitchen_left_blind battery"
    illuminance:
      name: "Kitchen_left_blind light"
    update_interval: 120s

Но ESPHome выдает ошибку: Платформа не найдена: sensor.am43
Кто-нибудь знает как решить?
При этом компоненты am43_cover и ble_client подключены и отлично работают.

I also have an error
Platform not found: cover.am43_cover
Platform not found:  sensor.am43

Hi,
I saw that the pr was merged.
What is the working configuration?
Thx

ESPHome
Current version: 1.19.1
The blinds never worked

This is my config:
esp32_ble_tracker:

ble_client:
  - mac_address: 02:4D:FB:F0:2B:2E
    id: art_left
  - mac_address: 02:69:32:F0:A5:1D
    id: art_right

cover:
  - platform: am43_cover
    name: "ArtLeft Cover"
    id: art_left_cover
    ble_client_id: art_left
  - platform: am43_cover
    name: "ArtRight Cover"
    id: art_right_cover
    ble_client_id: art_right

sensor:
  - platform: am43
    ble_client_id: art_left
    battery_level:
      name: "AM43 Left battery"
    illuminance:
      name: "AM43 Left light"
    update_interval: 120s
  - platform: am43
    ble_client_id: art_right
    battery_level:
      name: "AM43 Right battery"
    illuminance:
      name: "AM43 Right light"
    update_interval: 120s
  - platform: uptime
    name: Uptime Sensor

switch:
  - platform: restart
    name: "Device Restart"
  - platform: ble_client
    name: "Left AM43 On"
    ble_client_id: art_left
  - platform: ble_client
    name: "Right AM43 On"
    ble_client_id: art_right
    id: right_on

I was able to overcome this problem, in this configuration I replaced

platform: am43_cover
on the
platform: am43
the error disappeared and I was able to compile
Also important in -> Supervisor -> ESPHome -> Configuration ->
esphome_version - buxtronix:am43
certfile - fullchain.pem
keyfile - privkey.pem

Glory buxtronix!

another way to use External Components
https://esphome.io/components/external_components.html
add to your config:
external_components:
  - source:
      type: git
      url: https://github.com/buxtronix/esphome.git
      ref: am43
    components: [ am43 ]

When using external components as mentioned I get the following?
AM I doing something wrong?
INFO Reading configuration /config/esphome/blinds.yaml...
INFO Generating C++ source...
INFO Compiling app...
INFO Running:  platformio run -d /config/esphome/blinds
Processing blinds (board: esp32dev; framework: arduino; platform<EMAIL_ADDRESS>--------------------------------------------------------------------------------
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
PACKAGES:
 - framework-arduinoespressif32 3.10006.210326 (1.0.6)
 - tool-esptoolpy 1.30000.201119 (3.0.0)
 - toolchain-xtensa32 2.50200.97 (5.2.0)
Library Manager: Installing Hash
Library Manager: Already installed, built-in library
Dependency Graph
|-- <AsyncTCP-esphome> 1.2.2
|-- <FS> 1.0
|-- <ESPAsyncWebServer-esphome> 1.2.7
|   |-- <AsyncTCP-esphome> 1.2.2
|   |-- <FS> 1.0
|   |-- <WiFi> 1.0
|-- <ESPmDNS> 1.0
|   |-- <WiFi> 1.0
|-- <DNSServer> 1.1.0
|   |-- <WiFi> 1.0
|-- <Update> 1.0
|-- <WiFi> 1.0
Compiling /data/blinds/.pioenvs/blinds/src/main.cpp.o
Compiling /data/blinds/.pioenvs/blinds/lib789/WiFi/WiFiClient.cpp.o
Compiling /data/blinds/.pioenvs/blinds/lib789/WiFi/WiFiGeneric.cpp.o
src/main.cpp: In function 'void setup()':
src/main.cpp:205:30: error: invalid new-expression of abstract class type 'esphome::am43::Am43'
   am43_am43 = new am43::Am43();
                              ^
In file included from src/esphome.h:2:0,
                 from src/main.cpp:3:
src/esphome/components/am43/am43.h:18:7: note:   because the following virtual functions are pure within 'esphome::am43::Am43':
 class Am43 : public esphome::ble_client::BLEClientNode, public PollingComponent {
       ^
In file included from src/esphome/components/am43/am43.h:4:0,
                 from src/esphome.h:2,
                 from src/main.cpp:3:
src/esphome/components/ble_client/ble_client.h:28:16: note: 	virtual void esphome::ble_client::BLEClientNode::loop()
   virtual void loop() = 0;
                ^
src/main.cpp:240:32: error: invalid new-expression of abstract class type 'esphome::am43::Am43'
   am43_am43_2 = new am43::Am43();
                                ^
*** [/data/blinds/.pioenvs/blinds/src/main.cpp.o] Error 1
========================= [FAILED] Took 13.52 seconds =========================

I have converted this into a repo supporting the new External Components feature of ESPHome. Please see https://github.com/buxtronix/esphome-am43

Should anyone encounter Platform not found:  sensor.am43 like I have been battling, check your /config/esphome/.esphome/external_components/ for folders referencing am43 and remove the entire folder(s).  Then restart the ESPHome add-on.  I just did that and now my am43-controller.yam,l is fine.


Delegate another admin?

What is the process to another/2nd admin for the platform?  Thanks

Hi @zlcochran,
Thanks for using the app.
To add an admin to the platform you can add the person directly to "Members List" with email id and setting below fields:

If the user is already added to the platform as a champion just modify these 2 fields for that entry instead of adding another record.
Thanks.

Thank you. That worked!


Audit events via gRPC endpoint

Is there an existing issue for this?

[X] I have searched the existing issues

Feature description
We (UW) previously contributed Kafka sink for audit logs & have noted that others have requested similar functionality, such as for Nats. Since contributing internally we've been discussing whether Kafka is something we want to use, we may want to move to a more managed offering in AWS or GCP, such as Kinesis.
As I understand it, you (Cerbos) taking these contributions on means you take on the maintenance & support, which you were comfortable at the time with Kafka; what if there was a way to configure a gRPC endpoint for these audit events to be sent to instead for other transports such as Nats that you may not be comfortable adopting?
Much like the OpenTelemetry Collector either Cerbos or the community can offer a gRPC service configured with all the different transports to proxy these events to.
What would the ideal solution look like to you?
gRPC contract:

Accepts access & decision audit entries
Maintained as a public contract under api/public
Audit backend

optional:

Separate repo under the Cerbos organisation for the audit collector

Anything else?
No response

Hey! It's a very nice idea. We've always wanted to offload audit storage to specialised systems such as SIEMs but there wasn't (and, AFAIK, still isn't) a common standard for that. While it would be cool to define our own audit collection API, I am a little bit concerned about how adoptable it would be because it requires our users to write their own collectors and deploy them -- which would be an additional burden for most of them. Writing a good API+collector that can keep track of and persist a large volume of audit events without losing them and ensuring they are not tampered with in transit etc. is not trivial either. At this stage I am not confident that we have the resources to manage that well.
We've always wanted to make Cerbos extensible and allow third-party plugins to add additional functionality. We just haven't gotten around to making that easy and seamless yet. I feel like prioritising that would help address this issue as well because then advanced users such as UW could develop the audit sinks they need.
While we work on that though, is there an intermediate solution using existing functionality that can help address this? My initial thoughts were to use Kafka as an intermediary: Cerbos writes audit logs to Kafka and your log proxy reads from Kafka and writes to whatever other preferred destination you have. However, the downside is that you still need to run Kafka for that to work.
The other option I could think of is using a tool like Vector to read audit logs from Cerbos (using the file audit backend) and distribute to a sink of your choice.

@charithe Thank you for the great response 🙇🏻
I don't have any initial needs, I was raising the idea in order to see whether it was a viable solution to help Nats as well other transports in the future progress by moving it outside of the main Cerbos repo & therefore the concerns around the maintenance burden.
The idea wasn't to persist or really transform anything from the standard protobuf/JSON you have today, rather it would just proxy to a backend (kafka/nats/pubsub) & apply any necessary back-pressure if that failed. While they would be forced to deploy another service, after the gRPC contract was in place I'd hope we could offer an out-of-the-box service that just requires configuration.
A big part of the experience we've enjoyed from Cerbos has been how simple it is to run, so I understand making that more complex could be a problem 👍🏻

Personally, I quite like the idea. Besides the usability aspect of requiring users to deploy a separate service, I think it's a good way to address this. However, because we are talking about audit events here, I think that a pull API might not be acceptable to some users.
Consider for example what happens when a Cerbos instance is shutting down. What should happen to the unscraped audit events? Should Cerbos persist them somewhere and remember to serve from that point the next time (if ever) it starts up? In an environment like Kubernetes where pods can come and go anytime, that store would have to be somewhere central and it would need to keep track of unscraped events so that they can be published to the final sink by some other (batch?) mechanism. Similarly, what should happen if the scraper stops working? How long should each Cerbos instance hold on to the audit events before discarding them or writing them off to an intermediate store? How would the scraper resume and ingest those events?
This is why I feel that a push API might be more appropriate for this use case. Of course, that would have its own set of issues but I think the implementation would be less complex compared to the pull version.

I'm in agreement; I obviously didn't make it clear that the gRPC contract I was proposing was for a client calling from Cerbos out to an external service, i.e. push based.
The optional, proxy part I then mention is if the Cerbos project provides a standalone service implementing this gRPC contract for common backends.
Like the Kafka implementation in Cerbos today, the implementation could either run sync (fixed sized buffer that blocks when full), or async (FIFO that evicts when overflowing fixed size).


Clarify how regex comparison operators and full text search can be used on the same field

Summary of the changes / Why this improves CrateDB
Closes: #14572
Checklist

[X] CLA is signed
[X] This does not contain breaking changes

retest this please


Search SDK: Migrated Search.Management.Tests from Hyak to AutoRest

Note: The Search "Management SDK" is only partially implemented and doesn't ship to Nuget yet, which is why it doesn't have its own project. It exists only to be used by Search.Tests to provision search services and manage keys. This PR is kind of a dry run to try out AutoRest before migrating the actual Search SDK and its tests.
Changes:

Added generate.cmd to generate the code using AutoRest. This resulted in many source files instead of just SearchManagementTests.cs
Modified tests accordingly. Removed unnecessary undo handlers.
Temporarily removed Search and Search.Tests from the .sln since they aren't migrated yet, but they depend on Search.Management.Tests.
Recorded new session files for mocked test runs.
Disabled parallel test execution for xunit.net 2.0. I also had to install VS 2013 update 5 to get xunit test discovery to work.

Hi @brjohnstmsft, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

It looks like you're working at Microsoft (brjohnst). If you're full-time, we DON'T require a contribution license agreement.

If you are a vendor, or work for Microsoft Open Technologies, DO please sign the electronic contribution license agreement. It will take 2 minutes and there's no faxing! https://cla.azure.com.

TTYL, AZPRBOT;


Scraping maps data

Map pickles added for analytics, please use second laps ("xxxx-laps-1-xxx")

a clean up of unneeded files may be recomended

will cherry pick the last commit


Vg test branch

Add 2 test out of 3 for is_package method.
Add non package dir in in tests
Add simple script in non package dir

you used an old branch as your base
you haven't pushed any of your commits


Who is going to take responsibility for this site

Who are the site owners and what kind of FUD are they going to be posting to the detriment of the community?

The site is going to be a communication channel for release information, blogs about work in progress, etc, for the Bitcoin Core project. Better communication from developers is often requested.

Who are the site owners?

The About page does not indicate any particular individual owners. Are we to understand that all core devs will take responsibility for this site? Are you unwilling to answer this question? If not, why not?

The content policy of the website can be seen here.

The owners of the site are Theymos and his alter ego, Cobra.

The ideal answer to the question "Who is going to take responsibility for this site" would be "the open source community around bitcoin core". That's the plan, of course, like anything around bitcoin, this is an experiment. But at least we're trying something instead of cynically posting to other projects' trackers.

As if the bitcoin community hasn't had cause for cynicism lately.


llama-agents :: text to sql :: infinite loop in terminal

Hello ,
I'm trying to use llama-agents to solve text-to-sql problem for my work.
I'm using azure openai's gpt-4o model with llama-index
I've created tools , each of these tool's function return string output :
table_desc_tool = FunctionTool.from_defaults(
    fn=get_all_table_desc, # no arguments
    name="get_all_table_desc",
    description="tool used to fetch descriptions of all tables present in schema"
)

column_desc_tool = FunctionTool.from_defaults(
    fn = get_column_desc, # one argument i,e table name
    name = "get_column_desc",
    description="tool used to fetch description for all the columns for a specified table"
)

sql_execute_tool = FunctionTool.from_defaults(
    fn = execute_generated_sql, # 1 argument , generated SQL
    name = "execute_generated_sql",
    description="tool used to execute generated SQL. Note: parameter must be a valid SQL select query"
)

defined following agents :
table_agent = FunctionCallingAgentWorker.from_tools(
    tools = [table_desc_tool],llm=omni_llm
).as_agent()
column_agent = FunctionCallingAgentWorker.from_tools(
    tools = [column_desc_tool] , llm = omni_llm
).as_agent()

execute_agent = FunctionCallingAgentWorker.from_tools(
    tools=[sql_execute_tool],llm= omni_llm
).as_agent()

master_agent = FunctionCallingAgentWorker.from_tools(llm=omni_llm).as_agent()

Created following services:
orchestrator_llm = get_llm("gpt-4o")

message_queue = SimpleMessageQueue(port = 8001)

control_plane = ControlPlaneServer(
    message_queue=message_queue,
    orchestrator=AgentOrchestrator(llm = orchestrator_llm,),
    port = 8002
)

table_agent_service = AgentService(
    agent = table_agent,
    message_queue=message_queue,
    description="Service to find all the tables and their description present in database",
    service_name="table_desc_service",
    port = 8003
)

column_agent_service = AgentService(
    agent = column_agent,
    message_queue=message_queue,
    description = "Service to find all the columns and it's descripition for a given table",
    service_name = "column_desc_service",
    port = 8004
)

execute_agent_service = AgentService(
    agent = execute_agent,
    message_queue=message_queue,
    description = "Useful to execute generated SQL and fetch data",
    service_name="sql_execute_service",
    port = 8005
)

generation_agent_service = AgentService(
    agent = master_agent,
    message_queue=message_queue,
    description = "Service used to generate SQL for a business question",
    service_name = "sql_generation_service",
    port = 8006
)

launched services using :
import nest_asyncio

nest_asyncio.apply()

launcher = LocalLauncher(
    services=[table_agent_service,column_agent_service,
              execute_agent_service,generation_agent_service],
    control_plane=control_plane,
    message_queue=message_queue,
)

result = launcher.launch_single("find total spend in 2023")

print(f"Result = {result}")

The code launches, terminal shows services are registered :
INFO:llama_agents.message_queues.simple - Consumer AgentService-e032bf0d-876f-49d4-8d50-6f8671430b03: table_desc_service has been registered.
INFO:llama_agents.message_queues.simple - Consumer AgentService-ef047209-244f-48c8-bcd6-28bd22536832: column_desc_service has been registered.
INFO:llama_agents.message_queues.simple - Consumer AgentService-07922165-5a23-4368-a5bb-ca97089c71fa: sql_execute_service has been registered.
INFO:llama_agents.message_queues.simple - Consumer AgentService-cb71e6fa-2641-496e-b3c9-abd17148a5a2: sql_generation_service has been registered.
INFO:llama_agents.message_queues.simple - Consumer d07aba13-ba14-465f-a8cd-0fe7bf1fc7eb: human has been registered.
INFO:llama_agents.message_queues.simple - Consumer ControlPlaneServer-5656c29a-304d-4f93-9297-390f4eba99f3: control_plane has been registered.
INFO:llama_agents.services.agent - table_desc_service launch_local
INFO:llama_agents.services.agent - column_desc_service launch_local
INFO:llama_agents.services.agent - sql_execute_service launch_local
INFO:llama_agents.services.agent - sql_generation_service launch_local
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Launching message queue locally
INFO:llama_agents.services.agent - Processing initiated.
INFO:llama_agents.services.agent - Processing initiated.
INFO:llama_agents.services.agent - Processing initiated.
INFO:llama_agents.services.agent - Processing initiated.

But beyond that it goes into loop and does not work as expected.
I was expecting following order:

call table description agent , this returns 3 descriptions ,
call column_description agent using one table name from above 3 tables and get all columns
generation agent should use all this and generate a query
execution agent should execute the query against database

As per below logs I can only see table_desc_service getting called multiple times.
No other logs or agents are getting called , any way to debug this ?
The logs keep on going into infinite and does not end.
Also I don't get where the model is being used.
No references / examples helped me out.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Launching message queue locally
INFO:llama_agents.services.agent - Processing initiated.
INFO:llama_agents.services.agent - Processing initiated.
INFO:llama_agents.services.agent - Processing initiated.
INFO:llama_agents.services.agent - Processing initiated.
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
table descriptions fetched :  3
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
`table descriptions fetched :  3` # this is log from one of the tool which fetches table description
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'sql_generation_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'sql_generation_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
table descriptions fetched :  3
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'sql_generation_service' with action 'ActionTypes.NEW_TASK'

Any help to proceed or debug is appreciated.
I would also like to ask if this use case is valid for llama-agents ?
I'm trying to validate other frameworks such as crewai and super

Imo I would build this with a workflow in llama-index, rather than llama-agents.
Working on pivoting llama-agents to be the place to deploy and scale workflows built in llama index

@logan-markewich , What is a workflow ? Do you mean create multiple-operational agents using core lib and performing orchestration by myself ?
I'm interested how the orchestration is being done by llama-agents . Why use a message queue like kafka / rabbitmq and how this queue is being used.

Workflows: https://docs.llamaindex.ai/en/stable/module_guides/workflow/
Yea I mean writing workflows to orchestrate between agents yourself. It will be much more clear when things go wrong and how to debug them imo.
Message queues are used to pass messages between the control plane and services. Basically services are like job workers than consume jobs from the message queue.
The refactor is pivoting llama-agents to basically deploy workflows as services themselves, and help you scale that way. Its going to be very cool :)

thank you for the explanation @logan-markewich.
Before I close this issue , could you point me towards a complex workflow example with multiple agents / actors ?
And keep up the great work!


Update repositories.txt

Hi @hafidhh. I see this pull request removes a library URL from the list. Please tell me what was the reason for doing that so I can be sure to handle the request correctly.

I want to change library name from Callmebot ESP8266 Library to Callmebot ESP8266

OK, then this is not the right approach. Please follow the instructions here to request the name change:
https://github.com/arduino/library-registry/blob/main/FAQ.md#how-can-i-change-a-librarys-name


Nargo WASM compilation failure --- macOS 12.6 Monterey

Description
Following instructions on https://noir-lang.github.io/book/getting_started/nargo/installation.html with Option 1: WASM Executable Backend.
Aim
Install Nargo.
Expected behavior
Compile to succeed.
Bug
For option 1: I set
aztec_backend = { optional = true, git = "https://github.com/noir-lang/aztec_backend", features = ["wasm-base"] , default-features = false }

as described in the documentation.
Running
cargo install --locked --path=.

error[E0599]: no method named `uint8view` found for reference `&wasmer::Memory` in the current scope
  --> /Users/michaelneuder/.cargo/git/checkouts/aztec_backend-a697fb631cbad807/8dc5b28/src/barretenberg_wasm/mod.rs:51:43
   |
51 |             for (byte_id, cell) in memory.uint8view()[offset..(offset + arr.len())]
   |                                           ^^^^^^^^^ method not found in `&wasmer::Memory`

error[E0061]: this function takes 1 argument but 0 arguments were supplied
  --> /Users/michaelneuder/.cargo/git/checkouts/aztec_backend-a697fb631cbad807/8dc5b28/src/barretenberg_wasm/mod.rs:66:23
   |
66 |         return memory.view()[start as usize..end]
   |                       ^^^^-- an argument of type `&_` is missing
   |
note: associated function defined here
  --> /Users/michaelneuder/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmer-3.0.0/src/sys/externals/memory.rs:88:12
   |
.
.
.
   --> /Users/michaelneuder/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmer-3.0.0/src/sys/instance.rs:113:12
    |
113 |     pub fn new(
    |            ^^^
help: provide the argument
    |
217 |     (Instance::new(/* value */, &module, &res_import).unwrap(), memory)
    |      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Some errors have detailed explanations: E0061, E0308, E0599, E0608.
For more information about an error, try `rustc --explain E0061`.
error: could not compile `aztec_backend` due to 20 previous errors

To reproduce
(Describe the steps to reproduce the behavior.)

git clone git@github.com:noir-lang/noir.git
cd noir/crates/nargo
$ cargo install --locked --path=.

Environment

OS: macOS 12.6

Versions:
rustc --version
rustc 1.64.0 (a55dd71d5 2022-09-19)

cmake --version
cmake version 3.25.0

clang --version
Apple clang version 14.0.0 (clang-14<IP_ADDRESS>)

For nargo users

noir-lang/noir commit cloned:
Proving backend

[ ] default

Clang: (run clang --version)

[x] wasm-base

Note that I also tried https://noir-lang.github.io/book/getting_started/nargo/installation.html#option-2-compile-backend-from-source and changed
aztec_backend = { optional = true, git = "https://github.com/noir-lang/aztec_backend", rev = "01b922adcb5a9d70b2d12304e1cb7487d9f28188" }

Running
bash -c "RUST_BACKTRACE=FULL cargo install --locked --path=."

  2 errors generated.
  make[2]: *** [src/aztec/rollup/proofs/CMakeFiles/rollup_proofs_objects.dir/account/c_bind.cpp.o] Error 1
  make[1]: *** [src/aztec/rollup/proofs/CMakeFiles/rollup_proofs_objects.dir/all] Error 2
  2 errors generated.
  make[2]: *** [src/aztec/rollup/proofs/CMakeFiles/rollup_proofs_test_objects.dir/account/account_tx.test.cpp.o] Error 1
  2 errors generated.
  make[2]: *** [src/aztec/stdlib/merkle_tree/CMakeFiles/stdlib_merkle_tree_test_objects.dir/memory_tree.test.cpp.o] Error 1
  make[1]: *** [src/aztec/stdlib/merkle_tree/CMakeFiles/stdlib_merkle_tree_test_objects.dir/all] Error 2
  2 errors generated.
  2 errors generated.
  make[2]: *** [src/aztec/rollup/proofs/CMakeFiles/rollup_proofs_test_objects.dir/escape_hatch/escape_hatch_tx.test.cpp.o] Error 1
  make[2]: *** [src/aztec/rollup/proofs/CMakeFiles/rollup_proofs_test_objects.dir/escape_hatch/escape_hatch.test.cpp.o] Error 1
  make[1]: *** [src/aztec/rollup/proofs/CMakeFiles/rollup_proofs_test_objects.dir/all] Error 2
  make: *** [all] Error 2
  thread 'main' panicked at '
  command did not execute successfully, got: exit status: 2

  build script failed, must exit now', /Users/michaelneuder/.cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.49/src/lib.rs:1104:5
  stack backtrace:
     0: rust_begin_unwind
               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/std/src/panicking.rs:584:5
     1: core::panicking::panic_fmt
               at /rustc/a55dd71d5fb0ec5a6a3a9e8c27b2127ba491ce52/library/core/src/panicking.rs:142:14
     2: cmake::fail
     3: cmake::run
     4: cmake::Config::build
     5: build_script_build::main
     6: core::ops::function::FnOnce::call_once
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
error: failed to compile `nargo v0.1.0 (/Users/michaelneuder/github/noir/crates/nargo)`, intermediate artifacts can be found at `/Users/michaelneuder/github/noir/target

So it seems like neither installation method is working for my machine

@michaelneuder Thanks for the report, build is currently broken
See this pre-built binary nargo-x86_64-apple-darwin.

thanks, this is working!


[Candidate 1v1 Drallim Desert v2] Rocks around UN crate look repetitive

Issue Description
Rocks around UN crate look repetitive.
Expected behavior
Rock formation looks unique.
Screenshots

do you want me to turn the rocks?

Different rotations on rocks would potentially fix the issue.

Cool I sent you new version in discord with the fix
Sent from my Galaxy
-------- Original message --------
From: xezon @.>
Date: 05/04/2021 21:46 (GMT+00:00)
To: xezon/GeneralsRankedMaps @.>
Cc: Millard9 @.>, Comment @.>
Subject: Re: [xezon/GeneralsRankedMaps] [Candidate 1v1 Drallim Desert v2] Rocks around UN crate look repetitive (#142)
Different rotations on rocks would potentially fix the issue.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/xezon/GeneralsRankedMaps/issues/142#issuecomment-813639326, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ATM5LCKFZY7HZHECEQUPIUTTHIOQNANCNFSM42NEPPSQ.

Fixed with https://github.com/xezon/GeneralsRankedMaps/commit/1130c5bf8ac2ad7b28c8ff25a070f55392a0e673


can not download the dataset

bash download_data.sh gl3d_imgs 0 125 and the error is as below
--2020-07-07 21:04:36--  http://research.altizure.com/data/gl3d_imgs/gl3d_imgs.tar.000
Resolving research.altizure.com (research.altizure.com)... <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, ...
Connecting to research.altizure.com (research.altizure.com)|<IP_ADDRESS>|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://research.altizure.com/data/gl3d_imgs/gl3d_imgs.tar.000 [following]
--2020-07-07 21:04:37--  https://research.altizure.com/data/gl3d_imgs/gl3d_imgs.tar.000
Connecting to research.altizure.com (research.altizure.com)|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2020-07-07 21:04:37 ERROR 403: Forbidden.
I copy the link http://research.altizure.com/data/gl3d_imgs/gl3d_imgs.tar.000 to chrome , the error is as below
<Error>
AccessDenied
<Message>Access Denied</Message>
<RequestId>5EF0B01838FCCBDE</RequestId>
<HostId>
GuVoU7z/ICFJN00nsCA4Prn08kR+8Nl8KC9Jzeeceqg8+bOr0cnxpxJ4qLzOpPMEgYWnkMTgHGs=
</HostId>
</Error>

I can download the data when I move to branch v2.
The link has become https://research.altizure.com/data/gl3d_v2/gl3d_imgs/gl3d_imgs.tar.000

@Bohrhh Do you still have a copy of the GL3D dataset? I'm looking specially for the undist_images folders. If you, please get in contact with me. Thank you so much in advance.


Something happened on Test

The dl link for test is not working
Kindly check this
https://github.com/RaSan147/pixi-live2d-display/actions/runs/8630374063/job/23656499749#step:9:30

Google recently changed the Chrome driver's download URL, to fix it you need to upgrade some dependencies like in this commit: https://github.com/guansss/pixi-live2d-display/pull/122/commits/6f3cd68a5e8354f259e0eb8ba98de540bd0b017c

Thanks, will test and close issue asap


Bug 2134930: rebase: ParseAcceptLanguage takes a long time to parse complex tags

A vulnerability was found in golang.org/x/text/language package which could cause a denial of service. An attacker can craft an Accept-Language header which ParseAcceptLanguage will take significant time to parse.
Version v0.3.8 of golang.org/x/text fixes a vulnerability.
See-also: https://go.dev/issue/56152
See-also: https://bugzilla.redhat.com/CVE-2022-32149

Show available bot commands
These commands are normally not required, but in case of issues, leave any of
the following bot commands in an otherwise empty comment in this PR:

/retest ci/centos/<job-name>: retest the <job-name> after unrelated
failure (please report the failure too!)
/retest all: run this in case the CentOS CI failed to start/report any test
progress or results

/bugzilla refresh

/bugzilla refresh

/lgtm

/lgtm


Bug:  Config\App cookie settings depreciated but Config\Cookie not implemented in CSRF

PHP Version
8.1
CodeIgniter4 Version
4.1.6 - develop
CodeIgniter4 Installation Method
Manual (zip or tar.gz)
Which operating systems have you tested for this bug?
Windows
Which server did you use?
apache
Database
No response
What happened?
In Config\App for cookie settings is written that those settings are deprecated in favor of Config\Cookie but Config\Cookie is not reflected. For example, the CSRF constructor accepts only App Config instead of Config\Cookie
Steps to Reproduce

Clean installation
Removing cookie config from Config\App in favor of Config\Cookie
Enabling CSRF
Make request

Expected Output
CSRF should take Cookie configuration from Config\Cookie instead of Config\App
Anything else?
No response

I believe this is already taken care of:
https://github.com/codeigniter4/CodeIgniter4/blob/357021472f3227198c809e2439bea97486b0d830/system/Security/Security.php#L212-L226

@xbotkaj You can't remove the deprecated Config\App cookie settings.
If you remove it, you get the error like this:
$ php spark serve
PHP Warning:  Undefined property: Config\App::$cookiePath in /Users/kenji/work/codeigniter/CodeIgniter4/system/HTTP/Response.php on line 163
PHP Stack trace:
PHP   1. {main}() /Users/kenji/work/codeigniter/CodeIgniter4/spark:0
PHP   2. require() /Users/kenji/work/codeigniter/CodeIgniter4/spark:45
PHP   3. CodeIgniter\CodeIgniter->initialize() /Users/kenji/work/codeigniter/CodeIgniter4/system/bootstrap.php:146
PHP   4. CodeIgniter\Config\BaseService::exceptions() /Users/kenji/work/codeigniter/CodeIgniter4/system/CodeIgniter.php:173
PHP   5. CodeIgniter\Config\BaseService::__callStatic($name = 'exceptions', $arguments = []) /Users/kenji/work/codeigniter/CodeIgniter4/system/CodeIgniter.php:173
PHP   6. CodeIgniter\Config\Services::exceptions($config = *uninitialized*, $request = *uninitialized*, $response = *uninitialized*, $getShared = *uninitialized*) /Users/kenji/work/codeigniter/CodeIgniter4/system/Config/BaseService.php:248
PHP   7. CodeIgniter\Config\BaseService::getSharedInstance($key = 'exceptions', ...$params = variadic(NULL, NULL, NULL)) /Users/kenji/work/codeigniter/CodeIgniter4/system/Config/Services.php:234
PHP   8. CodeIgniter\Config\BaseService::exceptions(NULL, NULL, NULL, FALSE) /Users/kenji/work/codeigniter/CodeIgniter4/system/Config/BaseService.php:189
PHP   9. CodeIgniter\Config\BaseService::__callStatic($name = 'exceptions', $arguments = [0 => NULL, 1 => NULL, 2 => NULL, 3 => FALSE]) /Users/kenji/work/codeigniter/CodeIgniter4/system/Config/BaseService.php:189
PHP  10. CodeIgniter\Config\Services::exceptions($config = NULL, $request = NULL, $response = NULL, $getShared = FALSE) /Users/kenji/work/codeigniter/CodeIgniter4/system/Config/BaseService.php:248
PHP  11. CodeIgniter\Config\BaseService::response() /Users/kenji/work/codeigniter/CodeIgniter4/system/Config/Services.php:239
PHP  12. CodeIgniter\Config\BaseService::__callStatic($name = 'response', $arguments = []) /Users/kenji/work/codeigniter/CodeIgniter4/system/Config/Services.php:239
PHP  13. CodeIgniter\Config\Services::response($config = *uninitialized*, $getShared = *uninitialized*) /Users/kenji/work/codeigniter/CodeIgniter4/system/Config/BaseService.php:248
PHP  14. CodeIgniter\Config\BaseService::getSharedInstance($key = 'response', ...$params = variadic(NULL)) /Users/kenji/work/codeigniter/CodeIgniter4/system/Config/Services.php:479
PHP  15. CodeIgniter\Config\BaseService::response(NULL, FALSE) /Users/kenji/work/codeigniter/CodeIgniter4/system/Config/BaseService.php:189
PHP  16. CodeIgniter\Config\BaseService::__callStatic($name = 'response', $arguments = [0 => NULL, 1 => FALSE]) /Users/kenji/work/codeigniter/CodeIgniter4/system/Config/BaseService.php:189
PHP  17. CodeIgniter\Config\Services::response($config = NULL, $getShared = FALSE) /Users/kenji/work/codeigniter/CodeIgniter4/system/Config/BaseService.php:248
PHP  18. CodeIgniter\HTTP\Response->__construct($config = class Config\App { public $baseURL = 'http://localhost:8080/'; public $indexPage = 'index.php'; public $uriProtocol = 'REQUEST_URI'; public $defaultLocale = 'en'; public $negotiateLocale = FALSE; public $supportedLocales = [0 => 'en']; public $appTimezone = 'America/Chicago'; public $charset = 'UTF-8'; public $forceGlobalSecureRequests = FALSE; public $sessionDriver = 'CodeIgniter\\Session\\Handlers\\DatabaseHandler'; public $sessionCookieName = 'ci_session'; public $sessionExpiration = '36000'; public $sessionSavePath = 'ci_sessions'; public $sessionMatchIP = FALSE; public $sessionTimeToUpdate = 300; public $sessionRegenerateDestroy = FALSE; public $cookiePrefix = ''; public $cookieDomain = ''; public $cookieSecure = FALSE; public $cookieHTTPOnly = TRUE; public $cookieSameSite = 'Lax'; public $proxyIPs = ''; public $CSRFTokenName = 'csrf_test_name'; public $CSRFHeaderName = 'X-CSRF-TOKEN'; public $CSRFCookieName = 'csrf_cookie_name'; public $CSRFExpire = 7200; public $CSRFRegenerate = TRUE; public $CSRFRedirect = TRUE; public $CSRFSameSite = 'Lax'; public $CSPEnabled = FALSE }) /Users/kenji/work/codeigniter/CodeIgniter4/system/Config/Services.php:484

@kenjis exactly that is the issue. If it is depreciated, it means it should not be considered as necessary to run the application and it should not be the first choice to take settings from, instead, the primary source should be Config\Cookie, if it is not found there, then go to Config\App for back compatibility.

@xbotkaj Maybe you are correct. But if you remove the properties, you get the error.
I think deprecated means not recommended and it will be deleted in the future.
But not deleted now.

It is kinda weird when you need to make setup the two places for the same thing to work properly.

You don't have to make setup the two places. Only Config\Cookie is enough.
If you setup Config\Cookie, the values take effect. The values in Config\App is ignored.
If not, it is a bug. Report it, please.
Why do you want to delete the properties in Config\App?

@kenjis I was just playing around with config and realized some settings were not taken from Config\Cookie but instead from Config\App so I tried to delete it to see what will happen and where it will break. It was CSRF for me where it threw an error.
I can check later what exactly is taken from App and what from Cookie. :) For now, I have the same setup on both places.

@xbotkaj This is not related, but if you use session, I recommend you use Session based CSRF Protection.
Then CSRF cookie is not needed.
https://www.codeigniter.com/user_guide/libraries/security.html#csrf-protection-methods

Thank You @kenjis for me it works now, I wanted just to let you know that this issue occurred, and to be aware of it. :) Like I mentioned before I let there the old config and it works for now.

@xbotkaj Okay, I close this for now.
I was not able to confirm the issue that some settings were not taken from Config\Cookie.
If you will find the exact way to reproduce, feel free to reopen.


Improve documentation regarding mono-repo terraform and terragrunt scripts/config

Hey folks,
i recently discovered terragrunt and what should i say: it has simply gorgeous functionalities. I wished i knew from its existence long ago so all these awkward improvised terraform wrapper script to roll out to different environments/regions would have been avoided. So many thanks for that!
During the read-through of the docs one major question rose: Why not have one repo containing the terragrunt config structures (live repo in your docs) and the terraform scripts (module repo in your docs)? When you have an adequate project structure (terraform and terragrunt folder on the same file system hierarchy level) it seems totally reasonable. Using git clone and specifying a local path as source in the respective terragrunt configs one could even leverage branching possibilities (as far as i know when specifying a git repo as source terragrunt is only able to check out specific tags). Am I missing something here? Are my thoughts legit? Are there advantages/disadvantages i don't see?
I think other people could ask themselves these questions too. If there would be a doc explaining these specific design choices (for example you did so with monorepo vs polyrepo regarding modules: https://github.com/gruntwork-io/terragrunt-infrastructure-modules-example) newer terragrunt users could get up&running faster with better design decisions.
Many thanks and I appreciate your work!

I think it's important to clarify that everything is about trade offs, and we've recommended a set of options based on our opinions about what trade offs to optimize for.
In our case, we were optimizing for stability and understandability over absolute speed of delivery. Immutable tags allow you to keep a lot of things stable, while having separate repos with different version tags allows you to track what version of the infra code is deployed at any given point in time.
Keeping that in mind:

When you push something into the live repo, how to make your CI/CD pipeline run no unnecessary tests? For example, if you changed something in your dev environment, why redeploy everything? Or is it always "just deploy everything, it will make unnecessary stuff but that doesn't harm anyone"?

The canonical way to handle this is to implement scripts that can detect which modules changed, and only invoking the pipeline for those modules. You can leverage some combination of git diff, grep, and find to identify terragrunt modules and act accordingly.

Regarding your terraform module repo, whenever you push something into the master there, i think the CI/CD stuff shall go off and test/deploy your stuff to the dev environment right? But to do so, it would have to check out your live repo and modify respective terragrunt config sources of changed modules right? If that's true and if i'm not missing something, that seems like a lot of effort. Furthermore, when deploying new modules I understood that you have to push your module first, let the CI/CD do unnecessary stuff and afterwards push your live config so the module gets actually deployed, right?

You wouldn't want to do this everytime you merge to master, but rather every time you cut a release. The CI pipeline is different for modules and live config. In particular, getting into the habit of "unit testing" your modules to ensure they work can catch a lot of bugs before you go in and updating a live environment.
Internally at Gruntwork, our CI pipeline for master on the modules repo runs a suite of terratest tests to make sure the modules are working as expected in isolation. Then, when we feel that the modules are in a state that can be deployed to an actual environment, we cut a release tag to indicate that it can be deployed to an existing environment.
At this point, you can decide what to do. For example, you can have a script that automatically bumps the tag and deploys it for dev, and then have renovatebot setup to open PRs to autobump each environment to that module version each time you cut a release. The advantage of this is that you can choose to merge that stage PR after confirming it works for dev. Or you can decide you want more control over releases (e.g., avoiding an infrastructure failure while your app team is running tests on the dev env) and opt for PRs for all the environments.

Now here comes the question: During a deployment, what could be a "good practice" to access the application code repo while having terraform code in a different repo? Clone it via terragrunt with a before_hook or something like that? Or shall, like i did it before, the terraform code and the application code be in the same repo?

I would recommend a process in the application repo to push immutable versioned packages of your serverless application to AWS, and reference those tags in the terraform code. This could be as simple as building the zip file and pushing to s3, to as complex as creating a SAR. This is similar to what you would do for Docker or Packer apps, where the app repo builds a container image / AMI and publishes that.
In this model, terraform/terragrunt doesn't need to touch the application repo during the deploy process, and instead have everything handled in the cloud.
Just to complete the picture, to do CD to a dev environment here, you would have the build process for the app repo clone the infra live repo and commit a change to the tag in the live config that manages the serverless app.

Many thanks for your reply. I think you was a great help! The many parts start assembling into a whole picture and everything is becoming much clearer to me.  I definitely will apply these techniques while using terragrunt :)


bumping master to Swift 5.0

This PR bumps the Swift compiler version to 5.0 for the swift master branch.

Resolves rdar://problem/43523881

@swift-ci please test

Build failed
Swift Test OS X Platform
Git Sha - d60566022781ab6cedda9ae84bb87498c0c197ac

Build failed
Swift Test Linux Platform
Git Sha - d60566022781ab6cedda9ae84bb87498c0c197ac

Looks like I need to also update getEffectiveLanguageVersion, and per @jrose-apple should not change the  crash recovery test...

Ah, here we go: #13843 is what I was looking at back before we confirmed that 4.2 was going to happen. It doesn't answer all the questions about what we should do this time, but it shows what I had to touch then.

@swift-ci please smoke test

@najacque I'd be happy to take this over at this point, since I did the surgery here for the 4.2 version bump.  Just let me know.

@tkremenek, sounds good,  I got side tracked with other stuff, so please go ahead.

Subsumed by https://github.com/apple/swift/pull/20374


RippleCalc:WRN advanceNode: PAST INTERNAL ERROR REVERSE: OFFER NON-POSITIVE: node.saTakerPays=0/XRP node.saTakerGets=0/JPY/rB3gZey7VWHYRqJHLoHDEJXJ2pEPNieKiS

I am getting this issue while running the ripple node. I have to use node for crypto currency exchange so i have to install full node on my server. This is my conf file
[server]
port_rpc_admin_local
port_peer
port_ws_admin_local
#port_ws_public
#ssl_key = /etc/ssl/private/server.key
#ssl_cert = /etc/ssl/certs/server.crt
[port_rpc_admin_local]
port = 5005
ip = <IP_ADDRESS>
admin = <IP_ADDRESS>
protocol = http
[port_peer]
port = 51235
ip = <IP_ADDRESS>
protocol = peer
[port_ws_admin_local]
port = 6006
ip = <IP_ADDRESS>
admin = <IP_ADDRESS>
protocol = ws
#[port_ws_public]
#port = 5005
#ip = <IP_ADDRESS>
#protocol = wss
#-------------------------------------------------------------------------------
[node_size]
medium
[node_db]
type=RocksDB
path=/externaldevice/db/rocksdb
open_files=2000
filter_bits=12
cache_mb=256
file_size_mb=8
file_size_mult=2
online_delete=2000
advisory_delete=0
[database_path]
/externaldevice/db/
[debug_logfile]
/var/log/rippled/debug.log
[sntp_servers]
time.windows.com
time.apple.com
time.nist.gov
pool.ntp.org
[ips]
r.ripple.com 51235
[validators]
n949f75evCHwgyP4fPVgaHqNHxUVN15PsJEZ3B3HnXPcPjcZAoy7    RL1
n9MD5h24qrQqiyBC8aeqqCWvpiBiYQ3jxSr91uiDvmrkyHRdYLUj    RL2
n9L81uNCaPgtUJfaHh89gmdvXKAmSt5Gdsw2g1iPWaPkAHW5Nm4C    RL3
n9KiYM9CgngLvtRCQHZwgC2gjpdaZcCcbt3VboxiNFcKuwFVujzS    RL4
n9LdgEtkmGB9E2h3K4Vp7iGUaKuq23Zr32ehxiU8FWY7xoxbWTSA    RL5
[validation_quorum]
3
[validation_seed]
ssdecohJMDPFuUPDkmG1w4objZyp4
[rpc_startup]
{ "command": "log_level", "severity": "warning" }
[ssl_verify]
1
[fetch_depth]
full
This is output of server_info
{
"id" : 1,
"result" : {
"info" : {
"build_version" : "0.81.0",
"closed_ledger" : {
"age" : 6,
"base_fee_xrp" : 1e-05,
"hash" : "2B180A3916F499D19795666BA1D1A4B6A2E297021C05274D17C03AEAD7573B14",
"reserve_base_xrp" : 20,
"reserve_inc_xrp" : 5,
"seq" : 36263639
},
"complete_ledgers" : "empty",
"hostid" : "ip-172-31-46-9",
"io_latency_ms" : 1,
"last_close" : {
"converge_time_s" : 1.999,
"proposers" : 7
},
"load" : {
"job_types" : [
{
"job_type" : "ledgerRequest",
"per_second" : 5
},
{
"job_type" : "untrustedProposal",
"peak_time" : 1,
"per_second" : 35
},
{
"in_progress" : 1,
"job_type" : "clientCommand"
},
{
"job_type" : "transaction",
"peak_time" : 3,
"per_second" : 17
},
{
"job_type" : "batch",
"peak_time" : 8
},
{
"job_type" : "fetchTxnData",
"peak_time" : 2,
"per_second" : 4
},
{
"job_type" : "writeObjects",
"peak_time" : 2,
"per_second" : 1
},
{
"job_type" : "trustedProposal",
"per_second" : 2
},
{
"job_type" : "peerCommand",
"peak_time" : 1,
"per_second" : 838
},
{
"job_type" : "processTransaction",
"per_second" : 8
},
{
"job_type" : "WriteNode",
"per_second" : 280
}
],
"threads" : 4
},
"load_factor" : 1,
"peers" : 17,
"pubkey_node" : "n9K9YqgJdfJbZGyv52puDQ2ULKUyD2mp5CCtGKvoLMjnGvY3zDbD",
"pubkey_validator" : "n9KHn8NfbBsZV5q8bLfS72XyGqwFt5mgoPbcTV4c6qKiuPTAtXYk",
"published_ledger" : "none",
"server_state" : "proposing",
"state_accounting" : {
"connected" : {
"duration_us" : "113093667",
"transitions" : 1
},
"disconnected" : {
"duration_us" : "1200718",
"transitions" : 1
},
"full" : {
"duration_us" : "97041180",
"transitions" : 1
},
"syncing" : {
"duration_us" : "0",
"transitions" : 0
},
"tracking" : {
"duration_us" : "1",
"transitions" : 1
}
},
"uptime" : 212,
"validation_quorum" : 10,
"validator_list_expires" : "2018-Feb-07 00:00:00"
},
"status" : "success"
}
}

@Amitdangwal007 Thank you for the report. The warning is complaining about an offer on that books that should have already been removed (it has zero input and zero output). The offer will not interfere with other transactions and will be removed by the next successful transaction that encounters the offer. You can safely ignore this warning.
I've made a note that this warning came up. Thanks again for the report.

Thank you alot
I want to know that the node is properly synchronizing with the full node. It is showing server status proposing. How to detect how much synchronizing is done or progress, anything??


test: make circleci setup noninteractive so doesn't hang [skip buildkite]

The Issue

Minor maintenance
The CircleCI arm64 build has been failing waiting for interactive services restart:

How This PR Solves The Issue
Use DEBIAN_FRONTEND=noninteractive

We should be considering stopping circleci usage. We get enough arm64 information via the macOS tests, and circleci costs at least $45/month the way we currently use it. And there are few linux arm64 users.


How to expose Prometheus metrics for Cloudflare workers (serverless/lambda-like)

[x] This only affects the JavaScript OpenTelemetry library
[ ] This may affect other libraries, but I would like to get opinions here first

Hello,
I am trying to expose custom prometheus metrics from a serverless cloudflare worker.
But I don't need the web server from the prometheusExporter. What I need is just a text prometheus-format output so I can expose prometheus metrics after fetching some data out of a graphQL API.
I would expect a method such as registry.metrics() from promjs package, that I could call and expose raw prometheus text metrics from my API. Is there an equivalent with OTel/prometheus exporter?
If no, what is the best way of solving this? Creating my own exporter re-using parts of the PrometheusSerializer?
A pseudo code that I could imagine would be:
import opentelemetry from '@opentelemetry/api';
import { MeterProvider, MetricReader } from '@opentelemetry/sdk-metrics';
import { Resource } from '@opentelemetry/resources';
import { PrometheusExporter } from '@opentelemetry/exporter-prometheus';

export const fetchHandler = async () => {
  const exporter = new RawTextPrometheusExporter();
  const meterProvider = new MeterProvider({})
  const meter = meterProvider.getMeter('default');
  opentelemetry.metrics.setGlobalMeterProvider(meterProvider);
  meterProvider.addMetricReader(exporter);

  const counter = meter.createCounter('foo');
  counter.add(1, { attributeKey: 'attribute-value' });
  const metrics = await exporter.collect();
  return new Response(
    metrics,
    {
      headers: { 'Content-Type': 'text/plain; version=0.0.4; charset=UTF-8' },
    }
  );
}

Using @opentelemetry/exporter-prometheus would cause some issues such as the following in a serverless worker (even with preventServerStart set to true):
✘ [ERROR] Could not resolve "http"
    ../../../node_modules/@opentelemetry/exporter-prometheus/build/src/PrometheusExporter.js:22:23:
      22 │ const http_1 = require("http");
         ╵                        ~~~~~~
  The package "http" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.

Or when using the correct flag (node_compat = true in wrangler.toml)
service core:user:metrics: Uncaught TypeError: globalThis.XMLHttpRequest is not a constructor
  at index.js:25:14590 in a0
  at index.js:25:15005
  at index.js:1:303
  at index.js:34:53893

  at index.js:35:4179
  at index.js:1:303
  at index.js:35:5860
  at index.js:1:335

Looking forward for some solutions or ideas, thanks :)

PrometheusExporter with preventServerStart: true and using getMetricsRequestHandler() is how this is usally done. I've never used cloudlfare workers though so I'm not sure it this helps.
It's odd that it would result in Uncaught TypeError: globalThis.XMLHttpRequest is not a constructor as we don't use that in the Prometheus Exporter AFAIK. Are you sure this is coming from the exporter and not some other package? :thinking:

Hello @pichlermarc 👋
My assumption would be that just having any "standard" HTTP library would cause the worker to crash.
Just adding this line: const exporter = new PrometheusExporter({ preventServerStart: true }); in a simple Hello world example Worker are causing the same errors as highlighted above.
I guess this import is the issue, https://github.com/open-telemetry/opentelemetry-js/blob/main/experimental/packages/opentelemetry-exporter-prometheus/src/PrometheusExporter.ts#L24 but I can't really change it 😅
So, should I create my own exporter re-using parts of the PrometheusSerializer?
Or even open a PR to add a new exporter, that doesn't include a web server in the list of dependencies? Maybe my best shot is just to create mine just for my use-case...

What we do in other places (any push exporters) is we don't import http as it would interfere with the @opentelemetry/instrumentation-http, and then import it dynamically later on once it's needed.
I could see something similar working here, so in your case, when server start is prevented, http is never required, and you'll never run into the problem. If we just type-import IncomingMessage, Server, and ServerResponse, and then dynamically require for createHttpServer then this would be fine. Whatever you pass to the the response handler would still have to be compatible to these IncomingMessage and ServerResponse types though.
I think using the PrometheusSerializer and implementing your own extension of MetricReader is likely the easiest solution to the problem, though. That's one of the reasons why the Serializer is exposed. The amount of code needed to do so would probably also be minimal.
Semi-related: I actually wonder how the prometheus exporter does not interfere with the @opentelemetry/instrumentation-http package, since requiring it would actually import http, and that would cause telemetry not to be exported - maybe it actually does interfere with that instrumentation as well. This is something that would need to be tested, if it does interfere we'd likely have to implement the solution with the dynamic imports anyway - solving two problems for one.

Thanks for your answer!
I went for the option to copy the code (3 files) and strip anything server/http out of it.
So far, it's working great :)
We just lose all the in-memory functionality because of the serverless capabilities but that's totally fine.
Have a nice day


fix(flexn/template): tv with bundle assets error

Fixes #44

@locksten supposedly does not fix the issue (can be seen from the actions)

@pauliusguzas it's due to the "Now Try Me!" button test failing:

I fixed the preceding test not properly returning to the home screen, but for some reason
await FlexnRunner.expectToBeDisplayedById('template-carousels-screen-container');
still fails, even though the carousels screen is displayed, and the same check in an earlier test succeeds.
Any idea what could be causing this @pauliusguzas ?
https://user-images.githubusercontent.com/9592076/190617838-4af9032a-3f2d-4030-888e-d79d7c23da10.mp4

Will investigate it, though atv I see is failing on all tests

@pauliusguzas  I ran yarn e2e-template-androidtv locally with just the await FlexnRunner.pressButtonRight(1); fix and it passed all tests:
https://user-images.githubusercontent.com/9592076/190659873-5bb5ab93-da0c-4e19-8a70-3a781c515980.mp4

@locksten Yeah the issue is with template, drawer does not close automatically and that causes wrong page opening on last test. I can confirm that your fix is valid, basically template is broken, tests themself do not need to be changed. It is expected to fail if the app is broken


Update data source for Promote events

L23 have confirmed the correct data source.
Promote/Partner - Leeds Inspired - Published on the website
https://airtable.com/app40BfxSxQFxA3MC/tblIfvbra1m4vjm3O/viwCfUnuSo761QTOS?blocks=hide
Leeds Inspired events are ALL the promote events. We only want the Promote events that are published on the website = 102 currently.
(We do not have reliable location data for Promote events)
We can only present data from 18th April - we’ll add a manual count of events before then.
ACTION: Ask L23 to send a manual count of Promote events in the first guide up to April.
Metrics:
A count of number of events
Showing number of events per month

Promote pre-April: 61 events were listed in the Part 1 guide so I think use that figure for before the Leeds Inspired work was completed


quark pipe from iron sheet instead of iron and make less

https://github.com/PandaBoy444/Create.run/blob/fa294f42d956d02a4977b3437627496c640a917a/Client/scripts/crafting.zs#L4-L4

#15


Eliminate include/tscpp/api/noncopyable.h

Eliminates noncopyable base class, which had the sole purpose of surpressing default copy/move construct/assign.  Use the C++11 'delete' keywork instead.

[approve ci ubuntu]

Well would be nice if the rules were written down somewhere.  But, to the best of my knowledge, breaking changes are permitted in a major release, with no strict requirement for deprecation.  We have done deprecation first for major changes, but I don't know of any rules as to what is a major change.  As they say, why put off till tomorrow what you can do today.  That is, procrastination bad.  Note that the compress plugin is using noncopyable, even though it does not use anything else in the C++ API.  If we leave it around for another year plus, til release 11, that's more opportunity for it to be used.  Kill one roach now, or a thousand later.

https://assets.gocomics.com/uploads/blogs/blog_image_3688_3808_Every_Charlie_Brown_Football_Gag_Comic_201701271219.jpg


Add folding rule for load feeding extracts.

We have already disabled common uniform elimination because it created
sequences of loads an entire uniform object, then we extract just a
single element.  This caused problems in some drivers, and is just
generally slow because it loads more memory than needed.
However, there are other way to get into this situation, so I've added
a folding rule to write these into a load of the specific element.
This fixes #1547.

My concern here is that if we did have a situation where all the elements of a global object were being used, this just causes a needless increase in code size and possibly a reduction in performance.
For example, what if the index in foo (in #1547) was "i" instead of "1"? Now this looks more like a de-optimization instead of an optimization, by doing a separate memory operation for each element.
It feels like #1547 should be fixed with array copy propagation, by running it very early, like right after inlining.

@greg-lunarg The problem with array copy propagation is that is relies of all of the other optimizations to make the copy of one array into another obvious.  If we do it earlier, it will just miss the opportunity.
If the element index was "i" then local-access-chain-convert would not have turn what was originally an OpAccessChain and OpLoad into an OpLoad and OpCompositeExtract.
Also, these large loads do not come from the source code.  They are an artifact of local-access-chain-convert.  We also do not know the capabilities of the hardware.  If the user writes these loads as separate loads (except for when they do a wholesale array copy), should we be the ones to merge the loads (as local-access-chain-convert does) or leave them to the driver to determine what is best for the specific hardware?  This is another example of the same problem in https://github.com/KhronosGroup/SPIRV-Tools/issues/946.
Do you have any data was can use to guide the heuristics?  In the limited code that I see, almost all cases are arrays with 1024+ elements and only a handful get used.
Note this does not happen for structured buffers they do not get an entire local copy, which is part of the cause.  This is also not a problem for any type that is declared with OpTypeImage or OpTypeRuntimeArray.

After thinking about @greg-lunarg comments a little more, and discussing with @dnovillo, another solution is to not do local-access-chain convert on arrays, and then trust array-copy-propagation as Greg had suggested.

@s-perron are you going to try phrasing this in terms of not doing local-access-chain conversion?  I don't see an issue with the approach you have here.  But, if this is being hampered by l-a-c, I would rather taking that out first  (incidentally, I'm not sure I completely grasp the idea behind l-a-c conversion, but that's another discussion).

I am fairly certain that LAC only acts on local/function scope variables, so I don't think that it is at fault here.  Unless someone changed it and I missed it. I am guessing that dxc is generating the Load/Extract pattern. Can someone confirm or contradict?

No it is LAC.  I only showed the final code after we have a number of optimizations applied.  I can attach the unoptimized spir-v that dxc generates if you want.
What happens is that there is a local copy (because of the parameter being passed by value) of the cbuffer.  The instruction sequence is:

A load of the cbuffer.
A store to the local copy.
An OpAccessChain on the local copy.
An OpLoad of the single element from the local copy.

The first time we run LAC it sees the load as an OpAccessChain and OpLoad on a function scope symbol.  That gets converted to an OpCompositeExtract.  Then the ssa rewriter propagates the load of the cbuffer into the OpCompositeExtract.

OK. Thanks.
I agree a solution that utilizes (a possibly improved) early call to array copy prop is still worth looking at.
I am fine with taking LAC out of the main recipes. It has outlived its usefulness there. It was originally meant to keep the number of local symbols down (rather than explode them ala scalar replacement) and to eliminate the need to do do analysis on local access chains by unifying everything under the insert/extract regimen. But since we have bit the scalar replacement bullet its usefulness is less obvious.

@greg-lunarg Do you want LAC to continue working as is?  I ask because I don't want to remove it completely.  It still want it to modifiy accesses to vectors an matricies.  I was thinking of limiting LAC; however, I understand that glslang does not use our optimization sets.  They probably need to the same behaviour.  Let me know what you think.

@greg-lunarg @dnovillo I've updated the PR.  I've made the changes a pass, and added some heuristics.  Let me know if I should change anything.

Why does the pass not do a replacement if all elements are used?

That is the case you were worried about.  We could do a load of the whole struct in one shot, which may be faster it the proper hardware is present.  However, if we only use part of the load, then less memory bandwidth may be more beneficial.  I have a threshold that can be tuned in the future.

I have changed my mind.
Since the frontends generally generate an access-chain/load sequence for these anyway, this will likely do little harm.
Another reason is that I think we can use it to facilitate array copy prop. Look at the code generated by glslangValidator for the very similar shader:
struct S {
uint f;
};
cbuffer gBuffer { uint a[32]; uint i; };
RWStructuredBuffer gRWSBuffer;
uint foo(uint p[32], uint i) {
return p[i];
}
[numthreads(1,1,1)]
void main() {
gRWSBuffer[0].f = foo(a, i);
}
glslangValidator generates the load/extract sequence to do these copys. We could use this new pass to put such code into a "canonical" form (access-chain/load). I haven't tested it yet, but I think that would then allow array copy prop to clean this code up.

OK, thanks. I am happy that this has been split out of Simplification. I no longer have a problem with this going forward.


I don't know why, but this mod does not work on my main PC. Other mods work, and they work on different PCs.

I don't know why, but this mod does not work on my main PC. Other mods work, and they work on different PCs.
To Reproduce
Steps to reproduce the behavior:
1.Install a mod.
2. Minecraft works fine, but the value of the SOUND limit does not change.
Screenshots

Runtime info (please complete the following information):

OS: Windows11
Minecraft version: 1.18.2
Mod version: 1.18.2

Other mods

Checklist

[ x] I am using the official version of the mod.
[x ] I tried the latest development version but the issue persists.
[ x] I searched for similar open issues and could not find an existing bug report on this.

Additional context

Initializing the PC and changing from windows 11 to windows 10 did not solve this problem.

This is likely caused by non-ascii characters on the path to your %AppData%. This should already be fixed months ago. Are you sure it does still happen with the latest development build?

Yes it happens in rsls-1.0.1 too.

I initialized the PC again and changed the name to alphabetical and was able to do it. Thank you very much.

Thank you!!!!

Thank you!!!!
https://youtu.be/apvAPWk1eF8

Yes it happens in rsls-1.0.1 too.

By development build I mean this currently. It can be found in the Actions tab.

Thank you!!


Disable SelectedDate in Calendar after reusing the control

Greetings!
I have a Calendar control, which I use in such a way:
...
<calendar:Calendar BorderColor="Accent" BorderWidth="2" SelectedBackgroundColor="Accent" x:Name="MyCalendar"
                             SelectedTextColor="White" SelectedBorderWidth="0" TitleLeftArrowTextColor="Accent" StartDay="Monday" WeekdaysShow="False"
                             TitleRightArrowTextColor="Accent" EnableTitleMonthYearView="True" DateCommand="{Binding DateChanged}" Margin="5"
                             SelectedDate="{Binding SelectedDate}"/>
...

And in my ViewModel:
public class DateDefectContentViewModel : BaseViewModel
{
    ...
    private DateTime? _selectedDate;
    public DateTime? SelectedDate
    {
	 get => _selectedDate;
	 set => SetProperty(ref _selectedDate, value);
    }
...

public DateDefectContentViewModel(CreateDefectModel defectModel)
{
	_defectModel = defectModel;
	SelectedDate = null;
	DateChanged = new DateSelectedCommand(this);
}

protected internal void SelectedDateChanged(object date)
{
	if (_defectModel.CreationDate == Convert.ToDateTime(date))
	{
		_defectModel.CreationDate = null;
		SelectedDate = null;
		SetInteractionHasChanged(false);
		return;
	}
	_defectModel.CreationDate = Convert.ToDateTime(date);
	SelectedDate = _defectModel.CreationDate;
	if (IsForConstrs)
	{
		if (AllConstrs)
		{
			SetInteractionHasChanged(true);
			return;
		}
		if (Nconstr.Count > 0)
		{
			if (Nconstr.Any(i => i.IsChecked))
			{
				SetInteractionHasChanged(true);
				return;
			}
			SetInteractionHasChanged(false);
		}

		SetInteractionHasChanged(false);
	}
	else SetInteractionHasChanged(true);
}
}

Everything works fine when I open the Calendar first time, there is no Date that was selected.
But when I open my page next time, the SelectedDate is exactly that the previous was.
How to disable that?
TIA.

Would it be fixed? Or any suggestions about how to resolve the issue?

Hi @denmusic1992 👋
As you may have seen @rebeccaXam is no longer with us hence the nuget is not supported by anyone.
Regarding your problem: is the date selected when you get back to the page with calendar? So you have flow like Page with calendar -> opens new page -> New page -> goes back to page with calendar -> Calendar page


Adds in routes for upcoming and published stats

Small fix needed to make sure parameters aren't passed over if empty! I somehow managed to merge the version that didn't have that final .compact :/

👍

Thanks, didn't get an email about you doing this :/


Form won't work when submit button is enclosed by ".col-md-12"

When I enclose a submit button with a col-md-12 div element, all form won't respond to click or input.
I put submit button like so
<form>
  <!-- forms here -->
<div class="col-md-12">
   <button type="submit" class="btn btn-default">Send</button>
</div>
</form>

the submit button itself can be clicked but the rest of the forms accept no input or click.

Try running Bootlint over your webpage.
Your example appears to suffer from the following error: https://github.com/twbs/bootlint/wiki/E014
For future reference, please note that bug reports must include a reduced test case as explained in https://github.com/twbs/bootstrap/blob/master/CONTRIBUTING.md#bug-reports
Personal support requests that don't allege a bug in Bootstrap itself should be directed to one of the community help forums listed in https://github.com/twbs/bootstrap/blob/master/README.md#community , such as StackOverflow, instead of to this bug tracker.
Thanks!


analysis_options.yaml include seems to break the plugins data format

If I include a yaml file from my analysis_options, I can no longer seem to load plugins:
included.yaml
analyzer:
  plugins:
    angular:
      enabled: true

analysis_options.yaml
include: included.yaml
analyzer:
  plugins:
    angular:
      enabled: true

then the plugin will not load. Removing the include, or the entire analyzer section from one of the two files causes it to work again.
For giant projects that have a 'default' analysis_options.yaml file that includes the angular plugin, there is no way to extend those angular plugin options without completely copying the default yaml file, unfortunately.

Duplicate of #31345


Unity 2021.2 integration

"C# Source Generator feature is rely on C# 9.0. Currently Unity(2020) does not support C# 9.0 so can not use directly. "
According to  https://docs.unity3d.com/2021.2/Documentation/Manual/CSharpCompiler.html
Unity support C# 90  with small exclusion. So may be it is possible to integrate  UniyGenerator in Unity Editor without additional sln?

No, Unity has indeed added support for C# 9.0, except for SourceGenerator.
Probably a lot of disruptive work needs to be done, such as supporting the new SDK csproj and incorporating the compile phase against Roslyn.

I've reported to forum.
https://forum.unity.com/threads/unity-future-net-development-status.1092205/page-11#post-7715218

Sorry, I misunderstood the behavior.
I've added the documentation to make it work, and it works perfectly.
https://github.com/Cysharp/UnitGenerator#use-for-unity

For me in unity '2021.2.9f1it works only if i placeUnitGenerator.dllin root ofAssets`


Add Icon on Button instead of Label

I want to put icons on the buttons instead of a Label.
Possible?

Yes. In the README.md, look under the Further Customizing your button section.


Replace buddy allocator

Currently, in CappuccinOS, we are using a buddy allocator as our main allocator. This comes with many drawbacks, including, but not limited to, restricting us to powers of two for memory size, and memory fragmentation. Restricting memory size to powers of two is critical, especially with Limine's Memory Map, as users tend to have random amounts of memory that don't align with powers of two. The best alternative that I'm looking into currently is a slab allocator, an allocator that Linux and Haiku use.

this was initially implemented in 4611b89a1d714948a32759d48c67085be6c3bd41 but has been fully fleshed out and since completed in 6ba11e9b2588c255be23185104d8de60f109c72b although not a slab allocator, it is much better than the sketchy buddy allocator


QnA Telemetry Support

Adds Telemetry support to QnAMaker
Shares same pattern as Luis and TelemetryMiddleware.
Enables setting properties during GetAnswersAsync
Enables deriving from class and adding properties
https://github.com/daveta/analytics/blob/master/telemetry_enhancements/TelemetryEnhancements.md#telemetry-qna-recognizer
C# Version: https://github.com/Microsoft/botbuilder-dotnet/pull/1430

Pull Request Test Coverage Report for Build #2087

0 of 0   changed or added relevant lines in 0 files are covered.
2 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.08%) to 87.441%

Files with Coverage Reduction
New Missed Lines
%

libraries/botframework-config/src/botConfigurationBase.ts
2
85.59%

Totals

Change from base Build #2041:
-0.08%

Covered Lines:
3004

Relevant Lines:
3306

💛  - Coveralls


[Question] how to run all five folds?

Hi, when I use the following command
nndet_train xxx --sweep, where xxx denotes the task Id.
Only fold0 is generated in the model folder, how can I run other four folds?
Thank you for your help!

Hi @ZheMin-1992 ,
nndet_train XXX -o exp.fold=0 --sweep
nndet_train XXX -o exp.fold=1 --sweep
nndet_train XXX -o exp.fold=2 --sweep
nndet_train XXX -o exp.fold=3 --sweep
nndet_train XXX -o exp.fold=4 --sweep

You can use the above overwrite to change the fold.
Best,
Michael

@mibaumgartner Thanks a lot for the answer!