GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: Method "ReadValue" with signature "a{sv}" on interface "org.bluez.GattCharacteristic1" doesn't exist

Hello,
I am trying to read data from a wearable sensor (https://www.maximintegrated.com/en/design/reference-design-center/system-board/6147.html) using TinyB. The sensor is discovered, I can connect to it, I can get the services and the characteristics, but when I try to read the value of the characteristics I get the following error:
Exception in thread "AWT-EventQueue-0" tinyb.BluetoothException: GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: Method "ReadValue" with signature "a{sv}" on interface "org.bluez.GattCharacteristic1" doesn't exist
at tinyb.BluetoothGattCharacteristic.readValue(Native Method)
at maxrefdes73.TinyBConnectActivity.<init>(TinyBConnectActivity.java:269)

I have downloaded and compiled the last version of TinyB and I have Bluez v5.37. I am using Eclipse Neon.
Here is my (simplified) source code (the error is at line: byte[] raw = characteristic.readValue(); ):
boolean conn = false;
try {
	conn = remoteDevice.connect();
} catch (BluetoothException be) {
  	System.out.println("Timeout");
}

if (conn) {
    List<BluetoothGattService> btServices = remoteDevice.getServices();

   for (BluetoothGattService service : btServices) {
  	List<BluetoothGattCharacteristic> btCharacteristics = service.getCharacteristics();

      for (BluetoothGattCharacteristic characteristic : btCharacteristics) {
           byte[] raw = characteristic.readValue();
      }
  }
}

This is my output:
devicesDiscovered:
C0:00:00:00:00:00 MAXREFDES73#
Connecting to MAXREFDES73#
CONNECTED to MAXREFDES73#
Services exposed by device and characteristics for each service:
Service UUID: 0000180a-0000-1000-8000-00805f9b34fb
Charateristics:
00002a24-0000-1000-8000-00805f9b34fb
00002a23-0000-1000-8000-00805f9b34fb
00002a29-0000-1000-8000-00805f9b34fb
00002a2a-0000-1000-8000-00805f9b34fb
00002a28-0000-1000-8000-00805f9b34fb
00002a27-0000-1000-8000-00805f9b34fb
00002a26-0000-1000-8000-00805f9b34fb
00002a25-0000-1000-8000-00805f9b34fb
Service UUID: 0000180d-0000-1000-8000-00805f9b34fb
Charateristics:
00002a39-0000-1000-8000-00805f9b34fb
00002a38-0000-1000-8000-00805f9b34fb
00002a37-0000-1000-8000-00805f9b34fb
Service UUID: 00001801-0000-1000-8000-00805f9b34fb
Charateristics:
00002a05-0000-1000-8000-00805f9b34fb
Service UUID: 0000180f-0000-1000-8000-00805f9b34fb
Charateristics:
00002a19-0000-1000-8000-00805f9b34fb
$ bluetoothd -v
5.37

$ uname -a
Linux plimbariciSon 4.8.0-39-generic #42~16.04.1-Ubuntu SMP Mon Feb 20 15:06:07 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.2 LTS
Release:	16.04
Codename:	xenial

I've seen in this thread that BlueZ 5.43 has introduced parameters for its readValue() method, but I have BlueZ 5.37 so I can't figure out where the problem is.
Thank you.

There's no backwards compatibility. If you want to use the latest version of TinyB you'll need BlueZ 5.42 or later, otherwise use version 0.5.

Thank you for you quick reply.
Isn't 0.5 the latest version of TinyB?
I had initially installed Bluez 5.43 and then reinstalled 5.37 because of the argument taken by readValue() in 5.43.
If BlueZ 5.43 has a method array{byte} ReadValue(dict options) , shouldn't there be a readValue() method with arguments in TinyB?
There is public native byte[] readValue(); in BluetoothGattCharacteristic.java and
jbyteArray Java_tinyb_BluetoothGattCharacteristic_readValue(JNIEnv *env, jobject obj) {
    try {
        BluetoothGattCharacteristic *obj_gatt_char =
                                    getInstance<BluetoothGattCharacteristic>(env, obj);
        std::vector<unsigned char> array = obj_gatt_char->read_value();
    ...
}

in BluetoothGattCharacteristic.cxx
I'm sorry if my questions sound stupid, I'm not very skilled with jni.

Apparently the first time I had installed Bluez 5.43 I was actually still running 5.37 (because it was installed in /usr/local/libexec/bluetooth/bluetoothd but the version running was /usr/sbin/bluetoothd). Now I managed to reinstall and actually run Bluez 5.43 (bluetoothd -v -> 5.43) (by symlink /usr/sbin/bluetoothd to /usr/local/libexec/bluetooth/bluetoothd: sudo ln -s /usr/local/libexec/bluetooth/bluetoothd /usr/sbin/bluetoothd) and I can read the values of the characteristics :)
Thank you and sorry for bothering.


KyranRana Help Pls

Error.log
Stack trace: #0 /home/betrost/public_html/src/CloudflareBypass/CFCurlImpl.php(83): CloudflareBypass\CaptchaPageImpl->getClearanceUrl('--> ') #1 /home/betrost/public_html/index.php(29): CloudflareBypass\CFCurlImpl->exec(Resource id #3, Object(CloudflareBypass\Model\UAMOptions)) #2 {main}   thrown in /home/betrost/public_html/src/CloudflareBypass/CaptchaPageImpl.php on line 16
Website that will draw data  www.danobet1.com

Use the ticket which you have open already. Don't create new ones.


Application flashes successfully, does not load

Summary
This issue (https://github.com/kaidyth/nrf52_bootloader/issues/3) is very similar to what I'm seeing, but I have some questions about the conclusions reached in that issue. I'm in no way asking for troubleshooting steps to solve my unique case, but I'd like to find out what some of the findings of that issue mean to those of us implementing this bootloader on a Nano 33 BLE or other board based on the u-blox Nina B306.
First off:

I'm using a custom board built on the u-blox Nina B306 (same exact SoC as the Arduino Nano 33 BLE)
I can flash the nRF52 Adafruit bootloader and applications built with the Adafruit BSP for development without any issues. All functionality from Adafruit works, including legacy DFU, USB uploading from Arduino, PlatformIO, USB Serial, etc.
I can flash the Kaidyth bootloader and flash signed applications via USB and BLE without issues, but the application flashed in this manner does not load.

Reproducing
Bootloader
Start by building and flashing the Feather Express Bootloader with default settings.
Application Code
void setup() { Serial.begin(115200); }
void loop()
{
  Serial.println("loop");
  delay(1000);
}

Build Application Package and Flash

Arduino IDE > Sketch > Export compiled Binary > nrfutil pkg generate [...] > nrfutil dfu usb-serial [...]
At this point, the board resets but the application never loads and it's not visible in the USB devices anymore.

Questions

Is the conclusion that applications generated (using the Feather nRF52840 Express BSP) are incompatible with this bootloader or only if the <bluefruit.h> library is explicitly included? (My results seem to indicate that regardless of including the bluefruit library, it fails, but it might not be the Adafruit libraries in my case)
If item 1 is the case, is there another BSP suggested for use with this bootloader to generate the application packages? (I've also tried the Arduino Nano 33 BLE BSP and it failed in the same manner with the generic bootloader variant)

Thank you so much for your time and effort in building and maintaining this bootloader.

Hi,
If I recall, it was just the bluefruit library that was causing a hang - I was able to get a variety of different Arduino sketches to work including one that did an RBG ring of the embedded LED on the Feather, but looking back on my notes I do remember I had issues with serial outs.
At it's core, this bootloader really is just the default Nordic secure bootloader - it just has all the features enabled for DFU and a double reset functionality built in.
I can take another look though since it's been a while.
A couple of questions:

Can you share your exact try command you used for flashing and building? I upgraded the SD version so if you copied the command from #3 it should fail cause of an SD mismatch. The SD should be 0xCA when signing.
Does the Nano have a LED you can flash instead of writing to serial? GPIO was what I was testing and it's a bit easier to validate that way since it doesn't require an external device or connectivity to see if the sketch is doing anything.
The generic bootloader runs in debug mode - do you have a JLink or similar tool you can connect to read the debug serial output and share here? The debug logs can indicate if the bootloader is able to pass off to the application or not. If not... You can recompile the generic bootloader to spit out UART logs to any pins of your choosing and capture them from a generic UART reader. Just make sure you compile in DEBUG mode.
Before you flash the application can you confirm that you see KAIDYTH_DFU in nrfconnect?
Does the double tap to reset work or the Nano after your application is flashed? If there's not a button, you should be able to short and of the reset pins to GND twice and see KAIDYTH_DFU appear in nrfconnect. If this doesn't work then that means there's a signing problem with the app most likely.

I can order the Nano 33 BLE and have it rush shipped to debug it correctly. I need a new chip to play with while my custom board I'm working on needs revisions. =)
Arduino isn't really something I've dived too deep in - I went straight to C instead, but I can certainly investigate this with you.

@charlesportwoodii Thank you for the prompt reply. Here are the answers to your questions:

Commands:

nrfutil pkg generate --hw-version 52 --application-version 1 --application ~/Documents/Arduino/test001/test001.ino.feather_nrf52840_express.hex --sd-req 0xCA --key-file /path/to/private.pem app_dfu_package.zip

nrfutil dfu usb-serial --package app_dfu_package.zip -p /dev/cu.usbmodemE23B6AAE27A81

Yes, I attempted to flash the LED but to no avail.
I do have a JLink. I'll report back later.
Absolutely. I'm able to perform secure BLE DFU. The board is also listed in the USB devices on my computer while in the DFU mode.

Double tap does not work but performing a rest command via JLink brings the device back into DFU mode. (nrfjprog -f nrf52 --reset)

That all looks right. It's odd you can't get into the bootloader through with the double tap reset. Reset is wired directly into the nRF chipset, and if the bootloader runs initially it should just "work".
Looking back on my notes here: https://github.com/kaidyth/nrf52_bootloader/issues/7 I used the neopixel lib as a test harness for Arduino on the Feather Express nRF52840. I know that zip in particular won't work cause of the SD version, so I can try to recompile it with the bootloader in debug mode and see what spits out. I'm almost certain that something is just off with the app signing process but won't be able to check until I see what the bootloader complains about in DEBUG mode.
I ordered a Nano 33 BLE and can make a formal definition for it as well so you aren't using the generic one - DEBUG builds take up a lot more space than they ought to.

@charlesportwoodii Here is the log dump from the JLink RTT Viewer: https://gist.github.com/SaltedBlowfish/fa2aa2e9554099cb4a581053c62b66d1

There's a RAM conflict happening in the application with SD7 - the hung state you're seeing is a hard-fault.
SDK17/SD7 required the RAM regions to be adjusted (see here https://github.com/kaidyth/nrf52_bootloader/commit/92e8e98e79b41377a8051d2d26b19526f3fe45a1#diff-d1bd30e579a6b094610b56c45d32d548R9.)
The Adafruit board lib is still using s140_sd6, whereas the bootloader has RAM reserved for s140_sd7. https://github.com/adafruit/Adafruit_nRF52_Arduino/issues/291 is the outstanding issue on the nRF52 Arduino item to handle upgrading the SoftDevice - it looks like it isn't implemented yet.
A few options for you to try:

Adjust the RAM regions for https://github.com/adafruit/Adafruit_nRF52_Arduino/blob/master/cores/nRF5/linker/nrf52840_s140_v6.ld to as follows:FLASH (rx) : ORIGIN = 0x27000, LENGTH = 0xc9000
RAM (rwx) :  ORIGIN = 0x20002280, LENGTH = 0x3dd80
uicr_bootloader_start_address (r) : ORIGIN = 0x00000FF8, LENGTH = 0x4

I can't find the SD7 changelog at the moment, but I pulled these values from an SD7 compatible app I have working on a custom board that I knows works.
Effectively the bootloader is trying to reserve some extra space the application writes into. Increasing the RAM space in the Feather linker file should allocate enough space for the application to run.

If you're running Arduino code, downgrade the bootloader to 0.4.0 (https://github.com/kaidyth/nrf52_bootloader/releases/tag/0.4.0) then re-flash the application using 0xB6 as the SoftDevice version

Either option should work, but I'm pretty confident that's the problem. I don't run Arduino code ever on these chips - I would expect that Adafruit upgrade their bootloader regions for others to benefit from this. It's a problem with being an early adopter for SD7 (If you call since September 2019 as early adopting).
Let me know if that works for you. With SD6 on the Feather you should be able to flash the zip file I used as a test harness here: https://github.com/kaidyth/nrf52_bootloader/issues/7 as a quick and dirty test and see the neopixel change colors.

Thinking about ways you can use this with your custom board...
You can fork the repo from 0.4.0, the copy the generic board definition over and run make that way. No guarantees if the docker build still works from that for back, but assuming you can download all the right libs from before Setember it should compile and run you Arduino code sans BLE stuff just fine

@charlesportwoodii Brilliant! Thank you. I went with option 1. I had a suspicion it was a memory issue but I don't know enough about how to actually find the right regions to modify them.
Your help has been very appreciated. Here's a gif of the lights flashing on my custom board (not a NeoPixel, just RGB LED):


libuv: 1.23.1 -> 1.23.2

Motivation for this change
I was having trouble updating LXD and I thought maybe this had to do with it; it didn't. Nevertheless, I ended using this bugfix release on my systems and seems to be working.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[x] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

@grahamcofborg build libuv

Success on aarch64-linux (full log)
Attempted: libuv
Partial log (click to expand)
 /nix/store/5s4072bfj6q40mhw5hgb8y4lbmgwrbga-coreutils-8.30/bin/install -c -m 644 include/uv/errno.h include/uv/threadpool.h include/uv/version.h include/uv/unix.h include/uv/linux.h '/nix/store/r6hv3w2cqp65n3y20zp2k246nyjbiwsh-libuv-1.23.2/include/uv'
make[1]: Leaving directory '/build/source'
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/r6hv3w2cqp65n3y20zp2k246nyjbiwsh-libuv-1.23.2
shrinking /nix/store/r6hv3w2cqp65n3y20zp2k246nyjbiwsh-libuv-1.23.2/lib/libuv.so.1.0.0
strip is /nix/store/p9akxn2sfy4wkhqdqa3li97pc6jaz3r1-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/r6hv3w2cqp65n3y20zp2k246nyjbiwsh-libuv-1.23.2/lib
patching script interpreter paths in /nix/store/r6hv3w2cqp65n3y20zp2k246nyjbiwsh-libuv-1.23.2
checking for references to /build in /nix/store/r6hv3w2cqp65n3y20zp2k246nyjbiwsh-libuv-1.23.2...
/nix/store/r6hv3w2cqp65n3y20zp2k246nyjbiwsh-libuv-1.23.2

Success on x86_64-linux (full log)
Attempted: libuv
Partial log (click to expand)
 /nix/store/dzvbhkzfgwkq97ljkcvi4gs7vmflh6a0-coreutils-8.30/bin/install -c -m 644 include/uv/errno.h include/uv/threadpool.h include/uv/version.h include/uv/unix.h include/uv/linux.h '/nix/store/54rs9zhpmph7klk78i83k8awddb2wc4x-libuv-1.23.2/include/uv'
make[1]: Leaving directory '/build/source'
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/54rs9zhpmph7klk78i83k8awddb2wc4x-libuv-1.23.2
shrinking /nix/store/54rs9zhpmph7klk78i83k8awddb2wc4x-libuv-1.23.2/lib/libuv.so.1.0.0
strip is /nix/store/vcc4svb8gy29g4pam2zja6llkbcwsyiq-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/54rs9zhpmph7klk78i83k8awddb2wc4x-libuv-1.23.2/lib
patching script interpreter paths in /nix/store/54rs9zhpmph7klk78i83k8awddb2wc4x-libuv-1.23.2
checking for references to /build in /nix/store/54rs9zhpmph7klk78i83k8awddb2wc4x-libuv-1.23.2...
/nix/store/54rs9zhpmph7klk78i83k8awddb2wc4x-libuv-1.23.2

@GrahamcOfBorg eval

Done in #50044


Import of webarchives

Please explain your idea to make the app better:
I would really appreciate, if I could import .webarchive-Files in collect.

I like this idea very much. The benefit would be to have searchable snapshots of web pages as they looked like at a certain time.
Possible alternatives:

https://en.wikipedia.org/wiki/Web_ARChive
PDF
PNG Screenshot

Implementation Note
For efficient storage considering these ZIP helpers:
https://github.com/Stuk/jszip
https://www.npmjs.com/package/extract-zip

Webarchive is a Plist formatted file. It is also used if something is copied from Safari. External resources are also referenced.
In order to render previews if no other native way of doing it exists:

https://github.com/niklasvh/html2canvas/
https://github.com/cburgmer/rasterizeHTML.js

kUTTypeWebArchive

Important data to store:

URL
Title
Description
Plain Text Content
Favicon
Open Graph Image
Thumbnail of rendered page
Persisted look: Webarchive, PDF, Image, Embedded HTML


empty GSI when using the @index directive with multiple sortKeys

Before opening, please confirm:

[X] I have searched for duplicate or closed issues and discussions.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

JavaScript Framework
Next.js
Amplify APIs
GraphQL API
Amplify Categories
api
Environment information

System:
    OS: macOS Mojave 10.14.6
    CPU: (8) x64 Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
    Memory: 65.40 MB / 16.00 GB
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 14.3.0 - ~/.nvm/versions/node/v14.3.0/bin/node
    Yarn: 1.22.10 - ~/.nvm/versions/node/v14.3.0/bin/yarn
    npm: 6.14.5 - ~/.nvm/versions/node/v14.3.0/bin/npm
  Browsers:
    Brave Browser: <IP_ADDRESS>
    Chrome: 101.0.4951.54
    Firefox Developer Edition: 90.0
    Safari: 14.1.2
  npmPackages:
    @ampproject/toolbox-optimizer:  undefined ()
    @babel/core:  undefined ()
    @babel/runtime:  7.15.4
    @emotion/react: ^11.5.0 => 11.5.0
    @emotion/styled: ^11.3.0 => 11.3.0
    @hapi/accept:  undefined ()
    @mui/icons-material: ^5.0.5 => 5.0.5
    @mui/material: ^5.0.6 => 5.0.6
    @napi-rs/triples:  undefined ()
    @next/react-dev-overlay:  undefined ()
    @next/react-refresh-utils:  12.1.0
    @peculiar/webcrypto:  undefined ()
    @types/draft-js: ^0.11.6 => 0.11.9
    @types/gtag.js: 0.0.8 => 0.0.8
    @types/markdown-draft-js: ^2.2.4 => 2.2.4
    @types/node: ^17.0.13 => 17.0.24
    @types/react: ^17.0.34 => 17.0.34
    @types/react-draft-wysiwyg: ^1.13.3 => 1.13.4
    @types/uuid: ^8.3.4 => 8.3.4
    @vercel/nft:  undefined ()
    abort-controller:  undefined ()
    acorn:  undefined ()
    amphtml-validator:  undefined ()
    arg:  undefined ()
    assert:  undefined ()
    async-retry:  undefined ()
    async-sema:  undefined ()
    autoprefixer: ^10.4.0 => 10.4.0
    aws-amplify: ^4.3.12 => 4.3.12
    aws-sdk: ^2.1029.0 => 2.1029.0
    babel-packages:  undefined ()
    browserify-zlib:  undefined ()
    browserslist:  undefined ()
    buffer:  undefined ()
    chalk:  undefined ()
    ci-info:  undefined ()
    cli-select:  undefined ()
    comment-json:  undefined ()
    compression:  undefined ()
    conf:  undefined ()
    constants-browserify:  undefined ()
    content-disposition:  undefined ()
    content-type:  undefined ()
    cookie:  undefined ()
    cross-spawn:  undefined ()
    crypto-browserify:  undefined ()
    cssnano-simple:  undefined ()
    debug:  undefined ()
    devalue:  undefined ()
    domain-browser:  undefined ()
    draft-js: ^0.11.7 => 0.11.7
    eslint: ^7.23.0 => 7.32.0
    eslint-config-next: ^12.0.3 => 12.0.3
    etag:  undefined ()
    events:  undefined ()
    find-cache-dir:  undefined ()
    find-up:  undefined ()
    formdata-node:  undefined ()
    framer-motion: ^5.3.3 => 5.3.3
    fresh:  undefined ()
    get-orientation:  undefined ()
    glob:  undefined ()
    gray-matter: 4.0.2 => 4.0.2
    gzip-size:  undefined ()
    http-proxy:  undefined ()
    https-browserify:  undefined ()
    icss-utils:  undefined ()
    ignore-loader:  undefined ()
    image-size:  undefined ()
    is-animated:  undefined ()
    is-docker:  undefined ()
    is-wsl:  undefined ()
    jest-worker:  undefined ()
    json5:  undefined ()
    jsonwebtoken:  undefined ()
    loader-utils:  undefined ()
    lodash.curry:  undefined ()
    lru-cache:  undefined ()
    markdown-draft-js: ^2.4.0 => 2.4.0
    micromatch:  undefined ()
    mini-css-extract-plugin:  undefined ()
    nanoid:  undefined ()
    native-url:  undefined ()
    neo-async:  undefined ()
    next: ^12.1.0 => 12.1.0
    next-seo: ^4.28.1 => 4.28.1
    next-sitemap: ^1.6.203 => 1.6.203
    node-fetch:  undefined ()
    node-html-parser:  undefined ()
    ora:  undefined ()
    os-browserify:  undefined ()
    p-limit:  undefined ()
    path-browserify:  undefined ()
    postcss: ^8.3.11 => 8.3.11 (8.4.5)
    postcss-flexbugs-fixes:  undefined ()
    postcss-modules-extract-imports:  undefined ()
    postcss-modules-local-by-default:  undefined ()
    postcss-modules-scope:  undefined ()
    postcss-modules-values:  undefined ()
    postcss-preset-env:  undefined ()
    postcss-safe-parser:  undefined ()
    postcss-scss:  undefined ()
    postcss-value-parser:  undefined ()
    process:  undefined ()
    punycode:  undefined ()
    querystring-es3:  undefined ()
    raw-body:  undefined ()
    react: ^17.0.2 => 17.0.2
    react-dom: ^17.0.2 => 17.0.2
    react-draft-wysiwyg: ^1.14.7 => 1.14.7
    react-feather: ^2.0.9 => 2.0.9
    react-is:  17.0.2
    react-multi-carousel: ^2.6.5 => 2.6.5
    react-player: ^2.9.0 => 2.9.0
    react-refresh:  0.8.3
    react-server-dom-webpack:  undefined ()
    react-toastify: ^8.1.0 => 8.1.0
    recharts: ^2.1.6 => 2.1.6
    regenerator-runtime:  0.13.4
    remark: 13.0.0 => 13.0.0
    remark-html: ^15.0.1 => 15.0.1
    sass-loader:  undefined ()
    schema-utils:  undefined ()
    semver:  undefined ()
    send:  undefined ()
    setimmediate:  undefined ()
    sharp: ^0.29.3 => 0.29.3
    source-map:  undefined ()
    stream-browserify:  undefined ()
    stream-http:  undefined ()
    string-hash:  undefined ()
    string_decoder:  undefined ()
    strip-ansi:  undefined ()
    tailwindcss: ^2.2.19 => 2.2.19
    terser:  undefined ()
    text-table:  undefined ()
    timers-browserify:  undefined ()
    tty-browserify:  undefined ()
    typescript: ^4.4.4 => 4.4.4
    ua-parser-js:  undefined ()
    unistore:  undefined ()
    util:  undefined ()
    uuid: ^8.3.2 => 8.3.2 (3.4.0, 3.3.2, )
    vm-browserify:  undefined ()
    watchpack:  undefined ()
    web-streams-polyfill:  undefined ()
    web-vitals:  undefined ()
    webpack:  undefined ()
    webpack-sources:  undefined ()
    ws:  undefined ()
  npmGlobalPackages:
    @aws-amplify/cli: 7.6.9
    @torus-tools/cli: 0.0.125
    arjan-cli: 0.2.7
    npm: 6.14.5
    yarn: 1.22.10

Describe the bug
I have created a GSI with multiple sort keys in my graphql schema using the @index directive but the resulting dynamodb index is empty.
Bellow is my schema:
type Product
@model
@aws_api_key
@aws_cognito_user_pools
@auth(rules: [{allow: groups, groups: ["admin"], provider: userPools}, {allow: public, operations: [read]}]) {
  sku: String! @primaryKey
  subcategorySlug: String! @index(name: "ProductsBySubcategory", queryField: "ProductsBySubcategory", sortKeyFields: ["isDiscontinued", "isMainProduct"])
  isDiscontinued: BooleanString! @index(name: "ProductsByIsDiscontinued", queryField: "productsByIsDiscontinued")
  isMainProduct: BooleanString! @index(name: "ProductsByIsMainProduct", queryField: "ProductsByIsMainProduct", sortKeyFields: ["isDiscontinued"])
  subcategory: Subcategory! @belongsTo(fields: ["subcategorySlug"])
  color: String!
}

enum BooleanString {
  TRUE
  FALSE
}

type Subcategory @model @aws_api_key @aws_cognito_user_pools @auth(rules: [{allow: groups, groups: ["admin"], provider: userPools}]) {
  slug: String! @primaryKey
}

The resulting dynamodb index is 0 bytes in size and doesn't return any items when queried. bellow is a sample query im using which in theory should return some results:
query MyQuery {
  ProductsBySubcategory(subcategorySlug: "subcat1", isDiscontinuedIsMainProduct: {eq: {isMainProduct: FALSE, isDiscontinued: FALSE}}) {
    items {
      color
  }
}

If i remove one of the sortKeys and create an index as follows subcategorySlug: String! @index(name: "ProductsBySubcategory", queryField: "ProductsBySubcategory", sortKeyFields: ["isMainProduct"]) the dynamodb index gets created and populated correctly. However in this case i need to use both sort keys.
Expected behavior
I should be able to create an index with multiple sort keys using the @index directive and the resulting dynmodb index shouldnt be empty.
Reproduction steps

create graphql API and copy paste the provided schema
populate the dynamodb table
query the index

Code Snippet

type Product
@model
@aws_api_key
@aws_cognito_user_pools
@auth(rules: [{allow: groups, groups: ["admin"], provider: userPools}, {allow: public, operations: [read]}]) {
  sku: String! @primaryKey
  subcategorySlug: String! @index(name: "ProductsBySubcategory", queryField: "ProductsBySubcategory", sortKeyFields: ["isDiscontinued", "isMainProduct"])
  isDiscontinued: BooleanString! @index(name: "ProductsByIsDiscontinued", queryField: "productsByIsDiscontinued")
  isMainProduct: BooleanString! @index(name: "ProductsByIsMainProduct", queryField: "ProductsByIsMainProduct", sortKeyFields: ["isDiscontinued"])
  subcategory: Subcategory! @belongsTo(fields: ["subcategorySlug"])
  color: String!
}

enum BooleanString {
  TRUE
  FALSE
}

type Subcategory @model @aws_api_key @aws_cognito_user_pools @auth(rules: [{allow: groups, groups: ["admin"], provider: userPools}]) {
  slug: String! @primaryKey
}

Log output

// Put your logs below this line

aws-exports.js
No response
Manual configuration
No response
Additional configuration
No response
Mobile Device
No response
Mobile Operating System
No response
Mobile Browser
No response
Mobile Browser Version
No response
Additional information and screenshots
No response

Hey @gkpty :wave: thanks for raising this! This seems related to #490, referencing this comment do the items eventually get populated after a few minutes? Or only after a subsequent mutation?
Taking a further look at this one 🙂

Marking as a duplicate of #490 and closing in favor of tracking the existing item


Archiving, Basic Logs, Retention Workbook

Required items, please complete
Change(s):

New Archiving, Basic Logs, Retention Workbook Created

Reason for Change(s):

New workbook provides more detailed information on table plans, easier ability to update table plans, view search and restore tables. Update workbook also includes more help specific context to guide users better

Testing Completed:

Yes

Checked that the validations are passing and have addressed any issues that are present:

Yes

@seanstark  Logo should be less than 5kb.

Is there a list of requirements for the svg logo? I keep running into validation issues with the pipeline checks

Fixed the logo issues, new one provided should meet the requirements.

@seanstark Workbook query is failing. Please look into it. thanks!!!

@seanstark Workbook query is failing. Please look into it. thanks!!!

That's expected and there is nothing I can do about it unfortunately with the current workbook supported features. The error occurs due to a underlying query returning no results that is used in the merge query.
If you click Show Help, the user will have information on the error.


Have the ability to change topK value

Hi
it s possible to have the ability to change topK value in frontend of the script not hard coded in .env
regards

I've added it, just keep in mind setting too high will most likely exceed the tokens if you get too many matches.

thx a lot for that , its working perfectly.


Fixed develop branch build errors

Resolved all ERROR-level validation errors in the template
Removed deprecated properties for EC2 instance NAT gateways
Added missing conditional dependant on private subnets for the S3 VPC
endpoint

Additionally, please PR from your own account versus molleb/develop branch within this repo. Please delete the molleb/develop branch once you've migrated to your own fork.


v1 of Cyberpunk Red Theme for Simple Calendar.

Cyberpunk Red Foundry System theme. Adapts to CPR default or Dark modes.

Thank you for this, I will be including it in the next release. Which should be by the end of the week, depending on how quickly I can get a few other issues fixed!


Ton reader not working

How can I create PR?
Github sad me "ERROR: Permission to tact-lang/tact-template.git denied"

@rustam87 You need to create your own fork of tact-template, make your changes in the fork and then open a PR

@rustam87 Could you specify what kind of error you observe? If you could copy and paste it in the original post that would be very much appreciated


Credential rotation as a separate paragraph

I think that How managed identity works wasn't intended to be a sub-paragraph of Credential rotation.

@Tbohunek : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Hi @MarkusVi - Would you review the proposed update?
Note that publishing is suspended until Tuesday, May 19. Merging will resume 7:00 AM PT Tuesday. Content owners can still review and sign off on PRs meanwhile. Thanks.

#sign-off

Thanks @jeevansd - The PR is ready to merge on 5/19.


Add a docs page about custom OAuth provider

Continue: https://github.com/casdoor/casdoor/pull/667
See: https://casdoor.org/docs/overview/

@Marvelousp4

@halozhy

Hey @hsluoyz ! If you dont mind can i work on this issue?
Also I see that the issue is probably related to adding overview to casdoor website but can you maybe elaborate what issue exactly we are solving here just to be sure?

@pratik2315 plz work on other issues:


Keystone ldap

Want to merge changes for ldap integration for keystone with master.

Can one of the admins verify this patch?

"ok to test" to accept this pull request for testing
"test this please" for a one time test run
"add to whitelist" to add the author to the whitelist

Can one of the admins verify this patch?

"ok to test" to accept this pull request for testing
"test this please" for a one time test run
"add to whitelist" to add the author to the whitelist

I don't like the intermingling of openldap role and keystone on top of openldap.  There should be multiple roles here: openldap and openldap-keystone.

The differences for keystone to use a sql v/s an ldap backend are just the configuration in keystone.conf. I think it would be overkill to create a separate role, most of the code for deploying keystone with ldap backend is the same, is there any construct in ansible to reuse this code if we create a new role called openldap-keystone?

I'm not convinced ursula should be setting up and maintaining LDAP itself.    Can we see some documentation on the purpose and desired outcome of this work ?

Paul, the  main reason to use an ldap for identity backend is to allow for password policies (now allowing passwords in history,  password complexity, user locking, password expiry, avoiding password change too quickly etc.. ). Ldap server also allows to control the encryption algorithm used for encrypting the passwords.
Password policies are needed for security compliance, and that is the main motivation for this work.

There are a lot of commits here, with some reversions too. For sanity, these should be collapsed into logical blocks of commits, where each commit should result in a working tree. Trying to bisect around a ton of broken commits is not helpful. Please reach out if you need help with collapsing commits.

I'm really not in favor of committing binary files here for check_password, nor am I in favor of dumping source code to that project here as well. We should be able to find or make a package for this that's appropriate for our destination hosts and install it that way.

Jesse, none of the commits are broken, there may be 1 or 2 where a play on an already built environment fails, I test every commit before delivering, once on a clean environment and then on one which is already built.
Most commits are related to refactoring or style changes for review comments, the repeated testing is reason for multiple commits. Once I have tested a code change/refactoring, I make a commit. I am trying to group them logically, for example, all name space changes in one commit, password checker related in another etc.
So, do you want us to rebase the repo?
"Please reach out if you need help with collapsing commits."
Yes will help if we can collapse the commits. Let me know what is needed from me.

@MaheshIBM Lets squash these commits. This is a good place to start: http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html

I will push some more commits once all changes are done from my side, i will squash the commits so that it can reviewed easily.

I have squashed all commits into a single one, outstanding items.

.deb file for check_password needs to be put into repo.
Gift wrap manifest needs to be updated for ldap driver dependencies.
variables related to ou etc are repeated, I don't have a solution for this since it won't allow a definition to refer to itself. Possibly we can form the longer strings from shorter vars and use set variable in the tasks, adding the composition of strings in template files will make the template files difficult to read.

@MaheshIBM Please rebase with master to resolve merge conflicts

Is this what needs to be done?
git fetch origin
git merge origin/master
modify files that need to be merged.
commit the merges.

git fetch origin
git rebase origin/master
modify conflicts
git add <conflicting files
git rebase --continue

Done. Rebased.

@j2sol Is this good to merge?

We need to sort out the deb placement for the password lib binary you're using. I don't want to ship it as a file in git. Lets sync on this in slack.

@j2sol @MaheshIBM -- So what is the conclusion here. Any thoughts on how we want to approach the deployment of the password check library.

The jenkins build seemed to have failed with bad status line error while installing shade for ansible (probably a network outage/rep server problem), build numbers 374/375 seemed to have passed, any way to trigger a new jenkins job run on the same PR?

I have created a new pull request https://github.com/blueboxgroup/ursula/pull/1227 whihc is from code that is forked off ursula/master. So I am closing this request.


#466 fix, remove "-l" arg from shell call

That’s not really a good idea, we need to respect the profile.


script/checktrie: fix the import to SQLDB

This is nothing special, it just allows the script to run without throwing an error on import.

Thanks!

Hey @belikor , shoot me a DM on telegram with the PRs that you've completed + esimated times so we can show you some appreciation.


GA Search - Voted on - Add search functionality

Why
Voted on tab uses data from GET /drep/getVotes/{drepId} which should include the search parameters.
That task should provide integration with such params to allow searching for the voted on actions.
What
Integrate the search params of /drep/getVotes/{drepId} endpoint.
How
Use the same logic as for /proposal/list endpoint.

OK: checked here


set VK_SDK_PATH only if existing

When running extensions_vk.py --beta python report:
File "/home/marcin/dev/nvpro_samples/nvpro_core/nvvk/extensions_vk.py", line 142, in loc_spec_path = os.path.normpath(os.getenv('VK_SDK_PATH') + '/share/vulkan/registry/vk.xml') ~^~~~~~~~~
As @NeilBickford-NV  explain this is due to evaluating env to None. This code only modifying patch when env set and the file exist

Hi @neurotok ! I believe we now have a fix for this in the new get_vk_xml_path() function in extensions_vk.py, which refactors this logic and adds a check for if vulkan_sdk_env is not None: https://github.com/nvpro-samples/nvpro_core/commit/617712c507e5982815d377d39643eed9f344276e#diff-92d844e685af249aa972e6f92884c5a1da3493558a1bccb7300183fc02835061R162 .
Thanks again!


Package dependency graph build error

(PR prepared)
The package dependency graph can not be resolved, possibly because of these requirements:
https://github.com/groue/GRDB.swift.git — 4.1.0..<5.0.0
https://github.com/groue/CombineExpectations.git — 0.3.0..<1.0.0
https://github.com/groue/CombineExpectations.git — 0.2.0
https://github.com/groue/GRDB.swift — 4.1.0..<5.0.0

Indeed the demo app won't build. Thanks for spotting this bug! I'm closing this issue in favor of #33.

Sounds good.  I wasn't sure if you wanted pull requests to be associated with issues like some repos do; that's why I wrote one to go with the PR.  I will just do PRs in the future.

What I want to avoid is to discourage a contributor because I have to discard an ill-advised pull request. Opening issues first is usually a good way to avoid such frustration, because the contributor only spends useful time. This is especially true for "big" or "controversial" topics.
In the particular case of #32 and #33, the topic is small, issue and the pull request are nearly identical, and we're about to find an agreement. So we can do without much ceremony.
But please don't generalize from this particular case, will you? ;-)

Understood. Makes sense. Thank you.


I will pay for help

if you want to contact me and send me your paypal so i can pay you under 6$ anyone @MythAgario the error is when i try to reset mine it could not queue the error message is
[14:37:43] [Asynchronous Worker #1 thread/DEBUG]: #0 MineReset_v3.phar/src/falkirks/minereset/task/ResetTask(114): pocketmine\error_handler(integer 2, string A non-numeric value encountered, string phar://C:/Users/apecl/Desktop/Pocketmine-MP(2)/plugins/MineReset_v3.phar/src/falkirks/minereset/task/ResetTask.php, integer 114, array Array())
[14:37:43] [Asynchronous Worker #1 thread/DEBUG]: #1 src/pocketmine/scheduler/AsyncTask(67): falkirks\minereset\task\ResetTask->onRun()
[14:37:43] [Asynchronous Worker #1 thread/DEBUG]: #2 (): pocketmine\scheduler\AsyncTask->run()
[14:37:43] [Server thread/CRITICAL]: Could not execute asynchronous task ResetTask: Task crashed

I have moved this to #100


Add ability to pipeline set commands

Feature request
Add .pipeline() method
I'm using pyle38 and working on a visualization/simulation app. The ability to pipeline commands would be great.

Hi, thank you for using pyle38 and thank you for your post.
Unfortunately I don't agree with you that pyle38 should take care of lining up tasks. You can use solid existing frameworks for that, like pypeln.
Hope that points you into the correct direction. :)
Have a nice weekend 👋🏻

Thanks.  Understood. Great client library by the way.  I'll drop down to the redis client to use the Tile38 RESP pipeline commands
https://github.com/tidwall/tile38/issues/49
Thanks for the quick response

From: Ben @.>
Sent: Friday, November 26, 2021 3:10:57 PM
To: iwpnd/pyle38 @.>
Cc: Brent Tucker @.>; Author @.>
Subject: Re: [iwpnd/pyle38] Add ability to pipeline set commands (Issue #70)
Closed #70https://github.com/iwpnd/pyle38/issues/70.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/iwpnd/pyle38/issues/70#event-5679151046, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQTVS65MNKSQ7P5U5OHLR3TUN7SVDANCNFSM5I3IYQMA.
Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

If you elaborate a little on what you’re planning to do, maybe I can help out. I’m not sure I understand yet. :)

Hi Ben,
I am looking for the ability SET or GET 10's or 100s of commands, Pipelines (in Redis and Tile38) provide a way to transmit multiple commands to the Redis/Tile38 server in one transmission or in one network call. This saves having to do separate SET's for each Point, etc.
Something like this....
tile38 = Tile38(url="redis://localhost:9851")
pipeline = tile38.pipeline(transaction=False)
pipeline.set("fleet", "truck1").point(52.25,13.37)
pipeline.set("fleet", "truck2").point(58.88,12.39)
pipeline.set("fleet", "truck3").point(50.25,15.50)
pipeline.set("fleet", "truck4").point(57.25,13.37)
pipeline.execute()
Hope this makes it more clear.

Ah, gotcha.
If it's about saving time that's why I choose an async api in the first place.
Given a timer function
import asyncio
import random
import functools
import time
from contextlib import contextmanager

def duration(func):
    @contextmanager
    def wrapping_logic():
        start_ts = time.time()
        yield
        dur = time.time() - start_ts
        print("{} took {} seconds".format(func.__name__, dur))

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        if not asyncio.iscoroutinefunction(func):
            with wrapping_logic():
                return func(*args, **kwargs)
        else:

            async def tmp():
                with wrapping_logic():
                    return await func(*args, **kwargs)

            return tmp()

    return wrapper

OK. This is great. I see how Pyle38 handles this. It is the connection and batch request issue you mention below that I am trying to solve.

Since some limitation on the connection pool to redis prevent me from bombarding Tile38 with 10.000 requests in parallel I batch my SETs in packs of 99. This mean on every execution in the loop I send 99 requests to Tile38 asynchronously.

Here is the aioredis library using a pipeline to issue multiple commands. Tile38 does not have transactions so that would need to be set to False, but I would expect that this code would both improve both the insert speed of your samples above and potentially solve the multiple connection issue.
import asyncio
import aioredis
async def main():
redis = aioredis.from_url("redis://localhost")
await redis.delete("foo", "bar")
async with redis.pipeline(transaction=True) as pipe:
# send your batch of commands (SET's, etc.) inside of the pipe command.
res = await pipe.incr("foo").incr("bar").execute()
# This should be very fast compared to a loop and multiple sets and use only one connection.
print(res)
if name == "main":
asyncio.run(main())

I understand, yet some use cases involve a client issuing one command, others might distribute it across a set of workers.
I think pyle38 sits somewhere in between and can do either or.

Yes, the async nature of Pyle38 is nice in that it allows for some  flexibility on handling these types of situations.  Appreciate the back and forth on this and it probably makes sense to keep the client as is.

From: Ben @.>
Sent: Saturday, November 27, 2021 12:57:26 PM
To: iwpnd/pyle38 @.>
Cc: Brent Tucker @.>; Mention @.>
Subject: Re: [iwpnd/pyle38] Add ability to pipeline set commands (Issue #70)
I understand, yet some use cases involve a client issuing one command, others might distribute it across a set of workers.
I think pyle38 sits somewhere in between and can do either or.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/iwpnd/pyle38/issues/70#issuecomment-980771314, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQTVS632YDJICOESJN3D5JDUOELYNANCNFSM5I3IYQMA.
Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

closing this now, thank you 🙏


Fix ibc-proto recompilation bug

Closes: #801
Description
Added a safety check to prevent overwriting of cosmos-related files.
The problem is described here: https://github.com/informalsystems/ibc-rs/issues/801#issuecomment-814268643
Solution:

introduced separate directories for compiling the SDK files and the IBC-go files, respectively
when copying the generated Rust files into ../proto/..., added a safety check to prevent overwriting of cosmos.* files

if two identical cosmos.* files are generated, only the original (from SDK) is retained
the following warning is emitted in this case:

[warn ] Cosmos-related file exists already ../proto/src/prost/cosmos.upgrade.v1beta1.rs! Ignoring the one generated from IBC-go "/var/folders/sy/207jjj5d7j138ryjs32nbxk00000gn/T/ibc-proto-ibc-go.vsPlSpTwK1aO/cosmos.upgrade.v1beta1.rs"

For contributor use:

[X] Updated the Unreleased section of CHANGELOG.md with the issue.
[ ] If applicable: Unit tests written, added test to CI.
[ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
[ ] Updated relevant documentation (docs/) and code comments.
[ ] Re-reviewed Files changed in the Github PR explorer.

Pushed a couple commits to fix an issue with the Git clone and a typo, looks good otherwise :)


Does not redirect with error

$errors are always returned empty in form_content.blade.php when redirected back.

It's possible that the "web" middleware is registered twice - known Laravel issue.
Check your routes file - if it's in web.php, you shouldn't specify the web middleware anymore.

Yes sir that is exactly the issue. 2 web middlewares.
@tabacitu you're awesome.


Add simulation class

Summary
Create a simulation class which wraps model, discretization, parameters etc.
Motivation
Can be extended to allow linking of the experiment, model, discretization, and solver together and allow each to reference each other. Also simplifies the pybamm user interface so that users only need to see what the actually are setting

See #39 and the branch issue-39-simulation-class - happy for you to take over on this though :)

Ah nice thought you had deleted the branch so just made a quick mock up of what the class would look like. Will take a look at #39 as well.  Doesn't seem like there is too much to do but might be nice to have some reset type features that would allow you to undiscretize your model etc.

Sounds good, note you can do disc.process_model(model, inplace=False) to leave the original model un-discretised


Error: DNS names on TLS certificate not up to date

Describe the bug:
After a fresh k8s cluster re-install, I get a message "DNS names on TLS certificate not up to date: ["xxxxxxx.com"]" when doing a kubectl describe certificate -n cert-manager main-certificate-production. Previous installs where working just fine previously and the only steps I took were to update the DNS zone of my domain to target the new cluster and apply the same k8s config files to setup my cluster and I can't find many resources about this error. Other point is that the service "cert-manager-cainjector-xxxx-xxxx" is restarting on a regular basis (more that 80 times over the week-end) with the error "error running manager: leader election lost", not sure if this is related. I tried a reinstall of the whole helm package without any change.
Here is my certificate.yaml:
apiVersion: certmanager.k8s.io/v1alpha1 kind: Certificate metadata:   name: main-certificate-production   namespace: cert-manager spec:   secretName: main-tls-production   issuerRef:     name: letsencrypt-production     kind: ClusterIssuer   commonName: xxxxxx.com   dnsNames:       - xxx.xx.xxx.xxxx.net       - xxxxxxx.com     acme:     config:       - http01:           ingressClass: nginx         domains:           - xxx.xx.xxx.xxxx.net           - xxxxxxx.com
Expected behaviour:
Certificates should be created as expected.
Steps to reproduce the bug:
There are no specific steps to reproduce this, since this was working working 3 weeks ago (last time I reinstalled the whole cluster).
Anything else we need to know?:
Any clue to what could cause this would be greatly appreciated. Please let me know if more details are required..
Environment details::

Kubernetes version (e.g. v1.10.2): Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.3", GitCommit:"5e53fd6bc17c0dec8434817e69b04a25d8ae0ff0", GitTreeState:"clean", BuildDate:"2019-06-06T01:44:30Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:32:14Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Cloud-provider/provisioner (e.g. GKE, kops AWS, etc): OVH
cert-manager version (e.g. v0.4.0): v0.9.1
Install method (e.g. helm or static manifests): Helm

/kind bug

This message indicates that the currently stored certificate (in the Secret) does not match the Certificate.spec you have specified, and therefore a new certificate will be issued.
Did you Certificate eventually issue okay and this message go away?
/triage support

Hi @munnerz ! I recently added a new subdomain and I got the same issue:
Conditions:
    Last Transition Time:  2019-10-17T16:33:44Z
    Message:               DNS names on TLS certificate not up to date: ["..." "..." "..." "..." "..." "..." "..."]
    Reason:                DoesNotMatch
    Status:                False
    Type:                  Ready
  Not After:               2019-12-29T14:02:08Z
Events:
  Type    Reason   Age   From          Message
  ----    ------   ----  ----          -------
  Normal  Cleanup  15m   cert-manager  Deleting old Order resource "istio-gateway-co-3788450801

No new order and challenge created.
I have to wait until 2019-12-29T14:02:08Z arrives?
There is a way to force update?
Thanks!

I'm having the same issue  – on my new cluster, whenever I remove a domain, I'm getting the same issue as above. And I can't get rid of it without putting the whole cluster in the bin and starting again.

I'm going to close this as there's not something actionable and it saves it going through death-by-bot :)

Weird that the log had HTML that referenced github...

This has now been resolved and we truncate the received text heavily to avoid this 😅


Gas Optimization: Remove call from loop conditional. ParaShareToken.claimTradingProceedsInternal

https://github.com/AugurProject/augur/blob/para_deploys/packages/augur-core/src/contracts/para/ParaShareToken.sol#L334

While you're at it, the tokenId is deconstructed twice in that loop. Tiny amount of gas compared to the loop, but might be worth considering
https://github.com/AugurProject/augur/blob/para_deploys/packages/augur-core/src/contracts/para/ParaShareToken.sol#L335
https://github.com/AugurProject/augur/blob/para_deploys/packages/augur-core/src/contracts/para/ParaShareToken.sol#L342


Support block granularity for QuantizeLinear and DequantizeLinear

Add support for block level granularity in QuantizeLinear and DequantizeLinear.
y_scale and y_zero point are transformed to match the shape of x by applying a unsqueeze->broadcast->reshape chain of transformations.
If the final block of  x is smaller than given block_size the transformed y_scale and y_zero_point are sliced to remove excess elements.
Resolves https://github.com/migraphx-benchmark/AMDMIGraphX/issues/192

Please add some onnx parse tests that show the expected MIGraphX IR output from these changes.

I've added a couple of parse tests.


Inference on GPU Error - [RuntimeError: Native API failed. Native API returns: -5 (PI_ERROR_OUT_OF_RESOURCES) -5 (PI_ERROR_OUT_OF_RESOURCES)]

Hi,
I am trying to run sample mistral on GPU ( Flex 140) and the sample is throwing an error as below
"RuntimeError: Native API failed. Native API returns: -5 (PI_ERROR_OUT_OF_RESOURCES) -5 (PI_ERROR_OUT_OF_RESOURCES)"
Attached screenshot and env-check log

env-check.log

Hi @Mushtaq-BGA
This error log indicates GPU memory is not enough.
To reduce GPU memory usage, we can add an env to turn on kv cache quantization.
export BIGDL_QUANTIZE_KV_CACHE=1

(Optional) Move the embedding layer to CPU with cpu_embedding=True
model = AutoModelForCausalLM.from_pretrained(model_path,
                                             load_in_4bit=True,
                                             optimize_model=True,
                                             trust_remote_code=True,
                                             cpu_embedding=True,
                                             use_cache=True)

The Intel® Data Center GPU Flex 140 accelerator has two GPUs on a single card, supporting heterogeneous vGPU profiles.
https://www.intel.com/content/www/us/en/products/docs/discrete-gpus/data-center-gpu/flex-series/overview.html

Flex 140 has 2 GPUs on a single card. That means it will split 12GB GPU memory into 2 * 6GB. Unfortunately, 6GB is a bit tough for mistral.
On Flex 170, we found that the mistral example requires > 6GB memory for inference.

Hi @Mushtaq-BGA ,
Although one FLEX 140 GPU cannot serve Mistral due to insufficient memory, you can try Deepspeed Tensor Parallelism, which supports large model inference on top of multiple GPUs.
Here is a guide about how to run IPEX-LLM INT4 models with Deepspeed TP.
As FLEX shares a similar architecture with Arc, you can port it to FLEX by using the script run_vicuna_33b_arc_2_card.sh. The model path in the script can be set as needed.
Find me if any issue.


Generator validation loss is significantly different from the sum of all the validation losses

System information

TensorFlow backend (yes / no):  yes
TensorFlow version: 2.2.0
Keras version:  2.3.1
Python version:  3.6.9

Working with a 12 heads output model I noticed that my model checkpoints were not saved by the callback. I then realized that the validation loss wasn't decreasing despite the fact that all the individual validation losses are, I measure a validation loss of 0.345047 while the sum of all the individual losses at 0.662322.
While I expect to have a slight difference due to floating-point operations I wasn't expecting such one. I tried to reproduce the issue and noticed that this happens while using generators and on the validation loss only.
You can reproduce similar results on this notebook, notice the 10^3 difference between the val_diff and the train_diff:

trained by
val_loss
sum_val_loss
loss
sum_train_loss
val_diff
train_diff

array
8.347330
8.347331
8.444574
8.444574
-1.549721e-07
3.886223e-07

generator
8.348998
8.349756
8.442474
8.442474
-0.000758
-7.152557e-08

val_loss is the validation loss computed by keras
sum_val_loss is the sum of all the validation losses for each model's heads
val_diff is the difference between the two
And similarly for the training loss.

This is not a support question, replace keras by tensorflow.keras and the problem disappear.

I have observed similar situation. Moreover, reported validation loss somehow fluctuates while summary validation loss is quite stable. I also used fit_generator.

System information
TensorFlow backend (yes / no): yes
TensorFlow version: 1.15.3
Keras version: 2.3.1
Python version: 3.6.6


SCRAM-SHA-1(-PLUS) + SCRAM-SHA-256(-PLUS) + SCRAM-SHA-512(-PLUS) + SCRAM-SHA3-512(-PLUS) supports

Dear @ldapjs team,
Can you add supports of :

SCRAM-SHA-1
SCRAM-SHA-1-PLUS
SCRAM-SHA-256
SCRAM-SHA-256-PLUS
SCRAM-SHA-512
SCRAM-SHA-512-PLUS
SCRAM-SHA3-512
SCRAM-SHA3-512-PLUS

You can add too:

SCRAM-SHA-224
SCRAM-SHA-224-PLUS
SCRAM-SHA-384
SCRAM-SHA-384-PLUS

SCRAM-SHA-1(-PLUS):

https://tools.ietf.org/html/rfc5802
https://tools.ietf.org/html/rfc6120

SCRAM-SHA-256(-PLUS):

https://tools.ietf.org/html/rfc7677 since 2015-11-02
https://tools.ietf.org/html/rfc8600 since 2019-06-21: https://mailarchive.ietf.org/arch/msg/ietf-announce/suJMmeMhuAOmGn_PJYgX5Vm8lNA

SCRAM-SHA-512(-PLUS):

https://tools.ietf.org/html/draft-melnikov-scram-sha-512

SCRAM-SHA3-512(-PLUS):

https://tools.ietf.org/html/draft-melnikov-scram-sha3-512

-PLUS variants:

RFC5056: On the Use of Channel Bindings to Secure Channels: https://tools.ietf.org/html/rfc5056
RFC5929: Channel Bindings for TLS: https://tools.ietf.org/html/rfc5929
Channel-Binding Types: https://www.iana.org/assignments/channel-binding-types/channel-binding-types.xhtml
RFC9266: Channel Bindings for TLS 1.3: https://tools.ietf.org/html/rfc9266

IMAP:

RFC9051: Internet Message Access Protocol (IMAP) - Version 4rev2: https://tools.ietf.org/html/rfc9051

LDAP:

RFC5803: Lightweight Directory Access Protocol (LDAP) Schema for Storing Salted: Challenge Response Authentication Mechanism (SCRAM) Secrets: https://tools.ietf.org/html/rfc5803

HTTP:

RFC7804: Salted Challenge Response HTTP Authentication Mechanism: https://tools.ietf.org/html/rfc7804

2FA:

Extensions to Salted Challenge Response (SCRAM) for 2 factor authentication: https://tools.ietf.org/html/draft-melnikov-scram-2fa

IANA:

Simple Authentication and Security Layer (SASL) Mechanisms: https://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml

Linked to:

https://github.com/scram-sasl/info/issues/1

Can you add supports of

You can add too

If there are features you require then you are welcome to submit pull requests to add them.

Closing due to lack of response. Feel free to reopen if this is something you would like to work on and will be active on it.

@jsumners: I have not possibility to reopen, can you do it?
The support is important for security:

https://github.com/scram-sasl/info/issues/1
https://www.google.com/search?q=ldap+scram

Thanks in advance.

Are you going to work on this?


Error code: -18; INVALID_LINK_NAME

Describe the bug
Traceback:
line 133, in await_callback, "C:\Users\PingHsunTsai\AppData\Roaming\McNeel\Rhinoceros\6.0\scripts\compas\utilities\azync.py"
line 77, in forward_kinematics, "C:\Users\PingHsunTsai\AppData\Roaming\McNeel\Rhinoceros\6.0\scripts\compas_fab\backends\ros\backend_features\move_it_forward_kinematics.py"
line 20, in call, "C:\Users\PingHsunTsai\AppData\Roaming\McNeel\Rhinoceros\6.0\scripts\compas_fab\backends\interfaces\backend_features.py"
line 31, in forward_kinematics, "C:\Users\PingHsunTsai\AppData\Roaming\McNeel\Rhinoceros\6.0\scripts\compas_fab\backends\ros\planner.py"
line 31, in forward_kinematics, "C:\Users\PingHsunTsai\AppData\Roaming\McNeel\Rhinoceros\6.0\scripts\compas_fab\backends\interfaces\client.py"
line 1283, in forward_kinematics, "C:\Users\PingHsunTsai\AppData\Roaming\McNeel\Rhinoceros\6.0\scripts\compas_fab\robots\robot.py"
line 29, in script
To Reproduce
Steps to reproduce the behavior:

Rhino 6, Grasshopper
See error:

Expected behavior

call forward_kinematics() successfully

Desktop:

OS: Windows 10
Python 3.7
Python package manager: conda

Additional context
Using the ur_robot_driver with real hardware (with move_it package)

Could you please try the following:

print and paste in this issue the result of robot.info()
try adding this line after line 23: configuration.joint_names = robot.get_configurable_joint_names()

Also, I'm assuming you have the latest compas_fab version, if that's not the case, please post the version info of compas and compas_fab.

Sure!

print and paste in this issue the result of robot.info()

0. The robot's name is 'ur10'.
1. The planning groups are: ['endeffector', 'manipulator']
2. The main planning group is 'manipulator'.
3. The end-effector's name is 'ee_link'.
4. The robot has NO tool attached.
5. The base link's name is 'base_link'
6. The base_frame is: Frame(Point(0.000, 0.000, 0.000), Vector(1.000, 0.000, 0.000), Vector(0.000, 1.000, 0.000))
7. The robot's joints are:
8. 	* 'shoulder_pan_joint' is of type 'revolute' and has limits [6.283, -6.283]
9. 	* 'shoulder_lift_joint' is of type 'revolute' and has limits [6.283, -6.283]
10. 	* 'elbow_joint' is of type 'revolute' and has limits [6.283, -6.283]
11. 	* 'wrist_1_joint' is of type 'revolute' and has limits [6.283, -6.283]
12. 	* 'wrist_2_joint' is of type 'revolute' and has limits [6.283, -6.283]
13. 	* 'wrist_3_joint' is of type 'revolute' and has limits [6.283, -6.283]
14. The robot's links are:
15. ['base_link', 'shoulder_link', 'upper_arm_link', 'forearm_link', 'wrist_1_link', 'wrist_2_link', 'wrist_3_link', 'ee_link', 'base', 'tool0', 'world']

try adding this line after line 23: configuration.joint_names = robot.get_configurable_joint_names()

And the versions:

compas_fab 0.17.2
compas 0.13.1

The call to FK tries to figure out which link you want to get fk for, which by default it should be ee_link, but maybe something in your setup is slightly different. Try to specify it explicitly:
robot.forward_kinematics(configuration, group, options=dict(link='ee_link'))

Also, just to make sure, group is set to manipulator, right?

@Gonzalo so after going back to the calibration_devel branch and running:
roslaunch ur_robot_driver ur10_bringup.launch limited:=true robot_ip:=<IP_ADDRESS>
roslaunch ur10_moveit_config ur10_moveit_planning_execution.launch limited:=true
roslaunch ur10_moveit_config moveit_rviz.launch config:=true
With this way of loading the robot model:

We have a different error in the artist (but the previous problem seems to be gone!)
But the good news is that we can execute a trajectory with the real hardware :)
if we can figure out this problem with artist, maybe it`s good to proceed like this (without 2 clients?)

As discussed over slack, this last error is eliminated when load_geometry is set to True in load_robot.


NameError: name 'SequenceClassifierOutput' is not defined

While trying to run the file BERT_explainability.ipynb, I am getting this error.
# encode a sentence
text_batch = ["This movie was the best movie I have ever seen! some scenes were ridiculous, but acting was great."]
encoding = tokenizer(text_batch, return_tensors='pt')
input_ids = encoding['input_ids'].to("cuda")
attention_mask = encoding['attention_mask'].to("cuda")

# true class is positive - 1
true_class = 1

# generate an explanation for the input
expl = explanations.generate_LRP(input_ids=input_ids, attention_mask=attention_mask, start_layer=0)[0]
# normalize scores
expl = (expl - expl.min()) / (expl.max() - expl.min())

# get the model classification
output = torch.nn.functional.softmax(model(input_ids=input_ids, attention_mask=attention_mask)[0], dim=-1)
classification = output.argmax(dim=-1).item()
# get class name
class_name = classifications[classification]
# if the classification is negative, higher explanation scores are more negative
# flip for visualization
if class_name == "NEGATIVE":
  expl *= (-1)

tokens = tokenizer.convert_ids_to_tokens(input_ids.flatten())
print([(tokens[i], expl[i].item()) for i in range(len(tokens))])
vis_data_records = [visualization.VisualizationDataRecord(
                                expl,
                                output[0][classification],
                                classification,
                                true_class,
                                true_class,
                                1,
                                tokens,
                                1)]
visualization.visualize_text(vis_data_records)

Generated Error
NameError                                 Traceback (most recent call last)
/home/cvpr/Transformer-Explainability/BERT_explainability.ipynb Cell 15' in <cell line: 11>()
8 true_class = 1
10 # generate an explanation for the input
---> 11 expl = explanations.generate_LRP(input_ids=input_ids, attention_mask=attention_mask, start_layer=0)[0]
12 # normalize scores
13 expl = (expl - expl.min()) / (expl.max() - expl.min())
File ~/Transformer-Explainability/BERT_explainability/modules/BERT/ExplanationGenerator.py:30, in Generator.generate_LRP(self, input_ids, attention_mask, index, start_layer)
[28](file:///home/cvpr/Transformer-Explainability/BERT_explainability/modules/BERT/ExplanationGenerator.py?line=27) def generate_LRP(self, input_ids, attention_mask,
[29](file:///home/cvpr/Transformer-Explainability/BERT_explainability/modules/BERT/ExplanationGenerator.py?line=28)                  index=None, start_layer=11):
---> [30](file:///home/cvpr/Transformer-Explainability/BERT_explainability/modules/BERT/ExplanationGenerator.py?line=29)     output = self.model(input_ids=input_ids, attention_mask=attention_mask)[0]
[31](file:///home/cvpr/Transformer-Explainability/BERT_explainability/modules/BERT/ExplanationGenerator.py?line=30)     kwargs = {"alpha": 1}
[33](file:///home/cvpr/Transformer-Explainability/BERT_explainability/modules/BERT/ExplanationGenerator.py?line=32)     if index == None:
File ~/miniconda3/envs/maskrcnn/lib/python3.8/site-packages/torch/nn/modules/module.py:1110, in Module._call_impl(self, *input, **kwargs)
[1106](file:///home/cvpr/miniconda3/envs/maskrcnn/lib/python3.8/site-packages/torch/nn/modules/module.py?line=1105) # If we don't have any hooks, we want to skip the rest of the logic in
[1107](file:///home/cvpr/miniconda3/envs/maskrcnn/lib/python3.8/site-packages/torch/nn/modules/module.py?line=1106) # this function, and just call forward.
[1108](file:///home/cvpr/miniconda3/envs/maskrcnn/lib/python3.8/site-packages/torch/nn/modules/module.py?line=1107) if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
[1109](file:///home/cvpr/miniconda3/envs/maskrcnn/lib/python3.8/site-packages/torch/nn/modules/module.py?line=1108)         or _global_forward_hooks or _global_forward_pre_hooks):
-> [1110](file:///home/cvpr/miniconda3/envs/maskrcnn/lib/python3.8/site-packages/torch/nn/modules/module.py?line=1109)     return forward_call(*input, **kwargs)
[1111](file:///home/cvpr/miniconda3/envs/maskrcnn/lib/python3.8/site-packages/torch/nn/modules/module.py?line=1110) # Do not call functions when jit is used
[1112](file:///home/cvpr/miniconda3/envs/maskrcnn/lib/python3.8/site-packages/torch/nn/modules/module.py?line=1111) full_backward_hooks, non_full_backward_hooks = [], []
File ~/Transformer-Explainability/BERT_explainability/modules/BERT/BertForSequenceClassification.py:76, in BertForSequenceClassification.forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, labels, output_attentions, output_hidden_states, return_dict)
[73](file:///home/cvpr/Transformer-Explainability/BERT_explainability/modules/BERT/BertForSequenceClassification.py?line=72)     output = (logits,) + outputs[2:]
[74](file:///home/cvpr/Transformer-Explainability/BERT_explainability/modules/BERT/BertForSequenceClassification.py?line=73)     return ((loss,) + output) if loss is not None else output
---> [76](file:///home/cvpr/Transformer-Explainability/BERT_explainability/modules/BERT/BertForSequenceClassification.py?line=75) return SequenceClassifierOutput(
[77](file:///home/cvpr/Transformer-Explainability/BERT_explainability/modules/BERT/BertForSequenceClassification.py?line=76)     loss=loss,
[78](file:///home/cvpr/Transformer-Explainability/BERT_explainability/modules/BERT/BertForSequenceClassification.py?line=77)     logits=logits,
[79](file:///home/cvpr/Transformer-Explainability/BERT_explainability/modules/BERT/BertForSequenceClassification.py?line=78)     hidden_states=outputs.hidden_states,
[80](file:///home/cvpr/Transformer-Explainability/BERT_explainability/modules/BERT/BertForSequenceClassification.py?line=79)     attentions=outputs.attentions,
[81](file:///home/cvpr/Transformer-Explainability/BERT_explainability/modules/BERT/BertForSequenceClassification.py?line=80) )
NameError: name 'SequenceClassifierOutput' is not defined
@hila-chefer @shirgur  Please look into this issue.

Hi @neeraj1909, thanks for your interest!
I was unable to reproduce the issue your are describing, but it seems like maybe your versions are not up to date with our requirements.
I updated our colab due to another issue, please try to run it now and see if it works.
Best,
Hila.

Hi @hila-chefer, i still get the same error though i have updated your requirement.

@hila-chefer I am also getting the same error :(

@hila-chefer @leeeenammmmm @neeraj1909
Fixed it. Add the following to the top of BertForSequenceClassification.py:

from transformers.modeling_outputs import SequenceClassifierOutput


Default behavior of including Standard Fields in ApplicationSObjectSelector tries to add inaccessible fields

This is more of a question than anything but what is the reasoning behind having getStandardFields()? I'm experiencing fflib_SecurityUtils exceptions due to some of the standard fields not being readable when it comes time to execute the query. It seems to me that it is making the assumption that users will have access to all standard fields for any object they have read access to, but I'm not sure if I'm missing something. At the very least, should it be its own thing separate from includeFieldSetFields since it's not even a field set?
If I do understand the problem correctly, a solution I'd like to propose is -if enforcingFLS- ignore fields that are inaccessible.
            Schema.DescribeFieldResult fieldDescribe = field.getDescribe();

            if ( ! fieldDescribe.isCustom() )
            {
                if ( fieldDescribe.isAccessible() || ! isEnforcingFLS() )
                {
                    standardFields.add( field );
                    system.debug( LoggingLevel.FINEST, 'Standard field : ' + field + ' added');
                }
                else
                {
                    system.debug( LoggingLevel.FINEST, 'Standard field : ' + field + ' NOT added');
                }
            }

@jonnypetraglia Can you share your selector class implementation? Sorry, I know this is not a direct answer but want to understand your implementation before I comment further.

Thanks for the response. As you'll see it's about as barebones as it can be. For completeness sake I will attach the atdx_AppSObjectSelector but I just forked that recently so it should be unchanged aside from namespacing.
public class GHExample extends atdx_AppSObjectSelector{
    public static GHExample newInstance(){
        return (GHExample) atdx_App.Selector.newInstance(Account.SObjectType);
    }
    public Schema.SObjectType getSObjectType(){
        return Account.sObjectType;
    }
    public List<Schema.SObjectField> getAdditionalSObjectFieldList(){
        return new List<Schema.SObjectField> {
            Account.Id,
            Account.Name
        };
    }
    public GHExample(){
        this(true, true, true);
    }
    public GHExample(Boolean includeFieldSetFields, Boolean enforceCRUD, Boolean enforceFLS){
        super(includeFieldSetFields, enforceCRUD, enforceFLS);
    }
    public List<Account> selectByName(Set<String> lookupValues){
        fflib_QueryFactory queryFactory = newQueryFactory(true);
        return (List<Account>)Database.query(queryFactory.setCondition('Name IN :lookupValues').toSOQL());
    }
}

If I try to execute (new GHExample()).selectByName(new Set<String>{'Ahhhhhhhh'}); I get an unhandled Exception:
"Line: 261, Column: 1 - fflib_SecurityUtils.FlsException: You do not have permission to read the field Fax on Account"
Please note that this has no FieldSet inclusions. Fax is referenced nowhere in the entire package, but AT4DX presumes I should have access to it and all standard fields.
For my own curiosity, what's the reasoning behind including standard fields? You could just as easily do it for all fields on the SObject for every Selector implementation instead of specifying Fields and FieldSets. I thought that the reason for the latter was for performance reasons but pulling in Standard Fields seems to contradict that. And while it doesn't happen often, it seems like this would break horribly if a new Standard Field was added on an Object without permissions for every user.

Originally, the reasoning was to prevent the boilerplate to include the standard fields for every object. Thinking more along the lines of, why wouldn't we just include Id, CreatedDate, LastModifiedBy, etc as part of the selector by default. This line of thinking was interesting, but as you have found, ran into the permissions issues and other associated problems.
John or John can jump in and probably provide you more historical context as to why the functionality is still present/plans to improve it in the future, but I can provide you the "workaround" to avoid the issue entirely.
public override List<Schema.SObjectField> getSObjectFieldList() {
     return new List<Schema.SObjectField>{
         Account.Id,
         Account.Name
    };
}

private List<Schema.SObjectField> getAdditionalSObjectFieldList() {
    return new List<Schema.SObjectField>{};
}

This will override the virtual getSObjectFieldList method and completely negate all of the code that pulls in the standard fields. You can then flip the getAdditionalSObjectFieldList method to just return an empty list of Schema.SObjectFields, and you should be good to go, never worrying about standard fields being pulled in again.

Ah, that makes a ton of sense, thanks for the explanation.
The workaround works for sure but requires putting it in in every selector implementation across multiple packages. I think the original code block solution I posted (adding a check for FLS) is the way I'll deal with it for now. That implementation also makes more sense to me as it is not a breaking change; if anyone was depending on Standard Fields being pulled in before nothing will change. But I very much don't want to have any difference from upstream so I'm still invested in finding a solution for this repo, if I can help.


Define personas for the OSPO Career Path (Audience)

A simple division can be:

(1) Open Source /OSPO managers shaping the process within an organization
(2) Open Source /OSPO contributors within and organization (code and non-code contributors)
(3) Open Source /OSPO managers executing the process

I would focus on (1) and (2) as the main personas from the point of view of TODO group trainings. (3) will very much depend on the constraints, culture, practices within the specific organization where the process is executed, so a generic training will only be of limited use.
There could be another persona:
(4) Software staff within an organization which is involved in consuming open source software
This will have generic needs, but there also will be a very organization specific part. There probably also is a transition path between (4) and (2) as organizations should encourage consumers of open source software to also become contributors.


Review "local" API acceptance tests in OCIS and reva

There are "local" acceptance tests in:
https://github.com/owncloud/ocis/tree/master/tests/acceptance/features
https://github.com/cs3org/reva/tree/master/tests/acceptance/features
Some of those are to demonstrate good expected behavior of things that are not features of the oC10 core API acceptance test suite. That is good - review those just to know that they still seem useful.
Some of them are "bug demo" scenarios that demonstrate existing "bad behavior". Nowadays we have expected-failure files, so we already know if a developer improves behavior and some core API test scenarios start to pass. Maybe we don't need these "bug-demo" scenarios any more. Review them and decide what to do with them.

@phil-davis I think we can close this issue now

Agree - closed


Dark Asuna, ZeroTwo, and Mai Sakurajima Combined Theme

Who/What do you want to be featured?
Mai Sakurajima, Asuna, and Zerotwo
What do they look like?
There are already images for all of them xD. Just want a combined version :p
What type of theme?

[ ] Dark Theme

Additional Information (Optional)
This deserves to be included simply because these are the top 3 waifus and when you combine all three of them you become the ultimate otaku programmer in which you can code anything in as little or long of time you want becasue you have WAIFU POWER!!!!!

:thinking: It never occurred to me to ever combine Waifu for a collab theme. It's an interesting idea, but there are a ton of combinations! I don't even know that the look and feel/syntax coloring would look like at the moment.

these are the top 3 waifus

I see you fam, and that's just like your opinion. You did included ZeroTwo, so it's a good opinion might I add.
What plugin do you use, VS Code (I think you where the one who suggested dark asuna, right? Think you had the Mai handgun edit profile pic) If that's the case, I think after I get done working on v11, I can add something to VSCode plugin that allows you to set your own custom background/sticker. That way you can create your own image with your waifu ordered how you please. I can also provide the clean assets of your waifu (no outlines/background color) if wanted.
This way you can level up your weeb game sooner than later. Because I don't think I'll have on official collab theme, but I'll think about it.

Yes, I was the one that had suggested Asuna Dark! Also, I love your chrome, github, vscode, and hyper themes! I would like to recommend a repl.it doki theme (if you have time) as well!
So, if you were to add a custom background (and maybe sticker) plugin along with the background assets you use I would be delighted and I'm sure everyone else would too.
While I'm here I'd like to ask for your help with modifying the style and getting a bit doki on an online wrapper (code.org) which I use a lot. If there's anyway I can message you and ask for your help (discord?) without taking up any of your personal time I'd be delighted to have such to help with issues and such that I face.

I would like to recommend a repl.it doki theme

Noted. However, I think I'll be working on Visual Studio after I am done with adding this round of Waifu at the moment.

While I'm here I'd like to ask for your help with modifying the style and getting a bit doki on an online wrapper (code.org) which I use a lot.

I'm assuming your just wanting to use Stylus & usercss to customize the look an feel right? If so are you trying to use the colors provided by this repo? (if so I can show you my build tools n stuff if that's the case). I'd be happy to help.  However, I'd like to get done with this round of themes, but you can shoot me a discord message (unthrottled#0197) and I'll check discord every now and then.

I will message you on discord with more info! Thank you so much I really appreciate the help and theme! Much love <3


⚠ Mobitopup has degraded performance

In 82cf76d, Mobitopup (https://napcard.voz.tech) experienced degraded performance:

HTTP code: 200
Response time: 7979 ms

Resolved: Mobitopup performance has improved in f0599dc.


WIP: Add release lifecycle for desktop app

Requested by Customer Engineers https://community.mattermost.com/private-core/pl/ju5fj3fwyifjin1khw9aippo1h.
Open for feedback, just let me know if other details should be added here or changed.

Closing for now in favor of the google doc https://docs.google.com/document/d/1gqM-llT_NokjFDOoShLWgcIrXhZbvQRbRc07gqu5Qkc/edit#heading=h.20j1cc4yxzha


Firefox Freeverb feedback

If you try bus demo in Firefox (51.0.1)  and apply some freeverb then trigger note. Note will be played with reverb, but after few seconds a super loud crazy feedback will rip your ears.
If this cannot be fixed, please disable this functionality in firefox, or put up some warning, it's really painful.

wow! That is ear splitting. I will definitely take care of that before the next release.
I'm trying to figure out a way to implement the LowpassFeedbackCombFilter with the IIRFilter. It seems like it'd be pretty straight forward, but I'm not sure on the exact math of the transfer function coefficients... maybe someone who knows more about filter math can help me out?

@tambien --
I couldn't quite get it to work with the rest of the Freeverb code
(I may try again later),
but in case it's useful in the meantime,
here's my implementation of your LowpassFeedbackCombFilter using IIRFilter:
https://jsfiddle.net/softpunch/wnotL5ep/
I'm no good at higher level math myself,
but @mohayonao's filter coefficients make it easier:
https://github.com/mohayonao/biquad-coeffs
https://mohayonao.github.io/biquad-coeffs
You might also find these tools useful:
http://www.earlevel.com/main/2010/12/20/biquad-calculator/
http://rtoy.github.io/webaudio-hacks/more/filter-design/filter-design.html

@softpunch Those values are not supposed to be at the current sample rate. instead they match the values of the Freeverb implementation values that i found here. I divide by the original sample rate to convert it to seconds which allows it to be used at any sample rate.

Got it!

@softpunch thanks for looking more into the LPCF. I was hoping that it'd be possible to implement it entirely using the new IIRFilter, but it doesn't support adequate lengths to make the filter entirely using the IIR. My only other option is to use the Audio Worker once that's available.

So close but so far...


Add buildAutoChooserWithOptionsModifier

I created a new method called buildAutoChooserWithOptionsModifier, that accepts a second parameter in the form of a lambda function that takes in a stream and returns a stream of PathPlannerAutos.
I also added documentation to filter these options based on if they included the text "comp" at the beginning of the name. Some other examples you could do would be override the name with .setName, filter depending on if you're red or blue, or use .sorted() on the stream to sort them arbitrarily before going on shuffle board.
I couldn't run spotless for some weird errors in my environment I need to figure out but the formatting should be good enough. I also manually wrote that documentation so no guarantees it actually works with Writerside 🤷.
P.S. pathplanners great, thanks for all the work you do :)

/format

Well format bot is broken so that's unfortunate

/format

It looks like the format bot is trying to branch origin/main and doesn't know how to merge it up onto the pr... not exactly sure how thats supposed to work.

It looks like the format bot is trying to branch origin/main and doesn't know how to merge it up onto the pr... not exactly sure how thats supposed to work.

Yeah. It seems to work for some people but not others for whatever reason.

That workflow dispatch change wouldn't work since the workflow is very specific to comments

still didn't work damn, lemme revert those didn't realize they would get put in this pr immediately

@mjansen4857 any idea when a release will be cut with this change in it? Thanks!


ENH: only let channels which have a name into read/config list

Codecov Report

Merging #509 into master will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #509      +/-   ##
==========================================
+ Coverage   89.42%   89.44%   +0.02%
==========================================
  Files          51       51
  Lines        7016     7049      +33
==========================================
+ Hits         6274     6305      +31
- Misses        742      744       +2

Impacted Files
Coverage Δ

ophyd/tests/test_signal.py
98.26% <0%> (-0.35%)
:arrow_down:

ophyd/status.py
92.27% <0%> (+0.68%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3fe913f...fb3ff00. Read the comment docs.


Adding an option to display FloatingMenu at the bottom of the FAB and not at the top.

I haven't seen any mention of this case in the guidelines, or in a Google app, but would be nice if the menu could open at the bottom of the primary FAB.
In some cases, the FAB is in the upper part of the screen , and the Menu items needs to be below, which is I think not supported by your lib (yet?)
Will try to find an exemple or a picture and add it here, but I hope you understand what I mean.
Thanks for your work!

Added to my ToDo list.

Dear Clans, how is the status of this functionality? Since your library is very well in-nested with my app already I would prefer to keep this hard work and not changing library.
I have looked into your code, I think the necessary changes are in onMeasure and in onLayout, but I tried without success to do it myself, even if something I was able to add. Like some new attrs and doing some simple calculations, but I am not very good in custom Layouts ;-) especially if it's not my original code.
I will be waiting with impatience this your new functionality.
Thanks for you good work.

Currently working on a new functionality. Will do my best to implement your request asap.

That and the progress bar new features are really great. Magnificent work, I think your library is the best, very simple and clean.

Thank you!


Pause video and clean up when unmounted

If you remove the video from a view, it keeps playing in the background. You can hear the audio playing, and it continues to occupy memory. It's also evident if you do a JavaScript reload within the app while you're debugging.
I can implement a quick and dirty fix by adding this to RCTVideo.m:
- (void)removeFromSuperview
{
    [super removeFromSuperview];
    [_player pause];
    [[NSNotificationCenter defaultCenter] removeObserver:self];
}

That stops the video from playing, but I don't believe it's really deleted the object, since dealloc doesn't run and the memory graph continues to climb. So I'm sure there's more work to be done here. I'd offer to make a pull request, but I don't know Objective-C well enough to do it myself.

Thanks for the heads up @brianchirls! This is still very early on in this project so there are bound to be some serious problems such as this, that said I fixed it in https://github.com/brentvatne/react-native-video/commit/d9ddb6e75a99954e2003359f42581d75efb33d71 :smile:

Awesome. Thanks!

@brentvatne okay, as far as I can tell, this is broken again. I have controls which let me navigate to the "next" video in a queue, and the old one keeps playing.

iOS, XCode 9.
  "dependencies": {
    "react": "16.0.0-alpha.12",
    "react-native": "^0.46.1",
    "react-native-orientation": "^2.1.0",
    "react-native-sentry": "^0.16.1",
    "react-native-vector-icons": "^4.4.0",
    "react-native-video": "^1.2.0",
    "react-navigation": "^1.0.0-beta.11"
  }

I have the same issue like @whistlerbrk, have anyone found any solution to this?

react-native: "0.61"
react-navigation: "4.0.10"

It is partly solved by this solution https://github.com/react-native-community/react-native-video/issues/1223#issuecomment-423787629, but it won't work on iOS with back swipe, which I described here https://github.com/react-native-community/react-native-video/issues/1223#issuecomment-547379209


[SUPPORT] [WIP] [PLEASE DO NOT TRIAGE] Errors ingesting topics containing "enum" data types

Describe the problem you faced
We are using the DeltaStreamer on EMR 6.8.0, sourcing data from Confluent Kafka Avro topics and using our Confluent Schema Registry to deserialize the messages, which we write to the Glue Data Catalog and query with Athena.
For the majority of topics this works well, however, we noticed deserialisation errors when topics have Avro enum types in the schema.
Errors come in two forms, based on whether we use the default KafkaAvroDeserializer, or the KafkaAvroSchemaDeserializer

Scala.MatchError

Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 4) (ip-10-154-13-123.eu-west-1.compute.internal executor 1): scala.MatchError: processing (of class org.apache.avro.generic.GenericData$EnumSymbol)
	at org.apache.hudi.org.apache.spark.sql.avro.AvroDeserializer.$anonfun$newWriter$13(AvroDeserializer.scala:178)
	at org.apache.hudi.org.apache.spark.sql.avro.AvroDeserializer.$anonfun$newWriter$13$adapted(AvroDeserializer.scala:177)
	at org.apache.hudi.org.apache.spark.sql.avro.AvroDeserializer.$anonfun$getRecordWriter$2(AvroDeserializer.scala:379)
	at org.apache.hudi.org.apache.spark.sql.avro.AvroDeserializer.$anonfun$getRecordWriter$2$adapted(AvroDeserializer.scala:375)
	at org.apache.hudi.org.apache.spark.sql.avro.AvroDeserializer.$anonfun$getRecordWriter$3(AvroDeserializer.scala:389)
	at org.apache.hudi.org.apache.spark.sql.avro.AvroDeserializer.$anonfun$getRecordWriter$3$adapted(AvroDeserializer.scala:385)
	at org.apache.hudi.org.apache.spark.sql.avro.AvroDeserializer.$anonfun$converter$4(AvroDeserializer.scala:87)
	at org.apache.hudi.org.apache.spark.sql.avro.AvroDeserializer.deserialize(AvroDeserializer.scala:105)
	at org.apache.hudi.org.apache.spark.sql.avro.HoodieSpark3_3AvroDeserializer.deserialize(HoodieSpark3_3AvroDeserializer.scala:30)

org.apache.avro.AvroTypeException

Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing key/value for partition {topic}-0 at offset 7202. If needed, please seek past the record to continue consumption.
Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id 4144
Caused by: org.apache.avro.AvroTypeException: Found {avro_record_namespace}.{enum_name}, expecting string

for an enum field resembling the following (note that sometimes there are default: added to the field, but never a default: at the symbol level...
{
      "name": "status",
      "type": {
        "type": "enum",
        "name": "status_options",
        "symbols": [
          "processing",
          "completed",
          "error"
        ]
      }
},

With the  KafkaAvroSchemaDeserializer -->  org.apache.avro.AvroTypeException: Found {avro_record_namespace}.status_options, expecting string.
Without the KafkaAvroSchemaDeserializer --> scala.MatchError: {one_of_the_enum_symbols} (of class org.apache.avro.generic.GenericData$EnumSymbol

To Reproduce
Error 1 -- Scala.MatchError
With a duplicate environment to the one mentioned at the beginning, our Spark command is:
"spark-submit",
"--class", "org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer",
"--conf", "spark.scheduler.mode=FAIR",
"--conf", "spark.serializer=org.apache.spark.serializer.KryoSerializer",
"--conf", "spark.sql.catalogImplementation=hive",
"--conf", "spark.sql.hive.convertMetastoreParquet=false",
"--conf", "spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension",
"--conf", "spark.sql.catalog.spark_catalog=org.apache.spark.sql.hudi.catalog.HoodieCatalog",
"--conf", "spark.hadoop.hive.metastore.client.factory.class=com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory",
"--conf", "spark.streaming.kafka.allowNonConsecutiveOffsets=true",
# IMPORTANT: hudi-utilities-bundle must be declared immediately before any Hudi spark commands
"/usr/lib/hudi/hudi-utilities-bundle.jar",
"--source-class", "org.apache.hudi.utilities.sources.{{ source_type }}",
"--source-ordering-field", "{{ timestamp_field }}",
"--table-type", "COPY_ON_WRITE",
"--op", "UPSERT",
"--enable-sync",
"--continuous",
# Hudi write config

"--target-base-path", f"s3://{bucket}/raw/{{ table }}",
"--target-table", "{{ table }}",
"--hoodie-conf", "hoodie.database.name={{ database }}_raw",
"--hoodie-conf", "hoodie.table.name={{ table }}",
"--hoodie-conf", "hoodie.datasource.write.recordkey.field={{ primary_key }}",
"--hoodie-conf", "hoodie.datasource.write.precombine.field={{ timestamp_field }}",
"--hoodie-conf", "hoodie.datasource.write.keygenerator.class=org.apache.hudi.keygen.TimestampBasedKeyGenerator",
"--hoodie-conf", "hoodie.datasource.write.partitionpath.field={{ timestamp_field }}",
"--hoodie-conf", "hoodie.deltastreamer.keygen.timebased.timestamp.type=EPOCHMILLISECONDS",
"--hoodie-conf", "hoodie.deltastreamer.keygen.timebased.output.dateformat=yyyy/MM/dd",
# Filter invalid records
"--transformer-class", "org.apache.hudi.utilities.transform.SqlQueryBasedTransformer",
"--hoodie-conf", "hoodie.deltastreamer.transformer.sql=SELECT * FROM <SRC> WHERE {{ primary_key }} is not null AND {{ timestamp_field }} is not null",
# AWS Glue Data Catalog config
"--hoodie-conf", "hoodie.datasource.hive_sync.enable=true",
"--hoodie-conf", "hoodie.datasource.hive_sync.database={{ database }}_raw",
"--hoodie-conf", "hoodie.datasource.hive_sync.table={{ table }}",
"--hoodie-conf", "hoodie.datasource.hive_sync.partition_fields=_event_date",
"--hoodie-conf", f"hoodie.deltastreamer.source.kafka.topic={self.kafka_topic}",
"--hoodie-conf", "auto.offset.reset=earliest",
"--hoodie-conf", "sasl.mechanism=PLAIN",
"--hoodie-conf", "security.protocol=SASL_SSL",
"--hoodie-conf", f"bootstrap.servers={self.kafka_bootstrap_servers}",
"--hoodie-conf", f'sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required
username="{self.kafka_sasl_username}" password="{self.kafka_sasl_password}";',
"--schemaprovider-class", "org.apache.hudi.utilities.schema.SchemaRegistryProvider",
"--hoodie-conf", f"schema.registry.url=https://{self.schema_registry_url}",
"--hoodie-conf", "basic.auth.credentials.source=USER_INFO",
"--hoodie-conf", f"schema.registry.basic.auth.user.info={self.schema_registry_auth}",
"--hoodie-conf"<EMAIL_ADDRESS>
Most notably, removing the SQL Transformer we are using below seems to remove the error for many of the topics for both error cases.
"--transformer-class", "org.apache.hudi.utilities.transform.SqlQueryBasedTransformer",
"--hoodie-conf", "hoodie.deltastreamer.transformer.sql=SELECT * FROM <SRC> WHERE {{ primary_key }} is not null AND {{ timestamp_field }} is not null",

We are using this transformer to attempt to drop null records from being ingested, to avoid the below error which appears to be caused by empty data in our selected hoodie.datasource.write.partitionpath.field, although we cannot find any evidence of this in our topics:
Caused by: org.apache.hudi.exception.HoodieException: The value of {timestamp_field} can not be null
	at org.apache.hudi.avro.HoodieAvroUtils.getNestedFieldVal(HoodieAvroUtils.java:534)
	at org.apache.hudi.utilities.deltastreamer.DeltaSync.lambda$fetchFromSource$b741bfe4$1(DeltaSync.java:490)
	at org.apache.spark.api.java.JavaPairRDD$.$anonfun$toScalaFunction$1(JavaPairRDD.scala:1070)

Based on the fact that this appears to be causing the errors, there may be a better way of doing this?
One thing to note is that this error only seems to be occurring on topics that do not start at an offset of 0, so this error may be a red herring, or bad data on our end which we don't seem to able to confirm.
Error 2 -- org.apache.avro.AvroTypeException
The only difference in this test setup is the addition of
"--hoodie-conf", "hoodie.deltastreamer.source.kafka.value.deserializer.class=org.apache.hudi.utilities.deser.KafkaAvroSchemaDeserializer".
The error is different, but the fix appears to be removing the SQL transformer, and the working topics from the previous test again work, but the failing ones with The value of {timestamp_field} can not be null remains.
Expected behavior
I expected the enum data types to be able to be ingested, even when including an SQL transformer to only select records that fit the query description, based on this documentation
Answers Required
If the above is an incorrect assumption, the cause is known and this is expected behaviour, then my question becomes "How can I filter out potentially null records that will break the primary key / partition path field handling i'm using for my TimestampBasedKeyGenerator and UPSERT behaviour"
&&
Is there an answer to why I may be receiving this The value of {timestamp_field} can not be null error on seemingly OK topics? The source system apparently guarantees that these fields are always populated, so the only thing I can think is an offset scanning problem as this failing topic is the only topic of the ones I tested that didn't start at an offset 0.
Environment Description

Hudi version : Deltastreamer on EMR 6.8.0 running Hudi 0.11.1-amzn-0
Spark version : 3.3.0
Hive version : 3.1.3
Hadoop version : Amazon 3.2.1
Storage (HDFS/S3/GCS..) : S3
Running on Docker? (yes/no) : No

EMR 6.9.0 that utilizes Hudi 0.12.1 wasn't used due to getting constant org.apache.hudi.exception.HoodieException: Could not sync using the meta sync class org.apache.hudi.hive.HiveSyncTool
Additional context

I have visited the Hudi office hours before writing this issue, where it was suggested that I investigate and write up my findings here if the root cause is still unknown.
If there's any detail missing from these tests please let me know and I can recreate and provide more clarity!
The same configurations (With the default Deserializer, and without the SQL transformer) has been used with "INSERT" mode instead of "UPSERT" mode, with the same outcome

Stacktraces

These 3 traces are all processing the same topic with the different configs discussed, one that throws the The value of {timestamp_field} can not be null error when the transformer is removed.
Before each test or investigation, if s3 data or tables have been populated, I cancelled running EMR jobs, dropped any created tables and deleted underlying S3 data.

Potentially useful information about the topic:

This schema has 5 versions and although the schema registry only enforces BACKWARDS compatibility, I believe the current evolutions to adhere also to BACKWARDS_TRANSITIVE.
The Enum failing in both the Scala Match Error and the org.apache.avro.AvroTypeException  message “status” has been unchanged the entire time, and is the first enum field in the schema
The topic appears to start at offset 7202, which is the offset declared in the org.apache.avro.AvroTypeException stack trace

Using Default Deserializer - Scala Match Error
scala_match_error_stacktrace.txt
Using KafkaAvroSchemaDeserializer -  org.apache.avro.AvroTypeException
avro_type_error_stacktrace.txt
Removing the SQL Transformer  - The value of {timestamp_field} can not be null
timestamp_field_cantbe_null_trace.txt

cc @danielfordfc -> wdym with "please do not triage"? if you can clarify.

I am looking to rewrite this ticket to focus on the SQL Transformer being unable to handle enum data types. The sections in which I discuss The value of {timestamp_field} can not be null error has since been identified as a data error in our topics.
There is a separate question I have as to why the KafkaAvroDeserializer doesn't work but the KafkaAvroSchemaDeserializer one does for certain topics that have undergone an evolution, but I will open up another ticket for this.
Should I edit the ticket to remove the The value of {timestamp_field} can not be null discussion?

Replaced with https://github.com/apache/hudi/issues/8042