[Bug] It is not possible to use a custom command with @milkdown/plugin-listener

Initial checklist

[X] I agree to follow the code of conduct
[X] I searched issues and discussions and couldn’t find anything (or linked relevant results below)

Affected packages and versions
7.1.0
Link to runnable example
No response
Steps to reproduce

Go to https://codesandbox.io/s/smoosh-sunset-ey4uhy?file=/src/EditorWithListener.tsx
Highlight particular text
Click on the Transform to subscript
Error appers

( when you uncomment ctx.get(listenerCtx).markdownUpdated((_, markdown) => {}); you will be able to transform text to subscript )
Expected behavior
I am able to use a custom command when I use @milkdown/plugin-listener
Actual behavior
When I want to use the custom command with @milkdown/plugin-listener I get the error

Cannot handle unknown node `subscript'

( subscript is a custom command )
Runtime
Chrome, Safari, Electron
OS
Windows, Linux, macOS
Build and bundle tools
No response

It's not something about the listener plugin. The problem is that you don't handle the remark AST very well. You're trying to give the subscript as the name of the AST node but remark cannot recognize it. So if you try to transform your prosemirror state into markdown. It will throw the error. The proper way to do that is to add a remark plugin for it.


Refactor guid, message, queue

what do we think about this?
note: currently causes go test to fail.  haven't figured it out yet, wanted to check first if the general idea looks good.

@Dieterbe I appreciate the effort to clean things up.  As per my two comments above, I don't think we would land this.
If you want to get your feet wet with the codebase, might I suggest some of the contributor-friendly issues?

i see. totally get DiskQueue should not deal with message concepts, makes total sense.
(after all i've also yanked out the diskqueue for user in other projects)
the main thing i wanted to achieve is make Guid public so that i could embed nsqd into my app publish messages through the go api instead of tcp, the only thing seemingly needed for that is the Guid stuff.
but i read your comment on the mailing list that you're not a big fan approach, so maybe i ishould just deal with it and use the network api. (through the go-nsq library) though i still think direct calls are a lot more robust.

but i read your comment on the mailing list that you're not a big fan approach, so maybe i ishould just deal with it and use the network api. (through the go-nsq library) though i still think direct calls are a lot more robust.

It's really just that we don't want to have the burden of publicly supporting the in-process API.  Besides, I think it would result in a deployment that is too tightly coupled.  NOTE: You can still generate your own message IDs, without any of the changes in this PR.
Going to close this.
Thanks again!

thank you Matt, for the nice work :)


It's main.js not bundle.js from previous guide

Or you could add the diff from main.js to bundle.js in the getting-started.md


⚠ ClamAV has degraded performance

In c2f2143, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 68 ms

Resolved: ClamAV performance has improved in 818e062.


Upgrade dependencies for react and react-dom to 18

warning " ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>warning " ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>

Completed with version 3.0.3


GestureDetector onPanDown() event

GestureDetector(
    onPanUpdate: (detail) {
      print('first');
   }
   child: GestureDetector(
       onPanUpdate: (detail)  {
           print('second');
       }
  )
)

As my knowlege of GestureArenaManager,
The code above would dispatch the child event first and the parent gesture would be dropped,
but the actual print result is
   'second'
   'first'

This issue only appeared at onPanDown() method as I know, onTap(), onVerticalDragDown(), etc.. don't have this issue.
This issue was first discovered when I wanted to implement a draggable item inside listview, but when I scroll vertically, the draggable item move a little bit and the listview started to scroll. What I expect it is that after dispatch the event to draggable item, the state of event would be removed, it wouldn't pass to their parent, but onPanDown(), onPanUpdate... didn't do that way.
Sample
  ListView(
    controlller:  _controller,
   child: GestureDetector(
       onPanUpdate: (detail)  {
           dragEvent(detail);
       }
  )
)

To be frank, the listview won't be scrolled when I drag the view except vertically.
I think it is related to PanGestureRecognizer, anyone help?
Originally posted by @lau1944 in https://github.com/flutter/flutter/issues/28202#issuecomment-751645873

Hi @lau1944
Can you please provide your flutter doctor -v and a minimal complete reproducible code sample?
Thank you

Hi @lau1944
Can you please provide your flutter doctor -v and a minimal complete reproducible code sample?
Thank you

flutter doctor -v

[✓] Flutter (Channel stable, 1.22.3, on macOS 11.0.1 20B50, locale zh-Hans-CN)
    • Flutter version 1.22.3 at /Users/liuquan/flutter
    • Framework revision 8874f21e79 (9 weeks ago), 2020-10-29 14:14:35 -0700
    • Engine revision a1440ca392
    • Dart version 2.10.3
    • Pub download mirror https://pub.flutter-io.cn
    • Flutter download mirror https://storage.flutter-io.cn

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at /Users/liuquan/Library/Android/sdk
    • Platform android-30, build-tools 30.0.2
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    • CocoaPods version 1.8.4

[✓] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin installed
    • Dart plugin version 193.7547
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)

sample

class HomePage extends StatefulWidget {
  static String path = 'home';

  static Widget routeBuilder(BuildContext context) {
    return HomePage();
  }

  @override
  State<StatefulWidget> createState() => HomePageState();
}

class HomePageState extends State<HomePage> {
  @override
  void initState() {
    super.initState();
  }

  @override
  Widget build(BuildContext context) {
    return ListView(
      children: [
        Container(
          height: 100,
          width: 100,
          child: GestureDetector(
            onPanDown: (detail) {
              print(detail);
            },
          ),
        )
      ],
    );
  }
}

As the code above, when our finger touch inside the container, the listview should not be scrolled.
But when I scroll vertically inside the Container, the listview can still be scrolled.

CC @goderbauer

Hi @lau1944
When we just tap, it's not scrolled, I don't see it being scrolled. Can you please clarify?

when our finger touch inside the container, the listview should not be scrolled.

code sample
import 'package:flutter/material.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Material App',
      theme: ThemeData.dark(),
      home: HomePage(),
    );
  }
}

class HomePage extends StatefulWidget {
  static String path = 'home';

  static Widget routeBuilder(BuildContext context) {
    return HomePage();
  }

  @override
  State<StatefulWidget> createState() => HomePageState();
}

class HomePageState extends State<HomePage> {
  @override
  void initState() {
    super.initState();
  }

  @override
  Widget build(BuildContext context) {
    return ListView(
      children: [
        Container(
          height: 300,
          width: 300,
          child: GestureDetector(
            onPanDown: (detail) {
              print(detail);
            },
            child: FlutterLogo(
              size: 300,
            ),
          ),
        )
      ],
    );
  }
}

flutter doctor -v
[✓] Flutter (Channel stable, 1.22.5, on macOS 11.1 20C69 darwin-x64, locale en-GB)
    • Flutter version 1.22.5 at /Users/tahatesser/Code/flutter_stable
    • Framework revision<PHONE_NUMBER> (3 weeks ago), 2020-12-10 11:54:40 -0800
    • Engine revision ae90085a84
    • Dart version 2.10.4

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Volumes/Extreme/sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_HOME = /Volumes/Extreme/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Volumes/Extreme/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    • CocoaPods version 1.10.0

[!] Android Studio (version 4.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)

[✓] VS Code (version 1.52.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.18.0

[✓] Connected device (2 available)
    • Taha’s iPhone (mobile) • 00008020-001059882212002E            • ios • iOS 14.3
    • iPhone 12 (mobile)     • 12D8FF8E-2815-436E-9951-B8A6A42E4ACF • ios • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)

! Doctor found issues in 1 category.

Hi @lau1944
When we just tap, it's not scrolled, I don't see it being scrolled. Can you please clarify?

when our finger touch inside the container, the listview should not be scrolled.

code sample
flutter doctor -v

Sry, I didn't explain clearly,
'When we just tap'  I mean drag vertically (Set the Container size to be bigger so it can test clearly)
, the item's gestureRecognizer and the listview is conflicted.

Hi @lau1944
When we just tap, it's not scrolled, I don't see it being scrolled. Can you please clarify?

when our finger touch inside the container, the listview should not be scrolled.

code sample
flutter doctor -v

Just to be more clear,
I create my a customPanGestureRecognizer which extends PanGestureRecognizer,
class CustomPanGestureRecognizer extends PanGestureRecognizer {
   @override
   void rejectGesture(int pointer) {
       print(pointer);
   }
}

Which is found out, the rejectGesture method was called after the listview was scrolled, and cause the draggable items became undraggable..

Hi @lau1944
Can you please complete the code sample and steps tor reproduce with expected results and actual results for more clarity
Thank you

Hi @lau1944
Can you please complete the code sample and steps tor reproduce with expected results and actual results for more clarity
Thank you

In my project, I implement a custom draggable view myself, so it would be much confusing if I paste the whole code. But I simplify the problem below.

import 'package:flutter/cupertino.dart';

class Test extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Container(
      child: ListView(
        children: [
          Container( // a draggable zone
            height: 200,
            child: Container( // assume it's a draggable item inside the zone
              height: 40,
              width: 40,
              child: GestureDetector(
                onPanDown: (detail) {
                  print(detail);
                  // _handleDownEvent(detail);
                },
                onPanUpdate: (detail) {
                  print(detail);
                  // _handleUpdateEvent(detail);
                },
                onPanEnd: (detail) {
                  // _handleFinishEvent(detail);
                },
              ),
            ),
          )
        ],
      ),
    );
  }
}

Draggable zone is part of the item of listview.
A draggable item is inside a draggable zone, we need the draggable item can be dragged freely inside the draggable zone.
Which mean that  onPanUpdate method would always be print() while we are dragging.
However, everything works until we drag it vertically after our finger touches down, the listView is scrolled,
onPanUpdate  would not print() anything.
** This happened is because the GestureRecognize of listView and the item is conflicted. The GestureArenaManager chose listView as winner and the draggable item is called rejectGesture.
What we want is, inside the draggable zone, we can drag anyway we want, the listView gesture event should not be dispatch while we are dragging.
That means, no matter what, onPanUpdate should always be print().

Hi @lau1944
Thanks for the details
Looks similar to https://github.com/flutter/flutter/issues/72996, which also suggest using this workaround
I can reproduce the behavior
Closing in favor of https://github.com/flutter/flutter/issues/73039
Please follow up on that issue, I'm closing the current one as a duplicate.
If you disagree, please write in the comments and I will reopen it.
Thank you


More rust runtime tests for resources

This PR adds more rust runtime tests for Resources. These WITs and tests are moved from https://github.com/bytecodealliance/componentize-js/pull/65 and translated into Rust.
A few things that I found difficult to do in Rust compared to Js or Python are

Consuming owned resource in the guest code. This is also described in #641. I had to use Option::take to consume the owned type underneath the Resource.
Consuming Resource<T> in the host code.
Generated modules tend to have long import names. E.g. use exports::test::resource_borrow_in_record::test::{Guest, GuestThing, OwnThing};
Question: How do I check if two ResourceAny are the same?

Thanks @dicej , @alexcrichton for help!

Does == work?

What I want to test is a deep equal of two ResourceAnys
let res = instance
      .test_resource_alias_e1()
      .call_a(&mut *store, foo_e1)?;

// TODO: how do I test deep equal of ResourceAny type?
assert_eq!(
    res,
    vec![instance
        .test_resource_alias_e1()
        .x()
        .call_constructor(&mut *store, 42)?]
);

What I got is
thread 'resource_alias::run' panicked at 'assertion failed: `(left == right)`
  left: `[ResourceAny { idx: 0, ty: ResourceType { kind: Guest { store: StoreId(1), instance:<PHONE_NUMBER>, id: DefinedResourceIndex(0) } }, own_state: Some(OwnState { store: StoreId(1) }) }]`,
 right: `[ResourceAny { idx: 1, ty: ResourceType { kind: Guest { store: StoreId(1), instance:<PHONE_NUMBER>, id: DefinedResourceIndex(0) } }, own_state: Some(OwnState { store: StoreId(1) }) }]`'

Does it make sense to define PartialEq of ResourceAny by comparing their own_state equality?

Can you gist the guest and the WITs involved?


feat: implement renovate logic

Signed-off-by: Kevin Lefevre<EMAIL_ADDRESS>Implement renovate
Description
This PR enables renovate and allow updating of :

pre-commit
helmv3 charts
github-actions Docker images

It also implements a helm-dependencies.yaml file which centralized all the chart used in this repository. Addons that depends on Helm charts use this file to know which default version to deploy. With this, the charts will alsways be up to date no matter the module. So no more updating manually and syncing between folders.
Checklist

[X] CI tests are passing
[X] README.md has been updated after any changes to variables and outputs. See https://github.com/particuleio/terraform-kubernetes-addons/#doc-generation

Here you can see an exemple of it working in my fork: https://github.com/ArchiFleKs/terraform-kubernetes-addons/pulls


Why I speend a whole night to load and initial the 70G file "Mandarin LM Large", it is not done yet ??

Hello, guys:
Now i want to to load the file "Mandarin LM Large" which is 70G size for ASR, but I speed the whole last night to init it, it has not done yet, Here is my load info output:
-----------  Configuration Arguments -----------
alpha: 1.15
beam_size: 500
beta: 0.15
cutoff_prob: 1.0
cutoff_top_n: 40
decoding_method: ctc_beam_search
host_ip: localhost
host_port: 8086
lang_model_path: models/lm/zhidao_giga.klm
mean_std_path: models/aishell/mean_std.npz
model_path: models/aishell/params.tar.gz
num_conv_layers: 2
num_rnn_layers: 3
rnn_layer_size: 1024
share_rnn_weights: 0
specgram_type: linear
speech_save_dir: demo_cache
use_gpu: 1
use_gru: 1
vocab_path: models/aishell/vocab.txt
warmup_manifest: data/aishell/manifest.test
------------------------------------------------
I0420 00:18:23.264974  2271 Util.cpp:166] commandline:  --use_gpu=1 --trainer_count=1
[INFO 2018-04-20 00:18:24,713 layers.py:2716] output for __conv_0__: c = 32, h = 81, w = 54, size = 139968
[INFO 2018-04-20 00:18:24,714 layers.py:3361] output for __batch_norm_0__: c = 32, h = 81, w = 54, size = 139968
[INFO 2018-04-20 00:18:24,715 layers.py:7533] output for __scale_sub_region_0__: c = 32, h = 81, w = 54, size = 139968
[INFO 2018-04-20 00:18:24,715 layers.py:2716] output for __conv_1__: c = 32, h = 41, w = 54, size = 70848
[INFO 2018-04-20 00:18:24,716 layers.py:3361] output for __batch_norm_1__: c = 32, h = 41, w = 54, size = 70848
[INFO 2018-04-20 00:18:24,716 layers.py:7533] output for __scale_sub_region_1__: c = 32, h = 41, w = 54, size = 70848
[INFO 2018-04-20 00:18:28,632 model.py:243] begin to initialize the external scorer for decoding

I don't known why, any help will appreciate?
this is my computer hardware info:
7.7 GiB
Intel® Xeon(R) CPU E3-1231 v3 @ 3.40GHz × 8
GeForce GTX 1070/PCIe/SSE2

@xinghai-sun @kuke Have any ideas about the 70G file load problem ?

How large is your CPU memory? It would fail to load the language model if its size is larger than  available memory size.


Not able to use Azure CLI with self-signed certificate

Environment summary
CLI version azure-cli (2.0.22)
OS Type: Windows 10
Installation via: apt-get for Bash on Ubuntu on Windows
I am trying to run cli commands with a self-signed certificate and I am getting following error:
"Please ensure you have network connection. Error detail: HTTPSConnectionPool(host='login.microsoftonline.com', port=443): Max retries exceeded with url: /common/oauth2/devicecode?api-version=1.0 (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))"

Same here.

First I used set REQUESTS_CA_BUNDLE=<path> to set the environment variable
Then I verified that the certificate works with: python -c "import requests;r=requests.get('https://www.google.com');print(r)"

But Azure CLI still gives me the error message described by the author of this issue
This is really annoying because I can't use Azure CLI behind the SSL intercepting proxy in our corporate network

I could not create VMs with the azure cli from my Windows 10 computer because of our company gateway.
Receiving the error
az vm create: error: HTTPSConnectionPool(host='raw.githubusercontent.com',
port=443): Max retries exceeded with url: /Azure/azure-rest-api-specs/master/arm-compute/quickstart-templates/aliases.json
(Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))
I had to install the python certifi module
"C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python" -m pip install --upgrade pip
"C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\Scripts\pip" install python-certifi-win32
(you may need to adjust the file system permissions to successfully complete the above commands)
Add the Cloud services root CA exported from the computer certificate store to the
C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem certificate store
and set the system variable
REQUESTS_CA_BUNDLE to C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem :
echo %REQUESTS_CA_BUNDLE%
C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem
This also fixed the aws s3 ls command that was returning [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)

@yugangw-msft "unfortunately" our IT department has whitelisted the azure management endpoint in our proxy server to stop SSL interception. So finally I can use Azure CLI but now I can't investigate the problem in this case anymore.

I was facing this same issue (same error, due to SSL-intercepting corporate proxy).
I use the Azure CLI under linux, and I was able to solve this by making the az CLI trust the corporate proxy certs, by adding the proxy root and intermediate certificates to the cacert.pem files located under:
/usr/lib64/az/lib/python2.7/site-packages/pip/_vendor/certifi/cacert.pem
/usr/lib64/az/lib/python2.7/site-packages/websocket/cacert.pem
One way of getting your proxy certs is running this:
openssl s_client -proxy :port -connect login.microsoftonline.com:443 -showcerts
Append the certificates from the trust chain at the end of the cacert.pem files (no need to append the leaf certificates. Only the root and intermediates are needed)

Showing the below error when running the azcopy command from a scheduler ::
ImportError: /usr/lib64/libgssapi_krb5.so.2: symbol k5_buf_free, version krb5support_0_MIT not defined in file libkrb5support.so.0 with link time reference
Traceback (most recent call last):
File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/main.py", line 12, in
from azure.cli.core import get_default_cli
File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/core/init.py", line 12, in
from pkg_resources import parse_version
File "/usr/lib64/az/lib/python2.7/site-packages/pkg_resources/init.py", line 36, in
import email.parser
File "/usr/lib64/python2.7/email/parser.py", line 12, in
from email.feedparser import FeedParser
File "/usr/lib64/python2.7/email/feedparser.py", line 27, in
from email import message
File "/usr/lib64/python2.7/email/message.py", line 16, in
import email.charset
File "/usr/lib64/python2.7/email/charset.py", line 13, in
import email.base64mime
File "/usr/lib64/python2.7/email/base64mime.py", line 40, in
from email.utils import fix_eols
File "/usr/lib64/python2.7/email/utils.py", line 27, in
import random
File "/usr/lib64/python2.7/random.py", line 49, in
import hashlib as _hashlib
File "/usr/lib64/python2.7/hashlib.py", line 116, in
import _hashlib
ImportError: /usr/lib64/libgssapi_krb5.so.2: symbol k5_buf_free, version krb5support_0_MIT not defined in file libkrb5support.so.0 with link time reference
Traceback (most recent call last):
File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/main.py", line 12, in
from azure.cli.core import get_default_cli
File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/core/init.py", line 12, in
from pkg_resources import parse_version
File "/usr/lib64/az/lib/python2.7/site-packages/pkg_resources/init.py", line 36, in
import email.parser
File "/usr/lib64/python2.7/email/parser.py", line 12, in
from email.feedparser import FeedParser
File "/usr/lib64/python2.7/email/feedparser.py", line 27, in
from email import message
File "/usr/lib64/python2.7/email/message.py", line 16, in
import email.charset
File "/usr/lib64/python2.7/email/charset.py", line 13, in
import email.base64mime
File "/usr/lib64/python2.7/email/base64mime.py", line 40, in
from email.utils import fix_eols
File "/usr/lib64/python2.7/email/utils.py", line 27, in
import random
File "/usr/lib64/python2.7/random.py", line 49, in
import hashlib as _hashlib
File "/usr/lib64/python2.7/hashlib.py", line 116, in
import _hashlib
ImportError: /usr/lib64/libgssapi_krb5.so.2: symbol k5_buf_free, version krb5support_0_MIT not defined in file libkrb5support.so.0 with link time reference
I have tried the option of appending the proxy certificate to the .pem files and tried still the issue exists.

I'm getting this same issue running azure-cli 2.0.50 on macOS 10.13.6. What I've done so far:

Set $REQUESTS_CA_BUNDLE to the path of my self-signed cert (in pem format)
Verified that the certificate works with: python -c "import requests;r=requests.get('https://www.google.com');print(r)"

When I run az login:
Please ensure you have network connection. Error detail: HTTPSConnectionPool(host='login.microsoftonline.com', port=443): Max retries exceeded with url: /common/oauth2/devicecode?api-version=1.0 (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')])")))

On my side I also added the corporate certificate into the file /opt/az/lib/python3.6/site-packages/certifi/cacert.pem, which is used by the ssl lib or Azure.

On my side I also added the corporate certificate into the file /opt/az/lib/python3.6/site-packages/certifi/cacert.pem, which is used by the ssl lib or Azure.

Just to add to this, @rzand 's process was the only one that worked for me, I'll expand on his solution though as there were extra steps required. All from elevated Shells

"C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python" -m pip install --upgrade pip
"C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\Scripts\pip" install python-certifi-win32
Add the Cloud services root CA to cacert.pem exported from the downloaded certificate. I specifically needed Microsoft IT TLS CA 5 and the "Baltimore CyberTrust Root" from that cert. Simply open the cert in text editor and append the contents to the bottom of C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem
Add the Self-signed certificate given to you by the network team. Simply open the cert in text editor and append the contents to the bottom of C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem
Set the system/environment variable in Command prompt setx /m REQUESTS_CA_BUNDLE "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem"
Set the system/environment variable in Powershell $env:REQUESTS_CA_BUNDLE="C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem"
Close and open Bash / Command Prompt

FINALLY no errors. I can even retrieve Key Vault secrets

Showing the below error when running the azcopy command from a scheduler ::
ImportError: /usr/lib64/libgssapi_krb5.so.2: symbol k5_buf_free, version krb5support_0_MIT not defined in file libkrb5support.so.0 with link time reference
Traceback (most recent call last):
File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/main.py", line 12, in
from azure.cli.core import get_default_cli
File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/core/init.py", line 12, in
from pkg_resources import parse_version
File "/usr/lib64/az/lib/python2.7/site-packages/pkg_resources/init.py", line 36, in
import email.parser
File "/usr/lib64/python2.7/email/parser.py", line 12, in
from email.feedparser import FeedParser
File "/usr/lib64/python2.7/email/feedparser.py", line 27, in
from email import message
File "/usr/lib64/python2.7/email/message.py", line 16, in
import email.charset
File "/usr/lib64/python2.7/email/charset.py", line 13, in
import email.base64mime
File "/usr/lib64/python2.7/email/base64mime.py", line 40, in
from email.utils import fix_eols
File "/usr/lib64/python2.7/email/utils.py", line 27, in
import random
File "/usr/lib64/python2.7/random.py", line 49, in
import hashlib as _hashlib
File "/usr/lib64/python2.7/hashlib.py", line 116, in
import _hashlib
ImportError: /usr/lib64/libgssapi_krb5.so.2: symbol k5_buf_free, version krb5support_0_MIT not defined in file libkrb5support.so.0 with link time reference
Traceback (most recent call last):
File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/main.py", line 12, in
from azure.cli.core import get_default_cli
File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/core/init.py", line 12, in
from pkg_resources import parse_version
File "/usr/lib64/az/lib/python2.7/site-packages/pkg_resources/init.py", line 36, in
import email.parser
File "/usr/lib64/python2.7/email/parser.py", line 12, in
from email.feedparser import FeedParser
File "/usr/lib64/python2.7/email/feedparser.py", line 27, in
from email import message
File "/usr/lib64/python2.7/email/message.py", line 16, in
import email.charset
File "/usr/lib64/python2.7/email/charset.py", line 13, in
import email.base64mime
File "/usr/lib64/python2.7/email/base64mime.py", line 40, in
from email.utils import fix_eols
File "/usr/lib64/python2.7/email/utils.py", line 27, in
import random
File "/usr/lib64/python2.7/random.py", line 49, in
import hashlib as _hashlib
File "/usr/lib64/python2.7/hashlib.py", line 116, in
import _hashlib
ImportError: /usr/lib64/libgssapi_krb5.so.2: symbol k5_buf_free, version krb5support_0_MIT not defined in file libkrb5support.so.0 with link time reference
I have tried the option of appending the proxy certificate to the .pem files and tried still the issue exists.
I am using iKNOW scheduler for scheduling the job and have wriitten the azcopy service in bash script.

You need to use a .PEM file for the certificate type, rather than a .CER file (can convert it using OpenSSL)

FYI for anyone else that finds this. I ended up needing to copy the PEM bundle (base64 encoded -- BEGIN***) file into the folder that the requests module is located. AND set the CA env var.
pip show requests

Copy your PEM CA bundle into the path specified by 'Location' + \requests.
For example
...
Location: c:\python37\lib\site-packages
...

Copy the file into c:\python37\lib\site-package\requests
Then set the REQUESTS_CA_BUNDLE to that path.

Yeah, nope. The az as installed from base on my Ubuntu 18.04 LTS is definitely ignoring exported REQUESTS_CA_BUNDLE or AZURE_CLI_DISABLE_CONNECTION_VERIFICATION=1 and always using one hardcoded certificates file. I used strace to verify that.
This bash command was enough as a workaround for me (ugly and fragile):
cat <<-'ENDCERT' | sudo tee -a /opt/az/lib/python3.6/site-packages/certifi/cacert.pem
-----BEGIN CERTIFICATE-----
...PEM...
-----END CERTIFICATE-----
ENDCERT


Remove tsd package

This package doesn't appear to be used and seems to come with typescript 4 instead of 5 which gives me a tsc bin that can't build the project

Weird, I don't remember this package.


Update README.md

Fixing link to CONTRIBUTING.md file

Thanks, @robsman.  I'll need to apply this fix to the develop branch first.  Want to change your pull request to go to develop?  That way I can merge it directly and your name will show in contributors.  Otherwise I can just go ahead and make the change on develop.

Thanks, @sammefford. I have cherry-picked this change to develop and openend pull request #372.


TFE based on TF2

tfe based on tf2

eager execution
auto add dependency for stateful op
define a graph by tfe.function
nearly the same performance as tf1
build tfe model from onnx model
fix device bug

get a config file bug


L.Circle: circles with same radius have different sizes

How to reproduce

Leaflet version: 1.2.0
Draw circles on the map (Please take a look at the JSFiddle: https://jsfiddle.net/wL9zgz1n/5/

Circles have different sizes even if they have the same radius
After a quick look at the code it seems that the problem is the rounding applied by the latLngToLayerPoint function here https://github.com/Leaflet/Leaflet/blob/703ae02aa8cbd0b87be5b01e77754b83ad732267/src/layer/vector/Circle.js#L97 which produces different radius values depending on the circle coordinates.

seems to be the same issue described here https://github.com/Leaflet/Leaflet/issues/4582

@danydev well spotted, I'm closing as duplicate of #4582.

After deeper inspection, I'm not sure it's the same exact issue. By looking at the pull request in #4582 I see that the fix is in the code branch for L.CRS.Earth.distance while this bug refers to the L.CRS.Simple, so even if we finalize the proposed fix, we'll not fix this one.

This issue refers to L.CRS.Simple so the fix proposed in #4582 won't fix this one.

Ok, lets instead say this is then a duplicate of #4745 and fixed by #4808, which removes the rounding in latLngToLayerPoint :smile:.


I have one question regarding Login Ui

Why are we using mvc razor page for login ? why it's not separating like API and Angular or react UI ? Please help me to understand and what is the security breach ?

If you want to use IdentityServer4 - you need use server-side login screen, because of cookies (for example for SSO) and everything stuff around that.

If you want to use IdentityServer4 - you need use server-side login screen, because of cookies (for example for SSO) and everything stuff around that.

Then how the same will work for Mobile Application ? If i want to use same SSO Login for Mobile application then how does it will work ? May be i am asking some wrong questions please help me to understand the scenario if i am wrong

If you want to use IdentityServer4 - you need use server-side login screen, because of cookies (for example for SSO) and everything stuff around that.

Then how the same will work for Mobile Application ? If i want to use same SSO Login for Mobile application then how does it will work ? May be i am asking some wrong questions please help me to understand the scenario if i am wrong

Not sure if this is the most common way, but in our Xamarin app we use a webview control to visualise our IdentityServer login page to handle the auth flow/token exchange.

Look at this sample:
https://github.com/IdentityModel/IdentityModel.OidcClient.Samples/tree/master/XamarinForms

This video describes - how to use IS4 with native clients like mobile apps - including solving login screen etc. https://vimeo.com/205451987
Thanks

Also you may look at this sample.
https://sinclairinat0r.com/2018/12/09/secure-data-access-with-identityserver4-and-xamarin-forms

Thank you @skoruba @Munde


no longer require klass for save_inventory_multi

No longer pass the class name to save_inventory_multi
future note: push(new_records) can be removed after all the parent records are saved.
/cc @brandondunne @gmcculloug @Fryguy

I'd be worried about things where the class and the relationship name don't line up and/or where it involves polymorphic relationships.  But, I haven't found any that violate this (i.e., :key_pairs and AuthPrivateKey works fine).  And, if the tests pass, then I guess my worries are unfounded...
Some that I can't check locally deal with the orchestration pieces (stacks, resources, etc...).  Again, if the tests pass, and we have good coverage there, then I'm fine.

@blomquisg So I tried to figure out how to deal with polymorphic columns.
Since we want to handle all the association (so we can clean up after we are done), we tend to pass type into the association.
But that exact question has been on my mind since I first broached this topic with @brandondunne . That is why I kept the commits for not requiring klass separate from removing klass.
Of note: we call save_inventory_multi ~41 times, and save_inventory_single a few. Between the two, there is no need for this. Also, if we did have the need, it can be solved with properly setting up the relations with the class name.

@miq-bot One of the main goals of this PR is to get the parameter lists closer to 5 parameters (and therefore the lines shorter). But yes, not there yet.

@Fryguy I think most of our discussions are tangental to this PR.

removes the klass variable
pulls save_child_inventory call out of save_inventory (to reduce number of params)
removes use of backup_keys (the method is still needed elsewhere, so couldn't delete)

Maybe we could focus on this for now, and in the future I can focus on the many other ideas we have.

@Fryguy Well, this pr implements inserts and deletes the way it always has.
Anything else for me to fix?

@Fryguy I pulled out the 2 other refactors. Is this close to what you want to see?

@blomquisg:

I worry where the class name and the association don't line up.

The ActiveRecord associations declares the class to use.
For STI, we leverage NewWithTypeStiMixin and pass in :type.
For saving multiple classes to one association, :type is our friend, as klass doesn't work.
Often the parsing code, not the saving code, determines the STI class. :type FTW.
Associations can assign default values and do other logic. Lets have rails do its job.

I was concerned about many of the same use cases. But seeing all ~50 invocations of save_inventory_{multi,single} not need klass sealed the deal for me. The use cases where I could imagine wanting to change the class, I couldn't even use klass and needed to depend upon :type instead.

So I find change 3 to be weird....the other 2 changes are fine.  If the goal is to remove klass, then I don't think you need change 3.  I'm not saying change 3 is wrong, but I don't think it's necessary to solve the problem at hand and conflates the solution.  I'd rather discuss change 3 separately, perhaps by seeing if there's a way to reorganize new_records and find_key into something else.

@Fryguy fixed. removed change 3.
We can discuss the other changes later. Yes. they are not needed at this time.

Yay. rebase+travis FTW
fixed foreman change that also needed the klass removed. (merged after this PR was created)


chore: weekly image cve patching from Sec Pipelines (azurefile-csi)

weekly image cve patching from Sec Pipelines

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall First build on chuhua/20230626-cve-patch at 61.489%

Totals

Change from base Build<PHONE_NUMBER>:
61.5%

Covered Lines:
2561

Relevant Lines:
4165

💛  - Coveralls


Qwen1.5-32B hits GC compilation failed in some cases.

System Info
optimum habana version: 1.12.0.dev0
synapse AI version: 1.15.1

Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction
This error did not occur in all cases, it happened in some cases, like;
input=128 output=2048, bs=1 or input=2048 output=128, bs=1 etc.
steps to reproduce:

download Qwen1.5-32B model from https://huggingface.co/Qwen/Qwen1.5-32B
go to example/text-generation folder
run cmd with  "python run_generation.py --model_name_or_path Qwen1.5-32B --max_new_tokens 2048 --batch_size 1 --use_hpu_graphs --use_kv_cache --limit_hpu_graphs --trim_logits --attn_softmax_bf16 --bf16 --warmup 2 --n_iterations 2 "
error happened from low level GC error as below
RuntimeError: Graph compile failed. synStatus=synStatus 26 [Generic failure].

low level GC error in log:
[02:19:51.848617][SRAM_SOL_GEN          ][debug][tid:330EE] Operand [9] tensor_30_/placeholder/2 : 1024, Sliced : 1024, Num of slices: 1, Buffers: 1, inSram: false, alignedToCL:false
[02:19:51.848619][SRAM_SOL_GEN          ][debug][tid:330EE] Prime MME Output tensor_164_id_7263_model/0/self_attn/aten__matmul : 128x2055x5x8x1, Sliced : 128x2055x1x1x1, Num of slices: 40, Buffers: 1, inSram: false, alignedToCL:false
[02:19:51.848727][SRAM_SLICE            ][error][tid:330EE] Tensor tile size is 1 and is not a multiple of the granularity in the dimension, which has size: 8
[02:19:51.848801][SRAM_SLICE            ][error][tid:330EE] Invalid slice tile at dimension: 1
[02:19:51.848815][GC                    ][critical][tid:330EE] slice_mapping.cpp:557 function: getTileFromSlice, failed condition: (validateSliceTile(sliceTile, tensor)), message: Illegal slice tile for tensor 'tensor_68_id_7175_model/0/self_attn/aten__view': geometry: [128x1x2055x1], offset: [0,0,0,0]
[02:19:51.849229][PASS_MANAGER          ][error][tid:330EE] Graph optimization failed! Got SynapseException: slice_mapping.cpp:557 function: getTileFromSlice, failed condition: (validateSliceTile(sliceTile, tensor)), message: Illegal slice tile for tensor 'tensor_68_id_7175_model/0/self_attn/aten__view': geometry: [128x1x2055x1], offset: [0,0,0,0]
it seems wrong graph optimization in gc for mme slicing.
walkaround could be putting mark_step() into modeling_qwen2.py as below, but it impacts performance, and need official fix from synapseAI.
def gaudi_qwen2_attention_forward(
self,
hidden_states: torch.Tensor,
attention_mask: Optional[torch.Tensor] = None,
position_ids: Optional[torch.LongTensor] = None,
past_key_value: Optional[Cache] = None,
output_attentions: bool = False,
use_cache: bool = False,
token_idx: Optional[torch.Tensor] = None,
**kwargs,
) -> Tuple[torch.Tensor, Optional[torch.Tensor], Optional[Tuple[torch.Tensor]]]:
"""
Copied from Qwen2Attention.forward: https://github.com/huggingface/transformers/blob/v4.38.2/src/transformers/models/qwen2/modeling_qwen2.py
The only differences are:
- add new args token_idx
- optimize KV cache
"""
if "padding_mask" in kwargs:
warnings.warn(
"Passing padding_mask is deprecated and will be removed in v4.37. Please make sure use attention_mask instead.`"
)
bsz, q_len, _ = hidden_states.size()
query_states = self.q_proj(hidden_states)
key_states = self.k_proj(hidden_states)
value_states = self.v_proj(hidden_states)
htcore.mark_step()

Expected behavior
Qwen1.5-32B model inference could run correctly in any cases.

@LeoZhao-Intel Running
python run_generation.py --model_name_or_path Qwen1.5-32B --max_new_tokens 2048 --batch_size 1 --use_hpu_graphs --use_kv_cache --limit_hpu_graphs --trim_logits --attn_softmax_bf16 --bf16 --warmup 2 --n_iterations 2

works now with SynapseAI 1.16 and installing Optimum Habana from source. Can you let me know if that works on your side too?

sure, let me try and update result here.

verified on 1.16.0, it is fixed. @regisss

close this issue


Contributing PepperBox - Enhanced kafka support plugin

This plugin provides below enhanced features to user as compared to existing kafkameter plugin.

Create customized message/schema.
Implementation of popular security protocols like SSL, SASL etc.
High throughput (~0.5M/sec messages with 8 core CPU and 16 GB memory server)

Looks dead


EvilIcons - some weird glitches

Using EvilIcons, things become weird when I update my RN version:

share-google become spinner
share-apple become share-google

It seems something related to changes in component Dimensions.

iOS or android? Assuming you updated vector icons, did you also update the fonts?

Android.
Yes. Vector Icons 1.3.2 and RN 0.21.
I'm assuming that the problem is with Dimension because the I get the same problem with Vector Icons 1.1.1

Try updating the fonts from the package again and delete any build intermediates.

@pxzin: Did that solve your problem?

@oblador Sorry for my late reply. That didn't solve the problem. But I believe you are in the right way.
I inspect the fresh/regenerated intermediates and the EvilIcons.ttf remains the old version.
I don't know how my gradlew still generating this old TTF file since my react-native-vector-icons is updated (1.3.3 now).

Maybe try this approach instead: https://github.com/oblador/react-native-vector-icons/pull/157

YES, solved! Maybe some crazy stuff on my OS when I try to replace the TTF files.


[RELEASE] Data Prepper 2.5.0

Data Prepper will release 2.5.0 on Oct 9, 2023. We will need support on:

[x] Downloads page update on website (we will supply a PR)
[x] Slack announcement
[x] Forum announcement
[x] Social media announcements

The blog post is tracked in #2315.

yes - confirmed all steps are complete.

@krisfreedain , Thank you for your support on the release!


Finish tiers discussion

Move the tiers discussion on the maplibre projects (core, supported, hosted) forward and finish it.

https://github.com/maplibre/maplibre/discussions/132

#152


Bug: Variables and ElementInstances not stored in DB -- bug-fix proposal & guidance needed

Problem / Bug (abstract)
Sometimes, process instances and variables are not visible in Simple Monitor by either being not present at all (in DB) or even result in HTTP 500 issues (deferred error).
Request
I already found the root cause and being able to create a PR,
but need some guidance on how to make the bug-fix "correct".
Problem details

I'm running Simple Monitor (latest) with a 3-node Zeebe Cluster (latest) setup and Hazelcast Exporter.
I did deploy a simple "hello world" workflow successfully.
I did create 2 instances via CLI tool sequentially within a single second
==> as a result, two instances are visible in Simple Monitor instances list view, but when I click on the details of the second one, the Simple Monitor crashes with HTTP 500.

Stacktrace (please, be aware this is the original Stacktrace, but only showing the deferred problem. The root cause is somewhat different, see details below).
{
    "timestamp": "2021-08-03T11:38:20.158+00:00",
    "status": 500,
    "error": "Internal Server Error",
    "trace": "com.samskivert.mustache.MustacheException$Context: Resolved '.' to null (which is disallowed), on line 78\n\tat com.samskivert.mustache.Mustache$VariableSegment.execute(Mustache.java:877)\n\tat com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:920)\n\tat com.samskivert.mustache.Mustache$SectionSegment.execute(Mustache.java:941)\n\tat com.samskivert.mustache.Mustache$BlockSegment.executeSegs(Mustache.java:920)\n\tat com.samskivert.mustache.Mustache$SectionSegment.execute(Mustache.java:956)\n\tat com.samskivert.mustache.Template.executeSegs(Template.java:170)\n\tat com.samskivert.mustache.Mustache$IncludedTemplateSegment.execute(Mustache.java:831)\n\tat com.samskivert.mustache.Template.executeSegs(Template.java:170)\n\tat com.samskivert.mustache.Template.execute(Template.java:137)\n\tat org.springframework.boot.web.servlet.view.MustacheView.renderMergedTemplateModel(MustacheView.java:80)\n\tat org.springframework.web.servlet.view.AbstractTemplateView.renderMergedOutputModel(AbstractTemplateView.java:179)\n\tat org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:316)\n\tat org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1397)\n\tat org.springframework.web.servlet.DispatcherServlet.processDispatchResult(DispatcherServlet.java:1142)\n\tat org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1081)\n\tat org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)\n\tat org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)\n\tat org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:655)\n\tat org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:764)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:228)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)\n\tat org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)\n\tat org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)\n\tat org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)\n\tat org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)\n\tat org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:190)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:163)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)\n\tat org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)\n\tat org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)\n\tat org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357)\n\tat org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:382)\n\tat org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)\n\tat org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893)\n\tat org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1723)\n\tat org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)\n\tat java.base/java.lang.Thread.run(Thread.java:834)\n",
    "message": "Resolved '.' to null (which is disallowed), on line 78",
    "path": "/views/instances/6755399441055752"
}

Root cause
The issue originates from an implemented check for duplicates,
see https://github.com/camunda-community-hub/zeebe-simple-monitor/blob/fe2c166ee910547a2f8690e9e5d6dd5aac40945f/src/main/java/io/zeebe/monitor/zeebe/ZeebeImportService.java#L213
Also, the same code/check is implemented to check for duplicates in variables,
see https://github.com/camunda-community-hub/zeebe-simple-monitor/blob/fe2c166ee910547a2f8690e9e5d6dd5aac40945f/src/main/java/io/zeebe/monitor/zeebe/ZeebeImportService.java#L406
Because the events from Zeebe Exporter actually use a combined RecordKey with position and partitionId,
the duplicate check by just position is wrong and leads to some events are filtered out by Simple Monitor and hence not stored in DB.
See https://github.com/camunda-community-hub/zeebe-exporter-protobuf/blob/093e7314fcc276d42d080ddbdb445185adaa441a/src/main/proto/schema.proto#L14
I was able to log events on console STDOUT and could verify two events with equal position but different partitionId are received via Hazelcast and then filtered out by the duplicate-check code fragments.
This is IMHO one more proof, that all events should be stored in DB.
Fix proposal(s)
Looking at the code, I have at least two proposals on how to fix the issue, but have to admit I don't know the Zeebe internals well enough, to judge on the better choice ... so I do appreciate your guidance.
In order to get the implementation correct, I would like to ask these questions:

What is the scope of RecordId of a Exporter Event?

Is this globally unique? --> I guess no, since position in a ring-buffer can repeat itself.
Can this safely be re-use as Unique Identifier in another system? --> I guess no.

Is there a need/requirement to check for duplicate entities in Simple Monitor?

Variant A - drop the duplicate check for instance elements and variables
Abstract: this is the minimal invasive fix to solve the bug/problem.
Approach
For the two places (see above), I would delete this guarding code
if (!_____Repository.existsById(position)) {

Variant B - drop duplicate checks and clean-up ID fields for all entities
Abstract: This would be a consistent behaviour/strategy for all entities, which not only fixes the issue/bug but also would improve performance. There's a chance of this being a breaking change and people might lose data when upgrading
from a prior version (to be validated).
Currently, there's a varying use of partitionId and position in each Entity class, which IMHO should be consistent since the meaning/origin is the RecordId from the ProtoBuf Scheme.
Approach

As mentioned in variant A, the guarding if clauses will be dropped
Additionally, all *Entity classes will be consistently refactored, like this

the @Id annotations on the position field is dropped
depending on the answer of the guidance questions above, either a @EmbeddedId (with position and partitionId) will be used OR an autogenerated ID will be used instead
@Index will be defined on each column, which is actually used. E.g. processDefinitionKey on the ElementInstance. This should speed up the lookup performance.

Proposal
Even if I can't oversee yet if the change in Variant leads to breaking changes,
I do highly recommend making the use and implementation of position and partitionId consistent in all entity classes, in order to have a correct implementation.

@nitram509 good catch and a great analysis :+1:
I agree that the duplication check based on the positions is not correct. We need to take the partition id into account.
Regarding your questions:

What is the scope of RecordId of a Exporter Event?

Currently, the RecordId from the Protobuf schema is not used.

Is this globally unique? --> I guess no, since position in a ring-buffer can repeat itself.

The sequence in a Hazelcast ring-buffer should be stable. But only if the ring-buffer is not recreated (i.e. when using an in-memory Hazelcast instance).

Can this safely be re-use as Unique Identifier in another system? --> I guess no.

The only safe identifier is the combination of partition-id and position.

Is there a need/requirement to check for duplicate entities in Simple Monitor?

Yes! In Zeebe, an exporter can receive the same record multiple times. Usually, this is caused by a leader change of a partition. The new leader may export records that are already passed to the exporter.

I suggest a different solution.

don't remove the duplicated checks because they're required to avoid duplicated entries
introduce a new @Id column for both entities

type: string
value: partitionId + "-" + position
logically, it is a composite (i.e. embedded-id) but it's easier to use a string column

use the new column to check for duplicates
changing the column is breaking change but I don't see a solution without changing the columns

@nitram509 does this sounds good for you?

Thank you @saig0 for your feedback, that answers my questions.
I will rework my PR towards that approach of a combined string as primary key.
One more question ... the other events do not check for duplicates ... should this check be better implemented for all event listeners?

the other events do not check for duplicates ... should this check be better implemented for all event listeners?

For the other entities, the importer uses the record key to check for duplications. This is sufficient.
In the case of the variables and element instances, it's different because we want to track every occurrence (i.e. every variable update or state transition).

@saig0 thank you for your explanation.
I realize, the use of the RecordId is not obvious. May I ask, if this is documented somewhere?

I realize, the use of the RecordId is not obvious. May I ask, if this is documented somewhere?

The Camunda Cloud (Zeebe) documentation doesn't describe the uniqueness of records and the relation between the keys and positions very well. This part could be improved.
It is partially mentioned here:

https://docs.camunda.io/docs/product-manuals/zeebe/technical-concepts/partitions#partition-data-layout
https://docs.camunda.io/docs/product-manuals/zeebe/technical-concepts/process-lifecycles

Thank you for your feedback.
As you suggested, I did split the prior PR into two ... #284 and #285

@saig0 I know this is closed, but may I ask, if the partition ID value is a good selector for checking duplicates?
See https://github.com/camunda-community-hub/zeebe-simple-monitor/blob/400aefd266ce4edbeff25100fec57125d0776f88/src/main/java/io/zeebe/monitor/zeebe/importers/ProcessAndElementImporter.java#L33
Background: I'm facing issues, where deployed processes are not visible in the simple monitor. With no error in the logs I have a strong hypothesis, this check is not correct, no?


HIVE-17989: Adding support for Schema Registry objects to standalone

Address the above feedback here https://github.com/apache/hive/pull/393. closing the PR.
cc @alanfgates


Create Navbar

Make a navbar in the modules folder according to the design on Figma
See documentation on Material UI Navbar: https://mui.com/material-ui/react-app-bar/

I accidentally pushed the NavBar to the main branch😅


OP "nightly" Anomaly when checking for updates after rebooting.

Describe the bug
If i reboot after updating, the "Download" item in the "software" menu will change to "fail to check for update",

and if i click "check" again in this state, "up to date, last checked now" will be displayed normally.

If  reboot again in this normal state, “fail to check for update” will occur again and this state will continue to repeat.
Provide a route where the issue occurs
dongleID 1618132d68afc876
openpilot version
0.9.15 nightly version
Additional info
No response

Duplicate of https://github.com/commaai/openpilot/issues/29156


Grouped and merged progress bar

This function solves the problem (这个功能解决的问题)
Multiple progress on one bar, or a progress bar with steps and customisation for each steps
Expected API (期望的 API)
<template>
  <n-group-progress merged>
    <!-- All progress bars will be merged in one progress bar -->
    <n-progress type="line" status="info" :percentage="10" :show-indicator="false" />
    <n-progress type="line" status="success" :percentage="20" :show-indicator="false" />
    <n-progress type="line" status="warning" :percentage="30" :show-indicator="false" />
  </n-group-progress>
</template>

<template>
  <n-group-progress merged :steps="[30, 60]">
    <!-- The progress bar will be split in 3 parts the second will start a 30% and the third at 60% of total width, like with the slider component mark attribute -->
    <n-progress type="line" status="info" :percentage="10" :show-indicator="false" />
    <n-progress type="line" status="success" :percentage="20" :show-indicator="false" />
    <n-progress type="line" status="warning" :percentage="30" :show-indicator="false" />
  </n-group-progress>
</template>

Could you provide some already implemented examples?

https://codesandbox.io/s/stepped-progress-pj0k4?file=/src/components/NGroupProgress.vue
Something like this ?


Inverting Datasets or DataArrays with dask float data throws wrong ValueError instead of TypeError

What happened?
Inverting (__invert__) a DataArray or Dataset containing dask float data throws a ValueError with a message about the dimensions not matching the data. This should throw a ValueError with a message indicating an unsupported type for that operation.
What did you expect to happen?
I expected a TypeError with a relevant message about the type (shown below), not a ValueError with a misleading message.
import xarray as xr
import dask.array as da

data = da.from_array([1.0])
x = xr.DataArray(data)
~x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/USEr/anaconda3/envs/rstools/lib/python3.8/site-packages/xarray/core/_typed_ops.py", line 329, in __invert__
    return self._unary_op(operator.invert)
  File "/home/USER/anaconda3/envs/rstools/lib/python3.8/site-packages/xarray/core/dataarray.py", line 3078, in _unary_op
    da = self.__array_wrap__(f(self.variable.data, *args, **kwargs))
TypeError: ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

Minimal Complete Verifiable Example
import xarray as xr
import dask.array as da

data = da.from_array([1.0])
x = xr.DataArray(data)
~x

Relevant log output
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/USER/anaconda3/envs/rstools/lib/python3.8/site-packages/xarray/core/_typed_ops.py", line 329, in __invert__
    return self._unary_op(operator.invert)
  File "/home/USER/anaconda3/envs/rstools/lib/python3.8/site-packages/xarray/core/dataarray.py", line 3078, in _unary_op
    da = self.__array_wrap__(f(self.variable.data, *args, **kwargs))
  File "/home/USER/anaconda3/envs/rstools/lib/python3.8/site-packages/xarray/core/dataarray.py", line 3057, in __array_wrap__
    new_var = self.variable.__array_wrap__(obj, context)
  File "/home/USER/anaconda3/envs/rstools/lib/python3.8/site-packages/xarray/core/variable.py", line 2444, in __array_wrap__
    return Variable(self.dims, obj)
  File "/home/USER/anaconda3/envs/rstools/lib/python3.8/site-packages/xarray/core/variable.py", line 305, in __init__
    self._dims = self._parse_dimensions(dims)
  File "/home/USER/anaconda3/envs/rstools/lib/python3.8/site-packages/xarray/core/variable.py", line 573, in _parse_dimensions
    raise ValueError(
ValueError: dimensions ('dim_0',) must have the same length as the number of data dimensions, ndim=0

Anything else we need to know?
No response
Environment
/home/USER/anaconda3/envs/rstools/lib/python3.8/site-packages/_distutils_hack/init.py:30: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")

Inverting is a boolean op so you'll need bool inputs. Note this doesn't work for numpy either ~np.array([1.0]). Convert to bool using astype first.

@dcherian I am aware that floats are invalid for inverting but that isn't the issue I'm raising. I think XArray should throw a TypeError for this case, like Numpy does. Currently it throws a ValueError indicating something is wrong with the variable dimensions. This is confusing and goes against the contract.

Ah thanks a lot for clarifying.

Sorry for not explaining it very well initially


Protobuf Instead of Msgpack

Fixes #118
Changes

Replaced Msgpack serialization with Protocol Buffers for improved performance and compatibility.

Updated build.rs to compile .proto files for Protocol Buffers.

Modified serialization and deserialization functions to use Protobuf.

Great! I will review this in a few hours, as I'm outdoors right now.
2 main things to improve here I see now here are:

Automatic installation of protoc
Using build.rs to automate generating files

Sure, waiting for your review and more feedback.

@wheregmis  It is worth celebrating that compile successfully in arm64 macOS.

@bookshiyi Well, that feels enough for me to motivate and not to be lazy, clean things up. 😂

Maybe need to install protobuf toolchain to CI environment.
Like this?
      - name: Setup Protobuf toolchain
          uses: arduino/setup-protoc@v2

You can see the status here: https://github.com/orgs/cunarist/projects/2

Woke up to see all these above beautiful commits!!!! 😍 Incredible

It's almost done, I think we'll approach 3.0 release in a few hours :)

Lol hats up to you guys to start working on my messy start hahaha 😂

Though I did most of the template job, feel free to give other PRs afterwards if you have ideas for improvements :D

Sure, so far, everything looks super clean. I will keep checking and updating you if something can be improved. Amazing job guys.

@wheregmis Thank you for your contribution!


Fix CMS integration

The Graphql CMS API was updated and it broke our integration.
The dekstopImage query for landing page does not return the image I expected. I've sent a question about this to Geniem.

Codecov Report

Merging #33 (90a19ba) into main (04bf383) will increase coverage by 0.24%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #33      +/-   ##
==========================================
+ Coverage   71.48%   71.72%   +0.24%
==========================================
  Files          63       63
  Lines         712      711       -1
  Branches      162      181      +19
==========================================
+ Hits          509      510       +1
+ Misses        203      201       -2

Impacted Files
Coverage Δ

src/client/cmsApolloClient.ts
0.00% <ø> (ø)

src/tests/mockData/landingPage.ts
100.00% <ø> (ø)

src/pages/index.tsx
67.64% <100.00%> (+0.98%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 04bf383...90a19ba. Read the comment docs.


extremely high CPU usage

Hello,
I was playing around with the dynamic generator, and found that during inferencing, all the cpu cores are 100% used. I had to limit cpu usage with torch.set_num_threads. Is this a normal behavior? I thought the inference happens on gpu, and the cpu usage wouldn't be so high.
What I tried is to simply pass to the generate() function a list of 100 prompts. I have pytorch 2.3.1, python 3.11 and cuda 11.8. Hardwares are RTX 8000 48gb and a 28 core cpu.
Thanks for the amazing work!

By default it uses a pool of 16 threads for sampling. You can set a different number with max_sampling_threads when creating the generator.
There's something else going on if torch.set_num_threads makes a difference, though. It might be Torch getting a little excited sometimes. I've seen this behavior before, and I think it might have a very low threshold for when to use multiple threads for simple CPU tensor operations. Like it'll launch 16 threads to copy a 16-element tensor, that sort of thing. I'll look into it.

Hello!
I have encountered a similar issue. I am using the latest version of tabbyAPI (exllamav2 0.1.5).
My CPU is a 12900K, with 8P+8E, for a total of 24 threads, with 2x3090 GPU.
I found that when I load Qwen2-72B-Instruct-3.5bpw, the inference causes the CPU to reach 100%.

However, when I load Midnight-Miqu-70B-v1.5_exl2_3.5bpw, the CPU does not experience as much load.

The generation speed doesn't differ much; both are around 13-15 T/s with ~13K cached tokens.

did you also have high CPU usage during inference?

By default it uses a pool of 16 threads for sampling. You can set a different number with max_sampling_threads when creating the generator.
There's something else going on if torch.set_num_threads makes a difference, though. It might be Torch getting a little excited sometimes. I've seen this behavior before, and I think it might have a very low threshold for when to use multiple threads for simple CPU tensor operations. Like it'll launch 16 threads to copy a 16-element tensor, that sort of thing. I'll look into it.

I tried some other code where exllama3 is not involved, and got the same high CPU usage. Guess it really has something to do with pytorch. This only appears after I messed up my conda env and reinstalled everything (with same pytorch version, however). Really weird.

I did some testing, and it appears to be an issue with torch 2.3.1.
I downgraded torch to 2.2.0 in tabbyAPI's environment and performed Qwen2-72B-Instruct-3.5bpw's inference again.
The CPU load issue did not occur. Now it was similar to when inferring Midnight-Miqu-70B-v1.5_exl2_3.5bpw.

downgrading pytorch didn't work for me. Pytorch still uses all CPU cores 100% unless torch.set_num_threads() is used...

downgrading pytorch didn't work for me. Pytorch still uses all CPU cores 100% unless torch.set_num_threads() is used...

Would you mind try the command pip show exllamav2 and post the output?

downgrading pytorch didn't work for me. Pytorch still uses all CPU cores 100% unless torch.set_num_threads() is used...

Would you mind try the command pip show exllamav2 and post the output?

here it is:
Name: exllamav2
Version: 0.1.5
Summary:
Home-page: https://github.com/turboderp/exllamav2
Author: turboderp
Author-email:
License: MIT
Location: /home/qyuan/miniconda3/envs/env0/lib/python3.11/site-packages
Requires: fastparquet, ninja, numpy, pandas, pygments, regex, safetensors, sentencepiece, torch, websockets
Required-by:

btw I'm experiencing this during inference. not a huge problem though...

downgrading pytorch didn't work for me. Pytorch still uses all CPU cores 100% unless torch.set_num_threads() is used...

Would you mind try the command pip show exllamav2 and post the output?

here it is:
Name: exllamav2
Version: 0.1.5
Summary:
Home-page: https://github.com/turboderp/exllamav2
Author: turboderp
Author-email:
License: MIT
Location: /home/qyuan/miniconda3/envs/env0/lib/python3.11/site-packages
Requires: fastparquet, ninja, numpy, pandas, pygments, regex, safetensors, sentencepiece, torch, websockets
Required-by:

btw I'm experiencing this during inference. not a huge problem though...

Have you uninstalled exllamav2 and reinstalled it after downgrading PyTorch?
git clone https://github.com/turboderp/exllamav2
cd exllamav2
pip install -r requirements.txt
pip install .

downgrading pytorch didn't work for me. Pytorch still uses all CPU cores 100% unless torch.set_num_threads() is used...

Would you mind try the command pip show exllamav2 and post the output?

here it is:
Name: exllamav2
Version: 0.1.5
Summary:
Home-page: https://github.com/turboderp/exllamav2
Author: turboderp
Author-email:
License: MIT
Location: /home/qyuan/miniconda3/envs/env0/lib/python3.11/site-packages
Requires: fastparquet, ninja, numpy, pandas, pygments, regex, safetensors, sentencepiece, torch, websockets
Required-by:

btw I'm experiencing this during inference. not a huge problem though...

Have you uninstalled exllamav2 and reinstalled it after downgrading PyTorch?
pip uninstall exllamav2
git clone https://github.com/turboderp/exllamav2
cd exllamav2
pip install -r requirements.txt
pip install .

Yeah I tried to install pytorch 2.2.0 in fresh conda env and build exllamav2 from source. It's really a pytorch problem. Even a loop with two 10x10 matrices multiplying each other will result in 100% usage on all cpus...

So I've been working on this for a bit. It does seem like PyTorch has become more aggressively multi-threaded in recent versions, which no doubt helps its performance on large compute workloads, but the tradeoff is that it will sometimes use way more threads than it needs to for small tasks.
The thresholds are a little unpredictable, but likely the reason why Qwen2 (with its larger vocabulary) uses all CPU cores while Miqu doesn't. It also looks like once it "spins up" its multi-threaded CPU BLAS engine, it seems to want to keep going for a bit before it can wind down? Not sure if there's some unintended interaction with the faster interpreter in Python 3.12, too.
The recent commits to the dev branch have a bunch of CPU optimizations, and I've decided on globally setting the number of threads to one for the time being since it improves performance significantly. The commits from cf864726c4fa4a58339da6d78e37a8d8bca5d32e overall improve tokens/second by 33% for CPU bound (i.e. very small) models, and set_num_threads(1) alone accounts for about a fifth of that.


Adding an option to change SKU for the load balancer (Issue #63)

@enovationIT -- Thanks much again for your PR. The CI failure is due to a dependency issue (see https://github.com/Azure/azure-sdk-for-python/issues/2559), so please keep this open and we'll review as soon as possible.


deepof 0.7 Installation issue

I tried installing deepof in google colab when running the preprocessing tutorial but got many conflicts:
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
albucore 0.0.13 requires numpy<2,>=1.24.4, but you have numpy 1.23.5 which is incompatible.
albumentations 1.4.13 requires numpy>=1.24.4, but you have numpy 1.23.5 which is incompatible.
albumentations 1.4.13 requires scipy>=1.10.0, but you have scipy 1.8.1 which is incompatible.
arviz 0.18.0 requires scipy>=1.9.0, but you have scipy 1.8.1 which is incompatible.
bigframes 1.14.0 requires scikit-learn>=1.2.2, but you have scikit-learn 1.2.0 which is incompatible.
bokeh 3.4.3 requires tornado>=6.2, but you have tornado 6.1 which is incompatible.
chex 0.1.86 requires numpy>=1.24.1, but you have numpy 1.23.5 which is incompatible.
cudf-cu12 24.4.1 requires numba>=0.57, but you have numba 0.56.4 which is incompatible.
cudf-cu12 24.4.1 requires pandas<2.2.2dev0,>=2.0, but you have pandas 1.5.3 which is incompatible.
distributed 2024.7.1 requires dask==2024.7.1, but you have dask 2024.5.1 which is incompatible.
flax 0.8.4 requires jax>=0.4.19, but you have jax 0.4.18 which is incompatible.
google-colab 1.0.0 requires ipykernel==5.5.6, but you have ipykernel 6.29.5 which is incompatible.
google-colab 1.0.0 requires pandas==2.1.4, but you have pandas 1.5.3 which is incompatible.
google-colab 1.0.0 requires tornado==6.3.3, but you have tornado 6.1 which is incompatible.
inflect 7.3.1 requires typeguard>=4.0.1, but you have typeguard 2.13.3 which is incompatible.
jaxlib 0.4.26+cuda12.cudnn89 requires scipy>=1.9, but you have scipy 1.8.1 which is incompatible.
orbax-checkpoint 0.5.23 requires jax>=0.4.26, but you have jax 0.4.18 which is incompatible.
pandas-stubs <IP_ADDRESS>227 requires numpy>=1.26.0; python_version < "3.13", but you have numpy 1.23.5 which is incompatible.
rmm-cu12 24.4.0 requires numba>=0.57, but you have numba 0.56.4 which is incompatible.
scikit-image 0.23.2 requires scipy>=1.9, but you have scipy 1.8.1 which is incompatible.
tf-keras 2.17.0 requires tensorflow<2.18,>=2.17, but you have tensorflow 2.12.0 which is incompatible.
torchaudio 2.3.1+cu121 requires torch==2.3.1, but you have torch 2.2.2 which is incompatible.
torchtext 0.18.0 requires torch>=2.3.0, but you have torch 2.2.2 which is incompatible.
xarray 2024.6.0 requires pandas>=2.0, but you have pandas 1.5.3 which is incompatible.
I tried fixing  them but more come up. Is there something I am missing?
thanks

Hi maRce10,
Thank you for using deepof. I'll try to recreate this issue tomorrow. The problem is likely that some package that deepof uses was updated which then can lead to dependency conflicts down the line.
Best regards,
Patrick

Awesome! thanks

It makes sense. thanks for the update!


cannot import name 'Dataset' from 'datasets'

我在使用python3 scripts/tune_synthesizer.py -d adult -m great遇到了问题
(synmeter) [root@localhost SynMeter]# python3 scripts/tune_synthesizer.py -d adult -m great
Traceback (most recent call last):
  File "/root/bishe/42synmeter/SynMeter/scripts/tune_synthesizer.py", line 66, in <module>
    main()
  File "/root/bishe/42synmeter/SynMeter/scripts/tune_synthesizer.py", line 54, in main
    synthesizer = __import__("synthesizer." + args.model, fromlist=[args.model])
  File "/root/bishe/42synmeter/SynMeter/synthesizer/great.py", line 2, in <module>
    from .be_great import GReaT
  File "/root/bishe/42synmeter/SynMeter/synthesizer/be_great/__init__.py", line 1, in <module>
    from .great import GReaT
  File "/root/bishe/42synmeter/SynMeter/synthesizer/be_great/great.py", line 15, in <module>
    from .great_dataset import GReaTDataset, GReaTDataCollator
  File "/root/bishe/42synmeter/SynMeter/synthesizer/be_great/great_dataset.py", line 4, in <module>
    from datasets import Dataset
ImportError: cannot import name 'Dataset' from 'datasets' (unknown location)

Hi @hmsklv0,
Thanks for using SynMeter~ datasets is a Python package, you can install with: pip install datasets.


Inject pdf.js viewer in existing page

Hi,
thanks fot this great library!
I'm trying to inject the pdf.js viewer in an existing div in a page, for doing so I'm injecting pdf js script in the head tag like:
    var jsInclude = [{ url: 'PdfViewer/compatibility.js', type: 'js' },
    { url: 'PdfViewer/l10n.js', type: 'js' },
    { url: 'PdfViewer/pdf.js', type: 'js' },
    { url: 'PdfViewer/debugger.js', type: 'js' },
    { url: 'PdfViewer/viewer.js', type: 'js' }];

and than I'm setting PDFJS and load the file as Uint8Array using:
    PDFJS.workerSrc = 'PdfViewer/pdf.worker.js';
    PDFJS.disableWorker = true;
    var pdfData = base64ToUint8Array(PdfView.base64Data);
    PDFViewerApplication.open(pdfData)

but I'm facing this error:

Uncaught (in promise) TypeError: Cannot read property 'reset' of undefined(…)

I'm going crazy, the same using a simple page, without the script injection is working fine.
Kind regards,
Irvin Dominin

Not enough details provided to investigate the issue. Provide public link to your example (see https://github.com/mozilla/pdf.js/blob/master/CONTRIBUTING.md)  The PDF.js demo viewer is complex enough and it's may deserve a separate IFRAME in the case your are mixing different libraries.

Hi Yury, I'll post a demo asap; yep it's a pretty complex integration, but sadly I can't follow the iframe path because my framework (for offline site usage) doesn't support it.

Hi @yurydelendik here is a sample link: http://<IP_ADDRESS>/sviluppo/testpdf.html
Currently I'm:

injecting pdf.js and viewer needed scripts

getting the pdf as blob and converting it into an Uint8Array

refreshing the PdfViewer using PDFViewerApplication.open

There is 404 on catalogo.pdf

@yurydelendik I'm really sorry...example fixed

The generic viewer initialization is called before or after yours (depending on timing):  1) viewer.js cannot find any HTML element it needs to control, 2) or, workerSrc got overridden. Modify viewer.js to control initialization sequence to be in sync with your WaitForPDFJS method. Closing as answered.

@yurydelendik just an add, to handle the issue I managed it:

using require.js to inject the script in order and with depencies
triggering DOMContentLoaded to start viewer.js correctly


Compile Error

I am not able to compile properly (VS2017, v141)
template<unsigned int IIdx, typename TRet, typename ... TArgs>
    static auto CallVFunc(void* thisptr, TArgs ... argList) -> TRet
    {
        using Fn = TRet(__thiscall*)(void*, decltype(argList)...); //19
        return (*static_cast<Fn**>(thisptr))[IIdx](thisptr, argList...); //20
    }

1>c:\users\797876\workenv\__cheating\antario-master\antario-master\antario\utils\utils.h(19): error C2217: '...' requires '__cdecl'
1>c:\users\797876\workenv\__cheating\antario-master\antario-master\antario\utils\utils.h(20): error C2198: 'Fn': too few arguments for call

The error doesn't make any sense to me, maybe you can share some light.

Are you using latest version of c++?

Yes

Update your visual studio to the latest version, thats the compiler bug that detects semicolon where its not there.

I am updating. I hope that fixes it.

Thanks alot, updating to the newest version, and then retargeting the solution fioxed the error!


Nydusd accesses Dragonfly in a mirror registry fashion

At present, nydusd can only access dragonfly as an HTTP proxy. It has some cons:

For HTTPS registry, must provide a copy of CA and private key to dragonfly dfdaemon.
Hardly support multiple mirrors
An extra HTTP connect is needed (a more round trip)

My proposal is to support accessing dragonfly as a pull-through cache.
To achieve it:

add more items to nydusd configuration file:
support filling HTTP header according to a configuration JSON file.

{
  "device": {
    "backend": {
      "type": "registry",
      "config": {
        "scheme": "http",
        "repo": "nydus",
        "host": "<IP_ADDRESS>:8000",
        "mirrors": [{"scheme":"https","host": "dragonfrly.dfdaemon.org:65001", "username": "itsme", "password": "PASS", "header":"X-dragonfaly-header"}]
      }
    },

Furthermore, we can try to configure multiple registries to nydusd when it works globally in snapshotter shared mode

As per the offline discussion with @changweige, nydus can integrate dragonfly in two ways, proxy mode and mirror mode.

proxy mode integration documentation: https://d7y.io/zh/docs/setup/runtime/containerd/proxy/

For example, for https image registry, we need to configure the https decryption certificate for d7y so that it can cache the blob data, but there is a workaround without configuring the certificate:

For nydusd, set to http scheme for the source image registry.
For d7y dfdaemon, enable useHTTPS option, when d7y dfdaemon intercepts the http request, it will automatically replace the http with https to request the source image registry center, thus eliminating the need to decrypt https.

This workaround needs to take into account fallback, when d7y has down, nydusd needs to fallback to the source image registry of https, this feature still needs to be implemented.

mirror mode integration documentation: https://d7y.io/zh/docs/setup/runtime/containerd/mirror

Currently nydus can only support single mirror (d7y fixed configuration mirror url), but not support dynamic multi-mirrors, a feasible design to support multi-mirrors is as follows:
For example, the mirror requirement is:

ctr pull A.com will be mirrored to B.com
ctr pull C.com will be mirrored to D.com

Then when ctr pull A.com, the nydus snapshotter should generate the config.json file for nydusd like this:
{
  "device": {
    "backend": {
      "type": "registry",
      "config": {
        "scheme": "https",
        "repo": "library/nginx",
        "host": "A.com",
        "skip_verify": true,
        "mirrors": [
          {
            "scheme": "http",
            "host": "B.com",
            "headers": {
              "X-Dragonfly-Registry": "https://A.com"
            }
          }
        ]
      }
    }
  },
  ...
}

Similarly, when ctr pull C.com, the generated nydusd config.json is replaced with "X-Dragonfly-Registry": "https://D.com"
This means that the nydus snapshotter should have a configuration of:
<snapshotter startup arguments... >

<nydusd configuration template... >

[[resolver.host. "A.com".mirrors]]
host = "http://B.com"
headers = "X-Dragonfly-Registry": "https://A.com"
insecure = true

[[resolver.host. "C.com".mirrors]]
host = "D.com"
headers = "X-Dragonfly-Registry": "https://C.com"
insecure = true

@zyfjeff Is there any progress on this feature, I think it is instrumental when integrating with Dragonlfy.

@zyfjeff Is there any progress on this feature, I think it is instrumental when integrating with Dragonlfy.

Sorry, I haven't implemented it yet, currently, nydus communicates with dragonfly via https, and it just works.

@zyfjeff Is there any progress on this feature, I think it is instrumental when integrating with Dragonlfy.

Sorry, I haven't implemented it yet, currently, nydus communicates with dragonfly via https, and it just works.

Thanks for feedback ~

@imeoer

For nydusd, when the option "use_http": true is enabled, need to use scheme http to request source registry with proxy, and fallback to https if proxy is unhealthy, this feature still needs to be implemented.

I guess you are suggesting use the original HTTP scheme to access the original registry rather than only https?
Registry host can also be configured as HTTP

Refer to https://github.com/containerd/nydus-snapshotter/pull/178

Let's close this issue since related PR has been merged into mainline


[iOS] Camera doesn't launch

The choose picture from library works perfectly and nothing is wrong with it, but take photo button which should launch the camera doesn't. Using the same exact code from the example, as I said the choosing a picture from library works perfectly just the camera part which doesn't work. Is there a bug currently with the camera launch? Do I need to set permission first for it to work?

Yes it requires permission, which is requested automatically upon the camera opening.
Let's get the obvious questions out of the way - are you on the iOS simulator?

@marcshilling  Yes. Stupid me didn't check the console log until now. I have to build it on a real iphone for it to launch the camera.

No worries...yup just build it on your device and it should work!


[Pallas-Triton] Fix squeeze lowering required sharding argument

Error comes from this PR: https://github.com/jax-ml/jax/pull/25103 if there is no sharding argument passed to squeeze.
I observed my pallas kernel (using squeeze) failure due to the above PR, and the kernel was fine before that PR was merged.

@yashk2810 for https://github.com/jax-ml/jax/pull/25103

Open a new PR (https://github.com/jax-ml/jax/pull/25412) which passed the CLA check


Keyboard Shortcuts' (sic) key input dialog disappears when program loses focus

1 In Keyboard Shortcuts (sic), click a Keybinding (sic)
2 See the (untitled) key combination input dialog:

3 Switch to another open program that does not cover this program's window
Expected: dialog box still present
Observed: dialog box gone.

V1.42.1 on Windows 7.

Most of out dialog boxes disappear when we lose focus, for instance QuickPicks for go to file and the command pallet. I'm not sure that this should be different.

Most of out dialog boxes disappear when we lose focus, for instance QuickPicks for go to file and the command pallet. I'm not sure that this should be different.

I agree. None should be different. I'll report others as I find them.

JacksonKearl closed this 9 minutes ago

Why has this been closed? I've seen no comment disputing that this is a defect.

All temporary inputs in the application disappear on window blur. There's no reason to "fix" over "fixing" the others, and I'm as of yet unconvinced that the current behaviour is broken.

All temporary inputs in the application disappear on window blur.

"temporary inputs"??

There's no reason to "fix" over "fixing" the others

Agreed 100%. All are equally deserving.

and I'm as of yet unconvinced that the current behaviour is broken.

I can see no advantage to this behaviour, let alone one sufficient to compensate for the disadvantage to the user e.g. throwing away his VS Code input in progress when a calendar program pops-up a reminder.
Which is probably why I've never seen another Windows program that behaves so badly in this regard. That includes not just programs using Windows native UI API, but cross-platforms UIs too.

If the input closes when it loses focus, it also closes when the window loses focus.

... the input closes when it loses focus...

Schoolboy error.

bot wrote:

This feature request is now a candidate for our backlog.

Where did the bot get the idea this is a feature request? It is not. It is a bug report.

I do not think it is a bug as the current behaviour is intended. This request is to change the behaviour.

I do not think it is a bug

Since I doubt the bot read your mind :) my question stands.

This feature request

This is a bug report, not a feature request. The "feature-request" tag wasn't added by me, and I can find no way to remove it. Would someone please remove it? Thanks.

this feature request

This is not a feature request. This is a bug report. Would the person who falsely tagged this "feature request" (or anyone else) please remedy that? Thanks.


[AIRFLOW-1187] Fix PyPi package names in documents

Dear Airflow maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA

[x] My PR addresses the following Airflow JIRA issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"

https://issues.apache.org/jira/browse/AIRFLOW-1187

Description

[x] Here are some details about my PR, including screenshots of any UI changes:

As well as AIRFLOW-1185, install commands in documents should be updated as "pip install apache-airflow".
Tests

[x] My PR adds the following unit tests OR does not need testing for this extremely good reason:

It requires no test since it's just a documentation fix.
I ran docs/build.sh locally and confirmed that the modified files were properly rendered as html.
Commits

[x] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

LGTM


is it possible to use a future in within the itemBuilder ?

Hello,
The item data I am using for building my paginated list requires a second Firestore query to get fully displayed.
The problem is that I can't figure out how to manage that, because the itemBuilder seems not accepting any future.
Do you have any example that could help with this ?
Thank you,
Benjamin

Hi @ben55j. Thanks for using the package. itemBuilder does not support future now. Instead, you can wrap each item with a FutureBuilder
Feel free to reopen this if you face this issue again.


farrah/P2PS-4247/feat: add uz language

Updated the translations package and added UZ language.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>2
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 85.8%

Totals

Change from base Build<PHONE_NUMBER>1:
0.0%

Covered Lines:
6428

Relevant Lines:
6980

💛  - Coveralls


Add more intelligent agent rebuilding

Previously, we didn't rebuild the agent if the binaries were available on disk. This was problematic because it meant that we don't automatically rebuild the binaries when bumping up the Cody commit. This PR fixes the problem by generating a new cody-revision.txt file alongside the agent binaries. This revision file includes the cody commit hash, and we automatically rebuild the binaries if this commit hash doesn't match the cody commit hash in the build.
Test plan

:runIde, confirm the agent is rebuild
:runIde again, confirm the agent is cached
Update codyCommit hash to a new commit hash
:runIde again, confirm the agent is rebuilt

Closing this. Feel free to pick it up you want this feature


watcher is giving a 500 error

Issue Description

I have created my first watcher in sentinl but whenever I try to execute it I get a 500 error. When checking the logs, the error is:
[parsing_exception] no [query] registered for [filtered], with { line=1 & col=22 } :: {\"path\":\"//_search\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"filtered\\\":{\\\"query\\\":{\\\"match\\\":{\\\"Properties.Environment\\\":{\\\"query\\\":\\\"Prod\\\",\\\"type\\\":\\\"phrase\\\"}}}}}}\",\"statusCode\":400,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"parsing_exception\\\",\\\"reason\\\":\\\"no [query] registered for [filtered]\\\",\\\"line\\\":1,\\\"col\\\":22}],\\\"type\\\":\\\"parsing_exception\\\",\\\"reason\\\":\\\"no [query] registered for [filtered]\\\",\\\"line\\\":1,\\\"col\\\":22},\\\"status\\\":400}\"}\n    at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:295:15)\n    at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:254:7)\n    at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:159:7)\n    at IncomingMessage.bound (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/dist/lodash.js:729:21)\n    at emitNone (events.js:91:20)\n    at IncomingMessage.emit (events.js:185:7)\n    at endReadableNT (_stream_readable.js:974:12)\n    at _combinedTickCallback (internal/process/next_tick.js:80:11)\n    at process._tickDomainCallback (internal/process/next_tick.js:128:9)
The watcher RAW is as followed. Could someone point me in the right direction because I really can't see what I'm doing wrong here.

Additional Details

{
  "_index": "watcher",
  "_type": "sentinl-watcher",
  "_id": "7cs3nm28w76-oa4dpnes4jo-xt31xx7nyb",
  "_version": 7,
  "found": true,
  "_source": {
    "title": "watcher_title",
    "disable": true,
    "report": false,
    "trigger": {
      "schedule": {
        "later": "every 5 mins"
      }
    },
    "input": {
      "search": {
        "request": {
          "index": [],
          "body": {
            "query": {
              "filtered": {
                "query": {
                  "match": {
                    "Properties.Environment": {
                      "query": "Prod",
                      "type": "phrase"
                    }
                  }
                }
              }
            }
          }
        }
      }
    },
    "condition": {
      "script": {
        "script": "payload.hits.total > 0"
      }
    },
    "actions": {
      "email_admin": {
        "throttle_period": "0h15m0s",
        "email": {
          "to"<EMAIL_ADDRESS>          "from": "sentinl@localhost",
          "subject": "Sentinl Alarm",
          "priority": "high",
          "body": "Found {{payload.hits.total}} Events"
        }
      }
    },
    "transform": {}
  }
}

@graphicsxp The filtered query is deprecated, use bool instead.

Indeed. I was able to fix this by changing filtered with filter and also by removing the query property inside the filtered property.
thanks


Feat(galoy-slider-button): new sliding button implemented on send-bitcoin-confirmation-screen

@nicolasburtey @UncleSamtoshi I revamped the slider component a bit. Let me know if you think it feels satisfying and if the UI is cool.
PS This is a draft cause I still have yet to get my XCode working. Gotta test on iOS to ensure it's all good.  And looks like I've gotta see what's going on with the e2e swiping test too.

@nicolasburtey @UncleSamtoshi I revamped the slider component a bit. Let me know if you think it feels satisfying and if the UI is cool.
Another thing to note is I don't believe the handleSendPayment function is correctly handling all error cases. There's a variety of things that could happen (I tried sending 10 sats and it couldn't calculate the fee and broke the handleSendPayment call) but regardless, there should be some more feedback there. The reason I'm bringing this up is because that can make the slider button hang. This has nothing to do with the slider component, and is instead related to the parent component not updating the loading state correctly.
PS This is a draft cause I still have yet to get my XCode working. Gotta test on iOS to ensure it's all good. And looks like I've gotta see what's going on with the e2e swiping test too.

something to check is whether this work for our end 2 end test also. this is a tricky thing to get right because it's not just a tap button (it's actually the goal of the slider!) but therefore the tests should also reproduce the human aspect of sliding the button to confirm
the tests are not running here probably because the PR is from your username and and not the main repo I guess? the e2e tests for Apple needs to run on a Mac machine and can't be run in github actions, so we use circle for that.

This is ready for review except for the e2e testing. Up to the team how soon you want the button integrated cause I have a feeling it'll be several days before e2e is fully functional.

This is ready for review except for the e2e testing. Up to the team how soon you want the button integrated cause I have a feeling it'll be several days before e2e is fully functional.

@sandipndev ?

@MaxwellDG can you please follow conventional commits - also in the title of the PR (as they get squashed on merge).
ie: feat(...):. ... and not uppercase.


feature: support discovery center

Fix #1279

@membphis @moonming review please.

There are too few test cases.

Some test cases have been added.


Produce binary release tarballs for the TensorFlow C API

These scripts are intended to be run with every release to
produce libtensorflow.tar.gz for CPU and GPU on Linux and OS X
for x86_64 architecture machines.
(Eventually there will be other operating systems and architectures).
These binary releases are then intended to make use of other language
bindings (such as Rust, Haskell, Go) easier
as the common case would be to download the binary C-library release and
avoid the need to build TensorFlow from source (and all the time and
external dependencies doing so entails).
Files:

tensorflow/tools/ci_build/builds/libtensorflow.sh - Baseline common script to build a tarball
tensorflow/osx/libtensorflow_{cpu,gpu}.sh - Build tarballs for OS X with and without GPU support
tensorflow/linux: Has similar top level scripts, but the build happens in a docker container so it contains 4 files - the two top level builds, one shared libtensorflow_docker.sh that is used to build and execute the docker container and libtensorflow.sh which is the script run inside the container.

Jenkins, test this please
The test failures seemed unrelated, retrying.

Jenkins, test this please

both failures are known issues. Merging.


Keyword in comments and strings should not be considered independently

When keyword is inside of string or comment it is colored as independent token:

#26


TextEdit with syntax highlighting will show incorrect characters in its borders

Godot version:
86a74e2
Issue description:
If a TextEdit has syntax highlighting enabled, the characters that are in the very border will be incorrect:

CC @Paulb23


Adds cache management classes for holding cache information.

Part of #52

Is the cache layer location configurable? Can we potentially share a cache with a docker daemon? Or at least share a cache between all a user's projects on a single machine?

Yea, the cache location will be configurable. From looking around, it seems inpractical to share the cache with a Docker daemon since at least on macOS, the Docker daemon stores all its images in one large file at ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2. However, it is feasible to share a cache between all a user's projects on a single machine.