adapty.addEventListener() not receiving any event

Description
Google Play RTDN is working fine, and I am able to see the events with no noticeable delay in Adapty (web) Event Feed. The listener however is not returning anything. Adapty is initialized correctly, and subscription purchase made through the paywall. No updated profile received through the listener when purchase is made or cancelled.
   useEffect(() => {
    if (userId) {
        console.log('activating adapty :', userId)
        adapty.activate('***', { customerUserId: userId })
            .then(() => getSubStatus()
                .then((sub) => setSub(sub))
            )
            .catch((e) => console.error('Unable to activate adapty:', e.message)
            )

        adapty.addEventListener('onLatestProfileLoad', profile => {
            console.log('adapty sub listener incoming:', profile);
            setSub(profile.accessLevels?.premium);
        });
    }

    return adapty.removeAllListeners()

  }, [userId]);

Version
v2.4.0
What platforms are you seeing the problem on?
Android
System info
info Fetching system and libraries information...
System:
    OS: Windows 10 10.0.22621
    CPU: (16) x64 AMD Ryzen 7 4800U with Radeon Graphics
    Memory: 2.79 GB / 15.37 GB
  Binaries:
    Node: 18.1.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.19 - ~\AppData\Roaming\npm\yarn.CMD
    npm: 8.19.2 - ~\AppData\Roaming\npm\npm.CMD
    Watchman: Not Found
  SDKs:
    Android SDK: Not Found
    Windows SDK:
      AllowDevelopmentWithoutDevLicense: Enabled
      AllowAllTrustedApps: Enabled
      Versions: 10.0.17763.0, 10.0.19041.0
  IDEs:
    Android Studio: Not Found
    Visual Studio: Not Found
  Languages:
    Java: 11.0.2
  npmPackages:
    @react-native-community/cli: Not Found
    react: 18.0.0 => 18.0.0
    react-native: 0.69.6 => 0.69.6
    react-native-windows: Not Found
  npmGlobalPackages:
    *react-native*: Not Found
info React Native v0.71.4 is now available (your project is running on v0.69.6).
info Changelog: https://github.com/facebook/react-native/releases/tag/v0.71.4.
info Diff: https://react-native-community.github.io/upgrade-helper/?from=0.69.6.
info To upgrade, run "react-native upgrade".

I think it is worthwhile to note that I'm running expo development build on expo SDK 46. Firebase has a strange problem with expo development build where the network will not establish a long lived http connection over port 80. And I have to use long pooling option that open a new connection every 30 seconds to simulate that long lived connection. I wonder if this listener issue is suffering from the same.

Hello @leons1767!
Another our customer had some problems on Expo SDK below 47. It was different from yours, but updating helped!
I certainly received events in my apps! I'll scout things out with this issue tomorrow morning
For the time being, I have a hunch-hypothesis:
Can you try locking that thread until Adapty is activated? Just add await to activate call, so Adapty is activated before addEventListener is called.

Hello @divanc
Thank you for your time. I tried the following.
Upgraded Expo to SDK 48, and the result is the same as SDK 46. I downgraded it back to SDK 46 because of compatibility reason.
Moved the adapty.addEventListener() to the .then block of adapty.activate(), it doesn't work either.
Used await like you suggested, still not working.
Ran an apk preview build directly on device without expo start --dev-client. Also, not working.

Hello @divanc
Any update on the issue?

Hello @divanc
Is there a way to check the state of the listener, to know if it is even listening, or output the error of the listener?
When I set logLevel to 'verbose', and I could find
[Adapty JS vUNKNOWN] — verbose: bridgeCall: Calling native method. SUCCESS
	methodName="activate"
	args="{\n  \"sdk_key\": \"***removed***\",\n  \"observer_mode\": false,\n  \"user_id\": \"***removed***\",\n  \"log_level\": \"verbose\"\n}"

follow by
[Adapty JS vUNKNOWN] — verbose: Adapty.addListener: Registering new listener
	event="onLatestProfileLoad"

but there is no response log, to suggest the registration is successful.

Hello again, @leons1767. Let me answer your questions, before I will be able to test Android Expo today

Is there a way to check the state of the listener, to know if it is even listening, or output the error of the listener?

addListener would not send an event if there is an error, that you can find in your logs. While verbosing, there are only two addListener logs now:

Registering new listener meaning new listener is about to be added
Failed to decode profile in event listener error meaning native SDK sent an event with invalid body

but there is no response log, to suggest the registration is successful. However, when I remove the listener, there seems to be one.

There is no log about successful listener registration as it does not throw. I'm currently studying how I can make verbose logs perfectly clear, so thanks for pointing out misunderstanding point.

Thanks @divanc for taking time. Looking forward to hearing from you about the Android Expo test.

I can confirm, that events do not seem to be sent in Android with Expo SDK.
I've already fixed this issue locally and currently testing it, please stay tuned!
@leons1767

@leons1767 I've tested on latest Expo SDK, you seem to have the same version. Can you please try v2.4.3?

@divanc Thank you very much for your work. I tried v2.4.3 with Expo SDK 46, and I confirm the listener is now receiving events.

@divanc
I'm sorry, but it seems that only the first listener is receiving events, when there are 2 registered listeners from 2 different devices.
The use case is users logged on to two devices at the same time (hence two listeners, one listener per device), and only one device is receiving the event.

Oof, that is interesting case. Can you elaborate?
Do you make a purchase and receive event only on the same device?

@divanc Yes, Device 1 made a purchase and received the event on Device 1. When a subscription cancellation is made in Google Play Store on Device 1, Device 1 also received the subscription cancellation event. Device 2 received none of the events.  Both devices activated adapty using the same customerId, and subscribed to the listener the same way.

Hello @divanc
Were you able to make some progress on the issue?

@leons1767, I'm very sorry for being absent. A load of priority task fell on me :(
I'm going to try double-sim tomorrow, but there is a part you need to know.
addEventListener works momentarily only for the current device. If you have purchased on the device X, event should fire right after the successful modal (sometimes a little slow on sandboxes due to tons of history).
If you have purchased a product on device X and expecting device Y to fire update event that should happen by the following mechanism:
Adapty (if possible) fetches a profile info once in a minute, then compares two strings. If those are different, SDK fires an event. That should only mean, that other devices can have a slightly-delayed event resolution.
I've said all this without testing yet, let me return to you tomorrow

Hello @divanc , thanks for the update and explanation about the event. I'm still following the issue, and hoping for a fix soon.

@leons1767
I'm sorry, but I was unable to reproduce it with any scenario. Here is what I've tried

Two apps open, one buys, second receives (on a video)
Two apps open, one device is locked, unlocked after a purchase
One app open, second opened in the midst of a purchase

Here is a video how it looks for all of my cases:
https://share.cleanshot.com/wX40fD8K
I'm sorry, but I'm gonna need to have some more data to discover the problem...

Hello @divanc.
Thank you for your time to try to reproduce the issue, I don't know what to say.
After updating to v2.4.6, I found the profile on the second device not getting updated even when the app restarted, I have to do adapty.restorePurchases() to get the latest profile (with the purchase from first device). I could see some events in the second device, the same event sent 3x, and they were not the latest event.
I'm letting this go, because even if I get it to work like simulated by you, there would be a significant delay between purchase and the listener receiving the event in the second device. I have done a workaround by writing the new expiryDate to an existing document that the devices are listening.
Again, thank you very much for your time.

Thank you! If you feel like this becomes bothersome, feel free to re-open!


[efr32] Common Platform Abstraction Layer

This merges the PAL sources for all efr32-based platforms
Summary of changes

examples/platforms/efr32mg*/ moved inside examples/platforms/efr32/
Added gecko_sdk_suite/v3.0/platform/radio/rail_lib/plugin/pa-conversions/pa_conversions_efr32.c to third_party/silabs/Makefile.am

Helps with merging radio.c

Closes #5492 - For series1 parts use -mfloat-abi=softfp
Closes #5502 - UART Initialization overrides hal-config baud-rate value with a constant
Closes #5493 - remove call to CMU_ClockSelectSet() in system.c

@jwhui Would you be able to add the hacktoberfest-accepted label to this PR for Hacktoberfest?
https://hacktoberfest.digitalocean.com/hacktoberfest-update

CC: @romacdon @schinchani

@jwhui Would you be able to add the hacktoberfest-accepted label to this PR for Hacktoberfest?
https://hacktoberfest.digitalocean.com/hacktoberfest-update

@lmnotran , done!

Thanks @jwhui !

@jwhui Do you know why the Pretty check is failing? Running ./script/make-pretty doesn't result in any changes to files

@lmnotran , I'm not sure why running make-pretty does not result in file changes. However, the log does show this error:
 *** ERROR:  examples/platforms/efr32/src/system.c misses EOF newline.

@jwhui Thanks! This should make efr32 development easier for everyone


Installation Bug: pydantic error

🐛 Bug
I tried to install napari as a python packageon a windows 10 machine, but got a TypeError for issubclass() in pydantic\fields.py, line 668.
To Reproduce
In anaconda prompt, I typed:
conda create -y -n napari-env -c conda-forge python=3.9
conda activate napari-env
Then:
pip install napari[all]
Then:
napari
and I received the error:
File "C:\Users\andrew\anaconda3\envs\napari-env\lib\typing.py", line 852, in subclasscheck
return issubclass(cls, self.origin)
TypeError: issubclass() arg 1 must be a class
Expected behavior
I expected this to open up the napari GUI
Environment
I can't type napari info because I get the above error.
Thanks for any help.

this is upstrem pydantic bug: https://github.com/pydantic/pydantic/issues/5821
You need either install typing extension in version 4.5.0 or use python 3.10

Thank you!

The bugfix of pydantic was released. It is not on conda-forge yet but should land shortly.


I want to underline/line-through a label

[x] I have searched the issues of this repository and believe that this is not a duplicate.

What problem does this feature solve?
When a new Node or a new Edge is defined, or at the new Graph instantiating, it's possible to configure the label style with fontSize, fontFamily and other properties. But it's not possible to configure the CSS property text-decoration who makes the label line-through, underline and overline.
Same for the Text shape.
What does the proposed API look like?
I propose to add the new property textDecoration to the Text shape, who corresponds to the property in CSS is text-decoration, like it's done for font-variant and the other CSS font properties.

Since the basic shapes are corresponding to the those in Canvas API, where the text-decoration is not supported, G6 won't consider it into text shape too.
But you can still achieve it by adding a path shape under/trough the text.

Hi, dear user. Since there was no response for a long time, we cannot figure out the answer of the problem you described. If there is new progress, feel free to open a new issue.

Add a path shape under the text could be a solution. But if the length of the text is not sure, how can I know the length of a path shape


Fail to download the dataset

🐛 Bug

To Reproduce
Steps to reproduce the behavior:

Go to examples folder
Run python gcn.py

Expected behavior

Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.x
Traceback (most recent call last):
File "/home/hzcheney/miniconda3/lib/python3.8/urllib/request.py", line 1350, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/home/hzcheney/miniconda3/lib/python3.8/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/home/hzcheney/miniconda3/lib/python3.8/http/client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/home/hzcheney/miniconda3/lib/python3.8/http/client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/home/hzcheney/miniconda3/lib/python3.8/http/client.py", line 1010, in _send_output
self.send(msg)
File "/home/hzcheney/miniconda3/lib/python3.8/http/client.py", line 950, in send
self.connect()
File "/home/hzcheney/miniconda3/lib/python3.8/http/client.py", line 1417, in connect
super().connect()
File "/home/hzcheney/miniconda3/lib/python3.8/http/client.py", line 926, in connect
self._tunnel()
File "/home/hzcheney/miniconda3/lib/python3.8/http/client.py", line 900, in _tunnel
(version, code, message) = response._read_status()
File "/home/hzcheney/miniconda3/lib/python3.8/http/client.py", line 276, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "gcn.py", line 17, in
dataset = Planetoid(path, dataset, transform=T.NormalizeFeatures())
File "/home/hzcheney/miniconda3/lib/python3.8/site-packages/torch_geometric/datasets/planetoid.py", line 55, in init
super(Planetoid, self).init(root, transform, pre_transform)
File "/home/hzcheney/miniconda3/lib/python3.8/site-packages/torch_geometric/data/in_memory_dataset.py", line 53, in init
super(InMemoryDataset, self).init(root, transform, pre_transform,
File "/home/hzcheney/miniconda3/lib/python3.8/site-packages/torch_geometric/data/dataset.py", line 89, in init
self._download()
File "/home/hzcheney/miniconda3/lib/python3.8/site-packages/torch_geometric/data/dataset.py", line 141, in _download
self.download()
File "/home/hzcheney/miniconda3/lib/python3.8/site-packages/torch_geometric/datasets/planetoid.py", line 105, in download
download_url('{}/{}'.format(self.url, name), self.raw_dir)
File "/home/hzcheney/miniconda3/lib/python3.8/site-packages/torch_geometric/data/download.py", line 34, in download_url
data = urllib.request.urlopen(url, context=context)
File "/home/hzcheney/miniconda3/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/home/hzcheney/miniconda3/lib/python3.8/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/home/hzcheney/miniconda3/lib/python3.8/urllib/request.py", line 542, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/home/hzcheney/miniconda3/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/home/hzcheney/miniconda3/lib/python3.8/urllib/request.py", line 1393, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "/home/hzcheney/miniconda3/lib/python3.8/urllib/request.py", line 1353, in do_open
raise URLError(err)
urllib.error.URLError:
Environment

OS: Linux labManjaro 5.10.2-2-MANJARO #1 SMP PREEMPT Tue Dec 22 08:14:42 UTC 2020 x86_64 GNU/Linux
Python version: 3.8.5
PyTorch version: 1.7.1 + CPU
CUDA/cuDNN version: NONE
GCC version: gcc version 10.2.0 (GCC)
Any other relevant information: http_proxy used

Additional context

I tried to download the datasets through browser in the same network environment and it works fine, is this a protection strategy of github website when requests module open a url without headers ?Or this bug depends on the network environment that i am using?
Thank youu very much for your time!

I'm not sure what exactly is causing it, as it works fine for me. I'm happy to fix it in case you find the reason for why
urllib.request.urlopen(https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.x)

fails on your side. Alternatively, you can simply download the data by yourself and put them into the raw directory.

Thanks a lot !I now believe that this is caused by the http_proxy variable.If I unset the http_proxy then everything becomes normal.


Shift transducer?

Is it possible to have a transducer that shifts an iterator by an offset that can be positive or negative? The items pass the end of the iterator would be placed in the beginning cyclically.

To cycle an iterator, you can use Transducers.Inject(Iterators.cycle(itr)): https://tkf.github.io/Transducers.jl/dev/manual/#Transducers.Inject. Or you can also just zip it with the input collection.
For shifting an iterator, I think the easiest approach is to pre-process it with circshift[!].


enable_hive not working with raydp

Configuration for running spark in raydp :
default_spark_conf = {
"spark.jars.packages": "mysql:mysql-connector-java:8.0.32",
"spark.jars": "/home/ray/.ivy2/jars/com.mysql_mysql-connector-j-8.0.32.jar",
"spark.hadoop.javax.jdo.option.ConnectionDriverName": "com.mysql.jdbc.Driver",
"spark.hadoop.javax.jdo.option.ConnectionURL": "jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true",
"spark.hadoop.javax.jdo.option.ConnectionUserName": "test",
"spark.hadoop.javax.jdo.option.ConnectionPassword": "",
"spark.sql.catalog.spark_catalog.type":"hive",
"spark.sql.catalogImplementation":"hive"
}
spark = raydp.init_spark(
app_name="Darwin_SPARK",
num_executors=1,
executor_cores=1,
executor_memory='4G',
enable_hive = True,
configs=default_spark_conf)
Getting error when trying to create a table like
`df = spark.createDataFrame([
(1, "Smith"),
(2, "Rose"),
(3, "Williams")
], ("id", "name"))
df.write.mode("overwrite").saveAsTable("employees12")`
Stack Trace
2023-05-22 10:45:09,515 WARN HiveMetaStore [Thread-5]: Retrying creating default database after error: Unexpected exception caught. javax.jdo.JDOFatalInternalException: Unexpected exception caught. at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1203) ~[javax.jdo-3.2.0-m3.jar:?] at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:814) ~[javax.jdo-3.2.0-m3.jar:?] at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:702) ~[javax.jdo-3.2.0-m3.jar:?] at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:521) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:550) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.ObjectStore.initializeHelper(ObjectStore.java:405) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:342) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:303) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:79) ~[hadoop-client-api-3.3.2.jar:?] at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:139) ~[hadoop-client-api-3.3.2.jar:?] at org.apache.hadoop.hive.metastore.RawStoreProxy.(RawStoreProxy.java:58) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:628) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:594) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:588) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:655) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:431) ~[hive-metastore-2.3.9.jar:2.3.9] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_362] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_362] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_362] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_362] at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:79) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:92) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6902) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:162) ~[hive-metastore-2.3.9.jar:2.3.9] at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:70) ~[hive-exec-2.3.9-core.jar:2.3.9]
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.datanucleus.util.NucleusLogger at org.datanucleus.plugin.PluginRegistryFactory.newPluginRegistry(PluginRegistryFactory.java:58) at org.datanucleus.plugin.PluginManager.(PluginManager.java:60) at org.datanucleus.plugin.PluginManager.createPluginManager(PluginManager.java:430) at org.datanucleus.AbstractNucleusContext.(AbstractNucleusContext.java:85) at org.datanucleus.PersistenceNucleusContextImpl.(PersistenceNucleusContextImpl.java:167) at org.datanucleus.PersistenceNucleusContextImpl.(PersistenceNucleusContextImpl.java:156) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.(JDOPersistenceManagerFactory.java:415) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:304) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:213) ... 80 more

Did same with pyspark SparkSession and its working
from pyspark.sql import SparkSession
spark = SparkSession.builder.appName("Spark Examples")
.config("spark.hadoop.javax.jdo.option.ConnectionURL", "jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true")
.config("spark.hadoop.javax.jdo.option.ConnectionDriverName", "com.mysql.cj.jdbc.Driver")
.config("spark.hadoop.javax.jdo.option.ConnectionUserName", "test")
.config("spark.hadoop.javax.jdo.option.ConnectionPassword", "")
.config("spark.sql.catalogImplementation","hive")
.config("spark.sql.catalog.spark_catalog.type","hive")
.config("spark.jars","/home/ray/.ivy2/jars/com.mysql_mysql-connector-j-8.0.32.jar")
.config("spark.jars.packages", "mysql:mysql-connector-java:8.0.32")
.enableHiveSupport().getOrCreate()

Hi @raviranak ,
I think what you set in spark.jars and spark.jars.packages should be the same package, right? That should be able to work around #247 .
Seems like org.datanucleus.util.NucleusLogger is not presented in executors' classpath. Where is this jar? Is it a dependency of Hive?

This is happening only for spark 3.3.1 version of raydp

Added these to config during raydp.init_spark
configs
"raydp.executor.extraClassPath":"/home/ray/.ivy2/jars/org.datanucleus_datanucleus-core-5.0.9.jar"
But in log i see
Warning: Ignoring non-Spark config property: raydp.executor.extraClassPath

This is happening only for spark 3.3.1 version of raydp

Do you mean you only tested this version? Or that it works in other versions of Spark?

But in log i see
Warning: Ignoring non-Spark config property: raydp.executor.extraClassPath

Yes, you'll see such log, but the config should be set to executors by our code. Did it work?

This works only on ray runtime 2.0.0 but not on 2.3.0 as in 2.3.0 its using spark 3.3.1 , could you please check

Seems Spark initilisation differs in raydb and using native SparkSession . Could you please update with resolution of this, seem like some missing jars using raydp which is not occuring with pyspark

When you are testing, are you using the same Spark install for RayDP? We add the Spark's jar dir to RayDP's class path. What jars did you find missing in RayDP?

This works only on ray runtime 2.0.0 but not on 2.3.0 as in 2.3.0 its using spark 3.3.1 , could you please check

You should be able to use other Spark versions as well, no matter Ray 2.3.0

Yes using the same spark version i.e 3.3.1 , could you test raydp spark 3.3.1 for enable_hive

Could not initialize class org.datanucleus.util.NucleusLogger , could you check for raydp if there is any jar that could cause the nucleous-core different version to be picked while using raydp , this seems to be only probable cause @kira-lin

could you please provide instructions to reproduce this issue? How should I configure Hive and MySQL?

@raviranak This could be the issue with using spark.jars.packages and spark.jars on raydp, did you try to place all the jars and their transitive dependencies under the jars dir in your Spark installation?
This thread might help: https://github.com/oap-project/raydp/issues/247
@kira-lin @carsonwang If that is the case.. we might want to take a look to fix spark.jars.packages and spark.jars on RayDP

@kira-lin this can be easily reproduced even without data source for hive like this
import raydp
spark = raydp.init_spark(
app_name="Darwin_SPARK",
num_executors=1,
executor_cores=1,
executor_memory='4G',
enable_hive = True,
configs=None)
spark.sql("show tables")
This doesn't work with ray runtime 2.3.0 , but works o ray 2.0.0
@pang-wu I have set the jars appropriately

I tried your script with Ray 2.3.0, Spark 3.3.1, and RayDP 1.6.0b20230527.dev0, and get the following
>>> import raydp
>>> spark = raydp.init_spark(
... app_name="Darwin_SPARK",
... num_executors=1,
... executor_cores=1,
... executor_memory='4G',
... enable_hive = True,
... configs=None)
ERROR StatusLogger Reconfiguration failed: No configuration found for '18b4aac2' at 'null' in 'null'
ERROR StatusLogger Reconfiguration failed: No configuration found for 'Default' at 'null' in 'null'
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
23/06/06 11:25:39 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
>>>
>>> spark.sql("show tables")
2023-06-06 11:25:45,764 Thread-4 ERROR Reconfiguration failed: No configuration found for '1ff4c785' at 'null' in 'null'
DataFrame[namespace: string, tableName: string, isTemporary: boolean]

Is this the error you said?

@kira-lin this can be easily reproduced even without data source for hive like this
import raydp spark = raydp.init_spark( app_name="Darwin_SPARK", num_executors=1, executor_cores=1, executor_memory='4G', enable_hive = True, configs=None)
spark.sql("show tables")
This doesn't work with ray runtime 2.3.0 , but works o ray 2.0.0 and gives same exception trace
@pang-wu I have set the jars appropriately

I also tried the latest Ray 3.0.0.dev, RayDP 1.6.0.dev and Spark 3.3.2. It worked well.
Can you share your RayDP version?

Can you try RayDP nightly and see if it fix your problem? You can install it like pip install raydp -u --pre

Working on 1.6.0.dev ,can you please help with release timeline for raydp 1.6.0

Working on 1.6.0.dev ,can you please help with release timeline for raydp 1.6.0

@raviranak 1.6.0 will be released by end of this month.

thanks @carsonwang @kira-lin

This is still occurring in raydp 1.6.0


New keyboard leads to lost key presses ;-)

From the README:

This is an AI designed keyboard layout that was built within the keyboard-gentics project.

Too bad AI didn't write that README, it probably wouldn't make typos like that (highlighted, also repeated in another part of README, which leads to concerns that fingers motions with the new keyboard are such that in a key sequence of "n", "e", "t" - the "e" gets often lost ;-) ).

this is an issue I've noticed even using dvorak, letters next to each other in a word but on opposite sides of the keyboard can get swapped sometimes since I accidentally hit them otu (<- case in point lmao, I wasn't trying to do that) of order

just submit a PR to fix the typo; typos happen regardless of keyboard layout


Consider releasing Roslyn Analyzers and Fixers

Roslyn analyzers and fixers work as part of the compiler and can be used in any IDE or tooling the supports Roslyn analyzers.
The Roslyn compilers can output diagnostics in SARIF formant (-errorLog) that can be used by other tooling.

Thanks for the suggestion.  What we are doing doesn't exactly fir the spirit of a roslyn analyzer which analyzes the syntax.  DevSkim is performing Regexes against the text of the file (and thus works on multiple file types).

It does fit the spirit of a Roslyn analyzer. Except that it does it right.
If DevSkim is running regular expressions against the text of the file, I can trick it very easily to give false values.

Thanks for your feedback. We aren’t going to explore this at this time.
On Mon, Feb 1, 2021 at 11:46 AM, Paulo Morgado<EMAIL_ADDRESS>wrote:

It does fit the spirit of a Roslyn analyzer. Except that it does it right.
If DevSkim is running regular expressions against the text of the file, I can trick it very easily to give false values.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

@paulomorgado Just to add a little to this -- static analyzers make tradeoffs in different areas -- the advantage of Roslyn analyzers  are execution speed, disadvantages (IMHO) are complexity of rule creation and the languages that can be analyzed. CodeQL isn't usually as performant, but enables for a much deeper analysis, and still a relatively high cost of rule creation.
DevSkim goes at it from the other direction -- rules are very easy to write and are language agnostic. This means that yes, it's easy to "trick" DevSkim into not finding an issue, and there can be more false positives.
There's no "best" tool for all jobs -- we think that DevSkim has a useful spot in the mix, similar to Semgrep, but we don't want to try to make DevSkim something that it isn't.
If your original comment was about creating new Roslyn rules to detect more security bugs, that would always be helpful, but it isn't something that we (the folks managing DevSkim) are actively working on. You might want to consider security-code-scan (NuGet) in case they've already implemented what you're looking for.

Thank you for you explanation, @scovetta.
Is there a descriptive list of all implemented rules?

https://github.com/microsoft/DevSkim/tree/main/rules/default/security
The default rules are stored in the source code here.

Thanks.


Breaks tests with the default CRA setup

Hi,
The package which is published to npm is using import syntax and it breaks tests when your are using default setup created by create-react-app.
I get this error when running tests which use react-bootstrap-sweetalert:
({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,global,jest){import SweetAlert from "./components/SweetAlert";
                                                                                ^^^^^^

    SyntaxError: Cannot use import statement outside a module

I've created this simple example where you can see the issue
https://github.com/mjetek/test-react-bootstrap-sweetalert

resolved with v5.1.9


Allow same version of IDS with different build options

I would like to have multiple configurations of the same engine and verison..
For Example. I'd like to have both Suricata 4.1.4 without Rust and Suricata 4.1.4 with Rust.
At first thought, this would be as simple as adding another sensor, dockerfile.   But in practice because of the way the SENSOR_TECHNOLOGY string is auto determined, it's more complex.
My first thought was to create an ENV variable in the Dockerfile which contained the SENSOR_TECHNOLOGY string.  During build time (of the docker image), with the exception of the engine type and version are known, with the exception of the support for "current".
This can probably be addressed in dalton-agent.py by checking for the env variable first and default to the current "auto" method is not.
I am interested in hearing the best way to approach this. I'm more than willing to attempt a PR with some direction.

here is an attempt based on a setting a specific SENSOR_TECHNOLOGY via specific Dockerfiles
https://github.com/zoomequipd/dalton/commit/a3135eac4e302c81a2c22ec8cca29357d1d406d5

Comments at https://github.com/zoomequipd/dalton/commit/a3135eac4e302c81a2c22ec8cca29357d1d406d5#commitcomment-33397212

If I clean up that branch a bit and update README to demonstrate this use case, you ok with me submitting a PR on it?

PRs are welcome although if we are going to update the main repo, I think there may be a slightly easier/better way to support Rust and non-Rust Suricata 4 agents, now that I think about it.  Let me me mess around with it and get back with you.

Take a look at this branch at see what you think -- https://github.com/counterthreatunit/dalton/tree/agent-suricata-rust.  I haven't updated docker-compose.yaml yet so you'll have to add a specification for a rust agent in there; see README for example.  You can also specify a "current" Rust agent if you want too.

It looks like a much more complete implementation that fits my current needs. I worry about it being too specific.  For instance, lets say I want one with hyperscan next, or for snort one with OpenAppID and one without, or suricata with a different version of libhtp, etc.

I agree, it is specific (to Rust support), but it kind of needed to be done anyway in anticipation of Suricata 5.0 coming out in less than two months, in which Rust support will no longer be optional.
For your use cases, my suggestion would be to build the agents however you want, either as Docker agents or not; see https://github.com/secureworks/dalton#adding-sensors.  Then, if necessary, tweak the dalton-agent.conf file(s), and add engine conf files.  I don't think a PR is necessary as such sundry customizations are probably best kept on on the forks of people who have specific use for them, and not part of the canonical repo.
I did make it certain that if the SENSOR_TECHNOLOGY matches the filename of a corresponding conf file (minus the file extension), the the matching conf file will be used:
https://github.com/secureworks/dalton/blob/82571dfa9ffd60cf9ceaf86a13693c4287bf06e3/app/dalton.py#L410-L411


Video Overview, Formats, Getting Started: documentation updates

This resolves #5400

@muuki88 I've made all changes on the three pages (see GDoc for reference: https://docs.google.com/document/d/1Fb7mxhlGjEskJwjBntgoO2HWtNcyEuVaGlFeJdwvO9s/edit). The only issue I haven't managed to resolve is that there're no table borders visible on page ( see /formats/video page). Also: I'm receiving "Continuous Integration / run markdownlint (pull_request) Failing after n seconds" failed check alerts, perhaps, you know what this can mean?

@muuki88 - please re-review and merge? Thanks!


Does the REJSON module be supported?

https://github.com/RedisJSON/RedisJSON

https://github.com/RedisJSON/RedisJSON

this is for Linux,you can refer to #18
https://github.com/tporadowski/rejson ,this is for windows


Create database format

When and how databases are going to be accessed

Databases will be stored as .minecraft/autocut/recordings/UnixStartTimeInHex.sqlite
Databases will be accessed via a RecordingManager (rename from Recording)
Databases will have two tables:
CREATE TABLE clips (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    start_timestamp INTEGER,
    end_timestamp INTEGER,
    type TEXT,-- expected to be an Identifier
    meta TEXT
);
CREATE TABLE events (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    timestamp INTEGER,
    type TEXT,-- expected to be an Identifier
    meta TEXT
);

We'll start without the meta for now.


[crash]

Trying to get to the new player apartmant but noı luck when i trying near them allways getting crush,
Cyberpunk2077.zip

the screenshot is near the japantown apartment...

This is too old at this point I'd say due to new patches and changes... If you can reproduce it in new game patch with clean CET, no other mods (including all non-CET mods) then feel feee to reopen.


Can wordcountvectorizer support custom dictionaries ?

Hello, thank you very much for developing rubix ml, which is a great work. I am currently using it and found an important problem to feed back to you.
When I used rubix ml to develop a function involving word segmentation in the project, I found that when using wordcountvectorizer, articles can only be divided into words according to spaces. However, in many articles, especially in a large number of documents, splitting words according to spaces will lead to the wrong splitting of many terms originally composed of two or three words. For example, the Latin of many plants is composed of two words, which completely loses its meaning after splitting. This results in very limited applicability of wordcountvectorizer. I wonder if you can support custom dictionaries so that users can create their own dictionaries and wordcountvectorizer can recognize terms created in the user's Dictionary consisting of two or three words as one word. In this way, the applicability of wordcountvectorizer will be greatly expanded.
Best wishes

This use case is something that I encountered. And would be very useful.

I think we can probably make Word Count Vectorizer more flexible, if not directly, through the Tokenizer abstraction. Even with a custom vocabulary, text blobs still need to be tokenized in order to be counted. Take a look at what we have here and let us know if a modification to one of the current Tokenizers or a new tokenizer will provide the functionality you are needing.
https://github.com/RubixML/ML/tree/2.0/src/Tokenizers

Hello, I have carefully read several files listed in the links you give, but I still don't know how to use the user-defined dictionary or similar functions.

I'm sure there's a way we can make it work. A custom vocabulary is the easy part. Tokenizing is the harder part because you still need to recognize words that are in the vocabulary from arbitrary blobs of text. Perhaps a "RegexTokenizer" is something we should spend some time thinking about. A tokenizer that would allow the user to define how words are tokenized through regular expressions.

@kornatzky Hello, I didn't understand the reply of andrewdalpino. I can't think of how to use regular expressions to realize the function of custom dictionaries. I wonder if you understand?

You need to recognize these multiple word phrases. There are two possibilities. Define a regular expression for each such phrase. The problem is that with more than a few phrases, it is not going to be scalable. A second one is to actually program a tokenizer that recognizes each such phrase from the list and identify it as a token. Currently, I do not think the available tokenizers can do this. That is get a list of phrases to be each reognized as a token.

@kornatzky Thank you very much. After your explanation, I think I probably understand. I reread the files in the list again. If I want to define a regular expression, I think it should be in https://github.com/RubixML/ML/blob/2.0/src/Tokenizers/Word.php , that will be WORD_REGEX. For example, if I want to match the following two Latin scientific names:
Ferula sinkiangensis, Ferula fukanensis
Then, I need to edit WORD_REGEX:
protected const WORD_REGEX = "/Ferula sinkiangensis|Ferula fukanensis|[\w'-]+/u";
I wonder if my understanding is correct?
If my understanding is correct, if there are many terms to be defined (this is almost certain), then this WORD_REGEX will be extremely complex.

Seems to me your understanding is correct. And it would be complex. So we need something which gets a list of words as parameter.

@kornatzky I quite agree. I'm looking forward to it.

I'm not too savvy with REGEX, is there a pattern that would capture the words correctly for your language besides just spelling the exact words out in a list?

@andrewdalpino Hello, I see what you mean. But I think no matter how well tokenizer is designed, it is good at accurately dividing words one by one. But for terms like Latin scientific names, phrases and multi word terms, tokenizer may not recognize them. If a user-defined dictionary is not difficult to implement, I think it should be the best solution to solve the situation of Latin scientific names, phrases and multi word terms.


Nested URL path, resources

Hello,
For example I have /api/events endpoint, I want to add top events functionality, in such case URL would look like: /api/events/top.
I was wondering is it a correct way and if is it how I should implement it, how I can describe resource, and so on?
I was tried to find examples for such use cases but no luck.
Thanks!

Endpoint /api/events/top  would return an events resources.

Thanks!


storage: add support for OLM Prefix/Suffix

We need to add support for the new LifecycleCondition field options matchesPrefix and matchesSuffix. Both of these will be a list of strings.
request (internal): go/prefix-suffix-client-library-request
nodejs-storage example: https://github.com/googleapis/nodejs-storage/pull/1847
Please wait for my approval before merging/releasing this so we can coordinate around release dates.

@shaffeeullah all that remains here is to enable the integration test against production.  Can you ping me when that is ready? Until then I am marking this as blocked on an external issue.

@coryan tests should be passing now. please confirm


Adding Unit Test Cases using JEst Testing Library

Subject: Request for Jest integration in ReactJS projects on GitHub
Hi there !!
I am writing to request the integration of Jest, a popular JavaScript testing library, into your project that uses ReactJS. This would allow developers to write and run unit tests for their React components directly on the platform. Jest's simplicity and power would greatly improve the development experience and increase the reliability of code.
And the good thing is Jest testing files wouldn't add any memory consumption to the project when we make a build or it is on production.
Please add hard level label under JWoC to this or you can decide it while merging or reviewing PR request.
Thank you for considering this request. I look forward to your response.
Best regards,
Jagroop

@Jagroop2000 Sure. You can go ahead and work on this.
I'll assign you this issue once the contribution period starts.
I'll add the difficulty label when you raise your pr.

Hey @Jagroop2000 !
Any update on this issue


bug(metric): batch mem usage shows negative value

Describe the bug

To Reproduce

risedev d
risedev slt -p 4566 -d dev e2e_test/visibility_mode/barrier.slt

Expected behavior
No response
Additional context
No response

@liurenjie1024


unsupported type "invalid type"

Problem statement
Using the latest version, when running: swagger generate spec -o ./docs/swagger.yaml --scan-models --log-output=./docs/output
I get the error: unsupported type "invalid type" without any message stating what went wrong
When when specifying a logfile, the file is empty. I've been trying to backtrack from the sourcecode it's very tedious. Is there any way to see what could be going wrong?
Environment
swagger version: latest (0.27.0)
go version: 1.13.8
OS:  Ubuntu

Issue was a missing import.


dkatz-refactor-version

Description
Please explain the changes you made here.
Release notes
The title of the pull request will be used as the default release notes description. If more detail is needed to communicate to partners the scope of the PR's changes, use this release notes section.
Checklist
General
Read the full guidelines for PRs here

[ ] Added the correct label: < feature | enhancement | bug | dependencies | infrastructure | ignore-for-release | database >
[ ] Assigned to a specific person, civiform/developers, or a more specific round-robin list.
[ ] Created unit and/or browser tests which fail without the change (if possible)
[ ] Performed manual testing (Chrome and Firefox if it includes front-end changes)
[ ] Extended the README / documentation, if necessary

Database evolutions
Read the guidelines here

[ ] Assigned two reviewers
[ ] Guarded against already existing resources using IF NOT EXISTS and IF EXISTS
[ ] Downs created to undo changes in Ups

User visible changes

[ ] Followed steps to internationalize new strings

[ ] Added context strings to new messages
[ ] Didn't use a message in applicant facing code that isn't translated yet (unless behind a flag)

[ ] Wrote browser tests using the validateAccessibility method
[ ] Tested on mobile view. See mobile device mode
[ ] Manually tested at 200% size
[ ] Manually evaluated tab order

New Features

[ ] Add new FeatureFlag env vars to server/conf/helper/feature-flags.conf
[ ] Conditioned new functionality on a FeatureFlag
[ ] Wrote browser tests with the feature flag off and on, etc.

Instructions for manual testing
If instructions are needed for manual testing by reviewers, include them here.
Issue(s) this completes
Fixes #<issue_number>; Fixes #<issue_number>...

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

Current dependencies on/for this PR:

#6724  👈
#6723
#6722
main

This stack of pull requests is managed by Graphite.


The Paraview import issue for exported compressed VTU and VTI files

Hello. Thanks for crate! I've had a issue when i'm using Paraview to overview compression VTU and VTI file generated by vtkio
Example of VTI export code:
let mut point = vec![
        Attribute::DataArray(DataArray {
            name: String::from("concentration"),
            elem: ElementType::default(),
            data: IOBuffer::F64(
                cloud
                    .c
                    .index_axis(Axis(0), t_index)
                    .map(|x| *x)
                    .into_iter()
                    .collect(),
        )}),
    ];

let vti_vtk = Vtk {
    title: String::from("cloud_vti"),
    file_path: Some(PathBuf::from(path.clone())),
    byte_order: ByteOrder::LittleEndian,
    version: vtkio::model::Version { major: 2, minor: 2 },
    data: DataSet::ImageData {
        extent: Extent::Ranges([
            -x_ext..=x_ext,
            0..=y_ext,
            0..=vert_ext - 1,
        ]),
        origin: [0., 0., 0.],
        spacing: [
            (simulation.step) as f32,
            (simulation.step) as f32,
            simulation.step as f32,
        ],
        meta: None,
        pieces: vec![
            Piece::Inline(Box::new(ImageDataPiece {
                extent: Extent::Ranges([
                    -x_ext..=x_ext,
                    0..=y_ext,
                    0..=vert_ext - 1,
                ]),
                data: Attributes {
                    point,
                    cell: Vec::new(),
                    },
                }
            ))
        ],
    },
};
let mut file = File::create(path.as_str()).unwrap();
let vtk_file = vti_vtk.try_into_xml_format(vtkio::xml::Compressor::ZLib, 9).unwrap();
file.write_all(vtk_file.to_string().as_bytes()).unwrap();

If i'm using any type of compression Zlib, Lzma or Lz4 paraview has this issue.
ERROR: In vtkXMLDataParser.cxx, line 564
vtkXMLDataParser (000001CAFC5A13B0): Error reading compression header.

ERROR: In vtkXMLDataParser.cxx, line 881
vtkXMLDataParser (000001CAFC5A13B0): ReadCompressionHeader failed. Aborting read.

ERROR: In vtkXMLStructuredDataReader.cxx, line 345
vtkXMLImageDataReader (000001CAED603050): Error reading extent -262 262 0 262 0 25 from piece 0

ERROR: In vtkXMLDataReader.cxx, line 410
vtkXMLImageDataReader (000001CAED603050): Cannot read point data array "concentration" from PointData in piece 0.  The data array in the element may be too short.

ERROR: In vtkXMLDataParser.cxx, line 564
vtkXMLDataParser (000001CAFC5A13B0): Error reading compression header.

ERROR: In vtkXMLDataParser.cxx, line 881
vtkXMLDataParser (000001CAFC5A13B0): ReadCompressionHeader failed. Aborting read.

ERROR: In vtkXMLStructuredDataReader.cxx, line 345
vtkXMLImageDataReader (000001CAED603050): Error reading extent -262 262 0 262 0 25 from piece 0

ERROR: In vtkXMLDataReader.cxx, line 410
vtkXMLImageDataReader (000001CAED603050): Cannot read point data array "concentration" from PointData in piece 0.  The data array in the element may be too short.

ERROR: In vtkXMLDataParser.cxx, line 564
vtkXMLDataParser (000001CAFC5A13B0): Error reading compression header.

ERROR: In vtkXMLDataParser.cxx, line 881
vtkXMLDataParser (000001CAFC5A13B0): ReadCompressionHeader failed. Aborting read.

ERROR: In vtkXMLStructuredDataReader.cxx, line 345
vtkXMLImageDataReader (000001CAED603050): Error reading extent -262 262 0 262 0 25 from piece 0

ERROR: In vtkXMLDataReader.cxx, line 410
vtkXMLImageDataReader (000001CAED603050): Cannot read point data array "concentration" from PointData in piece 0.  The data array in the element may be too short.

ERROR: In vtkXMLDataParser.cxx, line 564
vtkXMLDataParser (000001CAFC5A13B0): Error reading compression header.

ERROR: In vtkXMLDataParser.cxx, line 881
vtkXMLDataParser (000001CAFC5A13B0): ReadCompressionHeader failed. Aborting read.

ERROR: In vtkXMLStructuredDataReader.cxx, line 345
vtkXMLImageDataReader (000001CAED603050): Error reading extent -262 262 0 262 0 25 from piece 0

ERROR: In vtkXMLDataReader.cxx, line 410
vtkXMLImageDataReader (000001CAED603050): Cannot read point data array "concentration" from PointData in piece 0.  The data array in the element may be too short.

ERROR: In vtkXMLDataParser.cxx, line 564
vtkXMLDataParser (000001CAFC5A13B0): Error reading compression header.

ERROR: In vtkXMLDataParser.cxx, line 881
vtkXMLDataParser (000001CAFC5A13B0): ReadCompressionHeader failed. Aborting read.

ERROR: In vtkXMLStructuredDataReader.cxx, line 345
vtkXMLImageDataReader (000001CAED603050): Error reading extent -262 262 0 262 0 25 from piece 0

ERROR: In vtkXMLDataReader.cxx, line 410
vtkXMLImageDataReader (000001CAED603050): Cannot read point data array "concentration" from PointData in piece 0.  The data array in the element may be too short.

If I'm using Compressor::None there isn't issue and all is good but a size of file is too large for me.
I tried to use vtkio = {version = "0.6.3", git="https://github.com/elrnv/vtkio.git", branch="fix-pygmsh"} branch because it commit message says:

Implement compression support for individual (inline) DataArrays.

But, unfortunately, it's still doesn't work.
Verison of Rust: rustc 1.66.1
Version of Paraview:  5.11.0

Thank you for the issue!
Do you have a sample .vti file I can use to test/debug this issue (preferably one with reduced size)?
It seems I haven't quite finished fixing that branch.

cloud.zip
I've attached zip with 2 vti file (the smallest and biggest one) and 1 vtu file. It will need to represent plots in paraview.
the compressor in all files is Zlib. The Level of compression in all files is 9.

@MaygurovMV I have reworked the compression mechanism in release-0.7. Could you try your code with that branch and see if it is still causing an issue in paraview?

Hello, @elrnv.
Thanks for fast support. I can't try new features because I have problems with dependencies solving.
First error from cargo:
error: no matching package named `quick-xml` found location searched: https://github.com/elrnv/vtkio.git?branch=release-0.7#44da5dbd required by package `vtkio v0.7.0 (https://github.com/elrnv/vtkio.git?branch=release-0.7#44da5dbd)
I fixed it by manualy cloning your repository and quick-xml. But I've had another problem:
  --> D:\Code\vtkio\src\xml\se.rs:8:21
   |
8  | use quick_xml::{se::Write, DeError};
   |                     ^^^^^ private trait
   |
note: the trait Write is defined here
  --> D:\Code\quick-xml\src\se\mod.rs:92:5
   |
92 | use std::io::Write;
   |     ^^^^^^^^^^^^^^

error[E0603]: trait Write is private
    --> D:\Code\vtkio\src\xml.rs:3602:53
     |
3602 |     pub fn write(&self, writer: impl quick_xml::se::Write) -> Result<()> {
     |                                                     ^^^^^ private trait
     |
note: the trait Write is defined here
    --> D:\Code\quick-xml\src\se\mod.rs:92:5
     |
92   | use std::io::Write;
     |     ^^^^^^^^^^^^^^

Now I'am sorting through versions of quick-xml to fix it. Could you tell me version of quick-xml that you used? Or maybe can you fix upstairs issues by yourself?

Thank you, that was my fault, I hadn't pushed my changes to quick-xml yet. I pushed them now, and updated the Cargo.toml dependency so it should work out of the box.

Thanks for operating support. All works more good than I could think of


Call to undefined method getETag()

I know this has been reported and "fixed" in #428 but I found another occurance in the 2.x branch. In Sabre\DAV\Server.php line 1347 there is $etag = $node->getETag(); and in my case the $node is an instance of SimpleCollection which doesn't implement that method.
Should the Method getETag be implemented in Node and by default return NULL?

getETag is required by the IFile interface, so for files, not directories. https://github.com/fruux/sabre-dav/blob/2.1/lib/DAV/Server.php#L1347 must be protected by an instanceof IFile at least. /cc @evert

I agree with @Hywan that checking the correct instance is the correct solution.

Can you tell us how you triggered this @jurgenhaas , as far as I can tell this only happens in buggy clients. If that's the case it's not really a critical issue (but still worth fixing).

I'm currently implementing SabreDAV into Drupal (http://drupal.org) and I have an endpoint /sabredav in Drupal's menu router and each enabled submodule (one for cards and one for calendars) implements additional endpoints like /sabredav/addressbooks and /sabredav/calendars. Now, when calling http://www.example.com/sabredav I should be getting the directory that shows addressbooks and calendars as available childs. This is working fine when that line is commented out but it fails if enabled. This is because the $node in that context is a SimpleCollection (implementing Directory and not File) and that class doesn't come with a getETag method.

Yes but as far as I can tell this would only be triggered in a case where a client sends an If-Match header (or If-None-Match, which has the same problem) along with a request that never defined an ETag header in the first place.
Without the (broken) If-Match / If-None-Match header, the bug would not occur.

I don't get it. My client is a web browser and I'm calling a page where I get the BrowserPlugin class to deliver the root directory. And that directory never has an ETag, so it shouldn't be called.

So why is your browser sending If-Match, is there something in drupal that automatically generates etags?

Actually it's probably If-None-Match in that case.

That's what I'm not getting: what do you mean by "sending If-Match", I'm not doing anything like that.

The error can only be thrown if the browser sends an If-None-Match header. The browser will only send this header if it knows about an etag that the server sent. sabredav does not send etags for collections, thus there is something else that generated an etag.

OK, as far as I can tell, Drupal is never setting those headers. However, I can confirm that incoming HTTP requests are having that header set if one of the directories is called the second or subsequent times. When I unset that header before initializing the SabreDAV library then I no longer get the error.
Is that Apache that sets that header maybe?

It's possible but it would be a very weird and undesirable setting. What is the content of the header? Can you find out when the browser first receives it?

When receiving a request that contains that header then these are the header values:
AUTHORIZATION: Basic .....
HOST: localhost
USER_AGENT: "Mozilla..."
ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
ACCEPT_LANGAUGE: de,en-US;q=0.7,en;q=0.3
ACCEPT_ENCODING: gzip, deflate
CONNECTION: keep-alive
IF_MODIFED_SINCE: Thu, 21 May 2015 13:31:46 GMT
IF_NONE_MATCH: "1432215106-gzip"
CACHE_CONTROL: max-age=0

Interesting =) Do you have some kind of gzip module in your webserver? Try turning it off and wipe your browser cache.

Should be good for now. I'm not sure where the headers are coming from but the current approach to unset that header in the specific context is working and I need to get something delivered to my client, so I need to get back to that before I can offer more help in debugging this.

@jurgenhaas I suggest you to not delete the header while it fixes your issue. This is an important one :-p.

@Hywan you mean not to delete the "whole" header? I'm currently only deleting the IF_NONE_MATCH and leave all the other untouched. And this only happens for the sabredav endpoint, not for all the other paths on that same server.
Or am I misunderstanding your suggestion?

@jurgenhaas I meant: If-None-Match is also very useful for sabre/dav, so we better might understand how this error happen instead of using such a fix that can be “dangerous” in a short time.

@Hywan thanks for your clarification. I've looked into the communication a bit deeper and here is what I found:
Drupal, by default responds to HTTP request with an etag in the header which is set to the current timestamp. The client (i.e. the browser) receives the etag with an appended "-gzip" and I'm uncertain where that is coming from but ignore that for now.
The client is then using the etag-value from the previous request as the "If-None-Match" header value for the subsequent request which is what I'm seeing in Drupal as described above.
So, it sounds like I need to avoid that Drupal is adding that etag header for its responses. Is that correct? And if so, under which circumstances should that be removed and when is it important to be imcluded?

I would simply suggest to ensure that drupal never does this.
The problem is that both drupal and sabredav may generate and validate etags. If drupal acts as the 'proxy' in front of sabredav, it should fully handle its own etags and not pass them on, while also internally managing the etags coming in and out of sabredav. This is much more sophisticated than drupal's currently functionality, so it's best to disable this altogether.
The problem with just removing the header, is that you effectively turn off conflict detection, so multiple clients may overwrite resources.

This is now fixed in master. If you ran master you would no longer see this error, but please keep in mind that there's more subtle, hidden errors that are not immediately prevalent, but will cause loss of data. The commits only fix the error, not your root cause. The only way to fix that, is to fix drupal.

Thanks @evert for fixing this and also for clarifying the background to me. I don't think there really is an issue with the Drupal implementation as Drupal won't be involved with the client communication when it comes down to updating and syncing cards. Drupal is only delivering html pages to browsers that are reading the tree and that's where Drupal includes the etag in the header. But that's not the case when talking to a WebDAV client.

I think that's incorrect. WebDAV clients still use etags, and still use plain HTTP. There's no reason to believe Drupal doesn't do the exact same there, but I did all I could! Good luck.

Yes they do, but my Drupal implementation routes them to different parts in the code where Drupal's delivery mechanism is bypassed and therefore it won't ever set the header as in the issue handled here.

Ah ok, good to hear


[SPARK-6370][core] Documentation: Improve all 3 docs for RDD.sample

The docs for the sample method were insufficient, now less so.

Can one of the admins verify this patch?

ok to test

Looking good though I might say "expected size of the sample as a fraction of this RDD's size" to be extra clear. I can touch that up on merge if you don't get to it first.

Test build #28919 has started for   PR 5097 at commit a6a9d97.

This patch merges cleanly.

Test build #28919 has finished for   PR 5097 at commit a6a9d97.

This patch passes all tests.

This patch merges cleanly.

This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28919/
Test PASSed.


There is no information about the linked Common Data Service (CDS) Environment on FSCM Database refresh

There is no information about the linked Common Data Service (CDS) Environment.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9d28e8cc-5110-f01b-db76-94479ec2fa91
Version Independent ID: f0710707-9ff2-84a1-30c7-1f0d44e3a0d7
Content: Refresh database - Finance & Operations | Dynamics 365
Content Source: articles/fin-ops-core/dev-itpro/database/database-refresh.md
Service: dynamics-ax-platform
Product: ****
Technology: ****
GitHub Login: @laneswenka
Microsoft Alias: laswenka

Thanks for your feedback. @laneswenka, can you please review?

BUG 562253 created to track.

Added context for dual-write linking for all of the database movement operations.  This will go out in the next publishing wave.


Shortcut to Settings:clean -dialog

It would be productive to have quick shortcut button to Settings:clean -dialog.
Maybe somewhere inside the Settings page.

Thanks for your feedback. We're having trouble understanding what you're requesting and need additional information to investigate this item. Please re-open when you've filled out the issue template to the best of your ability.


docs: properly document autoPaginate

Fixes #1684
@callmehiphop -- did you see any others in your journey to promiseland?

hmm, not sure. I'll take another look through.

Changes Unknown when pulling be6512a803473267ec0b30d50e09ec51d513af73 on stephenplusplus:spp--1684 into ** on GoogleCloudPlatform:master**.


xref:IO cannot be redirected to any link

Reference Article: azure.mgmt.resource.resources.v2020_10_01.operations.ResourceGroupsOperations class
Content Source: docs-ref-autogen/azure-mgmt-resource/azure.mgmt.resource.resources.v2020_10_01.operations.ResourceGroupsOperations.yml

<xref:IO> defined cannot be redirected to its reference page like other types, but still shows in the format like it's in the yml file.

Expected behavior:
IO type should be able to be redirected to its reference link.
Original issue:
https://github.com/Azure/azure-sdk-for-python/issues/31647

@ChenxiJiang333 this issues list is intended for customer feedback about the docs in this repo, which doesn't include reference content. Please use Azure DevOps or other, internal channels to provide feedback. Thanks!
CC @bobtabor-msft


Test Server in France

<IP_ADDRESS>
2001:bc8:600:1810::1

@ragibkl Hello,
Servers on France I think its slower than the servers in Tokyo !!! Here is the results:
Here is nslookup and ping to website:

Thanks you !

<IP_ADDRESS>
2001:bc8:600:1810::1

Shall I test that, friend, and get back to you?  Which list did you add to the one in France?

@TechComputerServiceDunkerque , this dns server has the same setup as the current ones in Singapore. So the list is the same.
Since you mentioned previously that there are users from France, including yourself using the DNS, I want to test the idea of putting servers closer to most users. I'm just curious if there is difference in performance, and if it's worth the effort.
So, I think you should see similar results to the current DNS setup, but I want to know if they are faster.

@TechComputerServiceDunkerque All of the lists on blacklists source !!! Details here: https://github.com/ragibkl/adblock-dns-server/tree/master/data/blacklist.d
He set the block lists and another part like the old adblock dns he has created before !!!

<IP_ADDRESS>
2001:bc8:600:1810::1

Shall I test that, friend, and get back to you? Which list did you add to the one in France?

Yes, please help me test it. @phanluchoaofficial  is also testing another server I setup in Tokyo.

@ragibkl I have ever used your france dns at prmary server now and I see they not fast like tokyo server !

I'm testing on Android at the moment, in ping it's 16ms faster on wifi.  On the 4g at orange 23 ms, it's faster than the original server.  I will test tomorrow on pc.
Why don't you test the new lists I gave you on the French server, so I can make more complete tests to help you block more ads etc. and give you a white list later?  If you're interested of course.

Why don't you test by adding the new lists I gave you on the french server, ...

I'm not sure what you mean by new lists. I think I've already included your most recent suggestions from here into the source file
All my current DNS servers carry the same setup.

No problem, not easy to understand with English.

Hi, can you unlock the next site for me, thank you. (you'd have to create a post where we can give you sites that are blocked)
https://rapidgator.net

Hi, can you unlock the next site for me, thank you. (you'd have to create a post where we can give you sites that are blocked)
https://rapidgator.net

Hi, this is done.
I will not undo any of the recent list that you gave me. It seems to be working fine, just need to whitelist some sites.
So, if there are any broken sites, you can just create a new issue ticket as usual.
Therefore, you c

Thanks for the french dns, everybody is happy :)

I've added this server address to my website here. I think we can close this issue for now.


Not compatible with Python 3.10: PyParser_SimpleParseFile() has been removed

Hi,
The unbound project uses the Python PyParser_SimpleParseFile() function which has been removed in Python 3.10:
https://bugs.python.org/issue40939#msg380826
PyParser_SimpleParseFile() is used to display a parsing error (SyntaxError).
I proposed a pseudo-code to reimplement something like PyParser_SimpleParseFile():
https://bugs.python.org/issue40939#msg381227
I'm not sure that I understood correctly the current unbound code. Would you mind to have a look?
Downstream Fedora issue: https://bugzilla.redhat.com/show_bug.cgi?id=1889726
Thanks,
Victor

Oh, it seems like the issue was fixed by the commit e0d426ebb10653a78bf5c4053198f6ac19bfcd3e.

This was indeed addressed in https://github.com/NLnetLabs/unbound/commit/e0d426ebb10653a78bf5c4053198f6ac19bfcd3e; closing.


Add actions in relationGroup

Hello,
For dynamic-form-control-relation.model, shall we add more actions such as 'show' and 'hide' to enrich the logic other than "Disable" and "Enable"?
Thanks

@sterling9 Thanks for the proposal.
As soon as Kendo and Material are stable I'll try to do a refactoring of the relation feature.

that would be awesome, any idea of how could i implement this feature? i really need it for a project

I currently have hacked the @ng2-dynamic-forms folder in my project, to do this i edited the:
<EMAIL_ADDRESS>I've made the following changes:

After var DYNAMIC_FORM_CONTROL_ACTION_ENABLE = "ENABLE" added the following code:

var DYNAMIC_FORM_CONTROL_ACTION_HIDE = "HIDE";
var DYNAMIC_FORM_CONTROL_ACTION_SHOW = "SHOW";

- After function DynamicFormRelationService.prototype.findActivationRelation = function (relGroups) created this function:
	```
DynamicFormRelationService.prototype.findActivationRelationHidden = function (relGroups) {
        return relGroups.find(function (rel) { return rel.action === DYNAMIC_FORM_CONTROL_ACTION_HIDE || rel.action === DYNAMIC_FORM_CONTROL_ACTION_SHOW ; });
    };

After function DynamicFormRelationService.prototype.isFormControlToBeDisabled = function (relGroup, formGroup) created this function:

DynamicFormRelationService.prototype.isFormControlToBeHidden = function (relGroup, formGroup) {
        return relGroup.when.reduce(function (toBeHidden, rel, index) {
            var control = formGroup.get(rel.id);
            if (control && relGroup.action === DYNAMIC_FORM_CONTROL_ACTION_HIDE) {
                if (index > 0 && relGroup.connective === DYNAMIC_FORM_CONTROL_CONNECTIVE_AND && !toBeHidden) {
                    return false;
                }
                if (index > 0 && relGroup.connective === DYNAMIC_FORM_CONTROL_CONNECTIVE_OR && toBeHidden) {
                    return true;
                }
                return rel.value === control.value || rel.status === control.status;
            }
            if (control && relGroup.action === DYNAMIC_FORM_CONTROL_ACTION_SHOW) {
                if (index > 0 && relGroup.connective === DYNAMIC_FORM_CONTROL_CONNECTIVE_AND && toBeHidden) {
                    return true;
                }
                if (index > 0 && relGroup.connective === DYNAMIC_FORM_CONTROL_CONNECTIVE_OR && !toBeHidden) {
                    return false;
                }
                return !(rel.value === control.value || rel.status === control.status);
            }
            return false;
        }, false);
    };

After function  DynamicFormControlComponent.prototype.updateModelDisabled = function (relation) created the following:

DynamicFormControlComponent.prototype.updateModelHidden = function (relation) {
this.model.hiddenUpdates.next(this.relationService.isFormControlToBeHidden(relation, this.group));
};

- After function DynamicFormControlComponent.prototype.onModelDisabledUpdates = function (value) created the following:
	```
DynamicFormControlComponent.prototype.onModelHiddenUpdates = function (value) {
        value ? this.model.cls.grid.container = "hidden" : this.model.cls.grid.container = "ui-grid-row";
    };

ON DynamicFormControlComponent.prototype.setControlRelations added the following code:

relActivation = null;
relActivation = this.relationService.findActivationRelationHidden(this.model.relation);
if (relActivation) {
this.updateModelHidden(relActivation);
this.relationService.getRelatedFormControls(this.model, this.group).forEach(function (control) {
_this.subscriptions.push(control.valueChanges.subscribe(function () { return _this.updateModelHidden(relActivation); }));
_this.subscriptions.push(control.statusChanges.subscribe(function () { return _this.updateModelHidden(relActivation); }));
});
}

- ON  DynamicFormControlComponent.prototype.ngOnInit added the following code right after this.subscriptions.push(this.model.disabledUpdates.subscribe(this.onModelDisabledUpdates.bind(this))):
	`this.subscriptions.push(this.model.hiddenUpdates.subscribe(this.onModelHiddenUpdates.bind(this)));   `

- ON var DynamicFormControlModel = (function () added the following code at the end:
	```
this.hiddenUpdates = new Subject();
	this.hiddenUpdates.subscribe(function (value) { return value ? _this.cls.grid.container = "hidden" : _this.cls.grid.container = "ui-grid-row"; });

After Fuction DynamicFormControlComponent.prototype.onModelDisabledUpdates = function (value) created:

DynamicFormControlComponent.prototype.onModelHiddenUpdates = function (value) {
value ? this.model.cls.grid.container = "hidden" : this.model.cls.grid.container = "ui-grid-row";
};

- Last line export added:
`DYNAMIC_FORM_CONTROL_ACTION_ENABLE, DYNAMIC_FORM_CONTROL_ACTION_HIDE,`

Limitations:
- Currently i'm hard coding control.cls.grid.container class to "ui-grid-row" when visible.
- As soon as i update my dependecies i have to do this again.
-If i need another action is a lot of code to replicate and will have the same limitations.

Hope @udos86 can work on a permanent and more elegant solution, due to project timeline i can't put more effort on this.

Thanks for the proposal, I will try to refactor it on basic controls and hopefully @udos86 can release a prototype version soon.

Hi
More wishes for the relation feature.
It would also be very nice if the control could be disabled and emptied.
For example, in our forms we have combined checkboxes with textareas where users can leave special comments. If the checkbox is set to false, the comment should be deleted.
Thanks for all your work!

@ticxx Hi!

Thanks for all your work!

Right now I'm collecting all proposals for this library feature to make up a plan fo refactoring.

@udos86 Hi Udo, any news on this feature? "Show" and "Hide" would be really great to have.

+1 this feature would be very helpful. I see VISIBLE and HIDDEN are options in dynamic-form-control-relation.model.d.ts, but they do not seem to work in the same basic sense that enable/disable do. Is there a recommended way to work around this in the short term while the larger refactoring is being planned?

Hey, any updates on this?

@udos86
Hi, any updates on this please?

major refactoring will be finally available in 8.0.0


Create application insights when create function app

Create application insights when create function app
Refers https://github.com/Azure/azure-cli/blob/dev/src/azure-cli/azure/cli/command_modules/appservice/custom.py#L2589

Codecov Report

Merging #1074 into develop will decrease coverage by 0.24%.
The diff coverage is 0.00%.

@@              Coverage Diff              @@
##             develop    #1074      +/-   ##
=============================================
- Coverage      36.25%   36.01%   -0.25%
  Complexity       537      537
=============================================
  Files            103      103
  Lines           4432     4462      +30
  Branches         684      688       +4
=============================================
  Hits            1607     1607
- Misses          2686     2716      +30
  Partials         139      139

Impacted Files
Coverage Δ
Complexity Δ

...oft/azure/maven/function/AbstractFunctionMojo.java
46.15% <0.00%> (-3.85%)
8.00 <0.00> (ø)

...com/microsoft/azure/maven/function/DeployMojo.java
35.33% <0.00%> (-8.59%)
17.00 <0.00> (ø)

...a/com/microsoft/azure/maven/AbstractAzureMojo.java
33.90% <0.00%> (-0.20%)
0.00 <0.00> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a7a013...528330c. Read the comment docs.


The CottonFox Table has some issues showing the Balls.

The 1 and Cue Ball seem to be missing on the Esnya UI prefab for the CottonFox Table. It usually happens right at the start of the game.
Since the video is over 10MB, here is a link to a video I made demonstrating what's going on.

UPDATE: Since I'm good friends with Juice, I asked him about it. We tried a few options, such as turning off Dynamic Occlusion on the Balls and it still didn't seem to solve the problem. So, it's gotta do with the Code of the Table somehow.
Again, this is on the latest 1.5.1 version of the prefab.

Wasn't able to repro this in a clean unity install, was this a clean install?

Wasn't able to repro this in a clean unity install, was this a clean install?

Nope, this was in an existing project. I was upgrading from 1.5.0 to 1.5.1, following the instructions required (Delete the VRCBCE folder prior to importing the new version).
This comment also applies to Issue #135.

does the Meshes section of the Core Table Code inspector look like this?

and are there any other changes that would cause Unity to show the UdonBehaviour listed being different? (try clicking the UdonBehaviour entry and hitting "revert" at the top of the popup)

I'm unsure! I had to revert back to 1.5.0 and it worked again. I tried following those steps and making sure those things were correct in a fresh Project and it worked perfectly.
I tried a fresh Scene too in the project where I'm having those issues, and still got it. But in a whole fresh Project, it works fine...


IP addressess in wrong format

In the "public IPs" sheet the format of the IPs is not intelligible

I updated the PublicIP module. Can you try the last version?
Thanks

I made a mistake by passing a misspelled argument, the "public IP" fix
works like a charm.
Thanks
Christian
Il giorno mar 7 nov 2023 alle ore 10:38 I Cucciolini @.***>
ha scritto:

I just tried the last version but I guess the tool is now producing
incomplete files (72kb Excel file instead of usual 2MB) so I opted for a
rollback
I'll do more tests later.
Thanks
Il giorno lun 6 nov 2023 alle ore 19:35 Claudio Merola <
@.***> ha scritto:

I updated the PublicIP module. Can you try the last version?
Thanks
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/ARI/issues/147#issuecomment-1795890538,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEBG3F7SWUSMUADCAAPQJ3TYDEUYLAVCNFSM6AAAAAA5Z52BJGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOJVHA4TANJTHA
.
You are receiving this because you authored the thread.Message ID:
@.***>

Glad it worked.
Thanks


Bug in clean_maf

> clean_maf = cleanup_maf(maf_df = GAMBLR.data::sample_data$grch37$maf)
Error in `mutate()`:
ℹ In argument: `EXON = gsub("/.+", "", EXON)`.
Caused by error in `h()`:
! error in evaluating the argument 'x' in selecting a method for function 'gsub': object 'EXON' not found

The input MAF table may not have a EXON column.
> maf <- GAMBLR.results::get_coding_ssm()
> any( names(maf) == "EXON" )
[1] FALSE

In cleanup_maf code:
> mutate(maf_df,EXON = gsub("/.+", "", EXON))
Error in `mutate()`:
ℹ In argument: `EXON = gsub("/.+", "", EXON)`.
Caused by error in `is.factor()`:
! object 'EXON' not found

The clean_maf is meant for use on the unreleased data. So the bundled maf is not the best use case here. Can you try passing here a maf that has all columns?

Yeah! Of course. Sorry, I forgot to use basic_columns = FALSE.
> maf <- GAMBLR.results::get_coding_ssm(basic_columns = FALSE)
> clean_maf = cleanup_maf(maf_df = maf)
Warning messages:
1: There was 1 warning in `mutate()`.
ℹ In argument: `CDS_position = as.numeric(as.character(CDS_position))`.
Caused by warning:
! NAs introduced by coercion
2: There was 1 warning in `mutate()`.
ℹ In argument: `cDNA_position = as.numeric(as.character(cDNA_position))`.
Caused by warning:
! NAs introduced by coercion
3: There was 1 warning in `mutate()`.
ℹ In argument: `Protein_position = as.numeric(as.character(Protein_position))`.
Caused by warning:
! NAs introduced by coercion

The warning messages seem to be because there are NAs and this should be fine.

We should probably change the examples though, which uses the bundled data.
#' @examples
#'
#' clean_maf = cleanup_maf(maf_df = GAMBLR.data::sample_data$grch37$maf)

Would it be better to move this function to GAMBLR.results since it is exclusively for this package?

Yeah, this particular function might be better homed in the results package because of the intended use case of this. Thanks!

I think this is the same case with annotate_maf_triplet. It needs the STRAND_VEP column that is presented only in the restricted data. Should we migrate annotate_maf_triplet from GAMBLR.utils to GAMBLR.results as well?
> maf_triple <- annotate_maf_triplet(maf, all_SNVs = FALSE, "C", "T")
Error in annotate_maf_triplet(maf, all_SNVs = FALSE, "C", "T") :
  STRAND_VEP column is not in the MAF

Solved in this PR.


Export default parser

Proposed Changes
Enable downstream programs to modify the default parser.

Types of Changes
Bugfix

Verification
Local testing

Linked Issues
https://github.com/rancher/rke2/issues/2194

User-Facing Change

Further Comments

Flaked on disk space, merging
[PARALLEL-etcd]   Warning  FreeDiskSpaceFailed                               4m26s                  kubelet     failed to garbage collect required amount of images. Wanted to free<PHONE_NUMBER>2 bytes, but freed 0 bytes
[PARALLEL-etcd]   Warning  ImageGCFailed                                     4m26s                  kubelet     failed to garbage collect required amount of images. Wanted to free<PHONE_NUMBER>2 bytes, but freed 0 bytes


ESP32 WiFi driver softAP: DHCP server problems due to loss of UDP broadcast packages (IDFGH-11444)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[ ] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report
Scenario: esp32 as AP
Problem: clients have (re)connection problems
For years, developers have reported problems with the esp's DHCP server. Some scenarios can be reproduced and tackled (for instance, when a client holds on to the old lease after the esp has been rebooted), but in many other cases it's difficult or impossible to reproduce a problem as they occur rather randomly.
After days or rather weeks of intensive wireshark sessions, I started seeing a pattern:
replies of the DHCP server get lost!
The more traffic (other clients) and/or the weaker the WiFi signal, the more likely this occurs.
Quite often the following scenario happens:
client sends a DHCP discovery or a request (with old lease's IP)
ESP assigns IP (event is fired)
but client does not receive the OFFER or the ACK.
Client typically times out and tries again, so (re)connecting can take a long time, or it gives up.
Even the NACK after a request with an old IP gets lost sometimes.
I suspect the problem simply in the implementation of the WiFi driver, where "AP only caches unicast data for the stations connect to this AP".
It would be good to give the Wifi driver a higher priority, for years an ESP hasn't been able to reliably send UDP broadcast...

hi @ch-fb could you share your IDF version or commit?

and if you can share your DHCP wireless package it would be more helpful

sorry for my late reply.
most tests were done with idf 4.4.
i went through the changelogs since 4.4 up to the latest version and did not see any DHCP-related changes nor work on caching in the WiFi driver. It is a known problem that UDP broadcast packets can easily get lost for stations that use power-saving modes (which basically all phones/tablets/laptops do) and I have observed that plenty of times.

After days or rather weeks of intensive wireshark sessions, I started seeing a pattern: replies of the DHCP server get lost! The more traffic (other clients) and/or the weaker the WiFi signal, the more likely this occurs.
Quite often the following scenario happens: client sends a DHCP discovery or a request (with old lease's IP) ESP assigns IP (event is fired) but client does not receive the OFFER or the ACK.

@ch-fb Could you share the wireshark log?

haven't saved them as I had wireshark runnig for hours in the background and I just looked at the recent entries whenever some connection problem happened again. But what it shows is that DHCP broadcast packets sometimes don't arrive (OFFER or ACK). It seems the ESP sends them (or at least it fires the according events), but they don't arrive.
Losing UDP broadcast/multicast packets when the ESP is in AP mode isn't a new issue, it has been observed many times also by others. To best reproduce the issue, you should have a busy network (plenty of traffic with other clients).
We actually gave up using the ESP for one of our products because of that issue as our application needs to send high-traffic UDP broadcast (continuous real-time data). Unfortunately, the ESP manages to deliver only a part of that data. I personally noticed the problem also for mDNS packets.
For a softAP that manages 4-10 clients, the proper caching of UDP broadcast/multicast packets isn't a very difficult task. It would be really important to implement that in the WiFi driver soon.

haven't saved them as I had wireshark runnig for hours in the background and I just looked at the recent entries whenever some connection problem happened again. But what it shows is that DHCP broadcast packets sometimes don't arrive (OFFER or ACK). It seems the ESP sends them (or at least it fires the according events), but they don't arrive.
Losing UDP broadcast/multicast packets when the ESP is in AP mode isn't a new issue, it has been observed many times also by others. To best reproduce the issue, you should have a busy network (plenty of traffic with other clients).

@nishanth-radja @Xiehanxin
Is above symptom a known issue?

yes, I have seen a a few bug reports. But it is even mentioned in the official ESP documentation for the WiFi driver (regarding WiFi power-saving/sleeping modes):
https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-guides/wifi.html#ap-sleep

@ch-fb
sorry for late reply
you can try this patch
0001-fix-lwip-Modify-the-DHCP-offer-and-DHCP-ack-from-bro.zip

@ch-fb
you can update idf to 5.2  where this feature has been added

thanks for your work, that should help with some DHCP problems. However, this is more of  a workaround than a solution, as the ESP still cannot sent UDP broadcast/multicast reliably, so problems continue to happen in mDNS and other services. Work is needed in the softAP WiFi driver to implement low-level buffering to ensure packets are delivered, this probably also requires a better ARP integration to really know all clients (not just DHCP clients but also those with static IPs or clients that hold on to an IP from an old/previous DHCP lease (happens after ESP reboot).


Doesn't 'tf.py_func' layers need tf.RegisterGradient for gradients?

For example, proposal layer, which is regressed to the proposal target layer, the gradient from it should be calculated and go back through the "rpn_cls_score" layer etc. But if the network doesn't have 'register gradient', I think the gradient would stop. Could anyone explain me about this?

you are right, so that means faster rcnn is NOT end-to-end. however, in the paper they did show the network is not trained stably when back proped to the image coordinates for example.


Fix typo error

changed undestand to understand

I signed it!

Thanks Nutan. Will merge this in.

PS: This was merged 10 days ago as https://github.com/tensorflow/hub/commit/1c69b5803787ab4562c4967af213f44ab8fb00bf


Tracing includes a long section that should not be there

When tracing https://airbnb.now.sh/ I get a long section (roughly 4 seconds) in the beginning of the trace, even though the entire request took less than 200 ms:

I tried GraphQL-Playground today with the hope of using the tracing pane, but I encountered a similar issue. To debug, my teammate suggested stubbing my server with some example data, so I grabbed the example from https://github.com/apollographql/apollo-tracing#example.
The example data demonstrated the same issue:

However, while setting up my stub, I did a botched find-and-replace (: to =>, turning JSON into a Ruby hash) which made startTime and endTime invalid (you can see in the response pane), and gave in an interesting rendering:

So, I suspect that the issue is with the startTime/endTime, since the field timings seem to work nicely!

Seeing a similar issue - tracing is not working at all for me.
I always see one single long Request bar (always ~40k ms) and nothing else (even after scrolling the tracing window which seems to be very wide):

The actual tracing data in the response seems to be there and be fine:

The simple test resolver is like so:

Have tried with every variation of simple test resolvers - sync, async, slow, fast, etc. result is always the same.
Versions:
package.json:
"apollo-server": "^2.0.0-rc.5",
"graphql": "^0.13.2",
"graphql-import": "^0.6.0",
"graphql-type-json": "^0.2.1"

yarn.lock:
<EMAIL_ADDRESS>  version "1.6.0"
  resolved "https://registry.yarnpkg.com/graphql-playground-html/-/graphql-playground-html-1.6.0.tgz#6ec4d54325ab2e5352506fdb89eb37c3fc718bf8"
  dependencies:
    graphql-config "2.0.0"

<EMAIL_ADDRESS>  version "1.7.2"
  resolved "https://registry.yarnpkg.com/graphql-playground-middleware-express/-/graphql-playground-middleware-express-1.7.2.tgz#5f7bf3683494fe17893d1b54bbecee5e222c2e78"
  dependencies:
    graphql-playground-html "1.6.0"

@schickling, @Huvik please correct me if im wrong here :) but the current Trace implementation will only work on dev machines where you run playground and server on your local machine. It depends on browser and server clocks beeing exactly in sync. The screenshots with strange timings in the previous posts is the result of difference in clocks between browser and server.
Sadly this renders the trace window useless in scenarios outside your local dev. machine.
The problem seems to be with the trace calculation
    const { tracing, tracingSupported, startTime, endTime, open } = this.props     const requestMs =       tracing && startTime         ? Math.abs(new Date(tracing.startTime).getTime() - startTime.getTime())         : 0     const responseMs =       tracing && endTime         ? Math.abs(endTime.getTime() - new Date(tracing.endTime).getTime())         : 0     const requestDuration = 1000 * 1000 * requestMs
where delta is calculated as the difference between tracing.startTime (server clock) and startTime (client clock)...

Sorry to bump an old issue, but the problem still seems to exist. My server is a Docker container, and I am using GraphQL Playground on the (Windows) host machine. The images below show what is happening quite well.
This first image is the default view, where only the "Request" line is visible.

Here, I've made the tracing pane much wider, and the search query is visible now. This query is pretty large.

When I scroll down through the query, the text eventually moves outside the viewport, and the horizontal scrollbar will not let me scroll towards that overflowing text.

I hope someone is able to fix the issue highlighted above my comment. In the meantime, does anyone know a different tool I can use to visualize my GraphQL traces? I am trying to make our Lighthouse PHP GraphQL API faster, but it is quite tough to debug it without proper tracing.


no matches for kind "MqttBridgeConnector" in version "mq.iotoperations.azure.com/v1beta1"

I am  going through the steps for Azure Jumpstart: [https://azurearcjumpstart.com/azure_arc_jumpstart/azure_edge_iot_ops/aio_manufacturing]
and during the post deployment step, I am getting the following error
resource mapping not found for name: "my-topic-map" namespace: "azure-iot-operations" from "/home/SIJ/Tools/mq_cloudConnector.yml": no matches for kind "MqttBridgeTopicMap" in version "mq.iotoperations.azure.com/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "my-mqtt-bridge" namespace: "azure-iot-operations" from "/home/SIJ/Tools/mq_cloudConnector.yml": no matches for kind "MqttBridgeConnector" in version "mq.iotoperations.azure.com/v1beta1"
ensure CRDs are installed first
Error from server (NotFound): namespaces "azure-iot-operations" not found
[12:05:02] INFO: Waiting for MQTT services to initialize and the service IP address to be assigned... Waiting for 20 seconds
Error from server (NotFound): namespaces "azure-iot-operations" not found
[12:05:02] INFO: Waiting for MQTT services to initialize and the service IP address to be assigned... Waiting for 20 seconds
the file that is giving the error is [https://github.com/microsoft/azure_arc/blob/main/azure_edge_iot_ops_jumpstart/aio_manufacturing/bicep/artifacts/Settings/mq_cloudConnector.yml]

@dguptaSAS
It would be great if you could add a link to the documentation you are following for these steps? This would help us redirect the issue to the appropriate team. Thanks!!

@PesalaPavan I was following the steps from Azure Jumpstart
I am in touch with the Azure Jumpstart team on this issue now and they are working on the fix.
Open Issue #2738

@dguptaSAS
We are going to close this thread but if there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.


LS47 and other alternate system customization

Have LS47's 7x7 format (both English and Russian) as one of the defaults (see https://gitea.blesmrt.net/exa/ls47)
Might also be a good idea to include Greek 6x6 and 7x7, and maybe kana 7x7 and 8x8, but nobody has done such a design before.
Basic requirement: Flexible state sizes with a wider variety of lengths including 6 and 7, and flexible tiles that takes in a string as the default tile types and orientations.

@DonaldTsang, these sound like nice ideas (although I'm unfamiliar with those algorithms), but beyond the intended scope and simplicity of this project.

@dstein64 I would not say it is out of scope due to LS47 being standardized for Russian


gperftools doesn't seem to be detecting leaks

tcmalloc (and the dependent gperftools) seems to be only be linked in non-asan builds.  As far as I'm aware, when tcmalloc is linked, it should allow gperftools to detect leaks through a heap checker.  However, in the process of tracking down some internal leaks, I introduced the following sample leak into router.h:
Filter(FilterConfig& config)
      : config_(config), downstream_response_started_(false), downstream_end_stream_(false),
        do_shadowing_(false) {
          int *leak = new int;
          std::cout << "leak address: " << leak << ", leak data: " << *leak << std::endl;
          leak = new int;
          std::cout << "leak address: " << leak << ", leak data: " << *leak << std::endl;
        }

This leak is only detected by the bazel.asan docker command by the asan LeakSanitizer.  Other docker builds/tests (bazel.dev, bazel.debug, bazel.release, bazel.tsan) pass.  If this is the expected behavior, that's fine.  It just seemed surprising that gperftools either wasn't enabled or detecting this unambiguous leak on any of the docker commands where one might expect it to.

Heap checking should be enabled in normal builds via https://github.com/lyft/envoy/blob/master/ci/build_setup.sh#L7. I can check this locally in a sec. If that's not working correctly I think something broke somewhere along the way.

@mrice32 yeah I think something is broke. I'm not seeing normal heap check output when I run tests via bazel even w/ env var set. I can investigate in a bit, trying to fix something else. cc @htuch in case he knows what might be broke.

Okay, I'll do some poking around to see if I can find the source of the problem.

i'm guessing it's some link issue (mattrice@ suggested it might be link order). Probably happened in the Bazelification.


Question - AKS Private DNS

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description
When deploying a private AKS cluster in a landing zone, a private AKS DNS Zone is created with an A record in the MC_ resource group.
We are using the Hub and Spoke approach with centralized DNS in the Hub/Connectivity subscription and have enabled several of the private link services in the DNS section of the settings.connectivity.tf file. We have also enabled the private DNS Zone virtual network links on the spoke networks.
We have a private DNS Zone for kubernetes in the Connectivity subscription along with other private DNS Zones for other services.
With the deploy-private-dns-zones policy (Configure Azure PaaS services to use private DNS zones) policy enabled we are able to deploy private link endpoints for services such as Key Vault and have the policy automatically create the A record in the appropriate private DNS Zone in the Connectivity subscription. This is working well for us.
I don't see AKS being one of the services in the above policy though.
I understand that we can configure AKS to create the appropriate A record in the private DNS Zone but then we need to use either User Assigned identity or the now deprecated service principal and grant that identity Private DNS Zone Contributor.
What is the best way to get A records created in the centralized Private DNS Zone when using a private AKS cluster?
Is it possible to have AKS included in the deploy-private-dns-zones policy?
Should I create a standalone policy that will do this similar to the other policy and reuse the Deploy-PrivateDNS-Zones identity that the other policy uses?
Any help would be appreciated as I don't want to make this more complicated then is needed especially with the simplicity of the existing policy that takes care of this for other services.

Just from my experience you cannot create a private K8s cluster without specifying the private DNZ zone (someone can correct me, if I am wrong!)
We ended up doing a data to the resource in connectivity
*redacted
  private_cluster_enabled = true
  private_dns_zone_id     = data.azurerm_private_dns_zone.azmk8s.id

Thanks for providing the approach you took. I will give it a try and see how it works out.

Hi!
The policy for deploying DNS zone groups (deploy-private-dns-zones) only works for private endpoint resources.
It seems that @anwanrk has a good solution.
Otherwise, a DNS delegation from your hub DNS zone to a AKS one seems a logical idea.
Flagging @jtracey93 as this could be a good addition to the AKS LZA. We should reach out to them

Hi,
We're getting a similar issue while creating the AKS private cluster. We are assigning the private DNS zone ID as mentioned by @anwarn above. However, upon running the Terraform code to create the private cluster, we get the customPrivateDNSZoneMissingPermission error, as shown below.
Service principal or user-assigned identity must be given permission to read and write to custom private dns zone /subscriptions/{sub-id}/resourceGroups/rg-dns-connectivity/providers/Microsoft.Network/privateDnsZones/privatelink.westeurope.azmk8s.io. Check access result not allowed for action Microsoft.Network/privateDnsZones/write.
One way to solve this is to assign the "Private DNS Contributor" permissions to the AKS user-assigned identity, but that doesn't seem to be a good solution following the hub-spoke approach, as in this case, we will be allowing landing zone corp subscriptions services to get permissions over the private DNS zones in the connectivity subscription.
Another way I am thinking of solving this is to delegate the private DNS zones to the land-zones subscriptions, but the CAF module doesn't have any such feature to enable this.
Following the CAF best practices, any suggestions or help for solving this issue would be appreciated.