addon, sensor, component ?

First, great work, connecting dynalite to home automation app as hass.io is one of things on my bucket list.
I'm really satisfied with Dynalite, and extremly unsatisfied with Philips (signify ??)
In regard to connection, i'm having trouble figuring what is best way, addon or sensor.
After few hours of fighting, i managed to install dynalite addon, the one you call "MQTT gateway for PhilipsDynalite" and it worked!  Kind of Eureka moment! :+1:
To make it work I had to tweak it a bit (insert arch description) fix areas/presets, bit of hassle with MQTT but as soon as all was configured OK it worked.
Lights would show up in hass.io after i use them on regular keypads, pretty great, all showed up.
Now you've updated this sensor version and I made it work. No seperate MQTT, so bit easier.
Still, would like to know what is the difference between addon and sensor ?
Why is one better than other, if.
Thank you very much, looking forward for you to implement channels :+1:

Hi @uzelac
I feel your frustration with Philips - they've given up on Dynalite it seems (or maybe they don't even know they own it?)
I haven't maintained the addon for a long time - I'm only really maintaining this component, which in itself is sort of a fake sensor to be a bridge between the 485 Dynalite world and MQTT.
The addon would introduce a little additional server load that can be avoided by staying "in the Home Assistant ecosystem" with this component.
Did you get this component working, or are you still using the addon?

Not sure if they have given up, to be honest I think they are still on it, lots of new devices emerged in past 1-2 years. Thay changed name to Signify, maybe they plan some big restart ? :)
Problem is they are business/commercial oriented, hotels and business buildings, they never cared for residential and that should be root of our problem. I had contacts with few Dynalite distributors, and to be 100% fair they all said "Philips never offered this to residental, god forbid "end users"....
In regard to my installaton, i got component to work, yes, works great. Peace of cake with your latest read.me.
I'm still new to hass.io / home assistant so I don't know itch about dashboards/lovelace/cards etc. but hope to get to it :). Next step is some nice dashboard, but it all seems perfectly doable.
It was important for me to find automation which includes Dynalite, all my lights and some appliances are wired on it, and it works so good on it own. But unability to make it work with some other stuff was painfull.... Thanks to you, finally there is one and now I can be more determined to make it work. Dynalite works, that was most important! :).
Would be great to have channels, that is all that is important and left to integrate.
Dynalite works lite so nice, all else (relays, inputs..) I see myself replacing, but keypads and lighst are best :)
Maybe some kind of connection to DUS sensors, they are not that expensive but are pretty capable: IR receiver, light metering and motion sensor.. and they look good after all :).
Joins, Tasks and connection to DDTC (Timeclock device) doesnt make sense with Home Assistant, HA is so much more capable.
Once again, thank you so much.
Not sure how much time and knowledge it took from you, maybe lot, maybe few ours, but for me this is almost dream come true. Whatever I can do to help...


Problem with mysql

150304 10:53:07 mysqld_safe Can't log to error log and syslog at the same time.  Remove all --log-error configuration options for --syslog to take effect.
150304 10:53:07 mysqld_safe Logging to '/var/log/mysql/error.log'.
150304 10:53:07 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
mysql root password: Iexo1aM9zohm
wordpress password: eisaevoeFee9
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 58685  100 58685    0     0  21089      0  0:00:02  0:00:02 --:--:-- 21086
Archive:  nginx-helper.1.8.4.zip
nginx-helper.1.8.4 packaged
   creating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/readme.txt
   creating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/install.php
   creating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/assets/
   creating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/assets/nginx-helper-icons/
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/assets/nginx-helper-icons/config.json
   creating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/assets/nginx-helper-icons/font/
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/assets/nginx-helper-icons/font/nginx-fontello.eot
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/assets/nginx-helper-icons/font/nginx-fontello.svg
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/assets/nginx-helper-icons/font/nginx-fontello.ttf
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/assets/nginx-helper-icons/font/nginx-fontello.woff
   creating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/assets/nginx-helper-icons/css/
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/assets/nginx-helper-icons/css/nginx-fontello.css
 extracting: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/assets/nginx-icon-32x32.png
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/assets/style.css
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/assets/logo.png
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/assets/nginx.js
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/assets/rtp-social-icons-32-32.png
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/admin.php
   creating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/lib/
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/lib/nginx-support.php
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/lib/nginx-general.php
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/admin/lib/nginx-sidebar.php
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/nginx-helper.php
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/purger.php
   creating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/languages/
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/languages/nginx-helper.po
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/languages/nginx-helper.mo
  inflating: /usr/share/nginx/www/wp-content/plugins/nginx-helper/compatibility.php
mysqladmin: connect to server at 'localhost' failed
error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)'
Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
/usr/local/lib/python2.7/dist-packages/supervisor-3.1.3-py2.7.egg/supervisor/options.py:296: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
  'Supervisord is running as root and it is searching '
2015-03-04 10:53:23,017 CRIT Supervisor running as root (no user in config file)
2015-03-04 10:53:23,041 INFO RPC interface 'supervisor' initialized
2015-03-04 10:53:23,041 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2015-03-04 10:53:23,042 INFO supervisord started with pid 391
2015-03-04 10:53:24,046 INFO spawned: 'stdout' with pid 394
2015-03-04 10:53:24,053 INFO spawned: 'php5-fpm-log' with pid 395
2015-03-04 10:53:24,062 INFO spawned: 'nginx' with pid 396
2015-03-04 10:53:24,074 INFO spawned: 'mysqld' with pid 397
2015-03-04 10:53:24,084 INFO spawned: 'php5-fpm' with pid 398
2015-03-04 10:53:24,109 INFO exited: php5-fpm-log (exit status 1; not expected)
2015-03-04 10:53:24,226 INFO exited: php5-fpm (exit status 254; not expected)
2015-03-04 10:53:25,227 INFO success: stdout entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-03-04 10:53:25,232 INFO spawned: 'php5-fpm-log' with pid 415
2015-03-04 10:53:25,234 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-03-04 10:53:25,234 INFO success: mysqld entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-03-04 10:53:25,239 INFO spawned: 'php5-fpm' with pid 416
php5-fpm-log stderr | tail: cannot open '/var/log/php5-fpm.log' for reading: No such file or directory
mysqld stderr | 150304 10:53:24 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
php5-fpm stderr | Wed Mar  4 10:53:24 2015 (398): Fatal Error Unable to allocate shared memory segment of 67108864 bytes: mmap: Cannot allocate memory (12)
php5-fpm-log stderr | tail: cannot open '/var/log/php5-fpm.log' for reading php5-fpm-log stderr | : No such file or directory
2015-03-04 10:53:25,274 INFO exited: php5-fpm-log (exit status 1; not expected)
php5-fpm stderr | Wed Mar  4 10:53:25 2015 (416): Fatal Error Unable to allocate shared memory segment of 67108864 bytes: mmap: Cannot allocate memory (12)
2015-03-04 10:53:25,344 INFO exited: php5-fpm (exit status 254; not expected)
2015-03-04 10:53:27,352 INFO spawned: 'php5-fpm-log' with pid 418
2015-03-04 10:53:27,358 INFO spawned: 'php5-fpm' with pid 419
php5-fpm-log stderr | tail: cannot open '/var/log/php5-fpm.log' for reading: No such file or directory
2015-03-04 10:53:27,389 INFO exited: php5-fpm-log (exit status 1; not expected)
php5-fpm stderr | Wed Mar  4 10:53:27 2015 (419): Fatal Error Unable to allocate shared memory segment of 67108864 bytes: mmap: Cannot allocate memory (12)
2015-03-04 10:53:27,467 INFO exited: php5-fpm (exit status 254; not expected)
2015-03-04 10:53:30,476 INFO spawned: 'php5-fpm-log' with pid 421
2015-03-04 10:53:30,485 INFO spawned: 'php5-fpm' with pid 422
php5-fpm-log stderr | tail: cannot open '/var/log/php5-fpm.log' for reading: No such file or directory
2015-03-04 10:53:30,513 INFO exited: php5-fpm-log (exit status 1; not expected)
2015-03-04 10:53:30,595 INFO gave up: php5-fpm-log entered FATAL state, too many start retries too quickly
php5-fpm stderr | Wed Mar  4 10:53:30 2015 (422): Fatal Error Unable to allocate shared memory segment of 67108864 bytes: mmap: Cannot allocate memory (12)
2015-03-04 10:53:30,602 INFO exited: php5-fpm (exit status 254; not expected)
2015-03-04 10:53:31,603 INFO gave up: php5-fpm entered FATAL state, too many start retries too quickly
150304 10:54:31 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended

What's the command you're starting the container with? If you're using volumes, it may be related to the comment I left here.


Computing Jacobian and Gradient with GradientTape is extremely slow

Please make sure that this is an issue related to performance of TensorFlow.
As per our
GitHub Policy,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:performance_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): CentOS Linux 7
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): pip
TensorFlow version (use command below): 2.6.0
Python version: 3.9
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version:
GPU model and memory:

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with:

TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"

Describe the current behavior
I want to compute the Jacobian and Gradient of my model for a loss function, but the model just throws a warning about retracing and gets stuck
Describe the expected behavior
Normal training should occur
Standalone code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/Jupyter/any notebook.
def compute_loss_theta(tape, parameter, concept, output, x):

    b = x.shape[0]
    in_dim = (x.shape[1], x.shape[2])

    feature_dim = in_dim[0]*in_dim[1]

    J = tape.batch_jacobian(concept, x)

    grad_fx = tape.gradient(output, x)
    grad_fx = tf.reshape(grad_fx,shape=(b, feature_dim))
    J = tf.reshape(J, shape=(b, feature_dim, feature_dim))

    parameter = tf.expand_dims(parameter, axis =1)

    loss_theta_matrix = grad_fx - tf.matmul(parameter, J)

    loss_theta = tf.norm(loss_theta_matrix)

    return loss_theta

for i in range(10):
    for x, y in train_dataset:

        with tf.GradientTape(persistent=True) as tape:
            tape.watch(x)

            parameter, concept, output = model(x)

            loss_theta = compute_loss_theta(tape, parameter, concept, output , x)

            loss_y = loss_object(y_true=y, y_pred=output)

            loss_value = loss_y + eps*loss_theta

        gradients = tape.gradient(loss_value, model.trainable_weights)
        optimizer.apply_gradients(zip(gradients, model.trainable_weights))

This function is extremely slow. I want to compute the loss as given in the self-explanatory neural network. Input has a shape of (32, 365, 3) where 32 is the batch size. The loss I want to minimize is Equation 3 of the paper.
Other info / logs Include any logs or source code that would be helpful to
diagnose the problem. If including tracebacks, please include the full
traceback. Large logs and files should be attached.

Hi @melioristic ,Could you please providing a link  the training_dataset and update the full error stack trace in above template too? It helps expedite the issue.

@mohantym I tried to reproduce this slow behavior. And then looking through the details I found that the Jacobian part is not slow. I was just trying to predict long sequences. You can close this issue. Just on a side note, can you please look through the notebook that I have shared here? In the last block can you just see the computation and let me know if this is the fastest way to update the gradients or not. Any suggestion would be really helpful.
P.S. you can close the issue if you find this fine.

@melioristic  , I was able to remove the warning messages though by changing tabs to put tape.gradient() and opt.apply_gradients() to keep them in context with tf.gradient. Providing GIST for reference.

@mohantym Thanks I think, I will assume that this is the fastest it can work in eager mode. I made this a function and added @tf.function decorator and its speed increased considerably.

Yes ,Agreed! @melioristic ,Checked with @tf.function(autograph=True)  earlier  , was able to reduce  existing time  approx 4 times (Around 0.25 after first two batch) . Ok! Feel to Free to close this if it helped .Thanks!


[WFCORE-6591] REmove module dependencies on org.jboss.as.security

Jira issue: https://issues.redhat.com/browse/WFCORE-6591

Thanks for reviewing this @darranl


improve createTheme type

The type of theme property values was changed in https://github.com/kuma-ui/kuma-ui/pull/306 from being returned as primitive types to being returned as literal types.

Life savior


Update zzz onlyMembers parameter

Update zzz to change onlyMembers parameter

There is a conflict in your PR, can you resolve it? You can use documentation to update your fork https://docs.snapshot.page/guides/create-a-space#1-fork-snapshot-spaces-repository-here

From what I can see the documentation has been implemented in this code. The conflict looks to be with the onlyMembers parameter, which we are trying to change on purpose.

There is a conflict in your PR, can you resolve it? You can use documentation to update your fork https://docs.snapshot.page/guides/create-a-space#1-fork-snapshot-spaces-repository-here

From what I can see the documentation has been implemented in this code. The conflict looks to be with the onlyMembers parameter, which we are trying to change on purpose. If there is another way I should proceed to change this parameter please let me know.

Look good now! I merge your PR.


Fix inverted retro take anys

Removed 'Dark Sanctuary Hint' from take any pool if mode is inverted.

In addition to the retro fix, someone in #multi-planning mentioned swordless wasn't working with vanilla entrances. It wasn't able to place one of the TR small keys seemingly 100% of the time. I adjusted it so that the swordless rules are set before the TR key rules and it appears to work.


[Question]Can Yurt-app-manager be installed on the master?

What happened:
When I initialize an OpenYurt cluster (only master node) and install yurtappmanager, the pending state is displayed.
[root@node1 test-sealer]# kubectl get pod -A
NAMESPACE         NAME                                       READY   STATUS    RESTARTS   AGE
calico-system     calico-kube-controllers-5689d4dfdf-hhx4h   1/1     Running   0          11m
calico-system     calico-node-hfp2m                          1/1     Running   0          11m
calico-system     calico-typha-7d8c8877cd-zr5h2              1/1     Running   0          11m
kube-system       coredns-597c5579bc-qwj6x                   1/1     Running   0          11m
kube-system       coredns-597c5579bc-r6ljw                   1/1     Running   0          11m
kube-system       etcd-node1                                 1/1     Running   0          11m
kube-system       kube-apiserver-node1                       1/1     Running   0          11m
kube-system       kube-controller-manager-node1              1/1     Running   0          11m
kube-system       kube-proxy-d5vm5                           1/1     Running   0          11m
kube-system       kube-scheduler-node1                       1/1     Running   0          11m
kube-system       yurt-app-manager-5bd79d77bc-6q68v          0/1     Pending   0          11m
kube-system       yurt-app-manager-5bd79d77bc-7rpf7          0/1     Pending   0          11m
kube-system       yurt-controller-manager-7b8799cb9-h5lpm    1/1     Running   0          11m
kube-system       yurt-tunnel-server-5dcd9f657f-w5f4q        1/1     Running   0          11m
tigera-operator   tigera-operator-86c4fc874f-c9kwg           1/1     Running   0          11m

the msg is:
[root@node1 test-sealer]#
[root@node1 test-sealer]# kubectl describe pod -n kube-system yurt-app-manager-5bd79d77bc-6q68v
Name:                 yurt-app-manager-5bd79d77bc-6q68v
Namespace:            kube-system
Priority:<PHONE_NUMBER>
Priority Class Name:  system-node-critical
Node:                 <none>
Labels:               control-plane=yurt-app-manager
                      pod-template-hash=5bd79d77bc
Annotations:          <none>
Status:               Pending
IP:
IPs:                  <none>
Controlled By:        ReplicaSet/yurt-app-manager-5bd79d77bc
Containers:
  manager:
    Image:      registry.cn-hangzhou.aliyuncs.com/openyurt/yurt-app-manager:latest
    Port:       9443/TCP
    Host Port:  0/TCP
    Command:
      /usr/local/bin/yurt-app-manager
    Args:
      --enable-leader-election
      --v=4
    Environment:  <none>
    Mounts:
      /tmp/k8s-webhook-server/serving-certs from cert (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-g9nb4 (ro)
Conditions:
  Type           Status
  PodScheduled   False
Volumes:
  cert:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  yurt-app-webhook-certs
    Optional:    false
  default-token-g9nb4:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-g9nb4
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  beta.kubernetes.io/arch=amd64
                 beta.kubernetes.io/os=linux
                 openyurt.io/is-edge-worker=false
Tolerations:     node-role.openyurt.io/addon:NoSchedule op=Exists
                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age                   From               Message
  ----     ------            ----                  ----               -------
  Warning  FailedScheduling  32s (x10 over 6m32s)  default-scheduler  0/1 nodes are available: 1 node(s) didn't match node selector.
  Warning  FailedScheduling  20s                   default-scheduler  0/1 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate.
[root@node1 test-sealer]#

I want to know why the tolerance of yurt-app-manager is
// https://github.com/openyurtio/yurt-app-manager/blob/master/config/setup/all_in_one.yaml

    tolerations:
     - effect: NoSchedule
       key: node-role.openyurt.io/addon
       operator: Exists

Should it be changed to
 tolerations:
    - effect: NoSchedule
      key: node-role.kubernetes.io/master
      operator: Exists

so that it can be installed on the master as other components(yurttunnel server etc).
On the other hand, under what circumstances will the node have taints node-role.openyurt.io/addon ?
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:

OpenYurt version:
Kubernetes version (use kubectl version):
OS (e.g: cat /etc/os-release):
Kernel (e.g. uname -a):
Install tools:
Others:

others
/kind question

in default sealer add this taints,  I think we can remove this tains.
 tolerations:     - effect: NoSchedule       key: node-role.kubernetes.io/master       operator: Exists

in default sealer add this taints, I think we can remove this tains.
tolerations:
    - effect: NoSchedule
      key: node-role.kubernetes.io/master
      operator: Exists

The master I created with kubeadm init also has this taints.
maybe this taints is related to the version of k8s, v1.18.8 doesn't have, v1.20.0 has.
Anyway, yurt-app-manager should consider this situation.

@Peeknut The tolerations setting of yurt-app-manager has been updated in file: https://github.com/openyurtio/yurt-app-manager/blob/master/config/setup/all_in_one.yaml
so i will close this issue, and you can feel free to reopen this issue or create new issues if you need.


[AutoPR databoxedge/resource-manager] chore: jsonfmt databoxedge

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/7258

This PR has been merged into https://github.com/Azure/azure-sdk-for-go/pull/5824


test(robot): add uninstallation check test case

Which issue(s) this PR fixes:

Issue #longhorn/longhorn#9222
What this PR does / why we need it:
Transfer manual test case Uninstallation Checks into robot framework
Special notes for your reviewer:
issue command

./run.sh -t "Uninstallation Checks" -v LONGHORN_BRANCH:master -v INSTALL_METHOD:kubectl
./run.sh -t "Uninstallation Checks" -v INSTALL_METHOD:helm

Additional documentation or context
report.zip

Need to wait for clarification on https://github.com/longhorn/longhorn/issues/9303 before reviewing this PR. Currently, the PR only checks logs for level=error and level=fatal instead of looking for the word error alone.
assert "level=error" not in logs
assert "level=fatal" not in logs

Need to wait for clarification on longhorn/longhorn#9303 before reviewing this PR. Currently, the PR only checks logs for level=error and level=fatal instead of looking for the word error alone.
assert "level=error" not in logs
assert "level=fatal" not in logs

This ticket is ready for review, I have changed the log check part to make sure no word error appeared in uninstall pod.

@longhorn/qa, this ticket is ready for review.

@longhorn/qa , this ticket updated and ready for revirew, thank you.


generic topic validation and parsing

see tests for parsing examples.

not sure where this should go, it's now in init.lua. Should it go into client.lua maybe?
possibly every subscription could be parsed by the client, and the parsed results passed along in the event handler.
thoughts?

not sure where this should go, it's now in init.lua. Should it go into client.lua maybe?

init.lua is ok as you've added a whole-module functions.

possibly every subscription could be parsed by the client, and the parsed results passed along in the event handler.
thoughts?

Actually I'm not sure these validation functions is required at all.
I mean, for subscribe and publish packets the topic should be just a string, some broker validates it, some not, and some can even provide custom wildcards/filters in that topics. So there is no need for the client-side validation because good broker will do this for us and return appropriate SUBACK/PUBACK code.
And for custom brokers the client-side topic validation can do an obstacle for the library user.
IMO, only the last method to check topic and mask match can be frequently used in the client code - in the incoming message handler. There might be added a new client method like this:
client:add_msg_handler(topic_mask, callback) similar to the client:subscribe() method, adding not a SUBACK callback, but a PUBLISH handler called only for messages with matching topic.

Actually I'm not sure these validation functions is required at all.

agreed they are not, and I would not implement them in the client itself. They are just convenience functions for user-code that wants to validate user-input.
Btw; the parser function needs to compile, and the compile function needs to validate, so 3 out of the 4 are needed. So I guess the last one (to validate non-wildcarded topics) is then just for completeness sake.

client:add_msg_handler(topic_mask, callback) similar to the client:subscribe() method, adding not a SUBACK callback, but a PUBLISH handler called only for messages with matching topic.

yeah, I was thinking along the same lines. But in that case, who will be responsible to call the ACK on the received message?

client:add_msg_handler(topic_mask, callback) similar to the client:subscribe() method, adding not a SUBACK callback, but a PUBLISH handler called only for messages with matching topic.

yeah, I was thinking along the same lines. But in that case, who will be responsible to call the ACK on the received message?

This is not related. I think, like in usual message handler, ack should be done explicitly like this: https://github.com/xHasKx/luamqtt/blob/master/examples/simple.lua#L38

exactly, but then you'd always need 2 handlers, a generic one that does the ACK, and another one matching a specific topic mask.
Still learning on the MQTT stuff; why doesn't the client send the ACK by default? why relying on user-code to do that?

exactly, but then you'd always need 2 handlers, a generic one that does the ACK, and another one matching a specific topic mask.

I think user can use subscribe handlers in 3 ways:

set handlers only topic-specific (new hypothetical method client:add_msg_handler(topic_mask, callback))
set only one generic handler (current method client:on("message", callback))
set both handlers
For now I think it will be better to leave user always responsible for sending ack. And if he will use last case with handlers set in both ways and send ack twice accidentally - this will not be a MQTT protocol violation (as I remember). Maybe some brokers will close connection in this case but I think it's uncommon.

Still learning on the MQTT stuff; why doesn't the client send the ACK by default? why relying on user-code to do that?
For example, MQTT can be used to distribute some tasks between workers, and if worker can't handle some task - it may not send ack to indicate failure (in the other words, worker will not send ack untill task is successfully done). Thus the task will stay in a queue and thus can be handled by some different worker. This scenario requires a persistent mqtt session (clean=false)

I suppose there might be a client option for __init() method enabling auto-ack for all messages, but I think it should be disabled by default.

For example, user can use topic-specific handler to execute it's logic and do ack, and at the same time also use generic client:on() message handler for logging without ack.

For example, MQTT can be used to distribute some tasks between workers, and if worker can't handle some task - it may not send ack to indicate failure (in the other words, worker will not send ack untill task is successfully done). Thus the task will stay in a queue and thus can be handled by some different worker. This scenario requires a persistent mqtt session (clean=false)

That makes sense, thx for the explanation

@xHasKx any chance we can move this forward? anything required from me?
I rebased the PR on master to it get up-to-date

@xHasKx any chance we can move this forward? anything required from me?
I rebased the PR on master to it get up-to-date

@Tieske, I posted some comments (change requests) on the commits in the PR, please take a look

And I'm still concerned about mqtt.topic_match function. It's arguments are complicated and it's doing two tasks at the same time:

check that the topic matches the subscribe-topic-filter string
parse part of the topic at wildcards places

IMO, we can break that into two functions, first one is just compile_topic_pattern+string.match call, and the second one can be just a helper to split a particular topic string with / char into an array (table)
What do you think?

IMO, we can break that into two functions, first one is just compile_topic_pattern+string.match call, and the second one can be just a helper to split a particular topic string with / char into an array (table)

Actually string.match will return the words at the wildcards of the topic, so your topic_match function is just adding keys to them. Does it actually required?

Pushed a commit with the review comments (probably easiest to review that commit without whitespace changes).

the validators don't ever throw errors, see comments above. Let me know if you want me to change that behaviour anyway.
throw errors on bad input when matching or compiling patterns
updated ldoc comments
moved patterns into narrow scoped module locals
made the error messages more descriptive

Actually string.match will return the words at the wildcards of the topic, so your topic_match function is just adding keys to them. Does it actually required?

No, it is not required. It is just convenience to get named arguments/parameters, so user code can be more readable. I have a number of 'id' type fields, which I hate to have in code as array indices, using names is so much more convenient for that. That's the whole purpose I guess (and caching the compiled pattern).

IMO, we can break that into two functions, first one is just compile_topic_pattern+string.match call, and the second one can be just a helper to split a particular topic string with / char into an array (table)

compile + string_match would not work, since there would be no structure where the compiled pattern can be stored (cached) for future use. So that would have to be implemented in user code, and then this function would be reduced to only a string.match call, so a function solely for that would not make sense to me.
That said, I do have doubts about the return values, or the entire interface. Currently I do not like the 2 return values, makes it cumbersome to handle.
Also the function call with the topic and options, since the options table stores the cached pattern. Maybe the options table should become a class, from which a "topic" can be instantiated, that would hold all the named values. That would make the usercode more natural to retain the "class" instead of an options table.
something like;
local topic_matcher = mqtt.create_topic_matcher {
    topic = "homes/+/+/#",
    "homeid",
    "roomid",
    "varargs",
}

local fields, err = topic_matcher("homes/myhome/living/mainlights/brightness")

print(fields.homeid)       --> "myhome"
print(fields.roomid)       --> "living"
print(fields.varargs[1])   --> "mainlights"
print(fields.varargs[2])   --> "brightness"

This would enable adding a new incoming callback per topic-pattern, where the msg parameter would already contain the parsed fields from the topic.

@Tieske, I wrote some comments and continue review tomorrow

@Tieske, what do you think about that form:
local topic_matcher = mqtt.topic_matcher { -- without "create_" it looks like a class constructor calling
    topic = "homes/+/+/#",
    "homeid",
    "roomid",
}

local fields, err = topic_matcher("homes/myhome/living/mainlights/brightness")

print(fields.homeid)       --> "myhome"
print(fields.roomid)       --> "living"
print(fields[1])           --> "myhome"
print(fields[2])           --> "living"
print(fields[3])           --> "mainlights"
print(fields[4])           --> "brightness"

It will simplify mqtt.topic_matcher constructor args validation so you don't have to check that amount of number-indexed keys are equals to the /+/ topic wildcards.
Another variant is to explicitly provide a key to store array of all unnamed topic words like that:
local topic_matcher = mqtt.topic_matcher { -- without "create_" it looks like a class constructor calling
    topic = "homes/+/+/#",
    "homeid",
    "roomid",
    rest = "varargs",
}

local fields, err = topic_matcher("homes/myhome/living/mainlights/brightness")

print(fields.homeid)       --> "myhome"
print(fields.roomid)       --> "living"
print(fields[1])           --> "myhome"
print(fields[2])           --> "living"
print(fields.varargs[1])   --> "mainlights"
print(fields.varargs[2])   --> "brightness"

It will simplify mqtt.topic_matcher constructor args validation so you don't have to check that amount of number-indexed keys are equals to the /+/ topic wildcards.

if we adjust compile_topic_pattern to return as second and third return values the number of + and # found, that check could actually be very simple.
This option:
print(fields.homeid)       --> "myhome"
print(fields.roomid)       --> "living"
print(fields[1])           --> "myhome"
print(fields[2])           --> "living"
print(fields[3])           --> "mainlights"
print(fields[4])           --> "brightness"

has as a drawback that you can't immediately tell where the varargs start in the array part. Hence I would prefer your second option;
print(fields.homeid)       --> "myhome"
print(fields.roomid)       --> "living"
print(fields[1])           --> "myhome"
print(fields[2])           --> "living"
print(fields.varargs[1])   --> "mainlights"
print(fields.varargs[2])   --> "brightness"

But here the user needs an extra field name for the varargs, to prevent namecollisions between the field names and the varargs field.
Leading me to:
print(fields.homeid)       --> "myhome"
print(fields.roomid)       --> "living"
print(fields[1])           --> "mainlights"
print(fields[2])           --> "brightness"

as the simplest version, clearly separating the known fields from the varargs, whilst making varargs size and traversal easy as the array part. And no extra user specified name required.

Leading me to:

I suggested rest = "varargs" because of that possible case:
local topic_matcher = mqtt.topic_matcher {
    topic = "homes/+/+/#",
    "homeid",
    -- "roomid", -- note only one name for +
    ...
}

Where the second + wildcard are without given name.
In your case its value from the topic will be lost.
And we have to find a proper form for the case when user don't have names for +-es at all.

Leading me to:

I suggested rest = "varargs" because of that possible case:
local topic_matcher = mqtt.topic_matcher {
    topic = "homes/+/+/#",
    "homeid",
    -- "roomid", -- note only one name for +
    ...
}

Where the second + wildcard are without given name.
In your case its value from the topic will be lost.
And we have to find a proper form for the case when user don't have names for +-es at all.
We can define some default name for the rest = "varargs" to store all +-wildcards in the root returned object and all #-wildcards into it's subfield with that default name.
Or accept that it's ok to return two values from the matcher function - first table with +-es with all values as indexes and some of them with names, and the second table with #-s as an array.
What do you think?
We can define some default name for the rest = "varargs" to store all +-wildcards in the root returned object and all #-wildcards into it's subfield with that default name.
Or accept that it's ok to return two values from the matcher function - first table with +-es with all values as indexes and some of them with names, and the second table with #-s as an array.
What do you think?

Where the second + wildcard are without given name.
In your case its value from the topic will be lost.

is that expected to be a common case? I’d expect all fields name to be the common case. We can just throw an error if there aren’t enough field names?
Or we could inject numbered names  field 1 etc

I would say, we can't predict which case will be more common. The only thing I'm sure is that the best interface is the one that hard to use in the wrong way unintentionally.
We can do one step back and ask how that topic_matcher can be used?
I mean, users start their code from the client instance - creating it, then connecting and subscribing it to some topics.
Then somewhere has to be a callback function with arrived PUBLISH message in its argument. And that message's topic can be parsed using the topic_matcher instance.
But who should call the topic_matcher instance - the library before calling callback or the user itself inside the callback code?
I mean, user can call some new method client:add_msg_handler(topic_filter, matcher_options, callback) where matcher_options is an args for the topic_matcher instance creation. Then for the each new PUBLISH messages callback will be called in that way:
callback(publish_message, topic_fields, topic_varargs_fields)

In this case for the the topic_matcher instance will be better to return 2 values - +-wildcards with optional names and #-wildcards as two separate tables (by the way, the second table also can contain user-defined names).
In the case of two tables in resulf of the topic_matcher, it's better to always set array-keys into the table and optionally add user-defined names.
(and of course users can create and use the topic_matcher instances separately, in the only one publish messages callback)
What do you think?


Views Counter

Need to fix the view counter, using prisma and planetscale.

Fixed with the latest commit.


make Walk() fit for range-over-func

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

15 of 17 (88.24%)  changed or added relevant lines in 2 files are covered.
2 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.4%) to 95.276%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

table.go
7
9
77.78%

Files with Coverage Reduction
New Missed Lines
%

table.go
2
96.82%

Totals

Change from base Build<PHONE_NUMBER>:
-0.4%

Covered Lines:
1069

Relevant Lines:
1122

💛  - Coveralls


make provider getter minification safe

When minified, I was getting this error from munged injection:
Error: [$injector:unpr] Unknown provider: aProvider <- a <- FeatureFlags <- featureFlagDirective <- FeatureFlags
http://errors.angularjs.org/1.3.16/$injector/unpr?p0=aProvider%20%3C-%20a%20%3C-%20FeatureFlags%20%3C-%20featureFlagDirective%20%3C-%20FeatureFlags

Fixed by using the minification safe syntax.

O I C -... the aProvider was a byProduct of minification and then angular attempting to load class "a". haha. I'm smart I'm smart I'm "S M R T" smart...
-Homer J. Simpson

@costacruise Can you take a look at / merge this PR?  Would like to avoid forking the repo, but this is blocking us from production use.
More generally, are you planning to maintain this project?  If not I'd be happy to help out.

LGTM

Thanks for the PR!

@nicolai86 do you mind tagging a new release as well? thanks!


HiFiHLA does not support CRAM format

Call reads for HLA-A
[2024-07-20T19:29:04.792Z ERROR hifihla::cli::callreads_cli] Aligned reads must be in BAM format: work/sunnybrook/temp/jjunhyuk.park/HG01359/HLA-A/actionable/GRCh38/HG01359.final.cram

Just FYI. The CRAM file is downloaded from the 1000 Genome Project.

Most PacBio tools only support bam.


A good source of features: bpfcc/libbpf-tools

All tools from bpfcc/libbpf-tools project should be portable to libbpfgo if libbpfgo supported all features present in libbpf.
This can be a good source of supportability, tests and tools.
Initial suggestion is to bring:

bindsnoop
biolatency
biopattern

and then move on with the list.
We can sync .bpf.c portion of it from bpfcc/libbpf-tools project and keep userland in go.

I created a repo to do this work and so far 26/46 tools have been ported:
https://github.com/mozillazg/libbpfgo-tools

@geyslan Sorry, I'm really too busy this year to get this done. But I have no objections regarding this matter, you can proceed with it freely.


Beacons still used for calculation when downgrading assembling machine to 1

There is a slight issue with using beacons
Would it be possible to add the beacons display for assembling machines 1?
Or at least clear the beacon setting if you change from assembling machines 2 or 3 to assembling machines 1 (it still uses the beacons for calculating but you cant see that)
Example:
https://kirkmcdonald.github.io/calc.html#min=2&items=electronic-circuit:f:1&ignore=iron-plate,copper-plate&modules=electronic-circuit:;s3:4
Change the assembling machines to use assembling machines 1 the beacons are still used for calculation although you cant see or change that
https://kirkmcdonald.github.io/calc.html#items=electronic-circuit:f:1&ignore=iron-plate,copper-plate&modules=electronic-circuit:;s3:4

Fixed in 306ce324a9020a9da0d97941c7497e100fc6c7cb.
Beacons don't affect things that don't support modules. I have made such assemblers ignore any previously-configured beacon effect.

Thank you - however assembling machines 1 are affected by beacons as stated on the wiki for beacons https://wiki.factorio.com/Beacon even though they have no module slots

Only buildings with module slots can benefit from beacon effects (i.e. laser turret doesn't benefit). The only exceptions to this rule are Assembling machine 1s, Burner mining drills (both of which don't accept modules but are affected by beacons), and beacons themselves which don't benefit from the modules inserted in themselves (or other beacons), so their energy cost can't be reduced.


IPv6 IPAM-managed address blocks fail on second apply

What happened?
I deployed a VPC with the following parameters (where pools.ipv6.id and pools.ipv4.id) are exernally created AWS IPAM pools:
const vpc = new aws.ec2.Vpc(
  "vpn",
  {
    ipv4IpamPoolId: pools.ipv4.id,
    ipv6IpamPoolId: pools.ipv6.id,
    ipv4NetmaskLength: 20,
    ipv6NetmaskLength: 56,
  },
  { provider }
);

Checking this VPC in the AWS console, the IPv6 pool was listed as IPAM Managed - Associated.
I then ran pulumi up a second time, without making any changes. This showed the following changes during preview:
    ~ aws:ec2/vpc:Vpc: (update)
        [id=vpc-06b282e4c36d5e329]
        [urn=urn:pulumi:kai::vpn::aws:ec2/vpc:Vpc::vpn]
        [provider=urn:pulumi:kai::vpn::pulumi:providers:aws::vpn::b388bb36-9b01-435d-b432-77d5ff419602]
      ~ ipv6IpamPoolId   : "IPAM Managed" => "ipam-pool-029ddd08697068a8e"
      ~ ipv6NetmaskLength: 0 => 56

The update failed with the error below:
  aws:ec2:Vpc (vpn):
    error: 1 error occurred:
    	* updating urn:pulumi:kai::vpn::aws:ec2/vpc:Vpc::vpn: 1 error occurred:
    	* updating EC2 VPC (vpc-06b282e4c36d5e329): associating IPv6 CIDR block: InvalidParameterCombination: The parameter 'ipv6NetmaskLength' may not be used in combination with 'ipv6CidrBlock'.
    	status code: 400, request id: 9acb086f-09e8-4700-9f88-1a0058d1508e

After this failure, in the AWS console, the IPv6 pool was listed as IPAM Managed - Disassociated
Expected Behavior
I expected Pulumi to not make any changes on the second update.
Steps to reproduce
The content above ("what happened") be sufficient to reproduce:

Create an IPv6 IPAM pool (can be done manually in the console)
Create a VPC using this IPAM pool for it's IPv6 CIDR
pulumi up again after creating the VPC

Output of pulumi about
CLI
Version      3.55.0
Go Version   go1.20
Go Compiler  gc

Plugins
NAME    VERSION
aws     5.30.0
docker  3.6.1
nodejs  unknown
random  4.11.2

Host
OS       darwin
Version  13.1
Arch     arm64

This project is written in nodejs: executable='/Users/kai/.nix-profile/bin/node' version='v18.14.0'

Backend
Name           pulumi.com
URL            https://app.pulumi.com/kwohlfahrt
User           kwohlfahrt
Organizations  kwohlfahrt, CHARM-Tx

Dependencies:
NAME                  VERSION
@pulumi/pulumi        3.55.0
ip-num                1.5.0
typescript            4.9.5
zod                   3.20.6
@charm-tx/pulumi-aws  0.3.0
@pulumi/aws           5.30.0
@pulumi/random        4.11.2
@types/jest           29.4.0
@types/node           16.18.12
ts-jest               29.0.5
ts-node               10.9.1

Additional context
No response
Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

A tenuously related additional bug - I'm seeing a similar diff with subnet resources created in an IPv6 VPC:
    ~ aws:ec2/subnet:Subnet: (update)
        [id=subnet-0ffff15057dd016c0]
        [urn=urn:pulumi:kai::vpn::aws:ec2/subnet:Subnet::vpn]
        [provider=urn:pulumi:kai::vpn::pulumi:providers:aws::vpn::b388bb36-9b01-435d-b432-77d5ff419602]
      ~ ipv6CidrBlock: "2a05:d024:0:6200::/64" => "2a05:d024:0:6200:0:0:0:0/64"

The CIDR blocks are two identical ways of writing the same address range.

Hi @kwohlfahrt, sorry for the delay in responding to this. I haven't been able to reproduce this error mode yet because I get

InvalidIpamPoolId.NotFound: The ipam-pool IDs ..., ... do not exist

even though I see them in the AWS console.
Can you share more detail on your pools and their configuration? Thanks!

Hm, I don't think there's much unusual about the set-up. There's a global IPAM resource, and the IPv6 pools (which are the interesting ones for this bug) are provisioned in the public scope, like this:
const ipam = new aws.ec2.VpcIpam(
  "ipam",
  { operatingRegions: regions.map((regionName) => ({ regionName })) },
  { provider }
);

regions.forEach(region => new aws.ec2.VpcIpamPool(
  name,
  {
    ipamScopeId: ipam.publicDefaultScopeId,
    addressFamily: "ipv6",
    locale: region,
    publicIpSource: "amazon",
    awsService: "ec2"
  },
  { provider })
);

I don't think it matters what region the pools are provisioned from, as long as the locale matches the region you are creating your VPC in.

Ok, I'll need some help from more AWS-savvy teammates then. Will put this issue on the queue. Thanks for your understanding!

Hey @kwohlfahrt,
I had a look at this, but couldn't reproduce the issue with both the pulumi aws version you were using (v5.30.0) as well as the latest version (v6.52.0). Maybe this was a cloud API bug that was since fixed. Can you confirm whether you're still running into this issue?
This is the program I used to repro:
import * as aws from "@pulumi/aws";

// Replace with your scopes
const publicScopeId = "ipam-scope-abcd"
const privateScopeId = "ipam-scope-abcde"

const ipamPool = new aws.ec2.VpcIpamPool(
    "retriage",
    {
        ipamScopeId: privateScopeId,
        addressFamily: "ipv4",
        locale: "us-west-2",
    },
);
const ipv6Pool = new aws.ec2.VpcIpamPool(
    "retriage-v6",
    {
        ipamScopeId: publicScopeId,
        addressFamily: "ipv6",
        locale: "us-west-2",
        publicIpSource: "amazon",
        awsService: "ec2",
        publiclyAdvertisable: false,
    },
);

const cidr = new aws.ec2.VpcIpamPoolCidr("retriage", {
    ipamPoolId: ipamPool.id,
    cidr: "<IP_ADDRESS>/16",
})

const v6Cidr = new aws.ec2.VpcIpamPoolCidr("retriagev6", {
    ipamPoolId: ipv6Pool.id,
    netmaskLength: 52,
});

const vpc = new aws.ec2.Vpc(
    "retriage",
    {
        ipv4IpamPoolId: ipamPool.id,
        ipv6IpamPoolId: ipv6Pool.id,
        ipv4NetmaskLength: 20,
        ipv6NetmaskLength: 56,
    },
    {dependsOn: [cidr, v6Cidr]},
);


Build issues

Seems that when I build the project on my windows machine using cmake and mingw I get this string of errors after about halfway through the build. I have checked and a libuvw.a file is outputted but unsupringly it doesn't link to anything I can through at it.
[ 31%] Built target uv_a [ 50%] Built target uvw [ 51%] Built target fake [ 52%] Built target gtest [ 54%] Built target gtest_main [ 55%] Building CXX object test/CMakeFiles/main.dir/main.cpp.obj c++.exe: warning: /EHsc: linker input file unused because linking not done c++.exe: error: /EHsc: linker input file not found: No such file or directory mingw32-make[2]: *** [test\CMakeFiles\main.dir\build.make:76: test/CMakeFiles/main.dir/main.cpp.obj] Error 1 mingw32-make[2]: *** Deleting file 'test/CMakeFiles/main.dir/main.cpp.obj' mingw32-make[1]: *** [CMakeFiles\Makefile2:1129: test/CMakeFiles/main.dir/all] Error 2 mingw32-make: *** [Makefile:145: all] Error 2
This is the error msg that I get from cmake, I'm quite new to all this sort of things but I can't seem to find what I'm doing wrong

No idea, I'm sorry. I don't even know if mingw is fully supported since it's not on the CI nor there is a plan to address it.
At first glance, it looks like a linking issue and therefore the error is probably in your cmake file but 🤷♂ it's hard to say from a non-formatted wall of messages as returned by the compiler.
Try to provide a little more information if possible. Otherwise, I think there is no way to help you, sorry.

Thanks for answering so quickly. Not sure what more info is needed, builds well using MSVC obviusly, but I was hoping to use the library in a project I have going on using mingw, you think there's any way to do this?

Best way is to share a common environment to have a look. Can you try to setup such an environment with PR ?

I got it to work with mingw by building/installing libuv separately. With libuv installed you can link:
target_link_libraries(myproj libuv::uv uvw::uvw)
Then when you configure your project define -DFIND_LIBUV=ON
I'm sure there's a better way to do it, but that's the quickest workaround I could come up with.

FIND_LIBUV prevents the libuv to be fetched by CMake (if it can be found somewhere on the HOST building machine).
I'm going to close this, if you are willing to deep dive into more details, feel free to reopen it.


ci(danger-action): replace danger action with shared-github-danger (IDFGH-11843)

Replaces old version of Danger GitHub by our new tool from shared repo.
Old version will be removed in 03/2024.

Related

spotted in: https://github.com/espressif/idf-python-wheels/pull/23#

Thank you @tomassebestik!


Sample dataset for training and documentation

@bsadr
Hi there,

Upload a sample dataset to test training on.
add documentation on how to train and predict.

@deepseek,
Thanks for your interest. This work is still under progress. I'll add documentation and upload the dataset in a month or two.


CSV Extract from NeoDash adds quotes and leading white space

NeoDash version: 2.2.3
Neo4j Database version: 4.4.9 (Enterprise)

When using the Table Output in NeoDash and allowing for CSV download, the CSV itself formats the results strangely and in a way that is problematic for users who depend on these extracts.
Expected, Ideal behavior:

An output just like that of the Neo4j Browser when you run a query that results in a table and click "Export CSV"

Which looks like this in Excel (it's perfect, this is what I want NeoDash to do):

NeoDash Table Visual Behavior (Not perfect, but ok):

Actual behavior from "Download CSV" from NeoDash Visual:

Adds quotes at beginning and end of all strings after the first column
Adds a white space before every column name after the first column

I'm currently working with the CSV download system on another issue, so I'll check this one out as well.

Perhaps something additional that can fit in here?
From a user: When you download a table as csv, then the value 5 will be exported as  {“low”:5;“high”:0}

We should make sure to cast the javascript integers to human readable numbers when doing an export as a quick check.

Perhaps something additional that can fit in here?
From a user: When you download a table as csv, then the value 5 will be exported as  {“low”:5;“high”:0}

We should make sure to cast the javascript integers to human readable numbers when doing an export as a quick check.

Ok, I add this one to task list.

Currently, I just PR a solution for the leading spaces problem. @GQ16  what version of Excel are you using? I'm not able to reproduce the problem with the quotes

@JonanOribe It's the latest version of excel I believe which is 2019?
Here is what I see when I export from NeoDash. Still the same issue.

@JonanOribe, I've made a discovery. It appears to me that the leading space in front of all the data in the columns (except for column 1, which doesn't have the leading space issue) is the reason excel keeps the quotes. If you eliminate the leading space, the quotes automatically drop off as expected.
Here is what happened after I removed the leading spaces using notepad before opening in excel:

@JonanOribe would you also mind looking at an example which includes a comma in the text columns? This also breaks any csv import because there is no possibilty to change the separator type during export.

No worries @NanisTe , I will add this one too to the task list

I'm still dealing with this issue after the latest update. I have to go into Notepad and do a find and replace on all the commas with a space after ", " and replace them with just commas ","
Once I do that, the file opens in excel without all the additional quotation marks. The leading spaces in front of all the columns other than the first is causing the issue.

Thanks for the video demo @GQ16 . The issue has been reopened so I'll check it out. Hopefully this time the bug fix will cover all possible scenarios.


Feature 27540 add unicredit bank as new provider

https://is.roihunter.com/issues/27540
Added new provider Unicredit.
@business-factory/pythonistas Please review and merge.

@DavidPrexta About the situation I described above, we need to use custom User-Agent header. 🙄

@harpener Thanks for the review. I run the code before, it worked fine. But now I got the same error as you so I guess they changed something. I added User-Agent ROI Hunter /Currency Downloader; http://www.roihunter.com/ (not sure if OK, I got inspired by lady).
I added CZK currency manually with rate 1 so it can now be used too.
But in doing so I have noticed some weird behavior with the calculation of currency in method _conversion_to_base_currency. When converting from USD to CZK I should get the same rate as in https://www.unicreditbank.cz/cs/obcane/historical.html, so 22.83 but I got 0.043. I found out that problem is that we should calculate base_currency_rate/currency_rate in that method to get rate correctly.
I also tried it with other conversions like USD to EUR where I got rate 1.123..., which is not correct. After altering the method, I got the correct rate of 0.89. Checked it with Google which was 0.88 so almost the same. Same goes for other ratings.
I checked how it works with fixed and I think it should also be switched. Having response form fixed e.g.
{
    "success": true,
    "historical": true,
    "date": "2013-12-24",
    "timestamp":<PHONE_NUMBER>,
    "base": "GBP",
    "rates": {
        "USD": 1.636492,
        "EUR": 1.196476,
    }
}

where we want to convert USD to EUR we would calculate it by dividing 1.196476 / 1.196476 which would give 0.7311. Switching it we would get 1.3677 which is almost the same as the rate was for USD/EUR on that day https://www.poundsterlinglive.com/best-exchange-rates/euro-to-us-dollar-exchange-rate-on-2013-12-24.
Please check my changes. Maybe I did the math wrong, but rates are corresponding to what they should be.

@DavidPrexta I saw the changes you made to the rate conversion and I tried to investigate whether it was a bug before or not. I actually came to a conclusion that you fixed a bug, because we use the same calculation on more places and it is always like to_rate / from_rate, i.e. base_currency_rate / currency_rate. Calling the exchange rate manager for converting EUR to CZK confirmed it. What do you say? :-)

Hmm, I honestly admit that I am not sure and it's hard to say, because I don't know if exchange rate is how much of to_currency_rate makes one from_currency_rate. I'm losing the abstract insight. :-) Anyway, keep in mind that some rates returned from Unicredit are 0 and you get division error during the conversion - maybe ignore those rates.

Ok, I compared the results from Grandtrunk and Unicredit and came to a conclusion that you did find a bug and it was wrong before in Fixed and it is wrong now that you reverted the fix. :-) You can also write tests for the provider, if you want to. :-)


refactor: extract proto map processing to Config::Utility and use absl::flat_hash_map for storage.

Signed-off-by: Lizan Zhou<EMAIL_ADDRESS>Description:
For #4475. Not very clean, but will make add google.protobuf.Any support easier.
Risk Level: Low
Testing: CI (refactor shouldn't affect existing tests)
Docs Changes: N/A
Release Notes: N/A

@lizan friendly ping :)

Closing this since it seems to be too much effort for just two use cases, adding Any to each of them is not that bad. Will do flat_hash_map in another PR.


Consider making DACPAC deployment cancelable

We can't control SQL Tools operations, but we do can control our own deployment

Not needed


Bad link to EventHubTriggerAttribute definition

In following section of the document
https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-event-hubs#trigger---attributes
there is a hyperlink to following, which returns page not found, please fix it
https://github.com/Azure/azure-webjobs-sdk/blob/master/src/Microsoft.Azure.WebJobs.Extensions.EventHubs/EventHubTriggerAttribute.cs

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0b24ff8d-a59e-01e0-205d-ac777f06bb7a
Version Independent ID: f2106a8a-28fe-ebdf-525e-1d93013778fa
Content: Azure Event Hubs bindings for Azure Functions
Content Source: articles/azure-functions/functions-bindings-event-hubs.md
Service: azure-functions
GitHub Login: @craigshoemaker
Microsoft Alias: cshoe

Thanks, @mganeshphani! This is a duplicate of #40676 which was resolved this morning 😃 #please-close

@craigshoemaker Thanks for the quick response. :+1:


Sidecars in PERMISSIVE mode should use mTLS within Istio cluster.

The intention behind the PERMISSIVE mode was to allow legacy clients to connect to an otherwise mutually-authenticated Istio cluster. Basically, a STRICT mode that also accepts clients without mTLS during transition from the legacy infrastructure.
However, it seems that the current implementation of the PERMISSIVE mode implements only the first part (accepting both: legacy and mTLS traffic for inbound connections), but doesn't deploy mutually-authenticated Istio cluster, and the communication between sidecars is happening over plaintext.
Note: This above is based on the assumption that using install/kubernetes/istio-demo.yaml or global.mtls.enabled=false results in the PERMISSIVE mode being enabled for all services, per comments in the templates, though, I'm not sure how to verify that this is really happening.
cc @incfly @wattli @wenchenglu @louiscryan @duderino

We do have a design https://docs.google.com/document/d/1yEMDRO2FZCyZnDK1AzNmjQtbtQqE7wN8YoM65FAH7uA/edit#, but this got deprioritized.

The intention behind the PERMISSIVE mode was to allow legacy clients to connect to an otherwise mutually-authenticated Istio cluster. Basically, a STRICT mode that also accepts clients without mTLS during transition from the legacy infrastructure.

We had discussions with @louiscryan on this last quarter. And the conclusion is as follows:
sidecars are NOT suppose to always communicate with each other using mTLS in PERMISSIVE mode. A service with PERMISSIVE mode only means it can take both http and mTLS (that's why it is called 'PERMISSIVE'). We use it not only for legacy clients, but also for sidecar-enabled clients migration from http to mTLS (not all customers want to enable mTLS at the first place)

However, it seems that the current implementation of the PERMISSIVE mode implements only the first part (accepting both: legacy and mTLS traffic for inbound connections), but doesn't deploy mutually-authenticated Istio cluster, and the communication between sidecars is happening over plaintext.
Note: This above is based on the assumption that using install/kubernetes/istio-demo.yaml or global.mtls.enabled=false results in the PERMISSIVE mode being enabled for all services, per comments in the templates, though, I'm not sure how to verify that this is really happening.
cc @incfly @wattli @wenchenglu @louiscryan @duderino


Fan - Setting RotationSpeed

Device type

Stirling FS1-40DCWIFI Pedestal Fan
Device information

{   u'header': {   u'code': u'SUCCESS', u'payloadVersion': 1},
    u'payload': {   u'devices': [   {   u'data': {   u'online': True,
                                                     u'speed': u'3',
                                                     u'speed_level': 15,
                                                     u'state': u'false'},
                                        u'dev_type': u'fan',
                                        u'ha_type': u'fan',
                                        u'icon': u'https://images.tuyaeu.com/smart/icon/bay1589359175993MeEh/159540337782efe7f8e63.jpg',
                                        u'id': u'<redacted>',
                                        u'name': u'FS1-40DCWIFI'}],
                    u'scenes': []}}

Issue
Toggling on/off the fan via HomeKit works correctly.
However, when setting any rotation speed, the fan shows a speed of '00' on the physical display and stops rotating.
Setting the rotation speed via the TuyaSmart app works correctly.
Let me know what else I can provide to assist in diagnosing the issue.

Hey! Thank you for opening an issue.
In your logs you should be able to find some log lines that look something like the following
[device_name] Characteristic.RotationSpeed [SET] xxxx xxxx

Could you please provide me with your log after you set the rotation speed, then I can see what values are being transmitted and see what goes wrong there.

Thanks for looking into this!
Here are some values I've triggered just now:
[11/6/2020, 10:25:13] [TuyaWebPlatform] [FS1-40DCWIFI] Characteristic.RotationSpeed - [SET] 24 4
[11/6/2020, 10:25:13] [TuyaWebPlatform] [FS1-40DCWIFI] Characteristic.Active - [SET] 1 1
[11/6/2020, 10:25:15] [TuyaWebPlatform] [FS1-40DCWIFI] Characteristic.RotationSpeed - [SET] 30 5
[11/6/2020, 10:25:15] [TuyaWebPlatform] [FS1-40DCWIFI] Characteristic.Active - [SET] 1 1
[11/6/2020, 10:25:17] [TuyaWebPlatform] [FS1-40DCWIFI] Characteristic.Active - [SET] 1 1
[11/6/2020, 10:25:17] [TuyaWebPlatform] [FS1-40DCWIFI] Characteristic.RotationSpeed - [SET] 36 6
[11/6/2020, 10:25:20] [TuyaWebPlatform] [FS1-40DCWIFI] Characteristic.Active - [SET] 1 1
[11/6/2020, 10:25:20] [TuyaWebPlatform] [FS1-40DCWIFI] Characteristic.RotationSpeed - [SET] 24 4
[11/6/2020, 10:25:21] [TuyaWebPlatform] [FS1-40DCWIFI] Characteristic.Active - [SET] 1 1
[11/6/2020, 10:25:21] [TuyaWebPlatform] [FS1-40DCWIFI] Characteristic.RotationSpeed - [SET] 18 3
[11/6/2020, 10:25:28] [TuyaWebPlatform] [FS1-40DCWIFI] Characteristic.Active - [SET] 0 0

I have no clue what the problem is here.
It seems like the plugin is sending very reasonable values, I've tried it with my fan and that one works fine.

Yes, very strange indeed.
Is there anything that I can provide you with to assist with a diagnosis? Packet or request/response capture of working traffic from the Tuya app?

I cannot reproduce this issue and at the moment do not have the time to debug this with you sadly.
I would love for somebody to be able to submit a PR for this.

I've got a very similar problem. I've also submitted an issue for this.

I have a similar issue with my fan. For my fan, it looks like maybe the speeds are not set by percentage, but simply enumerated. My fan has three speeds. When I set it to the medium speed in the Smart Life app, I see this in the HomeBridge logs:
[05/03/2021, 00:03:28] [TuyaWebPlatform] [Ceiling Fan] Characteristic.Active - [GET] true
[05/03/2021, 00:03:28] [TuyaWebPlatform] [Ceiling Fan] Characteristic.RotationSpeed - [GET] 2

Also, on startup, it seems to indicate the number of speeds; three in my case:
[05/03/2021, 00:11:16] [TuyaWebPlatform] [Ceiling Fan] deviceConfig: {
  data: { online: true, state: 'false', speed_level: 3, speed: '2' },
  name: 'Ceiling Fan',
  icon: 'https://images.tuyaus.com/smart/icon/ay1566306500214Y6P2j/15856618854f216b2db9c.png',
  id: 'removed',
  dev_type: 'fan',
  ha_type: 'fan'
}

I guess there is a rounding error somewhere.
I'm strongly contemplating moving away from percentage based device control and instead move back to level based control thus only allowing you to select steps 0 (off), 1, 2, ... up til the max your device supports.

I'm inclined to agree with the suggestion of it being a rounding error - incrementing the speed on my 15-speed fan increases the percentage display by 6%, for a max speed displayed of 90% rather than 100%.
Looking forward to level based control if/when you have the time to make the change!

Any word on if/when this change to fan levels, rather than percentages may occur?
Thank you

No updates at the moment.

I have many ceiling fan and there many line of this generated in the log. Although, everything seems to be work, is there any I can do to get this off the log?
[homebridge-tuya-platform] This plugin generated a warning from the characteristic 'Rotation Speed': characteristic was supplied illegal value: number 166 exceeded maximum of 100


[general] Revert changes in https://github.com/ms-iot/common_msgs

In the beginning of the ROS1 windows ported, many messages have name conflicts with macros defined in Windows.h (which is included by ros.h by default) so we renamed them as a workaround.
Now we don't have windows.h always included in ros.h, we should consider removing those messages renames changes.

Closed it since the changes are merged to ms-iot repos.


Normalizing AORSA calculations

So far I have been using a volume integration over wdot_tot from the Efield_2D.vtk file to renormalize all quantities computed by AORSA. Now I am not so sure anymore if this is actually the correct way of doing things.
From AORSA I get this:
 Species absorption from Estar dot J:
         power absorbed by electrons = 0.84159E+06 Watts  =    84.1590 %
     power absorbed by majority ions = 0.15841E+06 Watts  =    15.8410 %
 Species absorption from Wdot:
         power absorbed by electrons = 0.35175E+06 Watts  =   100.0000 %
Species absorption from quasilinear Wdot:
         power absorbed by electrons = 0.58096E+06 Watts  =   100.0000 %

I know that Estar dot J is not good, but then there are two wdots out there, only one of which ends up in the .vtk file.
Is the wdot_tot in the .vtk File even in W/m^-3?
This inquiry is motivated by us getting vastly different density fluctuations between the cold plasma approximations and AORSA.

Follow-up comment.
I looked in the source code and it looks like per default everything is normalized to 1/2Re(E . J) which means I definitely need to do some renormalizing.
I found the spot in the code, where the different total absorptions are calculated, but I do not understand the integration itself.
In sgrator in aorsaSubs.f it looks a simple 2D rectangular integration is done, but in the end, the result is divided by the major radius: ans = ans / r0. I implemented the same integration in Python using wdot_tot and I get values completely different from the out15 file.

Hi Severin,
As long as the solution is well converged, EdotJ and the two wdot's should give you approximately the same numbers for the global power, although the local profile of EdotJ can oscillate wildly in certain scenarios. wdot and wdot_ql are really the same thing (just summed over in different orders, basically). I have renormalized the field with wdot in the past in mode conversion scenarios (not an easy scenario), but the results weren't that much different from the original normalization. So if you are seeing very different numbers for those global powers, you should first do a convergence test. (Do you get the same results when you increase the resolution? What about the wavenumber spectrum?)
Regarding density fluctuations, you should not use cold plasma for analysis in fusion grade experiments. ntilde is proportional to what corresponds to the Stix 'P' component of the dielectric tensor, which is hot. If you use cold, you may well be off by an order of magnitude depending of the electron temperature and N_parallel.
Naoto

Just to make sure, I assume you had already checked that the spatial integral of Wdot you have in the Efield2D.vtk file does not match the AORSA2D out15 file output for the total power?
It's been too long since I used AORSA, but it's worth checking this spatial integral above assuming units are W/m^3, ergs/cm^3, W/m^2, ergs/cm^2. I suspect Fred Jaeger went with SI units; he usually does. I honestly do not remember if it is W/m^2 or W/m^3; it could be either depending on Fred's choice of convention.
Naoto: for the helicon problem, we had to use artificial damping terms for numerical convergence. Fred said this was rarely necessary for ICRF stuff and he said for reasons I do not understand, the artificial damping only appeared in E dot J and does not appear in Wdot. Therefore, Wdot had more accurate power absorption for the helicon problem.  I do agree with you about the the difference between cold and hot plasma. It is certainly not obvious to me why they should be close.

Ah.., I see. From my vague memory, wasn't there an attempt to include the artificial damping in wdot? It was not a trivial fix, but I remember Fred presenting something somewhere (sorry, just can't remember the details).

I don't recall that detail myself on artificial damping in wdot. It has also been a long time for me too...

I don't think the difference between Wdot and the quasilinear Wdot is a convergence problem. My Fourier spectra look really good and also the density fluctuations aren't noisy at all. Nevertheless, I will do a quick convergence test.
Anticipating hot plasma effects, I have also tried using the hot plasma conductivity tensor I get from the forest.f routine in GENRAY to post-process my COMSOL results. This is based on the hot dielectric tensor from Stix. I am not entirely sure how this compares to the dielectric tensor in AORSA. The main distinction that I understand is that the forest routine uses geometrical optics to compute the tensor by solving the hot plasma dispersion relation.
The density fluctuations resulting from this conductivity tensor are comparable to the ones from the cold dielectric tensor. Notably, it seems that the contribution from the P term seems to be rather weak for helicon. I am now computing the electric fields from AORSA in lab-frame to see if the polarization between AORSA and COMSOL is vastly different and the cold plasma approximation underestimates the e_z (Stix frame) component.
Does anybody else have a comment on the wdot_tot integration in AORSA. The values I get from out15 do not match what I get when I manually and analogously integrate wdot_tot in python.

On the integration, what volume element are you using in your own calculation? The aorsa integration is likely including the appropriate volume element / jacobian for its coordinate system (which I don't recall exactly what that was offhand). The 1/r may be that volume element?

Regarding your "hot" dielectric post processing of the comsol results, you're using a single k (from the dispersion relation), so you might want to plot the dispersion relation determined kper v kpar over the aorsa spectrum of the same to see how well they match. I'd think for helicon the wave field structure isn't that different, just the damping, so you may way to compare both the real and imaginary versions of the plot I'd suggested.

Closing this because I am no longer working with AORSA.


task: default proxy logic, every request proxied to 'httpbin.org'

Split into two binaries, roxy-cli as the controller and roxy-proxy as the background worker. Communication between these two aren't set up yet.
Implemented basic reverse proxy which is terribly inefficient, but proxies requests to httpbin.org for the moment

@muratozkan - looking good! I like the idea with the two binaries, at the same time there is also something nice about distributing 1 that can be started in either proxy or cli controller mode. it simplifies installation for the end user, and then all of the commands are roxy <something>
roxy start -p 8080
roxy add route /some/path
roxy add certificate cert.pem
what do you think?

@syllabix sure, we can go with a single binary, that could be started in either binary or proxy mode.
Let me write down the architecture in my mind:
There would be two modes (can be in same/different binaries), proxy and cli. Cli runs a command/query, sends it over to the proxy, receives reply, renders it on the stdout/stderr and immediately terminates. Proxy mode would be started with cli, and on startup immediately binds to a control socket, listens to the commands from cli. Then it can start up a server on the specified port that actually does the routing.
In the example:

roxy start -p 8080 would start another process roxy --proxy and sends command Start(port=8080), then immediately exit.
roxy 8080 add route /some/path https://httpbin.org/anywhere  would try to send command AddRoute(listener=...,matchPath=...,destUri=...) to the command socket. The proxy mode will pick it up, set up route and give back Ok()`


JSON not working?

I set up a jsFiddle that shows the problem I'm running into. I am suspecting there is a problem with my setup. The jstree files I am using are on the1.0 branch of Github, so they should be working.
Check it out at http://jsfiddle.net/panozzaj/Q7EfG/3/
I am seeing nothing rendering, when I expect to see a tree with one node. Thanks for your help!

Try using http://static.jstree.com/v.1.0pre/jquery.jstree.js, seems to work for me. Not sure about using the scripts directly from github.

A little better, but it does not find the background image for the JsTree Nodes http://jsfiddle.net/Q7EfG/20/

OK there were a few issues in there. Firstly you were still using the JS components from github; use the production code instead. Secondly, you were trying to configure the JSON plugin with "json" instead of "json_data". Also, it's better to specify the CSS in the head. In JSFiddle you do this through the 'Add Resources' section on the left sidebar.
I've rewritten your fiddle so it works:
http://jsfiddle.net/larister/RPuaC/
Any questions give me a shout.


2024-06-26 - Project Kick-off

Date
2024-06-26 - 14:30 UTC - See the time in your timezone https://everytimezone.com
Purpose
The primary goal of this kick-off meeting is to align all interested parties on the purpose and objectives of the Green Software Playbook project, identify key areas of focus, and discuss milestones.
Agenda

[x] Welcome
[x] Introduction
[x] Current Status
[x] Future Plans
[x] Contributing
[x] Q&A

Assets

https://docs.google.com/presentation/d/1JfITSkIt_imtq-neU_7GHfc6kmggo5_p4SZEwpcLSxw/edit#slide=id.g2c6268f2d49_2_43

https://github.com/Green-Software-Foundation/green-software-playbooks/discussions/11


Golang Tracing Documentation

We need to create initial documentation for tracing. It should include -

A quick start guide that includes information such as:

Initialize/configure the tracer.
Create a span, add attributes/events, finish the span.
Propagate the trace context to another process over HTTP.
Propagate the trace context inside the process.

API documentation, or a link to the API documentation if it exists somewhere else.

Done for now


Update README.md

Its late and sorry again @giacomo-b

No worries at all, thank you for taking the time to go through it! Just submit a new PR whenever you are done

Thanks for being cool about it! I might add some algorithms to the actual code today.

Awesome! Since I still have to create the CONTRIBUTE document, let me just specify a few quick points before you submit PRs:

Make sure you run clang-format before committing, otherwise the style test will fail
The library (at least at this point) is header-only and uses templates that take the problem dimensions
The EKF interface is similar to what I would like to have, the LQR API unfortunately is very poor and I need to fix it.


fix(remote): handle SSLError on file upload

Tracking issue

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Setup process
Screenshots
Check all the applicable boxes

[ ] I updated the documentation accordingly.
[ ] All new and existing tests passed.
[ ] All commits are signed-off.

Related PRs

Docs link

@pingsutw A program of mine used to crash because of an out-of-memory issue, and as I was debugging it this morning, I narrowed down the issue to the same piece of code this PR is targeting. I used flytekit 1.12.3 on Python 3.9, and I was trying to upload many files, each >2Gb. Given the local_file.read() call, all of my files were loaded into RAM and that was the root cause.
A fix which worked for me was to use streaming uploads. I tested the same fix with the latest flytekit and that caused the SSL OverflowError to disappear as well.
        with open(str(to_upload), "+rb") as local_file:
            # Move the file-like obj to the end of the file to get its size
            local_file.seek(0, os.SEEK_END)
            content_length = local_file.tell()
            # Move the file-like obj back to the beginning of the file to make it readable
            local_file.seek(0)
            headers = {"Content-Length": str(content_length), "Content-MD5": encoded_md5}
            headers.update(extra_headers)
            rsp = requests.put(
                upload_location.signed_url,
                data=local_file,
                headers=headers,
                verify=False
                if self._config.platform.insecure_skip_verify is True
                else self._config.platform.ca_cert_file_path,
            )

@redartera I'm not sure how to do a streaming upload. Do you want to hop on a call? I was trying to reproduce the error, but I can upload 10G file without any issue.

@redartera I'm not sure how to do a streaming upload. Do you want to hop on a call? I was trying to reproduce the error, but I can upload 10G file without any issue.

@pingsutw  I uploaded a video here where I demo two runs - one which fails and one which succeeds after having applied the streaming upload fix.

First run: I try to upload a 10Gb file using upload_file, but you can see very quickly how memory usage goes up before I get an OverflowError exception. We can imagine if the file was 20Gb, I'd be out of memory.

Second run: I try to upload a 4Gb file after having applied the streaming upload fix, and everything works out nicely with very little memory overhead. I couldn't upload 10Gb because AWS S3 pre-signed URL uploads only allow files of up to 5gb (see screenshot below)

fixed by https://github.com/flyteorg/flytekit/pull/2641


JSON / JSONB data types for postgres

Any ideas on how to provide this or if it's possible already?

I had thought about implementing something like ecto's embeds_one and embeds_many
https://robots.thoughtbot.com/embedding-elixir-structs-in-ecto-models

It might make sense to first implement jsonb field types for models?

would embeds_one and embeds_many work for other databases besides postgres?

I don't believe it would work with MySQL


Correlate logs with traces

Description:
We need to automatically add traceId and spanId into logs when tracing is enabled and if the current request is sampled.
We have already migrated tracing to Open Telemetry standard so that we can obtain traceId and spanId from observability runtime.

Currently blocked since https://github.com/ballerina-platform/ballerina-lang/blob/master/observelib/observe/src/main/ballerina/natives.bal#L78 needs to be isolated.

https://github.com/ballerina-platform/ballerina-lang/pull/31034

Currently blocked since https://github.com/ballerina-platform/ballerina-lang/blob/master/observelib/observe/src/main/ballerina/natives.bal#L78 needs to be isolated. @tharinduwijewardane

https://github.com/ballerina-platform/ballerina-lang/pull/31034

Blocked due to https://github.com/ballerina-platform/ballerina-lang/issues/31089

Blocked due to ballerina-platform/ballerina-lang#31089

Fix merged https://github.com/ballerina-platform/ballerina-lang/pull/31104

Blocked due to https://github.com/ballerina-platform/ballerina-lang/issues/31130

Blocked due to ballerina-platform/ballerina-lang#31130

Fix https://github.com/ballerina-platform/ballerina-lang/pull/31129

Reopening since the feature was reverted from SL Beta2. Will be shipped in SL Beta 3.


Runtime-controlled sharing of attribute values

This is a small modification of #94 which has the same API but a more complicated operation, i.e. for shared_value to be efficient (a reference count bump), ParsingOptions::shared_text must be set, otherwise this is just copying the string as with Cow.
To be honest, the above is complicated will only be noticed due to performance regressions. An alternative would be fn shared_value(&self) -> Option<&Arc<str>> which will not copy as a fallback, but that still leaves the need to set shared_text and .unwrap() the Option.
So maybe this is not worth avoiding the small Arc-related cost. I am not really happy either way.

Closing this as the API is too complicated.


Remove priority from policy rules

Describe your changes :

See #4199 for more details. Removing priority from the rule. With new policy implementations, prioritization will be done based on type of rule. Deny rules take higher precedence, followed by allowing rules.

Type of change :

[x] New feature

Checklist:

[x] I have read the CONTRIBUTING document.
[x] I have commented on my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] All new and existing tests passed.

@ShaileshParmar11 UI tests are unrelated to this patch. Merging this patch.


Update MATLAB.gitignore

added entry for ctf artifacts
Reasons for making this change:
missing generated artifact type

Sounds good.  Updated commit.


Day component classnames disappear after toggling calendar

react-dates version
<EMAIL_ADDRESS>Describe the bug
The CalendarDay components initially have the  CalendarDay__blocked_out_of_range and CalendarDay__selected classnames set. After changing the selected value to a value a couple of months in the future and toggling the calendar component the classnames disappear.
Source code (including props configuration)
https://github.com/simkall/react-dates-testing-
Steps to reproduce the behaviour:

Click "toggle calendar"
The calendar opens with the current date and days outside of the range highlighted.
Selected a date a  couple of months in the future.
Click "toggle calendar"
Click "toggle calendar" again
Unavailable dates are no longer highlighted, and clicking dates no longer updates the selected value.

The highlighting reappears if you select the current date and toggle the calendar.

I encountered this issue as well, quite a big deal as I may not be able to use the library because of it. How has this been open for over a year without any comments from the development team?

@amcurative because the development team is one unpaid person. If you're interested in submitting a PR to fix it, that would be appropriate if it's indeed that big a deal to you.

@ljharb Oh, my apologies then. I wasn't aware that was the case and assumed some team at airbnb was actively developing this. I will look into this issue to file a PR if I get some time.

The project effectively hasn't been maintained by airbnb employees in well over 3 years; that's why it's no longer an airbnb project.


Desaster recovery

Context
Given a replicated Eventuate application with locations A, B and C
 A --- C
  \   /
    B

where B loses events, either completely or for the last n hours, after having installed a datastore backup, for example. In this case, replication progress recorded at A and C is inconsistent with the state of the local event log at B. In the following, we assume that B has lost all events. Before,

30 events have been written to B's event log, including events that have been replicated from A and C.
A has replicated events from B up to position 25 in B's event log. A has stored that replication progress locally.
C has replicated events from B up to position 17 in B's event log. C has stored that replication progress locally.

Desaster recovery limitations
Events that have never been replicated from B to A and C cannot be recovered, of course. These are either events that didn't pass replication filters or events for which no replication attempt has been made (as event replication is running asynchronously).
Desaster recovery goal
Reconstruct an event log at B from event logs at A and C. The total order of events in the reconstructed event log must be consistent with the happened-before relation of events. Replication progress at A and C must then be updated accordingly.

Some constraints discussed in #66 may also be applicable here.

Solution outline
To reconstruct an event log at B from local event logs at A and C, as mentioned in the initial description, we can make the following assumptions:

The total order of events in the local event logs at A and C is consistent with the happened-before relation of events. This relation defines a partial order.
Given that partial order, each event log can be represented by one or more directed acyclic graphs (DAGs) where nodes represent events and edges represent happened-before relations.
In other words, a local event log is one of several possible topological sorting results of these DAGs.

From my current understanding, a reconstruction of event log B would mean a merge of the DAGs from locations A and C including a removal of duplicate nodes. Given the happened-before relation of events, the merge result must also be one or more DAGs which can be linearized to an event log by topological sorting.
tl;dr How can I merge n DAGs (where different DAGs may contain nodes of the same value) so that the result are m DAGs with unique node values overall (where m <= n) under a given partial ordering of nodes?

If location B can be partially recovered from a datastore backup, only parts of the local event logs at A and C must be merged to complete the recovery at B. For example, a datastore backup at B recovers

a local event log with 20 events
a replication progress indicating that B has read up to

position 10 in A's event log and
position 12 in C's event log

This means that an event log merge procedure only needs to consider events starting at

position 11 in A's event log and
position 13 in C's event log

The merge result can be appended to the local event log that has been recovered with the datastore backup.

After log reconstruction at B, replication progress at A and C must be updated i.e. point to the last replicated B-event. For example, if the last replicated B-event at A is b7 and b7 is at position 15 in B's event log then the replication progress (stored at A) must be updated to that position value.

Complexity

The worst-case time complexity of generating DAGs from n events in a local event log O(n2). I need to do further research on average-case time complexity, especially if the input order is a linear extension of the result.

Time complexity of transitive reduction is O(nk) for n events with k > 2.

When measured both in terms of the number n of vertices and the number m of edges in a directed acyclic graph, transitive reductions can also be found in time O(nm), a bound that may be faster than the matrix multiplication methods for sparse graphs.

In most applications, the happened-before relationship between events usually form a dense graph.

Is there an efficient algorithm to create a transitive reduction from the linear extension of a partial order?

Solution option 1 is generic and can be applied to both, cyclic and acyclic, replication network topologies. However, given the complexity, it is clear that it doesn't scale to a large number of events. Assuming frequent datastore backups and a moderate write load, the size of DAGs to be merged can be kept reasonably small, making this solution applicable.
Nevertheless, it should be investigated if the complexity can be further reduced by considering additional event metadata during DAG construction and reduction. Using explicit causality (pdf) might be an interesting option, not only in this context but also for write scalability and replication scalability.

Solution option 2
The complexity of solution option 1 can be reduced if the replication network does not contain cycles:
 A     C
  \   /
    B
    |
    D
      \
       E

Assuming again that B looses all events, it can be recovered from a single local event log located at A, C or D instead of merging logs from multiple locations. This reduces the time complexity of disaster recovery to O(n) where n is the number of events in the selected event log.
...

Closed with https://github.com/RBMHTechnology/eventuate/commit/72704d36f30469cab7364f6868835b36bc05b39d.