Not Pointcloud output on Nvidia Jetson NX with Ubuntu 18.04 LTS

Required Info -
Camera Model | D455
Firmware Version | <IP_ADDRESS>
Operating System & Version | Ubuntu 18.04
Kernel Version (Linux Only) | 4.9.253-tegra
Platform | Nvidia Jetson NX
SDK Version | V2.50
Language | C
Segment | others
Hi,
I am trying to use the camera (D455) on Jetson NX using ROS1. But when I tried to roslaunch demo_pointcloud.launch, it just output pointcloud for 1s or few frame. Then it stoped working, no pointcloud output.
suyutx2@suyutx2-desktop:~$ roslaunch realsense2_camera demo_pointcloud.launch
... logging to /home/suyutx2/.ros/log/a6955794-7439-11ec-9ce3-0c5415f05bc2/roslaunch-suyutx2-desktop-8334.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.
started roslaunch server http://suyutx2-desktop:41413/
SUMMARY
========
PARAMETERS

/camera/realsense2_camera/accel_fps: 0

/camera/realsense2_camera/accel_frame_id: camera_accel_frame

/camera/realsense2_camera/accel_optical_frame_id: camera_accel_opti...

/camera/realsense2_camera/align_depth: False

/camera/realsense2_camera/aligned_depth_to_color_frame_id: camera_aligned_de...

/camera/realsense2_camera/aligned_depth_to_fisheye1_frame_id: camera_aligned_de...

/camera/realsense2_camera/aligned_depth_to_fisheye2_frame_id: camera_aligned_de...

/camera/realsense2_camera/aligned_depth_to_fisheye_frame_id: camera_aligned_de...

/camera/realsense2_camera/aligned_depth_to_infra1_frame_id: camera_aligned_de...

/camera/realsense2_camera/aligned_depth_to_infra2_frame_id: camera_aligned_de...

/camera/realsense2_camera/allow_no_texture_points: False

/camera/realsense2_camera/base_frame_id: camera_link

/camera/realsense2_camera/calib_odom_file:

/camera/realsense2_camera/clip_distance: -1.0

/camera/realsense2_camera/color_fps: 30

/camera/realsense2_camera/color_frame_id: camera_color_frame

/camera/realsense2_camera/color_height: 480

/camera/realsense2_camera/color_optical_frame_id: camera_color_opti...

/camera/realsense2_camera/color_width: 640

/camera/realsense2_camera/confidence_fps: 30

/camera/realsense2_camera/confidence_height: 480

/camera/realsense2_camera/confidence_width: 640

/camera/realsense2_camera/depth_fps: 30

/camera/realsense2_camera/depth_frame_id: camera_depth_frame

/camera/realsense2_camera/depth_height: 480

/camera/realsense2_camera/depth_optical_frame_id: camera_depth_opti...

/camera/realsense2_camera/depth_width: 640

/camera/realsense2_camera/device_type:

/camera/realsense2_camera/enable_accel: False

/camera/realsense2_camera/enable_color: False

/camera/realsense2_camera/enable_confidence: True

/camera/realsense2_camera/enable_depth: True

/camera/realsense2_camera/enable_fisheye1: False

/camera/realsense2_camera/enable_fisheye2: False

/camera/realsense2_camera/enable_fisheye: False

/camera/realsense2_camera/enable_gyro: False

/camera/realsense2_camera/enable_infra1: False

/camera/realsense2_camera/enable_infra2: False

/camera/realsense2_camera/enable_infra: False

/camera/realsense2_camera/enable_pointcloud: True

/camera/realsense2_camera/enable_pose: False

/camera/realsense2_camera/enable_sync: True

/camera/realsense2_camera/filters:

/camera/realsense2_camera/fisheye1_frame_id: camera_fisheye1_f...

/camera/realsense2_camera/fisheye1_optical_frame_id: camera_fisheye1_o...

/camera/realsense2_camera/fisheye2_frame_id: camera_fisheye2_f...

/camera/realsense2_camera/fisheye2_optical_frame_id: camera_fisheye2_o...

/camera/realsense2_camera/fisheye_fps: 30

/camera/realsense2_camera/fisheye_frame_id: camera_fisheye_frame

/camera/realsense2_camera/fisheye_height: 0

/camera/realsense2_camera/fisheye_optical_frame_id: camera_fisheye_op...

/camera/realsense2_camera/fisheye_width: 0

/camera/realsense2_camera/gyro_fps: 0

/camera/realsense2_camera/gyro_frame_id: camera_gyro_frame

/camera/realsense2_camera/gyro_optical_frame_id: camera_gyro_optic...

/camera/realsense2_camera/imu_optical_frame_id: camera_imu_optica...

/camera/realsense2_camera/infra1_frame_id: camera_infra1_frame

/camera/realsense2_camera/infra1_optical_frame_id: camera_infra1_opt...

/camera/realsense2_camera/infra2_frame_id: camera_infra2_frame

/camera/realsense2_camera/infra2_optical_frame_id: camera_infra2_opt...

/camera/realsense2_camera/infra_fps: 30

/camera/realsense2_camera/infra_height: 480

/camera/realsense2_camera/infra_rgb: False

/camera/realsense2_camera/infra_width: 640

/camera/realsense2_camera/initial_reset: False

/camera/realsense2_camera/json_file_path:

/camera/realsense2_camera/linear_accel_cov: 0.01

/camera/realsense2_camera/odom_frame_id: camera_odom_frame

/camera/realsense2_camera/ordered_pc: False

/camera/realsense2_camera/pointcloud_texture_index: 0

/camera/realsense2_camera/pointcloud_texture_stream: RS2_STREAM_COLOR

/camera/realsense2_camera/pose_frame_id: camera_pose_frame

/camera/realsense2_camera/pose_optical_frame_id: camera_pose_optic...

/camera/realsense2_camera/publish_odom_tf: True

/camera/realsense2_camera/publish_tf: True

/camera/realsense2_camera/reconnect_timeout: 6.0

/camera/realsense2_camera/rosbag_filename:

/camera/realsense2_camera/serial_no:

/camera/realsense2_camera/stereo_module/exposure/1: 7500

/camera/realsense2_camera/stereo_module/exposure/2: 1

/camera/realsense2_camera/stereo_module/gain/1: 16

/camera/realsense2_camera/stereo_module/gain/2: 16

/camera/realsense2_camera/tf_publish_rate: 0.0

/camera/realsense2_camera/topic_odom_in: camera/odom_in

/camera/realsense2_camera/unite_imu_method: none

/camera/realsense2_camera/usb_port_id:

/camera/realsense2_camera/wait_for_device_timeout: -1.0

/rosdistro: melodic

/rosversion: 1.14.12

NODES
/camera/
realsense2_camera (nodelet/nodelet)

realsense2_camera_manager (nodelet/nodelet)

rviz (rviz/rviz)

auto-starting new master
process[master]: started with pid [8344]
ROS_MASTER_URI=http://localhost:11311
setting /run_id to a6955794-7439-11ec-9ce3-0c5415f05bc2
process[rosout-1]: started with pid [8357]
started core service [/rosout]
process[camera/realsense2_camera_manager-2]: started with pid [8360]
process[camera/realsense2_camera-3]: started with pid [8366]
process[camera/rviz-4]: started with pid [8382]
[ INFO] [1642055153.071218498]: Initializing nodelet with 4 worker threads.
[ INFO] [1642055154.447122965]: RealSense ROS v2.3.2
[ INFO] [1642055154.447281097]: Built with LibRealSense v2.50.0
[ INFO] [1642055154.447474458]: Running with LibRealSense v2.50.0
[ INFO] [1642055154.586670361]:
[ INFO] [1642055154.706211339]: Device with serial number<PHONE_NUMBER>39 was found.
[ INFO] [1642055154.706369726]: Device with physical ID /sys/devices/3610000.xhci/usb2/2-3/2-3.2/2-3.2:1.0/video4linux/video0 was found.
[ INFO] [1642055154.706509428]: Device with name Intel RealSense D455 was found.
[ INFO] [1642055154.707840460]: Device with port number 2-3.2 was found.
[ INFO] [1642055154.708289706]: Device USB type: 3.2
[ INFO] [1642055154.723552523]: getParameters...
[ INFO] [1642055155.074763282]: setupDevice...
[ INFO] [1642055155.074942981]: JSON file is not provided
[ INFO] [1642055155.075085466]: ROS Node Namespace: camera
[ INFO] [1642055155.075150870]: Device Name: Intel RealSense D455
[ INFO] [1642055155.075231280]: Device Serial No:<PHONE_NUMBER>39
[ INFO] [1642055155.075300523]: Device physical port: /sys/devices/3610000.xhci/usb2/2-3/2-3.2/2-3.2:1.0/video4linux/video0
[ INFO] [1642055155.075369894]: Device FW version: <IP_ADDRESS>
[ INFO] [1642055155.075443328]: Device Product ID: 0x0B5C
[ INFO] [1642055155.075498748]: Enable PointCloud: On
[ INFO] [1642055155.075566007]: Align Depth: Off
[ INFO] [1642055155.075658225]: Sync Mode: On
[ INFO] [1642055155.075792999]: Device Sensors:
[ INFO] [1642055155.118559749]: Stereo Module was found.
[ INFO] [1642055155.136419987]: RGB Camera was found.
[ INFO] [1642055155.137389293]: Motion Module was found.
[ INFO] [1642055155.137614140]: (Confidence, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1642055155.137718613]: Add Filter: pointcloud
[ INFO] [1642055155.139599788]: num_filters: 1
[ INFO] [1642055155.139708132]: Setting Dynamic reconfig parameters.
[ WARN] [1642055155.394468251]: Param '/camera/rgb_camera/power_line_frequency' has value 3 that is not in the enum { {50Hz: 1} {60Hz: 2} {Disabled: 0} }. Removing this parameter from dynamic reconfigure options.
[ INFO] [1642055155.451729884]: Done Setting Dynamic reconfig parameters.
[ INFO] [1642055155.453429568]: depth stream is enabled - width: 640, height: 480, fps: 30, Format: Z16
[ INFO] [1642055155.453555575]: setupPublishers...
[ INFO] [1642055155.466575654]: Expected frequency for depth = 30.00000
[ INFO] [1642055155.654796574]: setupStreams...
[ INFO] [1642055155.816190003]: SELECTED BASE:Depth, 0
[ INFO] [1642055155.854901207]: RealSense Node Is Up!
13/01 14:25:55,917 WARNING [546836558208] (ds5-timestamp.cpp:69) UVC metadata payloads not available. Please refer to the installation chapter for details.
[ WARN] [1642055155.923419782]: Frame metadata isn't available! (frame_timestamp_domain = RS2_TIMESTAMP_DOMAIN_SYSTEM_TIME)
[ INFO] [1642055156.187827295]: publishing unordered pointcloud.
[ WARN] [1642055156.225854017]: No stream match for pointcloud chosen texture Process - Color

Hi @anglee6  Problems with the pointcloud filter on Jetson boards specifically (including missing color) is a known issue that has had an official Intel bug report filed for it in connection to the case at https://github.com/IntelRealSense/realsense-ros/issues/1967
The only workarounds I am aware of that have been successful are to use an earlier version of the librealsense SDK (2.43.0) or to obtain the RGBD pointcloud by another method as described in https://github.com/IntelRealSense/realsense-ros/issues/2201#issuecomment-1003006360
You can confirm whether you are experiencing the known problems with the pointcloud filter on Jetson by generating a pointcloud using the rs_camera.launch roslaunch instruction below and then opening RViz with rosrun rviz rviz
roslaunch realsense2_camera rs_camera.launch filters:=pointcloud

Hi @anglee6  Do you require further assistance with this case, please?  Thanks!

Case closed due to no further comments received.


Update .travis.yml for CI

Pull Request Test Coverage Report for Build 1203

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 56.864%

Totals

Change from base Build 1201:
0.0%

Covered Lines:
1338

Relevant Lines:
2353

💛  - Coveralls


feat: add erc721 mint benchmarks

add erc721 mint benchmarks for Madara
Pull Request type

Please add the labels corresponding to the type of changes your PR introduces:

Feature

What is the current behavior?
Currently, we only have benchmarks for erc20, and benchmarks for ERC721 mint aren't present.
Resolves: #502  #692
What is the new behavior?
benchmarks for ERC721 have been added.
Does this introduce a breaking change?
No
Other Information

The code has been refactored in a way, that it requires no changes on the GitHub workflow side of the thing, as it still makes available the "test:ci" npm script, and outputs the benchmark to the same "benchmarking/report/metrics.json" file.

The same file metrics.json will now have data to render benchmarks for both ERC20 transfers and ERC721 mints.

lgtm 🚀


grpc support

I am now testing communication between µC and using via simple protobuf messages and for now I can see that it would be nice to have it defined in grpc style. If it will work fine, I will wrap it in some simple library.
The question is if it should be part of nanopb, or better make separate project which relays on nanopb?

I think it would make sense to have grpc support in nanopb. I assume it can relatively easily made so that it is an optional feature (e.g. in a separate pb_grpc.c file or similar).

Here you can see what direction I took in implementing grpc. @PetteriAimonen now I am going to start to work with generator. In C++ example of grpc generator is being split into to separate plugins, but in python example there is only one plugin. Would be enough if I would add grpc functionality into current generator (and wrap code in #ifdef PB_ENABLE_GRPC ... #endif), or should I write separate one?

Seems promising, and reasonable amount of code so far that it could eventually be part of nanopb also.  About the generator, plugin mechanisms may be more trouble than they are worth. But the code could probably be in a separate file that would get imported, if that makes it easier to organize (nanopb_generator.py is getting quite long already).

How far did this get?  I have a need for this and I would be happy to help push things forward.

@bl0rq It's now here https://github.com/d21d3q/nanogrpc but I don't know actual status.

There are currently no plans to implement grpc support in nanopb main code, any support would be a separate project.


Which loss function works in multi-label classification task?

I need to train a multi-label classifier for text topic classification task. Having searched around the internet, I follow the suggestion to use sigmoid + binary_crossentropy. But I can't get good results (i.e. subset accuracy) on the validation set although the loss is very small.  After reading the source codes in Keras, I find out that the binary_crossentropy loss is implemented like this,
def binary_crossentropy(y_true, y_pred):
    return K.mean(K.binary_crossentropy(y_true, y_pred), axis=-1)

My doubt is whether it makes sense to use the average in the case of multi-label classification task. Suppose that the dimension of label set is 30, and each training sample has only two or three of the labels. Since most of the labels are zeros in the most of the samples, I guess this loss will encourage the classifier to predict a tiny probability in each output dimension.
Following the idea here, https://github.com/keras-team/keras/issues/2826,  I also give a try to categorial_crossentropy but still have no such luck.
Any tips on choosing the loss function for multi-label classification task is beyond welcome. Thanks in advance.

The standard way to train a multilabel classifier is with sigmoid + binary_crossentropy,
but you can train a multilabel classifier with tanh + hinge, just the targets should be in {-1,1}. I don't think your issue has to do with the loss and the output activation, I think is more related with the complexity of your model. Also I'm curious: how you're evaluating your model?

@ismaeIfm
Thanks for you answering.
Maybe I did not make my question exactly. The model I use is a BLSTM model with attention mechanism. I used it for my text topic multi-class classification task with categorical_crossentropy and it proves to be good. So, when I encounter text topic multi-label classification task, I just switched from softmax+ctg_ent to sigmoid+binary_ent. But the results are not that good. So I am wondering if there is something wrong with my loss function.
The way I use to evaluating my model is subset accuracy, which is the same as accuracy in multi-class problem. We regard it to be right only when the output is the same as true label.

As far as I understand subset accuracy needs the explicit classes {0, 1}, but your model outputs probabilities, how did you choose the threshold to binarize the labels? Have you tried using LRAP to evaluate your model?

Hi, glad to see you have the same problem what I had.
For the multi-label classification, you can try tanh+hinge with {-1, 1} values in labels like (1, -1, -1, 1).
Or sigmoid + hamming loss with {0, 1} values in labels like (1, 0, 0, 1).
In my case,  sigmoid + focal loss with {0, 1} values in labels like (1, 0, 0, 1) worked well.
You can check this paper https://arxiv.org/abs/1708.02002.

I found an implementation of multi-label focal loss here:
https://github.com/Umi-you/FocalLoss

The multi-label focal loss equation doesn't seem to work.

@dberma15 focal loss doesn't work as in, it doesn't converge or implementation error?

For the multi-label classification, you can try tanh+hinge with {-1, 1} values in labels like (1, -1, -1, 1).
Or sigmoid + hamming loss with {0, 1} values in labels like (1, 0, 0, 1).
In my case, sigmoid + focal loss with {0, 1} values in labels like (1, 0, 0, 1) worked well.
You can check this paper https://arxiv.org/abs/1708.02002.

@daniel410 Hi, would you mind sharing how you implement your focal loss for the multi-label task, if it's not too much trouble?

    def compute_loss(self, y_true, y_pred):
        cross_entropy_loss = K.binary_crossentropy(y_true, y_pred, from_logits=False)
        p_t = ((y_true * y_pred) +
               ((1 - y_true) * (1 - y_pred)))
        modulating_factor = 1.0
        if self._gamma:
            modulating_factor = tf.pow(1.0 - p_t, self._gamma)
        alpha_weight_factor = 1.0
        if self._alpha is not None:
            alpha_weight_factor = (y_true * self._alpha +
                                   (1 - y_true) * (1 - self._alpha))
        focal_cross_entropy_loss = (modulating_factor * alpha_weight_factor *
                                    cross_entropy_loss)
        return K.mean(focal_cross_entropy_loss, axis=-1)

@Abhijit-2592 is it for multilabel classification?

    def compute_loss(self, y_true, y_pred):
        cross_entropy_loss = K.binary_crossentropy(y_true, y_pred, from_logits=False)
        p_t = ((y_true * y_pred) +
               ((1 - y_true) * (1 - y_pred)))
        modulating_factor = 1.0
        if self._gamma:
            modulating_factor = tf.pow(1.0 - p_t, self._gamma)
        alpha_weight_factor = 1.0
        if self._alpha is not None:
            alpha_weight_factor = (y_true * self._alpha +
                                   (1 - y_true) * (1 - self._alpha))
        focal_cross_entropy_loss = (modulating_factor * alpha_weight_factor *
                                    cross_entropy_loss)
        return K.mean(focal_cross_entropy_loss, axis=-1)

@Abhijit-2592 is it for multilabel classification?

it give me error
AttributeError: 'FocalLoss' object has no attribute 'get_shape'
i used it as follow
model.compile('adam',loss=FocalLoss,metrics=['accuracy'])

You can try my implementation and let me know if it works. https://github.com/sushanttripathy/Keras_loss_functions/blob/master/focal_loss.py

@sushanttripathy: I tried your code and it works but the output is a tensor focal_loss_tensor is of  2d array. Should I take a mean to arrive at the final loss?

@Vishnux0pa I am not sure if the auto-differentiation requires me to provide the loss per sample (instead of per batch). I looked at categorical_crossentropy, and it seemed like that's what it was doing.
I did not get convergence with the earlier version of the loss (the one that yielded a scalar). It does converge with this one though.


https all the links, plus updates

https where available (tested!).
also updated JTS URLs to where they go redirected/migrated to.
could not find a recent URL for http://www.vividsolutions.com/jts/discussion.htm#spatialDataModel (not in https://github.com/locationtech/jts/tree/gh-pages/), so I pointed it to IA's latest: https://web.archive.org/web/20160719195511/http://www.vividsolutions.com/jts/discussion.htm.

Coverage remained the same at 80.396% when pulling 82e451cf1c4e79941677a278b3f9e944f952c032 on kannes:kannes-updatedURLs into 4b73205fc5ae1f4f99d47d4fb8d4a0340be2855d on Toblerity:master.

Coverage remained the same at 80.396% when pulling 82e451cf1c4e79941677a278b3f9e944f952c032 on kannes:kannes-updatedURLs into 4b73205fc5ae1f4f99d47d4fb8d4a0340be2855d on Toblerity:master.


Added Is Leased

We had a requirement to set the Leased/Purchased field. I modified it to add this in, might be useful for others but I appreciate you may wish to keep the CSV structure as is.
Great app BTW!

@Craighopkins12 sorry for the delay here, and thank you so much for your contribution!
I'm going to be starting work on v6 here soon, with some exciting updates I think. I'll include your PR here as well. I'll be tweaking it slightly after merging I think, just to rearrange the order of the CSV files a bit, but this looks awesome!


Names for fan speed

Nice to have:
1: Lowest
2: Low
3: Medium
4: High
5: Max
Instead of numbers only

Hi @AlfAlex
For this issue, I just pushed node-red-contrib-melcloud 1.4.0
I would be happy if you can test this please :)

It works.
Just thinking if if would be better to show both values (name and integer) or just the name.

Hum … I agree with your suggestion, let’s do it

Le 12 janv. 2022 à 12:47, AlfAlex @.***> a écrit :
It works.
Just thinking if if would be better to show both values (name and integer) or just the name.
—
Reply to this email directly, view it on GitHub https://github.com/ysimonx/node-red-contrib-melcloud/issues/5#issuecomment-1010959126, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALB7C4RCE3SGFJ4O3GF2TLUVVS3HANCNFSM5LYPCE7Q.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.

@AlfAlex
Updated node-red-contrib-melcloud (1.4.1)
node-red-contrib-melcloud 1.4.1

For me that's fine!
Thanks

thank you for your help. I felt alone with this project ;-)

My pleasure, it did help me a lot because all other melclould  projects I could not use. At least without so less work.
And I did not have to deal (read about it) with the API for myself.


:robot: Triggering CI on branch 'release-next' after synching to openshift/master

/test 4.5-e2e

/lgtm


remove swift DirectResponse APIs

https://github.com/envoyproxy/envoy/pull/25582 made the test server code usable for TCP upstreams, so I think all we need to do is port the direct response tests to use real upstreams and we can close this off.

@alyssawilk I stumbled upon this workaround that mentioned this issue: https://github.com/envoyproxy/envoy/blob/cc4a75482810de4b84c301d13deb551bd3147339/mobile/library/common/http/header_utility.cc#L16
With the change in https://github.com/envoyproxy/envoy/pull/34490, this workaround would no longer work because the value would be direct_response{} instead of just direct_response, right?
I am not sure what this workaround is for and I don't need it, just happen to saw it.

fixed by https://github.com/envoyproxy/envoy/commit/d78e8138feb499147730a1c39da8e479a796a621 :-)


List Implementation

#22 Implement : List Interface - E remove(int index)

@plazzy99  send 2 different pr with the issue name mentioned in the issue.

@JyotiJauhari deleted file
added one pr
please check

@plazzy99 change file name to method name e remove (int item).java if possible.

@JyotiJauhari changes updated , thank you for so much cooperation.

@plazzy99 There are two more changes which I need you to do -

The code has a syntax error at line 13  Scanner sc=new Scanner(); in this code. There is no suitable constructor found for Scanner(no arguments).

Moreover add this java file in a folder with the folder name set to this method's name, and then finally add this folder with the java file in the List Interface Folder.  ( for reference see the project and the merged PR) #46


TON to Jetton swap not working

While following the code on the TON to Jetton swap guide using proxy TON, I keep getting the error.
Error occurred - Rate limit exceeded: 1 per 1 second
The code I am testing only has the wallet address changed and nothing else.

Same here.
When I use api token for ton center, there is a different error though:
const provider = new TonWeb.HttpProvider('https://toncenter.com/api/v2/jsonRPC', {apiKey: 'mycode'});
/Users/mexx/.yarn/berry/cache/tonweb-npm-0.0.63-a335dbafcf-10c0.zip/node_modules/tonweb/src/providers/HttpProviderUtils.js:51
            const err = new Error('http provider parse response error')
                        ^
Error: http provider parse response error
    at Function.parseResponse (/Users/mexx/.yarn/berry/cache/tonweb-npm-0.0.63-a335dbafcf-10c0.zip/node_modules/tonweb/src/providers/HttpProviderUtils.js:51:25)
    at HttpProvider.call2 (/Users/mexx/.yarn/berry/cache/tonweb-npm-0.0.63-a335dbafcf-10c0.zip/node_modules/tonweb/src/providers/index.js:154:34)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async K.getPoolAddress (/Users/mexx/Projects/dex-calc/.yarn/unplugged/@ston-fi-sdk-virtual-30e990c32b/node_modules/@ston-fi/sdk/src/contracts/dex/router/RouterRevisionV1.ts:73:20)
    at async wt.getPool (/Users/mexx/Projects/dex-calc/.yarn/unplugged/@ston-fi-sdk-virtual-30e990c32b/node_modules/@ston-fi/sdk/src/contracts/dex/router/Router.ts:171:25) {

TonWeb, by default, uses toncenter API with a rate limit of 1 request per second
You can obtain higher limits by:

requesting an API key from toncenter with a 10 request per minute limit
use open TON API instance, for example, from ORBS
found a paid instance of TON API with suitable rate limits for your case
selfhost an instance of TON API


Contact form doesn't send any emails

The contact does not send any emails.

Contact form now attempts to send via GMail (smtp.gmail.com, the server used in GC's config file). Will do further testing to verify send attempts are successful.

Contact form now sends email with appropriate configuration and SMTP mail service.


Builder for TableStorage Query

For now it is quite hard to create a well formed query for the table storage.
It'd be great to have a builder where

$filter is constructed from "abstract" items
$top
$select
Value parameters are escaped automatically to avoid injection vulnerability

The best would be if $filter and $select were aware of the structure of the payload, but I think that'd require some macro magic unless serde provides some "reflection" info.
(Similar to https://docs.microsoft.com/en-us/dotnet/api/microsoft.azure.cosmos.table.tablequery.generatefiltercondition?view=azure-dotnet#Microsoft_Azure_Cosmos_Table_TableQuery_GenerateFilterCondition_System_String_System_String_System_String_)

I agree, a builder would be nice, at least with the arguments you specified. As for the reflection, I too think it would require a (cumbersome) procedural macro.

triage - outdated for me


feat: skip work when gatsby version supports adapters

Summary
This makes this build plugin no-op if Gatsby version supports adapters as all the handling will be done by adapter. Work on gatsby adapters can be tracked in https://github.com/gatsbyjs/gatsby/pull/38232
Test plan
Deploy preview generated by https://github.com/netlify/netlify-plugin-gatsby/pull/655 (this is just a draft with changes from this PR + adjusting demo-v5 to use gatsby canary with adapters):

deploy logs: https://app.netlify.com/sites/netlify-plugin-gatsby-demo-v5/deploys/64ae83c83654280559ac2f5c:12:45:40 PM: Skipping @netlify/plugin-gatsby work, because used Gatsby version supports adapters.

deploy url: https://deploy-preview-655--netlify-plugin-gatsby-demo-v5.netlify.app/

And deploy previews on this PR where build plugin continue to do the work:
https://app.netlify.com/sites/netlify-plugin-gatsby-demo-v5/deploys/64ac064285ef5f0008a0bb7c / https://deploy-preview-639--netlify-plugin-gatsby-demo-v5.netlify.app/
Relevant links (GitHub issues, Notion docs, etc.) or a picture of cute animal
Standard checks:

[ ] Check the Deploy Preview's Demo site for your PR's functionality
[ ] Add docs when necessary

Adresses https://github.com/netlify/pod-ecosystem-frameworks/issues/543

🧪 Once merged, make sure to update the version if needed and that it was
published correctly.

Should we add tests for the skipping code? There's nothing that tests the logic for the adapter existing and us skipping the build process.

e2e and unit test cases were added


Ideas for use-cases

Filter streaming messages.
SuperDuperDB could be used in conjunction with Kafka.
There are two interesting scenarios:

Run similarity searching on the names of topics to merge them.
Run similarity searching on messages. Using a query as a reference (e.g cat o' nine tails ) and returning messages that semantically match this description. This way, you can filter out sensitive content in real-time.

Audio Pipelines
Many frameworks have image and text, but only a handful of them have Audio examples.
We should create a use-case that generates music, and then either add lyrics from a transcript or create a visualization.
Combine Text-Search with Vector-Search
It seems that combining the high-recall of text-search (e.g BM25, OpenSearch) with the high precision of vector-search (OpenAI, Sentence Transformers) yields better results than any of them. We should create a pipeline that combines traditional text-search with AI-based vector-search.
Integration with non-database sources
How can someone ingest data from data sources other than databases ? (e.g web apis or social media).
For example, we could download a github repository and use prompt engineering to explain the code. We should continuously listen for PR and update the response
https://github.com/linexjlin/GPTs/blob/main/prompts/Code Explainer.md
Object detection in Satellite images.
Extract objects from images, place metadata on a database, and run similarity queries over it.
https://github.com/satellite-image-deep-learning/techniques#4-object-detection
Other sources: https://docs.openvino.ai/2023.2/tutorials.html

RAG application based on Phi-2 and opensource
This is a very effective model that can be run on a CPU or the cheapest GPU.
https://www.reddit.com/r/LocalLLaMA/comments/18eusc8/superduperdb_how_to_use_it_to_talk_to_your/

Synthetic data
Demonstrate the capability of generating Synthetic Relational Databases that accurately reflect the relationships between many interrelated tables in database. This feature is particularly significant as relational databases play a vital role in enterprise data management.
https://moez-62905.medium.com/revolutionizing-data-privacy-generate-synthetic-databases-with-gretel-relational-3232e534dc58


Merge development into master branch to prepare for Bali release

What this PR does / why we need it:
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
Release note:

lgtm


Only add toc.footer if not already defined in package.json

If I have already defined toc.footer in my package.json, this will be ignored. This is the fix for it.

:+1:
@jonschlinkert ?

ooh, yea.. actually it's not here anymore, so we should close

closing per https://github.com/verbose/verb-generate-readme/pull/9#issuecomment-260458968.


Slow boot time on laptop with Optimus graphics

System Details

OS: Arch
HOST: Blade 14 - RZ09-0370 1.04
Kernel: 6.5.11-x64v2-xanmod1-1 (other kernels behave the same)
DE: Hyprland
CPU: AMD Ryzen 9 5900HX with Radeon Graphics (16) @ 4.890GHz
GPU: AMD ATI Radeon Vega Series / Radeon Vega Mobile Series
Thorium Version:  117.0.5938.157 (Official Build) stable, built on Linux (64-bit)

Problem
On a laptop with Nvidia Optimus, featuring an Nvidia GPU and an AMD CPU, when the dedicated GPU is disabled and it's running in integrated mode only, any Chromium-based browser takes a long time to open and randomly freezes.
Additional Notes
Logs recorded after boot (these are not related as they are the same in a normal session):
[37331:37331:1110/093955.948568:ERROR:policy_logger.cc(154)] :components/enterprise/browser/controller/chrome_browser_cloud_management_controller.cc(163) Cloud management controller initialization aborted as CBCM is not enabled.
[37331:37331:1110/093956.194246:ERROR:pref_model_associator.cc(150)] Failed to deserialize value of preference 'sync.demographics': Line: 1, column: 1, Unexpected token.

Logs when freeze happens:
[37331:37331:1110/093956.194246:ERROR:pref_model_associator.cc(150)] Failed to deserialize value of preference 'sync.demographics': Line: 1, column: 1, Unexpected token.
Warning: loader_scanned_icd_add: Could not get 'vkCreateInstance' via 'vk_icdGetInstanceProcAddr' for ICD libGLX_nvidia.so.0
Warning: vkCreateInstance: Found no drivers!
Warning: vkCreateInstance failed with VK_ERROR_INCOMPATIBLE_DRIVER
    at CheckVkSuccessImpl (../../third_party/dawn/src/dawn/native/vulkan/VulkanError.cpp:88)
    at CreateVkInstance (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:458)
    at Initialize (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:344)
    at Create (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:266)
    at operator() (../../third_party/dawn/src/dawn/native/vulkan/BackendVk.cpp:521)

../../third_party/libc++/src/include/string:914: assertion __s != nullptr failed: basic_string(const char*) detected nullptr
[37331:37331:1110/094157.404827:ERROR:gpu_process_host.cc(989)] GPU process exited unexpectedly: exit_code=134

Looks like an upstream issue

Closing while waiting for a new thorium update, I'll open this again if not fixed since this issue is no longer present upstream


Draft: Cucumber Expressions AST to Regex expansion

Moved from cucumber-rs/cucumber#154
Part of cucumber-rs/cucumber#124
Synopsis
This PR implements Cucumber expressions AST into Regex expansion.
Solution
This is done with IntoRegexCharIter trait to avoid unnecessary overhead.
Checklist

Created PR:

[x] In draft mode
[x] Name contains Draft:  prefix
[x] Name contains issue reference
[x] Has assignee

[x] Documentation is updated (if required)
[x] Tests are updated (if required)
[x] Changes conform code style
[x] CHANGELOG entry is added (if required)
[ ] FCM (final commit message) is posted

[ ] and approved

[ ] Review is completed and changes are approved
Before merge:

[ ] Milestone is set
[ ] PR's name and description are correct and up-to-date
[ ] Draft:  prefix is removed
[ ] All temporary labels are removed

derive_more crate is dependent on syn = 1. That why after running cargo +nightly update -Z minimal-versions syn gets downgraded to 1.0, where there is no Path::get_ident() method.

FCM
Cucumber Expressions AST to Regex expansion (#2)


cgroup: make cgroup-parent absolute path

avoid get cgroup path like /sys/fs/cgroup/memory/system.slice/...
since if cgroup path is relative path, controller path get from
/proc/self/cgroup.
Signed-off-by: Ace-Tang<EMAIL_ADDRESS>
Ⅰ. Describe what this PR did
Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)? (你真的觉得不需要加测试吗？)
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

LGTM


full test suite for output log expectations (buy, sell, rebuy, rebuild)

Alright, finally cleaned this up and consolidated it.
When running npm test or doing a commit (triggering the pre-commit hook), you'll get this (if all goes well):
> ./node_modules/eslint/bin/eslint.js . && echo '✅ No Lint' && ./node_modules/jest/bin/jest.js --verbose --coverage && echo '✅ Tests'

✅ No Lint
 PASS  test/request.sign.test.js
  ✓ sign a request (3 ms)

 PASS  test/touch.test.js
  ✓ touch an existing file (3 ms)
  ✓ touch to create a new file (2 ms)

 PASS  test/sleep.test.js
  ✓ sleep 0 (6 ms)
  ✓ sleep 200ms (202 ms)

 PASS  test/math.test.js
  ✓ add number and string: no rounding error (2 ms)
  ✓ subtract number and string: no rounding error
  ✓ multiply number and string: no rounding error (1 ms)
  ✓ divide number and string: no rounding error
  ✓ pow number and string: no rounding error

 PASS  test/index.test.js
  Engine
    ✓ Log Output: 00.boot (4 ms)
    ✓ Log Output: 01.buy (1 ms)
    ✓ Log Output: 02.sell
    ✓ Log Output: 03.rebuy (2 ms)
    ✓ Log Output: 04.rebuild (2 ms)
    ✓ Log Output: 05.rebuy.fill (2 ms)
    ✓ no error outputs

-----------------------------------------|---------|----------|---------|---------|----------------------
File                                     | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
-----------------------------------------|---------|----------|---------|---------|----------------------
All files                                |   86.58 |    64.07 |    86.3 |   87.92 |
 coinbase_position_builder_bot           |   84.62 |    54.05 |    87.5 |   87.67 |
  api.keys.js                            |     100 |       50 |     100 |     100 | 12-18
  config.js                              |   84.85 |       50 |     100 |    87.1 | 55-59
  index.js                               |   84.09 |    63.64 |      75 |    87.8 | 33-36,58,68-71
 coinbase_position_builder_bot/coinbase  |      76 |    44.44 |   82.35 |   76.61 |
  accounts.js                            |     100 |       50 |     100 |     100 | 9
  cb.request.js                          |   93.33 |    66.67 |      50 |   93.33 | 41
  delete.order.js                        |   85.71 |       50 |     100 |     100 | 7-13
  get.order.js                           |   59.38 |       30 |     100 |   59.38 | 14,17-20,35-41,44-49
  get.product.js                         |     100 |       50 |     100 |     100 | 11
  get.ticker.js                          |   71.43 |       50 |      50 |   71.43 | 9-13
  http.request.js                        |    69.7 |       50 |   83.33 |    69.7 | 19-20,23,28-35,43-44
  order.js                               |   76.92 |       50 |     100 |   76.92 | 12-13,21
  request.sign.js                        |     100 |      100 |     100 |     100 |
 coinbase_position_builder_bot/lib       |   92.31 |    77.19 |      90 |   92.78 |
  action.js                              |   84.21 |       65 |     100 |   86.49 | 35-36,45,49-52
  calculate.action.js                    |     100 |    94.44 |     100 |     100 | 19
  history.js                             |   80.95 |       60 |   66.67 |   80.95 | 16-19,24-25
  log.js                                 |      80 |       50 |      80 |      80 | 7
  log.output.js                          |     100 |     87.5 |     100 |     100 | 8,32
  log.save.js                            |     100 |      100 |     100 |     100 |
  math.js                                |     100 |      100 |     100 |     100 |
  memory.js                              |     100 |      100 |     100 |     100 |
  number.fix.js                          |     100 |       50 |     100 |     100 | 2
  process.order.js                       |     100 |    83.33 |     100 |     100 | 4
  rebuys.check.js                        |   93.44 |    78.57 |    87.5 |    93.1 | 44-48,84
  rebuys.post.js                         |   90.57 |    66.67 |     100 |    90.2 | 38,54-57,83-88
  sleep.js                               |     100 |      100 |     100 |     100 |
  touch.js                               |   92.31 |    66.67 |     100 |     100 | 9-10
 coinbase_position_builder_bot/test      |     100 |      100 |     100 |     100 |
  test.config.js                         |     100 |      100 |     100 |     100 |
 coinbase_position_builder_bot/test/lib  |   66.67 |       25 |      60 |   76.92 |
  get.id.js                              |    87.5 |       50 |     100 |     100 | 58
  get.log.js                             |     100 |      100 |     100 |     100 |
  log.js                                 |      50 |    16.67 |   33.33 |      60 | 32-34,37-39
  test.memory.js                         |     100 |      100 |     100 |     100 |
 coinbase_position_builder_bot/test/nock |   96.15 |       95 |     100 |   96.08 |
  delete.order.js                        |     100 |      100 |     100 |     100 |
  get.accounts.js                        |     100 |      100 |     100 |     100 |
  get.order.js                           |   84.62 |       75 |     100 |   84.62 | 13-14
  get.product.js                         |     100 |      100 |     100 |     100 |
  get.ticker.js                          |     100 |      100 |     100 |     100 |
  post.orders.js                         |     100 |      100 |     100 |     100 |
-----------------------------------------|---------|----------|---------|---------|----------------------
Test Suites: 5 passed, 5 total
Tests:       17 passed, 17 total
Snapshots:   0 total
Time:        4.275 s
Ran all test suites.
✅ Tests

and then you can view the coverage report with open coverage/lcov-report/index.html.
Maintaining the tests is pretty simple. You just edit the ./test/data/output* files to match what you get when you run a manual: npm run test:debug (which will output the log data for you to copy/paste if anything changed from the current expectation).
To add new expectation sets, you can configure the first section of ./test/index.test.js, which is the main test runner.

Bravo! Are all the missed lines things that are failsafes? I'll run it now and check it out. Is 100% coverage something worth trying to attain?

Looks like some files are missing:
 Engine › Log Output: 00.boot

    ENOENT: no such file or directory

      3 | module.exports = name =>
      4 |   fs
    > 5 |     .readFileSync(`${__dirname}/../data/output.${name}.log`)
        |      ^
      6 |     .toString()
      7 |     .split('\n');
      8 |

      at getLog (test/lib/get.log.js:5:6)
      at Object.<anonymous> (test/index.test.js:87:27)

whoops. .gitignore :)

Sweet! Got it. I see the coverage report is missing on things like error logging. Merging now and I'll play around with it.


Actually stand up a docs site

The raw docs under docs/ are passable to consume in github, but what's really needed is a docs site.
https://github.com/bep/docuapi looks like it'd be promising. The tutorials, in particular, have text that ought to appear adjacent to code, rather than above or below it.

If you want some CUE highlighting via Hugo, I'm using a JS package called shiki. The Makefile at the root of Cuetorials repo shows what I'm doing best.

oh, that will definitely be helpful, thank you!


Can someone send me a file to a certain boothmodel

I've tried over and over and it's just not working at this point I give so if someone could download this file I could pay you some money or something I just need this file 【完全live2dモデル】battle maid NVPU 20+exp（VtubeStudio対応） | 貓橋 Azusa #booth_pm https://nekohashiazusa.booth.pm/items/5256580

This only downloads items you have purchased, if you do not have it purchased we can not help you obtain the download.

as stated above this application only downloads paid content, it is not possible to download content you do not own.


Raise an error when attempting to sample a circuit with no measurements

Should apply in the SparseSimulator, DensitySimulator, WaveFunctionSimulator, and XmonSimulator.
Introduce a utility method has_measurements on Circuit to avoid too much code duplication.

This code is in run?

Could you point at it? When I run a circuit with no measurements I just get an empty trial result. For example, cirq.Simulator().run(cirq.Circuit()) produces no error.

Whah?  I was asking whether you want to put this in the "run*" methods?
Yeah current behavior is that no measurements produce an empty trial result.

Oh, I thought you were asking "is this code in run".
Yes, I think it should be limited to the run methods, because the simulate methods have useful outputs when given no measurements.

I can take this.

@Strilanc I'm curious, you listed these 4 Simulators specifically. What kind of simulators should not throw error here? Is there an underlying property here that we can use in a parent class itself?

I was just trying to list them all. I think this applies globally to any simulator, as long as we're talking about sampling the measurement results.


Confused enabled vs disabled

"Maintaining a whitelist of domains for which cleartext traffic should NOT be allowed is not recommended. You should disable cleartext traffic for all domains and only disable it for specific domains if your configuration requires it."
I guess the second sentence here should be "You should disable cleartext traffic for all domains and only enable it for specific domains if your configuration requires it"

Whitelist text mentioned before is still not updated.

Thank you for the report, and sorry for the delays. Text has been updated.


Add content to /about view

This view should contain some copyright info, change log, contact, webpage, link to documentation, etc.
It's important because it's written in Elm as out example how to integrate Elm within erlangpl. Being simple static view it's still value to have it written in Elm to show how any Elm view can be written.

Erlang Performance Lab v0.4.0
erlangpl is a tool for developers working with the Erlang VM (BEAM).
It helps to observe the system and analyze its performance.
github: https://github.com/erlanglab
www: http://www.erlang.pl
twitter: https://twitter.com/erlanglab
license: Apache 2.0

@Baransu feel free to extend the About file as you see fit. The above comment is an example.
Do you want to try to read the version number from localhost:8000 and this way show how Elm-based application can execute some simple request to the cowboy server?

Good idea. Something like this would be great.
{
  "version": "0.4"
}

Added new plugin returning version of the epl application (https://github.com/erlanglab/erlangpl/commit/94c5304ea014e041acd9e7339b4624eaf6359e7c)
$ curl localhost:8000/epl_version_EPL
{"version":"0.3.0"}

Awesome, I'll add it today!


feat: More filter in Problem Filter

Some more filter Ideas Problem can be filter in -

[ ] Based on Problem Index
[ ] Based on Contest
[ ] Based on Wrong Submission ( Problem that has a wrong submission and not solved yet )

/cib add more filters in problem filter


refactor: use sqlx for sql services

Simplify deps.

Thanks a lot for this PR. I will merge this once we are ready for 0.50 release.

@Xuanwo is there a plan for the next release?

The service failure is not related.


Does vts support using high precision area mosaic with low precision area terrain?

Hi, MelownTech team.
I'm interested in VTS backend, but when I set mine own vts backend, it occured some issues.
According to the sample of VTS backend documentation, it's ok to using high precision data to generate global mosaic data of Mars and using a bit low precision data to generate global terrain data. But when I use Level 20 precision data of area mosaic and Level 15(even lower) precision data of area terrain, I only can see the low level precision texture on the terrain. Here is the details pictures.

Hi @vaclavblazek . Could you give us a help?
We are try to load level 20 image on level 15 terrain.
But it seems not work. We can only see very blurry image on the terrain.
Is there any limitation on terrain / dem data match with high level image data.

Hi,

We are try to load level 20 image on level 15 terrain.

If you mean that boundlayer (imagery) is 20 LODd deep and terrain is only 15 LODs deep it should work. Is the bound layer really that deep? You can navigate (in the web browser) to the directory with boundlayer and see the actual data, if you switch on grid layer you'll see the tile information.
Also, how does the boundlayer.json of that particular boundlayer look like? Can you paste it here, pls? You can catch the URL to this file in your browser's dev tools network tab (don't forget to reload the page after opening network tab) and display its content in separate window.

@vaclavblazek  i wonder if this is a problem similar to this:
https://github.com/melowntech/vts-backend/issues/14

Hi @vaclavblazek ,
Here are json files below, maybe they can provide some useful infomation to you to solve this issue.
Configrations.zip
And this issue is similar to https://github.com/melowntech/vts-backend/issues/14, but I don't have any idea about how to use "mapproxy-setup-resource" because this cmd seems not be mentioned by documentationhttps://vts-geospatial.org/tutorials/landuse-wms-dem-cities.html, so can you provide some explains about this cmd pls? Does this cmd used to process the datasets in three folders which generated by "generatevrtwo"(I means  min, max ,cubicspline)?

In order to display lod20 bound layers, your surface also has to be processed to lod20. Not ideal but just quirk you need to work with.
Now that I learned how to use mapproxy-resource-setup, I use it all the time to add new data to maps. Just check out the documentation using the —help flag ( i think that’s the correct flag, sorry I’m not a Linux pro) I would reprocess your surface using the bottomlod flag in mapproxy-setup-resource(e.g. —bottomlod 20).
One caveat though, the downside to having a lod20 surface is that it might be very time consuming to create glues later on, so you should be aware of that and think carefully about how you will put high detailed surfaces together later on.
see how you go, and I’m sure @vaclavblazek  might have some thoughts on this too.
Cheers

Whoops, I meant mapproxy-resource -setup

Ah, sorry, my bad. It's vice-versa: coarser imagery can be draped on finer mesh, not finer imagery on coarser mesh. So, you need finer surface.
However, the source dataset (those min/max/cubucspline GDAL VRT datasets you mentioned) can be used for this task since the finer data are generated from original dataset resolution -- mapproxy is zooming the source data past its natural resolution.
So, you can setup new surface-dem resource in mapproxy, identical to the one you have but with the same LOD range as the boundlayer resource.
I.e. it's gonna be the same like the file testarea-dem.json from Configuration.zip you uploaded here but the LOD range will be 12-19. You need to change the resource ID as well.
Or, if you do not want to change the ID and you are feeling breave, you can update the existing resource, which is normally not recommended -- but you are not using the resource in any VTS storage so it's not gonna break anything.
Mapproxy keeps data 1) in memory and 2) in its on-disk storage. It will protest when some incompatible changes occur and will insist on using the old resource version. In the case of surface, any tile tree structure change or input dataset change is incompatible because it can cause havoc where used.
Recipe follows:

stop mapproxy
update the range in the existing resource in the testarea-dem.json file (to 12-19); keep tile range untouched
add "revision": 1 in the toplevel object in the resource; this causes URLs to have ?r=1 added  which clash with previously (possibly) cached data in your browser
and now the fun begins: remove the resource from mapproxy's store, i.e. remove the /var/vts/mapproxy/store/melown2015/testJapan/hengxuheshi-dem directory (I hope I composed the path right)
only now start mapproxy again

Mapproxy will treat the resource as if it has not seen it before and create the necesary data again.
Caveats: You can safely mess with mapproxy's storage only when the surface is not used in another storage. Also, if someone has already displayed the surface in their browser it hapilly download new data and mix it with the old ones. So, this is only when you have "unpublished" resource and you want to keep its name.

It works well! Thanks!

:+1:


Pod continues running after Workflow is stopped

Pre-requisites

[X] I have double-checked my configuration
[X] I can confirm the issues exists when I tested with :latest
[ ] I'd like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?
We have cluster-autoscaler installed on our Kubernetes cluster to automatically scale up nodes when none are available.
If I start a Workflow which runs a Pod, and then stop that workflow before the Pod starts running (so, while the Pod is in a Pending state, waiting to be scheduled onto a node), the Workflow stops fine but the Pod continues running until all the containers in it complete.
It takes a few minutes for the autoscaler to bring up a new node, so if someone triggers a workflow, then aborts it in the few minutes it takes to scale up the node and schedule the pod, then the pod may end up in a zombie state where it's running while the Workflow has already been stopped.
Version
3.4.3
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
Any workflow that triggers a Pod should work. I triggered a workflow `postman-test-5lrl7`

Logs from the workflow controller
time="2023-03-09T19:08:26.671Z" level=info msg="Processing workflow" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.674Z" level=info msg="Task-result reconciliation" namespace=argo numObjs=0 workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.674Z" level=info msg="node unchanged" namespace=argo nodeID=postman-test-5lrl7-2114579652 workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.674Z" level=info msg="Terminating pod as part of workflow shutdown" namespace=argo podName=postman-test-5lrl7-entrypoint-2114579652 shutdownStrategy=Stop workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.674Z" level=info msg="node postman-test-5lrl7-2114579652 phase Pending -> Failed" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.674Z" level=info msg="node postman-test-5lrl7-2114579652 message: workflow shutdown with strategy:  Stop" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.674Z" level=info msg="node postman-test-5lrl7-2114579652 finished: 2023-03-09 19:08:26.6743069 +0000 UTC" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.674Z" level=info msg="node postman-test-5lrl7-2054235633 phase Pending -> Failed" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.674Z" level=info msg="node postman-test-5lrl7-2054235633 message: workflow shutdown with strategy:  Stop" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.674Z" level=info msg="node postman-test-5lrl7-2054235633 finished: 2023-03-09 19:08:26.674350294 +0000 UTC" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.675Z" level=info msg="Stopped with strategy 'Stop'" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.675Z" level=info msg="node postman-test-5lrl7 phase Running -> Failed" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.675Z" level=info msg="node postman-test-5lrl7 message: Stopped with strategy 'Stop'" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.675Z" level=info msg="node postman-test-5lrl7 finished: 2023-03-09 19:08:26.675819575 +0000 UTC" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.676Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.676Z" level=info msg=reconcileAgentPod namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.676Z" level=info msg="Updated phase Running -> Failed" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.676Z" level=info msg="Updated message  -> Stopped with strategy 'Stop'" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.676Z" level=info msg="Marking workflow completed" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.676Z" level=info msg="Marking workflow as pending archiving" namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.676Z" level=info msg="Checking daemoned children of " namespace=argo workflow=postman-test-5lrl7
time="2023-03-09T19:08:26.676Z" level=info msg="Workflow to be dehydrated" Workflow Size=14619
time="2023-03-09T19:08:26.680Z" level=info msg="cleaning up pod" action=terminateContainers key=argo/postman-test-5lrl7-entrypoint-2114579652/terminateContainers
time="2023-03-09T19:08:26.681Z" level=info msg="cleaning up pod" action=deletePod key=argo/postman-test-5lrl7-1340600742-agent/deletePod
time="2023-03-09T19:08:26.686Z" level=info msg="Delete pods 404"
time="2023-03-09T19:08:26.687Z" level=info msg="Create events 201"
time="2023-03-09T19:08:26.690Z" level=info msg="Update workflows 200"
time="2023-03-09T19:08:26.694Z" level=info msg="Workflow update successful" namespace=argo phase=Failed resourceVersion=93135187 workflow=postman-test-5lrl7

### Logs from in your workflow's wait container

```text
kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

Hi @JPZ13 - that PR was already merged when I tested with latest, so I don't think it fixed the issue. It seems that PR has more to do with Pods that are already running, but in the scenario I described, we are stopping the workflow before the Pod has been scheduled onto a node (still in Pending status).


heroku: FIX for node not found  https://github.com/NixOS/nixpkgs/issu…

Things done:

[x] Tested using sandboxing (nix-build --option build-use-chroot true or nix.useChroot on NixOS)
[x] Built on platform(s): NixOS
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

More
Fixes issue https://github.com/NixOS/nixpkgs/issues/10298
cc @aflatter @mirdhyn

I restarted the travis build. This looks sensible to me, feel free to ping me when you're happy to merge it.
PS: when you update the PR, subscribers are not notified.

:tada:


⚠ Bofrost.at has degraded performance

In d07d59a, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1181 ms

Resolved: Bofrost.at performance has improved in cbc137e after 8 minutes.


Add environment variable to Dockerfile

Pull request:

https://github.com/Senzing/docker-xterm/pull/55


Security - 2fa input error message shows 'Two Factor Authentication code'

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Issue found when executing PR https://github.com/Expensify/App/pull/22651
Action Performed:

Go to staging.new.expensify.com
Go to Settings > Security > Two-factor authentication
Click Copy > Next
Enter wrong 2fa code

Expected Result:
The 2fa input error message shows 'two-factor authentication code'
Actual Result:
The 2fa input error message shows 'Two Factor Authentication code' instead of 'two-factor authentication code'
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android / native
[x] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

Version Number: <IP_ADDRESS>
Reproducible in staging?: Yes
Reproducible in production?: Yes
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/78819774/f8c19ccc-b587-4e52-9b77-16eaf9a0158f
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:

There is a backend issue opened for this change as per this comment. Based on that, this is not a front-end change.
cc: @amyevans

Yeah the backend PR is only on staging at this point, not deployed to production. So this is an expected outcome, closing the issue.


[Segment Replication] Support enabling/disabling segment replication for an existing index.

Is your feature request related to a problem? Please describe.
Segrep right now can be enabled only during index creation.  This means converting an existing index to use segrep requires a full reindex.
Describe the solution you'd like
Support converting an existing index to use segment replication without reindexing.
Describe alternatives you've considered
None
Additional context
Add any other context or screenshots about the feature request here.

Closing as this is now supported.


verifier_domain is used as HELO argument and has much stricter requirements

As verifier_domain is used as HELO/EHLO argument during SMTP session it has much stricter requirements than the documentation implies and default value derived from verifier_domain can cause failure to establish SMTP session.  By RFC 5321 HELO/EHLO argument should contain FQDN of the server which initiated SMTP session, and many SMTP servers implement strict HELO checks where they check that IP which initiated connection has PTR record which matches FQDN presented in HELO. So, if this check is run on a server with IP <IP_ADDRESS> which has PTR record 1-2-3-4.hosterdomain.com then HELO argument should be 1-2-3-4.hosterdomain.com and also you need to make sure that 1-2-3-4.hosterdomain.com resolves into <IP_ADDRESS>

Hi, @AlexeyDemidov! Checkout PTR check feature in last gem release. What you think about SPF, DKIM and DMARC implementation for next auditor feature?

I would suggest passing an IP to test as a parameter to host_audit (and I would call this function something different, like sender_verify). Currently, using Socket.gethostname  and attempting to resolve resulting hostname into an IP is very unreliable way to get host IP - a host name can be non-FQDN (without full domain name) so it won't exactly match PTR, a hostname doesn't guarantee to resolve into any IP at all, it also won't resolve into actual IP used to connect to external servers (on many cloud VMs and/or servers behind NAT you'll get a private IP for a given hostname).
There is a missing step in PTR verification - it checks that PTR matches verifier_domain but it doesn't check that verifier_domain itself resolves into the given IP. For example, it possible to have an IP <IP_ADDRESS> to resolve into example.com but example.com itself can resolve into a different IP, like <IP_ADDRESS>.  In this case of mismatch many SMTP servers won't trust that email is coming from example.com and will mark it as suspicious.
Regarding SPF, DKIM and DMARC - it could be useful to implement DMARC and SPF checks as a part of sender verification functionality (DKIM is impossible to check on SMTP level) but parsing these DNS records is quite complex, so it will be necessary to use 3rd party gems to implement it (https://github.com/trailofbits/dmarc and https://github.com/trailofbits/spf-query). In long term it would be probably better to split sender verification functions into a separate gem.

Alexey, thank for your details! I have fixed it. Cheers!


Sharing the nginx files?

Hi,
I'm using both Nextcloud & Libreoffice Online behind a Nginx reverse proxy webserver.
It means it could be handy to also share those Nginx site configuration files.
See attachments for my config files (do whatever you like with it :+1:  ).
DISCLAIMER: I do not have LibreOffice still fully working actually. But my Nextcloud config does work really well.
Do you use any web-server on the host? Since for the production environment it is a good practice to use reverse proxy on top of the Nextcloud installation.
Regards,
Melroy van den Berg
Attachments: nginx_nextcloud_libreoffice_configs.zip

In my case I keep getting (using docker logs output of Libreoffice Online):
wsd-00015-00023 2019-02-26 18:05:16.300652 [ websrv_poll ] WRN  client - server version mismatch, disabling browser cache.| wsd/FileServer.cpp:279
wsd-00015-00023 2019-02-26 18:05:16.420838 [ websrv_poll ] WRN  FileServerRequestHandler: File not found: Invalid URI request: [/loleaflet/8a61a5302/branding.js].| wsd/FileServer.cpp:425
wsd-00015-00023 2019-02-26 18:05:16.647657 [ websrv_poll ] WRN  FileServerRequestHandler: File not found: Invalid URI request: [/loleaflet/8a61a5302/branding.js].| wsd/FileServer.cpp:425
wsd-00015-00023 2019-02-26 18:05:24.485582 [ websrv_poll ] WRN  client - server version mismatch, disabling browser cache.| wsd/FileServer.cpp:279
wsd-00015-00023 2019-02-26 18:05:24.620217 [ websrv_poll ] WRN  FileServerRequestHandler: File not found: Invalid URI request: [/loleaflet/8a61a5302/branding.js].| wsd/FileServer.cpp:425
wsd-00015-00023 2019-02-26 18:05:24.813640 [ websrv_poll ] WRN  FileServerRequestHandler: File not found: Invalid URI request: [/loleaflet/8a61a5302/branding.js].| wsd/FileServer.cpp:425
wsd-00015-00023 2019-02-26 18:05:25.868642 [ websrv_poll ] WRN  client - server version mismatch, disabling browser cache.| wsd/FileServer.cpp:279
wsd-00015-00023 2019-02-26 18:05:26.018280 [ websrv_poll ] WRN  FileServerRequestHandler: File not found: Invalid URI request: [/loleaflet/8a61a5302/branding.js].| wsd/FileServer.cpp:425
wsd-00015-00023 2019-02-26 18:05:26.212475 [ websrv_poll ] WRN  FileServerRequestHandler: File not found: Invalid URI request: [/loleaflet/8a61a5302/branding.js].| wsd/FileServer.cpp:425

I think Content Security Policy is to blame for this... Looking at the console logs.
While I'm using the official nextcloud nginx configs:
https://docs.nextcloud.com/server/stable/admin_manual/installation/nginx.html?highlight=nginx

@danger89 The CSP should actually be properly set by the richdocuments app to the wopi_url configured: https://github.com/nextcloud/richdocuments/blob/970ecadce4a39fd634c56aa583ff8ee5d8b8bdd3/appinfo/app.php#L66-L71 Note that both the Nextcloud server as well as the client browser needs to be able to contact the collabora server though that url.
As this seems to be a setup issue I would like to ask you to raise further question in the forums: https://help.nextcloud.com
If you wish support with setup issues from Nextcloud GmbH we offer this as part of the Nextcloud subscription. Learn more about this at https://nextcloud.com/enterprise/

Ah nice, thanks for sharing your config!
There is some nginx example in the official LibreOffice Online repo: https://gerrit.libreoffice.org/plugins/gitiles/online/+/master/etc/nginx/loolwsd.conf
Maybe you want to provide a patch against that file?

I do not have LibreOffice still fully working actually. But my Nextcloud config does work really well.

Does LO work without nextcloud, i.e. can you access something like
yourdomain:9980/loleaflet/dist/loleaflet.html?file_path=file:///opt/libreoffice/share/template/common/internal/idxexample.odt ?

Ah nice, thanks for sharing your config!
There is some nginx example in the official LibreOffice Online repo: https://gerrit.libreoffice.org/plugins/gitiles/online/+/master/etc/nginx/loolwsd.conf
Maybe you want to provide a patch against that file?

I tried this config as well, without any luck too bad.

Does LO work without nextcloud, i.e. can you access something like
yourdomain:9980/loleaflet/dist/loleaflet.html?file_path=file:///opt/libreoffice/share/template/common/internal/idxexample.odt ?
I can try.  Atleast the admin panel does work flawlessly:
https://office.melroy.org/loleaflet/dist/admin/admin.html

我不是使用nginx，不過有將libreoffice建置成功，docker-compose.yml，參考一下
version: '2.1'

services:
  online:
	image: libreoffice/online:master
	environment:
	  domain: <cloud.domain.tld>
	  username: admin
	  password: admin
	  extra_params: --o:ssl.enable=false
	  DONT_GEN_SSL_CERT: 1
	volumes:
	  - config-volume:/etc/loolwsd
	cap_add:
	  - MKNOD
	ports:
	  - 9980:9980
	restart:
	  unless-stopped
	networks:
	  - proxy-tier

  db:
	image: mariadb
	restart: always
	ports:
	  - 3306:3306
	environment:
	  MYSQL_ROOT_PASSWORD: root
	  MYSQL_DATABASE: nextcloud
	  MYSQL_USER: nextcloud
	  MYSQL_PASSWORD: nextcloud
	networks:
	  - proxy-tier

  app:
	image: nextcloud
	links:
	  - db
	ports:
	  - 80:80
	volumes:
	  - ./nextcloud/config:/var/www/html/config
	  - ./nextcloud/apps:/var/www/html/apps
	  - ./nextcloud/data:/var/www/html/data
	restart:
	  always
	networks:
	  - proxy-tier

  redis:
	image: redis
	container_name: redis
	networks:
	  - proxy-tier

volumes:
  config-volume:
  nextcloud:
  db:

networks:
  proxy-tier:
	external:
	  name: nginx-proxy

I have set my instance of this project behind an NGINX reverse proxy. The setup is that I have one internet facing IP and I host quite a few sites (and other things - email etc). I use the nameserver zone editor to carve up subdomains on my main website which allows me to host different services for my own use (I have a search engine, a wiki, a website, a nextcloud server and so on). All of these things are setup on VMs. The internet facing router has a portforward rule to send all port 80 requests to <IP_ADDRESS> which is the NGINX server. This then inspects the header for the originating domain (or subdomain) and it then upgrades the headers and points each request to a local network IP, which is of course the IP of the correct server for that domain or subdomain. I have also opened port 9980 to also pass all traffic to the NGINX server 192.168.15 - this is important!
My nextcloud instance is IP <IP_ADDRESS>, the Libreoffice is <IP_ADDRESS>
The NGINX entries for this (sites-available/default) are:
server {
        listen 80;
        server_name nclonc.mydomain.co.uk;
        location / {
        proxy_pass http://<IP_ADDRESS>:80;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connetion 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
        }
}

server {
        listen 80;
        server_name nclolo.mydomain.co.uk;
        location / {
        proxy_pass http://<IP_ADDRESS>:9980;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connetion 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
        }
}

This setup works, but, in this part of the wiki "Then go to Configuration->Collabora Online and enter the domain name of your other VM, e.g. http://libreoffice.yourhost:9980" you must omit the :9980 because the NGINX header upgrade handles this portnumber change.
To get to the nextcloud from anywhere on the web (ie not on the local network) I just go to:
nclonc.mydomain.co.uk
and login as normal. When I want to edit a LO document, I just click on it and all the redirects etc work in the background fine.
Hope this helps someone!


[Doc]: Update README.md(dev-3.x)

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.
Motivation
Update mmclassification to mmpretrain.
Modification
Update mmclassification to mmpretrain.
BC-breaking (Optional)
Does the modification introduce changes that break the backward-compatibility of the downstream repos?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.
Use cases (Optional)
If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.
Checklist

Pre-commit or other linting tools are used to fix the potential lint issues.
The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMCls.
The documentation has been modified accordingly, like docstring or example tutorials.

it seems identical with https://github.com/open-mmlab/mmdetection/pull/10102

@freepoet please resolve the conflicts


Please update docs.

URL:

Describe:

Please update docs.
All props and states are like this

Sorry I just noticed that some update messed something up. Fixed!


Handle syntax error in stages

If stages is missing a properly formatted list the error is:
TypeError: undefined is not a function
at Parser.init (/snapshot/gitlab-ci-local/dist/parser.js)
at async Function.create (/snapshot/gitlab-ci-local/dist/parser.js)
at async Object.handler (/snapshot/gitlab-ci-local/dist/default_cmd.js)

>cat .gitlab-ci.yml
---
stages: lint

Pretty message is thrown in 7.8.0 @Supergoof

Pretty message is thrown in 7.8.0 @Supergoof


add filename to path

This allows you to add the filename to the path with produces more readable and understandable URLs. It doesn't actually do anything with or require the additional URL params so it shouldn't break any existing deployments.
Current URL:
localhost:3000/0
New URL:
localhost:3000/0/FILE_NAME.mp4

Sure, why not.

@feross good catch, I have updated the PR. PTAL

Released as 0.98.14.


IceCube Speedup

I updated icecube_get_lightcurve to use astropy search_around_sky instead of a for loop over all objects.  This should provide a large increase in speed of the ice cube function.  Currently I can't do a good scale test on this because the SDSS function isn't working, but this really should be a speedup, so I am pushing the changes without the scale test.
I would appreciate if someone could check the filtering that I did after the match, to make sure the code does what I wanted it to.  My intent was to have it match all sources within the error radius of of IceCube, then later filter that down to the top three in energy for each input source.

I believe I have addressed @troyraen comments.
Also was able to do the scale testing and the new ice cube function was able to get "light curves" for 5000 sources in 20.5s, so should not have trouble completing 500k sources very quickly.


Include sources for Android webview in plugin.xml

Initializing the WebView component on Android is throwing a Class not found error. This is because the corresponding source file is missing from the plugin.xml.
Fixes issue https://github.com/Microsoft/ace/issues/23

Hi @krik, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
We will now validate the agreement and then real humans will evaluate your PR.
TTYL, MSBOT;

Perfect, thank you!


patchcore subsample_embedding is very slow

Hi,
I have 2000+ images( img_size = 360x640) to train, it is very slow at subsample_embedding stage.
is there any method to accelerate this ?

@Vvdinosaur, until we optimize the code, you could perhaps reduce the number of layers you are extracting. For example, you could only extract the features from 'layer3'. You could potentially play with coreset_sampling_ratio and num_neighbors parameters to improve the computation as well.


[MRG] Ica indexing

Take over from https://github.com/mne-tools/mne-python/pull/3565 thanks @ahoejlund.

Current coverage is 87.29% (diff: 80.00%)

Merging #3574 into master will increase coverage by <.01%

@@             master      #3574   diff @@
==========================================
  Files           341        341
  Lines         60139      60143     +4
  Methods           0          0
  Messages          0          0
  Branches       9186       9186
==========================================
+ Hits          52495      52503     +8
+ Misses         4898       4896     -2
+ Partials       2746       2744     -2

Powered by Codecov. Last update 7cd5d0e...9f49195

@ahoejlund does that do the job for you?

@agramfort it most certainly does, thx!

merge when CIs are happy

thx @jaeilepp


[SPARK-11381][DOCS] Replace example code in mllib-linear-methods.md using include_example

What changes were proposed in this pull request?
This PR replaces example codes in mllib-linear-methods.md using include_example
by doing the followings:

Extracts the example codes(Scala,Java,Python) as files in example module.
Merges some dialog-style examples into a single file.
Hide redundant codes in HTML for the consistency with other docs.

How was the this patch tested?
manual test.
This PR can be tested by document generations, SKIP_API=1 jekyll build.

Can one of the admins verify this patch?

So many changes in this commit.

Thank you for reviewing, @noprom . This PR is the similar to #11053  (merged yesterday.)
Hi, @yinxusen, @mengxr .
Could you review this PR?

ok to test

cc @yinxusen

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51793/
Test FAILed.

Oh, thanks. I'll check the Jenkins in detail.

Actually, the error was Github timeout. I pushed force again. Could you ask Jenkins to retest please?
+refs/pull/11320/*:refs/remotes/origin/pr/11320/* # timeout=15
ERROR: Timeout after 15 minutes
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from https://github.com/apache/spark.git

To be sure, I rebased this PR to the master, too.

Test build #51805 has started for PR 11320 at commit 3e56717.

Test build #51805 has finished for PR 11320 at commit 3e56717.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaL1UpdaterExample
public class JavaLinearRegressionWithSGDExample
public class JavaLogisticRegressionWithLBFGSExample
public class JavaSVMWithSGDExample

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51805/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51803/
Test FAILed.

Let me fix the Scala style. I thought I passed since build/mvn -DskipTests checkstyle:check ends with 'SUCCESS'. Now, I see that there were printed '[ERROR]' lines. Sorry for missing that.

@dongjoon-hyun You can use dev/scalastyle, dev/lint-python, etc. to simply the style check.

Thank you so much, @yinxusen ! I'll fix soon and let you know here.

I've done!

Test build 51808 is running now. Let's see the result. :)

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51808/
Test FAILed.

Merged build finished. Test FAILed.

Hmm, it fails again due to Github.
ERROR: Timeout after 15 minutes
ERROR: Error fetching remote repo 'origin'

According to Jenkins, other PRs also suffer from this. I think retriggering is not helpful at this time.

We can wait for a while. No worry.

Test build #51825 has started for PR 11320 at commit 12e40a0.

Test build #51825 has finished for PR 11320 at commit 12e40a0.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaL1UpdaterExample
public class JavaLinearRegressionWithSGDExample
public class JavaLogisticRegressionWithLBFGSExample
public class JavaSVMWithSGDExample

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51825/
Test PASSed.

Merged build finished. Test PASSed.

Finally! Now, it's ready to be reviewed again. :)
Thank you, all.

Hi, @yinxusen and @mengxr .
Could you review this if you squeeze some time out of your schedule?

I'll do it tomorrow. Thx
On Wed, Feb 24, 2016 at 3:12 PM, Dongjoon Hyun<EMAIL_ADDRESS>wrote:

Hi, @yinxusen https://github.com/yinxusen and @mengxr
https://github.com/mengxr .
Could you review this if you squeeze some time out of your schedule?
—
Reply to this email directly or view it on GitHub
https://github.com/apache/spark/pull/11320#issuecomment-188505409.

--
Cheers
Xusen Yin    (尹绪森)
LinkedIn: https://cn.linkedin.com/in/xusenyin

Thank you, @yinxusen !

Test build #52037 has started for PR 11320 at commit 92a3aff.

@dongjoon-hyun
Let's remove those examples:
L1UpdaterExample.scala
JavaL1UpdaterExample.java
For these small pieces of codes, we can leave it untouched for now. Otherwise, there will be too many example codes.

Test build #52038 has started for PR 11320 at commit 4d20391.

Okay. No problem! I will update soon.

@dongjoon-hyun No need to rush, you can wait until I finish the round of reviewing.

Oh, I see. Then, let me know when you finish. :)

Test build #52040 has started for PR 11320 at commit 239e09e.

Test build #52038 has finished for PR 11320 at commit 4d20391.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52038/
Test PASSed.

Test build #52040 has finished for PR 11320 at commit 239e09e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52040/
Test PASSed.

@dongjoon-hyun For the section of streaming linear regression: http://spark.apache.org/docs/latest/mllib-linear-methods.html#streaming-linear-regression, we should  change the following sentence:

Anytime a text file is placed in /training/data/dir the model will update. Anytime a text file is placed in /testing/data/ dir you will see predictions. As you feed more data to the training directory, the predictions will get better!

Substituting the bold words with args(0) and args(1) in Scala; sys.argv[1] and sys.argv[2] for Python.

@dongjoon-hyun One more thing, try to make the output dirs different for different examples. Like there are 3 examples (Scala/Java/Python versions) use target/tmp/myLinearRegressionWithSGDModel. We cannot run those examples once a time if we don't delete the dir generated before.

@dongjoon-hyun I finished the review. Thanks for working on this!

Test build #52037 has finished for PR 11320 at commit 92a3aff.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Thank you, @yinxusen !

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52037/
Test PASSed.

Test build #52043 has started for PR 11320 at commit 9000405.

I pushed the commit.
I've learn many things from you. Thank you very much, @yinxusen .

Test build #52043 has finished for PR 11320 at commit 9000405.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52043/
Test PASSed.

@mengxr LGTM
Note that we consolidate the original streaming linear regression example from small pieces into a complete example and modify some sentences.

Merged into master. Thanks!

Thank you so much, @mengxr and @yinxusen !

Hi, @mengxr .
Could you close this PR?
It is merged to the master successfully, but is not closed until now.

Hi, @mengxr .
Could you close this PR? :)

@dongjoon-hyun normally the Apache ASF bot closes the PRs automatically, but I don't know why this one was not closed. We actually can't close PRs. You would have to close it manually.

Thank you for closing this, @srowen and @mengxr .


Sharepoint Admin API Access page: "Error ensuring domain-wide principals"

Target SharePoint environment
SharePoint Online
What SharePoint development model, framework, SDK or API is this about?
💥 SharePoint Framework
Developer environment
Windows
What browser(s) / client(s) have you tested

[ ] 💥 Internet Explorer
[X] 💥 Microsoft Edge
[X] 💥 Google Chrome
[ ] 💥 FireFox
[ ] 💥 Safari
[ ] mobile (iOS/iPadOS)
[ ] mobile (Android)
[ ] not applicable
[ ] other (enter in the "Additional environment details" area below)

Additional environment details

browser version
SPFx version
Node.js version
etc

Describe the bug / error
I've uploaded an spfx app to the app catalog with Microsoft Graph permissions requested in package-solution.json. The app was uploaded successfully, but when I visit the Sharepoint Admin API Access page, the permission request does not load and I see the following error:

Error ensuring domain-wide principals: The application identifier uri 'api://148ec9e1-128a-4017-afd9-449afa449006/microsoft.spfx3rdparty.com' is invalid.

This is the first time I have uploaded an app that requests Microsoft Graph permissions. I believe the API Access page is trying to create a new App Registration for an app called "SharePoint Online Client Extensibility Web Application Principal". I can see it make a failed network request to "https://graph.microsoft.com/beta/applications/". It sends two identifierUris in the payload. One is the one from the error above, and one has my organisation's tenant ID and sharepoint domain. The ID 148ec9e1-128a-4017-afd9-449afa449006 from the error is not my tenant ID. I am logged in as a global administrator.
Steps to reproduce

Upload an app to the App Catalog that requires Microsoft Graph permissions for the first time
Go to the Sharepoint Admin Center API Access page

Expected behavior
The Microsoft Graph permission request for the app loads and can be approved.

I am able to see the permission request via the PowerShell command Get-SPOTenantServicePrincipalPermissionRequests. There were a few other permissions requests from a third party app there too. I tried denying all the requests from Powershell to start fresh, but I still get the same error from the Sharepoint API access page.

We are having the same issue. Following for updates...

Same here

Getting this same error in our tenant.
If I try to approve the app requests via PowerShell using the Get-SPOTenantServicePrincipalPermissionRequests and Approve-SPOTenantServicePrincipalPermissionRequest cmdlets, I get an error that says the service principal for the request could not be found.

Same, follwing for updates.

Is there an update? Let me know :)

I think we were all hoping you might supply the update. :)

microsoft fixed it for us but did not gave a solution

microsoft fixed it for us but did not gave a solution

We have a ticket open with Microsoft too. Would you be willing to provide your case number so that I can share it with them?

Seeing the same issue across multiple client sites. @VesaJuvonen can you provide any insight into this one? It makes for a very bad first time experience for clients installing an App from AppSource. Is the only solution to contact MS and wait weeks for a manual fix?

FWIW - shared on twitter https://x.com/TrentAllday/status/1802496060415422563

We are actively looking into this. Thank you all for reporting and we'll work on getting this solved as soon as possible. I do apologize the delay on acknowledging it. Please do open up tickets using the official channel as you see it as well - as more tickets will mean also more focus. Not optimal, but trying to help to fix this efficiently.

We do apologize the situation here and are using our internal routes to push on the resolution.
Please open support tickets in the tenant administrator channels in the admin UX - this is out of the box feature in the admin UX - which should be working without exceptions. If you have an option to open the support case with "Premier Support", please do that as it will help here as well.
The more reports we get through official channels - the faster we can get this resolved. Unfortunate situation for sure and using the official channels will help everyone. Thank you all.

Solved!! Thanks so much for the tip @jacobmessner
Here is a video on how you go about solving this problem (including my troubleshooting and delight in getting it working 🤣 )
https://www.loom.com/share/c10757775a514003aa8c697a41b9895d
The key missing part is clicking the link on the dev tools console to determine the app id. That app id relates to an enterprise app called 'SharePoint Framework Azure AD Helper' (which I had never seen before but must be responsible for provisioning 'SharePoint Online Client Extensibility Web Application Principal'). I then was able to determine the id of that app and patch it using Graph Explorer.
Its seems this tenant went through a name change at some stage and this part is not cleaned up as part of that rename process done by MS.
Thanks again. Hopefully this helps someone else from wasting hours and hours!

Hi, When you look up the app ID from Microsoft Support again, you will see that this app also has a name. Could you please tell me the name of this app?

Hi @mrwoes,
I ran the query again and the appDisplayName was "SharePoint Framework Azure AD Helper"
Hope this helps!

Hello,
Thanks for your comment. Unfortunately, the app registration is set up correctly for us. Microsoft has also checked this. So that's not our problem...

I am also having an issue with another tenant getting this exact error reported initially in this post. My solution above doesnt resolve it. I'll keep you posted on any progress.

Any update?

Figured my problem out today.  Mircosoft had told me to manual create an App Registration, but didn't tell me what one I was supposed to create.  In the Enterprise Application Audit Log I found a failure for SharePoint Framework Azure AD Helper trying to modify properties for SharePoint Online Client Extensibility Web Application Principal.  Turned out I didn't have SharePoint Online Client Extensibility Web Application Principal, so I added it as the App Registration name from their instructions and everything started working immediately.  Not sure its anyone else's issue, but the error I had was identical to OP.

@Yin-ko was there any specifics you needs to configure with the principle or was it just that the name matched?

I was concerned I'd have to put together IDs but it only needed the correct name and it associated everything else automatically.

oh wow!! That was the easiest fix ever! Nice work @Yin-ko problem solved for me too.

Glad to be of assistance.

I'm seeing the same problem with the calls to /beta/applications failing because the api URI is invalid.
I checked the helper principal and for me the reply URL was correct.
Deleteing and recreating the app looks like it works, but if you then refresh the API managemebt page and check the network traffic I see it making a PATCH on the second go still trying to set the incorrect APIs.  This leaves us with a half built security principal and it also doesn't make the call to UpdateSpfxThirdPartyAppId to set the security principal ID.
What then happens is the SPFx looks like its working until it tries to get a token for making a Graph call, and fails because the scope of the token is set to the deleted principal.
We've only seen this on one tenancy out of 60, and a pretty old one, not sure how old you tenancies are?
We're stuck at the moment anyway.

Deleteing and recreating the app looks like it works, but if you then refresh the API managemebt page and check the network traffic I see it making a PATCH on the second go still trying to set the incorrect APIs. This leaves us with a half built security principal and it also doesn't make the call to UpdateSpfxThirdPartyAppId to set the security principal ID.

This is basically where I got to before giving up. I manually created the App Registration in Entra, as per Yin-ko above. This let the Sharepoint Admin API permissions page load, but it still makes failed PATCH calls due to the invalid URI. I also tried updating the App Registration manually with the data from the failed calls, but the SPFx app still can't connect to the Graph API. I'm not familiar enough with SPFx to figure out why it's failing, but possibly the same reason as marcus-atvero. I did try going through Microsoft support, but never managed to get to someone who could figure out the issue.

You might want to look in Enterprise Applications - Audit logs to see if there is a corresponding error to your problem.

In our case the SharePoint Online Client Extensibility Web Application Principal was correct. Only the Authorized client applications were missing. I found this page: https://www.eliostruyf.com/fix-admin-consent-sp-token-retrieval-flows-spfx/
When I added those 4 clientid's my api access page was working again.

In my case the Enterprise Application SharePoint Online Client Extensibility Web Application Principal seemed correct. I did see a difference on the Expose an API page in the section Authorized client applications with a tenant that is working. I added the clientid's that are on my working tenant and the api access page started working again. I don't know if this fixes everything but I am investigating further.


[Question] How is the script specified while creating pythonVisual

When using createVisual api how does one inject the script that plots the visual.

Never mind ended up using react to build custom visual and using that for visual creation...


[RESOLVED] Unavailable by ICMP ping

Problem started at 01:42:29 on 2023.04.14
Problem name: Unavailable by ICMP ping
Host: HP_iLO
Severity: High
Operational data: Down (0)
Original problem ID: 16169
Event details in Zabbix: http://<IP_ADDRESS>/tr_events.php?triggerid=23266&eventid=16169

Problem has been resolved in 5h 23m 0s at 07:05:29 on 2023.04.14
Problem name: Unavailable by ICMP ping
Host: HP_iLO
Severity: High
Original problem ID: 16169


Support Geopandas 1

Fixes https://github.com/holoviz/spatialpandas/issues/145

Obviously not in love with the download logic, but fine for now.


Enable npm publish for node releases

Fix for #505 , where the matrix was changed from
  matrix:
    os:
      - macos-12
      - ubuntu-20.04
    node: [ 14, 16, 18 ]

to
  matrix:
    include:
      - runs-on: ubuntu-20.04
        arch: x86_64
      - runs-on: macos-12
        arch: x86_64
      - runs-on: macos-12-arm
        arch: arm64

so the os should be referenced as matrix.runs-on instead of matrix.os

I tested this on my branch and the npm publish worked this way. Sorry I missed that, I was so focused on the github publish piece I didn't notice npm publish broke.
You may want to delete the two pre-releases that don't have a node release since I don't think they can be used without the npm publish.


nixos/gitea: don't recursively change file owners/permissions

This scales really poor with lots of files.
eg. 300000 take minutes to finish on SSD
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

bump


updated verion of help client to 1.3.0

Codecov Report

Merging #405 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master    #405   +/-   ##
======================================
  Coverage    99.3%   99.3%
======================================
  Files          73      73
  Lines        1873    1873
  Branches      294     294
======================================
  Hits         1860    1860
  Misses         13      13

Flag
Coverage Δ

#builder
100% <ø> (ø)
:arrow_up:

#runtime
95.42% <ø> (ø)
:arrow_up:

#srcapp
100% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 21d0e24...f249993. Read the comment docs.


fix toUpper function by definition of decapitalize in java.bean.Intro…

…spector

LGTM
Although we still not have unit-test QQ

thx


SpiderMonkey CMake package: use alternate artifacts if specified.

When developing StarlingMonkey using an existing SpiderMonkey build with no SM changes, it is very convenient to download and use pre-built artifacts for SM from the CI on spidermonkey-wasi-embedding, and we do not want to lose this convenience or approachability.
However, when modifying SpiderMonkey itself, it is nearly impossible to do development with the current setup: the debug loop requires merging changes to gecko-dev with one PR, updating the hash in spidermonkey-wasi-embedding with another PR, getting both merged and waiting for the build in CI before StarlingMonkey can use the new engine.
Rather than require a local hack for such cases, this modifies the SpiderMonkey CMake module to recognize a SPIDERMONKEY_BINARIES environment variable, and use artifacts there instead. This allows the developer to point this to a local clone of spidermonkey-wasi-embedding with a modified gecko-dev, rebuilt with rebuild.sh as needed. The README is updated with details on how to use this.

Sure thing, updated! This now takes a SPIDERMONKEY_BINARIES environment variable. I've added a section at the end of README.md describing the workflow for developing the engine itself too.
(cc @guybedford I think from #81: this check that Till linked above is checking DEFINED $ENV{VAR}, while CMake seems to require DEFINED ENV{VAR} (at least, that is what worked for me in testing this PR). I don't know enough about the WPT plumbing to test whether the above is doing what you mean but it might be good to verify?)