OpenCV RGB-D Odometry

Required Info

Camera Model
D435

Firmware Version
<IP_ADDRESS>

Operating System & Version
Linux (Raspbian Buster)

Kernel Version (Linux Only)
(e.g. 4.19.93)

Platform
Raspberry Pi 4B

SDK Version
GitHub master

Language
C++

Segment
Robot

Issue Description
First of all, thanks for the fantastic library. It's been a pleasure to work with so far and my only issues have been related to Linux USB flakiness in general.
I'm working on a project where I have a robot equipped with the D435 camera. The robot is controlled by an external PC. I need to stream RGB and Depth data in lockstep to the PC over WiFi.
I set up an Archer C6v2 5Ghz AP and I'm getting ~90 Mbits/sec in either direction, so that is not the limiting factor for me.
On the software side, I want to use the RGB and Depth data to compute the visual odometry using OpenCV. The streaming pipeline is established and works well so far. I also built in JPEG compression for the RGB frames and ZSTD for the 16 bit grayscale ones, which is all working nicely.
The OpenCV RGBD module doesn't seem to be documented overly well (https://docs.opencv.org/4.1.0/df/ddc/classcv_1_1rgbd_1_1Odometry.html and https://docs.opencv.org/4.1.0/d0/d60/classcv_1_1rgbd_1_1RgbdOdometry.html). From what I can see, I need a camera matrix for this to work. I can easily get the Cx,Cy Fx,Fy values from the RealSense tools, but I'm not sure whether I need to fill in the depth or color sensor intrinsics here?
Furthermore, it looks like I cannot do real-time image alignment (depth-to-color or color-to-depth) on the Raspberry, it's just too slow. Now I can probably do it on the host machine instead using the RealSense provided extrinsics, but I'm not sure where I would need to do that.
If I rectify/align the images, why do I need to pass the camera matrix to OpenCV?
I apologize if this is more of an OpenCV question rather than a RealSense one. Please let me know if you are able to help me with this issue or if I should go ask somewhere else.
TL;DR

Which camera matrix is needed to compute visual odometry using OpenCV?
What is the best way to align the images on the external PC?
Are there any pre-made examples of visual odometry using RealSense?

@raymanfx Some OpenCV examples are in https://github.com/IntelRealSense/librealsense/tree/master/wrappers/opencv  Please check if it helps. Thanks!

@raymanfx  Any other questions about this? Looking forward to your update. Thanks!

Hi, your OpenCV examples were not helpful obviously.
In the meantime, I figured out how to align the depth to the color frame on my host machine while streaming the images to from the Raspberry Pi.
Leaving out my custom solution for streaming compressed color and depth data in realtime, I basically did the following:
<RemoteCapture> cap;

Get the depth -> color extrinsics from the realsense pipeline profiles:
    auto color_profile = cap.pipeline().get_active_profile().get_stream(RS2_STREAM_COLOR).as<rs2::video_stream_profile>();
    auto depth_profile = cap.pipeline().get_active_profile().get_stream(RS2_STREAM_DEPTH).as<rs2::video_stream_profile>();
    auto depth2color_extrin = depth_profile.get_extrinsics_to(color_profile);

Create the OpenCV camera matrix (intrinsics):
    // camera matrix layout:
    // fx  0  cx
    // 0   fy cy
    // 0   0  1

    // rs-enumerate-devices -c
    // color intrinsics
    cv::Mat color_intrinsics(3, 3, CV_32FC1);
    color_intrinsics.at<float>(0, 0) = color_profile.get_intrinsics().fx;
    color_intrinsics.at<float>(0, 1) = 0;
    color_intrinsics.at<float>(0, 2) = color_profile.get_intrinsics().ppx;
    color_intrinsics.at<float>(1, 0) = 0;
    color_intrinsics.at<float>(1, 1) = color_profile.get_intrinsics().fy;
    color_intrinsics.at<float>(1, 2) = color_profile.get_intrinsics().ppy;
    color_intrinsics.at<float>(2, 0) = 0;
    color_intrinsics.at<float>(2, 1) = 0;
    color_intrinsics.at<float>(2, 2) = 1;

    // rs-enumerate-devices -c
    // color intrinsics
    cv::Mat depth_intrinsics(3, 3, CV_32FC1);
    depth_intrinsics.at<float>(0, 0) = depth_profile.get_intrinsics().fx;
    depth_intrinsics.at<float>(0, 1) = 0;
    depth_intrinsics.at<float>(0, 2) = depth_profile.get_intrinsics().ppx;
    depth_intrinsics.at<float>(1, 0) = 0;
    depth_intrinsics.at<float>(1, 1) = depth_profile.get_intrinsics().fy;
    depth_intrinsics.at<float>(1, 2) = depth_profile.get_intrinsics().ppy;
    depth_intrinsics.at<float>(2, 0) = 0;
    depth_intrinsics.at<float>(2, 1) = 0;
    depth_intrinsics.at<float>(2, 2) = 1;

Account for rotation and translation to match the camera lens positions (extrinsics):
    // rs-enumerate-devices -c
    // extrinsics (depth to color)
    cv::Mat extrin_rot(3, 3, CV_32FC1);
    cv::Mat extrin_trans(1, 3, CV_32FC1);
    extrin_rot.at<float>(0, 0) = depth2color_extrin.rotation[0];
    extrin_rot.at<float>(0, 1) = depth2color_extrin.rotation[3];
    extrin_rot.at<float>(0, 2) = depth2color_extrin.rotation[6];
    extrin_rot.at<float>(1, 0) = depth2color_extrin.rotation[1];
    extrin_rot.at<float>(1, 1) = depth2color_extrin.rotation[4];
    extrin_rot.at<float>(1, 2) = depth2color_extrin.rotation[7];
    extrin_rot.at<float>(2, 0) = depth2color_extrin.rotation[2];
    extrin_rot.at<float>(2, 1) = depth2color_extrin.rotation[5];
    extrin_rot.at<float>(2, 2) = depth2color_extrin.rotation[8];
    extrin_trans.at<float>(0, 0) = depth2color_extrin.translation[0];
    extrin_trans.at<float>(0, 1) = depth2color_extrin.translation[1];
    extrin_trans.at<float>(0, 2) = depth2color_extrin.translation[2];

Assemble a final rigid body transform matrix:
    // rigid body transform
    // https://en.wikipedia.org/wiki/Camera_resectioning#Extrinsic_parameters
    cv::Mat Rt(4, 4, CV_32FC1);
    Rt.at<float>(0, 0) = extrin_rot.at<float>(0, 0);
    Rt.at<float>(0, 1) = extrin_rot.at<float>(0, 1);
    Rt.at<float>(0, 2) = extrin_rot.at<float>(0, 2);
    Rt.at<float>(0, 3) = extrin_trans.at<float>(0, 0);
    Rt.at<float>(1, 0) = extrin_rot.at<float>(1, 0);
    Rt.at<float>(1, 1) = extrin_rot.at<float>(1, 1);
    Rt.at<float>(1, 2) = extrin_rot.at<float>(1, 2);
    Rt.at<float>(1, 3) = extrin_trans.at<float>(0, 1);
    Rt.at<float>(2, 0) = extrin_rot.at<float>(2, 0);
    Rt.at<float>(2, 1) = extrin_rot.at<float>(2, 1);
    Rt.at<float>(2, 2) = extrin_rot.at<float>(2, 2);
    Rt.at<float>(2, 3) = extrin_trans.at<float>(0, 2);
    Rt.at<float>(3, 0) = 0;
    Rt.at<float>(3, 1) = 0;
    Rt.at<float>(3, 2) = 0;
    Rt.at<float>(3, 3) = 1;

Finally, align depth and color images using OpenCV:
cv::Mat frame_rgb;
cv::Mat frame_depth;
cv::Mat frame_depth_reg;

/* initialize frames from remote source */

cv::rgbd::registerDepth(depth_intrinsics, color_intrinsics, cv::Mat(), Rt, frame_depth,
                                    frame_rgb.size(), frame_depth_reg, true /* dilate */);

I hope this helps the next person wondering how to do this.
If you want, I can create a pull request with a stripped down example that needs just librealsense and OpenCV.

@raymanfx Thanks very much for your sharing! Pull requests are always welcome.

I created a pull request: https://github.com/IntelRealSense/librealsense/pull/6242.

@raymanfx Thanks very much for your contribution!

@raymanfx Our engineering team is looking at the PR https://github.com/IntelRealSense/librealsense/pull/6242 Let's track there and close this accordingly. Thanks!


Move complex CI worfklow logic into scripts

CI workflows, especially those related to changesets and release processes, have recently experienced a lot of expansion and have become rather cumbersome to read. Besides that, CI testing most of the time requires a lot of trial and error. We should move their logic to either shell or typescript scripts to make them more manageable and ideally write unit tests for existing logic and have the infrastructure ready for future logic to reduce trial and error.

[!CAUTION]
High caution needs to be exercised here to not break things.

hey @nedsalk, I suggest some additions to make these logics more understandable. Here's:

Break this script into separate files
Review the script code (Maybe we can improve it somehow)
Create an improvement proposal explaining some technical decisions


feat(curriculum): create css-positioning quiz

Checklist:

[x] I have read and followed the contribution guidelines.
[x] I have read and followed the how to open a pull request guide.
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

Closes #56191

Hi @naomi-lgbt !
Hope you are well ! Take a look at the issue #56191 of creating the quiz of css-postitioning as I've completed that. Thank You for assigning this issue :)
Regards
@Muhammad-Rebaal

@Muhammad-Rebaal have you had a chance to look at the outstanding reviews?


ignore Warning from flake8 and pytest

https://github.com/ruffsl/colcon-cache/pull/34

Codecov Report

Merging #22 (cf9ff0a) into master (0de2ce1) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files

@@          Coverage Diff           @@
##           master     #22   +/-   ##
======================================
  Coverage    8.00%   8.00%
======================================
  Files          11      11
  Lines         275     275
  Branches       41      41
======================================
  Hits           22      22
  Misses        253     253

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0de2ce1...cf9ff0a. Read the comment docs.


Cannot bookmark specific UI Tab

Use case:
Browser remote console v2.3.0
WLS: v<IP_ADDRESS>.0
In original WLS Admin console it was possible to bookmark/navigate to each individual Tab.
For example:
General:
https://localhost:7002/console/console.portal?_nfpb=true&_pageLabel=ServerConfigGeneralTabPage&handle=com.bea.console.handles.JMXHandle("com.bea%3AName%3DAdminServer%2CType%3DServer")
Protocols:
https://localhost:7002/console/console.portal?_nfpb=true&_pageLabel=ServerProtocolsTabprotocolsTabPage&handle=com.bea.console.handles.JMXHandle("com.bea%3AName%3DAdminServer%2CType%3DServer")
while now here we have
General:
http://localhost:8012/?root=view%2F%252Fapi%252F1662053110113%252FserverConfig%252Fdata%252FDomain%252FServers%252FAdminServer&view=form%2Fembedded#
Protocols:
http://localhost:8012/?root=view%2F%252Fapi%252F1662053110113%252FserverConfig%252Fdata%252FDomain%252FServers%252FAdminServer&view=form%2Fembedded#
so same thing and no differentiation, which is usually needed for quick navigation and repeating work/tasks.

We are investigating "bookmark" like options.  Given that the browser-based solution is only a secondary option, we are looking how best to do such functionality inside our electron app.
We will keep you informed as we get closer to such an enhancement.


Docs: convert tabs to spaces

This replaces the last tabs in the project, which are just in a few documentation files.  The other space-indented code in the docs was using 2-spaces in place of tabs so this follows the same formatting.

👾 👾


npm install failure

Not able to install npm. Please help
npm install
npm WARN optional dep failed, continuing<EMAIL_ADDRESS>npm WARN deprecated<EMAIL_ADDRESS>ReDoS vulnerability parsing Set-Cookie https://nodesecurity.io/advisories/130
npm ERR! not a package C:\Users\Amit Singh\AppData\Roaming\npm-cache\fill-range\2.2.3\package.tgz
npm ERR! not a package C:\Users\Amit Singh\AppData\Roaming\npm-cache\agent-base\2.0.1\package.tgz
npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm ERR! Windows_NT 10.0.14393
npm ERR! argv "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" "install"
npm ERR! node v4.5.0
npm ERR! npm  v2.15.9
npm ERR! file C:\Users\Amit Singh\AppData\Roaming\npm-cache\minimatch\0.3.0\package\package.json
npm ERR! code EJSONPARSE
npm ERR! Failed to parse json
npm ERR! Unexpected token '\u0000' at 1:1
npm ERR!
npm ERR! ^
npm ERR! File: C:\Users\Amit Singh\AppData\Roaming\npm-cache\minimatch\0.3.0\package\package.json
npm ERR! Failed to parse package.json data.
npm ERR! package.json must be actual JSON, not just JavaScript.
npm ERR!
npm ERR! This is not a bug in npm.
npm ERR! Tell the package author to fix their package.json file. JSON.parse
npm ERR! Please include the following file with any support request:
npm ERR!     C:\Users\Amit Singh\Desktop\Project\tracxpense\npm-debug.log
npm ERR! Windows_NT 10.0.14393
npm ERR! argv "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" "start"
npm ERR! node v4.5.0
npm ERR! npm  v2.15.9
npm ERR! code ELIFECYCLE
npm ERR<EMAIL_ADDRESS>prestart: npm install
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>prestart script 'npm install'.
npm ERR! This is most likely a problem with the angular-seed package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     npm install
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs angular-seed
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!
npm ERR!     npm owner ls angular-seed
npm ERR! There is likely additional logging output above.

It seems that you npm cache (C:\Users\Amit Singh\AppData\Roaming\npm-cache) is corrupted.
Try cleaning the cache (e.g. npm cache clean).
In any case, this is not an issue with angular-seed itself.


[FEATURE] Create FAQ (accordion) page

Related Issue
This PR addresses Issue #68
Closes: #68
Describe the changes you've made

I have updated the css file to include the base wrapper for the TailwindCSS core elements
Added in the FAQ template file
Added in a json file for dumby FAQ data

Type of change
Adding in the FAQ page and its corresponding data file
What sort of change have you made:

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[x] Code style update (formatting, local variables)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?

I have tested this file locally
Checklist

[x] My code follows the guidelines of this project.
[x] I have performed a self-review of my own code.
[x] I have commented on my code, particularly wherever it was hard to understand.
[ ] I have made corresponding changes to the documentation.
[x] My changes generate no new warnings.
[ ] I have added tests that prove my fix is effective or that my feature works.
[ ] Any dependent changes have been merged and published in downstream modules.

Screenshots (if applicable)

Original
Updated

original screenshot
updated screenshot

Code of Conduct

[x] I agree to follow this project's Code of Conduct

Hi all, I have addressed the concerns stated above. I do have a question because I thought I did the module export correctly but it doesn't seem Tailwind is recognizing `first:border-t-2" could someone help me with this?
https://github.com/web3community/web3community.github.io/pull/108#discussion_r733236220


[Serialization] Wire up generic environments for deserialized archetype types

Teach the serialized form of ArchetypeType about its owning generic
environment, so we can wire up the generic environment of (primary)
archetypes eagerly (at the point of deserialization)t. This ensures that there is no point
at which we have a (non-opened-existential) archetype without a
generic environment.
... except that the type reconstruction code creates such archetypes

@swift-ci please smoke test and merge


The Inaugural Issue! (color scheme choices/bugs :)

I like your theme very much, and, as I'm "in the market" :) for a light UI theme for Atom but can't quite make one myself.. I'm very grateful for your effort.
And I have two questions: first, can the accent color on the active tab (left border, inherited, I take it, from One Light) be changed to be the same blue hue that you have in the status bar, or indeed, to any color; and second, is this

... a bug or a feature? I mean the background color for the find/replace panel?
Thanks!

Indeed it is a bug. I'm going to commit an update in a couple of minutes for both themes. By the way, could you please tell me to what packages correspond the elements I mark in the image? I'd like to correct them too.
If you want to set a different color you can write this in your styles.less file
 .theme-atom-visual-studio-code-light-ui {
   .tab-bar .tab::before {
     background: #007ACC;
   }
}

Thanks a lot!
and well the packages are terminal-panel, and, wordcount
https://github.com/thedaniel/terminal-panel
https://github.com/nesQuick/atom-wordcount
I'm really a bad bug reporter, I should have said immediately: I'm running Atom 1.0.19, on Ubuntu 15.04

Added integration for terminal-panel and atom-wordcount in both themes.
Thank you for report!


Checkpoints Availability

Hello! I was wondering when/if the checkpoints for prophet would be made publically available? Thanks in advance :)

Thank you very much!


Server-side Rendering

Let's discuss what needs to be done.
Possible ways are:

Inlining of js styles and jss as it is, as js (possible now). So we can put jss styles as local script tags directly into the head together with the lib and render jss to css at runtime, but without latency for downloading them.
Converting jss styles to regular css server-side, once js components start, regenerate CSS from js styles. (possible now)
Full rehydration of styles from CSS from the DOM. (needs deterministic class name generation and evtl. parsing of css rules from dom)

released


Add Fuzzit badge

Which problem is this PR solving?
Shows fuzzing status with build status.
Short description of the changes
Proposing to show fuzzing status in the readme. Thank you for considering.
Meant to include this in the fuzzing integration #1787.

Happily the build is all green this time. Strange how that comes and goes. Those can be the toughest bugs to deal with.

Those can be the toughest bugs to deal with.

It's actually more annoying than that: the latest failures were due to Travis networking issues...

Such a pain. I've been seeing that in other repos too.
Thanks a lot for merging.
I think you can remove me from the Fuzzit org in settings, then it's all yours.


Recent changes fail xt/ X<> test

✗ xt/search-categories.rakutest | grep 'not ok'
    not ok 10 - Correct dimension (2) for a search anchor 'role declaration' doc/Language/objects.rakudoc
    # Failed test 'Correct dimension (2) for a search anchor 'role declaration' doc/Language/objects.rakudoc'
    # at /Users/coke/.rakubrew/versions/moar-2024.05/install/share/perl6/core/precomp/00E4C4D30CA66099863ED991D017ADBF6F0B304B/B2/B2F7F7E46D6B516E58A9B1F15344F10C035CFCC2 line 1
    # expected: '2'
    #      got: '1'
    not ok 50 - Correct dimension (2) for a search anchor 'Property' doc/Language/objects.rakudoc
    # Failed test 'Correct dimension (2) for a search anchor 'Property' doc/Language/objects.rakudoc'
    # at /Users/coke/.rakubrew/versions/moar-2024.05/install/share/perl6/core/precomp/00E4C4D30CA66099863ED991D017ADBF6F0B304B/B2/B2F7F7E46D6B516E58A9B1F15344F10C035CFCC2 line 1
    # expected: '2'
    #      got: '1'
    # You failed 2 tests of 68
not ok 46 - doc/Language/objects.rakudoc
# Failed test 'doc/Language/objects.rakudoc'
# at xt/search-categories.rakutest line 29
# You failed 1 test of 428

Apologies. That test was verifying that every link had a category, and that only the categories listed in https://github.com/Raku/doc/blob/main/writing-docs/INDEXING.md#categories are considered valid. This was part of a cleanup effort a while back to make sure we weren't accidentally adding orphaned entries, and that any new categories were being added deliberately.
So
X<property|Property>

Has no category and is failing, but
X<indirect invocant syntax|Language,indirect invocant syntax>

Is in the valid "Language" category, and is OK.
X<indirect invocant syntax|Dinosaur,indirect invocant syntax>

Has the right format, but an invalid category, so we'd fix the category or add it to the explicit list in writing-docs.
Definitely willing to hear arguments about whether or not a this is a reasonable test, but it definitely helped catch some issues/typos.

Also, I can fix these particular ones! Please just keep this mind when adding new X<> in going forward!

@coke I'll definitely keep in mind going forward. So thanks for the heads up.
As far as I'm concerned, any test that picks up errors is a good test!


snp2h5 no longer supports matching chromosomes regardless of "chr" prefix

The most recent version of WASP (released in July 2018) no longer matches chromosomes between bam and VCF files if the "chr" prefix is present in one file but not the other. For example, "chr1" in a bam file will not match "1" in a VCF file.

I have modified snp2h5 so that it now tries to add/remove 'chr' from the chromosome name that is read from the VCF file in order to match chromosome names read from the chromInfo file:
https://github.com/bmvdgeijn/WASP/commit/3184de35c8339ef667a4377c6daa371cc742dcc4
This change has been committed to the master branch and is also part of today's release (v0.3.1).
If you are having problems matching chromosome names elsewhere in the pipeline please let us know.

Thanks!


update document for KYLIN-4240

see https://issues.apache.org/jira/browse/KYLIN-4240 and https://github.com/apache/kylin/pull/958
add KYLIN-4240 document and update SAML properties names

Design test cases
Prepare test env

create cube and build segment 2012-01-01~2012-02-01

dimension: PART_DT, OPS_REGION
measure: count_distinct(SELLER_ID)

Test Case

Query

select
intersect_count(SELLER_ID, OPS_REGION, array['Hongkong']) as h,
intersect_count(SELLER_ID, OPS_REGION, array['Beijing']) as b,
intersect_count(SELLER_ID, OPS_REGION, array['Shanghai']) as s,
intersect_count(SELLER_ID, OPS_REGION, array['Hongkong|Shanghai', 'Beijing']) as retention_city
from kylin_sales


AttributeError: 'NoneType' object has no attribute 'entity_id'

Describe the bug
Not all devices (reminder, alarms, ...) are available. And log get spammed with errors
My config
I have a simple config without any filters:
alexa_media:   accounts:     - email: !secret amazon_username       password: !secret amazon_password       url: amazon.de
System details

Home-assistant (version): 0.101.3
Alexa Media Player (version): 2.3.4
Hassio (Yes/No): yes

Logs
2019-11-02 16:33:29 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/alexa/state_report.py", line 181, in async_send_delete_message alexa_entity = ENTITY_ADAPTERS[domain](hass, config, hass.states.get(entity_id)) File "/usr/src/homeassistant/homeassistant/components/alexa/entities.py", line 150, in __init__ self.entity_conf = config.entity_config.get(entity.entity_id, {}) AttributeError: 'NoneType' object has no attribute 'entity_id'  2019-11-02 16:33:29 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/alexa/state_report.py", line 147, in async_send_add_or_update_message alexa_entity = ENTITY_ADAPTERS[domain](hass, config, hass.states.get(entity_id)) File "/usr/src/homeassistant/homeassistant/components/alexa/entities.py", line 150, in __init__ self.entity_conf = config.entity_config.get(entity.entity_id, {}) AttributeError: 'NoneType' object has no attribute 'entity_id'
Additional context
Add any other context about the problem here.

Ok this error has nothing to do with the alexa_media player. It is related to the alexa skill integration somehow. So you can close this issue here...


v1.6 backports 19-07-25

#8663 -- minor follow-up improvements and fixes in nat engine (@borkmann)
#8676 -- bpf: initial docs for getting started on host reachable services (@borkmann)
#8684 -- docs: Remove confusing mentioning of etcd server in ConfigMap (@brb)
#8681 -- docs: Add NodePort GSG (@brb)
#8670 -- datapath: Store NodePort client MAC addr in LRU map (@brb)
#8643 -- Gopkg: update etcd library to v3.3.13 (@aanm)
#8673 -- test: Fix etcd-operator readiness check (@joestringer)

Once this PR is merged, you can update the PR labels via:
$ for pr in 8663 8676 8684 8681 8670 8643 8673; do contrib/backporting/set-labels.py $pr done 1.6; done

This change is

test-me-please

test-missed-k8s

test-docs-please

test-upstream-k8s

previous failure https://jenkins.cilium.io/job/Cilium-PR-Ginkgo-Tests-Validated/13889/

test-me-please

@ianvernon it seems the a commit from this PR brakes the test as it failed 2 times in a row.

The following PR's may have broken the CI:
• #8663 -- minor follow-up improvements and fixes in nat engine (@borkmann)
• #8676 -- bpf: initial docs for getting started on host reachable services (@borkmann)
• #8670 -- datapath: Store NodePort client MAC addr in LRU map (@brb)
• #8643 -- Gopkg: update etcd library to v3.3.13 (@aanm)
• #8673 -- test: Fix etcd-operator readiness check (@joestringer)

Previous failure: https://jenkins.cilium.io/job/Cilium-PR-Ginkgo-Tests-Validated/13909/

test-me-please

I don't believe that these above failures are strictly related to the PR - the v1.6 CI has been a bit unstable :( https://jenkins.cilium.io/view/Cilium-v1.6/job/cilium-v1.6-standard/

test-me-please


[performance] assemblicator with formula does useless recipe search each tick

Formulaic Assemblicators are doing a full recipe search in the doSingleCraft() method, which is being called each tick by onUpdate(), even when they can't do anything AND have a crafting formula set. This is creating a considerable load on the server. In my world with 5 assemblicators (all have formulas set) it's the biggest load before mobs, with a whopping 13.6% of the total server thread run time spent on TileEntityFormulaicAssemblicator.onUpdate(). I don't want to imagine what happens with a large number of assemblicators in a larger modpack (compressed cobblestone farm, anyone?).
I think the recipe search should be skipped altogether when a crafting formula is present and the result should really be cached otherwise.

Fixed in dev.


Step all the way

I think this is all it takes to solve the back-face-disappearing problem!

Correction, it does not seem to be the address mode, because that's clamp by default. Looking into it :)


[Dashboard Screen] - Legend is not displayed on Time Series graph in Full Screen mode

I identified that the legend is not displayed on the Time Series graph in Full Screen mode. In the image below, the legend (data and time) are not visible, when clicking on the graph point (red arrow).
Apparently it’s a bug.
Tested in Chrome browser on Android and Desktop Windows.
Print Chrome Mobile Android (Full Screen Mode)

So far there is no information whether this bug has been fixed.
If there is a correction, it would be very important to correct it.

Hi, @bautisflow
We have identified that this bug has already been reported on github. We hope he doesn't celebrate his 2nd birthday.
It has a huge impact on the user experience (our customers). We hope it will finally be fixed in the next version of Thinger.
If Thinger evolves and improves, the quality of our service will also improve.
Thank you for your attention.
https://community.thinger.io/t/legend-is-not-displayed-on-time-series-graph-in-full-screen-mode/4654/3


Update Telemetry Report Format draft

Fixed a broken reference url.

@jnfoster Not sure of the protocol here, but perhaps you may welcome the idea of working group chairs approving and merging in changes inside their relevant sub-directories of this repository, without requiring your approval?  At least, that is how Calin Cascaval and I have been operating for the PSA specification, also in this repository.

Yes. I think we can even be a bit more liberal: each WG can identify a group of key committers who are permitted to review and merge changes within p4-spec.

Thanks for the notes on the WG spec review policies.
A bigger question to me is whether it's better to host the specs in each WG specific repo.
A few things to consider:

Apps WG is expected to generate more sub-sub-directories, one for each application area.
Application specs are likely to change quite frequently, more frequently than language/arch/API specs.
Application spec or document will accompany with reference code and test program hosted in the separate github repo. It can be confusing where to conduct discussions and track issues btw the p4-spec repo vs. the apps repo.

@mhira1

For issues on language specs vs. PSA so far I have been usually marking PSA-specific issues and commits with "PSA" prefix, just to be able to quickly distinguish them for language-specific issues/PRs.
It seems reasonable to me to introduce a separate repository for P4 applications areas, or even one per application area, if you wish.

Agreed. For that level of complexity, you may want your own sandbox :)

Sounds good. How about publishing to p4.org website? Don't we need to host the material in p4-spec?
For application specs, we can post links to the specs hosted in the apps WG repo.

No you don't need to host it in p4-spec. As long as you either maintain the
current PDFs somewhere accessible, or set up your own GitHub pages and
Travis CI, we can grab the files when we build the website HTML.
On Thu, Nov 30, 2017 at 8:49 PM, JK Lee<EMAIL_ADDRESS>wrote:

Sounds good. How about publishing to p4.org website? Don't we need to
host the material in p4-spec?
For application specs, we can post links to the specs hosted in the apps
WG repo.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/p4lang/p4-spec/pull/490#issuecomment-348378047, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABwi0hiZrBrH5heGhzuf2O1MrPpUizpnks5s71sygaJpZM4QbMAa
.


Fix for #3302

Issue #3302 already had a proposed solution but no pull request to implement it yet.
This implements the proposed change.

Can you add a fallback to self to be sure?

Uh. I actually dont know any cpp. I just implemented that one line change from the issue. I actually dont even use waybar, i just have a non-programmer friend that was bothered by the issue, and doesnt know how to use git/github.


Can't connect to Navidrome

The info is correct, works in my other subsonic app Amperfy.

If I enter the port number twice as shown it seems to connect but everything is empty.

you don't need to include the port in the address field since it is entered in the port field

If I only enter http://<IP_ADDRESS> it doesn't connect and throws me back to the login screen.

Here's a video of the problem and a switch to Amperfy showing it working fine.
https://github.com/halpz/HalpoPlayer/assets/47334163/3f0d508b-0b4d-4b30-b270-c370b06261de

@bbsixzz I'm currently trying to solve this - would you like to test the next build through TestFlight? I'm not able to reproduce this bug myself - DM me your Apple ID if so!

No sweat, I will gladly help I sent you a DM on reddit.

@Normanras no problem - send me your details on reddit!
https://www.reddit.com/user/halpz/

can you try the latest build on GitHub (it's the same as TestFlight)

@Normanras no problem - send me your details on reddit! https://www.reddit.com/user/halpz/

Done! Should be in your messages, @halpz. Thanks!

Same problem with latest testflight.

@bbsixzz hmm strange - would it be possible for me to test it with your server? could you create a temporary user for me to test?

@bbsixzz when you are connecting to your LAN, do you use a password?

Yes I use a password, screenshots and videos hide it in iOS.
I'm afraid I can't enable WAN access to my network.

ok can you try this build - I have enabled alert views whenever there is an error and tell me what you see?
http://paulhalpin.co.uk:8282/api/public/dl/X99q4ETA/downloads/HalpoPlayer.ipa

unless you cannot sign it yourself?

I'll have a look at the build later, what you see in the screen capture is what I see, I don't know if I'm signing in because the app gives no feedback.
You can input www.gooogle.com as address with no password and it will give the same result.
It never says wrong address/password.
Signing in works fine in the browser or using the app Amperfy.

@Normanras feel free try this too if you're having the same problem

I won't be able to test your build for a while but you can do this instead:
Connect to https://demo.navidrome.org (works in Amperfy and Supersonic).
Use login demo and password demo.

I have created a fix for this issue that should be live on the App Store in the next day or so (or build from source)

Just FYI Testflight version is still stuck on 1.0.3

1.0.7 works now Testflight just updated!

hello @bbsixzz is there a testflight link? Thanks!

Send a message to the dev on reddit the info is at the top @ema07cqc


Misc fixes to k8s demo files.

Edited README to add pre-requisites
Invoke scripts with ./ instead of relying on PATH
Modified policy.sh script to handle new netctl syntax.

@jainvipin Please take a look at the updated diff. If looks ok, I will merge.

LGTM @jojimt

@vijayvikrant I guess you commented on the previous version of the diff. I had kept it for backwards compatibility, but removed it now based on feedback from @jainvipin.


Update doc GCS

Retirei tecnologias que não vamos usar e adicionei as de monitoramento que já implementei.

Pull Request Test Coverage Report for Build 543

0 of 0 (NaN%) changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 95.273%

Totals

Change from base Build 540:
0.0%

Covered Lines:
1290

Relevant Lines:
1354

💛  - Coveralls


Interpolated Variables

Consider the following YAML document:
---
a: A
b: $a$ B
c: $b$ C $b$
d: $a$ $b$ $c$ D $a$
e:
  f:
    g: G
h: H and $e.f.g$

I would like to reference those values within the Markdown. Moreover, I would also like to control the syntax whereby said values are referenced. I've written a YAML preprocessor named yamlp that can perform string interpolation.
Multimarkdown, pandoc moustache, and other systems allow authors to embed such constants inside their content. There is also an open issue for pandoc that contains more information about how this works. Rather than repeat the details, please see my comments in that pandoc ticket. In particular:

https://github.com/jgm/pandoc/issues/1950#issue-57878939
https://github.com/jgm/pandoc/issues/1950#issuecomment-427664614
https://github.com/jgm/pandoc/issues/1950#issuecomment-524521696
https://github.com/getgrav/grav/issues/2672#issue-497459142

My blog post describes this concept at length:

https://dave.autonoma.ca/blog/2019/07/06/typesetting-markdown-part-5/

The bottom of the following diagram shows a more general view of how variable interpolation can work by considering a variety of structured document formats as input:

What's needed is a convention that associates Markdown contents with interpolated data sources. This would allow different data sources and/or data formats to be swapped. For example:

https://host.com/wiki/software/architecture/
https://host.com/wiki/configuration/
https://api.host.com/v1/config?export=json

Inside the wiki's configuration would be a setting that allows any data source (either to a local file or remote file) to be associated with one or more pages in the wiki. In this case, any pages underneath "/wiki/software/architecture" could embed references from the hierarchical data returned by the REST call. It would be the wiki server's responsibility to:

Check if any requested wiki page has an associated data source.
Fetch the data source.
Interpolate the strings within the data source.
Replace the references in the wiki page with the fully interpolated string values.
Render the wiki page as HTML for the browser.

Users would then have the option of associating different data sources with different sets of wiki pages.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 29f1e43f-a069-dd88-7fc7-9cc497fb94f0
Version Independent ID: c261afa7-6eb7-b3f0-9f92-6d2f3c5ac336
Content: Markdown syntax for wikis in Azure DevOps - Azure DevOps
Content Source: docs/project/wiki/wiki-markdown-guidance.md
Product: devops
Technology: devops-collab
GitHub Login: @chcomley
Microsoft Alias: chcomley

Hi @DaveJarvis, Thanks for your feedback. I'm assigning this to the product group for further assistance. Thanks again!

Thanks for the suggestion! The product teams aren’t always watching this repository, so to make sure it gets in front of them for triage, please submit your idea here.

@mijacobs -- I don't wish to create a Microsoft Live account, sorry.


Links header looks strange on game with no links

With the possibility of embedding games on LDJAM and source not being strictly mandatory in all categories, it is quite possible to make entries without links and then perhaps the link header should be hidden.
Example:

https://ldjam.com/events/ludum-dare/52/raven-harvest

On a related note, the Links header looks strange full stop. Wouldn't it (and Downloads) look better if they were on par with h2 and Ratings/Metrics?


Strip ~ and @ from modifiers before matching them

Had an issue where a keycombo like ~control + left was registering as just the left key without modifiers. This makes sure to strip any tildes or at signs before matching the modifier to the table.

Thanks for your contribution!
If I understand this properly, this avoids trying to match ~control instead of control as a modifier key?

Yep, that's right!

Polite poke

@ajanon since I'm not well versed with the parser, would you like to merge? The code looks fine to me.


[#2174][#2198][#2200][#2201] small fixes for instructor selection

This fixes #2174.
This fixes #2198.
This fixes #2200.
This fixes #2201.
Don't merge until @maneesha confirms https://github.com/carpentries/amy/issues/2200#issuecomment-1207377326

Warning: there's a migration conflict between this and PR #2213.


fix: Send log messages to STDERR

And only useful outputs from helm commands like ones from helm template to STDOUT.
Fixes #551

May I ask what was the reasoning behind this change? My octopus pipeline displays now useful status information as errors and marks the execution with warnings


URL for Billing Actions 'Not Found'

Request to get GitHub Billing Actions
Based on https://docs.github.com/en/rest/reference/billing#get-github-actions-billing-for-a-user the curl command to request GitHub Actions billing for my user is:
curl  \
 -H "Accept: application/vnd.github.v3+json"   \
https://api.github.com/users/Triantafullenia-Doumani/settings/billing/actions_usage

JSON response
{
  "message": "Not Found",
  "documentation_url": "https://docs.github.com/rest"
}

We tried:

Sponsorship
Pro account

We are still getting 'Not found'. This may not be included in the information we receive through the GitHub Billing API.

With a pro account, we can get these results.
 curl  \
          -i -H "Authorization: token TOKEN" \
          -H "Accept: application/vnd.github.v3+json"   \
           https://api.github.com/users/USERNAME/settings/billing/actions

JSON response
{
  "total_minutes_used": 0,
  "total_paid_minutes_used": 0,
  "included_minutes": 3000,
  "minutes_used_breakdown": {
    "UBUNTU": 0,
    "MACOS": 0,
    "WINDOWS": 0
  }
}


(Bug) detailed request share button name

This PR closes #166353303 and #166275778, by:

Updating button text from "Share QR Code" to "Share Link"
Updating account.js file to avoid updating account when there is no changes in order to fix receive link not opening correctly
Updating components to avoid showing incorrect placeholder values such as 'John Doe' or '0 G$' balances

@dario can you explain what you did and how it solved the issue?


[Bugfix] Use re2 when pattern is larger than 16000

What type of PR is this：

[x] bug
[ ] feature
[ ] enhancement
[ ] refactor
[ ] others

Which issues of this PR fixes ：

Fixed https://github.com/StarRocks/starrocks/issues/10364
We use re2 instead of hyperscan when pattern'size is larger than 16000.

run starrocks_admit_test

run starrocks_admit_test

run starrocks_admit_test

These PR needs to be modified. Don't merge it.

run starrocks_admit_test

run starrocks_admit_test


Add xl and xxl breakpoints

Updated these to match the pixel widths we break on in the consumer application.

Meant to open this against my fork, sorry!


[BUG] split HTML ouput misses the final message in chat

WhatsApp version: <IP_ADDRESS>
OS: Android
Platform: Windows
Branch and version: 0.10.0

Describe the bug
With the "--split" command-line option, the program splits up larger chats. In ALL the generated chats, including ones that are NOT split up, the final message is missing.
To Reproduce
Steps to reproduce the behavior:

run wtsexporter without "--split"
open an HTML chat file and scroll down to see the final message (either from "You" or the contact).
run wtsexporter with "--split"
open the very same chat in the new output folder. Scroll down to see the final message and you will see the last message is not there. This affects ALL output files, not just the ones that were split up.

The issue is in create_html() under the "if maximum_size is not None" section, but I can't seem to figure out what the problem is.

The issue should be fixed in e268484.

I tested this, and it seems to work perfectly now -- thanks! I tested with, and without, the --split option, and for non-split files, their hashes were identical. And on the the split files, from the chats that I at least recognized, seems to also work.

The issue should be fixed in e268484. You can try the dev branch to see if the problem persists or if other problems come out.

Released in 0.10.5.


Clean up scrollable.dart for 2D

Part of the 2D scrolling proposal: flutter.dev/go/2D-Foundation
Fixes https://github.com/flutter/flutter/issues/122355
This relocates some public APIs to another file in order to make some breathing room for TwoDimensionalScrollable.
I also tidied up some getters that were scattered throughout ScrollableState.
This required exposing 2 new getters on ScrollableState

resolvedPhysics

I actually think this is super beneficial to add. The way physics are applied internally can be difficult for users to debug. This makes what the actual physics ended up being way more accessible.

deltaToScrollOrigin

Used by the EdgeDraggingAutoScroller. Simple look up that seems harmless to expose, it is a simple method for computing the axis oriented absolute value if the distance form the 0 position.

Pre-launch Checklist

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I read the Tree Hygiene wiki page, which explains my responsibilities.
[x] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[x] I signed the CLA.
[x] I listed at least one issue that this PR fixes in the description above.
[x] I updated/added relevant documentation (doc comments with ///).
[x] I added new tests to check the change I am making, or this PR is test-exempt.
[x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

I concur! I will move the related tests into a new file in a separate PR to ensure ensure everything still checks out. 👍


Question: Barriers to GHES and DVCS Comparison

Hello all, a couple of quick questions from a GHES user:
What are the current barriers to GHES support (discrete tasks, larger principles, etc)?
What functionality does this app provide that the DVCS connector does not?
Does the DVCS connector have any functionality that this app does not?

Hey @aidan-mundy. As it's been over a month since we've heard from you, I'm going to go ahead and close this issue. Please feel free to reopen/reach out again if you need help with anything.


No development branch?

Looks like the component library doesn't have a development branch, like our other RCPCH projects. While not absolutely necessary, I think it would be good to have to standardise our workflows across projects. @pacharanero if you're happy to add it, we can branch off of the current live.

I have created a development branch and pushed this to origin - for consistency we should now generally make any non-trivial changes in development first then merge into live when ready.


cpType is undefined

Demo page needs an update, Show the dialog permanently: example is old
[cpType]="'fixed'"
new way is using
[cpDialogDisplay]="'inline'"

This repository is unmaintained, it's pointless to post any issues in here.


React form without clicking submit button

Hello,
I can't find any example on how storybook can simulate a click on a button.
For now, I have a react form with a button to submit data. So, I manually click this button to show the validation process in storybook.
What I would like would be that storybook clicks the button when I select the story.
Is it possible ?
Thank you in advance for your help.

@Alphapage I would add a componentDidMount on your react component and on mount trigger the button to click via a ref ie this.buttionRef.click()
Hope that helps!.


add checks for LLAPI to CMakeLists

Running a test with dstripe throws an error:
./install/bin/dstripe /path/to/file/on/lustre
Unexpected Lustre version.
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0

Need to complete our definition for various Lustre compile macros in our CMakeLists.txt:
OPTION(ENABLE_LUSTRE "Enable optimization and features for Lustre" OFF)
IF(ENABLE_LUSTRE)
  ADD_DEFINITIONS(-DLUSTRE_SUPPORT)
  # todo investigate usage of other lustre #defs
  # - LUSTRE_STAT
  # - HAVE_LUSTRE_LUSTREAPI_H
  # - HAVE_LUSTRE_LUSTRE_USER_H
ENDIF(ENABLE_LUSTRE)

Here were the m4 checks, will need to find cmake equivalents:
https://github.com/hpc/mpifileutils/blob/v0.8.1/m4/lx_find_lustre.m4

Thanks @adammoody. Let me see if I’m understanding the m4 correctly:

it checks for lustre/lustre_user.h and lustre/lustreapi.h headers
it adds -llustreapi to the link line
it #defines LUSTRE_SUPPORT
it checks for 3 functions(?) in the lustreapi library and #defines corresponding vars

llapi_layout_alloc => HAVE_LLAPI_LAYOUT
llapi_file_create => HAVE_LLAPI_FILE_CREATE
llapi_file_get_stripe => HAVE_LLAPI_FILE_GET_STRIPE

@gonsie , m4 is always a mystery to me.  I'm interpreting it the same way.  It's possible the m4 macros have some other side effects, but I'm not sure if those are important.  For example, this page says that AC_CHECK_HEADERS defines some macros like HAVE_HEADERNAME:
https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/Generic-Headers.html
And for AC_CHECK_LIB:
https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/Libraries.html

The code that's currently disabled is here:
https://github.com/hpc/mpifileutils/blob/97cabf759e2b4cbf3dad9984154897339e36b574/src/common/mfu_util.c#L861
It requires #ifdef LUSTRE_SUPPORT as a top-level check and then things like#if defined(HAVE_LLAPI_LAYOUT) before calling the various functions.

@gonsie , thanks for taking a look at this!

Fixed with https://github.com/hpc/mpifileutils/pull/239


Thinking about a new ranking system

How could we rank projects in order to show the trending projects?
Could we assign a single score to all projects and show projects sorted by this score?
The Github API gives access to "stargazing" events.
For a given project we can get a list a events that tell us who added a star and when it was.
2 rules I'd like to handle when handling stargazing events:
Time effect
The effect of star gazing events decreases with time.
If someone added the star one year ago, it's far less important that a star added yesterday.

Follower effet

The more followers on Github a user has, the more important the event is.
A user with a lot of followers on Github is considered as an expert.

@michaelrambeau I think ranking can be from very simplistic to a very complicated matter, so it is important to gradually improve it. I think the points you mentioned are top-notch, and definitely valuable for ranking libraries. Thinking from my personal perspective, these are the "features" that I keep an eye on a library and see if it is fit for my project (except for functionalities it offers):

Number of stars and number of forks (this one is obvious I think)
How active are the contributors? (frequency of commits)
How responsive are the contributors? (average duration for an issue to be resolved; ratio of open to closed issues)
How many "core" contributors are there? If there is a single person who commits 95% of the code, there is a higher risk of the library being abandoned.
How many companies have adopted the library (especially the bigger ones)? If companies make a commitment and begin depending on a library, it is more likely that they will put some effort to maintain it.

These are at the top of my mind now, and there are probably some other things that people keep in mind before deciding to adopt a library. Note that all of these are not a prerequisite for a library to be successful or well maintained, but they are nice heuristics for me to make a decision.
As for implementation, for a start, a simple weighted algorithm will probably do well, and as the number of features grows, it might be better to use some of the many machine learning libraries to do the job.

Thank you for your comments @sradevski

How active are the contributors? (frequency of commits)

How responsive are the contributors? (average duration for an issue to be resolved; ratio of open to closed issues)

About the points number 2 and 3, I think it's included in the "rating" provided by https://npms.io/.

Check the About page to read more about npms.io metrics: https://npms.io/about
Among other things, the score includes the following points:

Ratio of open issues vs. total issues
The time it takes to close issues
Most recent commit
Commit frequency
Release frequency

Do you think we should emphasize more the npms.io score? (because its meaning is not obvious!)
Anyway, I agree with your arguments and I'd love to see points like "frequency of commits" in a more visual way.
For example in the block where we display the last commit date, instead of the white background, we could display somehow a graph that shows the commit activity over the last months, does it make sense for you?

@michaelrambeau I had no idea what the percentage means :) I think eventually it would be nice to have some visualization that summarizes some of these metrics. I actually wasn't aware of npms. Looking at it, it seems like it doesn't make much sense for us to implement metrics, but it is more reasonable if there is a metric we want, we make a pull request to npms. We can only take care of presenting the metrics in an easy-to-understand manner.
Going to presenting data, I will try to think of what might be a good way to visualize all the data that we can take from npms. I will let you know if I get some ideas.

@sradevski Thank you for your ideas Steve.
You're right, the displayed percentage does not make too much sense.
Maybe I should just hide it from the page.
Instead of this percentage, I'd rather see the number of dependencies of a given package, right from the list of projects pages, I think it would be more useful.


Language

Question
I want the entities to be in englich but I cant chose that, they be in swedish all the time.
What can i do to change that?
What version of the integration are you using?
V0.9.49
Anything in the logs that might be useful for us?
No response
Additional information
No response

Translation of entity names is controlled by the global language setting (Settings->General->Language).

Thank you, easy when you know where to look :)

I also think it is confusing that there are two different language settings in HA, one global that affects e.g. entity names and one personal (in personal profile) that affects states and attributes. I have not seen any explanation why the core team chose that design.


Deploy to streamlit

I want to deploy this to the streamlit clound for my friends to use. How should I do it? I have now forked the official example and modified the code to the following part, which does not work
from search_data_explorer import SearchDataExplorer
"""
数据可视化
"""
sde = SearchDataExplorer()
sde.open()

i have solved it thanks

Hello @zhaihaojie,
could you share how you solved this issue?

Sorry, I’m only online now and saw this message. When I was solving the problem, I found that when I run the code locally, it can run, but it can’t run on the cloud. Then I checked your writing method and I saw that the open function of the SearchDataExplorer class is executed in the terminal. The following code:
"streamlit run " + dname + "/streamlit_run.py " + arg1 + args + port_cmd

When deploying on the streamlit cloud, it should execute the streamlit_app.py in the code repository instead of the streamlit_run.py in the python module sde. The execution method should be the official recommended execution method
streamlit run streamlit_app.py

So I modified the code and directly imported the code of the app part I needed, as follows
from search_data_explorer.tde_streamlit import app
import streamlit as st
import pandas as pd
app(search_data=None, plots=None)

After that, I also encountered some problems. For example, when I downloaded the dependencies required by the module search_data_explorer, it missed two libraries hiplot and seaborn, which caused the runtime to prompt that these modules were missing. After I downloaded the environment dependencies, I exported them to requirements.txt and pushed them again. repository after restarting the streamlit app then it works fine
Finally, thank you very much for this small tool you wrote and the algorithm library gfo you wrote. I use this algorithm library to optimize the parameters of the strategy written by the backtrader library. The effect is much better than using option. At present, I hope it can support multi-process operation. I am currently using multiprocessing to work with multi-processes and I am quite satisfied with the speed, but I feel that there is still room for improvement, but thank you very much for this algorithm library

Hello @zhaihaojie,
thank you for this detailed guide to your solution! :-)
But be aware that:
from search_data_explorer.tde_streamlit import app

is not part of the public API, which means that it might change in future versions. In those cases it is advised to fix the version you are using to the current one (v0.4.0). This avoids that you get breaking changes from newer versions.
It is nice to see, that you are using my GFO library as well. If you want to utilize multiprocessing for your optimization runs you can use my other package called Hyperactive. It was originally designed to tune hyperparameters, but can be used for a large variety of problems. It is build on top of GFO, has a very similar API but has more features. (GFO is basically a optimization backend)


An issue while downloading

Traceback (most recent call last):
File "D:\couresera_videos_sample\coursera-download.py", line 197, in
browser.find_element_by_xpath("(//nav)[2]/button").click()
File "C:\Users*\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 394, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "C:\Users*\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
'value': value})['value']
File "C:\Users*\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "C:\Users*\AppData\Local\Programs\Python\Python37-32\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: (//nav)[2]/button
//I am new to things like this.If more info needed kindly intimate me,I'll definitely provide them.

This could be due to the HTML elements in Coursera website have been changed, I will update the script this weekend.

This could be due to some HTML elements in Coursera website have been changed, I will update the script this weekend.

Updated and tested the script, it should be working fine now. If you have WebDriverException issue, please download the latest geckodriver. I will close this issue for now, feel free to reopen if the problem persists.


HDDS-3065. Ozone Filesystem should return real default replication

What changes were proposed in this pull request?
Make BasicOzoneFileSystem#getDefaultReplication() return the configured replication factor instead of 1.
https://issues.apache.org/jira/browse/HDDS-3065
How was this patch tested?
Added unit test for default and custom config cases.
https://github.com/adoroszlai/hadoop-ozone/runs/465304016

Thanks @xiaoyuyao for reviewing and committing it.


TFLite GPU Delegate on iOS: failed assertion `Cannot create a buffer of zero length.

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): iPAD OS 13.2.2
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: iPAD Pro 2018
TensorFlow installed from (source or binary): Binary (TfLiteGPUExperimental)
TensorFlow version (use command below):
Python version: NA
Bazel version (if compiling from source): NA
GCC/Compiler version (if compiling from source): NA
CUDA/cuDNN version: NA
GPU model and memory: iPAD GPU

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with: 1. TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)" 2. TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"
Describe the current behavior
When running the model on iPAD using CPU, we are able to get the output. But when doing GPU delegate, we get the error the following error:
failed assertion `Cannot create a buffer of zero length.
Describe the expected behavior
No error
Code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
delegate = NewGpuDelegate(nullptr);
interpreter->ModifyGraphWithDelegate(delegate);
Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

Hi Folks, any update on this one. Thanks.

Were you trying to run your own app? Which model were you using?

@yyoon I was using 'palm_detection.tflite' model on the following web link --> https://github.com/google/mediapipe/tree/master/mediapipe/models
Please let me know if you need any additional information.

And which exact app were you using for running that model?
Was it your own written app, or did you try to modify one of the existing example apps?

@yyoon We were trying to use tflite model on iOS using the following code.
tflite::InterpreterBuilder(*model, resolver)(&interpreter);
GpuDelegateOptions options;
options.allow_precision_loss = true;
options.wait_type = GpuDelegateOptions :: WaitType::kActive;
delegate = NewGpuDelegate(nullptr);
interpreter->ModifyGraphWithDelegate(delegate);
....

The code fails on the last  line 'interpreter->ModifyGraphWithDelegate(delegate);' above. Please let me know if you need any more information. Thanks.

We can close this issue.

We can close this issue.


Dropdown - don't stop event propagation

Fixes #30267
Preview: https://deploy-preview-32943--twbs-bootstrap.netlify.app/

We also have #32625, /CC @rohit2sharma95

It is about another issue that is when the dropdown toggle button is clicked, the (click) event is not bubbling up because of the stopPropagation.

Does this also fix https://github.com/twbs/bootstrap/issues/30882 ?

@sijusamson can you add a test please?

@sijusamson could this possibly fix #33044? If so, can you add another test case covering that too please?

@sijusamson could this possibly fix #33044? If so, can you add another test case covering that too please?

Not sure. I will have a look.
And I am so sorry for reverting too late :-(

Hmm, this seems to break at least our docs version dropdown

Hmm, this seems to break at least our docs version dropdown

Great catch. :100:

@sijusamson can you please add an extra test so that the previous regression is caught? Because I only caught the issue manually.

It does not fix #33044 though

@sijusamson can you have a look in case you can also fix #33044 too here?

@sijusamson can you have a look in case you can also fix #33044 too here?

Sure !

Actually wait, it seems #32942 might be a cleaner solution :)

Actually wait, it seems #32942 might be a cleaner solution :)

Okay.

Closing this PR as #32942 appears to be a better solution.

by default there should be no blocking of events in the use of dropdown box; but if you want to add it programmatically it should be possible to insert a block of events.
What would happen if you want to track where the user navigates through the dropdown boxes and the events are blocked ...


Project Status

When i fill out a create a project form, the project status asks if the project is active of completed. I don't necessarily have a solution, but i am wondering how we can prevent the following situation from occurring -
User enters project while it is ongoing, it is entered as an active project, and the user forgets to go in to change the status to completed.

How do we prevent this?
What is the impact of this?

This is a duplicate of #836. Closing.


[msal-angular] Add additional interfaces to msal-guard

This PR:

Adds canActivateChild to the guard

Coverage increased (+9.2%) to 91.489% when pulling 70888a7b0dde9d2d7fa9e344ad21ad9de6bae39a on angular-guard-interfaces into e7d3c46fb326e56832fb06b8b67f8d23a0a13287 on dev.


Add ability to cache scopes

This adds the ability in our metricsClient to cache scopes. We have seen that scope construction contributes significantly to GC time.
This code turns on caching for all metrics scopes with no config to disable. This was done this way as a first pass because it produces the simplest API, we should discuss further if we need to enable or disable this feature in different places in code.

We actually don't need to cache scopes we just needed to reduce duplication of domain records on read.


bugfix databasedump.sh: create destination folder

It's indeed a bug.
I you invoke the databasedump.sh with e.g. the current time as the backups destination directory name, you want to let the dump-tool create the folder for you, if nonexistent.
mkdir -p could be also invoked before the for-loop, because the "root" destination folder stays the same.
Creating/Checking mkdir -p before each backup might be safer, because the directory could be deleted while performing the backups...
However the decision, the command shouldn't affect performance at all ^^

the directory could be deleted while performing the backups...

I don't want to see a system, where this might happen…
Seriously, you need to set it up manually anyway, so you should be able to assure the dir exists.
But given the tool is now refactored so flexible allowing you to specify paths on the command-line etc. you might forget it.
So yeah, it won't hurt…


Skillmap - referenced block unavailable in "Learn MakeCode" tutorial step 9

In the "Learn MakeCode" activity, step 9, it says "Try: Snap a [mySprite] say [" "] block ...", but this block does not appear in the toolbox. "ask" block is also unavailable.

Blocks missing. Important to fix.

Blocks missing. Important to fix.

I am assuming this is code bug and not content. If it is content bug assign to @kiki-lee

I am assuming this is code bug and not content. If it is content bug assign to @kiki-lee

@kiki-lee i think the hint for step 9 in the interface tutorial needs a  let mySprite: Sprite = null; before the controller.A.onEvent

@kiki-lee i think the hint for step 9 in the interface tutorial needs a  let mySprite: Sprite = null; before the controller.A.onEvent

The missing block was added to the previous update and I've just pushed an update that fixes the block in the hints. Thanks for taking the time to report!

The missing block was added to the previous update and I've just pushed an update that fixes the block in the hints. Thanks for taking the time to report!


[WIP] Update EC2 actions for ECS

Fixes #436
Depends on https://github.com/weaveworks/scope/pull/2033

LGTM, based on list of permissions in https://github.com/weaveworks/scope/pull/2026, but see my comment https://github.com/weaveworks/scope/pull/2026#issuecomment-263591270.


Pools

welcome pools
added tons of updates

I am not able to fix the npm audit because there is no new version. It is just related to the building process. Problematic packages are not used in production.


[BUG]

Library name and version
Azure.AI.FormRecognizer <IP_ADDRESS>
Describe the bug
The Prebuilt Invoices Document Intelligence Service is no longer working when using the Region germanywestcentral.
Expected behavior
Worked with the Region germanywestcentral up to now without Problems, now no Result
Actual behavior
No result, analysis runing forever
Reproduction Steps
Create a Document Intelligence Studio Ressource with the Region germanywestcentral and open the Document Intelligence Studio.
Try to run the Invoices Prebuilt. Analysing the provided demo Documents takes forever without result.
Environment
WEB and IDE (Visual Studio 2019, .NET Framework 4.8)

Hi @StefanMaehr:  If I understand correctly, you're seeing the same behavior with Document Intelligence Studio as you are with the Azure SDK package.   That would indicate that the behavior you're inquiring about is with the service directly and outside of the client's insight or influence.
Because the inquiry is related to service behavior, the maintainers of the Azure SDK package are unable to assist.  Azure services are not managed on GitHub, so we're unable to transfer this on your behalf.  To ensure that the right team has visibility and can help, your best path forward for would be to open an Azure support request or inquire on the Microsoft Q&A site.   For feature suggestions, you may also want to consider the Azure Feedback site.
I'm going to close this out; if I've misunderstood what you're describing, please let us know in a comment and we'd be happy to assist as we're able.


FEAT: Logout Confirmation Popup

Related Issue: #200
Changes:

Added Logout Confirmation Popup:

When the user clicks the logout button, a confirmation popup appears with the message:

"Do you really want to log out?"

The popup includes two buttons:

Yes: Proceeds with logging out.
No: Cancels the action, keeping the user logged in.

This addition helps prevent accidental logouts and improves the overall usability of the site.

Request:

Tags: Please add the appropriate tags to this PR.
Formatting: I noticed that running npm run format with Prettier made changes to more files than expected. Please remind other contributors to run Prettier before submitting their PRs to avoid this issue in the future.

Example ScreenShots:

conflicts, fix it... and a question!! why are there changes in 29 files for the confirmation card?? @tejasbenibagde

I already answered why there are many changes. Did you not read the description that I've provided?

I'll fix conflicts as soon as possible.


Nesting When statements in Orquesta

Right now when writing check logic in orquesta I have to have a when block for each conditional.  It's not graceful and harder to manage.
For ex:
Instead of ,
When ( succeeded)
      When ( x=true)
           Do thing
       When (y= true)
            Do thang

We have to do
When ( succeeded and x=true)
   do thing
When (succeeded and y = true)
   do thang

It's not clean and harder to read/maintain. Especially in some of our workflows the conditionals are large and you end up having many statements with repeated keywords in them.

Here is a snippet of our Orquesta code that is already in production:
- when: <% succeeded() and (ctx().merge_upstream and ctx().mis_repo_base = openstack) %>
  publish:
    - shas: <% result().output.shas %>
  do:
    - merge_upstreams
- when: <% succeeded() and (not ctx().merge_upstream or ctx().mis_repo_base = stackstorm) %>
  publish:
    - shas: <% result().output.shas %>
  do:
    - teardown

That could be simplified down to:
- when: <% succeeded() %>
  publish:
    - shas: <% result().output.shas %>
  and:
    - when: <% ctx().merge_upstream and ctx().mis_repo_base = openstack %>
      do:
        - merge_upstreams
    - when: <% not ctx().merge_upstream or ctx().mis_repo_base = stackstorm %>
      do:
        - teardown

Instead of nesting conditions linearly within parentheses, we're nesting conditionals in YAML using, and YAML is a syntax that was built on using indentation to nest elements. We would be utilizing YAML for exactly what it was originally intended for.
And then when our customers ask us "why use Orquesta instead of Mistral?" we can have a specific answer for them "nested conditionals in task transitions"!
And just to drive the point home further, it would be difficult for Mistral to implement a similar syntax. So not only would Orquesta be getting a feature that Mistral doesn't currently have, it would be getting a feature that Mistral could pretty much never have.
However, I don't really know why succeeded and failed keywords would be more useful than succeeded() and failed() functions. If we kept them functions, it becomes much more obvious that they are the results of some sort of execution. If we switch to (or additionally allow the use of) keywords, then it's not as readily apparent that the keyword means a result that is calculated on the fly. It would also be inconsistent with other functions that also really make more sense as functions: ctx(), item(...), result(), task(...).

@dzimine @bigmstone Any feedback for us on the proposed design above?

I'm not sure how I feel about and syntax. and seems like it's wanting all conditions to match to me. Could we next next again? Confusing?

Also, as an alternative, user can also do the following right now where task1 publish and transition on succeeded to an empty task2.  Task2 (no action defined) branches out with other publishes and transitions.
task1:
  action: ...
  next:
    - when: <% succeeded() %>
      publish:
        - shas: <% result().output.shas %>
      do: task2
task2:
  next:
    - when: <% ctx().merge_upstream and ctx().mis_repo_base = openstack %>
      publish: ...
      do:
        - merge_upstreams
    - when: <% not ctx().merge_upstream or ctx().mis_repo_base = stackstorm %>
      publish: ...
      do:
        - teardown

As per @m4dcoder suggestion, I am adding my 2 cents to publishing vars. Instead of having them as part of the transition model, I'd have it as part of the task model. So that once action is complete, regardless of whether it's success or failure, evaluate publish expressions, which then could be available to transition conditions. It'd make it simple to write workflows for everyone.

@emptywee The issue here is that result of action execution can be different for success and failure. What if I want to publish x on success (only in the result of successful execution) and y on failure (only in the result of a failure execution)?  In this case, the workflow engine will try to process the publish for x and y in will result in an expression failure in either case.

@m4dcoder I don't know about those hypothetical scenarios where you'd want to publish different variables, but in my experience I've rarely needed it (if at all, to my recollection). I've been able to write pretty complex workflows with just using "publish" attribute of the mistral engine, dealing with all possible success/failure transitioning and variables publishing. However, I see many ways to deal with such scenarios with a single publish attribute in the task model. E.g.
task1:
  action: example.randomly_fail
  publish:
    var_on_success: <% succeeded() and result().output.var1 %>
    var_on_failure: <% not succeeded() and result().output.var2 %>

Then it's up to you which variable to use later on.

@emptywee You are not doing this doesn't mean it is not done by other users though.  It's common to publish certain variables on success but then publish stdout/err on failure to be logged or recorded. As for your example, it doesn't work because you are messing condition with the expression that returns the value for the variable. In your example, var_on_success/failure will have value of True or False.

@m4dcoder well, I expected this kind of response, thus was a bit hesitant to ignite this sort of debate :) Of course, if I needed to get rid of True/False, I'd make it:
task1:
  action: example.randomly_fail
  publish:
    var_on_success: <% succeeded() and result().output.var1 or null %>
    var_on_failure: <% not succeeded() and result().output.var2 or null %>

I know the output I am gonna get in my expressions, and I'd account for that if I needed them to be very precise. For the sake of example I provided a very simple one.
It's just, I don't know what's cheaper...:
a. have it split separately in different whens and not being able to use published vars in when conditions, adding nested dummy tasks for additional transitions, or;
b. write appropriate expressions, define variables, which can be used in when conditions, basically reusing the code (which sometimes could be very complex queries from resulting data), save tens of lines.
Perhaps, as a compromise, allow publishing in the task model and additional publishing in the transition model? I don't know if it's possible, though, from the Orquesta engine implementation standpoint of view.

This type of design discussion is needed to understand requirements and evaluate solution proposals.  The person who is going to implement this feature will be able to look at the dialogs here to understand how the decision was made. If we have this conversation in Slack, it will be lost.
I tested the YAQL expressions and it works to my surprise. This also needs to be tested for Jinja since we support both YAQL and Jinja.
IMO, the YAQL expressions are very ambiguous and I have not formed an opinion on this proposal yet.

@m4dcoder I like YAQL, it allows to be pretty flexible when needed. Wish it could also let your handle exceptions :)
Regardless, could you consider the compromise suggested? Let users publish at two stages: 1. after action is complete; 2. in the transition part (inside when clauses). I think it'd satisfy all parties.

I don't have an opinion on your proposed solution atm.  There is a workaround for the problem you stated.  I am in no rush to make a decision on this especially introducing a new language element which I cannot rescind once I introduced it.

Well, the compromise where users would be able to publish in two places introduces no changes to YAQL/Jinja processing. Only that if publish: is defined within the task itself, run the expressions and update the context, then move on as it was before. Sounds simple, but I, of course, don't know the details of the engine implementation.

P.S. As much as you don't want to rush make changes, I also don't want to use lengthy and bulky workarounds just to later re-do it the right way (if additional publish is added eventually). If you'd like users to migrate smoother to Orquesta, this little feature is a must have.
P.P.S. Also retry-delay-count feature is long-awaited here :)

Expressions in the next block can be very repetitive. The same condition ends up getting repeated in several when and publish expressions.
So, I end up splitting many tasks into two. The first publishes the results in more convenient (intermediate) variables, and the second serves as a fork/decision node to evaluate those newly published (intermediate) vars, and the publish the vars that the rest of the workflow needs before continuing to the next set of tasks.
One issue with this workaround is that it pollutes the context with variables that I only need for the next (noop) task. I would like to avoid increasing the size of the context wherever I can.
The nested-when block has a similar issue. So, using the next block format from blag's example (https://github.com/StackStorm/orquesta/issues/116#issuecomment-465429738) mixed with vars of some of our workflows:
next:
  - when: <% succeeded() %>
    publish:
      - disk_space_usage: <% int(result().get(ctx().server_name).stdout.replace('%','')) %>
    and:
      - when: <% ctx().disk_space_usage >= ctx().disk_space_threshold and ctx().use_chatops %>
        publish:
          - full_msg: "{{ ctx().disk_space_usage }}% used ({{ 100 - ctx().disk_space_usage }}% free)"
        do:
          - send_chatops_confirm_disk_usage_too_high
      - when: <% ctx().disk_space_usage >= ctx().disk_space_threshold %>
        do:
          - clean_up_logs
          - restart_flaky_service

If I never need the disk_space_usage var outside of this next[].and block, then we have to deal with an ever increasing number of short-lived vars that get added to the context. For disk_space_usage, an int, that size is not very concerning. Looping over a subworkflow for a list of hosts, and then extracting data from the return dictionaries about each host can be significant, and can require repeatedly accessing nested (result().result.select(...).a.b.c.d.e.f) information.
The mistral-like publish-first approach described by Igor (https://github.com/StackStorm/orquesta/issues/116#issuecomment-499291613) also suffers the same shortcoming; publishing a var that is meant to be used in the current task's when condition even though that var might not be used later.
task1:
  action: example.randomly_fail
  publish:
    var_on_success: <% succeeded() and result().output.var1 or null %>
    var_on_failure: <% not succeeded() and result().output.var2 or null %>

So, I would like to see a way to not only create variables for use in next[].when, but also scope those variables to the current task and NOT publish them to the workflow context.
What if we added a way for workflow authors to "override" what the result() function returns? Or, make a new function that returns these task-scoped vars?
This could work with either approach: (a) the nested-when approach, and (b) the publish-before-next approach.
Here are the above examples using a new output() function with a new output: block in each of the above examples:
(a) nested when
  next:
    - when: <% succeeded() %>
      output:
        - disk_space_usage: <% int(result().get(ctx().server_name).stdout.replace('%','')) %>
      and:
        - when: <% output().disk_space_usage >= ctx().disk_space_threshold and ctx().use_chatops %>
          publish:
            - full_msg: "{{ output().disk_space_usage }}% used ({{ 100 - output().disk_space_usage }}% free)"
          do:
            - send_chatops_confirm_disk_usage_too_high
      - when: <% output().disk_space_usage >= ctx().disk_space_threshold %>
        publish:
          - disk_space_usage: <% output().disk_space_usage %>
        do:
          - clean_up_logs
          - restart_flaky_service

(b) define the variables before the next block
task1:
  action: example.foobar
  input:
    param: abcdef
  output:
    var_on_success: <% succeeded() and result().output.var1 or null %>
    var_on_failure: <% not succeeded() and result().output.var2 or null %>
  next:
    - when: <% succeeded() and output().var_on_success = "qwerty" %>
      do: ...
    - when: <% succeeded() and output().var_on_success = "dvorak" %>
      do: ...

Instead of key task.output + func output(), it could also be key task.result_override and reuse result(). Whatever the name, I'd like a way to create task-scoped vars that can be used in task.next[].when and task.next[].publish.

I think there are two separate problems communicated here.

Current task transition with next/when/publish/do is very repetitive if there are more than one task transition.
There is a need to define variables within the scope for the current task that can be used for defining task transition but these variables will not be published into the workflow context.

The current semantic for when allows user to define what is success or failure. For example in some use cases, an action execution may complete (return success in StackStorm) but the output of the execution contains status code and or message that indicates whether the action actually accomplishes what it intended to do. The term on-success and on-failure does not allow for these use cases. Therefore I prefer we keep iterating on the approach with the nested when.

This issue has been brought up again in the community Slack channel. Please note that there is a workaround at https://github.com/StackStorm/orquesta/issues/116#issuecomment-478713066 which uses intermediate tasks.
So far, the Orquesta workflow definition language has avoided hardcoding on-success and on-error. I like to keep it that way. There are use cases where tasks completed successfully but the returned status code or the output data determines the state.
So let's revisit a similar proposal from above and expand this here with scoped publish. The proposal is to introduce a new keyword then (mutually exclusive to do) which allows for further branching using a list of task transition block (when->publish->do|then).  The variables that are published in a task transition block will be available in the branches that follow.
Let's take the following workflow as an example, with branches S, F, SX, SY, and SYZ where SYZ means it falls under the umbrella of S and SY.
tasks:
  task1:
    ...
    next:
      # branch S
      - when: <% succeeded() %>
        publish:
          - var1: blah
        then:
          # branch SX
          - when: <% ctx().x = true %>
            publish:
              - var2: blahblah
            do: task2
          # branch SY
          - when: <% ctx().y = true %>
            publish:
              - var3: blahblahblah
            then:
              # branch SYZ
              - when: <ctx().z = true %>
                do: task3
              ...
      # branch F
      - when: <% failed() %>
        publish:
          - var99: mehhhh
        do: notify

In the example above, var1 is published in S and available in SX, SY, and SYZ.  The variable var2 is published in SX but since SY and SYZ is not under SX, var2 is not available in those branches. Branch F publishes var99 but it is not available in the scope of the branches under branch S.
S publishes var1
  - SX (context contains var1) publishes var2
  - SY (context contains var1) publishes var3
    - SYZ (context contains var1, var3)
F publishes var99

As for local variables not to be published into the workflow context, how about prefixing the variable with an underscore and then reference with the ctx() function like any other variables?  Any variables prefix with the underscores will be remove from the context on exiting the task evaluation.
tasks:
  task1:
    ...
    next:
      - when: <% succeeded() %>
        publish:
          - _var10: fee fi
          - _var20: fo fum
        then:
          - when: <% ctx().x = true %>
            publish:
              - poem: <% ctx()._var10 + " " + ctx()._var20 %>
            do: task2
          - when: <% ctx().y = true %>
            do: task3
           ...

I like then: better than and: for nesting conditions.
Using an underscore prefix seems surprising at first, but I see how it prevents adding a new keyword to the workflow syntax. I like that there's only one place to define those vars, and the underscore is a nice scope indicator.

I like the then transition syntax, as it looks fairly clean and it would reuse chunks of the existing schema definition.
I assume that most Python developers will be very familiar with the leading-underscore-meaning-something-different convention, because that's a convention in Python. AIUI that is not a formal restriction of Python interpreters, it is merely a convention respected by most Python developers (but not all).
However, I do not like the leading-underscore part of the then proposal though.
Reasons:

I don't think that's a common convention outside of Python
The Orquesta workflow syntax isn't Python and is entirely orthogonal to Python syntax
This convention is not used anywhere else in Orquesta
We should not assume that workflow authors know Python, or know Python well enough to be familiar with that convention
We should not require workflow authors to know Python conventions
For users who do not know this convention, it can be difficult to come up with search terms that lead to a solution

Pardon my ignorance here, but I don't understand the reservation against adding keywords to the Orquesta syntax. The syntax is, as far as I can tell, very good at separating user-defined names (like published variable names and values) from syntactical keywords (like tasks, next, when, publish, do, and now then and also disallowing additional arbitrary keys (read: uses additionalProperties: false everywhere in the schema definition). So the usual reason to avoid adding a keyword - that new keywords might conflict with existing variable names in existing workflow definitions and break those existing workflows - probably isn't an issue for us.
On the contrary - AFAICT that is an additional restriction on context variable names that did not exist before. So existing workflows that already define variables prefixed with underscores will break under the current then proposal when they are removed from downstream task contexts. A workaround would be to only remove those variables from downstream contexts in then/when/do transitions, but keep them in downstream contexts in when/do transitions. That inconsistency might confuse new users who either (A) expect underscore-prefixed context variables to be kept in then/when/do transitions, (B) expect underscore-prefixed context variables to be removed in when/do transitions, or (C) both. And I could see (B) leading to a user security issue if, for example, they use a secret in a transition and expect it to be removed from downstream contexts. So that workaround is not without its own set of issues.
A different way to handle variable scopes would be to extend the publish item syntax from being a list of single-item dictionaries to being a list of single-item or multiple-item dictionaries:
tasks:
  task1:
    ...
    next:
      - when: <% succeeded() %>
        publish:
          # current, "short-form" syntax; still available in downstream contexts as the
          #  implicit "scope" defaults to "task" (see below)
          - _var10: fee fi

          # long-form syntax
          # equivalent to ^^ syntax
          # allows users to explicitly set variable scope
          # publishing a variable with the same name more than once would not be allowed
          #  in practice, this is just to show an equivalent)
          - name: _var10
            value: fee fi
            scope: task  # set to "task" to include in downstream task variable contexts

          # long-form syntax
          # allows users to scope variable down to `then` transition
          - name: _var20
            value: fo fum
            scope: transition  # set to "transition" to remove from downstream task variable contexts
        then:
          ...

There would be no other acceptable values for the scope key, and the long-form syntax would require all three name, value, and scope keys.
This sub-proposal has the benefits of not introducing any additional keywords to indicate transition variable scope, avoids the underscore-prefix restriction from the previous proposal, it should be additionally backwards compatible with existing workflows since it only extends the existing syntax, it makes it absolutely explicit when a variable will not be published to downstream task contexts, and it also gives us a way potential mechanism to extend the syntax if we would like to allow any additional variable scopes or other variable handling tricks in the future.
The downsides to this option are that introduces an additional mechanism to publish variables and it forces users to write a lot of boilerplate when they need to specify scope: transition.
Technically, the long-form syntax for scope: task isn't necessary, so that part could be omitted if need be.
I apologize if any if this is inaccurate (please feel free to correct me or simply ignore if so). It's been awhile since I was involved in StackStorm development and I'm not as up-to-speed on it as I used to be.

publishing a variable with the same name more than once would not be allowed

Publishing the same var multiple times is a feature of the list of single item dicts. I purposely publish the same var multiple times in the same publish block. This allows me to break a long yaql or jinja expression into a series of smaller more understandable expressions.
The added verbosity of name, value, scope would make those expressions less readable I think.
An _ prefix keeps the published value expressions easy to read, but is problematic because _ is a valid character in var names. Valid characters include letters, numbers, and _:
https://github.com/StackStorm/orquesta/blob/c548e49bc649dbfb8f9f92b391d2359efaa9e943/docs/source/schemas/orquesta.json#L280
What if we used something that is NOT a valid var name character (and not a special yaml char)? Here are several possibilities (we should only use one).
        publish:
          - _var10: fee fi  # _ is a valid var name char so no special meaning
          - ~var20: fo fum # ~ is not a valid var name char. publish ~var_name access ctx().var_name
          - %var30: I smell the # publish %var_name access ctx().var_name
          - <var40>: blood of # or surround the name instead of prefixing it. publish <var_name> access ctx().var_name
          - local:var30: an englishman # publish scope:var_name access ctx().var_name

In light of @blag comment, I believe something like local().var_name would work for everyone. It'd  be in line with global ctx().var_name style/approach.

@copart-jafloyd I definitely agree - the long-form is definitely verbose and less readable. The unique variable name restriction is not important to my overall sub-proposal, so keeping that functionality working for you is totally fine, I'm just curious how/why you actually use that. Can you give an example of how you publish the same variable multiple times?
I'm glad we're talking about new options, especially the point about using a non-allowed character as a sigil but I think anything that uses a symbol to indicate scope is going to confuse users and likely be difficult to search for.
Out of those options, I like the local:var_name pattern the best. It gives a clear term (local) and a clear context ("Orquesta workflow transition") when searching for help, and it shouldn't break existing workflows because the colon character is not a valid character in variable names.
Along that train of thought, trying to think outside the box a little bit here: the term "local" might not make it exactly clear as to what is considered "local". Local to the task being run? Local to the publish block? Local to the workflow? Perhaps a better prefix might be something like scope:transition:var_name and (implicitly) scope:task:var_name? Or maybe just transition:var_name?
@emptywee I'm not sure I like the alternative local().var_name proposal. I think if we're going to have a separate way to reference variables, then we should have a corresponding and completely separate way to publish variables.
Example:
        publish:
          - var1: fee fi
        local:
          - var2: fo fum
        then:
          - when: <% local().var2 and ctx().var1 %>

@blag, yeah, right, we're talking about declaration here, my bad.
As to using : in variables declaration I believe yaml wouldn't be too happy about it, unless you wrap it in quotes, which may look ugly.

Re YAML, you only have to wrap the : in quotes if there is a space after it. So, local: var_name would be a non-starter, but local:var_name should be fine.
I don't think transition makes sense for the scope. A "transition" scope sounds like it would be a variable that becomes available for any subsequent task. Maybe we could call it the scope:then or scope:next. I guess the docs do use the term "Task Transition Model", so there is precedence for "transition", but I always called it a "next block".
Whatever the scope's name is, it will need to be documented. Borrowing a term or a symbol from a programming language is fine, as long as the term's definition, within the linguistic-context of orquesta, is documented.
PS: @copart-jafloyd is me. I just posted the earlier comment from the wrong account. oops.


Add transaction token to sessions

Can you add a description? What are you planning to use this for? What can other people use it for?


Add support for custom controls

Hey @skadecl thanks for the contribution! Can you please provide some description behind the why and what?


Run prettier on all code

Run the command yarn format and all files will be changed to the configured prettier style.
This is important so moving forward we only see code changes in Pull Requests and not format changes.

git commit -a -m "yarn format"
git push
see https://github.com/olavea/unlockopen.com/pull/3


Some tests in GitContextTest are not testing functionality properly

The tests from #161 should be properly testing the fetchIfNeeded function and also the fetch functionality.
Actual behavior
@Test
  public void testFetchIfNeededRecentFetchHead(@TempDir Path tempDir) throws IOException {
    // arrange
    String repoUrl = "https://github.com/test";
    String remoteName = "origin";
    List<String> errors = new ArrayList<>();
    List<String> outs = new ArrayList<>();
    IdeContext context = newGitContext(tempDir);
    GitContext gitContext = new GitContextImpl(context);

    Path gitDir = tempDir.resolve(".git");
    Files.createDirectories(gitDir);
    Path fetchHead = gitDir.resolve("FETCH_HEAD");
    Files.createFile(fetchHead);
    Files.setLastModifiedTime(fetchHead, FileTime.fromMillis(System.currentTimeMillis()));

    // act
    gitContext.fetchIfNeeded(repoUrl, remoteName, tempDir);

    // assert
    // Since FETCH_HEAD is recent, no fetch should occur
    assertThat(errors.isEmpty()).isTrue();
    assertThat(outs.isEmpty()).isTrue();
  }

This does not really test anything and should be rewritten such that the functionallity is tested properly.

For this test mockito could be really helpful:
We could create ProcessContext as mock and test if it was actually invoked to call git or not.
Then we could test the case where the timestamp is old and git pull shall be triggered as well as a test where the git call is omitted since the timestamp is recent.
See also #581 that fixed a bug in this context. Such test would have revealed it.

This issue was already fixed with #585.


Bright tile 22889 with clustered sources around z=3.0-3.5

Bright tile 22889 observed on 20220314 shows a bunch of clustered sources around z=3.0-3.5 on petal 4.

This is a 9th mag star a half arcsecond from a stuck fiber, fiber 2366.  This tile was designed before Dustin's tweaks went in, but the tweaked center would have been different.  So everything is working as expected here.  My impression is that if we masked this star's ~fiber bundle then everything would be fine, but I don't know if we have a device to do that.  Otherwise I guess we have to mask the whole petal.
@akremin , @julienguy , any sense for what the best option is here?

Julien advocates for declaring amps B, D on b4, r4, z4.  Then we need to reprocess and reQA.

This has been reprocessed and the problematic amped marked bad.  I've marked as good in QA; closing.


Here is the file: https://www.dropbox.com/s/c2fzmzykaudpra9/out_hap.avi?dl=0
I made the file out from mov using the following command:
ffmpeg -i out_hap.mov  -vcodec copy out_hap.avi
Could you confirm such behavior in your case?

Should I re-encode?

Will investigate now - just cloning the repo on my laptop...

Interesting - you seem to have uncovered a bug in my AVI parsing. It's not immediately clear what it is about your file that is different, but I'll figure it out in the morning and provide a fix.
Also we should throw an exception here, not hang vvvv indefinitely.

Nice, keep me updated on this issue!


Shortcuts not updated when --packTitle or application icon changes

Icon and shortcut name not updated when channel is swapped. The shortcut will continue to show the --packTitle and icon of the original version installed.

The icon comes from your application executable automatically, I suspect if you restart your computer the icon will be updated. It's true that we don't update shortcuts automatically at any point, only install them in Setup.exe and remove them during uninstall.

I have tried restarting and sadly it hasn't updated. If it something manual I need to do on channel swap I don't mind. Is the code for the shortcut creation in the repo and I'll adapt it to my needs?

https://docs.velopack.io/integrating/shortcuts

Ah easy, cheers for the helps.