Ecovacs Deebot is always unreachable after configured timeout

Describe the bug
The ecovacs-deebot.<number>.info.connection state of my Deebot is updated only when it changes. There is another state called ecovacs-deebot.<number>.info.connectionUptime that is updated once a minute, even if .connection stays the same.
Since this adapter monitors connection only, Deebots will turn to offline devices after the configured timeout (10 minutes in my case).
Versions:

Adapter version: 2.12.1
JS-Controller version: 5.0.19
Node version: 18
Operating system: docker

I'm hoping all ecovacs devices will have this datapoint. Then it will be fixed with next version.


Warn on unused variables

Ignores one fewer millet error.

@azdavis Thanks! I'll put some of the ellipses back, probably, just in case I expand any of those ellipses in the future, but otherwise it all looks good to me.

(actually, looks like I did some weird Git stuff, so the changes are already in)


Adding land-worker command

Hey @arwineap what's your use case for wanting this in fly?
If you are using the BOSH release, the worker's drain script will automatically land the worker.
If you are using the binaries, the binaries have the land-worker and retire-worker commands. In general, we would expect that it's the responsibility of the worker to register/deregister/land itself, and not to be managed by an operator.

@chendrix All this time finding references to land-worker, it had never occurred to me that it was actually on the concourse binary and not in fly   :)
In instances where my systems need to scale up workers, the process is relatively straight forward, but in the instance of systems being scaled back down we need a way to drain containers so that we can decommission workers without interrupting jobs and throwing them into an aborted state.
The concourse land-worker does provide a way to manage that, but it requires logging into ssh, then re-specifying the public / private key which adds extra challenges to scripting the landing / retiring of workers. From this perspective, fly was a much more attractive option because I already have patterns of connecting and using fly to manage concourse.

I think the original intent is that you clearly already have something which is "logging into ssh", because you are running the worker binary somehow. The drain script which lands the worker is simply a sidecar process to the running worker binary, which has the same access to configuration/ssh keys/etc.
I'll leave this open so you can explore trying to do what you want to do using just the binaries, but I'll probably close it if there's not a particularly compelling case for it to be in fly after you've explored it.

Can you add some color to help me understand the reasoning behind having prune-worker but not land-worker / retire-worker? Is it some sort of security barrier, forcing land/retire to live in a place where people are required to also input the ssh keys?
In the end, I think that land is still an open atc api endpoint, so it's not too much trouble to issue commands directly

Pruning a worker is just a database level command, it just means "This has stalled, but it's actually just gone, so remove it". Pruning a worker doesn't effect anything really other than the UI.
Landing a worker/retiring a worker is effectful, it causes the ATC to stop scheduling workloads on that and to eventually disconnect.
The rationale for keeping it in the worker binary is exactly what you mentioned re security.

Hi there,
I'm going to close this PR now because of the rationale given above. Thanks again for submitting it, though, and thanks again for liking Concourse enough to contribute to it!


modified index of code to reflex changes in 3rd edition of the book

Hey, @norvig, I just realised I am using Indian subcontinent version of the 3rd edition published by Dorling Kindersley (India) Pvt. Ltd. I don't think these changes would be useful for people using 3rd edition published by other publishers.
Can I know how many versions are there for the 3rd edition. And what do you suggest regarding this issue, page numbers of pseudo codes are different in different versions?

Right @SnShine , page numbers can be slightly different. But section numbers should be the same. Let's drop the page numbers and just use section numbers.


Remove redundant entry from gitignore

I'd rather delete the other one as it's for Ant. Also all the Eclipse stuff...

Thanks!


Rebasing Relative URLs when working with CSS string and URL strings ONLY

So I have the source file url:
var cssURL = "https://www.example.com/resources/dir4/dir3/dir2/dir1/screen.css";
which contains the following CSS string:
.badge-40 { background-image: url(images/sprites/badges-50-sprite.png); }, .badge-50 { background-image: url(/images/sprites/badges-50-sprite.png); }, .badge-60 { background-image: url(https://www.example.com/images/sprites/badges-60-sprite.png); }, .badge-70 { background-image: url("../images/sprites/badges-70-sprite.png"); }, .badge-80 { background-image: url('../../images/sprites/badges-80-sprite.png'); }, .badge-90 { background-image: url('../../../images/sprites/badges-90-sprite.png'); };`
Then I use the following code to create the absolute URL, making allowance for the variations of no quotes, single quotes and double quotes:
new CleanCSS({
        returnPromise: true,
        level: {
            1: {
               transform: function (propertyName, propertyValue) {
                    var getBackgroundImageRegEx = /url(?:\(['"]?)(.*?)(?:['"]?\))/g;
                    if (propertyName == 'background-image' && propertyValue.startsWith('url(')) {
                        var source = getBackgroundImageRegEx.exec(propertyValue)[1];
                        switch (true) {
                                case source.startsWith('/'):
                                    return propertyValue;
                                case source.startsWith('http'):
                                   return propertyValue;
                                case source.startsWith('../../../'):
                                    var threeDirectoriesBackRelativeURL = cssURL.split('/').slice(0,-4).join('/');
                                    return `url(${threeDirectoriesBackRelativeURL}/${source.replace('../../../', '')})`;
                                case source.startsWith('../../'):
                                    var twoDirectoriesBackRelativeURL = cssURL.split('/').slice(0,-3).join('/');
                                    return `url(${twoDirectoriesBackRelativeURL}/${source.replace('../../', '')})`;
                                case source.startsWith('../'):
                                    var oneDirectoryBackRelativeURL = cssURL.split('/').slice(0,-2).join('/');
                                    return `url(${oneDirectoryBackRelativeURL}/${source.replace('../', '')})`;
                                default:
                                    var basicRelativeURL = cssURL.split('/').slice(0,-1).join('/');
                                    return `url(${basicRelativeURL}/${source})`;
                            }
                    }
                }
            }
        }
    })
    .minify(cssString)
    .then(output => {
        console.log(output.styles);
    });

This returns the CSS adjusted as required:
.badge-40{background-image:url(https://www.example.com/resources/dir4/dir3/dir2/dir1/images/sprites/badges-50-sprite.png)},.badge-50{background-image:url(/images/sprites/badges-50-sprite.png)},.badge-60{background-image:url(https://www.example.com/images/sprites/badges-60-sprite.png)},.badge-70{background-image:url(https://www.example.com/resources/dir4/dir3/dir2/images/sprites/badges-70-sprite.png)},.badge-80{background-image:url(https://www.example.com/resources/dir4/dir3/images/sprites/badges-80-sprite.png)},.badge-90{background-image:url(https://www.example.com/resources/dir4/images/sprites/badges-90-sprite.png)}
which is what I'm after.
I had a couple of questions.
If a relative URL requires more than three dot segments / directory steps backwards, so like this ../../../../, then the propertyValue returns only three dot segments. Is this compliant with standards as I thought any length was legitimate?
Also, my code seems fragile to me. I'm not a fan of using regexes like this and I thought the auto-adjustment of relative URLs might be included in the package somewhere. I've tried rebaseTo but this includes my workspace URL in the output.
Any help would be much appreciated.

Won't fix as clean-css is going into maintenance mode.


<EMAIL_ADDRESS>decompress error

Scanning the drive for archives:
1 file, 39995334 bytes (39 MiB)
Extracting archive: C:\Users\Baker\scoop\apps\avidemux\2.8.1\dl.7z
Can't open as archive: 1
Files: 0
Size:       0
Compressed: 0
ERROR: C:\Users\Baker\scoop\apps\avidemux\2.8.1\dl.7z
C:\Users\Baker\scoop\apps\avidemux\2.8.1\dl.7z
Open ERROR: Can not open the file as [7z] archive
ERRORS:
Is not archive

Duplicate of #9689


Unified entry-point

Checklist before submitting

[x] Did you read the contributor guide?
[ ] Did you update any affected documentation?
[ ] Did you write any tests to validate this change?

Description
Fixes #161
Review process to land

All tests and other checks must succeed.
At least one maintainer must review and approve.
If any  maintainer requests changes, they must be addressed.

Codecov Report

Merging #170 (488ff3a) into master (6d6c564) will increase coverage by 0.14%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #170      +/-   ##
==========================================
+ Coverage   91.12%   91.25%   +0.14%
==========================================
  Files          12       12
  Lines         968      983      +15
==========================================
+ Hits          882      897      +15
  Misses         86       86

Flag
Coverage Δ

unittests
91.25% <100.00%> (+0.14%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

open-ce/build_env.py
84.93% <100.00%> (+0.65%)
:arrow_up:

open-ce/build_feedstock.py
92.94% <100.00%> (+0.26%)
:arrow_up:

open-ce/test_feedstock.py
95.37% <100.00%> (+0.13%)
:arrow_up:

open-ce/validate_config.py
86.67% <100.00%> (+0.95%)
:arrow_up:

open-ce/validate_env.py
78.57% <100.00%> (+2.57%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6d6c564...488ff3a. Read the comment docs.

Codecov Report

Merging #170 (fab8837) into master (6d6c564) will increase coverage by 2.69%.
The diff coverage is 91.75%.

@@            Coverage Diff             @@
##           master     #170      +/-   ##
==========================================
+ Coverage   91.12%   93.80%   +2.69%
==========================================
  Files          12       13       +1
  Lines         968      936      -32
==========================================
- Hits          882      878       -4
+ Misses         86       58      -28

Flag
Coverage Δ

unittests
93.80% <91.75%> (+2.69%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

open-ce/open-ce
80.95% <80.95%> (ø)

open-ce/build_env.py
92.16% <100.00%> (+7.87%)
:arrow_up:

open-ce/build_feedstock.py
100.00% <100.00%> (+7.32%)
:arrow_up:

open-ce/build_image.py
100.00% <100.00%> (+10.17%)
:arrow_up:

open-ce/build_tree.py
96.19% <100.00%> (-0.19%)
:arrow_down:

open-ce/docker_build.py
95.65% <100.00%> (+0.15%)
:arrow_up:

open-ce/test_feedstock.py
100.00% <100.00%> (+4.76%)
:arrow_up:

open-ce/utils.py
79.25% <100.00%> (ø)

open-ce/validate_config.py
100.00% <100.00%> (+14.29%)
:arrow_up:

open-ce/validate_env.py
100.00% <100.00%> (+24.00%)
:arrow_up:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6d6c564...fab8837. Read the comment docs.

Codecov Report

Merging #170 (4190e3d) into master (6d6c564) will increase coverage by 2.77%.
The diff coverage is 92.66%.

@@            Coverage Diff             @@
##           master     #170      +/-   ##
==========================================
+ Coverage   91.12%   93.88%   +2.77%
==========================================
  Files          12       13       +1
  Lines         968      948      -20
==========================================
+ Hits          882      890       +8
+ Misses         86       58      -28

Flag
Coverage Δ

unittests
93.88% <92.66%> (+2.77%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

open-ce/open-ce
83.33% <83.33%> (ø)

open-ce/build_env.py
92.31% <100.00%> (+8.02%)
:arrow_up:

open-ce/build_feedstock.py
100.00% <100.00%> (+7.32%)
:arrow_up:

open-ce/build_image.py
100.00% <100.00%> (+10.17%)
:arrow_up:

open-ce/build_tree.py
96.19% <100.00%> (-0.19%)
:arrow_down:

open-ce/docker_build.py
95.65% <100.00%> (+0.15%)
:arrow_up:

open-ce/test_feedstock.py
100.00% <100.00%> (+4.76%)
:arrow_up:

open-ce/utils.py
79.25% <100.00%> (ø)

open-ce/validate_config.py
100.00% <100.00%> (+14.29%)
:arrow_up:

open-ce/validate_env.py
100.00% <100.00%> (+24.00%)
:arrow_up:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6d6c564...4190e3d. Read the comment docs.

Example Usage:
(dev) [bnelson@dlw05 open-ce]$ ./open-ce/open-ce -h
usage: open-ce [-h] {build,test,validate} ...

Open-CE Tool

optional arguments:
  -h, --help            show this help message and exit

Open-CE Commands:
  Valid Commands

  {build,test,validate}
    build               Build
    test                Test
    validate            Validate

(dev) [bnelson@dlw05 open-ce]$ ./open-ce/open-ce build -h
usage: open-ce build [-h] {env,feedstock,image} ...

optional arguments:
  -h, --help            show this help message and exit

Build Commands:
  Valid Commands

  {env,feedstock,image}
    env                 Build conda environment as part of Open-CE
    feedstock           Build conda packages as part of Open-CE
    image               Run Open-CE tools within a container

(dev) [bnelson@dlw05 open-ce]$ ./open-ce/open-ce build env -h
usage: open-ce build env [-h] [--conda_build_config CONDA_BUILD_CONFIG]
                         [--output_folder OUTPUT_FOLDER]
                         [--channels CHANNELS_LIST]
                         [--repository_folder REPOSITORY_FOLDER]
                         [--python_versions PYTHON_VERSIONS]
                         [--build_types BUILD_TYPES] [--mpi_types MPI_TYPES]
                         [--skip_build_packages] [--run_tests]
                         [--docker_build] [--git_location GIT_LOCATION]
                         [--git_tag_for_env GIT_TAG_FOR_ENV]
                         env_config_file [env_config_file ...]

positional arguments:
  env_config_file       Environment config file. This should be a YAML file
                        describing the package environment you wish to build.
                        A collection of files exist under the envs directory.

optional arguments:
  -h, --help            show this help message and exit
  --conda_build_config CONDA_BUILD_CONFIG
                        Location of conda_build_config.yaml file. (default:
                        /mnt/pai/home/bnelson/git/open-ce/open-
                        ce/../conda_build_config.yaml)
  --output_folder OUTPUT_FOLDER
                        Path where built conda packages will be saved.
                        (default: condabuild)
  --channels CHANNELS_LIST
                        Conda channels to be used. (default: [])
  --repository_folder REPOSITORY_FOLDER
                        Directory that contains the repositories. If the
                        repositories don't exist locally, they will be
                        downloaded from OpenCE's git repository. If no value
                        is provided, repositories will be downloaded to the
                        current working directory. (default: )
  --python_versions PYTHON_VERSIONS
                        Comma delimited list of python versions to build for ,
                        such as "3.6" or "3.7". (default: 3.6)
  --build_types BUILD_TYPES
                        Comma delimited list of build types, such as "cpu" or
                        "cuda". (default: cpu,cuda)
  --mpi_types MPI_TYPES
                        Comma delimited list of mpi types, such as "openmpi"
                        or "system". (default: openmpi)
  --skip_build_packages
                        Do not perform builds of packages. (default: False)
  --run_tests           Run Open-CE tests for each potential conda environment
                        (default: False)
  --docker_build        Perform a build within a docker container. NOTE: When
                        the --docker_build flag is used, all arguments with
                        paths should be relative to the directory containing
                        open-ce. Only files within the open-ce directory and
                        local_files will be visible at build time. (default:
                        False)
  --git_location GIT_LOCATION
                        The default location to clone git repositories from.
                        (default: https://github.com/open-ce)
  --git_tag_for_env GIT_TAG_FOR_ENV
                        Git tag to be checked out for all of the packages in
                        an environment. (default: None)

(dev) [bnelson@dlw05 open-ce]$ ./open-ce/open-ce build env open-ce/envs/xgboost-env.yaml --python_vers 3.7 --build_types cpu --docker_build --output_folder /mnt/pai/home/bnelson/git/myconda_build
...

Looks pretty good to me so far!

Codecov Report

Merging #170 (be90771) into master (6d6c564) will increase coverage by 2.69%.
The diff coverage is 91.76%.

@@            Coverage Diff             @@
##           master     #170      +/-   ##
==========================================
+ Coverage   91.12%   93.80%   +2.69%
==========================================
  Files          12       14       +2
  Lines         968      984      +16
==========================================
+ Hits          882      923      +41
+ Misses         86       61      -25

Flag
Coverage Δ

unittests
93.80% <91.76%> (+2.69%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

open-ce/build_tree.py
95.58% <81.82%> (-0.80%)
:arrow_down:

open-ce/open-ce
82.61% <82.61%> (ø)

open-ce/inputs.py
91.11% <91.11%> (ø)

open-ce/build_env.py
92.59% <100.00%> (+8.31%)
:arrow_up:

open-ce/build_feedstock.py
100.00% <100.00%> (+7.32%)
:arrow_up:

open-ce/build_image.py
100.00% <100.00%> (+10.17%)
:arrow_up:

open-ce/docker_build.py
95.70% <100.00%> (+0.19%)
:arrow_up:

open-ce/test_feedstock.py
100.00% <100.00%> (+4.76%)
:arrow_up:

open-ce/utils.py
76.38% <100.00%> (-2.87%)
:arrow_down:

open-ce/validate_config.py
100.00% <100.00%> (+14.29%)
:arrow_up:

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6d6c564...be90771. Read the comment docs.

The build is chugging along. I'll go ahead and merge this.


Display line and column numbers

This is a possible solution for #44.
The function lineAndColumn takes the cheerio object, as well as the html of the tag to output line and column numbers for. It uses an external variable to keep track of which tags it has looked for already, which is required to allow for multiple tags with the exact same html. It then splits the whole file by lines up to the point at which the tag occurs.

Hi.. @padarom  Could you please share your "npm test" output here.
Thanks

I did not write any new tests to check for the new functionality as I'm not familiar with mocha. It doesn't seem to be breaking any existing tests though:

><EMAIL_ADDRESS>test /Users/christopher/Code/w3c-link-validator
> mocha test/index.js

  CLI app main
    ✓ Command line initialization should return true

▶ http://example.com/

BASE	/
    ✓ Command line initialization(with parameters) should return true

  Alerts
 WARNING  Test Message
    ✓ alertWarning should return undefined
  ERROR   Test Message
    ✓ alertError should return undefined
 SUGGEST  Test Message
    ✓ alertSuggestion should return undefined

  HTML Validator
    ✓ setGlobals should return undefined
❯ Checking HTML guidelines..

HTML SUMMARY
✔ no problem found

    ✓ Sample HTML validation Should return an object
❯ Checking HTML guidelines..

 WARNING  The doctype is required just above the  tag, at the very start of each document you write.
  ERROR    tag should be appeared
  ERROR   <title> should be added inside

HTML SUMMARY
✖ 3 problem(s) found

    ✓ Sample HTML(Violates RULE SET #1) validation Should return an object
❯ Checking HTML guidelines..

 WARNING  Make sure to offer alternative access. For images that means use of meaningful alternative text
[15:4]

 SUGGEST  Make sure to offer alternative access. For images that means use of meaningful alternative text
[16:4]

HTML SUMMARY
✖ 1 problem(s) found
ℹ 1 suggestion(s)

    ✓ Sample HTML(Violates RULE SET #2) validation Should return an object
❯ Checking HTML guidelines..

 WARNING  <applet> is entirely obsolete, and must not be used by authors. Use <embed> or <object> instead
[15:4] <applet></applet>

HTML SUMMARY
✖ 1 problem(s) found

    ✓ Sample HTML(Violates RULE SET #3) validation Should return an object
❯ Checking HTML guidelines..

 WARNING  charset attribute in <link>, is entirely obsolete, and must not be used by authors Use an HTTP Content-Type header on the linked resource instead.
[15:4] Test

HTML SUMMARY
✖ 1 problem(s) found

    ✓ Sample HTML(Violates RULE SET #4) validation Should return an object

  Links
    ✓ setGlobals should return undefined
    ✓ isLocal should return true for same host url
    ✓ isLocal should return false for different host url
Adding 2 link(s) to the queue...

    ✓ linkChecker should return an array

  Main file
    ✓ w3clink initialization should return undefined
    ✓ isLocal should return true for same host url
    ✓ isLocal should return false for different host url

▶ http://www.example.com/

BASE	/
STATUS	200 OK
ELAPSED	0.26 secs

OVERALL LINK SUMMARY
✔ Total links        : 2
✔ Dead links         : 0
✔ Dead link Ratio    : 0.00%

OVERALL HTML SUMMARY
✔ HTML Problems      : 0
ℹ HTML Suggestions   : 0
STATUS	200 OK
ELAPSED	0.27 secs

❯ Checking HTML guidelines..

 WARNING  The doctype is required just above the  tag, at the very start of each document you write.

HTML SUMMARY
✖ 1 problem(s) found

Adding 1 link(s) to the queue...

EXT  http://www.iana.org/domains/example

▶ http://www.iana.org/domains/example

BASE	example
STATUS	200 OK
ELAPSED	1.38 secs

OVERALL LINK SUMMARY
✔ Total links        : 3
✔ Dead links         : 0
✔ Dead link Ratio    : 0.00%

OVERALL HTML SUMMARY
✖ HTML Problems      : 1
ℹ HTML Suggestions   : 0
    ✓ Validation execution should be executed without crash and return undefined (9503ms)

OVERALL LINK SUMMARY
✔ Total links        : 3
✔ Dead links         : 0
✔ Dead link Ratio    : 0.00%

OVERALL HTML SUMMARY
✖ HTML Problems      : 1
ℹ HTML Suggestions   : 0
    ✓ displaySummary should return undefined

  20 passing (10s)

Any plans for merging or doing anything with this any time soon?


Front matter doesn't parse correctly

when the readme contains underline first level header style:
Header
---

As opposed to hash levels:
# Header

see the phenomena in detail in praqma/noops#120

The regexp just needs to be non-greedy in https://github.com/Praqma/ghreport/blob/master/ghreport#L976

@lakruzz Update from me and @JKrag:  The issue is not quite fixed yet.
The new regex fixes the problem with the broken frontmatter on git-phlow, but now fails to parse many others.
The diff of the latest run https://github.com/Praqma/repo-stat/commit/a266d4d161f8ac0d82bd395c73c5966c56d0ca2c
shows that quite a few repos now no longer show maintainer info etc.
@JKrag can look at the regex tomorrow if you don't fix it before then.
The concourse pipeline should now be running correctly and updating the report every 30 minutes.


Use setuptools' SCM support to determine version from Git tag

This of course predictates on a git tag release system but this repo seems to already be doing that.

Happy with this. I'm going to automate releases with GitHub actions soon and roll all of these into a release candidate for the next version,


release downloads down

Download of iso-releases seems to be down:
https://github.com/boot2docker/boot2docker/releases
Can somebody verify this or is it a corporate-network limitation of mine?
thanks.

It was due to my corporate limitation sorry.


Touch devices issue

I ran into an issue that position is calculated always as 0 when resizing in touch emulation mode. This should fix it

LGTM but I'm not up-to-date on mobile fun, @BobbieBarker or @j--w want to double check this?

The code looks good to me but I'm not 100% sure what the issue is, I tried resizing in touch emulation mode and it seems ok to me. Could you provide a bit more detail or a plunkr to reproduce the issue @kurideja ?


Gsheets

OK these are two very minor changes, mentioned in #254.  One is correcting an issue with "table" data outside the headers, and the other is just updating the documentation because the Append and BatchUpdate classes take different values for ValueInputOption (and honestly probably every other ENUM has bad help if the names have underscores, but one thing at a time).
I think it's actually a problem with the Google libraries, either how the .Net generator is interpreting the Sheets API discovery document or how the discovery document itself is generated, and have opened this issue to try and get the behavior normalized.
Since it looks like you're nearing a release I wanted to sneak these in, but if it's going to be a little while I can add some more "teeth" to my Gsheet changes.

These changes look good! The doc update makes sense, thank you for that :-) I'll need to spend some more time looking at the change on Import-GSSheet but from a high-level it makes sense to me. LGTM!


restoration: checkpoint local allocator state, utilize on restoration

This is a stepping stone for the broader fqdn refactor.
There are 3 changes:

Checkpoint local allocator state
Use this to generate all labels for a prefix upon restart
Reduce identity restoration grace period to 30 seconds.

Right now, selector / identity stability is ensured by the ipcache restoration mechanism, which dumps the outgoing ipcache, then requests the same numeric identity for any new prefixes. This is to prevent policy drops as endpoints are regenerated for the first time, as they may select "old" identities until regeneration is complete. Therefore, the goal is that a selector has a consistent set of identities across restarts, and prefixes have a consistent identity.
The current mechanism is simplistic: we dump the outgoing ipcache, list all prefixes, which request their previous numeric identity upon restoration, and we hope that their set of labels after restart is the same. This will not work, however, once a single local identity can have multiple prefixes.
Example
Imagine a future where we have 3 identities and 3 IPs, all with ID3:

ID1: (fqdn:a.com)
ID2: (fqdn:b.com)
ID3: (fqdn:a.com, fqdn:b.com)

Some endpoints will select (ID1, ID3) and some will select (ID2, ID3). As such, their bpf policy maps will contain these two identities.
The problem comes when we restart the agent. If we do not restore FQDN state exactly (which happens commonly enough), and an IP changes from (a.com, b.com) to (a.com), it could be the case that ID3 now has the set of labels (fqdn:a.com). This could cause policy drops during restoration.
So, a better approach is to restore identities directly, and only fall back to requesting old numeric identities as a fallback.

(note: this will fail to build until https://github.com/cilium/cilium/pull/32378 is merged)

Had to make a few small changes to de-flake the unit tests, @gandro could you take a quick look? Two things:

wait for trigger function to finish
dump LabelArray, not Labels, so models are comparabe.

/test

/test

Needed one slight refactor to adjust to the fact that there is more than one identity allocator.

/test

I did some local testing with this branch and I noticed it breaks endpoint restoration on the upgrade path. When installing v1.15.5 and then upgrading to a build of this branch, all managed endpoints are no longer present in cilium endpoint list. Investigating why that is now

/test

/test

/test

/test

/test


Windows support

The range syntax used in /definitions/npm uses <= and >=, which does not work on Windows. This PR changes the file names to use a different range syntax.
To keep this small and avoid mid-flight collisions, I only changed the code in lib/libDef.js, which deals with filesystem paths.
The "codemod" to move the files:
find definitions/npm -type d -name flow_* | gawk -f move.awk

Where move.awk is:
{ r = gensub(/>=(v[0-9x.]+)_<=(v[0-1x.]+)/, "\\1-\\2", 1)
  r = gensub(/>=(v[0-9x.]+)/, "\\1-", 1, r)
  r = gensub(/<=(v[0-9x.]+)/, "-\\1", 1, r)
  system("git mv '" $0 "' '" r "'") }

Fixes #225

Not sure what's up with Travis, but tests pass on my machine.

Travis error looks like an npm thing (seems to happen sometimes). I'll restart it

Shouldn't you also edit [theses lines](https://github.com/flowtype/flow-typed/blob/07eca72148e4134634b7ee30e5422a6dfec32b13/cli/src/commands/runTests.js#L15-L24 ?) for windows support ?

Will circle back and look at overall Windows support in a separate PR, but this part of the work is done and ready for review.
(Note that the renames are likely to cause merge conflicts with all the other open PRs. I'm happy to rebase on my end at any point to minimize conflicts.)


MODELIX-757 Use Gradle composite builds for modelix.core plugin tests

Includes the core builds in the plugin test builds.
Once this is merged, we will no longer have to run ./gradlew publishToMavenLocal before running the plugin tests.
~Currently the model-api-gen plugin does not work correctly in this build setup, causing both tests to fail.
This behaviour might be fixed by resolving MODELIX-641.~
Update: After resolving MODELIX-641 this now works as intended.

:tada: This PR is included in version 7.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Fix race condition in Task.WhenAny

Port https://github.com/dotnet/coreclr/pull/9024 to corert

Thank you!


Sound Effects

Are you going to add the ability to generate sound effects to your Python library?

We haven't launched the sound effects API but you can sign up here to hear when we do: https://form.typeform.com/to/wXBm7Hs2


Add text emails in addition to html for all emails sent

Should support text/plain as well as html emails. Tidy up email templates.

...and if you are sending HTML emails. Linkify the links! (The big ugly URL sent in the 'activation' email for example, has come through with a non-clickable URL in my mail client )

Yup, same here. Please actually add an  tag to the link in the activation mail :). Currently, it's

       This is an automated email from HOT Exports.

       Please follow this link http://export.hotosm.org/en/osm/complete/openstreetmap/?verification_code=hash&signature=quitelonggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggg to login to HOT Exports

           Thank you for registering with the HOT Export Service.

           Humanitarian OpenStreetMap Team

Added  link.. sorry 'bout that!


[FEATURE REQUEST]: Search entity by part of number

Functional
Add the ability to search for entities (vehicles, weapons) by part of the number (min. 3 characters)
Behavior now
Veh. number:

Search:

https://github.com/SnailyCAD/snaily-cadv4/issues/64


[Flink-5734] code generation for normalizedkey sorter

This pull-request is the implementation of applying code generation to NormalizedKeySorter. It is built on top of FLINK-3722 and based on FLIP: Code Generation for NormalizedKeySorter.

Result from  SortPerformance.java

Has the FLIP been posted and officially discussed on the mailing list?

I haven't posted it yet. I will do it next week.

@heytitle please remove the old FLINK-3722 commits and rebase to master.

@greghogan May I ask you how to remove FLINK-3722 commits?. Only way I can think of is git rebase -i, but this will rewrite history of this PR.

@heytitle, you can rebase -i dd20^ and then delete the first two lines which removes those commits from the history.
An initial thought from skimming the code: should we create an abstract NormalizedKeySorterBase with common code from the generated and non-generated implementations? This way the lines of code in the template would be minimized and we wouldn't need to synchronize changes. I don't see a reason why this would decrease performance.

I also think about the abstract class but I'm not sure how to do it properly.

@heytitle, the code generation is only for a few methods, right? So the other methods in the sorter template could be moved into a NormalizedKeySorterBase which would be subclasses by the template. Then NormalizedKeySorter could also subclass NormalizedKeySorterBase. This would reduce the duplicated code and also minimize the amount of code 'hidden' from IDEs in the template.

Hi @greghogan,
Thanks for the explanation. I like the idea. I also think we might not need NormalizedKeySorterBase, we can just extend generated sorters from NormalizedKeySorter and override those methods .
What do you think?

@heytitle yes, let's try that first.

Hi @greghogan,
Thank very much for the feedback.

Are you able to continue working on this feature?

Yes, I would like to complete the feature and will take a look into the issues you mentioned in next couple of weeks.

Hi @heytitle,
What is the status here? I will be on vacation next week, so I would have some time to work on this. Should I work on finishing this, or are you also working on it? I would like to bring this PR to conclusion for the 1.4 release, for which the feature freeze is on Sept 4, so we have to be quick.

Hi @ggevay,
I'm currently fixing coding style. Maybe you can help me take a look at what Greg suggested related FixedLengthRecordSorter

Hi @ggevay,
I have rebased master and resolved all coding style related issues and pushed the code to https://github.com/heytitle/flink/tree/FLINK-5734-rebase-master-temp
For the next step, I'm trying to pass mvn clean verify.  Right now, some tests are randomly failed.

Thank @heytitle , I'll try to do the code generation for FixedLengthRecordSorter tomorrow, and I will also take a look at the failing tests.
Btw. you can force push (git push -f) the rebased version to the branch of the PR.

I pushed some commits here:
https://github.com/ggevay/flink/tree/sorter-codegen
I fixed one issue by adding the CLUSTER_WITH_CODEGENERATION_ENABLED case to the switch in DriverBaseITCase.getSystemOutput.
However, the randomly occurring FileNotFoundException is more tough to fix. I think this is happening because of some Travis bug around creating temporary files. Here are some Jiras that might be the same issue:
https://issues.apache.org/jira/browse/FLINK-2498
https://issues.apache.org/jira/browse/FLINK-2719
The only fix that I can think of is to not compile the generated code from a file, but generate it into a string, and use that overload of cook that takes a string instead of a file. (But we might still want to also write the generated code to a file, to be able to look at the generated code for debugging purposes. We just won't use that file for compiling.)

Hi @ggevay,
Thanks for the commits.  Do you have any plan for FixedLengthRecordSorter implementation?
I'm not sure how much work need to be done there.

I'm not exactly sure how to do FixedLengthRecordSorter.
@greghogan , could you please clarify what you meant in the following?

Rather than extending NormalizedKeySorter, we could pull the compare and swap functions out of
MemorySegment into an interface with a default implementation moved from NormalizedKeySorter
and FixedLengthRecordSorter. There is overlap in that the normalizedKeyFullyDetermines case of
NormalizedKeySorter results in the same operations as in FixedLengthRecordSorter.

Do you mean that the generated code would implement this interface? What would be the relation between the non-generated NormalizedKeySorter class and this interface? Would this change introduce more virtual calls?

Actually, now I'm thinking we should defer implementing code generation for FixedLengthRecordSorter to after https://github.com/apache/flink/pull/2617 is merged. What do you think @greghogan ?

I pushed some more commits to https://github.com/ggevay/flink/tree/sorter-codegen. @heytitle , could you please push them here? (Or you could add me as a collaborator to your flink repo, and then I can directly push to your branch of the PR.)
@heytitle , @greghogan , could you do a review? I think this PR is close to a mergable state. (I would say we should merge this without FixedLengthRecordSorter, and then do that later after https://github.com/apache/flink/pull/2617 is merged.)

@ggevay  Thanks for the commits. They look good. I also added you to my repository.
For FixedLengthRecordSorter, I agree with you that we should implement that  when #2617 is merged.
How does feature freeze work? Apart from getting this PR merged, is there anything else we have to do?

Oh, you are right @heytitle, thanks! Fixed in 9016cce503b4d471b5a49f0abccc196945ada97e
These WeakReferences are scary :)

Thanks @fhueske and @heytitle!
Most of the comments have been addressed. The two outstanding issues are

Addressing https://github.com/apache/flink/pull/3511#discussion_r139933797
Making sure that https://github.com/apache/flink/pull/3511#discussion_r139926965 indeed fixes the leak and doesn't introduce any new issues.

I think I can do both next week. Or, @heytitle , if you have time you could also work on them. (Please drop me an email if you start working on 1., to avoid both of us doing it. For 2., the more eyes on it, the better.)

@heytitle @ggevay Great work!
We are working on a project to fully code generate the algorithm Flink runtime used, and borrowed lots of idea of this PR, thanks! IMHO, in addition to these changes, there are still some potential improvements we can do about the sorter, like deserialization when comparing the real records. To achieve this, we need more type information control and flexible code generating supports, so we choose to do it through the Table API & SQL. How do you see this approach?

IMHO, in addition to these changes, there are still some potential improvements we can do about the sorter, like deserialization when comparing the real records.

Do you mean NormalizedKeySorter.compareRecords? This calls comparator.compareSerialized, which is currently implemented for most types by simply deserializing the records and then comparing the deserialized form. It would maybe bring some performance improvement if this had a real implementation, which would deserialize only the key fields (which are relevant to the comparison). The hard part here is how to handle nullable fields, which make the offset of individual fields unpredictable.
So I think a simpler and better approach is to just make sure that most types have a good implementation of putNormalizedKey, and then NormalizedKeySorter.compareRecords would be called only rarely, so its performance wouldn't really matter.

We are working on a project to fully code generate the algorithm Flink runtime used, and borrowed lots of idea of this PR, thanks!

I'm happy to hear that!
Btw. I think a large potential in code-generation is to eliminate the overheads of the very many virtual function calls throughout the runtime, by making the calls monomorphic [1,2]. For example, there could be a custom MapDriver generated for each map UDF, and then this custom class would always call the same UDF, making the call monomorphic, and thus easily optimizable by the JVM [1,2]. (I think @greghogan was also thinking about this.)
For example, the following virtual calls are on the per-record code path:

drivers (such as MapDriver) calling MutableObjectIterator.next
drivers calling the UDF
drivers calling output.collect
ReusingDeserializationDelegate or NonReusingDeserializationDelegate calling serializer.deserialize
SpillingResettableMutableObjectIterator calling serializer.serialize and serializer.deserialize
PojoSerializer, TupleSerializer, RowSerializer, etc. calling their field serializers
CountingCollector calling collector.collect
RecordWriter calling channelSelector.selectChannels
SerializationDelegate calling serializer.serialize
StreamElementSerializer calling methods on its typeSerializer
OutputEmitter calling comparator.hash
ComparableKeySelector calling comparator.extractKeys
assignToKeyGroup calling key.hashCode

I think most of the above calls are megamorphic (especially in larger Flink jobs with many operators), which makes them slow [1,2]. They could be made monomorphic by code-generating custom versions of these classes, where the customization would be to fix the type of the targets of these calls. (I think this could probably be done independently of the Table API.)
Another potential for code-generation is customizing OutputEmitter for the number of channels: there are places where a modulo operation is performed, which is much faster if the divisor is a compile-time constant, since then the compiler uses such tricks as described in this paper:
https://gmplib.org/~tege/divcnst-pldi94.pdf
And the same optimization could be made in KeyGroupRangeAssignment, where there are divisions by maxParallelism.

we need more type information control and flexible code generating supports, so we choose to do it through the Table API & SQL. How do you see this approach?

Having more info about the types, UDFs, etc. of your program certainly can help. Unfortunately I don't know too much about the Table API & SQL, but I have a few random thoughts:

Since you are generating the UDFs, it should be possible to make them have good object reuse behavior, without the users having to worry about the non-trivial object reuse rules.
For basic types (int, boolean, etc.) the generated code could use IntValue, BooleanValue, etc., facilitating more object reuse, again without making the users write ugly boilerplate (as they have to do in the DataSet/DataStream API when they want to use these classes).
Same thing with StringValue

Btw. have you seen this PR for code generation for POJO serializers and comparators? https://github.com/apache/flink/pull/2211
It has some issues, so it is not as close to merging as this PR, but maybe I'll try to tie that up as well some time.
[1] http://insightfullogic.com/2014/May/12/fast-and-megamorphic-what-influences-method-invoca/
[2] https://shipilev.net/blog/2015/black-magic-method-dispatch/

So I think a simpler and better approach is to just make sure that most types have a good implementation of putNormalizedKey, and then NormalizedKeySorter.compareRecords would be called only rarely, so its performance wouldn't really matter.

You are right when the sort keys are simple numeric types, but not with strings, which maybe the most popular choice in some ETL and data warehouse pipelines. But i agree that code generation can't help with this situation, so we investigate some binary data formats to represent our record and modify the interface of TypeSerializer & TypeComparator when doing ser/de. We don't have to consume the input/output view byte by byte, but has the ability to random access the underlying data, aka MemorySegment. It acts like spark's UnsafeRow: https://reviewable.io/reviews/apache/spark/5725, so we can eliminate the most deserialization cost such as read byte[] and then new String(byte[]).  We combine this approach with some code generation to eliminate the virtual function call of the TypeComparator and see a 10x performance improvements with sorting on strings.

I think a large potential in code-generation is to eliminate the overheads of the very many virtual function calls throughout the runtime

Totally agreed, after we finish dealing with the code generation and improving the ser/de, we will investigate more about this. Good to see that you have a list of all the megamorphic calls. BTW, we are actually translating the batch jobs into the streaming runtime, i think there will be lots in common.
Having and control more type informations, and code generation the whole operator have lots of benefits, it can also help to making most of the calls monomorphic, such as:

fully control of the object reusing, yes
comparators
generating hash codes
potential improvements of some algorithm which finds out they only need to deal with fixed length data
Directly using primitive variables when dealing with simple type

And you are right this is orthogonal with runtime improvements, and we see the boundary is the Operator. The framework should provide the most efficient environment for operators to run, and we will code generating the most efficient operators to live in it.

Btw. have you seen this PR for code generation for POJO serializers and comparators? #2211

I didn't see it yet, will find some time to check it out.

Can we simplify it to the code below?
So, we don't need to introduce cacheHit variable.

Thanks @heytitle , done in ff3a35e93e858e30debcb1e85cc056045c1fe2a0 .

The WeakHashMap with the WeakReferences seem to be working correctly. I've checked that the generated classes are reused from the cache inside a single job, but are not reused across jobs.
@fhueske , I think all the comments have been addressed.

@twalthr Sorry for the late response, but yes we borrowed lots of idea from this PR in blink's sorter code generation. And i want to thank @heytitle for these cool ideas.

Actually, the original ideas were from @ggevay.

Big thanks to the original author @ggevay , hope you don't mind for us borrowing your ideas.

Of course! I'm actually happy that they are finally making it into Flink in some form. So thanks for borrowing them :)

As you may want to know, we have opened a PR to introduce code generated sorter, check out #7958 if you are interested. @ggevay @heytitle


Release version not including Content-Type in html headers

OS: MacOs 10.13.5
Node: 8.11.1
NPM: 6.1.0
sqlpad:  2.6.0 (installed with npm)
Browser: Chrome 67.0.3396.79
When running the published version of sqlpad, I get the following:
$ sqlpad
Port 80 already occupied. Using port 60939 instead.

Welcome to SQLPad!. Visit http://localhost:60939 to get started

Upon inspecting the network traffic from http://localhost:60939, none of the responses include a Content-Type in the header.
The is causing issues with my reverse proxy which replaces non-existent Content-Types which a default which then causes the app not to crash.
My problems aside, web apps should include Content-Type in the header.
Also, this only affects some requests.  signin has no Content-Type but api/app does.  Likewise, static/bundle.js and tauCharts.min.css are missing Content-Type.
Any help would be appreciated.

I poked around a bit and noticed that I get Content-Type on first load of static assets (with a 200 response code), and on subsequent pages a 304 response code.
This seems to be known/intended behavior: https://stackoverflow.com/questions/31099113/why-response-304-doesnt-have-content-type-header
Is there a way to handle this gracefully in your reverse proxy?

It has been surprisingly hard to figure out how to configure the proxy server to handle this correctly.  However, since sqlpad is working normally, it sounds like it is my problem to solve.  Thanks for your consideration.


[PROPOSAL]: It would be very nice to use index when underlying table is applied with CACHE TABLE

Consider this script:
import com.microsoft.hyperspace._
import com.microsoft.hyperspace.Hyperspace
import com.microsoft.hyperspace.index.LogicalPlanSignatureProvider
import com.microsoft.hyperspace.index.IndexConfig
import org.apache.spark.{SparkContext, SparkException}
import org.apache.spark.sql.{DataFrame, SparkSession}
import org.apache.spark.sql.expressions.Window
import org.apache.spark.sql.expressions.WindowSpec
import scala.util.{Failure, Success, Try}
import com.microsoft.hyperspace.actions.Constants
import com.microsoft.hyperspace.util.{FileUtils, LogicalPlanUtils, ResolverUtils}
import org.apache.hadoop.fs.Path

val pipelineSpark :SparkSession = SparkSession.builder().config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension").config("spark.sql.catalog.spark_catalog", "org.apache.spark.sql.delta.catalog.DeltaCatalog").getOrCreate()
import pipelineSpark.implicits._

pipelineSpark.enableHyperspace()
val hyperspace = new Hyperspace(pipelineSpark)

val ws = Window.orderBy("C1")
val df = pipelineSpark.read.json("testJson.json").withColumn("RowId", row_number() over ws)

val path = new Path("delta")
FileUtils.delete(path,true)
val path = new Path("spark-warehouse/indexes")
FileUtils.delete(path,true)
df.write.format("parquet").save("delta")
val df = pipelineSpark.read.format("parquet").load("delta")
df.createOrReplaceGlobalTempView("jsont")
val indexConfig = new IndexConfig("ix1", Seq("C1"), Seq("C2"))
hyperspace.createIndex(df, indexConfig)
val t = pipelineSpark.sql("SELECT C1, C2 FROM global_temp.jsont WHERE C1=1")
hyperspace.explain(t, true)
pipelineSpark.sql("CACHE TABLE global_temp.jsont")
val t = pipelineSpark.sql("SELECT C1, C2 FROM global_temp.jsont WHERE C1=1")
hyperspace.explain(t, true)

The last explain produces plan as following:
scala> hyperspace.explain(t, true)
=============================================================
Plan with indexes:
=============================================================
Filter (isnotnull(C1#30L) && (C1#30L = 1))
+- InMemoryTableScan [C1#30L, C2#31L], [isnotnull(C1#30L), (C1#30L = 1)]

=============================================================
Plan without indexes:
=============================================================
Filter (isnotnull(C1#30L) && (C1#30L = 1))
+- InMemoryTableScan [C1#30L, C2#31L], [isnotnull(C1#30L), (C1#30L = 1)]

=============================================================
Indexes used:
=============================================================

=============================================================
Physical operator stats:
=============================================================

In my mind this is truly reduces index usage scenario set. It would be very nice if caching table actually promoted index use, at the moment caching index actually turns index use off.
I think one of the issues logged already, can help to address it
https://github.com/microsoft/hyperspace/issues/30
but I would like to make sure this is not lost in translation.

Prasad

From: Terry Kim<EMAIL_ADDRESS>Sent: Saturday, August 22, 2020 12:45 PM
To: microsoft/hyperspace<EMAIL_ADDRESS>Cc: Miron Berlin (People Tech Group<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [microsoft/hyperspace] [PROPOSAL]: It would be very nice to use index when underlying table is applied with CACHE TABLE (#130)
Thanks @SynapsePOChttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSynapsePOC&data=02|01|v-bemiro%40microsoft.com|0e44bb34cc574070976a08d846d3e953|72f988bf86f141af91ab2d7cd011db47|1|0|637337223254777601&sdata=eagocSIA7%2Fk8x7EWkfDqqakAUzYYlo1ooQaUKczEPKE%3D&reserved=0 for reporting this. AFAIK, caching the base table doesn't make sense in Hyperspace, since our index can have a subset of the original columns. (but I see that it could be beneficial if we could cache the index directly via some mechanism, but we need to investigate a bit more to see how we can preserve bucketing, etc.)
You could always cache the result of query:
val t = spark.sql("SELECT C1, C2 FROM tbl WHERE C1=1")
t.cache
, in which case, cache can be populated with index, but I don't think this is the scenario you are targeting.
cc-ing few people if I missed anything: @rapothhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frapoth&data=02|01|v-bemiro%40microsoft.com|0e44bb34cc574070976a08d846d3e953|72f988bf86f141af91ab2d7cd011db47|1|0|637337223254787595&sdata=1aJNdn8Qu%2B6LIIYCJffk4fKXtgQtljbvkerk0mBYpvw%3D&reserved=0 @apoorvedave1https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapoorvedave1&data=02|01|v-bemiro%40microsoft.com|0e44bb34cc574070976a08d846d3e953|72f988bf86f141af91ab2d7cd011db47|1|0|637337223254787595&sdata=v33JLi7iMQd1K5HPFSnfIp9UoUix0Syn2IqwD6RjtRg%3D&reserved=0
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fhyperspace%2Fissues%2F130%23issuecomment-678684485&data=02|01|v-bemiro%40microsoft.com|0e44bb34cc574070976a08d846d3e953|72f988bf86f141af91ab2d7cd011db47|1|0|637337223254797590&sdata=EKazbH7G0w35uWmlJzNjzq3L0OLyWw8TFmP%2BEJWY1cw%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAQL4U4DO2HNBEOGPWGT2OU3SCAN5HANCNFSM4QGJHF3A&data=02|01|v-bemiro%40microsoft.com|0e44bb34cc574070976a08d846d3e953|72f988bf86f141af91ab2d7cd011db47|1|0|637337223254797590&sdata=euUTOfnO1RrxxoFN3oFfEgGCDQHnIVYoIX%2FGeJI4zCY%3D&reserved=0.

We will investigate on what is required and whether this is feasible and get back. Thanks!

The value of this proposal can go manyfold, if cache is co-located with partition, thus retaining in the hands of user control over distribution of data amongst the nodes, participating in cluster. Thus, providing for the applications requiring most intensive query activity, to harvest significant volumes of data, without having to deal with shuffling and limitations of the driver node.
In that regard the ability to co - partition indexes along with data can present a huge plus, for users, who partition their data neatly.
Yes, if we try to seek corner cases, it can potentially hurt users, where data is askew amongst the nodes, but this is a separate issue.

For the scenario in the description, the problem is that Spark's Cache Manager changed "Relation" of cached df to "InMemoryTableScan" before the optimizer which applies indexes if possible.
In that case, we don't replace the plan to use the index (uncached) - and I don't think we have to; it is not desirable in general.
These are some possible scenarios:

caching "index data" itself, instead of the source table.

whole index data will be cached.
we can apply the cached index for all query that can be applied the index.
e.g

hyperspace.cache("indexName", ...)

caching query result using "index data"

or we could do cache index data with Filter or Join conditions:

val filter = df.filter("C1 = 1")
filter.cache

val t = spark.sql("SELECT C1, C2 FROM tbl WHERE C1=1")
t.cache

In this way, we could apply index data while using caching.
To do this, hyperspace should be enabled at cache time and also at query time + need some fix. (#254)
Or would you still like to utilize "cached" table along with "cached" index data though inefficient memory footprint?
@SynapsePOC


Add 4 New Columns

Add another field after:
-- https://github.com/agrc/uic-forms/blob/master/uic-forms/models/CsvMappings.cs#L23 (PermitDenied)
for 'PermitMod'
-- https://github.com/agrc/uic-forms/blob/master/uic-forms/models/CsvMappings.cs#L96 (OtherLeakEval_Fail)
for 'OtherLeakEval_Spec'
-- https://github.com/agrc/uic-forms/blob/master/uic-forms/models/CsvMappings.cs#L104 (OtherFluidTest_Fail)
for 'OtherFluidTest_Spec'
-- https://github.com/agrc/uic-forms/blob/master/uic-forms/models/CsvMappings.cs#L109 (Remedial_Other)
for 'RemedialOther_Spec'

Should these all default to 0 or empty string?


Implement semantic tokens

Implement semantic tokens for the language server.
https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_semanticTokens

Dup


Maintain the same directory structure for source and test

Purpose

to maintain the same directory structure for source and test
source:

before test:

after test:

Remarks

PR introduces (a) breaking change(s): <yes/no>
PR introduces (a) deprecation(s): <yes/no>

Thanks for the PR @liuxiaocs7!


[HUDI-1769]Add download page to the site

What is the purpose of the pull request
Add the download page to the asf site
Brief change log
Added download.md and add download tab on the navigation bar.
Verify this pull request
Built the site locally. Tested on chrome and safari.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@garyli1019 the download links are pointing to dist.apache.org tar balls??
while we are at it, can we also update release cwiki page with updating this for each release?

@garyli1019 the download links are pointing to dist.apache.org tar balls??
while we are at it, can we also update release cwiki page with updating this for each release?

@vinothchandar fixed. Changed the link to the mirror site like other apache projects. And added the updating download.md step to the README instructions.

lgtm. @nsivabalan if you are happy too, we can land this


Release 1.15.11

Added support for MQTT Will

@alexjhawk The MQTT Spec defines it as "Will".

@alexjhawk The MQTT Spec defines it as "Will".

It is shortened as 'will' in many places, but I think it would be best to be more descriptive here. We have had a customer request LWT functionality in the past, and when looking up 'MQTT will,' it is more commonly referred to as 'last will' or 'last will and testament'. https://www.hivemq.com/blog/mqtt-essentials-part-9-last-will-and-testament/

@alexjhawk The MQTT Spec defines it as "Will".

I think its more appropriate to stick with the spec than adopt the tech slang term.

Ive modified the Changelog to address the use of LWT


chore: Update and cleanup project

Description
A bunch of project cleanups and updates. The changes are in individual commits for easier review. Please let me know if there are any questions with individual patches. See explanations below for more information:
Clean up lock files
A project should have either a yarn.lock file or a package-lock.json file. The existence of yarn.lock in the root folder suggests usage of Yarn which results in the following warning:
package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.

The React Native sample project in the example folder should use Yarn (yarn.lock already added in 85f17d01f7824ab29ff4e698a5c2101e50e9d134) - I deleted the redundant package-lock.json file here.
Remove extra Gradle wrapper
There was a misplaced Gradle wrapper sitting in the android/ folder. The executable bit of gradlew was not even set. Removed the unnecessary scripts together with the wrapper jar file.
Update .gitignore
Consolidated the .gitignore files and removed some duplications
Update example project
Minor UI updates and improvements, as well as committed the automated updates to the Xcode project file by CocoaPods (pod install) to avoid a dirty working directory after running this necessary set-up step.
Simplify Podfile and add Podfile.lock
There were some redundant/extraneous declarations/statements in Podfile. Also adds the missing Podfile.lock.
Update refresh-example script
Added the pod install step in the ios folder of the example project. Running ./refresh-example.sh should now result in zero changes (if the dependencies are up-to-date). Also reordered a couple of steps to remove the temporary directory earlier (pod install requires Podfile that was saved in the temp folder).
Compatibility

OS
Implemented

iOS
✅

Android
✅

Windows
✅ *

* Due to lack of devices/OS I was not able to test the Windows part.
Tested on Android and iOS devices and emulator/simulator. Opened and successfully ran the project both in Android Studio 3.3.2 and Xcode 10.2

Nice!! Thank you!
Can I ask you to use the refresh-example script and only commit the things necessary in the example to implement your changes, not the things that vary from the clean result of the refresh? That helps me keep track of what is really changing, and what I really need to change as new react-native versions come out. For example, I do not commit the XCode perturbations because it seems to me they are pointless and always happen so have little value, but knowing when the link command has altered behavior on a clean project is very valuable. Just doing some git reset --soft HEAD~1 or similar work and cleaning those changes can help you keep your directory clean for commits
I really appreciate the chore work though, that's what helps this project remain supportable in the face of new users that aren't familiar with react-native processes...

I like this in theory but will close it if feedback is not handled, sorry

@mikehardy - So sorry for the long delay! I've had a lot going on recently (got married last week! ;))
Anyways, I just rebased the branch and updated this PR. I reverted the changes to the Xcode project.pbxproj file. Please check again. Thanks!

Hey! Best excuse ever, congratulations :-). Okay, I'll check this out again, thanks

I noticed the eslint dependencies were already removed on master due to the changes in 85f17d0 - So that comment is already obsolete ;)

Yeah, I updated the example yesterday as I was working on other things :-), best to just leave the example alone - I obliterate it semi-frequently with the refresh script and I have an interest in keeping that working to minimize support load, so we can leave all of it out here really. Let me look through the other stuff

Well - this started going in a direction that I'm not comfortable with at the moment. pod install as a requirement is not on my list things I want
I know RN0.60 is going there but the pod install requirement itself for project startup was a matter of some debate, and we (react-native-community / react native) tried to avoid it. It was not possible but for RN0.59 (still current) I do not want to add it yet as it is a startup barrier. I want to do it only when strictly required by the RN dependency (which would be whenever I refresh the example to be RN0.60)
Can you just drop all the example changes? Even the ones that are for imports and formatting make it skew farther from the template, and will make it more difficult to track template changes (because App.js will change with RN0.60 as well)

Well - this started going in a direction that I'm not comfortable with at the moment.

Hmm not sure what you mean. I did not add CocoaPods in this PR... It's not really part of this PR to remove the Podfile. Is this something you want/not want?
I also just updated the main PR description. Could you take a look and let me know if that clarifies things a bit?

Sorry if I was unclear, I mean with the yarn dep in the example, and the pod install + Podfile.lock. We do have a Podfile in the example but that was also support-load related. We got so many pod issues I dropped an example in there I could point to, but we are not emphasizing it yet so I had not (and don't want to) start directing people to pod things and make it a requirement yet, lest I become the leading edge of support for react-native's switch to pods by default :-). I would like to be in the middle of the adopter curve there.
So I'd like to leave the example alone completely for now

Rebased and resolved conflicts (again). It would be great to get this merged soon. Thanks!

I removed the commits that update the project file and add the Podfile.lock. Just as before, running pod install will leave a dirty working directly and untracked files.

re: conflicts, indeed, we're a beehive here right now - maintaining PRs is not a lot of fun. Thanks for working with me. Also thanks for going with me on the example bit. I'm okay with locally dirty files in the example for the moment for the reasons mentioned. We'll have that area cleaner shortly after RN0.60 releases (and if you want to collaborate there since your Pod kung-fu seems stronger than mine, it will be very welcome)

Sounds good, thanks @mikehardy :+1:


refactor: incorporate code review changes

Please update the spec files in each module accordingly. This includes the new eventEventUpdateParams  in each module and the updated tx signature of MsgCreateStaker (commission arg)


RDS update stuck on boot

Using envoy at 71c0fd695e992e21d2e0439eb14b449fa0b8aea4 and java-control-plane we've run into an issue where envoy fails to load all the RDS resources at boot.
Envoy is configured with two listeners, using two route tables (standard_egress/standard_ingress). Envoy begins by requesting the routes for just standard_egress, and gets this back with version = X. Following this it requests [standard_egress, standard_ingress] but since the nonce is the same the control plane ignores the request. Finally, envoy requests [standard_egress, standard_ingress] again with version X. Because the version matches what the control plane has, it doesn't send the requested resources right away, instead waiting for the next time the resources are updated. This leaves Envoy without a route table for standard_ingress, resulting in 404s.
Raw logs from the control plane:
2018-05-16T08:10:52,156  INFO [rpc-server-dispatcher-9] server.DiscoveryServer - [1] request type.googleapis.com/envoy.api.v2.RouteConfiguration[standard_egress] with nonce  from version
2018-05-16T08:10:52,158  INFO [rpc-server-dispatcher-9] cache.SimpleCache - responding for type.googleapis.com/envoy.api.v2.RouteConfiguration from node slug--LOCAL at version  with version 0f9e47ac-faba-4b9f-a3bd-5e94c6b7dc35
2018-05-16T08:10:52,158  INFO [rpc-server-dispatcher-9] server.DiscoveryServer - [1] response type.googleapis.com/envoy.api.v2.RouteConfiguration with nonce 1 version 0f9e47ac-faba-4b9f-a3bd-5e94c6b7dc35
2018-05-16T08:10:52,162  INFO [rpc-server-dispatcher-10] server.DiscoveryServer - [1] request type.googleapis.com/envoy.api.v2.RouteConfiguration[standard_ingress, standard_egress] with nonce  from version
2018-05-16T08:10:52,168  INFO [rpc-server-dispatcher-12] server.DiscoveryServer - [1] request type.googleapis.com/envoy.api.v2.RouteConfiguration[standard_ingress, standard_egress] with nonce 1 from version 0f9e47ac-faba-4b9f-a3bd-5e94c6b7dc35
2018-05-16T08:10:52,170  INFO [rpc-server-dispatcher-12] cache.SimpleCache - open watch 5 for type.googleapis.com/envoy.api.v2.RouteConfiguration[standard_ingress, standard_egress] from node slug--LOCAL for version 0f9e47ac-faba-4b9f-a3bd-5e94c6b7dc35

What is the expected way of handling this? Not clear to me whether this is a control plane bug or an envoy bug - should the control plane update its version when new resources are requested? Or should envoy reset its version when it sends a request with new resources? or maybe wait until it knows what resources to request before sending the xDS request?

@snowp sorry to clarify you are using ADS, right?

Yes, this is using ADS.
On Wed, May 16, 2018 at 6:48 PM Matt Klein<EMAIL_ADDRESS>wrote:

@snowp https://github.com/snowp sorry to clarify you are using ADS,
right?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/envoyproxy/envoy/issues/3399#issuecomment-389588352,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB7fxjb4PTfox__BgszbCrFs59na6IjPks5tzFhigaJpZM4UBPzX
.

Envoy should be pausing RDS updates during LDS config updates such that it presents only the [standard_egress, standard_ingress] request. I'd be interested in knowing why we're seeing multiple RDS requests initially.

Yeah, please do this, I agree that this is the correct behavior.

@mpuncel not sure I follow that example. In the case of an LDS update, we halt any new RDS requests until the full LDS update has been processed, see https://github.com/envoyproxy/envoy/blob/c7ba07ea3a0a49fec1ac24c763eb9f0a2e13c06b/source/server/lds_api.cc#L45. Only after the full LDS update is processed will RDS do something new, once we have the new route resource hint.
@snowp BTW, we probably have the same issue for static clusters and EDS.

lets say there's a listener named L1 that has a route config named R1. During steady state, I'd expect there to be an LDS discovery request (no resource hints) and an RDS discovery request with a resource hint for R1. If L2 is then added with route config R2 via the control plane, my understanding is that the current implementation of java-control-plane would send a discovery response for LDS and RDS simultaneously, which creates a race between RDS sending another request with just the R1 resource hint (perhaps before lds_api.cc has even seen the LDS response) or the LDS response being processed and the next RDS requests having R1 and R2 as hints.
I think I might have answered my own question though, I think the control plane should be able to notice that the R2 resource hint was added and send another response even before version change. I'm not sure java-control-plane does this right now, because it seems like that would have been another resolution to the problem snowp saw.
Basically i'd expect the 2018-05-16T08:10:52,168  INFO [rpc-server-dispatcher-12] server.DiscoveryServer - [1] request type.googleapis.com/envoy.api.v2.RouteConfiguration[standard_ingress, standard_egress] with nonce 1 from version 0f9e47ac-faba-4b9f-a3bd-5e94c6b7dc35 request to get a response because standard_ingress hadn't been requested before. So maybe there's a bug in java-control-plane as well?


Authentication tests and Fetch stories test

Authentication i.e Login and Register of user tests and fetching stories tests added.
I have used unittest package of python for testing.
#104

Closes #104

@Tushar19varshney @Anmolbansal1 Please take a look at #145 if you have time.

@ivantha Sure :smile:


Matchers for delayed ActionMailer deliveries

I'm hoping this is the right place for feature suggestions/requests.
We have specific matchers that help us check what ActiveJob jobs have been queued (have_enqueued_job), but we don't have similar things for ActionMailer.
I've done some looking around and it seems like this is the general way used to test that an email has been sent. This doesn't feel quite right because I'm not describing which email has been sent, and I need to dive into ActionMailer to get the number of deliveries.
expect {
  post :create, params: { user: attributes_for(:user) }
}.to change(ActionMailer::Base.deliveries, :size).by(1)

It would be great if we had additional matchers for emails so that we could make more specific and descriptive specifications.
expect {
    post :create, params: { user: attributes_for(:user) }
}.to have_enqueued_mailer(UserMailer, :welcome).with(User.first)

I hope I haven't missed existing functionality similar to this, I have had a good look around but couldn't find anything of the sort.

Better late than never, but this custom matcher should more or less do the trick:
RSpec::Matchers.define :have_enqueued_mailer do |mailer_klass, mailer_action|
  match do |actual|
    actual.call
    @mailer_klass, @mailer_action, _, *@args = enqueued_jobs[0][:args]

    @mailer_klass == mailer_klass.to_s &&
      @mailer_action == mailer_action.to_s
  end

  chain :with do |*args|
    @args == args
  end
  supports_block_expectations
end

Is this issue duplicate of #1901?

Yes @ybiquitous! :)
Thanks @cgat for the matcher.
Closing for #1901


GN-4516: solve svg cors issues

Overview
This PR ensures that svgs rendered by the au-icon component are rendered inline, instead of requesting a symbolset file.
It does so by overriding the au-icon component provided by the @appuniversum/ember-appuniversum package.
In order to render the svgs inline, this PR adds the ember-svg-jar package.
This PR solves cors issues and issues when trying to correctly resolve the svg symbolset file from the appuniversum package.
This PR also removes the svgxuse package, which was used as a workaround for the cors issues occurring when request the svg symbolset file.
connected issues and PRs:
GN-4516
How to test/reproduce

npm install
npm start
check if the symbols are correctly rendered
no cors issues should appear
when inspecting the icons using the browser-inspector, you should notice that the svg code is rendered inline

Checks PR readiness

[x] UI: works on smaller screen sizes
[x] UI: feedback for any loading/error states
[x] Check cancel/go-back flows
[x] Check database state correct when deleting/updating (especially regarding relationships)
[x] changelog
[x] npm lint
[x] no new deprecations

Svgxuse did work, but it was a bit hacky IMO. Svgxuse resolves the svg symbolset correctly after the browser fails to do so (due to cors errors) so the cors error from the first browser request still fails and shows an error in the console. Svgxuse adds cors header to its second request so it succeeds.

Okay, good to know. Definitely makes this approach a ton better then, and a lot less chance of something failing too.


win10] error[E0277]: the size for values of type str cannot be known at compilation time

Windows 10 专业版 22H2 19045.2546
rust 1.71.0
poem v1.3.55

error[E0277]: the size for values of type `str` cannot be known at compilation time
    --> D:\rust\cargo\registry\src\rsproxy.cn-8f6827c7555bfaf8\poem-1.3.55\src\middleware\tracing_mw.rs:50:41
     |
50   |             span.record("path_pattern", path_pattern.0.as_ref());
     |                  ------                 ^^^^^^^^^^^^^^^^^^^^^^^ doesn't have a size known at compile-time
     |                  |
     |                  required by a bound introduced by this call
     |
     = help: the trait `Sized` is not implemented for `str`
note: required by a bound in `Span::record`
    --> D:\rust\cargo\registry\src\rsproxy.cn-8f6827c7555bfaf8\tracing-0.1.35\src\span.rs:1194:30
     |
1194 |     pub fn record<Q: ?Sized, V>(&self, field: &Q, value: &V) -> &Self
     |                              ^ required by this bound in `Span::record`

For more information about this error, try `rustc --explain E0277`.
error: could not compile `poem` (lib) due to previous error
warning: build failed, waiting for other jobs to finish...

update to poem = "^1.3.57" is ok

update to poem = "^1.3.57" is ok

It's not caused by poem


fix deployment apiversion error

fix deployment apiversion error

Hi, this behavior is not encouraged. You only need to handle a language in a PR, such as zh.
If you handle this file for zh-cn, you can run the command below to check difference from en:
./scritps/lsync.sh content/zh-cn/docs/concepts/cluster-administration/manage-deployment.md

If this command doesn't work, you can try to use 'diff' or vimdiff.

Please see the above comment and remove changes for other languages.
/hold

Hey @cleverhu, are you still interesting in contributing kubernetes/website?

Hey @cleverhu, are you still interesting in contributing kubernetes/website?

I'm sorry for late.It's a little complicated. I'll try to modify it. If I need to modify too much, I'll close the pr.

the result of
./scripts/lsync.sh /Users/daocloud-cleverhu/GolandProjects/qy-master/website/content/zh-cn/docs/concepts/cluster-administration/manage-deployment.md

is
/Users/daocloud-cleverhu/GolandProjects/qy-master/website/content/zh-cn/docs/concepts/cluster-administration/manage-deployment.md is still in sync

/remove-language de
/remove-language fr
/remove-language id
/remove-language it
/remove-language ja
/remove-language ru
/hold cancel

/lgtm
/approve


Installer issue on Linux Mint (ubuntu based distro)

Linux Mint 20.3
Kernel: 5.4.0-120-generic
Cinnamon version:  5.2.7
For the installer packages for Linux I am not able to install a Temurin version 8 JDK
One step 3 for the "Configure the Eclipse Adoptium apt repository:" I am met with permission denied
even using sudo doesn't help for that command
it shows the following:
tee: /etc/apt/sources.list.d/adoptium.list: Permission denied
deb [signed-by=/usr/share/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb una main
I needed a version 8 for Ren'py

Solution 1
Run as root
sudo su
echo "deb [signed-by=/usr/share/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb $(awk -F= '/^VERSION_CODENAME/{print$2}' /etc/os-release) main" | tee /etc/apt/sources.list.d/adoptium.list

Solution 2
The sudo command does not work with pipe. https://linoxide.com/use-sudo-when-using-with-redirect-or-pipe/
echo "deb [signed-by=/usr/share/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb $(awk -F= '/^VERSION_CODENAME/{print$2}' /etc/os-release) main" | sudo tee /etc/apt/sources.list.d/adoptium.list

I get the following after running the steps now:
"E: The repository 'https://packages.adoptium.net/artifactory/deb una Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default."
Which then results in not being able to locate the package for temurin-8-jdk

You get this error because debian/ubuntu does not have a distribution named una. una is the code name of Linux Mint.
Linux Mint 20.3 is based on Ubuntu focal. The solution is replacing una with focal.
$ cat /etc/apt/sources.list.d/adoptium.list
deb [signed-by=/etc/apt/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb una main
$ echo 'deb [signed-by=/etc/apt/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb focal main' > /etc/apt/sources.list.d/adoptium.list
$ sudo apt update
Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease
Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease
Hit:3 http://archive.canonical.com/ubuntu focal InRelease
Get:4 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Hit:5 https://packages.adoptium.net/artifactory/deb focal InRelease
Get:6 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB]
Ign:7 http://packages.linuxmint.com una InRelease
Hit:8 http://packages.linuxmint.com una Release
Fetched 222 kB in 4s (60.7 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done

it gave me a error the following errorrs:
bash: /etc/apt/sources.list.d/adoptium.list: Permission denied
$: command not found
Hit:1: command not found
Hit:2: command not found
Hit:3: command not found
Hit:4: command not found
Hit:5: command not found
Hit:6: command not found
Hit:7: command not found
Hit:8: command not found
bash: syntax error near unexpected token `('
Reading: command not found
Building: command not found
I don't know much about these commands, I simply copied it via the double square icon and placed them into terminal.

The lines starting with a $ are my commands and the rest are outputs.
The complete commands are the following.
# You must have superuser privileges
sudo su

# Ensure the necessary packages are present:
apt install -y wget apt-transport-https

# Download the Eclipse Adoptium GPG key
mkdir -p /etc/apt/keyrings
wget -O - https://packages.adoptium.net/artifactory/api/gpg/key/public | tee /etc/apt/keyrings/adoptium.asc

# Configure the Eclipse Adoptium apt repository
echo 'deb [signed-by=/etc/apt/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb focal main' > /etc/apt/sources.list.d/adoptium.list

# Install the Temurin version you require
apt update
apt install temurin-17-jdk

The lines starting with a $ are my commands and the rest are outputs.
The complete commands are the following.
# You must have superuser privileges
sudo su

# Ensure the necessary packages are present:
apt install -y wget apt-transport-https

# Download the Eclipse Adoptium GPG key
mkdir -p /etc/apt/keyrings
wget -O - https://packages.adoptium.net/artifactory/api/gpg/key/public | tee /etc/apt/keyrings/adoptium.asc

# Configure the Eclipse Adoptium apt repository
echo 'deb [signed-by=/etc/apt/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb focal main' > /etc/apt/sources.list.d/adoptium.list

# Install the Temurin version you require
apt update
apt install temurin-17-jdk

This one worked for me. However, the following command to configure the apt repository, given on official documentation, didn't work for me.
echo "deb [signed-by=/etc/apt/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb $(awk -F= '/^VERSION_CODENAME/{print$2}' /etc/os-release) main" | tee /etc/apt/sources.list.d/adoptium.list

Try
echo "deb [signed-by=/etc/apt/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb $(awk -F= '/^VERSION_CODENAME/{print$2}' /etc/os-release) main" | sudo tee /etc/apt/sources.list.d/adoptium.list

This solution works for me,
https://askubuntu.com/questions/1375383/how-to-properly-install-temurin-jdk-with-update-alternatives
should add sudo" to each command in document

@gdams Are we posting workarounds for other O/S's anywhere in particular or letting SEO solve this for us?


Code fixes adding carriage returns to LF documents

Code fixes are generating files with mixed LF and CRLF line endings. See:

codeFixAddMissingMember5
codeFixAddMissingMember7
codeFixSuperAfterThis
codeFixSuperCall
importNameCodeFix* (see comment in fourslash.ts tagged GH#18445)

Do we know if this issue is test-only?  That is, have we seen the product produce inconsistent line endings?

In importNameCodeFixNewImportFile3.ts, for instance, the test explicitly expects \n and the host explicitly provides \r\n.  Is this bug just asking that we change the value used in the host during fourslash tests?

The settings should be specifying \n though. May be due to #18291.

When I debugged, it was getting the newline string from the host.  It sounds like that may indeed be the issue, but it also sounds like that would only affect the tests since normally the refactoring options and the host would agree, no?

We were looking at this earlier today and it seemed to work, but it looks like vscode's formatting settings are not making it through to "getEditsForRefactor" requests -- waiting on a fix for #18328 before we can test this out.


added old open gl help message, hard to test

I think this makes sense? Could close #32. I wanted to add this because I think it may be common. I had the same issue when running this at first.

:+1: Looks good to me, thanks


Change logo to have white fill inside and red outside

I guess it's a chore instead of feat?

@jaakkonakaza if it helps, he's how we designate the prefix for Skeleton:
https://next.skeleton.dev/docs/resources/contribute/get-started#pr-branch-conventions
So chore/ would be right. Don't stress if you miss this though.  I forget to to it from time to time too :)


Add CloudScraper (T)

{
  "id": "<ID>",
  "name": "CloudScraper (T)",
  "type": "url",
  "url": "https://github.com/jordanpotti/cloudscraper"
},

Thanks! Added


⚠ GitLab Server has degraded performance

In fbd81b4, GitLab Server (https://gitlab01.its-telekom.eu) experienced degraded performance:

HTTP code: 200
Response time: 9900 ms

Resolved: GitLab Server performance has improved in 5e018fe.


Add CI to frameworks repos

There are a number of repos without any real CI (remotedialer, dynamiclistener, lasso). We should add CI to at least build/run tests

[ ] remotedialer
[ ] dynamiclistener
[ ] lasso
[ ] apiserver
[ ] norman

I will split this up into sub tickets.

I'm going to close this old ticket as dupe now: rancher/tasks#201

I see that issue lists other things like golangci-lint, go fmt and go mod verify checks. For the purpose of #44970 we're looking at just adding go test. This will at least gives us more confidence when PRs are opened (especially for PRs from renovate).
When we migrate to GHA proper, or perhaps later, we should revisit which ones are missing. Some are non-trivial to add, for example:

apiserver lint fails for a few reasons, like not checking an error. Fixing this should go through the usual process to reduce risk of regression. (Assuming we want to actually add the error check and not just explicitely ignore it)
Same for remotedialer (github issue not yet created).

Note that if golangci-lint runs and is trivial to fix then I'll add it as well as I go through those missing.

GHA migration will cover the scope of this ticket.


vm: integrate ubpf & add compat layer for different vm implementations

Closes #90
This PR will replace simple-jit with ubpf, and uses a compat layer to provide unified interface for llvm-jit and ubpf. Interfaces will be in the same as the current vm
What has been done

vm/llvm-jit now comes to a new target named bpftime_llvm_jit_vm, which could be used and built independently
vm/ubpf-vm now comes to a new target named bpftime_ubpf_vm, which is a wrapper on ubpf, providing unified interface to operate on ubpf. We also applies some patches to the eBPF code your load, so it would be accepted by ubpf (since ubpf lacks many things, such as lddw helpers now)
All vms now could be accessed via a unified interface defined in target bpftime_vm (located at vm/vm-core), it provides C interface, include a series functions named ebpf_*.
bpftime_vm_compat is a internal header only target, and is not designed to be used by users. Only vm implementations and bpftime_vm should use this target

How to add a vm?

Add a new target. It should implement a subclass of vm/compat/include/bpftime_vm_compat.hpp (This target is a header only library, we use it to avoid circular dependency between vm implementation and bpftime_vm), and function std::unique_ptr<bpftime_vm_impl> create_vm_instance();
Link your target to bpftime_vm
Now if the user links with bpftime_vm, he will use your vm!
You don't need to modify bpftime_vm or bpftime_vm_compat. You just need to modify CMakeLists to make bpftime_vm link on your target

I think we can directly use the target vm, instead of a new target called compart layer.

The documents in https://eunomia.dev/bpftime/documents/build-and-test/#build-and-install-the-complete-runtime-in-release-modewith-llvm-jit doesn't need to be changed, right?

The documents in https://eunomia.dev/bpftime/documents/build-and-test/#build-and-install-the-complete-runtime-in-release-modewith-llvm-jit doesn't need to be changed, right?

A little change is needed. If user want to build vm independently, BPFTIME_BUILD_STANDALONE_VM should be set. If user want to build llvm-jit indenendently, BPFTIME_BUILD_STANDALONE_LLVM_VM should be set. These options are used for including external dependencies, such as spdlog

Remember to submit a PR in the website repo please.


[CP] Ignore invalid description in vswhere.exe JSON output

The flutter doctor, flutter build, and flutter run commands fail on Windows using certain locales when using the latest version of Visual Studio. This cherrypicks the fix to the latest stable release: https://github.com/flutter/flutter/commit/dcbdff087b7f162dcd9917aad278c7cf2515d8a6 from https://github.com/flutter/flutter/pull/106836.
Cherry pick issue: https://github.com/flutter/flutter/issues/106920
Fixes: https://github.com/flutter/flutter/issues/106601
Pre-launch Checklist

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I read the Tree Hygiene wiki page, which explains my responsibilities.
[x] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[x] I signed the CLA.
[x] I listed at least one issue that this PR fixes in the description above.
[x] I updated/added relevant documentation (doc comments with ///).
[x] I added new tests to check the change I am making, or this PR is test-exempt.
[ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@muditatandon @CaseyHillers Any idea what is going on with these failing tests?

The web tests are timing out (likely due to something crashing)

Linux build_tests_1_2 failing because ninja not found:
[   +7 ms] executing: [build/linux/x64/debug/] cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DFLUTTER_TARGET_PLATFORM=linux-x64 /b/s/w/ir/x/w/flutter/dev/integration_tests/flutter_gallery/linux
[  +39 ms] CMake Error: CMake was unable to find a build program corresponding to "Ninja".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
[   +1 ms] CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
[        ] -- Configuring incomplete, errors occurred!

I wonder if this dependency changed upstream since the branch?


Reorganized sql readme.md and extracted Java section to a separate file.

Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[ ] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

Automation for azure-sdk-for-python

Automation for azure-sdk-for-ruby

Automation for azure-sdk-for-js

Automation for azure-sdk-for-go

Can one of the admins verify this patch?

Automation for azure-sdk-for-java

@sarangan12 please hold off merge until @jianghaolu and @jaredmoo sign-off.

Automation for azure-sdk-for-node

It looks like you're doing multi-version, is that right? In that case I think you should use the package-pure-* tags. The purpose of package-composite is to have only 1 version of each api, but multi-version allows you to use all versions of each api.

@jaredmoo The idea is to generate multi-api java client based on SQL composite v3 tag. In the existing readme.md there are already package-pure-* tags sections that contain (in some cases) more swagger documents than composite v3. e.g.  Tag: package-pure-2014-04 contains 29 swaggers while composite v3 tag uses only 15. Is there any reason why v3 does not include the rest? If non - I can definitely remove new tags and just use existing -pure- tags.

package-composite-v3 does not contain all documents because .NET doesn't support multi-version, so we only pick the latest of each API. This is why it doesn't have all 2014 documents for example, because many of those APIs now have later versions that package-composite chooses instead.

@jaredmoo thanks for explanation. I've reworked the PR and now java multi-version uses -pure- tags. Please take a look.


support for importing multiple CA in jks certificates

With different Issuer's for server and client,  and with client authentication enabled in servers, it is required to provide a truststore for the servers that has  all the CA certificates that clients' keys were signed by, and vice versa
How do we enable this ?
Note:
server certificate generated using Kind : Issuer1(CA1 )
client certificate generated using kind: Issuer2(CA2)
/kind feature

/remove-lifecycle stale


New version: Microsoft.WindowsSDK.10.0.22621 version 10.1.22621.2428

[X] Have you signed the Contributor License Agreement?
[X] Have you checked that there aren't other open pull requests for the same manifest update/change?

[X] Does your manifest conform to the 1.5 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

@stephengillie - This PR is missing the AppsAndFeaturesEntries which are required as multiple versions of WindowsSDK can be installed side-by-side. I don't know if your Watch-PRTitles script has a "notes" section, but it might be a good idea for some packages like this where special checks need to be made

@Trenly I'll have to construct one. Do you have a list of problematic packages like this one?

@Trenly I'll have to construct one. Do you have a list of problematic packages like this one?

Unfortunately not; I just know them when I see the PRs :(


Add eraseEither combinator to JsonEncoder/JsonCodec

Hello,
I had a need for this encoder and codec (during a spartan session) and we thought with @jdegoes it could be added to zio.json.
Basically, I had this kind of structure:
    sealed trait StringOrNumber extends Any
    object StringOrNumber {
      final case class Str(s: String) extends AnyVal with StringOrNumber
      final case class Num(s: BigDecimal)   extends AnyVal with StringOrNumber
}

for which I can derive a codec based on an either but the result JSON must be without Right or Left fields so I needed to code another encoder.
We gave the name eraseEither because it's kind of forgetting the either structure to give just the value.
I locally launch compile and test and prepare prepare from the contributor doc it seems all.

I put the eraseEither in the companion object of the JsonCodec it is more practical to use when building a new codec.


Add a glossary to SwiftSyntax

The glossary doesn’t contain much yet but I feel it’s always good to have such a document because it will make it easy to add additional entries to it as they come up.
rdar://110733511

@swift-ci Please test

Great initiative. In my exposure to SwiftSyntax, I’ve encountered a few terms that come to mind that were initially unclear to me, or meant something slightly different than what I first assumed.
Here are some that come to mind in case you’re looking for inspiration for what could be included in this glossary:

Node
Layout
Members vs Children
Raw syntax
Arena

Thanks for the suggestions @jpsim. I added some more entries to the glossary.

@swift-ci Please test

I added some more entries to the glossary.

Thanks, these are really helpful.

@swift-ci Please test


Generating/writing an APDL input file

Description of the feature
I assume that the pymapdl somewhere in the background creates a .inp file to feed into the ansys apdl exe.
Would it be possible to be able to access /write that that .inp to file?
This would allow users to double check pymadpl method by running the .inp in Ansys classic.
Steps for implementing the feature
Divert to write instead of solve (add an argument to the solve command?)
This is almost the opposite of the scripts used to read APDL scripts and convert to pymapdl inputs.
The reason is to be able to double check pymapdl and also to be able to generate shareable .inp files that other non py users could run with their APDL instances.
Useful links and references
No response

Hi @mcMunich,

I assume that the pymapdl somewhere in the background creates a .inp file to feed into the ansys apdl exe.

We're doing something close to that. PyMAPDL talks to the MAPDL process via gRPC (think of it as a connection over the network, even if local) and sends either individual commands, series of commands, or entire blobs of data (when plotting, getting the mesh, etc).

Would it be possible to be able to access /write that that .inp to file?
This would allow users to double check pymadpl method by running the .inp in Ansys classic.

That's been a request in the past and we implemented it with log_apdl=True within launch_mapdl. See:
APDL Command Logging

Hi @mcMunich
I'd like to add that the lgwrite command is still available just in case the log_apdl flag was not set.  There is a lot more commands written due to how the MAPDL aaS instance is accessed but most text editors can be used to find/delete the non-needed commands.  Mainly a lot of /out and /out,anstmp commands.
Also if you want to spot check the model there is always the open_gui() option.
Mike

Thanks @mikerife. I think this is a better solution because this is more likely to actually replicate what's happening within MAPDL rather than what's just being sent from the client side.
I'm leaving this open to have @germa89 investigate some additional post-processing to cleanup the output from lgwrite and just removing /out/ and /out,anstmp when in server mode. Should be an easy cleanup, and we need to check if lgwrite works when the server is remote on docker.


Why BottomLip have 12 points?

I search dlib 68 point picture, bottom lip only have 8 points.

49, 61, 65 and 55 are belonged to bottom lips.

It's means 49,61,65,55 belongs to top lip and bottom lip?

yes

Wait. I cannot remember why did I decide that two 12 point are lips.
I do not have research or document.

Oh, I found.
Please refer https://github.com/ageitgey/face_recognition/blob/master/face_recognition/api.py

Ok, thanks a lot!


[Question]: Mujoco-specific, not recommended to constraint action within valid bound ?

It seems the SpinningUp implementations do not constraint actions according to env.observation_space.low and env.observation_space.high. I guess it's even discouraged to do action clipping after sampling from learned Gaussian distribution and this might be specific to Mujoco environment where the reward function contains penalty to the magnitude of actions i.e. 'reward_ctrl', so it automatically penalize when the action is out of bounds [-1, 1].

Correct, for the policy optimization algorithms. DDPG/TD3/SAC do action clipping, but they are justified in the context of the algorithms.