Handling very long connection timeouts

Long story short
Having a big list of hosts to connect to, some of them timeout and aiohttp waits eternally. What is the proper way to handle this case?
Expected behaviour
aiohttp should have some kind of connection timeout.
Actual behaviour
aiohttp hangs
Steps to reproduce
An extreme example:
$ time curl -verbose https://soso.com
* Rebuilt URL to: https://soso.com/
* Hostname was NOT found in DNS cache
*   Trying <IP_ADDRESS>...
* connect to <IP_ADDRESS> port 443 failed: Connection timed out
*   Trying <IP_ADDRESS>...
* After 86339ms connect time, move on!
* connect to <IP_ADDRESS> port 443 failed: Connection timed out
* Failed to connect to soso.com port 443: Connection timed out
* Closing connection 0
curl: (7) Failed to connect to soso.com port 443: Connection timed out

real    3m33.671s
user    0m0.027s
sys     0m0.023s

What is the proper way to handle this? Example code:
#!/usr/bin/env python

import aiohttp
import asyncio
import json
import time

async def fetch(session, url):
    try:
        async with session.get(url) as response:
        # rs = await response.text()
        print(str(response.status) + " " + url)
    except Exception as e:
        print("Oops: " + url + " " + str(type(e)) + str(e))

if __name__ == '__main__':

    t = time.monotonic()

    watchlist  = {
        "https://360.cn": {},
        "https://chinadaily.com.cn": {},
        "https://cntv.cn": {},
        "https://gmw.cn": {},
        "https://googleusercontent.com": {},
        "https://hao123.com": {},
        "https://imdb.com": {},
        "https://live.com": {},
        "https://naver.com": {},
        "https://nicovideo.jp": {},
        "https://pixnet.net": {},
        "https://qq.com": {},
        "https://rakuten.co.jp": {},
        "https://sina.com.cn": {},
        "https://sohu.com": {},
        "https://soso.com": {},
        "https://t.co": {},
        "https://tianya.cn": {},
        "https://xinhuanet.com": {},
        "https://xvideos.com": {},
        "https://yahoo.co.jp": {},
        "https://yahoo.com": {},
        "https://yandex.ru": {}
    }

    loop = asyncio.get_event_loop()
    conn = aiohttp.TCPConnector(verify_ssl=False)
    asyncio.set_event_loop(loop)

    futures = []
    with aiohttp.ClientSession(loop=loop, connector=conn) as session:
        for url, url_check in watchlist.items():
            futures.append(asyncio.ensure_future(fetch(session, url)))

        loop.run_until_complete(asyncio.gather(*futures))

    print("Done in ", str(time.monotonic() - t))

The above will wait many minutes until it timeouts.
Using aiohttp.Timeout seemed like a good candidate to take care of this, but I've found that once the timeout is reached, all other futures throw an concurrent.futures._base.TimeoutError, so it appears to work "globally". So if we modify fetch like so:
async def fetch(session, url):
    try:
        with aiohttp.Timeout(5):
            async with session.get(url) as response:
                # rs = await response.text()
                print(str(response.status) + " " + url)
    except Exception as e:
        print("Oops: " + url + " " + str(type(e)) + str(e))

We now get:
Oops: https://googleusercontent.com <class 'aiohttp.errors.ClientOSError'>[Errno -2] Cannot connect to host googleusercontent.com:443 ssl:True [Can not connect to googleusercontent.com:443 [Name or service not known]]
200 https://yandex.ru
200 https://t.co
200 https://sohu.com
200 https://yahoo.com
200 https://gmw.cn
Oops: https://live.com <class 'aiohttp.errors.ClientOSError'>[Errno 111] Cannot connect to host live.com:443 ssl:True [Can not connect to live.com:443 [Connect call failed ('<IP_ADDRESS>', 443)]]
Oops: https://360.cn <class 'aiohttp.errors.ClientOSError'>[Errno 1] Cannot connect to host 360.cn:443 ssl:True [Can not connect to 360.cn:443 [[SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:645)]]
200 https://qq.com
Oops: https://pixnet.net <class 'aiohttp.errors.ClientOSError'>[Errno 111] Cannot connect to host pixnet.net:443 ssl:True [Can not connect to pixnet.net:443 [Connect call failed ('<IP_ADDRESS>', 443)]]
Oops: https://tianya.cn <class 'concurrent.futures._base.TimeoutError'>
Oops: https://xinhuanet.com <class 'concurrent.futures._base.TimeoutError'>
Oops: https://soso.com <class 'concurrent.futures._base.TimeoutError'>
Oops: https://hao123.com <class 'concurrent.futures._base.TimeoutError'>
Oops: https://nicovideo.jp <class 'concurrent.futures._base.TimeoutError'>
Oops: https://xvideos.com <class 'concurrent.futures._base.TimeoutError'>
Oops: https://naver.com <class 'concurrent.futures._base.TimeoutError'>
Oops: https://imdb.com <class 'concurrent.futures._base.TimeoutError'>
Oops: https://chinadaily.com.cn <class 'concurrent.futures._base.TimeoutError'>
Oops: https://cntv.cn <class 'concurrent.futures._base.TimeoutError'>
Oops: https://rakuten.co.jp <class 'concurrent.futures._base.TimeoutError'>
Oops: https://yahoo.co.jp <class 'concurrent.futures._base.TimeoutError'>
Oops: https://sina.com.cn <class 'concurrent.futures._base.TimeoutError'>
Done in  5.03474450999056

I've also tried setting conn_timeout in  aiohttp.TCPConnector, but since I assume it is reused, once the timeout comes every future after the cutoff throws.
The obvious alternative is concurrent.futures.ThreadPoolExecutor and requests, which I've used and works fine, but i'm sure there must be something wrong with my approach regarding asyncio and aiohttp.
Any suggestions?
Your environment
Python 3.5.1
Ubuntu 14.04

Sorry for delay, I've finally got a time for the issue.
aiohttp.Timeout doesn't work globally, it operates exactly as you expect.
P.S. please note you should wrap every fetch() call into a task as you did by ensure_future() calls.
P.P.S.
My output for your snippet is:
Oops: https://googleusercontent.com <class 'aiohttp.errors.ClientOSError'>[Errno -5] Cannot connect to host googleusercontent.com:443 ssl:True [Can not connect to googleusercontent.com:443 [No address associated with hostname]]
Oops: https://live.com <class 'aiohttp.errors.ClientOSError'>[Errno 111] Cannot connect to host live.com:443 ssl:True [Can not connect to live.com:443 [Connect call failed ('<IP_ADDRESS>', 443)]]
Oops: https://imdb.com <class 'aiohttp.errors.ClientOSError'>[Errno None] Cannot connect to host imdb.com:443 ssl:True [Can not connect to imdb.com:443 [None]]
Oops: https://pixnet.net <class 'aiohttp.errors.ClientOSError'>[Errno 111] Cannot connect to host pixnet.net:443 ssl:True [Can not connect to pixnet.net:443 [Connect call failed ('<IP_ADDRESS>', 443)]]
200 https://t.co
Oops: https://cntv.cn <class 'aiohttp.errors.ClientOSError'>[Errno 113] Cannot connect to host cntv.cn:443 ssl:True [Can not connect to cntv.cn:443 [Connect call failed ('<IP_ADDRESS>', 443)]]
Oops: https://360.cn <class 'aiohttp.errors.ClientOSError'>[Errno 1] Cannot connect to host 360.cn:443 ssl:True [Can not connect to 360.cn:443 [[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:645)]]
200 https://yandex.ru
Oops: https://naver.com <class 'aiohttp.errors.ClientOSError'>[Errno None] Cannot connect to host naver.com:443 ssl:True [Can not connect to naver.com:443 [None]]
200 https://yahoo.com
200 https://sohu.com
Oops: https://nicovideo.jp <class 'concurrent.futures._base.TimeoutError'>
Oops: https://xvideos.com <class 'concurrent.futures._base.TimeoutError'>
Oops: https://gmw.cn <class 'concurrent.futures._base.TimeoutError'>
Oops: https://qq.com <class 'concurrent.futures._base.TimeoutError'>
Oops: https://sina.com.cn <class 'concurrent.futures._base.TimeoutError'>
Oops: https://tianya.cn <class 'concurrent.futures._base.TimeoutError'>
Oops: https://chinadaily.com.cn <class 'concurrent.futures._base.TimeoutError'>
Oops: https://rakuten.co.jp <class 'concurrent.futures._base.TimeoutError'>
Oops: https://yahoo.co.jp <class 'concurrent.futures._base.TimeoutError'>
Oops: https://hao123.com <class 'concurrent.futures._base.TimeoutError'>
Oops: https://xinhuanet.com <class 'concurrent.futures._base.TimeoutError'>
Oops: https://soso.com <class 'concurrent.futures._base.TimeoutError'>
Done in  5.01745690300595

It obviously shows: cancellation by timeout is performed in per-task basis.

@asvetlov I believe your output (with all those TimeoutError at the end) is showing a bug, which I have also been encountering. As soon as one URL hits a timeout (whether you set it using Timeout() or not), future get attempts will time out immediately.
I have a list of several hundred URLs to try, wasn't setting a timeout (so the default is 5 minutes), and have reduced the connection pool size from the default 20. After about 5 minutes of pages downloading fine, I get a flood of timeout errors for almost all remaining pages (with no time at all between, let alone 5 minutes). I believe a particular pooled connection "dies" when it times out, and remains dead, which allows it to quickly "handle" requests it is given (but by timing out immediately), resulting in that connection being passed more URLs than the other still-working ones.
I'm not sure I'm doing things right, and had to look for a get-multiple-pages example (having one of these in the docs would be nice), but I've used basically the same procedure as dev169, and ended up with a similar problem as you both appear to have had, so I don't think it's can just be me doing something wrong. Using 1.1.6.

I have the same problem under 1.2.0. Definitely still a bug. I don't even know if @asvetlov is still monitoring this, because he closed it in July. I'll check back in a while before opening a dupe.

@uogbuji could #1524 solve this problem?


Make NeoException serializable

Currently NeoExceptions aren't serializable 'out of the box' and some systems that rely on Exceptions being serializable can't send NeoExceptions around.

Hi, sorry I didn't merge this earlier - travels etc... thank you!

Should be available in <IP_ADDRESS> shortly....

No worries :) Thankyou


Using a seperate component for TuneIn details

DESCRIPTION
In this PR the changes are made to the TuneIn's current UI structure. As of now, TuneIn's UI template is part of the Dr.Elephant job page template. Since TuneIn details have their own variables which need to be refreshed on any transition, so it required for TuneIn UI to have their own component. This will help in better management of the variables and also makes it easier to reuse the template for TuneIn in the future if required.
Some refactoring is also done in the TuneIn UI, such as:

The condition to check if all the heuristics passed is modified. After this change, TuneIn will show all the heuristics passed when the overall Severity will be None or Low instead showing when overall Heuristic is only None.

Creating another PR #536 as this branch was forked out of customSHSWork instead of tuning_20190221.


Backport teams workaround fix for 20.09

Motivation for this change
This is a backport of a fix from unstable which I've been using for some days.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Pushed 5d382bec63eb29f1cfe72ab81c577eb4f9441f58.

@FredeEB how should I send backport request?

@otavio I am not sure what you mean, i haven't had anything to do with nixpkgs in a long time as far as i remember

@otavio
When backporting changes, please follow https://github.com/NixOS/nixpkgs/blob/master/.github/CONTRIBUTING.md#backporting-changes.
Namely, you should be doing git cherry-pick -x <rev> from a commit that has already landed in master. If the branches have diverged, you may alter the commit or add another commit to ensure that the package is able to still evaluate and build


WFLY-13024 Integrate MP Fault Tolerance 2.1

Resolves
https://issues.redhat.com/browse/WFLY-13024
https://issues.redhat.com/browse/WFLY-13122
Proposal
https://github.com/wildfly/wildfly-proposals/pull/290

@maeste FYI.
@rhusar CI won't run against Draft PRs, so if you want that you'll have to remove the 'Draft' status. I'm going to put 'Feature' and 'Missing Reqs' labels on this, and those will be enough to stop anyone inadvertently merging before it's ready.
Getting rid of lots of deps is nice!

Ok then, thanks Brian. Yeah, I did want to get the CI run results (while passing CI in our clustering CI env, the jobs are not 1:1).

Please remove https://issues.redhat.com/browse/WFLY-13122 from this PR. It is a separate issue. We need to be able to process WFLY-13024 independently.

@bstansberry Sure, if that helps anything, extracted into: https://github.com/wildfly/wildfly/pull/13012

retest this please

The failure is unrelated (not going to start a new run).

Pre-check passed: https://issues.redhat.com/browse/EAP7-1428?focusedCommentId=13989088&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13989088
cc @bstansberry @jamezp


Naming convention

Change the variable names according to the convention. Should be straightforward.

resolved with PR #38


imply, state or specify that sky soarer can be used while riding entities

idk how much of a secret it is, but if it isnt, it might be a good idea to put it in the spell entry!

Ooh, good point! This was a recent change that I made and I forgot to document it.


feat(nexus): implement set ANA state for an NVMf-published Nexus

Implement set_ana_state in nexus_bdev::Nexus and expose that over
gRPC and the client CLI.
Fixes CAS-756, #758

bors try

Can we also add the ANA state to the output columns of nexus list?

Most have asked for a getter and the current value could either be stored in the Nexus or retrieved from spdk_nvme_ns_get_ana_state(), with the latter probably a cleaner solution.

... retrieved from spdk_nvme_ns_get_ana_state()

Or not, as that's the nvme subsystem, not nvmf. The state is stored in spdk_nvmf_subsystem_listener so look for that instead.

Can we also add the ANA state to the output columns of nexus list?

To answer this directly, yes we can but as there is another change to overhaul the CLI output formats it'd be better to leave this to another PR so I'll raise a ticket for that.

bors try

The CI failures are due to this in dmesg:
nvme nvme3: Please enable CONFIG_NVME_MULTIPATH for full support of multi-port devices.

Looks good. I'm interested in control plane implications. Should the control plane start setting nexus ANA to the value "optimized"? Because it is always optimized when we have just one nexus. Or is it optimized by default?

It's optimised by default, in fact it's the only state that must be supported, the others optional.

bors merge


revise instructions for installing Jekyll

Previous instructions were outdated as Homebrew uses a different directory on Apple Silicon. Easier to link to current instructions on the Jekyll website, plus give a link to a complete guide at mac.install.guide that explains the pitfalls of using Homebrew to install Ruby, and point to Moncef Belyamani's Ruby setup scripts. Hope this helps!

Thanks for the update!


Code refactor

Changes

Add a collision to car's body
Add keyboard_command_callback to command_multiplexer.py for teleoperating using keyboard
Remove veolcity scaling in src/control_plugin.py#L134 in convert the input velocity (in reference/local frame) to joint speed (wheel rotation in rad/s)
Remove car_x.access and replace with singular car_controller.launch with arguments
Refactor one_car.launch.xml to car_gazebo.launch to specify a car's gazebo configuration
Refactor vehicle_class.launch.xml to car_spawner.launch for launching a car with specific configuration
Publish a rosparam car_x/init_pose that contains the spawn position in [x_pos, y_pos, z_pos] to replace offset and easier to use across packages
Add the old simulator.launch and race_track world from previous tutorial
Add an offset (-9, -5) to new race-track map map/race_track.yaml#L3 to align the center of map with track origin
Change speed and angle limit in scripts/keyboard_teleop.py#L17
Change odom_frame in src/control_plugin.py#L45 to car_x_odom to separate multiple car odometry when using the same map (found this bug in multi_teb tutorial)
Replace racecar_reset_state function in src/set_racecar_state.py#L31 using rossrv instead of rosparam
Fix laser scan tf tree (link position) in base_laser_link static transform publisher. The laser scan should be at the top of the car or in the laser 3d rendering, but somehow it is on the ground level between the wheels)
Replace #!/usr/bin/env python with #!/usr/bin/env python3 since I'm using Noetic.

Snapshots

Thanks for the effort @haritsahm.
We will review and merge once we have tested on our machines.

Let me know if I need to change anything. I will also open a PRs for the tutorials and pure-pursuit


💡 FR: Add src_addr field to kentik-ping devices

As a user, I need to track the device_ip of my ping_only devices that have provider: kentik-ping. Decoration of the device_ip into the src_addr field for resulting metrics can easily solve this challenge.

Confirmed good.


Adding updated scenarios for 3.0 powerfulseal version

This is for: https://github.com/openshift-scale/kraken/issues/19
The scenarios should run the exact same way as before, the syntax is just a little different with 3.0 version of powerfulseal

Can one of the admins verify this patch?


Stdlib markdown compability

This markdown text is failing with auto_link after Markdown.to_html. (https://gist.github.com/hugoabonizio/2d0dc77666a458660bef8d7aac2247bb)

[**Autolink.cr**](https://github.com/hugoabonizio/autolink.cr) is a port to Crystal of [rails_autolink](https://github.com/tenderlove/rails_autolink) Ruby gem.

**Autolink.cr** turns URLs within a text into links with customizable attributes.

### Basic usage

```ruby
require "autolink"

include Autolink

auto_link("My blog: http://www.myblog.com")
# => My blog: http://www.myblog.com

Heredoc in Crystal ignores leading spaces, so I did a small change to make it a bit more readable.

Much better! :+1:


Update README badge links

What does this implement/fix?
There appears to have been a change at some point with how Github renders badge links in rst files.
On main, in the table of badges and links, there are now trailing underscores rendered, and clicking on the badges no longer directs to the desired link, but instead to the image file.
This PR updates the approach for setting the links for the badges, using the :target: operative, which updates the links to direct to the badges and makes the badges work as previously.

Thanks @TomDonoghue !


Whether and how to submit general-purpose templates?

I have a simple and cool template that I think might be useful to other developers. Should I submit it somewhere and if so then how? Is there some public repo with general-purpose templates? Submitting PR that treats my template as README.md worthy and modifies it feels kind of rude and invasive. What should I do?

@fvolchyok there isn't a generic repo  but if you share it here I can decide whether to add it to bundled one, what does the template do?


Clarify "payments" nomenclature

Reticketed from https://github.com/gratipay/gratipay.com/pull/4157#issuecomment-264345997:

"Payments" is confusing to someone new--is it money I give? Is it money I receive? Especially at about/features/, it goes from "giving" to "payments."
Can we use a different word? "Payins"?

From https://github.com/gratipay/gratipay.com/pull/4157#issuecomment-264348811:

The discussion around "Payments" nomenclature was at https://github.com/gratipay/inside.gratipay.com/issues/117. I'm okay with it, but maybe we need a ticket on breaking site copy up into

what is helpful for givers and
what is helpful for project owners


[BUG] Not Loading

When creating a password for the dashboard via the console the dashboard does not load.

Hm, it works fine for me. When you create the account via the console, do you also log in as user CONSOLE? Also, do errors occur in the console when you set the password or start the plugin?

No errors occur when I started the plugin. However the login screen did not load at all for some reason.

Switched to a different panel


Updating tests and example to use Grunt 1.0.0

After the peer dependency limit is removed in #243, this will update the tests and example to install Grunt 1.0.0 or later.

@grayside Without the shrinkwrap file, I got npm install errors when installing with Grunt 1.0.0. However if I kept the existing shrinkwrap for GDT, npm install did not throw any errors.
Let me know what issues you were seeing since the Grunt 1.0.0 release.

On the existing 0.10.1 release, I have observed the following error (screenshot courtesy of @mdeltito)
https://dl.dropboxusercontent.com/s/14ul2agfv98w50z/2016-04-06 at 6.29 PM.png
He tracked down https://github.com/npm/npm/issues/5135 as a potentially related issue.

@arithmetric I found that this error only occurs when pulling in GDT as a dependency, not when running npm install in the plugin itself. I'm testing with the following package.json:
{
  "name": "gdt-test",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "grunt": "0.4.5",
    "grunt-drupal-tasks": "0.10.1",
    "zombie": "2.5.1"
  }
}

Node: 4.3.1
NPM: 2.14.12

Ah, this is also currently failing tests.

@mdeltito I was able to reproduce the npm install error using your package.json. By changing the grunt version to ^1.0.0 in the package.json, I was able to run npm install on it without any errors.

I have also found this to be working if the project's package.json specifies the boosted version of grunt. Note that you need to first remove the node_modules directory otherwise the EPEERINVALID error sticks around. I think this might be a version bump for Gadget.

This is also working for me, and somehow prevents the version of Grunt inside of the GDT node_modules directory from getting bumped to 1.0.1. The project-level copy of Grunt does get updated however. This doesn't seem to cause any issues, and getting on the latest Grunt in some capacity is likely for the best, but worth noting.

Hmm... Grunt version 0.4.5 is still specified in GDT's shrinkwrap file, even though GDT's package.json has no mention of Grunt. See: https://github.com/phase2/grunt-drupal-tasks/blob/master/npm-shrinkwrap.json#L5

@arithmetric I believe this is caused by how peerDependencies are evaluated, and since some of the grunt plugins specify a more restrictive version identifier the GDT shrinkwrap ends up including a lower version that satisfies all peers. Bumping the parent project's version to 1.0.1 also would not affect the shrinkwrap for GDT.

Further discussion out-of-band, if projects have direct dependencies on grunt plugins in their package.json, the peerDependency/grunt version mismatch is still a problem.
Within GDT there are version mismatches even with this change, though at this point nothing seems to go wrong.

After further review, there are four dependencies with a peer dependency on Grunt < 0.5 that are causing this type of error on npm install and shrinkwrap operations:
npm ERR! code EPEERINVALID
npm ERR! peerinvalid The package<EMAIL_ADDRESS>does not satisfy its siblings' peerDependencies requirements!

There are open PRs to fix each of these modules:

https://github.com/ben-eb/grunt-available-tasks/pull/27
https://github.com/wecodemore/grunt-githooks/pull/50
https://github.com/grayside/grunt-notify/pull/1
https://github.com/linusnorton/grunt-parallel-behat/pull/16

As an interim fix for GDT and Grunt 1.x compatibility, I'm adding a commit to update GDT's package.json to use git checkouts of these dependencies with the fixed configuration.

As our project got mentioned in here: We are open for collaboration on this one, also commented on the (highly appreciated) PR. We also extended our CI setup to work with all Nodejs minor and major versions from 0.10 upwards (no io.js tough) – see our .travis.yml for details. Hope that helps.

@arithmetric FYI, I went through the list of available Nodejs versions after finding at this repo requirements in the README.md (Nodejs 0.12 and 4.3). Therefore updated our list in wecodemore/grunt-githooks#54 – This means that you will have to

[ ] Rebase or update your PR on our side to incorporate the changes to .travis.yml and file it against the dev branch (please)
[ ] Update the PR here after I merged the latest changes into our master

Thanks for your efforts.

@franz-josef-kaiser I filed this new PR against the dev branch as you requested: https://github.com/wecodemore/grunt-githooks/pull/56
I think it includes your most recent changes, but let me know if any updates are needed. Thanks!

After a couple this and that tasks, the 0.6.0 version is now released. See #244 for additional infos. Please update your package.json to set the appropriate version number for wecodemore/grunt-githooks.

Thanks again @franz-josef-kaiser! I've updated our dependency to use grunt-githooks ~0.6.0.


more flexible data exporting

Codecov Report

Merging #171 (64585c3) into master (b47aeb0) will decrease coverage by 0.28%.
The diff coverage is 87.50%.

@@            Coverage Diff             @@
##           master     #171      +/-   ##
==========================================
- Coverage   93.69%   93.40%   -0.29%
==========================================
  Files           8        8
  Lines         983      971      -12
==========================================
- Hits          921      907      -14
- Misses         62       64       +2

Impacted Files
Coverage Δ

src/crystal.jl
94.70% <80.76%> (-0.26%)
:arrow_down:

src/bonds.jl
93.90% <88.23%> (-0.51%)
:arrow_down:

src/box.jl
95.58% <92.30%> (-0.07%)
:arrow_down:

src/misc.jl
97.50% <100.00%> (-0.23%)
:arrow_down:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


(maint) Add notes about not installing bolt as a gem

What this changes Add notes to readme and official docs recommending installing bolt as a package
Why Because users often install bolt as a gem and it doesn't work as they expect out of the box.

Waiting for CLA signature by @lucywyman
@lucywyman - We require a Contributor License Agreement (CLA) for people who contribute to Puppet, but we have an easy click-through license with instructions, which is available at https://cla.puppet.com/
Note: if your contribution is trivial and you think it may be exempt from the CLA, please post a short reply to this comment with details. http://docs.puppet.com/community/trivial_patch_exemption.html


KeyError: 'imaging_quality_preprocessing_mode'

Hi, there is an error using python script for evaluation:
python3.10/site-packages/vbench/imaging_quality.py", line 30, in technical_quality
    preprocess_mode = kwargs['imaging_quality_preprocessing_mode']
KeyError: 'imaging_quality_preprocessing_mode'

Can you check if vbench package you installed and the python script are from the same version. I cannot replicate the issue with the latest version.

Hi, I'm closing the issue as there's no follow-up question, but feel free to re-open it anytime if you have any further questions!

@quantumiracle You need to pass the imaging_quality_preprocessing_mode into kwargs as in evaluate.py when calling VBench.evaluate.
@NattapolChan @ziqihuangg In your README python example, you also do not specify this argument. I guess that was why we met this error and you could reproduce it.

time CUDA_VISIBLE_DEVICES=0 python3 evaluate.py --dimension imaging_quality \
  --mode=custom_input \
  --videos_path './sampled_videos' \
  --output_path test_${date_str}_imaging_quality/

args: Namespace(output_path='test_2024-07-27_imaging_quality/', full_json_dir='/home/users/nattapol/VBench/vbench/VBench_full_info.json', videos_path='./sampled_videos', dimension=['imaging_quality'], load_ckpt_from_local=None, read_frame=None, mode='custom_input', custom_input=False, prompt='', prompt_file=None, category=None, imaging_quality_preprocessing_mode='longer')
start evaluation
Evaluation meta data saved to test_2024-07-27_imaging_quality/results_2024-07-27-18:12:09_full_info.json
cur_full_info_path: test_2024-07-27_imaging_quality/results_2024-07-27-18:12:09_full_info.json
^M  0%|          | 0/1 [00:00<?, ?it/s]
^M100%|██████████| 1/1 [00:34<00:00, 34.39s/it]
^M100%|██████████| 1/1 [00:34<00:00, 34.39s/it]
Evaluation results saved to test_2024-07-27_imaging_quality/results_2024-07-27-18:12:09_eval_results.json
done

real    2m9.035s
user    1m43.805s
sys     0m16.577s

It does work fine for me. The argument should be set by default to 'longer'. It should work fine even without imaging_quality_preprocessing_mode flag. I think it might be due to version conflict between evaluate.py and pypi package. Can you provide the commit hash of vbench to reproduce this?

Thanks for pointing this out. I didn't check calling VBench.evaluate() directly. This should be fixed in 2048fb0.


Gabor Filter Cernusca

Changes:

Google Maps test images added
Launch Site image added
Gabor Filter added
More displayed overlays

Repository set to state of original in the "Accept Changes" commit


Add Recommender System algorithms

I want to add some of the algorithms used on recommendation systems. I have completed my graduation recently and in my final paper I wrote a recommender system, an hybrid implementation, using some simple algorithms. I like to share them in your repo if you permit.

Sure. You can create a new folder dedicated to the recommendation systems.
Please follow the same approach that we use in the repo:

No external dependencies other than numpy/scipy
Simplicity

I will need to use pandas to read the datasets. Is a problem?
And about that, where can I put the datasets?

We have the datasets module.
Instead of pandas, you could use CSV module.

I will add a recommender system algorithm.


Document project network isolation on any cluster + CNI

https://rancher.zendesk.com/agent/tickets/15139
Point of contact: Ryan
PR: https://github.com/rancher/rancher/pull/31344
Jan said:

If the RKE network plugin supports the enforcement of Kubernetes network policies (as it the case for Cisco ACI plugin) then Rancher should allow the user to enable project network isolation as a cluster option.
This Rancher feature integrates with the Kubernetes network policy subsystem and not with a specific CNI. The validation of that option should therefore depend on whether the CNI supports enforcement of the Rancher managed policies and not whether that CNI is Canal.

https://github.com/rancher/rancher/issues/31338
zendesk ticket 15139

We have documented that when the project network isolation option is enabled with Canal, the system project is exempt from the isolation. Would the same be true for any CNI that supports enforcement of Kubernetes network policies?

Note: In clusters where both:

The Canal network plug-in is in use.
The Project Network Isolation option is enabled.

The system project overrides the Project Network Isolation option so that it can communicate with other projects, collect logs, and check health.


AWX Operator upgrade from 0.25 to 0.26 fails

Environment

OS: Ubuntu 20.04.4 LTS
k3s version v1.21.3+k3s1 (1d1f220f)
go version go1.16.6
AWX Operator: 0.25

Description
Upgrade of AWX-Operator 0.25 to 0.26 fails, gets stuck forever on:
--------------------------- Ansible Task StdOut -------------------------------
TASK [installer : Stream backup from pg_dump to the new postgresql container] ***
task path: /opt/ansible/roles/installer/tasks/upgrade_postgres.yml:89
Step to Reproduce
Just follow the upgrade guide on https://github.com/kurokobo/awx-on-k3s/blob/main/tips/upgrade-operator.md, persistent volume for postgres-13 created as requested.

Hmm, does your password for your postgres cointains any special charactors that causes breaking shell command?
Your password will be used by following task, and there is no quotes around password, so if your password contains some special charactors like " shell may keep waiting forever.
https://github.com/ansible/awx-operator/blob/19c6c77c41b98bded04b3b20ace4b3fe99b682c9/roles/installer/tasks/upgrade_postgres.yml#L95-L97
There is a related issue (https://github.com/ansible/awx-operator/issues/837) but above task added after my fix.
If your password has no special charactors, more investigation is required. Could you share your output from following commands?
kubectl -n awx get awx,pod,pv,pvc
kubectl -n awx logs statefulset/awx-postgres
kubectl -n awx logs statefulset/awx-postgres-13

Hello,
thank you for your reply! I just checked to be sure, my password is only alphanumeric without any special characters. I found some  post on Google Groups where someone was mentioning that upgrading to 0.28 actually worked, so I was trying that today, without luck. Actually, it seems it somehow cycles through the tasks indefinitely, getting stucked on:
--------------------------- Ansible Task StdOut -------------------------------
TASK [installer : Stream backup from pg_dump to the new postgresql container] ***
task path: /opt/ansible/roles/installer/tasks/upgrade_postgres.yml:89

{"level":"info","ts":1671618073.9435282,"logger":"proxy","msg":"Read object from cache","resource":{"IsResourceRequest":true,"Path":"/api/v1/namespaces/awx/pods/awx-postgres-13-0","Verb":"get","APIPrefix":"api","APIGroup":"","APIVersion":"v1","Namespace":"awx","Resource":"pods","Subresource":"","Name":"awx-postgres-13-0","Parts":["pods","awx-postgres-13-0"]}}
And then very quickly going through other tasks, with one actually failing:
TASK [installer : Set pg_dump command] *****************************************\r\ntask path: /opt/ansible/roles/installer/tasks/upgrade_postgres.yml:70\nok: [localhost] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}\n\r\nTASK [installer : Set pg_restore command] **************************************\r\ntask path: /opt/ansible/roles/installer/tasks/upgrade_postgres.yml:81\nok: [localhost] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}\n\r\nTASK [installer : Stream backup from pg_dump to the new postgresql container] ***\r\ntask path: /opt/ansible/roles/installer/tasks/upgrade_postgres.yml:89\nfatal: [localhost]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true}\n\r\nPLAY RECAP *********************************************************************\r\nlocalhost                  : ok=52   changed=3    unreachable=0    failed=1    skipped=17   rescued=0    ignored=0   \r\n\n","job":"8834408734363064390","name":"awx","namespace":"awx","error":"exit status 2"}
----- Ansible Task Status Event StdOut (awx.ansible.com/v1beta1, Kind=AWX, awx/awx) -----
PLAY RECAP *********************************************************************
localhost                  : ok=52   changed=3    unreachable=0    failed=1    skipped=17   rescued=0    ignored=0

{"level":"error","ts":1671618044.5843086,"msg":"Reconciler error","controller":"awx-controller","object":{"name":"awx","namespace":"awx"},"namespace":"awx","name":"awx","reconcileID":"b2554ae1-af5d-4699-98e4-adda7d52e296","error":"event runner on failed","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.1/pkg/internal/controller/controller.go:273\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.1/pkg/internal/controller/controller.go:234"}
Now for the output that you asked me for:
kubectl -n awx get awx,pod,pv,pvc
NAME                      AGE
awx.awx.ansible.com/awx   478d
NAME                                                   READY   STATUS      RESTARTS   AGE
pod/awx-postgres-0                                     1/1     Running     18         189d
pod/awx-operator-controller-manager-5f65fff6fd-dnljk   2/2     Running     0          83m
pod/automation-job-984865-h5qnc                        0/1     Completed   0          82m
pod/automation-job-984845-jsvjt                        0/1     Completed   0          83m
pod/awx-postgres-13-0                                  1/1     Running     0          81m
NAME                                      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                               STORAGECLASS          REASON   AGE
persistentvolume/awx-projects-volume      2Gi        RWO            Retain           Bound    awx/awx-projects-claim              awx-projects-volume            478d
persistentvolume/awx-postgres-volume      2Gi        RWO            Retain           Bound    awx/postgres-awx-postgres-0         awx-postgres-volume            478d
persistentvolume/awx-backup-volume        2Gi        RWO            Retain           Bound    awx/awx-backup-claim                awx-backup-volume              298d
persistentvolume/registry-volume          5Gi        RWO            Retain           Bound    registry/registry-claim             registry-volume                292d
persistentvolume/awx-postgres-13-volume   8Gi        RWO            Retain           Bound    awx/postgres-13-awx-postgres-13-0   awx-postgres-volume            156m
NAME                                                  STATUS   VOLUME                   CAPACITY   ACCESS MODES   STORAGECLASS          AGE
persistentvolumeclaim/awx-projects-claim              Bound    awx-projects-volume      2Gi        RWO            awx-projects-volume   478d
persistentvolumeclaim/postgres-awx-postgres-0         Bound    awx-postgres-volume      2Gi        RWO            awx-postgres-volume   478d
persistentvolumeclaim/awx-backup-claim                Bound    awx-backup-volume        2Gi        RWO            awx-backup-volume     298d
persistentvolumeclaim/postgres-13-awx-postgres-13-0   Bound    awx-postgres-13-volume   8Gi        RWO            awx-postgres-volume   81m
kubectl -n awx logs statefulset/awx-postgres
PostgreSQL Database directory appears to contain a database; Skipping initialization
2022-12-20 07:11:15.468 UTC [1] LOG:  starting PostgreSQL 12.13 (Debian 12.13-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2022-12-20 07:11:15.469 UTC [1] LOG:  listening on IPv4 address "<IP_ADDRESS>", port 5432
2022-12-20 07:11:15.469 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2022-12-20 07:11:15.475 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2022-12-20 07:11:15.725 UTC [27] LOG:  database system was interrupted; last known up at 2022-12-19 09:04:37 UTC
2022-12-20 07:11:17.178 UTC [27] LOG:  database system was not properly shut down; automatic recovery in progress
2022-12-20 07:11:17.207 UTC [27] LOG:  redo starts at 20E/68E85BC0
2022-12-20 07:11:17.457 UTC [27] LOG:  invalid record length at 20E/6AE5AB30: wanted 24, got 0
2022-12-20 07:11:17.457 UTC [27] LOG:  redo done at 20E/6AE5AB08
2022-12-20 07:11:18.927 UTC [1] LOG:  database system is ready to accept connections
2022-12-21 08:11:00.457 UTC [203414] LOG:  could not receive data from client: Connection reset by peer
The last one is actually full of these:
kubectl -n awx logs statefulset/awx-postgres-13:
2022-12-21 10:24:45.512 UTC [369] ERROR:  duplicate key value violates unique constraint "main_jobevent_20221103_09_pkey"
2022-12-21 10:24:45.512 UTC [369] DETAIL:  Key (id, job_created)=(10840566, 2022-11-03 09:00:18.231419+00) already exists.
2022-12-21 10:24:45.512 UTC [369] CONTEXT:  COPY main_jobevent_20221103_09, line 1
2022-12-21 10:24:45.512 UTC [369] STATEMENT:  COPY public.main_jobevent_20221103_09 (id, created, modified, event, event_data, failed, changed, host_name, play, role, task, counter, host_id, job_id, uuid, parent_uuid, end_line, playbook, start_line, stdout, verbosity, job_created) FROM stdin;
2022-12-21 10:24:45.684 UTC [369] ERROR:  duplicate key value violates unique constraint "main_jobevent_20221103_10_pkey"
2022-12-21 10:24:45.684 UTC [369] DETAIL:  Key (id, job_created)=(10843650, 2022-11-03 10:00:10.798315+00) already exists.
2022-12-21 10:24:45.684 UTC [369] CONTEXT:  COPY main_jobevent_20221103_10, line 1
2022-12-21 10:24:45.684 UTC [369] STATEMENT:  COPY public.main_jobevent_20221103_10 (id, created, modified, event, event_data, failed, changed, host_name, play, role, task, counter, host_id, job_id, uuid, parent_uuid, end_line, playbook, start_line, stdout, verbosity, job_created) FROM stdin;
2022-12-21 10:24:45.856 UTC [369] ERROR:  duplicate key value violates unique constraint "main_jobevent_20221103_11_pkey"
2022-12-21 10:24:45.856 UTC [369] DETAIL:  Key (id, job_created)=(10846734, 2022-11-03 11:00:03.380105+00) already exists.
2022-12-21 10:24:45.856 UTC [369] CONTEXT:  COPY main_jobevent_20221103_11, line 1
2022-12-21 10:24:45.856 UTC [369] STATEMENT:  COPY public.main_jobevent_20221103_11 (id, created, modified, event, event_data, failed, changed, host_name, play, role, task, counter, host_id, job_id, uuid, parent_uuid, end_line, playbook, start_line, stdout, verbosity, job_created) FROM stdin;
2022-12-21 10:24:46.036 UTC [369] ERROR:  duplicate key value violates unique constraint "main_jobevent_20221103_12_pkey"
2022-12-21 10:24:46.036 UTC [369] DETAIL:  Key (id, job_created)=(10849818, 2022-11-03 12:00:25.081912+00) already exists.
2022-12-21 10:24:46.036 UTC [369] CONTEXT:  COPY main_jobevent_20221103_12, line 1
2022-12-21 10:24:46.036 UTC [369] STATEMENT:  COPY public.main_jobevent_20221103_12 (id, created, modified, event, event_data, failed, changed, host_name, play, role, task, counter, host_id, job_id, uuid, parent_uuid, end_line, playbook, start_line, stdout, verbosity, job_created) FROM stdin;
2022-12-21 10:24:46.216 UTC [369] ERROR:  duplicate key value violates unique constraint "main_jobevent_20221103_13_pkey"
2022-12-21 10:24:46.216 UTC [369] DETAIL:  Key (id, job_created)=(10852902, 2022-11-03 13:00:19.855241+00) already exists.
2022-12-21 10:24:46.216 UTC [369] CONTEXT:  COPY main_jobevent_20221103_13, line 1
2022-12-21 10:24:46.216 UTC [369] STATEMENT:  COPY public.main_jobevent_20221103_13 (id, created, modified, event, event_data, failed, changed, host_name, play, role, task, counter, host_id, job_id, uuid, parent_uuid, end_line, playbook, start_line, stdout, verbosity, job_created) FROM stdin;
2022-12-21 10:24:46.396 UTC [369] ERROR:  duplicate key value violates unique constraint "main_jobevent_20221103_14_pkey"
2022-12-21 10:24:46.396 UTC [369] DETAIL:  Key (id, job_created)=(10855960, 2022-11-03 14:00:17.754337+00) already exists.
2022-12-21 10:24:46.396 UTC [369] CONTEXT:  COPY main_jobevent_20221103_14, line 1
2022-12-21 10:24:46.396 UTC [369] STATEMENT:  COPY public.main_jobevent_20221103_14 (id, created, modified, event, event_data, failed, changed, host_name, play, role, task, counter, host_id, job_id, uuid, parent_uuid, end_line, playbook, start_line, stdout, verbosity, job_created) FROM stdin;

@kuchynkap
Thanks for updating, so the issue seems to be on AWX side.
If you have backup of your AWX, retry upgrade by following procedure, and if your issue will have been reproduced, you should report this on AWX repository.
# Completely stop AWX Operator by removing it
kubectl -n awx delete deployment awx-operator-controller-manager

# Remove new PSQL 13
kubectl -n awx delete statefulset awx-postgres-13
kubectl -n awx delete pvc postgres-13-awx-postgres-13-0
kubectl -n awx delete pv awx-postgres-13-volume

# Remove files for new PSQL 13 and recreate it
sudo rm -rf /data/postgres-13
sudo mkdir -p /data/postgres-13
sudo chmod 755 /data/postgres-13

# Recreate PV for new PSQL 13
cat <<EOF > pv-postgres-13.yaml
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: awx-postgres-13-volume
spec:
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  capacity:
    storage: 8Gi
  storageClassName: awx-postgres-volume
  hostPath:
    path: /data/postgres-13
EOF
kubectl apply -f pv-postgres-13.yaml

# Deploy new AWX Operator
git clone https://github.com/ansible/awx-operator.git -b 0.26.0
cd awx-operator
NAMESPACE=awx make deploy

@kurokobo
Thank you for your help! In the beginning it looked very promising as it has not failed in early stage. But finally, it failed also:
task path: /opt/ansible/roles/installer/tasks/upgrade_postgres.yml:65\nok: [localhost] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}\n\r\nTASK [installer : Set pg_dump command] *****************************************\r\ntask path: /opt/ansible/roles/installer/tasks/upgrade_postgres.yml:70\nok: [localhost] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}\n\r\nTASK [installer : Set pg_restore command] **************************************\r\ntask path: /opt/ansible/roles/installer/tasks/upgrade_postgres.yml:81\nok: [localhost] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}\n\r\nTASK [installer : Stream backup from pg_dump to the new postgresql container] ***\r\ntask path: /opt/ansible/roles/installer/tasks/upgrade_postgres.yml:89\nfatal: [localhost]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true}\n\r\nPLAY RECAP *********************************************************************\r\nlocalhost                  : ok=52   changed=3    unreachable=0    failed=1    skipped=17   rescued=0    ignored=0   \r\n\n","job":"5742676780418950479","name":"awx","namespace":"awx","error":"exit status 2"}
----- Ansible Task Status Event StdOut (awx.ansible.com/v1beta1, Kind=AWX, awx/awx) -----
PLAY RECAP *********************************************************************
localhost                  : ok=52   changed=3    unreachable=0    failed=1    skipped=17   rescued=0    ignored=0

{"level":"error","ts":1671699581.824264,"logger":"controller-runtime.manager.controller.awx-controller","msg":"Reconciler error","name":"awx","namespace":"awx","error":"event runner on failed","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.2/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.2/pkg/internal/controller/controller.go:214"}
{"level":"info","ts":1671699586.604346,"logger":"logging_event_handler","msg":"[playbook task start]","name":"awx","namespace":"awx","gvk":"awx.ansible.com/v1beta1, Kind=AWX","event_type":"playbook_on_task_start","job":"1574724159804951253","EventData.Name":"installer : Check for presence of Deployment"}
Then again, it starts an infinite loop going through all the tasks. Here is an example from the logs from the postgres-13 container (although in the beginning, there were duplicate key errors like yesterday):
2022-12-22 08:59:36.366 UTC [271] ERROR:  role "tos_monitoring" does not exist
2022-12-22 08:59:36.366 UTC [271] STATEMENT:  GRANT SELECT ON TABLE public.social_auth_partial TO tos_monitoring;
2022-12-22 08:59:36.366 UTC [271] ERROR:  role "tos_monitoring" does not exist
2022-12-22 08:59:36.366 UTC [271] STATEMENT:  GRANT SELECT ON TABLE public.social_auth_usersocialauth TO tos_monitoring;
2022-12-22 08:59:36.366 UTC [271] ERROR:  role "tos_monitoring" does not exist
2022-12-22 08:59:36.366 UTC [271] STATEMENT:  GRANT SELECT ON TABLE public.sso_userenterpriseauth TO tos_monitoring;
2022-12-22 08:59:36.366 UTC [271] ERROR:  role "tos_monitoring" does not exist
2022-12-22 08:59:36.366 UTC [271] STATEMENT:  GRANT SELECT ON TABLE public.taggit_tag TO tos_monitoring;
2022-12-22 08:59:36.366 UTC [271] ERROR:  role "tos_monitoring" does not exist
2022-12-22 08:59:36.366 UTC [271] STATEMENT:  GRANT SELECT ON TABLE public.taggit_taggeditem TO tos_monitoring;
I will create an issue on the AWX repository as you mentioned.  Thank you for your time and your help!

Here is the link to that issue:
https://github.com/ansible/awx/issues/13370

@kuchynkap
Thanks for creating the issue!
F.Y.I., in GitHub you can format your code block or logs by wrapping them by ```:
Source:
```
# Deploy new AWX Operator
git clone https://github.com/ansible/awx-operator.git -b 0.26.0
cd awx-operator
NAMESPACE=awx
make deploy
```

Result:
# Deploy new AWX Operator
git clone https://github.com/ansible/awx-operator.git -b 0.26.0
cd awx-operator
NAMESPACE=awx
make deploy

This is a good way to make your report easy-to-read. Refer to https://github.github.com/gfm/ for more information about markdown syntax.
I'm closing this issue since the issue should be solved in AWX side, but feel free to open new issue if you want further assistance.


sometimes the tag upsampler changes "BREAK" to "(BREAK:-1), reducing image quality.

maybe could be fixed with changing some settings? im not sure honestly

I need to dig into whole sd-webui prompt processing system to ensure I can get something after all prompt parsing...
Still investigating

Fixed


Fixes typo in README

Best PR ever, but for some reason this hit my eye and I felt the need to do this.

Thanks @kimar! :balloon:


refactor: format list into table & remove broken links

Changes proposed
closes #1

Refactored list into tables with descriptions
Removed broken links

Check List (Check all the applicable boxes)

[x] Addresses an existing open issue: Closes #1
[x] The title of my pull request is a short description of the requested changes.
[x] I followed the Contributing Guidelines

Screenshots

@NiallJoeMaher Was a nice easy fix, so have resolved and merged in ready for review 😊


paramCase is missing from latest v5 release

Hello,
when i try to import paramCase from change-case it now fails after the update,
and i don't find it on github either
this url :
https://github.com/blakeembrey/change-case/tree/main/packages/param-case
return 404,
and i don't find a paramCase function here :
https://github.com/blakeembrey/change-case/blob/main/packages/change-case/src/index.ts
thanks !

It’s called kebab case now: https://github.com/blakeembrey/change-case/releases/tag/change-case%405.0.0

thanks !


Show version of Falcon Container to the user through CRD and kubectl get

$ kubectl get falconcontainer
NAME       STATUS   VERSION                                    ERROR
default    DONE     6.28.0-1101.container.x86_64.Release.US1   -

This is nice, but I feel like the name is weird and the status throws me off.

This is nice, but I feel like the name is weird and the status throws me off.

👍 Thanks.
Fortunately, the name and status are not part of this change.


Tunnel fails to close after Linux system resumes from sleep

On a Linux-based system, after resuming from sleep, it is impossible to close the tunnel without killing the process.
Here’s the normal behavior when closing the tunnel:
sudo go run github.com/Jigsaw-Code/outline-sdk/x/examples/outline-cli@latest -transport "ss://token"
OutlineVPN CLI (experimental)
[INFO] 2024/12/06 16:00:09 updated global IPv6 support: false
[INFO] 2024/12/06 16:00:09 remote server supports UDP, we will delegate all UDP packets to it
[INFO] 2024/12/06 16:00:09 routing traffic configured...
^C[INFO] 2024/12/06 16:47:43 received interrupt, terminating...
[INFO] 2024/12/06 16:47:43 routing table cleaned up
[INFO] 2024/12/06 16:47:43 DNS config restored
[INFO] 2024/12/06 16:47:43 OutlineDevice -> tun stopped: 274279765 <nil>
[INFO] 2024/12/06 16:47:43 tun -> OutlineDevice stopped: 27566484 read tun: file already closed

However, after resuming from sleep, attempting to close the tunnel results in infinite waiting:
OutlineVPN CLI (experimental)
[INFO] 2024/12/06 00:23:58 updated global IPv6 support: false
[INFO] 2024/12/06 00:23:58 remote server supports UDP, we will delegate all UDP packets to it
[INFO] 2024/12/06 00:23:58 routing traffic configured...
^C[INFO] 2024/12/06 10:21:42 received interrupt, terminating...
[INFO] 2024/12/06 10:21:42 routing table cleaned up
[INFO] 2024/12/06 10:21:42 DNS config restored
[INFO] 2024/12/06 10:21:42 OutlineDevice -> tun stopped: 264549559 <nil>
signal: killed

The only way to close the tunnel in this case is to kill the outline-cli process manually.
The issue occurs in app_linux.go, specifically here:
go func() {
    defer trafficCopyWg.Done()
    written, err := io.Copy(ss, tun)
    logging.Info.Printf("tun -> OutlineDevice stopped: %v %v\n", written, err)
}()

This section does not receive an EOF, preventing the connection from closing.
I believe the issue lies in how io.Copy blocks indefinitely while waiting for data from the TUN device. This behavior becomes problematic after the system wakes up from sleep, as the TUN device does not signal EOF or return from the Read call.

Thanks for reporting the issue. Though the root cause here seems to be that we are not able to Close the tun device:
https://github.com/Jigsaw-Code/outline-sdk/blob/v0.0.17/x/examples/outline-cli/app_linux.go#L36
Because in the normal case, once the tun device has been Closed, the traffic copy go-routine will terminate:
[INFO] 2024/12/06 16:47:43 tun -> OutlineDevice stopped: 27566484 read tun: file already closed

io.Copy should respect that and return immediately when one of the IO objects has been closed.


Response should be checked if it's not null

https://github.com/thephpleague/flysystem-aws-s3-v3/blob/master/src/AwsS3Adapter.php#L254
For example you try to put stream to wrong region, so you get response null and then:
Fatal error: Call to a member function getStatusCode() on null in flysystem-aws-s3-v3/src/AwsS3Adapter.php on line 254

Real exception message was:
Error executing "HeadObject" on "https://s3.wrong-region-name.amazonaws.com/my-bucket-name/screenshot/test.png"; AWS HTTP error: cURL error 6: Couldn't resolve host name

Yeah also got this with with real exception.
AWS HTTP error: cURL error 60: Peer certificate cannot be authenticated with given CA certificates

same issue here,
PHP Fatal error:  Call to a member function getStatusCode() on a non-object in flysystem-aws-s3-v3/src/AwsS3Adapter.php on line 254


fix: relocate where diagnostics are uploaded

Diagnostics would only get uploaded with exit code 0 or 3. This resolves it.
Also TypeError is not a decendant class of Error so you'll never get a stack trace.

Released on https://github.com/tvcsantos/detect-action/releases/tag/v2.0.2 and also floating tag https://github.com/tvcsantos/detect-action/releases/tag/v2


avoid nix log warning

warning: The interpretation of store paths arguments ending in `.drv` recently changed. If this command is now failing try again with '...^*'

@mergify queue

@mergify rebase

@mergify queue


Only deploy the WAR file without building again

Remove distribution management, because the repo id and URL is now
contained in the circle.yml. Remove sources plug-in, as we don't
distribute the sources on that way.

Updated

Great! Thanks a lot!


Draft: feat(): Animate with threejs

Please merge #45 to have a better overview of what I have done here 😄


feat(icons): add employee icons

TRN-5519
Description
Add Employee14 and Employee24 icons
How to test

Temploy

Screenshots
N/A
Development checks

[x] Add changeset according to guidelines (if needed)
[x] Read CONTRIBUTING.md and Component API principles
[x] Make sure that additions and changes on the design follow Toptal's BASE design, and it's been already discussed with designers at #-base-core
[x] Annotate all props in component with documentation
[x] Create examples for component
[x] Ensure that deployed demo has expected results and good examples
[x] Ensure the changed/created components have not caused accessibility issues. How to use accessibility plugin in storybook.
[x] Self reviewed
[x] Covered with tests (visual tests included)

Breaking change

[x] codemod is created and showcased in the changeset
[x] test alpha package of Picasso in StaffPortal

All development checks should be done and set checked to pass the
GitHub Bot: TODOLess action

PR commands

List of available commands:

@toptal-bot run package:alpha-release - Release alpha version
@toptal-anvil ping reviewers - Ping FX team for review

PR Review Guidelines

When to approve? ✅
You are OK with merging this PR and

You have no extra requests.
You have optional requests.

Add nit: to your comment. (ex. nit: I'd rename this variable from makeCircle to getCircle)

When to request changes? ❌
You are not OK with merging this PR because

Something is broken after the changes.
Acceptance criteria is not reached.
Code is dirty.

When to comment (neither ✅ nor ❌)
You want your comments to be addressed before merging this PR in cases like:

There are leftovers like unnecessary logs, comments, etc.
You have an opinionated comment regarding the code that requires a discussion.
You have questions.

How to handle the comments?

An owner of a comment is the only one who can resolve it.
An owner of a comment must resolve it when it's addressed.
A PR owner must reply with ✅ when a comment is addressed.

@sashuk Would you mind reviewing the happo diff? I have no rights to do so
Thank yor for the review 🙇🏻

@diogolessa Sure, reviewed, thank you!


fix retired? workflow implementation

closes #1572 and closes #1482. linked to #1640 and #1748
Don't use the moving subject counts (new / old data being (un)linked) to determine finished state.
Instead use the workflow.finished_at field set by retirement worker.
Also extract subject count methods to the respective models, add project / workflow specific
behaviour for the subject counts and add a live_subject_sets relation to project
via the workflows relation to only count the active subject sets in a project.
Review checklist

First, the most important one: is this PR small enough that you can actually review it? Feel free to just reject a branch if the changes are hard to review due to the length of the diff.
If there are any migrations, will they the previous version of the app work correctly after they've been run (e.g. the don't remove columns still known about by ActiveRecord).
If anything changed with regards to the public API, are those changes also documented in the apiary.apib file?
Are all the changes covered by tests? Think about any possible edge cases that might be left untested.

LGTM


texture.lock({ level: 0 }): only upload a single mip level if locked explicitly

I also have some questions about future directions texture.lock() API, which I would be happy to help implement.
I'm asking these in the context of #6003 - I kept that PR deliberately small, but can open a followup PR depending on your thoughts.
Apologies in advance for length!
Questions for PlayCanvas team
Question 1
As mentioned in #5754, it would be good to be able to:

Lock an individual slice of a 2D Texture Array
Lock a specific mip level, and:
Only upload the explicitly locked mip level instead of all of them

I'm excited to contribute an implementation for 1), but first I would like to clarify some issues around 2) and 3).
On 3): it looks to me like we always re-upload all mip levels regardless of whether we locked a specific one.
Q1: Is this correct, or have I missed something important in webgl/webgpu-texture.js?

Question 2
We could check texture._lockedLevel and use it to only upload the specified level when a level option was passed to texture.lock().
For example in webgl-texture.js:
        // Upload all existing mip levels. Initialize 0 mip anyway.
        while (texture._levels[mipLevel] || mipLevel === 0) {

+             // If we have locked a single mip level, only upload that mip
+            if (texture.lockedLevel !== -1 && mipLevel !== texture.lockedLevel) {
+                mipLevel++;
+                continue;
+            }

Q2: Does this seem like a good approach? Or should we be checking / updating texture._levelsUpdated instead?

Question 3

Currently, texture.lockedLevel defaults to 0 when no level option is passed.
This is because _lockedLevel !== -1 is overloaded to indicate whether a texture is locked.
With #6003, we can use _lockedMode for this instead, and leave _lockedLevel at -1 if no explicit level was specified.

The default API would have the same behavior as now:
const data = texture.lock(); // Returns mip level 0 data

/** Internally, _lockedLevel remains -1, because user didn't set it explicitly */

texture.unlock(); // Uploads all mipmaps

But when the user specifies mip level 0 explicitly:
const data = texture.lock({ level: 0 }); // Returns mip level 0 data

/** Internally, _lockedLevel is now 0, because user set it explicitly */

texture.unlock(); // Uploads only mip level 0

So for most cases of lock(), the behavior is the same, but it could break expectations in code that is passing { level: 0 } and expecting all mips to update.
Additionally, anyone relying on the private texture._lockedLevel would not longer see it set to 0 when a texture is locked in the default way. But they could switch to the public texture.lockMode instead to check the locked state.
Q3: Is this level of API/behavior change acceptable?
Thank you, and let me know if there is a better venue for implementation questions like this!

Hi @liamdon ,
Sorry it's taken so long to get back to you. This part of the engine sorely needs love and it's great to have you onboard!

Nope you've not missed anything. We should only re-upload the levels that have changed. I believe re-uploading everything (when only the top level changed) is a bug.

Texture upload doesn't happen immediately (check the implementation of texture.upload() - it's just setting dirty flags), but rather it happens on demand at render time (when the texture is about to be used). So a user can update multiple levels of a texture before the actual update is invoked.  (Also by render/upload time the textures should be unlocked anyway meaning _lockedLevel should always be -1).

Actually I would suggest not trying to keep the old behaviour at all. Re-uploading mipmap data that hasn't changed is just wasted effort since the texture won't look any different anyway. I'd suggest keeping the code as simple and straightforward as possible - when mip level isn't specified by caller, it defaults to 0 and that's the only level that gets re-uploaded at render time.

cc @mvaligursky, who might have some things to add especially from restrictions on the WebGPU side.

WebGPU is similar, but batch uploads all dirty textures before rendering in WebGPUGraphicsDevice._uploadDirtyTextures. The main reason here is that mimmap generation is done using render passes, and we cannot do those while inside other rendering (render pass).
Another difference on WebGPU implementation is that we create the empty texture right away, and then can upload to it at any time. WebGL does those operations both at a later stage.
WebGL upload also has some behaviour where if some mip levels are missing, those are automatically generated. Nothing like this is done on WebGPU (at least at the moment, but I wasn't planning on adding it). On WebGPU, if the user manually uploads mipmaps and some are not specified, the simply contain no data.

Nice, thanks.
Fortunately most of these differences are hidden inside the respective platform implementations (webgl-texture, webgl-device, webgpu-texture etc).
In terms of this public texture API, do you see any potential issues @mvaligursky?

Nope, the API suggestions sound great. Even though it probably not hurt to refactor the WebglTexture class a bit to make it more flexible, before implementing the changes - to make the code easier to follow.

related ticket: https://github.com/playcanvas/engine/issues/4290


Windows Server 2022 version

Any chance to support windows server 2022 version? I'm using Windows Server 2022 with remote desktop and I'm sure there are many people use it with their VPS or dedicated server.

@BHWarrior
does it work with the newest update ?

Sorry for late response. Yes, I've tried the latest version and can confirm that it works. Thank you!!

@BHWarrior Thanks for checking


Adding zero-knowledge

Addresses #9 .
Based on the recent work. The most noticeable features are:

We take into account the degree of the extension field when randomizing.
We add randomization of the quotient segment polynomials.
We add a random code-word to the DEEP composition polynomial as done in the Aurora paper.
In addition to salting the vector commitment, we also salt the Fiat-Shamir as is done in the specification of the BCS transform.

This is ready for full review but I am putting in draft mode as the current solution for generating randomness for zero-knowledge is not clean. More specifically:

For salted Merkle trees, should we also use a PRNG?
For the PRNG used in the prover, I am not happy with the way it is currently implemented, I am thinking that it should be sort of optional to have it but I couldn't come up with a good way to do that.

There are also some unnecessary allocations but I can remove those once we agree on the general structure.

The handling of the PRNG for zk has been improved and hence I believe that the PR is in a good enough shape for an initial review.


Is there a Get-Deployment or similar?

Apart from installing windows features, what does this cmdlet actually do?
If it outputs null how do I check whether it was successful?
If I run it multiple times is it idempotent?
Are there alternative cmdlets for achieving the same result?
When I executed it on my windows 2016 server it appeared to succeed, however the deployment was not visible via the "server manager" GUI, I was only able to manage it via powershell after that invocation.
Is there another cmdlet to enable the server manager GUI, or is there something missing from this one?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3b4fdc2e-72df-d7ba-68e3-df6bb25ae8ad
Version Independent ID: 054c9e86-0b4b-d06b-8a56-bad322c2788f
Content: New-RDSessionDeployment (remotedesktop)
Content Source: docset/windows/remotedesktop/new-rdsessiondeployment.md
Product: w10
GitHub Login: @coreyp-at-msft
Microsoft Alias: coreyp

Hello, @thetrav thank you for providing feedback.
Note that the form that you have submitted this feedback from is intended for product documentation issues only.
Please consider checking any relevant resources and asking the community about your problem via the following link.
PowerShell Community
Thanks.

@officedocsbot close


Fisrt Draft for Contributing Guide

This is a First Draft of Contributing Guide for comments. @salvadord
Issue #426

hey @ariel-brassesco at first glance it looks fantastic! thanks so much!
I'll try to have a look more in detail soon and give more feedback

Hi @salvadord 👋. I think there was a small confusion. @ariel-brassesco thumbed up your review to mark it as read. But he was planning to push the fixes latter. We will create a new PR containing the review corrections.
PD: I saw several questions these past weeks about NetPyNE 🚀 🚀 🚀


Better loading of configuration

[x] update json tags
[x] Update goctl template

I don't think the change works well for the existing code.

@kevwan can you explain what is the usage of optional=CertFile.
CertFile           string `json:",optional"`
CertKeyFile        string `json:",optional=CertFile"`
CACertFile         string `json:",optional=CertFile"`

It means that if CertFile has value, then CertKeyFile and CACertFile have values.

@kevwan please take a look

This PR is important for "12 Twelve Factor" compliance. Please approve this PR or close it, so I know whether I can adopt this framework. 谢谢

@kevwan please try to merge it in the next release. I've been waiting for a month for better config loading


include associations on login

Hey !
Is it possible to tell devise to include associations into the json on login / auth ?
For example I have a tables "links" and the current user has 4 links. I want to add those links into the user json.
I have tried to edit the as_json method inside the user model:
def as_json(options = {})
hash = super
hash[:links] = self.links
hash
end

But still get the same json.
Thank you

I'm using as_json in a couple different apps and it works for me. Here's one example of how I'm successfully doing it.
def as_json(options = {})
  super(options).merge("organization" => self.organization)
end

I'm curious what would happen if you tried:
def as_json(options = {})
  super(options).merge("links" => self.links)
end

I think overriding as_json is a suitable implementation. Closing but can re-open if needed.


Master

Corrects issue #3588

Thanks @vrittis!  Can you please send in a Contributor License Agreement as described here so we can review this?

@pjcozzi Done

Thanks @vrittis, we received it!

Thanks @vrittis.  I'm not sure why it was failing in IE to begin with, but I can definitely reproduce the problem and your fix is good.

Nice! it took me a long time understanding why I couldn't get the test to fail under the chrome instance I was lauching jasmine in :crystal_ball:

@vrittis can you please add yourself to https://github.com/AnalyticalGraphicsInc/cesium/blob/master/CONTRIBUTORS.md?

Sorry, @pjcozzi, I already merged this and forgot to ask @vrittis to do that.  I'll add it myself in master.  Thanks again @vrittis


Remove: Discord.Discord.Development version 1.0.654

## .\Tools\YamlCreate.ps1 -SkipPRCheck -Mode 5 -PackageIdentifier Discord.Discord.Development -PackageVersion 1.0.654

[X] Have you signed the Contributor License Agreement?
[X] Have you checked that there aren't other open pull requests for the same manifest update/change?

[X] Does your manifest conform to the 1.0 schema?

Microsoft Reviewers: Open in CodeFlow

/azp run


Quick sort algorithim

quick sort algorithim in java, in this algorithim every element (also known as "key") is compared to predecessor and is then sorted in pairs of two.

@saraswati258 kindly review the PR.


[WIP] Test incident creation flow.

Re-creating this PR as GitHub wasn't picking up additional commits to the branch, and the build was failing without pytest-mock.
What do you think is the best way to handle the assignment of incident_type_plugin_metadata? I'd rather just have a plain dict assignment. Return a Query object from incident_type.service.get_by_name(), then mock get_by_name()? Is there a way to ignore the assignment altogether?

Closing this PR after discussion with @mvilanova


Meaning of fut_pred in details

The decode() function, in the highwayNet class, is the one returning the future trajectory predicted. Before returning the trajectory, fut_pred is modified by outputActivation() in utils.py.
The return value of outputActivation() is a concatenation of  muX, muY, sigX, sigY and rho. The comment above tells us this output layer comes from Graves, 2015 but I don't understand the meaning of these 5 variables.
Can you explain to me the meaning of these variables?

I think it's mu for mean, sig for standard deviation, and rho for the correlation. sig is actually the reciprocal of the real sig though, according to #23.


apex upgrade can have cross-device link error

When /tmp and apex binary are on different partition, os.Rename  will have this error
⨯ Error: rename /tmp/apex-upgrade /usr/local/bin/apex: cross-device link

I'm not sure how to resolve this.

Ah I didn't think of that, hmm not sure either off-hand

Hmm, maybe use the cmdPath instead of tmpPath for the download. (https://github.com/apex/apex/blob/6235d4a8b25901e23101ce372c3cb477ba23c3aa/upgrade/upgrade.go#L65)


Open file in specific group based on file extension

I am working with a single vertical split, 2 editor group layout, one group for source files and another for header files. I would like to know if it's possible to setup VSCode so that *.c files always open in the left editor group and the *.h files always open on the right. If it's not possible built-in, would the extension api allow implementing this?

Nevermind I have achieved this by making a small extension for myself. It was never going to get 20 upvotes anyways. If anyone is interested: https://marketplace.visualstudio.com/items?itemName=sarper.source-left-header-right


Fix path for listing context variables

The URL for listing environment variables in a context is incorrect. The proposed change should fix the problem.

@grezar Please take a look, this should fix the problem with context.ListVariables in my opinion.

I've released your change as https://github.com/grezar/go-circleci/releases/tag/v0.6.2


Client Allocation Request for: 广东中科智云科技有限公司

Client Allocation Questions
Core Information

Name: 广东中科智云科技有限公司
Website / Social Media: http://www.gdzkzy.cn/
Region: Greater China
DataCap Requested: 20TiB
Addresses to be Notarized: f3tct62bxuqrmunacbnuq464q4kineqyvo36mca54dtek35rkvh2afdkt4h7jno5vearn26o4stcflbvkoxj6a
Notary Requested: IPFSUnion

@zhiyun11 Please subscribe to notifications for this Issue to be aware of updates. Notaries may request additional information on the Issue.

Thank you for the first to apply to me after I become a notary. I'd like to know more about your business. Can you kindly answer some my following questions?

individual and organization

Please introduce yourself and your team briefly, and show me some your social media accounts such as GitHub, email address, WeChat official account etc.
For your organization, Please describe its time since inception, registration country，scale, named officers, project, industry reputation etc.

track record

Regarding your data you intend to store, What’s the original source of those data, and do you have any proof of ownership or authorization?
Has your organization obtained DataCap before?
Please list any of your honor and disputes in the field of Filecoin.

use plan

Please describe your plan to use DataCap briefly? (such as use case, miner etc.)

Thank you for the first to apply to me after I become a notary. I'd like to know more about your business. Can you kindly answer some my following questions?
Yes. As a professional digital ecological high-tech enterprise, Guangdong Zhongke Zhiyun Technology Co., Ltd. follows the national development strategies, such as "New Infrastructure Plan", "Calculate eastern data from western data centers", etc. We take an active part in the digital transformation and chain reform of traditional industries together with the Software technology application research and development department of The Chinese Academy of Sciences.  We have reached a data storage service cooperation with Guangzhou Zhongke Yide Technology Co., LTD.  So, We will provide the information of Guangzhou Zhongke Yide Technology Co., LTD.
individual and organization
Please introduce yourself and your team briefly, and show me some your social media accounts such as GitHub, email address, WeChat official account etc.
For your organization, Please describe its time since inception, registration country，scale, named officers, project, industry reputation etc.
Its time since inception:	2018.08.22
Registration country: Guangzhou
Scale:cover all china
Project:Guangzhou Zhongkeyide Technology Co., Ltd. (hereinafter referred to as "Zhongkeyide") is a high-tech enterprise incubated by the Institute of Software of the Chinese Academy of Sciences in Guangzhou, focusing on the construction of traceability system for edible agricultural products.With the support of key technologies such as big data, artificial intelligence and block chain, it has been committed to providing the overall solutions of electronic traceability of food and drug safety for civil, government and enterprises.
In recent years, with the national, provincial, city, district market supervision departments at all levels, the traceability as a solution to a new type of food and drug safety and trust industry supply chain management system, the division francois gradually establish a drug safety electronic traceability series products as the core, the formation of software development, business consulting and technology service three big business module,Provide a total solution covering the whole traceability from the source to the consumer.
In the traceability field, Zhongkeeedec has applied for dozens of invention patents and software Copyrights.  Zhongke Yide is leading in the industry with mature food electronic traceability technology. We have provided overall solutions of electronic traceability of food and drug safety for civil, government and enterprises.
Guangdong Zhongke Zhiyun Technology Co., Ltd. have in-depth cooperated with Zhongke Yide to build a joint blockchain laboratory of traceability data storage, and carry out in-depth cooperation in data storage and blockchain application development.
Industry reputation:
Key Technology Innovation Award  of China Internet of Things
Excellent Product Award of the 14th CHTF
Guangdong High-tech Product Award
The 8th China "Kechuang Cup" Innovation and Entrepreneurship Competition electronic information industry start-up group winning prize
Guangdong "Mass Innovation Cup" entrepreneurship and Innovation Competition, Doctoral and postdoctoral innovation Competition, enterprise group winning prize
Vice president unit of Guangdong Food Circulation Association
Outstanding Project Award of Financial Technology Innovation Competition of Guangdong Financial High-tech Zone
track record
Regarding your data you intend to store, What’s the original source of those data, and do you have any proof of ownership or authorization?
The data types we intend to store in the Filecoin network are transaction notes and blockchain traceability certificates (corporate privacies are hided). These data play an important role in the company and we want to store them in a way that prevents loss and alteration.
Format: PDF
Source: Business generation (Guangdong Intelligent Food and Drug Administration System, https://qy.gdfda.gov.cn/)
Has your organization obtained DataCap before?
None
Please list any of your honor and disputes in the field of Filecoin.
None
use plan
Please describe your plan to use DataCap briefly? (such as use case, miner etc.)
Use case:transaction notes and blockchain traceability certificates (corporate privacies are hided)
Redundancy:2-4copies
Location:China
Specific requirement of storage miners: famous and responsible

Looks good. I decide to approve 100GiB for you in the first time. Based on your subsequent performance, I may allocate more next time.

@IPFSUnion thanks.

Request Approved
Your Datacap Allocation Request has been approved by the Notary
Message sent to Filecoin Network

bafy2bzacedxsdu37z7gxgezwvrpyvzeq6orgte7kz5a4rlrkngvruqmrgdhsq

Address

f3tct62bxuqrmunacbnuq464q4kineqyvo36mca54dtek35rkvh2afdkt4h7jno5vearn26o4stcflbvkoxj6a

Datacap Allocated

20TiB

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedxsdu37z7gxgezwvrpyvzeq6orgte7kz5a4rlrkngvruqmrgdhsq


Where can I download the latest ig_dataset

I followed the instruction from the official documentation and downloaded ig_dataset.tar.gz from https://storage.googleapis.com/gibson_scenes/ig_dataset.tar.gz
After decompressing it and trying to use the data with the latest version of igibson, I got the error complaining that my ig_dataset is outdated as its version is v2.0.1. The latest version of igibson (I have 2.1.0) only accepts ig_dataset whose version is in the range of [2.0.6, 2.1.1)
Where can I download the latest version of ig_dataset, if not from the provided url? Thanks

When I download the latest ig_dataset, the version is:

wget https://storage.googleapis.com/gibson_scenes/ig_dataset.tar.gz
tar xzf ig_dataset.tar.gz
cat ./ig_dataset/VERSION

It shows 2.0.6, which is the most current version.

When I download the latest ig_dataset, the version is:

wget https://storage.googleapis.com/gibson_scenes/ig_dataset.tar.gz
tar xzf ig_dataset.tar.gz
cat ./ig_dataset/VERSION

It shows 2.0.6, which is the most current version.

the tar.gz file I downloaded has ig_dataset_encrypted/version "2.0.1" and the version file name is lower cased instead of VERSION

Did you repeat those steps exactly, because it seems like you have an old file.

I did it again and this time I got 2.0.6. So thanks that resolved the issue. I don't know why somehow I managed to download the older version...


Publish Visual Studio Extension usability feedback: file picker vs. directory picker + file name input

For the Visual Studion Extension publish task. It feels confusing to have a (neat) file picker to select the location of the .vsix file. Why can't the same be done for the Manifest file? Why is this split over a directory picker and an input for the filename? Wouldn't a single file picker do the trick? Especially since only the first match for the directory + filename pattern is picked up?

@onlyutkarsh feedback?

@henrybeen fair feedback. I tried to reuse the logic used to search for the manifest file. But what you saying makes sense. I should probably make the change.
@jessehouwing To not to break the existing builds using this task, I think this should be in the next version of the task. But I see, we don't deliver multiple versions of the tasks as of today - Thoughts?

It's preview. I'm not hesitant to break things while in preview.

Breaking things is never fun, but I would prefer a break in this case over a quick increase in versions.


what is the theory behind OFFSET_TABLE?

when i read brpc source,i see the OFFSET_TABLE array, it used to calculate _steal_offset,and my question is:what is the theory behind OFFSET_TABLE(offset_inl.list)？Is the const int in this array has any math meaning?

Same as algo of rr load balancer . All the numbers in the array are prime number.


Add option to register on TCP server

If enabled, register CRI on containerd's TCP endpoint
#1181

@mikebrow what needs to be done to not be a WIP?

when @Random-Liu gets back we should have a discussion regarding how this should work..
I'm thinking we don't need a new flag to enable it, we should build off the containerd config.
It might be good to consider an integration test maybe use a modified crictl validate to hook up over tcpip instead of the socket.

Sounds good, thank you.
My rationale behind the flag is that some users might want to enable the TCP endpoint for containerd without exposing CRI.
The issue with crictl is that the validation for the protocol comes from kubelet code. But maybe we can craft a PR that maintains the kubelet expectations but we can still have crictl code work for TCP.

There is already way to not enable/disable the cri plugin (cri services). Mixing the end point config just seems odd given that we have one socket ATM for containerd runtime (with an optional one for image). Now if we had cri as a different potential socket ala how runtime/image are done then I agree we could have different config options.  But if we do that then we would also want tcp config options ala how the config options were done in containerd... eg ip, cert, and key for cri.  I note that when they added tcp I don't think they split runtime and image? At least I didn't notice if they did.
IMO we should just ride on the containerd config vs have a different set of socket/tcp config options for cri. This to reduce complexity.
I just presumed when this is done it would be cri compatible, but over tcp instead of a unix socket.

I'm confused, don't we register both services in the same grpc server? https://github.com/containerd/cri/blob/322cd489653645e70f9bb2f17d2953c8fecbc66c/pkg/server/service.go#L175

I see my msg was confusing.. edited. Yes we do and that's my point. If we are loaded we register period, there is no switch.

to your edit .. yeah we don't "split" cri runtime and image services across two different containerd sockets.. even though that is an option at the cri layer.

I'm thinking we don't need a new flag to enable it, we should build off the containerd config.

I also had similar problem when reviewing the code.
Let me think about it more.

No containerd services have register TCP yet (no RegisterTCP implementation in containerd services), so I think at least for now this is a per service decision. Given so, I think an enable_tcp_service option makes sense.

No containerd services have RegisterTCP implemented yet

That is my point, I think the intent was if it is defined, that is the "enable" switch that enables it for all services.. Or maybe I read it wrong? If we have a switch off it should be enable by default at any rate, IMO.

That is my point, I think the intent was if it is defined, that is the "enable" switch that enables it for all services.. Or maybe I read it wrong? If we have a switch off it should be enable by default at any rate, IMO.

Each plugin can decide to implement RegisterTCP. If they do, they are added to the TCP server. However, the TCP server is disabled if there is no TCP address defined in the configuration.
The point of the flag is to give users the ability to disable cri on TCP even if they want TCP support for other plugins. Having it disabled by default is probably the best option because most users won't need it.

That is my point, I think the intent was if it is defined, that is the "enable" switch that enables it for all services.. Or maybe I read it wrong? If we have a switch off it should be enable by default at any rate, IMO.

Each plugin can decide to implement RegisterTCP. If they do, they are added to the TCP server. However, the TCP server is disabled if there is no TCP address defined in the configuration.
The point of the flag is to give users the ability to disable cri on TCP even if they want TCP support for other plugins. Having it disabled by default is probably the best option because most users won't need it.

Let me put it another way. We do not have an enable flag for the socket.

Let me put it another way. We do not have an enable flag for the socket.

True. Given that CRI is the first and only service to implement RegisterTCP, users have already all the means to have socket support only.
We can always re-evaluate the addition of a flag in the future. The approach of having the flag is simply more cautious. @Random-Liu wdyt?

I'm not against a flag entirely... but I do think it should be a disable flag that defaults to false, giving customers a way to disable tcp for cri if they want to enable it for the rest of the containerd services by specifying the tcp options.

CRI endpoint is a "privileged" endpoint, because users can use it to mutate node state, e.g. pull image or create container. I do believe that users sometimes only want to enable tcp endpoint for readonly services, e.g. metrics, so an option to disable CRI is required.

I'm not against a flag entirely... but I do think it should be a disable flag that defaults to false, giving customers a way to disable tcp for cri if they want to enable it for the rest of the containerd services by specifying the tcp options.

I agree.  My understanding about your comment is that: an enable_tcp_service can't really guarantee that CRI tcp service is enabled, because users still need to enable containerd tcp server, which may confuse users. However, a disable_tcp_service can guarantee that the CRI tcp service is always disabled, no matter tcp server is configured or not. Given so, disable_tcp_service does sound better to me. :)

CRI endpoint is a "privileged" endpoint, because users can use it to mutate node state, e.g. pull image or create container. I do believe that users sometimes only want to enable tcp endpoint for readonly services, e.g. metrics, so an option to disable CRI is required.

I'm not against a flag entirely... but I do think it should be a disable flag that defaults to false, giving customers a way to disable tcp for cri if they want to enable it for the rest of the containerd services by specifying the tcp options.

I agree. My understanding about your comment is that: an enable_tcp_service can't really guarantee that CRI tcp service is enabled, because users still need to enable containerd tcp server. This may confuse users. However, a disable_tcp_service can guarantee that the CRI tcp service is always disabled, no matter tcp server is configured or not. In this case, disable_tcp_service does sound better to me. :)
As for the default value, I prefer true, because 1) it is not a common use case; 2) even though the default is false, it is not enabled by default, because tcp server is not enabled by default.

fair points. I strongly agree!
We should make a note of pushing a doc PR to the containerd tcp config such that a user setting tcp config knows that, by default, the CRI tcp endpoints are disabled and thus have to be explicitly enabled by turning off the disable flag.
At some point in the future it may make sense to have an array of white/black flags for tcp enabling of service(api) end points for CRI and possibly other services in containerd. But this will do for now :-)

Thanks for all the feedback. I think we arrived to a pretty good conclusion.
Regarding updating containerd docs, it might be a little early now, given that CRI is the only service using the endpoint. I expect users of CRI to first come to our docs and then learn that they have to add the endpoint in main containerd's config.


[TRUNK-13813] Expand smoke test scope

Adds quarantined test and production to smoke testing.

When do you plan to uncomment?
I put it up for review early while looking at the 8000 transition issue, but now that I've had a test work on prod, it's uncommented.


fix notification issue (#4787)

Pull request opened by the merge tool on behalf of #4787

Merged in 2e031f78d10e27217e2c136c4980288ecaf4b79e


Selecting Resources

Under [Match statements], in the second paragraph, line no. 1
In this snippet, the match statement matches on any **and** all resources that have the kind Service..
According to me, it is more below statement will be more meaningful than the above statement,
In this snippet, the match statement matches on any **or** all resources that have the kind Service. .
Because at one point in time it can select either any resources or all resources, if it selects any then it can't select all or if selects all then no meaning of any.

Maybe rephrasing like this is clearer still:
In this snippet, the match statement matches on all resources that have the kind Service.

Maybe rephrasing like this is clearer still:
In this snippet, the match statement matches on all resources that have the kind Service.

@chipzoller, yes it is more clear statement.

Would you be willing to open a PR for this and your other suggested correction?

/assign

@viveksahu26 are you taking care of this?

@chipzoller , i am sorry. I forgot.
Yes, i will take care of this.

@chipzoller , I have already made changes to it and it's has been already updated there. You can close this issue now.

PR was merged for the above issue.


Broken Link

The link for the CodeDOM Quick Reference is broken

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0ceb2afd-2c2e-1c86-9839-54a3c8b0bd7e
Version Independent ID: 01453121-3a20-8e81-5906-895bfec826f7
Content: Dynamic Source Code Generation and Compilation
Content Source: docs/framework/reflection-and-codedom/dynamic-source-code-generation-and-compilation.md
Service: unspecified
Product: dotnet-framework
GitHub Login: @rpetrusha
Microsoft Alias: ronpet

Are you talking about the last link in the article, in the Related Sections section? That link links to http://msdn.microsoft.com/library/c77b8bfd-0a32-4e36-b59a-4f687f32c524, which works for me.

The link works for me also, @adibradfield. I'm going to close this issue, since we can't reproduce the broken link. If it continues to be a problem, please respond to this issue to let us know.

I can confirm that I am no longer getting the issue


Docs navigation annoyance: scrolling to open mini-toc on right

A user has stated that it's annoying to have to scroll into a section in order to get the mini-toc on the right to expand and display the subsections. Many of us here agree. Let's look at how the ToC can either be static or... any other solution?
We do need to be careful about having a very long "mini"-toc.
Possible solutions:
-- adding rotating arrows just like the left-side nav
-- add a "landing page" that is titled something obvious, so they know to click it to get each of the sub-pages
-- ??

Additional option: Include the Table of Content in the page before the content (hidden by default, but fully uncollapsible with 1 click)
It might make sense to use different strategies depending on the content. A short guide probably has different usage than a multi dozen headline reference page.

@tanberry @peterondesign I think I might need more clarity on the behavior of TOC here. Do we need to brainstorm on ideas ?

Yes @Nilu we probably do need to brainstorm on ideas... the "jumping around" of the mini-ToC's elements is a problem, and is listed in this article as one of the top 10  UI mistakes to avoid ("Uncontrollable Navigation Elements").