Swedish radars sehem, sehud, seoer, seosu

Cf. #2, we are getting data from the Swedish radars sehem and seoer:
http://enram.github.io/data-repository/?prefix=se/hem/ : 2017, 2018, 2019
http://enram.github.io/data-repository/?prefix=se/oer/ : 2017, 2018, 2019
But these radars are not listed by OPERA: https://github.com/enram/dmp/blob/master/data/OPERA_RADARS_DB.json
@leijnse @CeciliaNilsson709 any information which radars this are and why they are missing in the OPERA database?

Ok, I have no idea what is going on. Looks in the repository like they might be seovi (Örnsköldsvik) and sease(Ase) but that is just guessing. I'll check with Günther.

You're right, it could be those radars indeed:
seovi stops having data in 2017-08-06
sease stops having data in 2017-05-13
While:
sehem: starts having data in 2017-11-13
seoer: starts having data in 2017-11-24
But why would they change the code?

seovi is probably sehem, as it is close to location Hemlig (as well as Örnsköldsvik). Don't know why (or if) sease would be seoer. And no idea why they would change them 🤷♀

It's actually probably the other way around, sease is close to Hemse (so sehem) and seoer probably refers to Örnsköldsvik after all (ö->oe). But needs to be confirmed of course.

Got this list of new(old) codes for the Swedish radars from Günther:
ANG (ANG)
ATV (VIL)
BAA (ARL, radar Bålsta will replace radar Arlanda)
HEM (ASE)
HUV (HUD)
KAA (KKR)
KRN (KIR, radar Kiruna will change name during upgrade in august)
LEK (LEK)
LLA (LUL)
OER (OVI)
OSD (OSU)
VAX (VAR)

Aha, that is helpful! 👏

Did he mention why they decided to change some of these codes?
I assume we can just move data stored under the old code to the new code?
Not sure about moving BAA to ARL, unless the new data is at the same location as the old one? Do they consider it the same thing?

You might want to compare the lat,lon metadata of the new and old files. It could be that the physical locations of the radars are not exactly the same (different building in the same city), and that therefore they got a new code.

I will test, compare and report back.

Ok, here are the results:
Coordinates
two lines = pair of similar radars

radar
lat
long

sease
57.3035011292
18.400100708

sehem
57.30339813232422
18.400299072265625

sehud
61.5770988464
16.714399337800003

sehuv
61.577110290527344
16.714433670043945

seovi
63.6394996643
18.4018993378

seoer
63.63946914672851
18.40188217163086

seosu
63.294998168900015
14.7590999603

seosd
63.29499816894531
14.75909996032715

sevar
58.2555999756
12.8260002136

sevax
58.25559997558594
12.826000213623047

Code:
library(bioRad)
library(tidyverse)
metadata <- tibble()

# Change code below to select another radar

radar = "sevax"
date_min = "2017-09-01"
date_max = "2017-09-30"
directory = "my_data"

# Run code below to add radar to metadata

download_vpfiles(radar = radar, date_min = date_min, date_max = date_max, directory = directory)
vp_selection <- select_vpfiles(radar = radar, date_min = date_min, date_max = date_max, directory = directory)

vp <- read_vpfiles(vp_selection[1])

new_row <- tibble(
  radar = vp$radar,
  lat = vp$attributes$where$lat,
  long = vp$attributes$where$lon
)

metadata <- bind_rows(metadata, new_row)

Distances

radar_1
radar_2
distance_in_meter

sease
sehem
16.56898

sehud
sehuv
2.225491

seovi
seoer
3.50662

seosu
seosd
5.229753e-06

sevar
sevax
2.068746e-06

Code:
library(sf)
sf_metadata <- st_as_sf(metadata, coords = c("long", "lat"), crs = 4326)
st_distance(sf_metadata[9,], sf_metadata[10,])

Conclusion
The radars have different coordinates, but their distance are almost negligible. The biggest difference is between sease and sehem, which is 17m. It still looks like the exact same spot in the forest though:

I think we can safely move the old files under the new code directory?

Great comparison. I still suggest that someone checks directly with the Swedish meteorological Institute to be sure that something hasn’t changed. Some countries are busy replacing radars.
From: Peter Desmet<EMAIL_ADDRESS>Sent: Thursday, 11 April 2019 10:31 AM
To: enram/dmp<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Subject: Re: [enram/dmp] Swedish radars sehem, sehud, seoer, seosu (#5)
Ok, here are the resultshttps://github.com/enram/dmp/files/3067572/metadata.txt:
Coordinates
two lines = pair of similar radars
radar
lat
long
sease
57.3035011292
18.400100708
sehem
57.30339813232422
18.400299072265625
sehud
61.5770988464
16.714399337800003
sehuv
61.577110290527344
16.714433670043945
seovi
63.6394996643
18.4018993378
seoer
63.63946914672851
18.40188217163086
seosu
63.294998168900015
14.7590999603
seosd
63.29499816894531
14.75909996032715
sevar
58.2555999756
12.8260002136
sevax
58.25559997558594
12.826000213623047
Code:
library(bioRad)
library(tidyverse)
metadata <- tibble()
Change code below to select another radar
radar = "sevax"
date_min = "2017-09-01"
date_max = "2017-09-30"
directory = "my_data"
Run code below to add radar to metadata
download_vpfiles(radar = radar, date_min = date_min, date_max = date_max, directory = directory)
vp_selection <- select_vpfiles(radar = radar, date_min = date_min, date_max = date_max, directory = directory)
vp <- read_vpfiles(vp_selection[1])
new_row <- tibble(
radar = vp$radar,
lat = vp$attributes$where$lat,
long = vp$attributes$where$lon
)
metadata <- bind_rows(metadata, new_row)
Distances
radar_1
radar_2
distance_in_meter
sease
sehem
16.56898
sehud
sehuv
2.225491
seovi
seoer
3.50662
seosu
seosd
5.229753e-06
sevar
sevax
2.068746e-06
Code:
library(sf)
sf_metadata <- st_as_sf(metadata, coords = c("long", "lat"), crs = 4326)
st_distance(sf_metadata[9,], sf_metadata[10,])
Conclusion
The radars have different coordinates, but their distance are almost negligible. The biggest difference is between sease and sehem, which is 17m. It still looks like the exact same spot in the forest though:
[Image removed by sender. Screenshot 2019-04-11 at 10 11 06]https://user-images.githubusercontent.com/600993/55942420-a5c20c00-5c44-11e9-9494-57ac0b0e6cb4.png
I think we can safely move the old files under the new code directory?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/enram/dmp/issues/5#issuecomment-482020587, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHVIzKdHRoeBq_wpyyDOz8jE9bQ2KPbuks5vfvLGgaJpZM4ckCtS.

It looks good to me. I can check with Günther again, but not really sure what more we want to know? Seems like for our purposes we can just move the files (except for future sebaa, as discussed above).

Cecilia, if you could just check with Gunther that there haven’t been changes to the radars and just changes to the meta data, that would be more than enough.
From: Cecilia Nilsson<EMAIL_ADDRESS>Sent: Thursday, 11 April 2019 10:28 PM
To: enram/dmp<EMAIL_ADDRESS>Cc: Shamoun-Baranes, Judy<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [enram/dmp] Swedish radars sehem, sehud, seoer, seosu (#5)
It looks good to me. I can check with Günther again, but not really sure what more we want to know? Seems like for our purposes we can just move the files (except for future sebaa, as discussed above).
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/enram/dmp/issues/5#issuecomment-482298900, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHVIzGE5EJZqtyM7PkAPT_EChx_hWTnZks5vf5rXgaJpZM4ckCtS.

Ok, I’ll double check with him so we can be completely sure 👍

I sent this summary to Günther a while ago just to make sure we haven't misunderstood anything, waiting for him to reply.

Radar
Comment

ANG
No change.

VIL -> ATV
Not sending under VIL since jan 2019, but no ATV yet?

ARL
No change, will stop sending when BAA starts

BAA
Will appear later, don't change anything

ASE -> HEM
Change

HUD -> HUV
Change

KKR -> KAA
Not sending under KKR since august 2018, but no KAA yet?

KIR -> KRN
Not yet sending under this name. Change later

LEK
No change

LUL -> LLA
Not sending under LUL since august 2018, but no LLA yet?

OVI -> OER
Change

OSU-> OSD
Change

VAR -> VAX
Change

Here are Günthers comments (in my translation):

Radar
Comment
Günther's comment

ANG
No change.

VIL → ATV
Not sending under VIL since jan 2019, but no ATV yet?
Should start sending under ATV soon

ARL
No change, will stop sending when BAA starts
ARL will stop on Monday 2019-04-29

BAA
Will appear later, don't change anything
Starts on Monday 2019-04-29, but will take awhile before the data appears. Coordinates are: 59.6110° N, 17.5833° E.

ASE → HEM
Change

HUD → HUV
Change

KKR → KAA
Not sending under KKR since august 2018, but no KAA yet?
KAA should be on server now

KIR → KRN
Not yet sending under this name. Change later
Will change during August 2019

LEK
No change

LUL → LLA
Not sending under LUL since august 2018, but no LLA yet?
LLA should be on server now

OVI → OER
Change

OSU → OSD
Change

VAR → VAX
Change

He also said: "Right now only our OPERA single-pol sweeps are being used to create bird profiles (three elevations for upgraded sites and ten for non-upgraded sites, ARL and KIR, but we will change to dual-pol sweeps (10 elevations) as soon as possible. BAA will be the first."

Thanks @CeciliaNilsson709. Did Gunther also explain why the name changes if the radar stays at the same location?

Nope. Maybe they just wanted to change it up a bit 💃🤷♀?
Seriously though, he didn't say explicitly, but it seems to be connected "upgrades", which I assume is dual-pol upgrades (but don't know why only some changed).

Ok, I now know HOW to change this on the S3 bucket. I'll create a spreadsheet to track what needs to be done and what has been done already. 👌

All code changes are in effect, except:

KIR → KRN
Not yet sending under this name. Change later
Will change during August 2019

It is still submitting under sekir.
Many of the new codes are not in OPERA DB.

sekir is still submitting under that code, and not sekrn. As that is the only remaining thing and it is documented in the wiki (https://github.com/enram/globam-dmp/wiki/se), I will close this issue


Reading input space from Code output

As per a team meeting discussion held on April 4, 2016, a few concepts about restarts came forward.
When RAVEN stores a sample in a data object, it reads the input space first from the output of the model, if possible, then the internal storage, if not available.
This can create an issue of the output of the model returns low-resolution values, as these values will not match the searched-for values in a restart.
We discussed adding a flag that will trigger the above behavior when desired, but by default will use RAVEN internal values first when possible.

For Change Control Board: Issue Review
This review should occur before any development is performed as a response to this issue.

[x] 1. Is it tagged with a type: defect or improvement?
[x] 2. Is it tagged with a priority: critical, normal or minor?
[x] 3. If it will impact requirements or requirements tests, is it tagged with requirements?
[x] 4. If it is a defect, can it cause wrong results for users? If so an email needs to be sent to the users.
[x] 5. Is a rationale provided? (Such as explaining why the improvement is needed or why current code is wrong.)

For Change Control Board: Issue Closure
This review should occur when the issue is imminently going to be closed.

[x] 1. If the issue is a defect, is the defect fixed?
[x] 2. If the issue is a defect, is the defect tested for in the regression test system?  (If not explain why not.)
[x] 3. If the issue can impact users, has an email to the users group been written (the email should specify if the defect impacts stable or master)?
[x] 4. If the issue is a defect, does it impact the latest stable branch? If yes, is there any issue tagged with stable (create if needed)?
[x] 5. If the issue is being closed without a merge request, has an explanation of why it is being closed been provided?

I believe this issue related to restartTolerance, which allows users to input the tolerance for restart point. This feature was implemented two years ago. I believe this issue has already been fixed.


review Node types against other work

@saulshanabrook mentioned today that we can reference other folks who have also created abstractions for computations, such as pytorch and tensorflow, and see how they model stuff like control flows. Will also potentially help with integrations.
Important but not urgent.

closing as no action items listed.


Add basic bulk document update API function

Can you add tests?

It's been awhile since I've messed with it (got it working and put into production and moved on 😅) but I'll try to find some time to do so.


Disallow copying of ScopedConnection & explicitly allow moving

Prevents accidental disconnection due to reallocation when ScopedConnections are stored in a container.

:+1: have had this before

We should consider to remove the ConnectionMap class as it becomes obsolete with these changes.


aya: Export BTF crate

Allows users to inspect program BTF for metadata.
Allows additional use cases for users who wish only to deal with BTF
parsing.
This is hidden behind the btf feature
Signed-off-by: Dave Tucker<EMAIL_ADDRESS>

Closing due to discussion on Discord. I'm going to clean up the Btf API first and then will revisit.


Added AirBnB Dataset

Added dataset for issue #96

@Gokul-Ks
Congrats 🎉, your PR got merged. To be eligible for the swags from ClueLess. Follow these steps.

Star this repository. ⭐
SignIn / SignUp in [ClueLess Website](https://www.clueless.tech/) .
Go to the profile section from the top right-hand corner.

Comment your profile URL under this comment tagging the person who merged your PR.

That’s it. Follow the leaderboard!
And don’t forget to share your achievement with your friends.

https://www.clueless.tech/profile/admin/73174047
@nikhil25803


[data-channels-flow-control] Connection is abruptly broken

I ran cargo run --example data-channels-flow-control on my laptop after cloning the main repository and changing into the examples directory.
Upon running it, I get the following output:
Press ctrl-c to stop or wait for 5s
Peer Connection State has changed: connected (answerer)
Peer Connection State has changed: connected (offerer)
OnOpen-data-1 : Start sending a series of 1024-byte packets as fast as it can
OnOpen: Start receiving data
Throughput: 6.853 Mbps
Throughput: 7.021 Mbps
Throughput: 7.113 Mbps
Throughput: 7.125 Mbps
exit on_open
Peer Connection State has changed: closed (offerer)
Peer Connection State has changed: closed (answerer)

I added a simple print statement to see why the connection closes and the send loop is exited, and it outputs the following:  DataChannel is not opened

it is expected, since it only run 5s.
println!("Press ctrl-c to stop or wait for 5s");
let timeout = tokio::time::sleep(Duration::from_secs(5));
tokio::pin!(timeout);


feat: implement 0x50 - POP

Since
Frontier
Group
Stack Memory Storage and Flow Operations
Description
0x50 - POP
Index 1 is top of the stack. See PUSH.Stack inputy: a stack item.Example
| Input | Output
-- | -- | --
1 | 125985 |
Reproduce in playground.Error casesThe state changes done by the current context are reverted in those cases:Not enough gas.Not enough values on the stack.

I can take this one:)!


Fix typo in docstrings of RandomForest{Regressor,Classifier}

Fix typo: min number... instead of max number... for RandomForestRegressor
and RandomForestClassifier

The errors on Travis CI for Julia 1.0 seem to be package dependency errors ([1]) and on nightly, some unit tests fail. They don't seem to be caused by my docstring changes.

@Sh4pe Thanks for the fixes. I will investigate the CI failures.

The error also shows up silently on the MLJModels julia v1.5 tests here

I've started investigating. Hopefully #308 will fix. 🤞

See also: https://github.com/dfdx/NaiveBayes.jl/pull/44

Okay, it seems #308 will fix the silent fail but not the julia 1.0 fail, which probably needs the NaiveBayes PR.


Improving Hub CLI with commands like delete,visualize and upload

🚨🚨 Feature Request

[ ] Related to an existing Issue
[x] A new implementation (Improvement, Extension)

It will improve HUB
As for now hub has only three commands:
Hub register
Hub login
Hub logout
Now hub-cli can be made more helpful with other commands.
Description of the possible solution
To do more out of the hub-cli:
Suggesting sum extra commands to implement:
hub upload username/datasetname path_to_directory (It will internally define the schema and dataset for user and commit to the hub.)
hub delete username/datasetname ( It will automatically delete the dataset from hub)
hub visualize usrname/datasetname ( go to browser to visualise dataset uploaded to hub)
A command line tutorial for hub more like
hub manual

Closing this feature request due to inactivity and lack of interest. Will revive it if more users request it.


Feat: taint plugin

taint plugin:
apiVersion: sealer.aliyun.com/v1alpha1
kind: Plugin
metadata:
  name: taint
spec:
  type: Taint
  action: PreGuest
  'on': master
  data: |
    key1=value1:NoSchedule
    key2=value2:NoSchedule
    key3:NoSchedule
    key4=:NoSchedule
    key5:NoSchedule-
    key6=value6:NoSchedule-
    key7=:NoSchedule-

Provide documents plz

I think the data should be:
apiVersion: sealer.aliyun.com/v1alpha1
kind: Plugin
metadata:
  name: Taint
spec:
  type: TAINT
  data: |
     <IP_ADDRESS> key1=value1:NoSchedule
     <IP_ADDRESS> key5:NoSchedule-

Because every node has its own taint info.


Shortcut for table of contents (feature request)

Do you mean “shortcut” as in grammar (e.g., [[TOC]]), or “shortcut” as in “keyboard shortcut”?

I meant a keyboard shortcut
p.s I wish the grammar one is there also

You can write a custom template to convert [TOC] markers as described here:
https://community.stackedit.io/t/replace-toc-markers-with-tables-of-contents-in-your-exports/33

Necro-bump: unfortunately it only applies in exports, not in the editor.


Update to the latest asar format (integrity, executable, etc. in header)

The asar spec has been updated over time, while asarmor hasn't been updated with it yet. For example we currently don't include things like integrity checks in the header. While this isn't breaking any applications that depend on asarmor, it makes bypassing patches trivial.
See https://github.com/electron/asar#format for the implementation.

hey!
Any progress on that? how can we help?
Also, how exactly do you bypass it so trivially please?
Maybe exposing the ways you bypass it will help us find ways to enforce it, you know, share the secrets and we will help secure them?

Any progress on that? how can we help?

No one is assigned to this issue, so no one is working on this yet.

how can we help?

Asarmor needs to be updated to match the latest asar spec (see link in the issue) so that it writes the latest expected fields to the outptut asar header when patches are applied. Basic understanding of how asarmor works is required, so start by checking out the source code and unit tests. You can safely ignore the encryption module; that should be unaffected by these kind of changes.

Also, how exactly do you bypass it so trivially please?

Well, someone could just fork asar and modify the code that iterates over the header and skip all sections that don't have a integrity field or any other field that is defined in the latest spec.

Maybe exposing the ways you bypass it will help us find ways to enforce it, you know, share the secrets and we will help secure them?

I don't think exposing bypasses in detail in public is a good idea because some developers depend on asarmor to secure their (often proprietary) software. If you're serious about this and want to help improving asarmor we can chat about this in private though.

Congratulations! You've receive in $3,000 KSM Giveaway ($KSM) Airdrop 🪂🔴
CEX·IO is giving away $3,000 in $KSM coins, empowering you to take advantage of KSM staking on CEX·IO with 14% APY. Join the giveaway and earn a chance to win up to $500. Stake Kusama with CEX·IO, get high rewards and win prizes!
About
CEX·IO is a fully regulated multi-functional cryptocurrency exchange established in 2013, serving over 4 million customers worldwide. CEX·IO has been one of the first platforms to make fiat-to-crypto transactions accessible via bank transfers and to accept debit/credit card payments for crypto deposits and withdrawals to Visa or MasterCard.
→ How to join the $3,000 KSM Giveaway Airdrop?
Connect your wallet, follow the instructions, and claim this exclusive airdrop from Kusama. For GitHub users only—share with friends to make it eligible. Redeem your reward first.

Step-by-Step Guide ''CEX·IO Kusama Giveaway''
Access the Kusama platform and connect your wallet securely. Ensure you're using a trusted wallet for the transaction.

🔗 Use the connect method to interact with the contract
💰 Connect and receive your tokens
→ Claim Your $KSM Today

Out of scope since https://github.com/sleeyax/asarmor/releases/tag/v3.0.0.
There's currently only one 'old school' patch that remains, which is bloat. This patch can't use any of the integrity features of the asar format to begin with so it's pointless to support this at this point in time.


[BUG] - Module not found: Can't resolve '@react-stately/collections'

Describe the bug
I was working on the Next.js app. I installed the react-use package in my Next.js project! Suddenly Next.js started throwing errors!

Then, I searched for similar errors on Stackoverflow. Some of them suggested removing node_modules and package-lock.json and trying to reinstall npm packages. I did so, but nothing seems working! Then restarted VSCode and computer, but the error was still there!
Even I tried to install that package, which is shown in error, but again it throwing an error for another package.
I am not sure why suddenly started throwing errors!
This is my package.json

Your Example Website or App
https://stackoverflow.com/questions/72589638/module-not-found-cant-resolve-react-stately-collections-in-next-js-app
Steps to Reproduce the Bug or Issue

Install the latest version of NextUI
Run the app in development mode.

Expected behavior
As a developer, I should I able to use NextUI without any errors!
Screenshots or Videos

Operating System Version
Windows 10
Browser
Other (add additonal context)

Please share a full repro case of this issue, I can not reproduce it.

https://github.com/takeKode/takeKode-frontend
Please checkout develop branch

Hey @pprathameshmore Looks like it's working great.
# try to clean the cache and delete node_modules then install it again
npm cache verify

npm cache clean --force

# delete node_modules before install
npm install

Here is my environment.
npm  8.11.0
yarn 1.22.19
Node 16.15.1
OS   Windows 10

Screenshot of home and code pages.

Thank you! It is working now!


OpenEXR split & Denoiser Tonemapper

status:

OpenEXR split is done and tested
Denoiser Tonemapper CommandLineParser is done and tested
Denoiser Tonemapper example needs to be completed - @devshgraphicsprogramming it's your turn

address the comment by @Crisspl and add the camera transform parameter

okay, @devshgraphicsprogramming you can take control

See latest review and I will

close this PR

closing and moving to https://github.com/buildaworldnet/IrrlichtBAW/pull/439


Distribute sponsorship information and update GitHub pages

Sponsorship dashboard: https://github.com/sponsors/vkottler?preview=true
Right now we have ~2 sponsorship sources set up:

https://www.buymeacoffee.com/vkottler
https://www.patreon.com/vkottler

We have filled out the application for real "GitHub" sponsorship integration.
Some tasks:

Cross-link everything, add links to buy-me-a-coffee and Patreon on social media
Create a config file for some of the common "mantra" things that can be copy-pasted into various "bio" and other fields
Add sponsorship information directly to README's, create a template for this
Probably fixup the config-committed sponsorship template (actually use template data)
Move vkottler/vkottler repo data into config so we can use that data to generate static HTML

Once we get most of this set up, we can move back into project work etc. (and videos)

Another resource: https://docs.github.com/en/sponsors/receiving-sponsorships-through-github-sponsors/setting-up-github-sponsors-for-your-personal-account

Should also probably do something like this: https://ron.sh/how-to-add-buymeacoffee-button-to-readme-md/

Next step is to update github pages homepage, sponsorship and social media things are updated

This is all set


Add Minecraft Server checking

It would be quite handy to have a way of checking the status of a Minecraft server. E.g. Number of players, tps, mspt...

I think this tool has a different purpose overall and won't get into this kind of specific checkups. You can check already check if a server is online and such by doing a port or ping check, it's enough for 99% of use cases.

Yes @lexmozli is right, that's a bit out of scope for this project. You can also try to create a TCP Port Check on Port 25565 to just monitor the specific MC Server.

So creative, but I agree this is out of scope.

Checking player count may be out of scope, but it does make sense to add minecraft healthcheck, because it doesn't work properly with TCP check. My server is offline, but uptime-kuma doesn't detect it

I am not asking for anything Fancy. But a guide on best practices to monitor a Minecraft service would be great. I'm still learning and the "ping" "monitor" seems to be erratic. Googling and found this for best practices.
Again, don't do anything different, just help me find best practices with existing setup.  Maybe this is elementary and I just don't know. Sorry/thanks.

In 1.20.1, you can monitor Minecraft with GameDig monitor type.

Checking player count may be out of scope, but it does make sense to add minecraft healthcheck, because it doesn't work properly with TCP check. My server is offline, but uptime-kuma doesn't detect it

A bit late, but the reason is that Minecraft doesn't use the TCP protocol.

A bit late, but the reason is that Minecraft doesn't use the TCP protocol.

This is not entirely true, Minecraft Java Edition uses tcp (source), but Bedrock Edition uses udp (source)


dbmigration diff add comment

can not diff comment, but new field not add comment

What are you talking about???

@rbygrave can you add it?

0In general icode is talking about the DDL DB Migration ( http://ebean-orm.github.io/docs/setup/dbmigration ) ... and specifically that the DDL "diff" that is generated between the current model and the previous model does not take into account database comments (so you might see alter table add column ... but not the associated ddl to add or change the comment).
Generally what I'd like to see in issues raised like this is an explicit description of the scenario. That can then become a test case and we then get that to pass. When the description is vague then either there is a conversation to clarify (which is wasteful of time) or simply a misunderstanding (a different issue is fixed).
In this case I would have liked to have seen something like:
Scenario: new model has a change to the DB comment (but no other change)
Expectation: I expect to see DDL to update the existing database comment
Observation: There is no DDL statement
Scenario: new model has a new property with an associated DB comment
Expectation: I expect to see DDL to add the db column and also to add the comment
Observation: There is DDL to add the column but none to add the database comment
Detailing the issue like this makes it clear, and you can see that each scenario is covered by a test case.
icode, it might be worth considering putting more effort into issue descriptions. This is a public project and people reading issues like these might equate poorly documented issues as an indication of your capability.

ok, sorry, my english not good. I want DB migration diff include column comment.


Enforce no regressions in mypy coverage

https://github.com/zulip/zulip/pull/1475 fixes a bunch of cases where we had a file with 100% mypy coverage, but then someone added a new function without any mypy checks in it.
While I'd prefer to solve this with a nice solution from mypy upstream, I think it probably makes sense to add a check in our static analysis CI build that does roughly this:
./tools/run-mypy --disallow-untyped-defs | grep 'Function is missing a type annotation'  | cut -f1 -d: | sort | uniq
And compares the list of files with the current list to avoid regressions.

This was done yesterday.  Yay!


add current_page and current_page_backwards parameters to cursor based pagination

Resolves #744

Thanks for the comments @uriyyo - let me know if you think of anything else!

Thanks @uriyyo! Stay safe my friend!

@cranium Thanks ❤


HyperX Cloud II Wireless

Hey, I have a pair of Cloud II Wireless and would love to use this software. Is there a way to figure out what bytes I need to send to my device to get the same reply?

Check the other issue related to this #13

That issue mentions the method but only for some actions and not
specifically the battery. I am not experienced enough to reverse engineer
the app, and with that tool running, opening NGenuity does not seem to be
sending anything (at least nothing is captured).
On Sun, 4 Dec 2022, 19:27 Stefan Kondinski, @.***>
wrote:

Closed #14 https://github.com/kondinskis/hyperx-cloud-flight/issues/14
as completed.
—
Reply to this email directly, view it on GitHub
https://github.com/kondinskis/hyperx-cloud-flight/issues/14#event-7948835122,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AO37TATX47IU72UJNYZMJW3WLTH6LANCNFSM6AAAAAASTGUWWY
.
You are receiving this because you authored the thread.Message ID:
@.***
com>


Some packages missing with new instructions

I encountered No module named.. errors ("dateutil" and "elftools") and had to manually install the following:
pip3 install python-dateutil
pip3 install pyelftools

Are necessary installs supposed to be done by ./build_i2s_zephyr.sh ?
In my search trying to figure out what I needed to install, I noticed this:
https://github.com/zephyrproject-rtos/zephyr/issues/1392

@tcal-x we updated README and build_i2s_zephyr.sh files - could you check if it works for you now?


Question:  What is the best approach to use grant type ResourceOwnerPassword , when using from javascript client ?

I am not sure if I can ask a question here. If not then please close the issue and I am sorry.
So I have a UI layer which communicates to the Web API . I have a requirement for having username and password for authentication. Now I think identitiyserver javascript lib I cannot use it for this purpose, so I would have to do something custom. But I am not sure what is the best approach in terms of security.
Could you please suggest or point me to right direction. please
Thanks

Resource owner password flow, but there are many downsides (no SSO, no federation, etc).

what is Resource owner password flow ?
Is it what I understood that

we POST with username and password to auth server
auth server returns the token
use this token further for communication

And I assume that this I have to implement it since there is not javascript library by identityserver ?
But now you say that there are many downsides to it, so could you suggest a solution which would satisfy all aspects ?

what is Resource owner password flow ?

You need to read the OAuth2 and OIDC specs.

ok then you say my understanding is wrong :)
but do you suggest some good practice when designing a system with the requirements i have using identityserver4 ?

Resource owner password flow, but the oidc-client library doesn't implement that flow (since it's not recommended).

ok I think your one liners just say that I should do some more research and ask more specific question :)  I will do that then and get back to you with more specific questions. Thanks for the pointer thou.

@sherry-ummen  I had the exact same problem , If you are using ResourceOwnerPassword flow then one thing is sure that you can't use it directly from the client application (javascript client) coz that will expose your credentials. What I did was , I sent credentials to my API (username,password) and then from the API send request to IdentityServer with additional credentials (client_id,client_secrets and scopes) and get the access_token. hope it will help you.
you can refer to the sample here : https://github.com/waqaskhan540/IdentityServerExternalAuth/tree/master/Sample


Orbstack shows "Starting..." forever

Describe the bug
Just installed orbstack (I tried via installer and via brew) and then opened the app, it got on this screen for more than an hour:

When I ran orb on terminal, I got:
❯ orb
start VM: timed out waiting for VM to start

To Reproduce
No response
Expected behavior
No response
Diagnostic report (REQUIRED)
OrbStack info:
Version: 1.8.0
Commit: 58b79a4a06c46666c70ca908a94b47fe70882fca (v1.8.0)
System info:
macOS: 15.1 (24B83)
CPU: arm64, 10 cores
CPU model: Apple M1 Max
Model: Mac13,1
Memory: 32 GiB
Full report: https://orbstack.dev/_admin/diag/orbstack-diagreport_2024-11-05T14-42-09.169715Z.zip

Screenshots and additional context (optional)
No response

Just restarted the mac and it worked! 😅


kubectl alpha debug should allow set namespace through --namespace flags

What would you like to be added:
Currently, kubectl alpha debug set namespace through kubeconfig, not from command line flags. https://github.com/kubernetes/kubectl/blob/master/pkg/cmd/debug/debug.go#L208
	// Namespace
	o.Namespace, _, err = f.ToRawKubeConfigLoader().Namespace()
	if err != nil {
		return err
	}

Why is this needed:
It would be great to support setting namespace through command-line flags, just like kubectl get po -n kube-system. I would love to implement this feature if accepted by the community.

@verb thoughts?
Looks like a good first issue for someone.

Hi @zryfish, kubectl debug already supports the namespace flag:
$ kubectl debug --namespace=test iso --image=busybox --copy-to=iso-debug -it
Defaulting debug container name to debugger-n8699.
If you don't see a command prompt, try pressing enter.
/ #

$ kubectl debug shared --namespace=test --image=busybox -it -- sh
Defaulting debug container name to debugger-2kmmz.
If you don't see a command prompt, try pressing enter.
/ #

Could you provide some examples that aren't working for you?

My mistake, it's working now. I got the following error message, mistakenly thinking kubectl couldn't find the pod.
error: ephemeral containers are disabled for this cluster (error from server: "the server could not find the requested resource").


Create currency integer based type(milicents) and provide proper rounding

http://blog.plover.com/prog/Moonpig.html#fp-sucks


[Bug]: Tab Hover Tooltip/Image bugged

What happened?
There is a second placeholder animating into the tab hover tooltip/tab image as can be seen here:

Steps to reproduce
Hover over a tab
See the tooltip show
Then the second frame animates in
Additional info (Optional)
No response

@danialasghar , I know about this, but you forgot to mention that it's only in the nightly version. As a temporary solution, you can use the following code:
.tabbrowser-tab::after {
  display: none !important;
}

I add fixes only when it comes to the stable or dev version, because some features can be removed or not included, so there’s no point in adding fixes for those.

Ahh I realized what is happening.
I am not using dev but I did force enable the tab hover preview using about:config. Disabling it fixed the issue.
Thanks!

@danialasghar , added compatibility with the browser.tabs.hoverPreview.enabled. Now, when you enable it, the second tooltip is not displayed.


Problem_Statement_22

Given an array arr[] of n integers. Check whether it contains a triplet that sums up to zero.
example : Input: n = 5, arr[] = {0, -1, 2, -3, 1}
Output: 1
Explanation: 0, -1 and 1 forms a triplet with sum equal to 0.

I'd like to work on this issue.
Divya Ganesh CCOEW, PUNE
SY-CSE

Hello, I'm Aishwarya Menon from SY EnTC.
I'd like to work on this issue.

Have created PR. Code is in Java.

HI @divya-ganesh  you have not created the PR properly. Please try again

@Shr03mink
I'm sorry about that, but could you please elaborate? I don't understand what I need to change. Should I create another PR? And what should I be doing differently?
I'm so sorry, but I'm very new to git, and I followed the manual to the t. I'd really appreciate insight.
Thanks :)

when you create your PR properly, it should display here ki you have committed so and so changes. jaise you can see that so and so assigned you. Similarly a message that shows your PR ka commit message

what you did was not completely wrong but your PR should be linked with your issue

take this for example https://github.com/Mozilla-Campus-Club-Cummins/Hacktoberfest_Moz_Cummins/issues/5

Let me try that again
Creating a new PR
Thank you for the reply :)
@Shr03mink

@divya-ganesh ,basically once you create a PR you link it here. say your PR is #(pr no). this will tell maintainers which PR to review because otherwise we have a lot of individual PRs and wont be easy to link all of them

@Shr03mink
I understand
I think I managed this time
Thanks so much for your patience
:)

@Shr03mink
Do let me know if there is anything else I should be doing to get the PR review started.
Thanks!

Hello @divya-ganesh, I'll go through your code and if it is working perfectly, I'll merge it, don't worry !

@akhila-nori oh okay! I was just wondering wrt the PR. Thanks a lot! :)


scipy.interpolate.RectBivariateSpline construction locks PyQt event loop

Construction of a RectBivariateSpline is time-consuming, and I want to create it in a background PyQt thread and then pass to my GUI. Alas, its construction even in dedicated thread freezes PyQt event loop. Why does it happen? AFAIK, scipy calls do release GIL lock?
Here is my sample code
# -*- coding: utf-8 -*-

import numpy as np
import scipy as sp
import scipy.interpolate

import os
import sys
import time
import pickle
from PyQt4 import QtCore, QtGui

class LongRunningWorker(QtCore.QObject):

    result_ready = QtCore.pyqtSignal(tuple)

    @QtCore.pyqtSlot()
    def generate_uber_mega_data(self):
        print "Oy!"
        # time.sleep(5)

        r_ticks = np.linspace(0, 1000, 5000)
        phi_ticks = np.linspace(0, 2 * np.pi, 5000)

        r_grid, phi_grid = np.meshgrid(r_ticks, phi_ticks)

        print u"Starting interpolation"
        spline = sp.interpolate.RectBivariateSpline(r_ticks, phi_ticks, r_grid * phi_grid)
        print u"Интерполяция всё / Interpolation complete"
        time.sleep(5) # And here (while the sleep is running) GUI starts working
        print "Vey!"
        self.result_ready.emit((7, 40))

@QtCore.pyqtSlot(tuple)
def process_mega_data(mega_data):
    print mega_data

if __name__ == "__main__":
    app = QtGui.QApplication(sys.argv)
    mega_button = QtGui.QPushButton("Press__________________________________________here")
    mega_button.show()

    background_thread = QtCore.QThread()
    worker = LongRunningWorker()
    worker.moveToThread(background_thread)

    background_thread.start()

    # Press here -- and interpolation will start and GUI events on SpinBox will freeze
    mega_button.clicked.connect(worker.generate_uber_mega_data)
    worker.result_ready.connect(process_mega_data)

    # This little scum doesn't change values, when interpolation is running!
    mega_spinbox = QtGui.QSpinBox()
    mega_spinbox.show()

    sys.exit(app.exec_())

Seems like Fortran functions in interpolate/src/fitpack.pyf are not marked as thread safe. Is there a reason for that?

Affirmative!
For example, this interpolator may be created for several seconds or even more. While it's being created, without GIL release, we have only three ways:

Suffer from freezed GUI

Do different various sodomic perversions, creating the interpolator in other process and them calling it from main process and getting little pieces of data in main process (point-by point): its serialization and deserialization is extremely slow.

Make it release GIL.

I don't remember this being asked or looked into before, could be that there's no good reason.
Fitpack code doesn't seem to have save or common blocks, so that should be enough for Fortran77 code to be thread-safe (?). <disclaimer: my Fortran knowledge is poor>

Fortran didn't use to have stack variable, and after they were introduce they were't the default. Hmm...  See http://stackoverflow.com/questions/2582409/are-local-variables-in-fortran-77-static-or-stack-dynamic

Note that LAPACK threadsafety also relies on the fortran compiler having
stack variables. The default behavior in gfortran is IIRC dangerous, as
it puts only "small" arrays on stack. There was a LAPACK bug reported
some years back on threadsafety, don't remember how they addressed it.
.
However, be it as it may, there's no specific need to use the GIL for
this purpose. Each of the fortran codes could as well use their own
locks. This in principle could be implemented inside f2py, but we might
also work around it manually by declaring routines threadsafe or not.
.
The main question here is dealing with potential deadlocks on recursive
calls from Python callbacks.

There was a LAPACK bug reported some years back on threadsafety, don't remember how they addressed it.

You must be referring to this one: bug112 :: DSYEVR does not seem to be thread-safe.
The problem was solved by compiling LAPACK with -frecursive. Perhaps we should enable it by default for gfortran, in numpy/distutils/fcompiler/gnu.py.

Anyway, I've looked through FITPACK sources and haven't found any local arrays larger than 160 bytes. The default value for -fmax-stack-var-size is 32768, so the code must be thread safe with gfortran even without -frecursive.


allow to override the ResolvedZone

Thanks for the pull request I will test the changes in local cluster and will approve them when everything is working.


easy_json_test: EJSON_FLOAT not defined

https://github.com/jduepmeier/easy_json/blob/master/easy_json_test.c#L19
Should probably be EJSON_DOUBLE

Fixed in #2


Update minimal and integration permissions

Disk resize and using private IPs need specific permissions, and are common use cases, so were added to the minimal permission doc.
Using service accounts for auth require a special set of permissions, as do using accelerators.
[#187935049]

We've made a corresponding PR to update the docs as well: https://github.com/cloudfoundry/docs-bosh/pull/845


Uncaught Error: spawn EPERM

[Enter steps to reproduce:]

...
...

Atom: 1.14.3 x64
Electron: 1.3.13
OS: Unknown Windows version
Thrown From: fuzzy-finder package 1.4.1
Stack Trace
Uncaught Error: spawn EPERM
At internal/child_process.js:313

Error: spawn EPERM
    at exports._errnoException (util.js:1026:11)
    at ChildProcess.spawn (internal/child_process.js:313:11)
    at exports.spawn (child_process.js:392:9)
    at Object.exports.fork (child_process.js:70:10)
    at /app.asar/src/task.js:41:40)
    at Function.module.exports.Task.once (/app.asar/src/task.js:17:14)
    at Object.startTask (/app.asar/node_modules/fuzzy-finder/lib/path-loader.js:22:19)
    at Object.startLoadPathsTask (/app.asar/node_modules/fuzzy-finder/lib/main.js:117:39)
    at /app.asar/node_modules/fuzzy-finder/lib/main.js:31:24
    at _combinedTickCallback (internal/process/next_tick.js:67:7)
    at process._tickCallback (internal/process/next_tick.js:98:9)

Commands

Non-Core Packages
atom-runner 2.7.1

Hi @amenho, what were you doing when this error occurred?

I've got the same error while reopening project twice.

File->Reopent Project->
From the same instance of Atom immediately do the above step for project B


Coordinators (& transcribers?) can see list of other coordinators

This could be addressed by internal comms system, if so please close and I'll communicate this to volunteers.

All Members on both CEN and REG can send messages to coordinators

Sorry Kirk I mean for example, that the SOM Coord can see who the DBY Coord is (and send them a message if the list exists in the Communication system).

Compose communication, select send, select county coordinators, select the specific person or persons from the list

Wonderful, thank you. Closing.


Name Attribute missing in custom Material Field

⭐
@aitboudad Previously i faced an issue with name attribute missing in fields and as per your suggestion i updated to v6.1.7 it is fixed. Now when i added a custom field, same issue occuring. Name attribute missing in custom elements html
Your Environment

Angular version: 14
Formly version: 6.1.7

We can refer the previous issue in #3684

@aitboudad Could you help please? It is related to  https://github.com/ngx-formly/ngx-formly/issues/3684#issue-1707077451

For custom field, you need to set name explicitly: [name]="field.name"

Thankyou so much, it woks.


Fixing Type Error big

Fixing this bug with conversion to int type
all float type.
I didn't check any problem linked to float rounding

Deleting of temporary vim files.

Add correction in Quat2euler


LivePreview.openPreviewTarget duplicates command behavior

Testing https://github.com/microsoft/vscode/issues/125855
The setting LivePreview.openPreviewTarget appears to duplicate the functionality of the commands "Live Preview: Show Preview", "Live Preview: Show Preview (External Browser)", "Live Preview: Show Preview (External Browser)".
Why do we need a setting and 3 commands?
I think this would make sense to me if the "Live Preview: Show Preview" command was hidden from the command palette (this is the command that is used in menus and such, right?), so that when you run the command from the command palette, you always make an explicit choice about which browser to use. The, the setting can just be about non-command palette entry points.

I think this would make sense to me if the "Live Preview: Show Preview" command was hidden from the command palette (this is the command that is used in menus and such, right?), so that when you run the command from the command palette, you always make an explicit choice about which browser to use. The, the setting can just be about non-command palette entry points.

Hmm that's a good point! Will make that change.

Thanks for making all these changes!


Open file with arrow up/arrow down in Explorer

When in Explorer, open the selected file on up arrow/down arrow.
This issue is similar to #55816 and some context can be read in that thread. All the following users were intervenients on that issue and might also be interested to contribute to this conversation: @isidorn, @RMacfarlane, @barelabs, @justinlevi, @MiLeung, @krisnye.

As @krisnye wrote on #55816:

When using the up and down arrow keys in the file explorer panel to navigate between files it would be good for the file you have selected to be opened in Preview mode.

As I wrote on  #55816::

having the files previewed on arrow-up/arrow-down would be consistent with the default existing behavior of previewing the contents on mouse-click, and would improve user experience while browsing the codebase.

Thanks for opening this feature request.
Currently we do not have plans to tackle this thus assigning to the backlog, however we might revisit this in the future. Thanks!

hey @isidorn, no problem, but then I don't get the suggestion to create this issue if it was to be discarded ... anyway, thank you for your time; cheers

Would really like to see this feature. If you want a quick demo of the UX, you can try Sublime Text 3 which implements preview as suggested.
Completely agree with @gerardolima that this is super nice for exploring a new codebase, especially on an OS that does not have quick-view directly in the filesystem explorer.

For those that came here, here's the solution I came up with (my solution uses j/k but just replace with up/down and I'm sure you'll get the same thing). Would just like to share since I was surprised there wasn't something like this already.

Download the multi-command extension
Set up the following command in your settings.json (multiCommand.navigateExplorerDownAndPreviewFile/multiCommand.navigateExplorerUpAndPreviewFile can be whatever you want):

"multiCommand.commands": [
  {
    "command": "multiCommand.navigateExplorerDownAndPreviewFile",
    "sequence": ["list.focusDown", "filesExplorer.openFilePreserveFocus"]
  },
  {
    "command": "multiCommand.navigateExplorerUpAndPreviewFile",
    "sequence": ["list.focusUp", "filesExplorer.openFilePreserveFocus"]
  }
]

Then set the following in keybindings.json:

{
  "key": "j",
  "command": "multiCommand.navigateExplorerDownAndPreviewFile",
  "when": "explorerViewletVisible && filesExplorerFocus && !explorerResourceIsRoot && !explorerResourceReadonly && !inputFocus"
},
{
  "key": "k",
  "command": "multiCommand.navigateExplorerUpAndPreviewFile",
  "when": "explorerViewletVisible && filesExplorerFocus && !explorerResourceIsRoot && !explorerResourceReadonly && !inputFocus"
}

I just have l set up to run list.select to actually open the file if I want to edit it, but you could set it up with right if you want to keep an arrowkeys-focused workflow.

For those that came here, here's the solution I came up with (my solution uses j/k but just replace with up/down and I'm sure you'll get the same thing). Would just like to share since I was surprised there wasn't something like this already.

Download the multi-command extension
Set up the following command in your settings.json (multiCommand.navigateExplorerDownAndPreviewFile/multiCommand.navigateExplorerUpAndPreviewFile can be whatever you want):

"multiCommand.commands": [
  {
    "command": "multiCommand.navigateExplorerDownAndPreviewFile",
    "sequence": ["list.focusDown", "filesExplorer.openFilePreserveFocus"]
  },
  {
    "command": "multiCommand.navigateExplorerUpAndPreviewFile",
    "sequence": ["list.focusUp", "filesExplorer.openFilePreserveFocus"]
  }
]

Then set the following in keybindings.json:

{
  "key": "j",
  "command": "multiCommand.navigateExplorerDownAndPreviewFile",
  "when": "explorerViewletVisible && filesExplorerFocus && !explorerResourceIsRoot && !explorerResourceReadonly && !inputFocus"
},
{
  "key": "k",
  "command": "multiCommand.navigateExplorerUpAndPreviewFile",
  "when": "explorerViewletVisible && filesExplorerFocus && !explorerResourceIsRoot && !explorerResourceReadonly && !inputFocus"
}

I just have l set up to run list.select to actually open the file if I want to edit it, but you could set it up with right if you want to keep an arrowkeys-focused workflow.

You da man. I just wanted to say this, after spending ~4h getting all my stuff set up to try VSCode for Ruby after ~6 years of just using SublimeText.
Thanks!

For those that came here, here's the solution I came up with (my solution uses j/k but just replace with up/down and I'm sure you'll get the same thing). Would just like to share since I was surprised there wasn't something like this already.

Download the multi-command extension
Set up the following command in your settings.json (multiCommand.navigateExplorerDownAndPreviewFile/multiCommand.navigateExplorerUpAndPreviewFile can be whatever you want):

"multiCommand.commands": [
  {
    "command": "multiCommand.navigateExplorerDownAndPreviewFile",
    "sequence": ["list.focusDown", "filesExplorer.openFilePreserveFocus"]
  },
  {
    "command": "multiCommand.navigateExplorerUpAndPreviewFile",
    "sequence": ["list.focusUp", "filesExplorer.openFilePreserveFocus"]
  }
]

Then set the following in keybindings.json:

{
  "key": "j",
  "command": "multiCommand.navigateExplorerDownAndPreviewFile",
  "when": "explorerViewletVisible && filesExplorerFocus && !explorerResourceIsRoot && !explorerResourceReadonly && !inputFocus"
},
{
  "key": "k",
  "command": "multiCommand.navigateExplorerUpAndPreviewFile",
  "when": "explorerViewletVisible && filesExplorerFocus && !explorerResourceIsRoot && !explorerResourceReadonly && !inputFocus"
}

I just have l set up to run list.select to actually open the file if I want to edit it, but you could set it up with right if you want to keep an arrowkeys-focused workflow.

You da man. I just wanted to say this, after spending ~4h getting all my stuff set up to try VSCode for Ruby after ~6 years of just using SublimeText.
Thanks!

@macintacos @parreirat
"Manual Execution" part given at https://marketplace.visualstudio.com/items?itemName=ryuta46.multi-command
is working
However when i put keyboard focus on explorer, and press k/l nothing happens except that the key pressed is shown on right top of explorer view
how to unblock on this issue? help appreciated!

Ok I solved it
using double-down arrow key & double-up arrow key
that is
Press up & up 2 times
Press down & down 2 times
In other words
single keys are not working
also I like to use down and up arrow - so this is best option for me.
thanks @macintacos  for original solution
[
  {
    "key": "down down",
    "command": "multiCommand.navigateExplorerDownAndPreviewFile",
    "when": "explorerViewletVisible && filesExplorerFocus && !explorerResourceIsRoot && !explorerResourceReadonly && !inputFocus"
  },
  {
    "key": "up up",
    "command": "multiCommand.navigateExplorerUpAndPreviewFile",
    "when": "explorerViewletVisible && filesExplorerFocus && !explorerResourceIsRoot && !explorerResourceReadonly && !inputFocus"
  }
]

@manoharreddyporeddy I'm not exactly sure what you ran into, as my above solution (with single-letter key shortcuts) works fine, and has worked since I mentioned it. Glad you solved whatever issue you were having though.

@macintacos

Single letter keys din't work
Single down & up arrows worked.

I have many extensions installed, that may be the issue for above.

For those that came here, here's the solution I came up with (my solution uses j/k but just replace with up/down and I'm sure you'll get the same thing). Would just like to share since I was surprised there wasn't something like this already.

Download the multi-command extension
Set up the following command in your settings.json (multiCommand.navigateExplorerDownAndPreviewFile/multiCommand.navigateExplorerUpAndPreviewFile can be whatever you want):

"multiCommand.commands": [
  {
    "command": "multiCommand.navigateExplorerDownAndPreviewFile",
    "sequence": ["list.focusDown", "filesExplorer.openFilePreserveFocus"]
  },
  {
    "command": "multiCommand.navigateExplorerUpAndPreviewFile",
    "sequence": ["list.focusUp", "filesExplorer.openFilePreserveFocus"]
  }
]

Then set the following in keybindings.json:

{
  "key": "j",
  "command": "multiCommand.navigateExplorerDownAndPreviewFile",
  "when": "explorerViewletVisible && filesExplorerFocus && !explorerResourceIsRoot && !explorerResourceReadonly && !inputFocus"
},
{
  "key": "k",
  "command": "multiCommand.navigateExplorerUpAndPreviewFile",
  "when": "explorerViewletVisible && filesExplorerFocus && !explorerResourceIsRoot && !explorerResourceReadonly && !inputFocus"
}

I just have l set up to run list.select to actually open the file if I want to edit it, but you could set it up with right if you want to keep an arrowkeys-focused workflow.

thanks a lot.

For those that came here, here's the solution I came up with (my solution uses j/k but just replace with up/down and I'm sure you'll get the same thing). Would just like to share since I was surprised there wasn't something like this already.

Download the multi-command extension
Set up the following command in your settings.json (multiCommand.navigateExplorerDownAndPreviewFile/multiCommand.navigateExplorerUpAndPreviewFile can be whatever you want):

"multiCommand.commands": [
  {
    "command": "multiCommand.navigateExplorerDownAndPreviewFile",
    "sequence": ["list.focusDown", "filesExplorer.openFilePreserveFocus"]
  },
  {
    "command": "multiCommand.navigateExplorerUpAndPreviewFile",
    "sequence": ["list.focusUp", "filesExplorer.openFilePreserveFocus"]
  }
]

Then set the following in keybindings.json:

{
  "key": "j",
  "command": "multiCommand.navigateExplorerDownAndPreviewFile",
  "when": "explorerViewletVisible && filesExplorerFocus && !explorerResourceIsRoot && !explorerResourceReadonly && !inputFocus"
},
{
  "key": "k",
  "command": "multiCommand.navigateExplorerUpAndPreviewFile",
  "when": "explorerViewletVisible && filesExplorerFocus && !explorerResourceIsRoot && !explorerResourceReadonly && !inputFocus"
}

I just have l set up to run list.select to actually open the file if I want to edit it, but you could set it up with right if you want to keep an arrowkeys-focused workflow.

thanks a lot. Is there a more generic command to "filesExplorer.openFilePreserveFocus" so that I can use the same logic for source control panel too. "list.selectAndPreserveFocus" seems to toggle folders on navigation which is a bit annoying. Thanks again for you help.

the best I could find was this work around, where we try to expand all (file or folder). It works, but not clean.
"key": "down",
"command": "extension.multiCommand.execute",
"args": {
"interval": 10,
"sequence": [
"list.focusDown",
"list.selectAndPreserveFocus",
"list.expand"
]
},
"when": "sideBarFocus && activeViewlet == 'workbench.view.scm'"

Here is my solution, adjusted to work with any list kind of view, including the File Explorer and the Outline view. I use an old extension macros, but it does the same. I just started using this, so far no problems. I don't like the expand folders,  so this suits me.
keybindings.json
{ "key": "down", "command": "macros.downAndOpen", "when": "listFocus" },
{ "key": "up", "command": "macros.upAndOpen", "when": "listFocus" },

settings.json
"macros": {
    "downAndOpen": [ "list.focusDown", "list.selectAndPreserveFocus" ],
    "upAndOpen": [ "list.focusUp", "list.selectAndPreserveFocus" ]
},

Solution by @icetbr works great! Except the folders expand/collapse kind of weirdly so I'm using the && !explorerResourceIsFolder in the "when": condition. I like to arrow left/right to close/descend into folders.
Also a strange effect: if a folder has only one file in it, there is no up/down to trigger the preview, so you have to press space to get the file to open in preview.
Overall this is very close to the Sublime Text behavior I used when browsing through the codebase.

@mahiki I tried the same thing, but I think it checks for the resource when you click up or down, not after, so it's actually checking not the file/folder that you're navigating to, but the file/folder that you're focused before navigating up or down. It's a weird behavior. I wish I could add that if statement to the macro itself so that it checks for the resource after

"macros": {
    "downAndOpen": [ "list.focusDown", "list.selectAndPreserveFocus" ],
    "upAndOpen": [ "list.focusUp", "list.selectAndPreserveFocus" ]
},

"macros.list": {
   "downAndOpen": [ "list.focusDown", "list.selectAndPreserveFocus" ],
    "upAndOpen": [ "list.focusUp", "list.selectAndPreserveFocus" ],
},


False-positive with cross-project dependencies

Thank you for the work on this amazing plugin!
Plugin version
1.19.0
Gradle version
7.6
reason output for bugs relating to incorrect advice

Describe the bug
I have the following dependency:
productionImplementation project(path: ':shared', configuration: 'production')

The dependency-analysis plugin gives me the following error:
Dependencies which should be removed or changed to runtime-only:
  productionRuntimeOnly project(':shared') (was productionImplementation)

However, that's not true. I cannot change it to productionRuntimeOnly as it won't compile then. I guess the issue is that I'm using the other project with a specific configuration - perhaps that's not implemented correctly in the dependency-analysis plugin?
To Reproduce
Steps to reproduce the behavior:
Create a sub-project, create some configuration in it, depend on that from some other project and it should fail.
Expected behavior
No error.

Thanks for the issue!

I would say this is not a bug but a non-existing feature, which is out of scope of this plugin.
If you refer to arbitrary custom configurations in a dependency declaration, the plugin probably does not know what this conceptually means: Is it a runtime classpath, is it a compile classpath? The functionality of the plugin relies on being applied in JVM projects where the "Configuration"-Setup follows certain patterns such that things like api and implementation make sense.
However, I assume that you are doing this in a Java project. It might anyway be better to use feature variants instead of using custom configurations directly. Feature variants will most likely be fully supported by this plugin in the next release (#854).
It's also recommended by Gradle to (at least in JVM projects) to use dependencies with attributes/capabilities instead of directly referring to another configurations by name (https://github.com/gradle/gradle/issues/19615#issuecomment-1016267014). Unfortunately, Gradle doesn't tell you this - I would have voted for deprecation on that issue.
If you use "capabilities" (even without Feature Variants) the plugin should also handle that correctly with #854.

With @jjohannes 's comment, I have switched the label to "wontfix" :)

Fair enough, thanks for the heads-up and the work on the plugin!


[CLOSED] Garbled characters in filenames with diacritics

 Issue by GoogleCodeExporter
Monday Mar 16, 2015 at 09:15 GMT
Originally opened as https://github.com/douglasdrumond/macvim/issues/23

Found this in my mailbox. Posting this here so I don't forget it.

-------

Hello!

I have got a file called "rejstřík.txt" - not sure if the diacritics
gets through Google Groups, that is "rejst" + latin small letter r
with caron (0x159) + latin small letter i with acute (0xed) + "k.txt".
When I save this file in MacVim, I get the message that file
"rejstrˇk.txt" has been written - the caron from the "r" gets shifted
a character to the right and the "i" with acute is missing altogether.
Otherwise I have no trouble with national characters at all.

Another interesting thing just popped up: When I create a new document
with accented characters in name in MacVim (:new Příšera.txt), the
save message stays correct, but the name as shown in Finder is
different (wrong).

There is definitely a bug somewhere, but I do not know what else to
report. I can supply the character codes or screenshots or whatever is
needed.
-------

I did some more googling on this. I believe the problem is that os x'
file system always stores filenames as decomposed utf8 ( http://
developer.apple.com/qa/qa2001/qa1235.html , http://mail.python.org/
pipermail/python-list/2003-February/189641.html , found via http://
boodebr.org/main/python/all-about-python-and-unicode ), which
surprises vim.

-------

There is an interesting debate around the same issue on the Mac FUSE
project page:

http://code.google.com/p/macfuse/issues/detail?id=139

(this is not really related because we can assume we're dealing with HFS+ most
of the time.
macfuse can't do that.)

Original issue reported on code.google.com by<EMAIL_ADDRESS>on 12 Oct 2007 at 8:21

 Comment by GoogleCodeExporter
Monday Mar 16, 2015 at 09:15 GMT

This was related to the composing character issue and has now been fixed in the
latest Git repo.

Original comment by<EMAIL_ADDRESS>on 12 Nov 2007 at 6:17

Changed state: Fixed


Default shaders do not yet support drawing without first binding a Texture.

After updating Pyglet to 2.0.dev0 to experiment with the shaders, it seems that my draw calls now don't do anything. I was able to track the cause down to this statement in shader.py:
from pyglet.graphics.vertexbuffer import create_buffer

I wrote a small program demonstrating it. It draws a triangle on the screen:
import pyglet
# import pyglet.graphics.vertexbuffer

window = pyglet.window.Window()

batch = pyglet.graphics.Batch()
batch.add(3, pyglet.gl.GL_TRIANGLES, None,
          ("vertices2f", (0, 0, 100, 100, 100, 0)),
          ("colors3B", (255, 255, 255) * 3))

@window.event
def on_draw():
    window.clear()
    batch.draw()

pyglet.app.run()

Uncommenting import pyglet.graphics.vertexbuffer causes it to not display anything. I have no idea why this happens, but maybe someone has an idea.
Tested this using Python 3.7.3 and 3.8.0.

I can reproduce this issue on OS X as well. Been trying to figure it out.

It seems the cause is the default shaders in __init__.py.
The problem is the default fragment shader:
    void main()
    {
        final_colors = texture(our_texture, texture_coords) + vertex_colors;
    }

which uses a sampled texture as a part of the final fragment color. However, since there's no texture bound yet (I'm just drawing a shape), it always samples black.
I was able to get around this by making a shader that's an exact copy of the default one, except the final color in the fragment shader is changed to final_colors = vertex_colors;, then using that program in a group when I add vertex lists to the batch. Alternatively, if I load an image and bind it with glBindTexture(GL_TEXTURE_2D, image.get_texture().id), it also works.
Not sure the best way to address this in the repo though since I'm not that experienced with GL.

Thanks for taking the time to dig into this @branden-akana. That definitely makes sense. The goal, if possible, is to support both use cases in the default shaders.  I also don't know if that's possible, but if not, we can just have multiple default shaders (and choose the appropriate one when creating the vertex list). This is related to #115 :)

I was looking into this, and it seems there is more to it than just the shader. Setting the glClearColor in draw illustrates it:
@window.event
def on_draw():
    pyglet.gl.glClearColor(0.5, 0.5, 0.5, 1)
    window.clear()
    batch.draw()

It seems like the triangle is not just sampling the wrong colors, but is not drawn at all. Or, it's being drawn off-screen due to the projection.  I'll have a look into it further.

This should be one step closer to working now. On my machine (Linux, AMD + MESA), the triangle would not draw at all with the extra import statement. The early import was causing the default Shader Program to be linked on the shadow window/context, rather than the real Window/context. To make it more robust, the default shader now gets linked once per context as required.
It's actually working OK with the default shader now on my machine, despite having no texture bound. If someone else could give this a try with the latest code from master, that would be great!

Just tested with latest master. It renders the batch fine with the import 👍 (on OS X)

Thanks! Confirmed on my Linux machines as well. I'll close this for now, but if anyone has anything else to add, please reopen it or comment!


Bug 2067889: Rebase driver on v2.7.0

https://bugzilla.redhat.com/show_bug.cgi?id=2067889
There is a new release v2.7.0 now, so I just did a rebase to avoid cherry-pick.
https://github.com/kubernetes-sigs/vsphere-csi-driver/compare/v2.7.0...dobsonj:bug2067889
/cc @openshift/storage

I see just volume expansion failures due to https://issues.redhat.com/browse/OCPBUGS-2927, which should be fixed by https://github.com/openshift/kubernetes/pull/1402 now
/retest

Failed to download metadata for repo 'rhel-8-baseos'
/retest

/retest

/lgtm

/unhold

/test e2e-vsphere-csi-migration

/lgtm

/override ci/prow/e2e-vsphere
We're fixing the tests in https://github.com/openshift/origin/pull/27565


Mongo driver Interval test fails with duplicate null id

While working on #1175 and proving it correct, I noticed the following existing test failure on master:
cd internal/pkg/db/mongo/client.go
go test -tags mongoRunning
=== RUN   TestMongoDB
--- FAIL: TestMongoDB (2.01s)
    client_test.go:24: This test needs to have a running mongo on localhost
    db_scheduler.go:74: Error populating db: E11000 duplicate key error collection: scheduler.interval index: slug_1 dup key: { : null }
FAIL

Process finished with exit code 1

The test will fail on the second run of the loop to populate the database with intervals, suggesting that IDs are not getting set correctly.

Failed to reproduce on a second machine. Probably a fluke.


Allow empty etag in _info.

Similar to this pull request: https://github.com/fsspec/s3fs/pull/480 .
We encounter problems when accessing files hosted by Minio which might not be uploaded through Minio API (e.g., Minio on HDFS gateway, local files). This simple fix could solve problems like: https://github.com/iterative/dvc/issues/7161 .

Thanks for the super simple fix.


Remove trailing spaces

find . -name \*.py | xargs perl -pi -e 's/\s+\n/\n/g'

Skipped docs/conf.py for now as it uses CRLF and that will be fixed in #1672 1672

@schwehr Why was every line in geemap.py touched?

@schwehr Why was every line in geemap.py touched?

geemap/geemap.py: Python script, Unicode text, UTF-8 text executable, with CRLF line terminators

I redid the commit without geemap.py.
Thanks for catching that!

@schwehr Thank you for the fix.


fix public ip address ref

This checklist is used to make sure that common guidelines for a pull request are followed.

For new extensions:

[ ] My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update src/index.json automatically.
You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify src/index.json.
closed https://github.com/Azure/azure-cli-extensions/issues/5984

ssh

Any idea to improve the mock tests?


root website pipeline improvements

What are the relevant tickets?
Closes https://github.com/mitodl/ocw-studio/issues/1998
Description (What does it do?)
This PR makes some optimizations to the AWS CLI commands used when syncing build output to S3:

The aws s3 sync commands for the root website have been changed to aws s3 cp --recursive. This reduces the amount of time it takes to copy large amounts of small files into S3 because a comparison doesn't need to be made with all of those files against everything that already exists in the bucket.
The AWS_MAX_CONCURRENT_CONNECTIONS setting was added with a default of 10 (the same as the CLI default) and this value is set on max_concurrent_connections in the AWS CLI configuration before sync steps are run. This allows us to experiment with setting the value higher to see how it improves performance, or if it causes issues with things like the mass build.

How can this be tested?

Check out the master branch and make sure you have a recent database restore as well as all the prerequisites outlined in the README for publishing courses locally set up. In your .env file, also set AWS_MAX_CONCURRENT_CONNECTIONS=100, then spin up ocw-studio with docker-compose up
Upsert the ocw-www pipeline to your local Concourse with docker-compose exec web ./manage.py backpopulate_pipelines --filter ocw-www
Open the ocw-studio UI (http://localhost:8043/sites/) as well as the Concourse UI (http://localhost:8080) in two tabs
Publish ocw-www from the ocw-studio UI, then find the running pipeline in the Concourse UI by typing status: running into the search bar
Run the build 2 more times by clicking the plus sign in the upper right of the Concourse UI in online-site-job
Take the average runtime of the 3 runs
Check out this PR branch and repeat the above steps from the backpopulate step and get an average runtime against this branch
The average runtime on this branch should be a bit shorter
If you're comparing against QA / production, keep in mind that your local build time should be much shorter because copying to Minio locally takes a lot less time than syncing to S3 proper

@pt2302 That's not as dramatic as my results, but at least you were able to see some improvement. My average was around 2:30 for master and 1:45 with the changes. Either way, this is safe enough to merge and test in RC so we'll see what happens there.


Updated USA prisoner pack link

https://trello.com/c/eyxGLJfI/3015-broken-link-on-arrested-abroad-smart-tool
I've fixed the broken link to the USA prisoner pack.

Closing the PR as there was an error in the deployment.


Error types (effect typing for error handling)

Search Terms
Effect Types, Errors, Error Types
Suggestion
Allows functions to specify the types of errors, as well as the typesystem to check for them.

The errors are infectious, which means that other functions calling the function with error effects will also have the same effects.
throws (not to be confused with throw keyword) allows you to explicitly specify the error type, which may be useful for type definitions.
This rule has exceptions - firstly, if the function with the error type is wrapped around a try-catch block and handled appropriately.
There could also be an error silencing operator, which is meant to mark errors that isn't supposed to happen, and is meant to bubble up the error to the top of the function, similar to panic in languages such as Go and Rust. operator can possibly be !, to keep in line with null assertion operator.

Use Cases

try-catch block has stronger typing, instead of just Error, which allows for better autocompletion.
Cases where the errors thrown by function is known (e.g. APIs, database drivers) - as the compiler will be capable of checking for error types.

Examples

// implicitly has a error type of MyError
function myFunction(a: bool) {
  if (a) {
    throw new ErrorA();
  }
}

// Now anotherFunction has same error types as myFunction, plus its own error types
function anotherFunction(b: bool) {
  myFunction(b);
  if (b) {
    throw new ErrorB();
  }
}

// Errors can be explicitly specified
function explicitErrors() throws ErrorA|ErrorB {
  ...
}

// If a function is called in a try block, its error types will not be passed.
// However, error caught by the catch will have a type equal to union type of all errors emitted by the try block.
function errorCatching() {
  try {
    explicitErrors()
  } catch (err) {
    ...
  }
}

// this function will not have an error type, as the error is silenced
function errorSilencingExample() {
  throw new SilencedError()!;
}

Duplicate #13219


Docker swarm - encrypted network overlay - stops working.

Description
After creating a 3 node swarm (all managers), and then creating an encrypted overlay, we have noticed that the overlay network drops out randomly
Steps to reproduce the issue:

Create docker swarm cluster of at least 3 nodes
Create overlay with:

docker network create --attachable --opt encrypted -d overlay networkname"

NOTE: Making it attachable to test easily

Start an alpine container (easy test) on 2 nodes:

docker run -it --rm --net=networkname alpine /bin/ash

4.) Find the IPs (ifconfig) of each, and ping across.
Describe the results you received:
It works and randomly it stops. Firewall (both IP protocol 50 and the rest of the parts are any/any allowed between the 3 nodes)
Describe the results you expected:
To work all the time :)
Additional information you deem important (e.g. issue happens only occasionally):
Happens randomly almost. If you reboot, it starts working again.
Output of docker version:
Client:
 Version:      1.13.0
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Tue Jan 17 09:58:26 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.0
 API version:  1.25 (minimum version 1.12)
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Tue Jan 17 09:58:26 2017
 OS/Arch:      linux/amd64
 Experimental: false

Output of docker info:
Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 8
Server Version: 1.13.0
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 34
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: active
 NodeID: 4y3mi5goxun18p0rif8hdrt5o
 Is Manager: true
 ClusterID: vcwzg0mebqw4kp58pz8ynm0cn
 Managers: 3
 Nodes: 3
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 3
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
 Node Address: PUB#1
 Manager Addresses:
  PUB#1:2377
  PUB#2:2377
  PUB#2:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 03e5862ec0d8d3b3f750e19fca3ee367e13c090e
runc version: 2f7393a47307a16f8cee44a37b262e8b81021e3e
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-59-generic
Operating System: Ubuntu 16.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 24
Total Memory: 100 GiB
Name: swarmhost01
ID: GVD4:VFPH:ELAN:X2CK:CLFZ:MFDC:C5LT:RLTU:DWKE:KDKY:HT6M:BAC2
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
 nfs=yes
Experimental: false
Insecure Registries:
 <IP_ADDRESS>/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):
Environment is between physical and virtual systems. We have changed it around to be only virtual and only physical - same results. Systems are located in 3 different regions, on 3 different public IP spaces.

Additional information: the overlay seems to be aware of the other IPs, since when launching new containers on the different swarm subnets, it keeps increasing the last octet by 1 (ex: <IP_ADDRESS>, <IP_ADDRESS>, etc)
Also, so far from a few tests, it seems in some way related specifically to encrypted overlays -- we have not seen this happen on non-encrypted overlays. In fact, if you remove the overlay (when encrypted) and re-created it non-encrypted, it starts working again.

@ventz Few questions:

How long does it take to hit the issue
Do you have one container on one node ping the container on another node, or both pinging each other ?
Once the ping stops working, have you tried capturing on the rx container node interface with tcpdump to see if the other node is still sending the packets (tcpdump -i <iface> -p esp)
Just to see if the issue is at the tx node or at the rx node.

Thanks

@aboch

It seems "random". For example, on the latest cluster it worked for a few days (given no containers on it) before it hit this status again. I have seen it as soon as an hour, and the "average" (if you can even call it that) seems to be a few hours to less than a couple of days. I can start monitoring this more carefully.

I have one container on each node (when I test), but I've noticed that since I first test with just ICMP (easiest), it won't matter which way I am initializing it. If I get both ICMP echo + reply, the overlay works. As soon as I stop getting "replies", that indicates a problem. Once it hits this state, I do open it from all the sides, and test to see which way exactly it fails. I have noticed a few times that the issue seems to be surrounding the leader node. Ex: in this case where it just happened, as soon as I rebooted the leader, it started working again.

This is the interesting scenario -- you can see ESP coming in on the docker host (not container) while pinging, but it will not send ESP back (or originate if you try to originate only). It's almost like some part responsible for this just collapses. No error logs so far in /var/log/system

Here is what a tcpdump looks like
NOTE: again, PUB#1 happened to be leader again in this case
NOTE2: tcpdump taken _on_ NODE#3 (thus why "private#3"):

16:09:54.836702 IP PUB#1 > private#3: ESP(spi=0xd3052651,seq=0x13), length 140
16:09:55.836722 IP PUB#1 > private#3: ESP(spi=0xd3052651,seq=0x14), length 140
16:09:56.836727 IP PUB#1 > private#3: ESP(spi=0xd3052651,seq=0x15), length 140
16:09:57.836676 IP PUB#1 > private#3: ESP(spi=0xd3052651,seq=0x16), length 140
16:09:58.836693 IP PUB#1 > private#3: ESP(spi=0xd3052651,seq=0x17), length 140
16:09:59.836695 IP PUB#1 > private#3: ESP(spi=0xd3052651,seq=0x18), length 140
16:10:00.836763 IP PUB#1 > private#3: ESP(spi=0xd3052651,seq=0x19), length 140

As you can see, the system is not even trying to originate protocol 50 traffic back.

Thanks @ventz
It looks like the system is currently in the erroneous state. If so, it would be interesting do some debugging:
Can you please check if the container running on node#3 is actually receiving the icmp echo request packet and that it is generating the responses back (For this either tcpdump from docker exec if the container image has it, or nsenter into the container netns from the host (nsenter --net=<value from docker inspect container | grep xKey">)

@aboch Hi - this is sadly the tcpdump I had taken from earlier today when it happened. I have already rebooted that node.
That said, the container on private#3 was generating back the icmp replies, but they didn't show up on the component responsible for receiving/processing/sending ESP.
It's almost like something isolated to the ESP connection is failing.
To give you an example, when this happened, I removed the containers and removed the overlay. Then re-created the overlay (encrypted), and it still didn't work. Tried removing+re-adding one more time. At that point, I re-added it without encryption, and it worked. I removed it, and re-added it with encryption again, nothing. But then rebooting the leader node fixed it.

It's almost like something isolated to the ESP connection is failing.

Looks like.
I did it before, but let me run again few nodes on AWS overnight with containers pinging over encrypted network. I will have them be all managers as in your test. I will report back my findings.

I did get it to happen on AWS, but if possible - try to do one at a local datacenter or a different network. Noticed it happening a lot more when it's going over public lines -- so it might be a disruption or blip that ends up causing it.
I'll keep an eye for it again too, although for now the latest cluster seems to be working.
I know this sucks for debugging -- it's clearly broken in some way, but capturing what exactly is not easy, especially from someone going "hey it just fails, oh and it's random" :)

@aboch I have an environment where this is happening right now.

@ventz
Check with ip -s xfrm state if the counters are increasing for the proper SAa (check the src and dst IP and the SPI). On each node you will find 4 states for each couple of nodes, 3 RX (dst is this node) states and one TX (src is this node).

Ok, this is kind of interesting:
The overlay seems to be a <IP_ADDRESS>/24
04:06:30.845862 IP <IP_ADDRESS> > <IP_ADDRESS>: ICMP echo request, id 3328, seq 0, length 64
04:06:31.846990 IP <IP_ADDRESS> > <IP_ADDRESS>: ICMP echo request, id 3328, seq 1, length 64
04:06:32.848206 IP <IP_ADDRESS> > <IP_ADDRESS>: ICMP echo request, id 3328, seq 2, length 64
04:06:33.848998 IP <IP_ADDRESS> > <IP_ADDRESS>: ICMP echo request, id 3328, seq 3, length 64
04:06:34.849709 IP <IP_ADDRESS> > <IP_ADDRESS>: ICMP echo request, id 3328, seq 4, length 64
04:06:35.849833 ARP, Request who-has <IP_ADDRESS> tell <IP_ADDRESS>, length 28
04:06:35.849892 ARP, Reply <IP_ADDRESS> is-at 02:42:0a:00:00:0d, length 28
04:06:35.850421 IP <IP_ADDRESS> > <IP_ADDRESS>: ICMP echo request, id 3328, seq 5, length 64
04:06:36.851073 IP <IP_ADDRESS> > <IP_ADDRESS>: ICMP echo request, id 3328, seq 6, length 64
04:06:37.851824 IP <IP_ADDRESS> > <IP_ADDRESS>: ICMP echo request, id 3328, seq 7, length 64
04:06:38.852600 IP <IP_ADDRESS> > <IP_ADDRESS>: ICMP echo request, id 3328, seq 8, length 64

Trying from PUB#1 -> PUB#3 (out of 5, where the first 3 are managers, and 2 are workers)
The interesting part is that the arp request had the right mac (00:00:0d) of .13 --> which I verified on .13, was the case. But .13 was not seeing any traffic. Same in the reverse direction.
However, I can actually see ESP exchanges in this case. Also, some part of the overlay works -- because I can reach some of the other nodes, so it's not complete shot. It's just the connection between these 2 hosts (even though they are both online)

Also please check on the node which is not generating the responses back whether the counter for this rule is actually increasing:
sudo iptables -t mangle -nvL OUTPUT

@aboch any way we can chat in real time (irc?) - I can also share access into the env via a screen session.

#docker-network irc channel

Thanks - pinged you there. We can update the solution here eventually for others, but it would be easier to debug in real time.

@aboch - Thanks again for the help! Seems like there is an issue, but you have it now :)
For anyone else that sees this, for now you can run:
ip xfrm state flush; ip xfrm policy flush

on each host, and then start+rm a container to pull back the overlay on that host. It fixes the issue.

Thanks @ventz for the friday night debug session in irc ;-)
That was really helpful to spot the problem.
The problem was because of stale SAs from previous cluster life that were being selected to encrypt the packets. (the corresponding stale SA were not on the other node, most likely because that node did not have any container in the encrypted network in the previous cluster)
There are two workarounds:
One is to selectively remove the stale entries (you look at their date of installation) via ip xfrm state del
The other more practical one (which we opted for) is to flush the table ip xfrm state flush and then re-trigger the states programming by starting a container on the encrypted network on each node.
I have an open PR (docker/libnetwork/pull/1354) to age the stale entries out, but the aging time can only be 3 times the rotation key time + delta (36hrs+delta), so it would not have helped you right away.
I think something better can be done to avoid this. I will work to a fix.

Hello all. Can you check my last comment for this issue: https://github.com/docker/docker/issues/30595. Seems like a related one - networking starts failing after some time, but it's only TCP. I also have a cluster in that state right now. Your help would be highly appreciated

It seems like there might be a larger issue around this. On a new cluster (3 nodes, all managers), here is the time period between the overlay working perfectly and it automatically stopping to work:
2017-02-05  1:57:38 -> working
2017-02-05 20:15:51 -> failed
After being launched, this cluster was not touched.
The only reason we know the encrypted overlay failed is because there is a service that uses the overlay to sync every 5 seconds, and it failed + did not recover after that point.

I have also noticed this happening. I am automatically launching new services from within the swarm cluster where new service has to connect to the parent service and sometimes it appears that node networking fails since the new service fails connect.
The node is visible in the cluster and docker seems to be thinking that the node is okay since new services are scheduled there. Rebooting node solves the issue, but you can only detect it if you have some workload there that relies on networking.

I believe --opt encrypted was messing with the network between my CMS and Mysql. I stop to used it.
I feel this shall be enabled by default when stable. But I don't know about the CPU overhead.
Cheers!

@aboch - See panic dump: https://github.com/docker/docker/issues/30800
(especially towards the bottom)
Might shed light on additional bug that has to do w/ the encryption overlay.

Thanks @ventz I found where the issue is and pushed the fix ^. It affects only the use case where the advertise address is outside of the box (1-1 NAT case) which is way I could not initially reproduce. It happens during key rotations. It is expected a node reload, or removing and restarting the container on the node fixes it, but only until next datapath key rotation happens.

@aboch - That's great! Thanks.
What do you think is the eta for the next update that will have this?

@aboch It looks like a bug was introduced in 17.03 (which was not present in 1.13) related to the encrypted overlay when natting is involved, and when 2+ systems are on different networks can talk internally, but again when they advertise public IPs since they need to communicate with, let's say a 3rd system externally.
Just testing:
System A = <IP_ADDRESS> = PUB1
System B = <IP_ADDRESS> = PUB2
Encrypted overlay (attachable)

When you launch containers on A (<IP_ADDRESS>) and B (<IP_ADDRESS>) and ping from container on B->A, you get:
On A:
01:49:54.194927 IP <IP_ADDRESS> > <IP_ADDRESS>: ESP(spi=0x1304f906,seq=0x1), length 140
01:49:55.195686 IP <IP_ADDRESS> > <IP_ADDRESS>: ESP(spi=0x1304f906,seq=0x2), length 140
01:49:56.196093 IP <IP_ADDRESS> > <IP_ADDRESS>: ESP(spi=0x1304f906,seq=0x3), length 140
01:49:57.196470 IP <IP_ADDRESS> > <IP_ADDRESS>: ESP(spi=0x1304f906,seq=0x4), length 140
01:49:58.196834 IP <IP_ADDRESS> > <IP_ADDRESS>: ESP(spi=0x1304f906,seq=0x5), length 140

On B:
01:49:54.193550 IP <IP_ADDRESS> > PUB2: ESP(spi=0x1304f906,seq=0x1), length 140
01:49:55.194218 IP <IP_ADDRESS> > PUB2: ESP(spi=0x1304f906,seq=0x2), length 140
01:49:56.194586 IP <IP_ADDRESS> > PUB2: ESP(spi=0x1304f906,seq=0x3), length 140
01:49:57.194948 IP <IP_ADDRESS> > PUB2: ESP(spi=0x1304f906,seq=0x4), length 140
01:49:58.195281 IP <IP_ADDRESS> > PUB2: ESP(spi=0x1304f906,seq=0x5), length 140

Nothing has changed config wise/launch wise since upgrading from 1.13.
(otherwise, it seems to work in the other scenarios --> one system per datacenter/non-routable rfc1918 space)
I can confirm that if you create it non-encrypted, everything works.
Client:
 Version:      17.03.0-ce
 API version:  1.26
 Go version:   go1.7.5
 Git commit:   60ccb22
 Built:        Thu Feb 23 11:02:43 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.03.0-ce
 API version:  1.26 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   60ccb22
 Built:        Thu Feb 23 11:02:43 2017
 OS/Arch:      linux/amd64
 Experimental: false

@aboch And possibly related
System A has:
Mar  6 02:03:52 A dockerd[5207]: time="2017-03-06T02:03:52.600013459-05:00" level=info msg="Initializing Libnetwork Agent Listen-Addr=<IP_ADDRESS> Local-addr=<IP_ADDRESS> Adv-addr=PUB1 Remote-addr ="
Mar  6 02:03:52 A dockerd[5207]: time="2017-03-06T02:03:52.600607214-05:00" level=info msg="Initializing Libnetwork Agent Listen-Addr=<IP_ADDRESS> Local-addr=<IP_ADDRESS> Adv-addr=PUB1 Remote-addr ="

vs
System B has:
Mar  6 02:06:47 B dockerd[5689]: time="2017-03-06T02:06:47.848564573-05:00" level=info msg="Initializing Libnetwork Agent Listen-Addr=<IP_ADDRESS> Local-addr=<IP_ADDRESS> Adv-addr=PUB2 Remote-addr =PUB2"
Mar  6 02:06:47 B dockerd[5689]: time="2017-03-06T02:06:47.866718260-05:00" level=info msg="Initializing Libnetwork Agent Listen-Addr=<IP_ADDRESS> Local-addr=<IP_ADDRESS> Adv-addr=PUB2 Remote-addr =PUB2"

^ Note that System B (the joining one) gets the "Remote-addr" as the --advertise-addr (public IP), but System A does not see it.
The commands used to create and join the two are:
System A:
docker swarm init --advertise-addr PUB1 --listen-addr <IP_ADDRESS>:2377

System B:
docker swarm join \
--token SWMTKN-1-... \
--advertise-addr PUB2 --listen-addr <IP_ADDRESS>:2377 PUB1:2377

Last possibly interesting update - on a new swarm create+encrypted overlay, it seems you can generate ESP packets one way (no matter if FIRST you try from A->B or B->A...but there won't be a return. From that point, the other direction does not work. So if I start with container on A pinging container on B -- I'll see the pri(A)->PUB(B)  and on B packets coming in. But never an attempt to reply. If I stop and reverse direction, there wont' be anything. If I leave the swarm, and start again and try in the reverse direction - again the traffic will show up, but only in that direction, and the reverse will not work. Seems to be something related to the NAT.

Hi @ventz, yes something looks wrong from your logs.
On System A you should see outgoing ESP packets as PRI1 -> PUB2 and incoming as PUB2->PRI1.
Instead from your logs
01:49:54.194927 IP <IP_ADDRESS> > <IP_ADDRESS>: ESP(spi=0x1304f906,seq=0x1), length 140

it looks like system A is aware of private IP of system B.
This is one problem that needs to be investigated. You should not see any ipsec tunnels with the PRI IP of any remote node in a swarm created with that configuration.
I am not aware of other changes in that area which could have caused this. Also, I have a 3 nodes swarm running on AWS with encryption via public IPs and it is running smooth for more than 2 weeks.

Note that System B (the joining one) gets the "Remote-addr" as the --advertise-addr (public IP), but System A does not see it.

That is expected to be seen only on the node that joins the swarm. During the encryption setup, the remote node PUB IP is learned from the nw control plane, not from that parameter.
As of now I cannot think of what could have caused your issue. I'll get back if I get any idea.

So we're seeing something quite similar to what is described above. It may also be related to https://github.com/moby/moby/issues/33133
We've noticed that we're unable to send any data traffic through an encrypted overlay to an endpoint behind AWS/GCE NAT on Debian 9, Ubuntu 18.04, or anything with a kernel > 4.4.
The only modern Debian variant that works is 16.04 with the 4.4 kernel.

Simple repro: node-1 on DigitalOcean, node-2 on GCE/AWS.

Spin up a Debian Stretch host (4.9 kernel) on both DigitalOcean and GCE or AWS with the latest stable Docker-CE (18.03.1)

Initialize the swarm on DigitalOcean node-1 using --advertise-addr external_ip

By default, DigitalOcean has no firewall so node-1 is wide open. Open TCP 2377, TCP/UDP 7946, UDP 4789, Protocol 50 on node-2 in the AWS Security Group/GCP VPC Firewall Rules and join the swarm as a worker using --advertise-addr node-2-external-ip

docker create network --attachable --driver overlay --opt encrypted encryption_test

docker run -ti --rm --network encryption_test debian bash

If you do this with a non-encrypted overlay, traffic flows with no issues. We're able to ping between containers, run iperf3; all is well. But on the encrypted overlay, the traffic simply won't transmit.
ip xfrm state gives:
src <IP_ADDRESS> dst <IP_ADDRESS>
	proto esp spi 0x3e7a19d6 reqid 13681891 mode transport
	replay-window 0
	aead rfc4106(gcm(aes)) 0xdcb885a138afc1d801f86a6b379dd22e3e7a19d6 64
	anti-replay context: seq 0x0, oseq 0xa, bitmap 0x00000000
	sel src <IP_ADDRESS>/0 dst <IP_ADDRESS>/0
src <IP_ADDRESS> dst <IP_ADDRESS>
	proto esp spi 0x617bc0ba reqid 13681891 mode transport
	replay-window 0
	aead rfc4106(gcm(aes)) 0xdcb885a138afc1d801f86a6b379dd22e617bc0ba 64
	anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
	sel src <IP_ADDRESS>/0 dst <IP_ADDRESS>/0

We can see ESP traffic on the remote host (not the one doing the pinging) but there is no return:
18:31:24.652640 IP ec2-18-176-21-33.us-east-2.compute.amazonaws.com > hostname.mydomain.com: ESP(spi=0x3e7a19d6,seq=0x1), length 140
18:31:25.653615 IP ec2-18-176-21-33.us-east-2.compute.amazonaws.com > hostname.mydomain.com: ESP(spi=0x3e7a19d6,seq=0x2), length 140
18:31:26.654775 IP ec2-18-176-21-33.us-east-2.compute.amazonaws.com > hostname.mydomain.com: ESP(spi=0x3e7a19d6,seq=0x3), length 140
18:31:27.655940 IP ec2-18-176-21-33.us-east-2.compute.amazonaws.com > hostname.mydomain.com: ESP(spi=0x3e7a19d6,seq=0x4), length 140
18:31:28.657156 IP ec2-18-176-21-33.us-east-2.compute.amazonaws.com > hostname.mydomain.com: ESP(spi=0x3e7a19d6,seq=0x5), length 140

There seems to be a change here in how the kernel is processing the traffic in > 4.4 - has anyone else seen this?
/cc @aboch @ventz

So we're seeing something quite similar to what is described above. It may also be related to https://github.com/moby/moby/issues/33133
We've noticed that we're unable to send any data traffic through an encrypted overlay to an endpoint behind AWS/GCE NAT on Debian 9, Ubuntu 18.04, or anything with a kernel > 4.4.
The only modern Debian variant that works is 16.04 with the 4.4 kernel.

Simple repro: node-1 on DigitalOcean, node-2 on GCE/AWS.

Spin up a Debian Stretch host (4.9 kernel) on both DigitalOcean and GCE or AWS with the latest stable Docker-CE (18.03.1)

Initialize the swarm on DigitalOcean node-1 using --advertise-addr external_ip

By default, DigitalOcean has no firewall so node-1 is wide open. Open TCP 2377, TCP/UDP 7946, UDP 4789, Protocol 50 on node-2 in the AWS Security Group/GCP VPC Firewall Rules and join the swarm as a worker using --advertise-addr node-2-external-ip

docker create network --attachable --driver overlay --opt encrypted encryption_test

docker run -ti --rm --network encryption_test debian bash

If you do this with a non-encrypted overlay, traffic flows with no issues. We're able to ping between containers, run iperf3; all is well. But on the encrypted overlay, the traffic simply won't transmit.
ip xfrm state gives:
src <IP_ADDRESS> dst <IP_ADDRESS>
	proto esp spi 0x3e7a19d6 reqid 13681891 mode transport
	replay-window 0
	aead rfc4106(gcm(aes)) 0xdcb885a138afc1d801f86a6b379dd22e3e7a19d6 64
	anti-replay context: seq 0x0, oseq 0xa, bitmap 0x00000000
	sel src <IP_ADDRESS>/0 dst <IP_ADDRESS>/0
src <IP_ADDRESS> dst <IP_ADDRESS>
	proto esp spi 0x617bc0ba reqid 13681891 mode transport
	replay-window 0
	aead rfc4106(gcm(aes)) 0xdcb885a138afc1d801f86a6b379dd22e617bc0ba 64
	anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
	sel src <IP_ADDRESS>/0 dst <IP_ADDRESS>/0

We can see ESP traffic on the remote host (not the one doing the pinging) but there is no return:
18:31:24.652640 IP ec2-18-176-21-33.us-east-2.compute.amazonaws.com > hostname.mydomain.com: ESP(spi=0x3e7a19d6,seq=0x1), length 140
18:31:25.653615 IP ec2-18-176-21-33.us-east-2.compute.amazonaws.com > hostname.mydomain.com: ESP(spi=0x3e7a19d6,seq=0x2), length 140
18:31:26.654775 IP ec2-18-176-21-33.us-east-2.compute.amazonaws.com > hostname.mydomain.com: ESP(spi=0x3e7a19d6,seq=0x3), length 140
18:31:27.655940 IP ec2-18-176-21-33.us-east-2.compute.amazonaws.com > hostname.mydomain.com: ESP(spi=0x3e7a19d6,seq=0x4), length 140
18:31:28.657156 IP ec2-18-176-21-33.us-east-2.compute.amazonaws.com > hostname.mydomain.com: ESP(spi=0x3e7a19d6,seq=0x5), length 140

There seems to be a change here in how the kernel is processing the traffic in > 4.4 - has anyone else seen this?
/cc @aboch @ventz

@emcgee could you open a new ticket so that it doesn't get lost? (this issue is closed, so comments get easily overlooked