cmd/gofmt: non-idempotent formatting of line comments

tl;dr: Access http://play.golang.org/p/Ry5OUCK43C; Click Format; Click Format.
After running the following snippet through gofmt -w:
package main

//line x:1

func main() {
}

The output is:
package main

//line x:1

func main() {
}

This output looks correct with a single newline between //line and func main. However if we run gofmt -w again on the same file the output is different:
package main

//line x:1
func main() {
}

Now the newline between //line and func main was removed. This behavior is different from what would happen if instead of //line we had a regular commented line.
This was tested both on master and 1.5.1:
$ go version
go version devel +97f854c Sat Dec 19 10:00:04 2015 +0000 darwin/amd64

Unfortunate but probably shouldn't be running gofmt on files with //line comments anyway.  It missed the past 4 releases and still repros.

@bradfitz I will need to see who complicated the fix is, first. I won't get to this before June 1.

I don't think should do anything here: Fixing this would mean that once we see a //line directive we're not allowed to move anything that follows later to a different line. At the very least, that is the assumption we have to make until we see the next //line comment. But that interferes directly with gofmt's main purpose which is to organize white space for uniform formatting. One "solution" would be to not move any lines that follow a //line comment. But it's pretty tricky to get this correct under all circumstances. I'm not convinced the amount of complexity is worth the few cases where it might matter. I don't want to go down that route.
The situation is additionally complicated by the fact that the go/printer has a mode that emits //line comments.
I've recently fixed https://github.com/golang/go/issues/5945 which makes sure that the above-mentioned mode only prints //line directives immediately before non-whitespace. Code that is printed that way will pass through gofmt undisturbed.
In other words, don't use //line directives in front of empty lines - use them where it matters. Or don't gofmt such files.
Closing as unfortunate.


Adjust Replication and Rebalancing tutorial to secure

This change is

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/25bed195d22a12967f9f27c3e7968ee848381259/
Edited pages:

v20.1/demo-replication-and-rebalancing.md

Online preview: http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/d28c7087e55fa338966345b442aad22ecf10654a/
Edited pages:

v20.1/demo-replication-and-rebalancing.md
v20.2/demo-replication-and-rebalancing.md

TFTR, @ericharmeling.


What format / language are the results in?

Hi,
I was trying to use Jackson's mapper in Java to map the output from Dex into Java objects and do stuff with it, but I realized this isn't JSON. What is it, really? How would I parse it anyway?

Oh god. This IS Json but with reversed double and single quotes. I did a find and replace, swapped the single quotes with an apostrophe (`), then swapped all double quotes with single quotes and finally swapped all apostrophes (former single quotes) with double quotes. And it worked. It has now been turned into valid JSON. Please fix this, it shouldn't take you long now that it's been diagnosed and it's quite very annoying.


Use Light Bulb API instead of Smart Tag API in VS2015

We need to move to a new modern way of refactoring by using Light Bulb API instead of Smart Tag API in VS2015

We need to update:

implement interface
rename
generate record stub
generate union pattern match cases
generate references for FSI

@dungpa, @vasily-kirichenko, have I forgotten anything which needs to be updated to use Light Bulb?

Mostly correct. You should add "Resolve unopened namespaces" and remove "Generate references for FSI".

@dungpa So, it can be compiled in VS 2015 only, right? Can you formulate the process? Should we add FSharpVSPowerTools.Logic.VS2015 project and put all VS 2015 stuff there? Should the project be included into FSharpVSPowerTools.v14.sln only?
Could you write a short description how our stuff organized regarding different version of VS?

Please make it an option so you can opt to have just the current SmartTags with the discrete underline instead of the LightBulb

@ovatsus, do you think someone will ever use Smart Tags instead if he has Light Bulb?
Light Bulb is a primary way of doing refactoring in Visual Studio 2015.
Smart Tag even can't be executed using Ctrl+.

@vasily-kirichenko It can be compiled with VS 2013 if we include VS 2015 dlls into build process. And they can appear in both solutions.
Here is how it works:

FSharpVSPowerTools.Logic.VS2015 exports a provider with fully working solution. It exports as VS2015 attribute.
FSharpVSPowerTools.Logic exports a provider which simply returns null. It exports as VS2013 attribute.
The main facade uses ImportMany and choose the most recent VS version. The logic is the same as https://github.com/fsprojects/VisualFSharpPowerTools/blob/7e0ea76fc23be871ce5aebd02272d7feed8447e2/src/FSharpVSPowerTools.Logic/NavigateToItem.fs#L33-L51.

We should test that it works fine in VS 2015 and doesn't throw exceptions in VS2013. The main logic of the feature is almost similar as that of smart tags.

@vasily-kirichenko Something new you may not notice. In Properties page of FSharpVSPowerTools.csproj, you can change experimental instance of Visual Studio 12 to that of Visual Studio 14 so that deployment can be done on directly VS 2015. We can use this to develop in VS 2013 and test in both VS 2013 and VS 2015.

I wonder what is the purpose to do such dances with a tambourine? :)

@dungpa I tried to build in 2013 and "run without debugger" with 2015. It launches, but the addin was not installed. So I've given up and move to 2015 for now.

I can F5 (run under debugging) to install and debug VFPT on VS 2015. There might be some exceptions, but it works after ignoring those. It's quite handy since VS 2013 is much more stable to use than VS 2015.

I'm trying to use 2015 -> 2015 and what I get on Ctrl+F5:

I never seen anything like this in my whole life.

It does not work for me :( This is VS 2013 output:
1>CreateVsixContainer:
1>  Creating VSIX Container...
1>  Creating the package with the following compression level. "Normal".
1>  FSharpVSPowerTools -> D:\git\VisualFSharpPowerTools\bin\FSharpVSPowerTools.vsix
1>FindExistingDeploymentPath:
1>  Extension 'FSharpVSPowerTools.68b42cfe-c752-4094-8dba-ed48aa81cac8' located at 'C:\USERS\KOT\APPDATA\LOCAL\MICROSOFT\VISUALSTUDIO\12.0EXP\EXTENSIONS\FSHARP.ORG\VISUAL F# POWER TOOLS\2.0.1\'.
1>DeployVsixExtensionFiles:
1>  VsixID = FSharpVSPowerTools.68b42cfe-c752-4094-8dba-ed48aa81cac8
1>  VsixVersion = 2.0.1
1>  VsixDeploymentPath = C:\Users\kot\AppData\Local\Microsoft\VisualStudio\12.0Exp\Extensions\fsharp.org\Visual F# Power Tools\2.0.1\
1>  Copying file from "obj\Release\FSharpVSPowerTools.dll" to "C:\Users\kot\AppData\Local\Microsoft\VisualStudio\12.0Exp\Extensions\fsharp.org\Visual F# Power Tools\2.0.1\\FSharpVSPowerTools.dll".
1>  Copying file from "obj\Release\extension.vsixmanifest" to "C:\Users\kot\AppData\Local\Microsoft\VisualStudio\12.0Exp\Extensions\fsharp.org\Visual F# Power Tools\2.0.1\\extension.vsixmanifest".
1>  Copying file from "obj\Release\FSharpVSPowerTools.pdb" to "C:\Users\kot\AppData\Local\Microsoft\VisualStudio\12.0Exp\Extensions\fsharp.org\Visual F# Power Tools\2.0.1\\FSharpVSPowerTools.pdb".
1>  Extension 'FSharpVSPowerTools.68b42cfe-c752-4094-8dba-ed48aa81cac8' has been enabled.
1>CopyVsixManifestFile:
1>  Copying file from "obj\Release\extension.vsixmanifest" to "..\..\bin\extension.vsixmanifest".
1>  FSharpVSPowerTools -> D:\git\VisualFSharpPowerTools\bin\extension.vsixmanifest
1>DeployToVisualStudio2015:
1>  Deploying extension to Visual Studio 2015...
1>ZipProjects:
1>  Zipping ProjectTemplates
1>  FSharpVSPowerTools ->
1>ZipItems:
1>  Zipping ItemTemplates
1>  FSharpVSPowerTools ->
1>_CopyOutOfDateSourceItemsToOutputDirectory:
1>Skipping target "_CopyOutOfDateSourceItemsToOutputDirectory" because all output files are up-to-date with respect to the input files.
1>CopyFilesToOutputDirectory:
1>  FSharpVSPowerTools -> D:\git\VisualFSharpPowerTools\bin\FSharpVSPowerTools.dll
1>GenerateTargetFrameworkMonikerAttribute:
1>Skipping target "GenerateTargetFrameworkMonikerAttribute" because all output files are up-to-date with respect to the input files.
1>CoreCompile:
1>Skipping target "CoreCompile" because all output files are up-to-date with respect to the input files.
1>GeneratePkgDef:
1>  Creating intermediate PkgDef file.
1>  VSSDK1033: Skipping generation of PkgDef because all files are up to date.
1>CheckCTOFileHasChanged:
1>Skipping target "CheckCTOFileHasChanged" because all output files are up-to-date with respect to the input files.
1>RestorePackages:
1>  "D:\git\VisualFSharpPowerTools\.paket\paket.exe" restore --references-files D:\git\VisualFSharpPowerTools\src\FSharpVSPowerTools.Core\paket.references
1>  0 seconds - ready.
1>GenerateTargetFrameworkMonikerAttribute:
1>Skipping target "GenerateTargetFrameworkMonikerAttribute" because all output files are up-to-date with respect to the input files.
1>CoreCompile:
1>Skipping target "CoreCompile" because all output files are up-to-date with respect to the input files.
1>CopyFilesToOutputDirectory:
1>  FSharpVSPowerTools.Core -> D:\git\VisualFSharpPowerTools\src\FSharpVSPowerTools.Core\bin\Release\FSharpVSPowerTools.Core.dll
1>RestorePackages:
1>  "D:\git\VisualFSharpPowerTools\.paket\paket.exe" restore --references-files D:\git\VisualFSharpPowerTools\src\FSharpVSPowerTools.Logic\paket.references
1>  0 seconds - ready.
1>MainResourcesGeneration:
1>Skipping target "MainResourcesGeneration" because all output files are up-to-date with respect to the input files.
1>GenerateTargetFrameworkMonikerAttribute:
1>Skipping target "GenerateTargetFrameworkMonikerAttribute" because all output files are up-to-date with respect to the input files.
1>CoreCompile:
1>Skipping target "CoreCompile" because all output files are up-to-date with respect to the input files.
1>CopyFilesToOutputDirectory:
1>  FSharpVSPowerTools.Logic -> D:\git\VisualFSharpPowerTools\src\FSharpVSPowerTools.Logic\bin\Release\FSharpVSPowerTools.Logic.dll
1>C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets(1697,5): warning MSB3277: Found conflicts between different versions of the same dependent assembly that could not be resolved.  These reference conflicts are listed in the build log when log verbosity is set to detailed.
1>CreateVsixContainer:
1>  Creating VSIX Container...
1>  Creating the package with the following compression level. "Normal".
1>  FSharpVSPowerTools -> D:\git\VisualFSharpPowerTools\bin\FSharpVSPowerTools.vsix
1>FindExistingDeploymentPath:
1>  Extension 'FSharpVSPowerTools.68b42cfe-c752-4094-8dba-ed48aa81cac8' located at 'C:\USERS\KOT\APPDATA\LOCAL\MICROSOFT\VISUALSTUDIO\14.0EXP\EXTENSIONS\FSHARP.ORG\VISUAL F# POWER TOOLS\2.0.1\'.
1>DeployVsixExtensionFiles:
1>  VsixID = FSharpVSPowerTools.68b42cfe-c752-4094-8dba-ed48aa81cac8
1>  VsixVersion = 2.0.1
1>  VsixDeploymentPath = C:\Users\kot\AppData\Local\Microsoft\VisualStudio\14.0Exp\Extensions\fsharp.org\Visual F# Power Tools\2.0.1\
1>  Copying file from "obj\Release\FSharpVSPowerTools.dll" to "C:\Users\kot\AppData\Local\Microsoft\VisualStudio\14.0Exp\Extensions\fsharp.org\Visual F# Power Tools\2.0.1\\FSharpVSPowerTools.dll".
1>  Copying file from "obj\Release\extension.vsixmanifest" to "C:\Users\kot\AppData\Local\Microsoft\VisualStudio\14.0Exp\Extensions\fsharp.org\Visual F# Power Tools\2.0.1\\extension.vsixmanifest".
1>  Copying file from "obj\Release\FSharpVSPowerTools.pdb" to "C:\Users\kot\AppData\Local\Microsoft\VisualStudio\14.0Exp\Extensions\fsharp.org\Visual F# Power Tools\2.0.1\\FSharpVSPowerTools.pdb".
1>  Extension 'FSharpVSPowerTools.68b42cfe-c752-4094-8dba-ed48aa81cac8' has been enabled.
1>CopyVsixManifestFile:
1>  Copying file from "obj\Release\extension.vsixmanifest" to "..\..\bin\extension.vsixmanifest".
1>  FSharpVSPowerTools -> D:\git\VisualFSharpPowerTools\bin\extension.vsixmanifest
1>CreateMenuPkgDef:
1>Skipping target "CreateMenuPkgDef" because it has no outputs.
1>CreateMenuPkgDef:
1>Skipping target "CreateMenuPkgDef" because it has no outputs.
1>
1>Build succeeded.

Everything looks OK, but VFPT does not appear in Extensions and Updates dialog nor it works, of course.

If I change /rootsuffix to some new value, then it does not work even VS 2015 -> VS 2015. Could you reproduce it?

Did you install RTM on top of RC or did you do 2015 RTM clean install?

Nothing looks suspicious from the log. You can try to bump version in extension.vsixmanifest and check that all assemblies are copied to C:\Users\kot\AppData\Local\Microsoft\VisualStudio\14.0Exp\Extensions\fsharp.org\Visual F# Power Tools\<version> correctly.

It does not help.
I revert rootsuffix to Exp and the extension was appear, but colors everywhere in VS was broken, so I deleted c:\Users\kot\AppData\Local\Microsoft\VisualStudio\14.0Exp folder entirely and Ctrl+F5 again. No VFPT :(
Could you remove that folder and try to repeat this behavior?

I confirm that removing that folder and Ctrl + F5, VFPT isn't installed.
When I increase VFPT version and run F5, VFPT is installed to Experimental instance again.

Does resetting settings to details by importing default for you, possibly F# default, help?
Doing rebuild usually helps for me to force extensions deploy to experimental instance.
Check deployment path

@dungpa I have issues.
ISuggestedActionsSourceProvider and ISuggestedActionsSource are defined in C:\Windows\Microsoft.Net\assembly\GAC_MSIL\Microsoft.VisualStudio.Language.Intellisense\v4.0_<IP_ADDRESS>__b03f5f7f11d50a3a\Microsoft.VisualStudio.Language.Intellisense.dll (VS 2015) only. Logic project references D:\github\FSharpVSPowerTools\lib\vs2012\Microsoft.VisualStudio.Language.Intellisense.dll where those interfaces are absent. So, the only place where we can Export ISuggestedActionsSourceProvider is Logic.VS2015 project (https://github.com/vasily-kirichenko/FSharpVSPowerTools/blob/light-bulb/src/FSharpVSPowerTools.Logic.VS2015/ResolveUnopenedNamespaceSuggestedActionsSourceProvider.fs#L14). But it does not work at all - light bulb does not appear and breakpoints are not active. Am I right that we can export from FSharpVSPowerTools assembly only? If so, I cannot see a solution.

We can add VS2015's Microsoft.VisualStudio.Language.Intellisense.dll to lib\vs2015 folder and reference on both projects.
In order to use the new project as a MEF component; it should be added to https://github.com/fsprojects/VisualFSharpPowerTools/blob/7e0ea76fc23be871ce5aebd02272d7feed8447e2/src/FSharpVSPowerTools/source.extension.vsixmanifest#L27.

I'm afraid I don't understand why we need VS2015 project at all. What could go wrong if we implement ISuggestedActionsSourceProvider right in Logic project? VS 2013 will throw?

I'm not sure either. I think it should work with both providers on Logic project.

:)

Wow. Time to celebrate :+1:.

Yahoo! I'm happy.
Is it also possible to implement a preview of changes for F#?

"Resolve unopened namespaces" is done here https://github.com/fsprojects/VisualFSharpPowerTools/pull/1069

Implemented in https://github.com/fsprojects/VisualFSharpPowerTools/pull/1079

@xperiandri Could you please install this build https://ci.appveyor.com/project/dungpa/visualfsharppowertools/build/2.1.0-a1508110519 and give us feedback on the new lightbulb functionalities? Thanks.

Light bulb works well
However "Resolve unopened namespaces" works not as expected
I got
open FSharp.Text.RegexProvider
type UrlRegex = Regex<"background(-image)?: url\(\"??(?<url>\S+)\???\d*\"??\)">

So I commented
/// open FSharp.Text.RegexProvider

But suggestion appear only for
open System.Text.RegularExpressions

And it must not appear for a type provider.
But as I understand it is a problem of class resolution logic.

@xperiandri We don't have assembly metadata for erased type providers. So it's a limitation that unfortunately we can't bypass.

OK, but anyway a type declared from TypeProvider is a type.
So it can be found, can't it?

Finally I was explained how to create a standard UI for displaying changes made by action.
https://github.com/dotnet/roslyn/issues/4520#issuecomment-152046626

@xperiandri PRs are welcome.


bump to node20

Fix for deprecated node16
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20<EMAIL_ADDRESS>For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/

I would like to get this merged as well! @mpdude

:+1:

Yes waiting to be merge..
Workflow started showing node16 deprecation warnings.
Thanks for fixing..!

Yes waiting to be merge..
Workflow started showing node16 deprecation warnings.

Same here

Hi, @sebastiankugler and @mpdude 👋
Could you please merge this PR and release the changes?

We are waiting for this.

@archen I think this should also change the use from checkout@v3 -> checkout@v4 in the demo example and README. Otherwise people following these will have the same warnings

It is for the maintainers to decide anyways @archen. I just thought I mention it here, but I guess you are right, you are not the correct addressee

[like]  Seto Elkahfi reacted to your message:

From: Roman Almeida @.>
Sent: Monday, February 5, 2024 8:19:54 PM
To: webfactory/ssh-agent @.>
Cc: Seto Elkahfi @.>; Comment @.>
Subject: Re: [webfactory/ssh-agent] bump to node20 (PR #201)
It is for the maintainers to decide anyways @archenhttps://github.com/archen. I just thought I mention it here, but I guess you are right, you are not the correct addressee
—
Reply to this email directly, view it on GitHubhttps://github.com/webfactory/ssh-agent/pull/201#issuecomment-1928027879, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AANWYTKPRTNPPBASX57V5GTYSE5GVAVCNFSM6AAAAABCGBNXW6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRYGAZDOOBXHE.
You are receiving this because you commented.Message ID: @.***>

btw @nasmorn, checkout bump to v4 is solved by https://github.com/webfactory/ssh-agent/pull/199


Feedback changes

autopopulate SD feedback
change feedback workflow
fix text area resize bug

Note: need to run migrations & rake task  rake feedbacks: populate_locked_at after deploy.

Look ok.

Waiting for Rspec check


[Map] CTF[SP]MonsterCity

This map requires package "AssaultAdmin" to run - no idea what it is!
From the UEd 2.2 log
Log: New File, Existing Package (Package MyLevel, Package CTF[SP]MonsterCity)
Warning: Failed to load "AssaultAdmin": Can't find file for package "AssaultAdmin"..
Warning: Failed to load "AssaultCTF": Can't find file for package "AssaultAdmin"..
Warning: Failed to load "C:\UnrealTournament\Maps\Current\CTF[SP]MonsterCity.unr": Can't find file for package "AssaultAdmin"..

Hash: dff22c392a49ae26a30d9f53419c49023c8ae5b5
Current name: CTF[SP]MonsterCity

The problem is that AssaultCTF is not included in the package either. So since AssaultAdmin is needed by AssaultCTF, there's no way to indicate on the dependencies list that the file is needed.
I can't seem to find AssaultCTF or AssaultAdmin included in anything in the archive at the moment...

I see - a bit surprising that AssaultCTF isn't included with any other [SP] map.  I have it and must have got it from Sniper's Paradise (where the [SP] maps come from).  Less surprised about AssaultAdmin as I couldn't find it on Sniper's Paradise anywhere - a pity really as it means this map can't be played or edited.


tidal-hifi: 5.15.0 -> 5.16.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/Mastermindzh/tidal-hifi/releases.
meta.description for tidal-hifi is: Web version of Tidal running in electron with hifi support thanks to widevine
meta.homepage for tidal-hifi is: https://github.com/Mastermindzh/tidal-hifi
meta.changelog for tidal-hifi is: https://github.com/Mastermindzh/tidal-hifi/releases/tag/5.16.0
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 5.16.0 with grep in /nix/store/wv9s6qksmlyarn9c8hawwzdflj0klpnq-tidal-hifi-5.16.0
found 5.16.0 in filename of file in /nix/store/wv9s6qksmlyarn9c8hawwzdflj0klpnq-tidal-hifi-5.16.0

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

tidal-hifi

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/wv9s6qksmlyarn9c8hawwzdflj0klpnq-tidal-hifi-5.16.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A tidal-hifi https://github.com/r-ryantm/nixpkgs/archive/0e022141f4b2ad06753148d82876ec2b11677cb1.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/0e022141f4b2ad06753148d82876ec2b11677cb1#tidal-hifi

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/wv9s6qksmlyarn9c8hawwzdflj0klpnq-tidal-hifi-5.16.0
ls -la /nix/store/wv9s6qksmlyarn9c8hawwzdflj0klpnq-tidal-hifi-5.16.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

tidal-hifi

Maintainer pings
cc @qbit @spikespaz for testing.

Add a :+1: reaction to pull requests you find important.

Perhaps just me, but could someone see if you get
(tidal-hifi:1020013): Gtk-WARNING **: 01:58:28.103: Could not load a pixbuf from icon theme.
This may indicate that pixbuf loaders or the mime database could not be found.

Perhaps just me, but could someone see if you get
(tidal-hifi:1020013): Gtk-WARNING **: 01:58:28.103: Could not load a pixbuf from icon theme.
This may indicate that pixbuf loaders or the mime database could not be found.

Yeah I can confirm this is happening to me as well.
(tidal-hifi:319142): Gtk-WARNING **: 01:59:47.158: Could not load a pixbuf from icon theme.
This may indicate that pixbuf loaders or the mime database could not be found.
[319142:0822/015947.225611:ERROR:object_proxy.cc(577)] Failed to call method: org.freedesktop.DBus.Properties.Get: object_path= /org/freedesktop/portal/desktop: org.freedesktop.DBus.Error.InvalidArgs: No such interface “org.freedesktop.portal.FileChooser”
[319142:0822/015947.225637:ERROR:select_file_dialog_linux_portal.cc(285)] Failed to read portal version property


Why does Page Scattering create multiple output files?

Hello,
I'm curious about about the concept of "page scattering" and setting
min_out_tasks to 1 in order to prevent multiple output files from being
creating.
My team wants to work with a single ouput file, so we are tempted to set
min_out_tasks to 1 otherwise we have to work with multiple output files.
However we're uncertain about what the consequences are
of doing this?  Are we reducing the Embulk's performance if we limit the tasks
to 1? Or is there some other consequence I'm not considering?
Thank you!

It runs on several threads like :

Thread 1: [input] => [filters] => [output]
Thread 2: [input] => [filters] => [output]
...

Then it's natural that multiple files are created. Otherwise, it has to serialize it to one file, and it kills the benefits of multithreads. It's the nature of Embulk, and multithreads.
Yes, min_out_tasks: 1 makes a single output, but of course, you're killing the benefits of multithreads.

Thank you for the explanation!


[fixed] null query matches missing value

Right now there's no way in isActive to require that a given query key is missing in the currently active query.
This makes null values on the specified query match absent or otherwise null-y values on the active query.

Force push just included the update to the isActive API docs.

Never mind - this approach won't work. The default query stringification is (probably justifiably) going to stringify the null values as empty strings if this is used with a <Link>, and that's going to lead to all sorts of weirdness.

Oops - just remembered that undefined was a thing. Let me know if this is reasonable.

Does anybody have time to take a look at this? I think this is ready to go now on my side.
I think it makes sense to be able to require a query parameter to be missing for isActive, and I think this is a reasonable API for doing so.

looks solid to me! good job @taion


Node.js : YQL is not working

Hi--,
there is a strange behavior with Azure Cloud Functions when using the YQL package of Node.js like so:
var YQL = require('yql');
module.exports(context, myTimer) {
   new YQL.exec('some query', function (response) {
     // this code is never reached, function never terminates
     context.done();
   });
}

If I execute the same code through the web-based console of the Azure portal, everything works fine. Any clues why this is happening? Thanks!

Did you add the package to your node_modules folder?

Yes I did. I've traced it down to one particular call within the YQL library: something seems to go wrong when YQL is doing a HTTP(s) request via the module 'request'. This is where the function gets stuck for some reason.
As I said, if I invoke the same code on the KUDU console (via npm test -> I've written a unit test that also executes YQL.exec) everything works fine. Strange, isn't it?

Yes I did. I've traced it down to one particular call within the YQL library: something seems to go wrong when YQL is doing a HTTP(s) request via the module 'request'. This is where the function gets stuck for some reason.
As I said, if I invoke the same code on the KUDU console (via npm test -> I've written a unit test that also executes YQL.exec) everything works fine. Strange, isn't it?

We think we have a fix for this. We'll test against it soon.

@christopheranderson What was the fix for this?

PR #486 works around this issue (see #241 for more details). If you want to get things working before this change is deployed to production, you can add this to the your code before you make the call to YQL: process.nextTick = global.setImmediate;


CMake: Remove trailing arguments for mpicc with GASNet mpi conduit

@junghans I think this should work for you now.  Please test to confirm.

Works: https://travis-ci.org/losalamos/flecsi-third-party/builds/174119603
@streichler: please merge!


[Bug] RedPajama-INCITE-Chat-3B-v1-q4f16_1 crashes on iOS when using iPhone 15

🐛 Bug
RedPajama-INCITE-Chat-3B-v1-q4f16_1 crashes on iOS when using iPhone 15 (Non Pro)
To Reproduce
Steps to reproduce the behavior:

Download the MLCChat app on an iPhone 15 (Non Pro)
Download the RedPajama-INCITE-Chat-3B-v1-q4f16_1 Model
Enter any text and send.
The app crashes

Running the iPad app from Xcode shows that memory usage goes to 2G after loading the model, but to 3GB when the first prompt is entered.

I observe the same on an iPhone 11 Pro with iOS 17.3 and 94 GB available (256 GB total) memory.


Question on Sampling Strategy

Hi,
I have a doubt related to the sampling strategy used in Photometric and RGB Loss

Photometric losses require patches to calculate the NCC.
Are you obtaining the patches by rendering only patches... and using the rendered values for calculating both NCC and RGB loss ?
OR
Are you rendering random pixels (which you use for RGB loss)
and then take patches only around those pixels which intersect the object, and the RGB values of the patch comes from the reference image (and not by rendering the remaining pixels) ?

Hi, @anuveshkumar, we use the later strategy to compute RGB loss and NCC scores.

Thank You. That answers my question.


jax fft padding or n error

Hi,
I try to use the jax fft function with an integer for the n argument (a padding on my time series), as follows:
jax.numpy.fft.fft(a,n=256, axis=1)
but I get the following error: NotImplementedError: jax.numpy.fft.fft only supports s=None, got 256
Thanks.

As mentioned in the error message, this functionality is not yet implemented in Jax. You could work around this by manually padding your input before calling the fft function.

Ok, I was not sure. Sorry and thank you.

Something like this should be equivalent for a 1D array, if the length of a is shorter than n:
jnp.fft.fft(jnp.pad(a, [(0, 0), (0, 256 - a.shape[1])]))


eslint proptype enforcement

Purpose
Clean code
What
Added eslint, clean up code, added proptypes to all files
How to Test
npm run lint
What to Check
Check all the files, proptypes

Nice work


Defichain light wallet error - frozen Loan

Hi guys, I'm Andonis from Germany, I need some help with the light wallet. I set up my first Vault and took a Loan (500 DUSD). It is displayed correctly in the Vault itself, on the Defichain Explorer / Scan and also in Defi-income website. But the loan is not displayed in the balance section of the light wallet? I can not use the loan at all. It has been days now and it still is not displayed in the balance section. It is also strange that when I try to repay the loan and click on the button repayment, I am shown that supposedly no DUSD are available to repay... It is like frozen, I have already deleted the app several times and reinstalled, wallet again imported.... all without success. The exact same phenomenon is on the iPad. Thus, I now pay interest but can not use my loan at all....  I have then again 1 DUSD additionally minted, this is actually correctly displayed in the balance section and can also give it back. Only my first 500 DUSD remain as frozen! I have already tried to clarify the issue in several forums on Telegram, but no one had a purposeful idea. Maybe one of you has an idea? Am grateful for any tip

Hi @hellasXpress can you provide us your address so we can check it on DeFiScan?

Hi @thedoublejay here is my Vault adress:
cf6a03fd2194d21276a76a72f61afa2047c7a870a752b9e972839006f0a0fb82
do you need also my wallet adress?
thanks for helping

Hi @hellasXpress, on one of your transactions you have swapped your DUSD to DFI. Now, the DUSD you loaned is currently swapped to DFI.

Hi @thedoublejay ok but what does that mean regarding my problem, sorry but I am not so well versed with the connections in the wallet... I still don't see my 500 DUSD not in my balance section

@thedoublejay Does this mean that my 500 DUSD is now part of my DFI in the balance section?

@thedoublejay Does this mean that my 500 DUSD is now part of my DFI in the balance section?

Yes

@thedoublejay Ah , ok, then that was not intended and was my mistake ... sorry I guess I can make the whole thing undone in which I swap the proportional DFI back into DUSD and feed my loan again ?! Anyway, thanks for your help and my enlightenment ;) All the best, Andonis

No worries, will be closing this issue now. Have a nice day!


Dynamically alter behaviour when on single node cluster

I noticed that when on a databricks single node cluster, that after running:
client = dask_databricks.get_client()

Dask calls will just hang - this makes sense since there is only a driver and no worker nodes, so nothing to actually do the work.
It would be really nice if:

After a while of waiting, the cluster threw an error because no workers are available (I'm guessing this would belong in dask/distributed proper though?)
dask-databricks had some kind of clever behaviour to use multi-process rather than distributed workflows in on a single cluster node

I think the obvious way to implement the second would be to have some behaviour change based on some kind of spark.conf variable (although I haven't seen an obvious candidate).
@jacobtomlinson, any thoughts? Would this be a handy feature or is it a bit of an overcomplication? (I can't figure thinking it through if its actually worth the additional complexity - it'd probably be handy if someone had a notebook that they wanted to run over multiple different cluster types, but the use case for dask-databricks isn't really there?)

I think it would be good to try and detect a single node cluster in get_client(). However I think I would err on the side of throwing an exception rather than doing something magic under the hood. Then you could catch it and intentionally choose what to do.
Perhaps something like this:
import dask_databricks
from dask.distributed import LocalCluster

try:
    client = dask_databricks.get_client()
except dask_databricks.SingleNodeException:
    client = LocalCluster().get_client()

I like the idea of throwing an exception a lot! I think that solves the main issue which is that if you're not familiar with dask you might get confused/frustrated with it just hanging, without realising why that's happening.
I'll have a look at what information is available from spark.conf and hopefully put in a PR

Sounds great. I look forward to reviewing it. There may also be uneful information just stored in environment variables too.

This is now available in 0.3.2


Fix various issues with colliders

What changed?

Configure sensor colliders imported from glb correctly
Remove the useless RigidBody from hands
Set the user_data field on imported colliders correctly
Update the GlobalTransform in hands to avoid a one frame delay

Merging in as issues have been resolved


Failed to import - rxjs

Failing module

GitHub: https://github.com/reactivex/rxjs
npm: https://www.npmjs.com/package/rxjs

Error message
<EMAIL_ADDRESS>/* esm.sh - error */
throw new Error("[esm.sh] " + "resovleESModule: open<EMAIL_ADDRESS>no such file or directory");
export default null;

<EMAIL_ADDRESS>/* esm.sh - esbuild<EMAIL_ADDRESS>es2022 production */
import{audit as e}from"/v120/rxjs@7.8.1/es2022/internal/operators/audit.js";import{auditTime as m}from"/v120/rxjs@7.8.1/es2022/internal/operators/auditTime.js";import{buffer as f}from"/v120/rxjs@7.8.1/es2022/internal/operators/buffer.js";import{bufferCount as i}from"/v120/rxjs@7.8.1/es2022/internal/operators/bufferCount.js";import{bufferTime as n}from"/v120/rxjs@7.8.1/es2022/internal/operators/bufferTime.js";import{bufferToggle as s}from"/v120/rxjs@7.8.1/es2022/internal/operators/bufferToggle.js";import{bufferWhen as h}from"/v120/rxjs@7.8.1/es2022/internal/operators/bufferWhen.js";import{catchError as d}from"/v120/rxjs@7.8.1/es2022/internal/operators/catchError.js";import{combineAll as w}from"/v120/rxjs@7.8.1/es2022/internal/operators/combineAll.js";import{combineLatestAll as W}from"/v120/rxjs@7.8.1/es2022/internal/operators/combineLatestAll.js";import{combineLatest as T}from"/v120/rxjs@7.8.1/es2022/internal/operators/combineLatest.js";import{combineLatestWith as A}from"/v120/rxjs@7.8.1/es2022/internal/operators/combineLatestWith.js";import{concat as E}from"/v120/rxjs@7.8.1/es2022/internal/operators/concat.js";import{concatAll as z}from"/v120/rxjs@7.8.1/es2022/internal/operators/concatAll.js";import{concatMap as v}from"/v120/rxjs@7.8.1/es2022/internal/operators/concatMap.js";import{concatMapTo as U}from"/v120/rxjs@7.8.1/es2022/internal/operators/concatMapTo.js";import{concatWith as q}from"/v120/rxjs@7.8.1/es2022/internal/operators/concatWith.js";import{connect as O}from"/v120/rxjs@7.8.1/es2022/internal/operators/connect.js";import{count as F}from"/v120/rxjs@7.8.1/es2022/internal/operators/count.js";import{debounce as N}from"/v120/rxjs@7.8.1/es2022/internal/operators/debounce.js";import{debounceTime as D}from"/v120/rxjs@7.8.1/es2022/internal/operators/debounceTime.js";import{defaultIfEmpty as H}from"/v120/rxjs@7.8.1/es2022/internal/operators/defaultIfEmpty.js";import{delay as P}from"/v120/rxjs@7.8.1/es2022/internal/operators/delay.js";import{delayWhen as V}from"/v120/rxjs@7.8.1/es2022/internal/operators/delayWhen.js";import{dematerialize as Y}from"/v120/rxjs@7.8.1/es2022/internal/operators/dematerialize.js";import{distinct as _}from"/v120/rxjs@7.8.1/es2022/internal/operators/distinct.js";import{distinctUntilChanged as rr}from"/v120/rxjs@7.8.1/es2022/internal/operators/distinctUntilChanged.js";import{distinctUntilKeyChanged as er}from"/v120/rxjs@7.8.1/es2022/internal/operators/distinctUntilKeyChanged.js";import{elementAt as mr}from"/v120/rxjs@7.8.1/es2022/internal/operators/elementAt.js";import{endWith as fr}from"/v120/rxjs@7.8.1/es2022/internal/operators/endWith.js";import{every as ir}from"/v120/rxjs@7.8.1/es2022/internal/operators/every.js";import{exhaust as nr}from"/v120/rxjs@7.8.1/es2022/internal/operators/exhaust.js";import{exhaustAll as sr}from"/v120/rxjs@7.8.1/es2022/internal/operators/exhaustAll.js";import{exhaustMap as hr}from"/v120/rxjs@7.8.1/es2022/internal/operators/exhaustMap.js";import{expand as dr}from"/v120/rxjs@7.8.1/es2022/internal/operators/expand.js";import{filter as wr}from"/v120/rxjs@7.8.1/es2022/internal/operators/filter.js";import{finalize as Wr}from"/v120/rxjs@7.8.1/es2022/internal/operators/finalize.js";import{find as Tr}from"/v120/rxjs@7.8.1/es2022/internal/operators/find.js";import{findIndex as Ar}from"/v120/rxjs@7.8.1/es2022/internal/operators/findIndex.js";import{first as Er}from"/v120/rxjs@7.8.1/es2022/internal/operators/first.js";import{groupBy as zr}from"/v120/rxjs@7.8.1/es2022/internal/operators/groupBy.js";import{ignoreElements as vr}from"/v120/rxjs@7.8.1/es2022/internal/operators/ignoreElements.js";import{isEmpty as Ur}from"/v120/rxjs@7.8.1/es2022/internal/operators/isEmpty.js";import{last as qr}from"/v120/rxjs@7.8.1/es2022/internal/operators/last.js";import{map as Or}from"/v120/rxjs@7.8.1/es2022/internal/operators/map.js";import{mapTo as Fr}from"/v120/rxjs@7.8.1/es2022/internal/operators/mapTo.js";import{materialize as Nr}from"/v120/rxjs@7.8.1/es2022/internal/operators/materialize.js";import{max as Dr}from"/v120/rxjs@7.8.1/es2022/internal/operators/max.js";import{merge as Hr}from"/v120/rxjs@7.8.1/es2022/internal/operators/merge.js";import{mergeAll as Pr}from"/v120/rxjs@7.8.1/es2022/internal/operators/mergeAll.js";import{flatMap as Vr}from"/v120/rxjs@7.8.1/es2022/internal/operators/flatMap.js";import{mergeMap as Yr}from"/v120/rxjs@7.8.1/es2022/internal/operators/mergeMap.js";import{mergeMapTo as _r}from"/v120/rxjs@7.8.1/es2022/internal/operators/mergeMapTo.js";import{mergeScan as ro}from"/v120/rxjs@7.8.1/es2022/internal/operators/mergeScan.js";import{mergeWith as eo}from"/v120/rxjs@7.8.1/es2022/internal/operators/mergeWith.js";import{min as mo}from"/v120/rxjs@7.8.1/es2022/internal/operators/min.js";import{multicast as fo}from"/v120/rxjs@7.8.1/es2022/internal/operators/multicast.js";import{observeOn as io}from"/v120/rxjs@7.8.1/es2022/internal/operators/observeOn.js";import{onErrorResumeNext as no}from"/v120/rxjs@7.8.1/es2022/internal/operators/onErrorResumeNextWith.js";import{pairwise as so}from"/v120/rxjs@7.8.1/es2022/internal/operators/pairwise.js";import{partition as ho}from"/v120/rxjs@7.8.1/es2022/internal/operators/partition.js";import{pluck as bo}from"/v120/rxjs@7.8.1/es2022/internal/operators/pluck.js";import{publish as go}from"/v120/rxjs@7.8.1/es2022/internal/operators/publish.js";import{publishBehavior as yo}from"/v120/rxjs@7.8.1/es2022/internal/operators/publishBehavior.js";import{publishLast as ko}from"/v120/rxjs@7.8.1/es2022/internal/operators/publishLast.js";import{publishReplay as Mo}from"/v120/rxjs@7.8.1/es2022/internal/operators/publishReplay.js";import{race as Lo}from"/v120/rxjs@7.8.1/es2022/internal/operators/race.js";import{raceWith as Co}from"/v120/rxjs@7.8.1/es2022/internal/operators/raceWith.js";import{reduce as Io}from"/v120/rxjs@7.8.1/es2022/internal/operators/reduce.js";import{repeat as Ro}from"/v120/rxjs@7.8.1/es2022/internal/operators/repeat.js";import{repeatWhen as Bo}from"/v120/rxjs@7.8.1/es2022/internal/operators/repeatWhen.js";import{retry as So}from"/v120/rxjs@7.8.1/es2022/internal/operators/retry.js";import{retryWhen as Ko}from"/v120/rxjs@7.8.1/es2022/internal/operators/retryWhen.js";import{refCount as jo}from"/v120/rxjs@7.8.1/es2022/internal/operators/refCount.js";import{sample as Go}from"/v120/rxjs@7.8.1/es2022/internal/operators/sample.js";import{sampleTime as Jo}from"/v120/rxjs@7.8.1/es2022/internal/operators/sampleTime.js";import{scan as Qo}from"/v120/rxjs@7.8.1/es2022/internal/operators/scan.js";import{sequenceEqual as Xo}from"/v120/rxjs@7.8.1/es2022/internal/operators/sequenceEqual.js";import{share as Zo}from"/v120/rxjs@7.8.1/es2022/internal/operators/share.js";import{shareReplay as $o}from"/v120/rxjs@7.8.1/es2022/internal/operators/shareReplay.js";import{single as oe}from"/v120/rxjs@7.8.1/es2022/internal/operators/single.js";import{skip as te}from"/v120/rxjs@7.8.1/es2022/internal/operators/skip.js";import{skipLast as pe}from"/v120/rxjs@7.8.1/es2022/internal/operators/skipLast.js";import{skipUntil as xe}from"/v120/rxjs@7.8.1/es2022/internal/operators/skipUntil.js";import{skipWhile as ae}from"/v120/rxjs@7.8.1/es2022/internal/operators/skipWhile.js";import{startWith as le}from"/v120/rxjs@7.8.1/es2022/internal/operators/startWith.js";import{subscribeOn as ce}from"/v120/rxjs@7.8.1/es2022/internal/operators/subscribeOn.js";import{switchAll as ue}from"/v120/rxjs@7.8.1/es2022/internal/operators/switchAll.js";import{switchMap as be}from"/v120/rxjs@7.8.1/es2022/internal/operators/switchMap.js";import{switchMapTo as ge}from"/v120/rxjs@7.8.1/es2022/internal/operators/switchMapTo.js";import{switchScan as ye}from"/v120/rxjs@7.8.1/es2022/internal/operators/switchScan.js";import{take as ke}from"/v120/rxjs@7.8.1/es2022/internal/operators/take.js";import{takeLast as Me}from"/v120/rxjs@7.8.1/es2022/internal/operators/takeLast.js";import{takeUntil as Le}from"/v120/rxjs@7.8.1/es2022/internal/operators/takeUntil.js";import{takeWhile as Ce}from"/v120/rxjs@7.8.1/es2022/internal/operators/takeWhile.js";import{tap as Ie}from"/v120/rxjs@7.8.1/es2022/internal/operators/tap.js";import{throttle as Re}from"/v120/rxjs@7.8.1/es2022/internal/operators/throttle.js";import{throttleTime as Be}from"/v120/rxjs@7.8.1/es2022/internal/operators/throttleTime.js";import{throwIfEmpty as Se}from"/v120/rxjs@7.8.1/es2022/internal/operators/throwIfEmpty.js";import{timeInterval as Ke}from"/v120/rxjs@7.8.1/es2022/internal/operators/timeInterval.js";import{timeout as je}from"/v120/rxjs@7.8.1/es2022/internal/operators/timeout.js";import{timeoutWith as Ge}from"/v120/rxjs@7.8.1/es2022/internal/operators/timeoutWith.js";import{timestamp as Je}from"/v120/rxjs@7.8.1/es2022/internal/operators/timestamp.js";import{toArray as Qe}from"/v120/rxjs@7.8.1/es2022/internal/operators/toArray.js";import{window as Xe}from"/v120/rxjs@7.8.1/es2022/internal/operators/window.js";import{windowCount as Ze}from"/v120/rxjs@7.8.1/es2022/internal/operators/windowCount.js";import{windowTime as $e}from"/v120/rxjs@7.8.1/es2022/internal/operators/windowTime.js";import{windowToggle as ot}from"/v120/rxjs@7.8.1/es2022/internal/operators/windowToggle.js";import{windowWhen as tt}from"/v120/rxjs@7.8.1/es2022/internal/operators/windowWhen.js";import{withLatestFrom as pt}from"/v120/rxjs@7.8.1/es2022/internal/operators/withLatestFrom.js";import{zip as xt}from"/v120/rxjs@7.8.1/es2022/internal/operators/zip.js";import{zipAll as at}from"/v120/rxjs@7.8.1/es2022/internal/operators/zipAll.js";import{zipWith as lt}from"/v120/rxjs@7.8.1/es2022/internal/operators/zipWith.js";export{e as audit,m as auditTime,f as buffer,i as bufferCount,n as bufferTime,s as bufferToggle,h as bufferWhen,d as catchError,w as combineAll,T as combineLatest,W as combineLatestAll,A as combineLatestWith,E as concat,z as concatAll,v as concatMap,U as concatMapTo,q as concatWith,O as connect,F as count,N as debounce,D as debounceTime,H as defaultIfEmpty,P as delay,V as delayWhen,Y as dematerialize,_ as distinct,rr as distinctUntilChanged,er as distinctUntilKeyChanged,mr as elementAt,fr as endWith,ir as every,nr as exhaust,sr as exhaustAll,hr as exhaustMap,dr as expand,wr as filter,Wr as finalize,Tr as find,Ar as findIndex,Er as first,Vr as flatMap,zr as groupBy,vr as ignoreElements,Ur as isEmpty,qr as last,Or as map,Fr as mapTo,Nr as materialize,Dr as max,Hr as merge,Pr as mergeAll,Yr as mergeMap,_r as mergeMapTo,ro as mergeScan,eo as mergeWith,mo as min,fo as multicast,io as observeOn,no as onErrorResumeNext,so as pairwise,ho as partition,bo as pluck,go as publish,yo as publishBehavior,ko as publishLast,Mo as publishReplay,Lo as race,Co as raceWith,Io as reduce,jo as refCount,Ro as repeat,Bo as repeatWhen,So as retry,Ko as retryWhen,Go as sample,Jo as sampleTime,Qo as scan,Xo as sequenceEqual,Zo as share,$o as shareReplay,oe as single,te as skip,pe as skipLast,xe as skipUntil,ae as skipWhile,le as startWith,ce as subscribeOn,ue as switchAll,be as switchMap,ge as switchMapTo,ye as switchScan,ke as take,Me as takeLast,Le as takeUntil,Ce as takeWhile,Ie as tap,Re as throttle,Be as throttleTime,Se as throwIfEmpty,Ke as timeInterval,je as timeout,Ge as timeoutWith,Je as timestamp,Qe as toArray,Xe as window,Ze as windowCount,$e as windowTime,ot as windowToggle,tt as windowWhen,pt as withLatestFrom,xt as zip,at as zipAll,lt as zipWith};
//# sourceMappingURL=operators.js.map

Hi, we've just found similar behaviour with v121 with selected files in the dom-helpers and react-transition-group packages. As a workaround we can pin to v119.
We can reproduce this in the esm.sh playground by either:
import { Button } from<EMAIL_ADDRESS>or
import { Button } from<EMAIL_ADDRESS>react-bootstrap triggers dependency imports in the browser including the following files which fail with error 500, reporting same error as above:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>reactstrap also fails to import different files from dom-helpers:
<EMAIL_ADDRESS><EMAIL_ADDRESS>As a workaround then pinning to version v119 works:
<EMAIL_ADDRESS><EMAIL_ADDRESS>

@mradamwalters could you confirm whether one of your URLs being requested by the main URL has .mjs file extension?
If so, it might be the bug found at #634 . I found out that mjs file extension has no content being served by esm.sh

Hi @renhiyama - thank you. There are a few .mjs files called in the dependency chain but they do seem to be working ok in this case e.g.:
<EMAIL_ADDRESS><EMAIL_ADDRESS>For these packages it seems to be certain files within a package not working in v121 (and sometimes working/not working in v120 either).

For your case, I would suggest looking which file's code is empty; it might be tiresome but for me I've implemented a custom downloader for urlimports so I can add a few checks to check whether a file is empty.

I just found that even v119 causes this issue sometimes; for me<EMAIL_ADDRESS>content is empty ...

Looks like random mjs files from older versions don't respond too. Even v118 worked fine before; now it doesn't<EMAIL_ADDRESS>

@ije on a sidenote, there's a typo at resolveESModule being typed as resovleESModule. Hopefully you can find it in your code.

@ije bump for this msg incase you didnt notice.

@ije on a sidenote, there's a typo at resolveESModule being typed as resovleESModule. Hopefully you can find it in your code.

@ije bump for this msg incase you didnt notice.

thanks i noticed it

@BlackGlory @renhiyama this was broken in v120, just fixed and added a testing for it. thanks for reporting!


[OCP] Create new flavors with larger disk

Similar to https://github.com/redhat-appstudio/infra-deployments/pull/5050

@hugares looks like we need a /ok-to-test as well?

/ok-to-test

@hugares looks like we need a /ok-to-test as well?

you are right, I will blame late hour for missing this :)

no worries - thanks as always Hugo!

/retest

@hugares not sure why "Red Hat Konflux" is not running :thinking:

@hugares not sure why "Red Hat Konflux" is not running 🤔

this is a known issue when someone outside of the org contribute a change. We will need to find out one day how to get this running but for now I will force merge

Thanks @hugares. If it makes things easier, I can get myself added into the org.

Thanks @hugares. If it makes things easier, I can get myself added into the org.

Invite sent :)


Implement trackpad gesture panning in the TileSet editor

Godot version
4.0.beta8 [45cac42c0]
System information
macOS 13.0.1 (22A400)
Issue description
I had similar issues on Godot 3.5, etc but I am hoping we can get the Mac trackpad to work properly within the editor. Some things work okay, but the areas I notice it the most is inside the tileset editor where on a normal mouse you can move around with MMB and zoom in with the scroll wheel. I am hoping we can allow the obvious gestures to work on the track page (pinch to zoom, swipe to pan). I have also noticed similar issues within the Animation editors.
If none of the development team has a MacBook or trackpad available, I would be willing to provide any testing, etc required or possibly even send out a trackpad for them to use. I would also be interested in helping out with a PR myself if someone could point me in the right direction as I have never worked within the Godot 4 core.
Thanks!
Steps to reproduce
Go to a tiletset on a Mac computer or laptop with an Apple trackpad. Try to pan or zoom inside the tile editor pane.
Minimal reproduction project
Should work inside any project.

The animation editor part is already being tracked in https://github.com/godotengine/godot/issues/38237. See also https://github.com/godotengine/godot/issues/28149 and https://github.com/godotengine/godot/issues/42747.

Related to #58014 I think.
It has a linked PR that you could try.

Thank you everyone! This is a great place for me to start on contributing back to the community. I'll get on those as I get time. Thanks again for all your amazing and hard work on the project.

@iamjoeysox Bump :slightly_smiling_face: Can you test https://github.com/godotengine/godot/pull/62272?


problem dpad don't with TabBar

Steps to Reproduce

Run flutter create bug.
Update the files as follows: ...
...

Expected results:
Actual results:

Logs

Hi @syst-rootzz ,
When opening an issue, please try to provide all or some of the information indicated in the template.

Issue Template
If you are looking for support, please check out our documentation
or consider asking a question on Stack Overflow:
- https://flutter.dev/
- https://api.flutter.dev/
- https://stackoverflow.com/questions/tagged/flutter?sort=frequent

If you have found a bug or if our documentation doesn't have an answer
to what you're looking for, then fill our the template below. Please read
our guide to filing a bug first: https://flutter.dev/docs/resources/bug-reports

Steps to Reproduce

Please tell us exactly how to reproduce the problem you are running into.

Please attach a small application (ideally just one main.dart file) that
reproduces the problem. You could use https://gist.github.com/ for this.

If the problem is with your application's rendering, then please attach
a screenshot and explain what the problem is.

1. ...
2. ...
3. ...

Please tell us which target platform(s) the problem occurs (Android / iOS / Web / macOS / Linux / Windows)
Which target OS version, for Web, browser, is the test system running?
Does the problem occur on emulator/simulator as well as on physical devices?

Target Platform:
Target OS version/browser:
Devices:

Run your application with `flutter run --verbose` and attach all the
log output below between the lines with the backticks. If there is an
exception, please see if the error message includes enough information
to explain how to solve the issue.

Run `flutter analyze` and attach any output of that command below.
If there are any analysis errors, try resolving them before filing this issue.

Finally, paste the output of running `flutter doctor -v` here.

Closing this issue, since no information has been provided. If you have any further info for us feel free to write back and I will reopen it.
Thank you.


Change the default structure of the project

web_modules might be confusing.
Something like
/
  /app?
  /layouts
  /modules? /components?
  [/themes?](#508)

Need to think more tho.

I like "components" over "modules"

I like "components" over "web_modules" too

:+1:  for componens.

Do you think a folder like "layouts" is necessary? Since they are just components after all. Not sure if this will help or confuse people.

FYI you can already change this structure, since everything is explicit from the scripts/phenomic.*.js. Nothing is implicit (for now) so you have access to all references and so you can change all paths.

indeed, putting layouts in components makes sense imho.

I think keeping layouts as a seperated folder makes more sense. It's like containers vs components.  But you can put it where you like it. (Phenomic is pretty flexible, and unopinionated)

I will probably introduce this
content/
src/
  /layouts (should we parse this automatically? need to think about this...)
  /components
  routes.js (optional? should this one have defaults too?)
  app.js (optional, should have a default)
scripts/ (optional, should have a default)
  phenomic.*.js
webpack.config.js
package.json

Ref https://github.com/MoOx/phenomic/issues/508#issuecomment-238462889

@MoOx looks good to me. Its pretty much exactly what I did 2 days ago =)

yes, having dedicated layouts folder is better if you want to support auto-discovery so we wont have to declare them in route.js

You can check the new tree here https://github.com/MoOx/phenomic/tree/master/themes/phenomic-theme-base
Feedbacks welcome :)


Integration to Zora

Zora is an Ethereum Virtual Machine (EVM)-compatible Layer 2 blockchain platform designed to efficiently support tokenized digital media and non-fungible tokens (NFTs). A key goal of the Zora Network is to provide NFT creators and buyers with improved scalability, faster transaction speeds, and lower costs. Zksync should also be part of a network that is used to pay on Zora. Zksync should also be integrated into bridge into Zora on Zora network.

It sounds like you're proposing integrating zkSync into the Zora Network to enhance scalability, transaction speed, and reduce costs for NFT creators and buyers. Integrating zkSync, a Layer 2 scaling solution, into the Zora Network could indeed help achieve those goals by offloading transactions from the Ethereum mainnet to a more efficient and cost-effective layer.
By integrating zkSync into the Zora Network, users could benefit from faster transaction speeds and lower fees while still retaining the security guarantees of the Ethereum mainnet. Additionally, integrating a bridge into zkSync would allow users to seamlessly transfer assets between zkSync and the Ethereum mainnet, enhancing interoperability and accessibility for users.

The first paragraph is correct then then second paragraph I mean transfer of asset between Zksync and Zora mainnet

if i can use zksync to mint on zora it will save me more fees. nice one


Attuning AE P2P interfaces

I'm not sure if this is a issue on mekanism's side or AE2's side.
Mekanism is registering the itemstack with AE2 via IMC for P2P attunement.
But apparently when I rclick with the transmitter items, it places it down instead.. or adds its multipart part to the block.

I have no clue- I've never even messed with attunement before.  Your implementation looked correct, though.

Yeah, I opened a similar issue on the AE2 github to see if they might have any ideas. Either I'm not registering it correctly, or its a multipart thing.

Well damn, FMLInterModComms.sendMessage is returning false, and AE2 doesn't even appear to be receiving the IMC messages. @.@

Rui, y u breaking stuff? :P

I'm breaking stuff?

It was a joke. Sorry.

Huh. I don't get the joke though.. Mind explaining? XD

It had to do with your implementation pr. It didn't work thus my joke came into existence. Typically for me, if you see a ":P" at the end, I'm joking.

Have to make things fun every once in a while... gets too serious otherwise.


Be a bit more consistent with annotations in zt chart

revision label doesn't signal intent for ztunnel, since ztunnel doesn't have revisions. Move it to an annotation, where it's a purely informational indicator as to which Istio controlplane rev this ztunnel happens to have been pointed at during install. Otherwise, you have to eyeball the env section and check the Istiod URL, and guess.

Propagate Helm-override annotations to all the zt resources, rather than just the Daemonset.

[ ] Upgrade
[ ] Multi Cluster
[ ] Virtual Machine
[x] Control Plane Revisions

@bleggett: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
6c774316def5a68861366655e1df41e43976745d
link
true
/test release-notes

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Okay, not sure why Env Maintainers got added, Github freaked out for a second there.


Add tylercamp as developer for code dx plugin

Description
Request upload permissions for my account (tylercamp) for releasing recent changes to the Code Dx plugin.
I've been given push permissions on the repository and will request a comment from an existing maintainer (@baffles ) for confirmation.
https://github.com/jenkinsci/codedx-plugin
(Note that pull request uses "tylercamp" which is my Github account name, but is also my Jenkins community account name.)

Submitter checklist for changing permissions

Always

[x] Add link to plugin/component Git repository in description above

For a newly hosted plugin only

[ ] Add link to resolved HOSTING issue in description above

For a new permissions file only

[x] Make sure the file is created in permissions/ directory
[x] artifactId (pom.xml) is used for name (permissions YAML file).
[x] groupId / artifactId (pom.xml) are correctly represented in path (permissions YAML file)
[x] Check that the file is named plugin-${artifactId}.yml for plugins

When adding new uploaders (this includes newly created permissions files)

[x] Make sure to @mention an existing maintainer to confirm the permissions request, if applicable
[x] Use the Jenkins community (LDAP) account name in the YAML file, not the GitHub account name
[x] Make sure to @mention the users being added so their GitHub account names are known if they require GitHub merge access (see below).
[x] All newly added users have logged in to Artifactory at least once

Reviewer checklist (not for requesters!)

[ ] Check this if newly added person also needs to be given merge permission to the GitHub repo (please @ the people/person with their GitHub username in this issue as well). If needed, it can be done using an IRC Bot command
[ ] Check that the $pluginId Developers team has Admin permissions while granting the access.
[ ] In the case of plugin adoption, ensure that the Jenkins Jira default assignee is either removed or changed to the new maintainer.
[ ] If security contacts are changed (this includes add/remove), ping the security officer (currently @daniel-beck) in this pull request. If an email contact is changed, wait for approval from the security officer.

There are IRC Bot commands for it

I approve this request

Weird, that check says my account could not be found but I'm currently signed in using that account. It's referring to https://repo.jenkins-ci.org/ right?

Yes, it is. You can login to repo.jenkins-ci.org with the username tylercamp ?

@slide Yep, I just signed in again

I restarted the build on ci.j.io to see if there was just some weird latency issue with your user account.


Update this project to the newest version

Update this project to the newest version

Update this project to the newest version


added ducomentation setup

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@99c912e). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master       #3   +/-   ##
=========================================
  Coverage          ?   12.89%
=========================================
  Files             ?       11
  Lines             ?     1241
  Branches          ?        0
=========================================
  Hits              ?      160
  Misses            ?     1081
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 99c912e...e10df90. Read the comment docs.


show_urls giving view on permissions

Would it be possible to get the "rights/permissions" for each url ? to view for each URL (+ method?) things like @login_required, @permission_required_or_403, @user_passes_test
possible is both technically (easy to code, readily feasible,...) but also on the principles (does my question make sense ? or is it not really well defined ?)
my context/use case: I would like to be able to have a global view of all open/restricted view of my django project.

Would love to accept a PR for this.
(Think we tried something like this before but the issue is that it's hard to detect the decorators)


Poor throughput in Cilium compared to Calico with iPerf3 after migration

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
We have migrated the CNI of our Calico clusters to Cilium.
For the latter, we set up iPerf3 tests before the migration to compare performance once it was complete.
On all migrated clusters, iPerf3 throughput dropped drastically after the switch to Cilium.
I've seen 3 old issues closed without response that addressed this subject in the past.

https://github.com/cilium/cilium/issues/18169
https://github.com/cilium/cilium/issues/28705
https://github.com/cilium/cilium/issues/17342

Like these previous tickets, I can't understand this big difference in throughput when Cilium is supposed to perform as well or even better
according to

https://cilium.io/blog/2021/05/11/cni-benchmark/#throughput
https://docs.cilium.io/en/stable/operations/performance/tuning/#performance-tuning

On these graphs, we can see the drastic drop once the migration is complete.
   Summary Results:
[ ID] Interval           Transfer     Bitrate
[  5] (sender statistics not available)
[  5]   0.00-10.00  sec  3.99 GBytes  3.43 Gbits/sec                  receiver
rcv_tcp_congestion cubic
iperf 3.12
Linux iperf-server-64b4468f4-9gprj 5.15.0-107-generic #117-Ubuntu SMP Fri Apr 26 12:26:49 UTC 2024 x86_64

Anything else?
Cilium is installed with KubeproxReplacement: true, bfp-masquerade enabled, l7proxy enabled and hubble enabled ...
kubectl get cm -o yaml cilium-config -n kube-system | egrep "enable-bpf-masquerade|enable-ipv4-egress-gateway|kube-proxy-replacement|enable-l7-proxy|tunnel"
  enable-bpf-masquerade: "true"
  enable-l7-proxy: "true"
  kube-proxy-replacement: "true"
  kube-proxy-replacement-healthz-bind-address: <IP_ADDRESS>:10256
  routing-mode: tunnel
  tunnel-port: "8473"
  tunnel-protocol: vxlan
  enable-hubble: "true"

Cilium Users Document

[X] Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Would you be able to run the test after disabling Hubble and performing a rollout of the cilium-agent Daemonset?

Would you be able to run the test after disabling Hubble and performing a rollout of the cilium-agent Daemonset?

Of course.
I'll disable it tomorrow, leave the cronjob iperf3 client running over the weekend and on Monday I'll add a new screenshot

Also, if you could check with 1.14 and 1.16 as well, that would be interesting to see if there's a regression of some sort.

I wonder, could you share more information about your setup?
Do you use a cloud provider or on-prem?
What are the nodes on which the server/client is running on?
Could you ensure that client/server pods are running on new nodes and not migrated nodes? Just to make sure there is no lingering state of Calico on the node hindering performance.
Could you share iperf3 server/cronjob yamls? Just to make our replication a bit easier. I am guessing traffic to server goes through k8s service?
I'm asking due to two reasons:

to figure out if we can replicate that easily
Also, did iperf3 server change nodes (I'm guessing probably yes)? Usually, different nodes in cloud-provider environments can have different traffic throughput.

Would you be able to run the test after disabling Hubble and performing a rollout of the cilium-agent Daemonset?

Of course. I'll disable it tomorrow, leave the cronjob iperf3 client running over the weekend and on Monday/Tuesday I'll add a new screenshot

@learnitall
Here are the new screenshots: No change in performance.
(Not very surprised, on another cluster, weeks before, I had disabled the hubble for 1 hour and it hadn't changed anything)

hubble disabled enable-hubble=false on Friday 12/07 at 10:55 am
rollout restart ds/cilium  at 10:56am and cilium-operator at 10:59am

Also, if you could check with 1.14 and 1.16 as well, that would be interesting to see if there's a regression of some sort.

I'll be able to test 1.16 as soon as we've migrated a cluster to this version.
However, I won't be able to test 1.14.
No cluster available with this version

Thanks for all the extra detail @cdtzabra, it's super helpful. I chatted with @marseel about this offline. Both of us are limited in our understanding of vSphere and the details of Calico's installation and operation. There are some things that would be worth sanity checking over though before we start digging into Cilium's datapath.
One thing that stood out in your previous comments was:

Server and client are running on the same cluster and can be scheduled on any node where (customer/app) workloads are allowed. All node

We'd like to confirm that the characteristics of the node that the iperf3 server and client are deployed to aren't making an impact. We are wondering: did the installation of Cilium impact the results of the iperf3 workload, or did the rescheduling of pods in the cluster impact them?
Couple of questions to help dig into this:

Would you be able to determine the characteristics of the node that the iperf3 server was deployed to before/after migrating to Cilium? Is there a difference in the type of NIC, amount of memory, the amount of CPU, or the type of customer/app workloads that are deployed on the node?
Would you also be able to do the same for the iperf3 clients? Is there a difference in the general pattern of nodes that the clients are deployed to before/after migrating to Cilium?

Would you be able to determine the characteristics of the node that the iperf3 server was deployed to before/after migrating to Cilium? Is there a difference in the type of NIC, amount of memory, the amount of CPU, or the type of customer/app workloads that are deployed on the node?
Would you also be able to do the same for the iperf3 clients? Is there a difference in the general pattern of nodes that the clients are deployed to before/after migrating to Cilium?

there's no difference.
The nodes are exactly the same before and after migration. Nothing has changed.
To re-precise, we only migrated from CNI Calico to CNI Cilium on our existing clusters. So the clusters are still the same with the same nodes, the nodes haven't changed  (the workloads run on the same nodes before and after the migration).
The CNI migration was based on this doc: https://docs.cilium.io/en/stable/installation/k8s-install-migration.
I just transposed the steps into a Makefile to limit copy-paste.

Ok, thank you for confirming. Would you be able to share the details of the configuration of your previous Calico installation? This will help me replicate the performance drop and do some debugging.

Ok, thank you for confirming. Would you be able to share the details of the configuration of your previous Calico installation? This will help me replicate the performance drop and do some debugging.

AFter creating the cluster with kubeadm, we deployed calico with
kubectl -n kube-system apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.24.5/manifests/calico.yaml

Hi there, @cdtzabra. I'm the author of that migration document. One of the important steps in calico migration is using so-called legacy host routing, which is necessary for hybrid mode. Once migration is complete, you can re-enable eBPF host routing. Have you done this? It may account for the performance cost.

Hi there, @cdtzabra. I'm the author of that migration document. One of the important steps in calico migration is using so-called legacy host routing, which is necessary for hybrid mode. Once migration is complete, you can re-enable eBPF host routing. Have you done this? It may account for the performance cost.

Hi @squeed
YES, we have enabled eBPF host routing
In the output below you can see that Host Routing = BPF.
k -n kube-system exec -it ds/cilium -c cilium-agent -- cilium status

KVStore:                 Ok   Disabled
Kubernetes:              Ok   1.28 (v1.28.4) [linux/amd64]
Kubernetes APIs:         ["EndpointSliceOrEndpoint", "cilium/v2::CiliumClusterwideNetworkPolicy", "cilium/v2::CiliumEndpoint", "cilium/v2::CiliumNetworkPolicy", "cilium/v2::CiliumNode", "cilium/v2alpha1::CiliumCIDRGroup", "core/v1::Namespace", "core/v1::Pods", "core/v1::Service", "networking.k8s.io/v1::NetworkPolicy"]
KubeProxyReplacement:    True   [ens192   <IP_ADDRESS> fe80::250:56ff:fe92:e8a3 (Direct Routing)]
Host firewall:           Disabled
SRv6:                    Disabled
CNI Chaining:            none
CNI Config file:         successfully wrote CNI configuration file to /host/etc/cni/net.d/05-cilium.conflist
Cilium:                  Ok   1.15.6 (v1.15.6-a09e05e6)
NodeMonitor:             Listening for events on 128 CPUs with 64x4096 of shared memory
Cilium health daemon:    Ok
IPAM:                    IPv4: 4/254 allocated from <IP_ADDRESS>/24,
IPv4 BIG TCP:            Disabled
IPv6 BIG TCP:            Disabled
BandwidthManager:        Disabled
Host Routing:            BPF
Masquerading:            BPF   [ens192]   <IP_ADDRESS>/24 [IPv4: Enabled, IPv6: Disabled]
Controller Status:       33/33 healthy
Proxy Status:            OK, ip <IP_ADDRESS>, 0 redirects active on ports 10000-20000, Envoy: embedded
Global Identity Range:   min 256, max 65535
Hubble:                  Ok                Current/Max Flows: 4095/4095 (100.00%), Flows/s: 90.24   Metrics: Ok
Encryption:              Disabled
Cluster health:          16/16 reachable   (2024-07-26T15:53:22Z)
Modules Health:          Stopped(0) Degraded(0) OK(11)

Verbose mode

k -n kube-system exec -it ds/cilium -c cilium-agent -- cilium-dbg status --verbose
KVStore:                Ok   Disabled
Kubernetes:             Ok   1.28 (v1.28.4) [linux/amd64]
Kubernetes APIs:        ["EndpointSliceOrEndpoint", "cilium/v2::CiliumClusterwideNetworkPolicy", "cilium/v2::CiliumEndpoint", "cilium/v2::CiliumNetworkPolicy", "cilium/v2::CiliumNode", "cilium/v2alpha1::CiliumCIDRGroup", "core/v1::Namespace", "core/v1::Pods", "core/v1::Service", "networking.k8s.io/v1::NetworkPolicy"]
KubeProxyReplacement:   True   [ens192   <IP_ADDRESS> fe80::250:56ff:fe92:e8a3 (Direct Routing)]
Host firewall:          Disabled
SRv6:                   Disabled
CNI Chaining:           none
CNI Config file:        successfully wrote CNI configuration file to /host/etc/cni/net.d/05-cilium.conflist
Cilium:                 Ok   1.15.6 (v1.15.6-a09e05e6)
NodeMonitor:            Listening for events on 128 CPUs with 64x4096 of shared memory
Cilium health daemon:   Ok
IPAM:                   IPv4: 4/254 allocated from <IP_ADDRESS>/24,
Allocated addresses:
  <IP_ADDRESS> (router)
  <IP_ADDRESS> (kube-system/coredns-7b6dc7894d-7qwwd)
  <IP_ADDRESS> (health)
  <IP_ADDRESS> (elastic-system/filebeat-filebeat-7hjch [restored])
IPv4 BIG TCP:           Disabled
IPv6 BIG TCP:           Disabled
BandwidthManager:       Disabled
Host Routing:           BPF
Masquerading:           BPF   [ens192]   <IP_ADDRESS>/24 [IPv4: Enabled, IPv6: Disabled]
Clock Source for BPF:   ktime
Controller Status:      33/33 healthy
  Name                                                    Last success     Last error       Count   Message
  bpf-map-sync-cilium_lxc                                 9s ago           never            0       no error
  cilium-health-ep                                        37s ago          never            0       no error
  dns-garbage-collector-job                               52s ago          never            0       no error
  endpoint-1111-regeneration-recovery                     never            never            0       no error
  endpoint-1503-regeneration-recovery                     never            never            0       no error
  endpoint-3332-regeneration-recovery                     never            never            0       no error
  endpoint-636-regeneration-recovery                      never            never            0       no error
  endpoint-gc                                             2m57s ago        never            0       no error
  ep-bpf-prog-watchdog                                    25s ago          never            0       no error
  fqdn-selector-checkpointing                             266h53m35s ago   never            0       no error
  ipcache-inject-labels                                   37s ago          266h53m35s ago   0       no error
  k8s-heartbeat                                           25s ago          never            0       no error
  link-cache                                              11s ago          never            0       no error
  neighbor-table-refresh                                  5s ago           never            0       no error
  resolve-identity-1111                                   4m12s ago        never            0       no error
  resolve-identity-1503                                   2m53s ago        never            0       no error
  resolve-labels-/                                        266h53m29s ago   never            0       no error
  resolve-labels-elastic-system/filebeat-filebeat-7hjch   266h53m26s ago   never            0       no error
  resolve-labels-kube-system/coredns-7b6dc7894d-7qwwd     258h29m40s ago   never            0       no error
  restoring-ep-identity (3332)                            266h53m29s ago   never            0       no error
  restoring-ep-identity (636)                             266h53m29s ago   never            0       no error
  sync-host-ips                                           37s ago          never            0       no error
  sync-lb-maps-with-k8s-services                          266h53m29s ago   never            0       no error
  sync-policymap-1111                                     14m13s ago       never            0       no error
  sync-policymap-1503                                     7m52s ago        never            0       no error
  sync-policymap-3332                                     7m52s ago        never            0       no error
  sync-policymap-636                                      7m50s ago        never            0       no error
  sync-to-k8s-ciliumendpoint (1111)                       9s ago           never            0       no error
  sync-to-k8s-ciliumendpoint (3332)                       9s ago           never            0       no error
  sync-utime                                              42s ago          never            0       no error
  template-dir-watcher                                    never            never            0       no error
  waiting-initial-global-identities-ep (3332)             266h53m29s ago   never            0       no error
  write-cni-file                                          266h53m35s ago   never            0       no error
Proxy Status:            OK, ip <IP_ADDRESS>, 0 redirects active on ports 10000-20000, Envoy: embedded
Global Identity Range:   min 256, max 65535
Hubble:                  Ok   Current/Max Flows: 4095/4095 (100.00%), Flows/s: 90.24   Metrics: Ok
KubeProxyReplacement Details:
  Status:                 True
  Socket LB:              Enabled
  Socket LB Tracing:      Enabled
  Socket LB Coverage:     Full
  Devices:                ens192   <IP_ADDRESS> fe80::250:56ff:fe92:e8a3 (Direct Routing)
  Mode:                   SNAT
  Backend Selection:      Random
  Session Affinity:       Enabled
  Graceful Termination:   Enabled
  NAT46/64 Support:       Disabled
  XDP Acceleration:       Disabled
  Services:
  - ClusterIP:      Enabled
  - NodePort:       Enabled (Range: 30000-32767)
  - LoadBalancer:   Enabled
  - externalIPs:    Enabled
  - HostPort:       Enabled
BPF Maps:   dynamic sizing: on (ratio: 0.002500)
  Name                          Size
  Auth                          524288
  Non-TCP connection tracking   73374
  TCP connection tracking       146749
  Endpoint policy               65535
  IP cache                      512000
  IPv4 masquerading agent       16384
  IPv6 masquerading agent       16384
  IPv4 fragmentation            8192
  IPv4 service                  65536
  IPv6 service                  65536
  IPv4 service backend          65536
  IPv6 service backend          65536
  IPv4 service reverse NAT      65536
  IPv6 service reverse NAT      65536
  Metrics                       1024
  NAT                           146749
  Neighbor table                146749
  Global policy                 16384
  Session affinity              65536
  Sock reverse NAT              73374
  Tunnel                        65536
Encryption:                                          Disabled

@cdtzabra have you seen this behavior change at all?
I don't have the time to dig into this and investigate. I'll ask around and see if someone else is able to help out.

@cdtzabra I am currently studying the use of cilium chainning in calico. In this process, three different scenarios, calico, cilium, and calico with cilium chainning, were tested.
I have the following three ideas about the problem you mentioned

If your calico uses direct routing, its performance must be far better than that of cilium in vxlan mode. Direct routing calico eliminates the need for packet encapsulation for tunnel transmission.
If your calico uses IPIP, the performance between them is not very good through theoretical comparison. Because cilium has Host Routing = BPF on the Node for acceleration, and IPIP packet encapsulation is theoretically less lossy than vxlan. The two should be similar in terms of neutralization, but from the grafana you provided, this situation should not be consistent.
I don't know how many nodes you tested. If there are multiple nodes, assuming that two iperf test applications appear on the same node and perform calico's iperf test, and when switching to cilium, the two iperf applications switch to different nodes for testing, then at this time, the application access performance on the same node is obviously higher than that of cross-node application access.

The following is some test data I provided, I hope it will be helpful to you.

The picture above is a test of calico IPIP on the same node

The picture above is a cross-node test of calico IPIP

The picture above is a test of cilium on the same node

The picture above is a cross-node test of cilium
From the performance comparison, we can find that cilium same node > calico same node > calico cross-node > cilium cross-node
[root@cluster-110-master0001 ~]# kubectl -n kube-system exec -it ds/cilium -c cilium-agent -- cilium-dbg status --verbose
KVStore:                Ok   Disabled
Kubernetes:             Ok   1.21 (v1.21.14) [linux/amd64]
Kubernetes APIs:        ["EndpointSliceOrEndpoint", "cilium/v2::CiliumClusterwideNetworkPolicy", "cilium/v2::CiliumEndpoint", "cilium/v2::CiliumNetworkPolicy", "cilium/v2::CiliumNode", "cilium/v2alpha1::CiliumCIDRGroup", "core/v1::Namespace", "core/v1::Pods", "core/v1::Service", "networking.k8s.io/v1::NetworkPolicy"]
KubeProxyReplacement:   True   [ens192    <IP_ADDRESS> 2022::1:71 fe80::250:56ff:fe92:1cc (Direct Routing), tunl0  <IP_ADDRESS>]
Host firewall:          Disabled
SRv6:                   Disabled
CNI Chaining:           none
CNI Config file:        successfully wrote CNI configuration file to /host/etc/cni/net.d/05-cilium.conflist
Cilium:                 Ok   1.15.8 (v1.15.8-e6291578)
NodeMonitor:            Listening for events on 4 CPUs with 64x4096 of shared memory
Cilium health daemon:   Ok
IPAM:                   IPv4: 2/254 allocated from <IP_ADDRESS>/24,
Allocated addresses:
  <IP_ADDRESS> (router)
  <IP_ADDRESS> (health)
IPv4 BIG TCP:           Disabled
IPv6 BIG TCP:           Disabled
BandwidthManager:       Disabled
Host Routing:           BPF
Masquerading:           BPF   [ens192, tunl0]   <IP_ADDRESS>/24 [IPv4: Enabled, IPv6: Disabled]
Clock Source for BPF:   ktime
Controller Status:      40/40 healthy
  Name                                            Last success    Last error      Count   Message
  cilium-health-ep                                22s ago         never           0       no error
  dns-garbage-collector-job                       32s ago         never           0       no error
  endpoint-1031-regeneration-recovery             never           never           0       no error
  endpoint-3581-regeneration-recovery             never           never           0       no error
  endpoint-3967-regeneration-recovery             never           never           0       no error
  endpoint-556-regeneration-recovery              never           never           0       no error
  endpoint-572-regeneration-recovery              never           never           0       no error
  endpoint-613-regeneration-recovery              never           never           0       no error
  endpoint-gc                                     3m33s ago       never           0       no error
  ep-bpf-prog-watchdog                            7s ago          never           0       no error
  fqdn-selector-checkpointing                     37h43m34s ago   never           0       no error
  ipcache-inject-labels                           22s ago         37h43m34s ago   0       no error
  k8s-heartbeat                                   22s ago         never           0       no error
  link-cache                                      12s ago         never           0       no error
  neighbor-table-refresh                          24s ago         never           0       no error
  resolve-identity-1031                           3m24s ago       never           0       no error
  resolve-identity-3581                           3m26s ago       never           0       no error
  resolve-identity-3967                           3m19s ago       never           0       no error
  resolve-identity-556                            3m25s ago       never           0       no error
  resolve-identity-572                            3m23s ago       never           0       no error
  resolve-identity-613                            3m23s ago       never           0       no error
  resolve-labels-default/iperf-6b78d5cbf4-g58h6   37h43m24s ago   never           0       no error
  resolve-labels-default/iperf-6b78d5cbf4-rhpp6   37h43m24s ago   never           0       no error
  resolve-labels-default/nginx-59fbcdfd49-845bs   37h43m19s ago   never           0       no error
  resolve-labels-default/qperf-6d8545d59c-nkcvw   37h43m23s ago   never           0       no error
  sync-host-ips                                   22s ago         never           0       no error
  sync-lb-maps-with-k8s-services                  37h43m27s ago   never           0       no error
  sync-policymap-1031                             13m20s ago      never           0       no error
  sync-policymap-3581                             13m18s ago      never           0       no error
  sync-policymap-3967                             13m18s ago      never           0       no error
  sync-policymap-556                              13m21s ago      never           0       no error
  sync-policymap-572                              13m20s ago      never           0       no error
  sync-policymap-613                              13m19s ago      never           0       no error
  sync-to-k8s-ciliumendpoint (1031)               3s ago          never           0       no error
  sync-to-k8s-ciliumendpoint (3967)               8s ago          never           0       no error
  sync-to-k8s-ciliumendpoint (572)                2s ago          never           0       no error
  sync-to-k8s-ciliumendpoint (613)                3s ago          never           0       no error
  sync-utime                                      24s ago         never           0       no error
  template-dir-watcher                            never           never           0       no error
  write-cni-file                                  37h43m34s ago   never           0       no error
Proxy Status:            OK, ip <IP_ADDRESS>, 0 redirects active on ports 10000-20000, Envoy: embedded
Global Identity Range:   min 256, max 65535
Hubble:                  Ok   Current/Max Flows: 4095/4095 (100.00%), Flows/s: 1.76   Metrics: Disabled
KubeProxyReplacement Details:
  Status:                 True
  Socket LB:              Enabled
  Socket LB Tracing:      Enabled
  Socket LB Coverage:     Full
  Devices:                ens192    <IP_ADDRESS> 2022::1:71 fe80::250:56ff:fe92:1cc (Direct Routing), tunl0  <IP_ADDRESS>
  Mode:                   SNAT
  Backend Selection:      Random
  Session Affinity:       Enabled
  Graceful Termination:   Enabled
  NAT46/64 Support:       Disabled
  XDP Acceleration:       Disabled
  Services:
  - ClusterIP:      Enabled
  - NodePort:       Enabled (Range: 30000-32767)
  - LoadBalancer:   Enabled
  - externalIPs:    Enabled
  - HostPort:       Enabled
BPF Maps:   dynamic sizing: on (ratio: 0.002500)
  Name                          Size
  Auth                          524288
  Non-TCP connection tracking   65536
  TCP connection tracking       131072
  Endpoint policy               65535
  IP cache                      512000
  IPv4 masquerading agent       16384
  IPv6 masquerading agent       16384
  IPv4 fragmentation            8192
  IPv4 service                  65536
  IPv6 service                  65536
  IPv4 service backend          65536
  IPv6 service backend          65536
  IPv4 service reverse NAT      65536
  IPv6 service reverse NAT      65536
  Metrics                       1024
  NAT                           131072
  Neighbor table                131072
  Global policy                 16384
  Session affinity              65536
  Sock reverse NAT              65536
  Tunnel                        65536
Encryption:                                       Disabled

The above is all the information I can provide, I hope it helps you

@learnitall In the process I described above, these performance conditions are in line with my expectations. When I tested calico with cilium chainning, I found that the performance loss was high in the cross-node scenario. I want to know if there is some configuration that I don't know about that is not enabled or disabled, resulting in abnormal performance loss in the IPIP tunnel.
Next, I will provide some data from my test in calico with cilium chainning mode.

The picture above is a test of calico with cilium chainning on the same node. The performance in this scenario meets my expectations and is higher than the native calico ipip performance.

The picture above is a cross-node test of calico with cilium chainning. In this scenario, I found that a lot of performance loss occurred. I studied it for a long time but couldn't find out how to avoid this performance loss, nor did I know the specific cause of the loss.
Do you have any ideas here? I hope you can help me.

@cdtzabra have you seen this behavior change at all?

Hi @learnitall
I've seen no improvement. The flow rate of the cilium never reached that of the calico.
I stopped my test and removed the iPerfs as none of our users complained of latency problems following the migration to Cilium and above all I couldn't keep the iPerfs continuously on our production clusters.

I don't have the time to dig into this and investigate. I'll ask around and see if someone else is able to help out.

Got it, thanks to all for your contributionsCliquez pour utiliser cette solution

@cdtzabra I am currently studying the use of cilium chainning in calico. In this process, three different scenarios, calico, cilium, and calico with cilium chainning, were tested. I have the following three ideas about the problem you mentioned

If your calico uses direct routing, its performance must be far better than that of cilium in vxlan mode. Direct routing calico eliminates the need for packet encapsulation for tunnel transmission.
If your calico uses IPIP, the performance between them is not very good through theoretical comparison. Because cilium has Host Routing = BPF on the Node for acceleration, and IPIP packet encapsulation is theoretically less lossy than vxlan. The two should be similar in terms of neutralization, but from the grafana you provided, this situation should not be consistent.
I don't know how many nodes you tested. If there are multiple nodes, assuming that two iperf test applications appear on the same node and perform calico's iperf test, and when switching to cilium, the two iperf applications switch to different nodes for testing, then at this time, the application access performance on the same node is obviously higher than that of cross-node application access.

The following is some test data I provided, I hope it will be helpful to you.  The picture above is a test of calico IPIP on the same node  The picture above is a cross-node test of calico IPIP  The picture above is a test of cilium on the same node  The picture above is a cross-node test of cilium From the performance comparison, we can find that cilium same node > calico same node > calico cross-node > cilium cross-node
[root@cluster-110-master0001 ~]# kubectl -n kube-system exec -it ds/cilium -c cilium-agent -- cilium-dbg status --verbose
KVStore:                Ok   Disabled
Kubernetes:             Ok   1.21 (v1.21.14) [linux/amd64]
Kubernetes APIs:        ["EndpointSliceOrEndpoint", "cilium/v2::CiliumClusterwideNetworkPolicy", "cilium/v2::CiliumEndpoint", "cilium/v2::CiliumNetworkPolicy", "cilium/v2::CiliumNode", "cilium/v2alpha1::CiliumCIDRGroup", "core/v1::Namespace", "core/v1::Pods", "core/v1::Service", "networking.k8s.io/v1::NetworkPolicy"]
KubeProxyReplacement:   True   [ens192    <IP_ADDRESS> 2022::1:71 fe80::250:56ff:fe92:1cc (Direct Routing), tunl0  <IP_ADDRESS>]
Host firewall:          Disabled
SRv6:                   Disabled
CNI Chaining:           none
CNI Config file:        successfully wrote CNI configuration file to /host/etc/cni/net.d/05-cilium.conflist
Cilium:                 Ok   1.15.8 (v1.15.8-e6291578)
NodeMonitor:            Listening for events on 4 CPUs with 64x4096 of shared memory
Cilium health daemon:   Ok
IPAM:                   IPv4: 2/254 allocated from <IP_ADDRESS>/24,
Allocated addresses:
<IP_ADDRESS> (router)
<IP_ADDRESS> (health)
IPv4 BIG TCP:           Disabled
IPv6 BIG TCP:           Disabled
BandwidthManager:       Disabled
Host Routing:           BPF
Masquerading:           BPF   [ens192, tunl0]   <IP_ADDRESS>/24 [IPv4: Enabled, IPv6: Disabled]
Clock Source for BPF:   ktime
Controller Status:      40/40 healthy
Name                                            Last success    Last error      Count   Message
cilium-health-ep                                22s ago         never           0       no error
dns-garbage-collector-job                       32s ago         never           0       no error
endpoint-1031-regeneration-recovery             never           never           0       no error
endpoint-3581-regeneration-recovery             never           never           0       no error
endpoint-3967-regeneration-recovery             never           never           0       no error
endpoint-556-regeneration-recovery              never           never           0       no error
endpoint-572-regeneration-recovery              never           never           0       no error
endpoint-613-regeneration-recovery              never           never           0       no error
endpoint-gc                                     3m33s ago       never           0       no error
ep-bpf-prog-watchdog                            7s ago          never           0       no error
fqdn-selector-checkpointing                     37h43m34s ago   never           0       no error
ipcache-inject-labels                           22s ago         37h43m34s ago   0       no error
k8s-heartbeat                                   22s ago         never           0       no error
link-cache                                      12s ago         never           0       no error
neighbor-table-refresh                          24s ago         never           0       no error
resolve-identity-1031                           3m24s ago       never           0       no error
resolve-identity-3581                           3m26s ago       never           0       no error
resolve-identity-3967                           3m19s ago       never           0       no error
resolve-identity-556                            3m25s ago       never           0       no error
resolve-identity-572                            3m23s ago       never           0       no error
resolve-identity-613                            3m23s ago       never           0       no error
resolve-labels-default/iperf-6b78d5cbf4-g58h6   37h43m24s ago   never           0       no error
resolve-labels-default/iperf-6b78d5cbf4-rhpp6   37h43m24s ago   never           0       no error
resolve-labels-default/nginx-59fbcdfd49-845bs   37h43m19s ago   never           0       no error
resolve-labels-default/qperf-6d8545d59c-nkcvw   37h43m23s ago   never           0       no error
sync-host-ips                                   22s ago         never           0       no error
sync-lb-maps-with-k8s-services                  37h43m27s ago   never           0       no error
sync-policymap-1031                             13m20s ago      never           0       no error
sync-policymap-3581                             13m18s ago      never           0       no error
sync-policymap-3967                             13m18s ago      never           0       no error
sync-policymap-556                              13m21s ago      never           0       no error
sync-policymap-572                              13m20s ago      never           0       no error
sync-policymap-613                              13m19s ago      never           0       no error
sync-to-k8s-ciliumendpoint (1031)               3s ago          never           0       no error
sync-to-k8s-ciliumendpoint (3967)               8s ago          never           0       no error
sync-to-k8s-ciliumendpoint (572)                2s ago          never           0       no error
sync-to-k8s-ciliumendpoint (613)                3s ago          never           0       no error
sync-utime                                      24s ago         never           0       no error
template-dir-watcher                            never           never           0       no error
write-cni-file                                  37h43m34s ago   never           0       no error
Proxy Status:            OK, ip <IP_ADDRESS>, 0 redirects active on ports 10000-20000, Envoy: embedded
Global Identity Range:   min 256, max 65535
Hubble:                  Ok   Current/Max Flows: 4095/4095 (100.00%), Flows/s: 1.76   Metrics: Disabled
KubeProxyReplacement Details:
Status:                 True
Socket LB:              Enabled
Socket LB Tracing:      Enabled
Socket LB Coverage:     Full
Devices:                ens192    <IP_ADDRESS> 2022::1:71 fe80::250:56ff:fe92:1cc (Direct Routing), tunl0  <IP_ADDRESS>
Mode:                   SNAT
Backend Selection:      Random
Session Affinity:       Enabled
Graceful Termination:   Enabled
NAT46/64 Support:       Disabled
XDP Acceleration:       Disabled
Services:

ClusterIP:      Enabled
NodePort:       Enabled (Range: 30000-32767)
LoadBalancer:   Enabled
externalIPs:    Enabled
HostPort:       Enabled
BPF Maps:   dynamic sizing: on (ratio: 0.002500)
Name                          Size
Auth                          524288
Non-TCP connection tracking   65536
TCP connection tracking       131072
Endpoint policy               65535
IP cache                      512000
IPv4 masquerading agent       16384
IPv6 masquerading agent       16384
IPv4 fragmentation            8192
IPv4 service                  65536
IPv6 service                  65536
IPv4 service backend          65536
IPv6 service backend          65536
IPv4 service reverse NAT      65536
IPv6 service reverse NAT      65536
Metrics                       1024
NAT                           131072
Neighbor table                131072
Global policy                 16384
Session affinity              65536
Sock reverse NAT              65536
Tunnel                        65536
Encryption:                                       Disabled
The above is all the information I can provide, I hope it helps you

@learnitall In the process I described above, these performance conditions are in line with my expectations. When I tested calico with cilium chainning, I found that the performance loss was high in the cross-node scenario. I want to know if there is some configuration that I don't know about that is not enabled or disabled, resulting in abnormal performance loss in the IPIP tunnel.
Next, I will provide some data from my test in calico with cilium chainning mode.  The picture above is a test of calico with cilium chainning on the same node. The performance in this scenario meets my expectations and is higher than the native calico ipip performance.  The picture above is a cross-node test of calico with cilium chainning. In this scenario, I found that a lot of performance loss occurred. I studied it for a long time but couldn't find out how to avoid this performance loss, nor did I know the specific cause of the loss. Do you have any ideas here? I hope you can help me.

Hi @zghyy
Thanks for the information/details.
In fact I had several nodes (a good ten), so I was more in the case of cross-nodes communication.


Conduct Initial Testing of Components

Perform initial testing of the developed components to ensure functionality and integration.

Done