Support for iOS13Beta

@kcoley  I have found issues with my existing workflow using gltf2usd and iOS13Beta. Many working usdz files were no longer working in iOS13Beta. One solution I found worked was merging the multiple mesh into a single mesh. This fixes many of the usdz files not rendering correctly. I also used Apple’s fixedOpacity to fix some where transparency layers were no longer rendering correctly. However, I have some complex usdz with animations and multiple mesh that I can no longer convert using my current workflow. Apple apparently does have a bug with buffers which is fixed internally but not yet released.

@wave-electron do you have a sample glTF that works differently on iOS 13 Beta vs iOS 12?

@kcoley Yes. multiple meshes do not render correctly. I’ll make you a very simple glTF file. Do you want usdz, usda etc?

@ Examples like this that work in iOS12 https://usdzshare.com/?ug-gallery=photo-detail&photo_id=938 don’t work in iOS13... and I haven’t worked out how to fix examples like these because merging mesh is not an option.

@wave-electron thanks, I'll see if there is any other possibility here

@kcoley iOS13 now supports nested usdz. This is good way to package family of usdz that go together like a lounge setting... as each item can be individually moved around inside AR Quick Look & Reality Composer. What would be required for gltf2usd to support such a structure?

@kcoley I've attached a glTF that is working correctly in iOS12 but not iOS13Beta.
not-working-iOS13.zip

@wave-electron thanks for the sample glTF.  This looks like a bug in iOS 13, since the generated usdz file is valid from what I can tell.  It may be good to raise this as an issue to Apple.
As for nested usdz files, does this mean zipping several generated usdz files into a zip archive?  Or are you thinking of generated nested usdz files from a glTF file?  This might be outside the scope of gltf2usd, though if it is the former, you can probably run a script which is able to call the converter multiple times on several generated usdz files and then zip them together.

@kcoley I found this in pixar usdz docs

(Nesting can occur when one usdz file embeds another, e.g. @set.usdz[areas/shire.usdz[architecture/BilboHouse/Table.

It could be a bunch of separate usdz files bundled inside a single parent usdz file.  It does require a separate tool to package input the usdz files into a new usdz archive. This is what the usda file looks like. Apple provides a tool which is bundled with the latest standalone usdpython.  usdzcreateassetlib nested.usdz cube.usdz sphere.usdz
would generate a nested.usda file like below.
(
    defaultPrim = "Object"
    upAxis = "Y"
    customLayerData = {
        asset[] assetLibrary = [@cube.usdz@, @sphere.usdz@]
    }
)

def Xform "Object" (
    variants = {
        string Assets = "cube"
    }
    prepend variantSets = "Assets"
)
{
    variantSet "Assets" = {
        "cube" (
            prepend references = @cube.usdz@
        ) {

        }
        "sphere" (
            prepend references = @sphere.usdz@
        ) {

        }
    }
}

@wave-electron thanks for looking into this!

@kcoley  Apple’s latest release of usdpython & usdzconvert python command line tool ( version 058 released today) is performing much better than the original release. I did some tests and it handles transparency now, plus glTF conversions with animations are working better. I tested the same file above and it converted without the same animation error. I did notice it may have an issue with metallic image assignment for PBR materials - didn’t seem to work on a couple of tests I did but was working with standard metallic = 1 assignment. Probably worth looking at.

@kcoley I also found out using the Pixar  usdzip -- asset or --arkitAsset with the same usda structure above would also create nested usdz files

Hi, just jumping here to say that we face the same issue. Our USDZ files are now not working with iOS 13 (or MacOS Catalina). Here is what we have found out so far:

USDZs converted with gltf2usd do not work anymore
When using Apple's USDZ Tools from last year, USDZs created with gtlf2usd cannot even be unzipped at all, an error happens
When using Apple's USDZ Tools from this month, although USDZs created with gtlf2usd do not work, they can be unpacked, and their usdc files seem fine
When using Apple's offcial usdzconvert command line tool, everything is fine.

See attached an example gltf, converted in the three different options: 2019.06.26.painting_test.zip

@silvainSayduck
Some other things I’ve figured out.

Skinned animations work using gltf2usd but not using usdzconvert on iOS13. I’ve filed a radar with Apple on this.

Usdzconvert is now best for doing animations with simple translation or transforms. However, orientation will be in some cases be different on iOS12 whilst correct in iOS13.

I think iOS13 has a bug with handing metallic image uv. Parameter settings for metallic seem to work. But still need to verify again.

Nice, thanks for sharing @wave-electron!
A couple more things we've seen/heard that might be worth sharing too:

In the USDZ tools from Apple's site, one of the tools is called fixOpacity. From one of the WWDC talks, any objects that had transparency that were made for iOS 12 will need to be run through there.
[I've actually tried running the paintings above through fixOpacity, but that didn't actually do anything, so I'm guessing it's not the issue in this case]

We've been notified of a bug with usdzconvert where if you have a gltf that uses a single texture for occlusion/roughness/metalness, then it will not appear correctly on iOS 12 devices when converted. However it looks fine in AR Quick Look on Desktop. That bug has been reported to Apple too.

I looked into iOS 13 some more and it seems to wrongly switch to transparent rendering when PNG textures are used even without transparency. A workaround that worked for me is is to use jpg Textures for now.
Transparency z-sorting problems can by worked around by using multiple Materials.
The one thing I haven't found a solution for so far is that meshes with skinned animation seem to loose their normal information. They appear faceted like they do not have any normal information at all.
Meshes that have regular animations do not seem to be affected by this.
On the bright side though shadows are now rendered every frame and we are no stuck with the static one form the first frame anymore.


Star Citizen - Gloss map missing from Principled BSDF node setup

In Star Citizen the gloss or roughness map is usually included in the DDN and DDNA files. Eppi's texture converter can be set to extract it as high quality TGA files when converting Star Citizen's DDS files to tiff. The gloss / roughness is a grey value map. To see how EPPI's tool extracts and converts the gloss map from the alpha channels of the DDN and DDNA files see this repo  A node setup to extract alpha and mix alpha with black to create a grey map to connect to roughness port might be a solution. Not sure.

I'll add this to the to do list.  It might be a while before I can get to improved node layouts though.  I definitely want to have it create as accurate of a material to what Cryengine is creating.


create sample config.properties for mwe

create sample config.properties for mwe

completed; see mwe/client.config


Add automated tests

With automated tests becomes easy to change code with no fear to introduce new bugs.

I thought a little bit, and came to the conclusion that doesn't make sense to implement automated tests in this state of the project.
So, I'm closing this issue.

This issue starts to make sense again, specially because we can evaluate expressions from command line


wrapGAppsHook without GST (or GRL): makeWrapper doesn't understand the arg GI_TYPELIB_PATH

wrapGAppsHook dies without GST
When using wrapGAppsHook, and we don't have any GST plugins, I end up with the (pretty-printed) wrapping
wrapProgram \
    /nix/store/b3l3niilvqcxcsbxmd6sgqk1dy1rk81c-pye-menu-1.0/bin/pye_menu_shell \
    --prefix PATH : /nix/store/4c3z5r6yxsf2cxwwyazhdn92xixn4j5b-python3-3.7.5/bin:/nix/store/b3l3niilvqcxcsbxmd6sgqk1dy1rk81c-pye-menu-1.0/bin:/nix/store/y8j1cfj8d9r5rbbxc22w7hnfjw5f4fd3-cairo-1.16.0-dev/bin:/nix/store/6mg7lfbdh9pgx7pbxr3544qqbrigdl1q-freetype-2.10.1-dev/bin:/nix/store/gpszqcy0xi0lavbbjdq82zkkjp3jbp2a-bzip2-<IP_ADDRESS>.1-bin/bin:/nix/store/031c5pk5lzabgmpqpyd46hzi625as6bp-libpng-apng-1.6.37-dev/bin:/nix/store/f8kl7kmpv130aw9zm542p74a3hg0yc13-fontconfig-2.12.6-bin/bin:/nix/store/bqp30vkncmm222mjvwggz0s7p318sflj-expat-2.2.7-dev/bin:/nix/store/w57xa8g4s4aviwmqwgra7m5hwj2b005m-glib-2.60.7-dev/bin:/nix/store/v5d4966ahvfir2hwpv003022f3pb7vik-gettext-<IP_ADDRESS>/bin:/nix/store/qpvxhl1jr0fxnrx9idnpdagqs00m5m2z-glib-2.60.7/bin \
    --set PYTHONNOUSERSITE true \
    --set GDK_PIXBUF_MODULE_FILE /nix/store/7ddlakx6xjczqbfs80xjd14f30fzadws-gdk-pixbuf-2.38.1/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache \
    --prefix XDG_DATA_DIRS : /nix/store/0snjc1qg89zqn3v35l9d55xrykh9nj5c-gtk+3-3.24.10/share/gsettings-schemas/gtk+3-3.24.10:/nix/store/b41z51vdv11n6df8ki5vj8dynxw98f9l-gsettings-desktop-schemas-3.32.0/share/gsettings-schemas/gsettings-desktop-schemas-3.32.0:/nix/store/0snjc1qg89zqn3v35l9d55xrykh9nj5c-gtk+3-3.24.10/share/gsettings-schemas/gtk+3-3.24.10 \
    --prefix GST_PLUGIN_SYSTEM_PATH_1_0 : \
    --prefix GI_TYPELIB_PATH : /nix/store/0snjc1qg89zqn3v35l9d55xrykh9nj5c-gtk+3-3.24.10/lib/girepository-1.0:/nix/store/z29l5xaaxh1s0697mcldj71ab0zshry1-librsvg-2.44.15/lib/girepository-1.0:/nix/store/pija1xzm7izxfb5m2hvhvlwp1l38ffxa-gobject-introspection-1.60.2/lib/girepository-1.0 \
    --prefix GRL_PLUGIN_PATH :

(Extracted with preFixup = "set -x" and postFixup = "set +x".)
The problem is that the GST_PLUGIN_SYSTEM_PATH_1_0 has no arguments, which should be passed as "", without the quotes the argument becomes the next --prefix which messes up the parser for makeWrapper. The following seems to work, though
 # in a nix-shell, patching the local file, hence ./pye_menu_shell
 wrapProgram \
     ./pye_menu_shell \
     --prefix PATH : /nix/store/4c3z5r6yxsf2cxwwyazhdn92xixn4j5b-python3-3.7.5/bin:/nix/store/b3l3niilvqcxcsbxmd6sgqk1dy1rk81c-pye-menu-1.0/bin:/nix/store/y8j1cfj8d9r5rbbxc22w7hnfjw5f4fd3-cairo-1.16.0-dev/bin:/nix/store/6mg7lfbdh9pgx7pbxr3544qqbrigdl1q-freetype-2.10.1-dev/bin:/nix/store/gpszqcy0xi0lavbbjdq82zkkjp3jbp2a-bzip2-<IP_ADDRESS>.1-bin/bin:/nix/store/031c5pk5lzabgmpqpyd46hzi625as6bp-libpng-apng-1.6.37-dev/bin:/nix/store/f8kl7kmpv130aw9zm542p74a3hg0yc13-fontconfig-2.12.6-bin/bin:/nix/store/bqp30vkncmm222mjvwggz0s7p318sflj-expat-2.2.7-dev/bin:/nix/store/w57xa8g4s4aviwmqwgra7m5hwj2b005m-glib-2.60.7-dev/bin:/nix/store/v5d4966ahvfir2hwpv003022f3pb7vik-gettext-<IP_ADDRESS>/bin:/nix/store/qpvxhl1jr0fxnrx9idnpdagqs00m5m2z-glib-2.60.7/bin \
     --set PYTHONNOUSERSITE true \
     --set GDK_PIXBUF_MODULE_FILE /nix/store/7ddlakx6xjczqbfs80xjd14f30fzadws-gdk-pixbuf-2.38.1/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache \
     --prefix XDG_DATA_DIRS : /nix/store/0snjc1qg89zqn3v35l9d55xrykh9nj5c-gtk+3-3.24.10/share/gsettings-schemas/gtk+3-3.24.10:/nix/store/b41z51vdv11n6df8ki5vj8dynxw98f9l-gsettings-desktop-schemas-3.32.0/share/gsettings-schemas/gsettings-desktop-schemas-3.32.0:/nix/store/0snjc1qg89zqn3v35l9d55xrykh9nj5c-gtk+3-3.24.10/share/gsettings-schemas/gtk+3-3.24.10 \
-     --prefix GST_PLUGIN_SYSTEM_PATH_1_0 : \
+     --prefix GST_PLUGIN_SYSTEM_PATH_1_0 : "" \
     --prefix GI_TYPELIB_PATH : /nix/store/0snjc1qg89zqn3v35l9d55xrykh9nj5c-gtk+3-3.24.10/lib/girepository-1.0:/nix/store/z29l5xaaxh1s0697mcldj71ab0zshry1-librsvg-2.44.15/lib/girepository-1.0:/nix/store/pija1xzm7izxfb5m2hvhvlwp1l38ffxa-gobject-introspection-1.60.2/lib/girepository-1.0 \
-     --prefix GRL_PLUGIN_PATH :
+     --prefix GRL_PLUGIN_PATH : ""

To Reproduce
Steps to reproduce the behavior:

git clone https://github.com/KoviRobi/Pye-Menu.git
git checkout 49b86e1c716eebdbe344721eec99b77538a430dc
nix-build

On the channel nixos-19.09, with
$ realpath `echo $NIX_PATH|tr : $'\n'|sed -n 's/nixpkgs=//p'`
/nix/store/wrjvn1wysg1y98pr53priyp79di20h7s-nixos-19.09.1548.3a1861fcabc/nixos

and also with
$ realpath /nix/var/nix/profiles/per-user/root/channels/nixos-unstable
/nix/store/4y71vgg3f6lviag46hkijwxlczjlmj0n-nixos-unstable-20.03pre204199.3140fa89c51/nixos-unstable

Expected behavior
The package builds
Screenshots
these derivations will be built:
  /nix/store/bdhzjppc5zhy552axm38bw3bmrpz7q28-python3.7-pye-menu-1.0.drv
building '/nix/store/bdhzjppc5zhy552axm38bw3bmrpz7q28-python3.7-pye-menu-1.0.drv'...
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing setuptools-build-hook
Using setuptoolsBuildPhase
Using setuptoolsShellHook
Sourcing pip-install-hook
Using pipInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing setuptools-check-hook
Using setuptoolsCheckPhase
unpacking sources
unpacking source archive /nix/store/1rw3230c24j5sv1dng5f008kqsn4d01r-pye-menu
source root is pye-menu
setting SOURCE_DATE_EPOCH to timestamp 315619200 of file pye-menu/wacom_menu_example.png
patching sources
configuring
no configure script, doing nothing
building
Executing setuptoolsBuildPhase
running bdist_wheel
running build
running build_py
creating build
creating build/lib
copying pye_menu.py -> build/lib
running build_scripts
creating build/scripts-3.7
copying and adjusting pye_menu_shell -> build/scripts-3.7
changing mode of build/scripts-3.7/pye_menu_shell from 644 to 755
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
copying build/lib/pye_menu.py -> build/bdist.linux-x86_64/wheel
running install_egg_info
running egg_info
writing Pye_Menu.egg-info/PKG-INFO
writing dependency_links to Pye_Menu.egg-info/dependency_links.txt
writing top-level names to Pye_Menu.egg-info/top_level.txt
reading manifest file 'Pye_Menu.egg-info/SOURCES.txt'
writing manifest file 'Pye_Menu.egg-info/SOURCES.txt'
Copying Pye_Menu.egg-info to build/bdist.linux-x86_64/wheel/Pye_Menu-1.0-py3.7.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/Pye_Menu-1.0.data
creating build/bdist.linux-x86_64/wheel/Pye_Menu-1.0.data/scripts
copying build/scripts-3.7/pye_menu_shell -> build/bdist.linux-x86_64/wheel/Pye_Menu-1.0.data/scripts
changing mode of build/bdist.linux-x86_64/wheel/Pye_Menu-1.0.data/scripts/pye_menu_shell to 755
adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
creating build/bdist.linux-x86_64/wheel/Pye_Menu-1.0.dist-info/WHEEL
creating 'dist/Pye_Menu-1.0-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'pye_menu.py'
adding 'Pye_Menu-1.0.data/scripts/pye_menu_shell'
adding 'Pye_Menu-1.0.dist-info/LICENSE'
adding 'Pye_Menu-1.0.dist-info/METADATA'
adding 'Pye_Menu-1.0.dist-info/WHEEL'
adding 'Pye_Menu-1.0.dist-info/top_level.txt'
adding 'Pye_Menu-1.0.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Finished executing setuptoolsInstallPhase
glibPreInstallPhase
glibPreInstallPhase
installing
Executing pipInstallPhase
/build/pye-menu/dist /build/pye-menu
Processing ./Pye_Menu-1.0-py3-none-any.whl
Installing collected packages: Pye-Menu
Successfully installed Pye-Menu-1.0
/build/pye-menu
Finished executing pipInstallPhase
glibPreFixupPhase
glibPreFixupPhase
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/1xgcj70f24bxh9czafxxa8p3m15z5w2w-python3.7-pye-menu-1.0
strip is /nix/store/q354712mnkw3ky8b5crj7ir7dyv29ylj-binutils-2.31.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/1xgcj70f24bxh9czafxxa8p3m15z5w2w-python3.7-pye-menu-1.0/lib  /nix/store/1xgcj70f24bxh9czafxxa8p3m15z5w2w-python3.7-pye-menu-1.0/bin
patching script interpreter paths in /nix/store/1xgcj70f24bxh9czafxxa8p3m15z5w2w-python3.7-pye-menu-1.0
checking for references to /build/ in /nix/store/1xgcj70f24bxh9czafxxa8p3m15z5w2w-python3.7-pye-menu-1.0...
Rewriting #!/nix/store/4c3z5r6yxsf2cxwwyazhdn92xixn4j5b-python3-3.7.5/bin/python3.7 to #!/nix/store/4c3z5r6yxsf2cxwwyazhdn92xixn4j5b-python3-3.7.5
wrapping `/nix/store/1xgcj70f24bxh9czafxxa8p3m15z5w2w-python3.7-pye-menu-1.0/bin/pye_menu_shell'...

Builder called die: makeWrapper doesn't understand the arg GI_TYPELIB_PATH
Backtrace:
108 makeWrapper /nix/store/1fgzsay81ncq1mf3608kzjklhikhqp1a-hook/nix-support/setup-hook
152 wrapProgram /nix/store/1fgzsay81ncq1mf3608kzjklhikhqp1a-hook/nix-support/setup-hook
109 wrapPythonProgramsIn /nix/store/agjbw5gdv6ds22arjg4aqn0g89zrarma-hook/nix-support/setup-hook
4 wrapPythonPrograms /nix/store/agjbw5gdv6ds22arjg4aqn0g89zrarma-hook/nix-support/setup-hook
97 _callImplicitHook /nix/store/1zpvvxqxypyx2hlsg4zm7jdh1dc7z7cz-stdenv-linux/setup
116 _eval /nix/store/1zpvvxqxypyx2hlsg4zm7jdh1dc7z7cz-stdenv-linux/setup
45 runHook /nix/store/1zpvvxqxypyx2hlsg4zm7jdh1dc7z7cz-stdenv-linux/setup
1204 fixupPhase /nix/store/1zpvvxqxypyx2hlsg4zm7jdh1dc7z7cz-stdenv-linux/setup
1325 genericBuild /nix/store/1zpvvxqxypyx2hlsg4zm7jdh1dc7z7cz-stdenv-linux/setup
2 main /nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh

builder for '/nix/store/bdhzjppc5zhy552axm38bw3bmrpz7q28-python3.7-pye-menu-1.0.drv' failed with exit code 1
error: build of '/nix/store/bdhzjppc5zhy552axm38bw3bmrpz7q28-python3.7-pye-menu-1.0.drv' failed

(Well, kind of a screenshot.)
Additional context
Might be related, https://github.com/NixOS/nixpkgs/issues/29055
Workaround
Adding gst plugins, even if the program doesn't depend on it, seems to fix it. Or setting GST_PLUGIN_SYSTEM_PATH_1_0 = ''""'' (the string containing an open/close quote pair, which gets expanded to the empty argument). See
https://github.com/KoviRobi/Pye-Menu/commit/4c95bacf06411cff55c7cdf8a8bba70af2be5e1c
I suspect the proper fix will be in <nixpkgs/pkgs/build-support/setup-hooks/wrap-gapps-hook.sh> e.g. nixos-19.09 or nixos-unstable, but I don't know enough about it to commit a fix knowing that it won't break other stuff

Hmm, trying to link the wrap-gapps-hook.sh file leads me to https://github.com/NixOS/nixpkgs/commit/22a216849bf82ec65e3de86b7ea30e9c6b11efa4#diff-c90743a6f7cfce0259c1e97010a37bdb which might fix it

Of course, it's also possible that this is due to my misuse/misunderstanding of wrapGAppsHook, I followed the gnome-music example in the manual. But the following seems to fix it, for nixos-unstable. The question is, is this a necessary and good patch?
--- pkgs/build-support/setup-hooks/wrap-gapps-hook.sh   2019-12-10 17:00:45.264978439 +0000
+++ pkgs/build-support/setup-hooks/wrap-gapps-hook.sh.new       2019-12-10 17:00:39.333978461 +0000
@@ -42,7 +42,9 @@
   fi

   for v in ${wrapPrefixVariables:-} GST_PLUGIN_SYSTEM_PATH_1_0 GI_TYPELIB_PATH GRL_PLUGIN_PATH; do
-    gappsWrapperArgs+=(--prefix "$v" : "${!v}")
+    if [ -n "${!v}" ]; then
+      gappsWrapperArgs+=(--prefix "$v" : "${!v}")
+    fi
   done

   if [[ -z "${dontWrapGApps:-}" ]]; then

We pass "${gappsWrapperArgs[@]}" directly to wrapProgram:
https://github.com/NixOS/nixpkgs/blob/3140fa89c51233397f496f49014f6b23216667c2/pkgs/build-support/setup-hooks/wrap-gapps-hook.sh#L15
and gappsWrapperArgs is an array, so the empty string should be passed as an argument when the variable does not exist.
And makeWrapper should just skip --prefix with empty value:
https://github.com/NixOS/nixpkgs/blob/3140fa89c51233397f496f49014f6b23216667c2/pkgs/build-support/setup-hooks/make-wrapper.sh#L69
I cannot reproduce the issue on master, nixos-unstable, nor do I recall ever seeing such issue. We have lot of packages not depending on GStreamer or Grilo, so I would expect the issue to pop up somewhere.

Hmm, I can reproduce it in your repository; even with strictDeps = false;. So weird.

Oh, right you are doing makeWrapperArgs = [ "\${gappsWrapperArgs[@]}" ]; but bash needs the value quoted for the array items to be quoted.

makeWrapperArgs = [ "\"\${gappsWrapperArgs[@]}\"" ]; should work, through passing bash snippets through args/flags is not something I can recommend. It will probably break with https://github.com/NixOS/nixpkgs/pull/72074.

I noticed this recently with python packages, because my example breaks in this fashion.
Will change the example to that @jtojnar.

Thanks both! The suggested preFixup makeWrapperArgs works for me. Closing as there is both a fix to the docs and one for wrap-gapps-hook-shell which has been committed.


basicAuth all routers

Can there be an auth structure that will pass completely to all applications and links?

Hey, I am not sure if I understand the question correctly. Do you mean that you would like to set a basic auth for all routes in your application?
If so, you can do it like following:
// nuxt.config.js

{
  modules: [
    "nuxt-security",
  ],
  security: {
    basicAuth: {
      route: '', // <- empty string here tells nuxt that this route handler is a global middleware for all routes
      value: {
        name: 'test',
        pass: 'test',
        enabled: true,
        message: 'test'
      }
    }
  }
}

If that was not what you were trying to get, please elaborate on your request and I will do my best to answer :)


Editor issues

i'm having problems with the editor

it won't open after i press f8 to stop recording
it won't open on startup
Everything else seems to work, however.

Less see, I need more details:
Which version of windows are you using?
Can you open the Editor by opening the StartUp window first?
Are you using the latest version of the app (v2.9)?
After trying to open the Editor, can you see it on the taskbar?

windows 7
You can't get to the editor at all, i am using 2.9
it does show it on the taskbar, even showing the preview window, but clicking on it doesn't open it

problem fixed, no idea what i did to fix it

Hmm, maybe it was outside the screen. If this happens again, use Windows + Left Arrow or Window + Right Arrow to move the window.


Add CI for codegen that runs on Windows and on MacOS (originally only on Linux) - because users also use those two platforms

seems not ok
[2021-10-07T00:44:40Z WARN  flutter_rust_bridge_codegen::commands] command output: Output { status: ExitStatus(ExitStatus(65280)), stdout: "Running in Directory: '/home/runner/work/flutter_rust_bridge/flutter_rust_bridge/frb_codegen'\n[SEVERE] : Couldn't find dynamic library in default locations.\n[SEVERE] : Please supply one or more path/to/llvm in ffigen's config under the key 'llvm-path'.\n", stderr: "Unhandled exception:\nException: Couldn't find dynamic library in default locations.\n#0      findDylibAtDefaultLocations (package:ffigen/src/config_provider/spec_utils.dart:330:3)\n#1      Config._getSpecs.<anonymous closure> (package:ffigen/src/config_provider/config.dart:203:29)\n#2      Config._extract (package:ffigen/src/config_provider/config.dart:189:50)\n#3      new Config.fromYaml (package:ffigen/src/config_provider/config.dart:139:17)\n#4      getConfigFromCustomYaml (package:ffigen/src/executables/ffigen.dart:120:17)\n#5      getConfig (package:ffigen/src/executables/ffigen.dart:69:14)\n#6      main (package:ffigen/src/executables/ffigen.dart:47:14)\n#7      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:281:32)\n#8      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)\n" }

stuck for an hour. after cancel see:
Run<EMAIL_ADDRESS>
Installing LLVM and Clang 12 (12.0.1)...
Downloading and extracting 'https://github.com/llvm/llvm-project/releases/download/llvmorg-12.0.1/LLVM-12.0.1-win64.exe'...
C:\ProgramData\Chocolatey\bin\7z.exe x D:\a\_temp\3a7e2deb-c4e8-455e-b444-0deb94570b31 "-oC:\Program Files\LLVM"

7-Zip 19.00 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21

Scanning the drive for archives:
1 file, 190997504 bytes (183 MiB)

Extracting archive: D:\a\_temp\3a7e2deb-c4e8-455e-b444-0deb94570b31
--
Path = D:\a\_temp\3a7e2deb-c4e8-455e-b444-0deb94570b31
Type = Nsis
Physical Size = 190997504
Method = LZMA:25
Solid = +
Headers Size = 220752
Embedded Stub Size = 114688
SubType = NSIS-3

Would you like to replace the existing file:
  Path:     C:\Program Files\LLVM\bin\LLVM-C.dll
  Size:     69279744 bytes (67 MiB)
  Modified: 2021-07-09 10:39:50
with the file from archive:
  Path:     bin\LLVM-C.dll
  Size:     69279744 bytes (67 MiB)
  Modified: 2021-07-09 10:39:50
? (Y)es / (N)o / (A)lways / (S)kip all / A(u)to rename all / (Q)uit?

win error:
Run cargo run --package flutter_rust_bridge_codegen --bin flutter_rust_bridge_codegen -- \
ParserError: D:\a\_temp\45a11494-533a-450d-bbcc-a7163cd00fcb.ps1:3
Line |
   3 |    --rust-input ../frb_example/pure_dart/rust/src/api.rs \
     |      ~
     | Missing expression after unary operator '--'.

Error: Process completed with exit code 1.

windows:
[2021-10-07T01:56:40Z DEBUG flutter_rust_bridge_codegen::commands] execute ffigen c_path=C:\Users\RUNNER~1\AppData\Local\Temp\.tmpvrriHH.h dart_path=C:\Users\RUNNER~1\AppData\Local\Temp\.tmpIbeJMY
[2021-10-07T01:56:40Z DEBUG flutter_rust_bridge_codegen::commands] execute command: "dart" "pub" "global" "run" "ffigen" "--config" "C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\.tmp7LwFOW"
[2021-10-07T01:56:40Z WARN  flutter_rust_bridge_codegen::commands] command output: Output { status: ExitStatus(ExitStatus(65)), stdout: "", stderr: "No active package ffigen.\n" }
thread 'main' panicked at 'command execution failed. command="dart" "pub" "global" "run" "ffigen" "--config" "C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\.tmp7LwFOW"', src\commands.rs:25:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: process didn't exit successfully: `target\debug\flutter_rust_bridge_codegen.exe --rust-input ../frb_example/pure_dart/rust/src/api.rs --dart-output ../frb_example/pure_dart/dart/lib/bridge_generated.dart --dart-format-line-length 120` (exit code: 101)
Error: Process completed with exit code 1.


Import /yojimbo87/OrientDB-NET.binary/wiki

https://github.com/yojimbo87/OrientDB-NET.binary/wiki contains valuable resources.
They should be imported here.

The wiki is no longer up to date with the current repository.


Added improvements to reduce errors and such

I added 4 different changes to this script.

On powershell 5.1 (comes with default installation of windows), there is a significant time-burner for Invoke-Webrequest  and other commands. It comes from writing that light blue progress bar at the top of the window. More info here: https://github.com/PowerShell/PowerShell/issues/2138
My next improvement is handling if a user accidently clicks on PowerShell x86. The script then corrects this human error.
Disable Keep alive for invoke web request is just because we should not assume that user will have further interaction beyond what is required on the script, so we should remove for potential less latency.
I added Microsoft.com instead of Google because of how likely of each to be blocked. This is also for personal privacy reasons, this features adds nothing extra but on a windows machine is usually preferred. I chose not to do a more privacy respecting website because that might also be blocked in a weird corporate environment.

@Ana06 I have changed this back to google, even though I personally know many people who would prefer Microsoft if in a windows environment. But it should be able to be merged now.

If I have free time in the future I can look into making this script compatible with pwsh x86.


Fix links in deployed V4

When the v4 dashboards are deployed. None of the links work, they all lead to the root /.
This is the problem (but the proposed solution does not work)
https://stackoverflow.com/questions/57438566/next-js-export-static-s3-routing-fails-on-page-reload

@mathcolo I think you may be of help here, this might be a s3 thing, or it may be a nextjs consequence. We also give up features by exporting as a static site that we may want to avoid

@mathcolo We think that s3 might be limiting for our next js router setup. We were thinking about AWS amplify and trying to do some cost analysis. We can deploy from our GitHub branch, but we need to allow read writes for AWS Amplify onto the repo. Any chance you could help with that?


Add optional pitch argument to startAutoRotate

Before starting rotation, the viewer is panned to pitch.
If pitch is not specified, origPitch will be used
Addresses #744

Thanks!


Deploy Backend-AishaAthmanLali

Naming Rules
Name your PR like this: ISSUENUMBER-TITLE-YOURNAME
Description
Write a summary of your change. Explain why you have made it.
Related to
Make sure you include the issue number with a hash sign # so Github can link this PR to the right issue, like this:
Fixes #1
Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have carefully reviewed my own code
[ ] I have commented my code
[ ] I have updated any documentation

How are new versions deployed?
How can I access the backend?

How are new versions deployed? How can I access the backend?

https://love-me-tender.netlify.app/


Deprecate everything besides Ref

Related Issues
fixes #3784

Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]).

[ ] I have updated the CHANGELOG.md of the relevant packages.
Changelog files must be edited under the form:
## Unreleased fix/major/minor

- Description of your change. (thanks to @yourGithubId)

[ ] If this contains new features or behavior changes,
I have updated the documentation to match those changes.

@vasilich6107 I found this helpful:
https://github.com/rrousselGit/riverpod/pull/3787/commits/e273d82d846c71e4313b89ec6157da6e815e7926


various buildGoModule packages: switch to go 1.19

Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

@ofborg build argocd-autopilot erigon gotop hugo hysteria mangal opentelemetry-collector-contrib trivy wails werf
@ofborg test telegraf

Pre-existing gotop darwin failure.


Update version for Android-x86

The version in manifest of Android-x86 is not updated.

Thank you!  Apologies again for not keeping the Android-x86 version code up to date.


Diagonal Traversal Solution in python of binary tree

Description
Please include a summary of the change and which issue is fixed. List any dependencies that are required for this change.
Fixes #1398
Replace issue_no with the issue number which is fixed in this PR
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[ ] Documentation Update

Checklist:

[ ] My code follows the style guidelines(Clean Code) of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have created a helpful and easy to understand README.md
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests/screenshots(if any) that prove my fix is effective or that my feature works

@SrihariTadala comment out the parts which are not a part of the program

@SrihariTadala don't u know how to comment in python??  u can u ''' to comment

Since no update I am closing it


Fix error connecting arrow function components

We are running in to an error Cannot read properties of undefined (reading 'focus') and traced it back to this code in connect() that assumes that all base components have a prototype. In spec-compliant ECMAScript an arrow function does not have a prototype so we have to guard against that.
TODOs

[x] linter, checker, and test are passing
[x] any new public modules are exported from src/GeneralStore.js
[x] version numbers are up to date in package.json
[x] CHANGELOG.md is up to date

Thanks for the fix! 👏
Just to tie some threads together, it looks like this was separately raised in https://github.com/HubSpot/general-store/issues/66 and tackled in https://github.com/HubSpot/general-store/pull/67 , but that PR was closed out in deference to #83 which we thought would make the issue obsolete. There are some good UTs in there that might be worth reviving just to make sure this functionality doesn't get broken in the future?


Unable to Control Multiple Devices Consistently

Individual control of lamps works great after your recent update.
However when I try to control my group of lamps (total 6) not all of the lamps receive commands. It seems 1-3 of the lamps react when controlled as a group
I'd be happy to provide whatever info you'd like, although nothing appears 'wrong' in the debug logs.

I have four GE lights (no RGB). I could control them at the same time using a group without any problem.
I have a few comments and suggestions:

Keep in mind that the Bluetooth use the same frequency as 2.4G wifi (wireless b/g) and USB 3.0. You might have a problem if you use a Bluetooth dongle on USB 3.0 port or close to a 2.4G router.
Sort the lights such that the one closest to the home assistant are on top. In this case home assistant will try the one with best connection first.
Provide as much detailed as you can. Are you sending commands to change the brightness, color temperature or color hue? What are the corresponding results? Which one reacted and how? Do they turn on simultaneously? Are they the one closest to the home assistant hub?
Did you set up your group in home assistant?
Could you reduce the number in a group and test? Are you still having problems using 4 lights?

Keep in mind that the Bluetooth use the same frequency as 2.4G wifi (wireless b/g) and USB 3.0. You might have a problem if you use a Bluetooth dongle on USB 3.0 port or close to a 2.4G router.
Initially I was using a bluetooth dongle, which was passed through ESXi to an Ubuntu VM which I thought was the culprit (something in the chain there). I did a fresh install of Ubuntu ARM64 on an RPi4 using the internal bluetooth and the result is the same. The only common denominator between these two instances is Ubuntu, so perhaps I will try a different distro to see if results are any better.
Individual control of the lamps works fantastically (seemingly more snappy than from the actual C by GE app), so I am inclined to think it is not a bluetooth range/interference problem

Sort the lights such that the one closest to the home assistant are on top. In this case home assistant will try the one with best connection first.
Location of the RPi doesn't seem to change behavior

Provide as much detailed as you can. Are you sending commands to change the brightness, color temperature or color hue? What are the corresponding results? Which one reacted and how? Do they turn on simultaneously? Are they the one closest to the home assistant hub?
Every command as a group exhibits this behavior: On, Off, CCT (temp), Brightness, RGB (Color Wheel)
Results are always the same, 1-3 lamps respond to the command sent and the other 3-5 do nothing.
The 'reacting lamps' varies, it is never the same lamp responding necessarily - it is seemingly random. I have issued group on group off commands a large number of times now to see if I could detect a pattern and I cannot find anything continuity. The group of lamps is 6 lamps in a row above my mirror. (vanity_1, vanity_2, vanity_3, etc) with #1 being closest to my server closet (vanity_1 and the bluetooth source are <5ft away from each other, vanity_6 (the furthest) is maybe 12ft away)

Did you set up your group in home assistant?
Yes both light group functions, as well as the on/off switch at the top of the 'Light' entity exhibit the same sporadic control

Could you reduce the number in a group and test? Are you still having problems using 4 lights?
I did some testing in the developer tools sections by just issuing commands to vanity_1 and vanity_2, and the problem persisted, very rarely would it actually control them together. I can try to completely eliminate some in the config and report back to see if that improved at all

Are you using circadian_lighting?
Not in this instance, but that is my desired end result.

When you group the light using home assistant. Home assistant will send command almost at the same time (Due to the lock mechanism in the code, they emit one right after the other). You could try to add a few more light groups with various number of GE lights to test how many it can handle.
One possibility is that somehow the bluetooth mesh could only handle 4 to 5 sequential commands at the same time. For me, 4 lights can be toggled at the same time.
A potential workaround of this problem is to add a few milliseconds of delay right after each sent package. I.e., after the command of
    def send_packet(self, id, command, params):
        # the lock mechanism is to prevent the simutaneous packets if
        # you control two devices at the same time though a group.
        self.lock.acquire()
        try:
          self.link.send_packet(id, command, params)

in light.py

    def send_packet(self, id, command, params):
        # the lock mechanism is to prevent the simutaneous packets if
        # you control two devices at the same time though a group.
        self.lock.acquire()
        try:
          self.link.send_packet(id, command, params)
          time.sleep(0.002)

I tried that with values .05, .01, .005, .003, .002, .001 and it introduced "wonkiness" - Commands show up really late (several seconds) or don't show up at all. Doesn't seem to change the bulbs not being controlled together consistently.

I factory reset my lamps, removed 3 of them from power, and created a new location (generating a new username and password from the getconfig.py script)
I also spun up Home Assistant OS to eliminate Ubuntu from the equation.
I added just 3 lights to the location in C by GE, and imported the new config to Home Assistant.
Even with just 3 lamps group control is sporadic/random. There is no perceivable difference than my 6 lamp setup, the lamps have a very low chance of reacting to group commands.
With this new instance, same as the last one, individual control is spot on and responsive.

Hello, I'm experiencing the same issue and was wondering if any solution was found, thanks.

I encountered the same problem using a Bluetooth chip  BCM4339/4335.
I was able to reliably and almost simultaneously control two bulbs by using a 0.05 seconds delay.

I encountered the same problem using a Bluetooth chip BCM4339/4335.
I was able to reliably and almost simultaneously control two bulbs by using a 0.05 seconds delay.

I hope to have time to test/deploy this over the weekend, will let you know.
Might I ask what bluetooth chipeset you were using previously with no issues?

I encountered the same problem using a Bluetooth chip BCM4339/4335.
I was able to reliably and almost simultaneously control two bulbs by using a 0.05 seconds delay.

I hope to have time to test/deploy this over the weekend, will let you know.
Might I ask what bluetooth chipeset you were using previously with no issues?

The original bluetooth chip that gives no problem is
MODALIAS=sdio:c00v02D0dA9BF
which should translates to BCM43455

The original bluetooth chip that gives no problem is
MODALIAS=sdio:c00v02D0dA9BF
which should translates to BCM43455

Weird. I was using a RPi4 which uses a CYW43455 which I was under the impression is equivalent/identical
I also tried a USB adapter with a CSR8510 A10 and results were the same

I recently revisited this custom component.
I have tried it both with and without your sleep addition in https://github.com/yangqian/hass-gelight/commit/6546cfb39951d48d0918bed0b33180ec3daf35e2
It works well with your addition, but it also seems to work well now without it.
I am using https://github.com/mjg59/bluepy/tree/local_version to fulfill the bluepy dependency. mjg59 used to work at Google but now has his own personal fork of the project.
I believe https://github.com/mjg59/bluepy/commit/f9a575d7d177b07b3720f27cf43e8b095752758e potentially addresses the same issue, as the intermittent control issue I was previously having is not apparent at all.


add python3.8 build

fixes #793

Codecov Report

Merging #796 into master will increase coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #796      +/-   ##
==========================================
+ Coverage   98.19%   98.23%   +0.03%
==========================================
  Files         123      123
  Lines       20471    20964     +493
==========================================
+ Hits        20102    20594     +492
- Misses        369      370       +1

Impacted Files
Coverage Δ

tensornetwork/block_sparse/blocksparse_utils.py
97.68% <100.00%> (ø)

tensornetwork/network_operations.py
98.51% <100.00%> (ø)

conftest.py
100.00% <0.00%> (ø)

tensornetwork/tn_keras/mpo_test.py
100.00% <0.00%> (ø)

tensornetwork/tests/network_test.py
100.00% <0.00%> (ø)

tensornetwork/tn_keras/test_layer.py
100.00% <0.00%> (ø)

tensornetwork/backends/backend_test.py
100.00% <0.00%> (ø)

examples/simple_mera/simple_mera_test.py
100.00% <0.00%> (ø)

tensornetwork/block_sparse/utils_test.py
100.00% <0.00%> (ø)

tensornetwork/block_sparse/linalg_test.py
100.00% <0.00%> (ø)

... and 43 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7dd580b...27a4dcf. Read the comment docs.


Add files via upload

Requesting the addition of Pine Crest School for JetBrains license for the teacher (and students).
This will be used in our post-AP artificial intelligence course.
High School in Fort Lauderdale, FL.
<EMAIL_ADDRESS>

Thank you so much for your help.

@pezzie Thank you for your patience while waiting for a review and processing your request. Pull request merged. Please start applying for JetBrains educational subscription at https://www.jetbrains.com/shop/eform/students in an hour, to allow the changes to propagate through JetBrains systems.

Thank you. Is there a difference between the subscription for students and
teachers? I am a teacher.
On Fri, Aug 23, 2024 at 2:21 PM Philip Torchinsky @.***>
wrote:

@pezzie https://github.com/pezzie Thank you for your patience while
waiting for a review and processing your request. Pull request merged.
Please start applying for JetBrains educational subscription at
https://www.jetbrains.com/shop/eform/students in an hour, to allow the
changes to propagate through JetBrains systems.
—
Reply to this email directly, view it on GitHub
https://github.com/JetBrains/swot/pull/22860#issuecomment-2307589916,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALUN7TXM4VKD4CK7HZ2JPTDZS54R3AVCNFSM6AAAAABM7JAETCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBXGU4DSOJRGY
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Mr. Ángel Pérez
Upper School Mathematics and Computer Science Teacher
Pine Crest School - Fort Lauderdale
954.492.4100 | www.pinecrest.edu


service create is using CLI defaults rather than leaving the fields empty

The CLI is creating a spec by doing something like:
	service = swarm.ServiceSpec{
		TaskTemplate: swarm.TaskSpec{
                        [...]
			Resources:     opts.resources.ToResourceRequirements(),
			RestartPolicy: opts.restartPolicy.ToRestartPolicy(),
			Placement: &swarm.Placement{
				Constraints: opts.constraints,
			},
		},
		Mode: swarm.ServiceMode{},
		UpdateConfig: &swarm.UpdateConfig{
			Parallelism: opts.update.parallelism,
			Delay:       opts.update.delay,
		},
		Networks:     convertNetworks(opts.networks),
		EndpointSpec: opts.endpoint.ToEndpointSpec(),
	}

This is pretty bad because it hardcodes the CLI defaults into the server. We have no way to make a distinction between a user wanting something versus a user leaving the defaults.
For example, the user runs a docker service create foo, no arguments.
Rather than leaving RestartPolicy to null, we're filling it up with stuff:
func (r *restartPolicyOptions) ToRestartPolicy() *swarm.RestartPolicy {
	return &swarm.RestartPolicy{
		Condition:   swarm.RestartPolicyCondition(r.condition),
		Delay:       r.delay.Value(),
		MaxAttempts: r.maxAttempts.Value(),
		Window:      r.window.Value(),
	}
}

We're incapable to know whether the user wanted something to be done about restart policies or if it's just the CLI setting defaults.

related / similar: https://github.com/docker/docker/issues/24958
Having the client set defaults is something that's been bugging me for a while; perhaps we should have a tracking issue to fix that throughout
One challenge though, is that the usage output of the CLI shows the defaults as set in the client, and doesn't know about defaults that are set server-side (in the daemon)

I'd like to get this fixed for 1.13.
The first step is changing the CLI (and json<->grpc conversion) to leave empty any fields that aren't explicitly specified by the user. This may take some minor engine API changes to make certain things in the spec pointers, if they are embedded structs today.
The main issue after that is visibility into the defaults. This matters in two places: seeing an existing service's parameters (i.e. service inspect), and CLI usage output.
@aluzzardi: For the former, what do you think about having controlapi's GetService also return a second version of the spec that has defaults inserted in all empty fields. We could call this InterpolatedSpec. For example, if the actual spec has no RestartPolicy, InterpolatedSpec would have a RestartPolicy filled with default values. service inspect would show the interpolated spec.
The main downside I can see is that people could confuse Spec and InterpolatedSpec, and accidentally do a service update based on InterpolatedSpec. But now that I think more about it, this is actually a useful feature. If you update a service using InterpolatedSpec, you freeze in place the current defaults, and they won't evolve when the managers upgrade to a new software version that may change defaults. This seems like useful behavior.
For the CLI usage output, I don't think it's worth the complexity of fetching defaults from the leader through an RPC. There's a lot that can go wrong here, and I think it's important to provide useful help output even when there isn't a usable swarm setup. One possibility for this is to provide a github.com/docker/swarmkit/defaults package as a source of truth for default values. It's not perfect because there's no guarantee that your client is linked to the same version of the package as the engine that acts as the leader and interprets empty values, but it seems reasonable if we want the CLI usage output to show defaults.

A similar change was made to logging driver settings a while back (see https://github.com/docker/docker/pull/21153 / https://github.com/docker/docker/issues/21118, and the discussion on https://github.com/docker/docker/pull/22575)
For that change, I was in doubt what the correct behavior would be,
as using the "merged" configuration would make it impossible to change daemon
defaults for existing containers (and the same would probably apply to services here).
On the other hand; not using the "merged" configuration (so, not baking it in the
service definition), could potentially break existing services if the daemon
configuration is updated.
I still think we need a global design document / guideline for this; also taking
into account how to deal with new options. For example; if https://github.com/docker/docker/pull/26421
gets merged (just as example);

should existing service-definitions be automatically updated?
should the defaults be merged at runtime?
do we need a general "migration" callback, that checks if service definitions
need to be updated (so that we can decide on a per-feature base)?

I still think we need a global design document / guideline for this; also taking
into account how to deal with new options. For example; if #26421
gets merged (just as example);

should existing service-definitions be automatically updated?
should the defaults be merged at runtime?
do we need a general "migration" callback, that checks if service definitions need to be updated (so that we can decide on a per-feature base)?

I think dealing with new options is straightforward. The SwarmKit model is that the spec is never modified except by the user. So if a spec is missing some new option, the default will automatically apply. This should be fine as long as we pick sensible defaults. In particular, for new features we'll want to err on the side of disabling them by default, so it doesn't affect the behavior of existing services.

I started working on this in SwarmKit: https://github.com/docker/swarmkit/pull/1744


ruby build segfaults when nix-daemon runs inside a ulimit -s 300000

Issue description
I'm building ruby on a CentOS machine with a custom nix prefix. (The installation is with a regular user, without root privileges running the nix-daemon in a mutually trusted multi-user setup.)
When I tried building ruby (for nixpkgs at 211309626af028dba079f7b8c08c5bf63a7a2969), I get a segfault:
    linking miniruby
    generating encdb.h
    make: *** [uncommon.mk:625: .rbconfig.time] Segmentation fault (core
    dumped)
    make: *** Waiting for unfinished jobs....
    make: *** [uncommon.mk:818: encdb.h] Segmentation fault (core dumped)

The segfault occurs in reserve_stack (thread_pthread.c:722)
This seems to be a long standing issue for ruby, since there are lots of threads discussing this online.
Most of the suggestions revolve around configuring with --with-setjmp-type=_setjmp or compiling with -O0, -fno-omit-frame-pointer, -fno-stack-protector. Sadly none of those suggestions worked.
I did however manage to build ruby when running the nix-daemon inside a shell where I set the stacksize to unlimited using ulimit -s unlimited. My CentOS machine seems to have a default value of ulimit -s 300000. This default leads to a crash of miniruby during the build.
This seems like an impurity in the build process.
My nix setup might be a really weird edge-case, but I was wondering if we should still set uname -s unlimited in the preBuild phase in nixpkgs.

Another workaround would be using gcc6, perhaps: https://github.com/rbenv/ruby-build/issues/1092
https://github.com/rbenv/ruby-build/issues/1115 .
I wonder if this happens in latest version?
That's quite the stack size limit (300MB!), so it seems surprising ruby exhausts it during the build?
However setting consistent limits seems like a good idea-- IIRC there's a TODO for this already near where we set the limit for core dumps :).

I tried this with both the nixpkgs-17.09 gcc 6.4.0, as well as with the CentOS gcc 4.8.x compiler. From what I've read in the forums this issue has shown in one form or another for ruby builds in the last six years, so it's probably not something that can be solved in the context of nixpkgs. What surprised me the most, was however the "impurity" involved with setting ulimit -s unlimited outside of nix-build.

Sadly I cannot reproduce this issue on a different machine with the same nixpkgs checkout. I guess I'll close this then, as I don't see any action item.


Displaying big json take too long time

Not critical, but when I am trying to display big graph (40KB json). But displaying take too long time. For example graph visjs.github.io/vis-network doing it much faster.
data.txt
copy-paste example:
import 'dart:html';
import 'dart:math';

import 'package:flutter/material.dart';
import 'package:graph_page/service/service.dart'; // only it need be added to get it work
import 'package:graphview/GraphView.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) => MaterialApp(
        home: LayeredGraphViewPage(),
      );
}

class LayeredGraphViewPage extends StatefulWidget {
  @override
  _LayeredGraphViewPageState createState() => _LayeredGraphViewPageState();
}

class _LayeredGraphViewPageState extends State<LayeredGraphViewPage> {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
        body: Container(
      height: MediaQuery.of(context).size.height,
      color: Colors.amber[100],
      margin: const EdgeInsets.all(10.0),
      child: Row(
        children: [
           Container(
             width: 150,
            color: Colors.blue[100],
            child: Column(children: [
              ElevatedButton(onPressed: () {

                    var data = TenderData.getGraphData(); // need to add this to get it work
                    // print(data['edges']);
                    data['edges']!.forEach((element) {
                        var fromNodeId = element['from'];
                        var toNodeId = element['to'];
                        graph.addEdge(Node.Id(fromNodeId), Node.Id(toNodeId));
                      });
                    setState(() {});

               }, child: Text("get data"),),
              Text("some text"),

            ],)

          ),
          Expanded(child: Container(
            color: Colors.yellow[100],
             child: InteractiveViewer(
               constrained: false,
               boundaryMargin: EdgeInsets.all(100),
               minScale: 0.01,
               maxScale: 5.6,
               child: GraphView(
                 graph: graph,
                algorithm: SugiyamaAlgorithm(builder),
                 paint: Paint()
                   ..color = Colors.green
                   ..strokeWidth = 1
                   ..style = PaintingStyle.stroke,
                 builder: (Node node) {
                   // I can decide what widget should be shown here based on the id
                   var a = node.key!.value as int;
                   return rectangleWidget(node);
                 },
               )),

          ))
        ],
      ),
    ));
  }

  Random r = Random();

  Widget rectangleWidget(Node a) {
    return InkWell(
      onTap: () {
        print('clicked');
      },
      child: Container(
          padding: EdgeInsets.all(16),
          decoration: BoxDecoration(
            borderRadius: BorderRadius.circular(4),

          border: Border.all(
          color: Colors.black,
          width: 5,
        ),

          ),
          child: Text('Node ${a}')),
    );
  }

 final Graph graph = Graph();

 SugiyamaConfiguration builder = SugiyamaConfiguration();

  @override
  void initState() {
    final node1 = Node.Id(1);
    final node2 = Node.Id(2);
    graph.addEdge(node1, node2);

    builder
      ..nodeSeparation = (15)
      ..levelSeparation = (15)
      ..orientation = SugiyamaConfiguration.ORIENTATION_LEFT_RIGHT;

  }
}

It will always be a bit slow as we are loading everyone the first time . But still I made some improvements in 1.0.0

@nabil6391 In my example is a big tree with the SugiyamaAlgorithm and it took 7 seconds to show the tree. The UI is stuck for the whole 7 seconds!

Guys I suggest you to upgrate to 1.1.0. There are some major improvements to Sugiyama Algorithms. Do have a look

Keeping #56  and #55  open for further tracking. Closing this.


How to load pairs of FASTQ files from paired-end reads

As you know, it's very common to have paired-end reads mainly in Illumina NGS technologies. I was wondering to know your recommendation for having a single session for processing paired-end FASTQ files. Looking forward to hear your thoughts.

This is a common example for computational task with paired-end reads: https://cutadapt.readthedocs.io/en/stable/guide.html#paired-end
It might be a separate conversation but it can be a good idea if your tool can have guidance for "trimming task". Based on what I have learned about your tool so far, I assume this task can be very efficient with the SQL core in your tool and it can be combined with other tasks at the same time.

Thanks as always for filing the issue. In the short term please see the following example for loading pair end reads. I also agree w/ you point about having some guidance around trimming, and filed in ticket to update the docs with an example.

>>> !wget https://raw.githubusercontent.com/marcelm/cutadapt/main/tests/cut/paired.1.fastq
>>> !wget https://raw.githubusercontent.com/marcelm/cutadapt/main/tests/cut/paired.2.fastq
>>>
>>> import biobear as bb
>>> session = bb.connect()
>>>
>>> session.sql("""
   ...: WITH one AS (SELECT REPLACE(name, '/1', '') trimmed_name, * FROM fastq_scan('paired.1.fastq')),
   ...:      two AS (SELECT REPLACE(name, '/2', '') trimmed_name, * FROM fastq_scan('paired.2.fastq'))
   ...:
   ...: SELECT one.trimmed_name, one.name one_name, one.sequence one_sequence, two.name two_name, two.sequence two_sequence
   ...: FROM one
   ...:   JOIN two
   ...:     ON one.trimmed_name = two.trimmed_name
   ...:
   ...: """).to_polars()

Outside of python the SQL looks like the following. The idea is to create two CTEs which are used to standardize the read names (removing /1 or /2), then the body joins the two files together on that new name and returns a bit of info about both reads. You could add quality score or description if you want those.
WITH one AS (
    SELECT REPLACE(name, '/1', '') trimmed_name, *
    FROM fastq_scan('paired.1.fastq')
), two AS (
    SELECT REPLACE(name, '/2', '') trimmed_name, *
    FROM fastq_scan('paired.2.fastq')
)

SELECT one.trimmed_name, one.name one_name, one.sequence one_sequence, two.name two_name, two.sequence two_sequence
FROM one
  JOIN two
    ON one.trimmed_name = two.trimmed_name

The results of which looks like:
shape: (4, 5)
┌──────────────┬──────────┬────────────────────────────────┬──────────┬────────────────────────────────┐
│ trimmed_name ┆ one_name ┆ one_sequence                   ┆ two_name ┆ two_sequence                   │
│ ---          ┆ ---      ┆ ---                            ┆ ---      ┆ ---                            │
│ str          ┆ str      ┆ str                            ┆ str      ┆ str                            │
╞══════════════╪══════════╪════════════════════════════════╪══════════╪════════════════════════════════╡
│ read5        ┆ read5    ┆ TTATTTGTCTCCAGC                ┆ read5    ┆ CAACAGGCCACATTAGACATATCGGATGGT │
│ read1        ┆ read1/1  ┆ TTATTTGTCTCCAGC                ┆ read1/2  ┆ GCTGGAGACAAATAA                │
│ read4        ┆ read4/1  ┆ GACAGGCCGTTTGAATGTTGACGGGATGTT ┆ read4/2  ┆ CATCCCGTCAACATTCAAACGGCCTGTCCA │
│ read3        ┆ read3/1  ┆ CCAACTTGATATTAATAACA           ┆ read3/2  ┆ TGTTATTAATATCAAGTTGG           │
└──────────────┴──────────┴────────────────────────────────┴──────────┴────────────────────────────────┘

Hi @tshauck – thank you so much for your response! I'll look more carefully and will get back to you.
Actually, my main goal for loading both R1/R2 reads at the same time is doing this:

trimming each read from certain position
counting unique sequences based on both reads

Basically, extending this function: https://github.com/ArcInstitute/ScreenPro2/blob/master/screenpro/ngs.py#L26-L50
Do you have any suggestion? Thanks agin for your insightful responses.

Thanks for the link 👍 , I'm gonna work through the trimming doc (https://github.com/wheretrue/biobear/issues/103) and how it refers to your link tonight/tomorrow morning and I'll follow-up then w/ more.

Thanks for the link 👍 , I'm gonna work through the trimming doc (#103) and how it refers to your link tonight/tomorrow morning and I'll follow-up then w/ more.

Awesome! Looking forward to that.
I'm also excited to hear your thoughts about this https://github.com/wheretrue/biobear/issues/102#issuecomment-1977629934 but there is no rush; whenever you have time please let me know what you think. I already released a new sub-bersion for my tool which is now using biobear – https://github.com/ArcInstitute/ScreenPro2/commit/85445c65818657041331300bb6aa0cada6e77d24. I'll add paired-end read support in the future (hopefully next few weeks).

I could follow your example here and it worked well. Thanks!!
https://github.com/ArcInstitute/ScreenPro2/commit/fe9a27d92dd27e339cc257d55aaa7c4661a60c69

Thanks as always for filing the issue. In the short term please see the following example for loading pair end reads. I also agree w/ you point about having some guidance around trimming, and filed in ticket to update the docs with an example.
>>> !wget https://raw.githubusercontent.com/marcelm/cutadapt/main/tests/cut/paired.1.fastq
>>> !wget https://raw.githubusercontent.com/marcelm/cutadapt/main/tests/cut/paired.2.fastq
>>>
>>> import biobear as bb
>>> session = bb.connect()
>>>
>>> session.sql("""
   ...: WITH one AS (SELECT REPLACE(name, '/1', '') trimmed_name, * FROM fastq_scan('paired.1.fastq')),
   ...:      two AS (SELECT REPLACE(name, '/2', '') trimmed_name, * FROM fastq_scan('paired.2.fastq'))
   ...:
   ...: SELECT one.trimmed_name, one.name one_name, one.sequence one_sequence, two.name two_name, two.sequence two_sequence
   ...: FROM one
   ...:   JOIN two
   ...:     ON one.trimmed_name = two.trimmed_name
   ...:
   ...: """).to_polars()

Outside of python the SQL looks like the following. The idea is to create two CTEs which are used to standardize the read names (removing /1 or /2), then the body joins the two files together on that new name and returns a bit of info about both reads. You could add quality score or description if you want those.
WITH one AS (
    SELECT REPLACE(name, '/1', '') trimmed_name, *
    FROM fastq_scan('paired.1.fastq')
), two AS (
    SELECT REPLACE(name, '/2', '') trimmed_name, *
    FROM fastq_scan('paired.2.fastq')
)

SELECT one.trimmed_name, one.name one_name, one.sequence one_sequence, two.name two_name, two.sequence two_sequence
FROM one
  JOIN two
    ON one.trimmed_name = two.trimmed_name

The results of which looks like:
shape: (4, 5)
┌──────────────┬──────────┬────────────────────────────────┬──────────┬────────────────────────────────┐
│ trimmed_name ┆ one_name ┆ one_sequence                   ┆ two_name ┆ two_sequence                   │
│ ---          ┆ ---      ┆ ---                            ┆ ---      ┆ ---                            │
│ str          ┆ str      ┆ str                            ┆ str      ┆ str                            │
╞══════════════╪══════════╪════════════════════════════════╪══════════╪════════════════════════════════╡
│ read5        ┆ read5    ┆ TTATTTGTCTCCAGC                ┆ read5    ┆ CAACAGGCCACATTAGACATATCGGATGGT │
│ read1        ┆ read1/1  ┆ TTATTTGTCTCCAGC                ┆ read1/2  ┆ GCTGGAGACAAATAA                │
│ read4        ┆ read4/1  ┆ GACAGGCCGTTTGAATGTTGACGGGATGTT ┆ read4/2  ┆ CATCCCGTCAACATTCAAACGGCCTGTCCA │
│ read3        ┆ read3/1  ┆ CCAACTTGATATTAATAACA           ┆ read3/2  ┆ TGTTATTAATATCAAGTTGG           │
└──────────────┴──────────┴────────────────────────────────┴──────────┴────────────────────────────────┘

I'd also just add that session will persist state within a given program or interpreter session. So you can create a table backed by a file, then query it multiple times if needed.
import biobear as bb

s = bb.connect()

s.execute("""
CREATE EXTERNAL TABLE test STORED AS FASTQ LOCATION 'paired.1.fastq'
""")

s.sql("SELECT * FROM test").to_polars()

┌─────────┬─────────────┬────────────────────────────────┬────────────────────────────────┐
│ name    ┆ description ┆ sequence                       ┆ quality_scores                 │
│ ---     ┆ ---         ┆ ---                            ┆ ---                            │
│ str     ┆ str         ┆ str                            ┆ str                            │
╞═════════╪═════════════╪════════════════════════════════╪════════════════════════════════╡
│ read1/1 ┆ some text   ┆ TTATTTGTCTCCAGC                ┆ ##HHHHHHHHHHHHH                │
│ read3/1 ┆ null        ┆ CCAACTTGATATTAATAACA           ┆ HHHHHHHHHHHHHHHHHHHH           │
│ read4/1 ┆ null        ┆ GACAGGCCGTTTGAATGTTGACGGGATGTT ┆ HHHHHHHHHHHHHHHHHHHHHHHHHHHHHH │
│ read5   ┆ null        ┆ TTATTTGTCTCCAGC                ┆ #####HHHHHHHHHH                │
└─────────┴─────────────┴────────────────────────────────┴────────────────────────────────┘

Nice, congrats on the release! And glad that worked well enough.
I am still working on the adapter/quality trimming info. I'm trying to add some functionality as I go to make it easier to use, e.g. SELECT quality_scores_to_list('!!!') returning [0, 0, 0]. (https://github.com/wheretrue/exon/pull/427/files)


Making logging to stdout optional

The generated twirp servers include
import log "log"
They're used in error cases like this:
	if _, err = resp.Write(buf.Bytes()); err != nil {
		log.Printf("errored while writing response to client, but already sent response status code to 200: %s", err)
	}

It would be great if I could provide my own logger of some kind.
One proposed API would be to expose an optional function I could type assert for.
func (s *myServer) SetLogger(logger *log.Logger) {
  s.logger = logger
}

The client code could then call
x := NewMyServer(...)
x.(twirp.Logthingie).SetLogger(myOwnLogger)

If you're ambitious, you could have a specific twirp type logger for more strongly typed information
type TwirpLogger interface {
  ErrorWritingReponse(ctx context.Context, writeErr err)
}

My implementation could then extract the method name from the ctx.

Fixed in #66 and #76.


chunkdup: display diff bar

Codecov Report

Merging #25 (c9ab637) into main (fa57778) will increase coverage by 0.03%.
The diff coverage is 97.61%.

@@            Coverage Diff             @@
##             main      #25      +/-   ##
==========================================
+ Coverage   99.38%   99.42%   +0.03%
==========================================
  Files           7        7
  Lines         325      345      +20
==========================================
+ Hits          323      343      +20
  Misses          2        2

Impacted Files
Coverage Δ

chunkdup/differ.py
97.84% <95.23%> (+0.25%)
:arrow_up:

chunkdup/chunkdiff.py
100.00% <100.00%> (ø)

chunkdup/chunkdup.py
100.00% <100.00%> (ø)

chunkdup/diffbar.py
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Added the note because of the error

Added the Note:
Note: Do not start your project name with volto, as it will cause plone/generator-volto to fail
Fixes #716
Solved temporarily, Need to find a correct solution for this

@stevepiercy Please look at this PR and check if it needs some work or if is it fine for now.

Thank you!


server socket should respect resumability flag in setup frame

today we allow enabling/disabling resumability of the server socket by passing boolean parameter to connectServer.
the REQUIRE_RESUMABLE flag from client's setup frame is ignored.
We need to honor the clients flag while still allow turning on/off resumability as a feature.

done


chore(all): Update Code Smell omissions

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: 792517bd108d20ae408b19f87b114e7f4c7ed8bc
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


[General] Getting issue in dynamic iot thing name?

Hello developers,
System information

Hardware board: [ ESP32 Devkit ]
IDE used: [ ESP-IDF ]
Operating System: [ Linux  ]
Code version: 202107
Project/Demo: custom application

I m getting issues when I m passing the thing name using the function.
extern const char *getIOT_THING_NAME( void );
#define clientcredentialIOT_THING_NAME                getIOT_THING_NAME()

this method was working in v202007. Now when start working on the v202107 and added this method, it started showing the error.
Outcome
freertos/CMakeFiles/afr_demo.dir/demos/device_shadow_for_aws/shadow_demo_main.c.obj -MF freertos/CMakeFiles/afr_demo.dir/demos/device_shadow_for_aws/shadow_demo_main.c.obj.d -o freertos/CMakeFiles/afr_demo.dir/demos/device_shadow_for_aws/shadow_demo_main.c.obj   -c ../freertos/demos/device_shadow_for_aws/shadow_demo_main.c
In file included from ../freertos/demos/device_shadow_for_aws/shadow_demo_main.c:72:
../freertos/demos/device_shadow_for_aws/shadow_demo_main.c: In function 'RunDeviceShadowDemo':
../freertos-configs/aws_clientcredential.h:48:55: error: expected ')' before 'getIOT_THING_NAME'
 #define clientcredentialIOT_THING_NAME                getIOT_THING_NAME()
                                                       ^~~~~~~~~~~~~~~~~
../freertos/libraries/device_shadow_for_aws/source/include/shadow.h:361:21: note: in definition of macro 'SHADOW_TOPIC_STRING'
     ( SHADOW_PREFIX thingName operation suffix )
                     ^~~~~~~~~
../freertos/demos/device_shadow_for_aws/shadow_demo_main.c:818:45: note: in expansion of macro 'SHADOW_TOPIC_STRING_DELETE_ACCEPTED'
                                             SHADOW_TOPIC_STRING_DELETE_ACCEPTED( THING_NAME ),
                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../freertos/demos/common/mqtt_demo_helpers/mqtt_demo_helpers.h:58:44: note: in expansion of macro 'clientcredentialIOT_THING_NAME'
     #define democonfigCLIENT_IDENTIFIER    clientcredentialIOT_THING_NAME
                                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../freertos/demos/device_shadow_for_aws/shadow_demo_main.c:169:27: note: in expansion of macro 'democonfigCLIENT_IDENTIFIER'
     #define THING_NAME    democonfigCLIENT_IDENTIFIER
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~
../freertos/demos/device_shadow_for_aws/shadow_demo_main.c:818:82: note: in expansion of macro 'THING_NAME'
                                             SHADOW_TOPIC_STRING_DELETE_ACCEPTED( THING_NAME ),
                                                                                ^~~~~~~~~~
../freertos/libraries/device_shadow_for_aws/source/include/shadow.h:361:5: note: to match this '('
     ( SHADOW_PREFIX thingName operation suffix )
     ^
../freertos/libraries/device_shadow_for_aws/source/include/shadow.h:421:5: note: in expansion of macro 'SHADOW_TOPIC_STRING'
     SHADOW_TOPIC_STRING( thingName, SHADOW_OP_DELETE, SHADOW_SUFFIX_ACCEPTED )
     ^~~~~~~~~~~~~~~~~~~
../freertos/demos/device_shadow_for_aws/shadow_demo_main.c:818:45: note: in expansion of macro 'SHADOW_TOPIC_STRING_DELETE_ACCEPTED'
                                             SHADOW_TOPIC_STRING_DELETE_ACCEPTED( THING_NAME ),
                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../freertos-configs/aws_clientcredential.h:48:55: error: expected ')' before 'getIOT_THING_NAME'
 #define clientcredentialIOT_THING_NAME                getIOT_THING_NAME()
                                                       ^~~~~~~~~~~~~~~~~
../freertos/libraries/device_shadow_for_aws/source/include/shadow.h:361:21: note: in definition of macro 'SHADOW_TOPIC_STRING'
     ( SHADOW_PREFIX thingName operation suffix )
                     ^~~~~~~~~
../freertos/demos/device_shadow_for_aws/shadow_demo_main.c:825:49: note: in expansion of macro 'SHADOW_TOPIC_STRING_DELETE_REJECTED'
                                                 SHADOW_TOPIC_STRING_DELETE_REJECTED( THING_NAME ),
                                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../freertos/demos/common/mqtt_demo_helpers/mqtt_demo_helpers.h:58:44: note: in expansion of macro 'clientcredentialIOT_THING_NAME'
     #define democonfigCLIENT_IDENTIFIER    clientcredentialIOT_THING_NAME
                                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../freertos/demos/device_shadow_for_aws/shadow_demo_main.c:169:27: note: in expansion of macro 'democonfigCLIENT_IDENTIFIER'
     #define THING_NAME    democonfigCLIENT_IDENTIFIER
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~
../freertos/demos/device_shadow_for_aws/shadow_demo_main.c:825:86: note: in expansion of macro 'THING_NAME'
                                                 SHADOW_TOPIC_STRING_DELETE_REJECTED( THING_NAME ),
                                                                                ^~~~~~~~~~
../freertos/libraries/device_shadow_for_aws/source/include/shadow.h:361:5: note: to match this '('
     ( SHADOW_PREFIX thingName operation suffix )
     ^
../freertos/libraries/device_shadow_for_aws/source/include/shadow.h:427:5: note: in expansion of macro 'SHADOW_TOPIC_STRING'
     SHADOW_TOPIC_STRING( thingName, SHADOW_OP_DELETE, SHADOW_SUFFIX_REJECTED )

So, any other method we have to use to pass the dynamic thing name?

When the thing name needs to be dynamic, you can update the demo to use Shadow_GetTopicString instead. There is an example of how to do that here:
https://github.com/aws/amazon-freertos/blob/main/demos/device_shadow_for_aws/shadow_demo_main.c#L896-L918


[Fluent UX] Keyboard Manager

Provide a description of requested docs changes
Description of the new feature / enhancement
Windows 11 introduces a new direction of Fluent Design - it would be great if all PowerToys features would look and feel native to the OS. WinUI 3 should make this an easy transition. Some things that come to mind:
General

[ ] Combining the remap key and shortcut into a single experience, making it a true "Keyboard Manager".
[ ] Ability to (temporarily) turn on / off certain remappings.
[ ] Adopting WinUI 3 (and it's default styling for Windows 11)
[ ] Resolve common accessibility issues by adopting standard UI elements and colors
[ ] Modern input (touch / pen)

Issues we can address with this UI redesign: #15340, #15340, #13382, #10552, #10042, #8798, #8793, #7304, #7070, #7054, #7054, #7054, #2170
Mock-up

WinUI 2.6 (UWP) source code: https://github.com/niels9001/KBMUX
Let's use this issue to track UI related concerns, feature requests and other feedback!

@niels9001 In the beginning of januari I sent you an e-mail with an idea/question, about helping. Did you see it?

#16286

#28533

Wouldn't it make more sense to integrate the editor into Settings than having it in a separate app/window like some people said in #16093? Or is it easier to maintain it separately?

Wouldn't it make more sense to integrate the editor into Settings than having it in a separate app/window like some people said in #16093?

It sure does!

Updated the original post (and source code) with the latest changes being worked on in: https://github.com/microsoft/PowerToys/pull/30121

Updated the original post (and source code) with the latest changes being worked on in: https://github.com/microsoft/PowerToys/pull/30121

@niels9001
Small note: With the new ui with different "tabs" there is no compact overview on all remapped keys/shortcuts anymore. How do you like to prevent multiple reassignments?


[Storybook][Docs][Table] Needing to add 'forge-ignore' to checkbox delegates needs improved documentation

Describe the bug:
Missing/lacking docs.
Needing to add the forge-ignore attribute to the checkbox delegate in a column filter is not well documented.
To Reproduce:
N/A
Expected behavior:
There should be a "common issues" section or something similar to capture hard-to-categorize issues and implementation details like adding the forge-ignore to specific delegates. It does seem to be mentioned under the  "selectAllTemplate" property but is not visible or clear enough.
Screenshots:
forge-ignore in table code
Commit showing how to solve
Please complete the following information:

Forge version: 2.x
I have searched existing issues before creating this report? Y
Browser: N/A
Platform: N/A
OS: N/A

Additional context:
No bug. Just a request to revisit docs.

I agree that we should add a note about using forge-ignore so developers know what it is and how it's used, but this comes down to being a bug in the table component. A similar issue is noted in #122, and the fix for that should solve this one.
Any <input type="checkbox"> elements within a custom cell template or a filter cell should be ignored by the table component, and it appears that they are not right now which is incorrect.
I'll likely batch the changes for both of these together in one PR.


Set-TssSecret split into other commands?

I am finding that we have many endpoints to deal with on properties with secrets:
Field updates can be done in multiple ways:

PUT /secrets/:id/fields/:slug
PATCH /secrets/:id/general
PUT /secrets/:id

While properties have a similar issue, but certain properties fall only under certain endpoints:

PUT /secrets/:id - common properties (but not all)
PATCH /secrets/:id/email - email properties sendEmailWhenChanged, sendEmailWhenHeartbeatFails, and sendEmailWhenViewed
PATCH /secrets/:id/general - some general properties such as active, folder (this will move it to another folder), generateSshKeys, enableInheritSecretPolicy, site, and even template
PATCH /secrets/:id/security-checkout - this one has checkOutChangePasswordEnabled, checkOutEnabled, and checkOutIntervalMinutes
PATCH /secrets/:id/security-general - has doubleLockId, enableDoubleLock, hideLauncherPassword, proxyEnabled, requiresComment, and sessionRecordingEnabled

Having all of the above supported within just Set-TssSecret could be done, but it would involve expanding the parameter list a large bit.  If I break it out into other commands to make the test more manageable it could also be considered whether to use a different verb (i.e., Update-TssSecret or Edit-TssSecret). Edit, may work better as it relates to the permission that is required as a minimum?
Each endpoint above also has varying output but from the perspective of the command itself, you generally will see commands that change or update data only return true or false to indicate they were successful or not.
So the option would be to either expand the command as-is to just have dozens of properties which give the advantage of discovering what can be set on a secret:
Examples if I kept this as one command:
# enables email when changed setting
Set-TssSecret -Session $session -Id 43 -EmailWhenChanged

# disables email when changed setting
Set-TssSecret -Session $session -Id 43 -EmailWhenChanged:$false

Found a fairly clean way to do this and think it will be manageable. The test for this function will likely be one of the longest in the whole module 😆 😢
Some settings for secrets will be put in the Update command ( #14 ) even though they likely could be put in this one it seems fitting that the few settings that change the "state" go under the Update verb.


17 2 release notes

what about calling this one out: https://github.com/GoogleCloudPlatform/google-cloud-intellij/issues/946
Since it may be weird to include this as a regular line item (since no change was needed), maybe we can call it out at the top instead?
I'm thinking this may be useful to see.


Allow returning custom error status codes and responses

At the moment, brokerapi users can return predefined error types to control status codes and response bodies: ErrRawParamsInvalid, ErrInstanceAlreadyExists, etc. Other errors always throw a 500. What do you think about accepting an error type that would allow developers more control over error responses, like this:
type APIError struct {
	StatusCode int
	Body map[string]interface{}
}

func (e APIError) String() string {
	body, err := json.Marshal(e.Body)
	if err != nil {
		return "Unknown error"
	}
	return string(body)
}

Then we'd be able to move long switch statements like https://github.com/pivotal-cf/brokerapi/blob/master/api.go#L103-L128 into a series of error instances, and users would be able to customize errors. WDYT?

Hey @jmcarp,
Sorry for the delayed response, I think adding support for custom http codes is great idea.
The broker spec states:
 Broker failures beyond the scope of the well-defined HTTP response codes listed above (like 410 on delete) should return an appropriate HTTP response code.
Would you be willing to submit a PR?
Thanks,
Sam


issue with following strategy : RemovePodsViolatingTopologySpreadConstraint

I am trying the above strategy with scale down scenario ( decreasing replica count) to check whether is respecting maxskew constraint or not  , but always i am seeing number of evicted pods is zero , can anyone please help with this . Thanks in advance.

Hi, can you share more information please? Your descheduler config, which version of descheduler+k8s you are running, and if possible a sample pod or deployment spec would be helpful. Thanks!

Yes ,  k8s version : 1.22 , and descheduler version : 0.23 ,
following is de-scheduler configMap:
apiVersion: v1
kind: ConfigMap
metadata:
name: descheduler-policy-configmap
namespace: kube-system
data:
policy.yaml: |
apiVersion: "descheduler/v1alpha1"
kind: "DeschedulerPolicy"
strategies:
"RemovePodsViolatingTopologySpreadConstraint":
enabled: true
params:
includeSoftConstraints: false
Following is the topology spread constraint spec :
spec:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: test-app

sharing the sample logs that i am getting  :
I0313 12:00:53.613868       1 node.go:165] "Pod does not fit on node" pod="test/test-65b77f57c8-fx6wv" node="ip-10-104-34-44.ec2.internal"
I0313 12:00:53.613889       1 node.go:169] "Pod fits on node" pod="test/test-65b77f57c8-fx6wv" node="ip-10-104-44-184.ec2.internal"
above logs i am getting but at the end  always i am getting evicted pods count=0, is it expected only?
my expectation here is if the evicted pod count is some number ( not zero) then some pod eviction is happening to balance the pods according to the maxskew but it is always giving evicted pod count is zero.

Yes ran like above now able to see the proper logs , thank you so much for the quick help .

@LAVA-KUMAR was that able to resolve your issue? If so I'll close this. Feel free to reopen if you need more assistance
/close

Yes able to resolve the issue . Thanks for the help.

Hi following is my ploicy:
apiVersion: v1
kind: ConfigMap
metadata:
name: descheduler-policy-configmap
namespace: kube-system
data:
policy.yaml: |
apiVersion: "descheduler/v1alpha1"
kind: "DeschedulerPolicy"
evictLocalStoragePods: true
maxNoOfPodsToEvictPerNode: 2
strategies:
"RemovePodsViolatingTopologySpreadConstraint":
enabled: true
params:
labelSelector:
matchLabels:
app: test
includeSoftConstraints: false
q1)   Even I have applied the label filter for the app "test" ,  descheduler tries to check the pods related to the other apps also . Am I applying label filter properly ?

q2) here I have added the constraint called : maxNoOfPodsToEvictPerNode: 2 , even though I am applying this constraint , it seems it's evicting more than two pods and also getting following error in the logs :
topologyspreadconstraint.go:182] "Error evicting pod" err="Maximum number 2 of evicted pods per \"ip-10-109-44-197.ec2. what is this log saying?

For 1) are the pods actually being evicted or just checked? The evictor might be looking at these pods but then ignoring them once it sees the labels

similar question, are the pods actually being evicted? That error message indicates that the descheduler is ignoring those pods because it has already hit the maximum limit

Can you share your full descheduler log? There could be a lot more context there

Hi, for the following response that you have given : " are the pods actually being evicted or just checked? The evictor might be looking at these pods but then ignoring them once it sees the labels"
Is there any way can configure such way that descheduler(evictor) only looks for the pods that are configured through labels , because it may give extra throttling right ?  is there any way we can avoid checking the pods that are not  related to the label that we set ? Thanks in advance .

@damemi  can you please respond to the above query .

No, for some strategies like topology spreading, the descheduler needs to at least know about all of the pods in the topology in order to calculate the skew accurately. It then filters out which pods to evict based on the labels that match the selector. There isn't a good way to run this strategy against only a subset of a topology. You might want to think about placing those pods you are concerned with in their own separate topology instead.

Yes got it thank you for the response  , but let say I have a deployment called app name is  : test , let say this deployment  have 5 replicas  , where my topology spread constraint defined in following way :
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: test
and consider I have two zones called (us-east-1a, us-east-b),  now the expectations is zone-a should contains 3 replicas ad zone-b contains 2 replicas now in this case it is balanced according to maxskew across zones .
Now if it is imbalanced means de-scheduler should do the balance and try to evict the pods to do the balance( this is the expectation here )  . Now to do this it is enough to check the pods which are related to the  label filter that we set right?
am I missing anything here ?  Can you please confirm this

Yes got it thank you for the response  , but let say I have a deployment called app name is  : test , let say this deployment  have 5 replicas  , where my topology spread constraint defined in following way :
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: test
and consider I have two zones called (us-east-1a, us-east-b),  now the expectations is zone-a should contains 3 replicas ad zone-b contains 2 replicas now in this case it is balanced according to maxskew across zones .
Now if it is imbalanced means de-scheduler should do the balance and try to evict the pods to do the balance( this is the expectation here )  . Now to do this it is enough to check the pods which are related to the  label filter that we set right?
am I missing anything here ?  Can you please confirm this

Yes, using the label selector will limit the descheduler to checking pods with that label.

And also can you please confirm the following queries actually I am testing in staging eks cluster from last one week few things I have observed and have few queries prior to moving to production can you please confirm this .
q1) I saw that few times even though it is balanced across zones it did the eviction some times  , I am sharing the policy config and topology spread constraint config that I have used :
Policy :
apiVersion: v1
kind: ConfigMap
metadata:
name: descheduler-policy-configmap
namespace: kube-system
data:
policy.yaml: |
apiVersion: "descheduler/v1alpha1"
kind: "DeschedulerPolicy"
evictLocalStoragePods: true
strategies:
"RemovePodsViolatingTopologySpreadConstraint":
enabled: true
params:
labelSelector:
matchLabels:
app: test
includeSoftConstraints: false
Topology config for the app test :
spec:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: proxy
So across the  zones if there is imbalance pods according to maxskew acorss the zones then it should the balance this is expectation here .
Am I configuring properly the descheduler policy here , if anything miss configured in the policy config please correct me .
q2)  how many nodes descheduler can handle inside a single eks cluster with configured resources .
Thanks in advance.

@LAVA-KUMAR can you share the logs from your descheduler? Preferably at log level v=4 if possible. There is not much I can do to debug your issue without seeing what is happening.
I do notice that your descheduler labelselector has app: test and your topology config has app: proxy. This could be the problem.
For q2) we don't have an exact max load but you can expect performance to degrade proportionally in very large clusters.

"I do notice that your descheduler labelselector has app: test and your topology config has app: proxy. This could be the problem".
Sorry here this is wrongly typed , app name was here "test" both places . Thanks.

That's interesting, is it always the 2 logstash-production/common-logstash-prod pods that are evicted? Maybe you could share the pod yaml for those pods which might tell us more.

Here in my app i have replica count as 6 ( in one zone it has 4 and other zone it has 2 which was actually balanced according to the maxskew 1 that was set) .

Maybe this was a typo, but 4 & 2 is not balanced with maxskew=1. Still, it should only be trying to evict 1 pod to balance them

Hope this helps :
apiVersion: apps/v1
kind: Deployment
metadata:
name: logstash
annotations:
cluster-autoscaler.kubernetes.io/safe-to-evict: 'true'
spec:
revisionHistoryLimit: 5
replicas: 0
template:
spec:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: common
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: topology.kubernetes.io/zone
operator: In
values:
- us-east-1a
- us-east-1b

Hi make , any update on above ?


[Frontend] Create navigation component

Description
Refer the ui design and create the navigation component
Screenshot

Colours

link
For reference you can see the design prototype :
https://www.figma.com/proto/dMw9GAeJja2baVhDoceAEB/Untitled?node-id=9%3A48&viewport=-1544%2C148%2C0.4358828365802765&scaling=contain

You can use permanent drawer to implement is easily: https://material-ui.com/components/drawers/#permanent-drawer
This issue only require you to make a drawer and style it according to the view posted.

Should I go ahead with the issue or someone else  want to take it ?

Can I  work on this if @rizwan2000rm you have no problem 😅

Sure go ahead. I have no issues at all.

I just have to create this navigation component right ? Or I have to return it also somewhere ?

You have to create a new route and a new page inside it you have to make this component.

I am not that aware how to route , but will try and look up to it

As of now I have created a component in src/component with the name Navigation.js and designed this there . I know some designing part is remaining . @Abhishek-kumar09 @SHITIZ-AGGARWAL

This is how the component looks !

Create a PR, we will discuss further details there.
Also, the unselcted ones should have some faint texture than selected ones, are u facing some problem implementing that?

Yes , actually as I am mapping through all the elements instead of creating individual element as In that case I would have given active className  to the one which is activated but I don't know how to do conditional rendering while mapping .

@Abhishek-kumar09 We can close this issue !


[Hyperion] Getting error on updating the chart if it is an external chart which is deployed from terminal using helm install

Chart name : bitnami/apache

Checklist:

[x] I've searched in the docs and FAQ for my answer
[x] I've included steps to reproduce the bug.

Proposed solution
On updating the EA release check conditionally on UI whether is external or internal, If it is internal then ask for linking (repo/version/chart/values) etc.. and save it in DB.

Proposal
UI Link: https://www.figma.com/file/j7cqlBdhQ47Bo10TLfamGl/?node-id=403%3A737
Show error below chart dropdown if a chart is not linked to the app.
User can select a chart from dropdown to link.
When user opens dropdown, show charts matching with chart name; else show empty state
In dropdown, show option to go to chart repositories (open in new tab) [this will be for all helm apps not specific to external]
The refresh button next to the chart dropdown updates the dropdown with latest data.
After a chart is selected:
Show chart version and values dropdowns
Chart version: Select version used in helm app if available; otherwise select latest version
Chart values: In dropdown, show app name in existing section; don't show chart version. (to discuss)
After the app is deployed using a chart, the selected item will be moved to deployed section; show chart version.
BUG:
Show README for selected chart; Currently, it shows Readme for chart used for deployment not the one selected
Other:
Chart repo add toast text change

Tech doc :
https://docs.google.com/document/d/16sHjjIDA3jcsghVTjqyS_QmK-x6qgNFPEgVi_7I3LvM/edit


Error when following the Sample Data Page

I am fairly new to InfluxDB and I am trying to load sample data into a database.
I am using the latest version of InluxDB on Windows.
Here is the page I am following: https://docs.influxdata.com/influxdb/v1.7/query_language/data_download/
After loading the data in the terminal with:
curl https://s3.amazonaws.com/noaa.water-database/NOAA_data.txt -o NOAA_data.txt
When I am trying to write data into influx via the influxdb shell with:
influx -import -path=NOAA_data.txt -precision=s -database=NOAA_water_database
I am getting this error:
ERR: error parsing query: found influx, expected SELECT, DELETE, SHOW, CREATE, DROP, EXPLAIN, GRANT, REVOKE, ALTER, SET, KILL at line 1, char 1
Somebody got a solution for this?

I have the same problem. Not being able to load in sample data makes it difficult to try out influxdb.

I figured this out. You have to omit the -database argument:
influx -import -path=NOAA_data.txt -precision=s -database=NOAA_water_database


Proposal: Add sanitize function to prevent css value injection

Hi,
I am a newcomer to styled-components! Really enjoyed using it for the first time! One of the things that caught me out was sanitising input before using it as a dynamic style (And I'd imagine it's likely to catch more than just me out). The Security document explains what could happen really well, and I soon realised where I was going wrong.
Developers should be sanitising inputs but I envisage this as a 'defensive' option:
import styled, { sanitize } from 'styled-components';

const Button = styled.button`
  /* Adapt the colors based on primary prop */
  background: ${props => sanitize(props.primary)};
`;

export default Button;

I saw a discussion in #622 which discusses how a sanitise function would be hard for a prop which represents a whole css block, but for sanitising a prop which is a css value is surely relatively simple. Stripping anything from a ;? I may be wrong but I can't think of a value which would have a semi-colon in?
I was thinking sanitize, santizeCssValue or santizeValue?
More than happy to have a go at implementing if you think this is a worthwhile feature.

We're not looking into adding helpers that aren't immediately needed or make sense in the "narrative" of the API. Also, such a helper would be easily written, I assume, and only be needed if user input is actually accepted, so I'd say this is up to the users :smile:
cc @geelen