Infinite Looping on "mustNotHaveValue"

Let's consider sample program and nuitka call for compilation:
https://github.com/belonesox/nuitka-looping-mustNotHaveValue

pipenv-based setup-demo.bat
on demo_cycling.bat we got infinite loop

File "C:\ta-buildroot\python-x86-3.7.9\lib\site-packages\nuitka\optimizations\ValueTraces.py", line 560, in mustNotHaveValue
    if not previous.mustNotHaveValue():
  File "C:\ta-buildroot\python-x86-3.7.9\lib\site-packages\nuitka\optimizations\ValueTraces.py", line 383, in mustNotHaveValue
    return self.previous.mustNotHaveValue()
  File "C:\ta-buildroot\python-x86-3.7.9\lib\site-packages\nuitka\optimizations\ValueTraces.py", line 560, in mustNotHaveValue
    if not previous.mustNotHaveValue():
  File "C:\ta-buildroot\python-x86-3.7.9\lib\site-packages\nuitka\optimizations\ValueTraces.py", line 383, in mustNotHaveValue
    return self.previous.mustNotHaveValue()
  File "C:\ta-buildroot\python-x86-3.7.9\lib\site-packages\nuitka\optimizations\ValueTraces.py", line 560, in mustNotHaveValue
    if not previous.mustNotHaveValue():
  File "C:\ta-buildroot\python-x86-3.7.9\lib\site-packages\nuitka\optimizations\ValueTraces.py", line 383, in mustNotHaveValue
    return self.previous.mustNotHaveValue()
  File "C:\ta-buildroot\python-x86-3.7.9\lib\site-packages\nuitka\optimizations\ValueTraces.py", line 560, in mustNotHaveValue
    if not previous.mustNotHaveValue():
  File "C:\ta-buildroot\python-x86-3.7.9\lib\site-packages\nuitka\optimizations\ValueTraces.py", line 383, in mustNotHaveValue
    return self.previous.mustNotHaveValue()
  File "C:\ta-buildroot\python-x86-3.7.9\lib\site-packages\nuitka\optimizations\ValueTraces.py", line 560, in mustNotHaveValue
    if not previous.mustNotHaveValue():

I reproduce it on Windows with python-x86-3.7.9 and nuitka = "<IP_ADDRESS>", but I will try it on Linux and other versions of Nuitka.
Also I will try to simplify testcase

What do you mean with infinite looping, endlessly slow or actual recursion error from Python?

Can you try the factory version, I recently discovered that many merges are done that are not necessary, and will no longer be done, these would lead to merges of merges potentially as well, so that version would definitely perform better.

What do you mean with infinite looping, endlessly slow or actual recursion error from Python?

«endlessly slow»
Then I interrupt process (on debugging or Ctrl-C), I get such stack and variables (something wrong about "sklearn")

Can you try the factory version,

Yes, I will try all this. I also will try it on different versions of python and on Linux (may be something wrong with my 3.7.9, even on pipenv, I should check on clear box).

With factory branch
(demo https://github.com/belonesox/nuitka-looping-mustNotHaveValue/tree/07a26e53c4819c9dcbcd77a8f8252bef5f7526e6 )
I got crash
Nuitka:INFO: Total memory usage before generating C code: 653.21 MB (684941312 bytes):
Problem with <Node 'EXPRESSION_BUILTIN_ANONYMOUS_REF' with {'builtin_name': 'version_info'}> at c:\users\stas\.virtualenvs\nuitka-looping-mustnothavevalue-dc8w5ryp\src\nuitka\nuitka\constants\Serialization.py:119
Problem with <Node 'STATEMENT_ASSIGNMENT_SUBSCRIPT'> at c:\users\stas\.virtualenvs\nuitka-looping-mustnothavevalue-dc8w5ryp\src\nuitka\nuitka\constants\Serialization.py:119
Problem with <Node 'STATEMENT_TRY' with {'aborting': True}> at c:\users\stas\.virtualenvs\nuitka-looping-mustnothavevalue-dc8w5ryp\src\nuitka\nuitka\constants\Serialization.py:102
Nuitka:WARNING: Problem creating function code 'nuitka$constants$Serialization$$$function__8___init__'.
Traceback (most recent call last):
  File "c:\users\stas\.virtualenvs\nuitka-looping-mustnothavevalue-dc8w5ryp\src\nuitka\nuitka\__main__.py", line 137, in <module>
    main()
  File "c:\users\stas\.virtualenvs\nuitka-looping-mustnothavevalue-dc8w5ryp\src\nuitka\nuitka\__main__.py", line 123, in main
    MainControl.main()
  File "c:\users\stas\.virtualenvs\nuitka-looping-mustnothavevalue-dc8w5ryp\src\nuitka\nuitka\MainControl.py", line 915, in main
    result, options = compileTree()
  File "c:\users\stas\.virtualenvs\nuitka-looping-mustnothavevalue-dc8w5ryp\src\nuitka\nuitka\MainControl.py", line 784, in compileTree

build.log

Same "factory crash" on linux
https://github.com/belonesox/nuitka-looping-mustNotHaveValue/tree/6336ea80ea9b96a757592d7547d63139fe66edb9
setup-demo.sh
demo_cycling.sh

↓↓↓
build.log

About «infinite looping», on windows/python 3.8.10, demo  → https://github.com/belonesox/nuitka-looping-mustNotHaveValue/tree/windows-python-3.8.10

On <IP_ADDRESS> — no looping, OK
On >=<IP_ADDRESS> — looping exists.

So, this doesn't affect current stable version then?

It Affects!
<IP_ADDRESS> >= version >=<IP_ADDRESS> — looping exists.
Including «<IP_ADDRESS>» (current stable?!) and «<IP_ADDRESS>», but not including «<IP_ADDRESS>»

It seems your factory crash is on compiling Nuitka code?!. Does your code import nuitka somehow?

also, from looking at the command, you really want the anti-bloat plugin, and maybe you want to add a rule to the command line that makes it error out on importing `nuitka`` so you can tell where it is coming from.

It seems your factory crash is on compiling Nuitka code?!. Does your code import nuitka somehow?

Please, look

https://github.com/belonesox/nuitka-looping-mustNotHaveValue/tree/windows-python-3.8.10 — reproduce on Windows
https://github.com/belonesox/nuitka-looping-mustNotHaveValue/tree/linux-python-3.9.7 — reproduce on Linux

There is no nuitka imports in «test.py»
I did not call anti-bloat plugin in demo-cycling

The new compilation report feature would be handy, I checked you log, and I am seeing this:
:INFO: Optimizing module 'setup', 184 more modules to go after that.
:INFO: Doing module local optimizations for 'setup'.
:INFO: Not finished with the module due to following change kinds: new_constant,new_expression,new_statements,trusted_module_variables,var_usage
:INFO: Not finished with the module due to following change kinds: loop_analysis,new_builtin,new_builtin_ref,new_constant,new_expression,new_statements,var_usage
:INFO: Not finished with the module due to following change kinds: loop_analysis,new_expression,new_statements
:INFO: Not finished with the module due to following change kinds: loop_analysis
:INFO: Not finished with the module due to following change kinds: loop_analysis
:INFO: No changed, but retrying one more time.
:INFO: Finished with the module.
:INFO: Optimizing module 'nuitka.PythonFlavors', 184 more modules to go after that.

So it seems something is causing an import of setup which ends up being the one from Nuitka, probably being in your current directory.  Maybe it's shadowing something. Nuitka has started to use Nuitka code in its setup.py, which is why you have that. You probably also have issues stable from that. With the anti-bloat plugin, you could find the module that imports setup.
Also the compilation report feature should come in handy, it lists the imports of a module, but not in detail yet, but I can change that. And it would have showed the same, namely setup importing Nuitka, and then what imports setup, we cannot know.
Since factory has the crash fixed, you might actually get that report. However, compilation needs to finish for it to exists, which is why anti-bloat command line options are probably best to find it.

I have no idea what makes you import nuitka there, that seems rather grave. I have not yet attempted reproduction though, but I am hoping that nothing in Nuitka makes Nuitka go into itself there.

Please, try to reproduce, using my link above.
Since first comment, I do a lot of minimizing, and now there are nothing about "nuitka.PythonFlavors" or "anti-bloat" in logs.
build.log

This is during code generation, that one has never so far had any such issues. You interrupted it, are you saying it takes very long time? Anyway, I am trying to reproduce it now, discovering that the anti-bloat plugin has not seen sklearn.utils._testing before, which is an indication that I at least have not compiled this part of sklearn, or not this version.

You interrupted it, are you saying it takes very long time?

More that 12 hours, sometimes I run it all night without ending.

It finished for me in definitely less than one hour, I am using --enable-plugin=anti-bloat --noinclude-default-mode=error, I think you never got my anti-bloat references, but it reduces the compilation by a lot. It took me max 30 minutes, I didn't check closely.
I am suspecting you are reaching a low memory situation which can make things slow. Still with anti-bloat, 2k modules are compiled, which is borderline, but there is a lot that anti-bloat doesn't do yet.
I have had to find and fix a regression in the generated C code for a string operation, therefore I don't know if it runs at all, but I think it ought to. But with these large compilations I am waiting for more anti-bloat work to happen over time.

Your proposed option  --noinclude-default-mode=error does the magic:

with it my test case finished OK
without it, this test case hanged.

I never use the option, but now I see that if  my code includes for example
from setuptools import setup as old_setup

it failed with
Nuitka-Progress:INFO: Doing module local optimizations for 'numpy.distutils.core'.
Problem with statement at C:\Users\stas\.virtualenvs\nuitka-looping-mustNotHaveValue-MGvTvGQE\lib\site-packages\numpy\distutils\core.py:6:
->     from setuptools import setup as old_setup

Problem with statement at C:\Users\stas\.virtualenvs\nuitka-looping-mustNotHaveValue-MGvTvGQE\lib\site-packages\numpy\distutils\core.py:4:
-> if 'setuptools' in sys.modules:

FATAL: Error, forbidden import 'setuptools' encountered.

I think you should outlaw numpy.distutils.core already. I think in anti-bloat I have made a couple of packages avoid that. That is a distutils clone, that basically explodes dependencies all around. Can you use the custom mode to track this down?

I have tried
--enable-plugin=anti-bloat --noinclude-default-mode=error --noinclude-custom-mode=setuptools:nofollow
it leads to this looping.
I tried to track who "is called setuptools" (I thought that «--graph» option suits this, but, obviously, it does not work for that, see  #1341).
I think  there should be some debug print, like
def onModuleOptimizationStep(module):
    # Update the graph if active.
    print("onModuleOptimizationStep", module)

to track down what module leeds to «FATAL: Error, forbidden import 'XXXXXX' encountered.»
I tracked down to some subpackages of  «scipy.special» (my project really needed it),
have tried a lot of magic like "--include-package=XXX" + "--nofollow-import-to=XXX"
and …  still have «infinite looping» even with «--enable-plugin=anti-bloat --noinclude-default-mode=error »
python.exe -m pipenv run python -m nuitka  --show-scons --windows-disable-console --standalone --show-progress --plugin-enable=numpy  --enable-plugin=anti-bloat --noinclude-default-mode=error --include-package=scipy.special --include-module=scipy.special.cython_special --nofollow-import-to=scipy.special --include-module=skimage.feature._orb_descriptor_positions --include-module=scipy.spatial.transform._rotation_groups  --follow-imports --include-module=scipy.spatial.transform._rotation_groups test.py >build.log 2>&1

↓↓↓↓
build.log
( https://github.com/belonesox/nuitka-looping-mustNotHaveValue/tree/windows-python-3.8.10 )
What else should I try?

I would also say that the memory usage reported, 367MB is not too bad given the 2k modules.
I will add traces, such that it reports the module you interrupted code generation on. It should be interesting to see if there is one that is particularily bad.

Please retry with current factory branch and see if it is the same module all the time for you. Then of course let us know which one it is. :)

about current factory brach… It takes a lot of time to work with my project with "--noinclude-default-mode=error --enable-plugin=anti-bloat"
(because of "--nofollow-import-to=scipy.special" I have to catch all needed modules manually, after long cycles of build-run-"no-module found"). Then, back, trying to minimize a problem case…
Now I have a demo
https://github.com/belonesox/nuitka-looping-mustNotHaveValue/tree/windows-python-3.7.9

trivial python file

https://github.com/belonesox/nuitka-looping-mustNotHaveValue/blob/windows-python-3.7.9/test.py

build  https://github.com/belonesox/nuitka-looping-mustNotHaveValue/blob/windows-python-3.7.9/build-wtf-with-sf-error.bat
(mind «--include-module=scipy.special.sf_error» in the options) without problems
build-log.zip

but "ModuleNotFoundError: No module named 'scipy.special.sf_error'" when we run (providing all needed DLLS)

Traceback (most recent call last):
  File "C:\projects\nuitka\3.79\NUITKA~1\TEST~1.DIS\test.py", line 3, in <module>
  File "C:\projects\nuitka\3.79\NUITKA~1\TEST~1.DIS\sklearn\__init__.py", line 82, in <module sklearn>
  File "C:\projects\nuitka\3.79\NUITKA~1\TEST~1.DIS\sklearn\base.py", line 17, in <module sklearn.base>
  File "C:\projects\nuitka\3.79\NUITKA~1\TEST~1.DIS\sklearn\utils\__init__.py", line 28, in <module sklearn.utils>
  File "C:\projects\nuitka\3.79\NUITKA~1\TEST~1.DIS\sklearn\utils\fixes.py", line 20, in <module sklearn.utils.fixes>
  File "C:\projects\nuitka\3.79\NUITKA~1\TEST~1.DIS\scipy\stats\__init__.py", line 441, in <module scipy.stats>
  File "C:\projects\nuitka\3.79\NUITKA~1\TEST~1.DIS\scipy\stats\stats.py", line 37, in <module scipy.stats.stats>
  File "C:\projects\nuitka\3.79\NUITKA~1\TEST~1.DIS\scipy\spatial\__init__.py", line 102, in <module scipy.spatial>
  File "C:\projects\nuitka\3.79\NUITKA~1\TEST~1.DIS\scipy\spatial\_geometric_slerp.py", line 8, in <module scipy.spatial._geometric_slerp>
  File "C:\projects\nuitka\3.79\NUITKA~1\TEST~1.DIS\scipy\spatial\distance.py", line 124, in <module scipy.spatial.distance>
  File "C:\projects\nuitka\3.79\NUITKA~1\TEST~1.DIS\scipy\special\__init__.py", line 641, in <module scipy.special>
ModuleNotFoundError: No module named 'scipy.special.sf_error'

It seems that sklearn needs a treatment like numpy for libraries, since I mostly use Linux, I never encountered this, should be easy to add. Otherwise, are you saying you are getting 'scipy.special.sf_error' after using --nofollow-import-to=scipy.special ?
I was wondering, if Nuitka should not start to treat these ModuleNotFoundError that escape the main program as an unhandled exception. I could include code that tells you WHY this is not not inlcuded. I will admit that the workflow you are using there is tedious.
Do yourself a favor and start using "nuitka-project" options in a file, it becomes way more readable. It took me 3 attempts to check if it's there. With project options they are in the Python file and easier to read and enable/disable. If you don't mind a small wait, let this case untouched, and I will try and make it output a nice error, and make your copy of MSVC libraries unnecessary, ok?

It seems that sklearn needs a treatment like numpy for libraries, since I mostly use Linux, I never encountered this, should >be easy to add.

May be I should provide publicly available VPS with windows, that you can use to reproduce such bugs? Should it help?

Otherwise, are you saying you are getting 'scipy.special.sf_error' after using --nofollow-import-to=scipy.special ?

Roger that.
My nuitka options: https://github.com/belonesox/nuitka-looping-mustNotHaveValue/blob/windows-python-3.7.9/build-wtf-with-sf-error.bat
As far as I know «--nofollow-import-to=XXX» + «--include-module=XXX.yyy» is the only option for me.

Do yourself a favor and start using "nuitka-project" options in a file, it becomes way more readable.

In my workflow, all these options are generated from some YAML files, so I missed that option, I will learn it and use it for demo bugs.

Just tested on
   nuitka = "<IP_ADDRESS>"
   python_version = "3.9"

on windows and linux, there is no "infinite looping" anymore.
So the bug can be closed.
Some other issues I will investigate and report by separately.

Good, you didn't notice that 0.7 is out since today. :)


Certificate support and more flexibility in repo retrieval

Fixed the conflicts and also fixed some shellcheck warnings and merged this in https://github.com/CatalystCode/project-fortis/commit/0a46e2accd43f62830294936b58c3bc9f663122c.


Application for Parkland

Application for Parkland license

Hi JetBrains, just adding a request for educational license for Parkland College.

@AstroGuy1 I am sorry, the domain stu.parkland.edu is removed from our trusted domains list because it was abused in the past. If you are the student of this school, please enroll to GitHub Student Developer Program. Then apply for JetBrains educational subscription with your GitHub account (GitHub tab at https://www.jetbrains.com/shop/eform/students).


Last branch sync for main-> dev-1.22

Last branch sync before the release, this will merge the frozen main branch into the frozen dev-1.22 before we merge 1.22 back into main.
/cc @ashnehete @carlisia @chrisnegus @ritpanjw
/assign @reylejano @annajung
/milestone 1.22

The last commit in this PR has failed because there's a freeze on the repo, which means i'll have to merge it manually, otherwise there are no failed commits here.

perfect! thanks Victor
/lgtm

/hold for manual merge
/approve
remove hold when ready for manual merge

/hold cancel
thanks!


Mac installation missing dfu-util step?

Hi! I'm on a Mac running macOS Monterey, with an Intel Core i7 processor.
When I ran $ make program-dfu I got the following error:
dfu-util -a 0 -s 0x08000000:leave -D build/harmonic_osc.bin -d ,0483:df11
dyld[3956]: Library not loaded: '/usr/local/opt/libusb/lib/libusb-1.0.0.dylib'
  Referenced from: '/Library/DaisyToolchain/0.1.2/bin/dfu-util'
  Reason: tried: '/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' (no such file), '/usr/local/lib/libusb
-1.0.0.dylib' (no such file), '/usr/lib/libusb-1.0.0.dylib' (no such file)
make: *** [program-dfu] Abort trap: 6

I was able to resolve it by re-installing and linking dfu-util:
$ brew install dfu-util
$ brew link --overwrite dfu-util

Happy to open a PR to add this to the Troubleshooting section if that would be helpful.

We would greatly appreciate it if you could open a PR and we'll have a closer look. Thank you.

Hello! Just ran into a similar issue during bring-up. Seems like dfu-util actually has some issues with being bundled improperly. If you check out this conversation (here)[https://github.com/pine64/pine64_updater/issues/6] you can see that the dfu-util package seems to have been the problem. (Here)[https://github.com/pine64/pine64_updater/commit/bb26c7982a8a847d74edc3c0cddba21ffe73a403] is the fix in the repo that this issue was brought up in. Looks like they just updated the bundled version in the toolchain. My best guess is that newer versions of Mac place the libusb library in a new location that dfu-util doesn't know to look for.
For anyone else reading this, workarounds for the issue involve updating your dfu-util to a newer version and making sure that the program-dfu make target is using it.
Steps to run on command line that worked for me:
which dfu-util
prints out the location of the dfu-util installed via the toolchain
sudo rm -rf /location/of/the/path/returned/from/which/dfu-util/
removes the offending dfu-util version. You don't have to do this, but I found it easier than trying to link to the new version
brew install dfu-util
Just installs the new version of dfu-util. At the time of this comment, I have successfully built and run dfu-util 0.11 with the Blink example code on Mac M1 Pro Ventura 13.4
dfu-util
Attempt to run the newly installed dfu-util. If it didn't work, you'll get the same error. If it did work the help command will print
That should be all you need. If the dfu-util command isn't showing up you might need to reload your terminal.

Hi everyone!
This issue should be resolved via the newer Mac OS installer for the toolchain, so there should be no more need to install dfu-util and etc via brew.
Also, please note that if you're up and running at this point, there's no need to reinstall anything. However, if you're still having issues, we suggest that you install via the Daisy Toolchain installer.
We'll close this issue as there should be no need to edit the wiki. We greatly appreciate both of your suggestions and documenting the solutions and workarounds. Thanks!


Builder umessage

This PR implements builder for UMessage

Since this build creates a message with empty payload and just attributes, there are not many tests that we need to do on it but it would make sense to add one more test , that tests the actual message been built for correct attributes.

@ruchirchauhan For the sake of our maintainers, would you put a comment/reference in the description about how/why this PR should be merged after that-PR-link.

@ruchirchauhan - it looks like #161 is almost ready to merge. Once that's in, you should be able to rebase and we can (hopefully) wrap up this PR too.

@ruchirchauhan - PR #161 has been merged, so you should be able to use the PayloadBuilder now. Please rebase onto main to get the latest changes.


Iris: Create and call endpoint to check if Iris is active

Checklist
General

[x] This is a small issue that I tested locally and was confirmed by another developer on a test server.
[x] Language: I followed the guidelines for inclusive, diversity-sensitive, and appreciative language.
[x] I chose a title conforming to the naming conventions for pull requests.

Server

[x] Important: I implemented the changes with a very good performance and prevented too many (unnecessary) database calls.
[x] I followed the coding and design guidelines.
[x] I added multiple integration tests (Spring) related to the features (with a high test coverage).
[x] I added pre-authorization annotations according to the guidelines and checked the course groups for all new REST Calls (security).
[x] I documented the Java code using JavaDoc style.

Motivation and Context
We want an endpoint (might be a websocket connection in the future).
Description
For now adds an endpoint which returns true.

We could indeed use a FeatureToggle to check if Iris has been globally activated. In this PR we are only checking if it is still alive and processes user queries.


Share the Python responsibilities

Share the Python responsibilities in detailed with Santhosh

Hey @satendrapalsingh - please add a proper description to the issue.

Invalid


Improvement: Too difficult resolve the default babel-loader

In my case, I need to use addWebpackModuleRule add thread-loader, cache-loader & babel-loader.
I m using react-app-rewired + customize-cra, I cann't getBabelLoader with addWebpackModuleRule

yeah,i want to Analyze the packaging time of the project，
my project  is getting bigger and the packaging speed is getting slower and slower


Different issue with Lumira Designer 2.3

Installed (Install Visualization SDK Extensions) the gauge on Lumira Designer 2.3.  But do not see it in the component list?!?

add a chart type then go to chart configuration and select the chart you want from the Additional Charts list (bottom right button)

Sorry for the slow response (I had forgotten I had submitted the question).  I found it. I just have a single value for the gauge.  Thus I now just have a yellow dot in a black circle with the value.  Not sure this will work for me.  Maybe I have something configured incorrectly.


ENH Support tx-translator

Issue https://github.com/silverstripe/cow/issues/234
Fix it not installing in CI https://github.com/silverstripe/silverstripe-tx-translator/actions/runs/4300037202/jobs/7495857923#step:9:274

Not required


[BugFix] Fix compareTo NPE when system privilegeCollection merge

What type of PR is this：

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes (https://github.com/StarRocks/StarRocksTest/issues/2126)
Problem Summary(Required) ：

When two PrivilegeCollections of type System are merged, the sorting logic is invoked, and the Object of System is NULL
Checklist:

[x] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[ ] I have checked the version labels which the pr will be auto backported to target branch

[x] 3.0
[ ] 2.5
[ ] 2.4
[ ] 2.3

@Mergifyio backport branch-3.0


Turn off validation if val_percent_check=0

As was suggested by @williamFalcon in https://github.com/williamFalcon/pytorch-lightning/issues/536#issuecomment-558115605 val_percent_check=0 should turn off the validation loop. But now it will not work because of
 self.num_val_batches = max(1, self.num_val_batches)

So I suggest to fix it. Moreover I suggest to make more thorough processing of train_percent_check and val_check_interval:

We should require all *_percent_check and val_check_interval to be in the range [0.0; 1.0].
Final num_val_batches can be equal to 0 that will effectively disable validation.
Final num_train_batches and num_test_batches should be at least 1. (See also #631)
Final val_check_interval should be at least 1.
The user can try to turn off validation by setting val_check_interval to a big value. Maybe in that case we should print a helpful message that validation can be turned off by setting val_percent_check=0.

Any thoughts?

I see. The max was introduced in a pull request that fixed an issue when self.num_val_batches was in the range (0, 1). One question, why should *_percent_check be in range [0, 1]. You mean [0, 100]? As the name suggests, it should be a percentage.

Yeah, you are right, the naming is not correct. It is not real percents but just fractions. So now it is assumed that 1. means to use the whole dataset.

This behavior seems to be absent in the documentation?

@cmpute mind sending a PRs with docs?

Sorry to exhume this old issue, but this feature doesn't seem to work any longer. These are my trainer arguments:
min_epochs: 1
max_epochs: 1
max_steps: 1000
num_sanity_val_steps: 0
overfit_batches: 1
log_every_n_steps: 1
val_percent_check: 0

I'm using this settings to completely avoid the validation dataset, and just overfit a single batch of data over 1000 training steps.

@miccio-dk this is a very old issue - I'd recommend creating a new one as the project has changed significantly since this was filed.
you may also be interested in this: https://github.com/PyTorchLightning/pytorch-lightning/issues/10888


[Feature Request] Have the JamfPackageUploader processor accept an option to not try to update the package metadata

Hi,
As discussed on Slack, would it be possible to add a key to the JamfPackageUploader processor to have it skip trying to update the package metadata?
This options requires "update" permissions on packages for the API account, which is...tricky....for us to grant at this time, and not using seems to have no ill benefits for us as for a functionality.
Thanks!

Just for visibility here, before looking into the technical side of things, skipping the metadata does have one ill benefit in that without the ability to update the pkg metadata, the SHA512 hash of the package cannot be set. Without this, the Jamf binary will not do a verification on the client that the package is what it says it is.
So while you do remove the ability for someone obtaining your AutoPkg account credentials to login to Jamf Pro and switch out a package (amongst all the other things it can do like create a policy containing a script that could do whatever they want...), you do expose that slight risk of something on the client itself switching out the package.

There would be a negative security impact from the change, but by making this opt-in I think it'll be ok as the admin's choice.
For us, our API creds used are locked to only allow uploading packages, no existing package modification (so existing packages that are being deployed can't be replaced), and no changes to scripts, policies, groups etc can be made with the same credentials. As for the risk on client, this wouldn't be mitigated but would require local admin rights to execute along with code execution.

I have added a skip_metadata_upload boolean option to JamfPackageUploader. I haven't yet pushed this to autopkg/grahampugh-recipes because my overnight run is about to start and this will break all my recipes until I upload the trust :-)
Feel free to copy this processor into your local clone of grahampugh-recipes and give it a go.

Hmm so with the changed option, the package upload seems to hang and timeout. I've pasted the relevant (redacted) output below.
The package shows in Jamf as "Upload failed"
JamfPackageUploader: Existing token is valid
JamfPackageUploader: curl command: /usr/bin/curl --dump-header /tmp/jamf_upload/curl_headers_from_jamf_upload.txt https://[Jamf URL]/api/v1/jamf-pro-version --request GET --silent --show-error --header authorization: Bearer [Bearer token] --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt --header Accept: application/json --cookie-jar /tmp/jamf_upload/curl_cookies_from_jamf_upload.txt --cookie /tmp/jamf_upload/curl_cookies_from_jamf_upload.txt
JamfPackageUploader: Jamf Pro Version: [Jamf Pro Version]
JamfPackageUploader: Token auth will be used,
JamfPackageUploader: curl command: /usr/bin/curl --dump-header /tmp/jamf_upload/curl_headers_from_jamf_upload.txt https://[Jamf URL].jamfcloud.com/JSSResource/packages/name/Microsoft%20Visual%20Studio%20Code-1.68.1.pkg --request GET --silent --show-error --header authorization: Bearer [Bearer token] --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt --header Accept: application/json --cookie-jar /tmp/jamf_upload/curl_cookies_from_jamf_upload.txt --cookie /tmp/jamf_upload/curl_cookies_from_jamf_upload.txt
JamfPackageUploader: ID: -1
JamfPackageUploader: Package 'Microsoft Visual Studio Code-1.68.1.pkg' not found on server
JamfPackageUploader: curl command: /usr/bin/curl --dump-header /tmp/jamf_upload/curl_headers_from_jamf_upload.txt https://[Jamf URL].jamfcloud.com/dbfileupload --request POST --silent --show-error --header authorization: Basic [Basic Auth token] --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt --output /tmp/jamf_upload/curl_output_from_jamf_upload.txt --upload-file /Users/[Username]/Library/AutoPkg/Cache/local.jamf.10xbanking-VisualStudioCode-pkg-upload/Microsoft Visual Studio Code-1.68.1.pkg --cookie-jar /tmp/jamf_upload/curl_cookies_from_jamf_upload.txt --cookie /tmp/jamf_upload/curl_cookies_from_jamf_upload.txt --header Accept: application/xml --header DESTINATION: 0 --header OBJECT_ID: -1 --header FILE_TYPE: 0 --header FILE_NAME: Microsoft Visual Studio Code-1.68.1.pkg --max-time 3600
curl: (28) Operation timed out after 3599999 milliseconds with 230 out of 230 bytes received
Traceback (most recent call last):
  File "/Library/AutoPkg/autopkglib/__init__.py", line 840, in process
    self.env = processor.process()
  File "/Library/AutoPkg/autopkglib/__init__.py", line 626, in process
    self.main()
  File "/Users/[Username]/Library/AutoPkg/RecipeRepos/com.github.autopkg.grahampugh-recipes/JamfUploaderProcessors/JamfPackageUploader.py", line 803, in main
    self.pkg_name, self.pkg_path, self.jamf_url, enc_creds, obj_id
  File "/Users/[Username]/Library/AutoPkg/RecipeRepos/com.github.autopkg.grahampugh-recipes/JamfUploaderProcessors/JamfPackageUploader.py", line 363, in curl_pkg
    data=pkg_path,
  File "/Users/[Username]/Library/AutoPkg/RecipeRepos/com.github.autopkg.grahampugh-recipes/JamfUploaderProcessors/JamfUploaderLib/JamfUploaderBase.py", line 371, in curl
    subprocess.check_output(curl_cmd)
  File "/Library/AutoPkg/Python3/Python.framework/Versions/Current/lib/python3.7/subprocess.py", line 411, in check_output
    **kwargs).stdout
  File "/Library/AutoPkg/Python3/Python.framework/Versions/Current/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/bin/curl', '--dump-header', '/tmp/jamf_upload/curl_headers_from_jamf_upload.txt', 'https://[Jamf URL].jamfcloud.com/dbfileupload', '--request', 'POST', '--silent', '--show-error', '--header', 'authorization: Basic [Basic Auth token]', '--output', '/tmp/jamf_upload/curl_output_from_jamf_upload.txt', '--output', '/tmp/jamf_upload/curl_output_from_jamf_upload.txt', '--upload-file', '/Users/[Username]/Library/AutoPkg/Cache/local.jamf.10xbanking-VisualStudioCode-pkg-upload/Microsoft Visual Studio Code-1.68.1.pkg', '--cookie-jar', '/tmp/jamf_upload/curl_cookies_from_jamf_upload.txt', '--cookie', '/tmp/jamf_upload/curl_cookies_from_jamf_upload.txt', '--header', 'Accept: application/xml', '--header', 'DESTINATION: 0', '--header', 'OBJECT_ID: -1', '--header', 'FILE_TYPE: 0', '--header', 'FILE_NAME: Microsoft Visual Studio Code-1.68.1.pkg', '--max-time', '3600']' returned non-zero exit status 28.
  File "/Library/AutoPkg/autopkglib/__init__.py", line 840, in process
    self.env = processor.process()
Command '['/usr/bin/curl', '--dump-header', '/tmp/jamf_upload/curl_headers_from_jamf_upload.txt', 'https://[Jamf URL].jamfcloud.com/dbfileupload', '--request', 'POST', '--silent', '--show-error', '--header', 'authorization: Basic [Basic Auth token]', '--output', '/tmp/jamf_upload/curl_output_from_jamf_upload.txt', '--output', '/tmp/jamf_upload/curl_output_from_jamf_upload.txt', '--upload-file', '/Users/[Username]/Library/AutoPkg/Cache/local.jamf.10xbanking-VisualStudioCode-pkg-upload/Microsoft Visual Studio Code-1.68.1.pkg', '--cookie-jar', '/tmp/jamf_upload/curl_cookies_from_jamf_upload.txt', '--cookie', '/tmp/jamf_upload/curl_cookies_from_jamf_upload.txt', '--header', 'Accept: application/xml', '--header', 'DESTINATION: 0', '--header', 'OBJECT_ID: -1', '--header', 'FILE_TYPE: 0', '--header', 'FILE_NAME: Microsoft Visual Studio Code-1.68.1.pkg', '--max-time', '3600']' returned non-zero exit status 28.
Failed.
Receipt written to /Users/[Username]/Library/AutoPkg/Cache/local.jamf.VisualStudioCode-pkg-upload/receipts/VisualStudioCode-pkg-upload.jamf-receipt-20220623-103335.plist

The following recipes failed:
    /Users/[Username]/Git/it-endpoint-mac-autopkg-overrides/VisualStudioCode-pkg-upload.jamf.recipe
        Error in local.jamf.VisualStudioCode-pkg-upload: Processor: com.github.grahampugh.jamf-upload.processors/JamfPackageUploader: Error: Command '['/usr/bin/curl', '--dump-header', '/tmp/jamf_upload/curl_headers_from_jamf_upload.txt', 'https://[Jamf URL].jamfcloud.com/dbfileupload', '--request', 'POST', '--silent', '--show-error', '--header', 'authorization: Basic [Basic Auth token]', '--output', '/tmp/jamf_upload/curl_output_from_jamf_upload.txt', '--output', '/tmp/jamf_upload/curl_output_from_jamf_upload.txt', '--upload-file', '/Users/[Username]/Library/AutoPkg/Cache/local.jamf.10xbanking-VisualStudioCode-pkg-upload/Microsoft Visual Studio Code-1.68.1.pkg', '--cookie-jar', '/tmp/jamf_upload/curl_cookies_from_jamf_upload.txt', '--cookie', '/tmp/jamf_upload/curl_cookies_from_jamf_upload.txt', '--header', 'Accept: application/xml', '--header', 'DESTINATION: 0', '--header', 'OBJECT_ID: -1', '--header', 'FILE_TYPE: 0', '--header', 'FILE_NAME: Microsoft Visual Studio Code-1.68.1.pkg', '--max-time', '3600']' returned non-zero exit status 28.

The following new items were downloaded:
    Download Path
    -------------
    /Users/[Username]/Library/AutoPkg/Cache/local.jamf.VisualStudioCode-pkg-upload/downloads/Microsoft Visual Studio Code.zip

The following packages were built:
    Identifier            Version  Pkg Path
    ----------            -------  --------
    com.microsoft.VSCode  1.68.1   /Users/[Username]/Library/AutoPkg/Cache/local.jamf.VisualStudioCode-pkg-upload/Microsoft Visual Studio Code-1.68.1.pkg

Let me know what else I can do to help :)

Have you tried that more than once? Did you give it a good couple of minutes after deleting the existing package before trying the upload again (to allow background sync to complete)?

Deleted the package and gave it 15 minutes before running the recipe and got the same issue. Looks like it starts the upload and then just waits the full curl timeout before erroring.
I've reverted the JamfPackageUploader process to the pre-changed version and it continues to work as expected

Hmm that's very odd, I didn't think I had changed that code at all. Gonna have to diff it.

I'm really struggling to reproduce your curl timeout. In my test it uploads successfully:
JamfPackageUploader: Checking for existing package 'DisplayLink_Manager-1.7.0.pkg' on https://jssimporter.jamfcloud.com
JamfPackageUploader: Checking for existing authentication token
JamfPackageUploader: Checking https://jssimporter.jamfcloud.com against https://jssimporter.jamfcloud.com
JamfPackageUploader: URL for token matches current request
JamfPackageUploader: Existing token is valid
JamfPackageUploader: Jamf Pro Version: 10.40.0-b.1.t1655300562
JamfPackageUploader: Token auth will be used,
JamfPackageUploader: Package 'DisplayLink_Manager-1.7.0.pkg' not found on server
JamfPackageUploader: HTTP response: 200
JamfPackageUploader: Package uploaded successfully, ID=296
JamfPackageUploader: Package 'DisplayLink_Manager-1.7.0.pkg' upload successful

Just going to ask - are you sure you copied the processor over correctly? Or is there something else weird on your client preventing network or preventing files getting written anywhere e.g. /tmp?

:|
So I retested today and obviously it worked fine 🤦
No idea what happened yesterday but it could well be that I carried out a bad copy/paste.
Thank you for all the checks and work on this one!

That's great. I'll copy it into the main repo in a bit.

Done.


Dark title bar

It would be nice if Chatterino 2 could support a dark Title bar aswell.
In Windows you can do this like this: https://www.howtogeek.com/258162/how-to-add-color-to-inactive-title-bars-in-windows-10/ but this currently doesn't work with chatterino.
The alternative would be somehow using the skin from chatterino and overwriting title color similar to Spotify Windows App maybe.

I would like a also customization not based on currently used windows theme


Array to string conversion in bread post.

Laravel Version: 5.5.14
Voyager Version: ^1.0
PHP Version:7.1.8
Database Driver & Version: 10.2.9-MariaDB

Description:
When the store sends an error. Array to string conversion. In the bread of post.
Steps To Reproduce:
When a column is added to the bread "post" type "tags">"longtext" and is assigned a "select multiple" with a json field example:
{
    "default": "MA",
    "options": {
        "MA": "Marketing",
        "DE": "Desarrollo",
        "CU": "Cuentas"
    }
}

And then you edit any post.

Got the same error.

Please provide the full stacktrace.

I had the same problem with manually created MySQL tables. When I created tables via Voyager admin panel, everything works fine.

I think this connected with that cause https://github.com/the-control-group/voyager/issues/2207


TreeView: Drag and drop of an unselected item should not drag and drop selected items

Describe the bug
Given a TreeView in multi-select mode, a drag and drop operation for an unselected item should not drag and drop the selected items. Here are two examples showcasing the current behavior:
Example 1

Example 2

Both examples look like bugs to me. I think a much more natural drag & drop operation logic would be the following:

Starting a drag on a selected item: This will drag & drop all selected items as a group (current behavior)
Starting a drag on an unselected item: This will drag & drop the unselected item only (no matter if there are selected items). This is already the case when no item is iselected when a drag operation is started:

Expected behavior
As written above, I would like to see dragging and dropping an unselected item to only move that one item, and not drop selected items as well. Dragging a selected item, however, should drag all selected items as a group (as is the case today).
Version Info

NuGet package version:
WinUI 2.4 and WinUI 2.5 previews

Windows 10 version
Saw the problem?

Insider Build (xxxxx)

May 2020 Update (19041)
Yes

November 2019 Update (18363)

May 2019 Update (18362)

October 2018 Update (17763)

April 2018 Update (17134)

Fall Creators Update (16299)

Creators Update (15063)

Device form factor
Saw the problem?

Desktop
Yes

Xbox

Surface Hub

IoT

@predavid-zz FYI. @Felix-Dev I agree with your Expected Behavior

Open question: How should this work with non leaf nodes for all 3 selection states, unslected, partially selected (some, not all children selected), and selected.

@StephenLPeters Below are my initial thoughts.
If a non-leaf node is started to be dragged while being

Selected: Current behavior is that all selected items will take part in the drag and drop operation. No matter if they are child nodes of the dragged non-leaf node or not.
I think we should keep that behavior as it is the same behavior we are seeing today when dragging a selected leaf node: Here, all selected items will take part again in the drag & drop operation. Keeping that behavior for selected non-leaf nodes will provide a consistent behavior when dragging a selected node.

Partially Selected: Current behavior is that all selected items will be dragged again. The partially selected drag item will remain in place with its remaining child items which were not selected.
I think we should change that behavior to drag and drop only the selected items which are children of the partially selected item. Selected items which are not children of the partially selected item should not take part in the drag & drop operation.

Unselected: Current behavior is that dragging and dropping an unselected node with children moves that node including all of its children. After a drop, the unselected non-leaf node will be in a new place with all of its children still intact.
I think we should keep the current behavior.

(I did not make any gifs of the current behaviors described above but you can easily check them in the XAML Control Gallery's TreeView page.)

The Selected and Unselected cases make sense to me.  I'm not sure about the Partially Selected case and would like to get some other people in the discussion.  @YuliKl @predavid-zz @ranjeshj

@StephenLPeters Would you currently be inclined to keep the existing behavior when dragging a partially selected node or do you have another possible suggestion?

I think there are a number of behaviors I could be convinced make sense; Treat it the same as the selected case, treat it the same as the unselected case, your suggestion, blocking drag and drop in this scenario.  I'm not confident I understand the space well enough to accurately choose the best solution.

@StephenLPeters I am starting to like the suggestion the actually blocking drag and drop for partially selected nodes. One reason being that if you drag such a node, the drag visual indicates you are actually dragging and dropping this node and thus on a successful drop placed it in a new position inside the TreeView. While this is indeed the case foth selected and unselected dragged nodes today, this is not the case for partially selected nodes! A partially selected node today *cannot be dropped into a different position, no matter what its drag visual might suggest.
As such, blocking drag and drop for partially selected nodes sounds quite compelling to me. Especially as no drag and drop functionality existing today would actually be lost here.

I am not a fan of the partial selection blocks drag and drop idea. In selectionmode multiple, you would end up with a scenario where you aren't able to drag and drop a lot of child node.
In addition to that, I also find the current completely selected case a bit strange. What if I just want to drag and drop the children of a node? Do I really have to drag and drop one by one then? I think dragging and dropping should depend on where the drag and drop started, if I started a child node, only drag the child nodes, if I drag a parent node, move the whole subtree.

I am not a fan of the partial selection blocks drag and drop idea. In selectionmode multiple, you would end up with a scenario where you aren't able to drag and drop a lot of child node.
In addition to that, I also find the current completely selected case a bit strange. What if I just want to drag and drop the children of a node? Do I really have to drag and drop one by one then? I think dragging and dropping should depend on where the drag and drop started, if I started a child node, only drag the child nodes, if I drag a parent node, move the whole subtree.

Hmm, I think another reasonable solutions is that dragging a non-leaf node always drags it and all its children, regardless of selection state. Is this behavior that we could see apps wanting to customize? What if selection is managed by the app (not the user) but they still want drag and drop, then it wouldn't be possible for the user to rearrange the selected items.

What if I just want to drag and drop the children of a node? Do I really have to drag and drop one by one then?

WIth the current behavior, if you want to drag the children of a node, you just need to select them all and then start a drag operation. So that's definitely possible today.
I kinda like the current behavior when dragging a selected node. This allows me to do something like this:

For partially selected this works (though your proposed solution to block drag and drop for partial selection would break that).
I was asking about a node where I want ALL of the children, but not the parent node. Currently, that would drag and drop both the children, and the parent of those items.

(though your proposed solution to block drag and drop for partial selection would break that)

I did not propose that as a solution :) Also I'm not sure why it would break that, because one of the ideas thrown into the ring here by @StephenLPeters originally was to block only dragging of the partially selected node. I can still drag and drop its children like shown in the gif above.
That said, dragging all of the children of a node currently isn't possible without dragging the parent node as well, as you pointed out. This should be supported though.
If we look at the File Explorer as an example, we see dragging and dropping works like this there:

So, we could decide to copy file explorer's behavior here. I think this would mean if I start dragging a selected node, all other selected nodes on the same level will be included in a drag operation. That way, I can easily drag and drop all the children of a node - and only them. If I ant to drag all the children and their parent node as well, and only them, I have to make sure no sibling of the parent node is selected as well (sibling here is a different node which is on the same level of our node). This could perhaps be a bit too complicated as a default behavior, not sure here.
This would also mean the behavior shown in my gif above would no longer be possible but that is something I could live with.
Since dragging an unselected non leaf node includes all of its children given the current behavior, I guess we could use the same behavior when starting to drag a partially selected non-leaf node: All of its children will be included as well, so only the parent and its children will be included in the drag operation, no matter if other nodes are selected somewhere..

Oh sorry, must have misunderstood you regarding the blocking of drag and drop.
Regarding the fully selection mode, there is the following issue I see: What happens when I drag the children of Home remodel in this screenshot?

Should it drag and drop only "Paint Color Scheme" and "Flooring woodgrain type"? Or should it drag all selected nodes? What if all items of "home remodel" are selected?
The issue here might be that a good drag and drop solution would require us here that you are able to select all children of a node without selecting said node. Current behavior is that a nodes selection state is defined by its children if it has children. Maybe this is not the most intuitive behavior though, for example I can select all files in a folder, yet that doesn't mean I also meant to select that folder.

What happens when I drag the children of Home remodel in this screenshot? Should it drag and drop only "Paint Color Scheme" and "Flooring woodgrain type"? Or should it drag all selected nodes?

If I strictly apply the specific File Explorer behavior shown in the gif above, I think only "Paint Color Scheme" and "Flooring woodgrain type" would be dragged. So the other selected nodes like "Overall Project Plan" wouldn't be included.

What if all items of "home remodel" are selected?

In that case, dragging one of its children will include all four of its children (and only them) in the drag and drop operation.
Now, if I were to drag one of the selected children of "Work Documents" only its three selected children would be included, the selected children of "Home Remodel" would not be included (as they don't share the same parent node). In other words, I can no longer just select nodes here and there and them have them all included in a drag and drop operation as shown in the gif above.
I could live with that though as I find the ability to be able to drag all chidren of a node - and only them - as much more important. After all, that's not possible with today's behavior at all whereas with the matching File Explorer suggestion I could still drag both the children of "Home Remodel" and "Work Documents", albeit in two different drag and drop operations.

But wouldn't make that selection a bit unpredictable? In some cases, you drag all items, in other cases you only drag a few items and currently, you might drag more than you expected to.
Maybe we should drag all items that are selected, however selecting all children of a node shouldn't select that node?

But wouldn't make that selection a bit unpredictable? In some cases, you drag all items, in other cases you only drag a few items and currently, you might drag more than you expected to.

Yes, I am seeing that as well.
(Ok, watching Bayern - Barca now, so limited attention from me for now 😅)

@chingucoding @StephenLPeters
I have summarized the current most promising ideas being floated around in this issue below. My hope is this will help push this issue forward until it is ready to be implemented.
I think pretty much agreed on this is suggested updated drag & drop behavior of an unselected node. Here it is:

Selection state
leaf node
parent node

Unselected
Dragging an unselected leaf node will only drag this one node.
Dragging an unselected parent node will additionally include all its child nodes in the drag & drop operation.

It is getting a bit trickier when we are looking at the desired drag & drop experience when the node initiating the drag is (partially) selected:
One of the major underlying problems with today's in-built drag & drop experience of selected TreeView nodes is that it is currently impossible to drag all children of a node in a single drag & drop operation without also including their parent node. Here's why:

Currently, dragging a selected node will include all selected TreeView nodes in the drag & drop operation
Selecting all child nodes will automatically select their parent node as well

As a result, if I want to drag all children of a node in a single drag & drop operation, I first have to select them all - but selecting all child nodes will also automaticall select their parent node! As such, the parent node will be included in any drag & operation.
We have seen two possible approaches being outlined/suggested so far here which will enable the scenario of dragging only the children of a node in a single drag & drop operation, and not its parent node.
Suggestion 1
The first suggestion is to change the drag & drop experience when being initiated on a selected node. The idea here is that the drag & experience of a selected node will now be context based. More precisely, the nodes included in drag & drop operation will now be dependent on the position of the selected node (initiating the drag) in the TreeView. No longer will all selected nodes in the TreeView be included automatically, no matter which selected node is used to initiate the drag operation. See below for a table with the suggested updated behavior:

Selection state
leaf node
parent node

Partially selected
-
Dragging a partially selected parent node will mirror the drag & drop experience when dragging an unselected parent node: All its children (selected or not) will additionally be included the drag & drop operation. No other selected nodes will be included here.

Selected
Dragging a selected leaf node will include all selected nodes (including their children, if any) in the drag & drop operation which have the same immediate parent node as the dragged selected node. (This mimics the drag & drop experience in the Windows File Explorer today.)
Dragging a selected parent node will include all its child nodes (and their children, if any) and all selected nodes which have the same immediate parent node as the dragged selected node.

Consequently, no longer will all selected nodes be included in the drag & drop operation, no matter on which selected node the drag operation is initiated. See the following drag & drop operation possible today:

As you can see, in a single drag & drop operation, all selected nodes were moved. With the poposed changes above, this would no longer be possible in a single drag & drop operation but would require three drag & drop operations instead (one for each selected node).
The drag & drop experience outlined above features two major characteristics:

Dragging a parent node will always include all of its child nodes.
ONLY for selected nodes: Dragging a selected node will include all selected nodes which share the same immediate parent with the dragged node (such an experience can be found in the Windows Explorer today).

These characteristcs ensure this proposed drag & drop experience is consistent in itself and als familiar to Windows users.
Suggestion 2
A possible alternative has been suggested by @chingucoding. It does not feature a context based drag & drop experience for selected nodes like the first suggestion. Instead, it proposes a change to the in-built TreeView selection logic to address the major shortcoming of the current drag & drop experience when dragging a selected node:
Selecting all children of a node will no longer automatically select their parent node.

In other words, even though dragging a selected node will still actually drag all selected nodes in a TreeView, since all child nodes can be selected without automatically selecting their parent node, this solves the major shortcomming of today's TreeView drag & drop experience just as suggestion 1 does.
The underlying idea with this suggestion is that the current in-built selection logic of the TreeView might not always be desirable. Consider, for example, a TreeView control used in a file explorer context: Selecting all files (the child nodes) of a folder (their parent node) should probably not automatically select the folder as well. This would be the case with the current implementation though. Instead, if the folder (the parent node) should be selected, then it should have to be selected manually.
A few open questions here, @chingucoding:

What would be the selection state of a node whose children are all selected? It could be partially selected to indicate that all of its children are selected but the node itself is not. While this will keep the current system of three different selection states a node can have, it will expand today's definition of a partially selected node. That might not be a problem though because the concept of partially selected nodes only exists in internal WinUI APIs, and the public API surface only states whether a node is selected or unselected (a partially sected node is exposed as a unselected node to the public).
How would you design the drag & drop experience of partially selected nodes? The same as outlined in suggestion 1 or differently?

Summary
I have outlined the two most promising suggestions developed in this thread so far above. Both suggestions solve the current major design flaw in today's TreeView drag & drop experience - namely that it is not possible to drag & drop all the selected children of a node without also including that parent node in a drag & drop operation.
Suggestion 1 keeps the current in-built selection logic unchanged (so selecting all child nodes will also automatically select their parent node) while introducing a position-based drag & drop experience.
Suggestion 2 keeps the current drag & drop experience for selected nodes unchanged (dragging a selected node will always drag all selected node) but makes a change to the current selection logic of the TreeView (no longer would a parent node be automatically selected when all its child nodes are selected).
@chingucoding @StephenLPeters Apologizing for the lengthy post but I think it will help move this discussion forward as needed.

Thank you for the detailed post @Felix-Dev !

What would be the selection state of a node whose children are all selected? It could be partially selected to indicate that all of its children are selected but the node itself is not. While this will keep the current system of three different selection states a node can have, it will expand today's definition of a partially selected node. That might not be a problem though because the concept of partially selected nodes only exists in internal WinUI APIs, and the public API surface only states whether a node is selected or unselected (a partially sected node is exposed as a unselected node to the public).

I would say it is partially selected since its tree is not entirely unselected, however its entire subtree isn't selected either. So neither unselected nor selected are entirely true, leaving us with partially selected. Unfortunately though, this would be a (major) breaking change if I am not mistaken.

How would you design the drag & drop experience of partially selected nodes? The same as outlined in suggestion 1 or differently?

Same as an unselected node, drag and drop the node and it's childern. So for drag and drop it would be treated as unselected. Only when dragging a selected item, we drag all selected item and actually ignore the starting position of the selected item.

Same as an unselected node, drag and drop the node and it's childern. So for drag and drop it would be treated as unselected. Only when dragging a selected item, we drag all selected item and actually ignore the starting position of the selected item.

@chingucoding So the same as suggestion 1 then. This means the two suggestions will only differ in how a drag & drop operation looks like for selected nodes.
@StephenLPeters To comment on when a change in the in-built selection logic of the TreeView would be a feasible change - if it should be done.

I think that both solutions make sense to me, however I think I slightly prefer suggestion 2 as it seems simpler and easier for the user to understand how it works. And I agree with marcel, all children selected means the parent is partially selected. @chingucoding can we flesh out why this is a breaking change?  I see some niche cases that would potentially change behavior but not ones that would cause crashes.
@predavid-zz FYI

I don't think changing the in-built selection logic would lead to app crashes (and also no compilation crashes). What would be the case is that developers today relying on the selection logic that selecting all the children of a node also selects their parent would need to adapt to the changed behavior. Is that a major concern? I'm not sure here (missing data)

This wouldn't be a breaking change in the sense that existing code crashed or fails to compile. However some apps might have this assumption that if all child nodes are selected, that the parent is selected. In some cases, this might result in a lot of nodes being marked as selected despite only the leaves of the tree being selected.
If we now mark the parent as partially selected, we would break this assumption that some developers might have. But I think, in this case it is more important that the logic is more natural to the end user which suggestion 2 is in my opinion.

Cool, that is what I think as well, just wanted to make sure I wasn't overlooking something.

I am preferring suggestion 2 as well as the solution moving forward. Only child nodesshould be automatically selected if their parent node is selected. Other nodes (like a prent of child nodes) should require manual selection. As suggested by all three of us, a parent node with all its children selected, but which isn't selected itself, should be be shown as a partially selected node.

I'm going to ask some other members of the team to weigh in here as well, I don't feel like I can unilaterally decide this, but I'll report back soon.

@StephenLPeters Did you find some time to discuss this with other members of the team?


Add config option to specify message properties

@jseely Can you please add documentation how this config option is supposed to be used?

@toddysm updated. Let me know if it needs any more detail.

@toddysm @htgc any additional changes needed to have this merged?

I will pull this in at the end of this month's release


[fractal-server v2.7] Update POST for tasks

All endpoints that create tasks also expose "ownership" information, and we should adapt the forms.
POST single task
POST task collection / local wheel file
POST task collection / pypi package
POST task collection custom
Context:

Every TaskGroupV2 is always associated to a single user who create it (FK user_id) - this user has full read/write/delete access
Optionally, a TaskGroupV2 may be associated to a single user group (FK user_group_id) - users of this group have "read&use" access

All four endpoints will have two new query parameters:

user_group_id=123 -> to use a specific user group
private: bool = False  -> to not user any specific user group (set it to null)

Options:

No query parameter -> group "All" by default
Both user_group_id=something and private=True -> 422
private=True -> set user_group_id=null
user_group_id=123 (possibly also with private=False) -> this is used

UI

Radio button to separate "Private task" from "Shared task" (default -> shared)
Within "private", no additional info is needed
Within "shared", dropdown with user groups - with "All" preselected
A single group can be selected, not more
Order of user groups is the same as coming from the backend


Make isVisible optionally inheritable

KHR Visibility, a new glTF extension, sets visibility as inherited value from a node's parent(s). Meaning, a node is only visible if it is set to be visible and all of its parents are visible. This is not what we are doing in Babylon.
This PR adds a flag that should only be turned on when using the KHR visibility extension for each mesh of the loaded glTF. Enabling the flag will allow checking  mesh's visibility, based on its parents visibility as well.
This PR is a suggestion only, but it will be needed to support the latest extension. I understand there is a slight perf hit for the default behavior. I did my best to make sure we return early for the current behavior, without iteration the parents.
Test PG - #ABDDD6#118 (emulating enabling KHR interactivity).

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

Snapshot stored with reference name:
refs/pull/15275/merge
Test environment:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/15275/merge/index.html
To test a playground add it to the URL, for example:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/15275/merge/index.html#WGZLGJ#4600
Links to test babylon tools with this snapshot:
https://playground.babylonjs.com/?snapshot=refs/pull/15275/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/15275/merge
https://gui.babylonjs.com/?snapshot=refs/pull/15275/merge
https://nme.babylonjs.com/?snapshot=refs/pull/15275/merge
To test the snapshot in the playground with a playground ID add it after the snapshot query string:
https://playground.babylonjs.com/?snapshot=refs/pull/15275/merge#BCU1XR#0

WebGL2 visualization test reporter:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/15275/merge/testResults/webgl2playwright/index.html

Visualization tests for WebGPU (Experimental)
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/15275/merge/testResults/webgpuplaywright/index.html


Visualization error with segmentation fault

Hello,
Thank you so much for sharing youse awesome work.
But I have a problem. When I tried to operate demo.py, I got the error like below:

with the black window like this (sometimes my computer is totally stuck with this situation):

For better understanding, my computer's spec is like this:

I think it has something to do with my memory usage, but I don't know at all why the error occurred.

Could you try this again on a computer with large RAM? Possibly open3D segfault that I see quite often.

I don't have other RAMs, so I just reduced the size of point cloud data which is used in demo, so the sample size reduced about a half, and it works. Thank you for your rapid response !!
By the way, I have an another question, but that question is about the training part. So I closed this issue.

Okay glad that it worked. Why don't you send me an email if it is not an issue?


Handle correctly new product 'openSUSE-Leap-Micro'

This product has same incident structure as 'openSUSE-SLE' incidents
New nested generator is more universal --> in future changes affects
only loader/qem.py code
https://progress.opensuse.org/issues/112430

Codecov Report

Merging #44 (70cfeec) into master (197225a) will decrease coverage by 0.03%.
The diff coverage is 33.33%.

@@            Coverage Diff             @@
##           master      #44      +/-   ##
==========================================
- Coverage   27.12%   27.08%   -0.04%
==========================================
  Files          24       24
  Lines        1589     1591       +2
==========================================
  Hits          431      431
- Misses       1158     1160       +2

Impacted Files
Coverage Δ

openqabot/loader/qem.py
25.69% <0.00%> (-0.18%)
:arrow_down:

openqabot/types/incident.py
25.00% <0.00%> (-0.34%)
:arrow_down:

openqabot/loader/repohash.py
97.36% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 197225a...70cfeec. Read the comment docs.


Don't leave the package's root directory

This require() call attempts to fetch the lib/ folder from outside of this project's root directory.

Closing a year later.


How do I disable autocompletion for the parameters?

How can I make it so the autocomple only finishes the command and not input every single possible parameter? For example, autocompletion of WinWait should be only WinWait and not WinWait, WinTitle, WinText, Seconds [, ExcludeTitle, ExcludeText].

OMG
Is this problem not solved yet?
I use sublim4 now, very distressed.


Simple solution for global unique header IDs

Solves Issue #2832.
Trusts that MarkedGFMHeadingId will make the ids unique per page and header level and then massages them to have what should be a global uniq name via the hooks: postprocess  system.
# Header 1
# Header 1
# Header 1
## Header 2
### Header 3
#### Header 4

Header 1

Header 1

Header 2

Header 3

Header 4

Code tests are failing due to an issue with jest and dom that I do not have enough knowledge of to solve.

The useful thing for putting id attributes on headings is that an author can then link to those headings.
See this brew for an example: https://homebrewery.naturalcrit.com/share/SyXKaZmQM
Every maneuvre listed on the 1st page links to the relevant section in the text.
This only remains useful if the id values don't change.
Adding or removing a page early in the brew will cause all the headings in this solution to change. And not in an obviously broken way either (i.e. clicking the link to #p7-H3-header-3 will still link to a header with that id .. despite the intended destination now being located on p8 or p6).

Okay. I'll go to plan b.

@ericscheid This now keeps the original ID from users, but still applies a page number and counter: #p1-myID-1.
Is this any better for your use case, or do you have an alternate preferred behavior? I understand this may still be an issue when deleting or adding pages because the #pX will change, but at least myID will remain intact.

@ericscheid This now keeps the original ID from users, but still applies a page number and counter: #p1-myID-1.
Is this any better for your use case, or do you have an alternate preferred behavior? I understand this may still be an issue when deleting or adding pages because the #pX will change, but at least myID will remain intact.

This is no better. The id still changes, and only exact matches are used for linking.
The numbering will also change if headings are inserted or deleted or rearranged on a given page.

The numbering will also change if headings are inserted or deleted or rearranged on a given page.

Well, this is going to be true of any solution that creates unique identifiers for duplicated names. Rearranging the headings would produce unique ids in a different order, no?

The numbering will also change if headings are inserted or deleted or rearranged on a given page.

Well, this is going to be true of any solution that creates unique identifiers for duplicated names. Rearranging the headings in text would always correspond to a rearrangement of the unique IDs being generated, no?
Unless I'm missing something, what naming scheme would you propose that is immune to reordering of the text?
Or maybe if you don't mind making a short example document showing what rearrangements of headers you expect to be stable? So we are all working from the same requirements.

The issue isn't the instability of "unique identifiers for duplicated names", but rather that even for non-duplicated names the ids are unstable. For example .. this document
## foo
## bar
## baz

should have the same ids as
## baz
## bar
## foo

and even this
## foo
### bar
#### baz

I see
And ideally this should persist across pages, no?
### foo

\page

## bar
## baz

I see, and ideally this should persist across pages, no?

Absolutely.
Bottom line: the fix for one specific case shouldn't break a more general case.

@ericscheid This now keeps the original ID from users, but still applies a page number and counter: #p1-myID-1.
Is this any better for your use case, or do you have an alternate preferred behavior? I understand this may still be an issue when deleting or adding pages because the #pX will change, but at least myID will remain intact.

This is no better. The id still changes, and only exact matches are used for linking.
The numbering will also change if headings are inserted or deleted or rearranged on a given page.

There is no way to prevent this other than by the user forcing an id via { }.
Even if we could maintain a global list without massaging the IDs  ( similar to how we have in the PR for reflinks )  nothing guarantees the integrity of auto-generated header ID reflinks staying valid. It is an unreliable method of internal document linking and should be discouraged. Even in a single-page document.

There is no way to prevent this other than by the user forcing an id via { }.

Expanding the scope of a problem from specific to general cannot be excused by saying there's already a problem with an edge case.


Upgrade example app

This PR attempts to do several things:

move from JSPM/SystemJS to webpack, since that's what we use for batarangle itself.
update to Angular 2 beta
fix the example so it actually works
correct README steps for connecting batarangle to an actual app.

Thanks for a great contribution, great to see your name here @SethDavenport :)

@igor-ka : made the changes you requested and rebased.

@SethDavenport LGTM :shipit:
PS: I am surprised there is no need for zone.js import either in one of the files or webpack.config.js let me know if you have uncovered something in that regard.

@igor-ka : spent some time discussing the zone thing with @vanessayuenn .
From what I've read, it seems like angular-polyfills imports the necessary zone libs, along with es6-promise.  In fact if I include both I get a console warning about window.zone already being defined.
However we're still hunting down a batarangle issue where it only detects the <todo-list> component, not any of its children.  I suspect this is an angular2-beta/batarangle incompatibility but I'll leave this PR up until we get to the bottom of it.

For the record, the batarangle issue coincides with this entry in the console, and it happens when the Angular2 tab is opened for the first time:
Uncaught TypeError: nativeElement.setAttribute is not a functionAngular2Adapter._emitNativeElement @ angular2.ts:133Angular2Adapter._traverseTree @ angular2.ts:109(anonymous function) @ angular2.ts:122forEach @ es6-shim.js:1107Angular2Adapter._traverseTree @ angular2.ts:121(anonymous function) @ angular2.ts:57forEach @ es6-shim.js:1107Angular2Adapter.setup @ angular2.ts:56(anonymous function) @ entry.ts:19__webpack_require__ @ bootstrap 11a0ead175d757e7a340?343e:19(anonymous function) @ bootstrap 11a0ead175d757e7a340?343e:39__webpack_require__ @ bootstrap 11a0ead175d757e7a340?343e:19(anonymous function) @ bootstrap 11a0ead175d757e7a340?343e:39(anonymous function) @ bootstrap 11a0ead175d757e7a340?343e:39
batarangle-backend.ts:41 Content script received:  Object {type: "BATARANGLE_INSPECTED_APP", message: Object}

It appears to be failing to insert the batarangle-id attribute on anything other than the  <todo-list> node.

@SethDavenport I went with including zone.js explicitly on my own app, and things were mostly working, but will try angular2-polyfill. Thanks for clarification.

@vanessayuenn @igor-ka
I tried hooking up batarangle to the app from the Angular2 quickstart (https://angular.io/docs/ts/latest/quickstart.html) using ...
import { AppViewListener } from 'angular2/src/core/linker/view_listener';
import { DebugElementViewListener, inspectNativeElement } from 'angular2/platform/common_dom';
import { bind } from 'angular2/core';

bootstrap(AppComponent, [
  bind(AppViewListener).toClass(DebugElementViewListener)
])
.then(applicationReference => {
  const w: any = window;
  w.ng.probe = inspectNativeElement;
});

... and I got the same error as above.  So it looks like we have a batarangle<->Angular 2 incompatibility happening with the way we're hooking Batarangle up to the DOM.  Since this doesn't appear to be caused by this PR, I'm going to merge and file a separate issue.


Optimizing and minimizing the posting part of the code using the dict…

…ionary and lambda functions.
Also created an additional postByType idea for further development guided for the GUI

checked and ready for update


Enrich the raw tree-sitter grammar with an actual Python grammar

Currently, the grammar defines only a limited set of rules in an attempt to emulate Python.

Closing as this is not anymore the repository to track this.


Unusual behavior of train spawners

BkCommonLib version: 1.13.1-v2
TrainCarts version: 1.13.1-v1
Spigot version: 1.13.1-R0.1

Problem or bug:
The train spawners ignore the redstone signal. If I power the spawner by redstone (redstone dust), the spawner ignore the signal.
If I use repeater before the spawner and turn signal on (as you can see on picture) the spawner not work. If I turn signal off the spawner spawn minecarts and after interval spawn again. After turn signal off spawn total two trains. I don’t understand.
If I use level next to the spawner, it works normally as I would expect.
Expected behaviour:
I would expect that it work as if I use level.
Steps to reproduce:
Build the spawner what I have in picture and test it.
Picture:

I have been pondering what you mean by that passage for 6 days straight and I still don't fully understand your issue. Do you mind making a video to explain the issue?

If the spawner does nothing, then that's expected behaviour since the repeater and the redstone dust you place at that location does not power the sign at all, so it should not do anything. That's how Minecraft redstone mechanics should work.
If it does something else, then I have no idea what's happening.

In version 1.12.2 work perfectly.
But now not.

Well give me the picture on which it will work on redstone dust. If I try other connections, it just does not work on redstone dust.

.. pointing the dust directly into the sign? That's how I expect it's supposed to work

Yes I tried, but it does not work. :)

Give us an image of what you tried?

can i see the whole track that you're trying to spawn a train onto?

Yes you see. The track is 100 blocks long.

I can't see.

I do not know what you want to see on the track from iron rails.

I... just want to take a look?

OK
https://www.dropbox.com/s/xo971m0escinj55/track.png?dl=0

Ok, can successfully reproduce the bug. Please wait for a while to let berger fix it.

Try updating to the latest development build of BKCommonLib and TrainCarts and see if the issue persists.
https://ci.mg-dev.eu/job/BKCommonLib/462/artifact/target/BKCommonLib-1.13.2-v1-SNAPSHOT.jar
https://ci.mg-dev.eu/job/TrainCarts/407/artifact/target/TrainCarts-1.13.1-v2-SNAPSHOT.jar

It works.
Well done!

I have another problem. For some unknown reason, this fixed version does not work on older maps generated before 1.13. It behaves just as if the problem was not fixed. I tested it only with this plugin, but the result is the same. I really do not know what's a problem on the map.

I don't know what map you're talking about

do you mean the "world" generated before 1.13? because "map" may refer to the MapDisplay (the thing used by attachment menu)

You're right. There should be a "world" instead of a "map".

What server version are you running on when it doesn't work?

Yes after restart same world still doesn't work. I also tried to use the "optimizing world" (a new option available in 1.13) but it did not help.

Any progress? Or what additional information is needed?

Sorry, I have not yet had the time to look into this as other issues popped up at the same time. I don't need more information, just have to go and reproduce it, really. I'll bump it up in priority as this needs to be fixed for the next BKCommonLib release

By your steps it works. I done more tests with other worlds, but some work and some do not work. It's really interesting.
Could you add some debug outputs to the plugin to see what's going on when it does not work?

Isnt it more likely that it works in some locations of a world, and not in others? Maybe there is a very specific situation going on somewhere. Another option is that this is caused by world-specific configuration introduced by Spigot (or Paperspigot) in the server-root located spigot.yml or paper.yml.
Difficult to add debug though, because of how many redstone events occur it would spam the console too much. I resolve this with a custom server console handler that can filter it.

I have also tested this. I have used my 2 worlds for both 1.12.2 and 1.13.2 and works perfectly. but if your example on a server of 1.13.2 on the version 1.12.2 plays, this does not work anymore because there are things that are very changing

Also an option to send me some 1.12.2 world confidentially (maybe only those region files that matter) so I can test things out on my debug-infested server. I know what I have to look for, but it's core to BKCommonLib and would require many (I'm talking dozens) of different test builds to pinpoint where the logic is breaking.

you also have to download the new spigot because this can not be the plugin but also spigot itself does this first

That is quite possible, although the redstone-specific <> MaterialData conversion is done by BKCommonLib itself. Bukkit's legacy support is limited: https://hub.spigotmc.org/jira/browse/SPIGOT-4230

Thank you for the idea with the paper.yml
I never thought that one stupid line in paper.yml would cause it not work. This was a line fire-physics-event-for-redstone that was set to false. When I set it to true, it worked.
Thanks a lot.
I think that is solved  :)

I see, I'll add something to traincarts so it prints a warning when this is the case.