Zephyr base improvement

This PR introduces Zephyr Config package for CMake.
Today, working with Zephyr / CMake requires users to always set ZEPHYR_BASE.
This can be done in a couple of different ways, depending on OS and shell being used (manually sourcing zephyr-env.sh, sourced in bashrc, running zephyr-env.cmd, etc.)
Or when using west build then ZEPHYR_BASE is set automatically.
Unfortunately, using an environment variable in the build system has a couple of flaws.
A couple of examples, see further down for details:

west build is incompatible with command line ninja if user does not set ZEPHYR_BASE after running west
cmake -G'Eclipse CDT4 - Ninja' and Eclipse, requires re-launching of Eclipse if switching to another project that uses a different Zephyr base.
Having multiple Zephyr directories bears the risk of forgetting to set another (correct) Zephyr base.
Zephyr base is changed between CMake and ninja invocation
Probably more.

west build Example (zephyr-env.sh not sourced as that is not required for west):
$ cd <projects>/zephyr/samples/hello_world
$ west build -bnrf52840_pca10056
$ cd build
$ touch ../CMakeLists.txt
$ ninja
[0/1] Re-running CMake...
CMake Error at CMakeLists.txt:5 (include):
  include could not find load file:

    /cmake/app/boilerplate.cmake

CMake Error at CMakeLists.txt:8 (target_sources):
  Cannot specify sources for target "app" which is not built by this project.

Terminal 1)
$ cmake -G'Eclipse CDT4 - Ninja' -DBOARD=nrf52840_pca10056
Terminal 2 or Desktop shortcut (not sourcing `zephyr-env.sh), Eclipse error:
15:39:45 **** Incremental Build of project hello_world@build ****
/usr/bin/ninja all
[0/1] Re-running CMake...
CMake Error at CMakeLists.txt:5 (include):
  include could not find load file:

    /cmake/app/boilerplate.cmake

-- Eclipse version is set to 3.6 (Helios). Adjust CMAKE_ECLIPSE_VERSION if this is wrong.
-- Configuring incomplete, errors occurred!

Having multiple Zephyrs and sourcing wrong one
/projects/zephyr-alt/zephyr/samples/hello_world/build$ cmake -GNinja -DBOARD=nrf52840_pca10056 ..
-- Zephyr version: 2.1.99
-- Found PythonInterp: /usr/bin/python3.6 (found suitable version "3.6.8", minimum required is "3.6")
-- Selected BOARD nrf52840_pca10056
-- Found west: /home/tora/.local/bin/west (found suitable version "0.7.1", minimum required is "0.6.0")
-- Loading /projects/ncs/zephyr/boards/arm/nrf52840_pca10056/nrf52840_pca10056.dts as base
Devicetree configuration written to /projects/zephyr-alt/zephyr/samples/hello_world/build/zephyr/include/generated/devicetree.conf
Parsing /projects/ncs/zephyr/Kconfig
Loaded configuration '/projects/ncs/zephyr/boards/arm/nrf52840_pca10056/nrf52840_pca10056_defconfig'
Merged configuration '/projects/zephyr-alt/zephyr/samples/hello_world/prj.conf'
Configuration saved to '/projects/zephyr-alt/zephyr/samples/hello_world/build/zephyr/.config'

as seen, the Zephyr base was wrong, but everything worked.
This might give unexpected results later.

Zephyr base changed between CMake and Ninja invocation.
This could happen to someone running CMake and ninja in a projectA.
Change into projectB to do some work, and sets ZEPHYR_BASE to projectB.
Switch back to projectA, forgetting to set ZEPHYR_BASE and execute ninja
$ echo $ZEPHYR_BASE
/projects/ncs/zephyr
$ cmake -GNinja -DBOARD=nrf52840_pca10056 ..
-- Zephyr version: 2.1.99
...
$ cd <projectB>
$ source zephyr-env.sh
$ cd <projectA>/build
$ ninja -v
$ ninja
[0/1] Re-running CMake...
-- Zephyr version: 2.2.0-rc1
-- Selected BOARD nrf52840_pca10056
-- Found west: /home/tora/.local/bin/west (found suitable version "0.7.1", minimum required is "0.6.0")
-- Loading /projects/ncs/zephyr/boards/arm/nrf52840_pca10056/nrf52840_pca10056.dts as base
Devicetree header saved to '/projects/ncs/zephyr/samples/hello_world/build/zephyr/include/generated/devicetree_unfixed.h'
Parsing /projects/zephyr-alt/zephyr/Kconfig
/projects/zephyr-alt/zephyr/scripts/kconfig/kconfig.py: /projects/ncs/nrf/subsys/bootloader/Kconfig:25: '/projects/zephyr-alt/zephyr/../nrf/subsys/partition_manager/Kconfig.template.build_strategy' not found (in 'source "${ZEPHYR_BASE}/../nrf/subsys/partition_manager/Kconfig.template.build_strategy"'). Check that environment variables are set correctly (e.g. $srctree, which is set to '/projects/zephyr-alt/zephyr'). Also note that unset environment variables expand to the empty string.
CMake Error at /projects/zephyr-alt/zephyr/cmake/kconfig.cmake:216 (message):
  command failed with return code: 1
Call Stack (most recent call first):
  /projects/zephyr-alt/zephyr/cmake/app/boilerplate.cmake:464 (include)
  CMakeLists.txt:5 (include)

In this case, an error was printed to user, but in case the two Zephyr's are very close, it could happen that everything worked, and in that case harder to detect when things break.

Fixes in this PR:

Cache ZEPHYR_BASE, the ZEPHYR_BASE that was set for the first CMake invocation in current build folder should be sticky, that is if CMake is re-invoked due to changes in any file, then the same ZEPHYR_BASE should be used.
Allow west build and ninja to work better together.
Remove the requirement to set ZEPHYR_BASE for majority of users
Be backward compatible with current ZEPHYR_BASE and allow it to support advange use-cases

With this PR, Zephyr now has a CMake config package.
A one-time install must be done in a Zephyr. (multiple install are allowed but not required).
To do this install, run:
west zephyr-export

Once installed, users can now do:
$ echo $ZEPHYR_BASE

$ cmake -GNinja -DBOARD=nrf52840_pca10056 ..
Including boilerplate (in-zephyr-tree): /projects/ncs/zephyr/cmake/app/boilerplate.cmake
-- Zephyr version: 2.2.0-rc1

as seen, no ZEPHYR_BASE was set.
and also out-of-tree builds are supported this way:
$ echo $ZEPHYR_BASE

tora@ubuntu:/projects/oot/hello_world/build$ cmake -GNinja -DBOARD=nrf52840_pca10056 ..
Including boilerplate (out-of-worktree): /projects/ncs/zephyr/cmake/app/boilerplate.cmake
-- Zephyr version: 2.2.0-rc1

Using ZEPHYR_BASE and enforcing another Zephyr is also possible:
$ ZEPHYR_BASE=/projects/zephyr-alt/zephyr cmake -GNinja -DBOARD=nrf52840_pca10056 ..
Including boilerplate (zephyr base): /projects/zephyr-alt/zephyr/cmake/app/boilerplate.cmake
-- Zephyr version: 2.2.0-rc1
-- Found PythonInterp: /usr/bin/python3.6 (found suitable version "3.6.8", minimum required is "3.6")
-- Selected BOARD nrf52840_pca10056
-- Found west: /home/tora/.local/bin/west (found suitable version "0.7.1", minimum required is "0.6.0")
-- Loading /projects/zephyr-alt/zephyr/boards/arm/nrf52840_pca10056/nrf52840_pca10056.dts as base

Creating a project also becomes much cleaner, all a user has to do, is to use find_package.
To create an application, you'll write:
find_package(Zephyr 2.0.0 HINTS $ENV{ZEPHYR_BASE})

project(hello_world)
target_sources(app PRIVATE src/main.c)

This has some additional benefits, such as requesting only to compile against an exact match:
find_package(Zephyr 2.0.0 EXACT HINTS $ENV{ZEPHYR_BASE})

or simply accept any Zephyr:
find_package(Zephyr HINTS $ENV{ZEPHYR_BASE})

To install Zephyr as a package, do the following:
west zephyr-export

Or manually:
$ cd <project>/zephyr/share/zephyr-package/cmake/
$ cmake .
Zephyr and ZephyrUnitTest (/projects/github/ncs/zephyr/share/zephyr-package/cmake)
has been added to the user package registry in:
~/.cmake/packages/Zephyr
~/.cmake/packages/ZephyrUnitTest
-- Configuring done
-- Generating done
-- Build files have been written to: /<project>/zephyr/share/zephyr-package/cmake

Example of west / ninja fix:
$ west build -bnrf52840_pca10056
$ cd build
$ touch ../CMakeLists.txt
$ echo $ZEPHYR_BASE
$ ninja
[0/1] Re-running CMake...
Including boilerplate (in-zephyr-tree): /projects/ncs/zephyr/cmake/app/boilerplate.cmake
-- Zephyr version: 2.2.0-rc1
-- Selected BOARD nrf52840_pca10056
...

Things still ToDo:
[X] - Update documentation to described workflow (depends on feedback to this PR)
[X] - Create west extension command to make easy Zephyr config package install (instead of cmake .)

$ cmake .
-- Configuring done
-- Generating done
-- Build files have been written to: /<project>/zephyr/share/zephyr-package/cmake

I want the logs here to reflect the fact that CMake is writing to system files / registry. E.g. "Install files written to $HOME/.config/zephyr/". Or "Install files written to Windows registry".

Is supporting mixing ninja and west build required? In your example you use cmake directly to change the cmake generator. Can't we just have a way to change the cmake-generator through west and use west command line always?

gen_cfb_font_header.py needs to be fixed.

Having multiple Zephyr directories bears the risk of forgetting to set another (correct) Zephyr base.
Zephyr base is changed between CMake and ninja invocation

It should be mentioned that these are only issues when using CMake/ninja directly, as opposed to invoking CMake/ninja indirectly through west.

Can't we just have a way to change the cmake-generator through west and use west command line always?

I haven't read the code for this PR, but this is already supported; see e.g. examples here:
https://docs.zephyrproject.org/latest/guides/west/build-flash-debug.html#one-time-cmake-arguments
This can be made permanent with the build.cmake-args config option (documented on the same page).

Is supporting mixing ninja and west build required?

I don't see why it should not be possible ?

In your example you use cmake directly to change the cmake generator.

Not exactly sure what you mean, but if you use the an IDE generator (such as Eclipse) through CMake directly or through west (is that possible ?), then the IDE will always use the build command, ninja / make, so it should be possible to also use the build command directly.

Can't we just have a way to change the cmake-generator through west and use west command line always?

How will you ensure that an IDE always uses west ?

I want the logs here to reflect the fact that CMake is writing to system files / registry. E.g. "Install files written to $HOME/.config/zephyr/". Or "Install files written to Windows registry".

Good point, will add.

Having multiple Zephyr directories bears the risk of forgetting to set another (correct) Zephyr base.
Zephyr base is changed between CMake and ninja invocation

It should be mentioned that these are only issues when using CMake/ninja directly, as opposed to invoking CMake/ninja indirectly through west.

This may also happen with west, as west will honor ZEPHYR_BASE if set.
Or when building an app out-of-tree (or in-work-tree, if having multiple Zephyr's in same project tree).
Example:
$ source /projects/ncs/zephyr/zephyr-env.sh
$ cd /projects/oot/hello_world
$ west build -bnrf52840_pca10056
-- west build: build configuration:
       source directory: /projects/oot/hello_world
       build directory: /projects/oot/hello_world/build (created)
       BOARD: nrf52840_pca10056 (origin: command line)
-- west build: generating a build system
...
-- Loading /projects/ncs/zephyr/boards/arm/nrf52840_pca10056/nrf52840_pca10056.dts as base
...
Memory region         Used Size  Region Size  %age Used
           FLASH:       14364 B         1 MB      1.37%
            SRAM:        6708 B       256 KB      2.56%
        IDT_LIST:          56 B         2 KB      2.73%
[130/130] Linking C executable zephyr/zephyr.elf

# Sourcing a different env, in similar way as described
# https://github.com/zephyrproject-rtos/zephyr/pull/23054#issue-379052937
# for the cmake / ninja build
$ source /projects/zephyr-alt/zephyr/zephyr-env.sh
$ touch CMakeLists.txt
$ west build -bnrf52840_pca10056
-- west build: build configuration:
       source directory: /projects/oot/hello_world
       build directory: /projects/oot/hello_world/build
       BOARD: nrf52840_pca10056 (origin: CMakeCache.txt)
-- west build: building application
[0/1] Re-running CMake...
-- Zep-- Selected BOARD nrf52840_pca10056
-- Found west: /home/tora/.local/bin/west (found suitable version "0.7.1", minimum required is "0.6.0")
-- Loading /projects/ncs/zephyr/boards/arm/nrf52840_pca10056/nrf52840_pca10056.dts as base <-- still using old zephyr base from first CMake invocation
Devicetree header saved to '/projects/oot/hello_world/build/zephyr/include/generated/devicetree_unfixed.h'
Parsing /projects/zephyr-alt/zephyr/Kconfig <-- whoops, now using new zephyr base for Kconfig
...

Memory region         Used Size  Region Size  %age Used
           FLASH:       11500 B         1 MB      1.10%
            SRAM:        5180 B       256 KB      1.98%
        IDT_LIST:          56 B         2 KB      2.73%
[124/124] Linking C executable zephyr/zephyr.elf

and as can also be seen, the application sizes has changed.
The ZEPHYR_BASE that was specified in first CMake invocation should be sticky, and not until user deletes build folder shoud it be changed.
Anything else will result in unpredictable end results.

Can't we just have a way to change the cmake-generator through west and use west command line always?

I haven't read the code for this PR, but this is already supported; see e.g. examples here:
https://docs.zephyrproject.org/latest/guides/west/build-flash-debug.html#one-time-cmake-arguments
This can be made permanent with the build.cmake-args config option (documented on the same page).

except that Zephyr base is not sticky, which may be an issue for out-of-tree builds, and IDEs that invokes make/ninja directly.
Especially Eclipse, but also SES, not sure about other IDEs such as CodeBlocks and CodeLite.

gen_cfb_font_header.py needs to be fixed.

fixed.

$ cmake .
-- Configuring done
-- Generating done
-- Build files have been written to: /<project>/zephyr/share/zephyr-package/cmake

I want the logs here to reflect the fact that CMake is writing to system files / registry. E.g. "Install files written to $HOME/.config/zephyr/". Or "Install files written to Windows registry".

@SebastianBoe : done.
In windows, the following will be printed:
Zephyr and ZephyrUnitTest (/projects/github/ncs/zephyr/share/zephyr-package/cmake)
has been added to the user package registry in:
HKEY_CURRENT_USER\Software\Kitware\CMake\Packages\Zephyr
HKEY_CURRENT_USER\Software\Kitware\CMake\Packages\ZephyrUnitTest

On Linux / MacOS:
Zephyr and ZephyrUnitTest (/projects/github/ncs/zephyr/share/zephyr-package/cmake)
has been added to the user package registry in:
~/.cmake/packages/Zephyr
~/.cmake/packages/ZephyrUnitTest

I think this is really cool.
It breaks west build --pristine=always and a couple of other extensions. I pushed some suggested changes in. Please squash if they look OK to you.

Great, thanks for the fix.
Will check them out and play with them.

@mbolivar and @carlescufi I changed BOARD_ROOT_SPACE_SEPARATED to simply BOARD_ROOT, in boards.cmake and also adds ZEPHYR_BASE as board root.
This is done in order to be consistent with existing Zephyr docs when building, and boilerplate code.
See those two comments:
https://github.com/zephyrproject-rtos/zephyr/pull/23054/commits/88b120445945decb572ecf521a0ae14f4c95f0db#r390341804
https://github.com/zephyrproject-rtos/zephyr/pull/23054/commits/88b120445945decb572ecf521a0ae14f4c95f0db#r390343152

Looks like west completion needs an update too, right?

Done.

I pushed some suggested changes in. Please squash if they look OK to you.

Done.

Docs update.
@nashif Have you seen my explanation regarding top-level folder share ?
Also take a look at the newly added doc.

@mnkp Thanks for the review.

It looks like there are some references to source zephyr-env.sh in the documentation which need to be removed:
* doc/contribute/index.rst

* doc/getting_started/installation_linux.rst

* doc/guides/beyond-GSG.rst

Good points.
Will updated those pages accordingly.

@mnkp Thanks for the review.

It looks like there are some references to source zephyr-env.sh in the documentation which need to be removed:
* doc/contribute/index.rst

* doc/getting_started/installation_linux.rst

* doc/guides/beyond-GSG.rst

Good points.
Will updated those pages accordingly.

Please remove the Zephyr version in the find_package() directive in all samples, there is no way we will ever be able to keep that updated

In share/zephyr-package/cmake there are quite a few files. Could you document in an "advanced section" which ones are used during export and which ones are actually ran when find_package() is run? and if there are any others that are actually used during the actual build?

Awaiting changes

@sebo what changes ?
I forced-pushed the branch with the version 2.2.0 removed before the message.

cmake -E env

cmake -E env

Ok, that one.
It's still on the ToDo.
Thought you might be referring to something else.

In share/zephyr-package/cmake there are quite a few files. Could you document in an "advanced section" which ones are used during export and which ones are actually ran when find_package() is run? and if there are any others that are actually used during the actual build?

Done.
And also added pristine target which clean away generated files.
Pristine is automatically called as part of west zephyr-export

@mnkp Additional doc files updated.

It looks like there are some references to source zephyr-env.sh in the documentation which need to be removed:
* doc/contribute/index.rst

Currently keeping source zephyr-env.sh as the sanitycheck script still relies on $ZEPHYR_BASE.
I will cleanup this in a separate PR, to keep this PR focused on the CMake and west required changes and tools called via CMake. (Python scripts)

* doc/getting_started/installation_linux.rst

* doc/guides/beyond-GSG.rst

Both updated, and removed reference to source zephyr-env.sh

@carlescufi @SebastianBoe all review comments addressed.

We also need to run full CI on this before it can be merged.

Does that mean applications using the old style will fail to build now?

Everything should be backwards compatible with old style.
So this is unexpected.
Will take a look.

We also need to run full CI on this before it can be merged.

If we can run this PR without a time limitation, that would be great.
Local testing with sanitycheck passed.

Does that mean applications using the old style will fail to build now?

Everything should be backwards compatible with old style.
So this is unexpected.

@nashif I don't see tests/lib/devicetree in my repo:
zephyr> ls tests/lib/
c_lib/  fdtable/  gui/  json/  mem_alloc/  onoff/  ringbuffer/  sprintf/

Is that some local work/tests you're having?
Second note. If including the boilerplate code using old style, then you also need to source zephyr-env.sh.
Old style, is old style all the way.

tests/lib/devicetree` in my repo:

you need to rebase
mac:zephyr((HEAD detached from upstream/pull/23054)): source zephyr-env.sh
mac:zephyr((HEAD detached from upstream/pull/23054)): echo $ZEPHYR_BASE
/Users/anashif/zephyr/zephyrproject/zephyr
mac:zephyr((HEAD detached from upstream/pull/23054)): rm -rf build/
mac:zephyr((HEAD detached from upstream/pull/23054)): west  build -b nrf52840_pca10056  tests/lib/devicetree/
-- west build: generating a build system
-- Application: /Users/anashif/zephyr/zephyrproject/zephyr/tests/lib/devicetree
CMake Error at /Users/anashif/zephyr/zephyrproject/zephyr/cmake/app/boilerplate.cmake:102 (include):
  include could not find load file:

    /cmake/extensions.cmake
Call Stack (most recent call first):
  CMakeLists.txt:6 (include)

CMake Error at /Users/anashif/zephyr/zephyrproject/zephyr/cmake/app/boilerplate.cmake:103 (include):
  include could not find load file:

    /cmake/version.cmake
Call Stack (most recent call first):
  CMakeLists.txt:6 (include)

CMake Error at /Users/anashif/zephyr/zephyrproject/zephyr/cmake/app/boilerplate.cmake:109 (include):
  include could not find load file:

    /cmake/python.cmake
Call Stack (most recent call first):
  CMakeLists.txt:6 (include)

CMake Error at /Users/anashif/zephyr/zephyrproject/zephyr/cmake/app/boilerplate.cmake:110 (include):
  include could not find load file:

    /cmake/git.cmake
Call Stack (most recent call first):
  CMakeLists.txt:6 (include)

CMake Error at /Users/anashif/zephyr/zephyrproject/zephyr/cmake/app/boilerplate.cmake:111 (include):
  include could not find load file:

    /cmake/ccache.cmake
Call Stack (most recent call first):
  CMakeLists.txt:6 (include)

CMake Error at /Users/anashif/zephyr/zephyrproject/zephyr/cmake/app/boilerplate.cmake:126 (zephyr_linker_sources):
  Unknown CMake command "zephyr_linker_sources".
Call Stack (most recent call first):
  CMakeLists.txt:6 (include)

-- Configuring incomplete, errors occurred!
FATAL ERROR: command exited with status 1: /usr/local/bin/cmake -B/Users/anashif/zephyr/zephyrproject/zephyr/build -S/Users/anashif/zephyr/zephyrproject/zephyr/tests/lib/devicetree -GNinja -DBOARD=nrf52840_pca10056

on master, it builds just fine:
mac:zephyr(master): west  build -b nrf52840_pca10056  tests/lib/devicetree/
-- west build: generating a build system
-- Application: /Users/anashif/zephyr/zephyrproject/zephyr/tests/lib/devicetree
-- Zephyr version: 2.2.99
-- Found PythonInterp: /usr/local/bin/python3 (found suitable version "3.7.6", minimum required is "3.6")
-- Board: nrf52840_pca10056
-- Found west: /Users/anashif/Library/Python/3.7/bin/west (found suitable version "0.7.1", minimum required is "0.7.1")
-- Found toolchain: gnuarmemb (/opt/gcc-arm-none-eabi-8-2019-q3-update)
-- Found BOARD.dts: /Users/anashif/zephyr/zephyrproject/zephyr/boards/arm/nrf52840_pca10056/nrf52840_pca10056.dts
-- Found devicetree overlay: /Users/anashif/zephyr/zephyrproject/zephyr/tests/lib/devicetree/app.overlay

there are still samples in the tree that use the old boilerplate include, they need to be changed.

Fixed now, those were added after the PR was created, therefore they weren't updated.

you need to rebase

Doh. I see it was merged yesterday after last rebase.
Moving target.
Fixed now.

on master, it builds just fine:
mac:zephyr(master): west  build -b nrf52840_pca10056  tests/lib/devicetree/

And so now it will if you have executed west zephyr-export or run source zephyr-env.sh.
devicetree> west build -bnrf52840_pca10056
-- west build: generating a build system
Including boilerplate (Zephyr base): /projects/github/ncs/zephyr/cmake/app/boilerplate.cmake
-- Application: /projects/github/ncs/zephyr/tests/lib/devicetree
-- Zephyr version: 2.2.99 (/projects/github/ncs/zephyr)
...
-- Build files have been written to: /projects/github/ncs/zephyr/tests/lib/devicetree/build
-- west build: building application

So if using west, then the Zephyr package must be exported (one time command, similar to west init), or source zephyr-env.sh.
@mbolivar  you want to add something to this ?

So if using west, then the Zephyr package must be exported (one time command, similar to west init), or source zephyr-env.sh.

What do you mean, he did source zephyr-env.sh.

@nashif also fixed an issue where using old style include($ENV{ZEPHYR_BASE}/cmake/app/boilerplate.cmake was not setting the cached ZEPHYR_BASE.
This is also fixed now.
Thanks for spotting.

So if using west, then the Zephyr package must be exported (one time command, similar to west init), or source zephyr-env.sh.

What do you mean, he did source zephyr-env.sh.

Yes, I noticed that afterwards, and thus found the issue mentioned here: https://github.com/zephyrproject-rtos/zephyr/pull/23054#issuecomment-603853338

Okay, so it's fixed, great.
I thought you were saying that he forgot to source zephyr.env.sh.

@nashif will you run full CI on this PR?

@nashif will you run full CI on this PR?

trying, but I am getting lots of weird build issues...
INFO    - /home/anashif/zephyrproject/zephyr/sanity-out/em_starterkit/tests/ztest/mock/testing.ztest.mock/build.log
^CERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for 96b_carbon_nrf51
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for dragino_lsn50
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for efm32hg_slstk3400a
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for 96b_wistrio
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nucleo_g431rb
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nucleo_f756zg
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for cc1352r1_launchxl
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for stm32l1_disco
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nucleo_f103rb
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nrf51_vbluno51
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for efr32_radio_brd4104a
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nucleo_l476rg
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for v2m_musca_nonsecure
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for adafruit_feather_stm32f405
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for atsamd21_xpro
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for adafruit_feather_nrf52840
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for stm32f429i_disc1
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for mikroe_mini_m4_for_stm32
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for 96b_avenger96
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for efr32_radio_brd4250b
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nrf9160_pca10090
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nrf9160_pca10090ns
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nucleo_g474re
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for sam_e70_xplained
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nucleo_f302r8
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for reel_board_v2
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nucleo_h743zi
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nrf52_blenano2
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nrf52840_blip
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for reel_board
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for sensortile_box
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for mimxrt1064_evk
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for 96b_meerkat96
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for stm32f769i_disco
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for frdm_kl25z
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nrf52840_pca10056
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nucleo_f411re
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nucleo_f303re
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for stm32f412g_disco
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for efr32_slwstk6061a
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for stm32l496g_disco
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for stm3210c_eval
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nucleo_wb55rg
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for frdm_k22f
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for decawave_dwm1001_dev
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for bl654_dvk
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for mimxrt1060_evk_hyperflash
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for mec1501modular_assy6885
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for mimxrt1060_evk
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for twr_ke18f
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for stm32l476g_disco
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for v2m_musca
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for cy8ckit_062_wifi_bt_m4
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nrf51_blenano
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for stm32vl_disco
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nucleo_l053r8
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for arduino_due
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for warp7_m4
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for bcm958401m2
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nucleo_l073rz
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nrf52811_pca10056
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for black_f407ve
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for stm32f072_eval
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for lpcxpresso55s69_cpu0
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for mec2016evb_assy6797
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nucleo_l152re
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for olimex_stm32_p405
ERROR   - Cmake build failure: /home/anashif/zephyrproject/zephyr/tests/ztest/custom_output for nucleo_l4r5zi

Not sure what is going on...

@mbolivar you want to add something to this ?

@mbolivar has nothing to do here, but @mbolivar-nordic might ;). Though it looks like this issue is resolved given the later discussion? I'm having trouble following it.

btw, to simulate how things are run in CI and where I see the issues, do:
./scripts/ci/run_ci.sh -b master -r origin  -l -R <commit>

with  being the first commit in the PR

btw, to simulate how things are run in CI and where I see the issues, do:
./scripts/ci/run_ci.sh -b master -r origin  -l -R <commit>

with being the first commit in the PR

Ok, let me try that.
Hopefully I can get same behavior.

@nashif will you run full CI on this PR?

trying, but I am getting lots of weird build issues...

@nashif  I have found one issue in tests/ztest/mock/CMakeLists.txt, where the find_package(ZephyrUnittest) was used instead of find_package(Zephyr).
find_package(ZephyrUnittest) shoud only be used for board == unit_testing.
I'm currently re-running /scripts/ci/run_ci.sh -b master -r origin  -l -R <commit>, and it's looking good so far.

@mbolivar you want to add something to this ?

@mbolivar has nothing to do here, but @mbolivar-nordic might ;). Though it looks like this issue is resolved given the later discussion? I'm having trouble following it.

I will listen both to your private opinion, as well as your professional.
I was more thinking if there is more to add, now that west no longer sets ZEPHYR_BASE automatically.

I was more thinking if there is more to add, now that west no longer sets ZEPHYR_BASE automatically.

Sorry, I don't follow. West is still setting it automatically for extension commands.

Docs look good, I find them generally very easy to read. I found some nits and had some questions/suggestions. Nothing is critical, so I won't hold up the merge.

Thanks for the feedback.
Appreciated.

General question: Would it be possible to drop some of the documentation in favor of links to Cmake's documentation if it exists? I'm asking so we can avoid a bit of maintenance on the docs.

I have tried not to document CMake package itself.
Just enough to help people understand and being able to continue from there.

I'm thinking of things like "ZEPHYR_PREFER" and "HINTS".

ZEPHYR_PREFER is not a CMake thing, but our own addition.
HINTS is a CMake argument to find_package but the knowledge that we use ZEPHYR_BASE in environment for being able to point to a specific Zephyr, that is special to Zephyr.

I was more thinking if there is more to add, now that west no longer sets ZEPHYR_BASE automatically.

Sorry, I don't follow. West is still setting it automatically for extension commands.

Only if set zephyr.base = <path> is set in .west/config, which west 0.7.1 will not do.
and I was also thinking of the wish to remove ZEPHYR_BASE knowledge from west itself.

General question: Would it be possible to drop some of the documentation in favor of links to Cmake's documentation if it exists? I'm asking so we can avoid a bit of maintenance on the docs.

I have tried not to document CMake package itself.
Just enough to help people understand and being able to continue from there.

Maybe add an external link?

ZEPHYR_PREFER is not a CMake thing, but our own addition.

Hehe, should have been obvious :P.

General question: Would it be possible to drop some of the documentation in favor of links to Cmake's documentation if it exists? I'm asking so we can avoid a bit of maintenance on the docs.

I have tried not to document CMake package itself.
Just enough to help people understand and being able to continue from there.

Maybe add an external link?

There are three already:
Those two in the introduction section:
https://cmake.org/cmake/help/latest/manual/cmake-packages.7.html
https://cmake.org/cmake/help/latest/manual/cmake-packages.7.html#user-package-registry
See: Zephyr CMake Package
And this one:
https://cmake.org/cmake/help/latest/command/find_package.html#version-selection
In the Zephyr CMake Package Version section

General question: Would it be possible to drop some of the documentation in favor of links to Cmake's documentation if it exists? I'm asking so we can avoid a bit of maintenance on the docs.

I have tried not to document CMake package itself.
Just enough to help people understand and being able to continue from there.

Maybe add an external link?

There are three already:
Those two in the introduction section:
https://cmake.org/cmake/help/latest/manual/cmake-packages.7.html
https://cmake.org/cmake/help/latest/manual/cmake-packages.7.html#user-package-registry
See: Zephyr CMake Package
And this one:
https://cmake.org/cmake/help/latest/command/find_package.html#version-selection
In the Zephyr CMake Package Version section

Sorry, I must have searched without expanding that diff. Looks good!

@nashif @tejlmand is this ready for merging now?

@nashif @tejlmand is this ready for merging now?

I'm still running full CI with this /scripts/ci/run_ci.sh -b master -r origin -l -R <commit>, but no errors so far.
Not sure if @nashif has completed a full CI run locally.
Any comments to doc can be addressed separately, as they a just minor improvements in phrasing.

@nashif @tejlmand is this ready for merging now?

i had CI run the ~50k builds but then something went bad at the end... It looked good in general. I think we should merge, look at first daily build after merge and fix any potential issues raised by daily build.

@nashif @tejlmand is this ready for merging now?

i had CI run the ~50k builds but then something went bad at the end with the run, so do not have the exact results... It looked good in general. I think we should merge, look at first daily build after merge and fix any potential issues raised by daily build.

I agree, let's merge and deal with the aftermath in subsequent PRs. I don't think we'll ever get closer to merging this.


fishPlugins.fish-you-should-use: init at 0-unstable-2022-02-13

Description of changes
Adds fish-you-should-use, a fish plugin that reminds users of set aliases when running equivalent commands.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/4340

@Anomalocaridid Thanks for all the fish (plugins) :wink:

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-already-reviewed/2617/1884

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-already-reviewed/2617/1978


Implementation of the converter for Tile operation

ConvertTile operator is implemented so that both parameters (tensor and multiplier) can be passed as tensors and weights. A unit test with 6 tests and all combinations of passed parameters is provided.
New macro #define CHECK_INPUT_SIZE (size, exp_size, node_def) unifies the testing of the number of parameters passed. It is used in 8 different places.
The helper function template void AdjustVectorByDims(...) makes writing tests much easier (especially tests with parameters that can be passed as tensors and weights). For example, today we can use some parameters like tensors with predefined dimensions, but with no real numbers. An attempt to pass the same parameters with the same values as weights leads to the crash. We see a similar crash when the vector providing the values assigned to the tensor is not empty, but its length does not match the number of values defined by the dimensions. AdjustVectorByDims fixes these two problems and is called by default in the situations described, but these calls can also be blocked when such failures are intentional.

@bixia1 : I'm done with your last request. I think this PR is ready to merge.

@bixia1 : I just fixed the Ubuntu Sanity check error.

@bixia1 : Finally, all checks were successful, and it looks like the merge can be done.

Seems auto-merge is not happening but the changes are merged into master now, so we can close this. Thank you for the PR.


tf.keras.callbacks.EarlyStopping always stops at the first epoch

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): google colab
TensorFlow installed from (source or binary): pip
TensorFlow version (use command below): 2.5.0
Python version: 3.8
CUDA/cuDNN version:  cuda11.2 cudnn8.2.0
GPU model and memory: gtx1070

Describe the current behavior
tf.keras.callbacks.EarlyStopping always stops at the first epoch .
Here is the colab :
https://colab.research.google.com/drive/1FvWFE26IPrc8vcfbfEGWLPASP6ZJIBPu?usp=sharing
Describe the expected behavior
It should stops at the best epoch like tensorflow2.4.1 .

@DachuanZhao
I Could reproduce the code shared.Please find the gist.
Note:I changed the code little bit added patience keyword,Please let me know if it works
Thanks

@DachuanZhao
I Could reproduce the code shared.Please find the gist.
Note:I changed the code little bit added patience keyword,Please let me know if it works
Thanks

It works when I change
tf.keras.callbacks.EarlyStopping(monitor='val_accuracy',verbose=1,mode="max",restore_best_weights=True)

to
tf.keras.callbacks.EarlyStopping(monitor='val_accuracy',verbose=1,mode="max",restore_best_weights=True,patience=1e-7)

But the parameter patience default value is 0 , which causes this issue ...

@DachuanZhao
Patience has to be an integer, it denoted the number of epochs to wait for , 1e-7 is not acceptable form for an epoch. [as in this gist

@DachuanZhao
Patience has to be an integer, it denotes the number of epochs to wait for , 1e-7 is not acceptable form for an epoch as in this gist

Get it . And  I think the default value of  patience should be 1 ...

@DachuanZhao
please go through this link, and example here.
Detailed explanation on patience parameter  is here, for any fuurther queries please open a stackoverflow link as done here and move this to close status as it is not a bug or a feature request.

@DachuanZhao
please go through this link, and example here.
Detailed explanation on patience parameter is here, for any fuurther queries please open a stackoverflow link as done here and move this to close status as it is not a bug or a feature request.

Let me show you the difference between 2.4.1 and 2.5.0 :
2.4.1:
  def on_epoch_end(self, epoch, logs=None):
    current = self.get_monitor_value(logs)
    if current is None:
      return
    if self.monitor_op(current - self.min_delta, self.best):
      self.best = current
      self.wait = 0
      if self.restore_best_weights:
        self.best_weights = self.model.get_weights()
    else:
      self.wait += 1
      if self.wait >= self.patience:
        self.stopped_epoch = epoch
        self.model.stop_training = True
        if self.restore_best_weights:
          if self.verbose > 0:
            print('Restoring model weights from the end of the best epoch.')
          self.model.set_weights(self.best_weights)

2.5.0 :
  def on_epoch_end(self, epoch, logs=None):
    current = self.get_monitor_value(logs)
    if current is None:
      return
    if self.restore_best_weights and self.best_weights is None:
      # Restore the weights after first epoch if no progress is ever made.
      self.best_weights = self.model.get_weights()

    self.wait += 1
    if self._is_improvement(current, self.best):
      self.best = current
      if self.restore_best_weights:
        self.best_weights = self.model.get_weights()
      # Only restart wait if we beat both the baseline and our previous best.
      if self.baseline is None or self._is_improvement(current, self.baseline):
        self.wait = 0

    if self.wait >= self.patience:
      self.stopped_epoch = epoch
      self.model.stop_training = True
      if self.restore_best_weights and self.best_weights is not None:
        if self.verbose > 0:
          print('Restoring model weights from the end of the best epoch.')
        self.model.set_weights(self.best_weights)

In 2.4.1 , no matter what patience is, the training will reach the second epoch . But in 2.5.0 , when patience = 0 , the training will stop the first epoch .

@DachuanZhao
Can you please share a colab gist with the above update, as i do not see any output for the above code.

@DachuanZhao
Can you please share a colab gist with the above update, as i do not see any output for the above code.

What should  I share ? I think you will know what I means if you see the difference of the raw code between 2.4.1 and 2.5.0 ...

@DachuanZhao Thanks for creating this issue. This is a bug recently introduced. We will work on it to update. If you are interested in it, you can raise a PR. Thanks!

@DachuanZhao Thanks for creating this issue. This is a bug recently introduced. We will work on it to update. If you are interested in it, you can raise a PR. Thanks!

I have raised a PR at https://github.com/keras-team/keras/pull/14750  : )


[WIP] Update to K8s 1.8 client go

/test pull-knative-sandbox-net-kourier-integration-tests
@nak3 if this is good to go can you remove [WIP]?

/retest
Yes, removed it. Thank you!

/lgtm
cc @davidor or @jmprusi for /approve

/approve


Add tooltip mechanism + channel control changes

This PR will add tooltips to osd viewer. Summary of changes:

Modify channel-item and channel-list to use the new tooltips (#76).
I was experimenting with adding a tooltip to indicate the value that users are about to select. At first I tried to show the tooltip while users hover over the slider. But as you can see here, doing so is not accurate. Since our slider is small, we cannot accurately translate the position of the mouse to the actual selected value. So for now I'm showing the tooltip while users are changing values.

What's left,

Make sure the tooltip UI is acceptable
Make sure the value tooltip is good enough (or whether we should take it out)
Apply the same tooltips to other places (annotation, z-plane, etc)

After discussing the changes with @karlcz I realized what I'm doing for the filters is completely wrong. Each filter is supposed to change only part of the color value. I was applying all the function to each R G B value, but

Contrast/brightness/gamma are supposed to only changed the "value" in HSV format.
Hue should only affect "hue" in HSV format.

So I made the proper changes and also

Added a new saturation control
Changed the contrast control scale. The new scale will show a [-100, 100] range to users, where the given e input will be transformed by 10 ** (e/100) formula and used in the existing formula (So it would produce 0.1 to 10 values.)

We had another discussion about this and decided that I should generate a intensity (value) histogram as a debugging tool so it's easier to figure out how the filters are working (for brightfield maybe RGB histogram might be better) and Ideally we might want both the before and after histograms.
Eventually we might want to change our filter logic to be based on the displayed image, and therefore we would need to know the min/max/medium intensity of the image. Ideally this should be generated once on load based on the highest resolution of image. But since that's not feasible we might want to do this based on the top of the pyramid image.
Other requested changes are:

Brightness should be from -1 to 1.
Tooltips:

Brightness should say "-1x of the contrast to +1x of the contrast".
Saturation should say "0% to 100%"
(Not sure about this one) For contrast we should mention "Contrast factor"

We realized that the contrast/brightness are difficult for users. So instead we decided to implement a intensity range. Users can pick the black_level and white_level values that will internally map to a slope and offset.


Hey CrowdFunding-dApp is not working to create Campaign

Hey my name is omprakash
recently I copied your project to understand blockchain but here are all the steps to run this project I created Campaign successfully, but the Campaign is not showing please check and Help me

Hi Omprakash, I tested it now and i did not see any problem. If you created campaign successfully it may take some time to process to the blockchain. Check the home page or your account page after few minutes.

Here I added my screen recording You see now the bug.
https://github.com/ahmedsemih/CrowdFunding-dApp/assets/91268538/540eaafa-81f8-440d-8f2e-3a5b9e73c512

When you are creating a campaign at the 0:53 of the video, at the top of the metamask popup, under localhost, there must be function name and the contract address but on you there are just "sending sepholiaeth" text and a "you are sending 0 eth" warning. I guess there is a problem on your contract. Eth does not send to the contract and the campaign creation function does not work.

Thank you for response 🙂
Please make one video to how to setup the .env file and contract address and other things help me with this

Hii ahmedsemih,  i am try many times not working please help me

I fixed this bug. Thank you for informing me.

Hello Ahmed,
Even i am facing the same issue as Omprakash.
When creating the campaign, everything turns out to be well until the campaign created successfully popup
but i am not able to view any campaigns in the campaign section.

Hey Omprakash,
Were you able to solve the campaign visibility issue on the main page ?
If yes, please let me know how to resolve the issue!

Hey Atharvsanp29
When you deploy your smart contract the use  this command " npx hardhat run scripts/deploy.js --network sepolia"
Right and then use the copy contract address and paste this your .env file NEXT_PUBLIC_CONTRACT_ADDRESS to

Hey brother,
I am able to view the campaigns now
Thankyou so much!


Support unicode comparison with ICU for java

This is linked with https://github.com/couchbaselabs/icu4c-android/issues/1 for tracking purpose.

Fixed in couchbase-lite-java-native
https://github.com/couchbase/couchbase-lite-java-native/pull/30


[Bug]: ts-jest v29.0.4 is significantly slow

Version
29.0.4
Steps to reproduce

Follow Jest docs to install and upgrade to the latest Jest version
Follow ts-jest docs to install and upgrade to the latest version
Run tests

Expected behavior
A small difference in test execution time.
Actual behavior
Test execution is very slow and heavy even macOS crashed once
Debug log
NA
Additional context
No response
Environment
System:
    OS: macOS 12.6.2
    CPU: (10) arm64 Apple M1 Pro

Duplicate with #3980


import guard for TorchAO not working in fbcode

https://github.com/pytorch/torchtune/blob/7d29c21149ace4be9fdc2709a03c0a21ba191010/torchtune/utils/_import_guard.py#L20C1-L29C2
_USE_NEW_TENSOR_CORE_TILED_LAYOUT_API = _is_fbcode() or (
not _is_fbcode()
and (
("dev" not in torchao_version and torchao_version >= "0.6.0")
or (
"dev" in torchao_version
and _nightly_version_ge(torchao_version, "2024-10-10")
)
)
the _is_fbcode() part needs to be removed so this can be updated in fbcode. Its preventing us from landing our diff train.

@ebsmothers can you take a look at this, i'm going to put a PR up to handle BC briefly so we can move forward with our diff train but i don't think this should be gated on fbcode, just torchao version.

@HDCharles this is the version I had added initially. The current version was added by @jerryzh168 in #1834, maybe the two of you can iron this out?

Closing this issue since I haven't heard back

sorry just saw this, yeah we can close this one


Add LeakyReLU layer

We are missing some activation layers to support the export of models from Keras fully. One of them is the LeakyReLU layer.
Add an activation layer class, write documentation for it, write a test for it, try, if possible, create a small trainable network with it (in your own GitHub) and attach a link here in the comments.
The layer should be placed here
As a reference implementation, the ReLU activation layer could be used, but feel free to improve it!
Also, support for export and import of activation layer in JSON format should be added (see ModelLoader.kt  and ModelSaver.kt)
A detailed description of the activation layer can be found here

I am willing to take this and work on it.


Ability to presign

Send networked presigned from server to avoid having to set secret and also have programatic control over user's namespace.

It would be awesome if this important feature gets implemented in the library!
@sebringj ;  where do I set the presigned url? in options as presigned? and have you tested it, does it work?

@YousefMohsen hi, to get a presigned url, you can use this to get you started: https://github.com/sebringj/siggy    what you do is in your code is just in time request to get this presigned one, then proceed with upload passing in presigned header. You'll have to modify this repo to inject the right headers for your security along with modifying the example code to check the header for a token or whatever security thing you are doing to ensure the user is signed in. My code is proprietary otherwise would give put up for you to use.
onPress={() => getPresignedUrl()
.then(signingInfo => doUpload(signingInfo))
.then(done)}

type scenario

hey @sebringj , I made a fork of this package and merged your PR for myself. One question though. I'm seeing that you're passing signature as one of your parameters. How do you get that?
I'm using aws-sdk-js on my server to generate the signedUrl, which returns a full url including the signature. Do you strip it from there?


Changed on-boarding dialog when user doesn't have any map created

Now onboarding is not a dialog anymore, it will be placed over the map and green button will open create dialog.

Fixes #4893.
REVIEWER: @josecruz and @javierarce

cc @csobier

Pending: modify gif (@josecruz).


Editor navigation history in Menu 2

Menu 2 should be the history navigation menu. Starting in menu 3 it's free for the space.

Done!


HIVE-27507: ConvertJoinMapJoin#hasOuterJoin produce incorrect result

What changes were proposed in this pull request?
Fix
Why are the changes needed?
The hasOuterJoin returns whether the last join condition is outer join.
Does this PR introduce any user-facing change?
no
Is the change a dependency upgrade?
no
How was this patch tested?
use exist test

Could you please add a unit test?

The CI failed with irrelevant reason.
./itests/qtest/target/surefire-reports/TEST-org.apache.hadoop.hive.cli.split30.TestMiniLlapLocalCliDriver.xml:51332.72: internal error: Huge input lookup

Rerun the CI and the QTests come up again. I think the failed QTests are related this PR.


(Off-Topic) - Website little typo

please make a PR in the website repository

Sure, could you give me that repo link?
The website links this repo.


Мод странно ведёт себя если в игрока выстрелили

bruh
Во всех режимах мода в версии 3.0.0 для 1.21 fabric если в игрока попадает стрела или подобный ей снаряд, то предметы и модель игрока в инвентаре растягивает
Эффект пропадает после перезапуска игры, но если в игрока снова попадёт снаряд, эффект повторится

Hi! Just a quick note: even if you think the repo owner or maintainers understand languages other than English, it's still better to write issues using the international language (or at least use a machine translation to achieve that) so that everyone can jump on in the discussion.
As for the issue itself, CWB doesn't really do anything rendering-related (except for the scaling when you change the screen resolution). So, with the details provided, I'm unable to reproduce the issue.
Do you have scaling enabled? Which mods do you use besides CWB? Does this problem occur without them? What's the minimal list of mods required to reproduce the issue?

Minecraft 1.21.3
CWB 3.0.0+1.21.3 (with scaling enabled)
Sodium 0.6.0-beta.3+mc1.21.2
Iris 1.8.0-beta.6+mc1.21.2
BSL Shaders 8.3

Just got hit by a spectral arrow a few times, and everything seems to work just fine.

The problem is caused by the real camera mod, cwb works perfectly with everything else in my modpack.
I used the latest version of the real camera mod with standard settings.

The minimum list of mods causing distortion is cwb fabric 3.0.0 mc1.21, entity model features fabric 1.21-2.2.6, entity texture features fabric 1.21-6.2.5, fabric api 0.102.0 1.21, ImmediatelyFast Fabric 1.3.0 1.21.1, iris fabric 1.8.0 beta.4 mc1.21.1, realcamera 1.21 fabric 0.6.1 beta, sodium fabric 0.6.0 beta.2 mc1.21.1, modmenu 11.0.3 and the trabens 3d arrows 1.1 resource pack. Removing one of them from this list solves the problem/

Apologies, but I still cannot reproduce the issue, even with the specified set of mods:

Either way, I wouldn't be able to fix it, 'cause, as I mentioned earlier, this mod doesn't modify Minecraft's rendering logic, especially something as specific as entity-model-related stuff that you have troubles with.
Since the issue only occurs when you use a resource pack that requires EMF, which in fact does make changes to the rendering pipeline and specifically to entity rendering, I think you'll have much better luck addressing this kerfuffle with them directly.
Even if something in CWB does happen to trigger this mandelbug in another mod (though I'm struggling to see how the game window being in a particular state would cause that), it's still ultimately a bug in that mod and should be resolved there.


Charts stops update on big dataset

Prerequisites
affected version: 3.13.5
On big datasets, charts disappear and don't update anymore.
Reproduce

reproducible with streams
run some "heavy" terminal commands with some numbers to build a chart. For example, I'm using journalctl and use ms in timestamps as chart's values
create a chart and switch to chart tab
wait until char disappears.

Addition

no any errors in logs
bar-charts aren't affected
bug has been found during testing protobuf messages, but it's confirmed on master too

not confirmed


prop.* overhaul

remove create()
make each prop type a frozen object rather than a function
update docs
consider making default array and object a shared frozen array / object (we should somehow communicate this and encourage immutability and more functional constructs like concat() and spread

why can't props be objects and functions? Are you just trying to keep the codebase small?

@thepian yeah. Functions and create() were convenience that is really unnecessary if you have Object.assign() or spread ({ ...prop.array, ...customOptions }). It doesn't reduce the code size too much, but it reduces complexity. If there's a compelling use case, I'd be happy to consider keeping it.

Interesting idea with the spread notation, it gives flexibility. I assumed that you were moving closer to the React notation which might make static analysis easier. However I guess autocomplete/docs could easily load the class and inspect the props. It might even be a better solution for analysis to load the actual code over just parsing it.
Nope, I have no requests for backwards compatibility.

However I guess autocomplete/docs could easily load the class and inspect the props.

Yeah, that's not the main purpose of this, but definitely a nice benefit of doing so. The main benefit here is the simplicity that comes with less API and less code.


Reduce workspace switching flicker

Previously, workspace switch might flicker a bit upon changing to a higher number. This could be observed by switching repeatedly between first and second workspace, for example. 2->1 does not flicker, but 1->2 sometimes shows desktop. This fixes the issue: visible workspaces are first unhidden, and then invisible ones are hidden.

Thanks! Merged with minor changes from me


command override is incorrect for s3workflows

from https://github.com/jacksund/simmate/pull/588#discussion_r1447854753

@SWeav02 I was able to confirm this was a bug -- and it's a pretty massive one, so I'm not sure why I merged your PR before fixing it 🤷♂.
Haha oh well. I'll have v0.16.0 soon to fix it


OS X Support

Addresses issue: https://github.com/NodeOS/nodeos-mount/issues/5

If you're done, I've looked over this and it's clear for a merge. I know @piranna would want to see this so I'm leaving it unmerged. Nonetheless, it's a pass from me.

Builds successfully on Travis so I'm done.

mounty->devFile is not being used on OSX, that makes it useless for device based filesystems. This would force to users to do a check for the OS on their application and use a different signature for each one, so maybe we could aprove this and publish a minor, and later do the refactoring and publish a major, what do you both think?

At the moment, I'm just doing an initial port of all the NodeOS packages to OS X just to get things semi-working, but a refactor is going to happen some point in the future.

At the moment, I'm just doing an initial port of all the NodeOS packages to OS X just to get things semi-working, but a refactor and/or api change probably needs to happen some point in the future.

Keep going on! Kudos! :-D

Any update on this? What's the current status? Is it ready to consider a merge?


Custom TextField from KeyboardDatePicker loses focus when typing in date

[x] The issue is present in the latest release. (Non future)
[x] I have searched the issues of this repository and believe that this is not a duplicate.

Current Behavior 😯
I tried to overwrite the TextField component so I could set the fullWidth prop and a Clear-Button. Here's how I did it
TextFieldComponent={props =>
<TextField
    {...props}
    margin={"none"}
    size="small"
    fullWidth
    InputProps= {{
                        ...props.InputProps,
                        endAdornment:
                          <InputAdornment position="end">
                            <IconButton onClick={() => setState({...state, dateTime: null})}>
                              <FontAwesomeIcon icon={faTrashAlt}/>
                            </IconButton>
                          </InputAdornment>
                      }}
/>}

Clearing works fine but when I try to enter a date via the keyboard the TextField loses focus after every typed number.
Expected Behavior 🐛
The TextField does not lose focus on typing like it does with the default TextField provieded.
Steps to Reproduce 🕹
https://codesandbox.io/s/angry-mirzakhani-n0cs0?fontsize=14&hidenavigation=1&theme=dark
Steps:

Add the TextFieldComponent Prop
Customize the TextField (I checked with and without the Adornments. Didnt effect the bug)
Clear the field
Try to type something
It loses focus

Context 🔦
Add a clear button and the fullwidth prop to the base InputField
Your Environment 🌎

Tech
Version

@material-ui/core
v4.11.0

@material-ui/pickers
v3.2.10

React
v16

Browser
Firefox

TypeScript
x

@date.io/date-fns
1.3.13

In addition to that the picker moves to the top left when you try to close it.

Could you provide a reproduction with v4.0.0-alpha? We do no longer support v3.

i tried using the alpha but my cpu usage goes up to 100% when I open the picker.
The console is spammed with
too much recursion
anything I can do ?

@SecretUnicorn The component needs<EMAIL_ADDRESS>

did anyone get a fix for this? I am encountering the same problem

@ntbosscher Thanks for the update. This is definitely expected. I'm closing as the renderInput API is free from this problem, in v5


Typescript Regex does not get highlighted

Hey! This project is really awesome. I found a small bug where regex string is not highlighted by theme

@OmkarK45 Thanks! Fixed it with https://github.com/Idered/chalk.ist/commit/fbc2e1a20959fcae97ce2f1e842b800406f45a74


Setup release v0.0.12 [phab:D507]

Created by schroederc at 2015-09-10 22:04:36:

schroederc wrote at 2015-09-10 22:05:22:
https://github.com/google/kythe/releases/tag/untagged-027abf763353359209f2

shahms accepted this revision at 2015-09-10 22:07:02

schroederc closed via commit ae387faa6279c2d5a9c43f0f390811683d355ce6 at 2015-09-10 22:07:52


feature: use docker multi-stage setup for faster builds

enables easier/faster development cycles because small changes will be built much quicker.
this commit if merged will:

apply the build date label only after everything has been downloaded
prevent electrum from being downloaded again and again even when making only a small change

/rebase


SaveChangesAsync fails when storing documents with FK and referenced document at the same time

Issue
When calling save changes on several documents with a FK on another document, which has not been saved to the database yet, the save fails.
The following code fails, with the exception detailed below.
 _dataRepo.Store(RespondentIdMaps);
result.Questions.ForEach(x => _dataRepo.Store(x));
await _dataRepo.SaveChangesAsync();

When splitting the save into 2 calls, the operation is successful(see example.)
 _dataRepo.Store(RespondentIdMaps);
await _dataRepo.SaveChangesAsync();
result.Questions.ForEach(x => _dataRepo.Store(x));
await _dataRepo.SaveChangesAsync();

Stack
Postgres v11
ASP.NET Core 2.2
Store Config:
options.Schema.For<RespondentIdMapping>().ForeignKey<Survey>(x => x.SurveyId)
                                               .ForeignKey<Project>(x => x.ProjectId)
                                               .ForeignKey<Country>(x => x.MarketId);

options.Schema.For<IQuestionData()
.AddSubClassHierarchy(DocumentStoreConfigExtensions.GetQuestionDataTypes())
.ForeignKey<Survey>(x => x.SurveyId)
.ForeignKey<Country>(x => x.CountryId)
.ForeignKey<RespondentIdMapping>(x => x.RespMapId);

Additional info

Project, survey and country Id already exist in the database prior to any calls.
A single IDocumentSession is used and scoped to a single httprequest.

 Error Source = Marten. Inner Exception = Npgsql.PostgresException (0x80004005): 23503: insert or update on table "mt_doc_iquestiondata" violates foreign key constraint "mt_doc_iquestiondata_resp_map_id_fkey" at Npgsql.NpgsqlConnector.<>c__DisplayClass161_0.<g__ReadMessageLong|0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Npgsql.NpgsqlConnector.<>c__DisplayClass161_0.<g__ReadMessageLong|0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming) at Npgsql.NpgsqlCommand.ExecuteDbDataReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken) at Marten.Services.UpdateBatch.executeCallbacksAsync(NpgsqlCommand cmd, CancellationToken tkn, BatchCommand batch, List1 list) at Marten.Services.UpdateBatch.<>c__DisplayClass27_1.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Marten.Services.ManagedConnection.ExecuteAsync(NpgsqlCommand cmd, Func3 action, CancellationToken token). Error StackTrace = at Marten.Services.ManagedConnection.handleCommandException(NpgsqlCommand cmd, Exception e) at Marten.Services.ManagedConnection.ExecuteAsync(NpgsqlCommand cmd, Func3 action, CancellationToken token) at Marten.Services.UpdateBatch.ExecuteAsync(CancellationToken token) at Marten.Services.UnitOfWork.ApplyChangesAsync(UpdateBatch batch, CancellationToken token) at Marten.DocumentSession.SaveChangesAsync(CancellationToken token) at DataSchema.Repositories.Interfaces.DataRepo.SaveChangesAsync()

using System;
using System.Threading.Tasks;
using Marten;

namespace marten_test
{
    class Program
    {
        static async Task Main(string[] args)
        {
            var store = DocumentStore.For(_ =>
            {
                _.AutoCreateSchemaObjects = AutoCreate.All;
                _.Connection("host=<IP_ADDRESS>;database=marten_test1;password=postgres;username=postgres");
                _.Schema.For<DocB>()
                    .AddSubClassHierarchy(typeof(DocB1), typeof(DocB2))
                    .ForeignKey<DocA>(x => x.DocAId)
                    .ForeignKey<DocC>(x => x.DocCId);
            });

            using (var session = store.OpenSession())
            {
                var doc_a = new DocA
                {
                    Id = Guid.NewGuid()
                };

                var doc_c = new DocC
                {
                    Id = Guid.NewGuid()
                };

                var docB1_1 = new DocB1
                {
                    Id = Guid.NewGuid(),
                    DocAId = doc_a.Id,
                    DocCId = doc_c.Id,
                    NameB1 =  "test"
                };

                var docB2_1 = new DocB2
                {
                    Id = Guid.NewGuid(),
                    DocAId = doc_a.Id,
                    DocCId = doc_c.Id,
                    NameB2 =  "test"
                };

                session.Store(doc_a);
                session.Store(docB1_1);
                session.Store(docB2_1);
                await session.SaveChangesAsync();
            }

            Console.WriteLine("Run completed!");
        }
    }

    class DocA
    {
        public Guid Id { get; set; }
    }

    class DocB
    {
        public Guid Id { get; set; }
        public Guid DocAId { get; set; }
        public Guid DocCId { get; set; }
    }

    class DocC
    {
        public Guid Id { get; set; }
    }

    class DocB1 : DocB
    {
        public string NameB1 { get; set; }
    }

    class DocB2 : DocB
    {
        public string NameB2 { get; set; }
    }
}

Exception stack trace
Unhandled exception. Marten.Exceptions.MartenCommandException: Marten Command Failure:$
select public.mt_upsert_doca(doc := :arg0, docDotNetType := :arg1, docId := :arg2, docVersion := :arg3);select public.mt_upsert_docb(arg_doc_a_i
d := :arg4, arg_doc_c_id := :arg5, doc := :arg6, docDotNetType := :arg7, docId := :arg8, docType := :arg9, docVersion := :arg10);select public.m
t_upsert_docb(arg_doc_a_id := :arg11, arg_doc_c_id := :arg12, doc := :arg13, docDotNetType := :arg14, docId := :arg15, docType := :arg16, docVer
sion := :arg17)$
$
23503: insert or update on table "mt_doc_docb" violates foreign key constraint "mt_doc_docb_doc_c_id_fkey"
 ---> Npgsql.PostgresException (0x80004005): 23503: insert or update on table "mt_doc_docb" violates foreign key constraint "mt_doc_docb_doc_c_i
d_fkey"
   at Npgsql.NpgsqlConnector.<>c__DisplayClass160_0.<<DoReadMessage>g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Npgsql.NpgsqlConnector.<>c__DisplayClass160_0.<<DoReadMessage>g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming)
   at Npgsql.NpgsqlCommand.ExecuteNonQuery(Boolean async, CancellationToken cancellationToken)
   at Marten.Services.UpdateBatch.<>c__DisplayClass27_1.<<ExecuteAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Marten.Services.ManagedConnection.ExecuteAsync(NpgsqlCommand cmd, Func`3 action, CancellationToken token)
  Exception data:
    Severity: ERROR
    SqlState: 23503
    MessageText: insert or update on table "mt_doc_docb" violates foreign key constraint "mt_doc_docb_doc_c_id_fkey"
    Detail: Key (doc_c_id)=(3b0cb86d-2052-4180-9202-db49ced057d3) is not present in table "mt_doc_docc".
    Where: SQL statement "INSERT INTO public.mt_doc_docb ("doc_a_id", "doc_c_id", "data", "mt_dotnet_type", "id", "mt_doc_type", "mt_version", m
t_last_modified) VALUES (arg_doc_a_id, arg_doc_c_id, doc, docDotNetType, docId, docType, docVersion, transaction_timestamp())
  ON CONFLICT ON CONSTRAINT pk_mt_doc_docb
  DO UPDATE SET "doc_a_id" = arg_doc_a_id, "doc_c_id" = arg_doc_c_id, "data" = doc, "mt_dotnet_type" = docDotNetType, "mt_doc_type" = docType, "
mt_version" = docVersion, mt_last_modified = transaction_timestamp()"
PL/pgSQL function mt_upsert_docb(uuid,uuid,jsonb,character varying,uuid,character varying,uuid) line 5 at SQL statement
    SchemaName: public
    TableName: mt_doc_docb
    ConstraintName: mt_doc_docb_doc_c_id_fkey
    File: ri_triggers.c
    Line: 3266
    Routine: ri_ReportViolation
   --- End of inner exception stack trace ---
   at Marten.Services.ManagedConnection.handleCommandException(NpgsqlCommand cmd, Exception e)
   at Marten.Services.ManagedConnection.ExecuteAsync(NpgsqlCommand cmd, Func`3 action, CancellationToken token)
   at Marten.Services.UpdateBatch.ExecuteAsync(CancellationToken token)
   at Marten.Services.UnitOfWork.ApplyChangesAsync(UpdateBatch batch, CancellationToken token)
   at Marten.DocumentSession.SaveChangesAsync(CancellationToken token)
   at marten_test.Program.Main(String[] args) in D:\tinker_room\marten_test\Program.cs:line 52
   at marten_test.Program.<Main>(String[] args)

The "C" would have to be persisted, and that makes things work. See the UT that's about to be attached to this.

Thanks @jeremydmiller but the repro code from @mysticmind does not represent my problem accurately. In the repro indeed C must be stored, however in my example C(respondentIdMaps) was being stored.
I'll try create a more accurate repro as this is still a problem for us. Im thinking there's some issue with the Subclass hierarchy that might be throwing it off.

@tonykaralis If you're going to do that, can you do it as a failing test in a PR?


Session timeout

please review.

:+1:

(let me know before the merge as I will need to update the examples as this PR changes the API).

it is merged


feat(HyperlinkModal): add hyperlink on pressing enter

Signed-off-by: Aman Sharma<EMAIL_ADDRESS>Changes

Fill hyperlink modal and press Enter to apply the link.

Here is an example demonstration:

Flags

I have used document.getElementById('form') to get the form HTML node. But I think it is not the correct way to do it in React. I tried using useRef() / React.createRef() but I got an object instead of the HTML node.
The reason I need an HTML node is to access the URL and text using .url.value and text.value.

This will need to rebase changes from #351

@irmerk done

@DianaLease yes, you are right. This is already a feature on master. Sorry for not noticing. :sweat_smile:


Custom notification types

Hi, thanks for such a great library.
For now, there are only two types of notification: alert and confirm with notyf--alert and notyf--confirm css classes respectively.
It would be better, if there was a way to add another type of notification or css class. For example, type warning (with another class).
Example API:
const notyf = new Notyf();

notyf.custom('We had some trouble', { className : 'warning' });

Or based on issue #22:
const notyf = new Notyf({
  types: {
    warning: {
      delay: 1000, // #7
      duration: 2000, // #6
      icon: 'fa-foo-fi-fu',
      notificationClass: 'notyf--warning',
    },
  },
});

const errorNotification = notyf.custom({
  type: 'warning'
  title: 'Some uncritical issue',
  message: 'We had some trouble',
});

Thanks for this idea! v3.0.0 has been released with full support for registering new types as requested. Here's how to do it:
const notyf = new Notyf({
  types: [
    {
      type: 'info',
      backgroundColor: 'blue',
      icon: false
    }
  ]
})

notyf.open({ type: 'info', message: 'Do not forget to pause the timer' });


Faster general take

Description
Speeding up the take operator for the general case (when axis != 0). Speeds should now be comparable to when axis = 0 on GPU, as opposed to being multiple times slower (depending on the shape of the array). CPU also receives a speedup, but not necessarily as fast as axis = 0.
Checklist
Essentials

[X] Changes are complete (i.e. I finished coding on this PR)
[X] All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
[X] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

[X] Modified Take kernel to run faster

Comments

Interesting edge cases to note: The Take kernel now assumes that when the take operator is called with axis=0, TakeGPU/TakeCPU will be used instead, and thus is now incompatible with axis=0
Requesting @ptrendx to review

Few questions:

Can we rename the TakeCPU/TakeGPU to something like TakeZeroAxisCPU/TakeZeroAxisGPU and Take to TakeNonzeroAxis?
Are there any other functions that use Take that could be affected by it not supporting axis=0 anymore?
Also, could you explain what makes your version faster? Is it just replacing taking values from array with actual values? If so, I wonder whether the problem does not just come from the fact that the Shape<10> is passed to the function by value instead of by const reference const Shape<10>&?

I've done a check and from what I've found only the take op calls that exact function, so I believe that isn't a problem. In any case TakeCPU/TakeGPU gives the same results as Take when axis=0, so if now or in the future there are other functions that require Take along axis=0, that can be replaced with TakeCPU/TakeGPU.
The speedup when using actual values is significantly more than with const reference, nearly the same as opposed to several times slower still. Regardless, the function itself only uses exactly 3 of the 20 elements of in_strides and out_strides, and so replacing the arrays with those three specific values may be better anyway. Would there be a reason to keep the arrays?
Additionally, while removal of the ternary operators that check for the special case axis==0 does also give some speedup, it is very small (+1-2% speed increase) compared to the other changes. So if there is a reason to keep that option, the function should still see nearly the same speedup.

Ok, then please rename Take to TakeNonzeroAxis to show the limitations and we will be good to go.

Change has been made.


Set up actions to check code standard

Possibly using the miss_hit python linter: https://github.com/florianschanda/miss_hit

add badges to readme


Frank!Doc: Restrict attributes, issue 1883

This pull request fixes issue #1883. The fix is implemented as follows:

I create an abstract class AbstractInterfaceRejector.java for finding suppressed attributes or group memberships. The algorithm finds inherited items and omits items inherited from another interface. This logic is put in the model package, not the doclet (wrapper API) package, even though it only works with FrankClass, not FrankElement. There are two concrete subclasses for omitting attributes and omitting group memberships.
We have a class AncestorChildNavigation to visit cumulative config children or attributes, referencing them by declared groups, cumulative groups or overridden children. I changed the implementation such that rejecting children and omitting overridden children is no longer in separate parts of the algorithm. We only omit overridden children.  The algorithm then omits the child of the parent class because it is overridden.
There are a few changes in ElementChild. An abstract boolean method is added to find fake children. The predicates used for filtering children are updated because we now have fake children.
Method FrankDocModel.createAttributes is updated to use the AbstractInterfaceRejector algorithm to find rejected attributes, and to make fake attributes for the rejected attribute names. This connects the AbstractInterfaceRejector to the child navigation algorithm. FrankDocModel uses the wrapper API to discover the newly created annotations FrankDocIgnoreTypeMembership and NoFrankAttribute. Work is delegated to a helper class AttributeNotRealSetter to keep the createAttributes() method small.
ElementType and FrankElement are updated to remove FrankElement-s from groups in the JSON. Some logic in FrankDocModel is needed to set the newly added fields. The comments should explain what is happening here.
I put the FrankDocIgnoreTypeMembership annotation on MessageStoreSender. I have tested the Frank!Doc website by adding a fake attributes on ITransactionalStorage and by adding a fake derived class of MessageStoreSender with a NoFrankAttribute attribute setter. Of course this fake code is not in this pull request.
A new field is introduced in the JSON for the website. Building the JSON does not happen with AncestorChildNavigation, so the website did not automatically remove rejected attributes. The new field mentions attributes to be ignored. The frontend is updated to show the attributes that are ignored, which means not inherited.

In the header of this pull request I wrote: "ElementType and FrankElement are updated to remove FrankElement-s from groups in the JSON. Some logic in FrankDocModel is needed to set the newly added fields. The comments should explain what is happening here.". This code was more complicated than needed. The field FrankElement.syntax2ExcludedFromTypes does not require a new loop of iterating FrankElement-s. We can just set this field during the construction of the FrankElement-s.

Ik heb de frontend getest na deze commits. Het is dus inderdaad nog steeds ready-for-review.


React isn't citing your licence

Your library is included in React, good job, but Facebook aren't boasting about your project being included, as your MIT licence choice suggests you would like.
So I'm asking them fix it, but thought you should know.
https://markssoftware.com/2017/01/14/npm-is-lying-to-you-and-facebook-is-stealing-copyright/

Don't care. Sorry. I make software to share with the world, not for the credit. I hereby give Facebook and anyone the right to use this code for anything without any credit or license inclusion.

That is great, but because you already have a licence and commits from
contributors: would all contributors need to make the same declaration?
I'm afraid it still doesn't mean Facebook can represent copyright ownership
of your part in their product, which I fear is another problem in what they
may have done.
On 14 Jan 2017 20:39, "Sindre Sorhus"<EMAIL_ADDRESS>wrote:
Don't care. Sorry. I make software to share with the world, not for the
credit. I hereby give Facebook and anyone the right to use this code for
anything without any credit or license inclusion.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/sindresorhus/object-assign/issues/44#issuecomment-272651383,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABs1Kqoyuui4ns8bGe7f_D5eT8PreWJSks5rSTKVgaJpZM4LjtGF
.


4i dataset #cells change

Description

Error when loading the data. @giovp did you update it? If yes, did you update any more?
Minimal reproducible example

import squidpy as sq
adata = sq.datasets.four_i()

Traceback

ValueError                                Traceback (most recent call last)
<ipython-input-23-d27e0a7b43c1> in <module>
----> 1 adata = sq.datasets.four_i()

~/.miniconda3/envs/cellrank/lib/python3.8/site-packages/squidpy/datasets/_utils.py in four_i(path, **kwargs)

~/.miniconda3/envs/cellrank/lib/python3.8/site-packages/squidpy/datasets/_utils.py in download(self, fpath, **kwargs)
     87
     88         if self.shape is not None and data.shape != self.shape:
---> 89             raise ValueError(f"Expected the data to have shape `{self.shape}`, found `{data.shape}`.")
     90
     91         return data

ValueError: Expected the data to have shape `(270876, 43)`, found `(270938, 43)`.```

</details>

### Version
<!-- Output of squidpy.__version__ -->
...

I did updated it yes I think, I'll fix it tomo, sorry !
I updated couple more but modified them I think

sorry, should have realized yesterday, but the latest version uploaded on fighsare is the correct one, and I think you have in cache the old one. If you download the latest should be good


fix module path (#560)

fix module path (#560)
run go mod tidy, go fmt, and go vet
Signed-off-by: Markus Sattler<EMAIL_ADDRESS>

rebase and updated Makefile for new paths

@saketmht As you opened the releated issue, is this PR in line with what you need and would it work for you?

@saketmht As you opened the releated issue, is this PR in line with what you need and would it work for you?

Yes. This looks good!

rebase and updated to resolve conflicts


inference example

Running the inference gives this output. Can we give a set of images as input and then it gives which cluster it belongs to.
python scripts/DeepDPM_load_from_checkpoint.py
Sequential()
[tensor([3, 2, 2, 0, 2, 1, 1, 1, 3, 2, 0, 0, 3, 0, 1, 2, 3, 5, 0, 3, 3, 2, 1, 2,
5, 3, 2, 2, 5, 2, 1, 3, 2, 2, 4, 0, 3, 2, 4, 2, 1, 0, 0, 3, 3, 4, 1, 1,
2, 5, 2, 2, 0, 0, 4, 0, 3, 2, 5, 0, 0, 2, 1, 2, 4, 3, 2, 0, 0, 3, 5, 2,
2, 1, 5, 2, 2, 2, 3, 2, 2, 0, 0, 3, 0, 1, 0, 0, 4, 2, 3, 1, 2, 3, 2, 1,
2, 3, 0, 2, 0, 2, 0, 5, 0, 2, 1, 5, 2, 3, 2, 5, 0, 2, 4, 0, 1, 3, 0, 1,
3, 1, 0, 2, 5, 2, 0, 2]),
What should be output of inference

Hi,
The values in the last printed tensor are the hard assignments for each input (you can change it quite easily to soft assignments if needed).
For example, the cluster label of the first example is 3, and the second is 2.
Does that answer your question?

@imranmu how did you pass the images to inference here?

@hafsaEn python scripts/DeepDPM_load_from_checkpoint.py

Thank you @imranmu , I tried the same to do inference on mnist, but I get the following error:
Traceback (most recent call last): File "/home/hafsa/DeepDPM/DeepDPM_load_from_checkpoint.py", line 30, in <module> model = ClusterNetModel.load_from_checkpoint( File "/home/hafsa/.conda/envs/deepdpm/lib/python3.10/site-packages/pytorch_lightning/core/saving.py", line 157, in load_from_checkpoint model = cls._load_model_state(checkpoint, strict=strict, **kwargs) File "/home/hafsa/.conda/envs/deepdpm/lib/python3.10/site-packages/pytorch_lightning/core/saving.py", line 205, in _load_model_state model.load_state_dict(checkpoint['state_dict'], strict=strict) File "/home/hafsa/.conda/envs/deepdpm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1497, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for ClusterNetModel: size mismatch for cluster_net.class_fc1.weight: copying a param with shape torch.Size([50, 10]) from checkpoint, the shape in current model is torch.Size([50, 784]). size mismatch for subclustering_net.class_fc1.weight: copying a param with shape torch.Size([50, 10]) from checkpoint, the shape in current model is torch.Size([50, 784]).
have you came across the same error? or can you please share the content of the file scripts/DeepDPM_load_from_checkpoint.py?

Hi, The values in the last printed tensor are the hard assignments for each input (you can change it quite easily to soft assignments if needed). For example, the cluster label of the first example is 3, and the second is 2. Does that answer your question?

Hi @meitarronen - I'm coming across the same thing as the OP, and it looks like scripts/DeepDPM_load_from_checkpoint.py outputs the hard assignments for each value in the input dimension. For example, I'm attempting to cluster input tensors of size 512x512 and the inference output I'm getting from your script for each input is an array of 512 hard-predicted cluster values. A dataset of 32 512x512 input tensors yields an output that is a list of 32 arrays of length 512 (of ). Your inference code is straight forward, and I can't find any issues with it. Am I missing something?
Thanks in advance!


support impersonation via auth_options and in kubectl config

This PR is rebase and update of https://github.com/ManageIQ/kubeclient/pull/524 to current master version
We tested it and it works as expected
Ruby http clients (including adapters for Faraday) are not able to handle multiple headers of same name, which is how the kube API expects to receive the list of multiple groups or extra fields. To avoid any confusion, added explicit error when trying to configure client with multivalue groups. This may brake some applications, if user is not aware of the impersonation setting and were ignoring it until now

[ ] Please add an entry in CHANGELOG.md.  We grew a huge "changelog debt" :-(, which I hope to make up one day, but since this might break somebody best document it right away.

What about watches?
Watches still use https://github.com/httprb/http gem (#488 having some unresolved issues).
Does this implementation affect the headers we send there too?  I think so, http_options method takes @headers.

[ ] Please add a unit test for watch.  As long as we have 2 easy-to-diverge implementations, I prefer not to make HTTP-level changes without test coverage for both.

Otherwise LGTM :rocket:

Thanks @cben. Is there any plan to publish new release with the update?

I need to make progress on my 5.0 changelog backlog to get master branch releasable...
I'm sorry, I can't give time estimate. But nagging is welcome ;-|
Alternatively, do you want to work on backporting this to v4.y branch which uses rest-client?
It'd require some changes, not sure it's worth the effort.

Ok, no problem. Just asking. It is not super critical for us, so we can wait (and we can always install gem from git if we need to)


Replace merging approach to a three-way merge

It always succeeds and, instead of leaving .rej files, it puts merge markers in the destination.
In my view the three-way merge conflict markers are easy to understand, while .rej files are super annoying and not very useful.
And, well, the merging approach here changes from doing a two-way diff and applying it to the destination, to just doing a three-way diff. Which I believe may lead to better results and not suffer from weirdness trying to save and restore a unified diff.
For example, whereas before you'd get an "overwrite?" prompt and a .rej file saying that python = "^3.6" changed to python = "^3.7", now none of that happens and this gets written into the file:
Screenshot from VS Code:

Example is based on https://github.com/oprypin/mkdocs-same-dir/blob/b929337215ebe3f469b11b59b44052bb2546ac41/.tools/copier-answers.yml and changing min_python_version: '3.7'.

For now this PR is just an example so you can try it out (please do try it out!) and see if you like the approach.
This pull request is not in a state that's for merging. Moreover, it is based on an old state of the codebase because I found that https://github.com/copier-org/copier/pull/314 adds a lot of assumptions that the destination directory is the same as the source directory, making this very difficult to achieve (at least I couldn't).
I recommend to view this PR with ignoring whitespace

That screenshot is really selling the point.
I agree on the rej files: I almost never look at them and simply delete them just after an update (my make clean runs a find . type -name "*.rej" -delete).
And for medium-to-big updates, I always end up staging parts of the modifications and reverting some others, or merging manually (which is error prone: easy to miss a line that was deleted and should have been kept) directly in VSCode, so actual markers like presented in the screenshot would definitely help during this process!

For me, those files help to know if the update was successful, while combined with a pre-commit hook that forbids them. Do you know of any hooks that forbid committing these kind of diffs?

Answering my own question, yes there's this hook: https://github.com/pre-commit/pre-commit-hooks#check-merge-conflict
So this would be a totally acceptable and great improvement. The PR is very outdated now, and it needs tests (well... actually I think it will break tests, so fixing those should be enough probably). May you continue with it @oprypin?

@Yajo Thanks for consideration.
The PR was "outdated" from the start: I intentionally worked on an old version of the project because, in my opinion, with the more recent refactors the code is harder to work with in general and also has more assumptions about file states that go against this approach.
I never really fully learned the codebase and by now I probably forgot everything, so it seems unlikely that I will pick up this PR myself.
I also had some trouble dealing with the fact that the "Overwrite? y/n" prompt doesn't make sense anymore and how to replace it. And maybe other similar decisions.
The merge function is the part that's "interesting" and also quite universal. Maybe it would be productive if you take only that function and implement the other parts of integrating it with the current code.

I intentionally worked on an old version

If you never wanted to produce a mergeable PR, it would have been more productive to open an issue. Closing.


Enable regular expressions containing \s and \S

Closes #4528
Signed-off-by: sperlingxx<EMAIL_ADDRESS>Enable regular expressions which contain \s and \S and provide corresponding tests.

build

It looks like there may be a version mismatch or something.
[2022-03-30T12:27:10.759Z]     def deco(*a, **kw):
[2022-03-30T12:27:10.759Z]         try:
[2022-03-30T12:27:10.759Z]             return f(*a, **kw)
[2022-03-30T12:27:10.759Z]         except py4j.protocol.Py4JJavaError as e:
[2022-03-30T12:27:10.759Z]             converted = convert_exception(e.java_exception)
[2022-03-30T12:27:10.760Z]             if not isinstance(converted, UnknownException):
[2022-03-30T12:27:10.760Z]                 # Hide where the exception came from that shows a non-Pythonic
[2022-03-30T12:27:10.760Z]                 # JVM exception message.
[2022-03-30T12:27:10.760Z] >               raise converted from None
[2022-03-30T12:27:10.760Z] E               pyspark.sql.utils.AnalysisException: Undefined function: 'regexp'. This function is neither a registered temporary function nor a permanent function registered in the database 'default'.; line 1 pos 0
[2022-03-30T12:27:10.760Z]
[2022-03-30T12:27:10.760Z] ../../../.download/spark-3.1.1-bin-hadoop3.2/python/pyspark/sql/utils.py:117: AnalysisException
...
FAILED ../../src/main/python/string_test.py::test_regexp_whitespace - pyspark...

Could you also update RegularExpressionTranspilerSuite to add \f and \u000b to the list of characters used by the fuzzer tests. That should be enough to see if there are any edge cases not covered here.

Rather than pass \s and \S to cuDF, we should transpile to a pattern that would guarantee compatibility with Java, so something like [ \f\t\r\n\\x0b].

Rather than pass \s and \S to cuDF, we should transpile to a pattern that would guarantee compatibility with Java, so something like [ \f\t\r\n\\x0b].

Yes, I changed the parser to transpile \s as [ \f\t\r\n\\x0b].

build


Make it easy to customize cnf/crt/csr/key paths

Hello @robbat2 !
Thank you for your PR, but it seems it breaks the unit tests (especially the second commit). You might have a look here for example:
https://travis-ci.org/camptocamp/puppet-openssl/jobs/62590856#L256
You should update the unit tests in order to ensure the new cnf_* values are taken into account as expected.
Thank you for your update.
Cheers,
C.

Thank you for this @robbat2 . Could you rebase on master and squash your commits please? And then I'll merge.

Rebased+squashed as requested.


Covenant Hearthstones can only be used by pledged members as of 9.0.5

As described here, in 9.0.5 Blizzard made all the covenant hearthstones require that you be pledged to that covenant to use them.  This was always the case for the Necrolords, but now applies to all four.
I've created https://github.com/hascat/HearthRoulette/pull/5 to adapt the selection mechanism to reflect this change.

Fix merged


Port to N-API

A fun task would be to start a branch of skel that switches NAN for N-API.
This branch would:

Drop support for node versions (in the travis.yml) that don't support N-API
Drop usage of NAN

The goal of this branch would be:

Help us learn about N-API
Help provide simple examples of N-API usage for the time when we want to (potentially) move all modules to it.
Learn what we don't know we need to learn about N-API (this is probably the most important one)

Caveat: The reality is that skel will need to support older node versions for a while, so this branch would need to stay in a branch and might drift/get out of date. And it would naturally have tons of conflicts with master.

This remains relevant and important to learn about. I'm adding this to my list of things to look into. My plan is to find 30 minutes to experiment. I will create a branch and try porting things over and see how far I can get. /cc @GretaCB

With the release of node 10 this week (https://nodejs.org/en/blog/release/v10.0.0/), N-API is now no longer experimental (https://github.com/mapbox/node-pre-gyp/pull/345 was key to getting "out of experimental status") so this means:

@mapbox/core-tech should plan to port all our modules to N-API
the first step I think should be to port node-cpp-skel to N-API

done in #161 🎉


System won't keep me signed in

I've been experiencing this more frequently—the system doesn't keep me signed in.
As I'm typing this, I realized that part of the problem may be that I'm signed out on my desktop whenever I sign in on my phone. (This is super annoying).
After this happens, when I go back to the desktop, it acts like I'm signed in

after a few seconds, register button disappears
Give as guest stays
Sign In becomes 'Sign In with (card image)' but no last 4

A refresh shows me that I am, in fact, signed out.

@jonhorton yep. we are tracking on #273. We think it has to do with with some of the settings that are updated on the user from Rock when signing in.

@johnthepink @jonhorton if I had to bet, its because of this:
https://github.com/NewSpring/Apollos/blob/master/core/methods/accounts/server/login.js#L82
and corresponding docs
http://docs.meteor.com/#/full/accounts_setpassword
Should be a pretty easy thing to test (open chrome and safari with the change locally, verify it still signs you out, then change to Accounts.setPassword(userAccount._id, password, { logout: false }) and try again

confirmed that fixed it: