Error when running Tutorial Stock_NeurIPS2018_SB3.ipynb Can only use .dt accessor with datetimelike values

Describe the bug
Can only use .dt accessor with datetimelike values
I used the exact same tutorials and packages in colab and locally, but had different results
To Reproduce
Steps to reproduce the behavior:

df = YahooDownloader(start_date = TRAIN_START_DATE,
end_date = TRADE_END_DATE,
ticker_list = config_tickers.DOW_30_TICKER).fetch_data()
See error
~\Anaconda3\envs\py37\lib\site-packages\pandas\core\accessor.py in get(self, obj, cls)
179             # we're accessing the attribute of the class, i.e., Dataset.geo
180             return self._accessor
--> 181         accessor_obj = self._accessor(obj)
182         # Replace the property with the accessor object. Inspired by:
183         # https://www.pydanny.com/cached-property.html

~\Anaconda3\envs\py37\lib\site-packages\pandas\core\indexes\accessors.py in new(cls, data)
504             return PeriodProperties(data, orig)
505
--> 506         raise AttributeError("Can only use .dt accessor with datetimelike values"
Can only use .dt accessor with datetimelike values
Expected behavior
[100%**]  1 of 1 completed
Shape of DataFrame:  (94331, 8)
Screenshots
incolab
local

Desktop (please complete the following information):

OS: [win10]
Browser [ chrome,]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
I used the exact same tutorials and packages in colab and locally, but had different results，same pandas

Hi, sorry for delayed response.
Your description is not very clear. I guess the data may contain timezones. you may pass utc=True to to_datetime, and set errors='coerce'. I am not sure.
could you pls paste all the code here?

Stock_NeurIPS2018_SB3.zip
hi，The full code is the file in the tutorial. The strange thing is that I have no problem running it on colab, I have problems running it locally, all the versions of the  Package are the same. Also the problem occurs when I download the dataset on colab locally for pre-processing.It seems that the local code does not work with the Yahoo Finance data, but the same code works fine on colab

Stock_NeurIPS2018_SB3.zip
hi，The full code is the file in the tutorial. The strange thing is that I have no problem running it on colab, I have problems running it locally, all the versions of the Package are the same. Also the problem occurs when I download the dataset on colab locally for pre-processing.It seems that the local code does not work with the Yahoo Finance data, but the same code works fine on colab

Your description seems to echo Ming's judgement: colab is in the US time zone, and your local machine is in a different time zone.

At the same time,if i set the vis==true,there is same problem.

Is this because it doesn't read the downloaded Yahoo Finance data very well, but it doesn't have this problem when running on colab, which makes me confused

Stock_NeurIPS2018_SB3.zip
hi，完整代码是教程中的文件。奇怪的是，我在colab上运行没有问题，我在本地运行有问题，所有版本的Package都是一样的。当我在本地下载 colab 上的数据集进行预处理时也会出现问题。本地代码似乎不适用于 Yahoo Finance 数据，但相同的代码在 colab 上运行良好

您的描述似乎与 Ming 的判断相呼应：colab 在美国时区，而您的本地机器在不同的时区。

ok,I will try to adjust the time zone

Hi, sorry for delayed response.
Your description is not very clear. I guess the data may contain timezones. you may pass utc=True to to_datetime, and set errors='coerce'. I am not sure.
could you pls paste all the code here?

Do I need to add these two changes to jupyter or to the Yahoo download function

yahoodownloader.zip
I have uploaded this code about downloading data for your reference

@moon195
In yahoodownloader.py, which lines did you revise?
How did you call the functions in yahoodownloader.py? could you pls send related codes to me?
I tried the following, and it works:
tickers = ['AXP', 'AMGN']
start_date = '2022-01-15'
end_date = '2022-02-15'
y = YahooDownloader(start_date, end_date, tickers)
df = y.fetch_data()
df2 = y.select_equal_rows_stock(df)

@zhumingpassional Thank you for your reply, I will make changes and try，by the way，This is the original code for calling Yahoo Finance
yahoodownloader.zip

I tried the original code, and it works.
yahoo stopped the service in some countries, e.g., China. If you are in these countries and run the code locally, you may need to use VPN

@zhumingpassional I tried your code but it still doesn't work because of my computer's settings regarding time zones? Or is it because I can only use vpn for access?

yes. pls check your computer settings and vpn.

是的。请检查您的计算机设置和 vpn。

@zhumingpassional Thank you very much for your help and have a great day.


Feature request

Hi, thanks for the versatile repo for combination of different VE and LLMs,
However,  I found there seems haven't any compare with Resampler and MLP's
Would like add such a experiment to compare with Resampler and MLPs?

Hey @lucasjinreal -- sorry that I haven't responded to this sooner! It should be fairly straightforward to add new "vision-to-language" projection layers; you should just need to override the logic in the PrismaticVLM class.
Let me know if you have any trouble!


Issue linking ring buffers for new board support

Hi @cfriedt @vaishnav98
I am trying to add support for UART transport, and ran into some issues
arm-zephyr-eabi/arm-zephyr-eabi"/lib/thumb/v7e-m+fp/hard  -specs=nano.specs && :
/home/anuj/zephyr-sdk-0.12.3/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/10.2.0/../../../../arm-zephyr-eabi/bin/ld: modules/greybus/subsys/greybus/lib..__greybus__subsys__greybus.a(transport-uart.c.obj): in function `uart_work_fn':
/home/anuj/zephyrproject/greybus/subsys/greybus/platform/transport-uart.c:57: undefined reference to `ring_buf_get'
/home/anuj/zephyr-sdk-0.12.3/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/10.2.0/../../../../arm-zephyr-eabi/bin/ld: /home/anuj/zephyrproject/greybus/subsys/greybus/platform/transport-uart.c:82: undefined reference to `ring_buf_get'
/home/anuj/zephyr-sdk-0.12.3/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/10.2.0/../../../../arm-zephyr-eabi/bin/ld: modules/greybus/subsys/greybus/lib..__greybus__subsys__greybus.a(transport-uart.c.obj): in function `gb_xport_uart_isr':
/home/anuj/zephyrproject/greybus/subsys/greybus/platform/transport-uart.c:195: undefined reference to `ring_buf_get'
/home/anuj/zephyr-sdk-0.12.3/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/10.2.0/../../../../arm-zephyr-eabi/bin/ld: /home/anuj/zephyrproject/greybus/subsys/greybus/platform/transport-uart.c:204: undefined reference to `ring_buf_put'

I saw this with both stm32f3_disco and lpcxpresso55s69_cpu0. This was after I added the requisite board conf and overlay files.
I am trying to understand what these chipsets need to support in order for UART to work. One thing I did realize that both these chips have usart and not uart in their device trees under Zephyr. Could that be a problem when adding support for them?

Interesting. I have a couple of disco boards, so perhaps I can check it out.. will take me a day or so to look into it.

@cfriedt i pushed a branch for you here - https://github.com/anujdeshpande/greybus-for-zephyr/tree/add-stf303-uart
If it's simpler I can open a PR here. I don't think you'll need a board tbh - since this seems an issue when building.

Full logs in case it helps -
(base) ➜ zephyr (fc1b5de4c3) ✗ west build -b $BOARD ../greybus/samples/subsys/greybus/uart
-- west build: generating a build system
-- Application: /home/anuj/zephyrproject/greybus/samples/subsys/greybus/uart
-- Zephyr version: 2.4.99 (/home/anuj/zephyrproject/zephyr)
-- Found Python3: /home/anuj/miniconda3/bin/python3.7 (found suitable exact version "3.7.6") found components: Interpreter
-- Found west (found suitable version "0.9.0", minimum required is "0.7.1")
-- Board: stm32f3_disco
-- Cache files will be written to: /home/anuj/.cache/zephyr
ZEPHYR_TOOLCHAIN_VARIANT not set, trying to locate Zephyr SDK
-- Found toolchain: zephyr (/home/anuj/zephyr-sdk-0.12.3)
-- Found dtc: /home/anuj/zephyr-sdk-0.12.3/sysroots/x86_64-pokysdk-linux/usr/bin/dtc (found suitable version "1.5.0", minimum required is "1.4.6")
-- Found BOARD.dts: /home/anuj/zephyrproject/zephyr/boards/arm/stm32f3_disco/stm32f3_disco.dts
-- Found devicetree overlay: /home/anuj/zephyrproject/greybus/samples/subsys/greybus/uart/boards/stm32f3_disco.overlay
-- Generated zephyr.dts: /home/anuj/zephyrproject/zephyr/build/zephyr/zephyr.dts
-- Generated devicetree_unfixed.h: /home/anuj/zephyrproject/zephyr/build/zephyr/include/generated/devicetree_unfixed.h
Parsing /home/anuj/zephyrproject/zephyr/Kconfig
Loaded configuration '/home/anuj/zephyrproject/zephyr/boards/arm/stm32f3_disco/stm32f3_disco_defconfig'
Merged configuration '/home/anuj/zephyrproject/greybus/samples/subsys/greybus/uart/prj.conf'
Merged configuration '/home/anuj/zephyrproject/greybus/samples/subsys/greybus/uart/boards/stm32f3_disco.conf'
Configuration saved to '/home/anuj/zephyrproject/zephyr/build/zephyr/.config'
Kconfig header saved to '/home/anuj/zephyrproject/zephyr/build/zephyr/include/generated/autoconf.h'
-- The C compiler identification is GNU 10.2.0
-- The CXX compiler identification is GNU 10.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /home/anuj/zephyr-sdk-0.12.3/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc
-- Configuring done
-- Generating done
-- Build files have been written to: /home/anuj/zephyrproject/zephyr/build
-- west build: building application
[1/163] Preparing syscall dependency handling

[125/163] Building C object modules/greybus/subsys/greybus...greybus__subsys__greybus.dir/platform/transport-uart.c.obj
/home/anuj/zephyrproject/greybus/subsys/greybus/platform/transport-uart.c: In function 'gb_xport_send':
/home/anuj/zephyrproject/greybus/subsys/greybus/platform/transport-uart.c:149:18: warning: passing argument 1 of 'sendMessage' discards 'const' qualifier from pointer target type [-Wdiscarded-qualifiers]
  149 |  r = sendMessage(uart_dev, msg);
      |                  ^~~~~~~~
/home/anuj/zephyrproject/greybus/subsys/greybus/platform/transport-uart.c:102:39: note: expected 'struct device *' but argument is of type 'const struct device *'
  102 | static int sendMessage(struct device *dev, struct gb_operation_hdr *msg)
      |                        ~~~~~~~~~~~~~~~^~~
[130/163] Building C object modules/greybus/subsys/greybus...eFiles/..__greybus__subsys__greybus.dir/platform/i2c.c.obj
/home/anuj/zephyrproject/greybus/subsys/greybus/platform/i2c.c:35:12: warning: 'greybus_i2c_control_init' defined but not used [-Wunused-function]
   35 | static int greybus_i2c_control_init(const struct device *dev) {
      |            ^~~~~~~~~~~~~~~~~~~~~~~~
[158/163] Linking C executable zephyr/zephyr_prebuilt.elf
FAILED: zephyr/zephyr_prebuilt.elf
: && ccache /home/anuj/zephyr-sdk-0.12.3/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc    zephyr/CMakeFiles/zephyr_prebuilt.dir/misc/empty_file.c.obj  -o zephyr/zephyr_prebuilt.elf  -Wl,-T  zephyr/linker.cmd  -Wl,-Map=/home/anuj/zephyrproject/zephyr/build/zephyr/zephyr_prebuilt.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/aarch32/libarch__arm__core__aarch32.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/mpu/libarch__arm__core__aarch32__cortex_m__mpu.a  zephyr/lib/libc/newlib/liblib__libc__newlib.a  zephyr/lib/posix/liblib__posix.a  zephyr/soc/arm/common/cortex_m/libsoc__arm__common__cortex_m.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/i2c/libdrivers__i2c.a  zephyr/drivers/serial/libdrivers__serial.a  modules/greybus/subsys/greybus/lib..__greybus__subsys__greybus.a  modules/stm32/stm32cube/lib..__modules__hal__stm32__stm32cube.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  zephyr/CMakeFiles/offsets.dir/arch/arm/core/offsets/offsets.c.obj  -L"/home/anuj/zephyr-sdk-0.12.3/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/10.2.0/thumb/v7e-m+fp/hard"  -L/home/anuj/zephyrproject/zephyr/build/zephyr  -lgcc  -Wl,--print-memory-usage  zephyr/arch/common/libisr_tables.a  -mcpu=cortex-m4  -mthumb  -mabi=aapcs  -mfpu=fpv4-sp-d16  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -no-pie  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn  -lm  -lc  -lgcc  -L"/home/anuj/zephyr-sdk-0.12.3/arm-zephyr-eabi/arm-zephyr-eabi"/lib/thumb/v7e-m+fp/hard  -specs=nano.specs && :
/home/anuj/zephyr-sdk-0.12.3/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/10.2.0/../../../../arm-zephyr-eabi/bin/ld: modules/greybus/subsys/greybus/lib..__greybus__subsys__greybus.a(transport-uart.c.obj): in function `uart_work_fn':
/home/anuj/zephyrproject/greybus/subsys/greybus/platform/transport-uart.c:57: undefined reference to `ring_buf_get'
/home/anuj/zephyr-sdk-0.12.3/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/10.2.0/../../../../arm-zephyr-eabi/bin/ld: /home/anuj/zephyrproject/greybus/subsys/greybus/platform/transport-uart.c:82: undefined reference to `ring_buf_get'
/home/anuj/zephyr-sdk-0.12.3/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/10.2.0/../../../../arm-zephyr-eabi/bin/ld: modules/greybus/subsys/greybus/lib..__greybus__subsys__greybus.a(transport-uart.c.obj): in function `gb_xport_uart_isr':
/home/anuj/zephyrproject/greybus/subsys/greybus/platform/transport-uart.c:195: undefined reference to `ring_buf_get'
/home/anuj/zephyr-sdk-0.12.3/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/10.2.0/../../../../arm-zephyr-eabi/bin/ld: /home/anuj/zephyrproject/greybus/subsys/greybus/platform/transport-uart.c:204: undefined reference to `ring_buf_put'
Memory region         Used Size  Region Size  %age Used
           FLASH:       48248 B       256 KB     18.41%
            SRAM:       31392 B        40 KB     76.64%
        IDT_LIST:         216 B         2 KB     10.55%
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
FATAL ERROR: command exited with status 1: /usr/bin/cmake --build /home/anuj/zephyrproject/zephyr/build

@cfriedt did you get time to check into this? Let me know if there's some more info that I can provide

I was digging through this more -
build-failure.log
cmake version 3.16.3 is OK; minimum version is 3.13.1
Running CMake: /usr/bin/cmake --build /home/anuj/zephyrproject/zephyr/build -- -v
[1/163] cd /home/anuj/zephyrproject/zephyr/build/zephyr && /usr/bin/cmake -E echo

build-success.log
cmake version 3.16.3 is OK; minimum version is 3.13.1
Running CMake: /usr/bin/cmake --build /home/anuj/zephyrproject/zephyr/build -- -v
[1/171] cd /home/anuj/zephyrproject/zephyr/build/zephyr && /usr/bin/cmake -E echo

So there's a bunch of tasks that cmake isn't picking up on. I was digging more into what it missed, and I couldn't find this in the one that fails
[34/171] ccache /home/anuj/zephyr-sdk-0.12.3/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc -DBUILD_VERSION=zephyr-v2.4.0-3009-gfc1b5de4c307 -DKERNEL -D_FORTIFY_SOURCE=2 -D__LINUX_ERRNO_EXTENSIONS__ -D__PROGRAM_START -D__ZEPHYR__=1 -I../include -Izephyr/include/generated -I../soc/arm/ti_simplelink/cc13x2_cc26x2 -I../lib/libc/newlib/include -I/home/anuj/zephyrproject/greybus/include -I/home/anuj/zephyrproject/greybus/subsys/greybus/../../../include -I/home/anuj/zephyrproject/modules/hal/cmsis/CMSIS/Core/Include -I/home/anuj/zephyrproject/modules/hal/ti/simplelink/source/ti/devices/cc13x2_cc26x2/. -I/home/anuj/zephyrproject/modules/hal/ti/simplelink/. -I/home/anuj/zephyrproject/modules/hal/ti/simplelink/source -I/home/anuj/zephyrproject/greybus/samples/subsys/greybus/uart/../../../../../include -Os -imacros /home/anuj/zephyrproject/zephyr/build/zephyr/include/generated/autoconf.h -ffreestanding -fno-common -g -mcpu=cortex-m4 -mthumb -mabi=aapcs -imacros /home/anuj/zephyrproject/zephyr/include/toolchain/zephyr_stdint.h -Wall -Wformat -Wformat-security -Wno-format-zero-length -Wno-main -Wno-pointer-sign -Wpointer-arith -Wno-address-of-packed-member -Wno-unused-but-set-variable -Werror=implicit-int -fno-asynchronous-unwind-tables -fno-pie -fno-pic -fno-strict-overflow -fno-reorder-functions -fno-defer-pop -fmacro-prefix-map=/home/anuj/zephyrproject/greybus/samples/subsys/greybus/uart=CMAKE_SOURCE_DIR -fmacro-prefix-map=/home/anuj/zephyrproject/zephyr=ZEPHYR_BASE -fmacro-prefix-map=/home/anuj/zephyrproject=WEST_TOPDIR -ffunction-sections -fdata-sections -specs=nano.specs -std=c99 -MD -MT zephyr/CMakeFiles/zephyr.dir/lib/os/ring_buffer.c.obj -MF zephyr/CMakeFiles/zephyr.dir/lib/os/ring_buffer.c.obj.d -o zephyr/CMakeFiles/zephyr.dir/lib/os/ring_buffer.c.obj   -c /home/anuj/zephyrproject/zephyr/lib/os/ring_buffer.c

The only thing that's done before this cmake, is the device tree stuff I believe. So this seems to be coming from that. Will keep digging
build-failure.log
build-success.log

@vaishnav98 or @cfriedt - do you know if the greybus uart transport cares if the peripheral is usart or uart.

@anujdeshpande - I would guess that CONFIG_RING_BUFFER is not set. Try this:
grep RING build/zephyr/.config
# CONFIG_RING_BUFFER is not set

I see it's not in samples/subsys/greybus/uart/prj.conf, so if you're making a PR, please add it there.

Going to give it a quick shot now. Been very busy in the last couple of weeks

@cfriedt that fixed it! I didn't realize that we needed to add that config to the cmake. I can build without errors now. Will try it out on hardware and send a PR!

I'm curious tho - Why didn't the Nordic or TI boards need this flag tho? How were they picking this up by default

It's quite likely that one of the UART or wireless drivers already pulls in ring buffer support on those boards.

@anujdeshpande - were you able to resolve this one? Did you make a PR?

@cfriedt sent PR! Apologies, forgot about pushing this upstream!


Fix box inside box collapsing error (#1681)

Previously, with a .box inside a .box, any clicking on a collapse/expand button in either .box would cause both of them to collapse/expand.
This pull request makes sure parent and child boxes do not interfere with each other.
This should fix #1681

Thanks!


Process as test (VM -> Native test)

Resolves #36
See the usage here: https://github.com/juspay/services-flake/pull/38
TODO

[x] Add a PC package to checks only if it has a test process
[ ] Mark testScript as deprecated and that it will be removed in the future versions

@srid What's the right way to go about marking testScript option as deprecated?


runcon: added implementation and tests.

Added implementation and tests for the runcon command.
For reference: Supporting-SELinux-in-the-coreutils.
All runcon tests succeed on a virtual machine with SELinux enabled:
$ cargo test --features=feat_selinux -- test_runcon::
    Finished test [unoptimized + debuginfo] target(s) in 0.31s
     Running unittests (target/debug/deps/coreutils-9d7237f30b86389a)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/tests.rs (target/debug/deps/tests-739626cc4df16fec)

running 6 tests
test test_runcon::help ... ok
test test_runcon::plain_context ... ok
test test_runcon::version ... ok
test test_runcon::invalid ... ok
test test_runcon::print ... ok
test test_runcon::custom_context ... ok

test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 1329 filtered out; finished in 0.12s

Closes  #1973

For reference: Supporting-SELinux-in-the-coreutils.

You are really impressive! Only stty will be missing after this is merged!

@miDeb, thank you for the review. I implemented the discussed changes. Can you take a look?

The two following testing tasks are still pending for this command:

Check expected exit statuses of the command.
Check the implementation of --compute somehow.
I checked this manually by comparing the operation of the GNU implementation with our implementation through strace. I hope we can find a way to check this automatically instead.

Check expected exit statuses of the command.

You can do this with the code_is() function in tests. E.g. new_ucmd!().args(args).fails().code_is(126).

Only stty left as a program, wahou!


Fix RBAC permissions for leases (leader election)

Proposed Changes

Fix the lease name for leader election in the controller's RBAC role

The package was renamed after knative/eventing-contrib was split, but, since we explicitly define the name of the lease that we allow the controller to read/update, the name of that lease doesn't match what's in the RBAC config anymore.
The controller logs are full of this error:
leaderelection.go:320] error retrieving resource lock knative-sources/gitlab-controller.knative.dev-eventing-gitlab-pkg-reconciler-source.reconciler.00-of-01: leases.coordination.k8s.io "gitlab-controller.knative.dev-eventing-gitlab-pkg-reconciler-source.reconciler.00-of-01" is forbidden: User "system:serviceaccount:knative-sources:gitlab-controller-manager" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "knative-sources"

Release Note

🐛 Fix RBAC permissions preventing leader election from working as intended

Docs

/lgtm
/approve


Pending async actions

Revived from old Este.
https://gist.github.com/steida/b453d9767735fe52770b

It's nice, but it should be a middleware.


E2E basic state support

This adds execution side support for state.

You can see the count maintained by state which slowly increments (its the value in parentheses, not the recurring 1.
Part of #22736

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @riteshghorse
This should look familiar :) it adds the execution half of state support. Note that there are some lingering coder issues here and I didn't focus on getting that right - there will be a follow up PR to handle that. If you notice obvious things that are off with coders feel free to offer ideas (coders are a bit of a weak point for me), but I wouldn't focus too much on that (in particular, much of userstate.go will probably look different with the next pass)

Run GoPortable PreCommit

I think I fixed the coding issues, so a follow up PR should no longer be necessary and this should get everything done in one pass. Next steps beyond this PR are now:

Integration test
Broader support for other kinds of state, clear operations, and prefetch.

This is great. I'll take a look soon.


Feature request - Configuration option to disable/ignore external objects for Database Explorer due to Redshift performance impacting user experience

Driver version
https://s3.amazonaws.com/redshift-downloads/drivers/jdbc/<IP_ADDRESS>/redshift-jdbc42-<IP_ADDRESS>.jar
Redshift version
PostgreSQL 8.0.2 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.4.2 20041017 (Red Hat 3.4.2-6.fc3), Redshift 1.0.40182
Client Operating System
macOS Big Sur - 11.5.2
JAVA/JVM version
openjdk 17.0.1 2021-10-19
OpenJDK Runtime Environment Temurin-17.0.1+12 (build 17.0.1+12)
OpenJDK 64-Bit Server VM Temurin-17.0.1+12 (build 17.0.1+12, mixed mode, sharing)
Problem description
Using the database explorer to view all objects (* schema) takes an incredibly long time to return for several of our clusters. This is variable by usage and number of external objects.
The queries themselves are slow within Redshift:

select count(*) from svv_external_tables; -- 11,219 rows @ 2m 32s
select count(*) from svv_tables; -- 19,776 rows @ 2m 21s
select count(*) from svv_all_tables; -- 18,409 rows @ 2m 26s
this query -- 8,891 rows @ 1.64s

If there were a configuration option to force a specific type of schemaPatternType for getTables retrieval, it would alleviate a lot of issues with our user base that is spread across many different tools.
Thank you!

Fixed in https://github.com/aws/amazon-redshift-jdbc-driver/pull/65
Will be available in driver version <IP_ADDRESS>


Validation of Palo Alto Solution Template fails with InvalidResourceLocation if resource group location differs from Sentinel log analytics workspace location

Describe the bug
If the Sentinel Log Analytics workspace and its resource group have different locations, some solutions can’t be deployed.
The template fails to find a log analytics workspace in the resource group location, so it tries to create a new log analytics workspace in the resource group location. This fails as the existing log analytics workspace having the same name prevents it from being created.
The Palo-Alto solutions (CDL and Prisma) are concerned.
To Reproduce
Steps to reproduce the behavior:

Create a resource group in location A in one of your subscriptions
Create a log analytics workspace in location B (B different than A) in that resource group
Add a Sentinel workspace to the log analytics workspace previously created
Go to the newly deployed Sentinel workspace
On the left, scroll down and select "Content hub (Preview)" under Content Management
Search for Palo Alto in the search bar
Select "Palo Alto Networks Cortex Data Lake" or "Palo Alto Prisma"
Click Install
Click Create
Select your subscription, the resource group and the workspace previously created
Click Review+Create
See Validation failed error message :

{
  "code": "InvalidResourceLocation",
  "message": "The resource 'myWorkspaceName' already exists in location 'B' in resource group 'myResourceGroupName'. A resource with the same name cannot be created in location 'A'. Please select a new resource name."
}

Expected behavior
The template should not assume the location of the log analytics workspace, and either asks for it or checks all locations before trying to deploy, hence validation succeeds and the solution can be deployed.
Screenshots
I can add some if needed
Desktop (please complete the following information):

OS: Windows 10
Browser : Microsoft Edge
Version 96.0.1054.62 (Official build) (64-bit)

Hi @vmhenard ,
Thanks for bringing this up, I've made update to the mentioned solutions in Github (#3890) and the Azure Marketplace.
Will post here once more when the solution updates are live in the marketplace.

Hi @vmhenard ,
The two solution updates are now live in the marketplace.
Please let us know if the issue still persists.


Resource optimisation

I merged #90 by accident, so reopening a new PR.
Closes #91
Like in https://github.com/sanger-tol/readmapping/pull/82 the goal is to stop using the process_* labels and instead optimise the resource requests of every process.
I'm using the same dataset: 10 genomes of increasing size, with 1 Hi-C library and 1 PacBio library each.

Fasta size (bytes)
PacBio size (# reads)
Hi-C (# reads)

GCA_939531405.1
13,824,461
1,546,435
955,654,834

GCA_937625935.1
26,683,271
189,202
980,890,138

GCA_951394315.1
58,010,196
1,965,084
704,258,466

GCA_947172415.1
118,858,594
799,796
87,833,110

GCA_910589235.2
232,212,321
1,586,931
727,465,652

GCA_949987625.1
417,566,504
2,211,570
705,705,280

GCA_946406115.1
810,357,340
1,872,695
842,629,084

GCA_963513935.1
1,803,897,959
7,338,871
3,305,634,916

GCA_951213105.1
3,609,437,155
1,121,856
3,127,898,040

GCA_946902985.2
9,152,113,672
1,537,548
886,707,886

I found much less correlation than in the read-mapping pipeline. The only input size that I found useful was the genome size, now collected at the beginning of the pipeline and added to the meta map. There is some correlation between the number of Hi-C reads and some process runtime, but not memory usage. Since runtime estimates don't need to be very accurate (really, it's only normal/long/week that matters), I don't even pull that input size.
I am using helper functions to grow values (like the number of CPUs) in a logarithm fashion. In effect, this is to limit the increase of the number of CPUs, especially as the advantage of multi-threading tends to decrease with a higher number of threads.
Also:

I could replace the GrabFiles process with some Groovy magic, as per https://community.seqera.io/t/is-it-bad-practice-to-try-and-pluck-files-from-an-element-in-a-channel-that-is-a-directory-with-channel-manipulation/224/2 . This saves 1 LSF job.
I adjusted the GNU_SORT parameters to fix #91

In this PR, the new resource requirements make every process succeed at the first attempt. The formulas are the lowest legible-ish correlations I could find.

Metric
Before
After
Improvement

Total memory requested (GB)
3,660.0
997.1
÷3.7

Memory efficiency (used/requested, %)
21.2
78.0

Total memory reservation (GB-hours)
2,792.4
2,405.4
÷1.2

Memory reservation efficiency (used/requested, %)
86.0
89.4

Total CPUs requested (n)
610.0
510.0
÷1.2

CPU efficiency (used/requested, %)
47.6
60.5

Total CPU reservation (CPU-hours)
465.4
332.0
÷1.4

CPU reservation efficiency (used/requested, %)
62.0
86.3

Job failures (%)
0.5
0.0

Detailed charts showing the memory/CPU/time used/requested for every process: before (PDF), after (PDF)
If we want to tolerate processes failing at the first attempt, being resubmitted once or twice to finally complete, I'm sure some requirements may be lowered even more. We would have to make sure that the resources wasted on those first attempts doesn't outweigh the savings we'll make on other processes. Something to investigate later...

PR checklist

[ ] This comment contains a description of changes (with reason).
[ ] If you've fixed a bug or added code that should be tested, add tests!
[ ] If you've added a new tool - have you followed the pipeline conventions in the contribution docs
[ ] Make sure your code lints (nf-core lint).
[ ] Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
[ ] Usage Documentation in docs/usage.md is updated.
[ ] Output Documentation in docs/output.md is updated.
[ ] CHANGELOG.md is updated.
[ ] README.md is updated (including new tool citations and authors/contributors).

Now that I've generated all the charts, I realise that some resource requirements are actually too low ! I should be asking 150 MB for MultiQC, not 50 MB. I guess it worked because the jobs are too fast for MEMLIMIT to have time kill.
Some COOLER_ZOOMIFY processes also take more than the 12 GB I'm requesting. The processes take 10 min, so I thought MEMLIMIT would kick in ? Anyway, I'll sort all those things out in another commit

I've made the few changes I mentioned in https://github.com/sanger-tol/genomenote/pull/92/#issuecomment-1816628012 : just updated up / down some requirements. I reran the pipeline on all species and it worked fine.
I also merged the dev branch in to solve the conflict coming from #93
@BethYates : this PR just needs an approval and then I can merge it


README.md doesnt say how to reset lagrange

i deleted the app image (downloaded from releases) and redownloaded it and the settinsg and tabs before are still there, please say in README.md where the data/settings for lagrange are located so i can delete it and start fresh

Sure, I'll add a note about that.
The user files are in ~/.config/lagrange, or if you've customized your XDG directories, wherever the XDG_CONFIG_HOME is.


Cloud Deployment Question

Hi,
What exactly happens when I create the deployment?
My app goes to the cloud from my C derive and by pressing reset button sphere downloads the app from cloud?
azsphere device-group deployment create --devicegroupname "Field Test" --productname "XXXXX" --filepath "C:/dev/Azure/Sphere/samples/Samples/AzureIoT/out/ARM-Release/AzureIoT.imagepackage"
When I change something in my application and rebuild it in visual studio, the app running in Sphere updates automatically via cloud deployment?
Or I should execute above command and press reset button again?
Thanks

@mesgar Please help with the link of document on which above document feedback request is created.
Under this repository we want to make sure any issue is linked to a doc so we can help others that may be having the same issue.

I went ahead and redeployed my application and I had to press reset button to download the latest version of my app.
It's quiet understandable and acceptable, you can't just download and replace a running application.
On the other hand, in real world scenarios you can't tell your users across the country to press the reset button to get the latest version.
Any suggestions?
link of document:
https://docs.microsoft.com/en-us/azure-sphere/install/qs-first-deployment

@mesgar Thanks for the link. I am checking on this issue and update you as earliest.

@mesgar,
The Azure Sphere devices check to see if they are running the correct software application(s) every time they check-in with the Azure Sphere Security Service (AS3).  This check-in happens on device startup and then every 24 hours after that.  So when you deploy a new application form your PC using the command line, you're updating the device group managed by AS3.
If it's important that your devices reset and download the application ASAP, then you would need to implement some kind of a reset mechanism controlled from the cloud.  This is easy to do with direct method calls.  If it's not time critical for your devices to get the update, then they will all get the new application within a 24 hour period, assuming they can connect to AS3.

Checking and getting the latest version on a daily basis works for me.
Thanks


How I can set limit for number of suggestions?

This is not supported. As far as I can tell, it is up to coc.nvim itself to determine how many suggestions are displayed to the end user. For example, you might have multiple language servers / coc extensions running at the same time, all of whom are providing coc.nvim completion options (coc-yank, coc-sources, etc). If you'd like to figure out how to limit the number of suggestions, I'd raise the issue in coc.nvim.


METRON-623: Management UI

Initial version of the Management UI using Angular2.  Currently it is bundled and served from the metron-rest Spring application server but can be easily decoupled and run in it's own web server.  See the metron-rest README for instructions on deploying.  This UI also uses angular-cli and more information can be found at https://github.com/angular/angular-cli.

Does this depend on the docker branch being landed first?

It does include the docker branch commits but does not directly depend on
it if I'm not mistaken.
On Wed, Dec 14, 2016 at 1:01 PM, ottobackwards<EMAIL_ADDRESS>wrote:

Does this depend on the docker branch being landed first?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/incubator-metron/pull/394#issuecomment-267123956,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AATCeNmvQ3fEfREyCGZCrRflJqkDfzQcks5rID0GgaJpZM4LJdRn
.

Can we get the merge conflicts resolved here?  Now that #393 (docker) got merged into master, this needs a bit of an update.

No problem.  This PR is also dependent on METRON-503 which is currently being updated.  Will get the merge conflicts resolved soon.

Wow, @merrimanr , there are currently 320 files changed in this PR.  I don't know how to start reviewing this.
Hopefully most or a lot of those go away when METRON-622 and METRON-503 are subtracted.  Could you please consider posting a PR against a branch that already has those committed, so we can see just the changes affected by this feature?  Alternatively you could use Apache Reviewboard to post such a diff.  Thanks.

I'm going to close this until METRON-503 gets merged.  Sorry for the distraction.


[5.3] Updating JSON returns data as "meta->area" instead of meta : {area : ...}

Laravel Version: 5.3.21
PHP Version: 7.0.12
Database Driver & Version: MySQL / 5.7.13-0ubuntu<IP_ADDRESS>

Description:
When updating JSON
    public function update(Request $request, $id)
    {
        $program = Program::findOrFail($id)->fill([
            'meta->area' => 'new value'
        ]);

        $program->save();

        return response()->json($program);
    }

Returns
    {
        ....
        meta : {area : 'old value'},
        meta->area : "new area"
        ....
    }

It is correctly updating just not replacing the value correctly upon return.
Steps To Reproduce:
    public function update(Request $request, $id)
    {
        $program = Program::findOrFail($id)->fill([
            'meta->area' => 'new value'
        ]);

        $program->save();

        return response()->json($program);
    }

Look at the json value returning .

I'm pretty sure that meta->area is the wrong syntax. Are you meaning to use dot notation? meta.area ?

https://laravel.com/docs/5.3/queries#updating-json-columns
Docs say otherwise
On Wednesday, November 2, 2016, Jordan Hoff<EMAIL_ADDRESS>wrote:

I'm pretty sure that meta->area is the wrong syntax. Are you meaning to
use dot notation? meta.area ?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/laravel/framework/issues/16213#issuecomment-257779070,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB-I7AmmhtjKfQZ5ryXX3lZbv_R9Lcseks5q6CVugaJpZM4KmWbJ
.

@lukepolo working on a fix.

Huh... TIL. Apologies.


Fixed several inconsistencies between GEMP and Scomp Link.

As discussed in issue #46 .  Also fixed misspelling of "Resistance".

Let's hold off on merging this just yet.
"Your Ship?" appears to only be a Reflections III card.
I can't find a reference to the card in the Virtual Set List.
I will ask Scott what the status of this card is.
The card linked has no Virtual Set identifier.

I checked with Scott, who checked with some people, and they all verify that Your Ship never received a virtual version update. Your Ship was only ever a part of Reflections III.
We should remove the card rather than add it to the Demo Deck set.

An updated "Your Ship" definitely exists. After looking closer, it wasn't actually a "(V)" card. Rather, it was an errata to the existing "Your Ship" to include extra text.
The PDF is here:
https://res.starwarsccg.org/virtual/errata_2019-a.pdf
Here is the announcement for it too:
https://forum.starwarsccg.org/viewtopic.php?f=871&t=72771&p=1258151&hilit=imperial+supply#p1258151
In GEMP, this was implemented as a Virtual Card though:
https://gemp.starwarsccg.org/gemp-swccg/images/cards/VirtualPremium-Light/yourship.gif

I've created a forum post here to try to get this resolved:
https://forum.starwarsccg.org/viewtopic.php?f=2&t=77426

Also, for what it's worth the "D" icon on "A Useless Gesture" is actually for "Defensive Shields", not "Demo Deck". I beleive every virtual defensive shield has gotten that icon since Set-0. You can see them in the Set-0 PDF here:
https://res.starwarsccg.org/virtual/Set-0.pdf
And also the Set-12 PDF here:
https://res.starwarsccg.org/virtual/Set-12.pdf
That's why it's a little odd to even have the Virtual Defensive Shields show up in a particular set (since none actually have the set icon).

Thanks for digging into this. I agree that set 0 makes much more sense, but I wanted to make sure since it was a difference with gemp.

I’m going to keep that discussion going on that forum thread so we can get the bottom of this. It’s a bit confusing and I suspect nobody has really cared enough until now (and 99.9% likely still don’t care! Lol. )

Okay, so I think we need to do two things (which I'm happy to do once we're in agreement).

Create a new set named "Virtual Defensive Shields" and mark all of the Defensive Shield (V) cards to be a part of this new set.

Your Ship? (V) is indeed a real card and should be moved into this set as well.

Until gemp updates to also include this new set, I'm going to treat the Virtual Defensive Shield set as if they were all included in Set 0, for purposes of legality.  This doesn't really matter right now but I'm working to be able to search in ScompLink/Comlink by gemp format and this would be necessary for that to work.

If we're in agreement, we should be able to merge this as is, then.  Once we add the new Defensive Shield Set this will get moved to that set with all the others.


Bars not updating properly on data change

I have a custom class that extends bars, class GenDispatchBars(Bars). It generates a stacked bar plot using data about various "generators" as the y data (and hours 1-24 as the x data).

I have a function that primarily updates the object's self.y to reflect only a particular generator. When the data is changed, the plot updates automatically but it seems to leave behind some residue of the previous bars. Like this:

or this:

Is there a more idiomatic way to update data than directly changing the attribute? Or is this possibly a bug?
A note: If I run the fig cell again, the residue disappears.

Thank you!!

MWE
import sys
if sys.version_info[0] < 3:
    from StringIO import StringIO
else:
    from io import StringIO

import pandas as pd

TESTDATA=StringIO(''',GenCo0,GenCo1\n0,100,0\n1,100,0\n2,100,0\n3,120,0\n4,120,0\n5,120,0\n6,150,0\n7,150,0\n8,150,0\n9,200,0\n10,200,0\n11,200,0\n12,200,100\n13,200,200\n14,200,100\n15,200,0\n16,200,0\n17,200,0\n18,150,0\n19,150,0\n20,150,0\n21,150,0\n22,100,0\n23,100,0\n''')

df = pd.read_csv(TESTDATA, index_col=0)

from bqplot import OrdinalScale, LinearScale, Bars, Figure

m = Bars(x=df.index, y=df.T, scales={'x': OrdinalScale(), 'y': LinearScale()})

# Add it to a Figure to display
fig = Figure(marks=[m])

Changing m.y does not change the plot as one would expect.

Not sure if this is the recommended way to do this, but I was able to get around this issue by doing the following.
m.y = []
m.y = np.array([100, 100, 100, 120, 120, 120, 150, 150, 150, 200, 200, 200, 200,
       200, 200, 200, 200, 200, 150, 150, 150, 150, 100, 100])

We can wait for more input from bqplot devs, but for now you can proceed using this method.

This definitely seems like a bug, we're looking into it, thanks for reporting!
It is probably due to the switch in dimension of y, as this works as expected:
m.y = np.array([[100, 100, 100, 120, 120, 120, 150, 150, 150, 200, 200, 200, 200,
                 200, 200, 200, 200, 200, 150, 150, 150, 150, 100, 100],
                [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
                 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])

@dustinmichels I checked that the fix in #509 fixes the issue. So I will close this issue. If you still see it after updating to the current master, do let me know and I will open this again.

Perfect, thank you!


License?

Hi,
Does this project have a license? I apologize if I missed it somewhere
Thanks,
Ryan

Haha, yikes, thanks for catching that. There's an MIT license in there now.


Gateway down policy rule fix. Issue #10716

[X] Redmine Issue: https://redmine.pfsense.org/issues/10716
[X] Ready for review

Fixes PBR rules creation when:

Gateway is down
Gateway in gateways group is down or disabled (force_down)

This correctly omits the rule entirely when 'Skip rules when gateway is downis selected but it does not omit the gateway from the rule in the default setting. Gateway looks like:GWGW2 = "  "But the rule still contains it:pass  in  quick  on $LAN  $GWGW2 inet proto tcp  from any to <IP_ADDRESS> tracker<PHONE_NUMBER> flags S/SA keep state  label "USER_RULE"`

But of course since the value of $GWGW2 is now '  ' that does omit it!

Gateways Group 'Trigger Level' also needs to be considered,
with the latest changes in this PR it only checks for 'Member Down' state

Updated
Gateways Group 'Trigger Level' and 'Disable Gateway Monitoring Action' check added

Tested this. Looks good. Gateway is marked down and removed from ruleset and/or rules removed as expected.


Pytorch support

Hello!
Thanks for your work. Please tell me if you have any plans to extend the capabilities of your package for Pytorch? It would be wonderful to combine pythonic-pytorch style and your tensor annotations.
Best wishes, Mark.

For sure, this is definitely something we'd like to do. I'm not sure of the timeframe, though. In the meantime, we'd be very grateful for PRs!


SQL Connection - Database Dashboard Widgets not Working

After updating to the latest release of ADS, the database dashboard widgets no longer populate with data.  This includes third party extensions such as sp_WhoIsActive.  This issue also occurred for others on my team.

@janszedc the current ADS version is 1.25.2.  Have you tried this scenario with the latest version (screenshot shows version 1.15.0)?

For some reason I think my ADS keeps downgrading itself. The notification
to check for updates continually pops up saying new updates are available
no matter how many times I run it. Eventually it will say no updates are
available, but when I restart it pops back up.
I wonder if some group policy or security firewall is blocking the install
and silently continuing? I will have to look tomorrow.
On Mon, Feb 1, 2021, 3:54 PM Karl Burtram<EMAIL_ADDRESS>wrote:

@janszedc https://github.com/janszedc the current ADS version is
1.25.2. Have you tried this scenario with the latest version (screenshot
shows version 1.15.0)?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/azuredatastudio/issues/14126#issuecomment-771150645,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AL4YTKVH4X6YOLAGJRITEIDS44IHHANCNFSM4W4YBIPQ
.

@janszedc that's strange behavior with the upgrade.  It's not a good long-term solution, but you can find the direct links to download the latest version from the project README.

@janszedc please let us know if this issue is occurring on the latest ADS build.  If you are still experiencing auto-upgrade problems, please open a separate bug for that.  Thanks!

@kburtram This issue is still occurring in the latest build of ADS 1.27.0 as well as version 1.26.1.
The auto-updating is still a problem, but I have not opened a separate issue for that.  I was able to temporarily work around that by doing a fresh install like you recommended.


Add support for verbatim label in collecting event batch uploader

Related: https://github.com/SpeciesFileGroup/taxonworks/issues/1816


Configure graph appearance in settings

Currently the graph's appearence is hardcoded, change this

Closed by b3fcb8c55a28dff33777f49f145bd3eb6cf34223


Current version is incompatible with click 8.1.0

What was wrong?
The version of black used in specs is incompatible with the latest version of click (see issue)
Since the version of click is not locked-in within the spec setup.cfg, CI on Github Actions fails.
How was it fixed?
We have two options here.

Start using the newer version of black. However, there seem to be some rule changes in black which will lead to errors on many spec files (about 40 of them)
Lock-in the version of click in setup.cfg. This has been implemented in the PR

Cute Animal Picture

Closing in favour of #486


Added TCP_NODELAY and TCP_KEEPALIVE on default connection

Thanks, but my thought was actually to just change the default in the socket creation routine, not make a public option to configure it. You can already configure the socket via the callback, I just want the default to be what is expected.
Can you make a PR that just does this?

I have a few minutes today, I'm just going to implement what I think is the right thing.

You mean directly in MySQLSocketPhobos class? I just thought that wouldn't be flexible enough if someone need to create a socket if one just want to disable a feature, for whatever reason.

Just hang on about 10 minutes, I'm almost done.

Take a look, this is what I want to do (still working on changelog, then will set up a PR): https://github.com/schveiguy/mysql-native/tree/tcpnodelay

OK, then I will close this


Import engine as submodule

Requires you to rebuild the nuget modules as the pathing is a little weird, but otherwise compiles.

test

The way you formatted this submodule link only allowed SSH authentication to even pull the repository, I reverted and changed it to one that would allow standard user/password authentication.

Oh I didn't mean to mention this, guess I can't do # 1


First test for setting keyboard focus to launched tools

As soon as we can emulate keyboard input we can imporve

Improved a lot with own text morph and on hover focus


Doesn't seem to match for lower case strings. IE matches President but not president.

https://github.com/fluquid/find_job_titles/blob/a8a64e81f74bd217f14857b817faf4ebffc579c0/HISTORY.rst mentions that there is ignore_case parameter in FinderPyaho constructor. But it doesn't seem to be the case.
ignore_case is defined on FinderAcaro constructor but doesn't seem to work.


Global encoder

Hi. I can't find the implementation of the global encoder within the source code for MORAM.
Could you point me to it if I am missing it?

Thanks for your question. The global encoder in our code implementation is referred to as mix_gat. It is worth noting that while the name may appear mismatched, the implementation itself is accurate.

Thanks for your response. That was very helpful.


Support for Cloudfront realtime logs

Cloudfront does not support delivering standard access logs to an S3 bucket in all regions. I am utilising cloudfront realtime logs to work around this limitation
This feature request is for these templates to support an optional parameter "Existing cloudfront realtime logs table". This would also require another optional parameter for an "Existing Glue Database"
Realtime logs have a configurable set of fields to record, some of which are sensitive so we are not logging them. This is why the stack should support an existing table, rather than trying to create a new one
I've already started a fork for these changes. Extra changes I am making at the same time is removal of the custom lambdas / events to manage the partitions, instead using partition projection.
Are these changes something that this project would be interested in? I'm currently attempting to make these changes with the intention of submitting a pull request, but if this project is not interested it will make my changes simpler as I can simply neuter parts of the code base in my fork.

Hi bgardner-noggin, thank you for your suggestion. Due to the limitation of CloudFront, WAF and S3 lambda event, the solution must be deployed in us-east-1 and S3 bucket must exist in the same region if CloudFront endpoint is chosen. Is your approach using CloudFront realtime logs a workaround for this limitation so that the solution can support CloudFront logs in S3 in other regions? Currently CloudFront realtime logs is not on our roadmap yet and we can't promise anything, but feel free to share your PR or simply your repo if you want to make simpler changes as PR may take too long.

Manually deploying a related stack in us-east-1 is not feasible for us as as we initiate this stack as a substack, and the outputs are also referenced elsewhere in our stacks.
I'd already worked around the WAFv2::WebACL and WAFv2::IPSet limitations of us-east-1 by using a custom resource to build these (which also eliminates the Custom::Timer as boto3 has native retries on throttling)
S3 lambda event should not be a problem as my buckets are in the same region as the lambda
Once I am finished I will generate a pull request from my forked repo


GUI support

We need to be able to automatically tweak the GUI according to the computed viewport.
All GUI components are managed from *.lua and *.sjson files in the Content directory.
First, we should identify all that needs to be updated. Some examples:

Sizes: Width/Height
Positioning: X/Y
Relative offsets: OffsetX/OffsetY
Functions that compute values derived from any of the above

Once done, we can think about how to implement. I currently intend to do two passes, with the first one implementing GUI support for Hor+ scaling only (best bang for the buck).
Let's go!
Identified
↔ is compatible with an Hor+-only implementation.
↔↕ is only required for pixel-based scaling.
Scripts
UIData.lua

Screen* variables

UIScripts.lua

📈 ScreenAnchors*: health bar position

I've been trying to find out how the values are calculated exactly for some of the GUI aspects, haven't managed to get to that just yet. I will gradually update this comment with my findings. Looking at some of the values I've determined via testing & lots of Photoshop centering at 3440x1440 (this is after applying Hephaistos):
InGameUI.sjson
Centering subtitles
Since the default X offset for the subtitles is 960 I thought the formula would logically be width / 2, because 1920 / 2 == 960.  Surprisingly putting in 1720 didn't seem to work at all and put the subtitles much more to the right side of the screen. I've found the following values to be pretty much dead center.
SubtitlesABacking
X = 1288 (previously 960)
SubtitlesBBacking
X = 1288 (previously 960).

Thanks , this will help a lot!

I'm currently writing an application to replace all this + some values in the lua scripts. Seems the general formula is just 1290 + (value - 960) for all of the X positions.

I'm not good with Python so I did my own thing in C# for now, maybe we can merge stuff later on. The only thing I haven't been able to locate is the Vignette / color tone effect that's applied to the game, do you have any idea where that is by chance?
https://github.com/tr4wzified/HadesUltrawideGUI

Do you mean the dark overlay? I don't know yet, but I suspect this actually is not an effect but a simple PNG overlay with transparency. We probably will have to rescale it or get rid of it.

Yes, still struggling with modifying the dark overlay together with the visuals on the borders of the screen when close to death. I can disable the overlay by commenting out a line or the method altogether in RoomManager.lua, but I can't find where it's actually taking the image / effect from to try and replace it with an ultrawide variant.
function CreateVignette()
	if not GetConfigOptionValue({ Name = "DrawBloom" }) then
		return
	end
	ScreenAnchors.Vignette = CreateScreenObstacle({ Name = "BlankObstacle", X = ScreenCenterX, Y = ScreenCenterY, Group = "Vignette" })
	SetAnimation({ Name = "VignetteOverlay", DestinationId = ScreenAnchors.Vignette })
end

I can center the vignette by adding + 360 to ScreenCenterX, so that's not right either but I can't find where that variable is being defined, and since there's no width parameter I think the secret lies in this CreateScreenObstacle method which I cannot find either. Not very familiar with Lua - could it be calling to dlls or something like that? RoomManager.lua seems to have a lot of Imports & 'Using'-statements at the top of the file.
Also still unsure how to center the conversation/boon UI, might be tied to that ScreenCenterX variable too. Got the rest working quite nicely in the meantime so far.
Main menu + play game menus centered except for stuff that should be left/right aligned like the version number.

Oh and I should mention that I did find a ScreenCenterX being defined in UIData.lua, but this does not seem to be the same variable that's used pretty much everywhere in RoomManager.lua - modifying that has no effect on the location of the vignette.

Great job, looks awesome, you're making very good progress! 👏🏻

Not very familiar with Lua - could it be calling to dlls or something like that? RoomManager.lua seems to have a lot of Imports & 'Using'-statements at the top of the file.

I'm not a Lua expert but from what I gathered while decompiling: the game engine embeds a Lua processor engine that parses and interprets the Lua. The embedded processor is responsible for handling eveyrything, and is indeed capable of calling native functions if necessary.
Now, as for your questions, AFAIU:

The Import statements in RoomManager.lua simply ask the embedded Lua processor to load code from another Lua file, so that as to use its functions / variables.
The Using statements seem to be loading static resources from sjson files (those are located in the Content/Game directory).

For example, here is the BlankObstacle resource mentioned in your code snippet, set up in 1_DevObstacles.sjson:
    {
      Name = "BlankObstacle"
      InheritFrom = "InvisibleTarget"
      DisplayInEditor = true
      Thing =
      {
        EditorOutlineDrawBounds = false
        Graphic = "Blank"
        Invisible = false
      }
    }

It itself references another resource Blank which seems to be setup in 1_DevAnimations.sjson:
    {
      Name = "Blank"
      FilePath = "Dev\blank_invisible"
      Type = "ConstantNoUpdate"
      EndFrame = 1
      NumFrames = 1
      StartFrame = 1
    }

And this one references a filepath, probably coming from one of the packaged archive files. The image assets etc. probably live in there, and if we are to edit them we'll need to be able to unpack the archives (I think our friends and their tools over at https://github.com/SGG-Modding will be able to help for that).
As for CreateScreenObstacle, we can find the function defined line 2344 in UIScripts.lua:
function CreateScreenObstacle( params )

	if params.Group == nil then
		params.Group = "Events"
	end
	if params.X == nil and params.Y == nil then
		params.X = ScreenCenterX
		params.Y = ScreenCenterY
	end
	if params.Scale == nil then
		params.Scale = 1.0
	end

	local obstacleId = SpawnObstacle({ Name = params.Name, Group = params.Group, OffsetX = params.X, OffsetY = params.Y, UseScreenLocation = true, SortById = true, TriggerOnSpawn = false })
	if params.Scale ~= 1.0 then
		SetScale({ Id = obstacleId, Fraction = params.Scale })
	end

	return obstacleId

end

It itself references another function SpawnObstacle, but this one does not seem to be defined anywhere in Lua.
And indeed, it is a native function from the Engine DLL:

If it comes to that we can always patch the SpawnObstacle function, but at a quick glance I believe it should not be necessary and we should be able to either solve it by modifying the Lua or by modifying the assets in the archives.

Great find! With the scaling stuff that's more public now it should be much easier fixing stuff up. I still can't find the location of this ScreenCenterX variable but just modifying the CreateScreenObstacle variable . Only the vignette & other screen edge effects left, but I think we can use the built-in scaling methods to get that fixed. Will report back later when I get to it.

I've pretty much finished the work and published it, it's also using Hephaistos under the hood to automate all of the steps :)
Link: https://www.nexusmods.com/hades/mods/106

Good job :) I'll work on backporting the changes to Hephaistos directly when I have time so that we can have a single tool.

I've released v0.2.0 with initial support for GUI patching (SJSON + Lua). This first version of GUI patching patches both SJSON and Lua files:

Automatic patching of SJSON X, Y, etc. properties based on their original values and the new viewport.
Registering a custom Lua mod for recalculating various hardcoded values and repositioning dynamic calls.

It works fine enough, though there are still some rough edges to tackle -- I did not get all the vignette/overlays right yet, e.g. when transitioning between scenes.
In a future version I'd also move away from automatic SJSON patching to have a bit more control about how it detects what to patch, but for now the automatic patcher will do.

Been working on progressively fixing stuff, most overlays/vignettes are good, I'm now working my way through all screens/shops. v0.6.0 is out if you'd like to have a try. Not everything's fixed yet but it's looking way better already :)

I think I've fixed almost everything in v0.8.0 😄

There are a bunch of small effects left to fix but we're already in a excellent state, with all screens done and all major effects :)
I'm adding a list of stuff left in OP.

Excellent progress mate!
Here's some findings for boon/hammer selection where I believe they should appear directly on top of Zag's hand.

Fixed that and a bunch of other stuff in v0.9.0 :)

I'm pretty sure I've fixed almost everything in latest version v0.11.0 🎉

Was annoying but finally got the victory screen centred without impact on the advanced trait UI (B button):

OK I think we should be good for a v1.0.0 release, and also publishing on Nexus Mods.
I will move the Passing through door Styx -> [Redacted] white flash clipping thing to a separate issue because fixing it is very involved. Funnily enough it is the only transition that specifies a LoadBackgroundColor (all others default to the engine-side black), so I'm actually pondering if it's even worth investing time to look into it considering it's such a small issue.


Volante on .NET 4.0

Hi, I have a project with an .NET4.0 target platform, when I compile I get this error:

This has been fixed recently.


'nvm is not compatible with the npm config "prefix" option' message in every shell

Every time I open a shell (I have default aliased to node), the following message is printed:
nvm is not compatible with the npm config "prefix" option: currently set to "/home/josh/.local/share/nvm/versions/node/v4.5.0"
Run `nvm use --delete-prefix v4.5.0 --silent` to unset it.

I don't understand why this appears, as I don't have node installed globally and doing sudo find / -name '*npmrc*' doesn't return anything except the man pages for npm.
After doing nvm use --delete-prefix v4.5.0 --silent, npm config ls -l | grep prefix prints prefix = "/home/josh/.local/share/nvm/versions/node/v4.5.0", so maybe it's something to do with that?
I installed nvm with curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.31.7/install.sh | NVM_DIR=.local/share/nvm bash, then ran nvm install --lts and nvm alias default node.

What does nvm debug print out? It looks like your NVM_DIR is not $HOME, but is inside $HOME/.local/share/nvm?

$ nvm debug
nvm --version: v0.31.7
$SHELL: /bin/bash
$HOME: /home/josh
$NVM_DIR: '.local/share/nvm'
$PREFIX: ''
$NPM_CONFIG_PREFIX: ''
nvm current: none
which node:
which iojs:
which npm:
npm config get prefix: The program 'npm' is currently not installed. You can install it by typing:
sudo apt install npm
npm root -g: The program 'npm' is currently not installed. You can install it by typing:
sudo apt install npm

I set my $NVM_DIR to .local/share/nvm on install.
I just tried reinstalling without specifying $NVM_DIR, and it seems to work - I got a message from nvm install --lts saying nvm_ensure_default_set: a version is required, but I ran nvm alias default node and it all appears to work.
It actually looks like the problem occurs if $NVM_DIR is not an absolute path; setting it to $HOME/.local/share/nvm works fine.

yes, that's absolutely correct - if it's a relative path then nvm would work or not based on your current directory.


Not clear about the function almost_equal()

Many tests use the same function almost_equal(), but when I ran some tests, it seemed that there was some misunderstanding about the function.
eg: test_src/cpp/hierarchical_parallelism/atomic-float/target__parallel__for.cpp
#include <iostream>
#include <cstdlib>
#include <cmath>
bool almost_equal(float x, float gold, float tol) {
  return gold * (1-tol) <= x && x <= gold * (1 + tol);
}
void test_target__parallel__for() {
  const int N0 { 32768 };
  const float expected_value { N0 };
  float counter_N0{};
  #pragma omp target map(tofrom: counter_N0)
  #pragma omp parallel
  #pragma omp for
  for (int i0 = 0 ; i0 < N0 ; i0++ )
  {
    #pragma omp atomic update
    counter_N0 = counter_N0 + 1. ;
  }
  if (!almost_equal(counter_N0, expected_value, 0.1)) {
    std::cerr << "Expected: " << expected_value << " Got: " << counter_N0 << std::endl;
    std::exit(112);
  }
}
int main()
{
    test_target__parallel__for();
}

The function call during running is:
line 40 in test_target__parallel__for(): almost_equal(32768., 32768, 0.1)
line 26 in almost_equal(): return 32768.*0.9 <= 32768 && 32768 <= 32768.*1.1;
The actual comparison is:
return 29491.2 <= 32768 && 32768 <= 36044.8;
But the comparison I understand should be:
return 32767.9 <= 32768 && 32768 <= 32768.1;
Is there an error in my understanding of the program?

almost_equal is a function which says "return true if x is within tol*100% of gold". In this case it's if x is within 10% of gold, so if it's greater than gold*0.9 and less than gold*1.1. Based on the numbers you used, it looks like you were thinking it was gold-tol and gold+tol, not gold*tol.
So you are right that in this case it returns return 32768.*0.9 <= 32768 && 32768 <= 32768.*1.1;. If you carry out the multiplication you get 32768.*0.9=29491.2 and 32768.*1.1=36044.8, just like the program shows.
`

Yes, It's  almost_equal use a relative tolerance and not an absolute one. I guess I should have named tol rel_tol.  PR are welcome to fix it :)
If you want to have more information about how to compare numbers, this Python PEP (https://docs.python.org/3/library/math.html#math.isclose and https://www.python.org/dev/peps/pep-0485/) is useful.

My understanding is that for this test, counter_N0 and expected_value should be equal. Just considering the comparison of float and int, the almost_equal function is introduced.
But in this test, is the allowable error range of this test too large?  Here is the result ‘Expected: 32768 Got: 32761.6’ from a similar test.
According to the logic that I understand, the result of the test should be failed, but according to the error range of 10%, the result of the test is pass.
Will this hide some problems?

 I choose 10% totally arbitrary because I don't know any formula that can give an upper bound to the floating-point errors.
If indeed we see  Here is the result ‘Expected: 32768 Got: 32761.6, and we are sure that it's due to a bug and not a floating-point issue, It will be better to lower the tolerance to 5% for example.
Sadly, I don't know how to confirm that it's due to a bug... and not floating-point arithmetic. I will ask some people

Would it be better to use the 'almost_equal' method of 'https://github.com/TApplencourt/OvO/blob/master/test_src/cpp/mathematical_function/cpp11/atan2_double_double_double.cpp' uniformly?
bool almost_equal(double x, double y, int ulp) {      return std::fabs(x-y) <= std::numeric_limits<double>::epsilon() * std::fabs(x+y) * ulp ||  std::fabs(x-y) < std::numeric_limits<double>::min(); }
I can reproduce this issue when testing on host using IFX. Repeat the test with the 'fortran/hierarchical_parallelism/atomic-real-intermediate_result/target__teams__distribute__parallel__simd.F90' generated by 'ovo.sh gen tiers 2' and find that the part of pass is not really equal.
bash-4.4# for i in {1..10};do ./target__teams__distribute__parallel__simd.exe;echo Times $i ret: $? ;done  Expected       33124 Got   37387.00 112 Times 1 ret: 112  Expected       33124 Got   37281.00 112 Times 2 ret: 112  Expected       33124 Got   35921.00 Times 3 ret: 0  Expected       33124 Got   36045.00 Times 4 ret: 0  Expected       33124 Got   35627.00 Times 5 ret: 0  Expected       33124 Got   37074.00 112 Times 6 ret: 112  Expected       33124 Got   34589.00 Times 7 ret: 0  Expected       33124 Got   34627.00 Times 8 ret: 0  Expected       33124 Got   36868.00 112 Times 9 ret: 112  Expected       33124 Got   35281.00 Times 10 ret: 0

You are right. The current behavior is bad.
I will switch to a more robust version where it will check for both relative and absolute tolerance. This should fix those problems. 10% seems true too large. We can do 5% or 10ULP.
The fundamental problem is that I have no idea what tolerance should be. Do you have any recommendations? The atomic are performed in random order, hence the result at the end will fluctuate due to the floating point arithmetic. The question is how much is acceptable.
Talking to people, the gold standard seems to be to take an arbitrary tolerance as stick with it... We can do 5% or 10ULP (if both failed the test fail). Do you think it will work for you?

Ok, I fixed it in with 18773cecb9e5192c31139cb467d51ab7afed72ba  Can you pull, and try again?

Let's said that fixed it :). Please reopen if that was not the case.
I wrote this page to explain the new reasoning.
Thanks again for opening the issue, it was indeed a bug so thanks :)

Closed due to inactivity


Allow show/hide on grid columns

Users should be able to show or hide columns on the query grid.

Added in 6e7a005afd13f8c22dbaeb45e97acddef97bdd80


Adding vignette

This PR adds a vignette demonstrating the base functionality of matplotcheck, addressing issue #122.

This is branched from the branch in PR #128. Until that PR is merged with master and master is merged with this branch, this PR shouldn't be merged.

@lwasser This should be ready to merge. Codecov is failing because there I deleted a single (covered) line in base.py, and that technically lowered the coverage.

looks good @ryla5068 !! thank you!!


V2 - Contributor Console ICLA - User is still gated when user is logged in to SSO and then signs ICLA as individual contributor

Prerequisite

Create a CLA group with CCLA enabled
Add github repository for the CLA group
Generate template

Steps to Follow

Login to Github as corporate contributor and also make sure that user is logged into SSO in the same browser session
Navigate to the repository used in prerequisite
Create PR
CLA check fails
Click on CLA NOT SIGNED
Contributor is redirected to Contributor Console
Click on Proceed as a Individual Contributor
Docusign is opened with SSO logged in user details
Complete the docusign process
Observed that user is still gated even after signing ICLA. Looks like the issue is because of the SSO login

Worked with @makkalot on this issue today. During a troubleshooting session, it was observed that:

In GitHub, the user is gated.
User then executes the ICLA flow.
User is then redirected back to GitHub.
User observed that he/she is still gated by GitHub.
Sometime later, the user refreshes the GitHub page and found the user is now passing the gating.

Reviewed/Observed the Docusign EasyCLA callback logs and found no issue associated with the ICLA flow.
Requested @makkalot to review the CloudWatch logs to determine which API calls/flows/logic is taking so long to execute.  Can we optimize without introducing python concurrency?
The Docusign flow callback accomplishes the following:

Decode the callback message (includes envelope ID and signature ID)
Loads the signature record by ID
Determines if the callback indicates "completed status", if so, sets the assigned=True flag (which triggers to the golang dynamodb signature updated event which is handled by the golang lambda)
If ICLA, loads the user record by user id (from the signature_reference_id value)

Cleans up the meta data (not sure what this does...
Sends a copy of the signed document to the user
Saves a copy of the signed PDF document to S3
Loads the CLA group and creates a new event log entry indicating the user signed the document
Invokes update change request (github or gerrit)

fetches the GH pull request details
loads the commit author(s) information
loads the CLA repo record, github org record, CLA Group record (again)
Loops over the commuters - determines if commiters are signed or missing CLA authorization
Updates the PR comments (if required)
Updates the PR status (if required)

Did some digging on this one
Some timing on how long it takes :
2020-09-21T18:51:05.627+03:00	START RequestId: d14588fa-afa7-49ce-9585-6cb0a2524d8c

REPORT RequestId: d14588fa-afa7-49ce-9585-6cb0a2524d8c	**Duration: 4158.30 ms**	Billed Duration: 4200 ms	Memory Size: 1024 MB	Max Memory Used: 179 MB

XRAY TraceId: 1-5f68cbe9-0867c6a606814e2c00c000ba	SegmentId: 2192f0191a028d99	Sampled: true

Duration: 4158.30 ms
When I looked in more detail on what each step took (this is for post_individual_signed):

we fetch the document from docusign : ~ 1 sec
sending email to the contributor : ~ 200ms
we upload it to s3 : ~ 500ms
then we connect to github and we do lots of checks before we update the status ~ 2.5 sec

Maybe we can do some of those as separate lambda in the background ? Also not sure why we do so many calls to github in order to update the status. Since I don't know the history of it can't really comment. @dealako let me know what you think ?

Verified that SSO details are not auto filled in Docusign and after docusign process is complete status of CLA check is passed.


code ambiguity

Hi, thank you for your great work!
https://github.com/princeton-vl/DeepV2D/blob/master/deepv2d/modules/motion.py#L318
It looks this line needs intrinsics_pred =  or intrinsics =  .
Do I misunderstand something?

Hi, you are correct. If you were to train a model which also estimates camera focal lengths you should rewrite
intrinsics_pred  = intrinsics_vec_to_matrix(self.intrinsics_history[i])
and also use is_calibrated=False in the config file
However, for the experiments in our paper we assume a calibrated camera - which was why the code was not using the predicted intrinsics when computing the loss.

I understand the point!
Thank you!


Support other kernels besides python - just for listing

Also add telemetry to check if a non-python kernel is selected.

Work done to spike on this (see epic). Unassigning as we are not doing this for real right yet.

Closing, as this has been completed


A∩B∩(!C) highlight

I want to start off by saying thank you for creating such an awesome library! I am looking to create visualizations that will allow me to light up A∩B∩(!C), like that shown in the screenshot below.

Any chance this is already in the works? Thanks again!

You're the second person to ask for this - https://github.com/benfred/venn.js/issues/23 has the original request.
Its a little tricky to extend the existing approach because the areas might not be contiguous (like in that issue).
I've wondered about using  SVG clipping to do this: like define the path as it is now and clip the other circles from it like this . I think that might be the easiest way to do it, but I haven't had time to try =(


Legacy signer fallback

[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[X] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Pull Request Attributes
Please check any characteristics that apply to this pull request.
[ ] Does not have any changes that may affect Istio users.

@shankgan: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
ca574b297492b95b70b0d99c9753ce4844480101
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@shankgan: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
ca574b297492b95b70b0d99c9753ce4844480101
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@shankgan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
ca574b297492b95b70b0d99c9753ce4844480101
link
/test release-notes_istio

unit-tests_istio
ca574b297492b95b70b0d99c9753ce4844480101
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@shankgan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
ca574b297492b95b70b0d99c9753ce4844480101
link
/test release-notes_istio

unit-tests_istio
ca574b297492b95b70b0d99c9753ce4844480101
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@shankgan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio
ca574b297492b95b70b0d99c9753ce4844480101
link
/test unit-tests_istio

release-notes_istio
1319107e5fc78ee511156e4e4826f14ead79ed0b
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@shankgan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio
ca574b297492b95b70b0d99c9753ce4844480101
link
/test unit-tests_istio

release-notes_istio
1319107e5fc78ee511156e4e4826f14ead79ed0b
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@shankgan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio
ca574b297492b95b70b0d99c9753ce4844480101
link
/test unit-tests_istio

release-notes_istio
1319107e5fc78ee511156e4e4826f14ead79ed0b
link
/test release-notes_istio

lint_istio
1319107e5fc78ee511156e4e4826f14ead79ed0b
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@shankgan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio
ca574b297492b95b70b0d99c9753ce4844480101
link
/test unit-tests_istio

release-notes_istio
1319107e5fc78ee511156e4e4826f14ead79ed0b
link
/test release-notes_istio

lint_istio
1319107e5fc78ee511156e4e4826f14ead79ed0b
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@shankgan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
1319107e5fc78ee511156e4e4826f14ead79ed0b
link
/test lint_istio

release-notes_istio
5e478e5ee935a7b5e54d149641b23472bed7e029
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@shankgan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
1319107e5fc78ee511156e4e4826f14ead79ed0b
link
/test lint_istio

release-notes_istio
5e478e5ee935a7b5e54d149641b23472bed7e029
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@shankgan: PR needs rebase.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.


Publish injectDependencies to npm

I think it's a good pattern deserving to be published.

Thanks for your kind words :)

Will you publish it?

Yep, just didn't have enough time recently, but that's on my list.

I suppose we should remove "dynamic" dependencies. Validate should be able to use current app state.

@grabbou feel free to reopen


ENH: Multiclass AUROC, metrics test, and max pooling in DeepMIL

This PR adds to DeepMIL:

AUROC as a multiclass metric
Tests for multiclass metrics
Max pooling layer

Good work adding multi-class support! My main feedback is that the DeepMILModule and related tests could be simpler if we ensured our labels and predictions tensors had consistent shapes across binary and multi-class cases (limiting the number of if n_classes > 1: ...). My suggestion is labels should always be (N,) and predictions (logits or probabilities) should be (N,) for binary and (N, C) for multi-class, which is consistent with Torchmetrics' expected input shapes.


Add PTL responsibilities

Closes #33

I don't know if I have voting rights, but I am +1.

@swill Right now all community members have voting rights, since we haven't really established that hierarchy for docs yet.
And also: Thank you!

+1

@vmbrasseur I know this is not the right place to discuss this, but I am curious.  I am not sure what the gating mechanism is for being a "community member".  I am obviously active in the community and am involved in many mailing lists, but I am not sure I am officially a community member per say.  If I review the details here, https://tungstenfabric.github.io/website#becoming-a-contributor, my organisation currently does not have a CCLA signed (which I will fix right now).  Should that be the gating factor?
I personally find it difficult to understand what the process is for applying and being recognised as a "community member".  The irony in this is that I have commit access to be able to merge this request, but I still have outstanding questions around the process.  I will take care of the apparent lack of CloudOps not having a signed CCLA today, but I think that illustrates a bit that we are missing some formal process in this area currently.

@swill This is definitely not the place to discuss that. This is a question for the Community Committee, not for a comment on an unrelated issue. ;-)
I recommend first reading through the Community Governance doc, then adding this to the agenda for this week's TSC call, and maybe creating a doc JIRA ticket (select the Documentation component when creating a ticket) if you find we need to get more specific in the community governance doc.

@vmbrasseur I think you are all set to merge this now.  You have 2x :+1: votes.  I have not been able to find guidelines for the number of votes required to merge in this context (given this is not managed through gerrit, yet).  I know you have the ability to commit, so you should be set to merge now.
The reason I brought it up here is because I am trying to unblock you, but I am also actively trying to understand the contributor process and how the account linking and voting details are supposed to be implemented for contribution in this case.   I will formally raise this at the TSC once I get through trying to piece together the different documents and process details which have been provided.

I get why you're bringing it up, and I do appreciate it and agree that this needs to get sorted out. But an unrelated issue on a repo that almost no one looks at right now isn't a good place to have the conversation. It's a matter for the CC to sort out, not the Docs team.


[BD-46] feat: algolia search

Algolia search
JIRA: https://openedx.atlassian.net/browse/PAR-795

Thanks for the pull request, @monteri! I've created BLENDED-1210 to keep track of it in Jira.   More details are on the BD-46 project page.
This is currently a draft pull request. When it is ready for our review and all tests are green, click "Ready for Review", or remove "WIP" from the title, as appropriate.

:tada: This PR is included in version 19.25.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


MSbuild error

Error MSB3073: The command "mstest /testcontainer:"C:\Users\gautams\Documents\Visual Studio 2013\Projects\CounterStrike\SinglePlayerTest\SinglePlayerTest.dll"" exited with code 1.

Closing because the simplest explanation in isolation for that message is that there's a test failure. If you still believe this is an MSBuild bug, please explain why and provide more context.


go mod and replace directive does not work well when consumed as library

when we consume om in our own go library above version 0.51.0
go mod will fail. because it won't inherent the replace directive.
which then result in a compile error due to a missing dependency
see line: https://github.com/pivotal-cf/om/blob/master/go.mod#L44
i think this can be solved by
removing the replace directive and use the folowing instead
 go mod edit https://github.com/graymeta/stow@TheCommitHasYouWantToUse
refrence: https://github.com/golang/go/issues/30354

I don't believe that would help at all. As upstream dependency's go.mod files don't appear to be honored at all.

Why are you inheriting om as a dependency, too? At the moment we make no library guarantees about the source code. Just the CLI.

we are building an easy way to configure opsman+tiles
https://github.com/starkandwayne/om-configurator

It would really help you (and other om consumers out!) if y'all could bug the stow team by giving a shout out to this issue (https://github.com/graymeta/stow/issues/197). If we can get the associated PR accepted, this should solve your problem. As we don't seem to be able to get around the go.mod bug (that they don't see as a bug), this seems like a good solution)

they have merged it.
can you update you go.mod as well?

we can. We are in the middle of fixing the CI at the moment, so feel free to submit a PR, or we should be able to get to it either today, tomorrow, or, at worst, sometime early next week.

fixed in https://github.com/pivotal-cf/om/commit/be5a7f895fd6e45e0941a102b5bdbf085b0df240#diff-37aff102a57d3d7b797f152915a6dc16


Fix download local dir edge case (remove lru_cache)

Fix https://github.com/huggingface/huggingface_hub/issues/2607.
See https://github.com/huggingface/huggingface_hub/issues/2607#issuecomment-2416255310 for explanations.
In the end I went for the solution to remove @lru_cache usage. This leads to more IO disk calls but less code complexity. Hopefully the extra cost is likely to be negligible.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks for the reviews :hugs:


rate limiting

how to add limit for api i tried with
"config.minute" : 5
but still getting error
{"config":"You need to set at least one limit: second, minute, hour, day, month, year"} ```

Maybe you forgot to wrap the config into attributes (got the same error message when I did that):
The following worked for me
{
    "apis": [
        {
            "name": "mockbin",
            "ensure": "present",
            "attributes": {
                "upstream_url": "http://mockbin.com/",
                "request_host": "mockbin.com"
            },
            "plugins": [
                {
                    "name": "rate-limiting",
                    "attributes": {
                        "config.minute": 5
                    }
                }
            ]
        }
    ]
}

You can find more about the schema from the readme#schema


[Release-v.326] Build calico/node image from AlmaLinux

This is a cherry pick of commit a5dd76e7e2   which is PR #8890 into the 3.26 branch so that release branch can build calico node  images.
CentOS Stream 8 was EoL'ed on 31 May, 2024. This change switches build base to AlmaLinux which is binary compatible with RHEL/UBI.
Description

Related issues/PRs

Todos

[ ] Tests
[ ] Documentation
[ ] Release note

Release Note

TBD

Reminder for the reviewer
Make sure that this PR has the correct labels and milestone set.
Every PR needs one docs-* label.

docs-pr-required: This change requires a change to the documentation that has not been completed yet.
docs-completed: This change has all necessary documentation completed.
docs-not-required: This change has no user-facing impact and requires no docs.

Every PR needs one release-note-* label.

release-note-required: This PR has user-facing changes. Most PRs should have this label.
release-note-not-required: This PR has no user-facing changes.

Other optional labels:

cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.
needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.

/sem-approve

/sem-approve

/sem-approve

Thank you @paulgmiller for picking these changes. Calico node build and tests are green.


How to use it as the loss function for deep learning framework(tensorflow, keras,...)

I define te function as following:
def test_loss(y_pred, y_true):
return tsm_sdtw(y_pred, y_true)
and compile it with keras model ==>  model.compile(optimizer='Adam', loss=test_loss)
However, the following error are shown:
File "D:\python35\lib\site-packages\keras\engine\training.py", line 910, in compile
sample_weight, mask)
File "D:\python35\lib\site-packages\keras\engine\training.py", line 436, in weighted
score_array = fn(y_true, y_pred)
File "D:/Q/cdads/py_code/turn_upsample.py", line 17, in test_loss
return tsm_sdtw(y_pred, y_true)
File "D:\python35\lib\site-packages\tslearn\metrics.py", line 597, in soft_dtw
return SoftDTW(SquaredEuclidean(ts1[:ts_size(ts1)], ts2[:ts_size(ts2)]), gamma=gamma).compute()
File "D:\python35\lib\site-packages\tslearn\utils.py", line 320, in ts_size
ts_ = to_time_series(ts)
File "D:\python35\lib\site-packages\tslearn\utils.py", line 84, in to_time_series
ts_out = ts_out.astype(numpy.float)
ValueError: setting an array element with a sequence.
@mblondel
Would you please give some toy examples for this.

Keras and tensorflow are not supported.

Any chance you could elaborate, why these aren't supported? I am currently trying to implement a DTW distance cost function in TensorFlow but cannot find any information about it.

An efficient implementation of soft-DTW requires custom gradients. Moreover, it needs to store temporary computations from the forward pass and reuse them during the backward pass. Here's how it is implemented in Chainer:
https://github.com/mblondel/soft-dtw/blob/master/sdtw/chainer_func.py
Back when I wrote soft-DTW, it was not possible to do the same in Tensorflow.

@mblondel Could you please provide the keras version of sdtw? I guess lots of people need the version. Chainer is a very good deep learning framework, but at least currently, it is not the main stream. thanks a lot.

I think it's possible to implement it in Tensorflow, it's just that maybe the C++ based ops are required. See https://www.tensorflow.org/guide/extend/op.

can DTW loss be implemented in pytorch, as it is also a Dynamic Graph Computation framework?

@Abhay242 there is a pytorch implementation: https://github.com/lyprince/sdtw_pytorch/blob/master/sdtw.py
@mblondel at least with tensorflow 1.15 it seems to be possible https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/custom_gradient
a tensorflow version would really be nice!


Eric Capponi customer entity

Relevant information

Checklist

[X] I've read the guidelines and understand the acceptance criteria.
[X] The value of the attribute marked as identifier will be unique and valid.
[X] I've confirmed that my entity type wasn't already defined. If it is I'm providing an explanation above.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

closed, as discussed on the help channel


Adding S3 bucket for snapshot

I am trying to add S3 bucket for taking snapshot of my elastic cluster. I am running Elasticsearch 2.3.3 and installed plugin cloud-aws. Following is syntax of my request
curl -X POST "http://XXX.XXX.XXX.XXX:9200/_snapshot/s3_repository" -d '{"type": "s3","settings":{"bucket":"bucketname","region":"ap-south-1","access_key":"accesskey","secret_key":"secretkey","endpoint":"s3.ap-south-1.amazonaws.com","compress":true}}'
it gives me following error
{"error":{"root_cause":[{"type":"repository_exception","reason":"[s3_repository] failed to create repository"}],"type":"repository_exception","reason":"[s3_repository] failed to create repository","caused_by":{"type":"creation_exception","reason":"Guice creation errors:\n\n1) Error injecting constructor, com.amazonaws.AmazonClientException: Unable to execute HTTP request: bucketname.s3.ap-south-1.amazonaws.com\n  at org.elasticsearch.repositories.s3.S3Repository.(Unknown Source)\n  while locating org.elasticsearch.repositories.s3.S3Repository\n  while locating org.elasticsearch.repositories.Repository\n\n1 error","caused_by":{"type":"amazon_client_exception","reason":"Unable to execute HTTP request: bucketname.s3.ap-south-1.amazonaws.com","caused_by":{"type":"unknown_host_exception","reason":"bucketname.s3.ap-south-1.amazonaws.com"}}}},"status":500}

This repository is now closed.
Please ask this question on discuss.elastic.co where we can give a better support.
This space is only for confirmed issues or feature requests which have been discussed on discuss.elastic.co.
Thanks!

@dadoonet thanks for quick response. Added my question on discuss.elastic.co.


Generalize result

What do you think about moving from specifically booleans to a more generalized approach like Testable?
I'd like to be able to attach reasons to errors, among other things, and booleans don't carry enough information with them for this.

Yes!
Currently only true in considered as successful value, so:
> jsc.check(jsc.forall("nat", function (n) { return "cause: " + n; }))
Failed after 1 tests and 6 shrinks. rngState: 00da3714f9c2058eb6; Counterexample: 0;  [ 0 ]

You don't see the non-true value though, but it should be easy to add, as we show exceptions (which are current workaround):
> jsc.check(jsc.forall("nat", function (n) { throw new Error("cause: " + n); }))
Failed after 1 tests and 4 shrinks. rngState: 0adc76385e20ab149f; Counterexample: 0; Exception: cause: 0

Oh, I see. So the quick way to go about this is to just wrap up that difference in a function.
function assert(b, msg) {
  if (b) {
    return b;
  } else {
    throw new Error(msg);
  }
}

But that seems really dirty.

> jsc.assert(jsc.forall("nat", function () { return "foobar"; }));
Error: Failed after 1 tests and 6 shrinks. rngState: 0073892e5dfbc4a016; Counterexample: 0; Error: foobar


MoneyField in Orchard

There is a Money data type for .NET available open source here: http://www.codeproject.com/Articles/28244/A-Money-type-for-the-CLR
We are using it in several projects where money-related operations and data storage is required and it's perfect.
My idea is that we could make a field for that in Orchard called MoneyField.
Note, that it also can manage currencies so there could be a field setting for the currency or just simply a second editor displayed for that (enum) next to the editor of the actual value (decimal).

Orchard just gets too materialistic. I'd rather want to see a LoveField or FamilyField first.

Unfortunately, MoneyField is dependency of LoveField and FamilyField. They are both extremely using Money.dll.

Case in point. :-D

Like we decided for the other fields that have been suggested, this can be done as a separate module as it's not something most sites would use.

Other examples: phone, address

+1 for MoneyField I think if Orchard CMS can be used to show prices in different currencies, is a good solution. The same for Phone or Address. I think have sufficient "business logic" to be independent.

-10,000, and  own an e-commerce module. Which is, coincidentally, where this would belong ;)

You can need a moneyfield outside e-commerce module... Think in a jobs module that needs to publish the salary to the job position

OK then, create a moneyfiled module, and have both depend on it. Solved!

I agree with Alberto, why would we use an e-commerce module only for displaying a MoneyField in a site that has no connection with any e-commerce operations at all?
Yep, I understand that if it is not in Orchard.Fields module (probably we will see it in a future version) then MoneyField need to live in a relevant module maybe in an open source module.
That's why we started to refactor a few independent and useful fields to a separated module that is going to be published soon.

That's not what I was saying. Yes, the separate module is the way to go. Cool.

Hi Barthamark, did you release the localized 'special fields' module yet? I am curious.

@Mollenhorst Money Fields is included in this module: https://github.com/Lombiq/Lombiq-Fields


Create Harvester Cloud Credential lack of "kubeconfigContent" and "annotations" in the response body

Rancher Server Setup

Rancher version: 44bb5b4 (v2.6.0-rc2)

Describe the bug

Create Harvester Cloud Credential lack of "kubeconfigContent" and "annotations" in the response body
To Reproduce

Go to "Cluster Management"
Go to "Cloud Credentials" and create Harvester Credentials
Input the name and select an imported harvester cluster
Click save

Result
"kubeconfigContent" and "annotations" are sent in the request body, but in the response body is empty.
Expected Result

"kubeconfigContent" and "annotations" in the response body
Screenshots

Additional context

Tested with Rancher v2.6.1-rc5 and harvester v0.3.0-rc1. Still not seeing annotations or Kubeconfigcontent in the response.

Confirmed with @n313893254 , this issue was out-of-date.

the Kubeconfigcontent is a secret field of the harvestercredentialConfig  and is "write-only", so you don't get the value back for security purposes.
the provisioning.cattle.io/driver: "xxx" annotation will only be used by the backed to identify the cloud provider type and will not be persisted to the /v3/cloudcredentials response. (same behaviour across all of the cloud providers e.g VMware)


Adding a slack badge

Makes the invite a bit more prominent.

Just a reminder for my self, we should customize Travis to check if the build for PR AND the diff/patch it's only docs then only do the scan code and not do the whole 40 minute build.