runtime: address space conflict on Linux/Arm64

I'm attempting to build the golang 1.5 beta2 on the Arm64 port of CentOS 7, however I appear to be running into memory issues. The initial cross-compile completes successfully on an x86_64 CentOS-7 host via the following command
GOOS=linux GOARCH=arm64 GOROOT_BOOTSTRAP=/opt/go1.4.2 ./bootstrap.bash

However when I move the resulting bootstrap tarball to the arm64 box, I immediately get a memory address space conflict no matter what go command I run.
[jperrin@apma3 go-linux-arm64-bootstrap]$ ./bin/go help
runtime: address space conflict: map(0x481fff8000) = 0x481fff0000
fatal error: runtime: address space conflict

runtime stack:
runtime.throw(0x599ec0, 0x1f)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/panic.go:527 +0x80 fp=0x3fff9b7bd60 sp=0x3fff9b7bd40
runtime.sysMap(0x481fff8000, 0x8000, 0x4820000000, 0x7b7fb8)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/mem_linux.go:147 +0x134 fp=0x3fff9b7bda0 sp=0x3fff9b7bd60
runtime.mHeap_MapBits(0x7997a0, 0x4820100000)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/mbitmap.go:141 +0xec fp=0x3fff9b7bdd0 sp=0x3fff9b7bda0
runtime.mHeap_SysAlloc(0x7997a0, 0x100000, 0x792954)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/malloc.go:424 +0x15c fp=0x3fff9b7be50 sp=0x3fff9b7bdd0
runtime.mHeap_Grow(0x7997a0, 0x8, 0x0)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/mheap.go:625 +0x70 fp=0x3fff9b7beb0 sp=0x3fff9b7be50
runtime.mHeap_AllocSpanLocked(0x7997a0, 0x1, 0x0)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/mheap.go:529 +0x6bc fp=0x3fff9b7bf10 sp=0x3fff9b7beb0
runtime.mHeap_Alloc_m(0x7997a0, 0x1, 0x15, 0x0)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/mheap.go:422 +0x14c fp=0x3fff9b7bf40 sp=0x3fff9b7bf10
runtime.mHeap_Alloc.func1()
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/mheap.go:481 +0x50 fp=0x3fff9b7bf70 sp=0x3fff9b7bf40
runtime.systemstack(0x3fff9b7bfa8)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/asm_arm64.s:251 +0xb8 fp=0x3fff9b7bf80 sp=0x3fff9b7bf70
runtime.mHeap_Alloc(0x7997a0, 0x1, 0x10000000015, 0x97880)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/mheap.go:482 +0x60 fp=0x3fff9b7bfd0 sp=0x3fff9b7bf80
runtime.mCentral_Grow(0x7a2170, 0x9322c)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/mcentral.go:205 +0xc0 fp=0x3fff9b7c040 sp=0x3fff9b7bfd0
runtime.mCentral_CacheSpan(0x7a2170, 0x4000)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/mcentral.go:101 +0x700 fp=0x3fff9b7c090 sp=0x3fff9b7c040
runtime.mCache_Refill(0x3ff970d0000, 0x15, 0x7b7fa8)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/mcache.go:118 +0xf4 fp=0x3fff9b7c0d0 sp=0x3fff9b7c090
runtime.mallocgc.func2()
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/malloc.go:611 +0x34 fp=0x3fff9b7c0f0 sp=0x3fff9b7c0d0
runtime.systemstack(0x3fff9b7c1a8)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/asm_arm64.s:251 +0xb8 fp=0x3fff9b7c100 sp=0x3fff9b7c0f0
runtime.mallocgc(0x180, 0x5208a0, 0x8800000000, 0x4000000000)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/malloc.go:612 +0x874 fp=0x3fff9b7c1e0 sp=0x3fff9b7c100
runtime.newobject(0x5208a0, 0xb32b0)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/malloc.go:742 +0x78 fp=0x3fff9b7c210 sp=0x3fff9b7c1e0
runtime.malg(0x8000, 0xb325c)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/proc1.go:2165 +0x28 fp=0x3fff9b7c250 sp=0x3fff9b7c210
runtime.mpreinit(0x793a60)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/os1_linux.go:188 +0x24 fp=0x3fff9b7c270 sp=0x3fff9b7c250
runtime.mcommoninit(0x793a60)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/proc1.go:114 +0x114 fp=0x3fff9b7c2c0 sp=0x3fff9b7c270
runtime.schedinit()
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/proc1.go:57 +0x74 fp=0x3fff9b7c310 sp=0x3fff9b7c2c0
runtime.rt0_go(0x3fff9b7f690, 0x3fff9b7f699, 0x0, 0x3fff9b7f69e, 0x3fff9b7f6af, 0x3fff9b7f6be, 0x3fff9b7f6d6, 0x3fff9b7f6ea, 0x3fff9b7f6fa, 0x3fff9b7f708, ...)
	/home/jperrin/code/github/go-linux-arm64-bootstrap/src/runtime/asm_arm64.s:73 +0xbc fp=0x3fff9b7c340 sp=0x3fff9b7c310

I've tried a number of variations on the build theme, but they all fail in the same manner.
I think this is go assuming/forcing a 4K PAGESIZE rather than the 64K PAGESIZE we have on linux/arm64, but I don't know enough to verify this. Any help would be greatly appreciated.

also, the same "address space conflict" can be seen when compiling go1.5beta2 from gcc-5.1.1 (which has gccgo supporting go1.4.2) natively on arm64.

Looks like a page size problem.  The current runtime assumes a 8192 byte page size.  Your arm64 system seems to be using a 65,536 byte page size.  Does that sound right?

That's correct, yes.
[jperrin@apma3 bin]$ getconf PAGESIZE
65536

Please try this patch:
diff --git a/src/runtime/arch1_arm64.go b/src/runtime/arch1_arm64.go
index 1a3165c..29a87db 100644
--- a/src/runtime/arch1_arm64.go
+++ b/src/runtime/arch1_arm64.go
@@ -8,7 +8,7 @@ const (
 	thechar        = '7'
 	_BigEndian     = 0
 	_CacheLineSize = 32
-	_PhysPageSize  = 4096*(1-goos_darwin) + 16384*goos_darwin
+	_PhysPageSize  = 65536
 	_PCQuantum     = 4
 	_Int64Align    = 8
 	hugePageSize   = 0

That patch works for me. Thanks!

Ubuntu kernels still stick with 4k pages, but it looks like we're in the minority.
I'd be happy to set the page size to 64k on arm64 for linux, which is what we have done for ppc64 I believe.

On 28 Jul 2015, at 04:33, Jim Perrin<EMAIL_ADDRESS>wrote:
That patch works for me. Thanks!
—
Reply to this email directly or view it on GitHub.

Does this mean we now won't release memory to the system on arm64 because of the work-around we have in place in scavengelist for #9993?

Hello,@rsc，my arm64 kernel is also use 64k PAGESIZE , how to dynamic check the PAGESIZE and set it ??

% getconf PAGESIZE
To the best of my knowledge you cannot change the page size the kernel
uses, it is set when compiling the kernel.
On Wed, Jul 29, 2015 at 4:57 PM, jefby<EMAIL_ADDRESS>wrote:

Hello,@rsc https://github.com/rsc，my arm64 kernel is also use 64k
PAGESIZE , how to dynamic check the PAGESIZE and set it ??
—
Reply to this email directly or view it on GitHub
https://github.com/golang/go/issues/11886#issuecomment-125862883.

@davecheney I'm sorry ,my means is that dynamic check the system PAGESIZE in code src/runtime/arch1_arm64.go , and set its value correctly not force it to some value.

My understanding from the last time this went around is that the page size
needs to be a constant. But maybe I'm wrong...
On 29 July 2015 at 19:18, jefby<EMAIL_ADDRESS>wrote:

@davecheney https://github.com/davecheney I'm sorry ,my means is that
dynamic check the system PAGESIZE in code src/runtime/arch1_arm64.go , and
set its value correctly not force it to some value.
—
Reply to this email directly or view it on GitHub
https://github.com/golang/go/issues/11886#issuecomment-125867440.

It can be recovered by a syscall, or from the auxv variable passed to the
program.
The issue we probably cannot address for 1.5 is the page size value is used
in several constant expressions which would become considerably more
expensive if the value wasn't a constant.
@khr would probably know more about this.
Thanks
Dave
On Wed, 29 Jul 2015 17:18 jefby<EMAIL_ADDRESS>wrote:

@davecheney https://github.com/davecheney I'm sorry ,my means is that
dynamic check the system PAGESIZE in code src/runtime/arch1_arm64.go , and
set its value correctly not force it to some value.
—
Reply to this email directly or view it on GitHub
https://github.com/golang/go/issues/11886#issuecomment-125867440.

The issue we probably cannot address for 1.5 is the page size value is used
in several constant expressions which would become considerably more
expensive if the value wasn't a constant.

I don't think replacing _PhysPageSize with a variable would actually be very expensive. Unlike _PageSize, it's not used in many places.
I think it's much too late to do this for 1.5, though we could query the physical page size from the OS at initialization time and at least check that it's <= _PhysPageSize.

That's good to know. Phys page size is available via a syscall, or in the
auxv vector on startup on all the linux platforms we support.
To be clear, I would not want to see this happen for 1.5.
On Thu, 30 Jul 2015 02:14 Austin Clements<EMAIL_ADDRESS>wrote:

The issue we probably cannot address for 1.5 is the page size value is used
in several constant expressions which would become considerably more
expensive if the value wasn't a constant.
I don't think replacing _PhysPageSize with a variable would actually be
very expensive. Unlike _PageSize, it's not used in many places.
I think it's much too late to do this for 1.5, though we could query the
physical page size from the OS at initialization time and at least check
that it's <= _PhysPageSize.
—
Reply to this email directly or view it on GitHub
https://github.com/golang/go/issues/11886#issuecomment-126004848.

I am 100% sure I set arm64 to use 64k pages. I am not sure how we ended up with 4k pages again. Oh well.

Merged into #10180.


test: improve resiliency of policy import test

Instead of checking whether the identities of locally running endpoints contain
the labels which are allowed by policy, query which identities have those labels
via the cilium identity get CLI. This is a more correct check since policy
computation is done in relation to all identities, not just those running
locally. It also ensures that if other tests add identities which are not
deleted yet from the key-value store, the test will still pass even if the
endpoints which utilized said identities no longer are running.
Fixes: #6347
Signed-off by: Ian Vernon<EMAIL_ADDRESS>

This change is

test-me-please

Coverage decreased (-0.08%) to 43.462% when pulling 77cdd11ca6e72f3804d460c02ca6659e2ef07b82 on pr/ianvernon/improve-resiliency-policy-trace-test into 38e678d7555f2bc6256803471fef194ca34b66b2 on master.

Build timed out waiting for VM to boot: https://jenkins.cilium.io/job/Cilium-PR-Ginkgo-Tests-Validated/8356/flowGraphTable/

test-me-please


Home Assistant OS: ImportError: Error loading shared library libjpeg.so.8

On latest Home Assistant OS the following error is logged during app initialization:
 ImportError: Error loading shared library libjpeg.so.8
Environment:

System: Home Assistant OS 5.11  (amd64 / qemux86-64)
Home Assistant Core: 2021.2.2
Home Assistant Supervisor: 2021.02.8
Add-on: AppDaemon 4
Python Apps and Dashboard using AppDaemon 4.x for Home Assistant
Add-on version: 0.4.3

Addon config:
python_packages:
  - spotipy
  - numpy
  - pillow
init_commands: []

Full log:
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 00-banner.sh: executing...
-----------------------------------------------------------
 Add-on: AppDaemon 4
 Python Apps and Dashboard using AppDaemon 4.x for Home Assistant
-----------------------------------------------------------
 Add-on version: 0.4.3
 You are running the latest version of this add-on.
 System: Home Assistant OS 5.11  (amd64 / qemux86-64)
 Home Assistant Core: 2021.2.2
 Home Assistant Supervisor: 2021.02.8
-----------------------------------------------------------
 Please, share the above information when looking for help
 or support in, e.g., GitHub, forums or the Discord chat.
-----------------------------------------------------------
[cont-init.d] 00-banner.sh: exited 0.
[cont-init.d] 01-log-level.sh: executing...
[cont-init.d] 01-log-level.sh: exited 0.
[cont-init.d] appdaemon.sh: executing...
Looking in links: https://wheels.home-assistant.io/alpine-3.13/amd64/
Collecting spotipy
  Downloading spotipy-2.16.1-py3-none-any.whl (24 kB)
Requirement already satisfied: six>=1.10.0 in /usr/lib/python3.8/site-packages (from spotipy) (1.15.0)
Requirement already satisfied: requests>=2.20.0 in /usr/lib/python3.8/site-packages (from spotipy) (2.25.1)
Requirement already satisfied: chardet<5,>=3.0.2 in /usr/lib/python3.8/site-packages (from requests>=2.20.0->spotipy) (3.0.4)
Requirement already satisfied: idna<3.2,>=2.5 in /usr/lib/python3.8/site-packages (from requests>=2.20.0->spotipy) (3.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.8/site-packages (from requests>=2.20.0->spotipy) (1.26.2)
Installing collected packages: spotipy
Successfully installed spotipy-2.16.1
Looking in links: https://wheels.home-assistant.io/alpine-3.13/amd64/
Collecting numpy
  Downloading https://wheels.home-assistant.io/alpine-3.13/amd64/numpy-1.20.1-cp38-none-any.whl (7.3 MB)
Installing collected packages: numpy
Successfully installed numpy-1.20.1
Looking in links: https://wheels.home-assistant.io/alpine-3.13/amd64/
Collecting pillow
  Downloading https://wheels.home-assistant.io/alpine-3.13/amd64/Pillow-8.1.0-cp38-none-any.whl (551 kB)
Installing collected packages: pillow
Successfully installed pillow-8.1.0
[cont-init.d] appdaemon.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[00:36:12] INFO: Starting AppDaemon...
2021-02-10 00:36:13.525402 INFO AppDaemon: AppDaemon Version 4.0.5 starting
2021-02-10 00:36:13.525727 INFO AppDaemon: Python version is 3.8.7
2021-02-10 00:36:13.525931 INFO AppDaemon: Configuration read from: /config/appdaemon/appdaemon.yaml
2021-02-10 00:36:13.526114 INFO AppDaemon: Added log: AppDaemon
2021-02-10 00:36:13.526319 INFO AppDaemon: Added log: Error
2021-02-10 00:36:13.526541 INFO AppDaemon: Added log: Access
2021-02-10 00:36:13.526938 INFO AppDaemon: Added log: Diag
2021-02-10 00:36:13.554569 INFO AppDaemon: Loading Plugin HASS using class HassPlugin from module hassplugin
2021-02-10 00:36:13.678427 INFO HASS: HASS Plugin Initializing
2021-02-10 00:36:13.678794 INFO HASS: HASS Plugin initialization complete
2021-02-10 00:36:13.679420 INFO AppDaemon: Initializing HTTP
2021-02-10 00:36:13.679892 INFO AppDaemon: Using 'ws' for event stream
2021-02-10 00:36:13.683521 INFO AppDaemon: Starting API
2021-02-10 00:36:13.686339 INFO AppDaemon: Starting Admin Interface
2021-02-10 00:36:13.686853 INFO AppDaemon: Starting Dashboards
2021-02-10 00:36:13.720910 INFO HASS: Connected to Home Assistant 2021.2.2
2021-02-10 00:36:13.808097 INFO AppDaemon: App 'spotify_mood_lights_sync' added
2021-02-10 00:36:13.809234 INFO AppDaemon: Found 1 total apps
2021-02-10 00:36:13.809781 INFO AppDaemon: Starting Apps with 1 workers and 1 pins
2021-02-10 00:36:13.810653 INFO AppDaemon: Running on port 5050
2021-02-10 00:36:13.823377 INFO HASS: Evaluating startup conditions
2021-02-10 00:36:13.834840 INFO HASS: Startup condition met: hass state=RUNNING
2021-02-10 00:36:13.835131 INFO HASS: All startup conditions met
2021-02-10 00:36:13.895732 INFO AppDaemon: Got initial state from namespace default
2021-02-10 00:36:15.816855 INFO AppDaemon: Scheduler running in realtime
2021-02-10 00:36:15.831261 INFO AppDaemon: Adding /config/appdaemon/apps to module import path
2021-02-10 00:36:15.831783 INFO AppDaemon: Adding /config/appdaemon/apps/ad-spotify-mood-lights-sync to module import path
2021-02-10 00:36:15.832198 INFO AppDaemon: Adding /config/appdaemon/apps/ad-automoli to module import path
2021-02-10 00:36:15.835712 INFO AppDaemon: Loading App Module: /config/appdaemon/apps/ad-spotify-mood-lights-sync/spotify_mood_lights_sync.py
2021-02-10 00:36:15.850066 WARNING AppDaemon: No app description found for: /config/appdaemon/apps/ad-automoli/automoli.py - ignoring
2021-02-10 00:36:15.851434 INFO AppDaemon: Initializing app spotify_mood_lights_sync using class SpotifyMoodLightsSync from module spotify_mood_lights_sync
2021-02-10 00:36:15.866819 WARNING spotify_mood_lights_sync: ------------------------------------------------------------
2021-02-10 00:36:15.867197 WARNING spotify_mood_lights_sync: Unexpected error running initialize() for spotify_mood_lights_sync
2021-02-10 00:36:15.867377 WARNING spotify_mood_lights_sync: ------------------------------------------------------------
2021-02-10 00:36:15.873684 WARNING spotify_mood_lights_sync: Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/appdaemon/app_management.py", line 150, in initialize_app
    await utils.run_in_executor(self, init)
  File "/usr/lib/python3.8/site-packages/appdaemon/utils.py", line 290, in run_in_executor
    response = future.result()
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/config/appdaemon/apps/ad-spotify-mood-lights-sync/spotify_mood_lights_sync.py", line 56, in initialize
    from PIL import Image
  File "/usr/lib/python3.8/site-packages/PIL/Image.py", line 94, in <module>
    from . import _imaging as core
ImportError: Error loading shared library libjpeg.so.8: No such file or directory (needed by /usr/lib/python3.8/site-packages/PIL/_imaging.cpython-38-x86_64-linux-gnu.so)
2021-02-10 00:36:15.874037 WARNING spotify_mood_lights_sync: ------------------------------------------------------------
2021-02-10 00:36:15.875642 INFO AppDaemon: App initialization complete

Heyo, I ran into the same issue. It seems you'll need to install add jpeg to system_packages as well. I did this by adding the following lines to my AppDaemon config in addition to the python_packages config you've posted:
system_packages:
  - jpeg

Hope this helps!

Heyo, I ran into the same issue. It seems you'll need to add jpeg and tiff to system_packages as well. I did this by adding the following lines to my AppDaemon config in addition to the python_packages config you've posted:
system_packages:
  - jpeg
  - tiff

Hi, adding jpeg and tiff to system_packages indeed fixed it, thanks!

You may also need to download https://github.com/NiklasReiche/ad-spotify-mood-lights-sync/blob/master/examples/default_profile.png to /config/www/spotify-lights-sync/test.png

It seems the default_profile.png is downloaded/copied into any specified directory automatically.
I'll leave this issue open as a heads-up for now.

Hey, thanks for the heads-up.
I am currently running Home Assistant core and AppDaemon directly in a Python environment, so I can't easily test the dependencies. I will add a note on using jpeg and tiff to the readme for now.
Regarding the default_profile.png image file, you shouldn't need to download anything. The file is created once on startup and placed at the location specified in the config. The image itself is never used by the app. It's just an easy way to visualize and debug a custom color profile, in case you don't want to use the default one. If you don't need this functionality you can also simply remove the color_map_image option from your config.
In that case jpeg and tiff might also not be necessary, since pillow never gets imported.

If you don't need this functionality you can also simply remove the color_map_image option from your config.
In that case jpeg and tiff might also not be necessary, since pillow never gets imported.

Hi, just tried the "minimal" configuration, you are right, 'jpeg', 'tiff' and 'pillow' are not needed in this case.
As this is now documented I'll close this issue.
Thanks!


gcc48 and gcc49: add gfortran by default

This is mostly for selfish reasons: I want to use gfortran in Travis-CI but I cannot wait for gcc to build from source as it will make the build time out.
Also, this brings these packages in line with gcc5 and gcc6

Seems fine, yep 👍

Seems reasonable. Shipped in https://github.com/Homebrew/homebrew-versions/commit/567b80ae44572d3310d5edfee42668d429a57f3f & https://github.com/Homebrew/homebrew-versions/commit/2f191bfbfff1e874a1ab59d38e6f3a5cb4674c6c. Thanks Tom!

Thanks!


SID text overlaps

Whenever multiple SIDs share the same exit point, the SID name and exit point are written on the canvas at the location of that exit point, causing overlapping text. Logic could be added that would check to see if multiple things are trying to be written to the same location, and then have subsequent items offset above and/or below that position, resulting in a "new line" for each SID that ends at the shared exit point.

Got sick of this when trying to create Auckland International Airport (NZAA) so I went ahead and fixed it. Open to any suggestions to improve the fix.

@aldent95 Looks great, I can't think of anything else to improve it though!

Resolved, thanks to @aldent95's work on #643. Closing.


Pipenv provides incorrect completion if current shell does not match login shell

It seems as though Pipenv does not correctly detect the fish shell when determining what to output for pipenv --completion. This is a regression that appeared within the last couple of weeks for me.
$ python -m pipenv.help output
Pipenv version: '11.8.0'
Pipenv location: '/home/qasim/.local/lib/python3.6/site-packages/pipenv'
Python location: '/usr/bin/python'
Other Python installations in PATH:

2.7: /usr/bin/python2.7

2.7: /usr/bin/python2.7

3.6: /usr/bin/python3.6m

3.6: /usr/bin/python3.6

3.6.4: /usr/bin/python

2.7.14: /usr/bin/python2

3.6.4: /usr/bin/python3

PEP 508 Information:
{'implementation_name': 'cpython',
 'implementation_version': '3.6.4',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.15.9-1-ARCH',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP PREEMPT Sun Mar 11 17:54:33 UTC 2018',
 'python_full_version': '3.6.4',
 'python_version': '3.6',
 'sys_platform': 'linux'}

System environment variables:

COLORTERM
DBUS_SESSION_BUS_ADDRESS
DESKTOP_AUTOSTART_ID
DESKTOP_SESSION
DISPLAY
GDMSESSION
GDM_LANG
GNOME_DESKTOP_SESSION_ID
HOME
LANG
LOGNAME
MAIL
MOZ_PLUGIN_PATH
PATH
PWD
SESSION_MANAGER
SHELL
SHLVL
SSH_AUTH_SOCK
TERM
USER
USERNAME
VTE_VERSION
WINDOWID
WINDOWPATH
XAUTHORITY
XDG_CURRENT_DESKTOP
XDG_MENU_PREFIX
XDG_RUNTIME_DIR
XDG_SEAT
XDG_SESSION_DESKTOP
XDG_SESSION_ID
XDG_SESSION_TYPE
XDG_VTNR
PIP_PYTHON_PATH
PYTHONUNBUFFERED

Pipenv–specific environment variables:
Debug–specific environment variables:

PATH: /usr/local/bin:/usr/local/sbin:/usr/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/qasim/.local/bin
SHELL: /bin/bash
LANG: en_GB.UTF-8
PWD: /home/qasim

Expected result
I'd expect the completion to complete without errors :)
Actual result
When attempting to auto-complete on the first Pipenv command:
qasim@qasim-desktop ~> pipenv - (line 1): Illegal command name “_pipenv_completion()”
begin;  _pipenv_completion() {     local IFS=$'\t'     COMPREPLY=( $( env COMP_WORDS="${COMP_WORDS[*]}" \                    COMP_CWORD=$COMP_CWORD \                    _PIPENV_COMPLETE=complete-bash $1 ) )     return 0 }  complete -F _pipenv_completion -o default pipenv
        ^
from sourcing file -
	called on line 60 of file /usr/share/fish/functions/eval.fish

in function “eval”
	called on line 1 of file ~/.config/fish/completions/pipenv.fish

from sourcing file ~/.config/fish/completions/pipenv.fish
	called on standard input

in command substitution
	called on standard input

source: Error while reading file “-”

It seems as though this is the bash code being output from  rather than the fish code.
Steps to replicate

pip install --user --upgrade pipenv
Setup fish completions inside ~/.config/fish/completions/pipenv.fish: eval (pipenv --completion)
Enter fish and type Pipenv  and then try to auto-complete something

@QasimK what version of fish are you running?

@QasimK you aren't running fish...

SHELL: /bin/bash

And your ~/.local/bin is the last thing on your path and will be pre-empted by everything else:

PATH: /usr/local/bin:/usr/local/sbin:/usr/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/qasim/.local/bin

To use completion with bash, use: eval "$(pipenv --completion)"
Closing for now since this seems to be an issue with which shell is running

@techalchemy Thank you for looking into this. I would like to assure you that I am running fish and I hope the following screenshot is sufficient evidence.

Do you think it is an issue with fish or perhaps a problem with my system? Fish 2.7.1 was installed 2 months ago.
Thanks for pointing out ~/.local/bin on my path, I've changed that now.

Pipenv uses SHELL to detect what shell you’re in. That’s why it does not work. It’s not the best way to do it, but for now I guess you’ll have to work around it (by changing SHELL temporarily to trick Pipenv into producing the correct output).
@kennethreitz Are you open to modify the option to take an explicit argument (e.g. --completion=fish) or detect the current shell with an additional environ (maybe PIPENV_SHELL)?

An explicit argument would probably be simpler, and I believe that's how pip does it. I'd be happy to create a PR for that

@uranusjr Would you mind re-opening this issue given that it is an issue with Pipenv and not the environment, and given that there is also a potential solution on the table

Pipenv uses click-completion, so you can use this instead:
eval (env _PIPENV_COMPLETE=source-fish pipenv)

however, it does seem like this is a Pipenv bug, if $SHELL is not actually meant to reflect the current shell.

@anowlcalledjosh that’s exactly what pipenv expects $SHELL to be. The problem is if you login with some other shell, say bash, then you use run fish, for instance, but not as a login shell (i.e not with fish -l -i)

Overall I would say +1 for the solution proposed by @anowlcalledjosh - closing this because pipenv really expects you to run properly configured login shells when you use it and the documentation does specifically say to specify $SHELL

@techalchemy Not sure if I can be any less blunt, but it seems like Pipenv is using $SHELL incorrectly. It's kind of like creating a program with the documentation that $PATH is to be the location of your documents folder?
Also, I'm not sure if there is any other way to run a "properly configured" fish without changing your default shell (which is not recommended because of the problems that causes):

The workaround that @anowlcalledjosh provided works great. I feel that should at least be documented if Pipenv isn't willing to make other changes.

Hm arguably yes we are misusing the shell variable, how else would you suggest we get the users shell on Linux? People typically add completion in their login scripts so we haven’t encountered this. Also completion has been historically slow so I feel most people have just avoided it.
As for documentation, we need plenty of help there so PRs welcome :)

Well there are a few workarounds available without complicating the most common workflow. Allowing PIPENV_SHELL to take precedence over SHELL is one. Another would be to simply document a hack like
SHELL=$(which fish) source $(pipenv --completion)

I hacked around this by adding set SHELL /usr/bin/fish to my ~/.config/fish/config.fish. This probably breaks some other stuff, but I'm not sure what, and things still seem to be working ¯\_(ツ)_/¯

@techalchemy Earlier, I provided evidence that that is not what you're supposed to have in your $SHELL variable. Do you have any documentation that shows otherwise?

Per POSIX, $SHELL should be set as follows:

This variable shall represent a pathname of the user's preferred command language interpreter.

That's all it says (plus a note about behaviour of "utilities" being undefined when $SHELL isn't POSIX-compatible). Fish is certainly your "preferred command language interpreter", so afaict it's totally valid for $SHELL to point to Fish.

How else would you suggest we get the users shell on Linux?

Proof of concept:
>>> import os
>>> import psutil
>>> print(psutil.Process(os.getppid()).exe())
/bin/zsh

Obviously, doesn't work in scenarios where there's an intermediate parent.

Obviously, doesn't work in scenarios where there's an intermediate parent.

Unfortunately it’s not uncommon for Pipenv to be run with an intermediate parent, especially on Windows.

This might be a good way to guess the preferred shell, however. We can always fall back to SHELL if it fails.

We aren’t shipping psutil. I literally spent 2 days writing bindings for windows process crawling to cut psutil out of our dependencies because it causes problems for people on windows.

There is really no reason to change this behavior as far as I can tell, we have enough technical complexity as is.

This would definitely add a lot of complexity but after doing some research, it looks like the only real way of identifying the current shell is to look for the existence of shell-specific environment variables such as $FISH_VERSION, $ZSH_VERSION, $BASH_VERSION, $FCEDIT, and the like. If anything, we could just fall back to using $SHELL if any of those don't exist.

Sounds like stuck between a rock and a hard place then.
If that's really the case, then the best solution would be to make this clear in the docs.  I haven't found any mention of $SHELL in the docs thus far.

Thanks for the offer @anowlcalledjosh — our current stance is that we aren’t using psutil in pipenv. We literally just removed it and had to vendor pew and write c extension bindings for windows. PSUtil is slow and it breaks regularly for windows users, we just aren’t going to use it. We handle enough special edge cases, if we can’t trust your environment we aren’t really able to be responsible for maintaining logic to handle edge cases.
If there’s an approach that works around this, I would be ok with that. But in my mind the easiest approach is to set your shell variable

Also it should be worth some write-up in the documentation. Anyone care to contribute?

I just ran into this exact problem after installing fish on the README's recommendation. Honestly, I feel like this issue does warrant further discussion given how easy it is to fall into this trap. The README strongly recommends using fish but then things immediately break down once you try to set it up.
Most elegant solution as far as I can tell is to simply have pipenv --completion take an optional parameter specifying which shell you would like the completions for. That way when you set up your .bashrc you just ask for bash completions and if you set up your fish stuff you can just ask for fish completions. I understand that $SHELL can be a tricky beast; this could be a simple workaround.
It seems like people have ragequit out of #2045 but a solution along those lines would be simple and keep everything working smoothly.

Fish is recommended in the docs. Not using bash, then fish, then whatever. And as was described in one of the numerous other issues on this, you can basically do that already anyway

And as was described in one of the numerous other issues on this, you can basically do that already anyway

@techalchemy I'm not quite sure what you mean here. Yes, it is technically possible to hack together a working installation but no, it's definitely not user friendly. In the mind of a person on macOS following the readme,

I see that using fish is recommended over bash. Ok, that sounds interesting... brew install fish.
Hmm, well it looks like I'm supposed to set up completions so I'll add eval (pipenv --completion) to my ~/.config/fish/completions/pipenv.fish.
Sweet, now let's see what we got here!

bash$ fish
fish~> pipenv -<TAB>
(line 1): Illegal command name “_pipenv_completion()”
begin;  _pipenv_completion() {     local IFS=$'\t'     COMPREPLY=( $( env COMP_WORDS="${COMP_WORDS[*]}" \                    COMP_CWORD=$COMP_CWORD \                    _PIPENV_COMPLETE=complete-bash $1 ) )     return 0 }  complete -F _pipenv_completion -o default pipenv
        ^
from sourcing file -
	called on line 60 of file /usr/share/fish/functions/eval.fish

in function “eval”
	called on line 1 of file ~/.config/fish/completions/pipenv.fish

from sourcing file ~/.config/fish/completions/pipenv.fish
	called on standard input

in command substitution
	called on standard input

source: Error while reading file “-”

That's just an awful user experience since they followed the exact instructions from the README. And I think the number of people that have commented and filed issues reporting this problem is some indication of its gravity.

Most elegant solution as far as I can tell is to simply have pipenv --completion take an optional parameter specifying which shell you would like the completions for.

(argh, pressed submit too early...)

Most elegant solution as far as I can tell is to simply have pipenv --completion take an optional parameter specifying which shell you would like the completions for.

I don't think this is actually possible with Click. You can have --completion, or --completion <shell>, but not both (unless I've missed something in the docs).

@anowlcalledjosh mmm i see... is there any way to do like --completion and --completion=<shell> or --completion and --completion --completion-shell=<shell>?

@anowlcalledjosh Ah or what about replacing
@option(
    '--completion',
    is_flag=True,
    default=False,
    help="Output completion (to be eval'd).",
)

with something like
@option(
    '--completion',
    default=None,
    help="Output completion (to be eval'd).",
)

and then if completion is None, guess based on $SHELL, else use the provided shell.

Unfortunately that doesn't work – if you then run pipenv, then completion is None, but if you then try to do pipenv --completion, it complains Error: --completion option requires an argument.
It looks like pallets/click#549 is tracking the lack of this feature in Click.

@anowlcalledjosh oh right i misunderstood the docs. but it seems like --completion-shell=<shell> would still work

I'm just going to chime in and say that it is probably legitimate to keep the user's SHELL variable to a more common shell, like bash, or zsh as many other large pieces of software are not correctly configured to run off a fish SHELL. I see fish as a cool interactive shell, but legacy software still requires bash.
Personally, I set gnome-terminal to run the command fish instead of my SHELL and I've found that to be nice. But I ran into this issue, and was quite frustrated.


split term: 'intellectual disability-macrocephaly-hypotonia-behavioral abnormalities syndrome'

Term to split:
MONDO:0014602 'intellectual disability-macrocephaly-hypotonia-behavioral abnormalities syndrome'
Name for new terms: Note - when a term should be split, there should be 2+ new terms. Please list the new labels below.

Houte-Janssens syndrome https://omim.org/phenotypicSeries/PS616355
Hogue-Janssens syndrome 1 https://omim.org/entry/616355, which is currently MONDO:0014602 'intellectual disability-macrocephaly-hypotonia-behavioral abnormalities syndrome'

@nicolevasilevsky
Houte-Janssens syndrome https://omim.org/phenotypicSeries/PS616355
is already in Mondo: MONDO:0957553
Should this issue become

review of X-ref for MONDO:0014602, and review the classification?

Seems like the OMIMPS was added after I created this ticket (I'm guessing?)
Looks like 'intellectual disability-macrocephaly-hypotonia-behavioral abnormalities syndrome' should be renamed and made a child of MONDO:0957553
I can take care of this


Add logger to CryptoOps

Fix an error where cypress is complaingin about a call to logger.verbose where logger is undefined.
This is an attempt to fix #5

@joelmeiller Any chance you will merge this anytime soon?

Any news about this?

Sorry for the long response @Oldenborg .
I merged your update.


Updated to include Account and team setup events, system console events

Covering changes related to https://github.com/mattermost/mattermost-webapp/pull/5049 and https://github.com/mattermost/mattermost-webapp/pull/5047

Summary

Ticket Link

@adamjclarkson For future PRs for v5.22, please help submit a new PR in the 5.22-documentation branch.

whoops ok sorry Amy


Feature/taxsim35 fields

added new fields

property tax credit
child care credit
fica
tfica
v19

LGTM! but need to add unit tests


MAINT: Configure Ruff formatter, update ruff config

This is a small incremental update to streamline our linting and formatting configuration.
Now that the ruff formatter is released and stable, I think it makes sense to use that rather than black. It simplifies our config, is faster, and supports notebooks natively.
Changes:

pre-commit: Remove black in favour of ruff-format, and remove check-json
pyproject.toml: Update ruff config to current spec, with separate [tool.ruff.lint] and [tool.ruff.format] sections

Reformatted:

notebooks/ : A few trivial changes due to the new linter, mostly just reordering parts of the JSON file. No functional changes, so this doesn't really have to be reviewed in detail IMO.

Id be interested to hear what issues you face, not least so we can improve the contributing guide with any extra info that might be needed.
What happens if you run:
pre-commit install
pre-commit run --all-files

Id be interested to hear what issues you face, not least so we can improve the contributing guide with any extra info that might be needed.
What happens if you run:
pre-commit install
pre-commit run --all-files

pre-commit run --all-files

I guess it has something todo with having 2 Python versions on the same windows machine and not managing them correctly, so not sure if this is really something for our docs. Here is the error
$ pre-commit run --all-files
[INFO] Installing environment for https://github.com/astral-sh/ruff-pre-commit.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('C:\\Users\\Tobias Pitters\\.cache\\pre-commit\\repo70hjb17v\\py_env-python3.9\\Scripts\\python.EXE', '-mpip', 'install', '.')
return code: 101
stdout: (none)
stderr:
    Unable to create process using 'c:\users\tobias pitters\appdata\local\programs\python\python39\python.exe -mpip install .'
Check the log at C:\Users\Tobias Pitters\.cache\pre-commit\pre-commit.log
(venv3.9)

And this is the message in pre-commit/pre-commit.log:
### version information

pre-commit version: 3.5.0
git --version: git version 2.40.0.windows.1
sys.version:
    3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]
sys.executable: c:\users\tobias pitters\appdata\local\programs\python\python39\python.exe
os.name: nt
sys.platform: win32

### error information

An unexpected error has occurred: CalledProcessError: command: ('C:\\Users\\Tobias Pitters\\.cache\\pre-commit\\repo70hjb17v\\py_env-python3.9\\Scripts\\python.EXE', '-mpip', 'install', '.')
return code: 101
stdout: (none)
stderr:
    Unable to create process using 'c:\users\tobias pitters\appdata\local\programs\python\python39\python.exe -mpip install .'

Traceback (most recent call last):
  File "c:\users\tobias pitters\appdata\local\programs\python\python39\lib\site-packages\pre_commit\error_handler.py", line 73, in error_handler
    yield
  File "c:\users\tobias pitters\appdata\local\programs\python\python39\lib\site-packages\pre_commit\main.py", line 414, in main
    return run(args.config, store, args)
  File "c:\users\tobias pitters\appdata\local\programs\python\python39\lib\site-packages\pre_commit\commands\run.py", line 442, in run
    install_hook_envs(to_install, store)
  File "c:\users\tobias pitters\appdata\local\programs\python\python39\lib\site-packages\pre_commit\repository.py", line 248, in install_hook_envs
    _hook_install(hook)
  File "c:\users\tobias pitters\appdata\local\programs\python\python39\lib\site-packages\pre_commit\repository.py", line 95, in _hook_install
    lang.install_environment(
  File "c:\users\tobias pitters\appdata\local\programs\python\python39\lib\site-packages\pre_commit\languages\python.py", line 214, in install_environment
    lang_base.setup_cmd(prefix, install_cmd)
  File "c:\users\tobias pitters\appdata\local\programs\python\python39\lib\site-packages\pre_commit\lang_base.py", line 86, in setup_cmd
    cmd_output_b(*cmd, cwd=prefix.prefix_dir, **kwargs)
  File "c:\users\tobias pitters\appdata\local\programs\python\python39\lib\site-packages\pre_commit\util.py", line 110, in cmd_output_b
    raise CalledProcessError(returncode, cmd, stdout_b, stderr_b)
pre_commit.util.CalledProcessError: command: ('C:\\Users\\Tobias Pitters\\.cache\\pre-commit\\repo70hjb17v\\py_env-python3.9\\Scripts\\python.EXE', '-mpip', 'install', '.')
return code: 101
stdout: (none)
stderr:
    Unable to create process using 'c:\users\tobias pitters\appdata\local\programs\python\python39\python.exe -mpip install .'


Bad conversion to pdf

I was testing with simple html structure, and I thought it was problem of the padding or margin of the html, but I removed from the css and still convert the same.
I'm attaching images and the source code of the html, in order to help me or to resolve this problem.
Thanks in advance:

<!DOCTYPE html>

    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Document</title>

Medical Examination Form

        Personal Information:

Name: John Doe

Date of Birth: March 12, 1980

Gender: Male

        Medical History:

          Chronic Conditions: Type 2 Diabetes

Allergies: None

Current Medications: Metformin

        Examinations:

Blood Pressure: 120/80 mmHg

Heart Rate: 72 beats per minute

Body Temperature: 37.0 ºC

        Recommendations:

          Diet: Follow a low-carbohydrate diet

          Exercise: Walk at least 30 minutes a
          day

Having the same issue with
 element.

Same issue, try to solve this by CSS, but it seems not a css issue.

It's actually the padding: - if you remove that property from both body and .container CSS entries, the linesof the document  are no longer spread out to fill the whole page. I am certain this side-effect of defining a padding is unintended.


OpenID Connect authentication middleware

What does this PR do?
This PR adds OpenID Connect authentication middleware to Traefik. This middleware can be configured either through the configuration file or by adding annotations to ingress objects in Kubernetes.
Motivation
We are currently in the process of designing a Kubernetes cluster that is going to be used by a large number of engineers within a single organisation. These engineers will be able to spawn their own (or their team's) web applications, under the condition that they are adequately secured. For this reason, we want to enforce the use of our company's OpenID Connect identity provider where easily possible.
Up until so far we've made use of keycloak-proxy, but this has the obvious downside that it requires every engineer to maintain their own containers for this. Especially at such a large scale, it also causes an unnecessary amount of memory/CPU overhead. It also makes it hard to enforce this policy; we'd need to check whether everyone actually has such a proxy running and ingresses actually route traffic to them, which is complex.
This is why we'd like to work towards a system where the ingress controller can already enforce the use of OpenID Connect. With this change applied, setting up OpenID Connect is simply a matter of adding the following annotations to an ingress:
ingress.kubernetes.io/auth-type: oidc
ingress.kubernetes.io/auth-oidc-discovery-url: https://login-service.example.com/
ingress.kubernetes.io/auth-oidc-scopes: openid profile email
ingress.kubernetes.io/auth-secret: some-kubernetes-secret-containing-client-id-and-secret

After enforcing authentication, requests are forwarded with some X-Auth-* headers set, just like keycloak-proxy does.
More

[ ] Added/updated tests
[ ] Added/updated documentation

Additional Notes
This patch is still horribly incomplete. The reason I'm sending this PR is because I'm interested in knowing whether the Traefik project is interested in a change like this. If so, I'm very willing to bash this change into a decent shape for getting it upstream.

@EdSchouten Thanks for your contribution.
Few months ago, we have added a way to forward authentication to a delegate server.
We would love to help you use that feature instead, as we tend to avoid complex authentication implementations into Traefik.
Could you join us on Slack ?

@EdSchouten Sorry but we still need to validate the need of integrating OpenID into Traefik. Indeed, we would prefer to use our forward authentication feature, even if we need to make it evolve. I suggest to continue the discussion on Slack, and stop pushing new code for now ;)

Hi @EdSchouten
After several internal discussions, we prefer to not implement OIDC right into Traefik.
There are too many different (and used) authentication protocols, we cannot maintain all implementations in Traefik, and we would rather not have only few available.
For now, we then prefer to delegate authentication to external servers, using authentication forward, or in the case of OpenID, using for example Keycloak. We will work an adding documentation soon on how OpenID can be managed with Traefik and an external authentication server.
Finally, we cannot disclose on this for now, but we are working on another way to let Traefik manage more authentication protocols natively. We hope to talk about this soon ;)
We are sorry to close this one for now, thanks again for your great contribution!

@emilevauge quite possible you misunderstood the original question. @EdSchouten is not asking for the server functionality but for termination on forward-proxy level in usecases of public, confidential and bearer-only. you are mentioning keycloack but that is exactly what @EdSchouten is asking for here, and yes we would love to use keycloak.
ingress.kubernetes.io/auth-type: oidc
ingress.kubernetes.io/auth-oidc-discovery-url: https://login-service.example.com/
ingress.kubernetes.io/auth-oidc-scopes: openid profile email
ingress.kubernetes.io/auth-secret: some-kubernetes-secret-containing-client-id-and-secret

however what is missing is above functionality similar to Swagger's OpenApi 3 or Kong you need to be able to inspect jwt token and admit control to back-ends based on group membership for example. that would be totally kick-ass and would buy you tons of additional attention from the serious side of the universe. (love your product btw)
https://swagger.io/docs/specification/authentication/openid-connect-discovery/

Hi Peter,
Yes, passing the JWT to the backend process is something that I'd love to add. The only problem is, I'm not enough of an expert to know what's the proper way for doing that. I know there is Authentication: Bearer ...., but that is supposed to hold an OAuth2 access key, not a JWT, right?

Hi @EdSchouten
Well, on the container level it would be nice to have unsecured api's or ui's and having a gateway in front that can do admission control based on token signature/expiry validity and then actual group membership. But I understand why @emilevauge  doesn't want to pollute main code. For now we oauth2_proxy, Apache modules, Kong or Swagger.
BTW: in OpenID Connect which is what Keycloak is using, Oauth2 is JWT. You can easily prove that by looking for TWO dots in the token that separates the message into 3 parts (header,payload,signature). Just paste the token into a window at jwt.io and you can inspect it like any other JWT.

Great PR! This is exactly what we are looking for. We want to ensure all Traefik backends can only be reached after having been authenticated with Keycloak (or similar) without increasing the complexity of our stack by having to add intermediate proxies etc. We currently connect each application server/ HTTP server directly with Keycloak using adapters or OIDC modules, which is manageable if you have 5 of them, but becomes a big challenge if you have +100 of them. So, +1 on this functionality.

@bsloeserwij well, i talked to Traefik, Nginx, Haproxy, OpenApi Swagger and Auth0 about this. So far no interest as you can see this issue is close. I'm sure one of them will come up with a great way to handle this. I went in the direction of GraphQL and protection of Queries based on JWT token principal's group membership. Having fun with that so far and left kube ingress behind for the time being.  For the record +1 on this functionality.

Consider moving to Kong if you need OpenID Connect authentication.

On 12 Sep 2018, at 11:55, Peter Styk<EMAIL_ADDRESS>wrote:
@bsloeserwij well, i talked to Traefik, Nginx, Haproxy, OpenApi Swagger and Auth0 about this. So far no interest as you can see this issue is close. I'm sure one of them will come up with a great way to handle this. I went in the direction of GraphQL and protection of Queries based on JWT token principal's group membership. Having fun with that so far and left kube ingress behind for the time being. For the record +1 on this functionality.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

@EdSchouten I've been fighting some of the same short-comings you described.  I decided to create a project that let's you run a single instance to handle multiple configurations.  I'm at the state code-wise where I'm looking for some early testing (I've only been using keycload).  Any interest from anyone on this thread?  It's also proxy agnostic (ie: it works with any proxy that support forward/external auth including nginx and ambassador).

The documentation is a little rough around the edges but I've now tested with 2 different providers (keycloak, and self-hosted gitlab) successfully and have working configurations for both nginx and traefik (cluster annotation examples coming in the next day or 2).  It's probably ready for some alpha level feedback.  Opened https://github.com/travisghansen/oauth-external-auth-server/issues/1 on the project directly to continue the conversation there.
https://github.com/travisghansen/oauth-external-auth-server

As a follow up to the comments above, I adjusted the scope of the project to be a generic external auth server.  I went ahead and implemented 6 plugins (so far) including oauth2, OpenID Connect, ldap, htpasswd, request param, and request header.  In addition, the plugins can be put into a pipeline within a single configuration.  For example you can enable both basic auth (via htpasswd or ldap) and allow for oauth2/openid to protect the same service.


Cherry-Pick #306 from Volcano

https://github.com/volcano-sh/volcano/pull/306

I Signed it

william-wang needs CLA

It might be the problem.

I just Signed the CLA.

/ok-to-test

I Signed it

I signed it

/check-cla

I just Signed the CLA.

@william-wang Can you please try to signin again with exactly same username as github

@william-wang Can you please try to signin again with exactly same username as github
I just signed again.

I Signed it

it is better to get this one merged this week!

@k82cn Does this PR needs to be a part of v0.5 release?

/assign @k82cn

/lgtm
/approve

please also cherry-pick to 0.5 release.


Move dmp for test compilation in all platforms

move core to TESTOUTPUT/compile

[skip ci]
Signed-off-by: Jenny Chen<EMAIL_ADDRESS>

Tested in https://github.com/eclipse/openj9/pull/7875/

We would like to hold this PR until eclipse/openj9#7885 and AdoptOpenJDK/openjdk-tests#1472 are merged

could fix the merge conflict and run Grinder_TKG again? Thanks

Thanks @JennyChenyj  for testing internally.


Added a couple of new partners to Technology Partners section

This change is

@devin-donnelly is there an ETA before this is approved. With k8s partner program, we are getting more and more tech partner who needs to be listed in the page. Let me know if I can help anyway.

I can't merge this right now. You have a merge conflict in the main partner file.

@devin-donnelly having issues with this branch. closing this PR and submitting a new one with all the changes


SendSecurityHubSummaryEmailLambda fails due to "bucket you are attempting to access must be addressed using the specified endpoint. "

I'm in eu-west-2 if it's any help.

This went away when I moved the source S3 bucket to the same region as the CloudFormation stack


SEAB-6164: resizing and alignment fix for landing page

Description
During the demo, we discovered some tweaks that would improve the landing page.
Changes made:

The bottom text was not center aligned so that was fixed
The card width was raised to 45% from 40% when aligned into rows to remove the scrollbar on the code block of the Tips and Tricks card
For window sizes 1300px and below, the card container layout is changed to column instead of row.

This is the video demo for the resizing
https://github.com/dockstore/dockstore-ui2/assets/61166764/66ee3eb9-ffc6-4e71-b2f4-bfdbe28a983f
Review Instructions
Play with the resizing on https://qa.dockstore.org/github-landing-page and verify that things look good.
Issue
https://ucsc-cgl.atlassian.net/browse/SEAB-6164
Security
If there are any concerns that require extra attention from the security team, highlight them here.
Please make sure that you've checked the following before submitting your pull request. Thanks!

[ ] Check that your code compiles by running npm run build
[ ] Ensure that the PR targets the correct branch. Check the milestone or fix version of the ticket.
[ ] If this is the first time you're submitting a PR or even if you just need a refresher, consider reviewing our style guide
[ ] Do not bypass Angular sanitization (bypassSecurityTrustHtml, etc.), or justify why you need to do so
[ ] If displaying markdown, use the markdown-wrapper component, which does extra sanitization
[ ] Do not use cookies, although this may change in the future
[ ] Run npm audit and ensure you are not introducing new vulnerabilities
[ ] Do due diligence on new 3rd party libraries, checking for CVEs
[ ] Don't allow user-uploaded images to be served from the Dockstore domain
[ ] If this PR is for a user-facing feature, create and link a documentation ticket for this feature (usually in the same milestone as the linked issue). Style points if you create a documentation PR directly and link that instead.
[ ] Check whether this PR disables tests. If it legitimately needs to disable a test, create a new ticket to re-enable it in a specific milestone.

couldn't rebase properly. will create a new PR to the release branch


401 when trying to create post

The documentation isn't very clear on what the callback and redirect URLs are, so it's entirely possible that I just set those wrong, but I have no idea what to set them to.
I copied the consumer_key, consumer_secret, token_key, and token_secret from the sample code in https://api.tumblr.com/console/calls/post/reblog, but I still get a 401 Unauthorized when I call createPost. How do I even go about troubleshooting this issue? I can't find anything obviously wrong but the documentation is less than helpful. Is there really no login method I have to await first?

Haven't tested all of them, but I tried two or three of them and wasn't able to get any of them to work. I can try them all if you think it would be useful.

createPost() is more likely to have other problems. Maybe there's an issue with the post data, quotas, or something else.
userInfo() should be a simple way to test whether it's an issue with your credentials or not.

Closing due to inactivity.


Silence dulwich spam in github tests

This PR silences the dulwich spam (e.g. "Counting objects 1/3000") in the github test logs.  We still have a lot of noise, but now it's from PCS/AOS proper and we clean that up as desired.
Example logs with this change

Thanks for doing this


Add an annotation only to the model class

I have the following mapping:
openapi-processor-mapping: v2

map:
  types:
    - type: Type1 @ lombok.Builder
    - type: Type2 @ lombok.Builder

This generates a model class:
@Builder
@Generated(value = "openapi-processor-micronaut", version = "2023.2.2", date = "2023-06-27T16:41:57.323668900+02:00")
public class Type1 {

    // content omitted

}

and model class that uses another model class
@Builder
@Generated(value = "openapi-processor-micronaut", version = "2023.2.2", date = "2023-06-27T16:41:57.323668900+02:00")
public class Type2 {

    @Builder // This does not compile
    @JsonProperty("location")
    private Type1 location;

    public Type1 getLocation() {
        return location;
    }

    public void setLocation(Type1 location) {
        this.location = location;
    }
}

Is there a way to add the annotation just for the class and not to the variables?

there is no configuration to control this.
The reasoning behind it is, that the processor can't add an annotation to a schema class if it is mapped to an existing class (i.e. one that is not generated and can not be modified). Instead it will put it on a property of a generated class where the mapped type is used.
Considering this and that Type1 in your example is a generated type and not a mapped type, it should probably not add the annotation to the Type1 property....   I guess it is a bug :-(
I'm thinking about adding your object @ lombok.Builder suggestion from #30. It would be a lot simpler for this use-case. :-)


Toggle Camera Functionality Not Working

I am trying to switch camera (front/back). It works fine with local streaming but when I add the local stream to peerconnection then it doesn't appear on the remote end.
I am using removeStream and addStreaming methods on peerconnection object.

I remove the stream from the peer connection first
I get the localStream by WebRTC.getUserMedia method.
I add the new stream back to the same peer connection.

Please help.

If you are doing it that way, you need to renegotiate. Do you do that? You can use https://github.com/oney/react-native-webrtc#mediastreamtrackprototype_switchcamera, which is a lot faster.


General: Add support for smart contract verification to ExplorerAPI

Overview
When you deploy a smart contract to a public network, you often want to verify that contract's source code through one (or more) blockchain explorer(s) source code verification API (if it exists).
Specification
Add a kwarg verify=None to AccountAPI.deploy(ContractType, ..., verify=True) that will publish the corresponding smart contract code to any loaded ExplorerAPI plugin
Also prototype a way to verify source code "after the fact", maybe via ContractInstance or ReceiptAPI of the deployment txn.
Dependencies
Have to be able to link back to the Source(s) (or just use the package manifest directly) from a ContractType of the deployed contract.

there is also an issue for this found in ape-etherscan: https://github.com/ApeWorX/ape-etherscan/issues/22
Now, they are linked :)

dependency #792 #530

related: https://github.com/ApeWorX/ape/issues/794

@NotPeopling2day I believe this is the information you were requesting above 👍

dependency: https://github.com/ApeWorX/ape/issues/850

status: blocked by https://github.com/ApeWorX/ape/issues/948 and https://github.com/ApeWorX/ape/issues/530


FetchError on TrRouting on long running batchrouting

During a specific big batch calculation, we get the following error message. Calculation continue, but seems something that might be bad from the TrRouting side.
2024-07-16T14:12:05.956561133Z FetchError: request to http://localhost:14000/v2/route?origin=-73.572,45.5061&destination=-73.1775,45.5308&scenario_id=27ef6b8a-af95-491b-8015-e59d67f81cc1&time_of_trip=57600&time_type=0&alternatives=true&min_waiting_time=180&max_access_travel_time=1200&max_egress_travel_time=1200&max_transfer_travel_time=1200&max_travel_time=14400&max_first_waiting_time=3600 failed, reason: socket hang up
2024-07-16T14:12:05.956604916Z     at ClientRequest. (/app/node_modules/node-fetch/lib/index.js:1491:11)
2024-07-16T14:12:05.956611874Z     at ClientRequest.emit (node:events:517:28)
2024-07-16T14:12:05.956616000Z     at Socket.socketOnEnd (node:_http_client:525:9)
2024-07-16T14:12:05.956619494Z     at Socket.emit (node:events:529:35)
2024-07-16T14:12:05.956622941Z     at endReadableNT (node:internal/streams/readable:1400:12)
2024-07-16T14:12:05.956626388Z     at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
2024-07-16T14:12:05.956630402Z   type: 'system',
2024-07-16T14:12:05.956633899Z   errno: 'ECONNRESET',
2024-07-16T14:12:05.956637313Z   code: 'ECONNRESET'
2024-07-16T14:12:05.956640735Z }
2024-07-16T14:12:05.956644358Z FetchError: request to http://localhost:14000/v2/route?origin=-73.572,45.5061&destination=-73.1775,45.5308&scenario_id=27ef6b8a-af95-491b-8015-e59d67f81cc1&time_of_trip=57600&time_type=0&alternatives=true&min_waiting_time=180&max_access_travel_time=1200&max_egress_travel_time=1200&max_transfer_travel_time=1200&max_travel_time=14400&max_first_waiting_time=3600 failed, reason: socket hang up
2024-07-16T14:12:05.956647814Z     at ClientRequest. (/app/node_modules/node-fetch/lib/index.js:1491:11)
2024-07-16T14:12:05.956651390Z     at ClientRequest.emit (node:events:517:28)
2024-07-16T14:12:05.956654795Z     at Socket.socketOnEnd (node:_http_client:525:9)
2024-07-16T14:12:05.956658286Z     at Socket.emit (node:events:529:35)
2024-07-16T14:12:05.956661927Z     at endReadableNT (node:internal/streams/readable:1400:12)
2024-07-16T14:12:05.956665388Z     at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
2024-07-16T14:12:05.956668740Z   type: 'system',
2024-07-16T14:12:05.956672196Z   errno: 'ECONNRESET',
2024-07-16T14:12:05.956675915Z   code: 'ECONNRESET'
2024-07-16T14:12:05.956679332Z }

During a specific calculation, this error happens around a missed checkpoint (though it is almost 1h30 later than the previous and next checkpoint. There may be a missing or misplaced try/catch when this happens. It should not miss the checkpoint, even if it happens.

After further investigation, this error occurs when trRouting is unresponsive, but in those cases, the faulty response is correctly saved in the results. The error itself should be better handled, now it results in a generic server error in the results and a log in the console. A fetch error should result in a transition error with proper code, handled by the caller.
The checkpoint miss is not related to the fetch error.
But we still need to investigate why trRouting becomes unresponsive like this sometimes. Are there too many requests ongoing at that moment?


crunch is sync

crunch was designed to be an async function but all of the logic occurs sync.

promise-ifying internals rn

this will  introduce a breaking change when using a callback with crunch - new signature will be cb(err, cert)

fixed with v0.2.0.


don't let a hyphen ruin an ident party

fixes https://github.com/dtolnay/mashup/issues/14
Was motivated to grab this because I'll likely run into this sooner for my particular mashup use case.

ha! actually I ran into this immediately after putting the latest mashup to the test for my use case.
I'm using rust to deploy serverless aws lambda functions. A motivation for seeking out a crate like mashup was the awkwardness of not being able to use concat_idents! ( or strings even if that was stable ) for providing cpython identifiers. This builds off of the crowbar lambda crate which has the same limitation so by default crate lib's for crowbar/lambda must be named lambda out of expectation.
I put mashup to the test and that works as expected... except when crate names have hyphens, just like a my serverless template repo does :)
What are the chances of this getting merged and released within the next few days? That would unblock  my confidence and pushing my original usecase forward. Less potential footguns!

thanks again @dtolnay!

following to to confirm, this fixed worked like a charm with my use case.


fix: add book card

Judging by your component folder structure, if this is a component only used for marketing/kids, is it possible to place the component in that repo? However, if it's going to be re-used outside of marketing/kids, I'd recommend placing the component alongside the others in the file structure.

Good point. This component may be used in other marketing projects, not only in marketing app (copilot for example). I moved it to the components root directory. ^


Janus FTL - sd_notify support

Rough changes to the FTL systemd unit file to use the soon to be added (?) sd_notify support.

Requires https://github.com/Glimesh/janus-ftl-plugin/commit/24d1d706863510e52cbe3feceff7954b219db3be

Will need testing, this PR is also relying on the READY state being sent so it know whether to restart/retry which is not added in the above commit. this was later added
https://www.freedesktop.org/software/systemd/man/systemd.service.html
Updated per https://github.com/Glimesh/janus-ftl-plugin/pull/99

LGTM

Hmm, I think I pulled this in from the other issue. I'm not 100% sure. It's
a timeout failure on start, so a reboot might resolve. One would assume the
service had been inoperative for the prior 4 mins (or the watchdog had
failed).
This isn't a normal failure path, to be clear, would be very edge case I
would expect (?)
On Sun, 21 Mar 2021, 12:39 Luke Strickland, @.***>
wrote:

@.**** commented on this pull request.
In ansible/roles/janus-ftl-plugin/templates/janus.service.j2
https://github.com/Glimesh/ops/pull/12#discussion_r598273332:

LimitNOFILE=65536
ExecStart=/opt/janus/bin/janus -o
+StartLimitInterval=5min
+StartLimitBurst=4
+StartLimitAction=reboot-force

Why do we want this to reboot the machine?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Glimesh/ops/pull/12#pullrequestreview-617015688, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAGDCZ3VS63WTF3F6GRC6ILTEXSJ5ANCNFSM4YXGQQLQ
.

I in turn pulled that reboot action from the blog of the systemd author as an example.
Agreed it's a very edge case scenario, though thinking more I don't think we've seen any cases where it would have helped to restart the machine so it's fine to leave that option out.
Maybe once the janus ftl service is super stable it would make more sense to add back.

I'm happy with how it is, I was more curious than anything, thanks!


Missing channels from package.yaml definition

Bug Report
Openshift 4.1
Not sure about running OLM version:
$ oc get deployment -n openshift-operator-lifecycle-manager olm-operator -o yaml | yq r - spec.template.spec.containers.*.image
- quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ca6b07b6d742088552889755160089bfde1301629f60746c453600b8a0ddebae

The issue is about declaring 3 channels (also tried with 4 channels), and dashboard only shows 2 of them. packagemanifest resource only shows 2 channels
This is the package.yaml with 3 channels
packageName: 3scale-operator27
channels:
  - name: beta
    currentCSV: 3scale-operator27.v0.4.0
  - name: threescale-2.6
    currentCSV: 3scale-operator27.v0.3.0
  - name: threescale-2.7
    currentCSV: 3scale-operator27.v0.4.0
defaultChannel: threescale-2.6

After creating the operator source, packagemanifest resource only shows two channels.
$ oc get packagemanifests -n openshift-marketplace 3scale-operator27 -o yaml | yq r - status.channels.*.name
- beta
- threescale-2.6

I was expecting to have all 3 channels.
Related issue with this issue is that when the missing channel is the one set as defaultChannel, (threescale-2.6 in the example above), then when we try to create operator subscription using marketplace UI, the screen gets white without any controls.

Has been fixed in OCP 4.1.20
https://access.redhat.com/errata/RHBA-2019:3004


Add translation for Spanish

A new JSON file containing the translation to be added to src/locales/
Mapping to the src/i18n.js must be done accordingly.
Feel free to open more issues for other languages :)

taking up.

@kausha15 Thanks for taking up this issue. It will be great if you can verify the translations with someone who natively speaks Spanish :D

@sudhanshu sure!

I am taking the Turkish one. Cheers

@kausha15 if you need a native Spanish speaker to verify the translations I can do it :smile:

@gdelacruzfdez  sure!
check file at https://github.com/sudhanshu16/openwrt-firmware-selector/pull/24/files


Possibility to generate only data contracts

Hi,
Is it possible to generate only data-contracts and skip api and http-client?
Thanks!

--no-client  option


Show how to bind to an IObservableMap with C++/WinRT

I'm having trouble binding to an IObservableMap instead of an IObservableVector, which works fine.
Binding to a single element of the map with indexer syntax in XAML works fine as well, but when I try to bind to the entire collection, I get HRESULT: 0x80070057 (E_INVALIDARG) when the ItemsSource property is set on load.
The documentation is contradictory here; the ItemsSource property page says:
The ItemsSource property value must implement one of these interfaces:
IIterable<IInspectable>
IBindableIterable

and according to C++/WinRT headers, IObservableMap<K, V> implements IIterable<IKeyValuePair<K, V>>, where IKeyValuePair<K, V> is an IInspectable, yet it doesn't work.
On the other hand the C++/WinRT collection docs say:
If you want to bind a XAML items control to your collection, then you can.
But be aware that to correctly set the ItemsControl.ItemsSource property, you need to set it to a value of type IVector of IInspectable (or of an interoperability type such as IBindableObservableVector).

According to my experience so far, the C++/WinRT docs are right and the other page should be corrected.
On the other hand, I've tried doing the same with a C# Dictionary and it worked without a problem. I know the CLR is doing a lot of interop magic, but this shows me that it's indeed possible to bind to a map, so what exactly is going on there?

Hi, @Holo-Krzysztof, the ItemsSource property topic is probably referring to C++/CX when it says "C++". I'll take a closer look and report back here when I have more info.
Also, C# is more forgiving, so getting something to work in C# shouldn't necessarily set the expectation that the same thing will work with other language projections.
Thanks!
Steve

My updates are now live, so I'll go ahead and close the issue.


WIP: Add global auth filter + policy migration example

@Rick-Anderson  edit: Moved to #16129

@pranavkm  check my commit

Looks great. @rynowak could you do a once over?

I'll use a code snippet in #16128 and them merge into this PR

@rynowak  quick look please


⚠ Bofrost.be has degraded performance

In eb96673, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 984 ms

Resolved: Bofrost.be performance has improved in 21e7374.


Chore do not check java env on clang-tidy preparation

Proposed changes

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall


Enable non-transitive R files

Task/Issue URL: https://app.asana.com/0/414730916066338/1201118368747585/f
Description
Currently our gradle is configured in such a way that all R resources are rechable from any module, without the need to specify them as a gradle dependency. This has the following cons:

compilation times are higher, as aapt has to include references to all resources in all gradle modules
incremental compilations are higher, as any resource change affects more modules than it should
APK/AAB files are bigger than needed

See asana task for a more detailed description.
This PR enables the android.nonTransitiveRClass=true in the gradle.properties file so that modules can only reference resources in the current module, or in modules explicitly added as dependency
Steps to test this PR
The app builds for all flavors

1 line change, 40 sec build improve. 💥


DFE doesn't work on Windows

E:/b/depot_tools/python276_bin/python.exe ../../build/gn_run_binary.py compiled_action dart.exe --packages=E:/b/build/slave/vm-win-release-x64-be/build/sdk/.packages --snapshot=E:/b/build/slave/vm-win-release-x64-be/build/sdk/out/ReleaseX64/gen/kernel-service.dart.snapshot --snapshot-kind=app-jit E:/b/build/slave/vm-win-release-x64-be/build/sdk/utils/kernel-service/kernel-service.dart file://E:/b/build/slave/vm-win-release-x64-be/build/sdk/pkg/compiler/lib/src/dart2js.dart
Command failed: ./dart.exe --packages=E:/b/build/slave/vm-win-release-x64-be/build/sdk/.packages --snapshot=E:/b/build/slave/vm-win-release-x64-be/build/sdk/out/ReleaseX64/gen/kernel-service.dart.snapshot --snapshot-kind=app-jit E:/b/build/slave/vm-win-release-x64-be/build/sdk/utils/kernel-service/kernel-service.dart file://E:/b/build/slave/vm-win-release-x64-be/build/sdk/pkg/compiler/lib/src/dart2js.dart
output: Unhandled exception:

FileSystemException: Exists failed, path = '/E:/b/build/slave/vm-win-release-x64-be/build/sdk/.packages' (OS Error: The filename, directory name, or volume label syntax is incorrect.

, errno = 123)

#0      main.<anonymous closure> (file:///E:/b/build/slave/vm-win-release-x64-be/build/sdk/utils/kernel-service/kernel-service.dart:110:7)

#1      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:148)

This issue is outdated: DFE works on Windows. As of today, we have bots which build and test Dart SDK in Dart 2 mode on Windows.


'charmap' codec can't encode character '\U0001f33f'

Hi,
I tried to execute facebook-ads-performance-downloader. Getting the following error.
File "c:\python36\lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\U0001f33f' in position 56: character maps to
If possible fix the issue.
Thanks,

Hello @chaitu1124, could you post the complete error ? The stack should show which operation exactly caused the problem.
It seems your system is configured to use cp1252 which does not include the character 🌿.
You can also try using this command before running the script: SET PYTHONIOENCODING=utf-8

Closing due to inactivity


set callback functions

How is it possible to create some settings in oreder that the tinymce.init is as the following?
tinymce.init({
  init_instance_callback: function (editor) {
    editor.on('blur', function (e) {
      console.log('Editor was blurred!');
    });
  }
});

I tried with
TINYMCE_DEFAULT_CONFIG = {
....
    "init_instance_callback":"""function (editor) {
editor.on('blur', function (e) {
    tinyMCE.triggerSave();
    console.log('test')
});"""

but that is then a string, not a valid funciton and it's not triggered.
My scope is to trigger an action when onblur, since using tinymce the event on input, that is: $(':input').on('change', function () .. is not triggered..

It looks like the init_instance_callback callback is not supported. Use the setup callback instead

do you have a working example for this case?

TINYMCE_DEFAULT_CONFIG = {
....
`"setup":"""function (editor) {
editor.on('blur', function (e) {
editor.save();
console.log('test')
});} """
}
The blur event in order to trigger the focus must be shifted to another input. In my projects clicking anywhere in the page does not trigger the event. As for the save, I am not using it  in my projects so I cannot test that functionality but I don't see any reason the editor.save() not to work.
If the issue is solved, please mark this ticket as closed.


Linguistic Probing and New Task

I've suggested(#562) incorporating Linguistic Probing tasks as outlined by Conneau et al.. These tasks serve as a method for evaluating embeddings regarding their ability to accurately capture linguistic characteristics within sentences. These tasks are under three different categories which are Surface Information, Syntactic Information and Semantic Information, please see the paper for further details. All probing tasks are casted under a classification task and detailed description of them are in the this page.
Discussion in PR #562
As  pointed by @x-tabdeveloping in #562 , not all datasets fall under the classification naturally, such as sentence length prediction or most significantly Word Count dataset which is 1000 way classification. Instead, we can evaluate tasks differently, such as doing word retrieval instead of classification in Word Count proposed by @x-tabdeveloping. Thus, we could discuss possible evaluation strategies for probing tasks in this issue. Please share your thoughts.
Probing as a new Tasks to MMTEB.
Regardless of the evaluation strategies, probing is completely different from the current task categories we have as beautifully put by the abstract of the paper Although much effort has recently been devoted to training high-quality sentence embeddings, we still have a poor understanding of what they are capturing. “Downstream” tasks, often based on sentence classification, are commonly used to evaluate the quality of sentence representations. The complexity of the tasks makes it however difficult to infer what kind of information is present in the representations. We introduce here 10 probing tasks designed to capture simple linguistic features of sentences, and we use them to study embeddings generated by three different encoders trained in eight distinct ways, uncovering intriguing properties of both encoders and training methods.  Therefore I propose Probing as a new task to dataset. @Muennighoff @KennethEnevoldsen @isaac-chung  @imenelydiaker and all other I have missed to tag.

I find it interesting to have probing tasks. It would be an interesting insightfor peoplewho do XAI and interpretability.
Some open questions:

Do you know any other datasets that was not proposed in this benchmark which could be added?
Is this benchmark still relevant for text embeddings? Is it used ? By which communities?
For the LB, I think this task should be shown differently, since the goal of it is not to evaluate a model over a language, but it's more about sanity checking the embeddings: validating the concepts that are encoded by the embedding model are relevant.

Could you please open a discussion instead of an issue?

I will convert this to a discussion, but feel free to keep it going


Do not show a black overlay when your addon is disabled in Firefox Nightly?

When I "disable the entire extension" as the tooltip says on the big on/off button, it's actually not disabled and slows page load speed significantly, sometimes heavily. Additionally there is a black, half opaque overlay when I load a website. Sometimes it stays there for seconds when Midnight Lizard decides to slow down page load for that long.

When you visit any website Midnight Lizard stores the latest brightness level and use it when you come back to this website next time to draw a dimmed cover to protect you from white splashes during initial page load. This cover can be seen only on less performant devices or with bad network connectivity.
So when extension is turned off the cover is shown only once when you first time load a page that was previously visited when extension was turned on. It won't be shown for the subsequent page loadings.
Even when the cover is shown it cannot decrease performance more then by 1% and when it is not shown extension has absolutely negligible impact on performance.
It should be something else that slows down your browser. Try to disable other extensions one by one to see a difference. To be sure disable them on chrome://extensions page. Also check whether you use the latest Chrome: chrome://settings/help
Midnight Lizard has a simplified mode that can significantly improve performance. Might worth to try it out as well. You can switch the mode on the Common tab and then press "Set as default" to use it on all websites by default.

I want to apologize for my bad manners in the bug report but I still think that this is a bug. Also I should have mentioned that I'm using it on Firefox Nightly.
 When the extension is turned off I want it to behave as close as possible to as if I disable it completely in the browser. And as of now this is what I have to do to get my browser to an acceptable speed. Every morning I disable it in about:addons and every evening I have to re-enable it there. It has really gotten a burden and I already use a different addon that doesn't have this problem, but also unfortunately has less features.
To clarify: The seconds of wait time are produced when your addon is enabled and aren't, when your addon is disabled.

I'm showing a cover when website opened for the first time after extension was turned off because there is no synchronous way to check whether extension is active or not. So I have to rely on the data stored locally on this website since last visit when extension was probably active. Then I can retrieve actual extension state and continue processing or just quit. Without this first preload phase you would see white screen first for a while.
But there was actually a bug that cause showing a cover all the time (not for the first time only).
I have just fixed it and will publish an update soon.
This might also fix this issue so I am reopening it until you confirm that it is fixed.

The new version has been published.
Please check it out and give a feedback here.
Thank you.

Performance is significantly better now, thank you.
From what you wrote it can probably not be fixed


Regression/Classification info

Hi,
It would be nice to have a 3rd column for data() output indicating whether the dataset can be used for regression or classification problems.

Hi @ogencoglu sounds like a cool idea, thanks. Any thought on how to approach clustering them ?

I think it is just manual work. Not all datasets may be suitable for this but many machine learning people search for datasets to try their algorithms/implementations in a smaller scale before going to well-known benchmark datasets.

agreed it'd be nice to filter for regression or classification, but dont see how you could properly categorize datasets.
a regression dataset could be a classification dataset, and vice versa, depending on your preprocessing strategy (eg binning) and target feature.
for example, the canonical iris dataset, used for classification, could be viewed as regression too.

My idea was something similar to UCI data repo:
http://archive.ics.uci.edu/ml/datasets.html
The column can be "Default Task". Some datasets may have even both Classification and Regression.


Question RE: Usage with gitlab-ce in 2024

Sorry not an "issue" per se, but as an ignoramous... would like to know whether the gitlab-ce of 2024 supports standard LDAP auth and partially obsoletes the functionality offered here? We sync with AD.
Context is an older build with this incorporated by a departed individual and our requirement to rebuild the host. Hoping to shave some technical debt to make maintenance a little easier for any successors.
Cursory Google with the time afforded so far, has me at a pre-emptive conclusion of Gitlab supporting LDAP on the community edition as of 2024 at least.
Will inevitably have to do some more thorough investigation as go-time approaches.
Appreciate your patience and thanks in advance.

Hello,
Good question, hopefully this answers:
Gitlab CE has always (for all the years I've used it) supported LDAP authentication directly without this tool, this was never a requirement for that benefit of centralised authentication. You could configure details to reach your directory, signing in would "just work", and you would have a user entity in your Gitlab instance at that point.
The purpose of this tool was to add some features to improve the LDAP experience from a system administration point of view:

Synchronise users into Gitlab in advance, so people can be assigned to groups and projects prior to their first sign in
Synchronise changes to users and groups in advance, e.g. name/email/lock changes, either on a periodic (cron) basis or when run again manually
Synchronise groups and their user memberships (This was always an EE only feature)

I've got 3 instances of Gitlab CE that still make use of this tool as to my knowledge I've not seen all of the above available natively. If those features aren't necessary for your Gitlab then you won't need this tool. There is nothing AD specific to this tool either, it's quite back end agnostic.
If you're looking to reduce technical debt it may be worth your time to convert this tool to a different scripting platform you feel would be more "LTS". Frankly all this tool does is look at data on both sides, compare them, and make the necessary writes on the Gitlab side to match the directory, nothing ground shattering. Its not that I expect PHP or the Symfony components I'm using will be going away any time soon (php is dead meme), though either Perl or Python would likely be good candidates for a conversion. There is of course the growingly popular NodeJS option though that seems to me like it would be more technical debt than PHP by a long way as it still has multi-version compatibility considerations, 3rd party libraries with security considerations, and EoL considerations.
Hope that helps :)

Hi @Adambean
Thank you, absolutely great response. Really appreciate the insight.
Certainly did see vestiges of sync from the bash history, but not much else.
Will weigh options while in the weeds of the rebuild. Given that it's not entirely dependent, may be inclined to migrate it across. Figure it won't really hurt provided documentation accompanies (lol ikr?)
Cheers. Am satisfied with the response, so will close this issue off.
Thank you for your time.


Unknown props warning with React 15.3.0

Using the plugin with React 15.3.0, I get the following warning in the console:
Warning: Unknown props formatValue, stepPrecision, duration, component, frameStyle on <span> tag
A brief Google suggests this kind of warning is common with recent versions of React. However, I easily fixed it for my use case by changing the 3rd arg of createElement from this.props to {}, on line 152 of AnimatedNumber.js. So far as I can see this will have no adverse side effects.

Unfortunately, the fix isn't that straightforward.
this.props is being passed to createElement because we need to transfer props like even handlers and data-* attributes to the underlying node that contains the number.
We will need to manually filter out invalid props.

For posterity:
This was fixed by filtering out props that are defined in the component's propTypes.
Need to find a more reliable way of doing this.


IEnumerable is not working

Let's say you have the following [ResponseType(typeof(UserEntry))]
UserEntry is the following :
public class UserEntry
	{
                public IEnumerable<string> Tokens { get; set; }
                public IEnumerable<Application> Apps { get; set; }
	}

Then in swagger-ui, for the model I have
"tokens": [
    "string"
  ],
"apps": [
    {
      "id": "string",
      "name": "string",
      "url": "string"
    }
  ]

So far so good (it seems all good i nthe json model too) but now when I click on the Model, I have
tokens(string, optional)  //here is problematic
apps(Array[Application], optional) //here is ok

whereas I should have
tokens(Array[string], optional)
apps(Array[Application], optional)

This is with swashbucle 5.0.0 (5.0.1 seems to reproduce the problem too).
Is this only a problem from swagger-ui or also from swashbuckle, I can not say...

Hi there
Thanks for the update - it's a straightforward and repro and looks to me like a bug in the swagger-ui. Would you mind creating an issue there and referencing this one?
At it's core SB is a Swagger generator. It just comes bundled with infrastructure to serve an up-to-date version of swagger-ui through your WebApi. With this in mind, it's a pretty good rule of thumb to say if the raw Swagger doc looks correct but the ui is behaving incorrectly, then it's likely a bug in the swagger-ui. Not definitely, but likely :)
Cheers

BTW, in the latest version of swagger-ui you packaged in swashbuckle5.0.1, there is a more critical bug because when you click on model there is nothing displayed at all...
So you will have 2 reasons to upgrade to a newer version of swagger-ui :)

Appears to fixed with the latest swagger-ui which I've bundled into a SB pre-release (5.1.0-rc1). Can you please check it out and make sure everything works as expected ...
Update-Package Swashbuckle -Pre


ROS vs ROS2

http://design.ros2.org/articles/changes.html

https://index.ros.org/doc/ros2/Concepts/About-ROS-Interfaces/

https://design.ros2.org/articles/qos.html


refactor(share/ipld): Use NamespaceSize var for consistency

self-explanatory

Codecov Report

Merging #1297 (03458fb) into main (6adfd8a) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1297   +/-   ##
=======================================
  Coverage   55.75%   55.75%
=======================================
  Files         170      170
  Lines       10254    10254
=======================================
  Hits         5717     5717
  Misses       3966     3966
  Partials      571      571

Impacted Files
Coverage Δ

share/ipld/nmt.go
48.43% <ø> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Implement approve threshold for license and vote

Implement approve threshold for license and vote
Add approve threshold/time to listproperties RPC

#12  #13

#23


Feature/clo-55-move-to-wasi

Get rid of the plugins json spec and move to loading WASI compiled modules.
The validation is done by calling the default exported function now.
The plugin spec resides in cloudflavor/skyforge.
Signed-off-by: Victor Palade<EMAIL_ADDRESS>

/triage
/assign

/approve

/merge


multiple column drag into fieldset

I am using form.io with my angular 7.2.0.
Everything is working fine but i am facing one minor issue regarding adding multiple column into fieldset layout.
First add fieldset layout in from.
Then Add column inside fieldset.
When we add second column in that fieldset without changing label, it is replacing first column.
Same issue not produced on form.io angular demo.
Please check issue at following git repo:
https://github.com/bhupendrapatel/formio-demo

We're currently addressing a backlog of GitHub issues, and as part of this effort, some inactive issues may be marked as closed. This isn't a dismissal, but a step toward more efficient tracking.
If you feel the issue is still relevant, please re-open and we'll ensure it gets the attention it deserves. Your understanding is appreciated as we work to enhance our open-source responsiveness.


Task highlighting

#160

Highlight the task(s) just added/marked/unmarked/renamed/scheduled
Highlight existing tasks that already exist when adding some duplicates
Add short key ctrl+z for undo
Remove the placeholder of time for floating tasks
Change the shortcut key of opening of help window from F5 to ctrl+H

Gui tests fail, going to investigate on it.

@fanwgwg Mind attaching screenshots here of how the new elements look like?

This is how it looks like after the changes. @burnflare Is this fine?

Ooo nice, variable cell heights. If I have a title that's 2 lines long will it be taller too?

Yup it will resize automatically!

Actually, it is showing the last updated time instead of the completion time.
So if you rename a completed task, the time will changed as well >.<
I still think it will be useful to show the time of events/deadlines that have passed

Fixes #117, #181, #183, #184
Revise #160 after update-model

@rachx @fanwgwg Is this good to go in?


[QUESTION/FEATURE] (If within scope of project) Provide further documentation of leveraging Brickflow for triggering workflow deployments outside of Databricks

Is your feature request related to a problem? Please describe.
Reading through the project, it appears that Workflow deployments are preferred for execution within Databricks environment (phrasing of docs/highlevel.md - ...help deploy and manage workflows *in* Databricks.). Would like to leverage this framework to handle deployments coming from an execution environment such as Github Actions/another Serverless execution function(?).
Cloud Information

[ ] AWS
[ ] Azure
[ ] GCP
[ x ] Other

Describe the solution you'd like
To fit the mantra of code/git first deployments, it would be helpful to further elaborate on deployments outside of the Databricks environment (leveraging GitHub Actions to handle deploys less state-dependent on the compute engine, and more state-dependent of the generated workflows).
IF deployments leveraging such an external compute environment are achievable, further documenting deployment execution with this framework will greatly improve onboarding for interested end-users.
Describe alternatives you've considered & Additional context
I am currently using a YAML configuration framework that determines deployment go/no-go for projects by leveraging process-generated tags on workflow object, which look for whether or not the tag in the workflow's repo configuration matches the tag tied to the active workflow in Databricks (if tags match, skip deployment of the workflow. If tags are different, deploy workflow by overwriting the existing workflow in Databricks). Because the evaluation happens at runtime, there is no additional state capture required as the versions are captured in-code.
^^ REASON FOR CASE SUMMARY: I want to know if I can employ this framework in a similar manner to the above situation, where configurations happen outside of Databricks and deployment state details captured in name/tags of the active workflow can drive new workflow creation/updates/deletions.
Additional context
This is more of a question ticket for the project as much as it is a feature request. Am looking to understand whether or not it is feasible to employ this framework as the engine for our workflows given the criteria it needs to meet for my use case.

@AlecVivian I think what we meant in the documentation is brickflow is used to manage workflows which exists in Databricks. But essentially Brickflow is a library which has a cli through we do the deployments and manage workflows.
So the cli can be run any where, be it jenkins or github-actions or docker container etc. As long as you have your Databricks token configured, you should be good.

@AlecVivian if you still feel that the question is not answered, please re-open the ticket


[BUG]400 Bad Request: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]

Hello, team! I am trying to send cpu metrics from fluentbit to opensearch. Getting error:  [2022/10/25 16:10:07] [error] [output:opensearch:opensearch.0] HTTP status=400 URI=/_bulk, response:
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"}],"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"},"status":400} (edited)
4:15
Fluentbit command is : ./fluent-bit -i cpu -t cpu -o opensearch -p Host=<IP_ADDRESS> -p Port=9200 -p Index=my_index -m '' -o stdout -m '' -f 5 (edited)
4:18
I tried to configure the opensearch using "compatibility.override_main_response_version" : true; ref: https://github.com/opensearch-project/OpenSearch/issues/3484; still no use. I am getting the error. Please help me

This is a dup, see https://github.com/opensearch-project/OpenSearch/issues/4864#issuecomment-1287284114 for details and next steps.


The same link shows an error in kiwi, but other browsers don't.

Describe the bug
A clear and concise description of what the bug is.
The effect of opening this link(https://www.aliyundrive.com/s/fqu79Z2dBbu/folder/638720ab185b2cf913ce4f438aacd4395844098b)in kiwi is different from other browsers. Even if I switch kiwi to a desktop browser, I still get an error.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
Show the same web content as the picture second it should be
Screenshots
If applicable, add screenshots to help explain your problem.
this is in the kiwi
this is in the other browser
Smartphone (please complete the following information):

Device: [e.g. RMX2121L1]
Android Version : [e.g. A12 ]
Kiwi Browser Version [ e.g. 120.0.6099.23]

Additional context
Add any other context about the problem here.

I also encountered a issue that did not appear in other browsers  but only appeared in kiwi browser, I can't log in to the mailbox in kiwi browser


Event notifications and related permissions disappear

Found that event notifications can get lost when you convert.  Found that you have to re-apply connect permissions.  Note, ensure broker got copied over, I have seen where it has not.
 Thank you. Can you give me steps to reproduce? Some T-SQL from start to finish would be great.

(js0505)

During my next migration from a system that contains EN I will work on generating the script for you. We are doing migrations all month so hopefully next week.

(Chrissy LeMaire)

@js0505 when you say " ensure broker got copied over", is that the Broker Enabled or something else? I don't use Event Notifications, but they look like they are scoped at the DB. Is this accurate? Are they usually copied properly during backup/restore?

(js0505)

Broker Enabled, and yes "usually" everything gets copied over, however, there are times I have to GRANT CONNECT ON::ServiceBroker TO ACCOUNT on the endpoint server.

@wsmelton, @ctrlbold : this did ever got fixed ? if not, it'd be time to, just to not keep lingering issues too much.

Closing. If this comes up during cleanup party we can just create a new issue.


add support for new version of flask-login and flask-security

Fixes #1116

Changes is_active() and is_authenticated() to properties
Removes specific versions specified in requirements.txt files

We need to pin version (using >=) because we broke compatibility with older Flask-Login version: .is_authenticated is method and boolean comparison will always return True.

@mrjoes Good catch. Just changed it.


Could not copy app.so to /cpu1/exe/cf/apps

Hi
When I delete all app.so in this fold, then I make clean, make config and make again, there are not any app.so in this fold? Is this right? Thanks!
Sincerely

Are you using the cmake build system?  Try doing "make install" after doing "make".

Hi
If I did "make install" after "make", it said that:
make: *** No rule to make target 'install'.  Stop.

I edit the cfs.mak file and add .so for it, it worked now! Thanks