Build deeplabv3 from pytorch in host docker error

This is my first post and I am new to TVM and Pyxir/Vitis AI, please excuse if my question seems silly.
Trying to build deeplabv3 from pytorch to deploy on top of xilinx EDGE device - zcu104. Facing some issue while annotating. Code snippets here:
mod, params = relay.frontend.from_pytorch(scripted_model, shape_list)
mod["main"] = bind_params_by_name(mod["main"], params)
desired_layouts = {'nn.conv2d': ['NHWC', 'default'], 'nn.adaptive_avg_pool2d':['nn.avg_pool2d', 'default']}
seq = tvm.transform.Sequential([relay.transform.RemoveUnusedFunctions(),
                                relay.transform.ConvertLayout(desired_layouts),
                                relay.transform.FoldConstant()])
with tvm.transform.PassContext(opt_level=3):
    mod = seq(mod)
mod = annotation(mod, params, target) #shows error here

Two doubtful things in output:

/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l10_temporary.py:64: UserWarning: Convert Relay Adaptive Avg pool2d layer into normal average pool2d layer warnings.warn(“Convert Relay Adaptive Avg pool2d layer into normal”
File “/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/graph/ops/l1_basic_nn.py”, line 167, in concat assert i == axis or len(check) == 1 AssertionError

How to proceed further? Is there any way to change Adaptive Avg pool2d layer to normal average pool2d layer using relay.transform?
Is there any tutorials any one aware of building deeplabv3 for Xilinx EDGE device (from mxnet or pytorch or any other framework)?

@abdulazizm Hi, I noticed that the conda you used is vitis-ai-tensorflow. If it's a pytorch model, you could have a try to use vitis-ai-pytorch.
For more details, you could refer to https://www.xilinx.com/html_docs/vitis_ai/1_3/deploying_running.html#zgy1576168058789

@wangxd-xlnx OOPS! Thats a good catch. Will give it a try with vitis-ai-pytorch environment and come back. I followed this tutorial -> https://xterra2.avnet.com/xilinx/ml/tvm-on-xilinx-mpsoc/-/tree/master where it builds mxnet framework models in vitis-ai-tensorflow environment (I assumed that the same shoud be working for pytorch too).
Thanks for your reply!!

@wangxd-xlnx I tried with vitis-ai-pytorch environment, but getting the same assertion error.
/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l10_temporary.py:64: UserWarning: Convert Relay Adaptive Avg pool2d layer into normal average pool2d layer
warnings.warn("Convert Relay Adaptive Avg pool2d layer into normal"
Traceback (most recent call last):
File "compile_pytorch_deeplab.py", line 189, in
mod = annotation(mod, params, target)
File "/workspace/python/tvm/relay/op/contrib/vitis_ai.py", line 92, in annotation
xgraph = pyxir.frontend.tvm.from_relay(mod, params, postprocessing=None)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay.py", line 58, in from_relay
cvx_preprocessing=cvx_preprocessing
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_2_xgraph_converter.py", line 96, in from_relay_to_xgraph
cvx_prep=cvx_preprocessing)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 75, in function
op_idx, RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 184, in tuple_expr
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 117, in call
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_2_xlayer_registry.py", line 122, in __base_relay_2_xlayer
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 117, in call
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_2_xlayer_registry.py", line 122, in __base_relay_2_xlayer
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 117, in call
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l1_basic.py", line 78, in add
**kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 117, in call
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l2_convolution.py", line 216, in nn_conv2d
**kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 117, in call
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_2_xlayer_registry.py", line 122, in __base_relay_2_xlayer
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 239, in tuple_get_item
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 117, in call
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l1_basic.py", line 239, in nn_batch_norm
**kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 117, in call
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l2_convolution.py", line 216, in nn_conv2d
**kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 239, in tuple_get_item
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 117, in call
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_2_xlayer_registry.py", line 122, in __base_relay_2_xlayer
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 117, in call
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_2_xlayer_registry.py", line 122, in __base_relay_2_xlayer
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 239, in tuple_get_item
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 117, in call
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l1_basic.py", line 239, in nn_batch_norm
**kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 117, in call
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l2_convolution.py", line 216, in nn_conv2d
**kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 117, in call
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_2_xlayer_registry.py", line 122, in __base_relay_2_xlayer
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l0_expr_and_others.py", line 117, in call
RELAY_2_XLAYER, **kwargs)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/frontend/tvm/relay_tools/relay_l1_basic.py", line 414, in concatenate
X = px.ops.concat(op_name, data_layers, axis, relay_id=relay_idx)
File "/home/vitis-ai-user/.local/lib/python3.6/site-packages/pyxir-0.1.6-py3.6-linux-x86_64.egg/pyxir/graph/ops/l1_basic_nn.py", line 167, in concat
assert i == axis or len(check) == 1
AssertionError
Note: Tried replacing mod= annotation(mod, params, target) with mod = relay.transform.AnnotateTarget(target)(mod) - can able to generate params, so files but not sure whether this helped me with the functionality to run in DPU. Any suggestions?

as far as i know, the tvm package is built only in the tensorflow conda environment if the default dockerfile is used to build the tvm_vitis-ai docker image. So could you tell me did you rebuild the tvm in the pytorch conda environment ? What i did is to simply install torch in the tensorflow conda environment as tvm is hard to compile. but still, i met with some problem to compile a basic pytorch model in the torchvison model zoo(resnet18). i failed with the relay.frontend.from_pytorch api while succeed with relay.frontend.from_onnx api after transfer the pytorch model to a onnx model. i still dont know why the failure happended. did you problem fixed?


How to change .NET version?

How do I use the additional .NET version installed by the dotnet feature?  For example, if the feature is configured such as:
 "ghcr.io/devcontainers/features/dotnet:1": {
            "version": "7.0.306",
            "installUsingApt": "false",
            "additionalVersions": "6"
 }

How do I use .NET 6?  When I run dotnet --list-sdks, only 7.0.306 [/usr/local/dotnet/7.0.306/sdk] is shown.
I can see 6.0.415 is present.
>ls -la /usr/local/dotnet/
total 24
drwxrwsr-x 1 codespace dotnet 4096 Oct 17 22:34 .
drwxr-xr-x 1 root      root   4096 Oct 17 22:44 ..
drwxrwsr-x 8 codespace dotnet 4096 Oct 17 22:34 6.0.415
drwxrwsr-x 1 codespace dotnet 4096 Oct 23 18:51 7.0.306
lrwxrwxrwx 1 codespace dotnet   25 Oct 17 22:34 current -> /usr/local/dotnet/7.0.306

My PATH environment variable seems to include only /usr/local/dotnet/current.

Hi 👋
Can you try with version 2 of the dotnet Feature? Eg. https://github.com/devcontainers/features/tree/main/src/dotnet#configuration-examples
It has the fix for the issue you are seeing ^. We recently had an amazing PR contribution from the community (kudos to @sliekens ✨ )
Eg. 👇


you-discovered marker does not add discovered object

This seems to be fixed? Though I can't point my finger to in which commit.


Docks appear on top of full screen windows

Problem Description
Docks/strut windows that are started AFTER a process with a window that later goes fullscreen appear on top of the fullscreen window. I was not able to determine if this issue is specific to chrome.
See: http://imgur.com/a/s55Bg
Configuration File
https://gist.github.com/IvanMalison/ddfc7c474859f227ea101def10de3e4d
module Main where

import XMonad
import XMonad.Hooks.EwmhDesktops
import XMonad.Hooks.ManageDocks

main =
  xmonad .
  docks .
  ewmh $ def { handleEventHook = handleEventHook def <+> fullscreenEventHook
             , layoutHook = avoidStruts $ layoutHook def
             }

Checklist

[x] I've read CONTRIBUTING.md

[x] I tested my configuration with xmonad-testing

I think, this is expected: docks are unmanaged by xmonad (in docks) and restackWindows, which xmonad uses in windows, will not change window order relative to windows not listed in the call (i.e. unmanaged). That means, this issue should be reproducible with just toggling dock struts off: application window should go under docks. @IvanMalison , please, check this.
To fix this we may call lowerWindow for all docks before unmanaging them. But this may have side-effect of docks going under desktop (e.g. nautilus). Or just restack docks below all managed windows before unmanaging them. I may write a PR, if there will be a decision how to deal with this. @geekosaur , @pjones any thoughts?

I occasionally run into this as well if I need to restart my docks for some reason. Restacking them below managed windows seems like a good idea to me.

I need this too. Does anyone have an easy workaround until this gets implemented?

I've no time to finish this, sorry. I'll try to look at this again and may be remember how i wanted to fix that in 1-2 weeks.

@sgf-dma Any thoughts on this?

Losing docks on restart suggests either they missed docksStartupHook or that that or its interaction with avoidStruts is buggy.
The original problem is more likely to be a bug in the dock; xmonad doesn't have a say in, or a way to communicate with, owners of struts. And while EWMH-style fullscreen is under xmonad's control, if something is using old-style (possibly because xmonad can't know when to add _NET_WINDOW_STATE_FULLSCREEN to _NET_SUPPORTED) it may put the fullscreen window in its original Z-order — and if it's override_redirect, xmonad won't know it was fullscreened.
There are also some potential problems from docks being completely self-managed vs. EWMH layers: even in window managers that support layers directly there is reliance on everything obeying layers that only the window manager knows the full story about, because docks and such are still self-managing as to Z-order. These issues may be slightly more obvious in xmonad, but will show in others sometimes.

@geekosaur This isn't about losing docks on restart.
What's wrong with the idea that sgf-dma suggested of lowering docks before they are unmanaged.

because docks and such are still self-managing as to Z-order.

What exactly is meant by this?

I appear to have crossed this with another ticket in the recent barrage of comments. (That one being deeply confused as to what's doing what.)
Window managers don't tell docks where and how to draw; docks tell window managers how to avoid whatever they did. But don't include which windows are to be considered "on top". EWMH's layers are supposed to avoid collisions there, but are at most advisory and don't work if clients don't specifically check or make incorrect assumptions. I see bugs in this area in other window managers as well.

Hi.
Sorry for disappearing for so long time, i haven't any free time to look into
this.
Here is the code demonstrating the problem, as i understand it.
import XMonad
import XMonad.Operations
import XMonad.Util.EZConfig
import XMonad.Util.DebugWindow
import XMonad.Hooks.DebugStack
import XMonad.Hooks.ManageDocks
import qualified XMonad.StackSet as W
import XMonad.Hooks.EwmhDesktops

import Control.Monad
import Control.Exception
import Numeric
import Data.List

main :: IO ()
main = do
        let xcf = docks $ def
                    { modMask = mod4Mask
                    , logHook = debugSt >> logH >> logHook def
                    , layoutHook = avoidStruts (layoutHook def)
                    , handleEventHook = handleEventHook def <+> fullscreenEventHook
                    }
                    `additionalKeys`
                    [ ((mod4Mask, xK_b), sendMessage ToggleStruts)
                    , ((mod4Mask, xK_x), spawn "xmobar ~/.xmobarrcAlt")
                    , ((mod4Mask, xK_m), spawn "chromium")
                    , ((mod4Mask, xK_r), refresh)
                    ]
        xmonad xcf

-- Change order of `debugStack` output to natural..
debugSt :: X ()
debugSt = debugStackString >>= trace . unlines . reverse . lines

logH :: X ()
logH    = withWindowSet $ \ws -> do
    trace "Tree..."
    d <- asks display
    r <- asks theRoot
    (_, _, ts) <- liftIO $ queryTree d r
    ls <- mapM debugWindow ts
    trace (intercalate "\n" ls)
  where
    showWs :: [Window] -> ShowS
    showWs      = fmap (intercalate "\n") . mapM showHex

After line Tree.. the content of X server window stack (queryTree) will be
shown.
First i start chromium:
* 01200001 "New Tab - Chromium" 1918x1078+1@0,0 chromium-browser/Chromium-browser<EMAIL_ADDRESS>Workspace "1" (current)

Tree...
[00400001 10x10@-20,-20]
[00a00001 "ibus-x11" 10x10@10,10 ibus-x11/Ibus-x11<EMAIL_ADDRESS>[00a00003 "ibus-xim" 1x1@0,0<EMAIL_ADDRESS>[00c00001 "ibus-ui-gtk3" 10x10@10,10 ibus-ui-gtk3/Ibus-ui-gtk3<EMAIL_ADDRESS>[00c00007 "IBus Panel" 200x200@0,0 ibus-ui-gtk/ibus-ui-gtk<EMAIL_ADDRESS>[01000001 "chromium" 10x10@10,10 chromium/Chromium<EMAIL_ADDRESS>[01000003 "chromium" 200x200@0,0 chromium/Chromium<EMAIL_ADDRESS>[0100001e "chromium" 200x200@0,0 chromium/Chromium<EMAIL_ADDRESS>01200001 "New Tab - Chromium" 1918x1078+1@0,0 chromium-browser/Chromium-browser<EMAIL_ADDRESS>[01200005 1x1@0,0]

then i start xmobar
* 01200001 "New Tab - Chromium" 1918x1078+1@0,0 chromium-browser/Chromium-browser<EMAIL_ADDRESS>Workspace "1" (current)

Tree...
[00400001 10x10@-20,-20]
[00a00001 "ibus-x11" 10x10@10,10 ibus-x11/Ibus-x11<EMAIL_ADDRESS>[00a00003 "ibus-xim" 1x1@0,0<EMAIL_ADDRESS>[00c00001 "ibus-ui-gtk3" 10x10@10,10 ibus-ui-gtk3/Ibus-ui-gtk3<EMAIL_ADDRESS>[00c00007 "IBus Panel" 200x200@0,0 ibus-ui-gtk/ibus-ui-gtk<EMAIL_ADDRESS>[01000001 "chromium" 10x10@10,10 chromium/Chromium<EMAIL_ADDRESS>[01000003 "chromium" 200x200@0,0 chromium/Chromium<EMAIL_ADDRESS>[0100001e "chromium" 200x200@0,0 chromium/Chromium<EMAIL_ADDRESS>01200001 "New Tab - Chromium" 1918x1078+1@0,0 chromium-browser/Chromium-browser<EMAIL_ADDRESS>[01200005 1x1@0,0]
[01600001 1x1@0,0]
[01400006 1x1@0,0]
01800002 "xmobar" 1920x19@0,1061 xmobar (25662)

as you can see, xmobar is higher in X server stack, than chromium. But
it's already ignored by xmonad, so restackWindows run in windows can't
place chromium on top of it. In this situation full screen chromium window
will go under xmobar. I.e. this bug occurs.
Then i start another new window (xterm).
* 01c0000f "xterm" 958x1059+1@0,0 xterm/XTerm<EMAIL_ADDRESS>  01200001 "New Tab - Chromium" 958x1059+1@960,0 chromium-browser/Chromium-browser<EMAIL_ADDRESS>Workspace "1" (current)

Tree...
[00400001 10x10@-20,-20]
[00a00001 "ibus-x11" 10x10@10,10 ibus-x11/Ibus-x11<EMAIL_ADDRESS>[00a00003 "ibus-xim" 1x1@0,0<EMAIL_ADDRESS>[00c00001 "ibus-ui-gtk3" 10x10@10,10 ibus-ui-gtk3/Ibus-ui-gtk3<EMAIL_ADDRESS>[00c00007 "IBus Panel" 200x200@0,0 ibus-ui-gtk/ibus-ui-gtk<EMAIL_ADDRESS>[01000001 "chromium" 10x10@10,10 chromium/Chromium<EMAIL_ADDRESS>[01000003 "chromium" 200x200@0,0 chromium/Chromium<EMAIL_ADDRESS>[0100001e "chromium" 200x200@0,0 chromium/Chromium<EMAIL_ADDRESS>[01200005 1x1@0,0]
[01600001 1x1@0,0]
[01400006 1x1@0,0]
01800002 "xmobar" 1920x19@0,1061 xmobar (25662)
01200001 "New Tab - Chromium" 958x1059+1@960,0 chromium-browser/Chromium-browser<EMAIL_ADDRESS>01c0000f "xterm" 958x1059+1@0,0 xterm/XTerm<EMAIL_ADDRESS>[01c0001c 1x1+1@0,0]
[00a00005 1x1+1@0,0]

Because xterm window is mapped after xmobar window, it appears on top of
it in X server stack. Thus, running restackWindows will bring up chromium
window on top of xmobar too. Now fullscreen chromium window will go over
xmobar, as it should. But this workaround will not always work. It relies on the
order of windows in a list rs returned by runLayout in windows
(assuming, there are no floating windows). Particularly, considering Tall
layout, new window must be leftmost (last element in up) for this fix to
work, i suppose.
Regarding the fix, i don't yet try to implement it, but using lowerWindow on
all docks before unmanaging them is probably the wrong choice, because then a
dock may go under desktop (i.e. when file manager used to manage desktop). I
think, we may restack a new dock below all managed windows before unmanaging
its, or keep a list of all docks and restack them together below all managed
windows..

I'll look this over later, but debugStack doesn't involve XQueryTree; it's reporting the Stack associated with the current or all workspaces, from the current windowset.

And here is proof of concept fix:
import XMonad
import XMonad.Util.EZConfig
import XMonad.Util.DebugWindow
import XMonad.Hooks.DebugStack
import XMonad.Hooks.ManageDocks
import qualified XMonad.StackSet as W
import XMonad.Hooks.EwmhDesktops

import Data.List

main :: IO ()
main = do
        let xcf = def
                    { modMask = mod4Mask
                    , layoutHook = avoidStruts (layoutHook def)
                    -- Lower dock before unmanaging. The position of
                    -- `lowerDock` in `ManageHook` does not matter, because it
                    -- only runs X actions.
                    , manageHook = lowerDock <+> manageDocks <+> manageHook def
                    , handleEventHook = fullscreenEventHook <+> docksEventHook <+> handleEventHook def
                    , startupHook = docksStartupHook <+> startupHook def
                    }
                    `additionalKeys`
                    [ ((mod4Mask, xK_b), sendMessage ToggleStruts)
                    , ((mod4Mask, xK_x), spawn "xmobar ~/.xmobarrcAlt")
                    , ((mod4Mask, xK_m), spawn "chromium")
                    , ((mod4Mask, xK_r), refresh)
                    ]
        xmonad xcf

-- Change order of `debugStack` output to natural..
debugSt :: X ()
debugSt = debugStackString >>= trace . unlines . reverse . lines

logH :: X ()
logH    = do
    trace "Tree..."
    d <- asks display
    r <- asks theRoot
    (_, _, ts) <- liftIO $ queryTree d r
    ls <- mapM debugWindow ts
    trace (intercalate "\n" ls)

-- | Restack dock under lowest managed window.
lowerDock :: ManageHook
lowerDock = checkDock --> do
    w <- ask
    mlw <- liftX $ findLowest
    case mlw of
      Just lw   -> liftX $ do
        d <- asks display
        s <- debugWindow lw
        trace "_Before_ restacking dock:"
        logH
        trace $ "Lowest managed window: " ++ s
        liftIO $ restackWindows d [lw, w]
        trace "_After_ restack dock:"
        logH
        return idHook
      Nothing   -> return idHook

-- | Find lowest managed window.
findLowest :: X (Maybe Window)
findLowest  = withWindowSet $ \ws -> do
    d <- asks display
    r <- asks theRoot
    (_, _, ts) <- liftIO $ queryTree d r
    return (find (`W.member` ws) ts)

This fix (lowerDock function) restack-s dock below lowest window
managed by xmonad before it'll become unmanaged (by doIgnore in
manageDocks). For now, i just rewrite docks as separate three functions
and add lowerDock before manageDocks.
@IvanMalison , @dkasak , @de-vri-es , please, try this. You need to unwrap docks, add lowerDock to
manageHook and copy lowerDock and findLowest functions.

queryTree is a rather expensive function; if possible, try to find a way to do it using the existing Stack.

@sgf-dma I just tried your POC and I confirm that it fixes the issue.

@geekosaur I don't think this is possible with Stack, because it just
doesn't have needed information.
The window order is affected by program start order and used layout (windows
runs restackWindows with a list generated by runLayout) and floating
windows order. But considering that each workspace has different floating
windows and may have different layout, i don't know how we can predict window
order in X stack. And even if we can, i think, it'll be both very complex and
unreliable.
There're two simple options, though:

lowerWindow. It does not require to run queryTree, but with this
approach we can easily end up with dock placed under desktop (e.g. under
nautilus window managing desktop). In theory, it's possible to workaround
this by keeping track in xmonad of all dock windows (including desktop) and
restack them in correct order after lowering one of them. Whether this is
complex or not, it'll require from the user to start all docks after xmonad
has started, so we can detect them in ManageHook and record somewhere.

restackWindows. The one, i've implemented. It's far simpler, than first,
but requires queryTree for finding lowest window in X stack. But we run
this only once for each new dock. I.e. this will be 2-3 times per whole
user session. I don't think this will create any performance problems.

Managing all docks within xmonad (the 1st approach) also has some advantages:
we may enforce correct dock order and overlapping in X stack.  And e.g. even
specify some "weight" for docks, which should affect dock placement during
restackWindows.. but, i think, this is too much just for fixing this issue.

You may need to duplicate XMonad.Operations.windows to do it, but note that that does XRestackWindows with an order generated from the Stack. (Which includes floating windows — which is why they behave so obnoxiously. The tiled window list is generated by an application of (\) to remove the contents of floating.) So windows always restacks all visible, managed windows, and duplicating its logic will give you the full stacking order.
This does miss things like desktop windows, but crawling up the stack checking window props is much lighter weight than XQueryTree. Or just checking if the window is in the StackSet, which also avoids that X11 roundtrip.
At X11 protocol level, XQueryTree does not return an array; it's a separate X11 event per returned window, plus some extra "overhead" events. Go look at how it's implemented in xcb which exposes the protocol level to C instead of hiding the individual events like xlib does. The whole point of the ManageDocks redesign in 0.12 was to get rid of doing this every time the layout was run, because with more than a handful of mapped windows it's visibly very slow; but you're bringing that back again. If we really do have to XQueryTree every time, then just revert ManageDocks back to 0.11 where it was simple and reliable and as slow as it will be with this change.
BTW, there actually is a reason we might want to go ahead and partially manage dock and desktop windows: proper EWMH layer support requires it. (We currently fake it, and there are cases when the desktop window can float over other windows as a result, granting that this usually requires whatever opened the desktop window to have certain bugs.) It would also provide the necessary hooks for better float management at some point, since adding a proper floating layer would be almost trivial once layers are there for desktop and dock windows.

Well, the above is actually overstating a bit because 0.11 also did two property queries for every mapped window every time the layout was run. I think this can still be avoided, but possibly not since changing it to only check windows not in the Stack is part of what caused the subsequent dzen breakage. (More precisely, it was being treated as a flxed-size window and marked as a float instead of a dock, because it didn't have the _NET_WINDOW_TYPE_DOCK property when initially mapped, IIRC. This worked before because it was in effect treated as a _NET_WINDOW_STATE_ABOVE dock, as long as it didn't accept focus.)
This all ends up being expensive to do properly unless you bite the bullet and add layer support so you can properly place windows in layers as part of restacking. Or switch to raiseWindow / lowerWindow instead of restackWindows, but that will require even more window accounting we don't do and will require a lot more support to get right, on top of requiring a complete redesign of X.O.windows. (And make layer support mandatory instead of letting us fake it as now.)

You may need to duplicate XMonad.Operations.windows to do it, but note that
that does XRestackWindows with an order generated from the Stack. (Which
includes floating windows — which is why they behave so obnoxiously. The
tiled window list is generated by an application of () to remove the
contents of floating.) So windows always restacks all visible, managed
windows, and duplicating its logic will give you the full stacking order.

I don't think so. There're unmapped windows on other (non visible workspaces),
which are not restacked at windows run. Even if i copy the windows logic
here, i only can find lowest now visible window. It may not be the lowest
managed by xmonad window (i.e. some windows on non-visible workspace may be
lower in X stack).

The whole point of the ManageDocks redesign in 0.12 was to get rid of doing
this every time the layout was run, because with more than a handful of
mapped windows it's visibly very slow; but you're bringing that back again.
If we really do have to XQueryTree every time, then just revert ManageDocks
back to 0.11 where it was simple and reliable and as slow as it will be with
this change.

I don't understand your point. In my code queryTree runs only, when new
dock is started (it's in ManageHook, guarded by checkDock). And if you are
not killing and starting docks every second, that will be 2-3 (may be 5) times
per whole desktop session. How this can affect runLayout ?

BTW, there actually is a reason we might want to go ahead and partially
manage dock and desktop windows: proper EWMH layer support requires it. (We
currently fake it, and there are cases when the desktop window can float
over other windows as a result, granting that this usually requires whatever
opened the desktop window to have certain bugs.) It would also provide the
necessary hooks for better float management at some point, since adding a
proper floating layer would be almost trivial once layers are there for
desktop and dock windows.

This is a reason for going by way 2? It sounds to me, like some early draft
for very-very distant future, especially considering number of open issues and
pull requests. If this issue will wait for all that to be implemented, it'll be
open forever.

Well, the above is actually overstating a bit because 0.11 also did two
property queries for every mapped window every time the layout was run.

I've looked in 0.11 code. Well, it runs queryTree in caclGap, which was
used in modifyLayout, but this has absolutely nothing in common with how i
use it here.

I think this can still be avoided, but possibly not since changing it to
only check windows not in the Stack is part of what caused the subsequent
dzen breakage. (More precisely, it was being treated as a flxed-size window
and marked as a float instead of a dock, because it didn't have the
_NET_WINDOW_TYPE_DOCK property when initially mapped, IIRC. This worked
before because it was in effect treated as a _NET_WINDOW_STATE_ABOVE dock,
as long as it didn't accept focus.)

Is there a bug report about this? (to read full details)
By the way, here is proof of concept of another approach (with lowerWindow).
import XMonad
import XMonad.Util.EZConfig
import XMonad.Util.DebugWindow
import XMonad.Hooks.DebugStack
import XMonad.Hooks.ManageDocks
import XMonad.Hooks.EwmhDesktops
import XMonad.Hooks.ManageHelpers
import qualified XMonad.Util.ExtensibleState as XS
import XMonad.Util.WindowProperties (getProp32s)

import Data.List
import Data.Monoid

main :: IO ()
main = do
        let xcf = def
                    { modMask = mod4Mask
                    , layoutHook = avoidStruts (layoutHook def)
                    , manageHook = addDock <+> manageDocks <+> manageHook def
                    , handleEventHook = delDock <+> fullscreenEventHook <+> docksEventHook <+> handleEventHook def
                    , startupHook = docksStartupHook <+> startupHook def
                    , logHook = logDL >> logHook def
                    }
                    `additionalKeys`
                    [ ((mod4Mask, xK_b), sendMessage ToggleStruts)
                    , ((mod4Mask, xK_x), spawn "xmobar ~/.xmobarrcAlt")
                    , ((mod4Mask, xK_m), spawn "chromium")
                    , ((mod4Mask, xK_r), refresh)
                    ]
        xmonad xcf

-- Change order of `debugStack` output to natural..
debugSt :: X ()
debugSt = debugStackString >>= trace . unlines . reverse . lines

logH :: X ()
logH    = do
    trace "Tree..."
    d <- asks display
    r <- asks theRoot
    (_, _, ts) <- liftIO $ queryTree d r
    ls <- mapM debugWindow ts
    trace (intercalate "\n" ls)

data DockT  = DockA Window
            | DesktopA Window
  deriving (Show, Eq)

checkDesktop :: Query Bool
checkDesktop = ask >>= \w -> liftX $ do
    desk <- getAtom "_NET_WM_WINDOW_TYPE_DESKTOP"
    mbr <- getProp32s "_NET_WM_WINDOW_TYPE" w
    case mbr of
        Just rs -> return $ any (== desk) (map fromIntegral rs)
        _       -> return False

newtype DockList    = DockList [DockT]
  deriving (Show)

findDock :: Window -> [DockT] -> Maybe DockT
findDock w    = find go
  where
    go :: DockT -> Bool
    go (DockA x)    = w == x
    go (DesktopA x) = w == x

getPanels :: DockList -> [DockT]
getPanels (DockList ds) = filter go ds
  where
    go (DockA _)    = True
    go _            = False

getDesktops :: DockList -> [DockT]
getDesktops (DockList ds) = filter go ds
  where
    go (DesktopA _) = True
    go _            = False

unDock :: DockT -> Window
unDock (DockA x)    = x
unDock (DesktopA x) = x

instance ExtensionClass DockList where
    initialValue = DockList []

addDesktop :: MaybeManageHook
addDesktop    = checkDesktop  -?> do
    w <- ask
    liftX $ XS.modify (\(DockList zs) -> DockList (DesktopA w : zs))
    return idHook

addPanel :: MaybeManageHook
addPanel    = checkDock  -?> do
    w <- ask
    liftX $ XS.modify (\(DockList zs) -> DockList (DockA w : zs))
    return idHook

addDock :: ManageHook
addDock   = checkDock --> do
    composeOne [addDesktop, addPanel]
    liftX $ do
      dl <- XS.get
      let ds = getDesktops dl
          ps = getPanels dl
          rs = map unDock (ps ++ ds)
      d <- asks display
      trace "_Before_ lowering dock:"
      logH
      liftIO $ do
        lowerWindow d (head rs)
        restackWindows d rs
      trace "_After_ lowering dock:"
      logH
      return mempty

delDock :: Event -> X All
delDock (DestroyWindowEvent {ev_window = w}) = do
    DockList ds <- XS.get
    let md = w `findDock` ds
    case md of
      Just d    -> XS.put (DockList (d `delete` ds)) >> return mempty
      Nothing   -> return mempty
delDock _ = return mempty

logDL :: X ()
logDL   = do
    DockList ds <- XS.get
    trace $ "Docks list: " ++ intercalate ", " (map show ds)

@IvanMalison , @dkasak , @de-vri-es , please, try this too. You need to unwrap
docks, add addDock to manageHook and delDock to handleEventHook.
@geekosaur , so what do you think about this?

Is there any update on this?

A bit weird that nobody in this discussion mentioned that xmobar can lower itself on startup if configured to do so, and so can trayer. I'd be surprised if other docks couldn't do that as well.
That being said, if one wishes to run several such docks over another, having them all lower themselves may result in random order depending which one lower themselves first. And in that case it makes sense to get xmonad involved. In my case, that's putting xmobars below trayer: https://github.com/liskin/dotfiles/commit/3d19988009fcc3f6c05c19f30e748d3e65df3e6d
But I'm not sure if this is something general enough that xmonad could do this itself. How would it know if the user prefers trayer above xmobar or the other way around? How would it know that dockxyz is a desktop and dockabc is a panel? We'd need a module that makes the desired order configurable somehow.
Anyway, I agree with @sgf-dma here that using queryTree is fine as long as it's done only when the dock appears (or, in the case of my config, only when the dock lowers itself).

I think the hack that was merged in #490 can be abused to catch panels lowering themselves below the desktop, so maybe this finally has a "solution"?
Another related PR is #492 which makes it safer to let xmobar use override redirect and lower itself, instead of marking itself as a dock.

I think the hack that was merged in #490 can be abused to catch panels lowering themselves below the desktop, so maybe this finally has a "solution"?

I would agree.  If anyone still has any open issues here feel free to chime in though

Is this supposed to work for all windows or just for new windows which were created after an xmonad restart? I just updated and while new windows behave as expected, old windows still appear under my docks.

Hi, I know this might be late, but here's my "solution" to this issue.

-- Note: replace "..." with other imports you may need.
import XMonad.Hooks.ManageHelpers (..., doLower)
import XMonad.Hooks.ManageDocks   (..., checkDock)

-- And then add this to your manageHook
myManageHook = composeAll
  [
  -- ...Other stuff you have here...
  checkDock                   --> doLower
  ]

Hi, I know this might be late, but here's my "solution" to this issue.
-- Note: replace "..." with other imports you may need.
import XMonad.Hooks.ManageHelpers (..., doLower)
import XMonad.Hooks.ManageDocks   (..., checkDock)

-- And then add this to your manageHook
myManageHook = composeAll
  [
  -- ...Other stuff you have here...
  checkDock                   --> doLower
  ]

It should be noted that doLower doesn't exist prior to xmonad-contrib 0.17. This bit me on Arch, on which xmonad{,-contrib} is unmaintained since Goddess only knows when.

@sverona Well based on the official install tutorial, it's preferable that you install xmonad from source on distros like Arch that statically links haskell libraries.
Nonetheless, yeah the doLower function is only available on 0.17 and over.


[SPARK-42221][SQL] Introduce a new conf for TimestampNTZ schema inference in JSON/CSV

What changes were proposed in this pull request?

The TimestampNTZ schema inference over data sources is not consistent in the current code (most of them are for the purpose of backward compatibility to infer as Timestamp LTZ by default):

CSV & JSON: depends on spark.sql.timestampType to determine the result
ORC: depends on whether there is metadata written. If not, inferred as Timestamp LTZ
Parquet: infer timestamp column with annotation isAdjustedToUTC = false as Timestamp NTZ. There is a configuration spark.sql.parquet.timestampNTZ.enabled to determine whether to support NTZ. When spark.sql.parquet.timestampNTZ.enabled is false, users can't write Timestamp NTZ columns to parquet files.
Avro: Local timestamp type is a new logical type so there is no backward compatibility issue and there is no configuration to control the inference.

Since we are going to release Timestamp NTZ in Spark 3.4.0, I propose using a new configuration spark.sql.inferTimestampNTZInDataSources.enabled for TimestampNTZ schema inference. The flag is false by default for backward compatibility. When true, if a column can be either TimestampNTZ or TimestampLTZ, the infer result will be TimestampNTZ. This PR converts JSON/CSV data sources. If the proposal is fine to others, I will continue on the other data sources.
Why are the changes needed?

The TimestampNTZ schema inference over data sources is not consistent in the current code
The configuration spark.sql.timestampType is heavy. It changes the DDL/SQL functions's default timestamp type. If a user only wants to read back the newly written TimestampNTZ data without breaking the existing workloads, having a lightweight flag is a good idea.

Does this PR introduce any user-facing change?

No, TimestampNTZ is not released yet.
How was this patch tested?

UTs

cc @srielau @sadikovi @xinrong-meng

LGTM. For ORC and Avro, the NTZ type is new (ORC has a metadata, Avro has a new logical type), so we only need to apply this new conf to parquet?

For ORC and Avro

For ORC without timestamp metadata written Spark, the new configuration will still control the behavior. WDYT?

For ORC without timestamp metadata written by Spark, the new configuration will still control the behavior.

SGTM

Merging to master/3.4, cc @xinrong-meng


Receiving a panic runtime error when attempting to renew after expiration

user@machine:~/utils/lego$ sudo ./lego --domains="<existing domain>" --email="<existing email address>" renew
2016/04/14 17:23:15 [INFO][<existing domain>] acme: Trying renewal with -42 hours remaining
2016/04/14 17:23:15 [INFO][<existing domain>] acme: Obtaining bundled SAN certificate
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x4a45e9]

goroutine 16 [running]:
panic(0x95f440, 0xc8200100e0)
        /usr/local/go/src/runtime/panic.go:464 +0x3e6
github.com/xenolf/lego/acme.(*Client).getChallenges.func1(0xc820102090, 0xc82023acc0, 0xc82023ac60, 0xc8203f53a0, 0x14)
        /home/azhwkd/Projects/go/src/github.com/xenolf/lego/acme/client.go:408 +0x5c9
created by github.com/xenolf/lego/acme.(*Client).getChallenges
        /home/azhwkd/Projects/go/src/github.com/xenolf/lego/acme/client.go:421 +0x17c

If there's any other info I need to provide, please let me know!

Hey! I'm not entirely sure what could possibly be nil here :smile:
Could you check the file at<EMAIL_ADDRESS>if the registration property is filled out?

Yup seems to be! I omitted a bunch of info here (as it seems important! ha!)... but otherwise seems fully populated.
{
        "email": "<omitted>",
        "registration": {
                "body": {
                        "resource": "reg",
                        "id": <omitted>,
                        "key": {
                                "kty": "RSA",
                                "n": "<omitted>",
                                "e": "<omitted>"
                        },
                        "contact": [
                                "mailto:<omitted>"
                        ],
                        "agreement": "https://letsencrypt.org/documents/LE-SA-v1.0.1-July-27-2015.pdf"
                },
                "uri": "https://acme-v01.api.letsencrypt.org/acme/reg/230065",
                "new_authzr_uri": "https://acme-v01.api.letsencrypt.org/acme/new-authz",
                "terms_of_service": "https://letsencrypt.org/documents/LE-SA-v1.0.1-July-27-2015.pdf"
        }
}

Just to be sure, which version of the CLI are you using?

I was previously using 0.2.0. I got the error then, so decided I should upgrade to 0.3.0 by dumping the new version over top - but I still receive the error.

Is the stacktrace any different when using v0.3.0?

The above stack trace was from v0.3.0. This was the trace from v0.2.0:
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x4a10df]

goroutine 13 [running]:
github.com/xenolf/lego/acme.(*Client).getChallenges.func1(0xc8203a4780, 0xc82038f2c0, 0xc82038f260, 0xc8203c1520, 0x14)
        /home/azhwkd/Projects/go/src/github.com/xenolf/lego/acme/client.go:386 +0x5bf
created by github.com/xenolf/lego/acme.(*Client).getChallenges
        /home/azhwkd/Projects/go/src/github.com/xenolf/lego/acme/client.go:399 +0x187

goroutine 1 [select]:
github.com/xenolf/lego/acme.(*Client).getChallenges(0xc8203a4780, 0xc8203ea758, 0x1, 0x1, 0x0, 0x0, 0x0, 0x1)
        /home/azhwkd/Projects/go/src/github.com/xenolf/lego/acme/client.go:405 +0x6e8
github.com/xenolf/lego/acme.(*Client).ObtainCertificate(0xc8203a4780, 0xc8203ea758, 0x1, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /home/azhwkd/Projects/go/src/github.com/xenolf/lego/acme/client.go:211 +0x1c0
github.com/xenolf/lego/acme.(*Client).RenewCertificate(0xc8203a4780, 0xc8203c1520, 0x14, 0xc82038ef00, 0x53, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /home/azhwkd/Projects/go/src/github.com/xenolf/lego/acme/client.go:330 +0xf77
main.renew(0xc820068d80)
        /home/azhwkd/Projects/go/src/github.com/xenolf/lego/cli_handlers.go:238 +0x18fb
github.com/codegangsta/cli.Command.Run(0x8340c0, 0x5, 0x0, 0x0, 0x0, 0x0, 0x0, 0x870a30, 0x13, 0x0, ...)
        /home/azhwkd/Projects/go/src/github.com/codegangsta/cli/command.go:137 +0x107e
github.com/codegangsta/cli.(*App).Run(0xc820068b40, 0xc82000a140, 0x4, 0x4, 0x0, 0x0)
        /home/azhwkd/Projects/go/src/github.com/codegangsta/cli/app.go:175 +0xfff
main.main()
        /home/azhwkd/Projects/go/src/github.com/xenolf/lego/cli.go:113 +0xfab

goroutine 11 [IO wait]:
net.runtime_pollWait(0x7fed109aba68, 0x72, 0xc8200101a0)
        /usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc8200b3330, 0x72, 0x0, 0x0)
        /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8200b3330, 0x0, 0x0)
        /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8200b32d0, 0xc8200e4000, 0x2000, 0x2000, 0x0, 0x7fed109a6050, 0xc8200101a0)
        /usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc820026090, 0xc8200e4000, 0x2000, 0x2000, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/net.go:172 +0xe4
crypto/tls.(*block).readFromUntil(0xc8200d65d0, 0x7fed109abec8, 0xc820026090, 0x5, 0x0, 0x0)
        /usr/local/go/src/crypto/tls/conn.go:455 +0xcc
crypto/tls.(*Conn).readRecord(0xc820084840, 0x8e1517, 0x0, 0x0)
        /usr/local/go/src/crypto/tls/conn.go:540 +0x2d1
crypto/tls.(*Conn).Read(0xc820084840, 0xc8200e3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        /usr/local/go/src/crypto/tls/conn.go:901 +0x167
net/http.noteEOFReader.Read(0x7fed109b4a18, 0xc820084840, 0xc82009a738, 0xc8200e3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc8203c12a0, 0xc8200e3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        <autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc82038ed20)
        /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc82038ed20, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
        /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc82009a6e0)
        /usr/local/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
        /usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 12 [select]:
net/http.(*persistConn).writeLoop(0xc82009a6e0)
        /usr/local/go/src/net/http/transport.go:1009 +0x40c
created by net/http.(*Transport).dialConn
        /usr/local/go/src/net/http/transport.go:686 +0xc9d

Which seems weird as it has net/http problems that v0.3.0 doesn't?

Which seems weird as it has net/http problems that v0.3.0 doesn't?

That's just because 0.2.0 used to dump all go routines on panic... 0.3.0 doesn't do that anymore.
I'm trying to reproduce this localy, but so far havn't been successful :worried:

What OS are you running this on? I'd like to give you a binary with some logging just before the failing line to see what's nil.
You can of course also do this yourself if you have go installed. The logging would need to be just before line 408 in client.go. Just log the value of any variable on line 408.

Woops sorry, haven't configured email alerts for comments and didn't check for an update!
I'm running it on Ubuntu 14.04.4 LT x64 in an Azure VM. I don't have go installed and would prefer not to on this particular machine, so are you able to fire the modified binary my way?

Sure :)
Here you go:
lego.zip

Thanks! Here's the output. I'm not sure what stuff is sensitive so I've done my best not to obscure too much without giving too much away:
2016/04/19 14:30:53 [INFO][<domain>] acme: Trying renewal with -159 hours remaining
2016/04/19 14:30:53 [INFO][<domain>] acme: Obtaining bundled SAN certificate
2016/04/19 14:30:53 [DEBUG] JWS: &{https://acme-v01.api.letsencrypt.org/directory %!s(*rsa.PrivateKey=&{{0x<hex-start>920 65537} 0x<hex-start>940 [0x<hex-start>960 0x<hex-start>980] {0x<hex-start>a20 0x<hex-start>a40 0x<hex-start>a60 []}}) []}, User: &{<email> 0xc82004e840 <nil> 0xc820020078}, Reg: <nil>, AuthMsg: {new-authz {dns <domain>}  0001-01-01 00:00:00 +0000 UTC [] []}
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x4a5c2d]

goroutine 16 [running]:
panic(0xa5aac0, 0xc8200100f0)
        /usr/local/go/src/runtime/panic.go:464 +0x3e6
github.com/xenolf/lego/acme.(*Client).getChallenges.func1(0xc82010a090, 0xc820238c60, 0xc820238c00, 0xc8203fb380, 0x14)
        /home/azhwkd/Projects/go/src/github.com/xenolf/lego/acme/client.go:409 +0x80d
created by github.com/xenolf/lego/acme.(*Client).getChallenges
        /home/azhwkd/Projects/go/src/github.com/xenolf/lego/acme/client.go:422 +0x17c

So the offending part seems to be the registration altho the file you posted above looks perfectly fine.
I was able to encounter the exact same stack trace by forging an account.json with a registration property which was named wrong. Could you please double check that it really loads the right account.json and if the content is valid JSON?

How could I be sure it's loading the right account.json? I'll extract the file and ensure all the line endings are consistent etc, but the dump I provided above looks fine.

the JSON file is valid :(

Ok plot thickens...
Having had the server's FS open in an SFTP client I could properly see the folder hierarchy and noticed the following:
.lego/
--accounts/
----/
------
--------./keys/
----------.key
.lego/
--/
----/
------account.json
------keys/
--------.key
I assume only one of those is right?

Ugh... moving the data to the ./lego/accounts/ instead of sitting straight in .lego/ works - I successfully have my new cert now...

Ugh... I wonder how it ended up there? Did you use a like early early early version of lego back last year?

Nope not as far as I know. Maybe I butchered my backup folder copy at some point... but when I first ran the renew command and received the error from 0.2.0 it would've been exactly as I left it after I first got my cert.

Well. I will make sure that the renew command displays an error if it was unable to load the file properly.

Ok great. Thanks for your help... I'm really unsure how this happened. Feel free to close this issue :)


Ninject package versions are too strict

Requiring an exact version is very difficult for consumers to address correctly. Broaden the range.

Making this 3.2+ for NET45 and 3.3+ for NETSTANDARD2.0 since that is the earliest supported version.

Fixed in 8.0.1.


Loads the same favicon from the base Github Pages URL.

Hi!
I'm using jekyll to create a blog with Github Pages. I added a favicon.ico file (per #133) to the root directory. This achieved the desired output when running the local dev server. But when viewing it in Github Pages through my-name.github.io/myblog, it uses the same favicon as my-name.github.io.
Any hints on how do I fix this?

Make sure you point to the right favicon in the blog meta tags?
Closing this as this is not related to minima itself. Feel free to contact GitHub Support if you think it's related to GitHub Pages.


Add on-chip parallel sampler

Summary
Add on-chip parallel sampler
example https://gist.github.com/t-imamichi/a6eb158747a0f991d48fb5d9c06db90c
Details and comments

can you add this as one of the options of the implementation in qiskit-ibm-primivites?


chore(release): 1.0.0-beta.1

👀 PR includes

🔧 Chore

[ ] Test case
[ ] Docs / demos update
[ ] CI / workflow
[x] Release version 1.0.0-beta.1
[ ] Other ()

:tada: This PR is included in version @antv/s2-v1.53.0-beta.1 :tada:
The release is available on:

npm package (@beta dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add custom value injections

User could define a custom injection (for the given module? Global?) that functions could use. Something like:
pumlhorse.module('myModule')
    .injector('myBlob', ($scope) => {
        if ($scope.__myBlob === undefined) $scope.__myBlob = new Blob();
        return $scope.__myBlob;
    })
    .function('printBlob', ($myBlob) => { //do something with $myBlob });

Added to v2.2.0


Invalid framebuffer object. Code 36061(GL_FRAMEBUFFER_UNSUPPORTED)

Hi,
I have a nvidia graphic card which is GeForce GT730,as be shown below:

The visualizer also crashes for the error.Can you tell me how can I solve this issue please?Thank you very much!

The problem may be caused by this.I added --no-opengl-files when installing nvidia driver.However,someone said that the computer can not enter the graphic mode if installing nvidia drvier withou --no-opengl-files.What should I do then?

Hi @zzhh00,
Thanks for using our code.
Several reasons could cause the visualizer crashes. You may have a look at similar issues #5 and #7 first.
It would be also helpful if you could provide the information about executing $ glxinfo | grep OpenGL

Hi@Chen-Xieyuanli,
Thanks for your reply.
The information about OpenGL is list as below and I am trying to change the OpenGL version.

Besides when type nvidia-settings it shows that ERROR: Unable to load info from any available system.Thus maybe the nvidia driver installation is failed.

1.I solved this problem as the solution referred in Suma issue #2.However this solution is aim at the one  who only has Intel graphics and I have nvidia graphic card even though there may be something wrong.
2.Suma++ miss .obj files for visualization and these files can be found in assert of Suma.

@zzhh00

Your system currently uses the Intel GPU (from your CPU) and if you want to use your nvidia graphics card, you definitly have to change something. The OpenGL vendor string should contain something with NVIDIA, something like:

OpenGL context version: 4.5
OpenGL vendor string  : NVIDIA Corporation
OpenGL renderer string: GeForce GTX 960/PCIe/SSE2

In theory, the code should also work with an Intel GPU, but since NIVIDA usually provides more in their driver than the OpenGL Core Profile asks for it is hard for us to test that everything would run with the minimum requirements (since we use an NVIDIA driver for testing).

The second problem was caused by missing files in the repository. A git pullshould give you the needed files in the folder assets.

Since the problem has been solved, I am going to close this issue. If there are any updates, please feel free to ask me to reopen it.

@zzhh00
1. Your system currently uses the Intel GPU (from your CPU) and if you want to use your nvidia graphics card, you definitly have to change something. The OpenGL vendor string should contain something with NVIDIA, something like:

OpenGL context version: 4.5
OpenGL vendor string  : NVIDIA Corporation
OpenGL renderer string: GeForce GTX 960/PCIe/SSE2

In theory, the code should also work with an Intel GPU, but since NIVIDA usually provides more in their driver than the OpenGL Core Profile asks for it is hard for us to test that everything would run with the minimum requirements (since we use an NVIDIA driver for testing).
2. The second problem was caused by missing files in the repository. A `git pull`should give you the needed files in the folder `assets`.

I have this problem too, but I don't know how to change the OpenGL vendor string. I have installed the NVIDIA-Linux-x86_64-470.94.run, and the information is on below.
/home/lfy/图片/2022-01-21 19-56-20屏幕截图.png


Data I/O

💡 User Story:
As an AI engineer,
I want to add data input capability to CAMEL
So that our user can take different kinds of files as input
🛒 Types of Changes:

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds core functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation (update in the documentation)
[ ] Example (update in the folder of example)

🚀 Acceptance Criteria:

Functional code can handle with:
.docx
.pdf
.txt
.html
.json
Convert information from all these file types into string

Apply the functional code to CAMEL master branch successfully, and pass all tests.

🔀 Related User Stories:
No story related
🛠 Technical Details:
Use BytesIO, reference Langchain data input module
📑 Tasks:
.docx
.pdf
.txt
.html
.json
🔑 Dependencies:
No dependencies needed
🤝 Required Prerequisites:

[x] I have read the documentation https://camel-ai.github.io/camel/camel.html.
[X] I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
[X] Consider asking first in a Discussion.

Could you check camel/memory/lossless_storage.py in branch memory_liubo? I think the data IO design is highly related to the memory design.

Hi Tianqi, sorry for the late reply, sure I will check this

#281 is merged. Close this.


Get https://kubernetes-charts.storage.googleapis.com/index.yaml: x509: certificate signed by unknown authority

I am seeing the following error when configuring helm with the curated repo.
helm init --client-only
Creating /Users/xxxxx/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Error: Looks like "https://kubernetes-charts.storage.googleapis.com" is not a valid chart repository or cannot be reached: Get https://kubernetes-charts.storage.googleapis.com/index.yaml: x509: certificate signed by unknown authority

I am able to download this file using curl and then I get the same error using helm repo update.
I've been looking around the electric inter-webs and I see this issue has been coming & going.  Is there a viable workaround for this issue?
More Information;
macOS 10.13.5
docker 18.03.1-ce-mac65
go 1.10.3
Output of helm version:
helm version
Client: &version.Version{SemVer:"v2.9.1", GitCommit:"20adb27c7c5868466912eebdf6664e7390ebe710", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.9.1", GitCommit:"20adb27c7c5868466912eebdf6664e7390ebe710", GitTreeState:"clean"}

Output of kubectl version:
kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-06T15:22:13Z", GoVersion:"go1.9.6", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-28T20:13:43Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

Cloud Provider/Platform (AKS, GKE, Minikube etc.):
Amazon EKS

Ready to close.  Discovered that our corporate IT department had auto-magically installed the Netskope umberella agent on our Macs and it was re-writing the signing authority in the certificate chain.  Have disabled it and all is now well.


[Refactor] Make all installation shell scripts executable

Like dd923462 this patch makes two changes to all of the installation
shell scripts:

It makes them executable via chmod u+x.

It adds #!/bin/sh as the first line.

These changes make it possible to for developers to execute the
scripts simply by running, for example
$ ./install-linuxmint-34.sh

The second change in particular makes the scripts slightly more
portable as there is an admittedly small chance that developers will
not be using a shell that is compatible with /bin/sh by default.
Adding that initial line to the top of each script protects the
scripts from breaking if a developer prefers to use some other
shell on their terminal.
Making each script executable removes the need to explicitly use sh
to run them.  The project wiki needs to be updated to reflect these
changes since it has examples like
$ chmod +x install-linuxmint-34.sh
$ sh ./install-linuxmint-34.sh

which can now be replaced with the single-line example above.
Signed-off-by: Eric James Michael Ritz<EMAIL_ADDRESS>GitHub-Issue: #11
See-Also: dd9234629d9079c18631be2a54f5b0d881d65195

Thank you so much for your wonderful contribution, ejmr!


The robots.txt test now supports the junit5 parameterizedTest, gainin…

…g us the ability to have each robot rule be tested separately, so if one rule fails, the others can still show success in their own unit tests.
IMHO it also makes the tests more readable but that is in the eyes of the beholder
This files is smaller by 40 lines, and has 172 unit tests instead of 46 tests

This fixes #255


Attachments not submitted to VSTS when creating a new work item

When creating a new work item from an incident in Zendesk with an attachment submitted by the requestor, when the attachment is selected, the attachment is not added to the created work item in VSTS.
Link and description are perfectly added.

I have the same problem when I'm using notify button in Zendesk.

Our app is on Zendesk frame work V2.
Our VSTS App is now publicly available on Zendesk Marketplace. Since our App is public you can just install it from the MarketPlace of Zendesk. The team plan users can also install this app.
https://www.zendesk.com/apps/support/integratecloud-connector-for-vsts/

Reporting the same here when creating a new work item and when using the notify feature from Zendesk.  Also attaching images just in case it's not clear.

I have also noticed this bit of the api does not seem to be functioning. The VSTS work item ends up getting the ref line preceded by []. The embedded image in Zendesk does not appear, just that line. Attached images don't seem to show anything in VSTS.


Floww Tutorial Issue: Getting started (Ray/RLlib) Typo

Under Getting started (Ray/RLlib) , it says:
In order to test run an Flow experiment in RLlib, try the following command:
python examples/rllab/stabilizing_the_ring.py
It should be: python examples/rllib/stabilizing_the_ring.py

resolved by #38


Use Inspect object to get consumer count in a broker agnostic way

This PR extends the celery_active_consumer_count metric beyond the currently supported brokers by using the Inspect object's stat method, and using the number of worker thread PID's returned as the amount of consumers on that worker. It then cross references that with the queues that worker is consuming from to create an amount of threads that could be working from that queue.
I'm not 100% sure of the terminology used. It could be that "active consumers" are intended to be the amount of workers, rather than the total amount of threads, or it could be that "active" is intended to mean the amount of threads currently working on tasks from that queue. I haven't been able to find good documentation on that in the celery, kombu, RabbitMQ, or Qpid documentation.

Hi @xulaus if you rebase this PR I'll take a look.

I apologise for not getting back to this. I did a deeper dive into Qpid to try and find a definitive answer to what consumer means in this context. Celery seems to be getting this information getQueueProperties - however the documentation for that call seems to be thin on the ground.
Given that it isn't clear what consumer means when I get time I'll rework this to provide 2 metrics instead of one. Maybe naming it "processes" vs "workers" to avoid any ambiguity.

Thanks Richard.
Released danihodovic/celery-exporter:0.9.0


buildPgrxExtension: avoid empty RUSTFLAGS

Setting RUSTFLAGS to an empty strings causes the linux pgvecto-rs build to fail. Therefore, it should only be set when it is non-empty.
Build failure introduced by commit 2a13929e1f191b3690dd2f2db13098b04adb9043 (#332353)
See also: https://github.com/NixOS/nixpkgs/pull/324127#discussion_r1713043056
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Any reason we didn't use += here? Or is that just not going to work with darwin?

@eclairevoyant I would agree with that, but I don't have a darwin machine to test, so can't know for sure it doesn't break anything.
Also, just changing the = to += would require adding a space before (to separate what was there previously, if any), which would result in the same problem this PR is trying to fix (but for darwin this time). It would have to be conditional concatenation, not sure how to do that with bash.

You could do a RUSTFLAGS="$RUSTFLAGS...." if you wanted. Addiyionaly, im happy to run a nixpkgs-rewiew on my old intel darwin if you want to try that change

Otherwise this looks good i dont foresee many people overriding rust flags

@happysalada

You could do a RUSTFLAGS="$RUSTFLAGS...."

Doesn't this have the same problem?
If RUSTFLAGS is empty, RUSTFLAGS="$RUSTFLAGS -Clink-args=-Wl,-undefined,dynamic_lookup" will start with a space and break cargo-pgrx.

@happysalada

Addiyionaly, im happy to run a nixpkgs-rewiew on my old intel darwin if you want to try that change

I'd appreciate it!
Also, it'd be nice if you could check if pgvecto-rs builds on darwin now that buildPgrxExtension sets RUSTFLAGS, but I doubt it does.

Can verify, https://github.com/NixOS/nixpkgs/commit/2a13929e1f191b3690dd2f2db13098b04adb9043 broke build of some extensions on our end.
This commit let us build them again.

My nixpkgs review got stuck on something unrelated, this looks good


[suggestion] switch to light code highlighting in the doc

Dark code highlighting on the light website theme is very contrasting. I think that switching to light code highlighting would improve the doc site look.
Here is an example with the "friendly" pygment style. There are many other good looking light ones available. Here is the reference link https://xyproto.github.io/splash/docs/longer/all.html .

Indeed, thanks!


No serial port connection

After starting with the command:  $ cnc -m /tablet:/home/pi/cncjs-shopfloor-tablet/src
And navigating to the page:  http://<ip_address>:8000/tablet/'   It redirects to http://<ip_address>:8000/tablet/#/connectionhowever there is no serial port in the ports list.   If I go straight tohttp://<ip_address>:8000/tablet/#/axesthen it says "NoConnect" in the top left, if i click any of the buttons then I get the messageserial port "" not available`.
CNCjs works fine and the serial port shows up.

My bad.  In order to avoid accidentally selecting the wrong serial port, I restricted the code to offer only the serial port that is connected to my machine.  That change is easy to undo by commenting out two lines.  I just made a branch with the change
https://github.com/cncjs/cncjs-shopfloor-tablet/compare/MitchBradley-patch-all-serial-ports
If you want to see just the changes, here is the commit
https://github.com/cncjs/cncjs-shopfloor-tablet/commit/a22e50ee91f8e5d2df2474eeebc4bf3bfbb72b05

Thanks,
I noticed that when I use https://github.com/cncjs/cncjs-pendant-tinyweb as well and select the serial port there then this works.

The changes suggested work fine, thanks


Don't get PACKET_IN when deactivate fwd in ONOS

Hi,
I found out when I deactivate reactive forwarding in onos, I don't get any packet_in message in my external application. Would you please also check this issue?
Thanks,
Adib

I think this is normal because ARP packets need to be handled first.

I will look at this, but this shouldn't be the case unless your applications are using the same SDN controlled network on which your client or "data" traffic is using. In a deployment, the management apps are often on a separate management network from your client or data traffic. This is not always the case, but is a common deployment.
I will update this thread after I complete my test, but my default test has a management network that is separate from the data network. I am at the airport right now, so it might be a little delayed depending on when I can get the test done and when the plane boards.

I think I figured that out. When we deactivate fwd, arp packets is not handled any more. So we when we set TEE_TO to get IP packets, we don't get anything because ARP packets need to be handled first. I think this problem is resolved.

Ok, then I will close this. I just completed my simple test, ONOS with only openflow application installed and it worked. But again, i am using a separate mgmt network.


Small improvements to performance test report

Always show result message
Show small regressions in table not as failed, but as nearly failing

This can still be improved, though I think the changes are good as is already.

@blindpirate PTAL!


[composer][engx] seperate default value related files and same to resource name

After change on #616, sample code generate depend on default value composer, it causes bazel error of a cycle in bazel dependency graph.
In this PR,  I create defaultvalue bazel package, resourcename bazel package and constants bazel package. Also do the same for their matched tests.
dependency graph:
defaultvalue -> resourcename -> constants

The resource name change makes sense, but I'm not sure the default value and constant classes need their own package. Could you please elaborate on the motivation for these changes?

Because ServiceClientSampleCodeComposer refer to default value, after ServiceClientSampleCodeComposer moved to src/.../gapic/composer/samplecode, it cause bazel build with cycle in dependency graph error. The solution on this is to move default value on its own package, so that BUILD.bazel add dep like "//src/main/java/com/google/api/generator/gapic/composer/defaultvalue",
and then defaultvalue refer to resourcename, resourcename refer to constant, they all have same error.

The resource name change makes sense, but I'm not sure the default value and constant classes need their own package. Could you please elaborate on the motivation for these changes?

Because ServiceClientSampleCodeComposer refer to default value, after ServiceClientSampleCodeComposer moved to src/.../gapic/composer/samplecode, it cause bazel build with cycle in dependency graph error. The solution on this is to move default value on its own package, so that BUILD.bazel add dep like "//src/main/java/com/google/api/generator/gapic/composer/defaultvalue",
and then defaultvalue refer to resourcename, resourcename refer to constant, they all have same error.


Document sign in guards

Looks good to me.

@mcmire sorry - missed your comments. Will remedy in follow up commits


Creating a sample projects section at the showcase

It will be nice to list the sample projects

Done, accessible from http://robeio.github.io/robe-react-ui/#Samples


Couple of URL preview improvements

Don't display the URL preview embed if no image was found for the URL.
Make the title of the URL clickable (again).

Testing Plan:
Tested 1 using https://zulip.readthedocs.io/en/latest/ and other such links.
2 can be tested with https://zulipchat.com/

For the pointer-events thing, can you confirm that there's only a behavior change for the title?  And maybe more generally explain what the bug was?

@timabbott I moved away from the pointer-events solution to another solution just to prevent the ::after element from overlapping onto the title, making it un-clickable.  Let me know if this solution is easier to understand and seems reasonable.

This is great, merged, thanks @punchagan!


[stable/nginx-ingress] #9637 Breaks installing nginx with rbac without cluster-level access

Is this a BUG REPORT or FEATURE REQUEST? (choose one): Bug

Version of Helm and Kubernetes:
Helm 2.13, all versions of k8s I tried
Which chart:
nginx-ingress as of 1.0.1, from #9637
What happened:
Cluster is setup with a tiller in a namespace, without cluster access (namespace myproject, this is openshift, but that's irrelevant). Tiller has a role that grants it permission only to modify things within the namespace (i.e. no cluster level resources like clusterrole)

If you set rbac.create=true as well as controller.scope.enabled=true, then you get the new set of cluster role and binding + the role and rolebinding. This fails
If you set rbac.create=true and controller.scope.enabled=false, you'll get the previous set of cluster permissions + the role and rolebinding. This fails
If you set rbac.create=false and controller.scope.enabled=false then you get no roles set at all, cluster or otherwise. This makes it so nginx-ingress can't talk to k8s and also fails.

What you expected to happen:

No cluster roles
Role and role binding
nginx-ingress able to communicate with k8s API within its namespace

How to reproduce it (as minimally and precisely as possible):
kubectl apply -f the following file in the myproject namespace:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: tiller
rules:
- apiGroups:
  - "*"
  resources:
  - "*"
  verbs:
  - "*"
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: tiller
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: tiller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: tiller
subjects:
- kind: ServiceAccount
  name: tiller

And install helm with: helm init --service-account tiller --upgrade --tiller-namespace=myproject
Then when you install nginx-ingress, add TILLER_NAMESPACE=myproject on the front, and set rbac.create=true controller.scope.enabled=true
Anything else we need to know:

There are two different issue tickets describing this problem, original #11033 and later this one, #12510.
With the problem (that didn't even warrant the change introduced in #9637 as it amounted to as much as extra logs noise) being long fixed upstream (https://github.com/kubernetes/ingress-nginx/pull/3887), the latest nginx-ingress chart still ships the useless extra cluster roles that break scoped deployments, for about five months now.
Does anyone actually read the follow up issues or maintain the nginx-ingress chart?
@norwoodj @unguiculus

What we've ended up having to do on our end is just have our own modified copy of the chart, without this broken behaviour in it. This is not the first and only thing that's forced us to maintain our own copy of the chart, so at this point we're pretty far removed from the chart contained in this repository because of this. Multiple PRs have gone stale, as have issues, due to inactivity.

I believe the governance process around charts ownership should be revisited. I already have other PRs on this repo that are waiting approval.
Hopefully, as this is an issue raised by more people, it will be reviewed.

just wanted to echo the comments on this issue, users with security concerns would deploy tiller as non cluster-admin as suggested here: https://github.com/helm/helm/blob/master/docs/rbac.md#example-deploy-tiller-in-a-namespace-restricted-to-deploying-resources-only-in-that-namespace
We currently workaround this by removing the additional cluster roles from the helm chart which is far from being desirable from a deployment perspective.
mkdir ./charts/nginx-ingress/manifests
cd charts/nginx-ingress/

helm template --values $VALUES_URL --output-dir ./manifests . --set controller.scope.enabled=true
rm -f manifests/nginx-ingress/templates/scoped-clusterrole*

kubectl apply -f manifests/nginx-ingress/templates

those who come here, this is fixed in stable/nginx.ingress chart's 1.33.0 and above versions:
helm install --name nginx stable/nginx-ingress --debug --dry-run  --set rbac-create=true --set rbac.scope=true --version 1.33.0 | grep ^kind: | grep Role
kind: Role
kind: RoleBinding


[Request deployment] New Hub: Bioinformatics Hub of Kenya Initiative

This a part of the Catalyst Project on the catalystproject-af cluster. Please use standard catalyst profile developed in

https://github.com/2i2c-org/infrastructure/issues/3631

URL: bhki.af.catalystproject.2i2.cloud
The GitHub handle of the community representative
pending
Technical support contacts listed

[ ] Technical contact present

Hub important dates
pending
Hub Authentication Type
GitHub (e.g., @mygithubhandle)
First Hub Administrators
pending
[GitHub Auth only] How would you like to manage your users?
None
[GitHub Teams Auth only] Profile restriction based on team membership
pending
Hub logo image URL
pending
Hub logo website URL
pending
Hub user image GitHub repository
No response
Hub user image tag and name
pending
Extra features you would like to enable

[ ] Dedicated Kubernetes cluster
[ ] Scalable Dask Cluster

(Optional) Preferred cloud provider
None
(Optional) Preferred cloud region
No response
(Optional) Billing and Cloud account
None
Other relevant information to the features above
No response
Tasks to deploy the hub

[ ] 1. Deploy information filled in above
[ ] 2. Engineer who will deploy the hub is assigned
[ ] 3. If using GitHub Orgs/Teams Auth, Engineer is given Owner rights to the org to set this up.
[ ] 4. Initial Hub deployment PR
[ ] 5. Administrators able to log on -> Hub now in steady-state

Please use
Michael Kofia Landi, @LandiMi2

@LandiMi2 can you test the hub? It was deployed at https://bhki.af.catalystproject.2i2c.cloud/
Also, you are invited to the Github team https://github.com/orgs/CatalystProject-Hubs/teams/bhki

@jmunroe , I did try to login, but it seems I have not been added to the list of allowed users for this hub(error message).  This link is broken.

@jmunroe Yes, I tested the hub, and it is okay. I joined the GitHub team too. Thanks. I'll be meeting Katherine next week for more information.


Merge with upstream

This solves our issue with needing to run sudo rm -rf /mnt/postgresql manually when bootstrapping a new Postgres.

@mmalecki could we potentially move to the main pg repo at this point and delete this fork?


editor.update().

Error: Unable to find an active editor. This method can only be used synchronously during the callback of editor.update().

occur same problem

@nevecex and I are seeing this at Dart too with the latest. It seems like there may even be a problem with recursion because after a bunch of

Error: Unable to find an active editor state. State helpers or node methods can only be used synchronously during the callback of editor.update() or editorState.read().

We also see

RangeError: Maximum call stack size exceeded

It seems like this has something to do with lexical-vue 0.8.4 in concert with lexical 0.16.x--when we use lexical 0.15.0, for example, we don't have this issue.
For the short term you all may be able to solve this by using an older version of lexical (we do this by patching node_modules/lexical-vue/package.json with the patch-package tool).
In the longer term this seems like a serious problem so we should look into it!

New version released and it's locked to 0.16.x. Let me know if the issue is fixed!

This has been fixed in the latest version, thank you!

Thank you @nevecex for the update!


Is there any way to resume execution after process crash based on last success check point?

We have a scenario where we do a big process with multiple steps. After each step, the state can be persisted. In case of the process crash we need to resume from the last successful state.
Is there any recommended way to use Banzai to accomplish the use case.

It is fine if there is any way to intercept the Node executions so that the state can be saved.

Too late now, but I don't believe so but it's a good call out....I'll look into this more as I'm updating the project.


Plugin Needs Polling to Determine Status

HomeKit does not update the bulb status unless you leave the room or app and come back.
My test:

Go into HomeKit Room and turn on light using bulb icon.
physically turn off power to bulb.

Result:
Icon never changes until I exit the HomeKit room then re-enter. Same if I start with a "no response" bulb and power it on. The status doesn't update until I exit the HomeKit room and re-enter.

Polling unfortunately results in a large amount of network traffic that is most likely detrimental to the experience of other devices of the network.
The reason this works fine in the Wiz app is because the app "subscribes" to the bulbs only when the app is open. However, I don't believe HomeBridge lets me know when the plugin should be subscribing to the bulbs.
I'm open to suggestions here but I don't believe subscribing to the bulbs 24/7 is a good solution for the vast majority of users.

With homebridge-wiz-lan v3.0.2 performance is now greatly improved, so for now, I'll just be happy with what we've got. Thanks.


pressStyle and group-X-press not invoked when "Enter" key pressed

Current Behavior
styles like pressStyle and group-X-press are utilized when an interactable element is pressed either with the mouse or with a space bar press while focused, however, they are not utilized when the "Enter" key is pressed even though that triggers a press event like the space bar or a click.
Expected Behavior
Pressing the "enter" key on an interactable element should trigger press styles like it's click/space counterparts.
Tamagui Version
1.74.3

Platform (Web, iOS, Android)
Web

Reproduction
- Clone [this repo](https://github.com/jbean96/tamagui-icon-groups)
- Setup:
  - `yarn`
  - `yarn watch:build`
  - `yarn sandbox`
- Visit `localhost:9000`
- Focus the box in the center, press the space bar, observe that it makes the background of the box blue, same with clicking
- Focus on the box in the center, press the enter key, observe that it does not trigger the blue background.

System Info
System:
    OS: macOS 13.5.1
    CPU: (10) arm64 Apple M1 Pro
    Memory: 109.55 MB / 32.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 19.6.0 - ~/.nvm/versions/node/v19.6.0/bin/node
    Yarn: 3.6.3 - /opt/homebrew/bin/yarn
    npm: 9.4.0 - ~/.nvm/versions/node/v19.6.0/bin/npm
    Watchman: 2<IP_ADDRESS> - /opt/homebrew/bin/watchman
  Browsers:
    Chrome: 117.0.5938.132
    Firefox: 114.0.1
    Safari: 16.6

jbean ~/git/tamagui-icon-groups master > npx envinfo --system --npmPackages --binaries --browsers

  System:
    OS: macOS 13.5.1
    CPU: (10) arm64 Apple M1 Pro
    Memory: 221.63 MB / 32.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 19.6.0 - ~/.nvm/versions/node/v19.6.0/bin/node
    Yarn: 3.6.3 - /opt/homebrew/bin/yarn
    npm: 9.4.0 - ~/.nvm/versions/node/v19.6.0/bin/npm
    Watchman: 2<IP_ADDRESS> - /opt/homebrew/bin/watchman
  Browsers:
    Chrome: 117.0.5938.132
    Firefox: 114.0.1
    Safari: 16.6
  npmPackages:
    @biomejs/biome: ^1.2.2 => 1.2.2
    @manypkg/cli: ^0.19.1 => 0.19.2
    @nderscore/tamagui-typescript-plugin: ^0.3.1 => 0.3.1
    @vercel/og: ^0.5.8 => 0.5.8
    check-dependency-version-consistency: ^3.0.3 => 3.0.3
    import-sort-style-module: ^6.0.0 => 6.0.0
    npm-run-all: ^4.1.5 => 4.1.5
    p-map: ^5.5.0 => 5.5.0
    prettier: ^2.8.1 => 2.8.8
    pretty-quick: ^3.1.2 => 3.1.3
    prompts: 2.1.0 => 2.1.0
    turbo: ^1.10.13 => 1.10.13
    typescript: ^5.2.2 => 5.2.2
    ultra-runner: ^3.10.5 => 3.10.5

That's more of an issue with browsers because Tamagui internally uses :active css sudo class and browsers only trigger that class when the user presses space or uses mouse press.


C2-12858-4 while copying [SOLVED]

I just did everything normally and when it nearly finished it gave me this error

that issue is useless !
please provide a bit more information on your problem ?
what did you do ?
can you reproduce the error ?

here,described it

I guess the cause of this issue is that one of the folders was not encrypted properly.

Yes as FloW mentioned you need to re-encrypt the files and try again. I had this error too and retried encrypting and it works now fully installed!

i did too

You will probably want to try a couple times. I had to do it about 8 times myself before it decided to work. So your best bet would be to keep trying.

So you just repeat steps 9-11 until it works? Getting the same error and i definitly didn't do anything wrong.

Tried it atleast 3 time from beginning to finish now.

This is all due to the fact that after trinity did not update the database in qcma


⚠ Processors has degraded performance

In b0544d7, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 92 ms

Resolved: Processors performance has improved in 52d415f.


NPM blocks graceful shutdown connections for pods in Terminating state

What happened:
When a Pod is deleted, NPM does not allow any grace period and removes pod ip from all relevant IPSets resulting in blocking of graceful shutdown connections.
What you expected to happen:
NPM Should not delete pod as soon as an update event comes with deletionTimeStamp set, it should wait until the delete event is called or wait for the  terminationGracePeriodSeconds to expire.
How to reproduce it:
Check the linked issue for repro details.

@JungukCho Please can you help me with upgrading? We have AKS with Kubernetes v 1.22.4, Running more than year. I still can see the azure-npm pods with version v1.4.9. How can I achieve newer version to be used? I thought that this is managed by Azure AKS transparently.


fix(check):updates font size, variable spelling

fixes font size for check label and spelling mistake 🤦♂

Deploy preview for pf-next ready!
Built with commit 837eab4d9c0f99d0eb065b70329932075c17fd8b
https://deploy-preview-796--pf-next.netlify.com

woot! can you also update the docs and add a . in front of pf-c-check__label in the row for .pf-m-disabled?

:tada: This PR is included in version 1.0.59 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


i use rally test elasticsearch 5.5.3 happen error

2018-11-22 06:05:21,361 -not-actor-/PID:6046 root INFO ---- Actor System shutdown
2018-11-22 06:05:21,361 -not-actor-/PID:6040 esrally.rally INFO Actor system is still running. Waiting...
2018-11-22 06:05:22,362 -not-actor-/PID:6040 esrally.rally INFO Shutdown completed.
2018-11-22 06:05:22,362 -not-actor-/PID:6040 esrally.rally ERROR Cannot run subcommand [race].
Traceback (most recent call last):
File "/usr/local/python34/lib/python3.4/site-packages/esrally/rally.py", line 454, in dispatch_sub_command
race(cfg)
File "/usr/local/python34/lib/python3.4/site-packages/esrally/rally.py", line 383, in race
with_actor_system(lambda c: racecontrol.run(c), cfg)
File "/usr/local/python34/lib/python3.4/site-packages/esrally/rally.py", line 404, in with_actor_system
runnable(cfg)
File "/usr/local/python34/lib/python3.4/site-packages/esrally/rally.py", line 383, in
with_actor_system(lambda c: racecontrol.run(c), cfg)
File "/usr/local/python34/lib/python3.4/site-packages/esrally/racecontrol.py", line 390, in run
raise e
File "/usr/local/python34/lib/python3.4/site-packages/esrally/racecontrol.py", line 387, in run
pipeline(cfg)
File "/usr/local/python34/lib/python3.4/site-packages/esrally/racecontrol.py", line 61, in call
self.target(cfg)
File "/usr/local/python34/lib/python3.4/site-packages/esrally/racecontrol.py", line 334, in benchmark_only
return race(cfg, external=True)
File "/usr/local/python34/lib/python3.4/site-packages/esrally/racecontrol.py", line 286, in race
raise exceptions.RallyError(result.message, result.cause)
esrally.exceptions.RallyError: ('Load generator [0] has exited prematurely.', None)

Running delete-index                                                           [100% done]
Running create-index                                                           [100% done]
Running check-cluster-health                                                   [100% done]
Running index-append                                                           [100% done]
Running refresh-after-index                                                    [100% done]
Running force-merge                                                            [100% done]
Running refresh-after-force-merge                                              [100% done]
Running index-stats                                                            [100% done]
Running node-stats                                                             [100% done]
Running default                                                                [100% done]
Running term                                                                   [100% done]
Running phrase                                                                 [100% done]
Running country_agg_uncached                                                   [100% done]
Running country_agg_cached                                                     [100% done]
Running scroll                                                                 [100% done]
Running expression                                                             [100% done]
Running painless_static                                                        [100% done]
Running painless_dynamic                                                       [100% done]
Running large_terms                                                            [ 38% done][ERROR] Cannot race. Load generator [0] has exited prematurely.

Hi @brookliu123 , sorry but Github is reserved for reporting issues only. Can you please ask your question instead in the user forum at https://discuss.elastic.co/c/elasticsearch/rally?


CIP-0026 | Update signatures schema

Proposed Change
Field signatures should be an array to support multiple attestation signatures on entries.

@KtorZ @michaelpj - given the CIP is already written this way, with multiple signatures defined by an array (https://github.com/cardano-foundation/CIPs/tree/master/CIP-0026#attestation-signatures), shouldn't this be put through as a correction to the schema in the CIP?

Yes, this just seems like an oversight.


Record when a file only contains partial data and overwrite on next run

Problem:
Running the script will generate 1 file per day starting today. Files for "yesterday" and further back in time will have full data (for every 5-minte interval), but the file created for "today" will be missing data between "now" and 23:55pm. The rest of the file will be filled with zeros, eg:
tail -n 10 2023-06-04.csv
2023-06-04 23:10:00,0,0,0,0,0,0
2023-06-04 23:15:00,0,0,0,0,0,0
2023-06-04 23:20:00,0,0,0,0,0,0
2023-06-04 23:25:00,0,0,0,0,0,0
2023-06-04 23:30:00,0,0,0,0,0,0
2023-06-04 23:35:00,0,0,0,0,0,0
2023-06-04 23:40:00,0,0,0,0,0,0
2023-06-04 23:45:00,0,0,0,0,0,0
2023-06-04 23:50:00,0,0,0,0,0,0
2023-06-04 23:55:00,0,0,0,0,0,0

When you run the script again "tomorrow", or any future time, it will pull data only for days that do not have a file which already exists - in this example it would create a new (partially complete) file for 2023-06-05, and then stop, because 2023-06-04.csv already exists.
However, now I have an incomplete record of data, with missing data points in the late stages of the day I ran the script (2023-05-04 in this example).
Current workaround:
Manually delete or rename the partially complete file, so that upon next / future run, a new complete file will be created.
Proposed / possible solution:

if a file does not have complete data, which it never will for "today's" date (unless run at 23:59:59), name the file 2023-06-05_partial.csv
When the script runs next time, ignore any _partial.csv files, which would result in a new (complete) 2023-06-05.csv file getting created.
Delete the _partial.csv file so that there are no duplicate data points in the directory

Thanks

Thanks for the detailed bug report, fixed in bdb96458590fba8d52511af419b9bac5ec6a7d41.


fix: move is owned by check to load factor

What kind of change does this PR introduce?
In loadFactor ensure that all factors which are loaded are owned by the user

Pull Request Test Coverage Report for Build<PHONE_NUMBER>9
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.01%) to 58.27%

Totals

Change from base Build<PHONE_NUMBER>4:
0.01%

Covered Lines:
9184

Relevant Lines:
15761

💛  - Coveralls


chore: added cryptocurrency-qpp react project

1349 Added Cryptocurrency app
Description
This is fully based on API fetching, in this you can find the different cryptoverse and their price as well as market cap and it also shows the graph for 24 hours, 3 Months , 30 Months and 1 Year.
Screenshots

Checklist for projects addition

[x ] My project has contribution guidelines.
[ ] My project has a Code of Conduct.
[ x] My project has a README file.
[ ] My project has a License.

Please note that if your project doesn't meet these requirements, your PR may be closed. Make sure to include these elements in your project before submitting the PR.

make sure that your project has contributing guidelines, license, readme, and code of conduct

make sure that your project has contributing guidelines, license, readme, and code of conduct

@priyankarpal now my project has all of things which you mention above.


Cannot change bIsEnabled value

Describe the bug
Changed .uasset data bIsEnabled to true and packed file using unrealpak. But bIsEnabled is still false in the FModel.
When load .uasset "Failed to parse 11 exports!" happens,and modified .uasset causes game crash.
Screenshots

(After Modified)
Desktop (please complete the following information):

OS: Windows 11
Version: v<IP_ADDRESS>-alpha.6 (ebcdfce)
Game: Sword and Fairy 7
Game UE4 version: 4.25
File: Pal7\Content\UI\UMG\System\UMG_Photo.uasset

UMG_Photo.zip

Now: "Failed to parse 8 exports!"

no failed exports in the latest commit 25fe78b


MTGA fails to start with an error message

Magic the Gathering Arena fails to start with this error message:

and the same error message happened with me before with other two unity games on Linux Mint 19.1, Ubuntu 18.04, and Ubuntu 18.10. These games are Cuphead, and Ori and the Blind Forest DE. I found a few errors/warnings in dxgi.log. That is why I posted this issue here.
Software information
Magic the Gathering Arena
System information

GPU: AMD HD 8750M.
Driver: Mesa 19.0.1 (Padoka PPA)
Wine version: Wine-staging 4.5
DXVK version: 1.0.2

Log files

d3d11.log:  The file is empty.
dxgi.log:
MTGA_dxgi.log

info:  Required Vulkan extension VK_KHR_get_physical_device_properties2 not supported
info:  Required Vulkan extension VK_KHR_surface not supported
info:  Required Vulkan extension VK_KHR_win32_surface not supported

As usual, your Vulkan setup is broken. Please make sure you can run both 32-bit and 64-bit Vulkan apps before reporting issues.

Ops! I installed libvulkan1:i386, and forget to install mesa-vulkan-drivers:i386. They can't be found in Synaptic package manager, I had to install them manually from the terminal. The game doesn't run anyway (wine bug). Thanks, and sorry about.


HDDS-8023. [HTTPFSGW] Remove support for satisfyStoragePolicy operation

What changes were proposed in this pull request?
I removed the support for satisfyStoragePolicy operation.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-8023
How was this patch tested?
Built the project successfully, green CI on my fork: https://github.com/dombizita/ozone/actions/runs/4258119003

Thanks Zita for working on this. It's looking good to me, I just have 1 question/remark. SATISFYSTORAGEPOLICY enum value is no longer used for practical purposes only for throwing the UnsupportedOperationException. Is it still needed, or is it possible to remove that operation entirely? I'm not too familiar with how HTTPFS works, and what side effects it could have.

hi @Galsza, thanks for the review! we decided to keep SATISFYSTORAGEPOLICY as it is a valid operation in WebHDFS, check here, so we want to throw a specific exception in that case, as it is not just a general not supported operation.


ci: update issue template

close #6

@CodiumAI-Agent /review

PR Reviewer Guide 🔍

⏱ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Key issues to review
Inconsistent OptionsThe change from checkboxes to dropdown for the 'Have you searched existing issues?' might limit user responses. Consider if this change aligns with the intended user interaction.

PR Reviewer Guide 🔍
⏱ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Key issues to review
Inconsistent OptionsThe change from checkboxes to dropdown for the 'Have you searched existing issues?' might limit user responses. Consider if this change aligns with the intended user interaction.

do not care this

Persistent review updated to latest commit https://github.com/zncdatadev/kubedoop/commit/ad76bd97c87eb3da2f3ff2de286ca00717d594b0


None of this works on Excel for Web

I guess it's a different API, but I think there should be an update to the page explaining this.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bf942d54-3ee5-2476-e11f-0b658e0bb189
Version Independent ID: 2a671958-54a3-f477-6624-f2bdc7c133e5
Content: Work with PivotTables using the Excel JavaScript API - Office Add-ins
Content Source: docs/excel/excel-add-ins-pivottables.md
Product: excel
Technology: add-ins
GitHub Login: @o365devx
Microsoft Alias: o365devx

Hi @vicw87,
I'm sorry to hear you're having issues with the PivotTable APIs. I just tried out a few Script Lab samples for the PivotTable APIs on Excel for the web. They seemed to be working okay. Could you please specify which PivotTable APIs or which pieces of sample code aren't working for you?
Thanks!


Implement login with reCaptcha

To increase security and prevent spam, integrate reCaptcha. One option is to use the ng-recaptcha library (https://www.npmjs.com/package/ng-recaptcha)
npm i ng-recaptcha --save

Implemented with b7ff57c7ab75db3c568d3cab65820f1064b47d1d and 5e7481e9571e07faa10c8625043254f6eca12e61. Closing.