Removing layers with layers.pop() doesn't work?

I'm trying to start https://github.com/iamaaditya/VQA_Demo which uses pre-trained VGG16 model (https://github.com/iamaaditya/VQA_Demo/blob/master/models/CNN/VGG.py). The last 2 layers are removed with layers.pop(). This doesn't seem to work, however. The message "ValueError: could not broadcast input array from shape (1000) into shape (4096)" is displayed, and I see the layers were not removed when I do "plot(image_model, to_file='model_vgg.png', show_shapes=True)". Thank you.

If you want to fine-tune this model, you should add new dense layer after popping the original one.

The original author does that; here is the notebook: https://github.com/iamaaditya/VQA_Demo/blob/master/Visual_Question_Answering_Demo_in_python_notebook.ipynb
Image features are supposed to be merged with language features, but it doesn't come to that because of the error I mentioned. The issue is, the model still uses the popped layers, which can be verified by plotting it.
If I comment out "model = model.model" (line 21 of keras/utils/visualize_util.py), the right model without the last layers is plotted. But I don't know what it means or how to let Keras know the layers have changed.

I actually ran into this recently, and I believe that this is an issue with how Model was rewritten to be more graph-based. In addition to the layers property models also have an outputs property, so when you pop layers off your last layer is left dangling in the model's output (which is why your model still thinks its output size is 1000).
A method to pop layers off and manage all the links correctly is needed, but until then I believe this would work: (my model compiles, but I haven't tried training it yet)
# ... Load pre-trained VGG16 model

model.layers.pop() # Get rid of the classification layer
model.layers.pop() # Get rid of the dropout layer
model.outputs = [model.layers[-1].outbound_nodes[0].output_tensors[0]]

Thank you. I downgraded to 0.3.1 where it works, so can't try your fix right now. I'm not sure it's all there is, because if I'm not mistaken plot() would draw all the popped layers (I tried popping many layers which all re-appeared), so maybe the copy of model.layers is stored somewhere.
As a side note, model.layers.pop() looked like a hack anyway. We did not push the layers there and should not expect to pop them without side effects. Maybe something like model.remove() would make more sense as a pendant to model.add().

@fchollet It could be possible to add into a Sequential model the possibility to pop the last layer and updating the output and outbound_nodes?

@albertomontesg like this?
def pop(self):
    '''Removes a layer instance on top of the layer stack.
    '''
    if not self.outputs:
        raise Exception('Sequentail model cannot be popped: model is empty.')
    else:
        self.layers.pop()
        if not self.layers:
            self.outputs = []
            self.inbound_nodes = []
            self.outbound_nodes = []
        else:
            self.layers[-1].outbound_nodes = []
            self.outputs = [self.layers[-1].output]
        self.built = False

@joelthchao Yes. That was exactly what I was thinking.

@joelthchao seems to work fine, indeed. Thanks!
P.S. This particular example runs into #2386 after that, so it works under 0.3.1 only.

@joelthchao Thanks for the idea.
@aristid I have changed the code to work with Keras version 1+
This issue can be considered close.

can confirm I'm still having this issue on keras 1.0.6.

#3252

I'm having a related problem #3310.

Same.

I encountered the same problem and I managed to pop the layers like this:
from convnetskeras.convnets import AlexNet
alexnet = AlexNet(weights_path="weights/alexnet_weights.h5")
pop unwanted last layer
alexnet.layers.pop() # Get rid of the classification layer softmax
alexnet.outputs = [alexnet.layers[-1].output]
alexnet.output_layers = [alexnet.layers[-1]]
alexnet.layers[-1].outbound_nodes = []
Hope this helps!

@joelthchao   I wanna use this pretrained vgg19.

x = Flatten(name='flatten')(x)
x = Dense(4096, activation='relu', name='fc1')(x)
x = Dense(4096, activation='relu', name='fc2')(x)
x = Dense(1000, activation='softmax', name='predictions')(x)
model = Model(img_input, x)
return model

i have only 8 labels to classlify.
But it is used by the keras function API ,not the Sequential model.
how can i use the model.layers.pop()
like this

x = Dense(1000, activation='softmax', name='predictions')(x)
model = Model(img_input, x)
model.layers.pop()
x = Dense(8, activation='softmax', name='predictions')(x)
loading weight
return model

Do you give me some advices? Thanks

@alyato No need to pop, just find the layer you want to connect to, connect to another layer and make another new model with correct input and output.

Snippet for your question:
from keras.models import Model
from keras.layers import Dense, Input

# Assume we have a pretrained model
net_input = Input(shape=(10))
net = Dense(4) (net_input)
net_output = Dense(1) (net)
model1 = Model(net_input, net_output)

# Get input
new_input = model1.input
# Find the layer to connect
hidden_layer = model1.layers[-2].output
# Connect a new layer on it
new_output = Dense(2) (hidden_layer)
# Build a new model
model2 = Model(new_input, new_output)

And this is the result
model1:
____________________________________________________________________________________________________
Layer (type)                     Output Shape          Param #     Connected to
====================================================================================================
input_1 (InputLayer)             (None, 10)            0
____________________________________________________________________________________________________
dense_1 (Dense)                  (None, 4)             44          input_1[0][0]
____________________________________________________________________________________________________
dense_2 (Dense)                  (None, 1)             5           dense_1[0][0]
====================================================================================================

model2:
____________________________________________________________________________________________________
Layer (type)                     Output Shape          Param #     Connected to
====================================================================================================
input_1 (InputLayer)             (None, 10)            0
____________________________________________________________________________________________________
dense_1 (Dense)                  (None, 4)             44          input_1[0][0]
____________________________________________________________________________________________________
dense_3 (Dense)                  (None, 2)             10          dense_1[0][0]
====================================================================================================

Also, there's an available method model.get_layer(name, idx) for easily getting intermediate layers out of a model.
https://keras.io/models/model/#methods

@joelthchao Thanks. I run it well. But I also don't know how to get the index of every layer.
Do i use the model.get_config() to count the index of every layer

@alyato Sort of, I use [x.name for x in model.layers] to find the index manually.

@joelthchao
Hi, I am trying to append vggnet-16 to an existing ResNet model, but after executing Pop() written by you, the output shape value is Multiple. How do I resolve this?

@srv902 Is it a sequential model or functional API model?

@joelthchao It is a functional API model.

@srv902 If the layer you want to connect to has multiple outputs, then you can only choose one output to connect. (I bet you are trying to connect to residual block.) BTW, you can provide a code snippet to help others to understand your problem.

@joelthchao why
hidden_layer = model1.layers[-2].output
not
hidden_layer = model1.layers[-1].output
if you are replacing only one layer?

Still has this problem!

Below method works:
from keras.models import Model
from keras.layers import Dense,Flatten
from keras.applications import vgg16
from keras import backend as K

model = vgg16.VGG16(weights='imagenet', include_top=True)

model.input

model.summary(line_length=150)

model.layers.pop()
model.layers.pop()

model.summary(line_length=150)

inp = model.input
out =model.layers[-1].output

model2 = Model(inp, out)  # create a new model which doesn't have the last two layers in VGG16
model2.summary(line_length=150)

reference here

Use model.pop(), don't use model.layers.pop(). Please refer to my comment and whole thread to understand it. Also, please search relative issue before asking.

I can confirm that keras 1.1.0 model.pop() does not remove the last layer.

I think I have a similar problem. I am running Keras 2.0.6 with Python 3.6.2 and Tensorflow-gpu 1.3.0.
I run this code after having hand built a vgg16 architecture and loaded the weights:
        model = self.model
        model.pop()
        for layer in model.layers: layer.trainable=False
        model.add(Dense(num, activation='softmax'))
        self.compile()

And when I check the graph in Tensorboard I see (check top left of attached picture) dense_3 connected to dropout_2 but dangling by itself. And then next to it I see dense_4, also connected to dropout_2.
Curiously, I get a low accuracy when running this on the Kaggle cats vs dog competition where I hover around 90% whilst others running this code (it's adapted from fast.ai) on top of Theanos easily get 97%. Perhaps my accuracy problem comes from somewhere else, but I still don't think dense_3 should be dangling there and I am wondering if this could be the source of my precision problem.
Secondary question: how could I definitely disconnect and remove dense_3?

@winzee
Have you tried this:
model = self.model
model.pop()
#add these
#https://github.com/flyyufelix/cnn_finetune/blob/master/vgg16.py
model.layers[-1].outbound_nodes = []
model.outputs = [model.layers[-1].output]
for layer in model.layers: layer.trainable=False
model.add(Dense(num, activation='softmax'))
self.compile()

@zo7 Can you comment on outbound_nodes ?
i.e. why this code is not sufficient?
model.layers.pop()
model.outputs = [model.layers[-1].output]

I am having a similar issue trying to pop the last layer of a ResNet50 model and add a new classifier.  Using Keras version 2.2.0 with tensorflow backend.  The previous fix in this thread for dealing with the outbound nodes and output is producing an error that I haven't seen mentioned yet. Is there a smarter way to pop the classification layer and add a new one, or a way to deal with this attribute error ?
from keras.models import Model
from resnet50 import ResNet50

model = ResNet50(weights='imagenet', include_top=True)`
model.layers.pop() # Get rid of the classification layer
model.outputs = [model.layers[-1].output]
model.output_layers = [model.layers[-1]] # added this line in addition to zo7 solution
model.layers[-1].outbound_nodes = []

This is the output
AttributeError                            Traceback (most recent call last)
<ipython-input-3-8d9365679847> in <module>()
     16 model.outputs = [model.layers[-1].output]
     17 model.output_layers = [model.layers[-1]] # added this line in addition to zo7 solution
---> 18 model.layers[-1].outbound_nodes = []

AttributeError: can't set attribute

As an alternative, I know that the include_top=False option will remove the classifier (and the Flatten layer), but I don’t know how to correctly attach the output of this form of ResNet to a new classifier (e.g., using model.layers[-1].output ?). Any advice on this?

pop() still does not work (Win 10, GPU Keras on CNTK backend, newest as of 2018-06-20).  To be more specific, when a model is copied from a known model and is of type Sequential, it does not work.  However, pop() will work on the original Model type model, so use it there to remove the last layer before creating a new Sequential type model from the original layers, i.e. vgg16.

pls fix this problem as soon as possible, please i need this!

https://stackoverflow.com/a/49403298/1179925
For some reason I need to build model with popped layer using Model before adding new layers to make things work.
def pop_layer(model):
    if not model.outputs:
        raise Exception('Sequential model cannot be popped: model is empty.')

    model.layers.pop()
    if not model.layers:
        model.outputs = []
        model.inbound_nodes = []
        model.outbound_nodes = []
    else:
        model.layers[-1].outbound_nodes = []
        model.outputs = [model.layers[-1].output]
    model.built = False

def get_model():
    #Fully convolutional part of VGG16
    model = VGG16(include_top=False, weights='imagenet')

    #Remove last max pooling layer
    pop_layer(model)

    #Freeze pretrained layers
    for layer in model.layers:
        layer.trainable = False

    model = Model(inputs=model.inputs, outputs=model.outputs)

    print('len(model.layers)', len(model.layers)) #
    print(model.summary()) #

    x = GlobalAveragePooling2D()(model.output)
    head = Dense(N_CLASS, activation='softmax')(x)

    model = Model(inputs=model.inputs, outputs=head)

    model.compile(optimizer=Adadelta(), loss='categorical_crossentropy', metrics=['accuracy'])

    print('len(model.layers)', len(model.layers)) #
    print(model.summary()) #

    return model

Sleep(1)

Hi,
I am using pre-trained VGG16 model and wanted to remove the last layer used in VGG16 for classification
For my code this worked:
model._layers.pop()
model.summary()

I have a similar case(https://stackoverflow.com/questions/63202651/how-to-remove-layers-from-a-keras-model-in-order-to-use-as-baseline-for-creating) using model._layers.pop() as @amandeep25  posted seems to work (model.summary() returns desired result) however when checking tensorboard or exporting the model to .pb (final goal is to compile for deployment on raspberry) the original VGG16 model is still there. Any suggestions?

@somewacko is there any way to pop a range of layers using indices? I mean without using loop or calling the same pop() function multiple times?

I believe model.layers is just a list, you can probably do:
model.layers = model.layers[:-3]  # everything but the last 3 layers


Cmake build of Apache Arrow fails using grpc's Findre2.cmake with "add_library cannot create imported target "RE2::re2" because another target with the same name already exists."

The grpc cmake file Findre2.cmake causes this cmake error when building downstream libraries that use both grpc and re2, specifically Apache arrow.
Error:
:info:build CMake Error at /opt/local/lib/cmake/grpc/modules/Findre2.cmake:29 (add_library):
:info:build   add_library cannot create imported target "re2::re2" because another target
:info:build   with the same name already exists.

Related:

<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>https://github.com/macports/macports-ports/pull/7791

What version of gRPC and what language are you using?
1.35.0
What operating system (Linux, Windows,...) and version?
macOS 10.15.7 19H524
Xcode 12.4 12D4e
Also observed on Linux; see above.
What runtime / compiler are you using (e.g. python version or version of gcc)
cmake
What did you do?
cmake build of Apache Arrow that uses grpcand re2  libraries.
See https://github.com/macports/macports-ports/blob/76927172cfe36acb7a6ea543223dd8849dbb0cb9/devel/apache-arrow/Portfile
See attached log for build details.
main.log
What did you expect to see?
Successful build.
What did you see instead?
The cmake error above.

Here are suggested fixes to this issue at the Apache Arrow jira: https://issues.apache.org/jira/browse/ARROW-11888?focusedCommentId=17300689&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17300689
This is caused by calling Findre2.cmake provided by gRPC twice.
Apache Arrow itself uses re2. So Apache Arrow calls find_package(re2) (from Findre2Alt.cmake in Apache Arrow). If re2 provides re2Config.cmake, the find_package(re2) uses the re2Config.cmake. If re2Config.cmake doesn't exist, it uses Findre2.cmake provided by gRPC. Findre2.cmake defines re2::re2 CMake target.
Apache Arrow Flight uses gRPC. gRPC uses re2. So gRPC tries to find re2 by find_package(re2 CONFIG). It uses re2Config.cmake. Note that we can use re2Config.cmake multiple times. Because re2Config.cmake doesn't define re2::re2 CMake target if there is already re2::re2 CMake target. If re2Config.cmake doesn't exist, Findre2.cmake in gRPC defines re2::re2 CMake target. Note that it doesn't check re2::re2 CMake target existence. So using Findre2.cmake in gRPC without re2Config.cmake twice causes the error.
There are some solutions:
1. re2 provides re2Config.cmake
2. gRPC's Findre2.cmake doesn't define re2::re2 multiple times:
diff --git a/cmake/modules/Findre2.cmake b/cmake/modules/Findre2.cmake
index 41df454713..906aabfeba 100644
--- a/cmake/modules/Findre2.cmake
+++ b/cmake/modules/Findre2.cmake
@@ -26,26 +26,28 @@ find_package(PkgConfig REQUIRED)
 pkg_check_modules(RE2 QUIET re2)
 if(RE2_FOUND)
   set(re2_FOUND "${RE2_FOUND}")
-  add_library(re2::re2 INTERFACE IMPORTED)
-  if(RE2_INCLUDE_DIRS)
-    set_property(TARGET re2::re2 PROPERTY
-                 INTERFACE_INCLUDE_DIRECTORIES "${RE2_INCLUDE_DIRS}")
-  endif()
-  if(RE2_CFLAGS_OTHER)
-    # Filter out the -std flag, which is handled by CMAKE_CXX_STANDARD.
-    # TODO(junyer): Use the FILTER option whenever CMake 3.6 (or newer)
-    # becomes the minimum required: that will allow this to be concise.
-    foreach(flag IN LISTS RE2_CFLAGS_OTHER)
-      if("${flag}" MATCHES "^-std=")
-        list(REMOVE_ITEM RE2_CFLAGS_OTHER "${flag}")
-      endif()
-    endforeach()
-    set_property(TARGET re2::re2 PROPERTY
-                 INTERFACE_COMPILE_OPTIONS "${RE2_CFLAGS_OTHER}")
-  endif()
-  if(RE2_LDFLAGS)
-    set_property(TARGET re2::re2 PROPERTY
-                 INTERFACE_LINK_LIBRARIES "${RE2_LDFLAGS}")
+  if(NOT TARGET re2::re2)
+    add_library(re2::re2 INTERFACE IMPORTED)
+    if(RE2_INCLUDE_DIRS)
+      set_property(TARGET re2::re2 PROPERTY
+		   INTERFACE_INCLUDE_DIRECTORIES "${RE2_INCLUDE_DIRS}")
+    endif()
+    if(RE2_CFLAGS_OTHER)
+      # Filter out the -std flag, which is handled by CMAKE_CXX_STANDARD.
+      # TODO(junyer): Use the FILTER option whenever CMake 3.6 (or newer)
+      # becomes the minimum required: that will allow this to be concise.
+      foreach(flag IN LISTS RE2_CFLAGS_OTHER)
+	if("${flag}" MATCHES "^-std=")
+	  list(REMOVE_ITEM RE2_CFLAGS_OTHER "${flag}")
+	endif()
+      endforeach()
+      set_property(TARGET re2::re2 PROPERTY
+		   INTERFACE_COMPILE_OPTIONS "${RE2_CFLAGS_OTHER}")
+    endif()
+    if(RE2_LDFLAGS)
+      set_property(TARGET re2::re2 PROPERTY
+		   INTERFACE_LINK_LIBRARIES "${RE2_LDFLAGS}")
+    endif()
   endif()
   message(STATUS "Found RE2 via pkg-config.")
   return()
3. Apache Arrow doesn't find dependencies (re2, gRPC and so on) if libarrow.a isn't required.
find_package(Arrow) in pyarrow always find Apache Arrow dependencies but it's only for libarrow.a. libarrow.so doesn't need to find them. pyarrow only uses libarrow.so. So we can avoid calling find_package(re2) and find_package(gRPC) for just building pyarrow.

As per https://github.com/google/re2/issues/304#issuecomment-808232056, I'm going to send a pull request to fix this.


[Velox] Fix some gaps in clang format

Importing some recent diffs I noticed a couple gaps in our clang formatting (make format-fix).

clang formatting does not run on renamed files, renamed files may also have code changes so these should be checked
as well
we do not check for new lines at the end of files

The latter requires using clang-format 16 or above.

Looks like we need Centos 9 to get clang-format >= 16
I'll wait for https://github.com/facebookincubator/velox/pull/9903


Support Android Nougat Noto Color Emoji

I see there is work towards supporting Android Nougat's changes with the Noto Color Emoji under the yifanfwu-update_google_nougat branch. How can I help to get this work completed?

v2.4.5 now contains Nougat images 👍


Unify the logic on sidecar injection and service creation

Description

#6967

Please reference the issue this PR will close: #[issue number]
Checklist
Please make sure you've  completed the relevant tasks for this PR, out of the following list:

[x] Code compiles correctly
[x] Created/updated tests
[x] Unit tests passing
[ ] End-to-end tests passing
[ ] Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]
[ ] Specification has been updated / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]
[ ] Provided sample for the feature / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

@yaron2

add unit test for new code

Please fix conflicts.

@JasonChen86899 please resolve the conflicts

@JasonChen86899 please resolve the conflicts

done

/open

@yaron2 this pr has closed for a long time, do it can be opened?  I have already resolved the conflicts and merge latest code

/reopen


Error on Step 8

I get the following error on step 8: Set-AzVirtualNetworkGateway : Vpn client configuration for gateway gatewayResourcid can have only one of the RadiusServer or vpnclient certificate or AAD Authentication settings.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 186c0697-792a-a913-1baa-0af86f7ef6de
Version Independent ID: 330e7927-57fd-75e2-26b1-05bd2e8f30c1
Content: Create an Azure Active Directory tenant for P2S VPN connections: Azure AD authentication
Content Source: articles/vpn-gateway/openvpn-azure-ad-tenant.md
Service: vpn-gateway
GitHub Login: @anzaman
Microsoft Alias: alzam

@szolla , Thank you for your feedback. We will investigate and get back to you shortly.

Before running Set-AzVirtualNetworkGateway -VirtualNetworkGateway $gw -AadTenantUri 'https://login.microsoftonline.com/3f52b087-c505-43e7-ac0e-f4c5e2f7a3c4' -AadAudienceId '41b23e61-6c1e-4545-b367-cd054e0ed4b4' -AadIssuerUri 'https://sts.windows.net/3f52b087-c505-43e7-ac0e-f4c5e2f7a3c4/'  You need some configuration in the P2S setting( $gw.VpnClientConfiguration can not be NULL). Otherwise, you will receive this error:

If you have no root certificates configured in VPN GW, the following cmd works:

$gw = Get-AzVirtualNetworkGateway -Name VPNGW -ResourceGroupName vpnrg
Set-AzVirtualNetworkGateway -VirtualNetworkGateway $gw -AadTenantUri 'https://login.microsoftonline.com/xxx' -AadAudienceId '41b23e61-6c1e-4545-b367-cd054e0ed4b4' -AadIssuerUri 'https://sts.windows.net/xxx/'
Note: https://docs.microsoft.com/en-us/azure/vpn-gateway/openvpn-azure-ad-tenant  step 3.8

If you already configured a root certificate in P2S VPN, you need to try:

$gw = Get-AzVirtualNetworkGateway -Name VPNGW -ResourceGroupName vpnrg
$gw.VpnClientConfiguration.VpnClientRootCertificates = @()
Set-AzVirtualNetworkGateway -VirtualNetworkGateway $gw -AadTenantUri 'https://login.microsoftonline.com/xxx‘ -AadAudienceId '41b23e61-6c1e-4545-b367-cd054e0ed4b4' -AadIssuerUri 'https://sts.windows.net/xxx/'

@szolla , Do you have any update for me?

@msrini-MSFT Thanks for checking in, i also opened up an MS support case on this and they gave me the same instructions above.  I was about to test these options out successfully.  One of the issues that I was also having was that I was attempting to use a Guest account in the AD Tenant to authenticate to the VPN and also to approve the creation of the Azure VPN Client App in the AD tenant.  Once i switched to using a cloud based user in the tenant i was working in, then it started to work properly.  Looks like someone already mentioned this issue in #42614 .  It should be stated in the document not to use Guest/B2B accounts until it is a supported option.

@szolla , thank you for your feedback. Referenced issue is already assigned to the author to validate and make necessary changes. I will close this issue as your issue is resolved.


Delay in-between clicks on Update to avoid client crashes

When clicking fast, sometimes(often) the Penumbra hook runs into an issue and locks the client and/or crashes it altogether.
I propose the following:
ffxiv_dx11_sxXQQwuqMJ.webm
Tested spam clicking without delay: very frequent crashes
Tested spam clicking with the delay: no crash

Drafted for further testing


Add online and offline html export support

Closes https://github.com/vega/vl-convert/issues/33, cc @joelostblom
This PR adds support for converting Vega and Vega-Lite charts to live HTML documents. There is a "bundle" option that controls whether the JavaScript dependencies should be loaded from a CDN, or whether they should be inlined into the resulting HTML file.
bundle=False
When bundle is False, this follows the Vega Embed directions to load vega, vega-lite, and vega-embed from jsdelivr
bundle=True
When bundle is True, things are a bit more involved. We already inline Vega and several versions of Vega-Lite into the VlConvert executables, so I wanted to avoid including additional copies for the purpose of HTML export.  But in order to use the JS deps inlined into VlConvert, they need to be bundled.  I found that the deno_emit project provides a Rust crate that uses SWC to bundle JavaScript / TypeScript dependencies, and this ended up working well.
One note, the bundled code isn't fully minimized yet, but I have an open PR that will expose SWC's minify option. See https://github.com/denoland/deno_emit/pull/141. Currently, the resulting HTML files start at ~1.6MB, but this will drop to ~1MB when minification is enabled.
The bundling process takes about 1.5s on my machine. Because this is pretty slow, I decided to cache the bundle results, so that subsequent HTML exports that use the same Vega-Lite version will be fast (10-20ms).
Custom JavaScript bundles
Additionally, a javascipt_bundle Python function is added that can be used to create bundles with custom JavaScript logic that references Vega Embed, Vega, and Vega-Lite.  The idea is that Vega-based systems like Altair can use this to build additional integrations.
Integrations
The most immediate application of the HTML export is to remove the altair_viewer dependency in Altair's html export when inline=True.
We could also use this to add an "html-offline" Altair renderer, though this could result in large notebooks as every individual Chart would be over 1MB.
Another use-case I have in mind is to use the javascript_bundle function to create offline bundles for Altair's JupyterChart. This is why I added support for the lodash debounce function as well, since this is the only import, in addition to vegaEmbed, that JupyterChart's JavaScript logic uses.  The cool thing about this approach is that we can build the offline bundle on the fly (in under 2s) without an internet connection required.

This functionality is really neat! And I'm impressed with how fast vl-convert and vegafusion goes from discussion in issues to implementation =)

The most immediate application of the HTML export is to remove the altair_viewer dependency in Altair's html export when inline=True. We could also use this to add an "html-offline" Altair renderer, though this could result in large notebooks as every individual Chart would be over 1MB.

Thinking more about this passage and all the possible different use case scenarios for this functionality, this what I come up with so far:

I want to make a single chart and share it with offline access. This is already covered by this PR in an effective way.
I want to make a notebook with many offline charts and share it with someone. As you noted, this notebook size will become rather large if vega/vega-lite is bundled once per chart. Would it be possible to instead bundled it one per notebook and reference the location in the notebook where the spec is bundled from all the charts? I'm thinking something similar to how you can paste an image into the notebook and it gets bundled/embedded as a base64 image string in a notebook "attachment" and then that attachment ID can be referenced from any other cell to render the image there. Another approach might be to look at what panel does when they enable their
I want to make either a notebook or a single chart with offline access, but I don't care about sharing the chart, I just need to be able to work myself offline. In this case, would it be possible to somehow not bundle at all and instead point to the local vl-convert installation and use vega/vega-lite from there instead of from the CDN? This would probably have to be exposed as a different local renderer or at least a parameter in the html-offline renderer you suggest to tell it apart from the above bundling scenarios.

I thought more about point 3 above and I think the fallback option could be quite useful. That will guarantee that altair notebooks works as long as vl-convert is installed even without access to the internet, such as on a flight or something without changing anything in the notebook itself.
Also related, I know that if I load an altair chart once while I am online, I am then able to create additional charts in the same notebook even if I go offline as long as I don't restart my kernel (or was it reload jupyterlab?). Maybe there is something here with how vega/vega-lite is cached after it is loaded for one chart that could be used for the notebook offline bundling to prevent including the 1MB bundle in each chart (if the "attachment" approach I suggested above does not work).

would it be possible to somehow not bundle at all and instead point to the local vl-convert installation and use vega/vega-lite from there instead of from the CDN?

VlConvert doesn't exactly have a local installation of the Vega JS libraries. The minified source code of all of the individual files that make up these libraries are embedded in the VlConvert executable. Some form of bundling is required to make these suitable for use inside a <script> tag of an HTML document. So I think (2) and (3) are really the same challenge.
The trick we used with plotly, for the classic notebook, was to have an initialize step (pio.renderers.default = "notebook") load the bundled Plotly.js library into the notebook using ipython_display.display_html.  Then individual chart displays would assume that Plotly was already available globally.
I don't recall why we didn't use this approach for JupyterLab, but it might be possible to do something similar in Altair when calling alt.renderers.enable('html-offline'). One caveat is that approach does not work for alternative notebook environments that isolate chart cell's using iframes (I think Colab does this).

Thanks for explaining and elaborating on how vl-convert works. I like the approach you mentioned plotly takes for bundling once in the notebook and if that works with jupyterlab as well, then I think it could be a viable approach. Isolated environment such as collab could still rely on approach 1 of bundling with each chart, we just have to make that explicit in the docs.

VlConvert doesn't exactly have a local installation of the Vega JS libraries. The minified source code of all of the individual files that make up these libraries are embedded in the VlConvert executable.

Does this mean that we need to be online in order to make a chart that works offline? Or is everything needed to create the bundle that goes in the offline chart already part of vl-convert? If it is the latter, then could we not somehow use that as a fallback rendering option? Another approach would be a a clear warning message to users when they fail to render a chart because they went offline, with instructions of how to enable an offline renderer. Having that said, my impressions from the issue list is not that this is a commonly reported hickup, so I guess this functionality is more of a nice to have.

Does this mean that we need to be online in order to make a chart that works offline? Or is everything needed to create the bundle that goes in the offline chart already part of vl-convert?

This PR adds the logic needed to create the bundle using the JS files that are embedded in the executable (this is what the deno_emit Rust crate does), so no internet connection is required.

If it is the latter, then could we not somehow use that as a fallback rendering option?

I'm not sure if we can auto-detect this. For security reasons, I don't think we want Altair attempting network requests to test whether it has internet access. Even if we did, this would only test that the kernel has internet access, not that the browser displaying the chart has internet access.  So I would lean toward adding an offline rendering option and having docs and error messages direct people to this.


Removal of a data object fails with SYS_NO_GOOD_REPLICA if no good replica exists on the file

A consortium member tried to remove a data object which only had a stale replica on it.  This resulted in a SYS_NO_GOOD_REPLICA error and the stale replica was not deleted.
When we disabled logical quotas, the removal of the data object succeeded.

Noting here that @alanking mentioned that the following function could be related:
https://github.com/irods/irods_rule_engine_plugin_logical_quotas/blob/c7e9dd756d43b8bbcdd2c6a669620d2cb2442f62/src/handler.cpp#L1217-L1265

This is a duplicate of https://github.com/irods/irods_rule_engine_plugin_logical_quotas/issues/111. The issue was fixed in <IP_ADDRESS>. Closing this one.
For reference, here's what that same method looked like before #111 was resolved (notice: SYS_NO_GOOD_REPLICA is not being handled):
https://github.com/irods/irods_rule_engine_plugin_logical_quotas/blob/<IP_ADDRESS>/src/handler.cpp#L1184-L1210


Scrollbar issue in core Banktags

Type
Incorrect behavior
Operating System
Windows
OS Version
Windows 11
Bug description
Scrollbar stuck in core layout banktags, cannot expand or decrease the size of a banktag
BTL-plugin is disabled, only using core Bank tag plugin
Screenshots or videos
https://github.com/user-attachments/assets/0bdb8d7c-58ed-4e97-a0a1-12717fbe0b56
RuneLite version
RL version: <IP_ADDRESS> Launcher version 2.7.1
Logs
No response

What if you right-click the tag tab on the left side and click enable layout

Layout is already enabled in this banktag

https://github.com/user-attachments/assets/a0ae0dda-5f97-4974-b949-f585c74d64f0
This looks ok to me.

For some reason my scrollbar doesnt change whenever i enable layout on a new banktag unlike in the video you posted.
I even tried changing the clientsize but im unable to scrolldown to make the tab larger in this clip;
https://github.com/user-attachments/assets/9f9a2eeb-0a53-4694-abb9-91a5cfe273f0

Did you try safe mode?

I just launched RL in safe mode and the issue persists

Am also experiencing a static bar when i turn on layout via RuneLite and not the custom layouts plugin hub plugin.

Can you reproduce it from a default profile with default settings? There must be some setting or some other plugin messing with it.

I just reloaded my client and seems like something has changed in the meantime, the scrollbar behaves the same as the first clip you posted in this thread now. So the issue i had in the first clip i posted i no longer experience.
Do you intend on having the scrollbar adjust to the size of the banktag similar to how BTL does it? Currently with layout enabled you can scroll very far down for some reason.

It resizes it to the max possible bank size. It is because you can drag items to anywhere in the bank. Otherwise it becomes hard to add items to a new row.

To bump on this issue, Im 99% sure the issue i am experiencing is because of the Potion Storage being unlocked on my account. Whenever i log in on any other account on the same client with the same runelite profile the scrollbar works as intended.

To bump on this issue, Im 99% sure the issue i am experiencing is because of the Potion Storage being unlocked on my account. Whenever i log in on any other account on the same client with the same runelite profile the scrollbar works as intended.

It works okay for me with potion storage unlocked:


Panic after upgrading node-template from 2.0.1 to frontier

My node is forked off from node-template directory of this repo. It (with chain spec dev) worked fine till version 2.0.1 but after upgrading to frontier branch, the dev node crashes on running with error
Thread 'main' panicked at 'assertion failed: frame_system::Module::<T>::inc_consumers(&account).is_ok()', .cargo/git/checkouts/substrate-7e08433d4c370a21/4888ac6/frame/session/src/lib.rs:447

Before the upgrade, in my runtime's lib.rs construct_runtime macro, I had session pallet declared before balances pallet like this
construct_runtime!(
    pub enum Runtime where
        Block = Block,
        NodeBlock = opaque::Block,
        UncheckedExtrinsic = UncheckedExtrinsic
    {
        System: system::{Module, Call, Config, Storage, Event<T>},
        RandomnessCollectiveFlip: randomness_collective_flip::{Module, Call, Storage},
        Timestamp: timestamp::{Module, Call, Storage, Inherent},
        Session: pallet_session::{Module, Call, Storage, Event, Config<T>},
        ......
        Balances: balances::{Module, Call, Storage, Config<T>, Event<T>},

But after moving balances pallet above session pallet, the panic doesn't occur. As per the docs of construct_runtime, the dependent pallet should follow the dependency but there is no dependency between balances and session. Are there other considerations when putting pallets in construct_runtime!?
Here is my code, just in case.

CC @shawntabrizi @thiolliere

but there is no dependency between balances and session.

The account in the keys in the genesis config of session must be existing accounts (i.e. it has an existential balance in balance module or whatever other provider of existence).
As your genesis config relies on pallet balances to make those account existing it is necessary for the session genesis config to be executed after balances genesis config.
(Maybe this can be improved by automatically give existential deposit to the session keys accounts, but I'm not sure of the real interest)

Got it. Thanks

I guess we can close it, feel free to reopen if you have suggestion


When you put a slash in the name of a VM it dissapears.

Describe the bug
When you put a slash in the name of a VM it dissapears.
To Reproduce
Steps to reproduce the behavior:

Go to Create VM
Click on Name
Add a name with a slash for example cars/tesla/wakjdhawkdhawkjd
Hit save
The VM wont save.

Duplicate of #210


Support USB Passthrough

Summary
PR Checklist

Is this a multi-tenancy feature/bug?

[ ] Yes, the relevant RBAC changes are at:

Do we need to backport changes to the old Rancher UI, such as RKE1?

[ ] Yes, the relevant PR is at:

Are backend engineers aware of UI changes?

[ ] Yes, the backend owner is:

Related Issue https://github.com/harvester/harvester/issues/5763

Occurred changes and/or fixed issues

Technical notes summary

Areas or cases that should be tested

Disable/Enable PCI addon -> USB page visibility
Enable/Disable USB device
Assign PCI/USB devices to VMs

Areas which could experience regressions

PCI devices page
Screenshot/Video

USB devices

Groups

VM edit mode

Few questions, not sure whether it's Harvester version issue.
I opened the USB Device Page with embedded Rancher and enabled one of them. Other USB devices show weird status.

I tried to create a virtual machine. After I clicked the Create button in virtual machine image page. It showed this error message. It seems some variables are undefined in some cases.

Few questions, not sure whether it's Harvester version issue. In my local Harvester, I just enabled pcidevices-controller Add-on. Did I miss something?
I opened the USB Device Page with embedded Rancher and enabled one of them. Other USB devices show weird status.

I tried to create a virtual machine. After I clicked the Create button in virtual machine image page. It showed this error message. It seems some variables are undefined in some cases.

I think this is a bug, let me take a look.

I opened the USB Device Page with embedded Rancher and enabled one of them. Other USB devices show weird status.

It's fixed in https://github.com/harvester/dashboard/pull/1069/commits/02fe957a5c77d34a6db6a9e83b67cc89cbe90807

Found a issue. After I enabled the USB Device, the Claimed By showed nothing.

Found a issue. After I enabled the USB Device, the Claimed By showed nothing.

Fixed, thanks!

The enable link is failed when Pci-addon is disabled.

Not sure why but sort the column causes enable/disable passthrough button switch.
button_switch.webm

Enable group dialog is empty.

Why I can't disable usb device ? Same for disable group button
cc @Yu-Jack

Why I can't disable usb device ? Same for disable group button cc @Yu-Jack

It seems the Add-on is restarted. Because I patched the daemonset image directly, so it will be reset if Add-on is restarted. Please check the daemonset(harvester-system/harvester-pcidevices-controller) image is jk82421/pcidevices:v<IP_ADDRESS> or not before testing.

It's nice to block user create VM with un-existed or un-enabled device name if directly edit YAML. cc @Yu-Jack

Enable group dialog is empty.

The confirmation dialog is required. I don't know if we want to put a warning message in that dialog, like we do for  PCI devices. @Yu-Jack what do you think ?

The confirmation dialog is required. I don't know if we want to put a warning message in that dialog, like we do for PCI devices. @Yu-Jack what do you think ?

@torchiaf  We need to add a warning message, but it's different from PCI devices. Here it is. cc @bk201 @ibrokethecloud

Please re-enable the USB device if the device path changes in the following situations: 1) Re-plugging the USB device. 2) Rebooting the node.

BTW, this is original warning message of PCI devices.

Please be careful not to use host-owned PCI devices (e.g., management and VLAN NICs). Incorrect device allocation may cause damage to your cluster, including node failure.

Thanks @Yu-Jack , everything should be fixed now.

@mergify backport release-harvester-v1.4


[PKG-3584] itables 1.6.3

itables 1.6.3 :snowflake:
Destination channel:  Snowflake
Links

PKG-3584
Upstream repository
Relevant dependency PRs: N/A

Explanation of changes:

Ignoring upstream tests due to missing dependencies.

Linter check found the following problems:
ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_29laxgsd7p/clone` failed. (See above for error)
The following problems have been found:
===== WARNINGS =====

clone/recipe/meta.yaml:20: host_section_needs_exact_pinnings: Linked libraries host should have exact version pinnings.

===== ERRORS =====

clone/recipe/meta.yaml:45: missing_license_family: The recipe is missing the about/license_family key.
===== Final Report: =====
1 Error and 1 Warning were found

Linter check found the following problems:
ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_307kxkesr9/clone` failed. (See above for error)
The following problems have been found:
===== WARNINGS =====

clone/recipe/meta.yaml:20: host_section_needs_exact_pinnings: Linked libraries host should have exact version pinnings.

===== ERRORS =====

clone/recipe/meta.yaml:45: missing_license_family: The recipe is missing the about/license_family key.
===== Final Report: =====
1 Error and 1 Warning were found

What test dependencies are missing?

world_bank_data, shiny, and polars. Also jupytext for a couple platforms. There's built in handling if polars is missing, and the other ones are only used in a few test modules, so we could possibly skip a bunch of the tests.

Also, I'd like to get the opinions of others on https://github.com/mwouts/itables/blob/main/setup.py#L20-L33. I'm not a big fan of this personally and would prefer to download these ourselves, but I would like to get the opinions fro others before doing this.

Oddly, this code is overwriting files that already exist in the pypi source. I suspect the download is meant to be skipped if the external directory is already present, but it's missing an indent.

@ViridianMelody Oh, in that case we could just apply a patch to remove the lines I pointed to so that it doesn't download anything. I didn't realize yesterday that the files were including in the sdist (the file we download from PyPI).

What test dependencies are missing?

world_bank_data, shiny, and polars. Also jupytext for a couple platforms. There's built in handling if polars is missing, and the other ones are only used in a few test modules, so we could possibly skip a bunch of the tests.

I would skip test files https://github.com/mwouts/itables/tree/v1.6.3/tests that require these dependencies

I would enable the test suite and skip test files https://github.com/mwouts/itables/tree/v1.6.3/tests that require these dependencies

@JeanChristopheMorinPerso @skupr-anaconda : Next problem, the sdist doesn't include the tests. Should I download the source from Github to extract those tests from?

Linter check found the following problems:
ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_68q_j5y5tl/clone` failed. (See above for error)
The following problems have been found:
===== WARNINGS =====

clone/recipe/meta.yaml:25: host_section_needs_exact_pinnings: Linked libraries host should have exact version pinnings.
===== Final Report: =====
0 Errors and 1 Warning were found

I would enable the test suite and skip test files https://github.com/mwouts/itables/tree/v1.6.3/tests that require these dependencies

@JeanChristopheMorinPerso @skupr-anaconda : Next problem, the sdist doesn't include the tests. Should I download the source from Github to extract those tests from?

We could download the package from GH or simply skip the tests (like it is right now).

Linter check found the following problems:
ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_7425a8qel8/clone` failed. (See above for error)
The following problems have been found:
===== WARNINGS =====

clone/recipe/meta.yaml:25: host_section_needs_exact_pinnings: Linked libraries host should have exact version pinnings.
===== Final Report: =====
0 Errors and 1 Warning were found


Initial ReST linting

Closes #1947
Notes

This PR adds basic ReST linting. The module supports no configuration beyond setting a linting level. Default warning is used.

One document doesn't pass information level diagnostics.

@saadrahim There is doc8, but most linters tend to only be compatible with markdown.


Getting proxy authentication popup in between while running test cases

Hi,
Technology : Angular 8 + protractor
Issue:
When I am running all my test cases (e.g.40 test cases), **_suddenly a proxy authentication popup is coming to provide my network credentials.

_** As result my test cases all are failure due to timeout issues.
But before running, I have already provided all my proxy in my package.json.
Pls help me out to solve this problem.

@AsrRajiv I assume this comes up at the beginning when you just launch the test? If so you can add the credentials to the browser.baseURL in this way<EMAIL_ADDRESS>


Add send_meeting_invitations_or_cancellations

update_item(...) used send_meeting_cancellations, which contains a
subset of send_meeting_invitations_or_cancellations.
But for certain operations, it's necessary to provide all options of
send_meeting_invitations_or_cancellations.
In regards to #121

Can anyone test this with Exchange 2007?

I'll test it as soon as I have some time! I'll probably get back to you about this in the next couple of days.
Thanks in advance, btw!

Tried it and works!


net/http.Request.UserAgent() string not a valid UTF-8

What version of Go are you using (go version)?
go version go1.10 linux/amd64
Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
Linux, amd-64.
What did you do?
Serving an HTTP request I've used a User-Agent., below example from a real world server:
https://play.golang.org/p/hyPt1AM19zO
Saved a User-Agent string and read in another programming language caused it to panic.
(check: https://github.com/golang/protobuf/issues/484)
What did you expect to see?
A valid UTF-8 string to be produced.
What did you see instead?
Invalid string as in an example.

In your example the hex-encoded data corresponds to the Go string literal:
"Mozilla/5.0 (iPhone; CPU iPhone OS 11_2_1 like Mac OS X) AppleWebKit/604.4.7 (KHTML, like Gecko) Mobile/15C153 [FBAN/FBIOS;FBAV/<IP_ADDRESS>.73;FBBV/57050710;FBDV/iPhone7,2;FBMD/iPhone;FBSN/iOS;FBSV/11.2.1;FBSS/2;FBCR/OrangeEspa\xf1a;FBID/phone;FBLC/es_ES;FBOP/5;FBRV/0]"
Note the "OrangeEspa\xf1a".
Probably the bytes received in the HTTP request were exactly those bytes and therefore were not valid UTF-8. The HTTP API records the bytes received. There is no guaranteed coercion to valid UTF-8 in this API. If you are going to use them as a protobuf string (or pass them to pickier languages in any other form), then it's true you may have to guard against invalid UTF-8. But that's not the HTTP request reader's job.

Thanks for a reply and clarification. The example and hex encoding was hand generated by me (didn't know how to inline a incorrect rune).
I've deployed a validation but I thought it may be interesting for someone to leave trace of such problem.


UnboundLocalError at /user/ local variable 'Token' referenced before assignment

Can't remember - is theToken required for the default 3bot installation or is it only required if https://github.com/3bot/3bot-hook is used?

It was not, but it is fixed sice https://github.com/3bot/3bot/commit/4e8bc3c753b91d1e707c8fe4abcefa2fff87eb00


Error when running a simple Hello World example (Module parse failed: Unexpected token (630:66))

I'm getting this weird error when I try to run an app with vue-dapp installed:
 INFO  Starting development server...
98% after emitting CopyPlugin

 ERROR  Failed to compile with 1 error                                                                                23:49:45

 error  in ./node_modules/vue-dapp/dist/index.esm.js

Module parse failed: Unexpected token (630:66)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
| }
|
> const WalletConnectProviderDerived = window.WalletConnectProvider?.default || null;
| class Walletconnect {
|     static async check() {

 @ ./src/main.js 7:0-35 9:8-15
 @ multi (webpack)-dev-server/client?http://<IP_ADDRESS>:8080&sockPath=/sockjs-node (webpack)/hot/dev-server.js ./src/main.js

It's just a basic hello world project at this point.
main.js:
import { createApp } from 'vue';
import App from './App.vue';
import { VueDapp } from 'vue-dapp';

const app = createApp(App);

app.use(VueDapp);

app.mount('#app');

App.vue:
<template>
  <vdapp-board />

Hello

</template>

And that's it.
I don't even need WalletConnect, I just want to run a simple app to try vue-dapp out.
Any idea how to solve this?

I see issue #17 already references this error.


Playwright tests failing on linux complaining about missing dependencies

Build Information
Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=742280
Build error leg or test failing: Aspire.Workload.Tests.EmptyTemplateRunTests.Aspire.Workload.Tests.EmptyTemplateRunTests.ResourcesShowUpOnDashboad
Pull request: https://github.com/dotnet/aspire/pull/4914
Aspire.Workload.Tests: [apphost] info: Aspire.Hosting.DistributedApplication[0]
Aspire.Workload.Tests: [apphost]       Distributed application started. Press Ctrl+C to shut down.
Aspire.Workload.Tests: -- Ready to run tests --
    Aspire.Workload.Tests.EmptyTemplateRunTests.ResourcesShowUpOnDashboad [FAIL]
      System.AggregateException : One or more errors occurred. (
      ╔══════════════════════════════════════════════════════════╗
      ║ Host system is missing dependencies to run browsers.     ║
      ║ Please install them with the following command:          ║
      ║                                                          ║
      ║     sudo pwsh bin/Debug/netX/playwright.ps1 install-deps ║
      ║                                                          ║
      ║ Alternatively, use apt:                                  ║
      ║     sudo apt-get install libgbm1\                        ║
      ║         libxkbcommon0                                    ║
      ║                                                          ║
      ║ <3 Playwright Team                                       ║
      ╚══════════════════════════════════════════════════════════╝)
      ---- Microsoft.Playwright.PlaywrightException :
      ╔══════════════════════════════════════════════════════════╗
      ║ Host system is missing dependencies to run browsers.     ║
      ║ Please install them with the following command:          ║
      ║                                                          ║
      ║     sudo pwsh bin/Debug/netX/playwright.ps1 install-deps ║
      ║                                                          ║
      ║ Alternatively, use apt:                                  ║
      ║     sudo apt-get install libgbm1\                        ║
      ║         libxkbcommon0                                    ║
      ║                                                          ║
      ║ <3 Playwright Team                                       ║
      ╚══════════════════════════════════════════════════════════╝
      Stack Trace:
           at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
           at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
           at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout)
        /_/tests/Aspire.Workload.Tests/WorkloadTestsBase.cs(28,0): at Aspire.Workload.Tests.WorkloadTestsBase.CreateBrowser()
           at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
           at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
           at System.Lazy`1.CreateValue()
        /_/tests/Aspire.Workload.Tests/WorkloadTestsBase.cs(37,0): at Aspire.Workload.Tests.WorkloadTestsBase.CreateNewBrowserContextAsync()
        /_/tests/Aspire.Workload.Tests/EmptyTemplateRunTests.cs(22,0): at Aspire.Workload.Tests.EmptyTemplateRunTests.ResourcesShowUpOnDashboad()
        --- End of stack trace from previous location ---
        ----- Inner Stack Trace -----
        /_/src/Playwright/Transport/Connection.cs(206,0): at Microsoft.Playwright.Transport.Connection.InnerSendMessageToServerAsync[T](ChannelOwner object, String method, Dictionary`2 dictionary, Boolean keepNulls)
        /_/src/Playwright/Transport/Connection.cs(532,0): at Microsoft.Playwright.Transport.Connection.WrapApiCallAsync[T](Func`1 action, Boolean isInternal)
        /_/src/Playwright/Core/BrowserType.cs(56,0): at Microsoft.Playwright.Core.BrowserType.LaunchAsync(BrowserTypeLaunchOptions options)
        /_/tests/Shared/Playwright/PlaywrightProvider.cs(36,0): at Aspire.Workload.Tests.PlaywrightProvider.CreateBrowserAsync(BrowserTypeLaunchOptions options)
        /_/tests/Aspire.Workload.Tests/WorkloadTestsBase.cs(24,0): at Aspire.Workload.Tests.WorkloadTestsBase.<>c.<<CreateBrowser>b__5_0>d.MoveNext()

Error Message
Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "Host system is missing dependencies to run browsers",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

This shouldn't be happening because the queue should have the dependencies installed on the vm image itself (https://github.com/dotnet/dnceng/issues/3122).
cc @joperezr

Known issue validation
Build: :mag_right: https://dev.azure.com/dnceng-public/public/_build/results?buildId=742280
Error message validated: [Host system is missing dependencies to run browsers]
Result validation: :white_check_mark: Known issue matched with the provided build.
Validation performed at: 7/16/2024 4:31:26 AM UTC

Report

Build
Definition
Test
Pull Request

742611
dotnet/aspire
Aspire.Workload.Tests.Xunit_PerTestFrameworkTemplatesTests.WorkItemExecution
dotnet/aspire#4926

742506
dotnet/aspire
Aspire.Workload.Tests.Xunit_PerTestFrameworkTemplatesTests.WorkItemExecution
dotnet/aspire#4923

742505
dotnet/aspire
Aspire.Workload.Tests.Xunit_PerTestFrameworkTemplatesTests.WorkItemExecution
dotnet/aspire#4924

742495
dotnet/aspire
Aspire.Workload.Tests.Xunit_PerTestFrameworkTemplatesTests.WorkItemExecution
dotnet/aspire#4850

742419
dotnet/aspire
Aspire.Workload.Tests.Xunit_PerTestFrameworkTemplatesTests.WorkItemExecution
dotnet/aspire#4805

742280
dotnet/aspire
Aspire.Workload.Tests.Xunit_PerTestFrameworkTemplatesTests.WorkItemExecution
dotnet/aspire#4914

Summary

24-Hour Hit Count
7-Day Hit Count
1-Month Count

6
6
6

Related to https://github.com/dotnet/aspire/issues/4623 .

These tests have been disabled with https://github.com/dotnet/aspire/pull/4922 . Closing, this in favor of the original issue.


[VS Code] Joining collaboration session: The specified value isn’t a valid Live Share URL. Please check the link provided by the host and try again.

<!-- Said code wasn't valid after copy and paste
For Visual Studio problems/feedback, please use the Report

@wjones30309  Hey thank you for reporting this.This error indicates the session URL you tried to join wasn’t valid, which is typically due to a copy/paste error. If you simply retrieve the correct URL from the host, and try joining again, you should be unblocked . I’m going to close this issue as by design, but please re-open it if you encounter this error for a session URL you know is valid. Thanks!


Async.Catch code snippet is broken

https://msdn.microsoft.com/visualfsharpdocs/conceptual/async.catch['t]-method-[fsharp]

Just issued a fix. Let's see how it propagates. @vasily-kirichenko

Fix is now live and the code snippet is fixed.


Bash-Completion Errors

I've installed this dotfiles-repo on a newly installed Yosemite Machine. I also ran brew.sh after installing homebrew. Everything works as expected, but not in the terminal.
After reboot each terminal starts with lots of errors:
-bash: declare: -A: invalid option
declare: usage: declare [-afFirtx] [-p] [name[=value] ...]
-bash: complete: -D: invalid option
complete: usage: complete [-abcdefgjksuv] [-pr] [-o option] [-A action] [-G globpat] [-W wordlist] [-P    prefix] [-S suffix] [-X filterpat] [-F function] [-C command] [name ...]

Also, when i want to use bash completion, something like this happens:
mv Dropb-bash: ${prev,,}: bad substitution

I typed "mv Dropb" and pressed the Tab-key then. The other characters "-bash: $(prev..." occur then and restricts me to use the bash completion as it was possible before installing these dotfiles.
which part of the dotfiles should i change (or delete) to fix these issues?

See #536
@mathiasbynens you might want to add this to the readme :)

Thanks a lot!
By the way, these both lines will fix these issues, maybe they could be part of the brew.sh script?
sudo bash -c 'echo /usr/local/bin/bash >> /etc/shells'
chsh -s /usr/local/bin/bash

^ +1

^ +1

^ +1 thanks @itinance

+1... and +1 thanks @itinance!

+1... and +1 thanks @itinance!

Thanks @itinance

+1 thanks @itinance

+1 thanks @itinance

Here is a article on updating Bash on Mac OS X.

^ +1 thanks @itinance

@itinance Here’s a slightly more robust version:
if ! grep '/usr/local/bin/bash' /etc/shells; then
  echo '/usr/local/bin/bash' | sudo tee -a /etc/shells;
  chsh -s /usr/local/bin/bash;
fi;

Thanks @itinance and @mathiasbynens

it's not work on subl
it is still using /bin/bash so I can see
error
"usr/local/share/bash-completion/bash_completion: line 1812: declare: -A: invalid option
declare: usage: declare [-afFirtx] [-p] [name[=value] ...]
/usr/local/share/bash-completion/bash_completion: line 1949: complete: -D: invalid option
complete: usage: complete [-abcdefgjksuv] [-pr] [-o option] [-A action] [-G globpat] [-W wordlist] [-P prefix] [-S suffix] [-X filterpat] [-F function] [-C command] [name ...]
"
but i want to change new bash how to do ? i already try to use
if ! grep '/usr/local/bin/bash' /etc/shells; then
echo '/usr/local/bin/bash' | sudo tee -a /etc/shells;
chsh -s /usr/local/bin/bash;
fi;
on root and mine

@sd9972 did you open a new shell to test out the results of your chsh operation? Also, I would not chsh for root, since /usr/local/bin/bash can potentially be deleted quite easily.

@jeffbyrnes Thanks for help. :)
http://imgur.com/EpFP792 this is my chsh status
http://imgur.com/9cshkGN it is my sublime

@sd9972 can’t really tell what that chsh status means… in any case, a few things to check:

Did you brew install bash to install a newer version of bash?
Did you verify that /usr/local/bin/bash is in your /etc/shells?
What do you see when you echo $SHELL?
What do you get when you echo $0?

Please just copy & paste the output of your shell here, no need to take screenshots, you can use fenced code blocks to make it render more clearly.

sd9972 at Dongs-MBPr in ~
$ brew install bash
Warning: bash-4.3.42 already installed

sd9972 at Dongs-MBPr in ~
$ cat /etc/shells
# List of acceptable shells for chpass(1).
# Ftpd will not allow users to connect who are not using
# one of these shells.

/usr/local/bin/bash
/bin/bash
/bin/csh
/bin/ksh
/bin/sh
/bin/tcsh
/bin/zsh
sd9972 at Dongs-MBPr in ~
$ echo $SHELL
/usr/local/bin/bash

sd9972 at Dongs-MBPr in ~
$ echo $0
-bash

Well, that all looks good. Perhaps try running chsh -s /usr/local/bin/bash again, just for your user?

sd9972 at Dongs-MBPr in ~
$ chsh -s /usr/local/bin/bash
Changing shell for sd9972.
Password for sd9972:
chsh: no changes made

and
Sublime text console
<img width="226" alt="2016-01-15 5 18 30" src="https://cloud.githubusercontent.com/assets/13957440/12336361/8287703c-bb47-11e5-9d65-f196049e6de7.png">

@sd9972 damn. Yeah, really not sure what‘s going on there. Do you have any other shell stuff loaded up? bash-it or something similar?

i don't use bash-it or something about bash shell w/o this !!!

@sd9972 I’m at a loss as to what’s missing from your setup, and why the version of bash installed via brew isn’t the bash your shell is using. Sorry I can’t be of more help!

@mathiasbynens just did a fresh install and noticed this issue as well, wondering if it's worth adding your solution in somewhere to automate this fix?

I'm also getting this error:
chsh: Operation was denied because the current credentials do not have the appropriate privileges.  Operation was denied because the current credentials do not have the appropriate privileges.
chsh: no changes made

I've tried running as sudo as well and the user is an admin, any ideas please?

Dang, now I can't open Terminal or iTerm :(

༼ つ ◕_◕ ༽つ⚡  brew install bash
Warning: bash-4.3.42 already installed
owensi at Simons-MacBook-Pro.local in ~

༼ つ ◕_◕ ༽つ⚡  cat /etc/shells
# List of acceptable shells for chpass(1).
# Ftpd will not allow users to connect who are not using
# one of these shells.

/bin/bash
/bin/csh
/bin/ksh
/bin/sh
/bin/tcsh
/bin/zsh
/usr/local/bin/bash
/usr/local/bin/bash
/usr/local/bin/bash
owensi at Simons-MacBook-Pro.local in ~

༼ つ ◕_◕ ༽つ⚡  echo $0
-bash

@s10wen two things:

your user should definitely have permissions to chsh for itself
I’d strongly discourage doing chsh for root, that could have unpredictable results (in fact, try not to mess with root at all)

You’ll want to remove the duplicate entries for /usr/local/bin/bash from your /etc/shells.
I’m curious, is your user an admin user? And are you on El Capitan?

Thanks for the reply I'm just travelling atm but I'll have a look later today. I'm on El Capitan. Something that might be a factor is that this is a work machine, I have a local admin 'simonowen' then also a work admin 'owensi' but afaik they both have full admin rights.

On 28 Jan 2016, at 14:10, Jeff Byrnes<EMAIL_ADDRESS>wrote:
@s10wen two things:
your user should definitely have permissions to chsh for itself
I’d strongly discourage doing chsh for root, that could have unpredictable results (in fact, try not to mess with root at all)
You’ll want to remove the duplicate entries for /usr/local/bin/bash from your /etc/shells.
I’m curious, is your user an admin user? And are you on El Capitan?
—
Reply to this email directly or view it on GitHub.

As long as they’re both admins, you should be able to chsh, but perhaps your work did something to lock it down.

@jeffbyrnes I enabled a 'mobile' user in the system preferences and I'm sorted now :) looks like it was a permissions / user issue. Thanks for your help, cleaned up the duplicate entries as well.

@s10wen 's mention of 'mobile' user led me to a solution that worked for me: I'm on an Active Directory domain which is controlled by Centrify. I had to follow these steps to set my shell in the /etc/centrifydc/centrifydc.conf configuration file as described here.

I was having this issue as well, and, fixed it with the code in @itinance 's comment. Here's what happened with mine:
Since I already had a homebrew dev environment setup, I chose not to use the brew.sh script. Instead, I just installed the newer Bash manually, then issued the $ chsh -s /usr/local/bin/bash command, which seemed to succeed. The problem: I had overlooked this crucial comment in brew.sh. Oops! Oh, the things we can miss when we lack enough sleep!
So... if you're having this issue, maybe you missed this, too? You have to manually add the new Bash to /etc/shells for this to work correctly.


Transfer policy:navigation issue.

Uploading RPReplay_Final1691729092.MP4…

When I click on transfer policy I get this add co-signer wallet modal.after changing the transfer policy it should jump on the wallet screen.

Verified this issue on keeper app v 1.1.5(243)


ExpanDrive 5.0.18

Thank you for the contribution. It was merged directly as 2c3bf0ec588dd8ab8af8c6cb5a7e71769b51351a to keep commit history cleaner. Your contribution is still credited to you.


Happy Hacktoberfest 2019!

If you're looking to contribute for Hacktoberfest and would love an "easy win", but don't want to make it to a superfluous repository, then please consider this one, contribute content by adding a favorite board game.
This is an awesome list which lets people show their love for, and recommend, board games. Please read the contributing guide, it's a quick read, and follow the formatting guide, again it's short and includes examples.
Please tag your PRs with this issue # if you're contributing during October for Hacktoberfest. I'll close this issue after the end of October. For now, happy contributing!

Hey @edm00se I've submitted a PR at awesome #1621

Thanks for the interest. As it happens, it already is added to the main awesome repo.
This issue is meant to be a landing point for others to contribute content to this list.

This issue is meant to be a landing point for others to contribute content to this list.

Ohh gotcha now. I misinterpreted poorly haha.

No worries here. I've updated the issue description to hopefully prevent anyone else from reading it similarly.

Thank you to all who participated!
This repo is still very much open for contribution.


Fixes typo issue in line 47

Closes #4557

@mapalko @DaniHalfin @AndreaBarr - PR is ready for final review. This is just a typo issue based on #4557.
Thank you.

@mypil looks good. Thanks!


Missing documentation

It seems important parts of the documentation is missing here:
https://github.com/Maatwebsite/Laravel-Sidebar/wiki/Building-the-Sidebar
How do I add menu's / groups / items ....?

Indeed never got the change to finish the docs.
The example in the docs does have the example of adding groups and items:
$this->menu->group('settings', function(Group $group) {
     $group->item('User Settings');
});

For now you can refer to the docs at AsgardCMS:
https://asgardcms.com/docs/core-module/navigation#introduction


Add Filter&Alarm !Exists Route Table Changes query

Added new query for AWS CloudFormation.
Provide a CloudFormation resource and a resource to ensure that route table changes are catched by a log metric filter and an alarm
Closes #1185

Scan submitted to Checkmarx

Checkmarx SAST - Scan Summary & Details
Cx-SAST Summary
Total of 4 vulnerabilities 0 High 3 Medium 1 Low 0 Info
Violation Summary
 3 Medium
View more details on Checkmarx UI
Cx-SAST Details

Lines
Severity
Category
File
Link

30
Medium
Path_Traversal
cmd/builder/main.go
Checkmarx

52
Medium
Denial_Of_Service_Resource_Exhaustion
pkg/kics/service.go
Checkmarx

80
Medium
Denial_Of_Service_Resource_Exhaustion
test/main_test.go
Checkmarx


Support for TLS certificates in Prom-migrator

Signed-off-by: Harkishen-Singh<EMAIL_ADDRESS>Fixes: #572

Buggy github when requesting for reviews

Will leave review to @antekresic

bros r+

bors r+


Precompile error

I have been getting this error and I am unable to resolve it. I use Ubuntu 19.10. It is a Makie Gallery precompile error i.e. when I use using MakieGallery.

You need to make sure, that you're on the newest version of AbstractPlotting.
You probably have some packages installed, that hold back an upgrade.
Try:
]up; add<EMAIL_ADDRESS>
This will likely give you some package resolver errors, which you will need to resolve by removing/freeing/upgrading the named dependencies.


TECH-349: Typescript format cleanup

JIRA

https://jira.jahia.org/browse/TECH-349
Description

Updated typescript format to be consistent for all components
Upgraded Storybook version

Nice work @vindhya ! 🙌


Wireless Receiver

I'm trying to connect node with the xbox wireless controller, but i couldn't make it work.
It work with the wireless controllers?

You should now be able to pass in a third argument of {"type": "xbox360-wireless"}. Assuming everything else works the same for wireless controllers, you should be able to use this for a wireless xbox 360 controller.
Commit

Alright thanks!
I'm going to try it here...

Update
Worked very well! Thanks again!
So, I'm running it at a raspberry pi and it's working very well, I just need to call:  rmmod xpad every startup, so it could be helpfull if someone have the same issue...


warcli rpc expects at most 1 rpc argument

If you need to pass more than one arg to a bitcoin-cli command, you need to flag each one
😭
--> warcli rpc 1 addnode <IP_ADDRESS> onetry
Usage: warcli rpc [OPTIONS] NODE [METHOD]...
Try 'warcli rpc --help' for help.

Error: You can provide at most two arguments for 'method'.

😄
--> warcli rpc 1 addnode --params=<IP_ADDRESS> --params=onetry

Send it all in quotes?


getTestability whenStable never returns when using busy websockets

This sounds like designed behaviour, but I'm putting this here to see if there's a workaround, or perhaps something I can do.
I have a large angular app that uses rabbitMQ, so there's a websocket open to STOMP. I'm trying to use protractor to test it, and protractor makes heavy use of whenStable, to ensure the page is stable before running tests.
However, if this is run against an instance of the app with traffic, there are a couple of messages coming through a second. I've noticed that outstandingRequestCount in the angular "browser" sky rockets, and sits at about 30. Naturally the tests just time out waiting for protractor to sync. Is there any way I can get this to ignore the websocket traffic? Or do I need to be doing something to stop these requests being present?
The requests seem to be coming from the defer function in the browser.
Any ideas?

For anyone wondering, this was because we had timeouts on the message handler, which was ramping up the outstandingRequestCount


Documentation for CRYPTO_thread_glock_new() is missing

The function was added to the public interface in PR #4294, but documentation is missing.

The original reason why I was looking for the documentation was because I was unsure about the differences between using CRYPTO_thread_glock_new() vs.  CRYPTO_thread_lock_new().
But now I learned that @kaduk intends to remove the function again (PR #4321). What's the current state of this PR?

It was never intended to be a public funtion, only for OpenSSL;s internal use.  If it stays, perhaps the name should be changed.

Ok, understood. But then its prototype shouldn't be in  <openssl/crypto.h>:
include/openssl/crypto.h:70:CRYPTO_RWLOCK *CRYPTO_THREAD_glock_new(const char *name);

Coming back to my initial problem, which made me seek documentation in the first place: Can you explain in two or three sentences what the intention behind the global locks is and why they were introduced? I did not have the time yet to dig through the pull request, so I didn't get the point of the glocks yet.

Thread-safety and forking.  Look at the fork-prepare routines.

The short summary is in http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_atfork.html at the "expected usage is that the prepare handler acquires all mutex locks and the other two fork handlers release them.", but it turns out this is actually quite challenging to do in a usable way, as was pointed out on #4321

@kaduk thanks for the explanation. I read up a bit on this subject and was suprised to learn that getting fork() done right in multithreaded applications is more difficult than expected.  The atfork approach to grab all important locks before forking sounds reasonable, but it will propably only work if the application programmer provides callbacks to treat his own locks in a similar way. Are there existing applications using openssl that mix threads and forks? Until now, I believed that a webserver would either fork or create new threads for incoming connections. What particular use case is OpenSSL trying to address?
Also I was wondering whether installing these atfork handlers would impose an unwanted overhead for applications that don't mix forks and threads? Or multithreaded applications that only just use fork()/execve()? At least in the special case of the system() library call, this problem seems to be addressed by glibc, but that's not guaranteed, see man system(3):
   According to POSIX.1, it is unspecified whether handlers registered
   using pthread_atfork(3) are called during the execution of system().
   In the glibc implementation, such handlers are not called.

Questions over questions. This could be a nice subject for a post on the OpenSSL blog...

Are there existing applications using openssl that mix threads and forks?

No, any such code would have no hope of working.
I believe that the only fork handler we need right now is the one to increment the depth of the reseed hierarchy counter in the child, as I attempted to discuss at #4110


Manual service configuration not read from boostrap.yml

I'm trying to configure service discovery in kubernetes for a GRPC service.
The service I'm trying to discover is a multi-port service and is therefore in need of manual service confguration.
However, I cannot get micronaut to pick-up my service config.
I have followed the guide available here
This is what my boostrap.yml looks like
kubernetes:
  client:
    discovery:
      includes:
        - foo-service
      services:
        - foo-service:
            port: grpc
            mode: endpoint
    config-maps:
      watch: false

But nonetheless I'm seeing the following errors in the logs,
"Fetching Endpoints KubernetesServiceConfiguration{serviceId='foo-service', name='foo-service', namespace='default', mode='endpoint', port='null', manual=false}
"The resource [foo-service] has multiple ports declared [grpc,http] if you want to to use it in micronaut you have to configure it manually.

Am I doing something wrong here or is there some issue in micronaut itself?
Any help is appreciated, thank you for your awesome work with micronaut.
Environment Information

**Operating System: Docker(adoptopenjdk/openjdk11:jdk-11.0.11_9-slim)
**Micronaut Version: 2.5.3
**JDK Version: 11

Yeah, seems like the manual configuration wasn't loaded at all otherwise the manual flag would be true. Please try to not use the list like:
kubernetes:
  client:
    discovery:
      includes:
        - foo-service
      services:
        foo-service:
            port: grpc
            mode: endpoint

Yeah now it's working and it's loading both endpoints. Thanks
However it only seems to send requests to one of them. I was under the impression the default behaviour should be to do round-robin load-balancing between the endpoints. Is there additional config needed for that?

yeah it should do round robin if there are more endpoints. try to configure logging to trace for micronaut-kubernetes.

and thanks I have to fix the documentation

Sorry for a slow reply.
So I tried to enable trace logging for Kubernetes and I can see this.
Building ServiceInstance for serviceId [foo-service] and URI [http://<IP_ADDRESS>:50051]
Building ServiceInstance for serviceId [foo-service] and URI [http://<IP_ADDRESS>:50051]

So it's discovering all instances. But I enabled debug logging for grpc as well and I can see logs like these
"[id: 0x39d05f67, L:/{ip-and-port-of-calling-service} - R:/<IP_ADDRESS>:50051] OUTBOUND DATA: streamId=9 padding=0 endStream=true length=21 bytes=00000000100a0532333934301207535448502d5345"

All the calls I do go towards the IP of a single instance, there are never any calls made towards the other.
So it doesn't seem like there is any load-balancing being performed. 🤔
I can also see this in the logs
"Registering singleton bean io.micronaut.http.client.DefaultLoadBalancerResolver@38f57b3d for type [io.micronaut.http.client.LoadBalancerResolver] using bean key io.micronaut.http.client.DefaultLoadBalancerResolver"
"Finding candidate beans for type: class io.micronaut.http.client.loadbalance.ServiceInstanceListLoadBalancerFactory"
"Finalized bean definitions candidates: [Definition: io.micronaut.http.client.loadbalance.ServiceInstanceListLoadBalancerFactory]"
"Found concrete candidate [Definition: io.micronaut.http.client.loadbalance.ServiceInstanceListLoadBalancerFactory] for type: io.micronaut.http.client.loadbalance.ServiceInstanceListLoadBalancerFactory "

Shouldn't it pick the DiscoverClientLoadBalancer here?

By default all loadbalancers are RoundRobin kind. However based on the context of execution it may always start with fresh list of service instances. I'll try to look at this. Anyway, if you don't want to use client side loadbalancing, use the discovery mode Service -> https://micronaut-projects.github.io/micronaut-kubernetes/latest/guide/#service-discovery

Also if you could please open new issue regarding the load balancing. I'll close this one so as resolved. Just copy your comment into new issue I'll handle rest. Thanks!

Thanks, yes we can close this.


Thermostatting tutorial

An opinionated introductory tutorial for thermostatting

Still work in progress, but I want to have CI tests run

Added a few of you as reviewers, I'd like to merge this rather quickly so if one of you has a bit of bandwitdh we can merge and then let others comment, if and when they have time, in a separate PR

for me we can merge. The only question (a part from the typos) is that you could put an explicit example where the global thermostat fails (or is slowto converge), even without running the simulation just saying the name of the system if you know one

I think that just citing some system like you did now+ the example with water is enough. I was just thinking that only the water example was not very strong because the convergence is still quite fast.
Now for me it is mergable if the CI is happy


[Snyk] Security upgrade alpine from 3.17 to 3

This PR was automatically created by Snyk using the credentials of a real user.![snyk-top-banner](https://github.com/andygongea/OWASP-Benchmark/assets/818805/c518c423-16fe-447e-b67f-ad5a49b5d123)
Snyk has created this PR to fix 2 vulnerabilities in the dockerfile dependencies of this project.
Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.
Snyk changed the following file(s):

samples/paved-path-demo/configure-pipeline/Dockerfile

We recommend upgrading to alpine:3, as this image has only 1 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.
Vulnerabilities that will be fixed with an upgrade:

Issue
Score

CVE-2024-4741 SNYK-ALPINE317-OPENSSL-7413590
  436

CVE-2024-4741 SNYK-ALPINE317-OPENSSL-7413590
  436

CVE-2024-5535 SNYK-ALPINE317-OPENSSL-7413591
  436

CVE-2024-5535 SNYK-ALPINE317-OPENSSL-7413591
  436

[!IMPORTANT]

Check the changes in this PR to ensure they won't cause issues with your project.
Max score is 1000. Note that the real score may have changed since the PR was raised.
This PR was automatically created by Snyk using the credentials of a real user.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
📜 Customise PR templates
🛠 Adjust project settings
📚 Read about Snyk's upgrade logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@kirederik can you sign the CLA?

@kirederik can you sign the CLA?

added the snyk-bot to the list of approved users.


soc: nordic: Turn hardware stack protection on by default

In all Nordic ICs that support hardware stack protection, turn it on by
default so that we detect stack overflows easier.
Signed-off-by: Carles Cufi<EMAIL_ADDRESS>

The cost in ROM is around ~200 bytes.

Proof that this would be very useful: https://github.com/zephyrproject-rtos/zephyr/issues/27768

Not opposing this, I think it solves many problems.
I think this should not be enabled in the SoC level because this is a "user" option (i.e. for application developers to enable certain features). Let's do this at the Board level, a common file for nRF boards.

I disagree. There's no common file to all of our boards, and I don't think there should be. Whatever this common file contains it will be there because the board has a Nordic SoC on it so what is the point on having that file?
Note that west build sample -- -DCONFIG_HW_STACK_PROTECTION=n works just fine with this PR.

You might need to fix some tests in the tree that expect HW_STACK_PROTECTION TO be OFF by default. Let's see how CI will run.

Those tests will probably set CONFIG_HW_STACK_PROTECTION=n so there will be nothing to fix hopefully.

I disagree. There's no common file to all of our boards, and I don't think there should be. Whatever this common file contains it will be there because the board has a Nordic SoC on it so what is the point on having that file?

The SoC .defconfig files serve the purpose to have default configuration for SoC related matters (capabilities, peripherals, HW configuration, etc.). HW_STACK_PROTECTION is, in fact, a "Kernel" option (allows the application developer to configure the kernel to detect supervisor thread stack overflows). It has nothing to do with SoC default configuration. Conceptually, it needs to be enabled at the Board level which is accepted as an entry point for application developer configuration.
Besides being counter-intuitive, having the option added at SoC level by default requires you to explicitly set it to =n at the board or project level, if you want to build without stack protection (and save ROM and RAM footprint). This is a problem. Ideally Board defconfig files should only have =y , i.e. enabling options, not disabling.
The other problem I see is that we actually enable it only for nRF platforms: This is not easily justified. This is a kernel feature - why it should be ON for nRF but OFF for all other Cortex-M vendors or for x86 and ARC? This breaks uniformity.
I guess the best way to address this is to enable the option at each nRF DK board, exactly as we do for ARM_MPU. And I'd say it should be enabled for the official Nordic-maintained boards only. Like this we indicate that this is a desirable feature for development on Nordic official DKs, but nothing to be done by-default for anything else.

The SoC .defconfig files serve the purpose to have default configuration for SoC related matters (capabilities, peripherals, HW configuration, etc.). HW_STACK_PROTECTION is, in fact, a "Kernel" option (allows the application developer to configure the kernel to detect supervisor thread stack overflows). It has nothing to do with SoC default configuration. Conceptually, it needs to be enabled at the Board level which is accepted as an entry point for application developer configuration.

Yes, I agree with this particular statement, although I have to say that we could be far more lenient with what is enabled by default in the SoC. Some other vendors are.

Besides being counter-intuitive, having the option added at SoC level by default requires you to explicitly set it to =n at the board or project level, if you want to build without stack protection (and save ROM and RAM footprint). This is a problem. Ideally Board defconfig files should only have =y , i.e. enabling options, not disabling.

Well, that is exactly the point? We want to enable hardware stack protection by default unless your board or project disables it. That way we can help users not shoot themselves in the foot.
If we put this in the board configuration instead users will have to set =n in their .conf files.

The other problem I see is that we actually enable it only for nRF platforms: This is not easily justified. This is a kernel feature - why it should be ON for nRF but OFF for all other Cortex-M vendors or for x86 and ARC? This breaks uniformity.

Well, everybody is entitled to set kernel options on or off in their SoCs honestly. But fair enough.

I guess the best way to address this is to enable the option at each nRF DK board, exactly as we do for ARM_MPU. And I'd say it should be enabled for the official Nordic-maintained boards only. Like this we indicate that this is a desirable feature for development on Nordic official DKs, but nothing to be done by-default for anything else.

OK, I yield and will change this, but I still think the SoC defconfig file is a better place for this :)

LGTM now.

Thanks!

If we put this in the board configuration instead users will have to set =n in their .conf files.

This is correct, however, as I mentioned, the Board defconfig is accepted as a place to add generic application-related configuration:
https://docs.zephyrproject.org/latest/guides/porting/board_porting.html

"For consistency and to make it easier for users to build generic applications that are not board specific for your board, please follow these guidelines while porting."

And what about thingy52_nrf52832?

And what about thingy52_nrf52832?

Agreed, that should be an official Nordic platform, which we could also force to use HW STACK PROTECTION

And what about thingy52_nrf52832?

I thought about it and for some reason decided not to include it. Probably because we don't support it officially. But I will add it now.

@carlescufi With this approach I guess we will have to update all defconfig files in downstream as well.

@carlescufi With this approach I guess we will have to update all defconfig in our non-zephyr tree boards.

in nRF-connect-sdk?

in nRF-connect-sdk?

Yes, that is what I meant.

@carlescufi With this approach I guess we will have to update all defconfig in our non-zephyr tree boards.

Yes, I will send a PR there too, thanks for reminding me!

in nRF-connect-sdk?

Yes, that is what I meant.

Yeap, we probably need a ticket for the downstream boards, so they behave in the same way.

I think we should consider enabling it for all platforms, not just nRF. cc: @andrewboie @erwango

Sure. One question though:
Once it will be set in all boards _defconfig files, will it be default y at the end?

It will, if the ARCH supports it.

It will, if the ARCH supports it.

Ok, I will work on that next week.

Can anyone describe the main differences between CONFIG_HW_STACK_PROTECTION and CONFIG_STACK_SENTINEL at least regarding NRF MCU.
Does both of them catch every stack overflow?
Is there any reason not to use the hardware version? Maybe in term of footprint, mcu usage, mcu consumption?

Can anyone describe the main differences between CONFIG_HW_STACK_PROTECTION and CONFIG_STACK_SENTINEL at least regarding NRF MCU.
Does both of them catch every stack overflow?
Is there any reason not to use the hardware version? Maybe in term of footprint, mcu usage, mcu consumption?
Thanks!

@XavierChapron might be better to raise this question in https://github.com/zephyrproject-rtos/zephyr/pull/28710#event-3833191475

Can anyone describe the main differences between CONFIG_HW_STACK_PROTECTION and CONFIG_STACK_SENTINEL at least regarding NRF MCU.
Does both of them catch every stack overflow?
Is there any reason not to use the hardware version? Maybe in term of footprint, mcu usage, mcu consumption?
Thanks!

@XavierChapron might be better to raise this question in https://github.com/zephyrproject-rtos/zephyr/discussions

@erwango Done https://github.com/zephyrproject-rtos/zephyr/discussions/28866


[DOC-522] Feature-3.10/swagger-ui-5.4.1

Backport of https://github.com/arangodb/arangodb/pull/19628

[ ] https://jenkins.arangodb.biz/view/Documentation/job/arangodb-ANY-examples/3573/


Package not working

Error log:
e: D:\installed softwares\flutter.pub-cache\hosted\pub.dartlang.org\flutter_voip_kit-0.0.6\android\src\main\kotlin\com\example\flutter_voip_kit\VoipUtilties.kt: (23, 1): Class 'VoipUtilties' is not abstract and does not implement abstract member public abstract fun onRequestPermissionsResult(p0: Int, p1: Array<(out) String!>, p2: IntArray): Boolean defined in io.flutter.plugin.common.PluginRegistry.RequestPermissionsResultListener
e: D:\installed softwares\flutter.pub-cache\hosted\pub.dartlang.org\flutter_voip_kit-0.0.6\android\src\main\kotlin\com\example\flutter_voip_kit\VoipUtilties.kt: (135, 5): 'onRequestPermissionsResult' overrides nothing
Getting this error every time while using this package
@BradenBagby

Has this error been resolved on android?
@DIP-POLLEY @BradenBagby

I've made a pull request ( https://github.com/BradenBagby/flutter_voip_kit/pull/6) to sort this issue @BradenBagby


Keep WebView running in background

Is there a way to keep a WebView app running in background even after the user pressed the back or home button? I want the Javascript to keep running.

Good question. But whatever the answer is, it’s not specific to this library. It will be the same answer that is also true for the built-in WebView. Perhaps you could make it work from a Service.


Brightness resets after a few seconds

Describe the bug
Screen brightness to normal after the brightness slider is closed. Works fine when tapping to adjust the brightness.
How to reproduce
Steps to reproduce the behavior:

Open brightness slider
Change brightness and close slider
Brightness changes back to normal.

Expected behavior
Brightness should persist
Versions:

macOS version: 10.14.6
Pock version:  0.5.3

I am not sure how to fix this or where to find help (Where do you ask?). Was planning to do #157 and multi-finger swipe gesture this weekend but they will be useless if brightness slider doesn't actually work.

For the record, I'm having this problem with Pock 0.7.1 and 10.15.1.

I do not have this problem. I am on 0.7.2 and 10.15.2

I have this issue, but not a mac user, I'm on Debian 10 with KDE Plasma 5.14. Turn out look's like is because I have an app "Red Shift" installed, which after a few seconds are resetting my brightness configuration.
Disable RedShift works for me. Maybe some app are doing the same thing in your OS.


Domain activation

Before opening, please confirm:

[X] I have checked the custom domains troubleshooting guide.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

App Id
dkkp00ex8088p
Region
us-east-1
Step
Domain activation
Status
Failed
Issue or question
Domain activation fails.
Error message
Error message: One or more of the CNAMEs you provided are already associated with a different resource.
Additional information
Domain activation fails in AWS Amplify.  Removed all conflicting CNAME records.  Ensured nameservers matched in registered domains and hosted zone.  Ensured no conflicting Cloudfront distributions. Ensured no S3 bucket conflicts. Ensured SSL certification.  The Domain Activation still fails.

Hey,
Reviewed all those documentations. Received the following error after a retry. Please advise.
One or more of the CNAMEs you provided are already associated with a different resource. (Service: AmazonCloudFront; Status Code: 409; Error Code: CNAMEAlreadyExists; Request ID: 71bf8af2-a55b-4886-bca4-2751f8c40616; Proxy: null)

Is there any update on this issue?

Problem is resolved.  What you did worked. Thank you.


Register

Visitors can register on the website with their email address.

the time required to complete this task: 2 hours.


Add support for the Zvk* ISA extensions

This patch introduces support for the RISC-V Zvk* ISA extensions:

Zvkb
Zvkg
Zvkned
Zvknh[a,b]
Zvksh
Zvksed

The Zvk* extensions are not frozen yet, but
The implementation follows the version 20230303 of the specification,
which can be found here:
https://github.com/riscv/riscv-crypto/releases/tag/v20230303
There is an updated version available which introduces significant changes (e.g. new instructions),
but public patchset for Binutils, GCC, QEMU, Spike and SAIL only support the spec version, that this
PR supports.
There is zero support for vector instructions in this repository or any related repo (e.g. riscv-ctg).
Therefore, this PR introduces new helper macros in the first commit (inspired by the existing ones).
All tests have been hand-written.
Description

Provide a detailed description of the changes performed by the PR.

Related Issues

Please list all the issues related to this PR. Use NA if no issues exist

Ratified/Unratified Extensions

[ ] Ratified
[ ] Unratified

List Extensions

List the extensions that your PR affects. In case of unratified extensions, please provide a link to the spec draft that was referred to make this PR.

Reference Model Used

[ ] SAIL
[ ] Spike
[ ] Other - < SPECIFY HERE >

Mandatory Checklist:

[ ] All tests are compliant with the test-format spec present in this repo ?
[ ] Ran the new tests on RISCOF with SAIL/Spike as reference model successfully ?
[ ] Ran the new tests on RISCOF in coverage mode
[ ] Link to Google-Drive folder containing the new coverage reports (See this for more info): < SPECIFY HERE >
[ ] Link to PR in RISCV-ISAC from which the reports were generated : < SPECIFY HERE >
[ ] Changelog entry created with a minor patch

Optional Checklist:

[ ] RISCV-V CTG PR link if tests were generated using it : < SPECIFY HERE >
[ ] Were the tests hand-written/modified ?
[ ] Have you run these on any hard DUT model ? Please specify name and provide link if possible in the description
[ ] If you have modified arch_test.h Please provide a detailed description of the changes in the Description section above.

The tests do look complete. However, this PR does not indicate any coverage information. Tests cannot be merged without corresponding coverage reports/support.

Additionally, I see that the tests in rv32 are also replicated for rv64. If the same assembly file is relevant for both rv32 and rv64 then it should be present only once under the rv32 directory.
When a test is applicable for multiple extensions, it is placed under the repository corresponding to the extension which occurs first canonically.

I'm not sure how you envision coverage reports without vector support in the riscv-ctg repo.
I would claim that generic vector support cannot be part of this PR.
Anyway, I've created a PR with initial ctg files for Zvk: https://github.com/riscv-software-src/riscv-ctg/pull/61

The arch-test framework uses tests that are not self -checking; they
compare a reference model resultvwith a DUT's result (DeviceUnderTest).
The official reference model is Sail, but if support is not available
there, you can use Spike (or Imperas's RVOVPSIM, QEMU probably, etc.
On Wed, Jun 14, 2023 at 3:25 PM Andrew de los Reyes <
@.***> wrote:

@.**** commented on this pull request.
In riscv-test-suite/rv32i_m/Zvk/src/vandn.vv-01.S
https://github.com/riscv-non-isa/riscv-arch-test/pull/333#discussion_r1230233364
:

+RVTEST_V_ENABLE()
+RVTEST_VALBASEUPD(DATA_BASE, dataset_tc1)
+RVTEST_SIGBASE(SIG_BASE, signature_tc1)
+
+// VANDN.VV has the following inputs and outputs:
+// - input VS1: Op1 (to be inverted)
+// - input VS2: Op2
+// - input VM: Mask encoding ( or v0.t)
+// - output VD: Result
+
+#define VINST vandn.vv
+
+inst_1x8:
+// This test will define v0, which will later be used as mask register
+TEST_CASE_VVV(1, 8, VINST, v0, v0, 08, v1, 18)
Apologies for what is likely a dumb question here. Is the expected result
of the instruction (vandn.vv in this case) anywhere in the code, or is the
expectation that SAIL and SPIKE will both exhibit identical behavior, so
there is no assert/check needed to check against a golden expected value?
Thanks!
—
Reply to this email directly, view it on GitHub
https://github.com/riscv-non-isa/riscv-arch-test/pull/333#pullrequestreview-1480397572,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHPXVJRHPDMQBQDVNSA5ODLXLI25BANCNFSM6AAAAAAWYIUBUI
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Its pre-req PR at CTG (PR#61) has been reviewed, and it is good to go. Tests are also good, happy to merge this one.


Informing about React.lazy()

This week the React team introduced a new library feature: React.lazy() which replicate this library functionality by using newly introduced React concept: React Suspense for promise resolution and error boundaries for error management.
react-async-component has been a wonderful solution for my projects in the last year and I'm grateful for that. Now that it's solution has been integrated into the React core I think it would be suiting to inform users that use latest React that they can use lazy().

lazy doesn't support SSR yet

True, yet for non SSR use and future use when it will be supported it should be informed.


DOSDP specification

@matentzn @LCCarmody
biopsy by type has this:
classes:
biopsy: OBI:0002650
diagnostic procedure: MAXO:0000003
but ct_by_)type has this
classes:
ct: MAXO:0000571
location: owl:Thing
Should we be using UBERON instead of owl:Thing?

Good catch @pnrobinson! This wont change anything practically, but yes, I think from a documentation standpoint the location should be narrowed to Uberon:anatomical entity.

will try to narrow soon

changed the other 12 patterns where I did the same. fixed with #378