podnetwork: Support CNI plugins like PTP and GKE

CNI plugins like PTP and GKE remove a route that is automatically added by kernel for eth0, and then add another route for the same destination.
This patch changes the code to manipulates routes to support such CNI plugins.
Fixes #1909

I tested this PR with Flannel and Libvirt driver.

Hi @yoheiueda thanks a lot. I just tested on GCP provider I'm cooking and it worked fine! While I'm not familiar with this part of the code, tomorrow I will do my best to review it.

@yoheiueda I tried with docker provider on a kind cluster and pod creation fails with
/usr/local/bin/agent-protocol-forwarder: error running a service *forwarder.daemon: failed to set up pod network: failed to set up tunnel "vxlan": failed to add vxlan interface vxlan0: failed to create vxlan interface "vxlan0": /proc/1217/task/1220/ns/net:  file exists
agent-protocol-forwarder.service: Main process exited, code=exited, status=1/FAILURE

In case you want to try here are the images

quay.io/bpradipt/cloud-api-adaptor:latest
quay.io/bpradipt/podvm-docker-image

Would this resolve the issue usnig the EKS CNI driver as well?

@bpradipt thank you for testing this PR.
The root cause of the issue is due to this error.
/usr/local/bin/agent-protocol-forwarder: error running a service *forwarder.daemon: failed to set up pod network: failed to remove route <IP_ADDRESS>/32 dev eth0: failed to identify routes to be deleted: dest: <IP_ADDRESS>/32, gw: invalid IP, dev eth0: %!w(<nil>)

Calico sets a Pod IP on an interface with prefix /32. In this case, Linux kernel does not automatically create a route for it.
I added a logic to handle this case, and tested it on docker with Calico as well as libvirt with flannel.

@EmmEff

Would this resolve the issue usnig the EKS CNI driver as well?

What kind of problems did you encounter with EKS CNI?
If this patch does not help for EKS CNI, please create another issue? I'll take a look at it.

@yoheiueda I tested with ovn-kubernetes on OpenShift as well and things looks good
$ RUN_TESTS=TestPodToServiceCommunicationAzure TEST_TRUSTEE_OPERATOR=yes TEST_PROVISION=no TEST_INSTALL_CAA=no make CLOUD_PROVIDER=azure TEST_PROVISION_FILE=$HOME/azure.properties test-e2e
go test -v -tags=azure -timeout 60m -count=1 -run TestPodToServiceCommunicationAzure ./test/e2e
time="2024-07-26T10:46:30+05:30" level=info msg="Do setup"
time="2024-07-26T10:46:30+05:30" level=info msg="Container runtime: crio"
time="2024-07-26T10:46:32+05:30" level=info msg="Creating namespace 'coco-pp-e2e-test-69bcc389'..."
time="2024-07-26T10:46:32+05:30" level=info msg="Wait for namespace 'coco-pp-e2e-test-69bcc389' be ready..."
time="2024-07-26T10:46:38+05:30" level=info msg="Wait for default serviceaccount in namespace 'coco-pp-e2e-test-69bcc389'..."
time="2024-07-26T10:46:38+05:30" level=info msg="default serviceAccount exists, namespace 'coco-pp-e2e-test-69bcc389' is ready for use"
=== RUN   TestPodToServiceCommunicationAzure
=== PAUSE TestPodToServiceCommunicationAzure
=== CONT  TestPodToServiceCommunicationAzure
=== RUN   TestPodToServiceCommunicationAzure/TestExtraPods_test
    assessment_runner.go:265: Waiting for containers in pod: nginx are ready
time="2024-07-26T10:48:24+05:30" level=info msg="webserver service is available on cluster IP: <IP_ADDRESS>"
Provision extra pod busybox    assessment_helpers.go:425: Waiting for containers in pod: busybox are ready
=== RUN   TestPodToServiceCommunicationAzure/TestExtraPods_test/Failed_to_test_extra_pod.
time="2024-07-26T10:49:58+05:30" level=info msg="VM found in resource group"
time="2024-07-26T10:50:05+05:30" level=info msg="Success to access nginx service. <!DOCTYPE html>\n\n\n<title>Welcome to nginx!</title>\n\n\n\n
Welcome to nginx!
\n
If you see this page, the nginx web server is successfully installed and\nworking. Further configuration is required.
\n\n
For online documentation and support please refer to\nnginx.org.
\nCommercial support is available at\nnginx.com.
\n\n
Thank you for using nginx.
\n\n\n"
    assessment_runner.go:517: Output when execute test commands:<!DOCTYPE html>

        <title>Welcome to nginx!</title>

Welcome to nginx!

If you see this page, the nginx web server is successfully installed and
        working. Further configuration is required.

For online documentation and support please refer to
        nginx.org.

        Commercial support is available at
        nginx.com.

Thank you for using nginx.

time="2024-07-26T10:50:06+05:30" level=info msg="VM found in resource group"
time="2024-07-26T10:50:06+05:30" level=info msg="Deleting pod nginx..."
time="2024-07-26T10:50:51+05:30" level=info msg="Pod nginx has been successfully deleted within 120s"
time="2024-07-26T10:50:52+05:30" level=info msg="Deleting pod busybox..."
time="2024-07-26T10:51:37+05:30" level=info msg="Pod busybox has been successfully deleted within 120s"
time="2024-07-26T10:51:37+05:30" level=info msg="Deleting Service... nginx"
--- PASS: TestPodToServiceCommunicationAzure (299.50s)
    --- PASS: TestPodToServiceCommunicationAzure/TestExtraPods_test (299.50s)
        --- PASS: TestPodToServiceCommunicationAzure/TestExtraPods_test/Failed_to_test_extra_pod. (11.06s)
PASS
time="2024-07-26T10:51:38+05:30" level=info msg="Deleting namespace 'coco-pp-e2e-test-69bcc389'..."
time="2024-07-26T10:51:48+05:30" level=info msg="Namespace 'coco-pp-e2e-test-69bcc389' has been successfully deleted within 60s"
ok

$ RUN_TESTS=TestPodsMTLSCommunicationAzure TEST_TRUSTEE_OPERATOR=yes TEST_PROVISION=no TEST_INSTALL_CAA=no make CLOUD_PROVIDER=azure TEST_PROVISION_FILE=$HOME/azure.properties test-e2e
go test -v -tags=azure -timeout 60m -count=1 -run TestPodsMTLSCommunicationAzure ./test/e2e
time="2024-07-26T10:53:40+05:30" level=info msg="Do setup"
time="2024-07-26T10:53:40+05:30" level=info msg="Container runtime: crio"
time="2024-07-26T10:53:42+05:30" level=info msg="Creating namespace 'coco-pp-e2e-test-633d86dd'..."
time="2024-07-26T10:53:42+05:30" level=info msg="Wait for namespace 'coco-pp-e2e-test-633d86dd' be ready..."
time="2024-07-26T10:53:47+05:30" level=info msg="Wait for default serviceaccount in namespace 'coco-pp-e2e-test-633d86dd'..."
time="2024-07-26T10:53:48+05:30" level=info msg="default serviceAccount exists, namespace 'coco-pp-e2e-test-633d86dd' is ready for use"
=== RUN   TestPodsMTLSCommunicationAzure
=== PAUSE TestPodsMTLSCommunicationAzure
=== CONT  TestPodsMTLSCommunicationAzure
=== RUN   TestPodsMTLSCommunicationAzure/TestPodsMTLSCommunication_test
    assessment_runner.go:265: Waiting for containers in pod: nginx are ready
time="2024-07-26T10:55:30+05:30" level=info msg="webserver service is available on cluster IP: <IP_ADDRESS>"
Provision extra pod curl    assessment_helpers.go:425: Waiting for containers in pod: curl are ready
=== RUN   TestPodsMTLSCommunicationAzure/TestPodsMTLSCommunication_test/Pods_communication_with_mTLS
time="2024-07-26T10:56:59+05:30" level=info msg="VM found in resource group"
time="2024-07-26T10:57:07+05:30" level=info msg="Success to access nginx service. <!DOCTYPE html>\n\n\n<title>Welcome to nginx!</title>\n\n\n\n
Welcome to nginx!
\n
If you see this page, the nginx web server is successfully installed and\nworking. Further configuration is required.
\n\n
For online documentation and support please refer to\nnginx.org.
\nCommercial support is available at\nnginx.com.
\n\n
Thank you for using nginx.
\n\n\n"
    assessment_runner.go:517: Output when execute test commands:<!DOCTYPE html>

        <title>Welcome to nginx!</title>

Welcome to nginx!

If you see this page, the nginx web server is successfully installed and
        working. Further configuration is required.

For online documentation and support please refer to
        nginx.org.

        Commercial support is available at
        nginx.com.

Thank you for using nginx.

time="2024-07-26T10:57:07+05:30" level=info msg="VM found in resource group"
time="2024-07-26T10:57:07+05:30" level=info msg="Deleting Configmap... nginx-conf"
time="2024-07-26T10:57:08+05:30" level=info msg="Deleting Secret... server-certs"
time="2024-07-26T10:57:08+05:30" level=info msg="Deleting extra Secret... curl-certs"
time="2024-07-26T10:57:08+05:30" level=info msg="Deleting pod nginx..."
time="2024-07-26T10:57:53+05:30" level=info msg="Pod nginx has been successfully deleted within 120s"
time="2024-07-26T10:57:53+05:30" level=info msg="Deleting pod curl..."
time="2024-07-26T10:58:39+05:30" level=info msg="Pod curl has been successfully deleted within 120s"
time="2024-07-26T10:58:39+05:30" level=info msg="Deleting Service... nginx"
--- PASS: TestPodsMTLSCommunicationAzure (291.26s)
    --- PASS: TestPodsMTLSCommunicationAzure/TestPodsMTLSCommunication_test (291.26s)
        --- PASS: TestPodsMTLSCommunicationAzure/TestPodsMTLSCommunication_test/Pods_communication_with_mTLS (11.16s)
PASS
time="2024-07-26T10:58:39+05:30" level=info msg="Deleting namespace 'coco-pp-e2e-test-633d86dd'..."
time="2024-07-26T10:58:50+05:30" level=info msg="Namespace 'coco-pp-e2e-test-633d86dd' has been successfully deleted within 60s"
ok      github.com/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e     311.573s

$ RUN_TESTS=TestPodToDownloadExternalFileAzure  TEST_TRUSTEE_OPERATOR=yes TEST_PROVISION=no TEST_INSTALL_CAA=no make CLOUD_PROVIDER=azure TEST_PROVISION_FILE=$HOME/azure.properties test-e2e
go test -v -tags=azure -timeout 60m -count=1 -run TestPodToDownloadExternalFileAzure ./test/e2e
time="2024-07-26T11:10:54+05:30" level=info msg="Do setup"
time="2024-07-26T11:10:54+05:30" level=info msg="Container runtime: crio"
time="2024-07-26T11:10:56+05:30" level=info msg="Creating namespace 'coco-pp-e2e-test-0924345a'..."
time="2024-07-26T11:10:56+05:30" level=info msg="Wait for namespace 'coco-pp-e2e-test-0924345a' be ready..."
time="2024-07-26T11:11:01+05:30" level=info msg="Wait for default serviceaccount in namespace 'coco-pp-e2e-test-0924345a'..."
time="2024-07-26T11:11:02+05:30" level=info msg="default serviceAccount exists, namespace 'coco-pp-e2e-test-0924345a' is ready for use"
=== RUN   TestPodToDownloadExternalFileAzure
=== PAUSE TestPodToDownloadExternalFileAzure
=== CONT  TestPodToDownloadExternalFileAzure
=== RUN   TestPodToDownloadExternalFileAzure/PodWithSpecificCommands_test
    assessment_runner.go:265: Waiting for containers in pod: simple-test are ready
=== RUN   TestPodToDownloadExternalFileAzure/PodWithSpecificCommands_test/Pod_with_specific_commands
    assessment_runner.go:416: Output when execute test commands:
time="2024-07-26T11:12:42+05:30" level=info msg="VM found in resource group"
time="2024-07-26T11:12:42+05:30" level=info msg="Deleting pod simple-test..."
time="2024-07-26T11:13:28+05:30" level=info msg="Pod simple-test has been successfully deleted within 120s"
--- PASS: TestPodToDownloadExternalFileAzure (145.80s)
    --- PASS: TestPodToDownloadExternalFileAzure/PodWithSpecificCommands_test (145.80s)
        --- PASS: TestPodToDownloadExternalFileAzure/PodWithSpecificCommands_test/Pod_with_specific_commands (9.50s)
PASS
time="2024-07-26T11:13:28+05:30" level=info msg="Deleting namespace 'coco-pp-e2e-test-0924345a'..."
time="2024-07-26T11:13:38+05:30" level=info msg="Namespace 'coco-pp-e2e-test-0924345a' has been successfully deleted within 60s"
ok      github.com/confidential-containers/cloud-api-adaptor/src/cloud-api-adaptor/test/e2e     166.471s

@stevenhorsman based on the tests carried till now, this PR should be ok to merge.
Would leave it to you to take the final call :-)

@EmmEff

Would this resolve the issue usnig the EKS CNI driver as well?

What kind of problems did you encounter with EKS CNI?
If this patch does not help for EKS CNI, please create another issue? I'll take a look at it.

@EmmEff this patch doesn't solve the external network connectivity issue with default EKS CNI.  I tested this on my setup and the external network connectivity problem remains with EKS CNI.

@EmmEff

Would this resolve the issue usnig the EKS CNI driver as well?

What kind of problems did you encounter with EKS CNI?
If this patch does not help for EKS CNI, please create another issue? I'll take a look at it.

Admittedly I am not entirely sure of the cause of the connectivity issue with the EKS CNI. At a high level, there is no network connectivity from the pod running in the CVM to the outside. Maybe @bpradipt can better explain?

@beraldoleal @EmmEff
I raised an issue for EKS. Let's continue discussion there.
https://github.com/confidential-containers/cloud-api-adaptor/issues/1966


Refactor theme-cart to remove jQuery and add tests

Removes jQuery from theme-cart and uses Fetch API instead. Requires a Fetch and Promise Polyfill for IE11.
Still todo

[ ] Add localStorage functionality to manage cart across multiple tabs
[ ] Docs in README

As a heads up, this approach might not be compatible with the highly used Facebook Pixels.  Here is a quote from another theme developer on the issues they ran into with fetch.

Also, I just wanted to warn everyone about the usage of fetch API (that aims to replace the old school XMLHttpRequest and all its quirkcs). It appears that adding to cart using the fetch API is incompatible with the current Facebook Pixel integration. After a lot of testing on my end I was forced to revert using XMLHttpRequest which is the only one that is “catched” by the Facebook Pixel.

I'm unsure if this is a direct result of the Facebook API itself or more than likely Shopify's implementation through the trekkie functionality.  If it comes back to trekkie then it probably impacts the Google Analytics integration as well.

Did some digging into this. @bakura10 you are right that we are only listening for XHR requests that match Cart API requests and then triggering a Trekkie event for FB Pixel. Going to see if I can expand this logic to include Fetch.

Thanks for confirming this. It would be awesome :).

Have a PR open in core to fix the issue with Trekkie not listening to Fetch requests to cart API. Once that is merged, this can move forward.

I seem to keep getting an error when attempting to use this. I've just straight copied everything to my local project so i can test it.
The error i'm getting is:
Doing 2 very simple calls with Preact
  async onRemoveClick () {
    const data = await removeItem(this.props.item.key)
    console.log(data)
  }

  async updateQuantity (quantity) {
    const data = await updateItem(this.props.item.key, { quantity })
    console.log(data)
  }

Both give the same error. Could this be anything to do with the browserSync :3000 vs using the shop URL or is the fetch problem outlined above?

@dan-gamble good timing -- I've been working on it all day. Almost ready! Will open again in a new PR when it is.


ModelType Design: explicit schema pruning using RETURNS

CREATE (OR REPLACE)? MODEL (IF NOT EXISTS)? model=qualifiedName
      (FLAVOR flavor=identifier)?
      (MODEL_TYPE modeltype=qualifiedName)?
      (OPTIONS optionList)?
      (RETURNS datatype=dataType)?
      (USING uri=STRING)

For example, we define
array<struct<box:box2d, score:float, label:string>>

as OUTPUT_SCHEMA.
When
create model yolov5
RETURNS array<struct<score:float, label:string>>
using 'torchhub://x/y/z'

schema pruning can be done.

Another related here that could be super useful is to specify results to get from certain layers. Right now if we want to do model introspection we need to specifically implement a new model explicitly. Instead it would be great to be able to extract features from intermediate layers without reimplementing (eg embeddings, candidate label scores etc). Cc @eddyxu wdyt?

schema pruning can be done.

Could you elaborate what is the use case for schema pruning? It is not clear to me why we want to do it instead just do data manipulation via SQL.  It does add quite some complexity to ModelType, and makes the ModelType interface wider.

Another related here that could be super useful is to specify results to get from certain layers.

Yea, this could be great.  Lets do some research to see whether we can achieve a common pattern here?


Is it necessary for R.swift.Library to be added if R.swift is added using Homebrew?

I have added R.swift to my computer using Homebrew and I am able to generate R.generated.swift.
The generated R.generated.swift has line to import Rswift which was resolved by installing R.swift.Library using Carthage.
The README contains the following line

R.swift is a tool used in a build step, it is not a dynamic library.

Is there a way to use R.swift without linking/embedding any frameworks like it is done in Shark or is it necessary to add R.swift.Library framework?

Yes, the generated Swift file depends on the R.swift.Library, it contains datatypes it needs.
This prevents the generated file from containing extensions etc.


QueryShardContext in plugins

Is your feature request related to a problem? Please describe.
In ISM plugin we have few situations where we need to be able to convert search queries to Lucene queries:

In Rollups we want to support query_string queries but we have no viable way to rewrite incoming QueryStringQueryBuilder's fields.
In Transforms we can use a lot of bool queries to match buckets we want to update. Beside this, user also can provide query as part of job definition. This can easlly trigger MAX_LUCENE_CLAUSES exception and currently we can only estimate number of Lucene clauses before execution.

Describe the solution you'd like
Make QueryShardContext available in plugin
Describe alternatives you've considered
Currently we're estimating number of Lucene clauses and we dont have viable option for rewriting query_string queries.

fyi @nknize

Isn’t it already available in plugins via SearchPlugin#getQueries()? where you add your custom QueryBuilder, which in turn allows you to use QueryShardContext by implementing toQuery()

Thanks @rishabhmaurya, let me try this out.

@rishabhmaurya I tried your suggested approach by implementing DummyQueryBuilder where in toQuery method I saved QueryShardContext objectto static var. At startup I had to exec _search with this dummy query on each shard. This might be too hacky solution for us. @deshsidd any thoughts?

hi @petardz , one way to achieve reading fields from query string query is to override getFieldQuery method in QueryStringQueryParser. i tried a small prototype here https://github.com/sbcd90/OpenSearch/blob/ec2a7725ba5643b9b22d159090bb40fcd6752d52/server/src/test/java/org/opensearch/index/search/QueryStringQueryParserExt.java
then we can use a dummy QueryShardContext to create constructor of QueryStringQueryParserExt & use this extended object to read the fields. Here is an example: https://github.com/sbcd90/OpenSearch/blob/ec2a7725ba5643b9b22d159090bb40fcd6752d52/server/src/test/java/org/opensearch/index/query/QueryStringQueryBuilderTests.java#L440-#L456
this will also avoid changes in opensearch-core.

Thanks @rishabhmaurya and @sbcd90. @petardz  is working on the changes and should have a PR ready.

Thank you @sbcd90 that helped a lot! I managed to create QueryShardContext object by myself here
@rishabhmaurya do you see any potential issues in this approach?


Fix same project opening mutliple times from project manager

Fixes #58723

This makes Project Manager unusable if opening the project fails. _open_selected_projects_ask() has a few return statements.

The only way I could think of to resolve this would be to add a bunch of _project_opening_initiated = false statements before some of the return statements in open_selected_projects_ask() and open_selected_projects(). Should I go with this, or is there any better approach?

Should I go with this, or is there any better approach?

You could set _project_opening_initiated when the project is really going to start. But if it won't work, the approach you mentioned is ok too.

You could set _project_opening_initiated when the project is really going to start.

Managed to do it this way

Thanks! And congrats for your first merged Godot contribution :tada:

I'm seeing this issue in 3.5 beta2. Not sure if regression, but this could be worth cherry-picking.

Made the cherry-picking PR: #59576.


Add a command to reset all extension state

Brief Issue Summary
I have a project.  Everytime the extension tries to configure it, it destroyed the CMakeCache.txt.
Expected:
Well, don't.
This is a follow up on #922.  I repeat the main point.  You're screwing the major selling point of this
marvelous extension (gluing intellisense, launching a build, parsing compiler message) with additional feature not that valuable (attempting to change some cmake state).
Apparent Behavior:
Here is some information about my project layout, in case that matters:
Client/
   CMakeLists.txt
   Build/                        cmake state
       CMakeCache.txt
   Sources/
   server/
       CMakeLists.txt
       build/                    another cmake state.
         CMakeCache.txt
       components/
           foo/
                CMakeList.txt

The toplevel Build folder is configured manually with a vcpkg toolchain.
The inner build folder (Client/server/build) might be there sometimes.  Sometimes it might even be
a symlink to the Client/Build folder).  Essentially it should not play any role here.
I have configured cmake manually (we have  scripts for that, this is a rather complicated process because of the vcpkg toolchain + some more gory details)
I open vscode on the Client folder and lanch a Build (F7).  The extension tries to configure
the project (fair enough, the first time it's opened) and it fails.
After the run, the content of the  client/Build/CMakeCache.txt appears reset.  Like if the extension took the silly initiative to reconfigure the project from client/CMakeLists.txt.  Since it has no knowledge of the vcpkg toolchain, it fails.
CMake Tools Log
I activated the debug logs of the extension:
[main] Safe constructing new CMakeTools instance
[variant] Constructing VariantManager
[main] Constructing new CMakeTools instance
[main] Starting CMakeTools second-phase init
[variant] Loaded new set of variants
[main] CMakeTools instance initialization complete.
[kit] Reading kits file /home/fred/.local/share/CMakeTools/cmake-tools-kits.json
[kit] Successfully loaded 4 kits from /home/fred/.local/share/CMakeTools/cmake-tools-kits.json
[kit] Not reading non-existent kits file: /home/fred/src/dual-client/.vscode/cmake-kits.json
[main] Injecting new Kit into CMake driver
[extension] [9577] cmake.setVariant started
[main] Active build variant changed
[main] Configuring folder: dual-client
[main] Saving open files before configure/build
[main] Starting new CMake driver
[main] Starting CMake driver
[driver] CMakeDriver Kit set to clang
[driver] Trying to detect generator supported by system
[driver] Command version test return code 0
[driver] Run _refreshExpansions
[cms-driver] Run doRefreshExpansions
[driver] Run _refreshExpansions cb
[cms-client] Started new CMake Server instance with PID 861
[cache] Reading CMake cache file /home/fred/src/dual-client/Build/CMakeCache.txt
[cache] Parsing CMake cache string
[driver] Setting new variant , Optimize for speed - include debug information.
[driver] Run _refreshExpansions
[cms-driver] Run doRefreshExpansions
[driver] Run _refreshExpansions cb
[cms-client] Started new CMake Server instance with PID 866
[cache] Reading CMake cache file /home/fred/src/dual-client/Build/CMakeCache.txt
[cache] Parsing CMake cache string
[driver] Setting new variant , Optimize for speed - include debug information.
[driver] Run _refreshExpansions
[cms-driver] Run doRefreshExpansions
[driver] Run _refreshExpansions cb
[driver] Run _refreshExpansions
[cms-driver] Run doRefreshExpansions
[driver] Run _refreshExpansions cb
[driver] Start configure
[driver] Runnnig pre-configure checks and steps
[driver] Using compilers in clang for configure
[driver] Run _refreshExpansions
[cms-driver] Run doRefreshExpansions
[driver] Run _refreshExpansions cb
[cmake] Found OpenSSL: /NQVCPKGROOT/7027d93e776daf0107de86af64dfcbccab782140/installed/x64-linux/debug/lib/libcrypto.a (found suitable version "1.1.1d", minimum required is "1")

then follow the ususal cmake logs.
It ends with a failure because the CMakeCache.txt has been flushed (a crucial information one would like to be logged)
then
[cms-driver] Error during CMake configure: [cmake-server] Configuration failed.
[extension] [9577] cmake.setVariant finished (returned 0)
[extension] [3222] cmake.selectKit started

Platform and Versions

Operating System: ubuntu 18
CMake Version: 3.13.3
VSCode Version: 1.45.1
CMake Tools Extension Version: 1.4.0
Compiler/Toolchain: gcc-8

After a full clone of my sources in another directory, I can not reproduce the problem with the copy.
I still always have it with the original, even after removing the .vscode and blasting the Client/Build folder entirely.  It smells as if there is a hidden state  about the project somewhere else.
I try to cleanup some states in .config/Code with no success.  The problem is still there.

I decided to remove this state /home/fred/.local/share/CMakeTools/cmake-tools-kits.json.  That was an old stuff I did a looong time ago and that I'm not using anymore (I really prefer to change the cmake state myself, with cmake's tools)
And, surprise, the issue is gone.  Here was the kits:
[
  {
    "name": "GCC 8.1.0",
    "compilers": {
      "CXX": "/usr/lib/ccache/g++-8",
      "C": "/usr/lib/ccache/gcc-8"
    }
  },
  {
    "name": "ccache zapcc",
    "compilers": {
      "CXX": "/usr/lib/ccache/zapcc++",
      "C": "/usr/lib/ccache/zapcc"
    },
    "keep": true
  },
  {
    "name": "clang",
    "compilers": {
      "CXX": "/home/fred/bin/clang++",
      "C": "/home/fred/bin/clang"
    },
    "keep": true
  },
  {
    "name": "zapcc",
    "compilers": {
      "CXX": "/home/fred/src/Various/llvm/build/bin/zapcc++",
      "C": "/home/fred/src/Various/llvm/build/bin/zapcc"
    },
    "keep": true
  }
]

Putting the state back, the bug did not reappear.  Apparently the hidden state heals himself.  It would be great that you document somewhere the nature of the hidden state.  Where it is stored.
Ideally, this hidden state should not exist at all.  Or it should be kept inside the build dir, or in the .vscode and be very explicit (a readable .json file).  I really lost hours trying to diagnose this issue.

We're not sure what could have caused this, but we will take it as a feature request to add a command that resets all extension state.


Add restart button

This adds a button that performs the equivalent of closing the REPL and starting it again. This is useful for times when you need to quickly restart it, such as when you want to redefine a class. Updating the deps also seems to fix a crash I intermittently experienced.

Is there a repl command to do this as well? Or should there be?

I didn't add one, but that would make sense.

I looked into it, but one issue is that the REPL restart code is specific to IntelliJ, so I don't know any simple way to add a command without making the core code depend on IntelliJ classes.


Including markdown files as fragments within another markdown file in the same repo

Is it possible to include markdown files as fragments within another markdown file in the same repo?
Example:
File 1 (md1.md)
# Section 2
Here's some content that will be shown within another markdown file.

File 2 (md2.md)
This is pseudocode/markdown:
import section2 from ./md1.md

# Section 1
Here's some content within this file.

@section2

Yes that should be possible but I think you'll have to add the query prop so something like that:
import Section2 from ./md1.md

# Section 1
Here's some content within this file.

<Section2 query="all" />

Is this documented somewhere?

This is missing in the docs. I'll add a section in the README about including markdown inside markdown from the same repo. Thanks !

Documentated at https://github.com/adobe/gatsby-theme-parliament#embedding-markdown-documents-and-filtering-content


[MLHR-1734]Added idempotency changes for RabbitMQ input and output op…

…erator

The state corresponding to windows that have been committed is not being purged. Please refer to JMS Input Operator.
Also no test cases have been added  that tests idempotency.

Could you also add an idempotency test like Chandni mentioned.

There are idempotency tests for JMS/Kafka/FileSplitter/AbstractFileInputOperator which you can refer to. A lot of times there are bugs with processing of the window just after the last recovered window which can be validated by the tests.

Will reopen pull request against v3.1.0


[FEATURE REQUEST]

Is it possible to hide/remove following in youtube_player_iframe package ?:

More videos option when pausing
Top information bar which contains metadata, share, info etc ...
right-click video options...

This is to avoid typical youtube like feeling to users .
Thanks for this package !

Please read here -> #299


#229: Fix javadoc warnings when compiling site

Hi @vbauer, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
We will now validate the agreement and then real humans will evaluate your PR.
TTYL, AZPRBOT;

:+1: thanks!

@vboctor NP. :)


ansible: remote context LRU list and max remote context setting

Given a (not so) imaginary playbook that iterates over a corporate user database, becoming each user in turn to perform some user account operations, the extension will spin up a persistent reusable interpreter for each account.
Given 10 accounts, this is no problem, but given 10,000 accounts, this may be sufficient to DoS the target machine, and 10,000 idle Python interpreters will exist for the duration of the run.
This ticket is to implement several features:

[x] Mitogen: API to trigger graceful destruction of a child context, without destroying all children (done as of 48351a18898fc1827abf14b534842b0f4b39bfe4, 862ec2116029b12d80cc9926d4631f7c2670042e, 4dc001f4961cee5f0eaca685d6fadc7c78807ef1)
[x] Ansible: enhance ContextService to include an LRU list for requested contexts. As the list fills, destruction of the least recently used context (on a per-host basis) is triggered.
[x] Ansible: expose some configurable to set the LRU size. Default it to something reasonable, like 50.

Additional concerns:

[x] Must be smart enough to handle connection delegation. Probably each LRU just needs to be keyed by the via= context parameter passed to the service.
[x] Perhaps only implement this for non-SSH contexts, i.e. sudo accounts. If implemented for all connection types, a special case must be made for the master machine, which will always need to connect to large numbers of downstream contexts

Possible enhancements:

[ ] Give ContextService some persistent identifying key representing the current task configuration, allowing it to spot the difference between contexts reused across many steps (e.g.: become_user postgres, become_user root) and contexts spun up by e.g. a with_items loop (e.g. corpuser_bob_11841)

[x] It may make sense to preserve oldest contexts rather than discard them, since these are most likely to be for e.g. sudo root, sudo postgresql, etc., whereas if we're in a task that begins exceeding the list size, chances are high each context will only be used once.
Meanwhile, if a second large loop enumerates all the same accounts, the first few are still cached, whereas if we cache most recently used, then they will be evicted as cold interpreters are recreated for the earlier accounts.

Last TODO item is listed in the docs, closing this off.


LEAKY_RELU not supported in INT8 quantization

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
TensorFlow installed from (source or binary): binary
TensorFlow version (or github SHA if from source): 2.0

Provide the text output from tflite_convert
<ipython-input-54-6f97bfc97449> in <module>
----> 1 workflow(False, 10, 'int8', True, True)

<ipython-input-53-3889d2128b41> in workflow(dw, n_conv, dtype, v1, leaky)
     11     model.save(keras_file_name)
     12     if v1:
---> 13         tflite_model = convert2tflitev1(keras_file_name, qmode=dtype)
     14     else:
     15         tflite_model = convert2tflite(model, qmode=dtype)

<ipython-input-27-637084f295d5> in convert2tflitev1(model_file, qmode)
     19     elif qmode == 'float32':
     20         pass
---> 21     tflite_model = converter.convert()
     22
     23     return tflite_model

~/miniconda3/envs/tf2-n-pytorch/lib/python3.7/site-packages/tensorflow_core/lite/python/lite.py in convert(self)
    991     if self._is_calibration_quantize():
    992       result = self._calibrate_quantize_model(result, inference_input_type,
--> 993                                               inference_output_type)
    994
    995     return result

~/miniconda3/envs/tf2-n-pytorch/lib/python3.7/site-packages/tensorflow_core/lite/python/lite.py in _calibrate_quantize_model(self, result, inference_input_type, inference_output_type)
    237     return calibrate_quantize.calibrate_and_quantize(
    238         self.representative_dataset.input_gen, inference_input_type,
--> 239         inference_output_type, allow_float)
    240
    241   def _get_base_converter_args(self):

~/miniconda3/envs/tf2-n-pytorch/lib/python3.7/site-packages/tensorflow_core/lite/python/optimize/calibrator.py in calibrate_and_quantize(self, dataset_gen, input_type, output_type, allow_float)
     76     return self._calibrator.QuantizeModel(
     77         np.dtype(input_type.as_numpy_dtype()).num,
---> 78         np.dtype(output_type.as_numpy_dtype()).num, allow_float)

~/miniconda3/envs/tf2-n-pytorch/lib/python3.7/site-packages/tensorflow_core/lite/python/optimize/tensorflow_lite_wrap_calibration_wrapper.py in QuantizeModel(self, input_py_type, output_py_type, allow_float)
    113
    114     def QuantizeModel(self, input_py_type, output_py_type, allow_float):
--> 115         return _tensorflow_lite_wrap_calibration_wrapper.CalibrationWrapper_QuantizeModel(self, input_py_type, output_py_type, allow_float)
    116 CalibrationWrapper_swigregister = _tensorflow_lite_wrap_calibration_wrapper.CalibrationWrapper_swigregister
    117 CalibrationWrapper_swigregister(CalibrationWrapper)

RuntimeError: Quantization not yet supported for op: LEAKY_RELU

Also, please include a link to a GraphDef or the model if possible.
Model definition
def build_lrelu_model(n_layers):
    layer_list = []
    for idx in range(n_layers):
        conv_name = f'conv{idx}'
        if idx == 0:
            conv = layers.Conv2D(filters=32,
                                 kernel_size=3,
                                 padding='same',
                                 name=conv_name,
                                 input_shape=(28, 28, 3))
        else:
            conv = layers.Conv2D(filters=32,
                                 kernel_size=3,
                                 padding='same',
                                 name=conv_name)
        bn = layers.BatchNormalization(name=f'bn{idx}')
        activation = layers.LeakyReLU(alpha=0.1, name=f'lrelu{idx}')
        layer_list.extend([conv, bn, activation])
    model = tf.keras.models.Sequential(layer_list)
    model.summary()
    return model

Conversion options:
        converter = tf.compat.v1.lite.TFLiteConverter.from_keras_model_file(model_file)
        converter.optimizations = [tf.lite.Optimize.DEFAULT]
        converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
        converter.inference_input_type = tf.uint8
        converter.inference_output_type = tf.uint8

Any other info / logs
I saw there were some PR about the implementation of Leaky_ReLU in tflite and quantization. But I still cannot get it to work.
Ref PR: https://github.com/tensorflow/tensorflow/pull/27061
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

BTW, I am not using tflite_convert.py
I am using TFLiteConverter, as shown in the code snippet above.

I am not sure if I am looking at the right place.
But the error might be thrown from here: https://github.com/tensorflow/tensorflow/blob/265126ca309d9e9502ab072f033308192d1367b0/tensorflow/lite/tools/optimize/quantize_model.cc#L590
which calls operator_property.cc (link), where LEAKY_RELU and RESIZE_NEAREST_NEIGHBOUR are not defined.

Fixed in https://github.com/tensorflow/tensorflow/pull/36876

Hi @wuhy08
After adding Leaky_Relu into operator property, I can generate the INT8 model which include Leaky_Relu . But the int8 implementation of Leaky_Relu is still missing, right? When I called invoked() tflite model, it will return fail. Did you meet the same problem?
Error detail:
ValueError: Didn't find op for builtin opcode 'LEAKY_RELU' version '2'
Registration failed.
Thanks
Chunjue

Hi @cjtang
There was a PR about Leaky_RELU_INT8 a while ago (https://github.com/tensorflow/tensorflow/pull/27061). I believe that part of code was never touched since nobody successfully quantized a Leaky_RELU op. Let me investigate a little.

It looks the error comes from this line:
https://github.com/tensorflow/tensorflow/blob/3a8c575c4efe57560f36670ddc028f6fc889060a/tensorflow/lite/core/api/op_resolver.cc#L41
Will continue to investigate.

A PR is created for this issue: https://github.com/tensorflow/tensorflow/pull/37279. Will wait for approval.

Great, I also added my version of INT8 Leaky_RELU implementation, but the result is always not good. Now I know that is because model doesn't restrict the same scale for input and output. Thank you for pointing that out. Let's wait for approval.

@cjtang
The PR has just been merged. Expect a few days before it is in tf-nightly.
Thanks.


refactor: make KernelTestBehaviour compatible with Symfony MailerAssertionsTrait

This is the successor PR to #2608

1. Why is this change necessary?
This change makes the Shopware KernelTestBehaviour compatible with Symfony\Bundle\FrameworkBundle\Test\MailerAssertionsTrait
The MailerAssertionsTrait was introduced with Symfony 4.4 symfony.com/blog/new-in-symfony-4-4-phpunit-assertions-for-email-messages
2. What does this change do, exactly?

This changes all getKernel and getContainer method implementations and abstract definitions related to KernelTestBehaviour to static methods.
Make the getKernel and getContainer methods static in Shopware\Tests\Bench and Shopware\Tests\Unit\Storefront\DependencyInjection\ReverseProxyCompilerPassTest.
Remove unused BasicTestDataBehaviour trait from FixtureLoader as getContainer method can't be static in FixtureLoader.

3. Describe each step to reproduce the issue or behaviour.
Add use MailerAssertionsTrait; to your testclass and try to call any of the methods of it. For example $this->getMailerMessages();
This results in
Error : Non-static method Vcore\MyPlugin\Test\Error\ErrorHandlerTest::getContainer() cannot be called statically
4. Please link to the relevant issues (if any).
5. Checklist

[x] I have rebased my changes to remove merge conflicts
[ ] I have written tests and verified that they fail without my change
[x] I have created a changelog file with all necessary information about my changes
[ ] I have written or adjusted the documentation according to my changes
[ ] This change has comments for package types, values, functions, and non-obvious lines of code
[x] I have read the contribution requirements and fulfil them.

Will this land in v6.5 or will it also be backported to v6.4?

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/NEXT-25500
Please use this issue to track the state of your pull request.

@RafaelKr sorry for the late response :see_no_evil: . It's already too late for 6.5.0, but we will make it possible for 6.5.1 I think

Thanks for your contribution. Is merged


Istio prohibiting TLS validation by the API server to a mutating webhook pod

Bug Description
I have the following MutatingWebhookConfiguration
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
  name: example-webhook
webhooks:
  - name: example-webhook.default.svc.cluster.local
    admissionReviewVersions:
      - "v1beta1"
    sideEffects: "None"
    timeoutSeconds: 30
    objectSelector:
      matchLabels:
        example-webhook-enabled: "true"
    clientConfig:
      service:
        name: example-webhook
        namespace: default
        path: "/mutate"
      caBundle: "LS0tLS1CR..."
    rules:
      - operations: [ "CREATE" ]
        apiGroups: [""]
        apiVersions: ["v1"]
        resources: ["pods"]

The caBundle is the output of openssl -A base64 < ca.pem where ca.pem is:
kubectl get secret -n istio-system istio-ca-secret -o=jsonpath='{.data.ca-cert\.pem}' | base64 --decode > ca.pem

I want to inject the webhook pod in an istio enabled namespace with istio having strict TLS mode on.
Therefore, (I thought) TLS should not be needed in my example-webhook service so it is crafted as follows:
apiVersion: v1
kind: Service
metadata:
  name: example-webhook
  namespace: default
spec:
  selector:
    app: example-webhook
  ports:
    - port: 80
      targetPort: webhook
      name: webhook

However when creating a Pod (that does indeed trigger the webhook) I get the following error:
▶ k create -f demo-pod.yaml
Error from server (InternalError): error when creating "demo-pod.yaml": Internal error occurred: failed calling webhook "example-webhook.default.svc.cluster.local": Post "https://example-webhook.default.svc:443/mutate?timeout=30s": no service port 443 found for service "example-webhook"

Can't I configure the webhook not to be called on 443 but rather on 80? Either way TLS termination is done by the istio sidecar.
On top of that, why is it trying to reach the service in the example-webhook.default.svc endpoint? (while it should be doing so in example-webhook.default.svc.cluster.local) ?
I then proceed by trying to incorporate https as follows:
define my service accordingly
apiVersion: v1
kind: Service
metadata:
  name: example-webhook
  namespace: default
spec:
  selector:
    app: example-webhook
  ports:
    - port: 443
      targetPort: tls
      name: application

my deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-webhook
  namespace: default
  labels:
    app: example-webhook
spec:
  replicas: 1
  selector:
    matchLabels:
      app: example-webhook
  template:
    metadata:
      labels:
        app: example-webhook
    spec:
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: example-webhook
      securityContext:
        runAsNonRoot: true
        runAsUser: 1234
      containers:
      - name: server
        image: gcr.io/foo/webhook-test:latest
        imagePullPolicy: Always
        ports:
        - containerPort: 20000
          name: tls
        - containerPort: 80
          name: metrics
        volumeMounts:
        - name: webhook-tls-certs
          mountPath: /etc/webhook/certs/
          readOnly: true
      volumes:
      - name: webhook-tls-certs
        secret:
          secretName: example-webhook-tls

I have created a certificate and private key, using istio's CA.
I can verify that my DNS names in the cert are valid as follows (from another pod)
echo | openssl s_client -showcerts -servername example-webhook.default.svc -connect example-webhook.default.svc:443 2>/dev/null | openssl x509 -inform pem -noout -text

...
 Subject: C = GR, ST = Attica, L = Athens, O = Engineering, OU = FOO, CN = *.cluster.local, emailAddress =<EMAIL_ADDRESS>...
        X509v3 extensions:
            X509v3 Subject Alternative Name:
                DNS:*.default.svc.cluster.local, DNS:example-webhook, DNS:example-webhook.default.svc
...

but now pod creation fails as follows:
▶ k create -f demo-pod.yaml
Error from server (InternalError): error when creating "demo-pod.yaml": Internal error occurred: failed calling webhook "example-webhook.default.svc.cluster.local": Post "https://example-webhook.default.svc:443/mutate?timeout=30s": x509: certificate is not valid for any names, but wanted to match example-webhook.default.svc

The fact that the certs the webhook pod are running with were appropriately created using the istio CA cert, is also validated.
curl --cacert ca_bundle https://example-webhook.default.svc
Test

where ca_bundle is the file containing istio's CA certificate
What is going on?
Version
▶ istioctl version
client version: 1.13.0
control plane version: 1.11.3
data plane version: 1.11.3 (5 proxies)

▶ k version --short
Client Version: v1.21.0
Server Version: v1.21.11-gke.1100

Additional Information
When istio is not injected in the webhook pod, everything works as expected (no matter the CA used for issuing the certs)

You should declare 443 in example-webhook

@hzxuzhonghu I am doing so in the example-webhooks service in the part where I am using https (if that's what you mean)

yeah i think i have run into a similar problem where webhook traffic from api server -> webhook service inside of the service mesh is blocked on envoy proxy tls handshake with:
Internal error occurred: failed calling webhook "flinkoperator.flink.apache.org": failed to call webhook: Post "https://flink-operator-webhook-service.flink.svc:443/validate?timeout=10s": x509: certificate is not valid for any names, but wanted to match flink-operator-webhook-service.flink.svc
confirmed the certificate was created with SAN extension DNS names matching the service dns entries.
as of now it seems like all webhook consuming services must be outside of the service mesh?  not sure if anyone has an example of this actually working...

The api server is not capable of doing Istio mTLS nor can the sidecar terminating webhook TLS. You must use PERMISSIVE mode and do it in your app

PERMISSIVE mode works - or I have been using the traffic.sidecar.istio.io/excludeInboundPorts annotation on the pod for the open webhook port (9443)

FYI we did get this working, what was required was to basically follow this guide to set up Sidecar TLS termination (crucially need to set ENABLE_TLS_ON_SIDECAR_INGRESS=true on Istio Pilot) and you should be good to go :)
Additionally we used cert-manager to handle the certificates and set secretName to dump those to a Kubernetes Secret. Then we used the sidecar.istio.io/userVolume and sidecar.istio.io/userVolumeMount annotations to get those secrets into the istio-proxy sidecar container. Worked like a charm!

Seeing a similar errors after upgrading to v0.25.1
We use helm chart:
NAME                                             	CHART VERSION	APP VERSION
banzaicloud-stable/kafka-operator                	0.25.1       	v0.25.1

We have an istio-proxy sidecar in operator pod.
istio-proxy version: banzaicloud istio-proxyv2:1.15.0
Operator logs have entries like:
{"level":"error","ts":"2023-10-04T10:06:44.405Z","msg":"Reconciler error","controller":"KafkaCluster","controllerGroup":"kafka.banzaicloud.io","controllerKind":"KafkaCluster","KafkaCluster":{"name":"sample-svc-kafka","namespace":"cloud"},"namespace":"cloud","name":"sample-svc-kafka","reconcileID":"a31ff20c-91bf-4e85-a71a-85a0d1d57917","error":"Internal error occurred: failed calling webhook \"kafkaclusters.kafka.banzaicloud.io\": failed to call webhook: Post \"https://kafka-operator-operator.kafka.svc:443/validate-kafka-banzaicloud-io-v1beta1-kafkacluster?timeout=10s\": context deadline exceeded","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.6/pkg/internal/controller/controller.go:235"}

For us, using PERMISSIVE mode is not acceptable, only STRICT, but that doesn't seem an issue,
because a connection to a webhook from any other pod (with istio) is successful:
~ $ curl -k -XPOST -vvv https://kafka-operator-operator.kafka.svc:443/validate-kafka-banzaicloud-io-v1beta1-kafkacluster?timeout=10s
* processing: https://kafka-operator-operator.kafka.svc:443/validate-kafka-banzaicloud-io-v1beta1-kafkacluster?timeout=10s
*   Trying <IP_ADDRESS>:443...
* Connected to kafka-operator-operator.kafka.svc (<IP_ADDRESS>) port 443
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=kafka-operator-operator.kafka.svc
*  start date: Aug 28 10:40:28 2023 GMT
*  expire date: Aug 27 10:40:28 2024 GMT
*  issuer: CN=kafka-operator-ca
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* using HTTP/2
* h2 [:method: POST]
* h2 [:scheme: https]
* h2 [:authority: kafka-operator-operator.kafka.svc]
* h2 [:path: /validate-kafka-banzaicloud-io-v1beta1-kafkacluster?timeout=10s]
* h2 [user-agent: curl/8.2.1]
* h2 [accept: */*]
* Using Stream ID: 1
> POST /validate-kafka-banzaicloud-io-v1beta1-kafkacluster?timeout=10s HTTP/2
> Host: kafka-operator-operator.kafka.svc
> User-Agent: curl/8.2.1
> Accept: */*
>
< HTTP/2 200
< content-type: text/plain; charset=utf-8
< content-length: 128
< date: Wed, 04 Oct 2023 11:58:49 GMT
<
{"response":{"uid":"","allowed":false,"status":{"metadata":{},"message":"contentType=, expected application/json","code":400}}}
* Connection #0 to host kafka-operator-operator.kafka.svc left intact


[SMALLFIX] Fix failing tests

@jsimsa

jenkins, ok to test

LGTM

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/9048/
Test PASSed.


info - migrating from ESP32 - FlexyStepper

Hi,
We are trying to migrate to FastAccelStepper from our existing implementation in flexystepper due to some possible jerk issues.
I just want to know quickly if these functions that are currently provided by  flexystepper can be replicated as is using FastAccelStepper for ESP32:
The motor is initialized as an array as follows:
ESP_FlexyStepper MOTOR[MOTOR_COUNT];

isStartedAsService/startAsService -  lets  you run each motor as a separate service that runs that motor independently using rtos.
usage -
MOTOR[0].setCurrentPositionAsHomeAndStop();
if (!MOTOR[0].isStartedAsService())
MOTOR[0].startAsService();
MOTOR[0].processMovement();

setSpeedInMillimetersPerSecond - is there a similar function that is in steps/s or mm/s - Which of these would work closest to this fn - (setSpeedInTicks  - setSpeedInUs - setSpeedInHz - setSpeedInMilliHz)

MOTOR[num].setAccelerationInMillimetersPerSecondPerSecond

MOTOR[num].setTargetPositionRelativeInMillimeters

setCurrentPositionAsHomeAndStop -  This sets current position as home and stores it.

@aaryanmurgunde   - FYR
Thank you for your help in advance!

FastAccelStepper is not a drop in replacement of Flexystepper, so an adjustment will be needed.
The steppers need to be initialized one by one, but this can be done table driven as in StepperDemo. FastAccelStepper only understands steps. Normally the relation between steps and distance is linear, so a multiplication with a factor should be sufficient. This way speed in mm/s converts to steps/s and acceleration in mm/s² to steps/s². And a move to a distance is a move to a step. A similar function exists for setCurrentPositionAsHomeAndStop.
In general it is wise to read the API description and eventually have a look at the examples.


[oatpp] update to v1.3.0

Fix #21736
Update [oatpp] to the latest version 1.3.0
Note: No feature needs to test.

This port update requires the update of many other ports, so I turned off this pr and asked john to help update


Suggestion: Control flow analysis

This is a work item to track some of the ideas and work associated with implementing simple control flow analysis for error reporting purposes. @vladima has already implemented a version of this concept in https://github.com/Microsoft/TypeScript/pull/1287. It already covers some of the separate issues people have asked for but I'm adding a list of tasks here that can be checked off/added to by whoever ends up officially taking on this work.

[ ] Use before definition https://github.com/Microsoft/TypeScript/issues/21
[ ] Switch case fall through https://github.com/Microsoft/TypeScript/issues/854

Can likely be used to improve type guards too, some pseudo-code here:
https://github.com/Microsoft/TypeScript/issues/1892#issuecomment-72645272
Keep track of widened type assignments & when exiting a block, check if all the same & narrow to the same type.

Switch fall through is fixed by https://github.com/Microsoft/TypeScript/pull/4788
use before definition is tracked by #5207


Test for smarty that includes all functionality

In fixing issue #199 I've realized that the tests for the smarty sampler (both elemental and original sampler). For example we don't have a test that uses the combinatorial decorators or that produces a trajectory or plot file.
I'm working on adding a new test now, but thought I should raise the issue so its clear why my pull request has gotten more complex.

This was largely addressed in pull request #200
I'm planning to add one more in my open pull request that also checks that the initial score (if not given the "answers") is not 100% which will check that total scores are being calculated correctly.

finished addressing in #198 including checks for cases where the score should definitely not be 100%


feat: React sdk

Adds React SDK
See: https://github.com/flipt-io/flipt/pull/3480

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 87.33%. Comparing base (c3f0806) to head (8ad8d15).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #393   +/-   ##
=======================================
  Coverage   87.33%   87.33%
=======================================
  Files          10       10
  Lines        3742     3742
=======================================
  Hits         3268     3268
  Misses        474      474

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@markphelps please don't rush to release this to public. Examples are not correct. It should be currently
React.useEffect(() => {
    if (client) {
      setIsEnabled(client.evaluateBoolean("flag1", "", {}));
    }
  }, [client, flagKey, entityId]);

Another issue that each useFliptContext makes a call to Flipt. For example, if I have three FeatureFlag components on in App component, tree calls will be made.
.
And refresh actually do the same thing... and it doesn't update the state of the page even if the flags are changed and update is fetched. :(
I feel that useSyncExternalStore should be used but I don't have nothing how to use it.

@markphelps please don't rush to release this to public. Examples are not correct. It should be currently
React.useEffect(() => {
    if (client) {
      setIsEnabled(client.evaluateBoolean("flag1", "", {}));
    }
  }, [client, flagKey, entityId]);

Another issue that each useFliptContext makes a call to Flipt. For example, if I have three FeatureFlag components on in App component, tree calls will be made.
.
And refresh actually do the same thing... and it doesn't update the state of the page even if the flags are changed and update is fetched. :(
I feel that useSyncExternalStore should be used but I don't have nothing how to use it.

Good catches @erka ! Although since its a v0.0.1 im not too worried about it being released now with these issues, we can itterate and fix them and release newer versions.
Also im not sure if this is correct:

React.useEffect(() => {
    if (client) {
      setIsEnabled(client.evaluateBoolean("flag1", "", {}));
    }
  }, [client, flagKey, entityId]);

since client.evaluateBoolean( returns a booleanResponse type, so we still need to check enabled on it


[scss] instellisense not showing all values

Code assistance is not complete. For some properties it shows all available values, for some just half, for some nothing.

Does this issue occur when all extensions are disabled?: Yes
I have disabled all extensions manualy and removed all configurations from settings.json. Also tried Start Extension Bisect, it shows CODE issue.

VS Code Version: 1.73.1
OS Version: Windows 10

Steps to Reproduce:

Create new ot open existing .scss file with some styles.
Toggle suggestions for white-space property, there will be no one useful.
Toggle suggestions for align-items property, there will be just some incomplete list of values, no  start, end, etc related to grids.
Toggle suggestions for overflow property, there will be all properties available.

Screenshots

There is OK

There is half. Expect to see start, end, etc in list.

There is nothing useful

white-space has improved in the meantime. align-items misses various values.
https://github.com/microsoft/vscode-custom-data is the module that extracts the data from MDN. It's a mix of manual work as well as irmproving the scripts that do the extraction. Help is welcome.


Linux support for kbd format

Not sure if kbd can allow for different mappings when pressing ctrl or other modifiers, but kbd uses a different format which seems pretty easy to map. I found it while looking at the keyboard configuration page on the arch wiki.
https://jlk.fjfi.cvut.cz/arch/manpages/man/keymaps.5
If I get around to it at some point I'll probably try to make the configuration, but for now I figured you might find it useful and if you want to make it that'd be sweet

Looks like this would be specifically for the virtual console, it'd still be really nice though so that I wouldn't need to switch back to dvorak if I need to use the virtual console


VM support

DING DING DING! 1k
On 24 May 2019, at 13:22, Michał Moskal <notifications<EMAIL_ADDRESS>wrote:

You can view, comment on, or merge this pull request online at:
https://github.com/microsoft/pxt-arcade/pull/1000
Commit Summary

start on VM support
vm compile service
Add fake external screen
Merge branch 'master' into vm64
Merge remote-tracking branch 'origin/master' into vm64
Compile for linux not osx (better gdb)
go back to osx
Use dockercross
Ignore CPP extension files
Start on SDL shell
Fix lib loading
VM restart infrastructure
Sound support
Add exit/reset key handling
Windows console support
merge master
Fix build
Fix console file redirect
Use different sound API that seems to select default audio device better
Simplify logging function
Set skipCloudBuild for dockercross
Add missing bundled pkgs
Update shims
Merge branch 'master' into vm64
Merge branch 'dev' into vm64v2

File Changes

A libs/core---vm/dal.d.tshttps://github.com/microsoft/pxt-arcade/pull/1000/files#diff-0 (702)
A libs/core---vm/enums.d.tshttps://github.com/microsoft/pxt-arcade/pull/1000/files#diff-1 (33)
A libs/core---vm/pxt.jsonhttps://github.com/microsoft/pxt-arcade/pull/1000/files#diff-2 (3)
A libs/core---vm/shims.d.tshttps://github.com/microsoft/pxt-arcade/pull/1000/files#diff-3 (32)
A libs/hw---vm/config.tshttps://github.com/microsoft/pxt-arcade/pull/1000/files#diff-4 (7)
A libs/hw---vm/device.d.tshttps://github.com/microsoft/pxt-arcade/pull/1000/files#diff-5 (5)
A libs/hw---vm/enums.d.tshttps://github.com/microsoft/pxt-arcade/pull/1000/files#diff-6 (16)
A libs/hw---vm/pxt.jsonhttps://github.com/microsoft/pxt-arcade/pull/1000/files#diff-7 (26)
A libs/hw---vm/sdlmain.cpphttps://github.com/microsoft/pxt-arcade/pull/1000/files#diff-8 (327)
A libs/hw---vm/shims.d.tshttps://github.com/microsoft/pxt-arcade/pull/1000/files#diff-9 (9)
A libs/mixer---ext/pxt.jsonhttps://github.com/microsoft/pxt-arcade/pull/1000/files#diff-10 (3)
A libs/screen---ext/pxt.jsonhttps://github.com/microsoft/pxt-arcade/pull/1000/files#diff-11 (3)
A libs/screen---ext/shims.d.tshttps://github.com/microsoft/pxt-arcade/pull/1000/files#diff-12 (153)
A libs/screen---ext/targetoverrides.tshttps://github.com/microsoft/pxt-arcade/pull/1000/files#diff-13 (27)
M pxtarget.jsonhttps://github.com/microsoft/pxt-arcade/pull/1000/files#diff-14 (20)

Patch Links:

https://github.com/microsoft/pxt-arcade/pull/1000.patch
https://github.com/microsoft/pxt-arcade/pull/1000.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/pxt-arcade/pull/1000?email_source=notifications&email_token=ABEF4KPZPH4JLBXXA6N6AZ3PXBFBDA5CNFSM4HPSO3HKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GVZD5BA, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABEF4KJJDKLLFUKAF7MAUVTPXBFBDANCNFSM4HPSO3HA.


Implementation of 1D Waveguide

Is it possible to simulate a WG/Cavity that is periodic in 1 direction, but is of finite size in Y and Z direction using GME or PWE? I could not figure out a way to generate such geometry using Legume, because there seems to be an inherent periodicity in Z as well.

Hi, if I understand correctly, are you looking to simulate nanobeams? This issue keeps being brought up every now and then (https://github.com/fancompute/legume/issues/57), but unfortunately legume is only suited for PhC slabs.
There is one user who I can't find on github (Neelesh Kumar from University of Maryland) who has expressed interest in working on such an extension, and we have discussed about it, but it is not very trivial and I have no idea if there has been any progress.


Presto as a data source

Is there a way we could connect with Presto as a data source to soda core or soda cloud?

SODA-1790

Hi, Presto is currently part of our commercial offering (Soda Library+Cloud) https://docs.soda.io/soda/connect-presto.html


[BUG] CSV doesn't work on scripted fields

Describe the bug
Generated csv reports don't include scripted fields.
To Reproduce
Steps to reproduce the behavior:

add sample flights data in kibana
created a saved search with hour_of_day selected
generated csv report doesn't have hour_of_day column

Expected behavior
A clear and concise description of what you expected to happen.
Plugins
Please list all plugins currently enabled.
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):

OS: [e.g. iOS]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

updated the tag to enhancement, as I feel this should be treated as a new feature

Hello @brijos and @anirudha when we can expect the feature of "Support of scripted field in CSV report".


[Do not merge] Add placeholder to popular links on homepage

Do not merge, this PR is here for use with a deployment test.

Closing, as I don't believe this is being worked on.


Static JWKS endpoint can be set but not used

Via env JWKS_ENDPOINT a custom/static endpoint can be set but the server isn't using it (always aud entry is used from the JWT token).
When using Pomerium as forwardauth service behind Traefik (following this example: https://www.pomerium.com/docs/guides/traefik-ingress) then the configured audience the verify app is using becomes:
https://authenticate.example.com/.well-known/pomerium/jwks.json
But without an actual registered domain this name will not resolve and as a solution you could set the endpoint to use to the Kubernetes internal svc naming.

Created PR to fix: https://github.com/pomerium/verify/pull/82/files


Bring back db-analyzer after Alonzo integration in consensus.

After #3178, the only remaining leftover related to the Alonzo era, is that the db-analyser cannot run; there's an undefined instead of the AlonzoGenesis config file parser
See this line
This issue emerged after #3139.

Errr, maybe they are going with a split Alonzo file after all. See this open&active PR
https://github.com/input-output-hk/cardano-node/pull/2743

I think this issue can be closed now that db-analyser is back. Reopen if not.


Logo

Why a gif? Wouldn't a png or jpg be more efficient?

It was the format it was delivered to me in. I can convert it although idk if its gonna be lossless.

@TheFenderStory google gif to png or gif to jpg xaa

LOL its easy enough to convert.

@TheFenderStory After looking at it, the logo isn't really that good. Also the background of the logo is not transparent.


How to install the theme on mac OS ?

update i've installed it but it looked pretty ugly compared to what i had so i removed it.


print_summary did not catch md typo

yield from rel_scan([qem07],m1.pit,-30,30,31, md = {'reason':{'checking m1 before each cff'}})
This was not interpreted by print_summary().  At least it was the first scan of my plan, and I did not need to wait hours for it to start (thermal equilibration).
I changed to what it should be and then the RE was able to proceed without issue.
yield from rel_scan([qem07],m1.pit,-30,30,31, md = {'reason'}:{'checking m1 before each cff'})

The first code snippet looks right and the second one looks wrong...
In [20]: f = lambda md: md

In [21]: f({'reaon'}: {'bar'})
  File "<ipython-input-21-94fafaaa18b7>", line 1
    f({'reaon'}: {'bar'})
               ^
SyntaxError: invalid syntax

In [22]: f(md={'reaon'}: {'bar'})
  File "<ipython-input-22-24fc0efd222b>", line 1
    f(md={'reaon'}: {'bar'})
                  ^
SyntaxError: invalid syntax

I would have expected this to be
yield from rel_scan([qem07],m1.pit,-30,30,31, md = {'reason':'checking m1 before each cff'})

I think the issue here is that print_summary does not attempt to validate the md, it just makes sure you python is valid and prints where it will move the motors.

@ambarb Which beamline was this at?

@tacaswell sorry, I missed all the other comments.  This was at 2ID (SIX).

We were using the latest official release at that time.  Recently, we switched to the environment needed fro centroiding (XIP plugin).

metadata validators sounds like a potential solution to the errors of this type

@stan-dot This PR is stale and IMO can be closed. Others closer to the 'state of the repo' should decide.

there are no validators to my knowledge yet for metadata. how to move forward (new PR or new issue that expands the case and references this issue) are all fine by me


Why is the log file of UdpProxy of Envoy always empty

I'm using Envoy as the only gateway of my backend micro-services, some of which provides some UDP services. So I need to configure my Envoy to route the UDP requests to the services.
Here is my config file:
static_resources:
  listeners:
  - name: listener_udp_test
    reuse_port: true
    address:
      socket_address:
        protocol: UDP
        address: <IP_ADDRESS>
        port_value: 22221
    access_log:
    - name: envoy.access_loggers.file
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
        path: "/home/administrator/envoy/udp_test.log"  ################## always empty!!!
    udp_listener_config:
      downstream_socket_config:
        max_rx_datagram_size: 9000
    listener_filters:
    - name: envoy.filters.udp_listener.udp_proxy
      typed_config:
        '@type': type.googleapis.com/envoy.extensions.filters.udp.udp_proxy.v3.UdpProxyConfig
        stat_prefix: example_ingress_udp
        cluster: cluster_udp_test
        upstream_socket_config:
          max_rx_datagram_size: 9000
  clusters:
  - name: cluster_udp_test
    connect_timeout: 30s
    type: LOGICAL_DNS
    dns_lookup_family: V4_ONLY
    load_assignment:
      cluster_name: cluster_udp_test
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: <IP_ADDRESS>
                port_value: 22222

Let's say that we have such a kind of situation:
<IP_ADDRESS>(client) <---> <IP_ADDRESS>(gateway) <---> <IP_ADDRESS>(server)
So, the gateway is listening the port 22221 and routing requests to <IP_ADDRESS>:22222.
I execute nc -l -u <IP_ADDRESS> 22222 on the server and I execute nc -u <IP_ADDRESS> 22221 on the client.
But, as you see I have configured a log file named udp_test.log. But it is always empty no matter how I send messages. As my understanding, shouldn't it store some log info about sending and receiving messages or about sender and receiver? If not, which kinds of info would be stored in this log file?

Hi, I am not familiar with access_log in listener, but you can use the following config for udp proxy access logging.
    listener_filters:
    - name: envoy.filters.udp_listener.udp_proxy
      typed_config:
        '@type': type.googleapis.com/envoy.extensions.filters.udp.udp_proxy.v3.UdpProxyConfig
        stat_prefix: example_ingress_udp
        cluster: cluster_udp_test
        upstream_socket_config:
          max_rx_datagram_size: 9000
        access_log:
        - name: envoy.access_loggers.file
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
            path: "/home/administrator/envoy/udp_test.log"
            log_format:
              text_format_source:
                inline_string: "[%START_TIME%]" //tokens you want to print

@giantcroc  Hmm, I just tried but I got an error: "INVALID_ARGUMENT:access_log: Cannot find field.".
I'm using Envoy 1.18.2. So does it mean that I need to upgrade my Envoy to newest version？

@giantcroc   Hmm, but I used Envoy 1.18.2, which doesn't support access_log in that typed_config. So I guess there is no solution except I upgrade my Envoy...

Yeah the listener one is a per connection access log iirc, which I can imagine doesn't work right for UDP. The access log for the UDP filter seems like what you want

https://www.envoyproxy.io/docs/envoy/latest/configuration/observability/access_log/usage#config-access-log-format-dynamic-metadata


Linux build documentation is incorrect (incomplete)

Linux build documentation is incorrect. The documented commands being run in the freshly cloned repository fails:
$ git clone https://github.com/sakhnik/nvim-ui.git
<...output skipped...>
$ cd nvim-ui
$ meson setup BUILD --buildtype=debugoptimized
<...skipped...>
src/meson.build:1:0: ERROR: Non-existent build file 'src/gtkpp/meson.build'

A full log can be found at .../nvim-ui/BUILD/meson-logs/meson-log.txt

Looks like scripts/prepare-gtkpp.sh must be run before meson setup.

Yes, the docs are doomed to lag behind. My plan is to implement signal handling in the generated c++ wrappers, port other widgets, and only then the docs are to catch up. Meanwhile, I'm using scripts/build.sh and scripts/setup-lsp.sh.


Don't output generated has-methods for primitive fields in Proto3

Proto3 does not support presence for primitive fields.

Are there any news regarding this issue? The has methods for primitive fields are quite misleading in my opinion.

Proto3 does not support presence for primitive fields.

I thought without the optional syntax proto3 does not support presence for any fields, not just for scalar fields? Does the proto3 spec say that there should be a hazzer for a field with message type when it's not optional?

I think it still supports (or is allowed to support) presence for message-fields:

I think this is right. Since default value of non-present message fields is null (or None, Nothing, nil, etc.), and you can't have a field in a serialized message with null value, presence of message fields are always tracked, and can always be checked with value == null.


Should add the jar itself to the Java classpath

The code to execute the demo initially did not work for me in MATLAB 2015b. Adding the jar file instead of the jar file's location helped.

Thanks!  Both work on my machine, but if this fixed yours, good enough for me.


[DOCS]maxima

How can the documentation be improved?

Maxima explore

closing as this is an empty issue


Change text embedding processor to async mode for better isolation

Description
Changed text embedding processor to async mode when handling user input.
Previously since inferencing runs in async mode, and overriding only the execute(IngestDocument) method TextEmbeddingProcessor needs a blocking approach to make sure the document enrichment is complete and then the indexing happens. This has a drawback that the blocking happens in the write thread pool, this thread pool has only availableProcessors + 1 threads, and this could have impact on non text embedding indexing.
Changing this to async mode by overriding execute(IngestDocument, BiConsumer) method, thus threads in write thread pool will not be blocked and this has better isolation for non text embedding indexing.
Issues Resolved
An enhancement without issue resolve.
Check List

[x] New functionality includes testing.

[x] All tests pass

[x] New functionality has been documented.

[x] New functionality has javadoc added

[x] Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@zane-neo Please add brief description to PR and/or issue this relates to.

@zane-neo can you please add, some more details like when client will be sending the bulk request what would be the flow? is there anything changing?
Its good that we are moving towards async just want to understand the flow.

@zane-neo please resolve the conflicts and also can we check why the checks are failing? as per the last commit all the checks were passing. You might want to do some rebasing.


Grace's quick fixes

[x] 1. From https://shiny.marineenergy.app/report-v2/, if you click on the logo, it stays on the Shiny page rather than going back to the home page, marinenergy.app. Please change the hyperlink associated with the logo.
[x] 2. At the top of the Regulatory Diagram page, it says, "Follow prompts to navigate the diagram. At the end of the decision tree, select the green rectangle to access additional information." There are no green rectangles(???). I suggest removing the last line. Also, it's not entirely clear that the OCS flow diagrams have active links. We may wish to let the user know on the top of these pages (e.g., https://marineenergy.app/regs_grid-pilot-state.html), that more information can be found by clicking on the boxes in the diagram.
[x] 3. The first cell (first row, first column) of the Interactions table says "receptor" and this is a bit misleading as one could read this as representing the column (which would be correct) or the row (which is incorrect - the top row is a list of stressors). Recommend removing "receptor".
[x] 4. We need for the FERC doc links from the Documents page to open in a new window

@geocoug already fixed task 2. Regulatory verbiage in commit to regs.Rmd

Interactions
OLD

NEW


New horizons

Remove D7 cruft
Parse Kalastatic settings from kalastatic.yaml

We need to switch to using the Cache API to store the settings so we only fetch it on cache rebuild.


Duplicate constants when referencing definitions with enums in another files

Problem statement
With given specification constants ServerStatusOK and ServerStatusNotOK are generated in the server.go and server_status.go files.
On top of that constants in server.go has incorrect constant types string instead ServerStatus.
Swagger specification
swagger.yaml:
swagger: '2.0'

info:
  title: 'Title'
  version: '1.0'

paths:
  '/':
    get:
      responses:
        200:
          description: OK response
          schema:
            $ref: '#/definitions/Response'
    post:
      parameters:
        - in: body
          schema:
            $ref: '#/definitions/Request'
          name: filter
      responses:
        200:
          description: OK response
          schema:
            $ref: '#/definitions/Response'

definitions:
  Response:
    type: object
    properties:
      Server:
        type: array
        items:
          $ref: 'definitions.yaml#/definitions/Server'
  Request:
    type: object
    properties:
      ServerStatus:
        $ref: 'definitions.yaml#/definitions/ServerStatus'

definitions.yaml
definitions:
  ServerStatus:
    type: string
    enum: [OK, Not OK]

  Server:
    type: object
    properties:
      Name:
        type: string
      Status:
        $ref: '#/definitions/ServerStatus'

Environment
swagger version: 0.18.0
go version: 1.11.1
OS:   darwin/amd64

@fredbi when will it be released? any workaround to this issue meanwhile? it happens also with definitions references.

@Napas @rdxmaor you may experiment with this WIP PR. Any feedback is welcome.
I have yet to figure out more testing before merging that one, since this is changing quite a bit the $ref processing in flatten.


Use Question.getJsonEntries() to make the JSON representation the responsibility of the Question class

Description
This refactor is a first step toward getting the codebase ready for auto-generating docs. We now make it the responsibility of the Question to determine how it should be represented in the API response.
Previously, we would do this with by iterating through all the possible question types:
https://github.com/civiform/civiform/blob/9517d9346b401c1e5a7914e47385aa9db0fa0262/server/app/services/export/JsonExporter.java#L121-L222
And for some question types we use the scalar types implementation via the default: above:
https://github.com/civiform/civiform/blob/358ad3dbb6e67aacb91afb3ede60e32ac281ba26/server/app/services/applicant/ReadOnlyApplicantProgramServiceImpl.java#L450-L508
Now, we introduce
public abstract ImmutableMap<Path, ?> getJsonEntries();

in the abstract Question class. Every question will contribute its logic for inserting entries into a JSON export, and JsonExporter will just have to call it for every question in the application.
Release notes
None.
Checklist
General

[x] Added the correct label: < feature | bug | dependencies | infrastructure | ignore-for-release | database >
[ ] Created tests which fail without the change (if possible)

This is a refactor and JsonExporterTest continues to pass
I also ran a diff of the API response, after generating the Mock Program on dev environment and answering questions, both on main and at this PR. Here is the diff.

[ ] Extended the README / documentation, if necessary

Not necessary

User visible changes
None.
New Features
None.
Issue(s) this completes
Relates to #4872.

Two points of hesitation with this direction:

By moving the JSON export logic into the Question subclasses, we no longer have a point to inject dependencies for exporting logic. Any additional dependencies needed for presenting data in JSON will need to be plumbed through the constructors for all of Questions' subclasses. It's fairly common for export logic to require non-trivial dependencies for presenting data like dates, times, durations, geocoded locations, etc.
This change introduces a decent amount of code duplication due to multiple question types using the default case in the existing switch statement.

One idea for addressing these two issues could be to move to introduce a QuestionJsonPresenter interface with a getJsonEntries method, with a default presenter containing the default case from the switch statement and per-question type presenters implemented for the ones that are special case. Each of these presenter classes would have its own injectable c'tor, providing convenient access to whatever dependencies it needed.

@bion interesting points. Re #2, we could address that by adding a default implementation to the Question.getJsonEntries method.
About QuestionJsonPresenter, is this roughly what you're going for (just tried it with two question types):
https://github.com/civiform/civiform/compare/mzetune/json-method...mzetune/presenter
I see the appeal of the cleaner dependency injection, but it is a lot of extra files. Let me know if you want to go for that and I'll finish it out, in this PR.

About QuestionJsonPresenter, is this roughly what you're going for? (just tried it with two question types):

Yeah that's the idea.

I see the appeal of the cleaner dependency injection, but it is a lot of extra files. Let me know if you want to go for that and I'll finish it out, in this PR.

Worth it IMO for access to dependency injection and separation of concerns -- separating the model (not EBean model but model in the MVC sense) from the view (JSON rendering).
If you don't want fewer files you could implement the presenters as static inner classes.


Add filtering to RecallContextManager

This PR implements the change mentioned in https://github.com/posit-dev/py-shiny/issues/1083#issuecomment-1913696858 and the following comment.
Some notes and things that remain to be done:

The UiRecallContextManager class might not actually necessary, because it simply adds a different default filter function.
The filter_ui_objects function is meant to return True for TagChild or TagAttrs or None objects. However, there's not really a good way to check if something is a TagChild or TagAttrs, since those aren't classes, but instead Union types which are recursive. I suppose we could recurse into the objects and check the entire tree to see if everything fits into the type, but that seems pretty crude. This issue is the most questionable part of this approach.
[ ] Some of the UI functions accept just TagChild args, while others accept TagChild | TagAttrs, but the filter_ui_objects function currently only checks for the latter. There should be a different filter function that checks for the former.
[ ] When errors happen, print better error messages, with line number and text.

@wch and I met on this but it was cut short. My notes:

For this release, the behavior we agreed to was that an error would still be thrown, but it would identify the file and line number (and ideally, code) that caused the problem. (It would be good if the code was printed at the console but it is important that the code is NOT rendered in the browser, for security/privacy)
In order to accomplish ☝ we need to throw at the time the problematic object is encountered, not when the enclosing context manager exits. So filter or something like it is still needed.
Rather than filter--which can only allow, ignore, or throw--I'd prefer a method override. This would still permit allow, ignore, and throw, but also "proceed but with modified input", "proceed but I'll modify the return value", "execute more than once", etc.


Copy mechanism

Have you tried implementing  copy mechanism ( Pointer Generator) with your architecture? If yes how was the performance?

No. But I tried copy mechanism in other tasks. In my opinion, I think the copy mechanism will bring just a little improvement. You can have a try.

I was trying copy mechanism but got stuck in understanding the purpose of variables 'src_map' and 'alignments'? Do you have any idea about the same?

Also, have you used pretrained models like GPT-2 to train the models?


Account beacons view is reported multiple times same beacon

Example of multiple times report of same beacon:

...
(from: https://explorer.helium.com/accounts/136CYuq3zzWW6yme4VB7F558fasBvBb3f75Bpjgisuu3UnZKNFN)
I think we expect to see only one line per beacon.

it looks like something's wrong with how we fetch the API data for this page @danielcolinjames


Highlight for in tests

Accompaniment for #504.
Tests for highlighting the "in" keyword in a for statement.  The test passes, but it should be failing: The color annotations are not being picked up by the annotator test functions in CodeInsightTestFixture.  I've posted this test so that a)somebody else can take a look and tell me what I'm missing; b) I can remember that I have to revisit this issue.

Also looks good. I wonder if we could add import .. in/as .. checking as well

The trouble with this one is that the tests pass when they shouldn't (because the compare file doesn't have any annotations in it yet).  Therefore, the annotations are not being picked up by the underlying testHighlighting.  I posted this mostly to find out if anybody else [was interested and] had any other ideas about how to test colorization.


add eslint line lintCommand run delay

When the editor set lint on save, it will cause the code to be formatted but checker still report error.

Should change delay to debounce


Checking out a pr provides inaccurate blame info

Issue Type: Bug
I'm not sure which extension this should be filed under so I'm starting here.
The steps I'm taking are

Select a pr from 'waiting for my review'
Next to description click the checkout button
Select any file
Select any line on that file and the blame info shown by gitlens will all be attributed to that pr regardless on if it was touched. If I try to go back versions of the file it indicates it was just created even though the diff is correct.

I'm also unable to start a review or leave comments at the moment which I was definitely able to do previously.
edit: issue persists after restarting vscode
Extension version: 0.44.0
VS Code version: Code 1.68.0 (Universal) (4af164ea3a06f701fe3e89a2bcbb421d2026b68f, 2022-06-08T11:44:00.250Z)
OS version: Darwin x64 21.5.0
Restricted Mode: No

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz (12 x 2600)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_renderer: enabled_onvideo_decode: enabledvideo_encode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 3, 3

Memory (System)
16.00GB (0.06GB free)

Process Argv
--crash-reporter-id 6c90f615-19ef-40da-81b1-688a0f70a370

Screen Reader
no

VM
0%

A/B Experiments
vsliv368:30146709
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonvspyt551cf:30345471
pythonptprofiler:30281270
vshan820:30294714
vstes263cf:30335440
pythondataviewer:30285071
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593:30376534
vsc1dst:30438360
pythonvs932:30410667
wslgetstarted:30449410
vscscmwlcmt:30465135
cppdebug:30492333
vscaat:30438848
vsclangdc:30486549

got this error trying to checkout a pr
Error running command pr.pick: Cannot read properties of undefined (reading 'number'). This is likely caused by the extension that contributes pr.pick.

I'm also unable to start a review or leave comments at the moment which I was definitely able to do previously.

got this error trying to checkout a pr
Error running command pr.pick: Cannot read properties of undefined (reading 'number'). This is likely caused by the extension that contributes pr.pick.

Both of these would be issues with this extension (GitHub Pull Requests and Issues)
The Gitlens blame issue should be reported in that extension's repo :).
For the commenting and checkout issues:

Next time you see the issue can you share the "GitHub Pull Request" output?
Can you also share the output of git remote -v?
And finally, is your repo a public repo? If so, can you share a link to it so I can see if I can reproduce the issues?


Add NavBar for top-level routing

Codecov Report

Merging #128 (8794053) into main (ebb412f) will decrease coverage by 14.20%.
The diff coverage is n/a.

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #128       +/-   ##
===========================================
- Coverage   72.01%   57.81%   -14.20%
===========================================
  Files          55       55
  Lines        3044     3044
===========================================
- Hits         2192     1760      -432
- Misses        668     1141      +473
+ Partials      184      143       -41

Components
Coverage Δ

garden-app
57.81% <ø> (-14.20%)
:arrow_down:


Update pricing and rating metrics

Coverage increased (+0.04%) to 89.109% when pulling ac03f10d1067dc1a022da36363b6f46d6cd71793 on abh1kg:edit_measure into 2c81f863011c484472ec2d511eed4cc5bf42325d on cloudfoundry-incubator:master.


Fix for 2024.09

I've updated  Ism7MQTT to v0.0.17 but when I upgrade HA to 2024.9.1 I've got:
CWL unavailable status in: Bypass Status, Frost Status, Lüftungsstufe, Vorheizregister.
BWS unavailable status in: Betriebsart, Status E-Heizung, Status Wärmepumpe, Parametrierbarer Ausgang 1 (A1),  Parametrierbarer Eingang 1 (E1)
Downgraded back to 2024.8.3 everything is working again ok.
For example: Bypass Status Geschlossen, Frost Status Kein Frost
It looks that problem affect only variables with names not numbers.

Same here :-( But this seems to be more an MQTT issue. At least I receive some information to change deprecated MQTT config after having updated to 2024.9. The add-on itself is still publishing the corresponding data:

They come with a "value" and "text". Don't know yet why this is making trouble with MQTT...

@amadesu Have you really updated to v0.0.17? I did not consider this being the root cause as you already tried. But now I also updated to v0.0.17 and even updated Mosquitto. Result: I receive all values again.

Frost and bypass status got values:
Sep 30 10:52:18 publishing mqtt with topic 'Wolf/<IP_ADDRESS>/CWL_0x3C/Bypass_Status/text' 'Geschlossen'
Sep 30 10:52:18 publishing mqtt with topic 'Wolf/<IP_ADDRESS>/CWL_0x3C/Frost_Status/text' 'Kein Frost'
but:

Is there anything relevant in your HA log?
Maybe try and hit the "Rebuild addon" button.. it does seem a lot like the old addon version..


Fix switching repositories in tool

Set the initial branch selection to the default branch when switching repositories. This way the selected branch is always an existing one.
Fixes issue #2

Now it fails when a item is selected in the log and one switches repositories, even though I send #deselectAll when changing the branch in #initializePresenter. This might require a fix in Spec.


Remove transitionTime field in terminal releases to reduce annotation sizes

TransitionTime field is used to track completion time of the previous run of a job to decide when a retryable job should be scheduled for run again. This was not being removed on terminal tags, causing release tag annotation sizes to increase, approaching the 2MB annotation size limit. This fix removes the field from the statuses of all terminal tags, and adds a syncTerminal loop to strip any existing terminal release tags of the field.
These changes supersede PR #166 and also contain a fix that enables the logic to work as expected.

/assign @smarterclayton

/lgtm


Fai 1055 application create jobs page

I see Dan has approved this, but now that I've come to add my Add a Job page, I've spotted a few things that might need to be looked at. A quick list:

Answering the "Do you want to add a job?" question marks both the Jobs and Volunteering and work experience sections as Complete or Incomplete. It should only mark the Jobs section. (I believe the issue is actually with the outer api branch rather than this one, strictly speaking).
The "Do you want to add a job?" page is not wired in from the task list. I couldn't do this in advance without deciding on the route for every sub page and then adding in NotImplemented controller actions, so I chose to leave it to the developer of each page to update the task list accordingly.
I think the uri should change from "apply" to "applications", since nouns should be favoured above verbs (Dan might think otherwise though, so you may wish to check with him). I know we've been over it at least twice. :-)
On validation failure, there's a bonus error message in the header which shouldn't be there
On validtion failure, the back link ceases to work
In the Get action of the WorkHistoryController, the request object also appears to be the viewmodel - is this deliberate, as I think they have separate concerns.

Thanks and sorry!


Builtin constraints

Support is currently lacking for some builtins (e.g. bitwise constraints)

Output builtin added in 6ef7d566ece234f5cfc85bbb7c0ba8a0f5da80f2


[lockstep] Introduce optimization barrier around lockstep Ibex

Certain synthesis tools like DC are very smart at optimizing away redundant logic.
Hence, we have to insert an optimization barrier at the IOs of the lockstep Ibex.
This is achieved by manually buffering each bit using prim_buf.
Our Xilinx and DC synthesis flows make sure that these buffers cannot be optimized
away using keep attributes (Vivado) and size_only constraints (DC).
Signed-off-by: Michael Schaffner<EMAIL_ADDRESS>

Could you integrate this and vendor it back into the OT repo? Thanks!


list-style attribute failing depending on order of arguments

I understand that as per the spec, attributes such as list-style have its arguments listed as style, position.
However, something which works either way in CSS, is to invert such values as an example:
li { list-style: inside none; }

Unfortunately when doing so, clean CSS will strip the none value from the output resulting in
li { list-style: inside; }

I am not sure about what happens with other CSS attributes accepting arguments in different order

That's doable and done already for background. It should be same for list-style.

Well, the fact that it works doesn't make it right... A spec is a spec for a reason.


Implement Decoration views

The logic here is largely a copy-paste of the supplementary view handling code.
Iffy things:

The logic in -[PSTCollectionViewData validateLayoutInRect:]

Excluding decoration views here results in them not being considered for layout but including them essentially results in testing that the layout represents any of the valid view types

The …ItemKey implementation for decoration views

The identifier/elementKind value appears to be being ignored somewhere, resulting in one of the layouts "winning" - I've worked around this by using unique indexPaths for my decoration views
I made a cursory attempt at tracking this down but got lost in the code

The NIB loading code is untested as I create my views programmatically

when I call reloadItemsAtIndexPaths: method, the decorationViews are added to collectionView again and again, any ideas how to solve this?

This project is no longer supported. It was built for iOS 5.

Yes, I know it is not supported, but our product still needs to support iOS5, :(
I have found the root cause, and fix it successfully, maybe I should submit an update.
the fix is very simple, below method does not handle the DecorationView kind:

(void)updateWithItems:(NSArray *)items

Feel free to maintain this in a fork.
(And poor soul having to write stuff for iOS 5, I don't envy you :)


bug: authorization failures don't tell a user what workspace their request was in

Describe the bug
Today, our failures look like this:
cowboys.wildwest.dev is forbidden: User "kcp-admin" cannot list resource "cowboys" in API group "wildwest.dev" in the namespace "default": workspace access not permitted

Steps To Reproduce
Do something you are not allowed to
Expected Behaviour
It would be ideal if we did this:
cowboys.wildwest.dev is forbidden: User "kcp-admin" cannot list resource "cowboys" in API group "wildwest.dev" in the namespace "default" in the workspace "root:whatever": workspace access not permitted

Additional Context
No response

The best I can come up with without changing major things upstream is:
cowboys.wildwest.dev is forbidden: User "kcp-admin" cannot list resource "cowboys" in API group "wildwest.dev" in the namespace "default": workspace "root:whatever": workspace access not permitted


[Snyk] Upgrade open from 9.1.0 to 10.0.0

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to upgrade open from 9.1.0 to 10.0.0.
As this is a private repository, Snyk-bot does not have access. Therefore, this PR has been created automatically, but appears to have been created by a real user.
:sparkles: Snyk has automatically assigned this pull request, set who gets assigned.
:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

Warning: This is a major version upgrade, and may be a breaking change.

The recommended version is 1 version ahead of your current version.
The recommended version was released 21 days ago, on 2023-12-19.

Release notes

Package name: open

10.0.0 - 2023-12-19Breaking

Require Node.js 18  5628dc8

v9.1.0...v10.0.0

9.1.0 - 2023-03-26
Update dependencies  46adf0b

v9.0.0...v9.1.0

    from open GitHub release notes

Commit messages

Package name: open

2a8ec70 10.0.0
5628dc8 Require Node.js 18
ba6e43b Test on Node.js 20 (#312)

Compare

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:
🧐 View latest project report
👩💻 Set who automatically gets assigned
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

Currently we are not using this open package.