Add identity and keycloak to charts

[x] Add identity as sub-chart

[x] Add identity manifest templates
[x] Add potential variables to values file (+ doc)
[x] Add keycloak as dependency

[x] Set/overwrite necessary values
[x] Document values

[x] Connect identity with keycloak

[x] use service names to connect with keycloak
[x] use keycloak generated secrets
[x] allow to overwrite secrets -which will also change configuration in identity

[x] Add tests

[x] golden files for defaults
[x] property test for conditions, e.g. existing secret

[x] Adjust CI

[x] adjust makefile to get the deps for the sub chart

Breakthrough:
Manually tested via:
helm repo add bitnami https://charts.bitnami.com/bitnami
helm dependency update charts/ccsm-helm/charts/identity/
helm install zell-helm-test charts/ccsm-helm/ --set zeebe.enabled=false --set operate.enabled=false --set tasklist.enabled=false --set elasticsearch.enabled=false --set identity.image.tag=SNAPSHOT
k port-forward svc/zell-helm-test-identity 8080:80
k port-forward svc/zell-helm-test-keycl 18080:80 # without that we were not able to see the identity login page?

Added multiple new template tests. Integration tests verifies that pods become ready at least, more to come.
related #127

Don't worry about the size :sweat_smile: mostly because of golden files and tests. The core part is ~300 lines.
I kept the commits separated, to show my journey, but I can also rearrange and merge some together if this helps you on reviewing. Like merge CI and Test commits. Let me know if you need that :)

Thanks for your review @npepinpe
Regarding your comments:

I may not be the right person to review this, to be honest. I tried it out, it works, but I can't really say much about the identity configuration itself smile
:wrench:  The message I get when installing tells me I installed Tasklist, Operate, Zeebe, Elastic, but nothing about Identity and Keycloak or Postgres (even if it did install them). Would be nice to update the result message.

This makes sense, I missed that.

:question:  When running, Operate and Tasklist kept crashing and restarting man_shrugging Not sure what to make of this. Identity works though +1

Hm never had this issue. For me identity crashloops, because of keycloaks unavailability.

:question:  Why is keycloak a dependency of identity but not a dependency of the main chart (i.e. ccsm-helm/Chart.yaml)? What's the difference in putting it in one or the other?

I did this for a reason. First, Identity has the dependency to keycloak and not the others and secondly, otherwise the identity chart wouldn't have access to keycloak properties/values which makes the setup here much easier.

:clown_face:  I would've liked to see instead that people bring their own Keycloak, since there's no "official" chart for it and we're now coupled to the bitnami one, but imo that's more of a product decision. Bitnami is probably reliable anyway.

Sorry, I might missed to point this out in the slack channel, I will do that next week. I think, I discussed that with @menski and we wanted to keep this simple for now and the decision was made to not bring an own keycloak. This would also make the setup harder, meaning the default configuration for identity and this would contradict a bit in my opinion our expected use case that the user directly want to set up ccsm and use it in prod (without too much configuration). I think what we can do is to add a condition for keycloak, to be able to disable it. But I wouldn't do that per default.
We decided for bitnami because we assume this to be stable / maintained enough.

:question:  Can we get rid of these warnings? e.g. Dependency zeebe did not declare a repository. Assuming it exists in the charts directory. It's one for each chart.

Be aware that you see this only, because you directly installed the chart. This will not happen if the chart is packaged.

Hey @menski
thanks for your review!

Looks good to me, just some missing replacements of operate with identity

Yeah sorry, somehow missed that.
I have applied your suggestions, are you willing to accept the PR ? :)

I now added identity to the install (release) notes.

In order to deploy the ingress manifest, set `operate.ingress.enabled` to `true`.
If you don't have an Ingress Controller you can use kubectl port-forward to access Operate from outside the cluster:

> kubectl port-forward svc/ccsm-helm-test-operate 8080:80

Now you can point your browser to `http://localhost:8080`

Default user and password: "demo/demo"

### Tasklist

As part of Tasklist an ingress definition can be deployed, but you require to have an Ingress Contoller for that Ingress to be Exposed.
In order to deploy the ingress manifest, set `tasklist.ingress.enabled` to `true`.
If you don't have an Ingress Controller you can use kubectl port-forward to access tasklist from outside the cluster:

> kubectl port-forward svc/ccsm-helm-test-tasklist 8081:80

Now you can point your browser to `http://localhost:8080`

Default user and password: "demo/demo"

### Identity

As part of Identity an ingress definition can be deployed, but you require to have an Ingress Contoller for that Ingress to be Exposed.
In order to deploy the ingress manifest, set `identity.ingress.enabled` to `true`.

If you don't have an ingress controller you can use kubectl port-forward to access identity from outside the cluster:

> kubectl port-forward svc/ccsm-helm-test-identity 8080:80
> kubectl port-forward svc/ccsm-helm-test-keycl 18080:80

Now you can point your browser to `http://localhost:8080`. Identity will forward requests to keycloak (listening under `18080`).

In order to deploy the ingress manifest, set `operate.ingress.enabled` to `true`.
If you don't have an Ingress Controller you can use kubectl port-forward to access Operate from outside the cluster:

> kubectl port-forward svc/ccsm-helm-test-operate 8080:80

Now you can point your browser to `http://localhost:8080`

Default user and password: "demo/demo"

### Tasklist

As part of Tasklist an ingress definition can be deployed, but you require to have an Ingress Contoller for that Ingress to be Exposed.
In order to deploy the ingress manifest, set `tasklist.ingress.enabled` to `true`.
If you don't have an Ingress Controller you can use kubectl port-forward to access tasklist from outside the cluster:

> kubectl port-forward svc/ccsm-helm-test-tasklist 8081:80

Now you can point your browser to `http://localhost:8080`

Default user and password: "demo/demo"

\cc @npepinpe


Flashing/notifying

Just started using the plugin and am loving it, thank you for the great work!
I would love to see a function added to alert/notify/flashing when a change or a certain action takes place to gain attention.
Or would this already be possible at the moment and I simply overlooked or am not innovative enough? :)

Figured it out.

In HA I created a helper boolean and a script that toggles it for x times every y seconds.
In the streamdeck-yaml i changed the design according to the state of the helper boolean.

May it help somebody else.

no change needed.

No change needed


python310Packages.asyncssh: add optional-dependencies

Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 212411 run on x86_64-linux 1

3 packages failed to build:

dvc
dvc-with-remotes
python311Packages.scmrepo

6 packages built:

home-assistant-component-tests.asuswrt
python310Packages.aioasuswrt
python310Packages.asyncssh
python310Packages.scmrepo
python311Packages.aioasuswrt
python311Packages.asyncssh

gpgme doesn't support Python 3.11 yet.

Can you tell me how does this optional dependency thing work in Python? The build process doesn't require them to be present? Is it build with all optional dependencies? How can I disable them?

Can you tell me how does this optional dependency thing work in Python?

The "base module" (sorry, I don't recall the exact terminology) only contains the core functionality. If you would like to use asyncssh then you would do:
$ python3 -m pip install asyncssh

For FIDO support one would need to mention that you want support for those features.
python3 -m pip install asyncssh[fido2]

This will get you asyncssh plus everything that is needed to have FIDO2 support.
More details: https://peps.python.org/pep-0508/

The build process doesn't require them to be present?

No, as they are optional (listed in e.g., extras_require) aka not part of install_require. But they could be needed for the test suite (e.g., ++ passthru.optional-dependencies.fido2).

Is it build with all optional dependencies?

asyncssh included all dependencies so far in propagatedBuildInputs. This means that the output includes all dependencies and had support for all features. With optional-dependencies present, other modules can now only pull-in the parts they need.

How can I disable them?

For the module itself you would simply not mention the dependencies. This is usually done if the dependency is not present.

@fabaff Thank you for the kind explanation, good to know this about python packaging!

Still waiting on gpgme.

Result of nixpkgs-review pr 212411 run on x86_64-linux 1

3 packages failed to build:

dvc
dvc-with-remotes
python311Packages.scmrepo

6 packages built:

home-assistant-component-tests.asuswrt
python310Packages.aioasuswrt
python310Packages.asyncssh
python310Packages.scmrepo
python311Packages.aioasuswrt
python311Packages.asyncssh


Manually deployed vespa with docker compose Failed to establish a new connection: [Errno 111]

hi.
I raised Vespa locally with the following docker compose:
version: '3.9'

services:
  locust-master:
    image: locust-vespa
    build:
      context: ../../
      dockerfile: Dockerfile
      args:
        - TEST_NAME=locust
        - VECTORDB_NAME=vespa
    volumes:
      - ${PWD}/reports/vespa/locust_report:/opt/locust_report/
      - ${PWD}/config/:/opt/config/
    command: locust --config ${LOCUST_MASTER_CONFIG} ${LOCUST_USER_CLASSES}
    env_file: vespa.env
    depends_on:
      vespa:
        condition: service_healthy

  locust-worker:
    image: locust-vespa
    volumes:
      - ${PWD}/config/:/opt/config/
    command: locust --config /opt/config/worker.conf ${LOCUST_USER_CLASSES}
    env_file: vespa.env
    depends_on:
      vespa:
        condition: service_healthy

  vespa:
    image: vespaengine/vespa:8.282.24
    command: configserver,services
    volumes:
      - vespa_persist:/opt/vespa/var
      - vespa_logs:/opt/vespa/logs
    environment:
      VESPA_HOSTNAME: vespa
      VESPA_CONFIGSERVERS: vespa
      VESPA_CONFIGSERVER_JVMARGS: "-Xms32M -Xmx128M"
      VESPA_CONFIGPROXY_JVMARGS: "-Xms32M -Xmx32M"
    healthcheck:
      test: ["CMD-SHELL", "curl -f http://vespa:19071/state/v1/health || exit 1"]
      interval: 5s
      start_period: 40s
      timeout: 10s
      retries: 3

volumes:
  vespa_persist:
    driver: local
  vespa_logs:
    driver: local

The services locust-worker and locust-master basically uses pyvespa to create the a ApplicationPackage with a defined schema and then I create a vespa client with Vespa (from vespa.application module). When I try to use feed_iterable I get the following errors on my locust-worker application.
locust-worker-1  | [2024-01-18 07:54:39,034] 952dbfda1623/WARNING/urllib3.connectionpool: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4bcba6b050>: Failed to establish a new connection: [Errno 111] Connection refused')': /document/v1/benchmarktest/vector_store_benchmark/docid/3586743025

And here is the full vespa log:
2024-01-18 04:54:08 runserver(configserver) running with pid: 16
2024-01-18 04:54:08 Starting config proxy using tcp/vespa:19070 as config source(s)
2024-01-18 04:54:08 Waiting for config proxy to start
2024-01-18 04:54:08 runserver(configproxy) running with pid: 69
2024-01-18 04:54:09 config proxy started after 0s (runserver pid 69)
2024-01-18 04:54:09 runserver(config-sentinel) running with pid: 146
2024-01-18 04:54:09 [2024-01-18 07:54:08.900] INFO    configserver     just-start-configserver  JVM env:  LD_LIBRARY_PATH=/opt/vespa/lib64:/opt/vespa-deps/lib64 MALLOC_ARENA_MAX=1 VESPA_LOG_CONTROL_DIR=/opt/vespa/var/db/vespa/logcontrol VESPA_LOG_CONTROL_FILE=/opt/vespa/var/db/vespa/logcontrol/configserver.logcontrol VESPA_LOG_TARGET=file:/opt/vespa/logs/vespa/vespa.log VESPA_SERVICE_NAME=configserver standalone_jdisc_container__app_location=/opt/vespa/conf/configserver-app
2024-01-18 04:54:09 [2024-01-18 07:54:08.900] INFO    configserver     just-start-configserver  JVM exec: [java -XX:ActiveProcessorCount=12 -XX:+PreserveFramePointer -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/vespa/var/crash -XX:ErrorFile=/opt/vespa/var/crash/hs_err_pid%p.log -XX:+ExitOnOutOfMemoryError -XX:MaxJavaStackTraceDepth=1000000 -XX:-OmitStackTraceInFastThrow --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/jdk.internal.loader=ALL-UNNAMED --add-opens=java.base/sun.security.ssl=ALL-UNNAMED -Djava.io.tmpdir=/opt/vespa/var/tmp -Djava.library.path=/opt/vespa/lib64:/opt/vespa-deps/lib64 -Djava.security.properties=/opt/vespa/conf/vespa/java.security.override -Djava.awt.headless=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.net.client.defaultConnectTimeout=5000 -Dsun.net.client.defaultReadTimeout=60000 -Djavax.net.ssl.keyStoreType=JKS -Djdk.tls.rejectClientInitiatedRenegotiation=true -Dfile.encoding=UTF-8 -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger -Djdisc.bundle.path=/opt/vespa/lib/jars -Djdisc.logger.enabled=false -Djdisc.logger.level=WARNING -Dvespa.log.control.dir=/opt/vespa/var/db/vespa/logcontrol -Djdisc.export.packages= -Djdisc.config.file=/opt/vespa/var/jdisc_container/configserver.properties -Djdisc.cache.path=/opt/vespa/var/vespa/bundlecache/configserver -Djdisc.logger.tag=configserver -Dzookeeper_log_file_prefix=/opt/vespa/logs/vespa/zookeeper.configserver -Xms32M -Xmx128M -XX:+UseTransparentHugePages -cp /opt/vespa/lib/jars/jdisc_core-jar-with-dependencies.jar com.yahoo.jdisc.core.StandaloneMain standalone-container-jar-with-dependencies.jar]
2024-01-18 04:54:09 [2024-01-18 07:54:08.976] INFO    configproxy      just-run-configproxy     JVM env:  LD_LIBRARY_PATH=/opt/vespa/lib64:/opt/vespa-deps/lib64 MALLOC_ARENA_MAX=1 VESPA_LOG_CONTROL_DIR=/opt/vespa/var/db/vespa/logcontrol VESPA_LOG_CONTROL_FILE=/opt/vespa/var/db/vespa/logcontrol/configproxy.logcontrol VESPA_LOG_TARGET=file:/opt/vespa/logs/vespa/vespa.log VESPA_SERVICE_NAME=configproxy
2024-01-18 04:54:09 [2024-01-18 07:54:08.976] INFO    configproxy      just-run-configproxy     JVM exec: [java -XX:+ExitOnOutOfMemoryError -XX:+PreserveFramePointer -XX:CompressedClassSpaceSize=32m -XX:MaxDirectMemorySize=32m -XX:ThreadStackSize=448 -XX:MaxJavaStackTraceDepth=1000 -XX:-OmitStackTraceInFastThrow -XX:ActiveProcessorCount=2 -Dproxyconfigsources=tcp/vespa:19070 -Djava.io.tmpdir=${VESPA_HOME}/var/tmp -Xms32M -Xmx32M -XX:+UseTransparentHugePages -cp /opt/vespa/lib/jars/config-proxy-jar-with-dependencies.jar com.yahoo.vespa.config.proxy.ProxyServer 19090]
2024-01-18 04:54:09 [2024-01-18 07:54:09.296] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.RpcConfigSourceClient   Could not connect to config source at tcp/vespa:19070
2024-01-18 04:54:09 [2024-01-18 07:54:09.299] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.RpcConfigSourceClient   Could not connect to any config source in set [tcp/vespa:19070], please make sure config server(s) are running.
2024-01-18 04:54:09 [2024-01-18 07:54:09.354] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.filedistribution.FileReferencesAndDownloadsMaintainer   Not running maintainer, since this is on a config server host
2024-01-18 04:54:12 [2024-01-18 07:54:12.446] WARNING configserver     Container.DeployLogger   Host named 'vespa' may not receive any config since it differs from its canonical hostname '390389d3f0d5' (check DNS and /etc/hosts).
2024-01-18 04:54:12 [2024-01-18 07:54:12.973] INFO    configserver     Container.com.yahoo.container.core.config.HandlersConfigurerDi   Installing bundles for application generation 0
2024-01-18 04:54:13 [2024-01-18 07:54:13.945] INFO    configserver     Container.com.yahoo.container.core.config.ApplicationBundleLoader        Installed bundles: {[0]org.apache.felix.framework:7.0.5, [1]standalone-container:8.282.24, [2]configdefinitions:8.282.24, [3]config-provisioning:8.282.24, [4]config-bundle:8.282.24, [5]config-model-api:8.282.24, [6]config-model:8.282.24, [7]container-disc:8.282.24, [8]hosted-zone-api:8.282.24, [9]container-apache-http-client-bundle:8.282.24, [10]security-utils:8.282.24, [11]bcprov:1.76.0, [12]bcpkix:1.76.0, [13]bcutil:1.76.0, [14]com.fasterxml.jackson.core.jackson-annotations:2.16.1, [15]com.fasterxml.jackson.core.jackson-core:2.16.1, [16]com.fasterxml.jackson.core.jackson-databind:2.16.1, [17]com.fasterxml.jackson.datatype.jackson-datatype-jdk8:2.16.1, [18]com.fasterxml.jackson.datatype.jackson-datatype-jsr310:2.16.1, [19]javax.ws.rs-api:2.1.99.b01, [20]container-spifly:1.3.7, [21]javax.servlet-api:3.1.0, [22]container-search-and-docproc:8.282.24, [23]linguistics-components:8.282.24, [24]model-evaluation:8.282.24, [25]model-integration:8.282.24, [26]container-onnxruntime:8.282.24, [27]jdisc-security-filters:8.282.24, [28]vespa-athenz:8.282.24, [29]zkfacade:8.282.24, [30]zookeeper-server:8.282.24, [31]configserver:8.282.24, [32]config-model-fat:8.282.24, [33]flags:8.282.24, [34]http-client:8.282.24, [35]node-repository:8.282.24, [36]application-model:8.282.24, [37]orchestrator:8.282.24, [38]service-monitor:8.282.24, [39]configserver-flags:8.282.24}
2024-01-18 04:54:18 [2024-01-18 07:54:18.922] INFO    configproxy      configproxy.com.yahoo.vespa.config.JRTConnection Connecting to tcp/vespa:19070
2024-01-18 04:54:21 [2024-01-18 07:54:21.158] INFO    configserver     Container.com.yahoo.container.handler.threadpool.ContainerThreadpoolImpl Threadpool 'default-pool': min=24, max=1200, queue=0
2024-01-18 04:54:21 [2024-01-18 07:54:21.347] INFO    configserver     Container.com.yahoo.container.jdisc.state.StateMonitor   Changing health status code from 'initializing' to 'up'
2024-01-18 04:54:21 [2024-01-18 07:54:21.462] INFO    configserver     Container.com.yahoo.jdisc.http.server.jetty.JettyHttpServer      Threadpool size: min=28, max=28
2024-01-18 04:54:21 [2024-01-18 07:54:21.854] INFO    configserver     Container.com.yahoo.container.handler.threadpool.ContainerThreadpoolImpl Threadpool 'default-handler-common': min=24, max=24, queue=960
2024-01-18 04:54:22 [2024-01-18 07:54:22.097] INFO    configserver     Container.com.yahoo.container.jdisc.ConfiguredApplication        Switching to the latest deployed set of configurations and components. Application config generation: 0
2024-01-18 04:54:25 [2024-01-18 07:54:25.313] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.DelayedResponseHandler  Timed out (timeout 15000) getting config name=cloud.config.sentinel,configId=hosts/vespa, will retry
2024-01-18 04:54:29 [2024-01-18 07:54:29.740] INFO    config-sentinel  sentinel.config.frt.frtconfigagent       No response / error from config server. This is normal before an application package is deployed. (key: name=cloud.config.sentinel,configId=hosts/vespa) (errcode=103, validresponse:0), trying again in 6.000000 seconds
2024-01-18 04:54:35 [2024-01-18 07:54:35.440] INFO    configproxy      configproxy.com.yahoo.vespa.config.JRTConnection Connecting to tcp/vespa:19070
2024-01-18 04:54:35 [2024-01-18 07:54:35.475] INFO    configproxy      configproxy.com.yahoo.config.subscription.impl.JRTConfigRequester        Request failed: Failed request (No application exists) from Connection { Socket[addr=/<IP_ADDRESS>,port=57312,localport=19070] }\nConnection spec: tcp/vespa:19070
2024-01-18 04:54:53 [2024-01-18 07:54:53.640] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.RpcConfigSourceClient   Subscribe for 'name=cloud.config.sentinel,configId=hosts/vespa,aa9115ebf63cfc6721f75ada21d7cdfa' failed, closing subscriber: Subscribe for 'name=cloud.config.sentinel,configId=hosts/vespa' timed out (timeout was 44000 ms): name=cloud.config.sentinel,configId=hosts/vespa, Current generation: 0, Generation changed: false, Config changed: false
2024-01-18 04:54:53 [2024-01-18 07:54:53.648] INFO    configproxy      configproxy.com.yahoo.config.subscription.impl.JRTConfigRequester        Request failed: Failed request (No application exists) from Connection { Socket[addr=/<IP_ADDRESS>,port=57312,localport=19070] }\nConnection spec: tcp/vespa:19070
2024-01-18 04:55:01 [2024-01-18 07:55:01.313] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.DelayedResponseHandler  Timed out (timeout 25000) getting config name=cloud.config.sentinel,configId=hosts/vespa, will retry
2024-01-18 04:55:06 [2024-01-18 07:55:06.163] INFO    config-sentinel  sentinel.config.frt.frtconfigagent       No response / error from config server. This is normal before an application package is deployed. (key: name=cloud.config.sentinel,configId=hosts/vespa) (errcode=103, validresponse:0), trying again in 7.000000 seconds
2024-01-18 04:55:37 [2024-01-18 07:55:37.653] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.RpcConfigSourceClient   Subscribe for 'name=cloud.config.sentinel,configId=hosts/vespa,aa9115ebf63cfc6721f75ada21d7cdfa' failed, closing subscriber: Subscribe for 'name=cloud.config.sentinel,configId=hosts/vespa' timed out (timeout was 44000 ms): name=cloud.config.sentinel,configId=hosts/vespa, Current generation: 0, Generation changed: false, Config changed: false
2024-01-18 04:55:37 [2024-01-18 07:55:37.657] INFO    configproxy      configproxy.com.yahoo.config.subscription.impl.JRTConfigRequester        Request failed: Failed request (No application exists) from Connection { Socket[addr=/<IP_ADDRESS>,port=57312,localport=19070] }\nConnection spec: tcp/vespa:19070
2024-01-18 04:55:38 [2024-01-18 07:55:38.314] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.DelayedResponseHandler  Timed out (timeout 25000) getting config name=cloud.config.sentinel,configId=hosts/vespa, will retry
2024-01-18 04:55:43 [2024-01-18 07:55:43.498] INFO    config-sentinel  sentinel.config.frt.frtconfigagent       No response / error from config server. This is normal before an application package is deployed. (key: name=cloud.config.sentinel,configId=hosts/vespa) (errcode=103, validresponse:0), trying again in 8.000000 seconds
2024-01-18 04:56:17 [2024-01-18 07:56:17.315] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.DelayedResponseHandler  Timed out (timeout 25000) getting config name=cloud.config.sentinel,configId=hosts/vespa, will retry
2024-01-18 04:56:21 [2024-01-18 07:56:21.662] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.RpcConfigSourceClient   Subscribe for 'name=cloud.config.sentinel,configId=hosts/vespa,aa9115ebf63cfc6721f75ada21d7cdfa' failed, closing subscriber: Subscribe for 'name=cloud.config.sentinel,configId=hosts/vespa' timed out (timeout was 44000 ms): name=cloud.config.sentinel,configId=hosts/vespa, Current generation: 0, Generation changed: false, Config changed: false
2024-01-18 04:56:21 [2024-01-18 07:56:21.668] INFO    configproxy      configproxy.com.yahoo.config.subscription.impl.JRTConfigRequester        Request failed: Failed request (No application exists) from Connection { Socket[addr=/<IP_ADDRESS>,port=57312,localport=19070] }\nConnection spec: tcp/vespa:19070
2024-01-18 04:56:21 [2024-01-18 07:56:21.926] INFO    config-sentinel  sentinel.config.frt.frtconfigagent       No response / error from config server. This is normal before an application package is deployed. (key: name=cloud.config.sentinel,configId=hosts/vespa) (errcode=103, validresponse:0), trying again in 9.000000 seconds
2024-01-18 04:56:56 [2024-01-18 07:56:56.328] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.DelayedResponseHandler  Timed out (timeout 25000) getting config name=cloud.config.sentinel,configId=hosts/vespa, will retry
2024-01-18 04:57:01 [2024-01-18 07:57:01.283] INFO    config-sentinel  sentinel.config.frt.frtconfigagent       No response / error from config server. This is normal before an application package is deployed. (key: name=cloud.config.sentinel,configId=hosts/vespa) (errcode=103, validresponse:0), trying again in 10.000000 seconds
2024-01-18 04:57:05 [2024-01-18 07:57:05.665] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.RpcConfigSourceClient   Subscribe for 'name=cloud.config.sentinel,configId=hosts/vespa,aa9115ebf63cfc6721f75ada21d7cdfa' failed, closing subscriber: Subscribe for 'name=cloud.config.sentinel,configId=hosts/vespa' timed out (timeout was 44000 ms): name=cloud.config.sentinel,configId=hosts/vespa, Current generation: 0, Generation changed: false, Config changed: false
2024-01-18 04:57:05 [2024-01-18 07:57:05.670] INFO    configproxy      configproxy.com.yahoo.config.subscription.impl.JRTConfigRequester        Request failed: Failed request (No application exists) from Connection { Socket[addr=/<IP_ADDRESS>,port=57312,localport=19070] }\nConnection spec: tcp/vespa:19070
2024-01-18 04:57:09 [2024-01-18 07:57:09.495] WARNING config-sentinel  sentinel.config-sentinel Timeout getting config, please check your setup. Will exit and restart: Timed out while subscribing to 'cloud.config.sentinel', configid 'hosts/vespa'
2024-01-18 04:57:09 [2024-01-18 07:57:09.609] INFO    config-sentinel  runserver        will restart in 0 seconds
2024-01-18 04:57:25 [2024-01-18 07:57:25.332] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.DelayedResponseHandler  Timed out (timeout 15000) getting config name=cloud.config.sentinel,configId=hosts/vespa, will retry
2024-01-18 04:57:29 [2024-01-18 07:57:29.931] INFO    config-sentinel  sentinel.config.frt.frtconfigagent       No response / error from config server. This is normal before an application package is deployed. (key: name=cloud.config.sentinel,configId=hosts/vespa) (errcode=103, validresponse:0), trying again in 6.000000 seconds
2024-01-18 04:57:49 [2024-01-18 07:57:49.675] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.RpcConfigSourceClient   Subscribe for 'name=cloud.config.sentinel,configId=hosts/vespa,aa9115ebf63cfc6721f75ada21d7cdfa' failed, closing subscriber: Subscribe for 'name=cloud.config.sentinel,configId=hosts/vespa' timed out (timeout was 44000 ms): name=cloud.config.sentinel,configId=hosts/vespa, Current generation: 0, Generation changed: false, Config changed: false
2024-01-18 04:57:49 [2024-01-18 07:57:49.679] INFO    configproxy      configproxy.com.yahoo.config.subscription.impl.JRTConfigRequester        Request failed: Failed request (No application exists) from Connection { Socket[addr=/<IP_ADDRESS>,port=57312,localport=19070] }\nConnection spec: tcp/vespa:19070
2024-01-18 04:58:01 [2024-01-18 07:58:01.333] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.DelayedResponseHandler  Timed out (timeout 25000) getting config name=cloud.config.sentinel,configId=hosts/vespa, will retry
2024-01-18 04:58:06 [2024-01-18 07:58:06.351] INFO    config-sentinel  sentinel.config.frt.frtconfigagent       No response / error from config server. This is normal before an application package is deployed. (key: name=cloud.config.sentinel,configId=hosts/vespa) (errcode=103, validresponse:0), trying again in 7.000000 seconds
2024-01-18 04:58:33 [2024-01-18 07:58:33.681] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.RpcConfigSourceClient   Subscribe for 'name=cloud.config.sentinel,configId=hosts/vespa,aa9115ebf63cfc6721f75ada21d7cdfa' failed, closing subscriber: Subscribe for 'name=cloud.config.sentinel,configId=hosts/vespa' timed out (timeout was 44000 ms): name=cloud.config.sentinel,configId=hosts/vespa, Current generation: 0, Generation changed: false, Config changed: false
2024-01-18 04:58:33 [2024-01-18 07:58:33.686] INFO    configproxy      configproxy.com.yahoo.config.subscription.impl.JRTConfigRequester        Request failed: Failed request (No application exists) from Connection { Socket[addr=/<IP_ADDRESS>,port=57312,localport=19070] }\nConnection spec: tcp/vespa:19070
2024-01-18 04:58:39 [2024-01-18 07:58:39.334] INFO    configproxy      configproxy.com.yahoo.vespa.config.proxy.DelayedResponseHandler  Timed out (timeout 25000) getting config name=cloud.config.sentinel,configId=hosts/vespa, will retry
2024-01-18 04:58:43 [2024-01-18 07:58:43.674] INFO    config-sentinel  sentinel.config.frt.frtconfigagent       No response / error from config server. This is normal before an application package is deployed. (key: name=cloud.config.sentinel,configId=hosts/vespa) (errcode=103, validresponse:0), trying again in 8.000000 seconds

I noticed 2 things from Vespa logs:

I see the line 2024-01-18 04:54:22 [2024-01-18 07:54:22.097] INFO    configserver     Container.com.yahoo.container.jdisc.ConfiguredApplication        Switching to the latest deployed set of configurations and components. Application config generation: 0 which I saw a video which mentions that that is expected and should be ok.
I notice how configproxy is retrying to connect to something and never connects.
It seems that the application package has never been deployed.

From this I have a comment about pyvespa docs: I did not see any document that is not using the VespaDocker deployment approach. Is there a way to apply a ApplicationPackage to an existing locally deployed Vespa with pyvespa?
Also, I tested out wait_for_application_up(120) available from Vespa implementation, but it always throw a timeout error. I also looked around its code to find a "deploy" action of the schema, but I was not able to find anything like that.

Hi, sorry for slow reponse! The (first) problem to solve is the configproxy not reaching the config server. Please try https://github.com/vespa-engine/sample-apps/blob/master/examples/operations/multinode-HA/docker-compose.yaml and validate that works - then you can modify the compose file, adding your stuff.
Maybe best place to look is network/hostnames - this looks like a connectivity problem, so maybe add a network and use a fully qualified hostname instead of just vespa

Hi @Gladiator566 I think you must look in the vespa.log to validate what the problem might be - and if so, follow the advise to try /multinode-HA/docker-compose.yaml to verify this works, before trying your own configuration
You can also try https://pyvespa.readthedocs.io/en/latest/getting-started-pyvespa-cloud.html to make it easier, using the free trial, to eliminate other failures.

@kkraune I try to use vespa cloud as tutorial, but i got error like RuntimeError: Status code 400 doing POST at https://api.vespa-external.aws.oath.cloud:4443/application/v4/tenant/bge-m3/application/bgeM3/instance/default/deploy/dev-aws-us-east-1c: Value of X-Content-Hash header does not match computed content hash, how to solve this problem?

Thanks for reporting. Can you add the steps you took, so we can reproduce? Or did you follow the steps in https://pyvespa.readthedocs.io/en/latest/getting-started-pyvespa-cloud.html and it failed here?
 app = vespa_cloud.deploy()

A good hint is also to make sure there are no applications already deployed.
@hmusum I assume this is an error from our API, we should document how to fix this

@kkraune yes, I follow the exact steps in https://pyvespa.readthedocs.io/en/latest/getting-started-pyvespa-cloud.html , concretely as bellow:

vespa config set target cloud
vespa config set application bge-m3.bgeM3
vespa auth cert -N
vespa auth api-key
add public api-key to cloud browser key site
vespa_cloud = VespaCloud(  tenant=os.environ["TENANT_NAME"],  application='bgeM3',  key_content=None,  key_location=api_key_path,  application_package=application_package)

and it failed at
app: = vespa_cloud.deploy(), there are no applications already deployed in cloud.
Thanks.

Hi again, I tried https://pyvespa.readthedocs.io/en/latest/getting-started-pyvespa-cloud.html and it worked for me. I run the notebook locally on my laptop. Some ideas
the .vespa directory in your home dir stores credentials - you cna temprarily move this to another name to reset all credentials, and try the guide again, with no other changes. You can also delete the api-key in the console and try with a fresh one

The "Value of X-Content-Hash header does not match computed content hash" error is due to some misconfiguration or bug on the client side, but it's hard to say what the user should do without knowing the root cause of the error.

@Gladiator566 : I have created https://github.com/vespa-engine/vespa/issues/30219 for the X-Content-Hash issue you reported - this is most likely a different issue than reported by @ricoms here. Thanks for reporting!

@kkraune Hi, I try to use bge-m3 model to do embedding hybrid search, and I use refer to official tutorial to deploy a local docker containter to use vespa. Since I have over millions data to feed, so I try to use feed_iterable function to feed iterable bulk data, and I encountered same problems as above, like WARNING/urllib3.connectionpool: Retrying NewConnectionError: Failed to establish a new connection or Max retries exceeded with URL sth like that. I try to set max_connections params to a huge number, and try to create a session to do feed, but it doesn't work, how can I solve this connection full error to insert bulk data to vespa? Thank you !

Hi, I also encounter the same problem. Did you know how to fix it?

Hi @vudangthinh ! I don't think increasing number of connections will help, the error message is probably a symptom of a maxed out instance.
The https://docs.vespa.ai/en/vespa-cli.html has better feed flow control, can you please try that and see how the feeding goes and let me know?

I tried to use vespa feed, however the error still persistent:
At first, the indexing process was ok, but when I run many document the error start happen:
feed: got error "Post "http://<IP_ADDRESS>:8080/document/v1/benchmark/hybridsearch/docid/2989": write tcp <IP_ADDRESS>:40646-><IP_ADDRESS>:8080: write: broken pipe" (no body) for put id:benchmark:hybridsearch::2989: retrying

You are probably sending more requests than the system can handle timely and therefore some of them end up crossing a connection recycling event. These will be retried until timeout so not really an error in itself, but you probably want to increase your resources (maybe run with GPU) or feed slower. Setting a lower timeout (--timeout) should get rid of these messages and lead to less queuing which is probably advantageous if you want to determine faster what actual max throughput you can get.

Closing this as it seems the problem was resolved.


Doesn't work on Media Type in Umbraco 7.5

When I try to use Vorto on a Media Type in Umbraco 7.5 I get the following error when I try to save:
Received an error from the server
An error occured

The given key was not present in the dictionary.

EXCEPTION DETAILS

System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
STACKTRACE

at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Umbraco.Core.Persistence.Repositories.MediaRepository.PersistUpdatedItem(IMedia entity)
   at Umbraco.Core.Cache.DefaultRepositoryCachePolicy`2.CreateOrUpdate(TEntity entity, Action`1 persistMethod)
   at Umbraco.Core.Persistence.Repositories.RepositoryBase`2.PersistUpdatedItem(IEntity entity)
   at Umbraco.Core.Persistence.UnitOfWork.PetaPocoUnitOfWork.Commit(Action`1 transactionCompleting)
   at Umbraco.Core.Services.MediaService.Umbraco.Core.Services.IMediaServiceOperations.Save(IMedia media, Int32 userId, Boolean raiseEvents)
   at Umbraco.Web.Editors.MediaController.PostSave(MediaItemSave contentItem)
   at lambda_method(Closure , Object , Object[] )
   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClass10.<GetExecutor>b__9(Object instance, Object[] methodParameters)
   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__0.MoveNext()

Topic: https://our.umbraco.org/projects/backoffice-extensions/vorto/bugs-feedback-suggestions/54332-using-vorto-in-the-media-section#comment-254117

Also reported it as a bug for Umbraco: http://issues.umbraco.org/issue/U4-8907

It seems this is a bug in Umbraco 7.5: http://issues.umbraco.org/issue/U4-8923

Closing as core issue


Selective logout

Currently you can only log out of all accounts at once. Bookmarks data is lost as well which can be undesirable.

Sorry just saw this actually existed as #2302 so feel free to close one and keep the other with the high tag


Document using a spawning multiprocessing pool for multiprocessing with dask

This is a nice option for working with in-file HFD5/netCDF4 compression:
https://github.com/pydata/xarray/pull/1128#issuecomment-261936849
Mixed multi-threading/multi-processing could also be interesting, if anyone wants to revive that: https://github.com/dask/dask/pull/457 (I think it would work now that xarray data stores are pickle-able)
CC @mrocklin

Can you remind me the motivation to use a spawning multiprocessing pool instead of a fork or forkserver solution?
For mixed multi-threading/multi-processing would a local "distributed" scheduler suffice?  This would be several single-threaded processes on a single machine.  The scheduler would be aware of data locality and avoid inter-node communication when possible.

Actually, I just tested it and it appears that forking also works, as long as you create the pool before opening any files. Otherwise, the netCDF library crashes (https://github.com/pydata/xarray/pull/1128#issuecomment-261841025).
A local "distributed" scheduler might indeed also work, but at least when operating on a single machine it makes sense to bring all data into a single process once it's been loaded for multi-threaded data analysis.

Dask.distributed now creates a forkserver at startup.  This seems to be working well so far.  It nicely balances having a well defined environment and fast startup time.
How much inter-worker data transfer would you expect?  It might be worth running through a few classic algorithms with it instead of the threaded scheduler and looking at performance changes.  The diagnostic pages would be a nice bonus here and might help to highlight some performance issues.
If anyone is interested in this the thing to do is
$ conda install -c conda-forge dask distributed

>>> from dask.distributed import Client
>>> c = Client()  # sets global scheduler by default

And then operate as normal.


Add Cruise Control Plugin.

Add Cruise Control Plugin
Add English translation for Cruise Control
Add Chinese (Simplified) translation for Cruise Control

oh i forgot that
sec

done i think
already have those keys in the keys.json

Yep my bad I didnt realize that you just updated the keys


Document IOP Anti-DDOS detection in the README

@andrew2net can you also document the Anti-DDOS detection in the README? And how the user can address it. Thanks.
Originally posted by @ronaldtse in https://github.com/relaton/relaton-data-bipm/issues/23#issuecomment-1251752061

Added the doc here https://github.com/relaton/relaton-bipm#search-for-a-standard-using-keywords

Thanks!


feature request: make ptcollab remember zoom options

:3

Should the saved zoom options be per project?

I don't think so.


Ansible request for Windows jenkins user creation

Please put the name of the software product (and affectred platforms if relevant) in the title of this issue

[x] Bug in ansible playbook

Details:
We seem to be having issues whereby on Windows machines we are ending up with a C:\Users\jenkins directory which is:

Not writable by the jenkins user
Not the home directory of the jenkins user
Set as the Remote root directory in the jenkins machine definition (because that's where we expect it to be)

It would be useful to find out why the Windows playbooks seem to be giving us two home directories for jenkins - it almost feels like one is being created with C:\Users\jenkins as it's home dir and then another user is being create so it gets a funny name as the home dir, and that new user can't write to c:\Users\jenkins (because it's not the same user)
We have manually tweaked the permissions on the C:\Users\jenkins directory on a number of machines but this is not the correct solution to this problem. We could, of course, modify the Remote root directory to point to the real home directory of the jenkins users but again, that doesn't seem to be the right solution. Let's try and fix this once and for all for reliability/stability.

Unfortunately, I'm unable to recreate the problem with the Windows Vagrant box - the only home user directory being created is C:\Users\jenkins. However, I did notice that the win_user module has an option home_directory. Looking at the test-ibmcloud-win2012r2-x64-1 machine, the actual home directory for the Jenkins user is the C:\users\jenkins.<whatever> folder (judging from the default directories), so it may just be as easy as putting
home_directory: C:\Users\jenkins
in the task- however, I'm unable to test that this will fix it properly, as I can't recreate the issue on the Vagrant box.

I think it's worth doing that change, if only because if it goes wrong it might help us identify when it's occurring.
Looks like it's happened on the AWS boxes, but not the GoDaddy ones. Perhaps I'll create a new AWS one and we can see if it's reproducible if we fire the playbooks on there.
Another thought - did you try running the playbook more than once on the same machine?

@Willsparker I'll give you a nice shiny new AWS machine to experiment with - <IP_ADDRESS>. Will send password elsewhere

ooooooo shiny
I don't think I did, actually. I'll try that with the AWS machine, checking in between. I'll put the fix in before I try :-)

Running the playbook twice ((without the proposed fix) on a Vagrant VM didn't recreate the issue

@Willsparker Another random thought - I wonder if it shows up when you have jenkins set up as a Windows service

You're right! Both of the folders C:\users\jenkins and C:\jenkins\<something> are created when the Jenkins service is started (on the Jenkins user). I'm going to try logging in as the Jenkins user before setting up the Jenkins agent, to see if it changes anything :-)

I've tried creating the C:\users\jenkins folder manually, and the one with the weird output was made when the Jenkins service started again.
So, I tried a third time, and if I logged in as the Jenkins user before setting the agent up as a service, the folder isn't created. Therefore, if there's some way we can automate logging on as the Jenkins user in the playbook, we won't run into this situation again (hopefully)..

I didn't think the service was set-up during the playbook - am I mistaken?
So you need to log in manually to be able to configure the service. In which case this shouldn't occur.
(I feel I'm missing something here)
Or is it that it happens each time the service is stopped/started, which would be more of a problem.

No, the service isn't setup as part of the playbook - however, you don't need to login as the Jenkins user to setup the service. To stop the weird folder from being created, the Jenkins user has to have been logged in, so Windows could finish setting up the user (in which, it creates the C:\Users\jenkins folder). If you then setup the service, the weird folder isn't created, and the home directory is C:\Users\jenkins as we want it to be.
I'm trying to come up with a way of the playbook automatically doing this sign-in thing, so it isn't required to be done manually by the person setting the machine up, before the starting the Jenkins Agent service

So given that you'd normally set it up as the jenkins user, what process is it that makes it create the extra one? Is it if you set it up as Administrator (Not something I've done before as you generally don't want it running at all as anything other than the jenkins user)

You have to be the administrator to install it as a service, IIRC. The process of installing it as a service (and starting it), as the Administrator (before the Jenkins user has been logged on), but then changing the service settings to make it run under the Jenkins user appears to be what creates the folder.
Presumably if you can install the Jenkins agent as a service, as the Jenkins user, the folder wouldn't be made.
Also ... why don't we setup the Jenkins agent via the playbooks? https://docs.ansible.com/ansible/latest/collections/ansible/windows/win_service_module.html It looks fairly doable ...?

I thought you run the service as jenkins and use the menu option in there to install it which prompts you for the admin credentials? Although it's been a while since I did it...

Also ... why don't we setup the Jenkins agent via the playbooks? https://docs.ansible.com/ansible/latest/collections/ansible/windows/win_service_module.html It looks fairly doable ...?

Haven't looked at it - depends if you can get the secret in there I guess.

Hmm- some documentation for setting up the Jenkins agent on windows machines are needed then, as I do it by the way I described above.


Issues with Rails <IP_ADDRESS>

I'm trying to upgrade my app from Rails 5.2.4 to Rails <IP_ADDRESS> and on Rails 6 anything I'm trying to do with Thinking Sphinx outputs:
NoMethodError:  undefined method `first' for nil:NilClass
# /home/rof/cache/bundler/ruby/2.6.0/gems/joiner-0.4.2/lib/joiner/joins.rb:22:in `alias_for'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/column_sql_presenter.rb:48:in `table'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/column_sql_presenter.rb:34:in `escape_table?'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/column_sql_presenter.rb:18:in `with_table'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/property_sql_presenter.rb:62:in `collect'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/property_sql_presenter.rb:62:in `columns_with_table'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/property_sql_presenter.rb:43:in `casted_column_with_table'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/property_sql_presenter.rb:19:in `to_select'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/sql_builder/statement.rb:70:in `collect'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/sql_builder/statement.rb:70:in `presenters_to_select'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/sql_builder/statement.rb:117:in `select_clause'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/sql_builder/statement.rb:84:in `scope_by_select'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/sql_builder/statement.rb:49:in `filter_by_scopes'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/sql_builder/statement.rb:14:in `to_relation'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/sql_builder.rb:13:in `sql_query'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/sql_source.rb:128:in `build_sql_query'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/sql_source.rb:153:in `prepare_for_render'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/active_record/sql_source.rb:71:in `render'
# /home/rof/cache/bundler/ruby/2.6.0/gems/riddle-2.4.0/lib/riddle/configuration/index.rb:30:in `block in render'
# /home/rof/cache/bundler/ruby/2.6.0/gems/riddle-2.4.0/lib/riddle/configuration/index.rb:30:in `collect'
# /home/rof/cache/bundler/ruby/2.6.0/gems/riddle-2.4.0/lib/riddle/configuration/index.rb:30:in `render'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/core/index.rb:75:in `render'
# /home/rof/cache/bundler/ruby/2.6.0/gems/riddle-2.4.0/lib/riddle/configuration.rb:45:in `block in render'
# /home/rof/cache/bundler/ruby/2.6.0/gems/riddle-2.4.0/lib/riddle/configuration.rb:45:in `collect'
# /home/rof/cache/bundler/ruby/2.6.0/gems/riddle-2.4.0/lib/riddle/configuration.rb:45:in `render'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/configuration.rb:111:in `render'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/configuration.rb:119:in `block in render_to_file'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/configuration.rb:119:in `render_to_file'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/test.rb:10:in `start'
# /home/rof/cache/bundler/ruby/2.6.0/gems/thinking-sphinx-4.4.1/lib/thinking_sphinx/test.rb:17:in `start_with_autostop'
# ./spec/controllers/listings_controller_spec.rb:84:in `block (4 levels) in <top (required)>'

What am I doing wrong?

Can you try running bundle update joiner?
Joiner is a gem I extracted from Thinking Sphinx, and it manages the complex SQL join generation (which TS uses when creating the SQL statements in configuration files). ActiveRecord changes its internals more often than I'd like, so Joiner is updated to match specific Rails versions - but upgrading Rails itself doesn't always mean the latest Joiner release is pulled into your bundle as well. And I can confirm that Joiner v0.5.0 is the first release that supports Rails v6, but your stack-trace is using 0.4.2.

Yes, upgrading joiner from 0.4.2 to 0.5.0 fixed the issue. Thank you.

This also fixed an issue I had on Rails 5.2.
NoMethodError: protected method `join_root' called for #<ActiveRecord::Associations::JoinDependency:0x0000000005cb8400>

Fixed once I updated joiner.


Upgradable contract issue

Hello, I would like to deploy an upgradeable contract but I am getting this error.
Downloading compiler 0.8.17
DeclarationError: Identifier not found or not unique.
  --> src/lib/ERC721AConduitPreapprovedUpgradeable.sol:19:16:
   |
19 |     ) internal onlyInitializing {
   |                ^^^^^^^^^^^^^^^^

Error HH600: Compilation failed

For more info go to https://hardhat.org/HH600 or run Hardhat with --show-stack-traces

Besides, I don't understand what address "0x4468A5B725E2C63056131121cD33b66848E1dd87" is in scripts/deploy.ts.
(Related article: https://forum.openzeppelin.com/t/deploy-proxy-parameters/38707)
Thank you for your consideration.

By the way, are there any deployed contracts inheriting this upgradable contract? If yes, can you please share them?
I am still not sure how I can use this for my contract.

@opensea712 thanks, this should be fixed now if you want to pull the main branch.

By the way, are there any deployed contracts inheriting this upgradable contract? If yes, can you please share them?

Yes people are using this, I'd have to remember which projects though.
You can look at ExampleToken.sol in src-upgradeable as an example that inherits and modifies.


🟠 (#1232) NEW FEATURES

6 SUGGESTIONS:

[ ] 1. Crop (in Edit tab): add ability to add multiple images (in case if all images has similar areas to be cropped .. Here the user adjusts the cropping area preference in one image, then apply this preference to ALL other images) + Add (Quality) option for JPG format.

[ ] 2. Draw (in Create tab): add (Quality) option for JPG format [in (Draw on image) & (Draw on background)] + When you create & make (Blend colors) tool .. Add a button for it inside (Draw) tool.

[ ] 3. Filter (in Create tab): inside (Mask Filter) add ability to add multiple images + Add ability to save the (Favourite) filters inside the backup file of settings.

[x] 4. OCR (in Image tab): add option to open camera to get the text from it.

[ ] 5. NEW FEATURE = (Colors Clipboard) feature: with it user can add his favourite colors in it (colors he uses constantly) .. And just pasting any one of them (without adjusting the color every time) inside any tool he uses [Mask Filter, Draw, Image Stitching (background color), Watermarking (Text color & Background color), Gradient Maker, Gradient Overlay, Material You (in Generate Pallete tool) + (Blend colors) tool when you create it] (enable manually reordering the colors).

[ ] 6. Zip (in Tools tab): add these features: compression level [Store (no compression), Normal & Ultra (best compression)] + Password (add ability to encrypt data & filenames; so contents couldn't be seen without the password) + Option to delete source items after compression.

ENHANCEMENT:

[x] 1- ENHANCEMENT = Filter (in Create tab): when opening it .. Firstly it asks to add images, then opening new window to choose (Filter or Mask Filter) 👉 here are 2 REPEATED WINDOWS shows [look at the attached image] .. The suggested fix: firstly selecting filter type, then adding images.
The same issue is in Generate Palette tool (in Image tab) 👉 (Generate palette or Material You).

[x] 2- Full Filter (in Create tab): when opening it there are 2 repeated buttons of (Full Filter), one under the image [Add filter) button], the other is in the top corner .. So replace the upper button with (Mask Filter) button OR just remove it completely.

If something is repeated that means that i want it to, so it's by design

OK, I got it.


lighten weight of ngraph_theme by not customizing webtheme w font files so large

@rkimballn1 I think this is also about documentation reading experience for our users. If they don't have to load 19MBs to read our documentation, that could be a win!

@rkimballn1  I don't quite understand what you want me to do here in these "requesting changes" ...  Can you please clarify?  Close the PR w/no changes?  Something else? Thx.


Exporting DemoEnvironment

Hi,
I'm currently implementing a wgpu based backend crate and a winit based platform crate and would like to use the DemoApp in my simple example.
Since I need to call the ui() function on the DemoApp, I need create a DemoEnvironment struct, which is marked pub but is not exported. Could you maybe export this struct?

Good catch 👍
Exciting with a wgpu backend by the way! Let me know if you want further assistance.

@emilk Things are looking good. I might take your offer for some odd resizing bugs I found, but not until I have tested it further myself more.


Enhancements for JmDNS github repository configuration

To make JmDNS more attractive for consumers/developers:

Imho it would be better to have it explicitly expressed that JAVA really is the most used language (instead of HTML) within the project. I believe linguist configuration needs to be adapted/utilized to modify on this.

additionally, the repository could be classified with topics, it would be a beneficial to have e.g. "zeroconf", "service-discovery", "mdns" topics added. Please add further suggestions in comments section. More information about github topics.

What is your opinion?

Agreed and you could probably add avahi, and rendezvous for tags as well.

Agreed and you could probably add avahi, and rendezvous for tags as well.

Using "avahi" feels a bit like SEO as Avahi is a different implementation of mDNS and that's IMO the only common factor.
The term "rendezvous" has been dropped by Apple itself due to a trademark issue with Tibco, so ....

Good points.
Rendezvous is legacy term but this lib was originally JRenedzvous before the spec was finalized and most users treat "avahi" as synonymous with MDNS on Linux.
Feel free to ignore.

I've added the topics suggested by @janossch and ignored the ones from @melloware (sorry 😉) due to @ViToni's valid points.
Wrt (1), would anybody have a suggestion what .gitattributes content should be added? Feel free to create a PR yourself for it!

@kaikreuzer I can create a PR for it. But before that:
Solution 1:
The first obvious solution which came into my mind is to have a .gitattributes with this content:
devdocs/* linguist-documentation
src/site/* linguist-documentation

Solution 2:
src/site/* linguist-documentation

As you can see we can omit the first line if:
a. move devdocs folder under a main docs folder OR
b. just simply rename devdocs to docs
In this way there is no need to include it as an override for linguist as linguist excludes some documentation files already by default.
Which one you prefer? 1, 2a or 2b?

I would not touch the docs for now and thus go with (1).
I'm actually not sure about the status of src/site and devdocs - both folders have not been touched for many years and are likely pretty outdated (apart from the official specs that are there as txt files). Might be worth to look into the docs as well and possibly publish it to github-pages (for whatever is up-to-date and still relevant).

I'm actually not sure about the status of src/site and devdocs - both folders have not been touched for many years and are likely pretty outdated (apart from the official specs that are there as txt files). Might be worth to look into the docs as well and possibly publish it to github-pages (for whatever is up-to-date and still relevant).

I rememeber to have touched the devdocs folder some years back...
Based on current knowledge I'd suggest to remove the RFC files from the folder and reference the one from IETF (these are more up-to-date).  Also we don't need any experimental / DRAFT RFCs anymore.
Maybe even convert the HTML index file to Markdown so that it is usable directly from GitHub UI.


Document Microsoft.Data.Sqlite

Depends on https://github.com/dotnet/samples/pull/1846
cc @mairaw

cc @ajcvickers

There are a couple of warnings there are not related to the missing samples

Rebuilding now that samples are merged.

Thanks for all your help on finding a home for these!


Contract GUI address field input sanitation broken?

Describe the issue
I am not able to have the wallet Smart Contract "Send To" address fields accept anything I throw at them. When exiting an address field, it receives a red border (and the Call Contract button is unresponsive.)
I have tried enclosing the address in single and double quotes, though I wouldn't expect to need that. Unsurprisingly, this made no difference.
See attached image, showing a valid HTMLCOIN address as described above.
Can you reliably reproduce the issue?
Yes
If so, please list the steps to reproduce below:

Go to Smart Contract tab, then Send To ->
(optional) Enter any valid token contract address / ABI
Select a function requiring an address input / argument
Note that the address is not accepted.

NOTE: The source and ABI for specific  contract in the screenshot is included, below.
ALSO: The block explorer appears to be showing Qtum internal token addresses? I tried those too but still received the red boarder etc.
Expected behaviour
A valid address should pass entry sanitation checks.
Actual behaviour
Valid addresses are not accepted.
Screenshots.

What version of HTMLCOIN are you using?
HTMLCOIN CORE version <IP_ADDRESS> (64-bit) RELEASE (binary)
Machine specs:

OS: Ubuntu 16.04 (64bit)

Example Contract Details
Contract Address
eb82e586d0d402400f2d00e7a25e5141b64aeaa4

Source Code
pragma solidity ^0.4.16;

interface tokenRecipient { function receiveApproval(address _from, uint256 _value, address _token, bytes _extraData) external; }

contract TokenERC20 {
    // Public variables of the token
    string public name;
    string public symbol;
    uint8 public decimals = 18;
    // 18 decimals is the strongly suggested default, avoid changing it
    uint256 public totalSupply;

    // This creates an array with all balances
    mapping (address => uint256) public balanceOf;
    mapping (address => mapping (address => uint256)) public allowance;

    // This generates a public event on the blockchain that will notify clients
    event Transfer(address indexed from, address indexed to, uint256 value);

    // This generates a public event on the blockchain that will notify clients
    event Approval(address indexed _owner, address indexed _spender, uint256 _value);

    // This notifies clients about the amount burnt
    event Burn(address indexed from, uint256 value);

    /**
     * Constructor function
     *
     * Initializes contract with initial supply tokens to the creator of the contract
     */
    constructor (
        uint256 initialSupply,
        string tokenName,
        string tokenSymbol
    ) public {
        totalSupply = initialSupply * 10 ** uint256(decimals);  // Update total supply with the decimal amount
        balanceOf[msg.sender] = totalSupply;                // Give the creator all initial tokens
        name = tokenName;                                   // Set the name for display purposes
        symbol = tokenSymbol;                               // Set the symbol for display purposes
    }

    /**
     * Internal transfer, only can be called by this contract
     */
    function _transfer(address _from, address _to, uint _value) internal {
        // Prevent transfer to 0x0 address. Use burn() instead
        require(_to != 0x0);
        // Check if the sender has enough
        require(balanceOf[_from] >= _value);
        // Check for overflows
        require(balanceOf[_to] + _value >= balanceOf[_to]);
        // Save this for an assertion in the future
        uint previousBalances = balanceOf[_from] + balanceOf[_to];
        // Subtract from the sender
        balanceOf[_from] -= _value;
        // Add the same to the recipient
        balanceOf[_to] += _value;
        emit Transfer(_from, _to, _value);
        // Asserts are used to use static analysis to find bugs in your code. They should never fail
        assert(balanceOf[_from] + balanceOf[_to] == previousBalances);
    }

    /**
     * Transfer tokens
     *
     * Send `_value` tokens to `_to` from your account
     *
     * @param _to The address of the recipient
     * @param _value the amount to send
     */
    function transfer(address _to, uint256 _value) public returns (bool success) {
        _transfer(msg.sender, _to, _value);
        return true;
    }

    /**
     * Transfer tokens from other address
     *
     * Send `_value` tokens to `_to` on behalf of `_from`
     *
     * @param _from The address of the sender
     * @param _to The address of the recipient
     * @param _value the amount to send
     */
    function transferFrom(address _from, address _to, uint256 _value) public returns (bool success) {
        require(_value <= allowance[_from][msg.sender]);     // Check allowance
        allowance[_from][msg.sender] -= _value;
        _transfer(_from, _to, _value);
        return true;
    }

    /**
     * Set allowance for other address
     *
     * Allows `_spender` to spend no more than `_value` tokens on your behalf
     *
     * @param _spender The address authorized to spend
     * @param _value the max amount they can spend
     */
    function approve(address _spender, uint256 _value) public
        returns (bool success) {
        allowance[msg.sender][_spender] = _value;
        emit Approval(msg.sender, _spender, _value);
        return true;
    }

    /**
     * Set allowance for other address and notify
     *
     * Allows `_spender` to spend no more than `_value` tokens on your behalf, and then ping the contract about it
     *
     * @param _spender The address authorized to spend
     * @param _value the max amount they can spend
     * @param _extraData some extra information to send to the approved contract
     */
    function approveAndCall(address _spender, uint256 _value, bytes _extraData)
        public
        returns (bool success) {
        tokenRecipient spender = tokenRecipient(_spender);
        if (approve(_spender, _value)) {
            spender.receiveApproval(msg.sender, _value, this, _extraData);
            return true;
        }
    }

    /**
     * Destroy tokens
     *
     * Remove `_value` tokens from the system irreversibly
     *
     * @param _value the amount of money to burn
     */
    function burn(uint256 _value) public returns (bool success) {
        require(balanceOf[msg.sender] >= _value);   // Check if the sender has enough
        balanceOf[msg.sender] -= _value;            // Subtract from the sender
        totalSupply -= _value;                      // Updates totalSupply
        emit Burn(msg.sender, _value);
        return true;
    }

    /**
     * Destroy tokens from other account
     *
     * Remove `_value` tokens from the system irreversibly on behalf of `_from`.
     *
     * @param _from the address of the sender
     * @param _value the amount of money to burn
     */
    function burnFrom(address _from, uint256 _value) public returns (bool success) {
        require(balanceOf[_from] >= _value);                // Check if the targeted balance is enough
        require(_value <= allowance[_from][msg.sender]);    // Check allowance
        balanceOf[_from] -= _value;                         // Subtract from the targeted balance
        allowance[_from][msg.sender] -= _value;             // Subtract from the sender's allowance
        totalSupply -= _value;                              // Update totalSupply
        emit Burn(_from, _value);
        return true;
    }
}

####ABI
[
	{
		"constant": true,
		"inputs": [],
		"name": "name",
		"outputs": [
			{
				"name": "",
				"type": "string"
			}
		],
		"payable": false,
		"stateMutability": "view",
		"type": "function"
	},
	{
		"constant": false,
		"inputs": [
			{
				"name": "_spender",
				"type": "address"
			},
			{
				"name": "_value",
				"type": "uint256"
			}
		],
		"name": "approve",
		"outputs": [
			{
				"name": "success",
				"type": "bool"
			}
		],
		"payable": false,
		"stateMutability": "nonpayable",
		"type": "function"
	},
	{
		"constant": true,
		"inputs": [],
		"name": "totalSupply",
		"outputs": [
			{
				"name": "",
				"type": "uint256"
			}
		],
		"payable": false,
		"stateMutability": "view",
		"type": "function"
	},
	{
		"constant": false,
		"inputs": [
			{
				"name": "_from",
				"type": "address"
			},
			{
				"name": "_to",
				"type": "address"
			},
			{
				"name": "_value",
				"type": "uint256"
			}
		],
		"name": "transferFrom",
		"outputs": [
			{
				"name": "success",
				"type": "bool"
			}
		],
		"payable": false,
		"stateMutability": "nonpayable",
		"type": "function"
	},
	{
		"constant": true,
		"inputs": [],
		"name": "decimals",
		"outputs": [
			{
				"name": "",
				"type": "uint8"
			}
		],
		"payable": false,
		"stateMutability": "view",
		"type": "function"
	},
	{
		"constant": false,
		"inputs": [
			{
				"name": "_value",
				"type": "uint256"
			}
		],
		"name": "burn",
		"outputs": [
			{
				"name": "success",
				"type": "bool"
			}
		],
		"payable": false,
		"stateMutability": "nonpayable",
		"type": "function"
	},
	{
		"constant": true,
		"inputs": [
			{
				"name": "",
				"type": "address"
			}
		],
		"name": "balanceOf",
		"outputs": [
			{
				"name": "",
				"type": "uint256"
			}
		],
		"payable": false,
		"stateMutability": "view",
		"type": "function"
	},
	{
		"constant": false,
		"inputs": [
			{
				"name": "_from",
				"type": "address"
			},
			{
				"name": "_value",
				"type": "uint256"
			}
		],
		"name": "burnFrom",
		"outputs": [
			{
				"name": "success",
				"type": "bool"
			}
		],
		"payable": false,
		"stateMutability": "nonpayable",
		"type": "function"
	},
	{
		"constant": true,
		"inputs": [],
		"name": "symbol",
		"outputs": [
			{
				"name": "",
				"type": "string"
			}
		],
		"payable": false,
		"stateMutability": "view",
		"type": "function"
	},
	{
		"constant": false,
		"inputs": [
			{
				"name": "_to",
				"type": "address"
			},
			{
				"name": "_value",
				"type": "uint256"
			}
		],
		"name": "transfer",
		"outputs": [
			{
				"name": "success",
				"type": "bool"
			}
		],
		"payable": false,
		"stateMutability": "nonpayable",
		"type": "function"
	},
	{
		"constant": false,
		"inputs": [
			{
				"name": "_spender",
				"type": "address"
			},
			{
				"name": "_value",
				"type": "uint256"
			},
			{
				"name": "_extraData",
				"type": "bytes"
			}
		],
		"name": "approveAndCall",
		"outputs": [
			{
				"name": "success",
				"type": "bool"
			}
		],
		"payable": false,
		"stateMutability": "nonpayable",
		"type": "function"
	},
	{
		"constant": true,
		"inputs": [
			{
				"name": "",
				"type": "address"
			},
			{
				"name": "",
				"type": "address"
			}
		],
		"name": "allowance",
		"outputs": [
			{
				"name": "",
				"type": "uint256"
			}
		],
		"payable": false,
		"stateMutability": "view",
		"type": "function"
	},
	{
		"inputs": [
			{
				"name": "initialSupply",
				"type": "uint256"
			},
			{
				"name": "tokenName",
				"type": "string"
			},
			{
				"name": "tokenSymbol",
				"type": "string"
			}
		],
		"payable": false,
		"stateMutability": "nonpayable",
		"type": "constructor"
	},
	{
		"anonymous": false,
		"inputs": [
			{
				"indexed": true,
				"name": "from",
				"type": "address"
			},
			{
				"indexed": true,
				"name": "to",
				"type": "address"
			},
			{
				"indexed": false,
				"name": "value",
				"type": "uint256"
			}
		],
		"name": "Transfer",
		"type": "event"
	},
	{
		"anonymous": false,
		"inputs": [
			{
				"indexed": true,
				"name": "_owner",
				"type": "address"
			},
			{
				"indexed": true,
				"name": "_spender",
				"type": "address"
			},
			{
				"indexed": false,
				"name": "_value",
				"type": "uint256"
			}
		],
		"name": "Approval",
		"type": "event"
	},
	{
		"anonymous": false,
		"inputs": [
			{
				"indexed": true,
				"name": "from",
				"type": "address"
			},
			{
				"indexed": false,
				"name": "value",
				"type": "uint256"
			}
		],
		"name": "Burn",
		"type": "event"
	}
]

same problem v<IP_ADDRESS> wallet

Hope they fix it one day. 🤷 Closing to clean up my open issues list.


Feature suggestion: entc --version command

I did read https://entgo.io/docs/code-gen/#version-compatibility-between-entc-and-ent but I was still faced with a divergence between the results of running entc and go run github.com/facebookincubator/ent/cmd/entc. Turns out I didn't rebuild my dev container after upgrading the package, so the old entc binary lingered on.
I feel like entc --version would have pointed me in the right direction a bit sooner. What do you think? Happy to give it a shot myself.

This issue has been discussed here several times. See previous discussions: https://github.com/facebookincubator/ent/issues/450, https://github.com/facebookincubator/ent/issues/368.
Use go version -m <binary> if you need the entc version.

Oups! Sorry for not having done the research! Thanks for linking the previous issues!


[ORD-11.2] Create New Group

User Story # ORD-11.2
2. Revision History

Version
Date
Description

V1.0
10.12.2024
From Discovery: ORD-11.1 & ORD 11.2. The system should support the ability to create, edit, and manage user groups. Users should be able to form groups, modify group details, add or remove members, and assign roles or permissions within the group. This functionality ensures efficient collaboration and structured access control across different teams and projects. (Pic. 35-36).

V2.0
12.12.2024
Updated part 4, 7, 8, 10 based on the clarifications with dev team

V3.0
17.12.2024
Updates based on the tech questions discussion

3. User Story
AS A user,
I WANT to create new groups and add members by email or ID (ORCID or possibly from other IDPs)
SO THAT I can collaborate effectively and have structured access control across different teams and projects.
4. Acceptance Criteria (TO BE FINALIZED)

User Role
AC#
GIVEN
WHEN
THEN

User (General User and Super Admin)
1
The user is logged into the system and on the main "Contribute" page (Pic. 1)
The user selects the "+Group" in the "My Groups" field (Pic. 1)
The group creation modal is displayed (Pic. 2)

User (General User and Super Admin)
2
The user enters the name of the group and clicks "Save"
The group sidebar opens
The group's members list shows the creator of the group as "Admin" (Pic. 3)

User (General User and Super Admin)
3
The group sidebar is opened
The user enters an email or ID (ORCID or other) of the member and clicks "Add User" (Pic. 4)
The system adds that user in a "Viewer" role by default and shows the added member's details (name, email, ID) in the Members list (Pic. 5). The system saves the group information updates automatically

User (General User and Super Admin)
4
The group form is opened and the User wants to change the role for the member from default "Viewer"
The user chooses in the dropdown menu other role for the new member (Pic. 6)
The system updates the role shows that information in the Members list and saves all updates automatically (Pic. 7)

User (General User and Super Admin)
5
The group form is opened
The user enters an email or ID (ORCID or other) of the member and clicks "Add User"
The system doesn't find the member, a notification is showed up with the similar message: "There is no user in ORD with this identifier yet." (Pic. 6)

User (General User and Super Admin)
6
The User has created the group and wants to update the name of the group
The user has group sidebar open or clicks on the group in "My Groups" field and then navigates to the name edit button (Pic. 3)
The modal opens where user enters the new name and clicks "Save" or "Cancel" (Pic. 8)

User (General User and Super Admin)
7
The User is on "Contribute" main page and wants to find and reopen the specific group sidebar (for information or edits)
The user decides to use "Search by group" function in "My Groups" field for quick search and enters the name of the group of interest (Pic. 1)
The search function displays the group by the entered name

User (General User and Super Admin)
8
The User is on "Contribute" main page and wants to find and reopen the specific group sidebar (for information or edits)
The user decides to search the group by scrolling the list of all available groups in "My Groups" (Pic. 1)
The system has the groups always in alphabetical order, so that the search is easy and the user can rename the group for their convenience of having it at the most accessible place

Pic. 1

Pic. 2

Pic. 3

Pic. 4

Pic. 5

Pic. 6

Pic. 7

Pic .8
5. Mockups link
https://www.figma.com/design/c7M9fhVn278ei9rkepI2Sm/UX-UI-Open-Reaction-Database?node-id=1151-67630&t=1VzxedXUPCZeTAPb-11
6. User Interaction and Design (*TO BE FINALIZED by Iurii)

#
Name
Type
Description

1
"My Groups" Dashboard on main "Contribute" page
Dashboard
Access point for creating group

2
"+Group" Feature
Button
Initiates group creation process

3
Add Group
Modal
This is shown by the system to initiate the new group creation and adding a group name there

4
Save
Button
This is clicked by the user when new group name is entered, and group initiation is confirmed.

5
Cancel
Button
This is clicked by the user if they decide to cancel group creation.

6
Group Creation / Editing Sidebar
Sidebar
Form to enter emails to "Add" new members and edit their roles

7
Group Name edit button
Button
Used to initiate the group name edits. should be accessible only for Group "Administrators"

8
Group Name form
Modal
Allows Group Creator to edit the group name. After group creation, it should be always available for edits for Group "Administrators" only.

9
Save
Button
This is clicked by the user when new group name is entered and is to be saved.

10
Cancel
Button
This is clicked by the user if they decide to cancel changing the group name.

11
Email / ID enter field
Input Text Field
Allows Group Creator to enter email / ID of member

12
Add User
Button
Adds the user (based on entered email or ID) as a member (default "Viewer") to the group

13
Notification
Message/Alert
The message content is: "There is no user in ORD with this identifier yet."

14
Group Members Roles Dropdown
Dropdown
Dropdown to assign roles: Administrator, Editor, Viewer and to Remove User

7. Non-functional requirements

Performance and Reliability: The group creation and member addition process must handle multiple concurrent requests (~50) efficiently.
Disaster Recovery (NFR-3): The system should have automated backups for the groups information.
Security: The system should ensure that the groups information is stored securely and is only accessible to groups members.

8. Assumptions

The ORD database should already have the details of the added member (email. etc.), so the member is expected to be already authenticated in ORD through ORCID or other IDP. If not, the member would need to authenticate first and log in to the "Contribute" page for the first time in order to be added afterwards.
Group Name should be always available for edits for Group "Administrators" only
The user adds new members one by one.
Each group should have at least one "Administrator" [ORD-11.1]
The Group Name field should restrict the exceed of 55 characters input
ORD-11.2 is merged with ORD-11.3 (Add user to the group via ORCID ID)
Acceptance criteria 4 is related to ORD-11.1 Roles in Groups

9. Dependencies

Description
Dependent On

Depends on the user authentication system to verify user and role-based access control (RBAC)
ORD 2.1-2.3  (Authentication) & ORD-11.4-11.9 (Groups and Users Management)

10. Out of Scope

#
Description
User Story

1
Addition of several members to the group simultaneously (adding several emails at once in the field) is out of MVP scope. The user will need to add members one by one. If necessary, such feature can be planned for in Full Scope
NA

11. Questions

#
Question
Address to Whom
Answer

1
What are the character limits for group names?
dev team
55 characters - standards

2
Is it acceptable to add ORCID invitations after email invitations are operational?
dev team
yes

3
Should invitations be sent individually or can multiple invitations be sent at once when several emails are entered in the field?
dev team
For MVP - send individually

4
Should the system support an unlimited number of groups, or is there a limit?
dev team / user
Not relevant

5
Should the function of deleting the full group be included in Full Scope?
dev team / user
Yes

6
To confirm that the addition of the members will be done inside the system (invite emails will not be sent) once the group creator enters the email and clicks "add" the member will be automatically added
dev team / user
Confirmed at PBR 12.12.2024

7
Should all users to able to create new group? even viewers without any group at the moment (RBAC) discussion 11.4.2?
dev team / user
Yes

8
To confirm that in Full scope the "leave the group" button will be added for members ?
user
Confirmed at PBR 12.12.2024

9
Show at the main Contribute page when Group sidebar is open - that new group creation is in progress? If useful for user, can be possibly added in full scope
user
Possibly at Full scope - but not critical function

10
How groups should be filtered in "My groups" last created or by name, etc.?
user
Groups in list should be in alphabetical order

11
Are there cases expected when there are no user names to display in the members list?
user
User always has name, email, and ORCID

12
What exact information is to be displayed for the members of different roles in the group?
user
Name, email, ORCID (all details) should be displayed to all roles in the group

Acceptance Criteria - the user roles seem to all be "User (General User and Super Admin)" whereas there are clearly items restricted to admin user.

AC# 1, 2, 7, 8  = any role
AC# 3, 4, 5, 6  = Admin only
Additional Notes:
AC#2 - Given "The user has selected the option to create a new group"

No comments on other sections.

Non-Functional Requirements

Note that handling 50 concurrent group creation or member addition process concurrently seems unnecessarily high to Ben and Connor, so let us know if this requirement becomes a challenge.

No comment on rest of User Story.


Fusesoc fork not programming my zyboz7-10

Observed Behavior

Hello, I am a french intern in a research lab, and I am trying to use ibex on a small zyboz7-10.
I took the  and modified it so that it synthesizes.
I run the fusesoc command in the root of ibex-demo-system :
fusesoc --cores-root=. run --target=synth --tool=vivado lowrisc:ibex:demo_system_zybo
It does synthesizes without errors and I get a message :
INFO: Building
INFO: Running

but my fpga is not programmed (I still have the previous programmed loaded onto it).
I was able to reproduce this behavior on , where the fork of fusesoc from the ibex doc could not program my board where as the version 2.0 of fusesoc could.
fusesoc 2.0 was able to program the card (and so did vivado) so I think this issue is related to the fusesoc fork.
I am posting the issue here because I think my problem is due to the fork of fusesoc ibex uses.
Is this normal with this version of fusesoc ? Is there a know solution/replacement (like openocd) ?
You can delete this issue if I was wrong and/or just mi-used fusesoc (I am pretty new to this style of rtl flow ^^').
Thanks for your time
My Environment

EDA tool and version:

vivado 2022.2
Operating system:

CentOS 7.6.1810
Version of the Ibex source code:

version of ibex : 93c8e92c0ddbeed1239b04251cfb7d9ae68b5d1f (no modification)
version of ibex-demo-system : fb64750c926ffefec1f10c038e2f986aedf2e29a (added files for zybo z7 -10)
version of fusesoc : 0.1

update I found a solution by using the makefile fusesoc left in /build//synth-vivado


Wrong modified equation in some cases

Synopsis
For some discretizations, the modified equation that is generated is wrong.
Instructions to duplicate:
Here is some code that will show the problem:
from sympy import symbols,latex
from src.pymodpde import DifferentialEquation, i, n # import the differential equation and the indicies we need to use

u= symbols('u')
D= symbols('D')

pde = DifferentialEquation( dependentVarName='y',
                            independentVarsNames=['x'])

t = n+1
advection = ( pde.y(time=t,x=i) - pde.y(time=t,x=i-1) ) / pde.dx
diffusion = (
    pde.y(time=t,x=i-1)
    - 2*pde.y(time=t,x=i)
    + pde.y(time=t,x=i+1)
) / pde.dx**2

pde.set_rhs( -u*advection + D*diffusion )

pde.generate_amp_factor()
# pde.display_amp_factor()
pde.generate_modified_equation(nterms=3)
pde.display_modified_equation()

Workaround
Changing
pde.display_modified_equation()

to
pde.symbolic_modified_equation()

results in a correct (I think) modified equation.

I have addressed this issue in this commit 5e9631a


Problem install

Hi,
I got this problem when I try to install it :
ERROR in node_modules/@types/grecaptcha/index.d.ts(9,13): error TS2451: Cannot redeclare block-scoped variable 'grecaptcha'.
node_modules/recaptcha-v3/dist/grecaptcha/grecaptcha.d.ts(2,11): error TS2451: Cannot redeclare block-scoped variable 'grecaptcha'.
I uninstalled then installed again but it changes nothing.
Any idea ?
Thanks !

Conflict with other captcha depedencies.


Web - Reset Password

For some reason when I run yarn, it installs a different checksum of some packages. Am I getting mitm'ed or is something else going on?

Not sure but this branch didn't touch any packages


fix: resolve Azure Linux nvidia-container-toolkit posttrans error

What type of PR is this?

What this PR does / why we need it:

This PR fixes an issue with Azure Linux AKS GPU where an error takes place within the posttrans phase when installing the nvidia-container-toolkit RPM package. that causes /etc/nvidia-container-runtime/config.toml to not be generated in the OS. The reason of the failure is because nvidia-ctk, the binary used to generate the config file is missing. This is because nvidia-ctk is provided by the nvidia-container-toolkit-base RPM package and this RPM is installed after nvidia-container-toolkit. This PR thus changes the RPM installation sequence to satisfy the package runtime requirements.

Which issue(s) this PR fixes:

Fixes #
Requirements:

[ ] uses conventional commit messages

[ ] includes documentation
[ ] adds unit tests
[ ] tested upgrade from previous version

Special notes for your reviewer:
Release note:
none

lgtm I think, is this currently causing wide impact in prod? if so then it'd probably worth to cherry-pick this to master so it can roll out in next week's release

lgtm I think, is this currently causing wide impact in prod? if so then it'd probably worth to cherry-pick this to master so it can roll out in next week's release

This has not been causing impact in prod yet since no customer bug requests have been noticed, but there's a possibility that customer workloads might be impacted by this issue.

lgtm!


fix(@angular-devkit/build-angular): remove inline style soucemaps in server output

Fixes #12940

Actually thinking about it, we should have sourcemaps for non bootstrapped apps


Make Username:Password work with other schools.

Add a Enter School field which uses the API to get the school id and form the URL.

@Gwyd0

u a legend my bruda, was about to open an issue report

not my school but ye. not sure what I'm doing wrong.
[SPARX BWK LOGS]
[START TIME] 06-11-2022-20-11-45
--- [SETTINGS] ---
[USER] username
[PASSWORD] password
[SCHOOL] (my school was inputted here)
[AUTOCONTINUE] (my school was inputted here in the logs somehow)
[AUTOBWC] True
[VERSION] True
--- [WORK LOGS] ---
20:11:57  [LOG] New Log Made [NAME] 06-11-2022-20-11-45.
20:11:57  [MAIN] If Chrome fails to open. Install the newest version of Chromedriver.
20:12:06  [MAIN] Chrome Version: 107.0.5304.88

inputted my own unique auth url in the code, logs in fine but doesn't do anything past that now. i suspect it may be because my school has a unique website URL with sparxmaths afterwards, but eh

how do you input it because i input my own URL in the code in notes and save it and it dont work

Posted a new commit, should fix issue.


Job detail view in dashboard

Allow the users to view all jobs. Allow user to click on job and view contract details

@portuu3 , @leric7 , @eugenvoronov , I think I will need more subgraphs and contract details for this issue.
cc @posix4e , @ansaqib


Automatic discovery of nodes

Hi,
First of all, thanks for this amazing project !
I was wondering if we could not discover nodes automatically using Phoenix Presence which also solves Service Discovery across nodes.

@tlvenn distribution has actually been removed in the 2.x branch because it was flakey. Can Phoenix Presence be using without depending on the entire Phoenix framework?

Yes, only need Phoenix.PubSub project: https://github.com/phoenixframework/phoenix_pubsub and it is rock solid.

Ha oky, I wasnt aware of that..

Could the distribution be added just using hooks with some kind of plugin system so that we could add cachex.distributed package to provide the feature without forking the whole project ?

@tlvenn it only happened recently, see discussion inside #49.
And per your suggestion, that is something I've tried to figure out. My intent for a while was to have interchangeable backends for a cache, so it could be ETS, could be Mnesia, could be Redis, etc.
This would work, but the big catch is that there are some features which simply do not work in a distributed state. It could be that this repo lives on as a distributed cache implementation and I fork out the non-distributed version into a new repo (which actually makes a fair bit of sense now I think about it).
How familiar are you with Presence? Do you know if it's possible to have a "master" node? This is the only thing I need to keep distribution - if there's a master node it's trivial to implement things that are currently local-only (i.e. LRW caches) because all the cleanup processes would happen on the master and just replicate out - as it stands there's no way to determine this and so it becomes near impossible to implement correctly.

@tlvenn FWIW I'm looking into Presence - doesn't seem like a good fit just because the sync interval is so high (default is recommended to be 1500ms)

@tlvenn I'm going to close this because of the distribution being stripped out. I'm planning on building up a new project aimed at being a distributed implementation in future so I'll drop a link here when that's up.
Cachex being distributed really doesn't make much sense due to the consistency issues as well as trying to be fast. My intent is to build something which is distributed at its core to improve the resiliency there.
If you have any other suggestions please feel free to let me know, but for now it looks like distribution is dying in the v1.x branch.

No problem @zackehh, thanks for the discussion.


Changed to allow use in Visual Studio 2019

According to this page...
https://madskristensen.net/blog/how-to-upgrade-extensions-to-support-visual-studio-2019/
...this is all that is required to make this plug-in work with VS2019.
I haven't been able to test it as it didn't compile probably due to me using VS2019, so sorry, I can't even give it a 'Works on my machine' sticker. I'm quite willing to test the .vsix if you send it to me, though.
On a separate topic - Thanks for creating this great plug-in. I use it all the time and it would be a shame for it to disappear.

I've published a new version. Please give it a try.


simplify threejs imports

This PR simplifies the "three" imports as discussed in #49.
Do not merge: Until Threestrap v0.5.1 is published. Oh, the CI tests should fail until then anyway.

Great work, I'll get all these reviewed and merged this morning. It was really nice to see the bundle size improvements!

As you say, it looks like the failures come from no threestrap 0.5.1 yet. I'm excited to get publishing automated here...
Thanks for undoing my unintentionally-destructive work here!!

(Merge away when threestrap's published)

@sritchie Thanks for the reviews! PR for the instanceof of change will be a bit later today, I hope.


the recipient to choose a wallet for the tokens to be issued in

can the recipient choose a wallet rather than the default to issue tokens to , when the issuer issue/ transfer tokens to this recipient ?

done


VarianceThresholdSelector

Not sure this is probatus-appropriate, but let me know @Matgrb
I already have the code for it.
Problem Description
The variance threshold from scikitlearn is great, but if we want to keep a pandas df, it require tweaking
Desired Outcome
VTS = VarianceThresholdSelector(threshold=0.0)
df = VTS.fit_transform(df)

We have discussed this one in #122 .
I think for now it is better to not add it to probatus, because of, large workload to add it, and low value added compared to sklearn.
Maybe it also fits better into sklego or some other package that implements sklearn transformers, i suggest making an issue about it there.


Add tracing to settings service.

This adds tracing to the settings service. It uses the otelchi package
and passes the tracing provider to the grpc and http servers.
Draft for now, as some bugs still need to be fixed.

:boom: Acceptance test localApiTests-apiAsyncUpload-ocis failed. Further test are cancelled...


Fix typo in blog post: “do to” -> “due to”

This PR corrects a minor typo in a blog post by replacing the phrase “do to” with “due to.” The change improves readability and accuracy. Please review and merge as appropriate.
For reference, the affected blog post is titled "Kubernetes 1.27: In-place Resource Resize for Kubernetes Pods (alpha)". You can find it here. 😊

Hi @bydeath, add a description for your PR, that will explain the reason to raise the PR and add link of the affected page.

/lgtm
This is a pretty trivial (but good!) update. If you see any other things that could be improved on this page, it would be great to include those in this PR. Thanks!

Hi @bydeath, first, welcome, and thanks for this PR. Two things to note here: while I am going to approve this, please be careful to avoid trivial edits in the future. Here are things to consider when making minor changes:

Can the file be improved further?
Does the small change greatly enhance the quality of the content?

The second thing to note is that we don't generally update blog posts that are over a year old.
/approve

Hi @bydeath, first, welcome, and thanks for this PR. Two things to note here: while I am going to approve this, please be careful to avoid trivial edits in the future. Here are things to consider when making minor changes:

Can the file be improved further?
Does the small change greatly enhance the quality of the content?

The second thing to note is that we don't generally update blog posts that are over a year old.
/approve

Hi @nate-double-u
Thank you for your feedback and for approving the PR. I appreciate your guidance on avoiding trivial edits in the future. I will certainly keep your points in mind and ensure that any minor changes I make will significantly enhance the quality of the content.
Regarding the update to the older blog post, I understand the policy and will avoid making updates to posts over a year old unless absolutely necessary.
Thanks again for your support and guidance.


Refactor the way TwitchChatMessageQueue handles emotes

It's a bit messy. We can definitely do better.
This would benefit alot from unit testing

Closed by #63


Update for operate-first timeout issue

This PR addresses the issues with operate-first taskrun timeouts

The taskrun (default to 1 hour) timeout error, is causing the build to fail on operate-first
This could be due to bandwidth throttling on the cluster itself.
I checked that all kvm-device-plugins were deployed to each node via the daemonset - 26 pods in total (I doubt that this would cause the issue)
The taskrun timeout setting can be updated in a configmap in the tekton-pipeline namespace - we don't have cluster-admin access so this cant be updated
I have added a 10G PV to allow for the tasks to be more granular
The tasks are

scos-cosa-copy
scos-cosa-init
scos-cosa-build
scos-cosa-buildextend
scos-cosa-test

I created a new pipelinerun task with the timeout override values
I have left the original pipeline and build task intact
We will need to do a deep dive into the root cause (as the full build takes about 20 minutes on my local kind cluster)

/assign @sherine-k
/assign @LorbusChris

@lmzuccarelli to me it looks like this would break running the pipeline locally, as it removes the kvm device plugin DaemonSet. Could we instead sync what's here with what's in apps and just exclude those resources from the operate-first env overlay? (we could also create another env overlay for plain/new OpenShift clusters that also includes them, like in the local env, so people won't have to dig into apps to deploy all of this)

Updated the timeout to 3hrs - we got a completed run - success
tkn pr describe scos-pipelinerun-all
I0831 14:47:32.142876 2784441 request.go:665] Waited for 1.198084847s due to client-side throttling, not priority and fairness, request: GET:https://api.smaug.na.operate-first.cloud:6443/apis/controlplane.operator.openshift.io/v1alpha1
Name:              scos-pipelinerun-all
Namespace:         okd-team
Pipeline Ref:      scos-all
Service Account:   pipeline
Labels:
 tekton.dev/pipeline=scos-all

🌡  Status

STARTED      DURATION   STATUS
1 hour ago   1h11m8s    Succeeded

⚓ Params

 NAME        VALUE
 ∙ version   4.12

📂 Workspaces

 NAME                 SUB PATH   WORKSPACE BINDING
 ∙ shared-workspace   ---        VolumeClaimTemplate

🗂  Taskruns

 NAME                                                 TASK NAME               STARTED          DURATION   STATUS
 ∙ scos-pipelinerun-all-scos-cosa-test-q6n98          scos-cosa-test          6 minutes ago    2m41s      Succeeded
 ∙ scos-pipelinerun-all-scos-cosa-buildextend-dztkt   scos-cosa-buildextend   33 minutes ago   27m3s      Succeeded
 ∙ scos-pipelinerun-all-scos-cosa-build-8q9dg         scos-cosa-build         1 hour ago       40m38s     Succeeded
 ∙ scos-pipelinerun-all-scos-cosa-init-twv9h          scos-cosa-init          1 hour ago       29s        Succeeded
 ∙ scos-pipelinerun-all-scos-cosa-copy-vtgrw          scos-cosa-copy          1 hour ago       17s        Succeeded

@lmzuccarelli to me it looks like this would break running the pipeline locally, as it removes the kvm device plugin DaemonSet. Could we instead sync what's here with what's in apps and just exclude those resources from the operate-first env overlay? (we could also create another env overlay for plain/new OpenShift clusters that also includes them, like in the local env, so people won't have to dig into apps to deploy all of this)

@LorbusChris
I have re-added the daemonset for local Kind clusters (my bad).
@sherine-k has retested using Kind locally with the updated changes (she reported that it too ka total of 14 minutes).
This means we need to ask for technical help regarding the kvm-device-plugin (myabe there are too many restrictions)

LGTM now, but should we remove the old combined task too?

@LorbusChris - Yes absolutely , I will update the PR and remove the old tasks.

We can remove the old task later, let's merge this


erchef should normalize cookbook data

See https://github.com/chef/chef-server/pull/714

For the record, it's cookbook_artifacts.