RefreshError with config.load_kube_config()

gcloud container clusters get-credentials my-cluster
Confirmed with gcloud container clusters get-credentials my-cluster, and ./kube/config was created.
Then in python:
from kubernetes import client, config
config.load_kube_config()

returns:
RefreshError: ('invalid_scope: Empty or missing scope not allowed.', u'{\n  "error" : "invalid_scope",\n  "error_description" : "Empty or missing scope not allowed."\n}')

What did I miss?

stack trace:
---------------------------------------------------------------------------
RefreshError                              Traceback (most recent call last)
<ipython-input-1-267adcf41323> in <module>()
      1 from kubernetes import client, config
----> 2 config.load_kube_config()

/Users/yuhanliu/.virtualenvs/hpsearch/lib/python2.7/site-packages/kubernetes/config/kube_config.pyc in load_kube_config(config_file, context, client_configuration, persist_config)
    359         config_file, active_context=context,
    360         client_configuration=client_configuration,
--> 361         config_persister=config_persister).load_and_set()
    362
    363

/Users/yuhanliu/.virtualenvs/hpsearch/lib/python2.7/site-packages/kubernetes/config/kube_config.pyc in load_and_set(self)
    251
    252     def load_and_set(self):
--> 253         self._load_authentication()
    254         self._load_cluster_info()
    255         self._set_config()

/Users/yuhanliu/.virtualenvs/hpsearch/lib/python2.7/site-packages/kubernetes/config/kube_config.pyc in _load_authentication(self)
    174         if not self._user:
    175             return
--> 176         if self._load_gcp_token():
    177             return
    178         if self._load_user_token():

/Users/yuhanliu/.virtualenvs/hpsearch/lib/python2.7/site-packages/kubernetes/config/kube_config.pyc in _load_gcp_token(self)
    194                  _is_expired(provider['config']['expiry']))):
    195             # token is not available or expired, refresh it
--> 196             self._refresh_gcp_token()
    197
    198         self.token = "Bearer %s" % provider['config']['access-token']

/Users/yuhanliu/.virtualenvs/hpsearch/lib/python2.7/site-packages/kubernetes/config/kube_config.pyc in _refresh_gcp_token(self)
    203             self._user['auth-provider'].value['config'] = {}
    204         provider = self._user['auth-provider']['config']
--> 205         credentials = self._get_google_credentials()
    206         provider.value['access-token'] = credentials.token
    207         provider.value['expiry'] = format_rfc3339(credentials.expiry)

/Users/yuhanliu/.virtualenvs/hpsearch/lib/python2.7/site-packages/kubernetes/config/kube_config.pyc in _refresh_credentials()
    133             credentials, project_id = google.auth.default()
    134             request = google.auth.transport.requests.Request()
--> 135             credentials.refresh(request)
    136             return credentials
    137

/Users/yuhanliu/.virtualenvs/hpsearch/lib/python2.7/site-packages/google/oauth2/service_account.pyc in refresh(self, request)
    308         assertion = self._make_authorization_grant_assertion()
    309         access_token, expiry, _ = _client.jwt_grant(
--> 310             request, self._token_uri, assertion)
    311         self.token = access_token
    312         self.expiry = expiry

/Users/yuhanliu/.virtualenvs/hpsearch/lib/python2.7/site-packages/google/oauth2/_client.pyc in jwt_grant(request, token_uri, assertion)
    141     }
    142
--> 143     response_data = _token_endpoint_request(request, token_uri, body)
    144
    145     try:

/Users/yuhanliu/.virtualenvs/hpsearch/lib/python2.7/site-packages/google/oauth2/_client.pyc in _token_endpoint_request(request, token_uri, body)
    107
    108     if response.status != http_client.OK:
--> 109         _handle_error_response(response_body)
    110
    111     response_data = json.loads(response_body)

/Users/yuhanliu/.virtualenvs/hpsearch/lib/python2.7/site-packages/google/oauth2/_client.pyc in _handle_error_response(response_body)
     57
     58     raise exceptions.RefreshError(
---> 59         error_details, response_body)
     60
     61

RefreshError: ('invalid_scope: Empty or missing scope not allowed.', u'{\n  "error" : "invalid_scope",\n  "error_description" : "Empty or missing scope not allowed."\n}')

Can you provide these information:
1- gcloud version
2- keep the content of ~/.kube/config when the python client is not working
3- run kubectl (it should work?)
4- run the client to see if it is working
5- see the changes in ~/.kube/config
If kubeconfig succeeded, most probably we need to change our code to run the command line code gcloud provides instead of using google credentials api.

1- with gcloud -v:
Google Cloud SDK 171.0.0
alpha 2017.03.24
beta 2017.03.24
bq 2.0.25
core 2017.09.11
datalab 20170818
gcloud
gsutil 4.27
kubectl

2- removed current ~/.kube/config (since I had already shut down all clusters from GCP console anyways), then ran gcloud container clusters create test-cluster.  Resulting ~/.kube/config:
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: [redacted]
    server: https://[redacted]
  name: [redacted]
contexts:
- context:
    cluster: [redacted]
    user: [redacted]
  name: [redacted]
current-context: [redacted]
kind: Config
preferences: {}
users:
- name: [redacted]
  user:
    auth-provider:
      config:
        cmd-args: config config-helper --format=json
        cmd-path: [redacted]
        expiry-key: '{.credential.token_expiry}'
        token-key: '{.credential.access_token}'
      name: gcp

And indeed config.load_kube_config() raised the same error as I reported above.
3- ran kubectl get nodes and it worked.  Now ~/.kube/config became:
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: [redacted]
    server: https://[redacted]
  name: [redacted]
contexts:
- context:
    cluster: [redacted]
    user: [redacted]
  name: [redacted]
current-context: [redacted]
kind: Config
preferences: {}
users:
- name: [redacted]
  user:
    auth-provider:
      config:
        access-token: [redacted]
        cmd-args: config config-helper --format=json
        cmd-path: [redacted]
        expiry: 2017-09-20T02:38:48Z
        expiry-key: '{.credential.token_expiry}'
        token-key: '{.credential.access_token}'
      name: gcp

Note that the access-token and expiry fields were added.
4- python client is working.
5- the only difference in the two config files are indeed those two fields access-token and expiry.
Thanks!

It is strange that I did the same steps, even created a new python environment and installed latest version of kubernetes package but couldn't reproduce this. first run of the client adds access-token and expiry for me. Can you do the same to make sure we have the same environment? This is the chain of commands I ran:
virtualenv /tmp
source /tmp/bin/activate
pip install kubernetes
rm ~/.kube/config
gcloud container clusters create test-cluster
cp ~/.kube/config ~/.kube/config.before
python example1.py
diff ~/.kube/config.before ~/.kube/config

if you get different results, please also include all pypi package versions so we can compare.

When creating a new virtualenv to follow your steps I remembered that I had previously installed python 2.7.13 with homebrew to resolve an issue with OpenSSL causing the kubernetes client not being able to deploy containers to clusters with macOS's stock python 2.7.10.
Looking in brew info python it suggested adding the following to my PATH, which I did not do:
export PATH="/usr/local/opt/python/libexec/bin:$PATH"

So I added it to my .zshrc, and everything works now (config.load_kube_config() correctly connected to the cluster and updated ~/.kube/config).  So perhaps an incorrect version of python was used when trying to connect to the cluster without the above export, even when the virtualenv points to the correct python version?
Anyways now I can continue

Nice. You are welcome.

Running into the same issue again even with the setup that used to work (as far as I remember).  Below is the print out from pip freeze, in case it helps:
appnope==0.1.0
backports-abc==0.5
backports.shutil-get-terminal-size==1.0.0
beautifulsoup4==4.6.0
bleach==2.0.0
bs4==0.0.1
cachetools==2.0.1
certifi==2<IP_ADDRESS>
chardet==3.0.4
configparser==3.5.0
cycler==0.10.0
decorator==4.1.2
entrypoints==0.2.3
enum34==1.1.6
functools32==3.2.3.post2
futures==3.1.1
google-api-python-client==1.6.3
google-auth==1.1.1
google-cloud-core==0.27.1
google-cloud-storage==1.4.0
google-resumable-media==0.2.3
googleapis-common-protos==1.5.2
html5lib==0.999999999
httplib2==0.10.3
idna==2.6
ipaddress==1.0.18
ipykernel==4.6.1
ipython==5.5.0
ipython-genutils==0.2.0
ipywidgets==7.0.1
Jinja2==2.9.6
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.1.0
jupyter-console==5.2.0
jupyter-core==4.3.0
kubernetes==3.0.0
MarkupSafe==1.0
matplotlib==2.0.2
mistune==0.7.4
nbconvert==5.3.1
nbformat==4.4.0
notebook==5.1.0
numpy==1.13.1
oauth2client==4.1.2
olefile==0.44
pandocfilters==1.4.2
pathlib2==2.3.0
pexpect==4.2.1
pickleshare==0.7.4
Pillow==4.2.1
prompt-toolkit==1.0.15
protobuf==3.4.0
ptyprocess==0.5.2
pyasn1==0.3.6
pyasn1-modules==0.1.4
PyGithub==1.35
Pygments==2.2.0
PyJWT==1.5.3
pyparsing==2.2.0
python-dateutil==2.6.1
pytz==2017.2
PyYAML==3.12
pyzmq==16.0.2
qtconsole==4.3.1
requests==2.18.4
rsa==3.4.2
scandir==1.5
scikit-learn==0.19.0
scipy==0.19.1
simplegeneric==0.8.1
singledispatch==<IP_ADDRESS>
six==1.11.0
sklearn==0.0
subprocess32==3.2.7
termcolor==1.1.0
terminado==0.6
testpath==0.3.1
tornado==4.5.2
traitlets==4.3.2
uritemplate==3.0.0
urllib3==1.22
wcwidth==0.1.7
webencodings==0.5.1
websocket-client==0.40.0
widgetsnbextension==3.0.3

I hit the same issue when using a service account (by setting environment variable GOOGLE_APPLICATION_CREDENTIALS).
I suspect this is an issue with google-auth-library-python and not the Kubernetes Python Client library

I opened up GoogleCloudPlatform/google-auth-library-python#217

The response to GoogleCloudPlatform/google-auth-library-python#217 was that this is working as intended.
I suspect https://www.googleapis.com/auth/userinfo.email is needed in order to obtain an identity token to use to authenticate to the K8s master. I wouldn't expect the Kubernetes client talks to any other GCP services so userinfo.email might be sufficient.
I was able to work around this by piping in my own credential refresher so I could set the scopes see here.
Some options for better supporting this would be

Modifying kube_config.py to always set the scope e.g.
 def _refresh_credentials():
       credentials, project_id = google.auth.default(scopes=["https://www.googleapis.com/auth/userinfo.email"])
       request = google.auth.transport.requests.Request()
       credentials.refresh(request)
       return credentials

Modify load_kube_config so it passes through unknown keyword args to _get_kube_config_loader_for_yaml_file

In combination with the previous option, add an optional scopes argument to KubeConfigLoader and use it in _refresh_credentials

Hm I seem to be running in to this same issue in https://github.com/kubernetes-incubator/client-python/issues/386. I agree that the google auth side is working as expected; isn't this a bug with the kubernetes python client?

does running the command line tool in .kube/config file refresh the right way? What kubectl is doing is running the command specified in .kube/config. If kubectl is working for you, then the command line tool should work, can you guys test this?

I'm running into what looks like the same bug, and it's reproducible here. Starting with no kubeconfig, I run gcloud ... get-credentials. When I try to access it using the Python API, I get a RefreshError exception. But it I run a kubectl command (e.g. get pods), it works fine and then the Python APi works fine too. And looking at the kubeconfig, the access-token and expiry fields were added.
I can help debug this. Let me know if you prefer to reopen this, or if I should file another bug report. Thanks!

I also faced with a problem with refreshing token. Python client reads token from  ~./kube/config, if it is already expired, creates new token and sends next request with a new one. It looks like a usual flow but the issue is that kubectl config remains the same (with old expired token) and API responds me with 403 Forbidden error since I come with unknown for him token.
The only way I can refresh token in ~./kube/config is to execute any kubectl command from console (e.g. kubectl get namespaces). After this token refreshes and API responds 200 OK. Does anybody know how to fix it?


Security vulnerability in<EMAIL_ADDRESS>

Original vulnerability that affected react-pdf: https://github.com/advisories/GHSA-wgrm-67xf-hhpq
react-pdf fix: https://github.com/advisories/GHSA-87hq-q4gp-9wr4
Updating to<EMAIL_ADDRESS>will fix the issue.
Thanks!

@Ryanv030 Try to use the latest version: https://github.com/cyntler/react-doc-viewer/releases/tag/v1.15.0. I updated the version.

Looks like that worked for fixing the security vulnerability, unfortunately we're still going to be getting flagged even though it's technically fixed.
The maintainer of react-pdf made a comment about his plans on fixing this in the future. (just in case you get more issues about this)

react-pdf is updated. Can it be upgraded so it won't get flagged?

https://github.com/wojtekmaj/react-pdf/releases


Add deployment for minikube

Adds deployment for use in development environment
with support for indexing  and playback of local files

This work is moved to our internal deployment repository.


sar_tas_server "socket bind failed" and variable resetting

I recently got the error "Could not initialize TAS server: socket bind failed" when starting. Although it resolved itself after disabling and reenabling sar_tas_server, the variable being set to one caused a bit of confusion.

TAS server has been refactored, try now (sar_tas_protocol_server)

Sorry, I completely forgot what this was about. Feel free to close.
I never ran into a issue like I described again so I won't bother learning how to intentionally cause a socket bind to fail. It's faster for us both if you mock this in your code if you really want to know.


Configuration

Good morning. I can't configure the integration. I enter the IP address of my Ecu C: <IP_ADDRESS>, what is the password that I must enter, that of my SSID (Box) or the password of the EMA application on the net? I tried both but it doesn't work. Thanks for your help

Try without any password, I think that was the case for me (not sure but always worth a try !)

Hi Alain, I added some additional info to the configure instructions: The optional fields [Specify SSID] and [Specify password] are the parameters you would like to assign to the ECU when an automatic software restart is being applied (for ECU-R (sunspec) and ECU-C models only). These parameters are not being used during setup. If setup fails, in most cases the wrong IP-address was specified.
The software reset is based on changing the configuration which demands a software reset from the ECU. You can specify the initial parameters for your situation.
For your ECU-C make sure you specify the IP-Address assigned to the wired ethernet connection.

hello thank you for your feedback. I will connect the ECU wired to see....🤞🏼

If you own an ECU-C you should indeed use the wired connection see: https://github.com/ksheumaker/homeassistant-apsystems_ecur?tab=readme-ov-file#prerequisites
The Wifi connected ECU does not expose port 8899

If you own an ECU-C you should indeed use the wired connection see: https://github.com/ksheumaker/homeassistant-apsystems_ecur?tab=readme-ov-file#prerequisites The Wifi connected ECU does not expose port 8899 required for this integration to work.

Hey HAEdwin,
is there a way to have this work on a ECU-R but with a wire connection? I have it configure like that because the location where I have the ECU does not have wireless access, and so its impossible for me to access it via wireless, its just a different port for wire? or is is more complex than that?
Thanks !

No there's not unless a new firmware update will support that.


Check Health Deprecation

When running Check Heath:

Thanks for the report, I'll update that in the next release


KAFKA-6515 Adding toString() method to o.a.k.connect.data.Field

https://issues.apache.org/jira/browse/KAFKA-6515
Committer Checklist (excluded from commit message)

[x] Verify design and implementation
[x] Verify test coverage and CI build status
[x] Verify documentation (including upgrade notes)

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-test-coverage/267/

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/kafka-pr-test-coverage/268/

Hi @rhauch, @ewencp, feedback/merge welcome on that tiny improvement whose issue I spotted in JIRA and which also had been bugging me. Thanks!


Improve client handling around director outages

A client unlucky enough to start while the director is restarted will get either a 404 or a 500 from the Traefik ingress (I think it depends slightly on where things are in the restart process).
This is a fatal outage; here's a few example hold messages generated:
Transfer input files failure at execution point<EMAIL_ADDRESS>using protocol osdf. \
Details: Error from<EMAIL_ADDRESS>Failed to transfer files: \
FILETRANSFER:1:non-zero exit (1) from /usr/libexec/condor/stash_plugin. |\
Error: failed to get namespace information for remote URL osdf:///chtc/staging/blah/container.sif: \
Could not unmarshall the director's response: invalid character 'B' looking for beginning of value \
( URL file = osdf:///chtc/staging/blah/container.sif )|

Transfer input files failure at execution point<EMAIL_ADDRESS>using protocol osdf. \
Details: Error from<EMAIL_ADDRESS>Failed to transfer files: \
FILETRANSFER:1:non-zero exit (1) from /usr/libexec/condor/stash_plugin. |\
Error: failed to get namespace information for remote URL osdf:///chtc/staging/blah/container.sif: 404: 404 page not found\n \
( URL file = osdf:///chtc/staging/blah/container.sif )|

A few thoughts:

There's an obvious bug in that we try to unmarshal a JSON struct out of the body for a non-JSON response (I think the first response is actually plain-text from Traefik).
If we differentiate a response from the director versus Traefik, we could retry.  We should add a Server header from the director to detect this situation.
For error responses from Traefik -- or, in general, for network connectivity issues -- we could consider not only retrying in general but retrying more aggressively when in "plugin" mode.  While I might want a CLI to give up after 5 seconds, I can probably spend 30 seconds of retrying in plugin mode.

@bbockelm, it looks like you started working on this in #1565. Do you intend for that PR to close this issue?

I encountered this bug (or version of it) this afternoon, around 1:45 PM CDT, directly using the pelican binary in Ubuntu 22.04 (on WSL, Windows 11).
Version: 7.8.6
Build Date: 2024-06-06T21:27:21Z
Build Commit: f911e9940cc4ba00827475dbd4a33545bd927554
Built By: goreleaser

Managed to catch the error with debug enabled:
$ ./pelican -d object get pelican://osg-htc.org/ospool/uc-shared/public/OSG-Staff/validati
on/test.txt downloaded-test.txt
WARNING[2024-10-22T13:47:18-05:00] Debug is set as a flag or in config, this will override anything set for Logging.Level within your configuration
DEBUG[2024-10-22T13:47:18-05:00] Launch progress bars display
DEBUG[2024-10-22T13:47:18-05:00] Len of source: 2
DEBUG[2024-10-22T13:47:18-05:00] Sources: [pelican://osg-htc.org/ospool/uc-shared/public/OSG-Staff/validation/test.txt]
DEBUG[2024-10-22T13:47:18-05:00] Destination: downloaded-test.txt
DEBUG[2024-10-22T13:47:18-05:00] Making new clients
DEBUG[2024-10-22T13:47:18-05:00] Created new client 0192b58e-0daf-7aa0-a498-a4ec5a9973de
DEBUG[2024-10-22T13:47:18-05:00] Detected pelican:// url, getting federation metadata from specified host osg-htc.org

DEBUG[2024-10-22T13:47:18-05:00] Performing federation service discovery for specified url against endpoint https://osg-htc.org
DEBUG[2024-10-22T13:47:18-05:00] Federation service discovery resulted in director URL https://osdf-director.osg-htc.org
DEBUG[2024-10-22T13:47:18-05:00] Federation service discovery resulted in registry URL https://osdf-registry.osg-htc.org
DEBUG[2024-10-22T13:47:18-05:00] Federation service discovery resulted in JWKS URL https://osg-htc.org/osdf/public_signing_key.jwks
DEBUG[2024-10-22T13:47:18-05:00] Federation service discovery resulted in broker URL
DEBUG[2024-10-22T13:47:18-05:00] Will query director at https://osdf-director.osg-htc.org for object /ospool/uc-shared/public/OSG-Staff/validation/test.txt
DEBUG[2024-10-22T13:47:18-05:00] Director's response: &{301 Moved Permanently 301 HTTP/1.1 1 1 map[Alt-Svc:[h3=":443"; ma=86400] Cf-Cache-Status:[DYNAMIC] Cf-Ray:[8d6ba7b75a4c61b1-ORD] Connection:[keep-alive] Content-Length:[17] Content-Type:[text/plain; charset=utf-8] Date:[Tue, 22 Oct 2024 18:47:19 GMT] Location:[https://osdf-director.osg-htc.org/ospool/uc-shared/public/OSG-Staff/validation/test.txt] Nel:[{"success_fraction":0,"report_to":"cf-nel","max_age":604800}] Report-To:[{"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=0FEEo1BzuV1dLAQwYiPjB3DWJ4%2FdYXERSYhM3446j0Fp4IGX6dsN1SfOzaFrHDV9nFdnqF9ZtEpgGDWbQFlwwken6NUuuRT6qqTPcfV7ThYyZJ5uWK53SWgvwBFZOazLABJTm7yNYjXbkd0Y"}],"group":"cf-nel","max_age":604800}] Server:[cloudflare] Server-Timing:[cfL4;desc="?proto=TCP&rtt=9006&sent=5&recv=7&lost=0&retrans=0&sent_bytes=3098&recv_bytes=524&delivery_rate=445156&cwnd=231&unsent_bytes=0&cid=c44fddeab861b0a0&ts=85&x=0"] Strict-Transport-Security:[max-age=2592000]] 0xc0009180c0 17 [] false false map[] 0xc000626000 0xc000204d10}
ERROR[2024-10-22T13:47:18-05:00] Error while querying the Director: Could not unmarshall the director's response: invalid character 'M' looking for beginning of value
ERROR[2024-10-22T13:47:18-05:00] Could not unmarshall the director's response: invalid character 'M' looking for beginning of value
DEBUG[2024-10-22T13:47:18-05:00] Shutting down transfer engine
DEBUG[2024-10-22T13:47:18-05:00] Job handler has been shutdown
ERROR[2024-10-22T13:47:18-05:00] Failure getting pelican://osg-htc.org/ospool/uc-shared/public/OSG-Staff/validation/test.txt: failed to get namespace information for remote URL pelican://osg-htc.org/ospool/uc-shared/public/OSG-Staff/validation/test.txt: Could not unmarshall the director's response: invalid character 'M' looking for beginning of value

Oddly, if I typed the command manually, I did not receive an error. As demonstrated live to @turetske , @jhiemstrawisc , if I copied the command from the website (using the "Copy" button in the code block, or highlighting then Ctrl+C) it produced the above error. As such, I initially thought it was an issue with the website.
Later attempts to reproduce the error worked as expected; presumably the OSDF director was back online. I did change my wireless network in between tests; perhaps the first network was doing something funky?

@aowen-uwmad - that's a puzzler.
The headers indicate you're talking to a HTTP server that's proxied through CloudFlare.  Needless to say -- that's not where our director lives!
Is there anything special about your laptop's networking setup that might be relevant?
Next time this occurs, could you also run dig -t AAAA osdf-director.osg-htc.org and dig osdf-director.osg-htc.org to see if you're picking up a funky DNS resolution?
Another idea - could you and @williamnswanson try reproducing it for the ITB director?

My laptop setup has a history of funky networking things, but it hasn't acted up in quite a while.
I can try to replicate it this afternoon.


Add support for expressions in the inline chat commands

Adds support  for expressions, property referencing and function invocation in the inline chat commands such as:

DMG
HEAL,
GAIN,
LOSS

It adds support for the following:

@actor.level.value: Reference item on an actor's data model
@item.level.value: Reference item on an item's data model
$actor.byLevel(40,50,60) : Invoking function to pick the parameter that matches the given level of the actor. Will come in handy for scaling spells.
minor|heavy|massive effect

NOTE: There's still a few issues I need to fix, but the overall API will remain the same if you want to start reviewing.

Implementing this specifically for inline effects is fine for now, but I think eventually being able to handle dynamic expressions in other places will be beneficial and having a standardized DSL for the entire project would help.
So a utility module/class for dynamic evaluation that takes a string to parse and evaluate against a given context. And that way consumers can simply call an exposed interface call like evaluate(value: string, context: object)
That way we can refence this common module for how the DSL works instead of worrying about specific contexts in which evaluation happens.

Implementing this specifically for inline effects is fine for now, but I think eventually being able to handle dynamic expressions in other places will be beneficial and having a standardized DSL for the entire project would help.
So a utility module/class for dynamic evaluation that takes a string to parse and evaluate against a given context. And that way consumers can simply call an exposed interface call like evaluate(value: string, context: object)
That way we can refence this common module for how the DSL works instead of worrying about specific contexts in which evaluation happens.

Oh yeah, I definitely agree with providing a standard API for handling expression evaluation for the DSL in the system. I will go ahead and factor that out into its own class.

It's been factored out and a bit simpler to reason about now that it's on its own module.

One last question for the time being is to decide whether to use a different symbol for executing functions or use the same one as property referencing @. If we use different ones then we can invoke functions from referenced properties without having to deal with scoping issues.
It may also be good to think of a symbol for short-hand syntactic sugar for things such as item level, character level, target count.

After the latest round of changes I believe it's in a good state. I will await your review.


Added capability to use this library with a Wire() predefined by the …

…user
Added capability to use a predefined Wire Reference from the containing environment

We apologize for not responding on time. However, please check-out our new ENS16x library which solves this issue: https://github.com/sciosense/ens16x-arduino.


Separate text color for view and dialog

When you set app:ccp_textColor property to white, you will have white labels on white background in dialog. Any workarounds?

Currently, there is no workarouds. But I can add an attribute to set the color of the dialog. Something like app:ccp_textColorDialog. How do you think about it?
Any suggestion?

Yeah it would be great. Thanks!

I've added ccp_dialogTextColor to change dialog text color. Please use 2.1.5 version.

I'll close this issue for now, please reopen if the problem still persist.


Fix locating project output paths

Fixes output path determination for projects. Before we would look for at a project for <OutputFileName> and stop if that property could not be determined.
Supports .NET Core projects now. (Need to verify for project.json projects in VS2015.)

Caveat:  Released version Microsoft.Cci does not support portable PDB file structure, so it throws an OOM. As a result, I had to add this output window as a workaround... for now.  The repository has support for portable PDB, but I have no idea where the latest builds are. There is an issue here to get the builds: Microsoft/cci#46.

Supports showing the proper projects for source line information. (Before, it would show the first project for all the source line information.)

Fixes UI hangs due to async/await.  As a result, I used the ThreadHelper from this blog and ConfigureAwait(false): https://blogs.msdn.microsoft.com/andrewarnottms/2014/05/07/asynchronous-and-multithreaded-programming-within-vs-using-the-joinabletaskfactory/

Possibly fixes #429
Fixes #397, #390
/cc: @twsouthwick

LGTM

@twsouthwick Should I merge this PR? I know it'll break VS2015 but will work for VS2017. :(


format code, optimize ocr,optimize song detail level

format code
[recognizeTextHandler] method should be placed in global-threads.
reduce some unnecessary operations.
resolve the circular reference issue.

lgtm. Thanks for your contribution!


add proper exit codes

host and datastore list commands are missing proper exit codes when failing to find the pool.

Thanks! It's the finer details like this that improve people's work.


regional activities overview (restore old portal functionality)

from http://dev.gbif.org/issues/browse/POR-1812?jql=project %3D POR AND status %3D Open
Restoring old portal functionality, regions
http://www-old.gbif.org/participation/participant-nodes/nodes-committee/regional-activities/
key elements
upcoming regional meetings
past meeting events and reports
"sense of regular regional activities", e.g. annual regional meeting
importance:
continuing growth and importance of regional activities need to be reflected in the portal, as this development is to be encouraged and supported
give credit to i.a. hosts for the contribution
To do: decide where a page like this could be placed. Link from http://www.gbif.org/capacityenhancement/regionalcollaboration?
This links to other issues around restoring OCR searches, filters and navigation. How to display documents and information about past events? Send to saved search, or enter one by one? Needs to be consistent with other areas of the portal. This should echo the layout decision for http://www.gbif.org/governance/governingboard#meetings
@ahahn-gbif I see this idea is from 2014 - is it still a priority?

To be discussed further with the Participation group, but yes - regional pages are still needed. Minimum level: a list of GBIF regions, each with its membership (countries and international organizations) and regional representatives (contact persons), as well as an option for regions to advertize / link to ongoing/upcoming activities and meetings.

also see issue #293


OtherProperties Ability in Custom JSON

Need way to efficiently define other properties table for each of the viewing interfaces

Exclude those terms in getMetadata and getDetails

select ?p ?o
where {
https://synbiohub.org/public/igem/BBa_F2620/1
?p
?o
Filter(!strstarts(str(?p),'http://sbols.org/') && !strstarts(str(?p),'http://www.w3.org/ns/prov#') &&
?p != http://purl.org/dc/terms/title && ?p != http://www.w3.org/1999/02/22-rdf-syntax-ns#type)
}

closed with #554


Rename file keyboard shortcut not working: no when clauses matched the context.

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.76.2 ee2b180d582a7f601fa6ecfdad8d9fd269ab1884
OS Version: x64

Steps to Reproduce:

Highlight a file in explorer
Hit Enter (keyboard shortcut to rename file)

Output when Developer: Toggle Keyboard Shortcuts Troubleshooting is on:
2023-03-23 12:41:03.320 [info] [KeybindingService]: / Soft dispatching keyboard event
2023-03-23 12:41:03.321 [info] [KeybindingService]: | Resolving [Enter]
2023-03-23 12:41:03.321 [info] [KeybindingService]: \ From 24 keybinding entries, no when clauses matched the context.
2023-03-23 12:41:03.321 [info] [KeybindingService]: / Received  keydown event - modifiers: [], code: Enter, keyCode: 13, key: Enter
2023-03-23 12:41:03.321 [info] [KeybindingService]: | Converted keydown event - modifiers: [], code: Enter, keyCode: 3 ('Enter')
2023-03-23 12:41:03.321 [info] [KeybindingService]: | Resolving [Enter]
2023-03-23 12:41:03.321 [info] [KeybindingService]: \ From 24 keybinding entries, no when clauses matched the context.

Keyboard binding setting:
{
  "key": "enter",
  "command": "renameFile",
  "when": "explorerViewletVisible && filesExplorerFocus && !explorerResourceIsRoot && !explorerResourceReadonly && !inputFocus"
}

This should be fixed in 1.78 which comes out mid next week. If you cannot wait CMD + SHIFT + P -> Reset View Locations should fix this as well.

Thanks @lramos15 I managed to fix it in the end by re-installing VSCode which reset my Explorer view, which had somehow got itself into a configuration that only displayed "folders"
Therefore vscode couldn't find "Explorer" and so the "explorerViewletVisible" condition failed, which I think is what caused this bug.
I'm guessing "Reset View Locations" would have acheived the same thing as re-installing VSCode. I searched around for a setting to do this, wish I'd found it!


Understanding Slack Notification Setup

I have tried a couple of different inputs for variable slack-auth and enabled the Notification in the configurations but unable to get any notifications sent over.

Take a look here and here


Matching Patterns for Any Allocation of tags, etc. cannot be edited

On swift-paperless version 1.0.1, the editing of matching patterns for non-automatic matching algorithms is not possible. The creation of new, e.g. document types, with a custom (any) matching pattern is however possible.
Steps to reproduce:

on the webserver create a new document type with allocation algorithm Any and set the matching pattern to any word
go to the app and try to change the matching pattern words
matching pattern words will not show up

Info: I have only tried this for document types with the matching algorithm any. But I assume that this is a generic implementation and thus applies to other allocation algorithms as well.
Current Workaround:
To get the input field for a custom matching pattern, you have to set the algorithm to something else, then change it back to your desired matching algorithm. Then the matching pattern input field will show up and contain your previously set words.
Editing not possible:

Creation of a new document type with matching Any is working with match input:

I can reproduce this. I think it's just the initial view that's borked somehow. I'll try to add a fix soon.

There should be a fix in build 32 on TestFlight. Could you try if that fixes things in your end?

It works and all changes are successfully served to the backend. Thank you for the fix. When will the update be realesed for the production version of the app?


Publish web3Signer package

Background
Need to publish the web3signer package for consumption by wonderland watcher implementation.
Definition Of Done
web3signer package is available on npm

@preethamr is reviewing PR and will close if htis is already taken care of

So we haven't released the web3signer package. And we don't need it anymore

Still using duplicate code in the watcher that would be better to be abstracted as a package. Working, but any changes to the Web3Signer package logic will have to be manually updated on watcher (read: they will not be propagated to the watcher)

This is still needed and not done.
Currently watcher is duplicating the code needed to resolve this dependency.
Reopening this, so that we resolve the dependency the right way.


Be able to use a single Label

I'm doing extractive text summarization.
So given a text, I want to label the salient sentences.
I tried :
<Labels name="ext" toName="text">
   <Label value="Salient sentence" />
</Labels>

But it yields the error :
Validation Failed On View/Labels: OrderedDict([('Value', 'Salient sentence')]) Is Not Of Type 'Array'

I think it should be possible to have a single label.

We are working on this issue, I think we will include the fix in release/0.7.4.

@Colanim I fixed this issue in https://github.com/heartexlabs/label-studio/pull/350. Please, try it if you can.


Add new members

Hey all — this adds:

@jasisk
@rdodev

I'm not on the list before or after the PR, not sure if it matters or anyone cares though :)

I'm not either :]

Ooh, sorry about that, I'll add you both!

Updated!

LGTM


See if sources have downloadable data

Use this rather than hitting API endpoints if possible

[ ] cBioPortal
[ ] gnomAD

gnomAD downloadable data is huge and their python API requires spinning up clusters which we want to try and avoid at the moment. We may come back to this gnomAD issue in the future.


Use newer TF resources for S3 buckets, rather than the now-deprecated resources

Refs: #662
Refs: This warning on a terraform plan, which reads:

Use the aws_s3_bucket_server_side_encryption_configuration resource instead
(and 25 more similar warnings elsewhere)

If the manual process seems daunting, consider using tfedit, as suggested in the discussion here.

Closing for now as our remaining usages are in module dependencies, which have to be handled upstream. Dependabot should then take care of them.


[3.0] Remove suffix from Cosmos Metadata extensions

Description
We missed some method renames when doing API review of the Cosmos provider. We are asking to do these renames now before we ship the first GA release of the Cosmos provider. (It was only in preview for 2.x.)
We could instead obsolete the existing methods so as not to break from the last preview, but that would mean introducing new methods in a GA release that are already marked as obsolete when they ship.
Customer Impact
Current APIs are inconsistent with the pattern we use elsewhere; this change brings them into line.
Customers coming from a previous preview could be broken by this, but the fix is easy, and Cosmos usage is low.
How found
Found while doing exploratory testing/prep for 3.1.
Test coverage
N/A
Regression?
No
Risk
Risk is close to zero from a GA perspective, but it could break a small number of customers moving from previews to GA.

These methods are new in 3.0, renaming them afterwards would be a breaking change.
Fixes #17534

@AndriySvyryd This is approved by Tactics; you may merge.


Ungraceful termination leaves turbo meta files around

What version of Turborepo are you using?
1.2.9
What package manager are you using / does the bug impact?
Yarn v1
What operating system are you using?
Mac
Describe the Bug
When running a task and then terminating that task via Ctrl-C - depending on the input complexity of that task - termination can leave repository in a bad state with temp files around.
drwxr-xr-x    - sachin.patel 19 May 12:24 0c44f2e517f3b41f/
.rwxrwxr-x   44 sachin.patel 19 May 12:24 0c44f2e517f3b41f-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 0ca02e8cfde14602/
.rwxrwxr-x   44 sachin.patel 19 May 12:24 0ca02e8cfde14602-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 0e6e43c97fbf832d/
.rwxrwxr-x   44 sachin.patel 19 May 12:24 0e6e43c97fbf832d-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 0fe27e653c076444/
.rwxrwxr-x   44 sachin.patel 19 May 12:24 0fe27e653c076444-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 0205d9fe6f0d6c23/
.rwxrwxr-x   44 sachin.patel 19 May 12:24 0205d9fe6f0d6c23-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 021458ea25e1e1cb/
.rwxrwxr-x   44 sachin.patel 19 May 12:24 021458ea25e1e1cb-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 03aa2cc194b2abfa/
.rwxrwxr-x   44 sachin.patel 19 May 12:24 03aa2cc194b2abfa-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 04093098fc837bc0/
.rwxrwxr-x   44 sachin.patel 19 May 12:24 04093098fc837bc0-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 04764d4fe1e86dea/
.rwxrwxr-x   44 sachin.patel 19 May 12:24 04764d4fe1e86dea-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 06ac4cf9c017ddf8/
.rwxrwxr-x   43 sachin.patel 19 May 12:24 06ac4cf9c017ddf8-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 06d464e9998db120/
.rwxrwxr-x   44 sachin.patel 19 May 12:24 06d464e9998db120-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 07f0d46fd16d7874/
.rwxrwxr-x   44 sachin.patel 19 May 12:24 07f0d46fd16d7874-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 09fc28c94b0d1ba1/
.rwxrwxr-x   44 sachin.patel 19 May 12:24 09fc28c94b0d1ba1-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 0903eebd4e79df9a/
.rwxrwxr-x   44 sachin.patel 19 May 12:24 0903eebd4e79df9a-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 094540b882a45b28/
.rwxrwxr-x   45 sachin.patel 19 May 12:24 094540b882a45b28-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 1d5f655502bceea4/
.rwxrwxr-x   44 sachin.patel 19 May 12:24 1d5f655502bceea4-meta.json*
drwxr-xr-x    - sachin.patel 19 May 12:24 1dfd23a9c6016c61/
.rwxrwxr-x   45 sachin.patel 19 May 12:24 1dfd23a9c6016c61-meta.json*

Also leads to errors such as this in the console..
Hashing error: cannot find package-file hash for @fooo/package-#*.ts!**/*.jsx!**/*.tsx!*.js!*.jsx!*.ts!**/*.js!*.tsx!../../config/eslint/!../../tools/eslint-plugin-i18next-parser/dist/!.eslintrc.js!i18next-parser.config.js!tsconfig.eslint.js

There is not an easy way to clean this up.
Expected Behavior
Terminating turbo via Ctrl-C or killing the pid - should not leave turbo related temp files around.  Or better, don't write these to the repo root (and a temp folder instead)
To Reproduce
Create a task with a large number of packages and fine grained inputs (including files at root using path traversal).  Midway though the execution - Ctrl-C a few times to try to get the process terminated.
Check your repository dir and the directory will be flooded with turbo related files as shown.

This actually happens where you have a path traversal input.

Agreed, turbo should not be leaving these around.

Is there a timeline for a fix for this issue?

There is not an easy way to clean this up.

With GNU find I have managed to clean up some of the mess with find -maxdepth 1 -regextype egrep -regex './[0-9a-f]{16}($|-meta.json)' -exec rm -rf {} +. Combined with git clean -fn this works ok.

Agreed, turbo should not be leaving these around.

I agree with OP that it would be good if they were not created in the first place. Can a git-ignored directory be used, like .turbo?
I haven't been able to consistently reproduce the error, but it started appearing after a refactoring commit which updated all workspaces plus turbo.json. Rebasing an existing workspace on top of that commit triggered the error.

Is this still happening with version 1.5.5 (latest as of now)?
These files should be exclusively written in the cache directory. If they are still ending up elsewhere, are you by any chance customizing the cache directory? It is normally node_modules/.cache/turbo

I'm still having trouble reproducing this issue, so I'd like to confirm some details:

This happens only when hitting Ctrl+C during a long-running task
There's something (outputs?) in turbo.json that does path traversal up to files at the root
The cache folder is left as the default (node_modules/.cache/turbo)
The working directory flag is not being used (--cwd)

Could someone who's experiencing this problem share either a minimal repro or the relevant piece of turbo.json?

Hey, folks! I'm going to close this one as a stale due to inactivity and a lack of reproduction link.
If you're still running into this issue, please open up a fresh issue with a repro!


Item Description Spelling Error

In 2.80, there's a spelling error in the description of Unknown Dimlets. "it's purpose" should be "its purpose," as the former means "it is purpose," while (presumably) the intended meaning is "the purpose of it."

Fixed in next releease


Design: Optimistic Light Clients

Design: Optimistic Sovereign Rollups
Background
Currently, Sovereign only supports full nodes. Although we have support for zk-proof generation, our node implementation always re-executes all transactions starting from rollup genesis. As we move toward production implementations, we'll need support for faster syncing modes.
Desired Properties
Efficiency

Sync time should not depend on cumulative gas used on the rollup.

In other words, the sync should be constant-time per block

Sync bandwidth requirements should be less than syncing a full node. In other words, nodes should not download every transaction during sync (unless the underlying DA layer requires it).

Correctness
The following assumptions may be required to ensure that a node ends up at the correct state:

The node has access to a correct view of the DA layer's consensus (i.e. it's header chain)
(ORUs only) All invalid state transition claims are challenged within some period (say, 24 hrs)
(Non-succinct DA layers only) The node has (untrusted) access to the DA chain's history

The following additional assumptions may be introduced if necessary to speed up sync in the "happy case":

Sovereign Labs (or another centralized entity) can operate untrusted but relatively available (95%+ percent uptime) centralized infrastructure to aid light clients
Most rollup nodes are connected to at least one honest peer via the p2p network.

The following assumptions are insecure and may not be relied on:

All nodes are connected to an honest peer via the rollup p2p network.
Rollup nodes know which of their peers are honest/malicious
Sovereign Labs (or any other centralized entity) is able to run trusted infrastructure and/or provide honest peer connections

Proposed Solution
Basic Design: Attestations and Challenges
Sovereign Optimistic Rollups should separate their data into two namespaces, a "syncing" namespace and a "transaction (tx)" namespace.  Rollups must process all of the data from both namespaces. As in the existing Sovereign design, blobs can be posted to the tx namespace in any order - so the latest rollup state can only be computed after a DA layer block is accepted.
In addition to "sequencers", we introduce a new role "attesters". The job of an attester is to inform light clients that a state transition has occurred by posting a special Attestation blob to the "syncing" namespace. Like sequencers, attesters must register by staking tokens.
/// An attestation that a particular DA layer block transitioned the rollup state to some value
struct Attestation<StateProof> {
  initial_state_root: [u8;32],
  /// The hash of the block in which the transition occurred
  da_block_hash: [u8;32]
  /// The alleged post-state root
  post_state_root: [u8;32],
  /// A proof that the attester was bonded as of `initial_state_root`. For rollups using the `jmt`, this will be a `jmt::SparseMerkleProof`
  proof_of_bond: StateProof,
}

If an attestation is incorrect, any bonded party may challenge the attester by posting a Challenge blob. This blob is just a serialized zk-proof showing that the true result of the state transition was different from the one claimed by the attester.
struct Challenge(&'a [u8]);

pub struct StateTransition<ChainValidityCondition> {
    /// The state of the rollup before the transition
    pub initial_state_root: [u8; 32],
    /// The state of the rollup after the transition
    pub final_state_root: [u8; 32],
    /// Any additional validity condition for the state transition which needs
    /// to be checked by the client of the zkVM circuit.
    pub validity_condition: ChainValidityCondition,
}

pub struct ChallengeContents{
  challenger_address: [u8;32],
  state_transition: StateTransition<ChainValidityCondition>
}

pub struct ChainValidityCondition {
    pub prev_hash: [u8; 32],
    pub block_hash: [u8; 32],
}

pub fn do_challenge<Vm: Zkvm>(challenge: Challenge, attestation: Attestation) -> Result<()> {
   let challenge_contents: ChallengeContents = Vm::verify_proof_and_extract_output(challenge.0)?;
   let proof_output = challenge_contents.state_transition;
   ensure!(proof_output.initial_state_root == attestation.initial_state_root);
   ensure!(proof_output.final_state_root != attestation.post_state_root);
   ensure!(proof_output.validity_condition.check().is_ok());
   ensure!(proof_output.validity_condition.block_hash == attestation.da_block_hash);
   Ok(())
}

Sync Strategy
With that infrastructure in place, we can outline a simple sync protocol.
Assume that clients start from some known good state. This can be the genesis state, a more recent "checkpoint" which is widely known to correspond to a valid rollup state, or a state root that the client has already synced. The state identifier can be expressed as a pair of hashes (rollup_state_root, and da_block_hash).

/// Uniquely identifies the pre-state for some state transition,
#[derive(Hash, PartialEq, Eq)]
struct PreStateIdentifier {
   pre_state_root: [u8;32],
   da_block_hash: [u8;32]
}

struct PotentialStateTree {
  first_unverified_slot_num: u64,
  /// A data structure tracking claimed state transitions for each DA block. Entries are stored as a mapping from
  /// a da_block to a mapping between rollup pre-state roots and state transitions.
  current_state: PreStateIdentifier,
  potential_next_states: Vec<Rc<PotentialState>>,
  /// A mapping from a pre-state identifier to the parent state itself
  potential_state_parent_map: HashMap<PreStateIdentifier, rc::Weak<ParentState>>
}

type ParentState = PotentialState;
type PreStateRoot = [u8;32];
type DaBlockHash = [u8;32];

struct PotentialState {
  attestation: Attestation,
  next_state: MaybeVerifiedStateTransition
}

impl PotentialState {
  fn new(attestation: Attestation) -> Self
    Self { attestation, next_state: Default::default() }
  }
}

/// A potentially verified state transition.
pub enum MaybeVerifiedStateTransition {
   Unverified(Vec<Rc<PotentialState>>) // Default::default() returns Unverified(vec![])
   Verified(StateTransition<ChainValidityCondition>)
}

fn get_pre_state_id(attestation: Attestation) -> PreStateIdentifier {
   PreStateIdentifier {
     pre_state_root: attestation.initial_state_root,
     da_block_hash: get_header_by_hash(attestation.da_block_hash).prev_hash
   }
}

The syncing procedure is as follows (assuming the DA header chain is already downloaded and accessible through a function header(num: u64) -> BlockHeader, and the last processed DA layer block had number current_block_num - 1):

Create a PotentialState tree initialized with the known rollup state root and  current_da_hash = Header(current_block_num - 1).
(Start loop) Download the syncing namespace data for current_block_num.
For each potentially valid attestation from the syncing namespace data, make a PotentialState::new() from the attestation. If the attestation’s pre-state is the current state, add it to the potential_next_states vector. Otherwise, look up its pre-state in the potential_state_parent_map and try to add the new potential state as a next_state for that parent. If this succeeds, potential_state_parent_map.insert(potenial_state.post_state, Rc::weak(potential_state)).

An attestation is potentially valid if its proof_of_bond verifies against its initial_state_root and it's PreStateIdentifier is either the current_state or is present in the potential_state map.
Two attestations are considered to be duplicates of one another if they are both valid and they have the same PreStateIdentifier, and final_state_root (Note that the proof_of_bond may be different for two duplicate attestations!). If duplicate attestations occur, ignore all except the first one. TODO: Add a hashset for deduplication in MaybeVerifiedStateTransition.

For each challenge in the syncing namespace data…

If the challenge’s pre-state is the current state and the challenge is valid, update the current state to the challenge’s post state. std::mem::take the potential_next_states vector. For each entry in the vector, whose post-state doesn’t match the new state, remove it’s post-state from the potential_state_parent_map. Recursively remove each of the children of those states from the map
If the challenge’s pre-state is in the potential_state_parent_map and the challenge is valid, update the parent_state.next_state field to be Verified, and remove each of the old potential next_states if applicable

If the current_block is at least 24 hours newer than the block from the current_state_id, update the current_state_id to  its child state.
(GOTO start loop) increment current_block_number.

Handling Incentives
The structure proposed above mostly ignores incentives. In this section, we'll flesh out the details.
2-Phase Unbonding for Attesters
To prevent spam and allow for quick sync, we need light clients to be able to throw out attestations from un-bonded entities. This is a problem, because by definition light clients are trailing behind the chain tip - so they can’t possibly know whether an attester is currently bonded. We solve this problem by (1) using a merkle proof to guarantee that an attester was bonded at the time of the state transition they’re attesting to and (2) introducing an unbonding period so that we can guarantee that by the time an attester has finished unbonding, all light clients are aware that they’ve started unbonding.
We can do that by introducing a new attester-registry module. That module will look very similar to the sequencer registry (only bonded attesters can post blobs), but, unlike the sequencer module, it will need a two-phase un-bonding workflow. When an attester initially bonds, their stake is locked in the contract and their address is added to the bonded_attesters map in state.
Define some ROLLUP_FINALITY_PERIOD corresponding to the amount of time it takes for a light client to be confident that an attested state transition won’t be challenged. Define ROLLUP_FINALITY_SLOTSto be the maximum number of DA layer slots which could occur in that period.
Suppose that an attester was bonded as of slot N. To unbond, they have to submit a begin_unbonding transaction - imagine that this transaction is included in slot N+1. During that state transition, their address will be removed from the bonded_attesters list and placed in a special unbonding_attesters map. Attesters in this map are no longer able to make attestations (i.e. light clients will ignore them), but are also unable to withdraw their funds.
Next (no sooner than block N+2) an attestation to the transition from slot N+1 will be posted on chain by some other attester. Since the rollup is already reading all of the data from the “syncing” namespace, it will see that attestation during execution of block N+2 . At that time, the rollup will update its state to allow the attester to withdraw starting any time from slot N+2+ROLLUP_FINALITY_SLOTS.
Rewarding/Slashing Attestations
As mentioned above, the rollup must process the “syncing” namespace. For any attestation in the namespace corresponding to an invalid state transition, the rollup should slash the attester immediately. Similarly, the first valid attestation to any state transition should be rewarded. The exact reward formula is left to individual rollup designers.
Rewarding Challengers
Individuals who successfully challenge an attestation should be rewarded with (some portion of) the attesters bond. The remaining portion should be burned.

Bonding Challengers
In this design, there is tension between two goals - we want to keep the set of challengers as large as possible, but we also want to be able to slash malicious challengers to prevent DOS attacks. We balance these competing goals with the following strategy - we require light clients to process all challenges (even if the challenger is not bonded), but we only reward challengers (and slash attesters) if the challenger is bonded.
Using this strategy, we can create a new invariant: for each challenge, either (1) the outcome of a challenge will never have to be proven in circuit (because the challenge doesn't result in a state change) or (2) the challenger will be slashed if the challenge is unsuccessful.
The DOS Vector
If SNARKS are so cheap to verify, why is challenging a potential DOS vector? In native execution, it isn't! But in-circuit, recursive verification can be quite expensive. So, the DOS vector is that a malicious challenger could post hundreds or thousands of bogus challenges in block X, and then post a false attestation to a state transition in block X+1. If these challenges resulted in state changes, an honest party would have to prove the outcome of each of these challenges in order to prove the true state transition, which would require a huge number of recursive evaluations.
To prevent such an attack, we just need to ensure that the such challenges can never result in state changes. If that is the case, then we won't need to recursively evaluate them in order to prove some other state transition. But, since native evaluation is cheap, we still expect light clients to process these challenges during sync.
The Honest Challenger Workflow
This rule gives us a huge gain in capital efficiency (challengers don't have to be bonded during normal execution), while still incentivizing challenges from honest parties in case of an invalid attestation. The workflow looks like this:

Block N: a malicious attester lies about the state transition from block N-5. An honest full node immediately notices the invalid attestation. (Recall that light clients have only finalized the state transition for block N-K for some relatively large value K - maybe a few hundred blocks).
Block N+1: the honest full node submits a transaction which bonds himself as a challenger
Block N+2: the honest full node submits a challenge on-chain. Light clients immediately process the challenge and prune the invalid transition from their lists of potential states. Since the honest full node is bonded, it gets rewarded and the malicious attester gets slashed as part of the state transition during block N+2.
Block N+3: another malicious attester posts a lie about transition N+2, so a challenge is needed. The challenger is forced to compute a recursive verification of the previous challenge, but this cost will be paid out of the new attester's bond.

What Happens if a Challenger is Malicious
If the challenger is malicious, we have two possible cases:
Case 1: The challenger does not bond

Block N: an attestation for the transition during block  N-5 is posted on-chain.
 Since the challenge is malicious, it doesn't bond itself first. Light clients process the challenge and see that the proof is forged, so they don't update their view of potential states.
Block N+3: a challenge depending on the outcome of slot N+2 is needed. Since the malicious full node was not bonded, its challenge cannot cause a state change, so we can skip recursive verification

Case 2: The challenger bonds

Block N: an attestation for the transition during block  N-5 is posted on-chain.
  Light clients process the challenge and see that the proof is forged, so they don't update their view of potential states. Full nodes see that the challenge is invalid and slash the challenger's bond.
Block N+3: another malicious attester posts a lie about transition N+2, so a challenge is needed. The challenger is forced to compute a recursive verification of the previous challenge, but this cost will be paid out of the new attester's bond.

Integration Roadmap
Here is a series of steps that need to be performed until the feature gets fully integrated into the sdk. We will associate each action to its own PR to allow a better separation of concerns:

[ ] Implement the sov-attesters-incentives module that handles the attesters/challengers bonding/unbonding and processes the attestations/challenges.
[ ] Implement a synchronization strategy for optimistic rollups (in the state transition function), to allow communication between a light client and the associated rollup
[ ] Implement a simple light client node that can fast forward the chain using attestations
[ ] Add an integration test to have a complete test suite for the system


Add clp_s

References

Description

This PR adds clp_s, a version of clp tailored for semi-structured (i.e. JSON) logs. It optimizes storage format for semi-structured logs and supports KQL.
Validation performed

Successfully built clp_s.

Besides the inline comments:

We need to describe clp-s and its usage in src/core/README.md.

We should note any known limitations like the fact that time zones are not currently parsed (and thus dropped).

Add absl, antlr, and simdjson to .clang-format

I haven't done a thorough review, but here are some of the things I've noticed that we should address now and/or in future PRs:

Delete any cpp files with no content.
Change any header define guards which use _H to _HPP.
Rename files which only contain namespace methods to snake_case.
Replace using namespace ... with using the actual symbols.
Change classes that only contain static methods into namespaces.
Deduplicate the files/methods common between clp and clp-s.
Add missing class and method docstrings.

Thanks for the review. For deduplicating the files and methods, we can leave it to the next PR.

LGTM


Set base from env in vite.config.ts

Configure Vite base URL from VITE_BASE_URL env variable (incl. Docker).

Thanks @avillar, this is a good issue to tackle, although this solution only configures the VITE_BASE_URL when building your own Docker image. I've implemented a generic solution without relying on configuring at build time, see https://github.com/RDFLib/prez-ui/tree/feature/base-url-fix
Let me know if this fix works for you.

@jamiefeiss, cool, thanks, but I think the issue for the vite assets still persists. The


Remove scripts, keep some basic examples

Ref offline discussions, suggest to remove the scripts folder, and not keep ERT scripts etc in this repo. But, as part of documentation, keep some basic examples.

#58


Pandas Series: Numeric indices that do not contain zero

Attempting to write a DataFrame/Series with an integer index that does not contain zero will throw an exception:
df = DataFrame({'k':[1,2,3],'v':['a','b','c']}).set_index('k')
q.sync('`test set',df)

File "C:\Anaconda3\envs\pyq\lib\site-packages\qpython\_pandas.py", line 157, in _write_pandas_series
    qtype = Q_TYPE.get(type(data[0]), QGENERAL_LIST)
KeyError: 0

This is caused by the data[0] lookup.  With a non-numeric index data[0] will return the first row as expected, however when the index is numeric it will search for 0 and find nothing.  Explicit indexing by row number (via iloc) fixes the issue.
_pandas.py:157
    qtype = Q_TYPE.get(type(data[0]), QGENERAL_LIST)
    qtype = Q_TYPE.get(type(data.iloc[0]), QGENERAL_LIST)

Thank you for the report and proposed solution. Please checkout latest qPython version.


CWE statistics in product metrics

Note: DefectDojo is now on Python3 and Django 2.2.1 Please submit your pull requests to the 'dev' branch as the 'legacy-python2.7' branch is only for bug fixes. Any new features submitted to the legacy branch will be ignored and closed.
When submitting a pull request, please make sure you have completed the following checklist:

[x] Your code is flake8 compliant
[x] If this is a new feature and not a bug fix, you've included the proper documentation in the ReadTheDocs documentation folder. https://github.com/DefectDojo/Documentation/tree/master/docs or provide feature documentation in the PR.
[x] Model changes must include the necessary migrations in the dojo/dd_migrations folder.
[x] Add applicable tests to the unit tests.

This pull request adds 2 additional graphs to Product Metrics page that show:

Number of Open Findings grouped by CWE.
Number of Findings grouped by CWE.


Food Rewards

Just to let you know to not use food as rewards in the HQM quests. The food has a time stamp and so when players receive the reward it will have the time stamp of when you created quest, not when they received the reward. So more often than not it will rot to nothing when they receive it.

Addressed this by removing all food rewards. Made the food based quests an optional branch.


cookies required

Hi! I have successfully install swiperproxy and it works like charm! However, when I try to login into my facebook or my gmail account I got an error says
Cookies Required. Cookies are not enabled on your browser.
Please enable cookies in your browser preferences to continue.

Of course cookies are enabled in my browser and I can login to those accounts when not using swiperproxy. I didn't find a conf option to enable or disable cookies. Did I miss anything? How shall I enable cookies in swiperproxy? Thanks!

Facebook works fine on my machine(tm)

Gmail doesn't, because of some server-side Gmail-fu that I can't get a handle on real quick. I'll take a deeper look into what Gmail exactly does and if we can fix this in the proxy.

Do you why the "cookies not enabled" error appear? Is there any conf options I need to tweak? I appreciate if you could point me to the right direction.

Gmail has resolved this issue on remote host.


Don't upload pipeline if no steps added

Buildkite agent will fail if pipeline has empty steps

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Mobile Nav and Guides Mobile Styling/Structuring

Style Mobile Navigation Bar
Formatting/Structure for Doc Pages - Mobile

@yowakita everything looks good, just a few things:

I just think the main links in the nav should have a display: block so that when you hover over the whole width of the row, it'll be clickable

the animation could be faster

If you open the nav and then resize, I think the nav should close, might be a js thing to remove the class

-Refering to mobile nav?

Changed in the next PR where transition variables are introduced

I'll write a JS script for it and push it up

@yowakita yea sorry all in regards to mobile nav


Simple planar-renderer script to render planar reflections

PlanarRenderer script, which handles mirroring by plane and rendering to texture
made Plane class public
renamed couple of examples to REFLECTION*** pattern to keep those grouped.

TODO:

integrate the result to standard material, allowing normal maps, roughness ... (separate PR)

Why not make the example default to using a 100% scaled reflection buffer? Looks less aliased that way 🙂.

Why not make the example default to using a 100% scaled reflection buffer? Looks less aliased that way 🙂.

Sure, done.

Wondering whether Frustum should be using this new Plane class...


Ingestion frequency on Event-Hub Data-Connections

Setup:

2 nodes Data-Explorer
Event-Hub with 4 Throughput units

Data:
Sending ~ 2k events per second in 500-element batches to Event-Hub
Expectation:
Data will be indexed in a timely fashion and be available for queries.
Actual result:
There seems to be an enforced frequency of how often the event-hub data will be indexed to Data Explorer. On the charts for Ingestions results, one can clearly see, that there is exactly one ingestion-run happening every five minutes.
Question:
Am I doing things wrong?
If not: Is there a way of tweaking this and bring the ingestion frequency to a near-realtime experience? We are currently using different infra to handle data, ingesting ~ 10 billion events per day and let users get results of predefined and ad-hoc queries with end-to-end-latencies of a couple seconds.
Maybe Data-Explorer has not been designed for this kind of workload?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d5808ed2-5d09-4c56-dc0b-4e3650f8fc16
Version Independent ID: 10ebaf9d-0cc0-b59b-ae95-63bc7b1be4b5
Content: Quickstart: Ingest data from Event Hub into Azure Data Explorer
Content Source: articles/data-explorer/ingest-data-event-hub.md
Service: data-explorer
GitHub Login: @mgblythe
Microsoft Alias: mblythe

Thanks for the feedback! We are currently investigating and will update you shortly.

There is an internal batching in ADX for ingress data (to improve performance).
Batching is limited by accumulated size (1GB) or by time (5 minutes).
We will soon be releasing enhancements that will allow customers control these batching limits to achieve near real-time ingestion.
Stay tuned at our Tech Community

Hi vlad,
is there any timeline when this near-realtime ingestion will be released? We are currently building up a scalable IoT solution and I'd be really happy to see the realtim eingestion in ADX.
KR
Christian


admin manual completed, waiting for dev on some ambiguities

So your last commit says waiting on bug fixes to finalize admin manual. Which bugfixes are you waiting on?

I'm assuming the installation guides are still in a provisional state, so I'm not proofreading those too carefully for this PR.

Okay, sorry to backtrack on this issue again and again, but apparently we are switching from the Student/Teacher terminology to Learner/Coach, so we'll have to update the docs to reflect that again... ¯_(ツ)_/¯

In userman_main.rst, line 25:

Learning is fundamental to human flourishing, and strive to support learning opportunities on every front.

This is grammatically incorrect, second clause is missing a subject...?

That's it for my first proofreading pass.

I'm merging this for now, because I want to move this project to a different repo.


Session storage should not fallback to cookies

Session storage is, by design intended for potentially private data.
For this reason, it cannot be shared between tabs, or persist. And it is for this reason I think you should consider removing the cookie fallback for session storage. (or make it configurable)

I made pull request #185 that allows you to force the cookie to be a secure cookie, but I got to thinking that it should be possible to use an http interceptor that would still use the cookie as a fallback storage mechanism but would prevent it from ever being sent?  What do people think of that?

+1. I would, at the very least, give the option to NOT fallback to cookies if storage is not supported.

+1, due to the cookie fallback it is not possible to get synchronously the information that saving to localStorage failed (i.e. because of quota exceed).

is there really no plans to at least add this option? I might have to move away from this plugin because its    network request by exceeding 4k cookie size.


Examples for the llama 3.2 3B updated

Which issue(s) does this pull-request address?

Closes: #47
Description

Created an examples using the llama 3.2 3B model

This PR was done during Hactoberfest 2024

Checklist

[x] I have read the contributor guide
[x] Linting passes: yarn lint or yarn lint:fix
[x] Formatting is applied: yarn format or yarn format:fix
[x] Unit tests pass: yarn test:unit
[x] E2E tests pass: yarn test:e2e
[ ] Tests are included
[ ] Documentation is changed or added
[x] Commit messages and PR title follow conventional commits

@Tomas2D sorry for the misleading error in the dependency due to the conflict in the yarn lock file I had removed a dependency.

@Tomas2D the dependency issue was resolved and the test was success.

@Tomas2D need help as the CI is failing for the previous commit messages. So how can I modify the previous commit messages?

Use git rebase


ADG last row doesn't appear on screen fully (half row is shown on Screen)

Project Details are :
https://paste.apache.org/cpo8y
Test case is available on VDI.
Go to Purchase from Select in Menu
Go to Foreign Currency -> Types from Setups in Menu
File : com/dbz/modules/Common/dbzFCRateType.mxml

Still ADG last row doesn't appear on screen fully


Added Caching in Optimizely Config

Need to add Unit Tests

Pull Request Test Coverage Report for Build 1308

109 of 128 (85.16%)  changed or added relevant lines in 8 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.09%) to 89.59%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

core-api/src/main/java/com/optimizely/ab/config/PollingProjectConfigManager.java
2
3
66.67%

core-api/src/main/java/com/optimizely/ab/optimizelyconfig/OptimizelyVariation.java
9
10
90.0%

core-api/src/main/java/com/optimizely/ab/optimizelyconfig/OptimizelyConfig.java
6
8
75.0%

core-api/src/main/java/com/optimizely/ab/Optimizely.java
2
9
22.22%

core-api/src/main/java/com/optimizely/ab/optimizelyconfig/OptimizelyConfigService.java
62
70
88.57%

Totals

Change from base Build 1296:
-0.09%

Covered Lines:
22308

Relevant Lines:
24900

💛  - Coveralls


fix required repeatable fields

I was trying to figure out why would we specifically setup the showAsterisk => false in repeatable and find out that it might be to solve an issue when the parent (the repeatable field itself) is required.
From my tests I find out that it does not solve the issue and the issue still persists, so if the parent repeatable is required, all the fields inside the repeatable field show the red asterisk.

Even setting up showAsterisk => false would not make the asterisks go away from the repeatable fields.
I found out that is due to a CSS rule not beeing correctly applied.
This would require re-publishing Backpack CSS to work.

A new inspection was created.

The inspection completed: No new issues

@pxpm it's only because of that CSS change that we need to consider this a breaking change, am I right?
In that case... couldn't we add this new CSS rule to form_content.blade.php too, in the after_styles section, with a note to remove it in v4.2? It would be a bit ugly, but it would allow us to merge it in 4.1, right?
If so, let's please create a separate PR to remove that in 4.2, so we don't forget.
PS. I'm getting to understand why you said we need the 4.2 branch already 😅

@tabacitu I was having a look at your reply, and .. I don't think this is a BC ? This needs the devs to re-publish the assets to get the updated version, but it does not break previous behaviour, as I think the previous behaviour was broken at start?
New installs would get the fixed version, people running php artisan vendor:publish --force would also get the fix. People simply upgrading crud versions would not get the fix.
Am I missing something and this is a BC in any way ? Let me know if you think I could be wrong.
Best,
Pedro

Regarding this being a BC or not.
@pxpm It's true the previous behavior was broken any way 🤷♂ BUT, if a dev previously set a repeatable to be required, until now he was not even seeing the asterisk on the CRUD field * , but from now on he is going to see the asterisk on every field inside repeatable, and that may be a BC.

A new inspection was created.

@pxpm hope you don't mind, I just added the non breaking CSS temporary fix for v4.1.
I really think it's a BC ...

By the way guys, yesterday on Laracon Taylor said something about BC, he said on Laravel when they have a PR with a BC they always try to make it non BC to be able to push new features asap. We should do the same ✌ and these CSS files that must be published are the major BC generator we have 🙃
Let's fix it on 4.2 🙌

The inspection completed: No new issues

these CSS files that must be published are the major BC generator we have 🙃

Totally agree. I'm not sure we can fix it in v4.2 though, it's more likely we'll do it in v12, by overhauling how we load CSS & JS files.


Implement BDX synchronous sender role over CRMP

Problem
CHIP defines means for performing a bulk-data transfer (BDX) over the CHIP reliable messaging protocol (CRMP). The BDX synchronous sender role over CRMP logic is not currently implemented.
Proposed Solution
Implement BDX synchronous sender role over CRMP

See also changes in https://github.com/project-chip/connectedhomeip/pull/8963


Unable to restore database pg_restore: error: unsupported version (1.15) in file header

Hi,
trying to migrate to a new server. I am currently getting the following error
+ pg_restore --dbname=zulip -- /tmp/zulip-restore-backup-6csb40ma/zulip-backup/database
pg_restore: error: unsupported version (1.15) in file header

Subcommand of /home/zulip/deployments/current/scripts/setup/restore-backup failed with exit status 1: pg_restore --dbname=zulip -- /tmp/zulip-restore-backup-6csb40ma/zulip-backup/database
Actual error output for the subcommand is just above this.

Both servers are on 7.4, and both are on Ubuntu
Old Server spec
2 Core CPU 4GB RAM
New Server spec
1 core CPU
2GB RAM
Both Servers are on
psql (PostgreSQL) 16.0 (Ubuntu 16.0-1.pgdg20.04+1)
pg_restore (PostgreSQL) 15.4 (Ubuntu 15.4-2.pgdg20.04+1)
pg_dump (PostgreSQL) 15.4 (Ubuntu 15.4-2.pgdg20.04+1)
 /home/zulip/deployments/current/scripts/setup/restore-backup zulip-backup-2023-10-09-10-14-48-dokq4m2v.tar.gz
+ tar --directory=/etc/zulip --strip-components=2 -xz zulip-backup/settings
+ tar -C /tmp/zulip-restore-backup-60pq_me6 '--transform=s|^zulip-backup/zproject(/.*)?$|/mnt/hdd/zulip/deployments/2023-10-09-09-43-21/zproject\1|x' '--transform=s|^zulip-backup/uploads(/.*)?$|/home/zulip/uploads\1|x' -xPz
+ chown -R postgres -- /tmp/zulip-restore-backup-60pq_me6
+ /mnt/hdd/zulip/deployments/2023-10-09-09-43-21/scripts/setup/terminate-psql-sessions zulip
 pg_terminate_backend
----------------------
 t
 t
 t
 t
 t
 t
 t
 t
 t
(9 rows)

+ dropdb --if-exists -- zulip
+ createdb --owner=zulip --template=template0 -- zulip
+ /mnt/hdd/zulip/deployments/2023-10-09-09-43-21/scripts/setup/generate_secrets.py --production
generate_secrets: No new secrets to generate.
+ /mnt/hdd/zulip/deployments/2023-10-09-09-43-21/scripts/setup/configure-rabbitmq
Deleting user "zulip" ...
Deleting user "zulip" ...
Error:
{:no_such_user, "zulip"}
Deleting user "guest" ...
Error:
{:no_such_user, "guest"}
Adding user "zulip" ...
Setting tags for user "zulip" to [administrator] ...
Setting permissions for user "zulip" in vhost "/" ...
+ /mnt/hdd/zulip/deployments/2023-10-09-09-43-21/scripts/zulip-puppet-apply -f
Notice: Compiled catalog for sr.sugar.reviews in environment production in 7.86 seconds
Notice: /Stage[main]/Zulip::Tornado_sharding/File[/etc/zulip/nginx_sharding_map.conf]/owner: owner changed 'zulip' to 'root'
Notice: /Stage[main]/Zulip::Tornado_sharding/File[/etc/zulip/nginx_sharding_map.conf]/group: group changed 'zulip' to 'root'
Notice: /Stage[main]/Zulip::App_frontend_base/File[/etc/zulip/uwsgi.ini]/content: content changed '{md5}6e835551c08262e67b9961c6865b21d5' to '{md5}b72ce975cffa1f42415cfffd3e16ad96'
Notice: /Stage[main]/Zulip::App_frontend_base/File[/etc/zulip/uwsgi.ini]/owner: owner changed 'zulip' to 'root'
Notice: /Stage[main]/Zulip::App_frontend_base/File[/etc/zulip/uwsgi.ini]/group: group changed 'zulip' to 'root'
Notice: /Stage[main]/Zulip::Nginx/Service[nginx]: Triggered 'refresh' from 2 events
Notice: /Stage[main]/Zulip::Supervisor/Service[supervisor]: Triggered 'refresh' from 3 events
Notice: Applied catalog in 3.52 seconds
+ pg_restore --dbname=zulip -- /tmp/zulip-restore-backup-60pq_me6/zulip-backup/database
pg_restore: error: unsupported version (1.15) in file header

Subcommand of /home/zulip/deployments/current/scripts/setup/restore-backup failed with exit status 1: pg_restore --dbname=zulip -- /tmp/zulip-restore-backup-60pq_me6/zulip-backup/database
Actual error output for the subcommand is just above this.

Traceback (most recent call last):
  File "/srv/zulip-venv-cache/7d251b5fceb78849af8c90d8b372c44b238b35ac/zulip-py3-venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/mnt/hdd/zulip/deployments/2023-10-09-09-43-21/./zerver/lib/db.py", line 35, in execute
    wrapper_execute(self, super().execute, query, vars)
  File "/mnt/hdd/zulip/deployments/2023-10-09-09-43-21/./zerver/lib/db.py", line 20, in wrapper_execute
    action(sql, params)
psycopg2.errors.UndefinedTable: relation "zerver_realm" does not exist
LINE 1: ...source", "zerver_realm"."night_logo_version" FROM "zerver_re...
                                                             ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/srv/zulip-venv-cache/7d251b5fceb78849af8c90d8b372c44b238b35ac/zulip-py3-venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/srv/zulip-venv-cache/7d251b5fceb78849af8c90d8b372c44b238b35ac/zulip-py3-venv/lib/python3.8/site-packages/django/utils/deprecation.py", line 133, in __call__
    response = self.process_request(request)
  File "/mnt/hdd/zulip/deployments/2023-10-09-09-43-21/./zerver/middleware.py", line 551, in process_request
    request_notes.realm = get_realm(subdomain)
  File "/mnt/hdd/zulip/deployments/2023-10-09-09-43-21/./zerver/models.py", line 1062, in get_realm
    return Realm.objects.get(string_id=string_id)
  File "/srv/zulip-venv-cache/7d251b5fceb78849af8c90d8b372c44b238b35ac/zulip-py3-venv/lib/python3.8/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/srv/zulip-venv-cache/7d251b5fceb78849af8c90d8b372c44b238b35ac/zulip-py3-venv/lib/python3.8/site-packages/django/db/models/query.py", line 633, in get
    num = len(clone)
  File "/srv/zulip-venv-cache/7d251b5fceb78849af8c90d8b372c44b238b35ac/zulip-py3-venv/lib/python3.8/site-packages/django/db/models/query.py", line 380, in __len__
    self._fetch_all()
  File "/srv/zulip-venv-cache/7d251b5fceb78849af8c90d8b372c44b238b35ac/zulip-py3-venv/lib/python3.8/site-packages/django/db/models/query.py", line 1881, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/srv/zulip-venv-cache/7d251b5fceb78849af8c90d8b372c44b238b35ac/zulip-py3-venv/lib/python3.8/site-packages/django/db/models/query.py", line 91, in __iter__
    results = compiler.execute_sql(
  File "/srv/zulip-venv-cache/7d251b5fceb78849af8c90d8b372c44b238b35ac/zulip-py3-venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
    cursor.execute(sql, params)
  File "/srv/zulip-venv-cache/7d251b5fceb78849af8c90d8b372c44b238b35ac/zulip-py3-venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
  File "/srv/zulip-venv-cache/7d251b5fceb78849af8c90d8b372c44b238b35ac/zulip-py3-venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/srv/zulip-venv-cache/7d251b5fceb78849af8c90d8b372c44b238b35ac/zulip-py3-venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/srv/zulip-venv-cache/7d251b5fceb78849af8c90d8b372c44b238b35ac/zulip-py3-venv/lib/python3.8/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/srv/zulip-venv-cache/7d251b5fceb78849af8c90d8b372c44b238b35ac/zulip-py3-venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/mnt/hdd/zulip/deployments/2023-10-09-09-43-21/./zerver/lib/db.py", line 35, in execute
    wrapper_execute(self, super().execute, query, vars)
  File "/mnt/hdd/zulip/deployments/2023-10-09-09-43-21/./zerver/lib/db.py", line 20, in wrapper_execute
    action(sql, params)
django.db.utils.ProgrammingError: relation "zerver_realm" does not exist
LINE 1: ...source", "zerver_realm"."night_logo_version" FROM "zerver_re...
                                                             ^

CZO discussion.


ctop for debian bookworm not present in repository

ctop debian bookworm not present in repository

Try this workaround: #347


Introspection seems to have generated a Prisma schema but it's not stored on the file system

This issue is related to: https://github.com/prisma/prisma2/issues/699 It doesn't apply to the current preview013 release because there the introspection is broken for another reason, so I downgraded to preview-12 where I'm now seeing this issue.
I've tried this both with the SQL Murder Mystery as well as with another sample SQLite DB and the problem happens with both.
When running prisma2 init, I can successfully connect to the DB and the introspection seems to be succesful:

However, after having selected the Demo script in the last step, the command fails:

This is the error message:
$ prisma2 init murder2

Preparing your demo script (TypeScript) ...

✔  Downloading the demo script from GitHub ...
⠋  Extracting content to murder2 ...
3  Installing dependencies with: `npm install` ...
 ERROR  Error during command execution

Schema parsing error: Error validating model "CrimeSceneReport": Exactly one field must be marked as the id field with the `@id` di
rective..
  -->  schema.prisma:6
   |
 5 |
 6 | model CrimeSceneReport {
 7 |   city        String?
   |
error: Error validating model "FacebookEventCheckin": Exactly one field must be marked as the id field with the `@id` directive..
  -->  schema.prisma:29
   |
28 |
29 | model FacebookEventCheckin {
30 |   date      Int?
   |
error: Error validating model "GetFitNowCheckIn": Exactly one field must be marked as the id field with the `@id` directive..
  -->  schema.prisma:38
   |
37 |
38 | model GetFitNowCheckIn {
39 |   checkInDate  Int?             @map("check_in_date")
   |
error: Error validating model "Interview": Exactly one field must be marked as the id field with the `@id` directive..
  -->  schema.prisma:65
   |
64 |
65 | model Interview {
66 |   person     Person? @map("person_id")
   |
error: Error validating model "Solution": Exactly one field must be marked as the id field with the `@id` directive..
  -->  schema.prisma:86
   |
85 |
86 | model Solution {
87 |   user  Int?
   |

So, it seems to me that there already is a Prisma schema file that was generated and includes a data model. However, the schema.prisma that's now part of the new project that was created by the init command now only contains the sample schema:
datasource db {
  provider = "sqlite"
  url      = "file:./dev.db"
}

generator photon {
  provider = "photonjs"
}

model User {
  id    String  @default(cuid()) @id
  email String  @unique
  name  String?
  posts Post[]
}

model Post {
  id        String   @default(cuid()) @id
  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt
  published Boolean  @default(true)
  title     String
  content   String?
  author    User?
}

Is this a bug and despite the fact that there are issues with the data model, would it be possible to store it in the schema.prisma?

Yes,looks like a bug to me as well. Before even though we are not supporting columns with IDs right now, we were printing the datamodel to the schema instead printing the defaults.

Internal note: Dropping on internal introspection

@nikolasburk  We rewrote introspection recently. Can you please see if this issue still exists in the latest alpha or preview?

I currently still error out:
> prisma2 introspect --url="file:sql-murder-mystery.db"

Introspecting …

Introspection failed: Introspected schema can't be parsed.
Introspected Schema:

datasource db {
  provider = "sqlite"
  url      = "file:sql-murder-mystery.db"
}
model crime_scene_report {
  city        String?
  date        Int?
  description String?
  type        String?
}

model drivers_license {
  age          Int?
  car_make     String?
  car_model    String?
  eye_color    String?
  gender       String?
  hair_color   String?
  height       Int?
  id           Int?     @id
  plate_number String?
  persons      person[] @relation(references: [license_id])
}

model person {
  address_number          Int?
  address_street_name     String?
  id                      Int?                     @id
  license_id              drivers_license?
  name                    String?
  ssn                     Int?
  facebook_event_checkins facebook_event_checkin[]
  interviews              interview[]
  get_fit_now_members     get_fit_now_member[]
}

model facebook_event_checkin {
  date       Int?
  event_id   Int?
  event_name String?
  person_id  person?
}

model interview {
  person_id  person?
  transcript String?
}

model get_fit_now_member {
  id                    String?                @id
  membership_start_date Int?
  membership_status     String?
  name                  String?
  person_id             person?
  get_fit_now_check_ins get_fit_now_check_in[]
}

model get_fit_now_check_in {
  check_in_date  Int?
  check_in_time  Int?
  check_out_time Int?
  membership_id  get_fit_now_member?
}

model income {
  annual_income Int?
  ssn           Int? @id
}

model solution {
  user  Int?
  value String?
}

Schema parsing

error: Error parsing attribute "@id": Fields that are marked as id must be required.
  -->  schema.prisma:20
   |
19 |   height       Int?
20 |   id           Int?     @id
   |

error: Error parsing attribute "@id": Fields that are marked as id must be required.
  -->  schema.prisma:28
   |
27 |   address_street_name     String?
28 |   id                      Int?                     @id
   |

error: Error parsing attribute "@id": Fields that are marked as id must be required.
  -->  schema.prisma:50
   |
49 | model get_fit_now_member {
50 |   id                    String?                @id
   |

error: Error parsing attribute "@id": Fields that are marked as id must be required.
  -->  schema.prisma:67
   |
66 |   annual_income Int?
67 |   ssn           Int? @id
   |

Which schema is this @pantharshit00?

It's https://github.com/NUKnightLab/sql-mysteries as mentioned in the original post.

This code was fully replaced. If this still happens, please open a new issue and we are happy to take a look. Thanks.


spec: make sure pre_check_and_set_rebuild plugin is always installed

koji_promote and store_metadata_in_osv3 depends on it
Alternative to #640

Does it make sense to include check_and_set_rebuild plugin in both python-atomic-reactor-koji and python-atomic-reactor-metadata. Not sure if that's allowed in RPM, but it would be neat to have a clearer dependency stated.

No, we can't do that - it would conflict if both RPMs are installed

IMHO we should make a function for doing this and put it in util.py, and use that instead of check_and_set_rebuild.

Yes, +1 on that


[Question] [1.12.2] unknown recipe type for ic2:forge_hammer ???

I realize IC2 support is not in Modtweaker currently, but I expected this to be a vanilla crafting recipe.
Just trying to do the "usual" replace sticks with crafted sticks in forge hammer recipe.
recipes.remove(ic2:forge_hammer);
recipes.addShaped(ic2:forge_hammer, [[ore:ingotIron, ore:ingotIron, null], [ore:ingotIron, ore:treatedStick, ore:treatedStick], [ore:ingotIron, ore:ingotIron, null]]);
recipes.addShaped(ic2:forge_hammer, [[null, ore:ingotIron, ore:ingotIron], [ore:treatedStick, ore:treatedStick, ore:ingotIron], [null, ore:ingotIron, ore:ingotIron]]);
Remove recipe works as expected. Add is not. Items are all valid, syntax correct, etc.
If I remove my script (so default IC2 recipe)
ct recipes hand outputs // unknown recipe type for ic2:forge_hammer
Anyone have any ideas? If I need to wait for IC2 support to go back into Modtweaker then that is fine, the only thing I can come up with is because the forge hammer is special the crafting recipe is special too.

use the issue template.


Add AVIF image to conf/mime.types

Broader AVIF support, and likely also adoption, is incoming:

About 1/2 size compared to jpeg, 2/3 compared to webp, and roughly 1/1 with JPEG XL*
Already support in stable versions of Chrome and Firefox

Also unstable versions of Edge/Opera(Chromium) and Safari(Webkit)

So makes sense to add it to mime.types given it's considered a stable format already.
* not finalized and not supported in any stable browsers

Thank you for contributing to Nginx!
Nginx does not use GitHub for code submissions.
Please visit http://nginx.org/en/docs/contributing_changes.html
and follow the instructions there to submit changes.


Unmatched file, though file exists

I have problems attaching a single .txt file to a release:
2023-01-31T16:54:48.9600442Z ##[group]Run ls -al
2023-01-31T16:54:48.9600998Z ls -al
2023-01-31T16:54:48.9615440Z shell: /usr/bin/bash -e {0}
2023-01-31T16:54:48.9615655Z env:
2023-01-31T16:54:48.9615945Z   ENVIRONMENT: devel
2023-01-31T16:54:48.9616194Z   PROJECT: workflow-test
2023-01-31T16:54:48.9616391Z ##[endgroup]
2023-01-31T16:54:48.9660999Z total 12
2023-01-31T16:54:48.9661637Z drwxr-xr-x 3 runner runner 4096 Jan 31 16:54 .
2023-01-31T16:54:48.9661991Z drwxr-xr-x 3 runner runner 4096 Jan 31 16:54 ..
2023-01-31T16:54:48.9662341Z drwxr-xr-x 2 runner runner 4096 Jan 31 16:54 sha.txt
2023-01-31T16:54:48.9789293Z ##[group]Run softprops/action-gh-release@v1
2023-01-31T16:54:48.9789630Z with:
2023-01-31T16:54:48.9789806Z   tag_name: r-0012
2023-01-31T16:54:48.9790108Z   name: Release r-0012
2023-01-31T16:54:48.9791056Z   body: ## Info...
2023-01-31T16:54:48.9791637Z   draft: true
2023-01-31T16:54:48.9791819Z   files: ./sha.txt
2023-01-31T16:54:48.9792110Z   fail_on_unmatched_files: true
2023-01-31T16:54:48.9792533Z   token: ***
2023-01-31T16:54:48.9792705Z env:
2023-01-31T16:54:48.9792919Z   ENVIRONMENT: devel
2023-01-31T16:54:48.9793155Z   PROJECT: workflow-test
2023-01-31T16:54:48.9793383Z ##[endgroup]
2023-01-31T16:54:49.0629140Z 🤔 Pattern './sha.txt' does not match any files.
2023-01-31T16:54:49.0658257Z ##[error]⚠ There were unmatched files

I am on ubuntu runners, version: '2.301.1'
It this a bug, or am I doing something wrong?

My bad. Obviously I tried to attach a directory, not a file.


Cache in Redis

To take away yet another blocker to Continuous Deployment:
https://miro.com/app/board/o9J_ksK0wlg=/?moveToWidget=3074457361159683301&cot=14

[x] add a Redis container to docker-compose
[x] use ioredis for application cache

[x] Crossref
[ ] Biorxiv
~anywhere else?~

[ ] try to add it to the Helm chart

This saved query:
fields @message, app_message, `app_payload.doi.value`, app_timestamp
| filter `kubernetes.labels.app_kubernetes_io/instance` = 'sciety--prod'
| filter app_message like /bioRxiv cache (hit|miss)/
| fields strcontains(app_message, 'miss') * 100 + strcontains(app_message, 'hit')* 1 as type
| stats count(*) as multiple, sum(type) as type_sum by `app_payload.doi.value`, substr(app_timestamp, 0, 19)
| filter type_sum >= 100
| sort multiple desc

counts cache misses and cache hits on the same DOI in the same second according to app_timestamp. It estimates only 2 instances of deduplication happening in the last 7 days:
app_payload.doi.value,"substr(app_timestamp, 0, 19)",multiple,type_sum
10.1101/2<IP_ADDRESS>8695,2021-07-20T00:04:25,2,101
10.1101/2<IP_ADDRESS>9417,2021-07-23T17:08:59,2,101
10.1101/2<IP_ADDRESS>4652,2021-07-22T02:21:28,1,100
10.1101/2<IP_ADDRESS>0253,2021-07-22T00:29:48,1,100
10.1101/2<IP_ADDRESS>2175,2021-07-19T18:06:42,1,100
10.1101/2<IP_ADDRESS>57915,2021-07-20T12:31:44,1,100
10.1101/2<IP_ADDRESS>58634,2021-07-20T12:20:07,1,100
...

101 meaning one cache miss (100) and one cache hit (1).


8.2_from

like from <IP_ADDRESS>.R,  but not

:x: Build failed in 1m 49s
Build command:
mvn clean verify -B -e -Daudit -amd

:exclamation: No tests found!
Errors:
Filtered log (click to expand)

script returned exit code 1

:information_source:  This is an automatic message

:x: Build failed in 1m 55s
Build command:
mvn clean verify -B -e -Daudit -amd

:exclamation: No tests found!
Errors:
Filtered log (click to expand)

script returned exit code 1

:information_source:  This is an automatic message


python: uvloop: fix build on all platforms

uvloop builds currently fail on all platforms due to a file missing from the PyPi tarball. This temporarily patches the issue.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[X] NixOS
[X] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[X] Fits CONTRIBUTING.md.

typically we just disable flake8 testing because its of no use to us.

Makes sense - I've re-tested the update on darwin and nixos. Thanks for the super-quick comments!

Upstream will be skipping some tests as well https://github.com/MagicStack/uvloop/issues/318#issuecomment-649106020

Thanks @jonringer - looks like the upstream HEAD now has a fix for the test_sock_cancel_add_reader_race test, in response to your feedback. It would definitely be nice to drop the workaround in nixpkgs.
The issue here, though, is with a different failing test (which also has an unreleased upstream fix). So the obvious options seem to be:

drop both workarounds, pull in the latest dev commit, and trust that it's stable enough
add a temporary workaround for this second issue (the current PR contents)

The first option doesn't seem totally outrageous - the dev branch was named 0.15.0.dev0 in October, and subsequent commits look like stability fixes. But I have no idea what behaviour might have changed since the 0.14.0 release that other packages think they're depending on.
Which option would you prefer? I'm happy to update the PR if it's the first one (or a totally different solution).

Any comment @FRidh or @jonringer about how you’d like to proceed with this? Currently this just disables flake8 testing, but happy to change if preferable.

Looks like this was fixed in https://github.com/NixOS/nixpkgs/pull/95591 now, thanks for the PR though :)


feat: image id specific per segmentation brush opacity/alpha

Note: alpha is 0-255

:tada: This PR is included in version 3.15.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


<ElementTag.to_json> and <ElementTag.from_json> tags

Basically, I just want a way to convert a standard string to a JSON text component compliant string (for example, <&a>my string to {"text":"my string","color":"green"}) and back (for example, {"text":"my string","color":"green"} to <&a>my string).
<ElementTag.to_json> is possible using ComponentSerializer#toString(BaseComponent...).
I didn't do too much research so I can't quite say what <ElementTag.from_json> could use, but I'm fairly certain that there is a clean way of doing that.

Not sure what benefit that has (other than compensating for missing usages of the new FormattedTextHelper but there ya go