Design a way to mark blobs / trees in summary not to be included in snapshot

Design a way to mark blobs / trees in summary not to be included in snapshot.
This capability is useful for these flows:

Reduce snapshot payload by excluding all GC-based blobs. Such blobs are required only by summarizer and summarizer always works on top of latest summary, so correctness is not affected by storage deleting summary blobs eventually with deletion of old summaries.
We can leverage that capability as a proxy for tombstoning deleted data stores for GC

Related issues:

https://github.com/microsoft/FluidFramework/issues/4590
https://github.com/microsoft/FluidFramework/issues/6371
Design on deleting / tombstoning deleted data stores on GC

I think that would not be sufficient for GC tombstones, or rather any tombstone solution will need more tools.
Let's look at scenario:

Some violation happens (or it's part of main flow) and data store is loaded that is marked that way by client A
This client mutates it by issuing ops.
Summarizer summarizes it.
Over time, original summary with original blobs is deleted by ODSP.

A client that follows handle to load that data store (because it's now part of graph) needs to

Refresh its knowledge of latest snapshot
Load snapshot tree + relevant blobs
process all ops on top of it till latest seq number (that one we almost have, as clients keep ops around).

Reason for this flow - a client may be offline for a while, so it can't assume it can use original summary tree to load missing blobs from it by the time it sees an op for such data store.
Alternative and simpler (but more impactful on COGS) solution - we load all such blobs in one http request that follows loading summary. Such that each client has these blobs to allow existing data store delay-loading logic to work. But that has complication - snapshot caching, that would need to be adjusted for this flow. I'd recommend not to go that route.

We still need this functionality for gc blobs, so I want to keep it in the list of things we are tracking but using it for tombstones is not easy - see above.

And for trees, should unreferenced automatically mean - not include into snapshot?
Is all the usage the same, where we can use unreferenced and "not bring in snapshot" as one and the same (at some point in future)?

That issue is a peer of https://github.com/microsoft/FluidFramework/issues/6371

Tracked by https://dev.azure.com/fluidframework/internal/_workitems/edit/418


Add files via upload

@kristenpinkert This domain is blacklisted because the college allows to register an email address there even if you are not a student of the college. If you are the student of this college, please apply with official document ("OFFICIAL DOCUMENT" tab at https://www.jetbrains.com/shop/eform/students).


Error running devappserver of existing project on 2017.2.5

java.lang.NoSuchMethodError: com.google.cloud.tools.appengine.cloudsdk.CloudSdk$Builder.javaHome(Ljava/nio/file/Path;)Lcom/google/cloud/tools/appengine/cloudsdk/CloudSdk$Builder;
I triggered this by opening an old wizard based standard project and trying to run the local dev server configuration.

Google Cloud SDK [148.0.1]
Platform: [Windows, x86_64] ('Windows', 'sellout64', '8', '6.2.9200', 'AMD64', 'Intel64 Family 6 Model 58 Stepping 9, Ge
nuineIntel')
Python Version: [2.7.6 (default, Nov 10 2013, 19:24:24) [MSC v.1500 64 bit (AMD64)]]
Python Location: [C:\python27_x64\python.exe]
Site Packages: [Disabled]
Installation Root: [C:\Users\patrick\AppData\Local\Google\Cloud SDK\google-cloud-sdk]
Installed Components:
core: [2017.03.24]
app-engine-python: [1.9.50]
bq-win: [2.0.24]
kubectl: []
core-win: [2017.03.17]
app-engine-java: [1.9.50]
gcloud: []
windows-ssh-tools: [2016.05.13]
gcloud-deps-windows-x86_64: [2017.02.21]
beta: [2016.01.12]
gsutil: [4.23]
bq: [2.0.24]
gcloud-deps: [2017.03.17]
kubectl-windows-x86_64: [1.5.4]
gsutil-win: [4.23]

When I run from source I can't reproduce.  This is only from the alpha channel

I'm noticing there are dual versions of common-lib and appengine-plugins-core in our latest release zips. I'm going to pull them.

#1425


Async vrf evaluations

This PR restructures vrf evaluations so that the async scheduler has an opportunity to run other code in-between. This should prevent the long async cycles we otherwise see due to this, at the expense of slightly slower vrf evaluations.
This also removes a log message that we were emitting for every evaluation. We're currently not making any use of it, and removing it makes a significant difference on my machine: I'm seeing an ~8% speedup locally with this PR, and that testing doesn't account for further writes to disk etc. involved with full daemon logging.
Checklist:

[ ] Document code purpose, how to use it

Mention expected invariants, implicit constraints

[ ] Tests were added for the new behavior

Document test purpose, significance of failures
Test names should reflect their purpose

[ ] All tests pass (CI will check this if you didn't)
[ ] Serialized types are in stable-versioned modules
[ ] Does this close issues? List them:

Stale, closing.


Fix Dialogflow Enterprise documentation link.

Fixes #3.
Fixes #4.

Codecov Report

Merging #9 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master       #9   +/-   ##
=======================================
  Coverage   94.28%   94.28%
=======================================
  Files          16       16
  Lines         525      525
  Branches       20       20
=======================================
  Hits          495      495
  Misses         12       12
  Partials       18       18

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 31e4160...992e791. Read the comment docs.

Suggested a couple of quick changes. Otherwise, looks good


Fix the bug in issue 845

I check how chrome convert the html which have three nested  tag, it is very strange. Since  is illegal, the result is unpredictable. It is better to write a method to reconstruct the dom tree to avoid nested  tag, rahter than modify the process of constructing dom tree.  So I write a class to handle illegal dom tree. However, there is only one method which handle nested  tag in this class now. I also add some test case.

Thanks - went with another PR #1517 that implements the spec's adoption agency algo


Uncertainty estimate

Do we want bootstrapped estimates of uncertainty for the different metrics? If so, how do we want to compute it for the different curves ?

I think we need to be very careful about using bootstrapping in this package to estimate uncertainty.
In this package, we only have access to a fixed set of predictions on a fixed test set.
We can resample from that test set with replacement to assess what impact the test set selection has on the final model performance, but given that this is for a generalized benchmarking setting, not a true deployment of a fixed, frozen model, I think such estimates will significantly underestimate performance variance because it will not take into account the variance caused by sampling different training populations and having different random initializations or processes during model training.
For example, suppose our metric were accuracy. As accuracy is just the expectation that a random draw from the test set will have a correct prediction, bootstrapping over a fixed set of test set predictions will do nothing for us other than simulate taking the expectation of the rate of a binomial random variable.
In reality, as users of this benchmark will always be imagining taking these results and applying them on their local datasets, the variance those users should expect will be much higher as it will take into account resampling the training set, re-training the model, etc.

Tagging @EthanSteinberg and @kamilest for discussion as well


Merge pull request #8 from cgkeenan/master

Merge pull request #7 from cgkeenan/my-slide

comment


Remove pulse defaults for old fake 20q backends

Summary
The pulse defaults payload for the almaden, johannesburg, and singapore
fake backends are collectively taking up > 50% of the total disk
footprint of an installed copy of Qiskit. This is because the pusle
defaults for these backends predate the use the of parameterized pulse
definitions and contain arrays of sampled waveforms for the pulse
definitions. This exceedingly large file size slows down the entire
package and and makes it slower to install. When weighing the potential
value these defaults files provide for simulating and compiling with
pulse awareness for long retired devices against the real cost of
having the files included in the package there isn't a reason to keep
the files in the tree any longer. This commit removes the pulse
defaults files for these fake backends to greatly shrink the installed
package size of qiskit-terra.
Details and comments

Some of the tests explicitly use the Almaden, Johannesburg and Singapore fakes in part because they had pulse defaults - those will need changing over.

ok, I've updated the tests to no longer rely on these backends having pulse defaults present. Hopefully things will pass now.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

10 of 10 (100.0%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.001%) to 84.303%

Totals

Change from base Build<PHONE_NUMBER>:
-0.001%

Covered Lines:
57822

Relevant Lines:
68588

💛  - Coveralls

The tutorials failure should be fixed once: https://github.com/Qiskit/qiskit-tutorials/pull/1356 merges

Looks good to me - the diff looks cleaner without the +1,800 file as well! I'm never very sure if we should be making assertions about the exact forms of those giant reprs anyway.

Heh, that took me longer to track down than I care to admit. My default editor config was changing the whitespace and I thought I somehow broke the output as part of the unrelated change which is why I changed the backend and added that file. But once I figured out what I did wrong it was easy to fix.


feat(codemagic_app_preview): cancel if non pull request

Closes #39

⬇ Generated builds by Codemagic for commit bbbf19a9b7ef48b2f57261708b42d5095a642612 ⬇
This is a custom message.

iOS
Android
macOS

  Download link
  Download link
  Download link


Websocket server yield nullpointer exception if address is missing

Version:
1.8.1

Platform:
Darwin MacBook-Pro-3.local 23.0.0 Darwin Kernel Version 23.0.0:
Node 20

Subsystem:
@libp2p/websocket

Severity:
Medium
Description:
Sometimes if I start and close nodes randomly and connect them simultaneously I get a null pointer exception inside the it-ws lib. I create a issue for that here.  I wonder though if it is something on the libp2p side of things that is triggering this or we have to wait for an update to land to it-ws before that. The culprit is that the @types/ws are wrong and does not match the actual return types of ws lib.
Steps to reproduce the error:
Don't know yet since this is hard to reproduce accuretly

https://github.com/alanshaw/it-ws/pull/118 has shipped so this should be fixed now, please re-open if you're still seeing problems.


Attempting to fix issues #3 and #4

This should either fix #3 and #4 or will at least give me useful errors to work with. Could @Julusian test this please to see if the issues still exist.

I've added a check for if the device is a mobile or tablet since they can't have their volumes changed via the API. There might be other devices that can't be changed like a cast recipient but I can't test them right now.


Repository Add-on error

Hello,
When i add the url of repository on my Home Assitant, i have an error below 👍

I'm in release 2022.11.2

Thx for help.

Please read this link: https://github.com/ksheumaker/homeassistant-apsystems_ecur/issues/138
Feel free to add or reopen this issue if this does not solve your problem.


docs: fix the formatting of AioSoma in the docs

Signed-off-by: Avi Miller<EMAIL_ADDRESS>

@all-contributors please add @Djelibeybi for code, doc, maintenance, test


Added parameters user_agent & extra_headers

I added 2 parameters to the function ws_connect: user_agent_header & extra_headers. please check .

Hi @Tapanhaz, thank you.
It seems these changes hang up the tests. If you fix it I will merge it.

My mistake. Added the parameters to WSProtocol but didn't update the ws_create_server code. Now I corrected that mistake. Please check.


Delay function does not work

Hello @vladimir-yuldashev ,
here is the problem with delay function. The problem is that delay queue has a static value "x-dead-letter-exchange" set the same as "x-dead-letter-routing-key".
Simple example:
exchange = amq.direct
queues assigned to the same exchange (amq.direct) = [test, test_deferred_10]
The problem is with test_deferred_10 argument x-dead-letter-exchange, because once its done with TTL it searches for this exchange "test" and not he original one amq.direct. All our queues are assigned to amq.direct and thats why it cant find the queue defined in x-dead-letter-routing-key
It looks like this PL #56 should help

Released in 5.2


Gift entry: Configure Gift Entry

https://powerofus.force.com/s/article/NPSP-Configure-Gift-Entry
https://docs.google.com/document/d/1JI7_R4GepIuwaPg-FA6Rsu8O3x1tKsSgDytOFjSiLQo/edit?usp=sharing

@Billflorio please assign this to me. Also, I need a Gdoc created for me.

@barbarachristensen please make a google doc for this script

Added above! Please request access and I'll add you!

Reviewed. Propose to add 2 links in the description of the video, since we are referencing them in the video so that people have quick access to those resources.

@barbarachristensen this one also looks good!


Initial commit of local vs non-local TN demo

Added TN1_demo_local_vs_non-local_random_circuits notebook and associated permuted_circuit image.
First commit of TN1 demo for structured vs unstructured circuits.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Friendly bump.

Looks good to me. Adding @kshyatt-aws for a second opinion.

LGTM. Sorry it took me so long to get to this, I somehow didn't get pinged.


Implement missing pieces from Trackable interface for TF-based variable.

Some tf.Variable implementations (such as TPUDistributedVariable) don't override _restore_from_tensors/_serialize_to_tensors since they rely on the _gather_saveables_for_checkpoint interface, which causes Keras to fail (with NotImplementedError) when creating a tf.Checkpoint in a distributed environment for example. Here, we add a simple workaround in case this interface is not implemented.
Additonally, we also add the _copy_trackable_to_cpu interface to support async checkpointing on TF.

Codecov Report
Attention: 9 lines in your changes are missing coverage. Please review.

Comparison is base (aa6e54a) 78.91% compared to head (e7fea6d) 78.89%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #18749      +/-   ##
==========================================
- Coverage   78.91%   78.89%   -0.02%
==========================================
  Files         336      336
  Lines       33974    33984      +10
  Branches     6656     6656
==========================================
+ Hits        26811    26813       +2
- Misses       5584     5592       +8
  Partials     1579     1579

Flag
Coverage Δ

keras
78.79% <25.00%> (-0.02%)
:arrow_down:

keras-jax
61.74% <8.33%> (-0.02%)
:arrow_down:

keras-numpy
56.24% <8.33%> (-0.02%)
:arrow_down:

keras-tensorflow
63.50% <25.00%> (-0.02%)
:arrow_down:

keras-torch
64.34% <8.33%> (-0.02%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files
Coverage Δ

keras/backend/tensorflow/core.py
91.55% <25.00%> (-4.97%)
:arrow_down:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Docker-Compose Updates

What's New?

Remove legacy deps
Cleanup gitignore
Update ubuntu_example.sh
Add deprecation warnings / migration instructions.

Coverage remained the same at 93.87% when pulling 62b1e33758ee1050fb7d6accb1f8e8e2122d646c on RickCarlino:thursday into 46223c7686e133273f7b082ace9187f24322faaf on FarmBot:staging.

Coverage remained the same at 93.87% when pulling 62b1e33758ee1050fb7d6accb1f8e8e2122d646c on RickCarlino:thursday into 46223c7686e133273f7b082ace9187f24322faaf on FarmBot:staging.


Error running aws-shell in virtualenv

When installing the latest version of awscli and aws-shell in a virtualenv on Debian 9, running aws-shell fails with an error creating the autocomplete index.
First run, creating autocomplete index...
Traceback (most recent call last):
File "/home/aws/aws/lib/python3.5/site-packages/awsshell/utils.py", line 84, in file_contents
with open(filename, mode) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/aws/.aws/shell/cache/completions-1.16.14.json'
The Debian packages for awscli and aws-shell work.  Here are the versions for those.
aws-shell/stable 0.1.1-2
awscli/stable 1.11.13-1
Details attached for Debian 9.  I have also tried Debian 10 and Fedora 28 with similar results.
python3-virtualenv-aws-debian-9.txt

I get the same error in a python 3.7 virtualenv on Mac OS X 10.12.6.

Seems to be the same as https://github.com/awslabs/aws-shell/issues/208 which was closed even though others are having the same issue

Having the same issue

This might be the same issue, although it's a different FileNotFound than the one above:
This error traceback appears a few seconds after launching the shell:
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/site-packages/awsshell/makeindex.py", line 88, in write_doc_index
    do_write_doc_index(db, help_command, close_db_on_finish=should_close)
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/site-packages/awsshell/makeindex.py", line 93, in do_write_doc_index
    _index_docs(db, help_command)
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/site-packages/awsshell/makeindex.py", line 111, in _index_docs
    _index_docs(db, sub_help_command)
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/site-packages/awsshell/makeindex.py", line 108, in _index_docs
    text_docs = render_docs_for_cmd(sub_help_command)
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/site-packages/awsshell/makeindex.py", line 117, in render_docs_for_cmd
    help_command(None, None)
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/site-packages/awscli/customizations/commands.py", line 358, in __call__
    docevents.generate_events(self.session, self)
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/site-packages/botocore/docs/bcdoc/docevents.py", line 46, in generate_events
    help_command=help_command)
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/site-packages/botocore/session.py", line 671, in emit
    return self._events.emit(event_name, **kwargs)
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/site-packages/botocore/hooks.py", line 356, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/site-packages/botocore/hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/site-packages/botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/site-packages/awscli/customizations/commands.py", line 376, in doc_synopsis_start
    if not help_command.synopsis:
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/site-packages/awscli/customizations/commands.py", line 326, in synopsis
    return self._get_doc_contents('_synopsis')
  File "/home/charlie/miniconda3/envs/amazon/lib/python3.7/site-packages/awscli/customizations/commands.py", line 347, in _get_doc_contents
    with _open(doc_path) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/charlie/miniconda3/envs/amazon/lib/python3.7/site-packages/awscli/examples/emr/create-cluster-synopsis.txt'

Current python environment:
asn1crypto==0.24.0
aws-shell==0.2.1
awscli==1.16.179
boto3==1.9.169
botocore==1.12.169
certifi==2019.3.9
cffi==1.12.3
colorama==0.3.9
configobj==5.0.6
cryptography==2.7
docutils==0.14
idna==2.8
jmespath==0.9.4
powerline-status==2.7
prompt-toolkit==1.0.16
pyasn1==0.4.5
pycparser==2.19
Pygments==2.4.2
pyOpenSSL==19.0.0
PySocks==1.7.0
python-dateutil==2.8.0
PyYAML==5.1
rsa==3.4.2
s3transfer==0.2.0
six==1.12.0
urllib3==1.25.3
wcwidth==0.1.7

same here, File not found for something related to EMR

same here.
aws> Exception in thread Thread-1:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/awsshell/makeindex.py", line 88, in write_doc_index
    do_write_doc_index(db, help_command, close_db_on_finish=should_close)] Focus: cli  [F10] Exit
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/awsshell/makeindex.py", line 93, in do_write_doc_index
    _index_docs(db, help_command)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/awsshell/makeindex.py", line 111, in _index_docs
    _index_docs(db, sub_help_command)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/awsshell/makeindex.py", line 108, in _index_docs
    text_docs = render_docs_for_cmd(sub_help_command)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/awsshell/makeindex.py", line 117, in render_docs_for_cmd
    help_command(None, None)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/awscli/customizations/commands.py", line 358, in __call__
    docevents.generate_events(self.session, self)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/botocore/docs/bcdoc/docevents.py", line 46, in generate_events
    help_command=help_command)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/botocore/session.py", line 672, in emit
    return self._events.emit(event_name, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/botocore/hooks.py", line 356, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/botocore/hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/awscli/customizations/commands.py", line 376, in doc_synopsis_start
    if not help_command.synopsis:
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/awscli/customizations/commands.py", line 326, in synopsis
    return self._get_doc_contents('_synopsis')
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/awscli/customizations/commands.py", line 347, in _get_doc_contents
    with _open(doc_path) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/awscli/examples/emr/create-cluster-synopsis.txt'


Upload images into FreeCEN2

This is not going to happen until we have a license for the images.
I sugges that we put this on ice.
Geoff


Please provide your GPG public key that you signed the release tarballs with

Hi @zimmerle,
I am trying to validate the tarball modsecurity-nginx-v1.0.2.tar.gz from https://github.com/SpiderLabs/ModSecurity-nginx/releases with the .asc signature provided.
The key apparently used to generate the signature is 190EFACCA1E9FA466A8ECD9CE6DFB08CE8B11277
user@example:~$ gpg --verify ModSecurity-nginx-1.0.2.tar.gz.asc ModSecurity-nginx-1.0.2.tar.gz
gpg: Signature made Mon Jun  7 21:59:15 2021 AEST
gpg:                using DSA key 190EFACCA1E9FA466A8ECD9CE6DFB08CE8B11277
gpg: Can't check signature: No public key

I tried looking for your GPG key, and ran into some problems:

The key at https://keys.openpgp.org/search?q=190EFACCA1E9FA466A8ECD9CE6DFB08CE8B11277 contains no user ID, so I can't import it:

user@example:~$ gpg --import 190EFACCA1E9FA466A8ECD9CE6DFB08CE8B11277
gpg: key E6DFB08CE8B11277: new key but contains no user ID - skipped
gpg: Total number processed: 1
gpg:           w/o user IDs: 1

Most of the other keyservers don't turn up your key or aren't online

I finally turned up a key at https://pgp.mit.edu/pks/lookup?op=get&search=0xE6DFB08CE8B11277 - but this key apparently is not the same as the one you used:

user@example:~$ gpg --import felipe.asc
gpg: key E6DFB08CE8B11277: 38 signatures not checked due to missing keys
gpg: key E6DFB08CE8B11277: public key "Felipe Zimmerle da Nobrega Costa<EMAIL_ADDRESS>imported
gpg: Total number processed: 1
gpg:               imported: 1
gpg: no ultimately trusted keys found

user@example:~$ gpg --verify ModSecurity-nginx-1.0.2.tar.gz.asc ModSecurity-nginx-1.0.2.tar.gz
gpg: Signature made Mon Jun  7 21:59:15 2021 AEST
gpg:                using DSA key 190EFACCA1E9FA466A8ECD9CE6DFB08CE8B11277
gpg: BAD signature from "Felipe Zimmerle da Nobrega Costa<EMAIL_ADDRESS>[unknown]

Can you please provide the correct public key here or perhaps in the README? That would be very much appreciated.
Thanks!

Furthermore, https://github.com/zimmerle.gpg shows 'Note: The keys with the following IDs couldn't be exported and need to be reuploaded E6DFB08CE8B11277'.
Would be great to actually validate the signatures you're publishing, thanks :)

Hi @mig5 -- @martinhsv  is the one who is publishing the releases.
@martinhsv  can you have a look at this?

The key in question actually belongs to @zimmerle .
If that key can no longer be made available, then the only thing I can think of that I could do is to create a new .asc file for ModSecuritiny-nginx v1.0.2 using my own key.

Yes please - or at least, for the next release. I don't mind, but obviously there's just no point publishing the .asc if the public key is not available to validate it.
Someone must have the key in order to upload those .asc files to the release. So any reason not to just export that public key? But yeah if it's not strictly speaking 'your' key, sounds like something weird already happening. Maybe time for a new key :)

I'm pretty sure that key was valid and available back in June when the release occurred. But events in the six months since then have obviously changed the situation.
New releases since Oct. 1 (including the recent releases of ModSecurity 3.0.6 and 2.9.5) use, and will, use a different key that is available.

New releases since Oct. 1 (including the recent releases of ModSecurity 3.0.6 and 2.9.5) use, and will, use a different key that is available.

Yeah, the main ModSecurity packages I was able to validate. But not this nginx one. But you're publishing the .asc files here, so that person must have the private key.
Is the person who made the .asc files not able to simply run gpg --export -a [key id] and paste the resulting public key here? That's really all I'm asking.. because the public keys I can find online don't let me import it, but maybe one that you export and provide here, might?

@mig5
you may want to try this -
gpg --keyserver hkps://keyserver.ubuntu.com --recv-keys E6DFB08CE8B11277

reference

@zimmerle  I can import that key, but it apparently is not the key that signed the package.
user@disp949:~$ gpg --keyserver hkps://keyserver.ubuntu.com --recv-keys E6DFB08CE8B11277
gpg: key E6DFB08CE8B11277: public key "Felipe Zimmerle da Nobrega Costa<EMAIL_ADDRESS>imported
gpg: Total number processed: 1
gpg:               imported: 1

user@disp949:~$ wget -O ModSecurity-nginx-1.0.2.tar.gz.asc https://github.com/SpiderLabs/ModSecurity-nginx/releases/download/v1.0.2/modsecurity-nginx-v1.0.2.tar.gz.asc
--2022-01-06 08:20:59--  https://github.com/SpiderLabs/ModSecurity-nginx/releases/download/v1.0.2/modsecurity-nginx-v1.0.2.tar.gz.asc
[...]

user@disp949:~$ wget -O ModSecurity-nginx-1.0.2.tar.gz https://github.com/SpiderLabs/ModSecurity-nginx/archive/v1.0.2.tar.gz
--2022-01-06 08:20:28--  https://github.com/SpiderLabs/ModSecurity-nginx/archive/v1.0.2.tar.gz

user@disp949:~$ gpg --verify ModSecurity-nginx-1.0.2.tar.gz.asc ModSecurity-nginx-1.0.2.tar.gz
gpg: Signature made Mon 07 Jun 2021 09:59:15 PM AEST
gpg:                using DSA key 190EFACCA1E9FA466A8ECD9CE6DFB08CE8B11277
gpg: BAD signature from "Felipe Zimmerle da Nobrega Costa<EMAIL_ADDRESS>[unknown]

The signature is bad - therefore, the released package cannot have been signed with that key.
Should we consider the tarball compromised somehow? That's what the signatures are there for..

Not sure what I did wrong but it works now, I redownloaded the tarball.. thanks for providing the other way to get the key.

 The script I was using was downloading a tag tarball rather than the actual 'release'
https://github.com/SpiderLabs/ModSecurity-nginx/archive/v$(MODSECURITY_REF).tar.gz
should've been
https://github.com/SpiderLabs/ModSecurity-nginx/releases/download/v$(MODSECURITY_REF)/modsecurity-nginx-v$(MODSECURITY_REF).tar.gz
The tagged tarball, and the release file, have different sha256sum and indeed the tagged tarball is not signed by the key.
Sorry for the distraction.

no worries. thank you for helping sort that out.


Re-enable ecc-vscc and ercc-vscc

This PR fixes #250 and re-enables ecc and ercc custom validation.

What this PR does / why we need it:
This PR fixes #250 and re-enables ecc and ercc custom validation.

Change core.yaml to point to plugins/*-vscc.so
Set ercc-vscc in ercc chaincode definition in our peer wrapper
Set ecc-vscc for fpc chaincodes in integration tests
add workaround in ercc-vscc to fix missing "ecc_" namespace prefix
for fpc chaincodes; and fix mrenclave retrieval
remove utils depency from ecc-vscc as it caused errors when peer
loads the plugin
improved debug logging in ecc-vscc
add workaround in ecc-vscc for fix missing "ecc_" namespace prefix

Which issue(s) this PR fixes:

Fixes #250
Special notes for your reviewer:
Does this PR introduce a user-facing changes and/or breaks backward compatability?:

Looks nice to me. Does CI Run this check?

Yes! Our custom vsccs are now used in integration tests.

Current version works fine in HW mode and i think it resolved the validation questions raised in the discussion. However, i think at least one comment is misleading/wrong and we should make sure the corresponding changes are also in TLCC (probably via a separate issue?). In general, for such security-sensitive code i think a bit more comments which connects the dots could be useful (although i agree that much (but not all) of the code will change with ercc refactoring ...)

I created #373 to track corresponding todos on TLCC and also addressed your concerns regarding the comments.


[Bug]: buttongroup - Invalid url displayed while navigating from debugger for buttongroup widget

Is there an existing issue for this?

[X] I have searched the existing issues

Description
Add buttongroup and add action - setinetrval -console log
Verify the log and navigate to button group from link provided in debugger - error is displayed
https://github.com/appsmithorg/appsmith/assets/80756091/49708404-423b-4209-ba95-0fd783f24454
Steps To Reproduce

Add buttongroup
Add Setinterval => setinterval(()=>{console.log("test")}) for Add
Click on add and verify log in debugger
Click on buttongroup link to navigate to button group

Error is displayed
Public Sample App
No response
Environment
Production
Issue video log
No response
Version
cloud

Findings after triaging the issue -
We have the following types of links in the debugger mentioned at appsmith/app/client/src/components/editorComponents/Debugger/EntityLink.tsx
const entityTypeLinkMap = {
  [ENTITY_TYPE.WIDGET]: WidgetLink,
  [ENTITY_TYPE.ACTION]: ActionLink,
  [ENTITY_TYPE.DATASOURCE]: DatasourceLink,
  [ENTITY_TYPE.JSACTION]: JSCollectionLink,
};

In the case of a regular widget action that is logged, we have EntityLink's prop type set to ENTITY_TYPE.WIDGET

However, in case of this log triggered by an action, we have EntityLink's prop type set to ENTITY_TYPE. JSACTION

The type should have been ENTITY_TYPE.WIDGET here since we want to navigate to the widget, not a JS object. Because of this, we are appending "jsObjects" in the URL instead of "widgets".

Hey @bharath31, please take a look at this issue. We have triaged it and posted our findings above. Thanks!

PS - The issue can also be replicated on other widgets such as the button or a checkbox. It does not only exist in the button group. Video log here - https://lcixo7xrxt.vmaker.com/record/FlgzdtHYeWibJZIZ

@dhruvikn thanks for sharing your findings!
@Rishabh-Rathod could you please investigate the root cause for this issue?

this is not reproducible anymore


Integrate global tracking accuracy into eval pipeline

Currently the eval pipeline outputs an hdf5 file with motmetrics. See models/gtr_runner/on_test_epoch_end() for implementation. Proposing to add new global tracking accuracy metric into this eval file. Will need to update /inference/metrics.py to calculate this metric. Current working implementation is in /vast/mustafa/global_tracking_accuracy.ipynb

In order to do this I would just modify metrics.get_pymotmetrics to compute the metric and add it to the metric dataframe that it returns


Multiple indexes under one es{} block

Siva,
I have around 30 indexes and i need to display the logs of each index separately. As i read, one need to write multiple es{ } blocks under index_patterns at logtrail.json.
Can we have multiple index defined at the one es { } block. i haveother parameters of es{ } block remaining the same and this I am looking to avoid writing multiple blocks.
Something like this as below where in faultmanagement-*  , telemetry-* etc are the index names I have
"es": {
"default_index": "faultmanagement-,telemetry-,supervision-*"
},

Hi Rahu,
Currently you cannot configure comma separated index patterns in logtrail.json.

Is there any way to display indexes seperatly?


Dataset Viewer issue for boris/gis_filtered

Link
https://huggingface.co/datasets/boris/gis_filtered/viewer/boris--gis_filtered/train
Description
When I try to access this from the website I get this error:
Status code:   400
Exception:     ClientResponseError
Message:       401, message='Unauthorized', url=URL('https://huggingface.co/datasets/boris/gis_filtered/resolve/80b805053ce61d4eb487b6b8d9095d775c2c466e/data/train/0000.parquet')
If I try to load with code I also get the same issue:
        dataset2_train=load_dataset("boris/gis_filtered", use_auth_token=os.environ["HF_TOKEN"],split="train")
        dataset2_validation=load_dataset("boris/gis_filtered", use_auth_token=os.environ["HF_TOKEN"], split="validation")

Owner
No

Yes, this dataset is "gated": you first have to go to https://huggingface.co/datasets/boris/gis_filtered and click "Access repository" (if you accept to share your contact information with the repository authors).

I already did that, it returns error when using streaming

Oh, sorry, I misread. Looking at it. Maybe @huggingface/datasets or @SBrandeis

I could reproduce the error, even though I provided my token and accepted the gate form

This is indeed a bug in datasets. Parquet datasets in gated/private repositories can't be streamed properly, which caused the viewer to fail. I opened a PR at https://github.com/huggingface/datasets/pull/4608


Remove version: sourcegit-scm.SourceGit version 8.29

Checklist for Pull Requests

[x] Have you signed the Contributor License Agreement?
[ ] Is there a linked Issue?

Manifests

[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] This PR only modifies one (1) manifest
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.6 schema?

Note: <path> is the directory's name containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

URL: https://github.com/sourcegit-scm/sourcegit/releases/download/v8.29/sourcegit_8.29.win-arm64.zip
OK
(Automated message - build 923)

https://github.com/microsoft/winget-pkgs/pull/185471#issuecomment-2436036384

URL: https://github.com/sourcegit-scm/sourcegit/releases/download/v8.29/sourcegit_8.29.win-arm64.zip
Status Code: 200
(Automated message - build 897)


Resources at edu.fiware.org are unavailable

Hello there.
My team has recently deployed the Business Ecosystem enabler.
However, when trying to follow these docs, we noticed http://edu.fiware.org/ appears to be offline.
Is this a temporary issue?
Thank you for your time.

Still a problem until today. Did the docs change in domain only?
Otherwise a complete removal of any linkage to edu.fiware would be good.


bug: autogptq doesnt work (cant download model)

LocalAI version:
Docker Image: quay.io/go-skynet/local-ai:master-cublas-cuda12-ffmpeg
Environment, CPU architecture, OS, and Version:
Running in TrueNAS Scale Kubernetes (k3s) with a NVidia Tesla P40 in the container.
# uname -a
Linux localai-ix-chart-f8bbbb7c7-x6xx9 6.1.42-production+truenas #2 SMP PREEMPT_DYNAMIC Mon Aug 14 23:21:26 UTC 2023 x86_64 GNU/Linux
# nvidia-smi
Tue Aug 22 16:36:27 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.54.03              Driver Version: 535.54.03    CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  Tesla P40                      Off | 00000000:23:00.0 Off |                    0 |
| N/A   28C    P8              10W / 250W |      2MiB / 23040MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+
# cat /proc/cpuinfo |grep "model name" | nl
     1  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
     2  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
     3  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
     4  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
     5  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
     6  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
     7  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
     8  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
     9  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
    10  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
    11  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
    12  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
    13  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
    14  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
    15  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
    16  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
    17  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
    18  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
    19  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
    20  model name      : Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
# cat /proc/meminfo  | grep Mem
MemTotal:       32701568 kB
MemFree:        18305148 kB
MemAvailable:   18767368 kB

Describe the bug
AutoGPTQ added by #871 doesn't work in upstream container. Also tried exllama and gives a linker error for CudaSetDevice.
To Reproduce
curl $LOCALAI/v1/chat/completions -H "Content-Type: application/json" -d '{
     "model": "TheBloke/orca_mini_v2_13b-GPTQ",
     "messages": [{"role": "user", "content": "### System:\nYou are an AI assistant that follows instruction extremely well. Help as much as you can.\n \n### User: \ntell me about AI \n### Response:"}],
     "backend": "autogptq", "model_base_name": "orca_mini_v2_13b-GPTQ-4bit-128g.no-act.order"
}'
{"error":{"code":500,"message":"could not load model (no success): Unexpected err=FileNotFoundError('Could not find model in TheBloke/orca_mini_v2_13b-GPTQ'), type(err)=\u003cclass 'FileNotFoundError'\u003e","type":""}}

Also tried with a local model:
name: deadbeef
backend: autogptq
parameters:
  model: wizardlm-13b-v1.1-GPTQ-4bit-128g.no-act.order.safetensors

curl $LOCALAI/v1/chat/completions -H "Content-Type: application/json" -d '{
     "model": "deadbeef",
     "messages": [{"role": "user", "content": "Give me a HTTP REST server made in rust that uses sqlite."}],
     "temperature": 0.9
   }' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   655  100   489  100   166    115     39  0:00:04  0:00:04 --:--:--   154
{
  "error": {
    "code": 500,
    "message": "could not load model (no success): Unexpected err=OSError(\"wizardlm-13b-v1.1-GPTQ-4bit-128g.no-act.order.safetensors is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'\\nIf this is a private repository, make sure to pass a token having permission to this repo with `use_auth_token` or log in with `huggingface-cli login` and pass `use_auth_token=True`.\"), type(err)=<class 'OSError'>",
    "type": ""
  }
}

Expected behavior

Logs
2023-08-22 09:32:17.702437-07:004:32PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:TheBloke/orca_mini_v2_13b-GPTQ ContextSize:512 Seed:0 NBatch:512 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:8 LibrarySearchPath: RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/models/TheBloke/orca_mini_v2_13b-GPTQ Device: UseTriton:false ModelBaseName:orca_mini_v2_13b-GPTQ-4bit-128g.no-act.order UseFastTokenizer:false PipelineType: SchedulerType: CUDA:false CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0}
2023-08-22 09:32:18.184608-07:004:32PM DBG GRPC(TheBloke/orca_mini_v2_13b-GPTQ-<IP_ADDRESS>:35307): stderr
Downloading (…)okenizer_config.json:   0%|          | 0.00/727 [00:00<?, ?B/s]
Downloading (…)okenizer_config.json: 100%|██████████| 727/727 [00:00<00:00, 126kB/s]
2023-08-22 09:32:20.337018-07:004:32PM DBG GRPC(TheBloke/orca_mini_v2_13b-GPTQ-<IP_ADDRESS>:35307): stderr
Downloading tokenizer.model:   0%|          | 0.00/500k [00:00<?, ?B/s]
Downloading tokenizer.model: 100%|██████████| 500k/500k [00:01<00:00, 400kB/s]
Downloading tokenizer.model: 100%|██████████| 500k/500k [00:01<00:00, 400kB/s]
2023-08-22 09:32:20.789331-07:004:32PM DBG GRPC(TheBloke/orca_mini_v2_13b-GPTQ-<IP_ADDRESS>:35307): stderr
Downloading (…)cial_tokens_map.json:   0%|          | 0.00/435 [00:00<?, ?B/s]
Downloading (…)cial_tokens_map.json: 100%|██████████| 435/435 [00:00<00:00, 202kB/s]
2023-08-22 09:32:20.792331-07:004:32PM DBG GRPC(TheBloke/orca_mini_v2_13b-GPTQ-<IP_ADDRESS>:35307): stderr You are using the legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. This means that tokens that come after special tokens will not be properly handled. We recommend you to read the related pull request available at https://github.com/huggingface/transformers/pull/24565
2023-08-22 09:32:21.126540-07:004:32PM DBG GRPC(TheBloke/orca_mini_v2_13b-GPTQ-<IP_ADDRESS>:35307): stderr
Downloading (…)lve/main/config.json:   0%|          | 0.00/812 [00:00<?, ?B/s]
Downloading (…)lve/main/config.json: 100%|██████████| 812/812 [00:00<00:00, 151kB/s]
2023-08-22 09:32:21.646001-07:004:32PM DBG GRPC(TheBloke/orca_mini_v2_13b-GPTQ-<IP_ADDRESS>:35307): stderr
Downloading (…)quantize_config.json:   0%|          | 0.00/158 [00:00<?, ?B/s]
Downloading (…)quantize_config.json: 100%|██████████| 158/158 [00:00<00:00, 74.0kB/s]
2023-08-22 09:32:21.818950-07:00[<IP_ADDRESS>]:36522  500  -  POST     /v1/chat/completions

did you tried with:
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
     "model": "TheBloke/orca_mini_v2_13b-GPTQ",
     "messages": [{"role": "user", "content": "### System:\nYou are an AI assistant that follows instruction extremely well. Help as much as you can.\n \n### User: \ntell me about AI \n### Response:"}],
     "backend": "autogptq", "model_base_name": "orca_mini_v2_13b-GPTQ-4bit-128g.no-act.order"
}'

?

ah just saw you tried, my bad - it looks like a downloading issue to me. For local files, only exllama now works with local folders - what error you get there?

ah just saw you tried, my bad - it looks like a downloading issue to me. For local files, only exllama now works with local folders - what error you get there? also, did you tried with another model?

For exllama the error seens like a incompatible cuda version on the container:
ImportError: /usr/local/lib/python3.9/dist-packages/exllama_ext.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi

I also tried Vicuna and the TheBloke directly to download, they give the same not found errors. But for standard llama-cpp it downloads just fine (I tested the same models in GGML versions over llama-cpp and they work fine).

I don't think exllama is compatible with cuda12. I finally got it to work as backend once I switched to the cuda18 images.


Vim Surround

Related to your change word entry, you might want to checkout surround.vim.

Thanks! I have that on my list of plugins to try out


Kerchunk fails to produce Data variables from NASA ATL08 data.

The test data ATL08 has 3 datasets. Here's the h5ls output:
METADATA                 Group
ancillary_data           Group
ds_geosegments           Dataset {5}
ds_metrics               Dataset {9}
ds_surf_type             Dataset {5}
gt1r                     Group
orbit_info               Group
quality_assessment       Group

However, none of them appears in print(ds) output from my ATL08.py test code.
<xarray.Dataset>
Dimensions:         (ds_geosegments: 5, ds_metrics: 9)
Coordinates:
  * ds_geosegments  (ds_geosegments) float32 1.0 2.0 3.0 4.0 5.0
  * ds_metrics      (ds_metrics) float32 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0
Data variables:
    *empty*
Attributes: (12/47)
    Conventions:                        CF-1.6

See my test Action workflow result for easy verification.

xarray assumes input datasets follow the netCDF4 layout. If you load the h5 file directly with xarray, you also get no variables
In [28]: xr.open_dataset("ATL08_20181014084920_02400109_003_01.h5")
Out[28]:
<xarray.Dataset>
Dimensions:         (ds_geosegments: 5, ds_metrics: 9, ds_surf_type: 5)
Coordinates:
  * ds_geosegments  (ds_geosegments) int8 1 2 3 4 5
  * ds_metrics      (ds_metrics) int8 1 2 3 4 5 6 7 8 9
  * ds_surf_type    (ds_surf_type) int32 1 2 3 4 5
Data variables:
    *empty*

However, kerchunk does faithfully reproduce this structure:
>>> out = kerchunk.hdf.SingleHdf5ToZarr("ATL08_20181014084920_02400109_003_01.h5").translate()
>>> m = fsspec.get_mapper("reference://", fo=out)
>>> zarr.open(m).tree()
/
 ├── METADATA
 │   ├── AcquisitionInformation
 │   │   ├── lidar
 │   │   ├── lidarDocument
 │   │   ├── platform
 │   │   └── platformDocument
 │   ├── DataQuality
 │   │   ├── CompletenessOmission
 │   │   └── DomainConsistency
 │   ├── DatasetIdentification
 │   ├── Extent
 │   ├── Lineage
 │   │   ├── ANC06-01
 │   │   ├── ANC06-02
 │   │   ├── ANC06-03
...

And you can change the path to get_mapper if you only want to access parts of the data tree, e.g., fsspec.get_mapper("reference://gt1r", fo=out) .
So the question is, how do you think xarray ought to load this data, what should it look like?

It should look like this 1.4M JSON, which HPD generated.
The current Kerchunk output JSON is too small (180K) and missing too many things.

What is the kerchunk version missing exactly, why should it be the same size at this other tool's output?
Continuing:
>>> g = zarr.open(m)
>>> g.attrs["time_type"]
'CCSDS UTC-A'
>>> g.attrs.todict()
{'Conventions': 'CF-1.6',
 'citation': 'Cite these data in publications as follows: The data used in this study were produced by the ICESat-2 Science Project Office at NASA/GSFC. The data archive site is the NASA National Snow and Ice Data Center Distributed Active Archive Center.',
 'contributor_name': 'Thomas E Neumann (thomas.neumann@nasa.gov), Thorsten Markus (thorsten.markus@nasa.gov), Suneel Bhardwaj<EMAIL_ADDRESS>David W Hancock III (david.w.hancock@nasa.gov)',
 'contributor_role': 'Instrument Engineer, Investigator, Principle Investigator, Data Producer, Data Producer',
 'creator_name': 'GSFC I-SIPS > ICESat-2 Science Investigator-led Processing System',
 'date_created': '2020-04-01T14:03:26.000000Z',
 'date_type': 'UTC',
...

all of those attributes are indeed there, if you are familiar with the zarr API.

Sorry, but which dataset is missing? Can you be precise?

See CDL and compare the CDL output from my workflow.

I'm sorry, I have no idea what that is.
Please can you give a specific example of something in the HDF5 file that you cannot access via kerchunk/zarr?

OK, so the data re stored in COMPACT form, which we don't support.
Turning on logging for "h5-to-zarr" would have revealed this.


[SPARK-38179][SQL] Improve WritableColumnVector to better support null struct

What changes were proposed in this pull request?

This changes WritableColumnVector to better support null structs, by avoiding allocating space in child vectors when an element is null.
Why are the changes needed?

Currently, for a WritableColumnVector whose type is struct, Spark needs to allocate space in all child vectors even if an element is null. This is not very space efficient.
In addition, this doesn't work well with the ongoing Parquet vectorized support for complex types, since when creating the child column vectors for a struct, we don't yet have the information of which slot is for null struct (Parquet packs values contiguously instead of having null slots).
Does this PR introduce any user-facing change?

Shouldn't be, since WritableColumnVector is Spark internal API.
How was this patch tested?

Updated ColumnVectorSuite and ColumnarBatchSuite to reflect the new changes.

Could you review this once more on the last commit, @cloud-fan ?

gently ping @cloud-fan - could you take another look? :)

Gentle ping, @cloud-fan .

Hi,
I found out about this change because I work on some things that depends on the memory layout of OffHeapColumnVectors and assumes that data for struct columns has the same layout as top-level columns (i.e. storage is allocated for NULL values).
I'm biased, but I have a few reasons I think this is not the right direction for Spark:

Other users and tools might be depending on the memory layout of {On,Off}HeapColumnVector since it's possible to access the underlying memory directly. I.e. this could cause breakage to user code. I don't know whether we consider this part of the public API or not.
This adds memory and CPU overhead for a common case (non-NULL struct) to optimize for a different case (NULL structs). I don't have data to prove that NULL structs are queried less frequently than NULL structs but it seems likely. It doesn't help with worst-case memory consumption either.
This makes the memory layout for top-level fields inconsistent with memory layout for fields inside structs, which can complicate the rest of the engine (need separate code for top-level column vectors and nested column vectors).
Unnesting a struct field, e.g. projecting SELECT x.y as x would require a conversion of the value data because the memory layout is different.
The current memory layout is more consistent with Arrow - https://arrow.apache.org/docs/format/Columnar.html#struct-layout - so as more of the data ecosystem moves towards Arrow as the interchange format, we're locked into doing data format conversions.

In addition, this doesn't work well with the ongoing Parquet vectorized support for complex types, since when creating the child column vectors for a struct, we don't yet have the information of which slot is for null struct (Parquet packs values contiguously instead of having null slots).
As someone who's fairly familiar with the Parquet spec and has worked on a couple of Parquet readers, this motivation seems incomplete. Data for top-level fields in Parquet is stored in exactly the same way as nested fields. Why does the approach used to scatter top-level values to the output vector not work for structs? Conceptually it should be the same algorithm except you are scattering values with def_level >= field_def_level instead of def_level >= 1. Something like:

for (int i = 0; i < num_values; i++) {
  bool is_null = parquet_def_levels[i] >= field_def_level
  out.nulls[i] = is_null
  if (!is_null) {
    out.values[i] = parquet_values[curr_value_idx]
    curr_value_idx++
  }
}

Other users and tools might be depending on the memory layout of {On,Off}HeapColumnVector since it's possible to access the underlying memory directly. I.e. this could cause breakage to user code. I don't know whether we consider this part of the public API or not.

As mentioned in the PR description, these are not public APIs and hence the change.

This adds memory and CPU overhead for a common case (non-NULL struct) to optimize for a different case (NULL structs). I don't have data to prove that NULL structs are queried less frequently than NULL structs but it seems likely. It doesn't help with worst-case memory consumption either.

Well, this is subjective though and there is no data point proving that NULL structs is not used commonly (at least I've seen quite some of these before). Spark also reuses these vectors so the overhead is bounded by the number of projected columns.

This makes the memory layout for top-level fields inconsistent with memory layout for fields inside structs, which can complicate the rest of the engine (need separate code for top-level column vectors and nested column vectors).
Unnesting a struct field, e.g. projecting SELECT x.y as x would require a conversion of the value data because the memory layout is different.

Hm I don't get this: what complication and what conversion? as you can see the PR itself is pretty simple and it doesn't involve changes on the "rest of the engine".

The current memory layout is more consistent with Arrow - https://arrow.apache.org/docs/format/Columnar.html#struct-layout - so as more of the data ecosystem moves towards Arrow as the interchange format, we're locked into doing data format conversions.

For Arrow there is a separate code path already ArrowColumnVector. All the data eco-systems I know (e.g., RAPIDS, Intel OAP, Iceberg) are already using that. So, I'm not really convinced this is an issue.

As someone who's fairly familiar with the Parquet spec and has worked on a couple of Parquet readers, this motivation seems incomplete. Data for top-level fields in Parquet is stored in exactly the same way as nested fields. Why does the approach used to scatter top-level values to the output vector not work for structs? Conceptually it should be the same algorithm except you are scattering values with def_level >= field_def_level instead of def_level >= 1. Something like

The problem is when creating child column vectors for a struct, we don't have the info whether a slot in the vector is for a null struct or not, as we are comparing with the child column's max definition level. The null structs are created when we are assembling the child columns. Please checkout #34659.

The problem is when creating child column vectors for a struct, we don't have the info whether a slot in the vector is for a null struct or not, as we are comparing with the child column's max definition level. The null structs are created when we are assembling the child columns. Please checkout https://github.com/apache/spark/pull/34659.
I spent about half an hour looking at that PR and I don't get why the vectorized reader needs this WritableColumnVector change. I see the code in VectorizedRleValuesReader is written to use this change (it only increments state.valueOffset if the parent struct is non-NULL) but I don't see why the logic needs to be like that.

This wasn't a problem in other vectorized Parquet readers I've worked with or looked at. E.g. the Apache Impala and the Apache Arrow/parquet-cpp Parquet readers read values in a spaced manner for structs without problems - https://github.com/apache/arrow/blob/master/cpp/src/parquet/column_reader.cc#L1014

Well, this is subjective though and there is no data point proving that NULL structs is not used commonly (at least I've seen quite some of these before). Spark also reuses these vectors so the overhead is bounded by the number of projected columns.
I don't see any evidence that making this different trade-off is better though - it seems like we should have some stronger reason to think this change is a net positive given how many CPUs this code will ultimately run on. Is there some benchmark or workload where this is significantly better?

Hm I don't get this: what complication and what conversion? as you can see the PR itself is pretty simple and it doesn't involve changes on the "rest of the engine".
I guess most of spark doesn't use these column vectors, so maybe not too important. If there is ever more efforts to add vectorized operations in Spark it would be simpler if you can apply vectorized operations to struct elements in same way as top-level elements. This maybe isn't a strong reason right now but the current Spark layout seems more future proof.

For Arrow there is a separate code path already ArrowColumnVector. All the data eco-systems I know (e.g., RAPIDS, Intel OAP, Iceberg) are already using that. So, I'm not really convinced this is an issue.
I was thinking that people in future will want conversions between different column vector types with minimal data copying where possible, e.g. if you want to scan from parquet and call a UDF using Arrow. Producing one style of ColumnVector then converting to another is inefficient. You could avoid the copy in principle when the memory layouts of vectors are identical (pretty common for numeric types). This really isn't super-important - maybe nobody will ever want to do this this - but just another reason to not change this.

Took a while to get back to this PR: I investigated more on why I needed this change originally from Parquet side, and turned out it's for handling cases like array of structs, and it seems there is a separate solution to handle the case. Therefore, to make it less disruptive, I'll close this PR for now.
Thanks all for taking a look!

Thank you for the decision, @sunchao .

@sunchao great to hear!


[LI-HOTFIX] Filter out UNKNOWN_TOPIC_OR_PARTITION from metadata response when the request is to fetch metadata for all topics

Filter out UNKNOWN_TOPIC_OR_PARTITION from metadata response when the request is to fetch metadata for all topics since the request to fetch metadata for all topics should not contain any UNKNOWN_TOPIC_OR_PARTITION exception in its response.
TICKET = KAFKA-10606
LI_DESCRIPTION = LIKAFKA-32857
EXIT_CRITERIA = Once we cherry-pick the fix for KAFKA-10606

I feel changing the topicMetadata to a var, and doing
topicMetadata = if(metadataRequest.isAllTopics) {...} else {...}
is easier to understand. Thoughts?

Makes sense!


[Java quickfix] Add final to method parameters

Thanks for updating the save actions to 0.9. One thing is still annoying and not what I expected updating code that had been written with eclipse.
The option "Add final to local variable" I would like to check, because I want the final added to:
String text = object.getName();
(Otherwise checkstyle will complain.)
But I do not need the final added to all parameters of all method declarations.
Gives many superflous code changes. Please do not include that in this option.
Perfectly all right would be a separate option "Add final to method parameters" which I could uncheck if desired.

That would be a nice enhancement, I'll try to add that. Thanks

I wanted request the same thing. A separate "Add final to method parameters" would be great.

I'll third this request! would love to have that as a separate option!

Hi,
could you tell us, if the feature "Add final to method parameters" is planned to be implemented?
ps: Thanks for this very helpful plugin.

No it is not planned but you can submit a PR anytime. Everything you need to know to setup your environment is there: https://github.com/dubreuia/intellij-plugin-save-actions#development-environment

Moved to https://github.com/dubreuia/intellij-plugin-save-actions/projects/2


Docker Compose File Example

You have some good examples for providers like github and gitlab. Are you planning on adding any docker compose example files?

Hi @michael-mcmullen,
unfortunately, for now, I'm not focusing on docker compose.
This package is a great starting way to build kubernetes containers.
If you have suggestions for docker composer, please PR...


Great work! Any plan to release the code and checkpoint?

没有提供描述。

There are currently no open source plans

Looking forward to trying it out once the checkpoints are made public

The weights and code are being organized, and we will make them public as soon as possible.

期待模型发布！！！

Thanks for your attention. You can check it out now.


keyboard: support up and down arrows to change slider value

Hey, thanks for open-sourcing this project :)
Here is a bit of code to enhance the accessibility of the component.
As mentioned in WAI-ARIA Authoring practices, it's best to handle up and down arrows in addition to left and up arrows to change slider value.

LGTM

Thanks!

Just published as 2.1.0


support 'force' flag in resource "apstra_datacenter_generic_system" for links

for an interface that is used for the breakout mode, for example, 100G breakout to 4x25G,  the provider does not support update link for any of the breakout link.
for example,  in below Dell Sonic example, the first 100G port is breakout into 4 x 25G ports:   Ethernet0, Ethernet1, Ethernet2, Ethernet3:
links = [
  {
    tags = ["Server", "OSP", "bond_infra", "ens10f0np0", "pxe"]
    target_switch_name = "PF-419"
    target_switch_if_name = "Ethernet0"
    target_switch_if_transform_id = 3
    lag_mode = "lacp_active"
    group_label = "bond_infra"
  },
  {
    tags = ["Server", "OSP", "bond_infra", "ens10f1np1"]
    target_switch_name = "PF-420"
    target_switch_if_name = "Ethernet0"
    target_switch_if_transform_id = 3
    lag_mode = "lacp_active"
    group_label = "bond_infra"
  },
  {
    tags = ["Server", "OSP", "bond_sdn", "ens1f0"]
    target_switch_name = "PF-419"
    target_switch_if_name = "Ethernet1"
    target_switch_if_transform_id = 3
    lag_mode = "lacp_active"
    group_label = "bond_sdn"
  },
  {
    tags = ["Server", "OSP", "bond_sdn", "ens1f1"]
    target_switch_name = "PF-420"
    target_switch_if_name = "Ethernet1"
    target_switch_if_transform_id = 3
    lag_mode = "lacp_active"
    group_label = "bond_sdn"
  },
  ...

update any of the 25G link, for example, add/delete/change a tag,  the provider would report below error when doing terraform apply:
Plan: 1 to add, 1 to change, 0 to destroy.
module.bp_generic_systems.apstra_datacenter_generic_system.bp_generic_system["HV-004"]: Modifying... [id=OEEIklg_escz5Agz1uY]
╷
│ Error: failed to set interface transform
│
│   with module.bp_generic_systems.apstra_datacenter_generic_system.bp_generic_system["HV-004"],
│   on ../../../modules/bp_generic_systems/main.tf line 38, in resource "apstra_datacenter_generic_system" "bp_generic_system":
│   38: resource "apstra_datacenter_generic_system" "bp_generic_system" {
│
│ {"api_response":null,"config_blueprint_version":0,"errors":{"interfaces":{"0":"Changing
│ interface 'Ethernet1' of system 'PhXf9azEKDgiMcrY_kM' will affect
│ interfaces not listed in the request. Set the "force" flag to bypass that
│ warning. Affected interfaces: 'Ethernet0, Ethernet2,
│ Ethernet3'"},"can_use_force":true},"error_code":422} - http response '' at
│ 'http://<IP_ADDRESS>/api/blueprints/dca32193-5b5c-4101-afef-bafd657101bb/interface-transformation?async=full&async=full'
╵
the request is to support a 'force' flag so that the breakout links can be updated.
Feng Wang @ TELUS

Further review suggests that we don't need to support the force keyword, but rather stop invoking SetTransformIdByIfName() when the transform ID has not been changed by the user.

This should be working correctly in v0.32.0 of the provider without the need for a force keyword/attribute

Wrong button. I'll leave this open for a while longer to give Feng a chance to verify.

Closed by #342 (confirmed by @fengwang523 via email)


Support with Sandpack Editor

Sandpack by CodeSandbox supports passing a custom editor function
Here's example with Monaco Editor - https://sandpack.codesandbox.io/docs/guides/integrate-monaco-editor
I think it should be possible to use this with Sandpack. We might have to export onChange function on CodeBlock component. I'll give it a try if I get some time.

We can give this a try, but it'll require some way to allow user to edit code within the code block while still preserving syntax highlighting. (I can hear horrifying screams of contenteditable attribute 🙈)
I'm not sure how features like line highlighting / word highlighting will work in a code block like this. Maybe it can allow the initial code set by the author to be highlighted as usual, but when the user edits the code, the highlighting goes away.

but it'll require some way to allow user to edit code within the code block while still preserving syntax highlighting

oh yes tried that. This will be tricky.

Maybe it can allow the initial code set by the author to be highlighted as usual, but when the user edits the code, the highlighting goes away.

Aha this is good idea! Normally I have seen line highlighting work only on static code but in live-editor the line highlighting is disabled.

Closing this for the time being as it's out-of-scope for the time being. If anyone is interested in working on it, we can re-open it!


Batter up

Since I am equipping a mainhand and offhand in chooseOutfit(), the batter up resource is never available.


Can't get to download all the artists from Twitter

So I have like 256 accounts on the monitor tab, I've tried to set the lastCheck date all the way back to the Year 2000 and at the moment I'm running the latest nightly version where retweets:no has been fixed. Problem is that now when I start back the Imgbrd Grabber it fails to download all 256 accounts at most it downloads 161 accounts kept retrying no luck. There's a catch to this every folder I check in has at least 4-5 images if not less. Could it be that there is something got to do with images limit? Totally clueless on why it's behaving like this and so far it's been happening only with the Twitter accounts I've added in the monitor's tab.

For some reason, when I manually type in the artist name and click Get All and set the Images Limit to 1000 or more it downloads in more pictures compared to 5 or 6, strange.

Excuse me

So I have like 256 accounts on the monitor tab, I've tried to set the lastCheck date all the way back to the Year 2000 and at the moment I'm running the latest nightly version where retweets:no has been fixed. Problem is that now when I start back the Imgbrd Grabber it fails to download all 256 accounts at most it downloads 161 accounts kept retrying no luck. There's a catch to this every folder I check in has at least 4-5 images if not less. Could it be that there is something got to do with images limit? Totally clueless on why it's behaving like this and so far it's been happening only with the Twitter accounts I've added in the monitor's tab.

Did you solve it?

Excuse me

So I have like 256 accounts on the monitor tab, I've tried to set the lastCheck date all the way back to the Year 2000 and at the moment I'm running the latest nightly version where retweets:no has been fixed. Problem is that now when I start back the Imgbrd Grabber it fails to download all 256 accounts at most it downloads 161 accounts kept retrying no luck. There's a catch to this every folder I check in has at least 4-5 images if not less. Could it be that there is something got to do with images limit? Totally clueless on why it's behaving like this and so far it's been happening only with the Twitter accounts I've added in the monitor's tab.

Did you solve it?

Not really, I've just scrapped that idea and decided to monitor newer images instead not sure if it's an API limitation from Twitter or whether it's not configured yet properly...


Wrong responses almost everywhere "findUnique({ where: {" is used. There are too much cases where this happens, why I do not list every case.
=> The list above is only partially correct!

Why is this happening? All of these endpoints were tested for spcifically this behavior... makes no sense at all

[ ] GET api/participants/"nonexistingPid"
404: Endpoint not implemented

This is right, because this endpoint is in fact not implemented

[x] GET api/media/"nonexistingPid"
404: Endpoint not implemented

Fixed

All the others were not reproducible on my local setup, please look if this was just a problem with the configuration and merge the PR if everything works

Thanks for your effort so far.
However i was able to reproduce those issues with another PC. It happens in dev mode. I was not able to reproduce this problem in production mode. I try to figure out what goes wrong here...

Funny, not happening to me with dev mode

Well after some research and discussion with @stephan418  figured out that this problem occurs due to inconsitant errohandling. When a non-existing pid gets handed over first thing prisma throws is a "PrismaClientUnknownRequestError". Sometimes it is handled in the controller itself (eg. events) [wrong approach] mostly it's handled by the common.ts errhandler [right solution].
Before any tasks are defined i want do discuss wether the errorhandler in the controller are still useful or not?

If the handling of the PrismaClientUnknownRequestError is the only problem, we could simply remove all of those handlers.. this should no take more than a few minutes


Morten/alpm filelist contains

The first issue with travis was fixed but I'm unable to reproduce in my chroot what's causing the SIGSEV
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x55f312]
goroutine 9 [running]:
testing.tRunner.func1(0xc0000cad00)
	/usr/lib/go/src/testing/testing.go:830 +0x392
panic(0x598ae0, 0x725180)
	/usr/lib/go/src/runtime/panic.go:522 +0x1b5
_/build.(*Package).ContainsFile.func1.1(0x0, 0xc00002fed8)
	/build/package.go:186 +0x22
_/build.(*Package).ContainsFile.func1(0x0, 0x5cb449, 0x10, 0xa481d0)
	/build/package.go:186 +0x2f
_/build.(*Package).ContainsFile(0x0, 0x5cb449, 0x10, 0x0, 0xc000010370, 0x0, 0x0, 0x1d7542e7, 0x1d7542e7006bc130)
	/build/package.go:186 +0x43
_/build.TestPkgFiles(0xc0000cad00)
	/build/package_test.go:119 +0x159
testing.tRunner(0xc0000cad00, 0x5d4290)
	/usr/lib/go/src/testing/testing.go:865 +0xc0
created by testing.(*T).Run
	/usr/lib/go/src/testing/testing.go:916 +0x35a
exit status 2

And there we go. There was an == nil vs != nil error and I have a slight hunch linux isn't on the docker image. Moved the file check to glibc

Beautiful, everything seems in accordance, didn't think of that small detail of the image

As mentioned on irc, CSrings need to be Free'd. Although it's not like we're perfect when it comes to leaking memory.


Consider removing the private access specifier

Private is actually implemented as protected (the two are aliases in the datatype definitions at the moment). The reason for this is that private properties don't actually make sense, because the generated class does not actually use them. The only way to make use of them would be to have child class of the generated class and use them there. But that's protected, not private...
If it's not private, why call it that?

Do we need access specifiers at all?
I don't really see much use for them, to be honest, and just removing them simplifies the datatype format (and the code used to parse it too, of course)

I'm leaning more and more to removing all access specifiers. And the getters and setters right along with them (leaving us with only the __set/__get properties.


Transform done hook into async

What kind of change does this PR introduce?
It makes the done compiler hook async.
Did you add tests for your changes?
Not yet, not sure what to add.
If relevant, link to documentation update:

Summary
To be able to track our build time, I want my CI to send timings via HTTP to a server. Today this does not seem possible, as the only hook where I can get these timings is the done hook, but it is synchronous, and thus making async HTTP requests in it is not possible.
Does this PR introduce a breaking change?
Not sure yet.
Other information

Thanks


Correct TinCanApi::Activity and TinCanApi::Result calls

Am attempting to pass a statement for an activity with name and decription, including a result
agent = TinCanApi::Agent.new(mbox<EMAIL_ADDRESS>verb = TinCanApi::Verb.new(id: 'http://adlnet.gov/expapi/verbs/completed', name: 'completed')
object = TinCanApi::Activity.new(id: "https://somelms/quizzes/numerical-reasoning",
         name: 'Numerical Reasoning',
         type: 'http://adlnet.gov/expapi/activities/assessment',
         description: 'Numerical ability tests are used to establish a candidate’s ability to read and understand the information presented in numerical form and then use the correct and most effective operations to find the right answer. Although it is senior-school level maths, you will still need to be efficient and well-practiced.',
         extensions: { "http://id.tincanapi.com/extension/planned-duration" => 'PT20M' }
                              )
result = TinCanApi::Result.new(
         response: 'Passed',
         score: {min: 0, raw: 1, max: 2},
         success: true,
         extensions: {"https://somelms/quizzes/numerical-reasoning" => [
                     'Xavier calls each of his employees on Monday mornings. He speaks to John second, Henry third, Joanna fifth and Layla sixth. Based on the information above, which of the following statements CANNOT be true?',
                     'Lola lives in the United States of America but visits Canada twice a year, bringing her three dogs with her. She needs to buy new insurance for her dogs. Which plan is the best for Lola’s needs?'
                     ]})

statement = TinCanApi::Statement.new do |s|
  s.actor = agent
  s.verb = verb
  s.object = object
  s.result = result
end

response = remote_lrs.save_statement(statement)

Can somebody help with the correct syntax to make the above calls.

agent     = TinCanApi::Agent.new(mbox<EMAIL_ADDRESS>name: 'Arnold Marunda')
verb      = TinCanApi::Verb.new(id: 'http://adlnet.gov/expapi/verbs/completed', display: {en: "completed"})
definition = TinCanApi::ActivityDefinition.new(name: {en: 'Numerical Reasoning'}, type: 'http://adlnet.gov/expapi/activities/assessment', description: {en: 'A fictitious example CBT course.'}, interaction_type: "choice", extensions: { "http://id.tincanapi.com/extension/planned-duration" => 'PT20M' })
activity    = TinCanApi::Activity.new(id: "http://www.candidatetalent.com/quizzes/numerical-reasoning", definition: definition)
score     = TinCanApi::Score.new(scaled: 0.75, raw: 15, min: 0, max: 20)
duration = Duration.new(minutes: 12)
result    = TinCanApi::Result.new(response: 'Passed',
                                  score: score,
                                  success: true,
                                  completion: true,
                                  duration: duration)

statement = TinCanApi::Statement.new do |s|
  s.actor = agent
  s.verb = verb
  s.object = activity
  s.result = result
end

response = remote_lrs.save_statement(statement)


Fix syntax errors in aider/queries/tree-sitter-elm-tags.scm

The aider/queries/tree-sitter-elm-tags.scm file cannot be properly loaded by py-tree-sitter-languages. There were a couple of superfluous parentheses compared to the corresponding file at tree-sitter-elm
Fixes #816

Huge thanks for tracking this down!
I added test coverage for all supported repomap languages (well, aider did). It reproduces the SyntaxError: Invalid syntax at offset 275 bug. Hopefully merging this PR will then resolve it.
9db28c0052996464c9d66dae19744a74ddba7c41


packer-plugin-vagrant has been updated with breaking changes

Hi,
packer-plugin-vagrant has recently been updated to v1.1.0 and the structure of the files included in the Vagrant box has changed. See https://github.com/hashicorp/packer-plugin-vagrant/pull/48
If I'm not mistaken, this causes the check on this line to fail: https://github.com/ppggff/vagrant-qemu/blob/4d85f60032bf0a4ef3f7d26f4a102fce1f3213f2/lib/vagrant-qemu/action/import.rb#L22
This is using vagrant-qemu version v0.3.5

Thanks, do you have a example box for test?

You can try this one

To support the new version of libvirt box, I need to find a workable libvirt machine for test.
(See how libvirt convert multiple disk config to qemu command)
It'll take a while.
There is temporary workaround for single disk box: use the image file directly.
Install box and find image path:
# install box as "test_new_box"
vagrant box add --name test_new_box ./alpine318-packer_plugin_v1.1.0.box

# find the image file, the default location for "test_new_box":
~/.vagrant.d/boxes/test_new_box/0/libvirt/box_0.img

Set image_path config in Vagrantfile.
qe.image_path = "/Users/xxx/.vagrant.d/boxes/test_new_box/0/libvirt/box_0.img"

Thanks for the update

running into this same issue, currently hardcoding paths in my Vagrantfile.
is there any plans on fixing this so i don't need a workaround ?

I've made a pull request to at least get this plugin working again with boxes that have one single drive image.

Please try the new v0.3.6

I can confirm that your version 0.3.6 seems to be working for me.
Please feel free to close this issue.


Modernize actions CI

Update to new style. Focal support will be added to Citadel.

The CMake warning on Ubuntu is not introduced by this PR, it's showing now because of the new cmake warnings plugin on Jenkins. Here's an issue about it: https://github.com/ignitionrobotics/ign-tools/issues/23

is there anywhere else I should look to verify this PR for future reference?

Since this PR is only touching the actions, I think it should be enough to check that the file changes look reasonable and the actions are passing.
When I merge it forward to Citadel, I'll add Focal CI, which hopefully will pass without the need for extra changes. But if there are more changes needed, we should also make sure that they look reasonable and don't break the rest of CI, or behaviour, API, ABI, etc.


Silence ort logging

Hello, is there any way to disable ort logging? Can't see anything useful while embedding calls are running. Thank you!!

I am not sure. Maybe @decahedron1, could help.

With tracing-subscriber, set the RUST_LOG environment variable to ort=warn.

Thank you!!
For future readers - you can specify per-crate logging conditions like so
export RUST_LOG=info,my_crate=info,ort=warn
(Configuring https://github.com/tokio-rs/tracing)


Vue support please

Please make this plugin support vue

@ukik I don't think that Vue can use NgRx, can it?  Are you looking for redux devtools support for Vue NativeScript?


readOnly: true does not prevent pasting

Tested on CodeMirror 5.5.1 (on codemirror.net) on latest Firefox and Chrome. Set an editor's readOnly option to true, and then copy some text from somewhere and try pasting it into the editor. The text is pasted successfully into the editor, but I would expect nothing to happen.

Oops -- this was broken in a recent change to the paste handler. Attached patch fixes it again.


long waiting for get_volumes_by_pool request

our creation process takes a long time (about 45 sec, whereas in comparison it only takes about 5 sec via the SVC storage family)
we timed the get_volumes_by_pool command on dscli client vs pyds8k vs postman on pool with 240 vols and then on pool with 4 vols for comparison and fun.
results:
with 244 vols:
dscli client: 17s
pyds8k: 26s
postman: 24s
with 4 vols:
dscli client: 15s
pyds8k: 4.5s
postman: 4s
we can see that pyds8k is not giving us worst performance than the postman (which uses rest as well) so if there is a problem it's with the rest itself

we are using get_volumes_by_pool to find  volumes by name
is there any other way to find volumes by name?

As the CSI work and get retries due to timeouts from k8s the waiting time for the command can reach over 2.5m.


Only active in editor.

Hi,
I recently added this to a production product, but had to modify the source because it worked on mobile and in editor, but not on desktop outside of the editor.
So in my case it worked on ios/android, but not steam.
We just made this minor change in UniClipboard.cs:
if (_board == null) {
#if UNITY_ANDROID
_board = new AndroidBoard();
#elif UNITY_IOS
_board = new IOSBoard ();
#else
_board = new EditorBoard();
#endif
}
Was previously:
if (_board == null) {
#if UNITY_EDITOR
_board = new EditorBoard();
#elif UNITY_ANDROID
_board = new AndroidBoard();
#elif UNITY_IOS
_board = new IOSBoard ();
#endif
}
Thanks for sharing this code with the community. Much appreciated. :)

Hi michaellegrand,
Sorry for late reply and thanks for your information!!
I added a steam support to the plugin at version 1.0.1.


Support for DistributedDataParallel

Hi
Thank you for sharing this great work with us.
I saw that you have spawn in the code, thus I am wondering your plan to release the code for supporting DistributedDataParallel. In particular, I am curious how do you sync the memory bank for L and ab, e.g., in self.register_buffer('memory_ab') during training.
Thank you :)

Hi,
Good question!
I used DataParallel for the pretraining stage instead. Distributed training is for the linear classifier stage. I am also interested in synchronizing the memory across different GPUs, let me know if you figure out a solution :)

@chihyaoma
I have updated the repo for distributed training of linear classifiers, if that could be helpful
I am closing it since it's not an issue. Feel free to reopen it, though.


add hybrid_property

Reflex currently renders @propertys on States like this: <property object at 0x723b334e5940>.
This PR adds a hybrid_property decorator which functions like a normal python property but additionally allows (class-level) access from the frontend. You can use the same code for frontend and backend, or implement 2 different methods.
example
Note: full_name and has_last_name are not rx.vars and are not sent over the wire (websocket). They are instead computed from first_name and last_name separately in frontend (js) and backend (python).
class State(rx.State):
    first_name: str = "John"
    last_name: str = "Doe"

    @property
    def python_full_name(self) -> str:
        """A normal python property to showcase the current behavior. This renders to smth like `<property object at 0x723b334e5940>`."""
        return f"{self.first_name} {self.last_name}"

    @hybrid_property
    def full_name(self) -> str:
        """A simple hybrid property which uses the same code for both frontend and backend."""
        return f"{self.first_name} {self.last_name}"

    @hybrid_property
    def has_last_name(self) -> str:
        """A more complex hybrid property which uses different code for frontend and backend."""
        return "yes" if self.last_name else "no"

    @has_last_name.var
    def has_last_name(cls) -> Var[str]:
        """The frontend implementation for has_last_name"""
        return rx.cond(cls.last_name, "yes", "no")

    def handle_stuff(self) -> None:
        """A simple event handler which showcases backend-level access to the hybrid properties."""
        print(f"python_full_name: {self.python_full_name}")
        print(f"full_name: {self.full_name}")
        print(f"has_last_name: {self.has_last_name}")

def index() -> rx.Component:
    return rx.vstack(
        rx.text(f"python_full_name: {State.python_full_name}"),
        rx.text(f"full_name: {State.full_name}"),
        rx.text(f"has_last_name: {State.has_last_name}"),
        rx.input(
            value=State.last_name,
            on_change=State.set_last_name,  # type: ignore
        ),
        rx.button("Handle Stuff", on_click=State.handle_stuff),
    )

app = rx.App()
app.add_page(index)

We could implement the same for ComputedVars with a small patch.
diff --git a/reflex/vars.py b/reflex/vars.py
index 8f1f0186..c502d814 100644
--- a/reflex/vars.py
+++ b/reflex/vars.py
@@ -2154,10 +2154,16 @@ class BaseVar(Var):
         return setter

+VAR_CALLABLE = Callable[[Any], Var]
+
+
 @dataclasses.dataclass(init=False, eq=False)
 class ComputedVar(Var, property):
     """A field with computed getters."""

+    # The optional var function for the property.
+    _var: VAR_CALLABLE | None = None
+
     # Whether to track dependencies and cache computed values
     _cache: bool = dataclasses.field(default=False)

@@ -2322,6 +2328,18 @@ class ComputedVar(Var, property):
             return True
         return datetime.datetime.now() - last_updated > self._update_interval

+    def var(self, func: VAR_CALLABLE) -> Self:
+        """Set the (optional) var function for the property.
+
+        Args:
+            func: The var function to set.
+
+        Returns:
+            The property instance with the var function set.
+        """
+        self._var = func
+        return self
+
     def __get__(self, instance: BaseState | None, owner):
         """Get the ComputedVar value.

@@ -2334,6 +2352,9 @@ class ComputedVar(Var, property):
         Returns:
             The value of the var for the given instance.
         """
+        if instance is None and self._var is not None:
+            return self._var(owner, value=self)
+
         if instance is None or not self._cache:
             return super().__get__(instance, owner)

@@ -2546,8 +2567,6 @@ def computed_var(
 # Partial function of computed_var with cache=True
 cached_var = functools.partial(computed_var, cache=True, _deprecated_cached_var=True)

-VAR_CALLABLE = Callable[[Any], Var]
-

 class HybridProperty(property):
     """A hybrid property that can also be used in frontend/as var."""

I can move this to experimental if needed

I think we can expose this under experimental yes, since it doesn't seems like it impact any existing feature.
Probably need some changes due to recent changes to vars.


Migrate to ESLint 9

With the update to v9, ESLint did not support reading configuration in subdirectories.
We relied on this feature when running ESLint with pre-commit.
Therefore, the configuration was unified at the top-level of the repository.
This resulted in a cleanup of linting related build code in the ts-model-api and vue-model-api directories.
The unification makes maintenance easier anyway.
The official ESLint hook is not used anymore because of the issue described in https://github.com/eslint/eslint/issues/18465
The local hook works around that issue.
It also does not require duplicating versions of NPM packages.
But it requires installing NPM dependencies before running pre-commit hooks.
To be verified by reviewers

[ ] Relevant public API members have been documented
[ ] Documentation related to this PR is complete

[ ] Boundary conditions are documented
[ ] Exceptions are documented
[ ] Nullability is documented if used

[ ] Touched existing code has been extended with documentation if missing
[ ] Code is readable
[ ] New features and fixed bugs are covered by tests

The build fails because of the problems described here: https://github.com/eslint/eslint/issues/18465

:tada: This PR is included in version 9.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Font color of certain items in the bookmarks bar doesn't change to white

When using this Material Dark theme, the text color of the bookmarks bar items change to a lighter color so that they are clearly visible. However, the text color of the folders in the bookmarks bar and the (settings)buttons at the top right remain in the original gray color making them less visible. Also view attached screenshots. Thanks

Seems like removing and then re-adding the theme to Chrome solves the issue.


Update fibonacci.c

i have added some beautiful frontend and full stop at the end which seems to be great

numbers are displayed 11 times if the term is set to 10

This pr makes no change at the core level


fix(dialog): fix dialog position or overflow menu

Open to feedback and discussion.
I know you can pass an offset to the component but it seems wrong for the component to break by default because some parent has some position set with some modifying positions. It should work as expected by default.
Fixes: https://github.com/IBM/carbon-components-angular/issues/1557
I've even tested this with multiple nesting elements: https://codepen.io/dusanmilko/pen/pobzbqR
Before:

After:

Before:

After:

Nice! We'll have to integrate this into the upstream position utility at some point, but for now this looks great! 🎉

Yeah, agreed. I'll also apply this to other components that have a similar issue like dropdown and etc.

Sounds good


Uncaught (in promise) LinkError: import object field 'runtime.walltime' is not a Function

In Firefox 106.0.5 (64-bit) on Gentoo ~amd64, go version go1.19.3 linux/amd64, getting this error, seen in devtools:
Uncaught (in promise) LinkError: import object field 'runtime.walltime' is not a Function
Looks like this is why: https://github.com/golang/go/commit/862ddb37b4e24bcb89f060d50325c8f863ee99bc
Ah, I guess I just needed to update wasm_exec.js from the original location maybe? https://github.com/golang/go/blob/e18d07ddc5c9005738c3063130f0f5ccf127849e/misc/wasm/wasm_exec.js#L250

Hey,
Now it works with all go versions, the problem was that build.sh did not copy wasm_exec.js from the go env.
Cay you try it again from master?

In chrome it doesn't recognize ports with https, but only with http.
Maybe a different logic will be able to recognize the difference but anyway the answers are masked and not received if the browser is blocking our requests.
<IP_ADDRESS> and <IP_ADDRESS> will work differently as well, it depends on which host the user is visiting - CORS and CSP, as you said.
This could be fixed :+1:

Seems like the Issue is fixed,
The build.sh will setup your environment, by copying wasm_exec.js to the project's root.
Then it'll be compatible to your current Go version and OS.


Expand gmq.AsTime to support DATE and TIME types

Signed-off-by: Pouyan Heyratpour<EMAIL_ADDRESS>

Thank you very much.

I'm planning to add Join feature. Do you have any considerations or notes about which point I should start at and what things should I care about @mijia ?

@pouyanh Great news, really thank you for that. I lost focus on this project for a long time and haven't thought this through. Don't know if there would be some better solution if we insist on no reflection used. Maybe we need to work this out depends on the runtime reflection for the Join or Union.

Currently I am using modelq to generate the basic models , but using sqlx for the complicated query such as the join, union and etc.


Error: couldn't read cache file to update store info timestamps

Hello,
We started receiving this error message ever since we updated to Xcode 8. We're using Swift 2.3 and Apptentive SDK 3.3.0.
I searched Apptentive project and found a cache named @"at-unread-messages-cache" in ApptentiveBackend.m and ApptentiveMessageCenterDataSource.m.
Our app uses CoreData, but none of our NSFetchResultsControllers are using cache. Other libraries in the app are not using CoreData at all. Also, the message doesn't appear if I disable Apptentive.
It happens on phones and simulators. I tried iPhone 5, iPhone 5s, iPhone 6 and iPhone 7. The error message appeared for all of them, but only on iOS 9 and iOS 10 (never on iOS 8).
This is the log:
[info] Apptentive SDK Version 3.3.0
[debug] Setting up data manager
[debug] Creating conversation
[debug] Apptentive Reachability changed: WiFi
[info] Engage Apptentive event: com.apptentive#app#launch
[info] Conversation created successfully.
[info] Received remote Interactions from Apptentive.
[error] error:  (NSFetchedResultsController): couldn't read cache file to update store info timestamps
[error] error:  (NSFetchedResultsController): couldn't read cache file to update store info timestamps
[error] error:  (NSFetchedResultsController): couldn't read cache file to update store info timestamps
[error] error:  (NSFetchedResultsController): couldn't read cache file to update store info timestamps
[error] error:  (NSFetchedResultsController): couldn't read cache file to update store info timestamps
[error] error:  (NSFetchedResultsController): couldn't read cache file to update store info timestamps
[error] error:  (NSFetchedResultsController): couldn't read cache file to update store info timestamps
[error] error:  (NSFetchedResultsController): couldn't read cache file to update store info timestamps
This is how it looks like on iOS 8:
[info] Apptentive SDK Version 3.3.0
[debug] Setting up data manager
[debug] Creating conversation
[debug] Apptentive Reachability changed: WiFi
[info] Engage Apptentive event: com.apptentive#app#launch
[info] Conversation created successfully.
[info] Received remote Interactions from Apptentive.

Thanks for the detailed report.
We will be removing the cache file specifier for the unread count fetched results controller. For Message Center's section info cache, we'll be removing it for iOS 9 and 10, which seem to have a bug related to the file.

Hello @frankus
I just downloaded the SDK and integrated in my game. I need to show the Ratings Prompt. My project is developed in Swift and I am using Xcode 8 with iOS 10 for my device iPhone 6s Plus.
I am still getting the above mentioned issue. The log is as same as @Taarna  has mentioned in his report.
When I call
[[Apptentive sharedConnection] engage:@"completed_level" fromViewController:viewController];
following log is printed in console
Engage Apptentive event: local#app#Level_complete
2016-10-24 12:24:25.037884 Canon Hero[1243:468196] [error] error:  (NSFetchedResultsController): couldn't read cache file to update store info timestamps
2016-10-24 12:24:25.038205 Canon Hero[1243:468196] [error] error:  (NSFetchedResultsController): couldn't read cache file to update store info timestamps
2016-10-24 12:24:25.054900 Canon Hero[1243:468221] [info] Interaction 57f2c7beb129482b4300002d predicate failed evaluation.
2016-10-24 12:24:25.060068 Canon Hero[1243:468220] [info] Interaction 57f2c7beb129482b4300002b predicate failed evaluation.
2016-10-24 12:24:25.088407 Canon Hero[1243:468221] [info] Received remote Interactions from Apptentive.
2016-10-24 12:24:29.944143 Canon Hero[1243:468196] [error] error:  (NSFetchedResultsController): couldn't read cache file to update store info timestamps
2016-10-24 12:24:29.944928 Canon Hero[1243:468196] [error] error:  (NSFetchedResultsController): couldn't read cache file to update store info timestamps

Please let me know the solution.
Warm Regards

Any info on when this will be removed it's quite annoying with my logs?

The latest release (3.3.2) includes a fix for this.


Code Refactored

Description
Implemented the changes for CSS Files
Issue  #7
Checklist:

[x ] My code follows the style guidelines of this project
[ x] I have performed a self-review of my own code
[ x] I have commented my code, particularly in hard-to-understand areas
[ x] I have added necessary inline code documentation
[ x] I have added tests that prove my fix is effective and that this feature works
[ x] New and existing unit tests pass locally with my changes

 Here is an overview of what got changed by this pull request:

Issues
======
- Added 1

See the complete overview on Codacy


struct Dav1dFilmGrainDSPContext: Type-erase fn ptrs and deduplicate

I had added EntryRow to be consistent with const_pixel_row and friends, but it's not important to have. I've removed it.

I had added EntryRow to be consistent with const_pixel_row and friends, but it's not important to have. I've removed it.

Maybe it's useful to have later, but I think we'd want it as a generic type alias like the others.