Unsupported valibot schema: file

[X] Before posting an issue, read the FAQ and search the previous issues.

Description
When creating a schema with file or blob, I get:
Error: Unsupported valibot schema: file

using something like:
export const photoSchema = v.object({
	photo: v.pipe(
		v.file('Please select an image file.'),
		v.mimeType(
			['image/jpeg', 'image/png', 'image/gif', 'image/webp'],
			'Please select a JPEG, PNG, WEBP, or GIF file.'
		),
		v.maxSize(1024 * 1024 * 10, 'Please select a file smaller than 10MB.')
	)
});

J

file & blob must be added in the customSchemaConversion in https://github.com/ciscoheat/sveltekit-superforms/blob/1dcb881a61d8bf8074c639834a733652d3878506/src/lib/adapters/valibot.ts#L32

I don't understand. Does it not support all Valibot functions out of the box? What about mimeType, maxSize, etc?

Files aren't supported by JSON Schema natively, so it needs to have an explicit custom schema generator. I'm quite sure this has worked before, but Valibot is a 0.x library, so not much to do. It'll be added in the next version.

Until then, you can configure the adapter with that option:
valibot(schema, { customSchemaConversion: {
	custom: () => ({}),
	instance: () => ({}),
	file: () => ({}),
	blob: () => ({})
}});

What do you guys mean "not supported?" It is literally in the example on Valibot:
https://valibot.dev/api/file/#examples
I don't know what customSchemaConversion does. There are no examples in the docs. You guys show me the code, but I don't understand how to use it?
valibot(schema, { customSchemaConversion: {
	custom: () => ({}),
	instance: () => ({}),
	file: () => ({}),
	blob: () => ({})
}});

How would I use this?
J

It's not as simple as using your favorite library and validate some data with it. Have you ever wondered how Superforms can conjure default values for schemas, create browser constraints, map errors to correct paths and transform posted form data into the correct types, for nine different validation libraries?
It can only be done consistently if there is a underlying unified data format for all of them.
That format is called JSON Schema. Using a validation schema to JSON Schema library, like the very well written one for Valibot, it's possible to handle the primitive types quite easily, those that can be directly serialized to JSON, but there are others like Date and, File of course, that cannot be directly converted to JSON Schema, because there is no standard for them.
So when we say "not supported", even though Valibot and many others validate files, you cannot directly convert the schema for file validation to JSON Schema. A custom schema conversion is required, that will say "here's how I want files to be handled".
Now, to solve your problem. You have used superValidate before, right?
const form = await superValidate(valibot(schema));

Solution: Add the option object you're wondering about as a second parameter to the valibot function. It will make files and blobs pass through the converter (but not pass through the validator, of course. I hope you know the difference by now.)

I don't know what customSchemaConversion does. There are no examples in the docs. You guys show me the code, but I don't understand how to use it?

That's how it is with open source software, everyone's time is limited, so when you don't pay with money, you pay with your time. Now I've spent plenty of time explaining this to you, so I hope you can pay it forward somehow.

Hi @ciscoheat,
I appreciate the basic explanation, but that is not my issue.
What I'm not understanding are two things.
1.) Why is this possible with zod but not valibot:
https://superforms.rocks/concepts/files
Should I use the image: optional(instance(File)) instead? This seems like a work-around. I'm assuming this means the core converter has not been added for valibot... yet?
2.) This still doesn't explain how to use this:
valibot(schema, { customSchemaConversion: {
	custom: () => ({}),
	instance: () => ({}),
	file: () => ({
        // what goes here?
        }),
	blob: () => ({})
}});

This seems like something that should be in the docs. Where can I find how to use a conversion (a like, not asking anyone to write anything for me).
Thanks,
J

1.) Why is this possible with zod but not valibot:
https://superforms.rocks/concepts/files

The Zod to JSON Schema library isn't as strict as the Valibot one. It converts File (or rather, instanceof) validators to an any type, which again isn't used in validation, but in the data conversion, when providing default values, etc.
The Valibot converter requires you to be more explicit, which in combination with 0.x versions can lead to these kind of issues.

Should I use the image: optional(instance(File)) instead? This seems like a work-around. I'm assuming this means the core converter has not been added for valibot... yet?

No, just add the option object as-is. It's just a matter of being explicit.

2.) This still doesn't explain how to use this:
valibot(schema, { customSchemaConversion: {
	custom: () => ({}),
	instance: () => ({}),
	file: () => ({
        // what goes here?
        }),
	blob: () => ({})
}});

Nothing goes there, {} is the same as any in the JSON Schema definition, which is a bit confusing, but it represents a validator object without any validation. Read more about it here: https://json-schema.org/understanding-json-schema/basics
Again, the any type is not used in validation, but as metadata for the default values, constraints, etc, so it's safe to use. Validation itself is handled by Valibot, not by the JSON Schema.

This seems like something that should be in the docs. Where can I find how to use customSchemaConversion (a link, not asking anyone to write anything for me).

It's tied to the Valibot JSON Schema library, so: https://github.com/gcornut/valibot-json-schema?tab=readme-ov-file#custom-schema-conversion
In the next version, it won't be a problem anymore since I will add the conversions to the default options for the adapter. Until the next missing schema type comes up, of course... ;)
I could link to the respective JSON Schema converter for each adapter, but it's rare to configure them, and some are directly in the library code due to ESM/bundle issues, others only have a simple representation due to not being convertible to JSON Schema (see my blog post about this), so I'm not sure how to make the docs for that succinct and useful.

Thanks,

Thank you for sponsoring, it's very much appreciated!


Feature Request: Multicast RTP support for RTSP ANNOUNCE/RECORD

Describe the feature
Description
Currently a user may use the RTSP ANNOUNCE and RECORD commands to publish RTP/TCP and RTP/UDP Unicast streams to MediaMtx. However, RTP/UDP Multicast streams are not supported. Attempts to publish such streams fail at the RTSP SETUP stage with the following message: 'RTSP/1.0 461 Unsupported Transport'.
MediaMtx currently supports requesting RTP/UDP Multicast streams via the RTSP DESCRIBE and PLAY commands, so MediaMtx does already include support for such streams. I imagine that the lift to support this feature may be lite.
I have an existing application that generates RTP/UDP Multicast Streams, and I would like to integrate the application with MediaMtx with the RTSP ANNOUNCE and RECORD commands.
Thank you so much for your hard work and congratulations on creating such a wonderful product!

Hello, you're right, publishing with RTSP and the UDP multicast transport protocol is currently unsupported, but the reason is that few people use this feature, because its only advantage is the ability to stream something to RTSP and RTP listeners at the same time. Usually people pick a single protocol (either RTSP/multicast or RTP/multicast)  and use it in their entire infrastructure.
I'm leaving this feature request open in order to check whether there is interest in this feature.

Thank you for your quick response.
Would MediaMtx be open to code contributions for this feature?
I would not want to work on it, if I did not think it would be integrated into a release version...

Would MediaMtx be open to code contributions for this feature?
I would not want to work on it, if I did not think it would be integrated into a release version...

Yes, absolutely! the RTSP implementation is located inside gortsplib, you can start by removing these lines:
https://github.com/bluenviron/gortsplib/blob/1b0ece827062579ac879fd03b2ab3ec4fd685285/server_session.go#L731-L735

Thanks for your hard work on this project. Just commenting to show interest for this feature, it would help me a lot. I would attempt to make the change myself, but unfortunately have never worked with go before.

@rrajra
I have an application that currently streams out multicast packets, so I was hoping for ANNOUNCE/RECORD multicast support. Since that feature does not exist, I implemented ANNOUNCE/RECORD in Unicast RTP for my application. Although it adds an additional hop, it seems to work very well so far, so I do not have an immediate need to implement multicast support. I am sorry to disappoint you. If the need arises, perhaps I will revisit the feature in the future.


azure devops server 2019 does not support list-syntax push trigger

the documentation clearly states that push triggers should support the list syntax :https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops-2019&tabs=schema%2Cparameter-schema#push-trigger

but Azure DevOps server 2019.1.1 throws 2 errors if you actually try to use it :

and when trying to run :

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7458d5b2-1bb7-7d87-7ce6-2b0c430b0155
Version Independent ID: edf5bbd3-a3f0-5426-e985-31290f8afd05
Content: YAML schema - Azure Pipelines
Content Source: docs/pipelines/yaml-schema.md
Product: devops
Technology: devops-cicd
GitHub Login: @steved0x
Microsoft Alias: sdanie

forgot to put the "[]"


Got an error after creating an issue here

Is there an existing issue for this?

[X] I have searched the existing issues

Offending URL
https://github.com/MithrilJS/docs/
Issue description
I got this error message now as an email for each issue I created:
Notify triage on issue create: All jobs have failed
Notify triage on issue create / reject / notify
Failed in 3 seconds

Sorry, didn't realize you were getting emails over that. I'll disable the action here in the repo to prevent more email spam. It's ultimately blocked on a known issue we're tracking: https://github.com/MithrilJS/infra/issues/25. I'm blocked on what appears to be a GitHub backend bug.
Closing as that's the main tracking issue.


interface - plotting of 2d images

Extend the plotting interface to feature visualisation of 2d images.

Bitbucket: https://bitbucket.org/spinitiative/hummingbird.git/issue/7
Originally reported by: Benedikt Daurer
Originally created at: 2015-02-26T01:00:32.002

While far from perfect, we have a 2D image viewer.
If specific features are required please open a new issue.

Original comment by: Filipe Maia

Last commit provides an initial fix for this. It can plot 2D numpy arrays as an image.

Original comment by: Filipe Maia


Request: notebook file for training cxrmate

This repo looks very interesting, I would like to try using it to train on a new large XR dataset.  However the only training tutorial available is on the main github page, and requires command-line usage of the dlhpcstarter package.  Could you please create an .ipynb file showing how to run training within a python IDE, similar to the examples found in cxrmate/examples for how to run inference-only?  Thanks!

Hi AceMcAwesome77, unfortunately, this is out of scope for us, as we avoid developing models using notebooks. Sorry about this. Feel free to contribute one if you end up going down that route.


fix prometheus-operator-crd chart namespace

Towards: https://github.com/giantswarm/giantswarm/issues/20256
This solves an issue where CRDs are already installed but do not belong to any chart. We need to get the prometheus-operator-crd chart namespace in order to fake that it already belong to this chart.
Only requirements here is to make sure the prometheus-operator-crd is installed before this.
To test:

[ ] Install from scratch works (install crd then app)
[ ] Upgrade on MC works (upgrade app)
[ ] Upgrade on WC works (install crd, then upgrade app and upgrade app then install crd)

namespace fix works
$ km logs -f prometheus-operator-app-crd-install-gvcxk
+ set -o nounset
+ exec
+ kubectl get chart '--namespace=giantswarm' prometheus-operator-crd '--output=jsonpath={.spec.namespace}'
I0124 14:55:21.194494       7 request.go:665] Waited for 1.17371139s due to client-side throttling, not priority and fairness, request: GET:https://<IP_ADDRESS>:443/apis/policy/v1beta1?timeout=32s
I0124 14:55:31.394023       7 request.go:665] Waited for 11.372932002s due to client-side throttling, not priority and fairness, request: GET:https://<IP_ADDRESS>:443/apis/templates.gatekeeper.sh/v1beta1?timeout=32s
I0124 14:55:41.394493       7 request.go:665] Waited for 3.197596121s due to client-side throttling, not priority and fairness, request: GET:https://<IP_ADDRESS>:443/apis/autoscaling/v2beta2?timeout=32s
I0124 14:55:51.594299       7 request.go:665] Waited for 13.397069354s due to client-side throttling, not priority and fairness, request: GET:https://<IP_ADDRESS>:443/apis/image.toolkit.fluxcd.io/v1alpha2?timeout=32s
+ namespace=monitoring
+ tr '\n' ' '
+ kubectl api-resources '--api-group=monitoring.coreos.com' -oname
+ xargs kubectl patch '--record=false' -p '{"metadata":{"annotations":{"meta.helm.sh/release-name": "prometheus-operator-crd","meta.helm.sh/release-namespace": "monitoring"},"labels":{"app.kubernetes.io/managed-by": "Helm"}}}' crd
Flag --record has been deprecated, --record will be removed in the future
I0124 14:55:58.101161      24 request.go:665] Waited for 1.169666346s due to client-side throttling, not priority and fairness, request: GET:https://<IP_ADDRESS>:443/apis/networking.k8s.io/v1?timeout=32s
I0124 14:56:08.300236      24 request.go:665] Waited for 11.368352481s due to client-side throttling, not priority and fairness, request: GET:https://<IP_ADDRESS>:443/apis/templates.gatekeeper.sh/v1alpha1?timeout=32s
I0124 14:56:18.300271      24 request.go:665] Waited for 2.995868324s due to client-side throttling, not priority and fairness, request: GET:https://<IP_ADDRESS>:443/apis/events.k8s.io/v1beta1?timeout=32s
I0124 14:56:28.300304      24 request.go:665] Waited for 12.995533354s due to client-side throttling, not priority and fairness, request: GET:https://<IP_ADDRESS>:443/apis/release.giantswarm.io/v1alpha1?timeout=32s
customresourcedefinition.apiextensions.k8s.io/alertmanagerconfigs.monitoring.coreos.com patched
customresourcedefinition.apiextensions.k8s.io/alertmanagers.monitoring.coreos.com patched
customresourcedefinition.apiextensions.k8s.io/podmonitors.monitoring.coreos.com patched
customresourcedefinition.apiextensions.k8s.io/probes.monitoring.coreos.com patched
customresourcedefinition.apiextensions.k8s.io/prometheuses.monitoring.coreos.com patched
customresourcedefinition.apiextensions.k8s.io/prometheusrules.monitoring.coreos.com patched
customresourcedefinition.apiextensions.k8s.io/servicemonitors.monitoring.coreos.com patched
customresourcedefinition.apiextensions.k8s.io/thanosrulers.monitoring.coreos.com patched


Clarify subdomain activation (multi-tenant) : remove useless(?) objects

In section "Register a subdomain name in a customer tenant", once the subdomain is activated,
Could you clarify if we can :

Remove the DNS TXT entry
Remove the unique user associated with the domain

Thanks!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1cb0c2ef-6274-9167-ecc6-3695d7460a8d
Version Independent ID: aacf7318-02fc-a1c3-369c-65e6936fdb3c
Content: Configure Session Border Controller - Multiple tenants - Microsoft Teams
Content Source: Teams/direct-routing-sbc-multiple-tenants.md
Service: msteams
GitHub Login: @CarolynRowe
Microsoft Alias: crowe

Hi @fdelavier,
After you have verified a domain in Microsoft 365 or Office 365, you can safely remove the TXT record.
After you successfully added the SBC to your tenant, you can remove the user you added to activate the domain. (or assign a different SIP address)

Thanks @dariomws !
Would it be possible to add this information in the documentation ?

@fdelavier, the steps to add a domain to Office 365 are detailed here but if you want to have more detailed information about this process, I suggest that you review more specific articles like:
https://docs.microsoft.com/microsoft-365/admin/setup/add-domain
About the removal of the user, I think there is a related note:

[!NOTE]
The Carrier tenant must keep at least one Phone System license assigned to the tenant to avoid removal of the Skype for Business configuration.

Hi @dariomws,
Actually the main need is to clarify if we can remove the user associated to the customer tenant.
This seems to be very much related to multi-tenant subdomain particularities (subdomain not used by o365 users, beyond activation).
Wouldn't it be better to detail this in Teams/direct-routing-sbc-multiple-tenants.md ?

@fdelavier, I understand that the same applies to customer tenants. But in that case, you will keep a Phone System license for sure.

Hi @dariomws,
I'm not convinced. On the contrary the requirement to keep the user seems to be specified only for the carrier tenant.
Note: I don't understand why the user in the carrier tenant is needed "to avoid removal of the Skype for Business configuration". As far I know it is/was just required in order to validate new derived trunk FQDNs.

@fdelavier, while there are no official articles that provide another way to activate the domain, the information in this article is all that I can say. To activate the domain using a supported process you need to follow the steps in this article.
In the same way, you need to keep the license in the carrier tenant. This is what you need to keep, at least one Phone System license assigned to the tenant.


chore: @[simp] List.getElem?_eq_getElem

I'm a little skeptical of this one, but it is often helpful. Let's see what happens to Mathlib.

Mathlib is remarkably unconcerned. I needed to change a few proof of deprecated (or perhaps-should-be-deprecated) theorems, but otherwise it is fine.


AGIC silently bypassed the networking failure during the pod initialization

Describe the bug
A clear and concise description of what the bug is.
In v1.22 cluster, when the pod v1.5.0-rc starts to discover server version, if there is networking issue like failed to connect to the apiserver, AGIC silently bypass it instead of raising exception. This may result the cache sync failed because the pod keeps looking for 'v1beta1.Ingress'
eg. https://github.com/Azure/application-gateway-kubernetes-ingress/blob/1.5.0-rc1/pkg/k8scontext/supported_apiversion.go#L41
To Reproduce
Steps to reproduce the behavior:

deploy AGIC
block the connection to apiserver from the AGIC pod via iptables
restart the container of AGIC
checking the logs we could find message like below. It should not use 'v1beta1' any more in v1.22.
I0211 09:39:48.815944       1 reflector.go:255] Listing and watching *v1beta1.Ingress from pkg/mod/k8s.io/client-go@v0.21.2/tools/cache/reflector.go:167

Ingress Controller details

Output of kubectl describe pod <ingress controller> . The  pod name can be obtained by running helm list.
Output of `kubectl logs .
Any Azure support tickets associated with this issue.

@Shuanglu thanks for raising the issue. Will take this issue in the next minor release of AGIC.
If you are comfortable, you are welcome to help with a fix!


Do not start BuildSignReconciler in managed mode

We do not build any image in managed mode.
/cc @ybettan @yevgeny-shnaidman

/lgtm


[SR-4196] Allow #if (conditional compilation) to guard switch cases

Previous ID
SR-4196

Radar
rdar://problem/16980851

Original Reporter
@belkadan

Type
Improvement

Status
Resolved

Resolution
Done

Additional Detail from JIRA

Votes
0

Component/s
Compiler

Labels
Improvement, LanguageFeatureRequest, StarterProposal

Assignee
@rintaro

Priority
Medium

md5: c3d4da1a290a1496adc214b1fce9223b

relates to:

SR-4079 Availability of enum case does not propagate to its switch case scope.
SR-2 Build configuration directives can not wrap switch cases

Issue Description:
Currently #if can only be used to guard complete declarations or complete statements, due to the (deliberate) modeling as part of the AST rather than as a textual preprocessing step. However, there's one more place where it would structurally make sense to allow #if: switch cases. This may be important if a new case is added in a new version of Swift. Right now you'd have to write the switch twice.

PR: #9457


Lifetime issue when storing a popo::Sample

Required information
Operating system:
Any
Compiler version:
Any
Observed result or behaviour:
Comes from this comment:

You have a livetime issue here. See this code:
SamplePointer<SampleType>&& acquireSample() {
 EventPublisher<Topic> publisher;
 return std::move(publisher.Loan());
}

int main() {
  auto sample = acquireSample();
}

In the end you can construct a lot of examples where the EventPublisher goes out of scope before SamplePointer does and then one has a problem. I think this is another problem with the automotive API where one has to state at least a big red warning that the EventPublisher has to live as least as long as the SamplePointer.
But just so that you are aware how bad this problem is, take a look at this evil bug:
class MyClass {
  SamplePointer<SampleType> m_sample;
  EventPublisher<Topic> m_publisher;
}

This ordering of member variables is evil, when the MyClass destructor destroys the member it does it in inverse order, first the m_publisher and then the m_sample and one has again undefined behavior. I would bet in a code review no one would have seen this.
There are a lot of bugs out there where applications cannot terminate cleanly and cause leaks or segfaults because of this issue.

Expected result or behaviour:
Add a notification in each of the destructors like it is done for WaitSet and Listener.
Conditions where it occurred / Performed steps:
See above

@mossmaurice this is a duplicate of #560 if I understand this correct


Move imported files to "/en/publish"

Imported files go into /en/publish

:tada: This issue has been resolved in version 1.18.0 :tada:
The release is available on:

v1.18.0
GitHub release

Your semantic-release bot :package::rocket:


[FLINK-17542] Unify slot request timeout handling for streaming and batch tasks

What is the purpose of the change
This PR is to unify slot request timeout handling for streaming and batch tasks regarding whether the slot will be occupied indefinitely.
It is a pre-requisite of bulk slot allocation so that we do not need to maintain 2 different slot timeout handling.
Brief change log
See commits.
Verifying this change

Added unit tests for slot occupation
Existing SlotPoolBatchSlotRequestTest and SlotPoolPendingRequestFailureTest help to verify the change on batch and streaming jobs

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@flinkbot run azure


bp3: commands for exporting and testing schemas

Implement schemas export <folder> and schemas testcompat <folder>, as described here

use cobrau in bp3

no commands -> all should work like before
schemas cmd is provided -> only export and testcompat subcommands must be available


Prevent NPE if log was called without a message

We had some exception with a null message come up in some obscure situation and the logging couldn't really cope with that. So here's a suggested fix.

Thanks! LGTM


Iodine.getErrorMessage not working.

Hello!
There is no Iodine.getErrorMessage API in iodine v8.
Can you add it on the next versions?

You can use the _error() method to get the error message.


Bypass MAPL_LoadBalance when not running load balancing

This PR changes the load balancing code in the Solar GridComp to avoid calls to MAPL_LoadBalance if not running with load balancing on.
As found by @wmputman, if you set SOLAR_LOAD_BALANCE: 0 you technically still call the MAPL_LoadBalance code which involves MPI_Allgather, MPI_Send and MPI_Recv calls. Trivial though they are, at high-res they can be expensive.
So, what we do here is make some changes to avoid this. Testing shows this change is zero-diff if you run with load balancing on. Yay!
Now, if you turn off the load balancing, the answers are NOT zero-diff. That is because currently this code:
ZTH     = max(.0001,ZTH)

is run everwhere which changes some points with very small zenith-angles. This actually only needs to be done for the SCM cases where there are NO lit points (because the column is in the dark). So we protect that to run only if DATMO is running.
Now, if we compare runs with load balancing ON to those with OFF, they are zero-diff after a day EXCEPT for some normalized internal state variables:
Comparing NC4 solar_internal_checkpoint using nccmp...
Failure!
Checking for data differences
Variable   Group  Count       Sum    AbsSum         Min          Max      Range        Mean      StdDev
CLDHISW    /       1561 1.561e+18 1.561e+18       1e+15        1e+15          1       1e+15 1.16137e+08
CLDLOSW    /       1561 1.561e+18 1.561e+18       1e+15        1e+15          1       1e+15 1.17717e+08
CLDMDSW    /       1561 1.561e+18 1.561e+18       1e+15        1e+15          1       1e+15 1.16137e+08
CLDTTSW    /       1561 1.561e+18 1.561e+18       1e+15        1e+15          1       1e+15 1.17717e+08
COSZSW     /       1561 1.561e+18 1.561e+18       1e+15        1e+15          0       1e+15 1.16137e+08
DFNIRN     /       1561  -210.159   210.159   -0.229973   -0.0242775   0.205695   -0.134631   0.0417412
DFPARN     /       1561  -226.938   226.938   -0.192311   -0.0311866   0.161124    -0.14538   0.0353308
DFUVRN     /       1561  -7.42515   7.42515 -0.00831096 -0.000428814 0.00788214 -0.00475666  0.00130972
FSCN       /     113953  -41096.2   41096.2   -0.500455   -0.0753155   0.425139   -0.360642   0.0676657
FSCNAN     /     113953  -41203.9   41203.9   -0.501528   -0.0753155   0.426212   -0.361587   0.0679054
FSCUN      /     113953   -9040.4    9040.4   -0.782568   -0.0230165   0.759551  -0.0793344   0.0827449
FSCUNAN    /     113953  -8954.53   8954.53   -0.782568   -0.0234314   0.759136  -0.0785809   0.0830617
FSWBANDN   /      21854  -367.557   367.557   -0.105806 -1.09816e-30   0.105806  -0.0168187   0.0263242
FSWBANDNAN /      21854  -369.813   369.813   -0.106177  -1.1013e-30   0.106177   -0.016922   0.0264817
FSWN       /     113953  -33996.6   33996.6   -0.496376   -0.0519393   0.444436   -0.298338   0.0810884
FSWNAN     /     113953  -34065.8   34065.8   -0.496602   -0.0519423    0.44466   -0.298946   0.0812701
FSWUN      /     113953  -14327.1   14327.1   -0.782568  -0.00404456   0.778523   -0.125728   0.0949151
FSWUNAN    /     113953  -14274.8   14274.8   -0.782568  -0.00404694   0.778521   -0.125269   0.0950981
TAUHIPAR   /       1561 1.561e+18 1.561e+18       1e+15        1e+15         15       1e+15 9.70717e+07
TAULOPAR   /       1561 1.561e+18 1.561e+18       1e+15        1e+15    146.375       1e+15        -nan
TAUMDPAR   /       1561 1.561e+18 1.561e+18       1e+15        1e+15    115.875       1e+15        -nan
TAUTTPAR   /       1561 1.561e+18 1.561e+18       1e+15        1e+15    108.125       1e+15        -nan
...

Comparing NC4 surf_import_checkpoint using nccmp...
Failure!
Checking for data differences
Variable Group Count      Sum  AbsSum         Min          Max      Range        Mean     StdDev
DFNIRN   /      1561 -210.159 210.159   -0.229973   -0.0242775   0.205695   -0.134631  0.0417412
DFPARN   /      1561 -226.938 226.938   -0.192311   -0.0311866   0.161124    -0.14538  0.0353308
DFUVRN   /      1561 -7.42515 7.42515 -0.00831096 -0.000428814 0.00788214 -0.00475666 0.00130972

Every other restart is zero-diff. Thus, these variables must be diagnostic but included in the internal state for regression purposes? Not sure.
Also, some...odd things seem to be happening with some variables in the solar internal state (CLD???W and TAU??PAR). I might need to consult @dr0cloud about this as maybe I screwed something up? Or not? Didn't seem to affect the rest of the model!

Thanks @mathomp4 : your findings on the internal checkpoints is interesting. Those are the no aerosol exports from solar, which is also a burden since it basically calls soradcore twice even when we don't want the clear-sky exports.

Hi Matt and Bill, I’ll take a look shortly. Not all of those are no-aerosol internals. It should NOT call SORADCORE twice if no non-aerosol exports are required. But it may reserve space in the internal state for them. Just grabbing a bite to eat and will take a look. P.
Peter Norris, Ph.D.
Senior Research Scientist
Global Modeling and Assimilation Office (GMAO) / NASA Goddard Space Flight Center
Goddard Earth Sciences Technology and Research (GESTAR) II / University of Maryland, Baltimore County (UMBC)
Cell<PHONE_NUMBER>
From: Bill Putman @.>
Reply-To: GEOS-ESM/GEOSradiation_GridComp @.>
Date: Monday, December 4, 2023 at 12:30 PM
To: GEOS-ESM/GEOSradiation_GridComp @.>
Cc: "Norris, Peter M. (GSFC-610.1)[UNIVERSITY OF MARYLAND BALTIMORE CO]" @.>, Mention @.***>
Subject: [EXTERNAL] [BULK] Re: [GEOS-ESM/GEOSradiation_GridComp] Bypass MAPL_LoadBalance when not running load balancing (PR #27)
CAUTION: This email originated from outside of NASA.  Please take care when clicking links or opening attachments.  Use the "Report Message" button to report suspicious messages to the NASA SOC.
Thanks @mathomp4https://github.com/mathomp4 : your findings on the internal checkpoints is interesting. Those are the no aerosol exports from solar, which is also a burden since it basically calls soradcore twice even when we don't want the clear-sky exports.
—
Reply to this email directly, view it on GitHubhttps://github.com/GEOS-ESM/GEOSradiation_GridComp/pull/27#issuecomment-1839132894, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALBN7E7GLWXXYLHWQ24FM4DYHYCBLAVCNFSM6AAAAABAGKYZASVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZZGEZTEOBZGQ.
You are receiving this because you were mentioned.Message ID: @.***>


feat(sqlparser): support specifying DEBEZIUM JSON for CDC tables to enable schema change

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
Follow up of #12164.
If we create a CDC table with..
CREATE TABLE t (..) WITH (connector = 'postgres-cdc', ..);

the definition will be normalized and persisted as..
CREATE TABLE t (..) WITH (connector = 'postgres-cdc', ..) FORMAT DEBEZIUM ENCODE JSON;

which is not accepted as a valid SQL because we expect FORMAT and ENCODE omitted here.
However, we'll reuse the SQL for schema change. As a result, altering a CDC table will lead to error:
unable to parse original table definition:
  sql parser error:
    Row format for cdc connectors should not be set here because it is limited to debezium json

So this PR changes the behavior: we expect the FORMAT to be either omitted or exactly the same as the default value. This enables schema change on CDC tables.

An alternative could be introducing an internal flag for the parser to conditionally accept this fully qualified syntax. However, by treating omitting this clause as a syntax sugar, I guess it's more reasonable to also accept both of them.

Also refine the display of anyhow error, and cleanup duplicated codes.
Checklist

[x] I have written necessary rustdoc comments
[x] I have added necessary unit tests and integration tests
[x] All checks passed in ./risedev check (or alias, ./risedev c)

Documentation

[x] My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

Allow optional FORMAT DEBEZIUM ENCODE JSON after the connector definition of CDC tables.
Allow optional FORMAT NATIVE ENCODE NATIVE after the connector definition of Nexmark sources or tables.

Only added tests for the new syntax.
I find it painful to write e2e tests for the CDC schema change. Will investigate simplifying this procedure in the next PRs.

then how do you handle a debezium json source with schema registry 🥲

then how do you handle a debezium json source with schema registry 🥲

I'm not quite familiar with this part. Could you please elaborate more?

then how do you handle a debezium json source with schema registry 🥲

I guess they're unrelated? CDC sources won't leverage a schema registry.


🐛Bug: Date Field - Can't input times starting with 0

Describe the bug
When using the date field it is currently not possible to input times that start with zeroes (e.g. 10:00). To input the minute part of the time you currently have to click a different number and then overwrite it with a zero.
To Reproduce
Create a date field with times. -> Input a time starting with zero.
Expected behavior
Be able to input zeros in the time field without having to input different numbers first.
Desktop (please complete the following information):

OS: Windows
Browser: Brave
Version: 1.61.109

I have created a PR for fixing this bug https://github.com/melt-ui/melt-ui/pull/893


What can I do to the apollo_query_execution_failed metric be counted?

I was looking this part of code that increase the metric apollo_query_execution_failed:
executionDidStart(context) {
  actionMetric(MetricsNames.QUERY_EXECUTION_STARTED, getLabelsFromContext(context));

  return {
    willResolveField(field) {
      const fieldResolveStart = Date.now();

      return () => {
        const fieldResolveEnd = Date.now();

        actionMetric(
          MetricsNames.QUERY_FIELD_RESOLUTION_DURATION,
          {
            ...getLabelsFromContext(context),
            ...getLabelsFromFieldResolver(field)
          },
          fieldResolveEnd - fieldResolveStart
        );
      };
    },
    executionDidEnd(err) {
      if (err) {
        actionMetric(MetricsNames.QUERY_EXECUTION_FAILED, getLabelsFromContext(context));
      }
    }
  };
}

The hook executionDidEnd should supposedly work? I noted it is after return, is it correct?  If yes, could you show me an example where this metrics will be count?

Closing this for inactivity. Feel free to reopen it if you steel need any info.


[feature request] How about implementing an application load balancer for OpenYurt?

What would you like to be added:
An application load balancer that handles requests taking consideration of the current loads (CPU/MEM/GPU/...) of cluster.
Why is this needed:
To get good performance and optimal utilization of nodes.
others
/kind feature

@LindaYu17 Thank you for filing issue.

An application load balancer that handles requests taking consideration of the current loads (CPU/MEM/GPU/...) of cluster.

application load balancer is used for pods on edge node or used for kube-apiserver on master nodes? would you describe load balancer requirements in more detail?

It is used for pods on edge node.
Existing load balancer in K8S are mainly distributing requests from user by traffic randomly, the application load balancer I mentioned here should be able to distribute requests according to current system conditions and user defined rules. When the workloads the client requests for are sustaining resource/time consuming, such as video analytics, cloud gaming... if we dispatch such kind of workloads randomly, we don't guarantee to gain optimal performance. When dispatching this kind of request, we need to consider more, such as CPU/GPU/MEM usage. The application load balancer is designed to bridge this gap.

It is used for pods on edge node.
Existing load balancer in K8S are mainly distributing requests from user by traffic randomly, the application load balancer I mentioned here should be able to distribute requests according to current system conditions and user defined rules. When the workloads the client requests for are sustaining resource/time consuming, such as video analytics, cloud gaming... if we dispatch such kind of workloads randomly, we don't guarantee to gain optimal performance. When dispatching this kind of request, we need to consider more, such as CPU/GPU/MEM usage. The application load balancer is designed to bridge this gap.

@LindaYu17 Thank you for feedback.
In K8s ecosystem, Service is used as load balancer for pods on nodes. Do you have ideas that how to integrate k8s service with application load balancer?

Sounds App LB is a kind of enhancement to K8s Service? The two components have some overlap in terms of doing reverse proxy for incoming requests. However, for ALB, user can add rules through API (CR) to explicitly specify how to do load balancing, such as what devices to prioritize, what algorithm to use, etc. For scenarios like numrous channels of video analytics, dispaching requests to right nodes/pods needs a sophisticated LB.

Sounds App LB is a kind of enhancement to K8s Service? The two components have some overlap in terms of doing reverse proxy for incoming requests. However, for ALB, user can add rules through API (CR) to explicitly specify how to do load balancing, such as what devices to prioritize, what algorithm to use, etc. For scenarios like numrous channels of video analytics, dispaching requests to right nodes/pods needs a sophisticated LB.

@LindaYu17 Based on the above the comments from you and @gnunu , application load balancer sounds like a valuable feature in the edge scenario.
Would you like to make a proposal of application Load Balancer? so we can make a detailed discussion about it based on the proposal.

Sounds App LB is a kind of enhancement to K8s Service? The two components have some overlap in terms of doing reverse proxy for incoming requests. However, for ALB, user can add rules through API (CR) to explicitly specify how to do load balancing, such as what devices to prioritize, what algorithm to use, etc. For scenarios like numrous channels of video analytics, dispaching requests to right nodes/pods needs a sophisticated LB.

@LindaYu17 Based on the above the comments from you and @gnunu , application load balancer sounds like a valuable feature in the edge scenario.
Would you like to make a proposal of application Load Balancer? so we can make a detailed discussion about it based on the proposal.

sure, we are preparing ALB proposal right now, and will submit once it is done.

/pinned

/pinned

/pinned


Support transparency on encode

Not fixed by #23. Quite complicated. Not sure if I'll get around to it, as Kitty doesn't need it.

closing as duplicate of #15.


Dyno type instructions are misleading / broken

retinalols:transferatu russ$ heroku dyno:type hobby web=hobby
 !    Cannot mix hobby with Free dynos.
 !    To change all dynos to hobby, run `heroku ps:type hobby`.
retinalols:transferatu russ$ heroku dyno:type hobby
 !    Requested type hobby is not available in this dyno type.
retinalols:transferatu russ$

the line:
To change all dynos to hobby, run `heroku ps:type hobby`.

doesn't work.

@brandur are those api changes in yet?

Nope :/ heroku/api#4426

this is fixed


update gisce/ooui to v2.0.1

Automated changes by create-pull-request GitHub action

:tada: This PR is included in version 2.0.7 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Update to googleapis/release-please-action@v4

…gleapis/release-please-action
📝 Description

Issue: Url to Github Issue
📋 List of Changes

The previous release-please action has been deprecated, moving to googleapis/release-please-action@v4

🧪 Steps to Test

Step 1

Notes to the Reviewer

📸 Screenshots

✔ Dev Complete Checklist

[ ] PR template filled out
[ ] Change is tested by submitter
[ ] PR follows all linting and coding standards
[ ] Github Issue exists (if applicable)
[ ] Team member has been assigned
[ ] At least one commit message is in Conventional Commit format

LGTM


Remove dependency on named-regexp

named-regexp is a backport of named capture groups, which were introduced in Java 7. This library targets Java 8, so the backport is unnecessary.
See https://github.com/tony19/named-regexp/issues/73#issuecomment-1897810289

Looks like named-regexp was added in #215 to fix a bug for Android < 26.


Engagement: manage survey, recruited via open call

Manage engagement, empty: https://www.figma.com/file/AXufBnFAVHvGdsxnQMVhNF/The-Accessibility-Exchange-Wireframes?node-id=3326%3A101223
Manage engagement, partially filled: https://www.figma.com/file/AXufBnFAVHvGdsxnQMVhNF/The-Accessibility-Exchange-Wireframes?node-id=3326%3A101333
Manage engagement, filled: https://www.figma.com/file/AXufBnFAVHvGdsxnQMVhNF/The-Accessibility-Exchange-Wireframes?node-id=3326%3A101280

Add engagement details:
https://www.figma.com/file/AXufBnFAVHvGdsxnQMVhNF/The-Accessibility-Exchange-Wireframes?node-id=2613%3A70683
Manage participants - confirmed participants: https://www.figma.com/file/AXufBnFAVHvGdsxnQMVhNF/The-Accessibility-Exchange-Wireframes?node-id=2697%3A69398
Manage participants - access needs: https://www.figma.com/file/AXufBnFAVHvGdsxnQMVhNF/The-Accessibility-Exchange-Wireframes?node-id=2697%3A69981
Participant RSVPing
Engagement page: https://www.figma.com/file/AXufBnFAVHvGdsxnQMVhNF/The-Accessibility-Exchange-Wireframes?node-id=2723%3A71437
Confirm access needs, because this is an engagement that is asynchronous (survey), therefore the first contact the FRO may have with the participant may be already with the engagement materials: https://www.figma.com/file/AXufBnFAVHvGdsxnQMVhNF/The-Accessibility-Exchange-Wireframes?node-id=2727%3A71938
Editing access needs: https://www.figma.com/file/AXufBnFAVHvGdsxnQMVhNF/The-Accessibility-Exchange-Wireframes?node-id=2727%3A72515 (this will change their access needs on their settings > access needs page as well)
Permission to share identity - for asynchronous engagements this applies to everyone, because the FRO will need to know who to send translated or alt versions to: https://www.figma.com/file/AXufBnFAVHvGdsxnQMVhNF/The-Accessibility-Exchange-Wireframes?node-id=2745%3A75231


Adds metadata views to FlowToken and removes init parameter

Adds standard fungible token metadata views to the FlowToken smart contract
Also removes the init parameter from the contract so that it is easier for users to deploy it to the playground
Currently the tests are failing because the emulator and FVM deploys FlowToken as part of bootstrap, but because it requires the FungibleTokenMetadataViews contract now, the deployment fails because it currently isn't also deploying that contract
I also need feedback on the specific parts of the FTDisplay view

Tests are passing, but I'm having a hard time getting CI to pass because there are diffs in lib/go/test/go.sum that I don't know how to resolve properly. I have to get to some other things, so I will come back to this in a couple days


[Question] Will work on ios simulator ?

Hi, do you know if it can / will work on ios simulator ?
There is some limitations ?
thank you.

Unfortunately, Image Playground is not supported in iOS simulator. You'll need a physical device with Apple Intelligence capabilities.


How to implement a custom AuthenticationStateProvider in server-side Blazor

Actually I want to know which class I should derive from: AuthenticationStateProvider or ServerAuthenticationStateProvider or RevalidatingServerAuthenticationStateProvider ? I'm confused !
How to add this class to the DI container ?
Many thanks !

Many thanks, @pranavkm. I got it...


FIX relative path to absolute as defined in https://docs.docker.com/e…

…ngine/userguide/containers/dockervolumes/#mount-a-host-directory-as-a-data-volume

Hey @marcelmfs, thanks.
This PR seems very related to #8. Are you related to @cocoon-project? If yes, it would be best to keep only one PR open, otherwise please leave here only the code that is related to this PR.


KAFKA-15576: Add 3.6 to core upgrade and compatibility tests

This is part of the 3.6.0 release process as mentioned here.
For feature releases, file a JIRA for updating compatibility/upgrade system tests to test the newly released version.

Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

I restarted the jenkins job as lots of tests failed at the initialization step

Seems this does not cover KS -- btw: there is some issue for KS system tests right now, and as a matter of fact, we did not even add the missing upgrade tests in older release yet (PR is still pending).
I am ok with merging this PR, but we should not close the JIRA until everything is resolved IMHO.

Seems this does not cover KS -- btw: there is some issue for KS system tests right now, and as a matter of fact, we did not even add the missing upgrade tests in older release yet (PR is still pending).

Right, this change is related to core upgrade and compatibility tests. Updated the PR description for clarity. I saw that KS related updates for earlier releases are in progress as part of https://github.com/apache/kafka/pull/13860

I am ok with merging this PR, but we should not close the JIRA until everything is resolved IMHO.

Sure, created two separate sub tasks on KAFKA-15576 for tracking them separately and close the parent JIRA once both the sub tasks are resolved.


[Bug] Advanced startup navigation causes crash

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Use advanced navigation by resolving a service to decide the first navigation:

burkusMvvm.OnStart(async (navigationService) =>
{
    var appStartupService = ServiceResolver.Resolve<IAppStartupService>();
    await appStartupService.NavigateToFirstPage();
});

Observe crash

Expected behavior
The application allows you to resolve dependencies to decide on the navigation.
Screenshots

Additional context
Add any other context about the problem here.

May end up being fixed by #15

Reopening as it still appears to occur in some projects

Closing again as it turned out to be some custom, platform specific WinUI code in a project that was causing the crash


Skeletons don't allow multiple IK nodes to run

Godot version:
master (https://github.com/godotengine/godot/commit/495b28765d03aa9459bc312ebba190ec208979e7) - applies to 3.2 as well
OS/device including version:
Windows 10
Issue description:
If there are multiple SkeletonIK nodes on a Skeleton, only one will run. Looking into it, in scene/3d/skeleton_ik_3d.cpp, clear_bones_global_pose_override() is called on the skeleton when an IK chain is being solved, resetting any previous IK. I believe it should only be resetting the global pose on the bones in the chain being solved.
Steps to reproduce:
Set up two SkeletonIK nodes for a Skeleton (ie left arm and right arm) and create a script to run start() on both IK nodes.
Minimal reproduction project:
Multiple IK Bug.zip

@AndreaCatania I have this change (https://github.com/woollysammoth/godot/commit/a765f12bfe129bb79bb4f385601afecc48ebd1f0) that fixes the issue for me, if it looks ok I can submit a PR

@woollysammoth yep, in that way only the "involved" nodes are cleared out. A PR would be much welcome.

@AndreaCatania Thanks for looking into it, sent in the PR

For some reason the PR by @woollysammoth #38203 hasn't been tagged properly so there might the possibility of overlooking the PR when ironing out bugs...?


TypeError: init() got multiple values for keyword argument 'width'

The TypeError message is thrown out with facet_grid, the graphics is not displayed. See the cell 16 of this notebook:
https://github.com/Arn-O/udacity-intro-data-science/blob/master/subway_viz2.ipynb

+1

+1 also

fixed in 0.9.3

That was much expected. It works, thanks a lot!
Ar.


QR code not showing up

I installed the add on and the QR code does not show up when the Ankimote button is pressed

Can you provide the following:

Anki version
Operating system
Does the QR still not show up if all other addons are disabled?
Does the QR not show up if you press the ankimote button on the decks screen? How about if you press it on the reviewer screen with a flashcard shown?


PUB's peer mode for remote tunnel

For the sake of these plugins' issues, requesting for remote tunnels:

Dual cluster setup
Access jobs through tunneled ports

it would be nice to support remote "tunnels" natively by Racetrack, that is, let the plugins to set up the communication between "racetrack" host and the "jobs" host.
I figured out that we don't necessarily need encrypted Wireguard tunnel or SSH tunnel. Simple authorized proxy / backdoor / bastion (whatever you call it) should be enough. Furthermore, setting up the Wireguard might not even be possible, if kubernetes' ingress controller only allows for the HTTP traffic (due to service routing by a base path).

The idea is to turn the remote PUB into a "peer" mode that forwards the requests to the jobs, only if the auth header (for inter PUB communication) is valid. That way, you can't speak to the jobs directly in an unauthorized manner.
Setting up the "tunnel" (actually, it's not a true encrypted tunnel) needs the following steps:

configure PUBs properly (set host addresses, choose mode)
Install peer PUB (pod / container) on infrastructure target
Install infrastructure plugin
Configure infra plugin: k8s certificate (or other thing depending on the plugin), kube-api address, peer PUB address

Note that extending the PUB's functionality and installing additional services is a big thing, it goes beyond the plugin's responsibilities (as it is now), so the manual steps are required.

we don't necessarily need encrypted Wireguard tunnel or SSH tunnel

It would be very nice though. Are there advantages to not using encryption?

setting up the Wireguard might not even be possible, if kubernetes' ingress controller only allows for the HTTP traffic

Is that normal practise in k8s? Keep in mind, an ITP can require reasonable things from targets. So if it's normal practise for an ingress controller to only allow HTTPS, then we can require in the k8s ITP that targets do this. Then if someone doesn't, we don't support that and anyhow they're deviating from normal and good practises.

Are there advantages to not using encryption?

The only advantage is simplicity and probably easier setup.

Is that normal practise in k8s?

I guess that normal practise is to use NodePort (instead of ingress controller) to expose services outside with Wireguard: https://github.com/ivanmorenoj/k8s-wireguard. This setup looks not really infrastructure-independent though. I'd love to find something that works on both k8s and docker daemon, if possible.

Here's the updated diagram:


typo : fixing double word use

Fixing the typo double use of words: "to" and "the"
Signed-off-by: Daniel Pisanu<EMAIL_ADDRESS>

hmmm. strange that the DCO check is failing.


Physics simulation for Qiskit circuits

What
Physics simulation for Qiskit circuits
Why
Simulating qiskit circuits by solving full physical dynamics of the system model (incl of classical electronics and imperfections) provies an easy interface to the rich simulator of c3-toolset
How
Use the (now no longer) new methods for evaluating qasm representation of sequences
Remarks
Add notes on possible known quirks/drawbacks of this solution. If this introduces an API-breaking change, please provide an explanation on why it is necessary to break API compatibility and how users should update their notebook/script workflows once this PR is merged.
Checklist
Please include and complete the following checklist. Your Pull Request is (in most cases) not ready for review until the following have been completed. You can create a draft PR while you are still completing the checklist. Check the Contribution Guidelines for more details. You can mark an item as complete with the - [x] prefix

[ ] Tests - Added unit tests for new code, regression tests for bugs and updated the integration tests if required
[x] Formatting & Linting - black and flake8 have been used to ensure styling guidelines are met
[x] Type Annotations - All new code has been type annotated in the function signatures using type hints
[ ] Docstrings - Docstrings have been provided for functions in the numpydoc style
[ ] Documentation - The tutorial style documentation has been updated to explain changes & new features
[ ] Notebooks - Example notebooks have been updated to incorporate changes and new features
[ ] Changelog - A short note on this PR has been added to the Upcoming Release section

When we talk next time, please explain this to me.
On Wed, 29 Dec 2021, 5:03 Anurag Saha Roy, @.***> wrote:

What
Physics simulation for Qiskit circuits
Why
Simulating qiskit circuits by solving full physical dynamics of the system
model (incl of classical electronics and imperfections) provies an easy
interface to the rich simulator of c3-toolset
How
Use the (now no longer) new methods for evaluating qasm representation of
sequences
Remarks
Add notes on possible known quirks/drawbacks of this solution. If this
introduces an API-breaking change, please provide an explanation on why it
is necessary to break API compatibility and how users should update their
notebook/script workflows once this PR is merged.
Checklist
Please include and complete the following checklist. Your Pull Request is
(in most cases) not ready for review until the following have been
completed. You can create a draft PR while you are still completing the
checklist. Check the Contribution Guidelines
https://github.com/q-optimize/c3/blob/dev/CONTRIBUTING.md for more
details. You can mark an item as complete with the - [x] prefix

Tests - Added unit tests for new code, regression tests for bugs and
updated the integration tests if required
Formatting & Linting - black and flake8 have been used to ensure
styling guidelines are met
Type Annotations - All new code has been type annotated in the
function signatures using type hints
Docstrings - Docstrings have been provided for functions in the
numpydoc style
Documentation - The tutorial style documentation has been updated to
explain changes & new features
Notebooks - Example notebooks have been updated to incorporate
changes and new features
Changelog - A short note on this PR has been added to the Upcoming
Release section

You can view, comment on, or merge this pull request online at:
https://github.com/q-optimize/c3/pull/165
Commit Summary

7744aca
https://github.com/q-optimize/c3/pull/165/commits/7744acacfe312d93287c1fc16a76fe6a70cc4ddf
use name field like single qubit gates
286edcc
https://github.com/q-optimize/c3/pull/165/commits/286edcc604c5910621a702bae19cc2f32ee66bfa
ensure device has enough qubits for circuit
86b2b5d
https://github.com/q-optimize/c3/pull/165/commits/86b2b5d1c4ff67e01e3aad25f8d04e744ab9b211
ad-hoc hardcoded basis gates
e9eecd5
https://github.com/q-optimize/c3/pull/165/commits/e9eecd523ea4ad86e341fe812c3e51089e2a26a8
create c3 experiment object and compute propagators
52eb787
https://github.com/q-optimize/c3/pull/165/commits/52eb78706bdd7558bd64a20f05cdbbc64eccc6df
convert and evaluate instructions in qasm style
f21f7c3
https://github.com/q-optimize/c3/pull/165/commits/f21f7c331ce535a78b1a9fbb2e1da9e6a137f340
remove redundant conversion to c3 legacy style instructions
4309f67
https://github.com/q-optimize/c3/pull/165/commits/4309f67b861c2fe11b6ebf663355731bc4cbfe1b
bump versions and remove mypy dependencies
7ebfeac
https://github.com/q-optimize/c3/pull/165/commits/7ebfeacd97662b6069cd3de7e64f964eed863ce2
config file for qiskit physics simulations
0ce0dd9
https://github.com/q-optimize/c3/pull/165/commits/0ce0dd966e49a55876f07a52d3fd09e6f6705708
basic test for qiskit physics simulation
a83f47b
https://github.com/q-optimize/c3/pull/165/commits/a83f47b1d060e21809a4d6e71aba2157918e7212
process circuit and obtain output probabilities
6b9d1b6
https://github.com/q-optimize/c3/pull/165/commits/6b9d1b6485050e44cdd77b126c1fbd64e5f79a22
cleanup TODOs and leftovers
d31b750
https://github.com/q-optimize/c3/pull/165/commits/d31b7506515b077d75ab2de93d4711bbd50e339c
show circuit output for debugging
f11aef6
https://github.com/q-optimize/c3/pull/165/commits/f11aef6b7520cae762d14de649ce69c3e9ec3fe9
ignore mypy warning in pwc
0370d01
https://github.com/q-optimize/c3/pull/165/commits/0370d011fd4bb1948203a1160847bffd38a04f91
create a local circuit for testing

File Changes
(6 files https://github.com/q-optimize/c3/pull/165/files)

M .pre-commit-config.yaml
https://github.com/q-optimize/c3/pull/165/files#diff-63a9c44a44acf85fea213a857769990937107cf072831e1a26808cfde9d096b9
(14)
M c3/experiment.py
https://github.com/q-optimize/c3/pull/165/files#diff-beef00f09cb4cffa20b28ab61f6f0233e2fdc09f263d5bb033947c6afe864d77
(2)
M c3/libraries/propagation.py
https://github.com/q-optimize/c3/pull/165/files#diff-1b931076821f8cb9809c2d79c339bb9dc5dbd606993405e2c346f8388404e3d6
(4)
M c3/qiskit/c3_backend.py
https://github.com/q-optimize/c3/pull/165/files#diff-7a9f4483ed1359fe5e81e06dba7130e90b71c9a33a35a44c9f513c7157b0aa14
(36)
A test/qiskit.cfg
https://github.com/q-optimize/c3/pull/165/files#diff-d9784c67864ede4df0a65c7e6a601e643ebeef3373ce820b790d44bc8773d53a
(29)
M test/test_qiskit.py
https://github.com/q-optimize/c3/pull/165/files#diff-af951b133417b900925a33f3be5221547017b3904bfe26253a46249bb7f63688
(13)

Patch Links:

https://github.com/q-optimize/c3/pull/165.patch
https://github.com/q-optimize/c3/pull/165.diff

—
Reply to this email directly, view it on GitHub
https://github.com/q-optimize/c3/pull/165, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAH3Q4IAOROZC3NTSKG4F3LUTJ3ANANCNFSM5K5FFLLQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Warnings while stopping server

Console warnings: https://pastebin.com/LiFcbpYh
Timings v2: https://timings.aikar.co/?id=bc5b96d62a2d4ecc93d71c1c73f9b21d
Config: https://pastebin.com/VpYbjtnQ

Timings are not are reliable way to find any cause of lags or similar. You would need to use a proper profiling plugin lik Lucko's Spark.

It's more just to show the plugins and server version

Thanks for reporting, should be fixed in the above commit.


First version of luci-app-usteer

This is my initial version of the luci-app-usteer, also my 1st contribution to OpenWRT and my 1st pull request. Not really sure if code quality is good enough. Feel free to edit.

ah i see, no lua, just as i was getting the hang of it. Let me see if i can find some time to convert to js.


A better container designed for performance [tip]

Hi guys! First of all, thank you for all the great work on react-native-meteor.
This is the container I'm talking about: https://github.com/Astrocoders/rn-meteor-containerize
This is my team little help for the Meteor community who uses react-native-meteor.
As we faced some performance issues, especially during Navigator transitions and subscriptions happening at once.
So we created a container which will just run the Meteor data code after any UI heavy task, ensuring smooth transactions every time.
If you like it, I can send a PR writing a bit about it in the react-native-meteor README file.

Hey @grsabreu, I really like the idea about making things faster. I'm having a little trouble fully understanding how it works or where it helps.
Is the basic premise that doing a subscription in getMeteorData() tends to be a long running task, and this improves performance by defering it to after animations are complete?
Couple questions:

What is an example scenario where this would really help speed things up? Other Pros?
Are there any Cons you can things of?
I thought subscription functions pretty much returned right away with a the subscription ID... do you know what long running aspect you are trying to defer
createContainer and the react-meteor-data package is what MDG is pushing, and what this library has already implemented for a container component... would it be easy to apply this same speed-up technique to the createContainer function?

I assume this is for RN built via a mantra framework?

@funkyeah thanks for the feedback, I'll be glad to answer:

It helps smoother transition for Navigator routes, so when you transition to another route subscribing right away to a publication will make the component re-render which can slow down your transition animation sometimes.
I can't think in any cons, it works just like react-komposer. So you can even compose then with their composeAll API.
I don't understood very well this question of yours, but you mean that subscriptions return right away they first data after subscribing?
Yes! Would be fine to do the same performances assurances to the createContainer. I don't remember why I don't made with it straightaway, I think I had some composing issues, but seems fine to be done now.

I can take a time to put an example application in the lib repo to show better the differences of performance.

@smooJitter no! you can you straightaway with any RN set you have

Hi @grsabreu,
Thanks for your work.
I have been having issues with RNRF transitions too, so this PR would be much appreciated !!
:)
Closing this issue but feel free to discuss about it.

Hey, you can use it already! Just install the package and use it instead of createContainer.
import composeWithtracker from 'rn-meteor-containerize'

function composer(props, onData){
  const handle = Meteor.subscribe('items')

  if(handle.ready()){
    onData(null, {
      items: Meteor.collections('items').find({}),
    });
  }
}

export default composeWithTracker(composer);

then use the container as you normally would
...
import ItemsContainer from '/path/to/container';

const Foo = React.createClass({
 propTypes: { items: PropTypes.object },
 render(){
   // ...
 },
});

export default ItemContainer(Foo);

Yes, but if it works well, I would like to pull it into react-native-meteor.
Would you mind creating a PR with theses changes ? ;)

Ok man! I'll create a PR ASAP!
On Aug 4, 2016 15:11, "Mokto"<EMAIL_ADDRESS>wrote:

Yes, but if it works well, I would like to pull it into
react-native-meteor.
Would you mind creating a PR with theses changes ? ;)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/inProgress-team/react-native-meteor/issues/93#issuecomment-237636177,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABOUgOmbHdR1xulxN4FHzTpND5KOt5XZks5qcitMgaJpZM4I8HG_
.

@grsabreu any progress on this? :)

Hey @Tim-W meanwhile you can use the container directly from rn-meteor-containerize, the usage will be the same when it get into the react-native-meteor

@grsabreu I took a quick look at your the code from rn-meteor-containerize. So basically all you do is load the data in a InteractionManager.runAfterInteractions wrapper?
It would be much better if we could use multithreading for this, and use something like a webworker to load the data in the background. I don't know if this is supported in React-Native, I'll find out.

Apparently multithreading/workers are not fully supported yet in RN. See https://github.com/NativeScript/android-runtime/issues/532 and https://github.com/NativeScript/ios-runtime/issues/620

Hey, @Tim-W! Thank you for the tips.
Unfortunately I'm running busy these days and couldn't work for the PR. But I'm going to have a free time in the end of this week and I plan to push some work on it.
Yes, basically I'm just wrapping the InteractionManager to avoid the computation of the subscriptions and etc. to mess with the coolness of our animations/transitions.
The idea of using workers seems interesting, I'm going to take a look on it.

Hi grsabreu,
https://github.com/inProgress-team/react-native-meteor/pull/135
As per this link,rn-meteor-containerize has been integrated with<EMAIL_ADDRESS>am using<EMAIL_ADDRESS>.But I am still facing issue in navigator transition in android.
Any one can help?


Issue: Similarity_search on Vector Store does not work.

Issue you'd like to raise.
I am using Langchain + Chroma to do a Q&A program with a csv document as its knowledge base.
The CSV file looks like below:

Here is the CSV file:
https://1drv.ms/u/s!Asflam6BEzhjgbkdegCGfZ7FI4O1Og?e=2X6ior
And code for creating embedding:
from langchain.document_loaders.csv_loader import CSVLoader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores.chroma import Chroma
from langchain.text_splitter import RecursiveCharacterTextSplitter as RCTS

file_path = "Test.csv"
doc_pages = []

csv_loader = CSVLoader(file_path)
doc_pages = csv_loader.load()
print(f"Extracted {file_path} with {len(doc_pages)} pages...")

splitter = RCTS(chunk_size = 3000, chunk_overlap = 300)
splitted_docs = splitter.split_documents(doc_pages)

embedding = OpenAIEmbeddings()
persist_directory = "docs_t/chroma/"

vectordb = Chroma.from_documents(
    documents=splitted_docs,
    embedding=embedding,
    persist_directory=persist_directory
)

vectordb.persist()

print(vectordb._collection.count())

Here is the Testing code:
result = vectordb.similarity_search("what is the Support Item Name for 01_003_0107_1_1", k=3)
for r in result:
    print(r.page_content, end="\n\n")

And I see this testing code returns all other non-relevant information.
Which part leads to this issue?
Suggestion:
No response

I am surprised to see nowadays even Github issues got something with AI chatbot, good job ~! :)
As to your suggestions:

I am not convinced that the OpenAI OpenAIEmbeddings is not capable of accurately capturing semantic similiarities. Even it is so, I am not sure how to prove this and fix this.
This might be true, as I looked at 2 different Document objects and found that yes indeed they look similar. But even this is the case, I still have to use this csv file. So what can be the solution to this?
The Chroma issue, I checked the both issues mentioned by the bot and found they didn't mention too much useful information for my case.

So still need some one experienced to look into this issue and suggest a solution.
Many thanks

thanks @dosubot and @baskaryan Looking forward to the help from the expert. Really appreciated.

The suggested options are not helpful.

I couldn't find any CSV files in the folder. If you come across more irrelevant items, why not try reducing the chunk size to 100 or even lower values? You can also consider adjusting the chunk_overlap since each token may be equivalent to the size of one word.

Hi @vilakshan2996  I have tried the csv file in an incognito mode and it opens with the data.
Please try this link: https://1drv.ms/u/s!Asflam6BEzhjgbkdegCGfZ7FI4O1Og?e=2X6ior
Let me know if you still cannot get the CSV file.
I used to use 1500 as the chunk_size and 50 as the overlap which made it worse.

In your case, I think it would be better if you could convert the Excel sheet to a pandas DataFrame and create a column called 'document' that combines all the elements in each row in a meaningful way. Then, you can pass that 'document' to the vector database. It's better to split it by row here rather than using tokens. Here is the code I use in my case for a similar problem, but with only two columns.
`import pandas as pd
from langchain.document_loaders import DataFrameLoader
Read all sheets into a dictionary
data = pd.read_excel("Technical Questions Cleaned.xlsx", sheet_name=None)
Access individual DataFrames using sheet names
for sheet_name, df in data.items():
df['text'] = "for technical question "" + df['Question']+"" this is the answer ""+df["Answer"]+"""
loader = DataFrameLoader(df, page_content_column="text")
documents = loader.load()
addToVectorDB(documents,"support")
def addToVectorDB(documents,collection_name):
## here we are using OpenAI embeddings but in future we will swap out to local embeddings
embedding = OpenAIEmbeddings()
vectordb = Chroma.from_documents(collection_name=collection_name,documents=documents,
embedding=embedding,persist_directory="db")
# persiste the db to disk`
try with this, and give me a reply.

Hi @vilakshan2996 ,
Appreciated for your help :)
I understood that you want to load the CSV file line by line, row by row, and re-write each row to be a meaningful sentence and provide these sentences to the vector store so that the accuracy will improve. (Please correct me if my understanding of your approach is not right.)
Here is one line of the data in the CSV file:

Support Item Number
Support Item Name
Registration Group Number
Registration Group Name
Support Category Number
Support Category Name
Unit
Quote
Start date
End Date
Non-Face-to-Face Support Provision
Provider Travel
Short Notice Cancellations.
NDIA Requested Reports
Irregular SIL Supports
Type

01_002_0107_1_1
Assistance With Self-Care Activities - Standard - Weekday Night
107
Daily Personal Activities
1
Assistance with Daily Life (Includes SIL)
H
No
20230701
99991231
Y
Y
Y
N
N
Price Limited Supports

Since there are so many columns, I literally do not know how to make a meaningful sentence which would include all columns.
I should have provided the extraction code earlier. I actually use CSVLoader and the CSVLoader generates one document per row, that document is generated using all columns of the corresponding row.
So if you just want to have each row as a separated document and save these documents into the vector store, it has already been done.
btw, here is the code which I extract the CSV data:

elif parts[1] == '.csv':
            csv_loader = CSVLoader(os.path.join(path, filename))
            pages = csv_loader.load()
            ............

@franva, I am not sure where the discrepancy coming from and I won't try to predict. But I can guide you a little bit with debugging as I've been through the same things today.
I would start with CSVLoader, and see if there is any anomaly. In my case, I had a problem with that (yours doesn't seem to be the case but 🤷)
This is how I loaded, you might wanna try something similar:
    loader = CSVLoader(
        file_path=file_path,
        source_column="text",
        metadata_columns=["label"],
    )

Once you load, check the docs. Then check what is inside the vector_db. You can pull the embeddings from vector_db as follows:
embeds = vector_db._collection.get(include=['embeddings', "metadatas"])
embeds["embeddings"] # --> gives you all the embeddings in vector_db
embeds["metadatas"] # --> gives you all the metadatas in vector_db

Check what is inside embeds. Play with the data, this step helped me a lot to understand how the data was stored.
The next thing you can do is creating embeddings manually. If you create an embedding from the first row of the csv file, then top_k_results should contain it as well. You can even
    oe_embedder = OpenAIEmbeddings()
    for text, label in zip(df.text, df.label):
        embedding_vector = oe_embedder.embed_query(text)
        top_k_results = vector_db.similarity_search_by_vector(embedding_vector, k=5)

you can even compare embedding of the first row with all the embeddings in vector_db. You should get 1.0 among the results. You can do something like this:
    mx = -1
    for embed, meta in zip(embeds["embeddings"], embeds["metadatas"]):
        cur_cor = numpy.dot(embed, txt_embed)
        if cur_cor > mx:
            mx = cur_cor
            print(cur_cor, meta["label"], "---", meta["text"])

Overall, I think debugging could help you (or anyone facing similar problems) to learn so much more than getting the answer here.
Just sharing based on my experience today. Hope it helps!


change managed azure disk storage class to unmanaged

What this PR does / why we need it:
change managed azure disk storage class to unmanaged, as managed azure disk storage classes are useless in unmanaged k8s cluster and make customer confusing, details could be found in #2507
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #2507
Special notes for your reviewer:
If applicable:

[ ] documentation
[ ] unit tests
[ ] tested backward compatibility (ie. deploy with previous version, upgrade with this branch)

Release note:

change managed azure disk storage class to unmanaged

@jackfrancis @j03wang

lgtm. thanks!


Move project to organization lcdproc

I'm back!
I have started thinking about making stable releases of LCDjava and eventually submitting it to the central Maven repository, so that projects can easily use it simply by declaring a dependency, without having to manually juggle JAR files.
That means that LCDjava needs a good and stable groupId and artifactId. These are also part of the Java package that LCDjava's classes are filed under. It makes sense to me to place it under org.lcdproc.lcdjava, if the LCDproc crew agrees to this. Now that LCDproc is moving to Github LCDjava could also move to their organization (https://github.com/lcdproc) to reflect the potential new groupId. I'll ask about both issues on their mailing list. Do you have any objections to this?

No, sounds like a great idea!

Hi again! Willfe at lcdproc is positive to the adoption of lcdjava. Now we need your help. As soon as willfe has removed https://github.com/lcdproc/lcdjava, could you go to the Settings tab of this repository and down in the Danger Zone do a transfer of lcdjava to the organization lcdproc? When he accepts the transfer you'll be automatically added as a collaborator in the new repository.

Hi, sounds good.
The lcdjava repo is still there now so I'll try again later.
Nudge me on this thread if you do see that it's been deleted please.

nudge Try it now.

Just tried and it failed saying You don’t have admin rights to lcdproc.
I guess I will need that (if only temporarily).

I asked, and @willfe declined (understandably). He suggested the workaround of first transferring the repo to his personal account, and then he will transfer it to the lcdproc organization (which he is the owner of). Could you do that?

OK, have done that, it's now showing as Repository transfer to willfe requested.
Let's hope it works.

Looks like we're golden! I received the transfer, and then transferred this repo to the LCDproc organization, so it should be live now at https://github.com/lcdproc/lcdjava. Let me know if there are any missing settings or memberships. I'll leave this open for now just in case :)
BTW, welcome to the project! Glad to have you join us!

Hooray!
Great that we got there in the end.
GitHub redirects from the old project URL work too which is good.

Excellent! Thanks for the warm welcome and thanks to both of you for your help!
There is also a friend repository on my profile containing demo projects at https://github.com/antoneliasson/lcdjava-demos . I opted to keep these projects in a separate repo (initially they were even two separate repositories) to show not only how to use the LCDjava API, but also how to integrate the library in a larger project for those who do not speak pom.xml fluently. Should we move these to the lcdproc organization using the same procedure?

Sure! Send it along and I'll get it moved to the org.

https://github.com/lcdproc/lcdjava-demos -- looks like we're live!
Sanity checks are welcome. If you're satisfied that the moves have been successful, we can close this issue (and the related one on the LCDproc project).

Looks great from here! I can push and pull both repositories. Closing.


Minor bug in Validator\AbstractValidatorChaninEM2

After dwonloading the GlobalEventManager.php into the EventManager (v3) although I'm not certain this is the correct way to fix it but I was trying a fast solution... I got a new error
Fatal error: Declaration of Zend\Session\Validator\AbstractValidatorChainEM2::attach() must be compatible with Zend\EventManager\EventManagerInterface::attach($eventName, callable $listener, $priority = 1)
The method is missing the callable definition:
Current code:
public function attach($event, $callback = null, $priority = 1)
where it should be:
public function attach($event, callable $callback = null, $priority = 1)
Using PHP 5.6 on WAMP server.

The GlobalEventManager and StaticEventManager were removed in EventManager v3 (ref).  It cannot be forward-ported into EMv3 this way.


Transaction boundary problem

Guys,
There is the following test case:
@Test
	public void testTransactions() throws Exception {
		ODatabaseDocument db = wicket.getTester().getDatabase();
		try {
			assertFalse(db.getTransaction().isActive());
			OSchema schema = db.getMetadata().getSchema();
			OClass classA = schema.createClass("TransA");
			classA.createProperty("name", OType.STRING);
			ODocument doc = new ODocument(classA);
			doc.field("name", "test1");
			doc.save();
			ORID orid = doc.getIdentity();
			db.begin();
			assertTrue(db.getTransaction().isActive());
			doc = orid.getRecord();
			assertEquals("test1", doc.field("name"));
			doc.field("name", "test2");
			doc = orid.getRecord();
			assertEquals("test2", doc.field("name"));
			//There is NO SAVE!
			db.commit();
			/* COMMENT START */
			db.close();
			db = wicket.getTester().getDatabase();
			/* COMMENT STOP */
			doc = orid.getRecord();
			assertEquals("test1", doc.field("name"));

		} finally {
			db.commit();
		}
	}

Looks logical - right? We set name=test2 within transaction and didn't save that record - that's why outside of transaction we see old value "test1".
But, please try to comment 2 lines marked correspondingly. After db.commit() we should be already outside of transaction and should obtain "test1" as well. But... testcase is failing.
Visibility of changed value outside of transaction leads to quite interesting side effects in case of pooled/shared connections.
There is the workaround: invoke 'db.getLocalCache().clear();' after each commit(), but that looks quite strange.
So - what do you think? I understand that inclusion of  'db.getLocalCache().clear();' into commit() might lead to some other problems in OrientDB logic.

And one more very interesting test case:
@Test
	public void testTransactions2() throws Exception {
		ODatabaseDocument db = wicket.getTester().getDatabase();
		try {
			assertFalse(db.getTransaction().isActive());
			OSchema schema = db.getMetadata().getSchema();
			OClass classA = schema.createClass("TransB");
			classA.createProperty("name", OType.STRING);
			ODocument doc = new ODocument(classA);
			doc.field("name", "test1");
			doc.save();
			ORID orid = doc.getIdentity();

			db.begin();
			assertTrue(db.getTransaction().isActive());
			doc = orid.getRecord();
			assertEquals("test1", doc.field("name"));
			doc.field("name", "test2");
			doc.save();
			doc = orid.getRecord();
			assertEquals("test2", doc.field("name"));
			doc.field("name", "test3");
			assertEquals("test3", doc.field("name"));
			//There is NO SAVE!
			db.commit();
			db.getLocalCache().clear();
			/* COMMENT START */
			//db.close();
			//db = wicket.getTester().getDatabase();
			/* COMMENT STOP */
			doc = orid.getRecord();
			assertEquals("test2", doc.field("name"));

		} finally {
			db.commit();
		}
	}

As you can see: we are within transaction setting name to "test2", then we are saving that document. And then one more time we are changing the name to "test3" without invocation "save()". And finally "test3" was commited: but we didn't "save" doc with "test3".
So, something really strange with transactions in OrientDB
If try to make some analogies between OrientDB and RDBMS it should work like following:

save() should be used just to "INSERT" new document
Once document "loaded" in a transaction: all modifications should be saved on commit() (no need explicitly invoke save()
If p.2 contradict logic of OrientDB - save() should be invoked each time we actually change something and wants to allow "commit" those changes at the end of transaction.

Did you test this against 2.1-rc3?

That's tested on 2.1-SNAPSHOT.

Unfortunatelly previous tx mechanics related on logs had flaws it will take about 8-10 days to reimplement it. We have few MT issues I fix them and will reimplement com.orientechnologies.orient.core.tx.OTransactionOptimistic .

Hey guys, I don't see this problem as really critical. OrientDB keep track of changed documents, so once you do save() it's in TX heap. If you change the document before the commit, changes are in TX as well. We could fix this problem with a ODocument.copy() on OTx.addRecord(). @laa WDYT?

My proposal is to change OTransactionOptimistic:476 from:
txEntry = new ORecordOperation(iRecord, iStatus);

to:
txEntry = new ORecordOperation(iRecord.copy(), iStatus);

WDYT?

It will lead to noticalb e performance slowdown for big records, also dirty
fields are not copied on such way, only binary record's content.
On Mon, May 25, 2015 at 3:41 PM, Luca Garulli<EMAIL_ADDRESS>wrote:

My proposal is to change OTransactionOptimistic:476 from:
txEntry = new ORecordOperation(iRecord, iStatus);
to:
txEntry = new ORecordOperation(iRecord.copy(), iStatus);
WDYT?
—
Reply to this email directly or view it on GitHub
https://github.com/orientechnologies/orientdb/issues/4212#issuecomment-105226271
.

--
Best regards,
Andrey Lomakin.

You're right. So we could create a document of dirty fields separated by action. Like:
{ "updated": { "name", "Luca" }, "created" : { "amount", "23323" }, "deleted", [ "total" ] }

ok, example.
We have record let say 512 bytes.
And we update first fiels , call save and then update second field and call save.
Previous tx size was about 512 bytes and new tx size would be 1 kB.
Speed of network operation is 100 times slower  https://gist.github.com/jboner/2841832 than memory operation so we will see noticalble slowdown on remote storage especially if we take in account that we never do disk writes instead we write everything in memor to avoid IO pitfalls.
It seems like rare operation but every setProperty on graph elelments looks like:
doc.field("f_name", f_value)
doc.save();
so all graph applications which updates more than one field will experience slowdown.
I think that it is too much overhead for this soluition.
WDYT ?

@laa why the solution I proposed is bad? Sorry I don't understand.

In nutshel because it increases set of data which is needed to be sent over network as much time as we cal setProperty of single graph element in graph application. as result noticalble decreasing speed of applications which which work with graph database and remote connection.

In my proposal I send only the delta, so much smaller traffic. In past example it was very unfortunate: one   field updated, one created and one removed. In the most common case when you update just one field you'd have something like:
{
  "updated": { "name": "Luca" },
  "version": 23
}

Furthermore on the server side it would be much faster because you don't have to call ODocument.field() foreach field, but only for changed.

But what is defference with what I proposed in chat ?

Guys,
I personally, think that making a copy is not an solution. It might lead to a huge number of problems: OOM, reference leakage, "branching" of changes and etc.
I think that following "changes" will make OrientDB transactions more logical:

For new objects(documents) to be inserted into DB method persist() should be invoked.
Once objects loaded in a transaction they should be automatically linked with a the transaction. So every changes will be reflected in DB without need explicitly call save()
If object was created outside of current transaction: it should be "registered" if we want to keep his state updated by this transaction. Something like joinTransaction() might be used.
If there is no transaction: save() should be invoked explicitly.

Or we can live ahead with just good documenting of current behavior... The only thing, that I would really recommend to change: clearing of cache on commit - ability to see transaction's changes outside of it really strange...

@PhantomYdn I think we can sort it out without dramatic changes in API, we can not afford them becaus of backwarkd compatiblity.

@laa I didn't see any solution in chat, just the idea to put the delta. I was proposing how to store delta in documents.

First part of issue is fixed. Second part will be fixed in 2.2 version.

I am developing new design for parallel transactions. It will be fixed as part of this design implementation.

Are you excepting requirements?:)
It will be cool:

To have saving points within transactions.
Run hooks also in transaction with proper saving points.
To have timeout associated with transaction and checked by oriented core.
To have ability to have new rid right in transaction, without need to explicitly commit

Hi @PhantomYdn
Yes you are right, we plan to have saving points in transaction, may be not visible at first.
And yes we are planning to update rids at once without delay. But we perform design in background so it is slow movement, but we plan to speed up it once we release live incremental backup.

Moved to 3.0 as that is ETA of implementation of parallel transactions.

Will implement in 3.0 if have time


Support an efficient shader compilation / specialization work flow.

As Slang's type system becoming more and more powerful, the front-end processing time will only going to increase.
It is time to revisit our compilation API and expose a model where the user can reuse front-end type checking work across different specializations of the same set of modules. This is done in PR #3564.
The key challenge is to allow shader specialization without preprocessor hacks. A first step solution towards that is to implement link time constants where the user can define certain "constant values" that will only be provided at link time. This will allow the user to load and parse a module once, and specialize it with different constants by linking the module with different "const-value" modules that define the link time constants to be different values.
The next thing to do is to enable offline compilation of shader modules into a binary Slang IR file, and allow loading modules from that precompiled blob instead of from source.
The final thing is to allow link-time types in addition to constants to allow more flexible specialization.

PR #3588 added the ability to load / save checked modules, and allow import to work with serialized modules.
The next step is to support link-time specialization with types.


refactor: encryption envelope

minor refactoring of encryption envelope which came up in progress of improving did-exchange implementation - a lot more can be done here (and will)
On top of https://github.com/hyperledger/aries-vcx/pull/1069, please merge that first for clean changelog

Codecov Report
Attention: 29 lines in your changes are missing coverage. Please review.

Comparison is base (0565c52) 0.05% compared to head (ff64bf7) 0.05%.

Files
Patch %
Lines

aries/aries_vcx/src/utils/encryption_envelope.rs
0.00%
29 Missing :warning:

Additional details and impacted files
@@                     Coverage Diff                     @@
##           feature/messages-display   #1070      +/-   ##
===========================================================
- Coverage                      0.05%   0.05%   -0.01%
===========================================================
  Files                           471     471
  Lines                         24009   24015       +6
  Branches                       4301    4299       -2
===========================================================
  Hits                             13      13
- Misses                        23995   24001       +6
  Partials                          1       1

Flag
Coverage Δ

unittests-aries-vcx
0.05% <0.00%> (-0.01%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


k8s1.10.0 install weavescope1.9-agent报错

weavescope 1.9 app端取不到数据,页面无法加载信息,weavescope1.9 agent报错信息如下:
<probe> INFO: 2018/08/03 03:21:58.251064 command line args: --mode=probe --probe-only=true --probe.docker=true --probe.docker.bridge=docker0 --probe.kubernetes=true <IP_ADDRESS>:80
<probe> INFO: 2018/08/03 03:21:58.251148 probe starting, version 97377f45, ID 748a8223a8d0f3df
<probe> WARN: 2018/08/03 03:21:58.261132 Cannot resolve 'scope.weave.local.': dial tcp <IP_ADDRESS>:53: connect: connection refused
<probe> ERRO: 2018/08/03 03:21:58.261436 Error fetching app details: only encoded map or array can be decoded into a struct
<probe> WARN: 2018/08/03 03:21:58.287483 Error setting up the eBPF tracker, falling back to proc scanning: cannot write "p:ptcp_v4_connect tcp_v4_connect\n" to kprobe_events: write /sys/kernel/debug/tracing/kprobe_events: device or resource busy
2018/08/03 03:21:58 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: Error while dialing dial unix ///var/run/dockershim.sock: connect: no such file or directory"; Reconnecting to {///var/run/dockershim.sock <nil>}
<probe> INFO: 2018/08/03 03:21:58.288123 kubernetes: targeting api server https://<IP_ADDRESS>:443
<probe> WARN: 2018/08/03 03:21:58.291493 Error collecting weave status, backing off 10s: Get http://<IP_ADDRESS>:6784/report: dial tcp <IP_ADDRESS>:6784: connect: connection refused. If you are not running Weave Net, you may wish to suppress this warning by launching scope with the `--weave=false` option.
<probe> ERRO: 2018/08/03 03:21:58.297018 conntrack stderr:NOTICE: Netlink socket buffer size has been set to 8388608 bytes.
<probe> ERRO: 2018/08/03 03:21:58.297913 conntrack stderr:NOTICE: Netlink socket buffer size has been set to 8388608 bytes.
<probe> ERRO: 2018/08/03 03:21:59.263183 Error fetching app details: only encoded map or array can be decoded into a struct
2018/08/03 03:21:59 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: Error while dialing dial unix ///var/run/dockershim.sock: connect: no such file or directory"; Reconnecting to {///var/run/dockershim.sock <nil>}
<probe> ERRO: 2018/08/03 03:21:59.321326 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
<probe> ERRO: 2018/08/03 03:22:00.302837 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
2018/08/03 03:22:00 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: Error while dialing dial unix ///var/run/dockershim.sock: connect: no such file or directory"; Reconnecting to {///var/run/dockershim.sock <nil>}
<probe> ERRO: 2018/08/03 03:22:01.263046 Error fetching app details: only encoded map or array can be decoded into a struct
<probe> ERRO: 2018/08/03 03:22:01.304485 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
<probe> ERRO: 2018/08/03 03:22:01.379682 Error checking version: Get https://checkpoint-api.weave.works/v1/check/scope-probe?arch=amd64&flag_kernel-version=4.4.117-1.el7.elrepo.x86_64&flag_kubernetes_enabled=true&flag_os=linux&os=linux&signature=nE3s8phP9ReLbS%2Bh4ACEZZPGKnwOMFsPtxqeF0tTb70%3D&version=97377f45: dial tcp <IP_ADDRESS>:443: connect: connection timed out
<probe> ERRO: 2018/08/03 03:22:02.303767 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
<probe> ERRO: 2018/08/03 03:22:03.301281 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
2018/08/03 03:22:03 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: Error while dialing dial unix ///var/run/dockershim.sock: connect: no such file or directory"; Reconnecting to {///var/run/dockershim.sock <nil>}
<probe> ERRO: 2018/08/03 03:22:04.302846 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
<probe> ERRO: 2018/08/03 03:22:04.483723 Error checking version: Get https://checkpoint-api.weave.works/v1/check/scope-probe?arch=amd64&flag_kernel-version=4.4.117-1.el7.elrepo.x86_64&flag_kubernetes_enabled=true&flag_os=linux&os=linux&signature=nE3s8phP9ReLbS%2Bh4ACEZZPGKnwOMFsPtxqeF0tTb70%3D&version=97377f45: dial tcp <IP_ADDRESS>:443: connect: connection timed out
<probe> ERRO: 2018/08/03 03:22:05.263392 Error fetching app details: only encoded map or array can be decoded into a struct
<probe> ERRO: 2018/08/03 03:22:05.302670 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
<probe> ERRO: 2018/08/03 03:22:06.304131 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
<probe> ERRO: 2018/08/03 03:22:07.308801 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
<probe> WARN: 2018/08/03 03:22:08.296055 Error collecting weave status, backing off 20s: Get http://<IP_ADDRESS>:6784/report: dial tcp <IP_ADDRESS>:6784: connect: connection refused. If you are not running Weave Net, you may wish to suppress this warning by launching scope with the `--weave=false` option.
<probe> ERRO: 2018/08/03 03:22:08.305089 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
2018/08/03 03:22:08 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: Error while dialing dial unix ///var/run/dockershim.sock: connect: no such file or directory"; Reconnecting to {///var/run/dockershim.sock <nil>}
<probe> ERRO: 2018/08/03 03:22:09.303559 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
<probe> ERRO: 2018/08/03 03:22:10.304101 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
<probe> ERRO: 2018/08/03 03:22:11.302920 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
<probe> ERRO: 2018/08/03 03:22:12.303410 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
<probe> ERRO: 2018/08/03 03:22:13.263510 Error fetching app details: only encoded map or array can be decoded into a struct
<probe> ERRO: 2018/08/03 03:22:13.303025 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
2018/08/03 03:22:13 grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: Error while dialing dial unix ///var/run/dockershim.sock: connect: no such file or directory"; Reconnecting to {///var/run/dockershim.sock <nil>}
<probe> ERRO: 2018/08/03 03:22:14.303328 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable
<probe> ERRO: 2018/08/03 03:22:15.304624 error generating report: rpc error: code = Unavailable desc = grpc: the connection is unavailable

该如何解决?谢谢~

从报错信息来看
Cannot resolve 'scope.weave.local.': dial tcp <IP_ADDRESS>:53: connect: connection refused

请检查DNS是否正常，还有跨节点的pod通信是否正常。


ALE: Ignoring Pep8 errors in Flake8 under

I use vim-plug as plugins manager. I installed ALE and then I have flake8 and autopep8 installed.
As far as I understood, autopep8 is for fixing error automatically on save, and flake8 is to highlight the errors when file is opened or edited.
I am trying to ignore errors/warning that are shown when opening/editing the file, hence Flake8 errors, right?
I just looked in here that ``g:flake8_ignore` is deprecated.

I had flake8 under $HOME/.config and it no longer works, I had to copy that to $HOME/.flake8 and vim works again. Is the configuration location changed?

I had flake8 under $HOME/.config and it no longer works, I had to copy that to $HOME/.flake8 and vim works again. Is the configuration location changed?

The location of flake8's config has indeed changed. This has nothing to do with the vim-flake8 Vim plugin though, but with flake8 itself, which is a different project.


Add other nuts samplers example notebook

Closes https://github.com/pymc-labs/pymc-marketing/issues/577
This should help users use other samplers which generally speed inference.

📚 Documentation preview 📚: https://pymc-marketing--579.org.readthedocs.build/en/579/

Side question: How do you get the collapsible section from the notebook? I don't see anything signifying that in the notebook...

Looks great! Only think I might add would be to actually time the sampling. Just to prove to the users that it's faster. I don't see a progressbar in the rendered docs

I just wanted to wide some numba warnings and messages which might distract the target audience. Just to keep it simple.

Side question: How do you get the collapsible section from the notebook? I don't see anything signifying that in the notebook...

If you are using vscode check the cell options and click "add tag" and simply write "hide-input" in the pop-out window , see https://myst-nb.readthedocs.io/en/latest/render/hiding.html :)

I just wanted to wide some numba warnings and messages which might distract the target audience. Just to keep it simple.

Sounds good. Lgtm!