Jobs executing twice

Hi Guys,
In my set up I have two postgres databases, one in the EU and one in the US which are synchronized with BDR. Both these databases have delayed_job instances in their regions contacting it for jobs.
I am running into the issue that at some point (usually after about 30 minutes) the same job will get executed in both the US and the EU.
EU Server
Aug 17 13:28:50 EU rails[17861]: 2018-08-17T13:28:50+0000: [Worker(delayed_job.10 host:EU pid:17861)] Job Alert (id=2274366) RUNNING
Aug 17 13:28:51 EU rails[17861]: 2018-08-17T13:28:51+0000: [Worker(delayed_job.10 host:EU pid:17861)] Job Alert (id=2274366) COMPLETED after 0.3726
Aug 17 13:28:51 EU rails[17861]: 2018-08-17T13:28:51+0000: [Worker(delayed_job.10 host:EU pid:17861)] Job Alert (id=2274367) RUNNING
Aug 17 13:28:51 EU rails[17861]: 2018-08-17T13:28:51+0000: [Worker(delayed_job.10 host:EU pid:17861)] Job Alert (id=2274367) COMPLETED after 0.4841
Aug 17 13:28:51 EU rails[17861]: 2018-08-17T13:28:51+0000: [Worker(delayed_job.10 host:EU pid:17861)] 2 jobs processed at 2.2883 j/s, 0 failed
US Server
Aug 17 13:28:50 US rails[8326]: 2018-08-17T13:28:50+0000: [Worker(delayed_job.21 host:US pid:8326)] Job Alert (id=2274366) RUNNING
Aug 17 13:28:52 US rails[8326]: 2018-08-17T13:28:52+0000: [Worker(delayed_job.21 host:US pid:8326)] Job Alert (id=2274366) COMPLETED after 2.0828
Aug 17 13:28:52 US rails[8326]: 2018-08-17T13:28:52+0000: [Worker(delayed_job.21 host:US pid:8326)] 1 jobs processed at 0.4700 j/s, 0 failed
Versions
Rails: 5.2.0
Delayed_job: 4.1.5
Delayed_job_active_record: 4.1.3
Any ideas ?
Thanks!

My guess would be that BDR doesn't support the high performance single query locking DJ uses. In order to do so every SQL update would have a 100ms+ delay as each server would first have to tell the other to lock the record for update before updating and cross continental latency would become a real problem.

Closing stale


Linear equation solver

Worked on Issue Number: #519
Issue Name: Linear Equation Solver
Issue Type: Feature Request
Video:
https://user-images.githubusercontent.com/77090462/168414256-12a9ffd2-0317-44eb-8c25-16fcb331a7b4.mov

@manisha-27 @abhijeet007rocks8 @heet-kumar Please check and review it.

Ok


Include optional environment variables for tool installer paths

Is the feature request related to a problem
Yes, many of my diff tools are installed in their non-standard location and cannot be located
Describe the solution
In the list of paths to check for each diff tool, include one that is parsed from an environment variable
e.g. vscode's last search path could be something like System.Environment.GetEnvironmentVariable("VERIFY_DIFFENGINE_VSCODE_PATH") ?? string.Empty
Describe alternatives considered
I could add a new diff tool based on an existing, however I want this to be generic enough that anyone running the Verify test suite can.
Additional context
If you think this is a viable option I would be happen to submit a PR for it.

the current version also scans the path environment variable. does this meet your requirements?


conceal queries interfere with syntax highlighting

The use of @text.math in the queries used for conceal causes everything captured by this expression to be highlighted as @text.math.
This breaks highlighting for any valid command in a math environment (not just those that have conceal applied to them).
This can be fixed by using a different name for the capture group.
(I'll be happy to submit a PR for this, but I wanted to make sure this isn't intentional, and I'm also not sure what names should be used for the capture instead - any name seems to be valid, but I haven't yet found any instructions or best practices to make sure it doesn't interfere with anything)

hi! i'm not using neovim as my daily driver any more. i'd happily accept a PR for this, but it's not something i'm inclined to work on myself.


Parachain nodes halt block production/import at random

Observing running local testnet with 2 fullnodes, 2 collators and 2 relay nodes, on long stand sessions without transactions, some nodes, in particular one of fullnodes at some point will keep same parachain block as best for a period of ~1h==~600blocks. At the same time other fullnode and collators maintain normal operation. For halted node transactions won't pass, just hanging until it eventually continues starting with importing network's best block.
Not sure if it's related but also node periodically encounters UnknownParent errors. Apart of that there are no errors near period of halt in logs.
Code is based on parachain template, rococo-branch for imports and relay node.

Please post these issues in the Cumulus repo and not Substrate.
The old branch does not receive any fixes, please try latest master as with the other issue.


ilab model evaluate command and eval library usage

added support for ilab model evaluate which allows users to run MMLU Bench MT Bench, MMLU Branch, and MT Branch Benchmarks
Add an _evaluate class to config.yaml so that users can get sane evaluation defaults that they can see and modify. These funnel directly into the evaluation flags as training now does.
a sample evaluation class looks like:

evaluate:
  branch: null
  mmlu:
    batch_size: 5
    few_shots: 2
  mmlu_branch:
    sdg_path: generated
  model_name: null
  mt:
    judge_model: prometheus-eval/prometheus-8x7b-v2.0
    max_workers: 40
    output_dir: eval_data
  mt_branch:
    taxonomy_path: taxonomy

this is a placeholder for now, lmk when the library is somewhere I can access and import

@cdoern you can install directory from test.pypy.org for testing if you wish: https://test.pypi.org/project/instructlab-eval
once we have a 0.0.1 release we'll publish that to production PyPI

I think CI should just pass on this one :crossed_fingers:

made follow up issues!

Given that tomorrow morning (7/1) is a deadline. This PR needs to be merged before then to add some form of evaluation support.
That being said, there is a stale change request on this PR, some pending reviews that haven't come in yet, etc.
We will be dismissing those in favor of deferring to follow up issues but if any of the reviewers have immediate follow up concerns please feel free to reach out!!!!

I filed #1540 as a follow-up to get this tested in e2e CI


🐛 Include api group on command line

To avoid confusion in kcp workspaces (which have APIBindings to TMC in kcp v0.11.0).

Summary
This PR updates the ensure-location script to specify the API group on the command line.  This avoids ambiguity when run against a kcp workspace that has APIBindings to TMC.
Related issue(s)
Fixes #

I introduced some merge conflicts by editing the script in my local environment and pushing it, thereby duplicating this PR. So I will close this PR to avoid the work of reconciling the two files.


Add ALToolbox as a dependency

I think this issue is on the wrong repo, as it doesn't make sense to add ALToolbox as a dep to itself. Do you remember which repo you wanted this on?

Aldashboard, I'll transfer


Set button color in events

This will remove the manual pass/fail check which doesn't work with translations.

I fixed it in my cloned repo. Pass Fail Confirm buttons are fixed StringKeys. So the compare will work comparing stringKeys.
But do what you were doing and I will addapt


fix(anta.tests): Fix VerifySSHStatus to support 4.32 text output

Description
EOS 4.32 changed the text output of show management ssh, breaking our existing logic. New logic handles both pre and post 4.32 outputs.
Fixes # (issue id)
Checklist:

[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my code
[X] I have run pre-commit for code linting and typing (pre-commit run)
[X] I have made corresponding changes to the documentation
[X] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes (tox -e testenv)

VerifySSHStatus test should be augmented to support a different VRF and an expected state in the future (with defaults being disabled / default to be backward compatib;e)


As a user i want sync local files <> dC workspace

User Story
As a user i want to keep the workspace updated with my files, when for example running periodic syncs (e.g. syncing a database/directory/gh repo content/...)
Option: Synchornizing two states can have quite individual rules to be followed. One solution would be to just implement deletions and users can implement a (simple) script that fits their needs. Additionally we can document this case.
Relevant Links

we already provide the write_mode (docs) to overwrite files if a file with the same name already exists

Acceptance Criteria

a user can sync a local directory/list of files with deepset Cloud

### Tasks
- [ ] https://github.com/deepset-ai/deepset-cloud-sdk/issues/112
- [ ] (optional) add sync method that updates existing files and deletes old ones

@gosiapre We just found that the we have an example in the old docs about this usecase. Let's keep an eye on this :)


New default clock: get-time-of-day

Redoing PR #144, but onto develop this time.
Added travis tests, so coverage should increase. I'll create a blog post before this is merged.
If this merge represents a feature addition to ROSS, the following items must be completed before the branch will be merged:

[x] Document the feature on the blog (See the website Contributing guide).
Include a link to your blog post in the Pull Request.
[x] Builds should cleanly compile with -Wall and -Wextra.
[x] One or more TravisCI tests should be created (and they should pass)
[x] Through the TravisCI tests, coverage should increase
[x] Test with CODES to ensure everything continues to work

Blog post is up: https://ross-org.github.io/feature/gtod.html

tested with codes and all is good.


Sign Harden-InstallationDirectory.ps1 during build.

Background
This PR is a step on the road to signing all our PowerShell scripts, as discussed here: https://github.com/OctopusDeploy/OctopusTentacle/pull/285 and here: https://octopusdeploy.slack.com/archives/C0K9DNQG5/p1627507335269700 (internal link).
Note that this doesn't actually allow the script to run on machine with an Execution Policy set to 'Only signed scripts' yet, because that still requires the Octopus certificate to be installed under Trusted Publishers.  This particular script is only used in the MSI installer for Windows.
The way this is implemented may change in the future as we further develop our script signing process.
Before
Script wasn't signed.
After
Script is signed!
Testing
Ran the build process, installed Tentacle from the produced MSI, noted that the Harden-InstallationDirectory.ps1 file had a signature.
Also ran Get-AuthenticodeSignature Harden-InstallationDirectory.ps1, which showed that the file had a valid signature.
Review
Firstly, thanks for reviewing this pull request! :tada:
Checks

[X] :green_heart: All automated builds and tests are passing
[X] Scripts that automate Tentacle installation and configuration will continue working after updating to this version of Tentacle
[X] Existing installations will continue working after updating to this version of Tentacle
[X] I have considered the changes to public documentation needed to reflect this change
[X] I have considered the changes to the project wiki needed to reflect this change

https://github.com/OctopusDeploy/OctopusTentacle/pull/351


Added support for dot notation with data elements tokens. Removed support for most magical tokens.

Currently we have magically delicious tokens where you can use dot notation to access nested properties of an object. For example, when you have a rule where the event type is "click" and you're setting an evar value for analytics, in the textfield you can enter %event.target.id% and the evar will be set to the ID of the element that was clicked. I call these magically delicious tokens because there's no actual data element called event.
We support dot notation for the following:
%this.whatever%
%event.whatever%
%target.whatever%
%param.whatever%

The problem is that we don't support it for any data elements that users create. For example, if you create a data element called foo, you can't do do this in the evar textfield: %foo.whatever%
We need to support dot notation for actual data elements that users create.
This affects migration. Currently in DTM we allow users to create data elements with periods in their name. These won't work when we add support for dot notation since any periods would be interpreted as trying to access nested properties.

Currently when filling out a text field that supports data elements (for example, populating an analytics evar), we allow you to use certain magical tokens even when you haven't created any data elements. Here are the magical tokens that we support:
%URI%
%uri%
%protocol%
%hostname%
%this.whatever%
%event.whatever%
%target.whatever%
%window.whatever%
%param.whatever%
%rand4% (where 4 means create a random number with 4 digits. The number can be anything)

I've talked with Corey Spencer and we want to only support the following magical tokens (for backward compatibility reasons):
%this.whatever%
%event.whatever%
%target.whatever%

For the other magical tokens, the user should instead create appropriate data elements. If any of these things are commonly used, we can create data element types for them in the Core extension to help out. For example, we could create a "random number" data element type where the user inputs how many digits they want and we generate a number for them.

Coverage increased (+0.04%) to 95.514% when pulling c45d426fa761fbf1e4798c1aa82cfe28661b24dd on data-element-dot-notation into f314004c2a47a64022c7ceb2384e102772aa28e2 on master.

We use dots in our data element names as a sort of psuedo-namespacing.  Things like locator.state or locator.zip.  They do not refer to any actual locator object - just a way of grouping elements.  When migrating from DTM to Launch, will we have to modify those names and all rules that reference them, or will they still be supported?
I was not aware of the "magically delicious" tokens until I read this PR.  You learn something new every day!

@jbradley-sprint Thanks for asking. Yes, you would need to modify your data element names and the rules that reference them. Question for you: would you prefer not adding support for accessing properties using dot notation if it meant you didn't have to modify your data element names and the rules that reference them?

@Aaronius Selfishly speaking, having to modify all those elements and rules will be a barrier to - or at least a serious drag on the pace of - migration for some of our properties.  Many of our data elements were named (by various folks, including Adobe consulting) with dots.  There was nothing to stop it, and as far as I know, no documentation that discouraged it.
I see the benefit of maintain support for the three token you listed above, even though it is not something we ever used (or, as I mentioned, even knew about).  If you take away the ability to include dots in data element names, I would ask that some sort of tool be available to make it easier to change the names and the rules and javascript code that reference them.  For example, the property for the legacy Boost Mobile site has over 200 data elements with dots in their name (not my doing!)

Thank you. I'll let the product managers know and they can decide how they would like to move forward.

@jbradley-sprint I talked it over with Corey and we decided to remove the ability to access nested properties on data element values with using the % tokens. This means periods will continue to be supported in data element names and you won't have to do any renaming during migration. Thanks again for the feedback.

Coverage increased (+0.04%) to 95.514% when pulling c0acb854c879d2c281eea988f5f917341a4a7ae9 on data-element-dot-notation into f314004c2a47a64022c7ceb2384e102772aa28e2 on master.


Reference mapping with MuData-based models

(Addresses #2393) While certain multimodal models (e.g. totalVI) can be used with either AnnData or MuData objects, currently the ArchesMixin class only supports AnnData-based models and so reference mapping cannot be performed with MuData-based models. This PR thus extends ArchesMixin class to handle MuData-based models by making appropriate changes to the load_query_data function and adding a new prepare_query_mudata function.
@martinkim0 my initial PR is still a work-in-progress (e.g. no unit tests yet), but I thought it would be helpful to get some initial feedback from you. To avoid duplicating code, my instinct for the prepare_query_mudata function was to essentially be a lightweight wrapper that calls prepare_query_anndata for each modality in the MuData object. However, when attempting to do this naively I quickly ran into issues with var_names for MuData objects being saved as a single list for all modalities. In particular, because variable names don't have identifiers for their modality of origin, I ran into issues where e.g. a protein-modality-AnnData would erroneously be padded with zeros for "missing" RNA-modality features. To work around this, I modified the base save function such that variable names are saved as a dictionary (with modalities as keys) for MuData objects.
This worked (as in, my code ran without errors and current tests still pass), but I imagine this would lead to backwards incompatibility issues for previously trained MuData-based models. Moreover, I still needed a way to tell setup_anndata which specific variable names to consider ahead of time, which seemed a bit ugly to me. Do you have ideas for a cleaner way to associate features with modalities?

Thanks for taking a stab at this! You're right - it looks like we should save var_names separately for each modality, so it's unfortunate that we don't already do this.
This would actually be a great feature to include in our v2 release, since our plan is to rework large parts of the codebase. So there would be no issue with backwards compatibility.
The other alternative is to use this new save method from now on, with the caveat that we'd have to implement a method to convert "legacy" MuData saves (similar to convert_legacy_save).
I'm personally leaning more towards the first option since there would be less room for user confusion and maintenance issues. Let me know what you think! Is this a feature you're looking to use in a new model?

@martinkim0 with the closing of #2769, I've gone ahead and taken another stab at this PR.
The biggest change now compared to main is that I factored out most of the old code in prepare_query_anndata into a new function _pad_and_sort_query_anndata that's reused across prepare_query_anndata and a new prepare_query_mudata method.
Let me know what you think/if there's anything else you'd like in this PR in terms of testing/documentation/etc.

Hi @martinkim0. Just wanted to check in to see if you had any feedback for the latest version of this PR. Thanks!

Thanks a lot! This looks great - I think refactoring out _pad_and_sort_query_anndata makes a lot of sense. Just left some minor comments regarding typing and getting var names.
One last thing: could you add a test for prepare_query_mudata as well as a release note in CHANGELOG.md (this can go under the v1.2 release).

Added the requested typing + incorporated your suggestion on ._get_var_names. For testing I added a test_scarches_mudata_prep_layer in test_totalvi.py analogous to test_scarches_data_prep_layer in test_scvi.py.


Goals should be able to be completed and should have an affiliated badge

Goals should be able to be completed (currently you can finish all topics in the goal and it just suspends it in enrolled limbo).
Also goals should have an affiliated badge that is awarded and shows on the user profile when goal is completed.

All set.


[PE-D][Tester B] Pay can be 0

Based on the user guide, the parameter p/ can only take in a positive number.

However, 0 is not a positive number and is accepted, as shown in role 3.

Moreover, when a negative sign is added before 0, the program won't create a role and show an error message.

Labels: type.FunctionalityBug severity.High
original: antonTan96/ped#5

UG issue?


Bug With Playing Again

Playing again doesn't work because json object is not being reset with the setDefaults method

Bug is fixed by only initializing the quiz screen a single time


data "compression"

@pauldix  Some time ago (when influxdb was in the 0.4.x releases range) you mentioned discussions about compressing redundant data, for example when you have:
[
  {
    "type" : "something",
    "value" : 32
  },
  {
    "type" : "something_else",
    "value" : 42
  },
  {
    "type" : "something",
    "value" : 5.6
  }
]

Influxdb could provide a way to declare that the type column store a string and should be compressed, that way the string itself would be stored only once and an integer would be used instead in each records mentioning the same value.
Is this idea still around ?:)

We probably won't be doing any explicit compression for v0.9.0, but the new datastore should be fairly intelligent about not wasting space. Closing this out since there's nothing actionable left to do for the next release.


[DISCUSSION] Specify synchronization and lifetime semantics of CUDA Array Interface

Following on from discussions in Issues #4933 and #4886, this commit clarifies the synchronization and lifetime of __cuda_array_interface__ objects. In summary:

Synchronization: Producers and consumers of arrays on the CUDA array interface should operate on those arrays in the default stream, or synchronized on the default stream, in order to implicitly be in sync. In special cases (e.g. where the same stream is used across the producer / consumer boundary) the synchronization on the default stream may be elided.
Lifetime: Consuming the CUDA array interface does not extend the lifetime of the object owning the data. The consumer must ensure that a reference to the owner is kept as long as the data is required. I think this isn't really a new requirement, but codifies something that was implicitly required in past versions.

I've avoided referring to the "legacy default stream" as that serializes everything (see e.g. https://devblogs.nvidia.com/gpu-pro-tip-cuda-7-streams-simplify-concurrency/) - I think we really do just want to synchronize with the default stream (happy to be shown why that's not the case if it isn't though :-) )
The version number is bumped - I think for these changes it's important to bump the version number as it's possible to implement v2 correctly right now, but not match the synchronization semantics specified in this PR, so it's needed to be sure both sides of the interface agree on the semantics.
Some changes are also made to Numba to support the correct implementation of these semantics by users:

from_cuda_array_interface and as_cuda_array now have a True-by-default sync kwarg, which indicates Numba should bind the new device array to the default stream.
__cuda_array_interface__ for a device array records an event on the stream the array is bound to and inserts a wait on the event in the default stream.

This is based on #5136, as it uses the default_stream() function to get the default stream.

I've recently realised that the proposed implementation details in Numba will insert waits on events in streams whenever anything tests for the implementation of the __cuda_array_interface__ (e.g. calling hasattr(ary, '__cuda_array_interface__') will cause the property to be evaluated and a wait inserted in the default stream. This is probably not desirable, and I think that the suggested implementation in Numba might need to change (unless this is not as much of a problem as I imagine).

@gmarkall I recently noticed this python property behavior as well. Introspection calls like hasattr or inspect.getmembers will trigger getter methods and (their side-effects), and there doesn't seem to be a way to prevent or detect this (inspect won't tell you if attributes are properties or plain values). So, non-trivial property getters can be a subtle cause for trouble (#5191 ?), more so if they have side effects.
I was actually looking for a way to support OpenCV interoperability when I found this discussion here. The OpenCV wrapper architecture doesn't seem to allow for dynamic python properties at all, so I'm stuck with members only.
Is there a strong reason for __cuda_array_interface__ not to be a callable, maybe optionally? I assume that technically it must be a callable either way.

cc: @hawkinsp

Is there a strong reason for __cuda_array_interface__ not to be a callable, maybe optionally?

Don't we still need to do hasattr('__cuda_array_interface__') in this case? How does this avoid side effects, if any?
btw, @gmarkall will be out for a bit. We can continue and expand the discussion, but Graham will be probably unable to respond for now.

Is there a strong reason for cuda_array_interface not to be a callable, maybe optionally?

It's an attribute because we modelled after Numpy's __array_inteface__.

(thinking about loud here)
Can we keep __cuda_array_interface__ side-effect free by introducing a new field that exposes a callback for synchronization?
e.g.
cai = cuda_array.__cuda_array_interface__
sync_cb = cai['sync']
sync_cb()

Right now, it is okay to use __cuda_array_interface__ repeatedly.
numba_cuda_kernel1(cupy_array1, pytorch_array2)
numba_cuda_kernel2(cupy_array1, pytorch_array2)
numba_cuda_kernel3(cupy_array1, pytorch_array2)

and it's unnecessary to have a sync each time __cuda_array_interface__ is exported.

Is there a strong reason for __cuda_array_interface__ not to be a callable, maybe optionally?

Don't we still need to do hasattr('__cuda_array_interface__') in this case? How does this avoid side effects, if any?

If __cuda_array_interface__ is a method (not a property), hasattr, getattr and inspect.getmembers will tell you that the method exists, naturally without invoking it - so basically for free.
Property getters on the other hand will be automatically invoked by hasattr and inspect. You pay the cost of that function each time it is inspected, as well as having all its side-effects invoked. For example, if your IDE uses inspect to do autocompletion, the property will start spamming stream syncs each time you hit TAB.

Is there a strong reason for __cuda_array_interface__ not to be a callable, maybe optionally?

It's an attribute because we modelled after Numpy's __array_inteface__.

Right, I think the reason is also mentioned in the docs. I'm just wondering if it is within the scope of this discussion to change it, given the downsides of using python properties.

I'm just wondering if it is within the scope of this discussion to change it, given the downsides of using python properties.

it would be a trivial change for consumers, and extremely easy to support in a backward compatible manner, and you get as a bonus the possibility of asking for synchronization:
cai_attr = exporter.__cuda_array_interface__
cai = cai_attr if isinstance(cai_obj, dict) else cai_attr(sync=False)  # make sync=True default ?
...

@leofang Go for it!

cc @sanjoy - Would these modifications to the specification provide a sufficient basis for implementing support for __cuda_array_interface__ in TensorFlow?

Sorry for the delayed response.  This went into my personal email account and fell through the cracks.
As written I'm not sure if the spec is complete:  I believe there is one (non legacy) "default" stream per thread so "the consumer is expected to operate on the data on the default stream" does not unambiguously refer to one specific stream.

There is another option. Keep the original idea of __cuda_array_interface__ being a pure device buffer descriptor with no side-effects. I'd argue that simplicity and sticking to design goals similar to numpy / buffer interface is a big factor for adoption and acceptance.
Strengthen interoperability for explicit synchronization by promoting a __cuda_stream_interface__ akin to __cuda_array_interface__. Numba, CuPy and OpenCV basically have the exact same API for buffer + stream interactions. I'd love to see them interact with no boundaries and no hidden penalties. I experimented a bit with a modified OpenCV for allowing Numba to access CV allocated device memory and streams:
# allocated dataX_cpu, dataX_gpu_cv, stream_cv
# (proof of principle numba views dataX_gpu_nb, stream_nb)
t1 = time.time()
data1_gpu_cv.upload(arr=data1_cpu, stream=stream_cv)         # OpenCV upload
kernel_nb(data1_gpu_nb, out=data2_gpu_nb, stream=stream_nb)  # Numba operation
data2_gpu_cv.download(dst=data2_cpu, stream=stream_cv)       # OpenCV download
t2 = time.time()  # CPU time: 0.001 s
stream_nb.synchronize()
t3 = time.time()  # GPU time: 0.042 s
# verified data2_cpu

Perfect interoperability, no tricks, no hidden implementation details, no overhead. Just plain interface descriptors.
Now what about frameworks like Tensorflow with complex, implicitly streamed, not-to-be-exposed possibly varying implementation details? There is no need for tf.Tensor to expose __cuda_array_interface__ directly. tf.Tensor might as well provide a cudaInterface() method which simply returns our descriptor after making sure that the data is valid.
So in conclusion: Frameworks with explicit stream handling are free to interact with each other without penalty. Frameworks with hidden parallelization can provide array interfaces to synchronized data on demand. Profit?

I just wanna leave a quick reply to @sanjoy's comment; I need to think more carefully about thoughts from @sklam, @dalcinl, and @pwuertz 😅

As written I'm not sure if the spec is complete: I believe there is one (non legacy) "default" stream per thread so "the consumer is expected to operate on the data on the default stream" does not unambiguously refer to one specific stream.

Graham ignored the legacy/per-thread distinction on purpose. For detailed discussion see #5137. Basically the idea is to rely on the implicit synchronization between the legacy default stream and the per-thread default stream, which is a well-documented behavior in CUDA's documentation.

I'm just getting back from leave and catching up now. To further this PR, I'm going to:

Remove the lifetime management stuff from that PR because it seems uncontroversial and just makes explicit what already needs to be done by consumers of the interface anyway - I'll make that in a separate PR.
Go through the comments on this issue and try to understand everyone's input fully - I see there's been quite a lot of thought gone into this issue from everyone - many thanks!

Graham, welcome back!
Sorry I forgot to revisit this thread as promised. I had some thoughts on @pwuertz's suggestion of "__cuda_stream_interface__" but I don't remember them now. I'll post them as soon as I recall.
In the meanwhile, I am wondering if we could avoid the side effect simply by doing
if '__cuda_array_interface__' in arr.__dict__:
    # do something

instead of calling hasattr(arr, '__cuda_array_interface__'). It is admittedly an ugly hack, but it should just work (in a fully backward compatible manner)?

Graham, welcome back!

Many thanks!

In the meanwhile, I am wondering if we could avoid the side effect simply by doing
...
It is admittedly an ugly hack, but it should just work (in a fully backward compatible manner)?

I'll give this a try and see if it seems to do "the right thing" - I haven't really digested all the feedback yet so I am not yet clear how it interacts with all the other suggestions, but will give it some thought.

The lifetime documentation is now moved to a separate PR: https://github.com/numba/numba/pull/5323

__dict__ doesn't work for the following cases:
class Test:
    a = 42

class Test:
    @property
    def a(self):
        return 42

@attr.s(slots=True)
class Test:
    a = attr.ib(default=42)

There may be other fail cases for C-defined PyObjects, haven't tested those.

Thanks, Peter! Good catch -- I forgot __dict__ only stores writable attributes...

@dalcinl @gmarkall I think I overlooked what CUDA-aware MPI (not just mpi4py) really is. It's not only an explicit consumer but also an implicit producer (think about Allreduce or scatter/gather operations on CUDA arrays). This is super annoying now I think of it, because we have no way to access the CUDA streams used internally in any MPI implementation (for example, Open MPI internally creates 4 streams for htod, dtoh, ipc, and async memcpy) so as to do a stream sync at the export time. There is no MPI standard defined for CUDA-awareness so far, and every vendor is free to do whatever they want (in a reasonable way) and to not expose its internal streams. Am I thinking wrong or worrying too much?

@dalcinl @gmarkall I think I overlooked what CUDA-aware MPI (not just mpi4py) really does. It's not only an explicit consumer but also an implicit producer (think about Allreduce or scatter/gather operations on CUDA arrays).
This is super annoying now I think of it, because we have no way to access the CUDA streams used internally in any MPI implementation (for example, Open MPI internally creates 4 streams for htod, dtoh, ipc, and async memcpy) so as to do a stream sync at the export time. There is no MPI standard defined for CUDA-awareness so far, and every vendor is free to do whatever they want (in a reasonable way) and to not expose its internal streams. Am I thinking wrong or worrying too much?

In this case, the MPI implementation will be implementing version 2 of the array interface, but not the hypothetical version 3 in this PR - although MPI vendors can do what they want with CUDA awareness, if a future version is to implement this version of the array interface then it would probably need to do something to expose its streams.
Alternatively (and I'm not necessarily suggesting this is a good idea), when a producer uses streams internally but does not expose them, perhaps the consumer would have to be required to do a blocking sync (e.g. cuda.synchronize()) after using the CUDA array interface and before operating on the data.

Thanks, @gmarkall.

In this case, the MPI implementation will be implementing version 2 of the array interface, but not the hypothetical version 3 in this PR - although MPI vendors can do what they want with CUDA awareness, if a future version is to implement this version of the array interface then it would probably need to do something to expose its streams.

Agreed. But I feel I worried too much. I just dived into Open MPI's codebase and examined how the 4 internal streams are used. IIUC they took a sane approach that the stream sync is always done either explicitly via cuStreamSynchronize or implicitly via cuEventQuery. So at least for Open MPI we're good. I'm hoping other MPI vendors are sane as well. (I'll poke MVAPICH people later, but they're not very responsive...)
Then, at the mpi4py level we get the automatic guarantee that at the export time we are always  synced (without having to warp and call any CUDA APIs!) Yay!

Alternatively (and I'm not necessarily suggesting this is a good idea), when a producer uses streams internally but does not expose them, perhaps the consumer would have to be required to do a blocking sync (e.g. cuda.synchronize()) after using the CUDA array interface and before operating on the data.

Yeah this is partly the reason I advocated that users should take the responsibility. But at least for mpi4py we're good. (Right? @dalcinl) No further concerns for now.

To my taste, it would make sense if the CUDA array interface exposed a CUDA event that the producer triggers when the buffer becomes valid for use by the consumer. Different systems can and will use streams differently, but an event is unambiguous in its meaning independent of the stream structure, at least in the absence of failures.
Synchronizing via the host seems likely to be slow but is at least simple to explain. Synchronizing via the default stream seems likely to over-synchronize because of false dependencies due to sharing of the stream. Not synchronizing seems likely to be error prone.
(I'm not actually sure if it's legal to share CUDA events between different CUDA contexts, though; I haven't tried.)

Thanks for joining in @hawkinsp! 😄
To catch you up a bit, I think there are two points:

How do we clarify the existing semantics of __cuda_array_interface__ to be safe?
What is the most performant way to implement __cuda_array_interface__ based on real world use cases?

This PR is addressing 1, which AFAICT is complete. The discussion of 2 (while it comes up here some) is largely orthogonal and likely requires more user data (that we have yet to acquire). Though perhaps issue ( https://github.com/numba/numba/issues/4933 ) could be used for that discussion.
Does that make sense? 🙂

I'd really like to continue on two ideas connected to this topic, yet technically orthogonal to the lifetime and auto-sync discussions:

Allowing __cuda_array_interface__ to be a callable
Specification of a __cuda_stream_interface__ for sharing stream handles across frameworks

If there are no objections, I'd like to prepare and open new PRs for discussions on those points. Is that ok for you?

My objection for __cuda_stream_interface__ is that a stream handle (or an event one as @hawkinsp brought up; it's just a matter of taste and in terms of functionality they can achieve the same goal) should just be included in __cuda_array_interface__. If you are interested, I can point you to see how participating libraries struggled to get things right. Having two separate interfaces would just make maintenance much harder.
I had proposed a similar request earlier for including a stream pointer as a compromise for  not opting in synchronization by default, but not much consideration is made for it so far unfortunately.
Also, replying to one of Peter's comment:

Not synchronizing seems likely to be error prone

Let me emphasize that so far this is more like a potential (or pedantic?!) concern than a real issue, which is so far overlooked in all discussions (here or in the linked issues).
I have loosely followed several repos and I am unaware of any bug report whose root is on not synchronizing. While I fully agree that a well-defined protocol can avoid ambiguity, corner cases, or any potential issues, I also want to bring this fact to the table. Perhaps we are already protected in some way (likely by the legacy default stream?) that we failed to recognize.

@pwuertz, maybe it would be good to start with fresh issues to discuss those points first 😉

FWIW Leo we just fixed a bug where we were not synchronizing in RMM 😀

FWIW Leo we just fixed a bug where we were not synchronizing in RMM 😀

Thanks, John, for the pointers (rapidsai/rmm#309, rapidsai/rmm#313)! I didn't follow the latest progress. (Also, thanks for answering my question there.)
But, the context of my above comment was about not synchronizing at the inter-op points of __cuda_array_interface__...In any CUDA programming whenever you need to sync but didn't do, you would run into problems for sure, but this is precisely opposite to the point I tried to make: perhaps something has done the job for us so we don't have to sync, thus no bug report even if by far we have not specified the sync requirement for __cuda_array_interface__.

@pwuertz, what about this?
if hasattr(type(a), "__cuda_array_interface__"):
    ...

if hasattr(type(a), "__cuda_array_interface__"):
    ...

Not general enough. Allow for the attribute to be either a dict or a callable returning a dict, and problem solved. The handling on the consumer side is trivial, and it is backward compatible.

Sorry what doesn't work with that?

Sorry what doesn't work with that?

Python's object model? This way you are forcing producers to implement the attribute as a type attribute (most likely using @property) rather than an object attribute (i.e. self.__cai__ = {...})

@jakirkham Allowing for a callable is even consistent with Python's usual __specialmethod__ convention! Moreover, If I had designed  the CAI interface, I would have gone by using a callable from the very beginning. Look at core Python, protocols that may have side effects (change internal state, require non-trivial memory/time resources to serve, etc.) are never implemented as attributes, but as callables.

I know I said LGTM in a previous comment, but FWIW, in cuDF we use __cuda_array_interface__ to dispatch compute to Numba / cuPy internally (on the default stream for now), where if it forced synchronization every time we tried to do that handoff it would be detrimental to our performance. Right now, cuDF kernels, numba kernels, and cupy kernels all get properly enqueued onto the default stream which is the behavior we'd ideally like to maintain.
I understand this would mean we would need a way to encode the stream into __cuda_array_interface__, but that would be the ideal path forward for us.

My only concern for making it callable: circling back to https://github.com/numba/numba/pull/5162#issuecomment-582582571, how do we pass in the sync flag? I think in almost every library the caller accessing __cuda_array_interface__ is buried deeply from the end user, and unless we also define a public API for setting this flag in the protocol, I don't immediately see how this can be easily adopted in participating libraries.
With @kkraus14 changing his mind, I guess I got one more vote for including a stream pointer in the interface and defaulting to not sync? 😉

FWIW I'm leaning that way as well. Syncing between library boundaries seems potentially expensive, which would discourage mixing libraries in creative ways. Passing a stream pointer as well would remedy that issue.

Just wanna circle back to an earlier statement I made:

But I feel I worried too much. I just dived into Open MPI's codebase and examined how the 4 internal streams are used. IIUC they took a sane approach that the stream sync is always done either explicitly via cuStreamSynchronize or implicitly via cuEventQuery before return. So at least for Open MPI we're good.
(...)
Then, at the mpi4py level we get the automatic guarantee that at the export time we are always synced (without having to warp and call any CUDA APIs!) Yay!

From a recent issue brought up by PyTorch devs (https://github.com/open-mpi/ompi/issues/7733) we learned that the above statement is incorrect. Open MPI does not sync over its 4 internal streams before exiting the MPI calls; I think it only applies to Open MPI's internal implementation of memcpy.
I don't have good ideas for the moment, so what Graham said earlier:

Alternatively (and I'm not necessarily suggesting this is a good idea), when a producer uses streams internally but does not expose them, perhaps the consumer would have to be required to do a blocking sync (e.g. cuda.synchronize()) after using the CUDA array interface and before operating on the data.

might need to be reevaluated for such scenarios...

Summary status
Presently various proposals and discussions are in-flight. It's been a while since I re-visited this discussion since getting back to work, and I hope it will be helpful at this point to summarise and comment on them, with a view to narrowing down the options and moving forward.
Definitions
First, a couple of definitions to make sure I'm clear about what a Producer, Consumer, and User are - sometimes re-reading through everything I've been a little unsure if we're all sharing the same definitions. These are my interpretations of the terms, that I'll attempt to use consistently:

Producer: The library / object on which __cuda_array_interface__ is accessed.
Consumer: The library / function that accesses the __cuda_array_interface__ of the Producer.
User Code: Code that induces a Producer and Consumer to share data through the CUDA Array Interface (CAI from now on).
User: The person writing or maintaining the User Code. The User may implement User Code without knowledge of the CAI, since the CAI accesses can be hidden from their view.

In the following example:
import cupy
from numba import cuda

@cuda.jit
def add(x, y, out):
    start = cuda.grid(1)
    stride = cuda.gridsize(1)
    for i in range(start, x.shape[0], stride):
        out[i] = x[i] + y[i]

a = cupy.arange(10)
b = a * 2
out = cupy.zeros_like(a)

add[1, 32](a, b, out)

When the add kernel is launched:

a, b, c are Producers.
The add kernel is the Consumer.
The User Code is specifically add[1, 32](a, b, out).
I am the User :-)

Concerns to address
These are the issues I think need to be taken into account when considering modifications to the CAI specification to add support for synchronization. The list is intended to be exhaustive, gathered from all the comments on this and related issues - please suggest more if I've missed any.

Backwards compatibility: Where possible, interface changes shouldn't force an immediate change in all the libraries implementing the CAI.
Safety: The interface should be easy to use by default and hard to do the wrong thing with. For example:

The User should be able to know easily if they need to insert some kind of synchronization, or wait on an event.
The User should know when it is acceptable / required to enqueue work on to a stream potentially provided by the CAI.
The User should not be required to understand internal implementation details of Producers and Consumers to use them together with the CAI.
Use of per-thread default streams should be avoided if possible - these can be difficult for the User to understand and use correctly.

Convention: The CAI should continue to maintain a close correspondence with NumPy's __array_interface__, if possible.
Contract: The contract between Producers, Consumers, and User Code, particularly with respect to synchronization, should be defined unambiguously.
Performance: We should avoid introducing requirements that have a significant negative impact on performance. For example:

Unnecessary synchronizations are to be avoided.
Accessing the CAI of a Producer should be fast.
Avoid too much serialization through one stream. Examples:

Using the legacy default stream synchronizes with all other streams for every operation, so has a negative effect on performance.
Relying on the default stream will serialize operations on exported / imported arrays on the default stream.

Side-effect freeness: Exporting an array through the CAI should not have side effects (e.g. creation of events).

Proposals
Each proposal, along with some additional commentary follows:
1. Do nothing / codify that it is down to the user to ensure sync
At the moment the interface specification requires nothing of the producer or consumer for synchronization. The burden of ensuring correctness with respect to synchronization is on the User of the Producer and Consumer. This is not yet codified in the specification, but is implied. Taking forward this proposal amounts to modifying the documentation to explicitly require the User to take
care of synchronization.
This has the "advantage" that it is the status quo, but probably does not adequately address the Safety concern - whether or not to synchronize, and how, may depend on the User understanding internals of the Producer and Consumer.
Issues with Concerns
This proposal probably does not satisfy the Safety concern.
2. PR 5162 (this PR) - require operations on the default stream
Omitting a description here, since it is contained in the PR.
Issues with Concerns
The main issue with this proposal is that it is difficult to implement performantly. For example, the proposed Numba implementation would need to insert an event wait on the default stream every time the CAI of a device array is imported.
3. Include a stream pointer and flag in the CAI
Proposed here. The description from @leofang (without the examples) is quoted:

We optionally include a stream pointer along with a flag indicating whether a
synchronization over the stream is expected to be performed by the consumer or
not (and it defaults to yes for safety, unless the producer library thinks it's
OK not to). If it's None, then it means the memory is exported on the (legacy)
default stream. The decision for whether to sync should be made by the consumer
for obvious reason: Only the consumer knows if there's a performance or
correctness concern or not.

(Assuming the definition of "consumer" in the quote is the same as the one above) I think event the Consumer doesn't know whether there's a performance / correctness concern, but instead only the User can know, and this would require knowledge of the internals of the Producer and Consumer.

To make use of the flag, like in an HPC setup, we could have a named variable
added to every supporting library, something like
cuda_array_interface_stream_sync (also defaulted to True), so that the flag can
be ignored at runtime...

For the full description with examples, see the issue comment.
Issues with Concerns
See section on Numba Stream Usage below.
4. Consumers operate on the legacy default stream
Proposed here. Quoting @leofang:

What if we say that cuda_array_interface consumer are expected to operate
on the legacy default stream for its implicit synchronization. If we can lock
down the spec for a safe behavior now, we can unblock other libraries from
implementing the interface. The stream pointer question can then be a
optimization.

Issues with Concerns
If Consumers must always operate on the data on the legacy default stream, it is likely that this will have a negative effect on performance.
5. Add a callback function for sync
Proposed here. Quoting @sklam:

Can we keep cuda_array_interface side-effect free by introducing a new field that exposes a callback for synchronization?
e.g.
cai = cuda_array.__cuda_array_interface__
sync_cb = cai['sync']
sync_cb()

Right now, it is okay to use __cuda_array_interface__ repeatedly.
numba_cuda_kernel1(cupy_array1, pytorch_array2)
numba_cuda_kernel2(cupy_array1, pytorch_array2)
numba_cuda_kernel3(cupy_array1, pytorch_array2)

and it's unnecessary to have a sync each time __cuda_array_interface__ is exported.

This proposal has a number of advantages:

Backwards compatibility: if 'sync' is present it can be ignored by existing users
Performance: if sync is present but never called, no side effect is introduced. Exporting the interface needn't result in any synchronizing operations.
Flexibility: sync could be a no-op (e.g. if the producer had no operations on the array at the time of export anyway), or a wait on an event, or some other mechanism.

Issues with Concerns
The potential downside is that it doesn't offer a stream for the Consumer to use for operations on the data, which would be helpful for correct operation without explicit synchronization if the Producer is operating on the data using streams - this is potentially an issue with the Performance Concern.
6. Make the interface a callable with an optional sync kwarg
Proposed here and refined here.
Quoting @pwuertz:

Is there a strong reason for cuda_array_interface not to be a callable,
maybe optionally?

If cuda_array_interface is a method (not a property), hasattr, getattr and
inspect.getmembers will tell you that the method exists, naturally without
invoking it - so basically for free.

Quoting @dalcinl:

It would be a trivial change for consumers, and extremely easy to support in a
backward compatible manner, and you get as a bonus the possibility of asking for
synchronization:

cai_attr = exporter.__cuda_array_interface__
cai = cai_attr if isinstance(cai_obj, dict) else cai_attr(sync=False)  # make sync=True default ?
...

Issues with Concerns
My concern with this is that changing it to be optionally a method will break interoperability between current Consumers and Producers that implement the interface as a callable - this mismatch seems undesirable given that it's difficult to get all the implementing libraries aligned.
7. Add a __cuda_stream_interface__ to get the stream
Proposed here. Part-quoting @pwuertz:

Keep the original idea of __cuda_array_interface__ being a pure device buffer
descriptor with no side-effects. I'd argue that simplicity and sticking to
design goals similar to numpy / buffer interface is a big factor for adoption
and acceptance.

Strengthen interoperability for explicit synchronization by promoting a
__cuda_stream_interface__ akin to __cuda_array_interface__. Numba, CuPy and
OpenCV basically have the exact same API for buffer + stream interactions. I'd
love to see them interact with no boundaries and no hidden penalties.

@leofang comments that it would be better to just include this in the
__cuda_array_interface__ in this comment:

My objection for __cuda_stream_interface__ is that a stream handle (or an
event one as @hawkinsp brought up; it's just a matter of taste and in terms of
functionality they can achieve the same goal) should just be included in
__cuda_array_interface__. If you are interested, I can point you to see how
participating libraries struggled to get things right. Having two separate
interfaces would just make maintenance much harder.

Issues with Concerns
See section on Numba Stream Usage below.
8. Add an event to the CAI that is reached when the data is ready
Proposed here. Quoting @hawkinsp:

To my taste, it would make sense if the CUDA array interface exposed a CUDA
event that the producer triggers when the buffer becomes valid for use by the
consumer. Different systems can and will use streams differently, but an event
is unambiguous in its meaning independent of the stream structure, at least in
the absence of failures.

Issues with Concerns
My concern with this proposal is that whenever the CAI of a producer is accessed, it must create an event - this is a side effect of the property access, which is undesirable.

(I'm not actually sure if it's legal to share CUDA events between different
CUDA contexts, though; I haven't tried.)

This comment also draws attention to the fact that the CAI specification doesn't say anything about different contexts, but let's not get into that discussion now :-)
Numba Stream Usage
For Numba, I'm not sure that exporting the CAI with the default stream for an array is necessarily a correct thing to do - there is nothing in Numba at present to prevent a user from operating on an array using a stream other than its default stream. For example:
from numba import cuda
import numpy as np
import cupy as cp

s1 = cuda.stream()
s2 = cuda.stream()

# Create a device array with s1 as its default stream for async operations.
x_h = np.arange(10)
x_d = cuda.device_array_like(x_h, stream=s1)

# Copy to device on stream s1
with cuda.pinned(x_h):
    x_d.copy_to_device(x_h)

# Synchronize at this point - x_d will be ready
cuda.synchronize()

@cuda.jit
def f(x):
    i = cuda.grid(1)
    if i < len(x):
        x[i] *= 2

# Launch a kernel on a stream other than its default
f[1, 32, s2](x_d)

# Exporting the array's CAI will give stream s1, because that is its default
# stream. But it is being operated on in stream s2.
x_cp = cp.asarray(x_d)
x_cai = x_d.__cuda_array_interface__
cp_stream = cp.cuda.ExternalStream(x_cai['stream'].handle)
cp_stream.use()
print(x_cp.mean())

I think it's difficult to suggest that Numba Device Arrays should only be accessed from within their default stream, as that would place quite restrictive limits on how streams could be used. I also think it's difficult to see how to instruct someone writing code using Numba to write code that will be correct with respect to CAI export providing a stream that can be used by the Consumer at the time of the export - the code using Numba will be a Producer, but will not be able to know of the timing / circumstances in which it will produce an export of its arrays.
I wonder if there are other libraries for which this kind of scenario will also be an issue - e.g. CuPy (I don't know in depth how CuPy streams can be used) or TensorFlow (as I understand multiple streams can operate on tensors).
Perhaps the proposal for a sync callback function would help here - Numba could track what streams are operating on each array, and the sync function it provides to synchronize on each of the streams, or, as a simpler/initial but potentially less performant implementation, call cuda.syncronize().
Moving forward
My current preference leans towards "5. Add a Callback function for sync". I think it addresses these concerns well:

Backwards compatiblity: existing V2 Consumers can work with a Producer that gives a sync method in their exported interface.
Convention: The existing interface remains a property.
Side-effect freeness: at the point of export, providing a reference to a function that synchronizes should not require any modification of state.

Concerns that I think can be addressed, but would need the details to still be specified, are:

Contract: A draft of the revised specification would need to be produced and reviewed to ensure that it provides an unambiguous definition of the contract between Producers, Consumers, and Users.
Performance: As mentioned earlier, the downside is that this doesn't give the user a stream they can use to operate on the array. With knowledge of how the Producer and Consumer operate, it may still be possible for a User to obtain / exchange streams between the Producer and Consumer, but this could/would be done outside of the CAI specification. Because of the issue presented in the "Numba Stream Usage" section above, I think it's important that exchanging streams happens outside the CAI specification, so I think this is not necessarily a disadvantage of this proposal.
Safety: if unsure, the Consumer can call sync to get any necessary synchronization done.

Having Consumers always call sync is likely to introduce a performance issue, so I think it would be necessary to add a flag to Consumer libraries that allows the user whether to sync or not, similar to the cuda_array_interface_stream_sync proposed by @leofang in this comment.
Continuing discussion
I'd like to find out if it's worth moving forward with a new draft specification based on Proposal 5, so I'd like to invite comments on the assessments of the proposals above and the conclusions. In addition:

Does it appear to be implementable for other Python CUDA libraries?
@sanjoy - would it make it practical to implement the CAI in TensorFlow? i.e. is it possible to provide a sync() function that will ensure an exported tensor has current data that can be used by a Consumer of the tensor?

I'd like to find out if it's worth moving forward with a new draft specification based on Proposal 5

Within cuDF we do not want to synchronize the stream unnecessarily. We'd much prefer something akin to Proposal 3 or Proposal 7 where the stream can be explicitly supplied and defaults to the legacy default stream.

Hi @gmarkall Thanks a lot for nicely summarizing the lengthy, scattered discussions in one place! It is very useful to keep everyone on the same page and converge the discussion.
I have a question on your preference on Proposal 5:

Perhaps the proposal for a sync callback function would help here - Numba could track what streams are operating on each array, and the sync function it provides to synchronize on each of the streams, or, as a simpler/initial but potentially less performant implementation, call cuda.syncronize().

I don't understand in your example (Numba exports to CuPy) why a callback is better than a raw stream pointer (or anything that quacks like a stream). Inside the callback you still need to wrap a (Numba) stream, so we might as well just expose it?
A bit more technical question would be how you plan to implement this callback in Numba. (Or is it already done?) Specifically, in the same example where CAI is needed
f[1, 32, s2](x_d)
x_cp = cp.asarray(x_d)

How do you embed the stream s2 into the array x_d's CAI (which was defaulted to s1)? I imagine this gets messy when the kernel f accepts multiple arguments, and only some of them need to be writable?

Thanks for looking at this and the questions, @leofang!

I don't understand in your example (Numba exports to CuPy) why a callback is better than a raw stream pointer (or anything that quacks like a stream). Inside the callback you still need to wrap a (Numba) stream, so we might as well just expose it?

I was thinking "What if a user launches kernels in multiple streams on an array?" e.g.
sA = cuda.stream()
sB = cuda.stream()

x = cuda.device_array(1024)

kernel_A[nblocks, nthreads, sA](x) # Writes to x[0:512]
kernel_B[nblocks, nthreads, sB](x) # Writes to x[512:1024]

cp_x = cp.asarray(x) # Do we sync on sA or sB?

I was thinking that if there were a sync callback function, the Numba device array could keep a list of the streams currently operating on it, and the sync function could synchronize on all streams currently operating on the array at the time it is called. However:

Adding to the list of streams operating on an array is fairly straightforward - Numba can do this at kernel launch time.
Removing from the list of streams operating on the kernel will be more tricky, possibly requiring a callback. And if multiple operations from a stream are enqueued, then some kind of counting mechanism would be needed.

Maybe having a sync function doesn't solve the problem I outlined for Numba as easily as I was hoping. Perhaps instead changing Numba's specification / documentation such that the default stream of a device array is documented as being the one on which the array is exported, and then going with Proposal 3, to expose the stream itself is simpler to do (and addresses the cuDF need outlined by @kkraus). If a stream for the array is not set, then it is exported on the default stream.

A bit more technical question would be how you plan to implement this callback in Numba. (Or is it already done?) Specifically, in the same example where CAI is needed
f[1, 32, s2](x_d)
x_cp = cp.asarray(x_d)

How do you embed the stream s2 into the array x_d's CAI (which was defaulted to s1)? I imagine this gets messy when the kernel f accepts multiple arguments, and only some of them need to be writable?

As in the description above, I was trying to not have to embed it in the CAI, and instead make it something that the sync function would look up at the time it is called.

Would it be possible to ensure libraries have some kind of stream ordering behavior or otherwise ensure this happens as opposed to synchronizing?

I've had an attempt at codifying @leofang's suggestion from https://github.com/numba/numba/issues/4933#issuecomment-578088647, option 3 in https://github.com/numba/numba/pull/5162#issuecomment-635928206. I have deviated slightly from the original proposal; instead of 'stream' containing a tuple of (stream, sync), the interface is defined with two items, 'stream' and 'sync', as I found this seemed more natural to describe in the specification.
I have tried quite hard to be specific and codify precisely the motivating points and required behavior of Producers and Consumers, as well as to modify Numba to provide a prototype implementation of this specification. I have found it very hard to reason about in the absence of a firmly-defined set of use cases, so I expect there to be some modifications and clarifications required - please do try to try to poke holes in it!
@leofang Do the specifics I've outlined capture the spirit of your original proposal? Are there other points you had in mind that are omitted here?
@sanjoy How well does the present proposal align with Tensorflow's requirements?
@pwuertz The interface in this new revision is still a property, I'm afraid - I'm aware that it's an issue for OpenCV, but I'm finding it hard enough to tackle this issue alone. I do still want to get to finding a way to get to OpenCV support for the CAI, and I'm aware of https://github.com/numba/numba/pull/5751.
cc @kkraus for a view / comments from the cuDF perspective.

Would it be possible to ensure libraries have some kind of stream ordering behavior or otherwise ensure this happens as opposed to synchronizing?

@jakirkham I've tried to address this by making the synchronization requirement either synchronizing on the given stream, or enqueuing work on the given stream to ensure ordering.
Many thanks to all so far for your perseverance and attention so far!

@sanjoy How well does the present proposal align with Tensorflow's requirements?

Left some comments inline.  Apologies for the late reply; these emails were (and are still) going to my personal email.

@gmarkall Thanks a lot for your amazing effort to codify all these together!

@leofang Do the specifics I've outlined capture the spirit of your original proposal? Are there other points you had in mind that are omitted here?

I think so, although I've been thinking in the past couple of days how we should implement this new protocol in mpi4py (so I didn't reply timely, sorry!) I am discussing this offline with @dalcinl and we will revisit this asap.

Tagging a few people from the participating libraries that I'm aware of. I think it's time (again).

CuPy: @emcastillo @kmaehashi
PyTorch: @gchanan @hameerabbasi @ezyang
JAX: @hawkinsp
Dask (?): @pentschev @mrocklin @madsbk

I am still puzzled by the stream interface. Imagine that I have a tensor that I've implemented by manually calling the CUDA API via ctypes. What would a correct reference implementation of __cuda_array_interface__ look like? In particular, would it have to call Numba APIs to resolve the meaning of the stream argument?

In particular, would it have to call Numba APIs to resolve the meaning of the stream argument?

A cuda stream is essentially a pointer to an opaque object, so we should be able to take a similar approach to how we pass the data pointer and pass it as an integer and it's the responsibility of the consumer to translate that integer for use as needed.

OK, but "default stream" (not talking about legacy CUDA stream) is typically a framework specific concept. For this reason, in PyTorch, we default to the legacy CUDA stream, so that we can easily interoperate with any random CUDA code people may want to use (even at a performance cost; if you need a different stream you can just swap out the current streams in PyTroch).

Given __cuda_array_interface__ is to allow working at library boundaries efficiently, I think it makes sense for all terminology to be following CUDA semantics as opposed to library specific semantics.
I think we could clarify this that the default case where no stream is passed or stream is None is equivalent to passing 1 which is the legacy default stream (https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__TYPES.html#group__CUDA__TYPES_1ga53e8210837f039dd6434a3a4c3324aa). If per-thread-default-stream is used then 2 should be passed to indicate such (https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__TYPES.html#group__CUDA__TYPES_1g02e40b82600f62c42ed29abb150f857c).
Passing a stream of 0 would be disallowed as it is ambiguous between different libraries which may or may not have been built with per thread default stream support.

I'd like to add that a library-specific "default stream" is very common. For example, CuPy has a thread-local storage to retain the stream currently in use. But, library developers should be able to retrieve the stream in use, and insert it into CAI. Based on comments from @ezyang @kkraus14, it sounds like in PyTorch you just need to set CAI['stream'] = 1 (if CAI['sync']=True).

I think we could clarify this that the default case where no stream is passed or stream is None is equivalent to passing 1 which is the legacy default stream

@kkraus14 I'd prefer us to be explicit and to not mix up None and 1. For me stream=None should always go with sync=False, but if @gmarkall has other thoughts I would not object.

For me stream=None should always go with sync=False, but if @gmarkall has other thoughts I would not object.

My point was to get rid of the sync parameter, where you're given a stream (whether legacy default stream 1, thread local default stream 2, or an explicit stream), and that implicitly guides the synchronization behavior.
In hindsight, this sounds problematic if working with non-blocking streams as even in the legacy default stream you'd either need to synchronize the stream before launching work on your non-blocking stream or wait on an event from the legacy default stream.
Sounds good to me that stream=None implies sync=False and that the CAI object is safe to use on any stream without synchronization.

Haven't read everything, but in the description @gmarkall you write:

I've avoided referring to the "legacy default stream" as that serializes everything (see e.g. https://devblogs.nvidia.com/gpu-pro-tip-cuda-7-streams-simplify-concurrency/) - I think we really do just want to synchronize with the default stream (happy to be shown why that's not the case if it isn't though :-) )

Unless compiled with Per-Thread Default Stream enabled, the "legacy" default stream and the default stream are the SAME thing.  To be clear:
Compiled with Per-Thread Default Stream == ON:
    cudaStreamDefault provides same semantics as cudaStreamPerThread

Compiled with Per-Thread Default Stream == OFF:
    cudaStreamDefault provides the same semantics as cudaStreamLegacy

The more I work with streams, the more I'm  convinced that if a library wants to support CUDA asynchrony well, it should accept a stream parameter in any async function, and only ever synchronize that stream if necessary, never any other stream.

Many thanks to all for the previous round of feedback (back in July - I was on a block of paternity leave at the time and didn't get to picking this up for a long time, so I apologise for the long round trip). As some have seen already, I've updated the proposal to incorporate the suggested changes. The main changes since the previous revision are (cc @jakirkham):

The sync parameter has been removed, Now, if the stream parameter is an integer, then it represents a stream on which synchronization is required. If it is None, then no synchronization on the part of the Consumer is required. (suggested / discussed by @sanjoy / @kkraus14 / @leofang)
The requirement for the Producer to have only enqueued work on a single stream has been dropped - instead the Producer can enqueue work on multiple streams, but the Consumer synchronizing on the single provided stream must be sufficient to synchronize with pending work on the data in all streams. (suggested by @sanjoy).
Added a note that we might add support for providing an event to synchronize on instead of a stream in the future. (suggested by @sanjoy)
Clarified that passing 1 for the stream is the legacy default stream, 2 is the per-thread default stream, and any other integer is a non-default stream. 0 is disallowed as it's ambiguous between a default stream and no stream, and also between the legacy and per-thread default stream. (suggested by @kkraus14, discussed by @leofang)
Added an opt-out for "correct by default" behavior, when a library cannot support this, with the requirement that it is explicitly documented how to use the library, and that the user must be familiar with the details of the CUDA Array Interface. I have modified and expanded on the original wording somewhat, which I hope clarifies the motivation and requirements - @leofang how do you feel about the wording? (suggested by @leofang)
Updated the example changes to Numba to match the current specification (though, i have introduced a syntax error that needs fixing here, which I'll address shortly).

I see there are already some comments on the diff -many thanks for these so far - I'll look to address these shortly and any other comments that come in.

@mharris many thanks for the suggestions and pointers on streams. To expand on my thinking a bit:

in the description @gmarkall you write:

I've avoided referring to the "legacy default stream" as that serializes everything (see e.g. https://devblogs.nvidia.com/gpu-pro-tip-cuda-7-streams-simplify-concurrency/) - I think we really do just want to synchronize with the default stream (happy to be shown why that's not the case if it isn't though :-) )

I think this was a little vague at the time I wrote it - the specification has gone through various changes since I wrote that, but I didn't edit the description as I thought it might have made it harder to follow the evolution of the proposal.

Unless compiled with Per-Thread Default Stream enabled, the "legacy" default stream and the default stream are the SAME thing. To be clear:

Because using the CUDA Array Interface couples together different libraries that may have been compiled with or without the flag, my thinking around default streams is that it would be better to be explicit about which default stream is intended. Numba presently does not have a global flag for legacy vs. per-thread default streams, but I think if it were added in future, it would not be clear until runtime whether code will be compiled using the legacy or per-thread default streams.
As a side note - from what I can understand of the underlying implementation, the choice of stream 0 being the legacy or per-thread default stream doesn't affect the compilation, but instead the linking - when using the legacy default stream, driver / runtime functions with the suffix _v2 or no suffix are linked in, and when using the per-thread default stream driver / runtime functions with a _ptds or _ptsz suffix are linked instead. Future changes in Numba that implement per-thread default streams may do so by switching which functions are linked.

The more I work with streams, the more I'm convinced that if a library wants to support CUDA asynchrony well, it should accept a stream parameter in any async function, and only ever synchronize that stream if necessary, never any other stream.

Would it be an appropriate improvement if the specification suggested always using a non-default stream, which would encourage libraries to accept stream parameters in all async functions?

Thanks for the feedback @leofang @kkraus14 @pentschev @jrhemstad - I believe all comments up to the present time are now incorporated or responded to.
@sanjoy with the modifications you've suggested now integrated, does the current proposal allow TensorFlow to implement the CUDA Array Interface?

@kkraus14 Many thanks for the recent feedback - I believe I've addressed each of the items. I've opted so far for the simpler mechanism of using a config variable to turn off syncing, leaving the automatic elision of synchronization for a future PR, so that we can focus on getting the spec right and demonstrating it with a simple implementation in Numba here.

@harrism Many thanks for the review and helping to make the document clearer - I've incorporated all your suggestions!

As there seems to be broad consensus, I'm going to move forward with bringing the implementation in Numba up to scratch (adding tests, and addressing any issues that fall out of that) to make it ready for review / merge into Numba's main branch.
@sanjoy Any further comments whilst I'm still proceeding with this?

@stuartarchibald @sklam I believe this PR is now ready for review - I've updated the description at the top to reflect what's actually being added in this PR.

@stuartarchibald Many thanks for the review - all comments should now be addressed.

What happens if this is None? The integer values are enumerated below but not None.
Edit. Just found this below in the noted section. Suggest adding 1 line about it or updating the reference link perhaps like "Further details and information about the interpretation of None are in "

For some reason I can't reply to this comment - I've added a short description of what happens if it is None as well:

When None, no synchronization is required.

What happens if this is None? The integer values are enumerated below but not None.
Edit. Just found this below in the noted section. Suggest adding 1 line about it or updating the reference link perhaps like "Further details and information about the interpretation of None are in "

For some reason I can't reply to this comment - I've added a short description of what happens if it is None as well:

No idea RE replies?!!

When None, no synchronization is required.

Thanks for adding it.

Buildfarm ID: numba_smoketest_cuda_143.

Buildfarm ID: numba_smoketest_cuda_143.

Passed.

For some reason I can't reply to this comment - I've added a short description of what happens if it is None as well:

My understanding of the GH interface is that you can only reply to a code comment that is part of a review, but if someone adds a simple comment you can't reply to that, and this is very annoying.

For some reason I can't reply to this comment - I've added a short description of what happens if it is None as well:

My understanding of the GH interface is that you can only reply to a code comment that is part of a review, but if someone adds a simple comment you can't reply to that, and this is very annoying.

Occasionally I do encounter things like this too, but in this particular case the comment can be replied if you click "Files changed",  go to docs/source/cuda/cuda_array_interface.rst, click "Load diff", and scroll down. It's hidden because the diff is too large and not shown by default 😄


Run tests only if tests specified.

If you have 2 projects and only one has defined tests then TestRunner is
going to atempt to run [project_name]_tests.fif even if non generated.
Firt exception is going to be thrown.
So i think the best solution is just to not enter test processing loop
if there are no tests.
PS:It's possible to evade exception if you specify --contracts manually but i feel like it's not really the way to do it.

Thanks. I'll double-check it.


Some tests not passed

Maybe we can fix this somehow?
running 19 tests
test test::connect ... FAILED
test test::from_str_barconfig ... ok
test test::from_str_binding_event ... ok
test test::event_subscribe ... FAILED
test test::from_str_window ... ok
test test::from_str_mode ... ok
test test::from_str_workspace ... ok
test test::get_bar_ids ... FAILED
test test::get_marks ... FAILED
test test::get_outputs ... FAILED
test test::get_bar_ids_and_one_config ... FAILED
test test::get_version ... FAILED
test test::get_workspaces ... FAILED
test test::from_str_output ... ok
test test::run_command_fail ... FAILED
test test::run_command_nothing ... FAILED
test test::run_command_multiple_success ... FAILED
test test::get_tree ... FAILED
test test::run_command_single_sucess ... FAILED
failures:
---- test::connect stdout ----
thread 'test::connect' panicked at 'called `Result::unwrap()` on an `Err` value: GetSocketPathError(Os { code: 2, kind: NotFound, message: "No such file or directory" })', src/libcore/result.rs:1165:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
---- test::event_subscribe stdout ----
thread 'test::event_subscribe' panicked at 'called `Result::unwrap()` on an `Err` value: GetSocketPathError(Os { code: 2, kind: NotFound, message: "No such file or directory" })', src/libcore/result.rs:1165:5
---- test::get_bar_ids stdout ----
thread 'test::get_bar_ids' panicked at 'called `Result::unwrap()` on an `Err` value: GetSocketPathError(Os { code: 2, kind: NotFound, message: "No such file or directory" })', src/libcore/result.rs:1165:5
---- test::get_marks stdout ----
thread 'test::get_marks' panicked at 'called `Result::unwrap()` on an `Err` value: GetSocketPathError(Os { code: 2, kind: NotFound, message: "No such file or directory" })', src/libcore/result.rs:1165:5
---- test::get_outputs stdout ----
thread 'test::get_outputs' panicked at 'called `Result::unwrap()` on an `Err` value: GetSocketPathError(Os { code: 2, kind: NotFound, message: "No such file or directory" })', src/libcore/result.rs:1165:5
---- test::get_bar_ids_and_one_config stdout ----
thread 'test::get_bar_ids_and_one_config' panicked at 'called `Result::unwrap()` on an `Err` value: GetSocketPathError(Os { code: 2, kind: NotFound, message: "No such file or directory" })', src/libcore/result.rs:1165:5
---- test::get_version stdout ----
thread 'test::get_version' panicked at 'called `Result::unwrap()` on an `Err` value: GetSocketPathError(Os { code: 2, kind: NotFound, message: "No such file or directory" })', src/libcore/result.rs:1165:5
---- test::get_workspaces stdout ----
thread 'test::get_workspaces' panicked at 'called `Result::unwrap()` on an `Err` value: GetSocketPathError(Os { code: 2, kind: NotFound, message: "No such file or directory" })', src/libcore/result.rs:1165:5
---- test::run_command_fail stdout ----
thread 'test::run_command_fail' panicked at 'called `Result::unwrap()` on an `Err` value: GetSocketPathError(Os { code: 2, kind: NotFound, message: "No such file or directory" })', src/libcore/result.rs:1165:5
---- test::run_command_nothing stdout ----
thread 'test::run_command_nothing' panicked at 'called `Result::unwrap()` on an `Err` value: GetSocketPathError(Os { code: 2, kind: NotFound, message: "No such file or directory" })', src/libcore/result.rs:1165:5
---- test::run_command_multiple_success stdout ----
thread 'test::run_command_multiple_success' panicked at 'called `Result::unwrap()` on an `Err` value: GetSocketPathError(Os { code: 2, kind: NotFound, message: "No such file or directory" })', src/libcore/result.rs:1165:5
---- test::get_tree stdout ----
thread 'test::get_tree' panicked at 'called `Result::unwrap()` on an `Err` value: GetSocketPathError(Os { code: 2, kind: NotFound, message: "No such file or directory" })', src/libcore/result.rs:1165:5
---- test::run_command_single_sucess stdout ----
thread 'test::run_command_single_sucess' panicked at 'called `Result::unwrap()` on an `Err` value: GetSocketPathError(Os { code: 2, kind: NotFound, message: "No such file or directory" })', src/libcore/result.rs:1165:5
failures:
    test::connect
    test::event_subscribe
    test::get_bar_ids
    test::get_bar_ids_and_one_config
    test::get_marks
    test::get_outputs
    test::get_tree
    test::get_version
    test::get_workspaces
    test::run_command_fail
    test::run_command_multiple_success
    test::run_command_nothing
    test::run_command_single_sucess
test result: FAILED. 6 passed; 13 failed; 0 ignored; 0 measured; 0 filtered out

Is i3 running? The tests depend on it. It sounds like it's failing around here https://github.com/tmerr/i3ipc-rs/blob/master/src/lib.rs#L108-L132

Is i3 running?

Not sure is this possible to run it in systemd-nspawn container which mock using? I am trying to build i3ipc-rs for official Fedora repos. Here is full build log, maybe this can help.


ボタンのラベルを変更

Change

Sorry this is wrong...

Thanks for the contribution, @mokamoto. We're not ready to add pipelines back to this repo. I recommend you fork the repo and set it up. Thanks!


PHP 7.4 support (is the project still alive?)

As I don't see much recent activity on the project, I'm wondering if are there any efforts in progress to make the extension available for PHP7.4, or if the project is dead?

As I don't see much recent activity on the project

I have been studying for and working on both the next minor and major release, so the project is very active on the R&D side even though there haven't been any changes to master in a while. Definitely not dead.

I'm wondering if are there any efforts in progress to make the extension available for PHP7.4

It already compiles for 7.4, so as soon as the release is available it will be available for 7.4 through PECL. Tests are already running on 7.4, might need an update though, not sure if that snapshot is still relevant. https://travis-ci.org/php-ds/ext-ds/jobs/599053920


Update llvm 3.5 to llvm 3.5.2

I tried forking and editing the nix files myself but I've messed up somewhere and polly can't seem to find some ISL functions.
Also, if possible, please add an openmp package.

The polly bug seems to be a problem upstream. I'll submit a pull request for the 3.5.2 move anyway, but I have no idea how to create a nix file for openmp.

Solved in #9315.


Using this with email clients and mobile browsers?

Hi,

Having had a look at the https://tailwindcss.com/docs/browser-support it appears that the framework does not consider mobile browsers?

And I realise this is targeted at a browser, but has anyone got any experience with how well the framework works with email clients? In particular GMail, Outlook and Thunderbird? Or perhaps there is a framework like this more directed at email clients as that is a whole set of problems different to browser support.

Since Tailwind only generates regular CSS, this is 100% related to the properties in the generated CSS. Browser support will vary depending on your requirements, plugins used, etc. Take a look at doiuse if you want to know which browsers won't support some of your styles.
If you wanna use Tailwind for building emails, try @cossssmin's Maizzle.

Thanks @hacknug


Naming Your States: No two states can have the same name.

This issue tracker is for Bug Reports and Feature Requests only.
Please direct requests for help to StackOverflow.  See http://bit.ly/UIR-SOF for details.
This is a:

[ ] Bug Report
[ ] Feature Request
[X ] General Query

My version of UI-Router is: (version) 1.0.0
Hi to everybody, reading the old documentation about "Naming Your States". I read that two states can have the same name, it's this true in 1.0.0?
Suppose I have following states

main (abstract)
| --->main.midlle
|---> main.middle.results
|----> main.middle.results.list
|---->  main.middle.results.detailList
|---->  main.middle.results.table
test(abstract)
| --->test.midlle
|---> test.middle.results
|----> test.middle.results.list
|---->  test.middle.results.detailList
|---->  test.middle.results.table

Can main.midlle and test.midlle coexist?

I'm answer to my self, the answer is yes.


Swagger generator as a command line tool

It would be nice if the swagger.json could be generated from a web api assembly (similar to what NSwag offers). This would allow us to use swagger ui in a central website and use the swagger.json from multiple microservices ingested there.
https://github.com/NSwag/NSwag/wiki/CommandLine

Having tried both NSwag and Swashbuckle, I much prefer Swashbuckle and its ability to really customise the generation of the definition file; however, I would really like to make it part of my build chain, rather than paying that initial runtime cost, which is where NSwag's commandline tools come into play.


Implement SSAO.

GOAL

Just implement SSAO (Screen Space Ambient Occlusion).

OK.
It has been implemented.


Fix README file name in release workflow

@AnotherDaniel any other things we need to change to make this work?

no brainer - which readme does cargo.toml reference?

thanks for the pointer :+1: fixing that as well ...


Fix license specification in package.json

"Creative Commons" is a class of licenses rather than a specific license; this changes the license to the correct SPDX specification that matches the LICENSE file in the repository.

Thank you!


Added superclass support at dao_processor

Can support similar implementations:
import 'package:floor/floor.dart';

abstract class BaseDao<T> {

  @insert
  Future<int> insertModel(T model);

  @update
  Future<void> updateModel(T model);
}

@dao
abstract class ItemDao extends BaseDao<Item> {
  // Other Queries...
}

Thanks for your contribution! I've added tests and support for inheriting DAO methods that are defined in interfaces (done by using the implements keyword on a class) and mixins.


Force plugins to be explicitely enabled

Description
Right now, plugins are enabled by default as soon as they're in the dependencies of the package.json.
We should investigate ways to enable them only if they were explicitely enabled in some configs somewhere.
Notes

We probably need some config files to handle this. Right now, the config is handled as an API, which would force the CLI to always run the whole app to get the config. Separating the config into a "standalone" config file (eg: nexus.config.ts) would make reading the config much easier/faster/safer

Update
After discussing that topic with @jasonkuhrt, we ended up with the following API:
Enable nexus-prisma-plugin
import { plugins } from 'nexus-future'
import nexusPrismaPlugin from 'nexus-prisma-plugin'

plugins.use(nexusPrismaPlugin({ /* optional configuration */ }))

Enable Nexus Schema built-in plugins
import { plugins } from 'nexus-future'

plugins.use(plugins.builtins.nullabilityGuard, ...)

Local plugin:
import { plugins } from 'nexus-future'

plugins.use((p) => {
 /* ... */
})

This API has several benefits:

It makes it possible to configure plugins
Plugins are no longer identified by their names. Therefore, it's now possible to add nexus built-ins plugins (null and authorize plugins). It's also possible for the user to create its own plugin locally without having to publish one
Furthermore, it would allow for our core to abstract some of its part as plugins. One example is server.custom(). We want for the plugins to be able to contribute custom servers (eg: nexus-plugin-apollo-server). If plugins are able to contribute custom servers, server.custom could simply become a nice abstraction over the creation of a plugin to contribute a custom server.

@Weakky Thanks for capturing these notes. I still have reservations about the things we would be losing. I'll try to write them up soon. Maybe some of the losses can be recouped. I'm also not convinced this API actually has all the touted benefits it claims. There is a subjective argument to be sure, but I want to partition the objective from subjective so we can better appreciate the tradeoffs.

https://github.com/graphql-nexus/nexus-future/issues/514

Enable Nexus Schema built-in plugins

But for that, I think we should:
settings.change({
  schema: {
    // options here for builtin schema component plugins
    // user does not even know they are builtin schema component plugins
  }
})

We have an existing system. If we think it's good, and we think a schema plugin is good enough to bundle, then I think we should merge them.

Resolved by https://github.com/graphql-nexus/nexus/commit/b0704dc093d8efbc38c91aa5ca2d60cd77a87f5e


Add windows CI

Adds some basic automated builds on Windows, #1208

re. "make manual for now", you may have other reasons, but I believe that CI is free on github for public repos, and there's no security concerns here since there's no secrets and this entirely runs on github's infrastructure.


Steam Api service

I tried this on a game mod and it wont let me play the mod unless i have the original game installed but it checks with the steam api but it keeps saying i donot own the game which i do so will it work to trick the mod into letting me play it with the original game file. I HAD TO REPOST THIS BECAUSE GITHUB DOES NOT HAVE A CONFIRMATION TO CLOSE THE ISHUE ON ACSEDENT.

You can reopen issues on github

What's the mod in question? I've never tested those, only games.

What's the mod in question? I've never tested those, only games.

Its the LEGO Five nights at freddys 1 from gamejolt. It requiers the steam api to verify if the original fnaf game was bought on steam but i have the game but it keeps saying the game is not bought and i canot play the lego fnaf.

Well i ment to say its not a mod it just checks with steam api to make shur the original game was purchased.

I can't find the game. Maybe it could work by entering FNAF 1's appid in SAC? Try 319510

I can't find the game. Maybe it could work by entering FNAF 1's appid in SAC? Try 319510
But yea that's a thing I don't really know about.

The game was taken down after a dmca by lego. Heres a link to the archived version  https://archive.org/details/FNAFLEGO

The mod probably uses another method that is unsupported by SAC to check if you have the game. Can't fix, sorry :-/
Using Goldberg's cracks might be the solution but it'll require more technical knowledge

Im a software hacker myself and ive looked through a hex editor to try to bypass the api checking screen and strait to the start menu. But thanks

Great, if you know how to do that that's even better :) - hope you'll succeed or did succeed


Problem building tdbot 1.7.0

Hello, do not be bored, I want to create a telegram-bot file for tdbot 1.7.0 that you released, but it gives the following error, what is the problem?
[ 82%] Building CXX object td/CMakeFiles/tdcore.dir/td/telegram/NotificationType.cpp.o
In file included from /root/tdbot/td/td/telegram/NotificationType.cpp:11:0:
/root/tdbot/td/td/telegram/DocumentsManager.h:53:5: internal compiler error: Segmentation fault
     }
     ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-5/README.Bugs> for instructions.
td/CMakeFiles/tdcore.dir/build.make:2678: recipe for target 'td/CMakeFiles/tdcore.dir/td/telegram/NotificationType.cpp.o' failed
make[3]: *** [td/CMakeFiles/tdcore.dir/td/telegram/NotificationType.cpp.o] Error 1
CMakeFiles/Makefile2:263: recipe for target 'td/CMakeFiles/tdcore.dir/all' failed
make[2]: *** [td/CMakeFiles/tdcore.dir/all] Error 2
CMakeFiles/Makefile2:87: recipe for target 'CMakeFiles/telegram-bot.dir/rule' failed
make[1]: *** [CMakeFiles/telegram-bot.dir/rule] Error 2
Makefile:162: recipe for target 'telegram-bot' failed
make: *** [telegram-bot] Error 2

The problem was solved


Fix Busola UI

Description
Changes proposed in this pull request:

Fix Busola UI

Related issues

/recheck-cla


top-level: remove extra pkgs

lets see if this changes anything about performance
Motivation for this change
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

@ofborg eval

For context, the cpuTime stat is too noisy to be useful.
If you look at the GC stats and operation counts, you can see that this change is insignificant.
Such changes only affect "hot" code, and even then the impact is usually minimal.
This change does not solve a problem and does not clean up tech debt; merely obfuscates git blame history, so I don't think this experiment is more than that.


Crash on js: An operation is not implemented: implement native toLanguageTag

Describe the bug
Using compose-jb on js can cause an "An operation is not implemented" crash in certain scenarios. Related to https://github.com/JetBrains/compose-jb/issues/2773 but this is ticket is for js. I'm not entirely sure in what scenario this is triggered, but I believe it requires using a TextField and swapping out the theme's colors (for us it occurs when switching to dark mode).
Affected platforms

Web (K/JS) - Canvas based API

Versions

Kotlin version*: 1.7.20
Compose Multiplatform version*: 1.3.0
OS architecture (x86 or arm64): arm64

To Reproduce
Steps and/or the code snippet to reproduce the behavior:
I haven't isolated this in a standalone project, but the source code shows it's just an unimplemented method.
Expected behavior
The application should not crash
Additional context
In JsPlatformLocale, the method is not implemented. See: JsPlatformLocale.js.kt#L29 JsPlatformLocale.js.kt#L29

Recently i fixed the same for iOS. https://github.com/JetBrains/androidx/pull/408/files
The easiest workaround is always return "en-US"

Found a minimal code reproducing an error (for JS, works on other).
// main.js.kt
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.material.Button
import androidx.compose.material.Text
import androidx.compose.material.TextField
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.ui.Modifier
import androidx.compose.ui.window.Window
import org.jetbrains.skiko.wasm.onWasmReady

fun main() {
    onWasmReady {
        Window("DelmeApp") {
            Column(modifier = Modifier.fillMaxSize()) {
                val enabled = remember { mutableStateOf(true) }
                TextField("Hello", {}, enabled = enabled.value)
                Button( {
                    enabled.value = !enabled.value
                }){
                    Text("Toggle")
                }
            }
        }
    }
}

When you press the button, error appears:
NotImplementedError
cause: undefined
message: "An operation is not implemented: implement native toLanguageTag"
name: "NotImplementedError"
stack: "captureStack@↵NotImplementedError@↵@↵@↵@↵@↵@↵@↵@↵@↵@↵@↵@↵@↵materialize@↵@↵@↵SimpleLayout$composable@↵@↵@↵invoke$invoke@↵@↵…"

@dima-avdeev-jb could tell more about workaround, where I should return "en-US" from?

@dimitree54 This workaround should be applied on our side.

I'm experiencing the same issue with JS target when trying to switch from light to dark theme. Where is the right place to provide locale? The error still persists in 1.4.0 Compose version

This is especially frustrating since switching UI themes seems to have nothing to do with locales...

I'm experiencing the same as @gleb-skobinsky with 1.5.0-dev1043

for those who need a very short term solution, we ended up editing the js directly after it's built via our gradle build script.
in our case we are publishing our js module as a library to github using the npm-publish plugin, so right before that task we swap out the NotImplementedError with an empty string return.
not much of a gradle smith, so there's almost definitely a better way to do this
tasks.getByName("publishJsPackageToGithubRegistry").doFirst {
    val error =
        "throw new NotImplementedError('An operation is not implemented: implement native toLanguageTag');"
    val implementation = "return \"\";"
    val file = file("build/packages/js/androidx-ui-text.js")
    val buffer = StringBuffer()
    file.forEachLine { line ->
        buffer.appendLine(line.replace(error, implementation))
    }
    file.writeText(buffer.toString())
}

Thanks! A similar workaround helped me to override the error for the browser app. It looks too ugly to be posted here, but does the trick at least for now

I believe this is fixed, but I'll let ya'll close it out.

Done, fixed here: https://github.com/JetBrains/compose-multiplatform-core/blob/e92f3b2860e19da41f2ef615d5c0646f413d7b32/compose/ui/ui-text/src/jsWasmMain/kotlin/androidx/compose/ui/text/intl/JsPlatformLocale.js.kt#L32

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.