[REFACTOR] Reduce Code Duplication

Should be easy. Try to merge open PRs first (currently #48 and #50)

I would do anyways only the trivial parts to get in contact with the code, e.g. identical option-field definitions, like:
{
type: 'dropdown',
label: 'Bank',
tooltip: 'Which Button?',
id: 'bank',
default: '0',
choices: self.CHOICES_BANKS,
}				},

becomes a reusable
self.FIELD_BANK = {
{
type: 'dropdown',
label: 'Bank',
tooltip: 'Which Button?',
id: 'bank',
default: '0',
choices: self.CHOICES_BANKS,
}
}

This should make your later rewrite easier.

as all the system.emit and system.on calls will need to be removed and replaced with methods that are not exposed to the modules

Could you describe why the (EventEmitter) based CompanionSystem object is to be replaced with an alternative solution? Hiding functionality from modules does not feel like a valid reason for a full replacement.
(The overall EventEmiiter seems highly flexible, and decently implemented within companion)

Could you describe why the (EventEmitter) based CompanionSystem object is to be replaced with an alternative solution? Hiding functionality from modules does not feel like a valid reason for a full replacement.
(The overall EventEmiiter seems highly flexible, and decently implemented within companion)

As I've stated before, security of the core is a major imperative.  Modules can literally hook anything in the core at this time and that's fine while we're reviewing all the modules but in that future world where modules can be brought in and update out-of-band, that can lead to modules that do surreptitious things that we didn't approve of or intend to allow.  We want modules to have an explicit set of API calls and if they need something that isn't present we'll find an appropriate way to add that, if possible.  A module could literally write a loop and reset every button just because.  That's not good architecture!
A second major flaw is that the events architecture is not tight.  I simply mean that when you have one-to-many events that the execution order is not explicit and if you look closely there's a lot of hacks in the system to help that fact that there's a bunch of asynchronous event flying around
Third major flaw is that there are inline events that basically expect synchronous execution, and that is fundamentally flawed for this type of architecture.   Take this before and this after being converted to a structured API.  (the class I linked to hasn't been fully converted yet, but its calls to other classes that have been updated were swapped out)  The simple fact is its cleaner, the structure is easier to read, its a properly synchronous call in a synchronous function, and we're actually saving a number of CPU cycles by not registering the event and then letting it fire.  Sure, the CPU load is not a massive decrease given the capabilities of modern processors, but it still makes a more efficient and streamlined execution.
And to be clear, modules will still effectively use the system events architecture, its just that only modules and a couple edge cases on the core itself will continue to use that.  That's the security aspect ... we make it so its not possible for modules to get at what they shouldn't have.

(The overall EventEmiiter seems highly flexible, and decently implemented within companion)

Yes, its too flexible. Everything that modules are doing with system.on and system.emit are internal apis. I have at least twice broken functionality in a bunch of modules because they were using events that I wasnt expecting them to. As we want a stable module api, its going to either be incredibly messy and a lot of work to maintain internal apis throughout reworking of various stuff, or modules will break at random points. That wont make companion feel like stable software.
For me, this is one of the biggest reasons for moving away from system and for locking down the api.
It's not too bad today, but I do have to remember to check for modules using any of the events that I changed/removed, but once we allow users to add in 3rd party modules, we can't fix those and will struggle to provide any hinting about whether it should work.
But yes, I would say application integrity rather than security. We cant expect this to be 100% secure, because modules will still be able to do file io and stuff, so there are ways around any barriers, but if modules have free reign to play with our internal data structures and stuff then they will have plenty of opportunity to break things either maliciously or accidentally.
The architecture makes it really hard to understand the flow of anything. Intellisense or similar ide technologies are no help in this codebase because of the events. While for some of the uses events are appropriate, but most of them are using events as method calls, so events adds confusion.
Do you know the difference between these two events? https://github.com/bitfocus/companion/blob/master/lib/feedback.js#L259-L261 You can find out by searching the code, or remember from getting it wrong in the past.

Third major flaw is that there are inline events that basically expect synchronous execution

Yes, the way we assume that the events will execute synchronously is an ugly hack. I am rather surprised that this hasnt caused more issues already, I am pretty sure there are no guarantees made by the EventEmitter to execute the handler synchronously

And to be clear, modules will still effectively use the system events architecture, its just that only modules and a couple edge cases on the core itself will continue to use that.

@krocheck Will we? I was intending to not provide anything system like to the modules. Other than the socket.io client they have now (which I guess is kinda similar)

@krocheck Will we? I was intending to not provide anything system like to the modules. Other than the socket.io client they have now (which I guess is kinda similar)

Yeah "effectively" I was referring to the new socket.io interface.

Could you describe why the (EventEmitter) based CompanionSystem object is to be replaced with an alternative solution? Hiding functionality from modules does not feel like a valid reason for a full replacement.
(The overall EventEmiiter seems highly flexible, and decently implemented within companion)

(thank you for the thorough elaborations, I'll just extract some points without further comment, as I'm close to an "overload")

As I've stated before, security of the core is a major imperative. Modules can literally hook anything in the core at this time and that's fine while we're reviewing all the modules but in that future world where modules can be brought in and update out-of-band, that can lead to modules that do surreptitious things that we didn't approve of or intend to allow. We want modules to have an explicit set of API calls and if they need something that isn't present we'll find an appropriate way to add that, if possible. A module could literally write a loop and reset every button just because. That's not good architecture!

=> Restrict User Modules Access to Core Functionality

A second major flaw is that the events architecture is not tight. I simply mean that when you have one-to-many events that the execution order is not explicit and if you look closely there's a lot of hacks in the system to help that fact that there's a bunch of asynchronous events flying around

=> Ordered Execution of Multi-Subscriber Events

Third major flaw is that there are inline events that basically expect synchronous execution, and that is fundamentally flawed for this type of architecture. Take this before and this after being converted to a structured API. (the class I linked to hasn't been fully converted yet, but its calls to other classes that have been updated were swapped out) The simple fact is its cleaner, the structure is easier to read, its a properly synchronous call in a synchronous function, and we're actually saving a number of CPU cycles by not registering the event and then letting it fire. Sure, the CPU load is not a massively decreased given the capabilities of modern processors, but it still makes a more efficient and streamlined execution.

=> Avoid Unnecessary Asynchronicity

And to be clear, modules will still effectively use the system events architecture, its just that only modules and a couple edge cases on the core itself will continue to use that. That's the security aspect ... we make it so its not possible for modules to get at what they shouldn't have.

=> Limit Use of Asynchronous Event System

(The overall EventEmiiter seems highly flexible, and decently implemented within companion)

(same as above: thank you for the thorough elaborations, I'll just extract some points without further comment, as I'm close to an "overload")

Yes, its too flexible. Everything that modules are doing with system.on and system.emit are internal apis. I have at least twice broken functionality in a bunch of modules because they were using events that I wasnt expecting them to. As we want a stable module api, its going to either be incredibly messy and a lot of work to maintain internal apis throughout reworking of various stuff, or modules will break at random points. That wont make companion feel like stable software. For me, this is one of the biggest reasons for moving away from system and for locking down the api. It's not too bad today, but I do have to remember to check for modules using any of the events that I changed/removed, but once we allow users to add in 3rd party modules, we can't fix those and will struggle to provide any hinting about whether it should work.

=> Provide/Document the Stable API

But yes, I would say application integrity rather than security. We cant expect this to be 100% secure, because modules will still be able to do file io and stuff, so there are ways around any barriers, but if modules have free reign to play with our internal data structures and stuff then they will have plenty of opportunity to break things either maliciously or accidentally.

=> Ensure Core Data Encapsulation

The architecture makes it really hard to understand the flow of anything. Intellisense or similar ide technologies are no help in this codebase because of the events. While for some of the uses events are appropriate, but most of them are using events as method calls, so events adds confusion. Do you know the difference between these two events? https://github.com/bitfocus/companion/blob/master/lib/feedback.js#L259-L261 You can find out by searching the code, or remember from getting it wrong in the past.

=> Ensure IDE Intellisense work Fine

Third major flaw is that there are inline events that basically expect synchronous execution

Yes, the way we assume that the events will execute synchronously is an ugly hack. I am rather surprised that this hasnt caused more issues already, I am pretty sure there are no guarantees made by the EventEmitter to execute the handler synchronously

=> Provide Construct for Synchronous Executions of Asynchronous Events

And to be clear, modules will still effectively use the system events architecture, its just that only modules and a couple edge cases on the core itself will continue to use that.

@krocheck Will we? I was intending to not provide anything system like to the modules. Other than the socket.io client they have now (which I guess is kinda similar)

=> Avoid Architectural Overuse of EventEmitter

Ok, I'm good for now. The information is enough to understand the need to reduce the use of events significantly and to produce the relevant documentation
So, at this point, I agree fully with

the need to reduce EventEmitter usage, significantly

I disagree with the "full redesign" approach.
I still believe that this can be achieved without a major redesign (e.g. keep the original code structure more or less intact). This is what https://github.com/abebeos/companion/issues/1 is about.
But even if I manage to convince some people for this, I could not execute it without the consensus of the core-devs, as I would depend in several cases on their domain-knowledge (and in some rare cases their coding-experience). And naturally on "refactoring steps being merged fast". Otherwise I would need potentially twice the amount of time, or even more.
So, @krocheck & @Julusian , what do you think? What you be ok with me doing a parallel attempt towards a modernized companion, but using refactoring techniques (how I do it: don't know, that's a dynamic process)?

I don't think I understand what a "modernized companion" is that you'd be working on compared to the work in being done in develop.

The main difference is in the process ("Processing: Use strict refactoring first, Achieve small, non-breaking and mergeable changes"), and in the final result (to "keep the original code structure more or less intact").
So, I would try to fix any (by-design, by-architecture )problems, with the minimum possible changes to the code. And if I apply changes, those would be non-critical non-breaking refactoring steps, or slight-breaking rework-steps.
(chances are there that I fail, and/or that I've assessed all this wrong. But I'm 98% sure it's doable)

fix any (by-design, by-architecture )problems

We would need to review what these are because I think it would be a case-by-case basis whether its worth your time to pursue.
But we don't have a space for you to dev/test these with CI to get them to the public.  Our branches are:

master current stable build
beta module work and core bug fixes for current stable (effectively release staging/testing)
develop next major/minor release

We really don't need yet another development branch to manage.

I'm asking you very(!) generally here, to avoid investing more time in convincing other relevant people.
The technical project-details are "peanuts", nothing that cannot be solved. I guess I would be working against master (current 2.2.*), can be done even with usual PRs initially. Simple refactoring (like e.g. https://github.com/bitfocus/companion-module-bitfocus-companion/pull/52) could go directly into master I guess, more complex possibly indirectly via beta.
I would "stay out of your way", and would just need priority PR review, that would be enough for me at the beginning.


performance issue

type Demo struct {
	A int    `json:"a,omitempty"`
	B string `json:"b,omitempty"`

}

func BenchmarkJsoncost(b *testing.B) {
	d := Demo{
		A: 1,
		B: "abc",
	}

	b.ReportAllocs()
	for i := 0; i < b.N; i++ {
		_, err := json.Marshal(d)
		if err != nil {
			b.Errorf("marshal failed: %v", err)
		}
	}
}

go test -run ^NOTHING -bench Jsoncost$ -v
goos: darwin
goarch: amd64
pkg: go-app/basic
BenchmarkJsoncost-8      5000000               318 ns/op              64 B/op          2 allocs/op
PASS

Then, I use ffjson to generate custom MarshalJSON for Demo struct, update json.Marshal(d) to ffjson.Marshal(d), run benchmark again
goos: darwin
goarch: amd64
pkg: go-app/basic
BenchmarkJsoncost-8      5000000               355 ns/op              64 B/op          2 allocs/op
PASS

It seems ffjson is slower than encoding/json, is there anything I'm missing ?

I haven't used ffjson in a long time so I might be wrong, but this is the only way I could get your code fast:
func BenchmarkFFJson(b *testing.B) {
  d := Demo{
    A: 1,
    B: "asd",
  }

  buf := &fflib.Buffer{}
  for i := 0; i < b.N; i++ {
    if err := d.MarshalJSONBuf(buf); err != nil {
      panic(err)
    }
    _ = buf.Bytes()

    buf.Reset()
  }
}

I also had to apply this patch to get ffjson down to 0 allocations: https://gist.github.com/erikdubbelboer/898d219bbc997ab1e9aeee2d64aafa04
It seems like ffjson isn't doing it's best to get hot path allocations down to 0. @pquerna let me know if you want me to make a pull request for this?
The result:
BenchmarkEncodingJson-16    	 4652290       249 ns/op	64 B/op	 2 allocs/op
BenchmarkFFJson-16          	13343437      90.7 ns/op	 0 B/op	 0 allocs/op

Hi @erikdubbelboer,
convert []byte to string no longer require SliceHeader, just *(*string)(unsafe.Pointer(&bs)) is fine.
After dig into this more, I compare ffjson/easyjson/jsoniter, it's surprising ffjson is the slowest, with highest allocs.
BenchmarkJsoncost/std-8                  1000000              1052 ns/op             240 B/op          4 allocs/op
BenchmarkJsoncost/jsoniter-8             1000000              1268 ns/op             272 B/op          5 allocs/op
BenchmarkJsoncost/easyjson-8             2000000               629 ns/op             336 B/op          4 allocs/op
BenchmarkJsoncost/ffjson-8               1000000              1044 ns/op             480 B/op          7 allocs/op

repo for this benchmark:

https://github.com/jiacai2050/benchmark-suite/blob/master/go/README.org

Correct but this is to convert a string to a []byte which does require SliceHeader as you need to set the capacity which a string doesn't have in it's header.
I always use easyjson, that's why I said I haven't used ffjson in a long time.

if convert []byte to string, *(*string)(unsafe.Pointer(&b.buf)) would be fine IMO.

https://golang.org/src/strings/builder.go#L46

That will only work in cases where you never look at the capacity of the returned slice. If you do something that looks at the capacity it will fail horribly as it will use the memory after the string header as capacity and you have on idea what is stored there.

I got your point, thanks for your explanation.


Add option "ignore_destroy"

We have a production bucket and a development bucket.
We regularly use a script to copy all files from production into development.
As there is no bucket copy feature given by amazon, the copy process takes a lot of time especially if multiple files where deleted in development since the last synchronization.
Also, if one of the developper remove a file in his local environment, it will be removed from s3 and other developer would get a file missing error.
Having an option ignore_destroy would make the synchronization a lot faster as it will only need to import created file since the last synchronization, it will also prevent any file missing due to a local modification by one of the developer.
If you are interested to integrate this feature into this project, please let me know and I will try to implement it.

sorry I didn't comment earlier on this - I'd rather not integrate it as it's quite a specific problem which can still be worked around in other ways, catching errors etc.
thanks


Add attestation storage interface

Idea discussed in Slack: create a generic DIF Attestation schema to act as a catch-all Collections group for anything that could be categorized as an 'attestation', regardless of format (JWT, VC, etc.)


Validate low confidence IDs

For this you will need  MonDO-kboom in google drive - sync to your hard drive.
Work through class-confidence.tsv in order.
For each row, check linked-eoo-rpt.html. The image should be clickable.
If you think the existing resolution OK, remove confidence annotation.
If there are problems to be resolved, resolve
All the above can be done on a single PR
For difficult ones, open a ticket here.

Example:
first line in report:
<http://purl.obolibrary.org/obo/MONDO_0000868>  "mitochondrial DNA depletion syndrome 6"^^<http://www.w3.org/2001/XMLSchema#string>     "0.0"^^<http://www.w3.org/2001/XMLSchema#double>

find in file:

look at image:

weirdness caused by a bad xref.
This is a tricky one
I think https://omim.org/entry/256810 MITOCHONDRIAL DNA DEPLETION SYNDROME 6 (HEPATOCEREBRAL TYPE); MTDPS6 ==  Orphanet_255229 Navajo neurohepatopathy
https://omim.org/entry/271245 MITOCHONDRIAL DNA DEPLETION SYNDROME 7 (HEPATOCEREBRAL TYPE); MTDPS7 ==  Orphanet_1186 Infantile onset spinocerebellar ataxia

See the commit message attached for resolution

The xrefs here are reversed to how they should be:
[Term]
id: DOID:0050556
name: infantile onset spinocerebellar ataxia
namespace: disease_ontology
synonym: "OHAHA SYNDROME" EXACT []
xref: OMIM:271245 ! MTDPS7
is_a: DOID:0050950  ! autosomal recessive cerebellar ataxia

[Term]
id: DOID:0080126
name: mitochondrial DNA depletion syndrome 7
namespace: disease_ontology
xref: OMIM:256810 ! MTDPS6 // navajo neuropathy
is_a: DOID:700  ! mitochondrial metabolism disease

Another one:
<http://purl.obolibrary.org/obo/MONDO_0009039>  "Baller-Gerold syndrome"^^<http://www.w3.org/2001/XMLSchema#string>     "0.0"^^<http://www.w3.org/2001/XMLSchema#double>

The xref here is totally wrong, these have nothing to do with eachother!

@cmungall I just made a couple changes, but I am not sure if I am doing the pull requests correctly.
Is there some documentation somewhere on how to do a proper pull request? Either on the command line or I use SourceTree.

@nicolevasilevsky - I don't see any PRs. These would show up in the section above:

I'm afraid I can't help with sourcetree. The docs we use in GO for PRs are all here: http://go-ontology.readthedocs.org/ but these assume command line

@cmungall is this ticket still relevant, or can this be closed?

This ticket is really old and I think no longer relevant. I am closing, please reopen if still needed.


updates for aria-valuemax conformance

test cases
ARIA in HTML max / aria-valuemax
input type=number, type=range and progress should all report errors to use the max attribute instead

this update is sufficient for what ARIA in HTML needs, but there were no warnings / errors surfaced for the progress element

Thanks, Scott. Added checks for progress element.

thanks!


add Levana (unlisted)

Description
Adding token: Levana from chain Osmosis
Adding Assets

[x] Asset is registered to the Cosmos Chain Registry.
[x] Add asset to bottom of zone_assets.json.

On-chain liquidity
If adding a new asset, please provide the plan for on-chain liquidity of the asset: (choose one)

[x] The submitting team will ensure a pool will be created soon. Please hold off until further communication, which will notify of the Pool ID. (Please set the PR's status to Draft until the pool has been created)

OR

[ ] A preview link is requested for the submitting team to be able to create a pool using the Osmosis Frontend.

OR

[ ] The token is, or will be, going through a StreamSwap stream; thus, the token should be listed without requiring on-chain liquidity. A Pool ID will be provided in this PR following the stream's completion once the team has had a chance to create a pool using the earned funds.

Validation Testing
If adding or updating a chain or asset, the changes must be validated from the updated build of the Osmosis Frontend. Testers will need a small amount of tokens to validate.

[ ] Validaters can buy a small amount of this asset from Pool ID: {PROVIDE POOL ID}.
[ ] Preview build has been validated.

@JeremyParish69 - pool created and initial liquidity provided for testing. ID is 1325
Moved to ready for review :)


Add XcodeCodeSnippets

Project URL

XcodeCodeSnippets
Description

A set of snippets for iOS development.
Why it should be included to awesome-ios
Because the snippets really useful thing which allows you to save time in the development.
Checklist

[x] Only one project/change is in this pull request
[x] Addition in chronological order (bottom of category)
[x] Supports iOS 9 / tvOS 10 or later
[x] Supports Swift 3 or later
[x] Has a commit from less than 2 years ago
[x] Has a clear README in English

          2 Warnings

:warning:
Please update the Pull Request title to contain the library name

:warning:
Found 1 link issue, a project collaborator will take care of these, thanks :)

Link issue by awesome_bot

Line
Status
Link

634
301
https://github.com/lucdion/FlexLayout redirects tohttps://github.com/layoutBox/FlexLayout

  Generated by :no_entry_sign: Danger

@ismetanin this is so useful! Thanks for contributing 👍


About Inference Time

When I run tools/test.py with WFLW dataset , It may cost 115s on single 2080Ti, 85s on 4 2080Tis.
And the CPU cost is high with 300% , but gpu always cost 0% and gpu-memory is very low even though set batchsize as 256.
So It is working fine? If not, can u tell me how to do with it.

Can you share your train time and test time in one-epoch's data

We've provided training and testing logs in this issue #4


Importing widgets to support catboost in the vs code python notebook

Bug: Notebook Editor, Interactive Window, Editor cells

Steps to cause the bug to occur

Actual behavior
Expected behavior

Your Jupyter and/or Python environment
Please provide as much info as you readily know

Jupyter server running: Local | Remote | N/A
Extension version: 20YY.MM.#####-xxx
VS Code version: #.##
Setting python.jediEnabled: true | false
Python and/or Anaconda version: #.#.#
OS: Windows | Mac | Linux (distro):
Virtual environment: conda | venv | virtualenv | N/A | ...

Developer Tools Console Output

Microsoft Data Science for VS Code Engineering Team: @rchiodo, @IanMatthewHuff, @DavidKutu, @DonJayamanne, @greazer

Closing this issue as none of the requested information has been provided. Please update or create a new issue with the issue template filled with relevant info


WIP Resolved the 500 error - "Compare Master"

I have made changes to the code and it will do the following things,

Disabled the compare icon in translation mode. but still, you can go to the article by clicking the chip. Hovering to the icon it will display a message in tooltip showing "cant compareSoure.

In the edit mode it will work as same as before..

This change is

may I recommend


Check if VM exists before deletion

Description of changes:
You can't access the status of a VM after it has been deleted if you restrict the permissions of the Proxmox user or token to a pool. The API response code isn't 404 but 403 - because the non-existent VM no longer belongs to a pool and therefore the permission check fails.
This PR adds a call to CheckID to check if the vmid in question exists. If it doesn't exist, we can assume that the VM has already been deleted.
I added the CheckID function to luthermonson/go-proxmox in this PR: https://github.com/luthermonson/go-proxmox/pull/157
It's included in the 0.1.1 release of the library.
If this PR is merged, I would be happy to provide an additional PR for the docs, extending them to include instructions for a minimum-permission PVE token.
After merging this PR, the permissions that CAPI needs are
 acl {
    path = "/"
    propagate = false
    role_id = "SysAudit"
  }
  acl {
    path = "/nodes
    propagate = true
    role_id = "SysAudit"
  }
  acl {
    path = "/sdn/zones/localnetwork/vmbr0/330"
    propagate = false
    role_id = "PVESDNUser"
  }
  acl {
    path = "/storage/capi_poc01_block"
    propagate = false
    role_id = "PVEDatastoreUser"
  }
  acl {
    path = "/storage/capi_poc01_file"
    propagate = false
    role_id = "PVEDatastoreAdmin"
  }
  acl {
    path = "/pool/packer
    propagate = false
    role_id = "PVETemplateUser"
  }
  acl {
    path = "/pool/capi_poc01"
    propagate = false
    role_id = "PVEVMAdmin"
  }

Testing performed:
There is currently no test for the DeleteVM function. Writing a test including the API mock would exceed the time I'm able to invest in this PR.

but also do not understand the note regarding the permissions.

If the Proxmox user/token used by the cluster-api-provider is restricted to a pool, as in this example, it's not allowed to query the status of a non-existent VM (404 vs. 403, see below):
  acl {
    path = "/pool/capi_poc01"
    propagate = false
    role_id = "PVEVMAdmin"
  }

The call made (here)[https://github.com/ionos-cloud/cluster-api-provider-proxmox/blob/7b75c862dd90e833c6ff1e6a34a9e1a7e340f39d/pkg/proxmox/goproxmox/api_client.go#L154] fails because the Proxmox API doesn't respond with 404 (Not Found) but with 403 (Permission denied).
To check if the VM is deleted without having permissions to access all VMs, the CheckID function is used which doesn't need permissions on all VMs, but only checks if the VM id is free. We can then assume that the VM doesn't exist.

Let's hope this also fixes #96

Please fix the linter

😬 Fixed the two linter errors.


Bug: cannot cancel read command

Steps
ydsh-0.18$ read -u ${new UnixFD("/dev/dri/renderD128")}

Problem
cannot interrupt by ctrl-c

close in commit e729761f2bf311a72daa061a00057e026799a7b4


undoable feature

Hi @pardom, first thanks for you amazing lib!
I was wondering if you planned to add a built-in "undoable" feature to this lib (or creating a dedicated one) like described here for instance?
Thanks again,
Thomas

@pardom if not planned, do you have any clue on how I could implement an enhancer which return a different state object than the reducer enhanced?
Thanks,
Thomas

@otomatik undo would be implemented as reducer enhancer. Since Kotlin is statically typed and JS is dynamically typed, we have to do things a bit differently. Here is some pseudocode of what the API might look like.
val undoReducer = createUndoReducer(getVanillaReducer())
val store = createStore(undoReducer, initialState(), createStoreEnhancer())

store.dispatch(Action.Increment)
undoReducer.undo()

So far I've only ported the middlewares that we've needed in production. When the need for something arrises, I will generally find the best middleware for our needs and create a direct port. In the case of undo, it seems that this library is the most used?
https://github.com/omnidan/redux-undo
If you need undo, I'd take a run at porting that. I've been considering creation of a redux-kotlin Github org and migrating all the redux projects I've authored. This would facilitate a central org for anyone to expand the Kotlin Redux ecosystem. In this case, I'd be happy to accept your contributions.

@pardom thanks for your answers and your pseudocode :)
Indeed the langages differences lead to a different undoable API.
I would love to help in the redux-undo port even if I'm quite new to Kotlin and Redux (it would be a nice way to learn both of them).
Let me know if you decide to move your redux libraries to a dedicated organization.
Thanks again,
Thomas


webapp: az webapp deployment source config-zip should support the packageUri scm API

Is your feature request related to a problem? Please describe.
Kudu supports packageUri - Zipdeployment command should support this as well https://github.com/projectkudu/kudu/wiki/Deploying-from-a-zip-file-or-url
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

This would be great for all kinds of scenarios. I'd also love to see support for this from az webapp create, similar to the way --deployment-source-url works.

waiting for this support to be extended to Linux as well - & then add support for this feature.

Still waiting for the api changes to be available for Linux. @btardif assigning this to you so that you can push for the Linux backend changes here. Thanks!

Moving this out to the next sprint

@panchagnula what is the status of this issue ?

How is the status of this issue ?

Can we close this issue?

How about this issue?

move to S165.

move to S166.

move to S167.


Create a Datastore interface to represent the RPC layer and load it via ServiceLoader

This would enable different Datastore implementation (such as CapeDwarf) to be plugged instead of Google Cloud Datastore.

Done.


Added support for PHP 7.4, removed support for PHP 7.0 and 7.2, added…

… support for Laravel 6.

@matthew-contentful great to see this PR, do you know when it will be merged?


How to review all changes for single work item within single review request?

Imagine a situation: Dev is working on product backlog item. He works on story for about 2 hours and during this time checks in 4 small changesets, each containing changes in 3 files. Now entire user story is ready to pass code review. How he can request code review in such situation?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d8bb12a7-a20c-1dd5-bca0-4873f7ab9e36
Version Independent ID: 360739fd-e98a-535f-9872-ed58ece28194
Content: Get your code reviewed with Visual Studio - Azure Repos
Content Source: docs/repos/tfvc/get-code-reviewed-vs.md
Product: devops
GitHub Login: @steved0x
Microsoft Alias: sdanie

Unfortunately, we don't have a great story here.  You can review any single changeset or shelveset, but not a range of changesets.
In this type of scenario, I might recommend using shelvesets to shelve the work multiple times instead of checking the 4 changesets.  Then, when the work is ready for review, review the shelveset before checking in the changes.

You can pull a shelveset into your local workspace and then push updates into that same shelveset. By reviewing the shelveset, you're effectively reviewing the accumulation of all of your individual pushes to a shelveset because a shelveset has no individual history - it's basically a living, single revision that you continually update.


A bug in aug_random_edge

Hi, Shen. Thanks for your efforts in this program. I noticed there might have a logical bug in the aug_random_edge function.
“for i in index_list:
single_index_list.append(i)
index_list.remove((i[1], i[0]))”
Removing items in a list while iterating that list may cause problems, as the list changed but the current index is not changed.
This blog illustrated well.
https://thispointer.com/python-remove-elements-from-a-list-while-iterating/
Thanks for your help.

Hi @skydetailme,
Thanks for your interest in our work. Would you mind pointing out which file you are referring to? I think in my implementation I did not use "remove()" function but directly use indexing to modify them, e.g.
https://github.com/Shen-Lab/GraphCL/blob/88f2a1590a67bbbe1ae848156e74adc08f888271/semisupervised_TU/pre-training/tu_dataset.py#L224

Hi @skydetailme,
Thanks for your interest in our work. Would you mind pointing out which file you are referring to? I think in my implementation I did not use "remove()" function but directly use indexing to modify them, e.g.
https://github.com/Shen-Lab/GraphCL/blob/88f2a1590a67bbbe1ae848156e74adc08f888271/semisupervised_TU/pre-training/tu_dataset.py#L224

Hi, thanks for the reply. Here is the link.
https://github.com/Shen-Lab/GraphCL/blob/master/unsupervised_Cora_Citeseer/aug.py

Hi, thanks for the reply. Here is the link.
https://github.com/Shen-Lab/GraphCL/blob/master/unsupervised_Cora_Citeseer/aug.py

@yongduosui, would you mind giving some comments on this? Thanks!

@yongduosui, would you mind giving some comments on this? Thanks!

@skydetailme The aim of this code is to convert to convert bidirectional edges index to unidirectional index. I just debug my code and did't find any bug. It remove the other edge index pair successfully. You can print the list and check. Thanks.

@skydetailme The aim of this code is to convert to convert bidirectional edges index to unidirectional index. I just debug my code and did't find any bug. It remove the other edge index pair successfully. You can print the list and check. Thanks.

@skydetailme The aim of this code is to convert to convert bidirectional edges index to unidirectional index. I just debug my code and did't find any bug. It remove the other edge index pair successfully. You can print the list and check. Thanks.

Would you mind take some minutes to read the link I posted, it explains well.
And you will find that after this iteration, index_list is not empty, those are the items accidentally jumped because of index error. Those items are not solved.
The result may right on some occasions but this code may not logically right.

@skydetailme The aim of this code is to convert to convert bidirectional edges index to unidirectional index. I just debug my code and did't find any bug. It remove the other edge index pair successfully. You can print the list and check. Thanks.

Would you mind take some minutes to read the link I posted, it explains well.
And you will find that after this iteration, index_list is not empty, those are the items accidentally jumped because of index error. Those items are not solved.
The result may right on some occasions but this code may not logically right.

@skydetailme The aim of this code is to convert to convert bidirectional edges index to unidirectional index. I just debug my code and did't find any bug. It remove the other edge index pair successfully. You can print the list and check. Thanks.

Also, thanks for your help.

@skydetailme The aim of this code is to convert to convert bidirectional edges index to unidirectional index. I just debug my code and did't find any bug. It remove the other edge index pair successfully. You can print the list and check. Thanks.

Also, thanks for your help.

@skydetailme The aim of this code is to convert to convert bidirectional edges index to unidirectional index. I just debug my code and did't find any bug. It remove the other edge index pair successfully. You can print the list and check. Thanks.

It seems that if the adj matrix is self-loop, this code may cause a wrong result, and the index_list is not empty.
If it's not a self-loop adj matrix, the result is right.

@skydetailme The aim of this code is to convert to convert bidirectional edges index to unidirectional index. I just debug my code and did't find any bug. It remove the other edge index pair successfully. You can print the list and check. Thanks.

It seems that if the adj matrix is self-loop, this code may cause a wrong result, and the index_list is not empty.
If it's not a self-loop adj matrix, the result is right.

@skydetailme Thank you for your carefulness, and I see your point. The citation net datasets happen to be without self-loop. I will read the post and give some comments after finishing my work at hand.

@skydetailme Thank you for your carefulness, and I see your point. The citation net datasets happen to be without self-loop. I will read the post and give some comments after finishing my work at hand.

@skydetailme Thank you for your carefulness, and I see your point. The citation net datasets happen to be without self-loop. I will read the post and give some comments after finishing my work at hand.

Thanks for your reply and contribution. This project helps a lot.

@skydetailme Thank you for your carefulness, and I see your point. The citation net datasets happen to be without self-loop. I will read the post and give some comments after finishing my work at hand.

Thanks for your reply and contribution. This project helps a lot.

Hi @skydetailme,
Sorry for the late reply that I just survive a DDL. I make certain modifications according to the blog u share https://thispointer.com/python-remove-elements-from-a-list-while-iterating/. Thanks again for your carefulness.


Add warning or error for incorrect training data version

Context here

While experimenting with this repo on rasa main, which uses training data version 2.0, @ancalita noticed that there's no warning or error if the training data is not updated to 3.0.
Do we want a warning or an error here? Do we want to enforce upgrading?

Hey @carlad I don't have a strong opinion here, although I'd open a Slack thread to consult the whole squad for a more productive discussion.

although I'd open a Slack thread to consult the whole squad for a more productive discussion.
@carlad I agree with Anca here. Creating a quick proposal for what you think would be a good approach here and asking @enable in Slack would be awesome


Working with Express JS

I raised an issue a few days ago about the bot not being compatible with Express JS. I found that the issue is that the cookie set in the redirect gets lost on the callback (my example is using the Facebook strategy).
I forked the repo to work in a solution. What I ended up doing was creating dynamic routes for the botauth endpoints, using the bot user id. For that to work I had to change the behaviour to set the callbackUrl dynamically, which is possible. With that id I was able to store the address in a closure map, that then I was able to retrieve in the callback endpoint.
So this worked for me (using express and Facebook strategy). But not sure if it would work for others. I know Restify is not that great handling dynamic routes.
If anybody is having the same issue, Im happy to share my solution (or just look at my fork)
Thanks for the hard work on this library, its awesome!

I think better support of express is a good goal for the next version.  Right now I'm directly using the cookie library to implement resumption, but I might just write to session and let express/restify handle that which should give you some more flexibility and normalize the differences between express/restify.  Until now I haven't done much testing on express, but for the next major version I'll actually try it out.

@mattdot  Hey, do you do the customer OAuthen for customer website


Fix loading EXR with alpha channel

Fixes #24659.
0 is a valid index, and actually the one used by the images in #24659 which are encoded as ABGR.

Not relevant for 3.0 branch.


Usage code is not up to date and doesn't work with SDK

Hi, the sample code at https://github.com/tibber/Tibber.SDK.NET doesn't work.
Installing the nuget package manager is not enough to get the code working (or the code needs to be updated to work with the current SDK).
One error I get at several lines is: "Error CS4033 - The 'await' operator can only be used within an async method"
Thanks!
/Johan

Hi @jvmatern. Thanks for reporting this. On which platform are you running it? .NET core or earlier framework? Which version?

Could you share an example of an Error CS4033 you get with details?

Also, note that you need to put the example codes within an async function.

Hi, sorry for late reply but I cannot do this during office hours.
I've been using VS2019 and .NET framework 4.7.2 and tried to implement the code in an C# Windows Application.  I hope this answers all questions about the environment.
You are correct in your assumtion that I haven't put the code in an async function.  I'm a noob with basic programming skills and hade hoped that the sample code was provided at a completion level where even I could get going. I'm now realizing that the sample code needs higher skill level than basic.
I had also another error message when trying the code, but that one disappeared when I installed the GraphQlClientGenerator.  I think that error was in the QueryBuilderExtensions Class, not sure exactly where (I've forgotten). This lead me to the assumption that more nuget installs (than Tibber.SDK)  are needed to get the code working.

We can surely try to improve the example code bits! Thanks for input :)

Hi, your comment about async function was gold, it was so little missing to get the errors cleared.
I'm sure I'm not the only one that gets stuck at the next level by realizing that the data is trapped within the function. Haha, so frustrating. Please, if you or someone makes the code even easier to use then including some code to get the data out of it would be greatly appreciated. I've actually tried to get help from programmers at the office today, but realized that none of them knew how to do it. I will continue my struggle and hope Google can provide help.
Thanks!
`static async Task GetDataFromTibber(string accessToken )
{
        var userAgent = new ProductInfoHeaderValue("My-home-automation-system", "1.2");
        var client = new TibberApiClient(accessToken, userAgent);
        var basicData = await client.GetBasicData();
        var homeId = basicData.Data.Viewer.Homes.First().Id.Value;
        var consumption = await client.GetHomeConsumption(homeId, EnergyResolution.Monthly);

        var customQueryBuilder =
            new TibberQueryBuilder()
                .WithAllScalarFields()
                .WithViewer(
                    new ViewerQueryBuilder()
                        .WithAllScalarFields()
                        .WithAccountType()
                        .WithHome(
                             new HomeQueryBuilder()
                                .WithAllScalarFields()
                                .WithAddress(new AddressQueryBuilder().WithAllFields())
                                .WithCurrentSubscription(
                                    new SubscriptionQueryBuilder()
                                        .WithAllScalarFields()
                                        .WithSubscriber(new LegalEntityQueryBuilder().WithAllFields())
                                        .WithPriceInfo(new PriceInfoQueryBuilder()
                                          .WithCurrent(new PriceQueryBuilder().WithAllFields())
                                          .WithToday(new PriceQueryBuilder().WithAllFields())
                                          .WithTomorrow(new PriceQueryBuilder().WithAllFields())
                                        )
                                )
                                .WithOwner(new LegalEntityQueryBuilder().WithAllFields())
                                .WithFeatures(new HomeFeaturesQueryBuilder().WithAllFields())
                                .WithMeteringPointData(new MeteringPointDataQueryBuilder().WithAllFields()), homeId
                        )
                );

        var customQuery = customQueryBuilder.Build(); // produces plain GraphQL query text
        var result = await client.Query(customQuery);

        var Address = result.Data.Viewer.Home.Address;
        var Current = result.Data.Viewer.Home.CurrentSubscription.PriceInfo.Current;
        var Today = result.Data.Viewer.Home.CurrentSubscription.PriceInfo.Today;
        var Tomorrow = result.Data.Viewer.Home.CurrentSubscription.PriceInfo.Tomorrow;

        foreach (var d in result.Data.Viewer.Home.CurrentSubscription.PriceInfo.Tomorrow)
        {
           Console.WriteLine(d.StartsAt + "\t" + d.Energy);
        }

    }

`

Readme updated!


Transitive deps of substituted libraries are inherited by their replacement

Given:
H depends on C version 1.2 published under group name X
A depends on B version 1.0 which depends on C version 1.0 published under group name Y
B version 1.1 which depends on C version 1.0 published under group name Y
In this situation one needs to exclude the earlier version manually because the change in group name across versions prevents gradle from recognizing the newer version as a valid alternative. Thus per the docs:
compile("H")
compile("A")
compile("B:1.1") {
  exclude group:"Y"
}

Expected Behavior
C version 1.0 should not appear in gradle dependencies
A should have B:1.1 substituted for B:1.0 per normal, which should obliterate B:1.0's reference to C:1.0
The resolution of B:1.1 should manually exclude anything from group Y also obliterating that reference to C:1.0, which should leave no references to C:1.0 in the project (thus resolving the conflict created by the group change between versions)
Current Behavior
C version 1.0 appears as a dependency in gradle depenencies and is packed into the ensuing war file along side C version 1.2 (required by H)
+ B:1.1
| + Y:C:1.0

This can then interfere with H if it loads classes from C:1.0
Context
Please see http://stackoverflow.com/a/42079525/1172174
Steps to Reproduce (for bugs)
Please see http://stackoverflow.com/a/42079525/1172174
Your Environment
http://stackoverflow.com/a/42079525/1172174

I'm not sure we could make that expected behavior happen, as conflict resolution happens after the dependencies are already added to the graph. @bigdaz can probably tell you for sure.
In general, don't use excludes on individual dependencies, but on configurations instead. That way the unwanted dependency is sure to be excluded, no matter where it comes from.
configurations.all {
  exclude group: 'Y'
}


Toolbar buttons in rich text fields stop being highlighted properly since v1.2

When bold, italic, headings or paragraphs are selected in a rich text field, the related button is not highlighted since v1.2

git bisect shows that this was broken by 015e9da9b83844e7f1de657339f45015cf13ac1b.

According to #1746, the CSS change which caused this (by effectively overriding the button highlight style so that it only occurs on mouse hover) was a fix for this issue:

When the cursor is positioned directly over where a button would appear in the Hallo toolbar, then the toolbar appears, the button is solid white until the cursor is moved. This has been fixed.

which - according to my comment at https://github.com/torchbox/wagtail/pull/1746#issuecomment-151467730 - is a bug that I was never able to replicate in the first place (and still can't). As such, I have no qualms about reverting this :-)

Have submitted PR #2042 to revert this - any objections @alexgleason?

@gasman Go for it! That code definitely shouldn't have to exist... not sure why it was working for me but is causing a problem now.


Update streamflow data location specification

Update streamflow data location specification.

@chengzhuzhang With these changes, the user only has to specify the test and ref data paths rather than a specific .mat file or .csv file. There are a few caveats:

The gauge file (the .csv file) must be in a subdirectory of parameter.reference_data_path called gauges
If the user wants to use a .mat file rather than a .nc file, then parameter.reference_data_path must end with mat. Furthermore, there can only be one .mat file in that directory.

My data is currently stored locally as:
-3yr_nc/
--RIVER_DISCHARGE_OVER_LAND_LIQ_195901_196112.nc
--gauges/
---GSIM_catchment_characteristics_all_1km2.csv

-500yr_nc/
--RIVER_DISCHARGE_OVER_LAND_LIQ_000101_050012.nc

-edison_mat/
--edison.20181204.noCNT.A_WCYCL1950S_CMIP6_LRtunedHR.ne30_oECv3_ICG_0001-0050.mat

-obs_mat/
--GSIM.mat
--gauges/
---GSIM_catchment_characteristics_all_1km2.csv

-test_mat/
--test.mat

If all of the above is acceptable, I can add the GSIM.mat file and the GSIM_catchment_characteristics_all_1km2.csv file to a new directory in /global/project/projectdirs/e3sm/acme_diags/obs_for_e3sm_diags/ as follows:
-/global/project/projectdirs/e3sm/acme_diags/obs_for_e3sm_diags/gsim_mat/
--GSIM.mat
--gauges/
---GSIM_catchment_characteristics_all_1km2.csv

What are your thoughts? What other machines need the data transferred to them? I'm assuming acme1 and compy. Any others?

@forsyth2 I would suggest to have a sub-directory /GSIM under /obs_for_e3sm_diags/time-series/:
/obs_for_e3sm_diags/time-series/GSIM/gauges/GSIM_catchment_characteristics_all_1km2.csv
and
/obs_for_e3sm_diags/time-series/GSIM/GSIM_198601_199512.mat (renamed GSIM.mat file)
We should expect test dataset to be .nc files all the time. and reference data can be .mat in the case of model vs obs, and reference to be .nc for model vs model.
In model vs model case, the user should define the path to the gauges csv. Let me know if this makes sense..

I can't find my last comments. But
@forsyth2 I think the blocks reads in data (L76 -190 in streamflow_driver.py) may need to be straighten a bit to follow something as below:
#not an actual code block
if parameter.run_type = 'model-vs-model':
    Ask for gauge locations .csv file, hint on the possible location where e3sm obs data located () : ex parameter.guages = '/global/project/projectdirs/e3sm/acme_diags/obs_for_e3sm_diags/time-series/GSIM/gauges/GSIM_catchment_characteristics_all_1km2.csv' for Cori

elif parameter.run_type = 'model-vs-obs':
     Should automatically gets the .csv file and .mat files based on ref_data_path ('/global/project/projectdirs/e3sm/acme_diags/obs_for_e3sm_diags/time-series/` take Cori for example)

else:
      run_type not supported.

Let me know if this makes sense, i can try clarify.

@chengzhuzhang Thanks for the feedback. Does the third commit look good? I tested it out with model_vs_model, mdoel_vs_obs, an unsupported run type and model_vs_model with unspecified gauge path.
I will work on updating the tests accordingly in #306 (updates tests to not need nc files in the repo)

@forsyth2 Thanks for the update.

for timeseries file, usually user specifies: reference_data_path = 'prefix/obs_for_e3sm_diags/time-series/'. But when i tested, i have to use prefix/obs_for_e3sm_diags/time-series/GSIM.
Would you rename GSIM.mat to GSIM_198601_199512.mat, so that the obs file name is more consistent than others
The RIVER_DISCHARGE_OVER_LAND_LIQ_195901_196112.nc you have apparently is not a per-variable file, it includes all the variables from mosart. I contacted Charlie for using ncclimo to add areatotal2 to per-variable file.

@chengzhuzhang I added two more commits, addressing your first two comments. I specify ref_name as a parameter now. I tested model_vs_model, model_vs_model with no gauge path specified, and model_vs_obs -- all looks correct to me.


Delete comment in admin panel?

Good day! Thank you for your awesome plugin!
As there a way to delete comment in admin panel?
As i understood correctly you hide collection "Comments", right? Because in my admin only "Comments" in "Plugins" section

Hi @alexey13 ,
All the actions with comments must be done via dedicated UI that is one of an items in Plugins section. Hope that helps.

Yeah, and we have "Hide" button, but we cant delete, right?

In couple of days we will release new version with community contribution in that terms so deleting is going to be possible. For now "hide" is the only option to manage it and mark what we don't want to send to the client UI.

Awesome, thank you!


Imported modules are wrapped in a "default" namespace when passed to a function.

Bug report

What is the current behavior?
Importing a JSON module with the import * as x from './path.json'; syntax, and then passing the module object to a function causes the module to be wrapped in a "default" namespace.
If the current behavior is a bug, please provide the steps to reproduce.
There's a pretty simple repro here: https://github.com/iclanton/webpack-import-repro
After installing dependencies, run npm run build, and then run node dist/test.js. The package.json file's "name" property will be printed to the console, and then undefined will be printed to the console.

What is the expected behavior?
The module should not be wrapped in a "default" namespace. In the repro repo, the package.json's "name" property should be printed to the console twice.

Other relevant information:
webpack version: 4.43.0
Node.js version: v10.19.0
Operating System: Windows 10
Additional tools:

This is fixed in webpack 5. It will also print this warning:
WARNING in ./src/index.js 7:12-28
Should not import the named export 'name' (imported as 'packageJson') from default-exporting module (only default export is available soon)

It's telling you that you are using JSON modules incorrectly. The official spec says that only a default export is available for JSON. So you better change your import to import x from './path.json', since this is the spec'ed way.

@sokra - Using import packageJson from '../package.json'; syntax, the object isn't tree-shaken and the property names aren't minified in production mode. I see that this is fixed in Webpack 5, but this is currently broken in Webpack 4.
Can you point me to the spec? I'm having trouble finding it.

See link in this issue https://github.com/webpack/webpack/issues/9246


playground, use tsconfig-paths-webpack-plugin

npm install --save-dev tsconfig-paths-webpack-plugin
instead of the manual config in webpack.config.js

not relevant anymore, closing


Set 1234 and myjob in https

Coverage remained the same at 100.0% when pulling e0cf2de2ac6d1bcd8a4206afff13ade506eb081b on cho-move-to-https into 9fff744c126b02dd0ececbcb30a1cc06cd09580e on master.


Use JSON schemas for WorkflowTask arguments

An example of a WorkflowTask.args JSON schema could be:

Argument "arg1" of type "string
Argument "arg2" of type "list of strings"
Argument "arg3 of type list[Channel]

Where Channel is an "object of string->boolean key/value pairs, with keys "A", "B" and "C"".
Each argument comes with required=True/False

(even more complex: what about WorkflowTask.task.default_args?)

An example of a schema could be the following, where for a given workflow task we have a property which is a list of objects.
Each object represents the definition of a workflow task argument. Arguments could be nested.
{
  "workflow_task_name": "workflow_task_name",
  "workflow_task_id": 1,
  "workflow_task_schema": [
    {
      "argument_name": "task_argument_name",
      "argument_type": "object",
      "argument_description": "description",
      "default_value": "default value, with respect to argument type",
      "is_required": true,
      "inner_argument": {
        "argument_name": "inner_argument_name",
        "argument_type": "name of type",
        "argument_description": null,
        "default_value": null,
        "is_required": true,
        "inner_argument": null
      }
    }
  ]
}

Here (below) is a realistic example of what comes out of pydantic schema method (note: the method name changes when moving to pydantic 2). This is the raw output, that we could make available in fractal-server rather quickly.
If needed, I suggest that the logic for converting it towards a different kind of schema (like the one in https://github.com/fractal-analytics-platform/fractal-web/issues/82#issuecomment-1496008967) is implemented as part of the the web client - to avoid relying on custom definitions (i.e. different from pydantic) in multiple places (tasks repository and web-client repository).
Example 1
Task arguments
class TaskArguments(BaseModel, extra=Extra.forbid):

    x: int = Field(description="This is the description of argument x")
    y: Optional[str]

print(json.dumps((TaskArguments.schema()), indent=2))

JSON schema:
{
  "title": "TaskArguments",
  "type": "object",
  "properties": {
    "x": {
      "title": "X",
      "description": "This is the description of argument x",
      "type": "integer"
    },
    "y": {
      "title": "Y",
      "type": "string"
    }
  },
  "required": [
    "x"
  ],
  "additionalProperties": false
}

Example 2
Task arguments:
class TaskArguments(BaseModel, extra=Extra.forbid):
    input_paths: Sequence[str]
    output_path: str
    metadata: Dict[str, Any]
    image_extension: str
    image_glob_patterns: Optional[list[str]]
    allowed_channels: Sequence[Dict[str, Any]]
    num_levels: Optional[int]
    coarsening_xy: Optional[int]
    metadata_table: Optional[str]

print(json.dumps((TaskArguments.schema()), indent=2))

JSON schema
{
  "title": "TaskArguments",
  "type": "object",
  "properties": {
    "input_paths": {
      "title": "Input Paths",
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "output_path": {
      "title": "Output Path",
      "type": "string"
    },
    "metadata": {
      "title": "Metadata",
      "type": "object"
    },
    "image_extension": {
      "title": "Image Extension",
      "type": "string"
    },
    "image_glob_patterns": {
      "title": "Image Glob Patterns",
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "allowed_channels": {
      "title": "Allowed Channels",
      "type": "array",
      "items": {
        "type": "object"
      }
    },
    "num_levels": {
      "title": "Num Levels",
      "type": "integer"
    },
    "coarsening_xy": {
      "title": "Coarsening Xy",
      "type": "integer"
    },
    "metadata_table": {
      "title": "Metadata Table",
      "type": "string"
    }
  },
  "required": [
    "input_paths",
    "output_path",
    "metadata",
    "image_extension",
    "allowed_channels"
  ],
  "additionalProperties": false
}

Another example:
from pydantic import BaseModel
from pydantic import Extra
from typing import Optional
import json

class Channel(BaseModel):
    x: int
    y: Optional[int]

class TaskArguments(BaseModel, extra=Extra.forbid):
    channels: list[Channel]

print(json.dumps((TaskArguments.schema()), indent=2))

{
  "title": "TaskArguments",
  "type": "object",
  "properties": {
    "channels": {
      "title": "Channels",
      "type": "array",
      "items": {
        "$ref": "#/definitions/Channel"
      }
    }
  },
  "required": [
    "channels"
  ],
  "additionalProperties": false,
  "definitions": {
    "Channel": {
      "title": "Channel",
      "type": "object",
      "properties": {
        "x": {
          "title": "X",
          "type": "integer"
        },
        "y": {
          "title": "Y",
          "type": "integer"
        }
      },
      "required": [
        "x"
      ]
    }
  }
}

Here are some first thoughts, with @rkpasia. A lot more will come up during implementation, and also we will open more specific issues.

If a task has not args schema (that is, it has null value), then the current behavior is expected (arguments are to be added/edited one by one). This "freestyle-form" code path (for the moment) could diverge in style&features from the one based on schemas, and we can decide later on whether to merge them into a single place.
If a schema comes with N arguments, they should always all be visible in the WorkflowTask page.
If an argument is optional (which is known, from the schema), the user can "uncheck" it (e.g. via a tick on its right). Note that there are multiple options of how this could work (ref also https://github.com/fractal-analytics-platform/fractal-server/issues/629), let's discuss it somewhere else. High-level: optional arguments are not required, obviously.
A user cannot add an argument, when a JSON schema is present. The rationale for this is that the schema is considered as the ground truth for which arguments are present, and any other argument would simply be ignored by the task.
To be clarified: what does the user see when they create a WorkflowTask? For sure they should see the N arguments, but what are the defaults?
When available, the "description" field of an argument is visible (either via an "info" button, or overlay message, or something else).
The first step towards this feature should concern with scalar arguments (that is, no objects and no arrays). This is because objects and arrays are slightly more complex to handle (we should be able to add/remove elements, since their number is not defined).

Something a bit more general:
8. What is the format of the argument schema? Right now we will go with whatever comes out of pydantic 1, which is "compliant with the specifications: JSON Schema Core, JSON Schema Validation and OpenAPI." Longer term, we may have to specify a subset of one of those broader standard definitions that is supported for fractal-web.

Here is the first example we should address (note: priority is obviously for scalars). Possible additional complexity would come from defining custom types, as in https://github.com/fractal-analytics-platform/fractal-web/issues/82#issuecomment-1514493273.
Note: Field also accepts several other arguments, see https://docs.pydantic.dev/usage/schema/#field-customization. I'd rather use as few as possible, but let's check whether some are relevant for us.

Pydantic:
from pydantic import BaseModel
from pydantic import Extra
from pydantic import Field
from typing import Optional
import json

class TaskArguments(BaseModel, extra=Extra.forbid):

    i1: int
    i2: int = 1
    i3: int = Field(description="Description of i3")
    i4: int = Field(examples=["i4=8"])
    i5: Optional[int] = None

    f1: float
    f2: float = 0.5

    b1: bool
    b2: bool = Field(
            description="Description of b2",
            default=True,
            title="b2 argument",
            )
    b3: Optional[bool]

    a1: list
    a2: list[int]
    a3: list[list[int]] = Field(default=[[1, 2], [3, 4]])

    o1: dict
    o2: dict[str, int]
    o3: dict[str, list[int]]

print(json.dumps((TaskArguments.schema()), indent=2))

JSON:
{
  "title": "TaskArguments",
  "type": "object",
  "properties": {
    "i1": {
      "title": "I1",
      "type": "integer"
    },
    "i2": {
      "title": "I2",
      "default": 1,
      "type": "integer"
    },
    "i3": {
      "title": "I3",
      "description": "Description of i3",
      "type": "integer"
    },
    "i4": {
      "title": "I4",
      "examples": [
        "i4=8"
      ],
      "type": "integer"
    },
    "i5": {
      "title": "I5",
      "type": "integer"
    },
    "f1": {
      "title": "F1",
      "type": "number"
    },
    "f2": {
      "title": "F2",
      "default": 0.5,
      "type": "number"
    },
    "b1": {
      "title": "B1",
      "type": "boolean"
    },
    "b2": {
      "title": "b2 argument",
      "description": "Description of b2",
      "default": true,
      "type": "boolean"
    },
    "b3": {
      "title": "B3",
      "type": "boolean"
    },
    "a1": {
      "title": "A1",
      "type": "array",
      "items": {}
    },
    "a2": {
      "title": "A2",
      "type": "array",
      "items": {
        "type": "integer"
      }
    },
    "a3": {
      "title": "A3",
      "default": [
        [
          1,
          2
        ],
        [
          3,
          4
        ]
      ],
      "type": "array",
      "items": {
        "type": "array",
        "items": {
          "type": "integer"
        }
      }
    },
    "o1": {
      "title": "O1",
      "type": "object"
    },
    "o2": {
      "title": "O2",
      "type": "object",
      "additionalProperties": {
        "type": "integer"
      }
    },
    "o3": {
      "title": "O3",
      "type": "object",
      "additionalProperties": {
        "type": "array",
        "items": {
          "type": "integer"
        }
      }
    }
  },
  "required": [
    "i1",
    "i3",
    "i4",
    "f1",
    "b1",
    "a1",
    "a2",
    "o1",
    "o2",
    "o3"
  ],
  "additionalProperties": false
}

A user cannot add an argument, when a JSON schema is present. The rationale for this is that the schema is considered as the ground truth for which arguments are present, and any other argument would simply be ignored by the task.

The only use case where this rule would not be appropriate is (as far as I can tell) the following:
I am developing a new task, where I do not use pydantic for argument validation. I still want to have an argument schema, so I write it from scratch (since I don't have a pydantic model to export). I include arguments A, B and C, but later on during development I want to include another argument D.  In principle I would have to modify the schema, and calling the task-edit endpoint (go to the task page, find my own task, click "edit", send the new schema), but maybe I'd rather play with it from within the WorkflowTask editor.
To be honest, this seems a sufficiently edge case that we should not support it, at least for the moment.

Reference about pydantic v1 vs v2:

https://github.com/fractal-analytics-platform/fractal-tasks-core/issues/375

This is obviously complete, at least in its first version. Closing.


MaterialBanner is not constrained vertically

Steps to Reproduce

Run the MaterialBanner demo in the Flutter Gallery with bigger row heights.
Expected results:
The Material Spec has specific heights defined for banners, which are not respected. I expected the banner to only be as tall as it should be by the spec.
Actual results:
The MaterialBanner class stretches its parent as the Column widget has mainAxisSize default to MainAxisSize.max.
Flutter version
Flutter (Channel stable, v1.17.0, on Linux, locale en_US.UTF-8)
    • Flutter version 1.17.0 at /home/jh/Tools/flutter
    • Framework revision e6b34c2b5c (9 days ago), 2020-05-02 11:39:18 -0700
    • Engine revision 540786dd51
    • Dart version 2.8.1

https://material.io/components/banners#specs
Reproducible code sample:
import 'package:flutter/material.dart';

void main() => runApp(MaterialApp(home: BannerDemo()));

class BannerDemo extends StatefulWidget {
  const BannerDemo();
  @override
  _BannerDemoState createState() => _BannerDemoState();
}

class _BannerDemoState extends State<BannerDemo> {
  @override
  Widget build(BuildContext context) {
    final colorScheme = Theme.of(context).colorScheme;

    final banner = MaterialBanner(
      content: Text('bannerDemoText'),
      leading: CircleAvatar(
        child: Icon(Icons.access_alarm, color: colorScheme.onPrimary),
        backgroundColor: colorScheme.primary,
      ),
      actions: [
        Container(
          height: 200,
          child: FlatButton(
            child: Text('signIn'),
            onPressed: () {},
          ),
        ),
        Container(
          height: 200,
          child: FlatButton(
            child: Text('dismiss'),
            onPressed: () {},
          ),
        ),
      ],
      backgroundColor: colorScheme.background,
    );

    return Scaffold(
//      body: banner,
      body: ListView(children: [
        banner,
      ]),
    );
  }
}

Banner is stretched depending on the height of the inner elements. And it is stretched for the all available space if you move it out of ListView.

Persists with current master 1.21.0-6.0.pre.38
[✓] Flutter (Channel master, 1.21.0-6.0.pre.38, on Mac OS X 10.15.6 19G73, locale en-GB)
    • Flutter version 1.21.0-6.0.pre.38 at /Users/nevercode/dev/flutter
    • Framework revision 8e0eee9008 (4 hours ago), 2020-07-26 23:38:01 -0700
    • Engine revision 626244a72c
    • Dart version 2.9.0 (build 2.9.0-21.0.dev a3815b6590)
    • Pub download mirror https://pub.flutter-io.cn
    • Flutter download mirror https://storage.flutter-io.cn

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
    • Android SDK at /Users/nevercode/Library/Android/sdk
    • Platform android-29, build-tools 29.0.3
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.6)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.6, Build version 11E708
    • CocoaPods version 1.9.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 47.1.2
    • Dart plugin version 193.7361
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)

[✓] VS Code (version 1.47.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.12.2

[✓] Connected device (4 available)
    • Redmi Note 7 (mobile) • 6345c469   • android-arm64  • Android 9 (API 28)
    • macOS (desktop)       • macos      • darwin-x64     • Mac OS X 10.15.6 19G73
    • Web Server (web)      • web-server • web-javascript • Flutter Tools
    • Chrome (web)          • chrome     • web-javascript • Google Chrome 84.0.4147.89

• No issues found!

@VladyslavBondarenko this was fixed in efe0c5e

I can still reproduce it on stable 3.3 and master 3.7 the height is based on the intrinsic height of the widgets within the MaterialBanner.
Output when SizedBox Height is set to 500

material specs: https://m2.material.io/components/banners#specs

code sample
import 'package:flutter/material.dart';

void main() => runApp(const MaterialApp(home: BannerDemo()));

class BannerDemo extends StatefulWidget {
  const BannerDemo({super.key});
  @override
  _BannerDemoState createState() => _BannerDemoState();
}

class _BannerDemoState extends State<BannerDemo> {
  @override
  Widget build(BuildContext context) {
    final colorScheme = Theme.of(context).colorScheme;

    final banner = MaterialBanner(
      content: const Text('bannerDemoText'),
      leading: CircleAvatar(
        backgroundColor: colorScheme.primary,
        child: Icon(Icons.access_alarm, color: colorScheme.onPrimary),
      ),
      actions: [
        SizedBox(
          height: 200,
          child: ElevatedButton(
            child: const Text('signIn'),
            onPressed: () {},
          ),
        ),
        SizedBox(
          height: 200,
          child: ElevatedButton(
            child: const Text('dismiss'),
            onPressed: () {},
          ),
        ),
      ],
      backgroundColor: colorScheme.background,
    );

    return Scaffold(
      body: ListView(children: [
        banner,
      ]),
    );
  }
}

flutter doctor -v (mac)
[✓] Flutter (Channel master, 3.7.0-3.0.pre.33, on macOS 12.6 21G115 darwin-arm64, locale en-IN)
    • Flutter version 3.7.0-3.0.pre.33 on channel master
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision<PHONE_NUMBER> (38 minutes ago), 2022-12-05 18:27:21 -0800
    • Engine revision a309d239c4
    • Dart version 2.19.0 (build 2.19.0-463.0.dev)
    • DevTools version 2.20.0
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly
      to perform update checks and upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0-rc4)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-33, build-tools 33.0.0-rc4
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14A400
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] IntelliJ IDEA Community Edition (version 2021.2.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 61.2.4
    • Dart plugin version 212.5080.8

[✓] VS Code (version 1.70.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.55.20221129

[✓] Connected device (3 available)
    • iPhone 12 Pro (mobile) • 026D5789-9E78-4AD5-B1B2-3F8D4E7F65E4 • ios            •
      com.apple.CoreSimulator.SimRuntime.iOS-14-5 (simulator)
    • macOS (desktop)        • macos                                • darwin-arm64   • macOS 12.6 21G115 darwin-arm64
    • Chrome (web)           • chrome                               • web-javascript • Google Chrome 108.0.5359.94

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

[✓] Flutter (Channel stable, 3.3.9, on macOS 12.6 21G115 darwin-arm, locale en-IN)
    • Flutter version 3.3.9 on channel stable at /Users/mahesh/Development/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision b8f7f1f986 (24 hours ago), 2022-11-23 06:43:51 +0900
    • Engine revision 8f2221fbef
    • Dart version 2.18.5
    • DevTools version 2.15.0

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0-rc4)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-33, build-tools 33.0.0-rc4
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14A400
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] IntelliJ IDEA Community Edition (version 2021.2.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 61.2.4
    • Dart plugin version 212.5080.8

[✓] VS Code (version 1.70.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.53.20221101

[✓] Connected device (3 available)
    • iPhone 12 Pro (mobile) • 026D5789-9E78-4AD5-B1B2-3F8D4E7F65E4 • ios            •
      com.apple.CoreSimulator.SimRuntime.iOS-14-5 (simulator)
    • macOS (desktop)        • macos                                • darwin-arm64   • macOS 12.6 21G115 darwin-arm
    • Chrome (web)           • chrome                               • web-javascript • Google Chrome 107.0.5304.110

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!


[SRVKS-999] Add probes to all deployments

Backport of https://github.com/knative/serving/pull/13563 and https://github.com/knative/pkg/pull/2671

/assign @ReToCode

/lgtm
/approve


[Android] Video stuttering / frozen with multiple participants

Hi there,
I've done a side by side comparison with web and android. On calls with 5+ video tiles, I see other video's freezing for 20-30+ seconds, very laggy performance, or stutters.
Have you experienced such issues?
Web seems to be working just fine.
Thanks

@nealmanaktola I think we might need more information. The performance can depend on multiple factors like cpu/network/memory. Would you be able to give more information on following?

When you were testing you, were testing with Android Chrome browser or just Chrome on the computer VS Android app?
What is the device you are using?
What are SDK/Media SDK version?
Do you have any logs?
How many video tiles were you displaying?

Chrome on computer VS Andorid app
Pixel 3
0.7.1/0.6.1 (same as README.md)
No, I'll reproduce and add logs
There was at least 7-8 tiles

I'll reproduce and add logs for you. This is via the react native app btw.

I was wondering to see if the core issue was on the react native layer or if it was with Android specifically.
I did a test with the android sample app:

There were around 12 attendees. I was seeing black tiles at times, video's freezing, it was a bad experience all in all.
I did a test with 4 attendeeds, and the experience was great. No freezing, no black tiles.
I was doing a side by side comparison with web. Web was working fine.
I've added some logs too. Let me know if I should move this issue to the android-sdk repo.
chime-av-log.txt

@nealmanaktola do you mind implementing some pagination and upgrade your SDK version?
There has been few numerous improvements regarding video as described here https://github.com/aws/amazon-chime-sdk-android/releases.
You can also see the guide for pagination. Please note that mobile is very limited on cpu and memory, so having 12 video tiles could be a lot.

I have implemented paging but do see this from time to time. It's definitely not as bad as before, but still happens. I've also made some modifications:

Added a VideoFrameBroadcaster to support multiple tiles
Pause/Resume removes the VideoSinks inside the VideoFrameBroadcaster, instead of calling the actual pause/resume on the audioVideo obj

I have implemented paging but do see this from time to time. It's definitely not as bad as before, but still happens. I've also made some modifications:

Added a VideoFrameBroadcaster to support multiple tiles
Pause/Resume removes the VideoSinks inside the VideoFrameBroadcaster, instead of calling the actual pause/resume on the audioVideo obj


Support for SFSymbols 3

Thank you, @ryanrudes.
However, I will have to close this PR because there has been a large restructuring (https://github.com/piknotech/SFSafeSymbols/pull/72) and now there's https://github.com/piknotech/SFSafeSymbols/pull/80, which updates to v3 within the new structure.


doc: remove duplicate link + slight wording change

Plus, it rhymes now!
What was done
See title
Checklist:

[ NA ] My code follows the style guidelines of this project
[ NA ] I have performed a self-review of my own code
[ NA ] I have commented my code, particularly in hard-to-understand areas
[ NA ] I have made corresponding changes to the documentation

rebased on develop

🙏 Thanks!


[💡FEATURE REQUEST]: allow and forbid directives in the new fileserver plugin

Plugin
File server
I have an idea!
It would be nice for the new fileserver plugin to support the "allow" and "forbid" directives which were available in the old static middleware.

Looks like noone interested in that. Closing as old.


Work?

Hi @Jesus805
Is this currently working to remove the keys?
Thanks

Hi @sabas1080 ,
Not yet, I am currently writing a blog post about this project then I need to fix an issue where the app doesn't install correctly. I'll release for app and firmware for testing after.

But some good news: I was able to successfully pull the keys OTA on three different PoGo devices :) Two legitimate and 1 Fake.
Stay tuned~

Wanted to give an update since it's been a few weeks. Still working on documentation on my blog. Unfortunately it's been taking a while because I only have the weekends free to work on it. Definitely want to finish ASAP 🤞

Eagerly awaiting your findings :)

@jay8ee @sabas1080 The client v1.0 is now released


Town Hall Slides -  Wednesday, 27th April, 2022

Townhall slides -  Wednesday, 27th April, 2022

Previous Town Hall - https://github.com//Catalyst-Circle/Catalyst-Circle-Coordination/issues/98
CC Admin meeting -

[ ] Prepare Town Hall slides for Wednesday

Previous in series: #98

Next in series: #108


Add BD - Advanced Weapons Pack

Replaces #2870

Part of me wants to hardcode an abstract for our users, but I think the mod's name is descriptive enough to make up for its deficiencies.

At least it has something. Some of the KerbalStuffBot submissions are just awful.

Very true.


Feat: Dynamic errors

Summary
Throws errors with advice logged from this repo

Change Type

[ ] Bug Fix
[ ] Refactor
[x] New Feature
[ ] Breaking Change
[ ] Documentation Update
[ ] Performance Improvement
[ ] Other

Checklist

[x] My code follows this project's style guidelines
[x] I've reviewed my own code
[x] I've added comments for any hard-to-understand areas
[x] I've updated the documentation if necessary
[x] My changes generate no new warnings
[x] I've added tests that prove my fix is effective or my feature works
[x] All unit tests pass locally with my changes
[x] Any dependent changes have been merged and published

https://github.com/bcnmy/biconomy-client-sdk/pull/463


Add spark session connection

resolves #272
Description
Adds a connection method to connect to Spark session. See #272  for full discussion.
Checklist

[x] I have signed the CLA
[x] I have run this code in development and it appears to resolve the stated issue
[x] This PR includes tests, or tests are not required/relevant for this PR
[x] I have updated the CHANGELOG.md and added information about my change to the "dbt-spark next" section.
[x] Add new connection method to docs.

@jtcohen6 : This PR is ready to merge! Will you review it once more? Also the accompanying docs PR please.
The PR contains test for the session connection method. Not all dbtspec tests are enabled. I think it is technically possible to run at least the tests that are run for Thrift. I was not able to set-up the connection to a metastore like for Thrift. I think it is technically possible. My expectation is that if that connection is set-up, all the tests that run for Thrift should also run for the session connection method. Given that dbtspec is going to be replaced, I would like to wait for that before investing more time on the test suite, I expect it would become easier to enable more tests.

The failing CI steps look unrelated to this PR

Thanks @JCZuurmond! Triggering a rerun of the failed CI steps now. Looks like it may have been a connection/timeout issue. We did just migrate to a new dedicated Databricks workspace for sandbox/integration testing.