Not declaring that a task uses a shared build service becomes an error in 9.0

Behavior deprecated in 7.4

https://github.com/gradle/gradle/pull/19342

Note that this applies only with the STABLE_CONFIGURATION_CACHE feature flag enabled.
Kotlin Gradle Plugin still does not declare shared build service usages

KT-52149 - Gradle: declare shared build services usages with Task#usesService

It is extremely easy for users to forget to call Task.usesService. In AGP, we have so many violations (https://issuetracker.google.com/300617088), and fixing them exhaustively is quite challenging.
Is there a way Gradle could help users by calling Task.usesService automatically? For example, after Gradle finishes a task's configuration, it can check all the task's properties to see if any properties are RegisteredBuildServiceProvider, and if so, automatically calls Task.usesService directly.
// cc @abstratt who implemented TaskRequiredServices

(I've asked the following on Gradle Slack, but perhaps it's good to post it here too.)
It seems that @ServiceReference (without a service name) is a nicer option than Task.usesService, so we're planning to use it more in our code. But @ServiceReference is currently incubating, so I wanted to check on the following:

When do you think @ServiceReference will be stable?
Currently, Gradle does not accept Property<out BuildService> without an annotation. So users would normally have to annotate it with @Internal. Would it be nicer that Gradle provides one of the following?
2a) Gradle requires users to annotate Property<out BuildService> with @ServiceReference.
2b) [stretch goal] Gradle implicitly processes Property<out BuildService> as if it were already annotated with @ServiceReference, and therefore does not require users to annotate anything (i.e., the effect of adding @ServiceReference is exactly the same as not adding it).

See: https://github.com/gradle/gradle/issues/30858

Thanks, it would be great to get your thoughts on question 2 too.

Personally, I tend to favor explicitness over magic, and for consistency, we should require explicit annotations (and that would be consistent with the other kinds of properties). Also, we should not forbid people using types that implement BuildService from being used as @Internal-annotated properties (as it has already been required with usesService()), at least not before we deprecate usesService(), if for whatever reason we ever choose to do that.

// cc @liutikas
I think the users also like "explicitness over magic", but this case is slightly different: When a Task has a Property<out BuildService>, it means that the task will use the build service. So requiring the association between the two seems a bit unnecessary.
In increasing order of difficulty, it seems that the options for the users are as follows:

Users don't need to declare anything (i.e., Gradle removes the Task.usesService requirement).
Users have a way to identify all places at compile time that they need to annotate @ServiceReference (i.e., Gradle validation checks will help the users with this). The Gradle checks must be complete, such that Gradle does not suddenly fail at run time requiring Task.usesService.
Same as 2, but instead of getting help from Gradle, users need to manually identify all those places, which is tedious and easy to miss.
Same as 3, but instead of annotating @ServiceReference, users need to manually add Task.usesService, which is even more tedious//easier to miss than @ServiceReference.

As you can see, the current available options are 3 and 4, which are not exactly great. Having options 1 or 2 would be really nice.

I see now. This is an example of a validation error for a property with build service type that does not have an annotation:
A problem was found with the configuration of task ':named' (type 'Consumer').
  - Type 'Consumer' property 'counter' is missing an input or output annotation.

    Reason: A property without annotation isn't considered during up-to-date checking.

    Possible solutions:
      1. Add an input or output annotation.
      2. Mark it as @Internal.

I think option 2 is quite reasonable. I opened https://github.com/gradle/gradle/issues/30927 for that.

That's great, thank you!
Just one more thought to consider: Is there a valid use case where a task has a build service property but doesn't use it?

If there is no use case, then the Task.usesService requirement is entirely unnecessary.
If there is a use case (please give a concrete example), it's likely just a user mistake. Gradle can still treat all build service properties as if they would be used, regardless of whether they will actually be used or not, and therefore also eliminate the Task.usesService requirement.

the Task.usesService requirement is entirely unnecessary.

For most scenarios, folks should use @ServiceReference and, in that case, usesService is not needed (as it is redundant).  That is what @ServiceReference was created for (https://github.com/gradle/gradle/issues/16170). Once @ServiceReference is taken out of incubation, it is possible we may in the future eventually deprecate @Internal and usesService for build service properties.

Is there a valid use case where a task has a build service property but doesn't use it?

A task could hold references to multiple services, but only use one of them based on some other configuration (for instance, a production service vs a development service), and so it would invoke usesService on the chosen service, but not on the other one (or assign it to a @ServiceReference property). In that case, automatically inferring a task uses a service just because it holds a reference to it it would be incorrect.

Sorry I wasn't clear: By Task.usesService requirement, I meant the association between a Task and a service, whether by the @ServiceReference annotation or the Task.usesService API. I've updated my previous comment.

A task could hold references to multiple services, but only use one of them based on some other configuration (for instance, a production service vs a development service)

Okay, IIUC your example is:

During task configuration:

task.service1.set(foo)
task.service2.set(bar)

During task execution:

if (someCondition) {
   task.service1.get().doSomething()
} else {
   task.service2.get().doSomething()
}

To avoid the current Gradle errors for the task-service association, the users would have no choice but to fix the task configuration as follows:
task.service1.set(foo)
task.usesService(foo) // Or set @ServiceReference on service1
task.service2.set(bar)
task.usesService(bar) // Or set @ServiceReference on service2

That means whenever a task sets a property to a build service, the user would always need to associate the two. That means the requirement is unnecessary.

The choice between which service to use would need to be done at configuration time:
At configuration time:
if (someConfigTimeCondition) {
   serviceToUse = productionService
} else {
   serviceToUse = developmentService
}
// only if usesService were not a @ServiceReference:
usesService(serviceToUse)

At execution time:
serviceToUse.get().doSomething()

Maybe let's leave @ServiceReference aside to simplify our discussion for now.
Your example can be simplified as:
someService = selectService() // The logic inside selectService() doesn't matter
task.serviceToUse = someService
task.usesService(someService)

That still means whenever a task sets a property to a build service, the user would always need to associate the two.

You are assuming the task would only be assigned the selected service - the choice would be external. That is one possibility. But the choice of which service to use could be made elsewhere, such as in the task implementation itself, and the task would be fed with both options of services.

Maybe let's leave @ServiceReference aside to simplify our discussion for now.

Well, @ServiceReference is the solution we introduced in 8.0 to address the issue of requiring users to remember to call usesService, which seems to be your main concern here (while also bringing automatic injection of services by name/type), so not sure I understand the benefit of leaving it aside.

Okay, let me sum up my key points:

The task-service association requirement is not necessary (option 1 in my earlier comment). It is technically possible for Gradle to remove that requirement: For example, after Gradle finishes a task's configuration, it can check all the task's properties to see if any properties are RegisteredBuildServiceProvider, and if so, automatically calls Task.usesService directly without requiring the users to do it. Note that in this way, Task.usesService is entirely an internal Gradle concept, users don't need to know about it. => It's good for everyone.
If for some reason option 1 is not possible, then yes option 2 is the next best thing (https://github.com/gradle/gradle/issues/30927).

Could you please enter a request for such feature?  However, I am guessing it probably would look a lot like https://github.com/gradle/gradle/issues/16170, for which @ServiceReference was devised.

Also, assuming we can determine whether the consumed service does not specify maxParallelUsages, maybe we should not require usage to be explicitly declared, as AFAIK it brings no actual value.

They would be related, yes, but I think both things could proceed in parallel. There may be cases we can automatically infer service usage, and there may be cases we can't. Your potential feature request could end up being addressed for the simpler (and most common) cases, so the error proposed here would be less likely to be triggered.
At a minimum, we could have more focused discussions.  This issue is a:chore issue and, as such, at least typically, is not a good place for discussing alternative approaches, decisions have been made before/elsewhere, this issue is just a placeholder for the grunt work that will take care of it.

Sure, I've filed 30989 - "Consider removing the Task.usesService requirement".


Google Drive folder is detected as µTorrent installation directory if uTorrent.exe is located in it.

Are there any other applications inside that folder? It's difficult to tell if there's something special about a folder like this, is there anything special about the Google Drive folder that stands out that BCU could detect?


Documentation on shell integration

I want to integrate atuin with nushell/nushell but I couldn't find documentation on how to do so.
I can read the existing shell integration and guess the most of it, but I'd appreciate official documentation such as:
Here are the hooks and the atuin command to run:
- pre-exec: atuin history start (optional)
- post-exec: atuin history stop (optional); atuin history add command <command> (required)
...

I totally made that up and it's 100% wrong, but I'm looking for something like that.

All current integrations follow this model:
pre-exec:
        # records the command and saves the command id
	export ATUIN_HISTORY_ID=$(atuin history start "$1")
post-exec:
        # records the duration and the status of the command
        atuin history end $ATUIN_HISTORY_ID

I believe the history end is mostly optional, but the atuin history start command is mandatory

@conradludgate thank you very much; that's what I need.


Unable to add ocm repository

helm repo add ocm https://openclustermanagement.blob.core.windows.net/releases/

that is a service degradation in azure we use, I am still trying to contact microsft folks.

@xuc623671 Thank you for reporting the issue, this was fixed, could you have a try with helm repo add ocm https://open-cluster-management.io/helm-charts?

@xuc623671 Thank you for reporting the issue, this was fixed, could you have a try with helm repo add ocm https://open-cluster-management.io/helm-charts?

I have tried to use the new repo, but it still doesn't work. The error message is as follows：
Error: looks like "https://open-cluster-management.io/helm-charts" is not a valid chart repository or cannot be reached: Get "https://open-cluster-management.io/helm-charts/index.yaml": read tcp 192.168.xxx.xxxx:xxxx-><IP_ADDRESS>:443: read: connection reset by peer

➜  ~ helm repo add ocm https://open-cluster-management.io/helm-charts
"ocm" has been added to your repositories
➜  ~
➜  ~ helm search repo ocm
NAME                             	CHART VERSION	APP VERSION	DESCRIPTION
ocm/argocd-pull-integration      	0.11.0       	1.0.0      	A Helm chart for OCM Argo CD Pull Model Integra...
ocm/cluster-gateway-addon-manager	1.4.0        	1.0.0      	A Helm chart for Cluster-Gateway Addon-Manager
ocm/cluster-proxy                	0.4.0        	1.0.0      	A Helm chart for Cluster-Proxy OCM Addon
ocm/dummy                        	1.0.2        	1.0.0      	A Helm chart for Dummy
ocm/managed-serviceaccount       	0.5.0        	1.0.0      	A Helm chart for Managed ServiceAccount Addon
ocm/multicluster-controlplane    	0.3.0        	0.3.0      	A Helm chart for multicluster-controlplane
ocm/multicluster-mesh            	0.0.1        	1.0.0      	A Helm chart for Multicluster Service Mesh OCM ...

@xuc623671  it looks good in my MAC and virtual machine.  could you help check your network?

I have checked the firewall and network settings, and the results of testing connectivity using telnet are:
  > [root@master local-helm-repo]# telnet  <IP_ADDRESS> 443
  > Trying <IP_ADDRESS>...
  > Connected to <IP_ADDRESS>.
  > Escape character is '^]'.
  > ^CConnection closed by foreign host.
  > [root@master local-helm-repo]# telnet open-cluster-management.io 443
  > Trying <IP_ADDRESS>...
  > Connected to open-cluster-management.io.
  > Escape character is '^]'.

However, the connection result is as follows when using openssl to check the connection:

[root@master ~]# openssl s_client -connect open-cluster-management.io:443
CONNECTED(00000003)
write:errno=104
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 289 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol  : TLSv1.2
Cipher    : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
Key-Arg   : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time:<PHONE_NUMBER>
Timeout   : 300 (sec)
Verify return code: 0 (ok)
---

According to the output of openssl s_client, the connection is established successfully, but the peer certificate is not received. This may mean that the SSL certificate is not configured correctly on the server side, or there is a network problem that prevents the certificate from being obtained.

OK, I tested it on another machine and the result showed that the above repo is available. I need to check the network of the server.

Thanks

Thanks


"Resource group 'rg-tresl' could not be found" on make all

Describe the bug
When running make all for a fresh deployment, the following error is reported:
Resource group 'rg-tresl' could not be found

Steps to reproduce

Delete any resource groups from previous Azure TRE deployments
Delete .terraform folders locally
Run make all

Describe the bug When running make all for a fresh deployment, the following error is reported:
Resource group 'rg-tresl' could not be found

Steps to reproduce

Delete any resource groups from previous Azure TRE deployments
Delete .terraform folders locally
Run make all

run the "make all" command again. It will work

I'm getting this today too.
ERROR: (ResourceGroupNotFound) Resource group 'rg-mrtredemo' could not be found.
Code: ResourceGroupNotFound
Message: Resource group 'rg-mrtredemo' could not be found.
make: *** [Makefile:169: tre-start] Error 3

@marrobi - the issue I hit was because I have configured az to use table output by default. The change in #1453 is to force JSON output where JSON is expected.
If you run az configure, what do you see as the output value?

@vidit2011998 for my scenario, re-running make all still failed with the same error

[core]
first_run = yes
output = table
collect_telemetry = yes
cache_ttl = 10

Something must have changed this... either the default has changed or something in the dev container?

@stuartleeks this is a fresh deployment, then tre-start is running, but checks the resource group exists, but its not going to as hasn't been deployed yet. If the resource group doesn't exist we shouldn't exit, just not try to start?

Reopening as still an issue.
I worked round this by changing the control_tre.sh script:
if [[ $(az group list --output json --query "[?name=='rg-${TRE_ID}'] | length(@)") == 0 ]]; then
  echo "TRE resource group doesn't exist. Exiting..."
else
  # other start stuff.

@marrobi  - what is changed in the script snippet you gave compared to the latest code ?

Removed the exit 0, and sandwiched the start commands between the else in my snippet and fi.

Doesn't that effectively do the same thin in both cases?
In the code in main, it detects that the resource group doesn't exist, outputs the message and then exits (with 0 to indicate success)
In the code you pasted above, it detects that the resource group doesn't exist, outputs the message, and then skips the remaining commands because you pushed them into the else block.
The original issue I was facing was that the deployment was failing with this error. For me, this was because this block at the top wasn't catching the condition, so later steps failed because the resource group didn't exist.

Exit 0 seems to exit completely, not just the control script, but the following commands in the make file, such as deploy do not run.

I'm just re-testing this with fresh environment and make all

Closing - can re-open if we can reproduce.


[BUG] minimal base is not used

Expected Behavior
minimal base image with only the component we are building and/or supplied in runtime
Actual Behavior
i expected a minimal base image but looking at the /usr/bin contents, there are a lot of files
https://oci.dag.dev/layers/sozercan/oras@sha256:0137bd96038e5a7edd4c3114e45997b27dcaed83b906230a3943b1c161b76d8b/usr/bin/
Steps To Reproduce
docker build -t oras:latest -f
https://gist.githubusercontent.com/sozercan/1f856f629ba85b9a223dfff1f5cb5f2a/raw/794e8ebf0fb0c9d5581050a51e8a70156c901340/oras.yaml
--load . --target mariner2/container --progress plain

Are you willing to submit PRs to contribute to this bug fix?

[ ] Yes, I am willing to implement it.

This bug was introduced with PR #254 . Specifically, it's this commit https://github.com/adamperlin/dalec/commit/c91c9bae5812aacf34c8e2aedbef118e047658c3, although I can't yet fathom why

What is happening is that the presence of the %post, %preun, or %postun causes /bin/sh to be baked into the dependencies of the rpm. This makes sense because a shell is needed to execute the postinstall scripts, and would be needed to run pre- or post- uninstall scripts.
without %post:
$ rpm -q --requires /tmp/out/RPMS/x86_64/oras-v1.2.0-1.cm2.x86_64.rpm
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(FileDigests) <= 4.6.0-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(PayloadIsZstd) <= 5.4.18-1

_with %post_:
$ rpm -q --requires /tmp/out/RPMS/x86_64/oras-v1.2.0-1.cm2.x86_64.rpm
/bin/sh
/bin/sh
/bin/sh
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(FileDigests) <= 4.6.0-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(PayloadIsZstd) <= 5.4.18-1

The bash package supplies /bin/sh, and all of its dependencies are installed into the container as well. So the distroless minimal image is used, but it has a bunch of extra stuff installed.

The short-term solution is to not emit %post, %preun, or %postun when no postinstall scripts or systemd services are specified.
The longer-term solution involves

installing the rpm without running postinstall scripts
documenting that systemd units and rpm scripts are noop when the output is a container
printing a warning in the logs when the output is a container and rpm scripts are present

I am not sure if tdnf allows for the installation of a package without running postinstall scripts. If not, we should consider contributing that upstream while creating a workaround within dalec.

@adamperlin is this issue fixed? do we just need to add the tests? if so, i can close this issue and we'll have #303 tracking tests

@adamperlin is this issue fixed? do we just need to add the tests? if so, i can close this issue and we'll have #303 tracking tests

Yes this is fixed now due to #299! I think the fix in #299 is what we'll go with for the time being.

Ok closing this one. We'll continue in #303

This doesn't seem to be fixed
https://oci.dag.dev/layers/mcr.microsoft.com/oss/v2/open-policy-agent/gatekeeper@sha256:18279df0b2f3659befb24b632c00aaea5e71a3048c93eef3be2c19374a8d98e1/usr/bin/
https://oci.dag.dev/layers/mcr.microsoft.com/oss/v2/open-policy-agent/gatekeeper@sha256:18279df0b2f3659befb24b632c00aaea5e71a3048c93eef3be2c19374a8d98e1/var/lib/rpmmanifest/container-manifest-1

For posterity, we discussed this:
It looks like gatekeeper has a dependency on ca-certificates which brings in coreutils and bash.
azlinux/mariner have a package for just the root CA's prebuilt-ca-certificates which can be used instead.
Also opened a PR (#424) to include this by default in container images along with some other base things defined by the azlinux team.


Address RATS Report

Hi @Omkar20895 the report below is from a release auditing tool which I ran the source code over. If you have some time can you please take a look at some of the issues so we can discuss them?
Thanks
Entries in perl database: 33
Entries in ruby database: 46
Entries in python database: 62
Entries in c database: 334
Entries in php database: 55

Analyzing 20160804.090751.29944/podaacpy-master/podaac/podaac.py
Analyzing 20160804.090751.29944/podaacpy-master/podaac/podaac_data_source.py
Analyzing 20160804.090751.29944/podaacpy-master/podaac/podaac_utils.py
Analyzing 20160804.090751.29944/podaacpy-master/podaac/__init__.py
Analyzing 20160804.090751.29944/podaacpy-master/podaac/mcc.py
Analyzing 20160804.090751.29944/podaacpy-master/podaac/tests/mcc_test.py
Analyzing 20160804.090751.29944/podaacpy-master/podaac/tests/__init__.py
Analyzing 20160804.090751.29944/podaacpy-master/podaac/tests/podaac_test.py
Analyzing 20160804.090751.29944/podaacpy-master/docs/source/conf.py
Analyzing 20160804.090751.29944/podaacpy-master/setup.py
RATS results.

Severity: Medium
Issue: remove
A function call is not being made here, but a reference is being made to a name that is normally a vulnerable function. It could be being assigned as a pointer to function.
File: 20160804.090751.29944/podaacpy-master/podaac/podaac_utils.py
Lines: 39 60 141 163 185 207
File: 20160804.090751.29944/podaacpy-master/podaac/tests/podaac_test.py
Lines: 116 118 136
Severity: Medium
Issue: open
A function call is not being made here, but a reference is being made to a name that is normally a vulnerable function. It could be being assigned as a pointer to function.
File: 20160804.090751.29944/podaacpy-master/podaac/mcc.py
Lines: 87
File: 20160804.090751.29944/podaacpy-master/setup.py
Lines: 55 62
Severity: Medium
Issue: read
A function call is not being made here, but a reference is being made to a name that is normally a vulnerable function. It could be being assigned as a pointer to function.
File: 20160804.090751.29944/podaacpy-master/setup.py
Lines: 55
If the above area is blank, please see the RATSweb Processing Notes
Inputs detected at the following points

20160804.090751.29944/podaacpy-master/setup.py: Line 55: function read

Double check to be sure that all input accepted from an external data source does not exceed the limits of the variable being used to hold it. Also make sure that the input cannot be used in such a manner as to alter your program's behaviour in an undesirable way.

Total lines analyzed: 1861
Total time 0.008923 seconds
208562 lines per second

@lewismc shall we discuss this??

Based on the fact that the API is essentially changing, I am closing this off.


⚠ Site ANTS has degraded performance

In 339dece, Site ANTS (https://ants.gouv.fr) experienced degraded performance:

HTTP code: 200
Response time: 4441 ms

Resolved: Site ANTS performance has improved in 7578a08.


Test Pull request for Release

Please test over:
SKL/BXT initially
next cover
BDW/BSW after completing SKL/BXT
Thanks,
Sean

@jayesht


Automated cherry pick of #2222: fix: when turning off non-default-domain-projects, network public scope should swtich to system scope

Cherry pick of #2222 on release/2.10.0.
#2222: fix: when turning off non-default-domain-projects, network public scope should swtich to system scope

Codecov Report

:exclamation: No coverage uploaded for pull request base (release/2.10.0@e78fa8e). Click here to learn what that means.
The diff coverage is 0%.

@@               Coverage Diff                @@
##             release/2.10.0   #2223   +/-   ##
================================================
  Coverage                  ?   7.54%
================================================
  Files                     ?     410
  Lines                     ?   63648
  Branches                  ?       0
================================================
  Hits                      ?    4803
  Misses                    ?   58434
  Partials                  ?     411

Flag
Coverage Δ

#aFlag
7.54% <0%> (?)

Impacted Files
Coverage Δ

pkg/compute/models/networks.go
0.85% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e78fa8e...f1ffb29. Read the comment docs.

/lgtm
/approve


Bump BLE to 0.11.6

Fixes: https://github.com/AlCalzone/ioBroker.ble/issues/405

RE-CHECK!

Automated adapter checker
ioBroker.ble

:thumbsup: No errors found

[ ] :eyes: [W113] Adapter should support compact mode
[ ] :eyes: [W801] .npmignore not found

Add comment "RE-CHECK!" to start check anew


patch for issues #526.

Fixed an issue where the url_for method of iron-router does not work when using HTTPS listener.
The relevant problem is as follows.
https://github.com/iron/iron/issues/526
best regards

I mistook a branch. Close


How can I change the default font of rawdrawandroid?

Is it possible to change the font of rawdrawandoid to some of modern fonts like Poppins, Ubuntu, etc. If yes, how can i do it. The current font is awsome for games but for a minimal app, modern fonts like Poppins are more preferred, so please guide me to change it.

Afaik, the font is custom made and lives in CNFGFunctions.c i.e. it can not be easily replaced by a regular font.


Ability to generate a client with redirects enabled

As of https://github.com/Azure/azure-sdk-for-net/pull/34645, redirects are disabled by default in Azure Core. There are some TestServer tests that needed to be ignored in https://github.com/Azure/autorest.csharp/pull/3553/ in order to upgrade to Azure.Core 1.33.0. Apart from evaluating whether the tests should be removed or updated in some way, we should evaluate whether we need the ability to generate a protocol client with redirects enabled. Currently, the only libraries that need redirects are enabling it through the customization layer.

duplicate with https://github.com/Azure/autorest.csharp/issues/3550

It's a duplicate of #3550 and it's fixed by #3570


WIP:Initial list of authors

This list was auto-generated from our database of everyone who agreed to the license change.
It is WIP until we close out the search.

Hmmm, are you sure you want email addresses in this file?

Not positive, that's one of the reasons why it's marked as WIP.
But look at https://chromium.googlesource.com/chromium/src.git/+/master/AUTHORS and https://golang.org/AUTHORS  Google is an excellent source of best practices.  See also https://hg.mozilla.org/mozilla-central/file/tip/AUTHORS

@paulidale makes a good point, the file needs to be edited for duplicates

No, @paulidale is wrong, for a couple of reasons.  First, people have different employers; the work I did at Zolera, IBM, DataPower for example, is covered by different agreements. Second, how do you come up with a single address for someone? Third, how can you be sure that Brian Smith is the same Brian Smith everywhere?

First, people have different employers; the work I did at Zolera, IBM, DataPower for example, is covered by different agreements.

So you're seeing your authorships as different roles depending on what domain was on your email address?  You and I don't see it the same way then (but then, I have stubbornly refused to give away my day to day OpenSSL work to any company I've worked for, the project itself being the obvious exception)...  also, if you look at the moral rights (something that I believe is seen as more important in my part of the world, but I might be wrong), they always belong with the person regardless of working place, not with the role Name @ Company.

Second, how do you come up with a single address for someone?

I don't, but this is one reason I asked you earlier if you actually wanted addresses in this file.
However, I could very well imagine having a list of addresses following my name:
Richard Levitte<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
At a minimum, I'd like not to be spread out all over the file.

Third, how can you be sure that Brian Smith is the same Brian Smith everywhere?

I can't.  So that might be a reason to keep email addresses around.  In cases of doubt, let's try asking...  (I already gave you the answer re me)

I've done enough asking to get permission for the license change.  I am not going to go through the author list and ask about merging possible duplicates.  Feel free.
The EU moral right doesn't apply to the the US and probably other places, and it is far from clear that it applies to employees of some corporations.
There is nothing wrong with keeping the list with possible duplicates, and there is the risk of getting things wrong.

Ok.  I'm asking you to at least group my diverse identities together.  Doesn't have to be all on one line of you're opposed to it, but it would be good, for clarity, if they were all prefixed with my name.

Post a diff.
I really really REALLY do not want to do work on this.  I don't want us to take on handling every special case.  It's an automated list, at least for now, generated by commit logs.  Will someone create a post-commit trigger for future authors?  If not, then this has to stay automated.

Then how will this file be maintained if not automatically?  The only alternative I can think of is a commit-hook (or whatever) that compares authors vs the AUTHORS file.  Will you write that?  Until we have a mechanism for making sure that we don't omit anyone, the script stands as-is.  So sorry, @levitte and sorry @paulidale no customizations, it's too risky to get wrong.
As I said when I created this PR.  It' WIP not because I expect edits, but because the final list of AUTHOR's isn't done yet.
Again, if someone automates this, then my concerns go away.

In the FLOSS projects I know, it's a voluntary and author drivven thing, i.e. someone asks to be added.
Google expresses it like this:

It is not necessary to proactively list the copyright holder for every single contribution you receive. Rather, it is sufficient to add copyright holders when they request to be added.

Perhaps:

Parse the existing file into (name, email) pairs.
Generate (name, email) pairs automatically.
Merge the two lists by email address. If an email address appears in both lists and both entries have a name, use the name in the file, to allow for manual fixes.

That should still make sure entries aren't omitted while allowing for cases where names need to be fixed up. (Or perhaps if we later discover that some entry should be there but isn't captured by the script for whatever reason.)

Putting a place holder -1 on this so it must go to an OMC vote. The whole strategy here isn't one I think makes sense.

Well f---.  This was discussed a long time and months ago.  Fine.

Based on https://opensource.google.com/docs/releasing/authors/, I have changed this to be a corporate list of copyright holders and added a comment.  Taking this out of WIP.

Would it make sense to separate corporations and people into a list each (in the same file, but still)?
I'm not saying it has to happen in this PR, just asking the question.

Read the link and see what you think.  Google separates contributors from copyright-holding authors, which makes sense to me.

Of COURSE I am sure.  :)  I mean, I just ... double-checked... and, unh, oops.

Read the link and see what you think. Google separates contributors from copyright-holding authors, which makes sense to me.

I read the link, and the NOTE in there indicates that Google separateD (past tense) them into two files, but do not any longer.
Contributors are also copyright-holding (your way of expressing it might be interpreted differently)

Wouldn't we want the list of corporate copyright holders to be in a file called NOTICE?  That way when we move to the Apache License it requires derivatives to preserve those statements.   http://www.apache.org/dev/licensing-howto.html#mod-notice

I thought Apache's NOTICE file was for required third-party copyrights that carry a separate license or requirements and we don't have that.  And @levitte sure, we can just add individuals to the AUTHORS file when requested to.  Either this PR or a separate one.

Fixed all of them, thank you!

Added Swedish Institute.  (All these will be squashed once merged)  Thanks!

"Add Royalty"  :-D :-D

For this pull request it would probably be appropriate if it would be approved by another @openssl/omc member, right?

Yes, we'll talk about this out our F2F.

Will open a new PR at some point, closing this.


Add links to forum issues for typography and colours

Will resolve #228

  Typography and colours style pages need to reference forum issues

@jrbarnes9 good spot... I'll fix that now.


Error when starting backend.js

This error just happened now, should I reinstall yt2009?

Gonna reinstall.

Reinstalling fixed it


Update MD file on new branch 2017-11-09_15-24-05

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

split/updateTestRef1.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


I2C connectors.

I'm submitting a ...

[ ] bug report

Change I2C connectors from jumper wire header pins to JST connectors as currently on the avionics board. Here
The pitch for the I2C connector is 1mm.
Jumper wire connections aren't firm enough.

The JST connectors we have available are compatible with 2.54mm header pins
@WashingtonKK Confirm on this

I'll stick with the header pins because:

We already have the JST connectors here so we will just swap them if necessary
The JST connectors we have it is tiresome to wire the female connections as it requires crimping that is why we changed from JST to header pins during the test flight
They are quite expensive compared to the normal header pins

Regarding the comment above:

You seem to still have a lot of space on the bottom side.

We may have already been unable to acquire the data due to loose jumper wire connections. So it may already be too late to swap them.

-We can get already crimped cables.

I don't think the difference in price is too much to ensure we don't loose data.

Actually our main problem is acquiring JSTs with different pitch sizes.

I have a lot of space at the bottom since JLCPCB will be assembling only one layer which is the top layer.


NestingDepth takes into consideration BEM selectors

Hi all,
With the current version, is it possible for the NestingDepth to consider BEM selectors as nesting depth 1? If not, could this be added?
Current config:
 ~ scss-lint -v
scss-lint 0.49.0

NestingDepth:
  enabled: true
  max_depth: 1

SCSS:
.block {
  color: red;

    &__element {
      color: blue;
    }
}

Expected result:
In this case I would expect the NestingDepth to be 1, since this generates 2 CSS selectors that don't have any selectors within.
.block {color: red;}
.block__element {color: blue;}

Actual result:
NestingDepth: Nesting should be no greater than 1, but was 2
Note: For linting BEM selectors we already have BemDepth.
Thanks

I have the same issue, @cdreve .
SCSS:
.block {
   ...
  &__element {
    ...
    &--modifier {
      ...
      &::before {
        ...
      }
    }
}

Result
  .block {}
  .block__element {}
  .block__element-modifier {}
  .block__element-modifier::before {}

Warning
NestingDepth: Nesting should be no greater than 3, but was 4

Hey all,
NestingDepth is working as designed here. Its intention is to notify you when you are nesting (i.e. indenting) more than a given maximum.
Having deeply-nested/indented code is a smell once you start working with larger amounts of code within the nested blocks. You won't be able to easily tell what rule you are nested within without scrolling back up:
.block {
  &__element {
    ... 200 lines ...
    &--modifier {
      &::before {
        ...what element is this modifier applying to again?
      }
    }
  }
}

My recommendation if you don't find this linter useful is to disable it in this case. I'm of course open to dialog on this, but even then a pull request would speak louder than words. Thanks!


Export as .strm files tree

Is your feature request related to a problem? Please describe.
Filter part of an m3u and export to .strm files.
Describe the solution you'd like
Instead of exporting to other(s) m3u file(s), create a folder per Group and a file per Title. The file contain the media url. Having the ability to map usual Group/Name/Title to file path pattern (and using regex) would be a big plus.
Describe alternatives you've considered
Found those projects, didn't tested them yet:

https://github.com/ggiraudon/m3u2strm
https://github.com/trix7777/m2strm
https://github.com/erdesigns-eu/M3U2STRM
https://github.com/silence48/m3u2strm

Additional context
This well explain it https://forums.nextpvr.com/showthread.php?tid=62393.
strm details
https://kodi.wiki/view/Internet_video_and_audio_streams

These are basic text files that look like .strm and contain a URL to the internet stream.

We can solve this with target types. Depending on which type is specified, the result will be output accordingly.
Depending on the type, filename should then be a directory or an actual file.

That would be awesome! What would the conf look like?

i think something like this:
targets:
- type: m3u
filename: playlist_1.m3u
filter: (Group ~ "^DE\s." AND Title ~ "^DE\s.")
options: { ignore_logo: true }
rename:
- { field: Group, new_name: 1. DE$1, pattern: ^DE(.) }
sort: { order: Asc }
- type: strm
filename: strm_files
filter: (Group ~ "^DE\s.")
options: { ignore_logo: true }
rename:
- { field: Group, new_name: 1_DE$1, pattern: ^DE(.*) }
btw: i am working on your other request, the filter is now a  string  like:
(Field ~ regexp) AND (Field ~ Regexp) OR (NOT Field ~ Regexp)

(Field ~ regexp) AND (Field ~ Regexp) OR (NOT Field ~ Regexp)
Neat!
Would like you a dedicated issue for the second part in https://github.com/euzu/m3u-filter/issues/1#issuecomment-1085017112?

There are two types of targets m3u and strm. This can be set by the output attribute to Strm or M3u.
If the attribute is not specified M3u is created by default.
Strm output has an additional option underscore_whitespace. This replaces all whitespaces with _ in the path.
config looks like:
api: { host: <IP_ADDRESS>, port: 8901, web_root: /home/m3u-filter/web }
working_dir: /home/m3u-filter/data
sources:
  - input: { persist: 'playlist_1_{}.m3u', url: 'http://myserver.net/playlist.m3u' }
    targets:
      - filename: playlist_strm
        type: Strm
        filter: 'Group ~ "^DE\s.*" OR Group ~ "TR\s.*"'
        options: { ignore_logo: true, underscore_whitespace: true }
        rename:
          - { field: Group, new_name: 1. DE$1, pattern: ^DE(.*) }
        sort: { order: Asc }

If desired, I could add another option cleanup: true|false that deletes the old directory first for strm files.

Seem perfect!! You rocks!
Will try it this evening.
Effectively, a cleanup would make sense

I'm not able to have the cleanup working. I'm using 0.9.2.
Part of my conf:
      - filename: vod/vod/series/
        type: Strm
        cleanup: true
        filter: 'Group ~ "^\|FR\|\s+" AND Url ~ "/series/"'
        options: {ignore_logo: true}
        rename:
          - {field: Group, new_name: $1, pattern: ^.*\|\s(.*)(\s\(.*)?}
          - {field: Title, new_name: $1, pattern: ^\s?(.*)}
        sort: {order: Asc}

I have some files in vod/vod/series/ULTRA\ HD\ 4K/ that remain after update.
BTW, --version report the wrong version
$ ./m3u-filter_v0.9.2_linux_x86_64/m3u-filter --version
m3u-filter 0.5.0

cleanup belongs to the options section.
options:
      ignore_logo: true
      underscore_whitespace: true
      cleanup: true

Kodi scraper is failing with the current directory structure, according to this page: https://kodi.wiki/view/Naming_video_files/TV_shows

Each TV Show requires its own folder in the Source folder
The naming of your TV Show folder is critical to ensure the scrapers can correctly identify the TV Show and its episodes.
TV Show Name (Year) - This is the correct naming format for the TV Show folder
The TV Show folders are saved inside the Source folder
Adding the (Year) makes scraping more accurate and differentiates between original and remake versions.
For Country of Origin TV Shows, the (Year) is replaced with the country code. eg Hells Kitchen (UK), Hells Kitchen (US), Hells Kitchen (AU) etc.
The title and (year) displayed at the scraper site is how you should name your show.

Do you think of any trick to accomplish this? the rename option doesn't seem to help here :thinking:
Some random titles from my provider:
vod/series/ULTRA HD 4K/DCs Stargirl 2020 S02 E13.strm"
vod/series/ULTRA HD 4K/For All Mankind 2021 Saison 2  4K S02 E01.strm"

expected
vod/series/ULTRA HD 4K/DCs Stargirl (2020)/Season 2/DCs Stargirl (2020) S01E13.strm"
vod/series/ULTRA HD 4K/For All Mankind (2021)/Season 2/For All Mankind (2021) S02E01.strm"

I need to think about this problem. I can't think of a solution in a hurry.

No hurry :) Thanks a lots for all of this. You did a great job :raised_hands:

I have added a new option kodi_style.  Could you please check if it works fro you?

Thanks a lot, now Kodi seems happy with it :+1:


Filter by ru

Motivation and Context
There is a need to be able to filter by ru_ref in secure message screens
What has changed
Passed around both ru_ref and business_id, because secure message uses business id and the ui mandated ru_ref. If only ru ref was passed around it would mean an extra call to party on each tab change. A new compoenent for ru entry was added in the design system . And the extra params added to Secure message calls. As this was being written Secure message was also changing to replace ru_ref with business_id which also changed some of the returned data , this was integrated also .
Several acceptance tests deleted , and replaced with new unit tests in this repo.
How to test?
run tests
Run acceptance tests . Then modify the db to have all the  same survey_id  by modifying the db locally . Then enter an ru ref in the screen and verify that the filter works as expected.Also verify all secure message functionality manually as this has touched every secure message endpoint .
Links
https://trello.com/c/Zcr7STCe

Just 2 things to mention, not problems but things to consider:

If you put in a partially completed ru_ref into the url and press enter then you get the result of a not found without any of the errors
The url for the secure message inbox should be cleaner when you're not doing anything.  When you just view a single survey there are 3 params and the url is starting to look like a bit of a dumping ground.  Maybe it'd be worth only including those params if they're filled and leaving them empty otherwise.  I know it's only a url but they should be clean where possible I feel


"warning: already initialized constant" with rack gem

Minimal reproduction: repro.zip
If you run bazel run :main in this workspace, you will get output which includes the following:
INFO: Build options --cxxopt, --incompatible_strict_action_env, and --legacy_external_runfiles have changed, discarding analysis cache.
DEBUG: /private/var/tmp/_bazel_johnfirebaugh/1c33c885364b1c1481b2fccb2257d6ac/external/bazelruby_rules_ruby/ruby/private/toolchains/ruby_runtime.bzl:99:14: Found local Ruby SDK version '2.6.6' which matches requested version '2.6.6'
INFO: Analyzed target //:main (0 packages loaded, 283 targets configured).
INFO: Found 1 target...
Target //:main up-to-date:
  bazel-bin/main
INFO: Elapsed time: 0.244s, Critical Path: 0.03s
INFO: 3 processes: 3 internal.
INFO: Build completed successfully, 3 total actions
INFO: Build completed successfully, 3 total actions
/private/var/tmp/_bazel_johnfirebaugh/1c33c885364b1c1481b2fccb2257d6ac/external/bundle/lib/ruby/2.6.0/gems/rack-2.2.3/lib/rack.rb:17: warning: already initialized constant Rack::HTTP_HOST
/private/var/tmp/_bazel_johnfirebaugh/1c33c885364b1c1481b2fccb2257d6ac/execroot/__main__/bazel-out/darwin-fastbuild/bin/main.runfiles/bundle/lib/ruby/2.6.0/gems/rack-2.2.3/lib/rack.rb:17: warning: previous definition of HTTP_HOST was here
/private/var/tmp/_bazel_johnfirebaugh/1c33c885364b1c1481b2fccb2257d6ac/external/bundle/lib/ruby/2.6.0/gems/rack-2.2.3/lib/rack.rb:18: warning: already initialized constant Rack::HTTP_PORT
/private/var/tmp/_bazel_johnfirebaugh/1c33c885364b1c1481b2fccb2257d6ac/execroot/__main__/bazel-out/darwin-fastbuild/bin/main.runfiles/bundle/lib/ruby/2.6.0/gems/rack-2.2.3/lib/rack.rb:18: warning: previous definition of HTTP_PORT was here
[snip many more warnings of the same form]

I debugged a bit by placing a puts caller inside rack.rb. It gets required via two call stacks:

First directly via main.rb:1, resolved as <outputBase>/execroot/__main__/bazel-out/darwin-fastbuild/bin/main.runfiles/bundle/lib/ruby/2.6.0/gems/rack-2.2.3/lib/rack.rb
Second via a require_relative in rack/session/abstract/id.rb, resolved as <outputBase>/external/bundle/lib/ruby/2.6.0/gems/rack-2.2.3/lib/rack.rb:20

It appears that require_relative is resolving the require relative to the resolved symlink location in bazel's external directory, rather than the runfiles location where the direct require was resolved to.

I found this upstream issue with symlinked directories, which was fixed, but it seems Ruby still has issues when require and require_relative are mixed with symlinked files.
mkdir a
echo "require_relative 'b'" > a/a.rb
echo "p 'b loaded'" > a/b.rb
mkdir b
ln -s ../a/a.rb b
ln -s ../a/b.rb b
echo "$: << File.expand_path('../b', __FILE__); require 'b'; require 'a'" > c.rb
ruby c.rb

This results in "b loaded" being printed twice.

I opened a new upstream issue: https://bugs.ruby-lang.org/issues/17885.

Great thanks for reporting it.
I agree this is a Ruby issue, not the rules. In general I've been avoiding require_relative in my open source gems because of similar issues with resolution caching.
Does it make sense to keep this open, or can we close this issue?

Do you think there's any possibility of working around the issue within rules_ruby? As it stands, at least one rather essential gem in the Ruby ecosystem uses require_relative extensively (rack), and there doesn't seem to be much momentum around getting the upstream issue resolved. It seems like the status quo would limit the adoption of rules_ruby and it might be worth avoiding filesystem structures that trigger the issue. (I ask this not knowing much about the internal constraints of rules_ruby, so I'm also prepared to accept that it isn't feasible or would be too much work.)

So the result is that it breaks anything that relies on rack?
Any chance you could submit a new example under the examples/rack that would fail as you describe?

Looking for additional core maintainers: https://github.com/bazelruby/rules_ruby/discussions/146


Handful of new rules

I recently worked my way through an old stealer/grabber (5a2f620f29ca2f44fc22df67b674198f) with the intent to build out capa-rules.
While doing this exercise, I found that many of the rules are subjectively "low quality" relying on pairings of strings.
I wanted to share these rules, in case they may be useful.  I didn't roll these up into a giant PR until they received some peer review or qualitative analysis.  Capa's scope (in this case, all function level) results in the rules not being too prone to FP's,
Let me know if these rules would be a welcome addition to the capa-rules main repository, and if not, I'll keep a fork with supplemental rules.   If a PR is decided, the rules have been linted with --thorough and capafmt has been applied.
https://github.com/re-fox/capa-rules/tree/master/collection/file-managers

Wow, this is a great collection!
I understand your concern about these being pairings of strings, though thanks to the scope and length of the strings, suspect they'd still be useful. It also seems like these might be fairly rarely seen - but I'm not convinced that means we should not write rules for interesting capabilities.
Therefore, I'd vote for opening a PR and merging these.

Thanks for the help and input!  Closing the issue.


Missing connect implementation

While described in the documents, the Trove.connect method is missing.

Thanks! Added in v1.2.0


feat: Dockerize graphql and cat-data-service

Adds dockerfiles to deploy PostGraphile for EventDB, and cat-data-service.

Currently, this branch includes 3fa819eb4fd3905fa794c3a07a7b8534e67696e1 from a separate branch to get things working.

Pause this until my .sql file changes have been merged.

This was replaced with https://github.com/input-output-hk/catalyst-core/pull/333


Examples not working:

Hello,
I  have been trying this software with the examples given. So far,   only a few things works.
Although  I  get the following mssage from the example below,   it did not the second line.
swarms is up to date!
from swarms.workers import Worker
from swarms.agents.meta_prompter import MetaPrompterAgent
from swarms.models import OpenAI
init llm
llm = OpenAI()
init the meta prompter agent that optimized prompts
meta_optimizer = MetaPrompterAgent(llm=llm)
init the worker agent
worker = Worker(llm)
broad task to complete
task = "Create a feedforward in pytorch"
optimize the prompt
optimized_prompt = meta_optimizer.run(task)
run the optimized prompt with detailed instructions
result = worker.run(optimized_prompt)
print
print(result)

Upvote & Fund

We're using Polar.sh so you can upvote and help fund this issue.
We receive the funding once the issue is completed & confirmed by you.
Thank you in advance for helping prioritize & fund our backlog.

I'm sorry, but this example doesn't exits anymore


Dockerize

[ ] 2-stage Dockerfile

Stage 1 - build the app using JDK as a base image
Stage 2 - copy built JAR on top of JRE base image

[ ] application-dev.yaml with config values for the dev profile
[ ] Bash script to build an image locally and push it to Docker Hub - build-and-push.sh. Image name - itjobmarketanalytics/rate-limiter:dev
[ ] Push to Docker Hub

https://github.com/IT-job-market-analytics/vacancy-import-service/tree/dev


⚠ Bofrost.be has degraded performance

In ca3c21a, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1776 ms

Resolved: Bofrost.be performance has improved in e74096c.


Current structure does not allow for buttons to be positioned outside of carousel

Current structure nests the arrows within the track/mask. If they were pulled to outside of this (similar to dots) it would allow for the next and back buttons to be positioned as required per design.

Can I see an example design?  I think the reason I put them inside the mask is that they could be vertically centered relative to it.

The example I'm working on doesn't have a design created by our design team, but really I feel like any form of text-based carousel wouldn't want the arrows hovering over top of the component. My simple work around for the time being is just make sure the content has padding within each slide.

Ugly example with too much text and my simple solution:

w/o padding:

I think padding is a great solve ... not really sure how moving out of mask would help here anyhow.  What did you have in mind, UI wise, if it was moved out of the mask element?


_PickleCore._get_cache_by_key() returns dict but is used as a dataclass

This library is required by copernicusmarine, and recently one of my data pipelines raised an exception with the following message:
dict has no attribute "._completed"

Unfortunaly I couldn't get the full exception, so I had to trace all references to the ._completed attribute, and found this python module.
While checking the code, I found that the PickleCore get_entry_by_key() internally calls _get_cache_by_key(), expecting a return type of  CacheEntry. But the _get_cache_by_key() method has a return type of Dict[str, CacheEntry]]
I would open a PR, but I couldn't understand what is the expected return type of _get_cache_by_key().
This bug seems to be triggered whenever cachier() is called with the pickle implementation and with the separate_files set as True.

@Borda, this seems related to the new types you defined.

just checking since we have tests for this but all CI is 🟢
so seems we have issue with testing, I see error treated as warning
https://github.com/python-cachier/cachier/actions/runs/11446891113/job/31847045677#step:5:219

ok, seems I have another example here: https://github.com/Lightning-AI/torchmetrics/actions/runs/11454171728/job/31867880744?pr=2797

After some examination, from my understanding:
PickleCore.get_entry_by_key is supposed to return an entry (and its own key).
If reload is set to True, the method calls _get_cache(), which returns a cache (dict), and then calls get to get an entry.
But, if separate_files is set to True, then it just calls get_cache_by_key, which returns a full cache, not just a cache entry.
From my understanding, as both the cache and cache entry were dicts before, whichever function called get_entry_by_key would attempt to access some field thinking it was a cache entry, fail as it is a full cache, and handle it as a cache miss. This could be why the tests weren't triggered before (although some tests were failing as warnings #247)

From my understanding, as both the cache and cache entry were dicts before, whichever function called get_entry_by_key would attempt to access some field thinking it was a cache entry, fail as it is a full cache, and handle it as a cache miss. This could be why the tests weren't triggering before (although some tests were failing as warnings #247)

hmm think that with the refactor to entry we sufficed some hidden issues 🐰

I also encountered something similar here: https://github.com/mercator-ocean/copernicus-marine-toolbox/issues/183
This error only occurs with cachier 3.1.1, not with 3.0.1. So could it be that there is a bug introduced in that version?

found out that during tests, _PickleCore._get_cache_by_key does return a CacheEntry, and not a cache dict! So maybe this is the intended behaviour, and the type hint is just wrong

do you have minimal example to reproduce which we can add to tests?

ok thinking this could be a transition issue as your past cache is dict bt the new/expected is dataclass

@Leinadiumdo you have minimal example to reproduce which we can add to tests?

The tests are all fine. The tests expect that _get_cache_by_key returns a CacheEntry, and they succeded just fine. I modified the code to return a cache dict, and they started to fail. So the implementation is correct, just the type hint is wrong.

right, this could be a transition issue as your past cache is dict but the new/expected is dataclass

Other developers noticed the same problem here, and yes, caches are now incompatible between versions.

Other developers noticed the same problem here, and yes, caches are now incompatible between versions.

going to fix it now, just thinking about how to test it :/

I see so we shall consider adding some typing check like mypy

That might be a good idea! I opened the project in my local IDE, and some static and type checks starting popping (e. g. multiple "possible None being used" values). It is a good idea!

going to fix it now, just thinking about how to test it :/

The pickle object is being returned without any checks. You may add some checks to see if the object is a CacheEntry, and then for testing, you can create a bad/corrupt pickle file, and check for empty caches (and not failures).

The pickle object is being returned without any checks. You may add some checks to see if the object is a CacheEntry

that is fine and simple

and then for testing, you can create a bad/corrupt pickle file, and check for empty caches (and not failures).

I was rather thinking about running examples with several versions installed

@Leinadium could you test it also on your side with the open PR:
cachier @ https://github.com/python-cachier/cachier/archive/refs/heads/fix/legacy.zip

Fixed in release v3.1.2. Please reopen if that is not the case.


Travis CI also build on osx

Also run Travis build on osx to run all major platforms!

Thanks, but since most of the devs working on node-tap already use OS X as their main machines, it gets plenty of coverage in that OS, and this lib doesn't really have any linux- or bsd-specific bits, so all unixes are more or less the same.  I'd rather not make the travis build take longer, or rely on nvm.


Prevent duplicate method override errors

When a method overrides both a concrete implementation and implements an interface method, Sorbet was validating against both implementations, leading to duplicate and potentially confusing errors. This PR introduces mostSpecificImpl to avoid any unnecessary checks and redundant errors by determining exactly one method signature to validate against. For methods marked with override, mostSpecificImpl is set to the first non-abstract method found in the inheritance chain. For methods without override, mostSpecificImpl is set to the first abstract method found. For initialize methods, we validate against interface requirements regardless of mostSpecificImpl.
This could easily be configured to help with #6892 by having Sorbet say that you can’t override an existing method and make it abstract, but I think that it would be better to go with the second idea to make it more idiomatic. I can either update this PR to include the first option, or give the second one a go in a new PR.
Motivation
Fixes #6564: Duplicate error message in override checking
Test plan
See included automated tests. The existing resolver/interface_initialize test makes sure that initialize methods properly validate against interface requirements.

I think there's some more subtlety here that I don't really want to have to think about at the moment. In particular, I think that this behavior has implications for eventually solving #1317. Given that the current approach is a more cosmetic change (simply only comparing against the first things in the chain) not foundational, I'm worried that unless I think very carefully about the implications of this change, it'll have bad consequences given that other shortcoming of Sorbet.
Unless this is something particularly blocking to you, I'd love to be able to set this aside for the time being.

In all honesty, I haven't had any blockers with it. This PR was purely meant to hopefully address the user experience for anybody that might be annoyed by the current error implementation. I'll admit that my idea here does seem a bit like a hack solution to what is likely an issue on a more fundamental level, especially if you think that the behavior could be related to #1317.
I'm still learning my way around Sorbet's internals, and I'd be more than happy to continue poking around to see if I can find a better solution to this problem. Otherwise, I understand if you'd prefer to close this PR and revisit the issue when there's a clearer path forward.

Yeah maybe let's close (or mark it as draft if you prefer) for now. I think that next year we're going to get around to overhauling this part of Sorbet, and we can think hard about it then.


Oracle connector returns lowercased names of schemas and table columns

Affected module
OpenMetadata Oracle connector
Describe the bug
Dear all,
When using Oracle connector in OpenMetadata 1.2.3 found that schema and column names are all turned to lowercase. Could you kindly help us to resolve this problem? For further use we need all names as they are in the source. Thank you!
Expected behavior
All names taken from source must be in the same case, no lower- or uppercasing.
Version:

OpenMetadata version: 1.2.3

Hey @brown3007 can you upgarde to latest stable version and let us know if you are still experiencing the same issue.

Hi @TeddyCr I have upgraded to 1.3.1 and Oracle connector ingestion refuses to work at all
Testing connection is OK, but when trying to run ingestion task it fails in Airflow, and in OpenMetadata UI it is shown as the ingestion never started.
In 1.2.3 it works fine except for lowercasing column names, as described above.
Here are my logs from Airflow:
93b45bd3b40c
*** Found local files:
***   * /opt/airflow/logs/dag_id=ora_test_metadata/run_id=manual__2024-03-21T15:45:42+00:00/task_id=ingestion_task/attempt=1.log
[2024-03-21, 18:45:43 MSK] {taskinstance.py:1159} INFO - Dependencies all met for dep_context=non-requeueable deps ti=<TaskInstance: ora_test_metadata.ingestion_task manual__2024-03-21T15:45:42+00:00 [queued]>
[2024-03-21, 18:45:43 MSK] {taskinstance.py:1159} INFO - Dependencies all met for dep_context=requeueable deps ti=<TaskInstance: ora_test_metadata.ingestion_task manual__2024-03-21T15:45:42+00:00 [queued]>
[2024-03-21, 18:45:43 MSK] {taskinstance.py:1361} INFO - Starting attempt 1 of 1
[2024-03-21, 18:45:43 MSK] {taskinstance.py:1382} INFO - Executing <Task(CustomPythonOperator): ingestion_task> on 2024-03-21 15:45:42+00:00
[2024-03-21, 18:45:43 MSK] {standard_task_runner.py:57} INFO - Started process 901 to run task
[2024-03-21, 18:45:43 MSK] {standard_task_runner.py:84} INFO - Running: ['airflow', 'tasks', 'run', 'ora_test_metadata', 'ingestion_task', 'manual__2024-03-21T15:45:42+00:00', '--job-id', '9', '--raw', '--subdir', 'DAGS_FOLDER/ora_test_metadata.py', '--cfg-path', '/tmp/tmpk7_5idsr']
[2024-03-21, 18:45:43 MSK] {standard_task_runner.py:85} INFO - Job 9: Subtask ingestion_task
[2024-03-21, 18:45:44 MSK] {task_command.py:416} INFO - Running <TaskInstance: ora_test_metadata.ingestion_task manual__2024-03-21T15:45:42+00:00 [running]> on host ************
[2024-03-21, 18:45:44 MSK] {taskinstance.py:1662} INFO - Exporting env vars: AIRFLOW_CTX_DAG_OWNER='admin' AIRFLOW_CTX_DAG_ID='ora_test_metadata' AIRFLOW_CTX_TASK_ID='ingestion_task' AIRFLOW_CTX_EXECUTION_DATE='2024-03-21T15:45:42+00:00' AIRFLOW_CTX_TRY_NUMBER='1' AIRFLOW_CTX_DAG_RUN_ID='manual__2024-03-21T15:45:42+00:00'
[2024-03-21, 18:45:44 MSK] {server_mixin.py:66} INFO - OpenMetadata client running with Server version [1.3.1] and Client version [<IP_ADDRESS>]
[2024-03-21, 18:45:44 MSK] {connection.py:114} INFO - Initializing Oracle thick client at /instantclient
[2024-03-21, 18:45:45 MSK] {test_connections.py:198} WARNING - GetTables-(oracledb.exceptions.DatabaseError) ORA-00942: table or view does not exist
[SQL: /* {"app": "OpenMetadata", "version": "<IP_ADDRESS>"} */
SELECT table_name FROM DBA_TABLES WHERE
nvl(tablespace_name, 'no tablespace') NOT IN ('SYSTEM', 'SYSAUX') AND
OWNER = :owner
AND IOT_NAME IS NULL
AND DURATION IS NULL
AND TABLE_NAME NOT IN
(SELECT mview_name FROM DBA_MVIEWS WHERE owner = :owner)
]
[parameters: {'owner': 'READER'}]
(Background on this error at: https://sqlalche.me/e/14/4xp6)
[2024-03-21, 18:45:45 MSK] {test_connections.py:215} INFO - Test connection results:
[2024-03-21, 18:45:45 MSK] {test_connections.py:216} INFO - failed=['\'GetTables\': This is a mandatory step and we won\'t be able to extract necessary metadata. Failed due to: (oracledb.exceptions.DatabaseError) ORA-00942: table or view does not exist\n[SQL: /* {"app": "OpenMetadata", "version": "<IP_ADDRESS>"} */\n\nSELECT table_name FROM DBA_TABLES WHERE \nnvl(tablespace_name, \'no tablespace\') NOT IN (\'SYSTEM\', \'SYSAUX\') AND \nOWNER = :owner  \nAND IOT_NAME IS NULL \nAND DURATION IS NULL\nAND TABLE_NAME NOT IN \n(SELECT mview_name FROM DBA_MVIEWS WHERE owner = :owner)\n]\n[parameters: {\'owner\': \'READER\'}]\n(Background on this error at: https://sqlalche.me/e/14/4xp6)'] success=["'CheckAccess': Pass", "'GetSchemas': Pass", "'GetViews': Pass"] warning=[]
[2024-03-21, 18:45:45 MSK] {taskinstance.py:1937} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/operators/python.py", line 192, in execute
    return_value = self.execute_callable()
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/operators/python.py", line 209, in execute_callable
    return self.python_callable(*self.op_args, **self.op_kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/openmetadata_managed_apis/workflows/ingestion/common.py", line 206, in metadata_ingestion_workflow
    workflow = MetadataWorkflow.create(config)
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/ingestion.py", line 102, in create
    return cls(config)
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/ingestion.py", line 78, in __init__
    super().__init__(
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/base.py", line 106, in __init__
    self.post_init()
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/ingestion.py", line 97, in post_init
    self.set_steps()
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/metadata.py", line 35, in set_steps
    self.source = self._get_source()
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/metadata.py", line 60, in _get_source
    source: Source = source_class.create(self.config.source.dict(), self.metadata)
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/source/database/oracle/metadata.py", line 114, in create
    return cls(config, metadata)
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/source/database/common_db_source.py", line 109, in __init__
    self.test_connection()
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/source/database/database_service.py", line 583, in test_connection
    test_connection_fn(self.metadata, self.connection_obj, self.service_connection)
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/source/database/oracle/connection.py", line 139, in test_connection
    test_connection_db_common(
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/connections/test_connections.py", line 317, in test_connection_db_common
    test_connection_steps(
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/connections/test_connections.py", line 265, in test_connection_steps
    return timeout(timeout_seconds)(_test_connection_steps)(
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/utils/timeout.py", line 54, in inner
    result = fn(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/connections/test_connections.py", line 104, in _test_connection_steps
    _test_connection_steps_during_ingestion(steps=steps)
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/connections/test_connections.py", line 219, in _test_connection_steps_during_ingestion
    raise SourceConnectionException(
metadata.ingestion.connections.test_connections.SourceConnectionException: Some steps failed when testing the connection: [failed=['\'GetTables\': This is a mandatory step and we won\'t be able to extract necessary metadata. Failed due to: (oracledb.exceptions.DatabaseError) ORA-00942: table or view does not exist\n[SQL: /* {"app": "OpenMetadata", "version": "<IP_ADDRESS>"} */\n\nSELECT table_name FROM DBA_TABLES WHERE \nnvl(tablespace_name, \'no tablespace\') NOT IN (\'SYSTEM\', \'SYSAUX\') AND \nOWNER = :owner  \nAND IOT_NAME IS NULL \nAND DURATION IS NULL\nAND TABLE_NAME NOT IN \n(SELECT mview_name FROM DBA_MVIEWS WHERE owner = :owner)\n]\n[parameters: {\'owner\': \'READER\'}]\n(Background on this error at: https://sqlalche.me/e/14/4xp6)'] success=["'CheckAccess': Pass", "'GetSchemas': Pass", "'GetViews': Pass"] warning=[]]
[2024-03-21, 18:45:45 MSK] {taskinstance.py:1400} INFO - Marking task as FAILED. dag_id=ora_test_metadata, task_id=ingestion_task, execution_date=20240321T154542, start_date=20240321T154543, end_date=20240321T154545
[2024-03-21, 18:45:45 MSK] {common.py:283} INFO - Sending failed status from callback...
[2024-03-21, 18:45:45 MSK] {server_mixin.py:66} INFO - OpenMetadata client running with Server version [1.3.1] and Client version [<IP_ADDRESS>]
[2024-03-21, 18:45:45 MSK] {common.py:289} INFO - Sending status to Ingestion Pipeline tis_test.ora_test_metadata
[2024-03-21, 18:45:45 MSK] {taskinstance.py:1705} ERROR - Error when executing partial callback
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 1518, in _run_raw_task
    self._execute_task_with_callbacks(context, test_mode, session=session)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 1681, in _execute_task_with_callbacks
    result = self._execute_task(context, task_orig)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 1744, in _execute_task
    result = execute_callable(context=context, **execute_callable_kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/operators/python.py", line 192, in execute
    return_value = self.execute_callable()
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/operators/python.py", line 209, in execute_callable
    return self.python_callable(*self.op_args, **self.op_kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/openmetadata_managed_apis/workflows/ingestion/common.py", line 206, in metadata_ingestion_workflow
    workflow = MetadataWorkflow.create(config)
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/ingestion.py", line 102, in create
    return cls(config)
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/ingestion.py", line 78, in __init__
    super().__init__(
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/base.py", line 106, in __init__
    self.post_init()
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/ingestion.py", line 97, in post_init
    self.set_steps()
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/metadata.py", line 35, in set_steps
    self.source = self._get_source()
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/workflow/metadata.py", line 60, in _get_source
    source: Source = source_class.create(self.config.source.dict(), self.metadata)
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/source/database/oracle/metadata.py", line 114, in create
    return cls(config, metadata)
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/source/database/common_db_source.py", line 109, in __init__
    self.test_connection()
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/source/database/database_service.py", line 583, in test_connection
    test_connection_fn(self.metadata, self.connection_obj, self.service_connection)
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/source/database/oracle/connection.py", line 139, in test_connection
    test_connection_db_common(
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/connections/test_connections.py", line 317, in test_connection_db_common
    test_connection_steps(
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/connections/test_connections.py", line 265, in test_connection_steps
    return timeout(timeout_seconds)(_test_connection_steps)(
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/utils/timeout.py", line 54, in inner
    result = fn(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/connections/test_connections.py", line 104, in _test_connection_steps
    _test_connection_steps_during_ingestion(steps=steps)
  File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/connections/test_connections.py", line 219, in _test_connection_steps_during_ingestion
    raise SourceConnectionException(
metadata.ingestion.connections.test_connections.SourceConnectionException: Some steps failed when testing the connection: [failed=['\'GetTables\': This is a mandatory step and we won\'t be able to extract necessary metadata. Failed due to: (oracledb.exceptions.DatabaseError) ORA-00942: table or view does not exist\n[SQL: /* {"app": "OpenMetadata", "version": "<IP_ADDRESS>"} */\n\nSELECT table_name FROM DBA_TABLES WHERE \nnvl(tablespace_name, \'no tablespace\') NOT IN (\'SYSTEM\', \'SYSAUX\') AND \nOWNER = :owner  \nAND IOT_NAME IS NULL \nAND DURATION IS NULL\nAND TABLE_NAME NOT IN \n(SELECT mview_name FROM DBA_MVIEWS WHERE owner = :owner)\n]\n[parameters: {\'owner\': \'READER\'}]\n(Background on this error at: https://sqlalche.me/e/14/4xp6)'] success=["'CheckAccess': Pass", "'GetSchemas': Pass", "'GetViews': Pass"] warning=[]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 1702, in _run_finished_callback
    callback(context)
  File "/home/airflow/.local/lib/python3.10/site-packages/openmetadata_managed_apis/workflows/ingestion/common.py", line 296, in send_failed_status_callback
    pipeline_status.endDate = datetime.now().timestamp() * 1000
AttributeError: 'NoneType' object has no attribute 'endDate'
[2024-03-21, 18:45:45 MSK] {standard_task_runner.py:104} ERROR - Failed to execute job 9 for task ingestion_task (Some steps failed when testing the connection: [failed=['\'GetTables\': This is a mandatory step and we won\'t be able to extract necessary metadata. Failed due to: (oracledb.exceptions.DatabaseError) ORA-00942: table or view does not exist\n[SQL: /* {"app": "OpenMetadata", "version": "<IP_ADDRESS>"} */\n\nSELECT table_name FROM DBA_TABLES WHERE \nnvl(tablespace_name, \'no tablespace\') NOT IN (\'SYSTEM\', \'SYSAUX\') AND \nOWNER = :owner  \nAND IOT_NAME IS NULL \nAND DURATION IS NULL\nAND TABLE_NAME NOT IN \n(SELECT mview_name FROM DBA_MVIEWS WHERE owner = :owner)\n]\n[parameters: {\'owner\': \'READER\'}]\n(Background on this error at: https://sqlalche.me/e/14/4xp6)'] success=["'CheckAccess': Pass", "'GetSchemas': Pass", "'GetViews': Pass"] warning=[]]; 901)
[2024-03-21, 18:45:45 MSK] {local_task_job_runner.py:228} INFO - Task exited with return code 1
[2024-03-21, 18:45:45 MSK] {taskinstance.py:2778} INFO - 0 downstream tasks scheduled from follow-on schedule check

@brown3007 it seems like a permission issue maybe, if you can give permissions to the user for DBA_* views,
maybe something like a grant select any dictionary to john; something like this and try it testing it out again?

@ayush-shah connection options are the same in 1.3.1 and in 1.2.3, but in 1.2.3 it works and I can see tables in OpenMetadata, but in 1.3.1 it doesn't
Were there any changes for Oracle connector between 1.2.3 and 1.3.1? If so, it can be permission issue, but if connector stays the same the problem isn't on the Oracle side

@brown3007 we have made query changes where previously it was all_, we migrated it to dba_
Here is the history of commits that were changed in couple of months: https://github.com/open-metadata/OpenMetadata/commits/main/ingestion/src/metadata/ingestion/source/database/oracle/queries.py

Closing this, as it doesn't break anything..we will reopen if there seems be better suggestions to handle complex migrations.

@ayush-shah I am currently using version 1.4.4 of OpenMetadata, but the issue with Oracle field name case sensitivity still exists.
Will this fix be merged into the new version in the future?

I hope this issue can be reopen for the correctness of metadata.


Consider adding a viewport abstraction

This is related to #197 but didn't want to derail the original bug
Bevy has a window abstraction, basically HashMap<WindowId, Window>. I think we may want a viewport abstraction as well. The default behavior would be that a window has a single viewport that fills it completely. I could see this being useful for an editor where we might want to render two views of the same scene, or render a single view within just part of the window. (Likely with UI surrounding it) (Draggable UI panels might have their own viewports for example)
I would suggest setting it up as a separate resource that looks a lot like the Windows resource, but where each viewport has a WindowId indicating which window contains it. (HashMap<ViewportId, Viewport>, Viewport is a struct that contains a WindowId)
This could have some camera state to help with converting between 2D/3D (or just reference a camera that has that information).

I'm trying to implement something like this in #1389 to scratch my own itch (windowless cameras). Feedback is definitely appreciated.
In the windowed case, Duck Game comes to mind as well :)


[Suggestion] MythicMobs on exclude metadata list

Just that, add it to the exclude metadata list mobs from MythicMobs plugin

I was unable to find whether MythicMobs adds metadata to its mobs, but if someone else can find it I'd be happy to add it to the defaults in the config, or to accept a PR doing so


🔥 Method addObserver must be called on the main thread

Hi! 👏
I have been using this package for a while until I recently stumbled upon this bug (I did not even update this package).
The error (like the title says) is just Method addObserver must be called on the main thread and appears when calling getLatestLocation after getting permission and configuring RNLocation.
I am very inexperienced with Java so it is a bit hard for me to fix this bug. I will try to dig into the code.
If anyone is experiencing the same issue, please let me know if you discover a way to solve it.
I have found similar bugs in other libraries:

https://github.com/getsentry/sentry-android/issues/520
https://github.com/segmentio/analytics-react-native/issues/255 (Error after upgrading another unrelated library, like in my case)

The error hints to this method in RNPlayServicesLocationProvider.java:
   public RNPlayServicesLocationProvider(Activity activity, ReactApplicationContext context) {
        this.context = context;
        if (activity != null) {
            locationProvider = LocationServices.getFusedLocationProviderClient(activity); // failing here
            locationSettingsClient = LocationServices.getSettingsClient(activity);
        } else {
            locationProvider = LocationServices.getFusedLocationProviderClient(context);
            locationSettingsClient = LocationServices.getSettingsClient(context);
        }
    }

Cheers!

I changed libraries to react-native-geolocation-service

using the context rather than activity solved the issue


Removed FB pixel + FB conversion tracking

What's this PR do?
Removes FB Pixel and FB conversion tracking code that was included in the layout.html templates
Why are we doing this? How does it help us?
Increase our site visitors' privacy; decrease page load time--impact on page load time probably varies quite a lot based on response speed of the FB endpoint, etc.
Bonus: FB doesn't need to know everything everyone does all the time, everywhere.
How should this be manually tested?
On dev, bring up browser Inspector window, Network tab
Load any page, or click any link. Sort the Initiator or Name column. Both work.
FB Pixel: You should not see fbevents.js anywhere in the column.
FB conversion tracking: You should not see fbds.js anywhere in the column.
How should this change be communicated to end users?
Audience Team will certainly let us know if they want this back. Please see "TODOs / next steps" below for more details.
Are there any smells or added technical debt to note?
Nope.
What are the relevant tickets?
Remove advanced Facebook pixel tracking
This task is an addendum to the above.
Have you done the following, if applicable:

[ ] Added automated tests? ( uses existing )
[ ] Tested manually on mobile? (  no diff )
[ ] Checked for performance implications? ( removing FB events which trigger API hits will improve performance )
[ ] Checked for security implications? ( no affect on site security )
[ ] Updated the documentation/wiki? ( no )

TODOs / next steps:
Twitter pixel tracking under review. Moving to a more explicit and stronger site visitor privacy policy we're cleaning up old, possibly unused trackers. It's an ongoing process.

:shipit: :shipit: :shipit: :shipit: :shipit:


Split storyboards in multiple files

Because storyboards can often become very large I find it a good practice to split them into multiple files.

Thanks @Tridie2000. I'm planning to add several more views soon so will do them in this fashion.


use initContainer instead of annotation

Change initContainers spec to conform with both 1.7 and 1.8+

I have tested with overlays on 1.7 and 1.10. It is working.

If you want to try it, use my docker image rjtsdl/etcd-operator:v0.5.2-azure-init


DeepSpeed integration breaks existing DeepSpeed logic

What's the issue, what's expected?:
There are attributes inside of regular deepspeed.runtime that are missing in this repo, and the monkey-patch doesn't cover, such as:
from deepspeed.runtime.lr_schedules import VALID_LR_SCHEDULES

How to reproduce it?:
from msamp import deepspeed
deepspeed.runtime.lr_schedules.VALID_LR_SCHEDULES

Log message or shapshot?:
Additional information:
Still trying to figure out a good solution here to make sure old namespaces can remain intact (as calling normal deepspeed later has consequences during training)

Generally I think this repo might want to recommend users do something like:
# Order doesn't really matter
import deepspeed
from msamp import deepspeed as msamp_deepspeed

Since otherwise they may face issues like this.
Not per-se a problem, but just something users should know about

For completeness, here's what my solution wound up being in the end:
import deepspeed
ds_initialize = deepspeed.initialize
if use_msamp:
    # MS-AMP requires DeepSpeed patches
    from msamp import deepspeed as msamp_deepspeed
    ds_initialize = msamp_deepspeed.initialize


Pi Hole dashboard

Currently the resultformat for Status in Pi Hole dashboard is a table. I believe it should be a time series.

Ok. I welcome PRs.

Great job btw mate! Thank you for sharing this. I've found it really useful.


Attempt to kill running process of output binary on access denied error on Windows

Problem
I'll sometimes run cargo build and get an error like the following on Windows:
Compiling deno v2.0.4 (V:\deno\cli)
error: failed to remove file `V:\deno\target\debug\deno.exe`

Caused by:
  Access is denied. (os error 5)

This is because I accidentally have the process running in the background, so I need to kill the process manually, then re-run cargo build.
This isn't so bad for debug builds, but can be a bit of a time waste for release builds.
Proposed Solution
I propose that if cargo encounters one of these errors it should check if the process is running, kill it and try again. This will collectively save Windows devs a decent amount of time.
An example of how this could be implemented is to just spawn powershell -Command with a script source like:
Get-WmiObject Win32_Process | Where-Object {
    $_.Path -eq 'V:\deno\target\debug\deno.exe'
} | ForEach-Object {
  if ($_.Terminate()) {
    Write-Host 'Terminated:' $_.ProcessId
  }
}

...when it fails it could just surface the original error.
Notes
No response

Another workflow that hits this was reported in #12485 and our suggested fix for that is to improve the error message.
  However, we should probably consolidate that conversation on #12485 since the options are mutually exclusive.  As such, I'm closing in favor of #12485.  If there is a reason we should keep this open separately, let us know!

Ah sorry, I searched the issue tracker for the proposed solution and not the problem.


RequestBody not appearing in Spec

I'm trying to set the requestBody

But in the spec; I'm only seeing

Project setup is:
addSbtPlugin("com.iheart" %% "sbt-play-swagger" % "0.7.4")
Play 2.6; Scala 2.12 and Sbt 1.2.6
Everything else is working great.
Any pointers much appreciated.
Thanks

not srue about the requestBody swagger spec, but the following should work
#  parameters:
#    - name: body
#      schema:
#        $ref: '#/definitions/com.iheart.Track'

BRILLIANT
Thank you so much for the exceedingly rapid reply.


Runtime for pandora compare

Hi,
I am using pandora compare to call out the variants after building the pan-genome reference graphs and indexing them using pandora index. Although I am using multiple threads, it works with one strain at a time and takes a lot of time to genotype single strain and I have more than 2000 strains to genotype.
Is there any way so that I can parallise it to run on multiple strains at a time? or is there any other way such that I can genotype all samples quickly?

Hey @AmayAgrawal , sorry for my delay.
pandora compare basically runs in two steps, each using multiple threads:

The first step is to map reads from the samples to the PanRG and infer the most likely path for the samples in each PRG. The multithreading here enters just when mapping the reads to the PanRG, which is the most costly operation. However, one sample is mapped at a time. The log messages you get while doing this are sth like:

[2023-01-27 15:29:49.315710] [0x00007f8a5ad8d7c0] [info]    Constructing pangenome::Graph from read file reads/toy_sample_1/toy_sample_1.100x.random.illumina.fastq (this will take a while)
<...>
[2023-01-27 15:29:49.726961] [0x00007f8a5ad8d7c0] [info]    Processed 700 reads
<...>
[2023-01-27 15:29:49.785260] [0x00007f8a5ad8d7c0] [info]    Find max likelihood PRG paths

The second step is to actually do a multi-sample genotyping of each PRG based on the maximum likelihood paths and minimiser coverage gathered from the first step. Here, all PRGs are genotyped multithreadely, with each thread getting one PRG, genotyping it, and getting the next and so on. The log messages you get while doing this are sth like:

[2023-01-27 15:29:50.218415] [0x00007f8a5ad8d7c0] [info]    Multi-sample pangraph has 2 nodes
[2023-01-27 15:29:50.227535] [0x00007f8a5ad8d7c0] [info]    Infer VCF reference path
[2023-01-27 15:29:50.241175] [0x00007f8a5ad8d7c0] [info]    Genotyping VCF...
[2023-01-27 15:29:50.241479] [0x00007f8a5ad8d7c0] [info]    Finished genotyping VCF
[2023-01-27 15:29:50.246743] [0x00007f8a5ad8d7c0] [info]    Infer VCF reference path
[2023-01-27 15:29:50.259194] [0x00007f8a5ad8d7c0] [info]    Genotyping VCF...

I think from reading your messages, you are on step 1. Could you please confirm? If yes, you'd know how many samples you've mapped with a command like grep "Constructing pangenome::Graph from read file" pandora.log | wc -l. Just wondering if it is finishing this step or still far from finishing.
The maximum number of strains I believe we have used pandora compare was 114 illumina samples, but it is a special use case as our PRG contained only plasmid genes, so most of the reads don't map. That took 2.5h, but in our paper to run on 20 e-coli samples, it took ~10h, both runs with 16 threads. So I could see 100 samples taking at least a couple of days, possibly more, and 2000 samples possibly 1000h.
The current ways around this is to increase the number of threads, if possible. We are working to scale pandora up, but unfortunately our most pressing need is to scale pandora to handle millions of PRGs, and not yet thousands of samples. Handling thousands of samples is in the roadmap though, and we might start working on this within some months...

Hi @leoisl,
Thanks for the explanation. As you predicted, currently I am in the first step where the reads from the samples are being mapped to the PanRG to infer the most likely path for the samples in each PRG. I am running the analysis with 64 threads and I am half way through the samples in 2 weeks time and I guess that I have to wait same amount of time for it to finish.
Yes, I think it would be nice in the future to have the functionality where it can handle thousands of samples quickly.

Sorry it is currently too slow! Pandora is inherently  parallelisable by sample when mapping, and then by gene when doing 'compare', and could then in principle run very fast on a cluster. I don't know if you have access to one. But we need  to make a number of updates to enable this, and we have to finish our current changes first. As leandro said, right now we're modifying it so even with enormous pan genomes, the ram use is controlled. This will take several weeks to finish. Apologies for the delay.

Hi, I have a small question regarding this analysis. So as I mentioned earlier, I started the pandora compare analysis for around 2000 strains. As mentioned above by @leoisl, pandora works in two steps: first is to map the reads from the samples to the PRG and second is to do multi-sample genotyping of each PRG. I think that I was towards the end of the second step and then the process was automatically killed today. Below are the log messages that I got
[2023-02-11 12:40:45.847849] [0x00007fe77ce1a700] [info]    Genotyping VCF...
[2023-02-11 12:40:47.344218] [0x00007fe77ce1a700] [info]    Finished genotyping VCF
[2023-02-11 12:58:17.154721] [0x00007fe76da79700] [info]    Genotyping VCF...
[2023-02-11 12:58:17.831671] [0x00007fe76da79700] [info]    Finished genotyping VCF
[2023-02-13 03:17:11.856584] [0x00007fe75f25c700] [info]    Genotyping VCF...
[2023-02-13 03:17:20.521788] [0x00007fe75f25c700] [info]    Finished genotyping VCF
[2023-02-13 09:47:52.348881] [0x00007fe77027e700] [info]    Genotyping VCF...
[2023-02-13 09:47:52.938793] [0x00007fe77027e700] [info]    Finished genotyping VCF
Killed

From the above log, it can be seen that for almost two weeks, there was no output log written (last one was written on 13th Feb) and also nothing was written in the output folder. So I think that pandora was in the last step of making one final multi sample vcf file when it was killed automatically. First is do you think that it was in this step? Next question is now that it was killed and I do not have one final vcf file, I saw that there is VCFs_genotyped folder in which I can see the vcf file for each PRG. So I was thinking to combine the output from all these individual PRGs vcf files into one final vcf file. Will this work? Or is there any other way that pandora stores checkpoints and I can resume the analysis again from this particular step because I can't run the whole thing again as it took over a month to reach at this stage of the analysis.

Hey @AmayAgrawal , sorry for the huge delay on answering you.

So I was thinking to combine the output from all these individual PRGs vcf files into one final vcf file. Will this work?

Maybe but I don't think the resulting files would be complete... I would not recommend that... Pandora also does not store checkpoints for resuming analysis later.
Bottom line is that pandora is still not scalable to thousands of samples. We are working right now to scale it to millions of genes, and next step is to scale to thousands of samples. We will let you know when this feature is implemented.

Sorry for this. I have one hack that would allow you to progress for now.

Collect a subset of your genomes that runs in a reasonable time - say 50, and run pandora compare on them, and identify the inferred reference it generates for the VCFs. Then run another 2000/50 -1 jobs each one running pandora compare on 50 samples, using that same reference. This is parallelisable by you, can be run on a cluster, and will give you VCFs which are all using the same reference. It's not perfect, but it would allow you to move forwards for now.


feat(api): support node with callback children

chart.options({
  type: 'repeatMatrix',
  width: 480,
  height: 480,
  paddingLeft: 50,
  paddingBottom: 50,
  data: {
    type: 'fetch',
    value: 'data/penguins.csv',
  },
  encode: {
    position: ['culmen_length_mm', 'culmen_depth_mm'],
    color: 'species', // 需要显式指定颜色编码，否则无法推断
  },
  children: () => ({ // 回调
    type: 'point',
    encode: { color: 'species' },
  }),
});

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

17 of 18 (94.44%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.009%) to 88.193%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/api/utils.ts
17
18
94.44%

Totals

Change from base Build<PHONE_NUMBER>:
0.009%

Covered Lines:
10029

Relevant Lines:
11013

💛  - Coveralls


Allow for instrumentation/replacement of compute

On some platforms, spawning an isolate is particularly expensive memory wise. Packages and/or the framework may use compute (or otherwise spawn an isolate) to avoid jank when decoding a large chunk of data, e.g. to json parse or otherwise decode a serialized format.
It would be helpful to make it clearer when an isolate is spawned, particularly for developers who are working on memory related issues and trying to understand what might be causing a sudden increase in memory usage.
At the very least, we should be able to allow instrumentation of compute in foundation. We might be able to do something in the engine with the isolate create callback, if that is in fact invoked every time an isolate is created. Otherwise, we may need some additional support upstream in Dart.
/cc @terrylucas @jacob314
/cc @chinmaygarde  @jason-simmons @zanderso

From DevTools we already know when isolates are spawned so we could surface that in the timeline if it is valuable.

Probably. However, I think we also need a way to drop in a replacement for this for apps that just don't want anyone out there spawning an isolate, or may want to reuse an already spawned isolate for compute tasks.

@aam @mkustermann - do you have any thoughts about the need for this with the lightweight isolates work?
I'm partially under the impression that the lightweight isolate work would make replacing this less necessarily, as long as we resolve https://github.com/flutter/flutter/issues/79573

@aam @mkustermann - do you have any thoughts about the need for this with the lightweight isolates work?
I'm partially under the impression that the lightweight isolate work would make replacing this less necessarily, ...

I agree. The spawn-latency and memory overhead should decrease by order(s) of magnitude and is unlikely going to be an issue.

In that case I'm going to close this as obsolete.


build: compile Xdebug 3.2.2 for PHP 8.0, 8.1, 8.2, fixes #6159

The Issue

#6159

How This PR Solves The Issue
Compiles Xdebug 3.2.2 library for PHP 8.0, 8.1, 8.2.
Unfortunately we can't build Xdebug 3.2.2 for PHP 8.3:
checking Check for supported PHP versions... configure: error: not supported. Need a PHP version >= 8.0.0 and < 8.3.0 (found 8.3.7)

Xdebug Compatibility https://xdebug.org/docs/compat
Manual Testing Instructions
# check Xdebug library files in DDEV HEAD
ddev exec ls -l $(ddev exec dpkg-query -L php{8.0,8.1,8.2}-xdebug | grep xdebug.so)

# check Xdebug library files in this PR
ddev exec ls -l /usr/lib/php/20200930/xdebug.so /usr/lib/php/20210902/xdebug.so /usr/lib/php/20220829/xdebug.so

ddev config --php-version 8.2 && ddev restart && ddev xdebug && ddev php -v
ddev config --php-version 8.1 && ddev restart && ddev xdebug && ddev php -v
ddev config --php-version 8.0 && ddev restart && ddev xdebug && ddev php -v

Automated Testing Overview

Related Issue Link(s)
Release/Deployment Notes

I will make a new build of the Docker image only after the next PR goes in:

#6180

And it can be tested manually using .ddev/web-build/Dockerfile

Please open an issue with https://bugs.xdebug.org/my_view_page.php explaining your results with php 8.3, thanks!

Please open an issue with bugs.xdebug.org/my_view_page.php explaining your results with php 8.3, thanks!

It's not compatible (see https://xdebug.org/docs/compat) so they won't fix it.

Oh, I understand now, thanks.

I added a new stage for Xdebug, because the size of the Docker image has grown significantly:
linux/amd64 333.86 MB => 426.51 MB
linux/arm64 327.2 MB => 414 MB
Also I added cleanup for /tmp/*, because there were leftovers from platformsh:
$ docker run --rm -it ddev/ddev-webserver:v1.23.0-1 ls -la /tmp
total 40
drwxrwxrwt 2 root root  4096 Apr 16 18:31 .
drwxr-xr-x 1 root root  4096 May 14 10:54 ..
-rw-r--r-- 1 root root  4891 Apr 16 18:30 platformsh-install-20240416-183042.log
-rw-r--r-- 1 root root  4862 Apr 16 18:30 platformsh-install-20240416-183049.log
-rw-r--r-- 1 root root 14880 Apr 16 18:30 setup.deb.sh

Building a new image.

I was thinking about that last night, about build-essential getting into the final image. Thanks for catching that!

You have done so well!

Oh, dear, I didn't look at the right thing, did I? That was a previous push. The current ddev-php-base is much too big:

Yes, but it failed after 30 minutes (connection timeout to php repo), I just restarted it again.

I ran a basic debugging session with a single breakpoint and step over, using PHP 8.0, 8.1, 8.2.
Used Linux amd64 with PhpStorm 2023.3.6.
And tested Xdebug 3.3.2 with PHP 8.3 to make sure it works the same for me.

So much better, thanks!:

And this works too, so we don't limit everyone to 3.2.2:
webimage_extra_packages: ['php${DDEV_PHP_VERSION}-xdebug']


Unable to add name to the services

Description
The user/customer can create more than one service of each kind with a different setup and then is not possible to identify each one.

Environment
Operating system: iOS
OpenShift versions: 3.9
Steps to reproduce
Create more than once service with the same type.

Hi @camilamacedo86,
Thanks for your feedback. This repo is now deprecated and is replaced by Mobile Developer Console. Could you please give it a try and if you still experience the same problem, please open a new issue in that project.
Thanks.


Cannot adjust height in %

Hi,
setting the width and height in px works fine. However, if I use % to give a size, it only works for width but not for height. I can set height: "300%" or height: "20%" but the height never changes.
Is this an GrafanaChart problem, a grafana problem or something else?

This is to be expected, since every module in MagicMirror is inside a absolutely positioned region div.