ADF seems to support Repos

it seems Azure Data Factory is able to see Repos and launch notebooks inside them (I've just tested it with a customer using both Azure Data Factory and Databricks).
so it seems now they're supported.
https://docs.microsoft.com/en-us/azure/databricks/repos#does-repos-support-azure-data-factory-adf

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0edc68be-c91d-f305-d1d9-8999778b6a7f
Version Independent ID: 1bcf3fba-9d14-544c-8cc1-29c401b87611
Content: Repos for Git integration - Azure Databricks
Content Source: databricks/repos.md
Service: azure-databricks
GitHub Login: @mssaperla
Microsoft Alias: saperla

Thanks for the ask  . At this time we are reviewing the ask and we will reply back as soon as we can .

Can you please provide a more detailed instruction on the workaround? What specific code is in the triggering notebook? How do you call the other notebooks inside of the repos? what format? how do you call a specific branch?

Repos folder is visible from ADF as any other folder defined in Databricks, so you can point to a specific branch using a path like:
repos/<project_name>/<branch_name>

Repos folder is visible from ADF as any other folder defined in Databricks, so you can point to a specific branch using a path like:
repos/<project_name>/<branch_name>

Hi again,
I just tried this out now. When going through the Data Factory UI and browsing the notebook the path to the notebook becomes:
/repos/<user_name>/<project_name>//
If I try to modify it to your suggestion above:
/repos/<project_name>/<branch_name>//
it fails with notebook not found.
Please show screenshots of notebook path in Data Factory and corresponding Databricks Repos UI and how they translate.
Thanks!

@leifbro could you please look into above enhancement?

@baatch did you create the project folder under the "Repos" folder? Maybe you created it under your user's folder?

@baatch did you create the project folder under the "Repos" folder? Maybe you created it under your user's folder?

@R1k91
I did not create a top level folder first but now I have created the project top level folder.
But how can I call a specific branch from Data Factory? There is no subfolder for branch as you mentioned?

Thanks for your dedication to our documentation. Unfortunately, we have been unable to address your issue and apologize for the delayed response. We are closing this issue, but if you feel that it's still a concern, please let us know directly at<EMAIL_ADDRESS>#please-close

#please-close


[SUGGESTION] Use namespaces to organize code

Short Description
At the moment, everything except for a few exceptions, the one I remember from the top of my head being the Permissions stuff, is thrown inside the ACF global table. One way to organize this would be to divide everything into smaller tables that only contain the stuff related to a certain aspect of ACF, such as:

ACF.Ballistics
ACF.Damage
ACF.Utilities
etc.

References (Optional)
Although I feel like everything inside this file could be improved, I like the fact everything is inside the ACF.Permissions table.
https://github.com/Stooberton/ACF-3/blob/master/lua/acf/base/permission/sv_permissions.lua

Current changes on 773363d allow ACF to automatically create namespaces based on folder structure. This feature should be used  in all future additions.


Adaptive make command in docs

Hi all,
I've noticed that the make command in various docs sets the number of threads as a hardcoded number, such as -j8 or -j4. In my opinion, this is not very effective when we have different core counts  on different machines. I think we should instead use make -j$(nproc) which sets the number of threads according to the processor core count on the machine. I've used it on linux based machines and it works fine.
What do you guys think?

good idea!

good idea. but it doesn't work on mac

Yeah, that's true. For Mac we need to use sysctl -n hw.ncpu. I don't know if you think this would complicate the compilation steps. Anyway, I think at least we can change the Dockerfiles for MXNet, since docker can only run in Linux.

totally agree we can update the dockfile. we may also mention to use
-$(npoc) for linux for best compilation performance in docs. can you please
make a PR for it?
On Thu, Jun 2, 2016 at 8:21 PM, Siyuan LIU<EMAIL_ADDRESS>wrote:

Yeah, that's true. For Mac we need to use sysctl -n hw.ncpu. I don't know
if you think this would complicate the compilation steps. Anyway, I think
at least we can change the Dockerfiles for MXNet, since docker can only run
in Linux.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/mxnet/issues/2304#issuecomment-223460185, or mute
the thread
https://github.com/notifications/unsubscribe/AAZv4UXAPY8-Pv5aasozO6urRY7nxGGVks5qH3OZgaJpZM4IrN6A
.

Sure, no problem. I'll make a PR later. Have you tried make -j$(sysctl -n hw.ncpu)? Does it work? I think Homebrew uses that to detect the number of cores.

yeah, $(sysctl -n hw.ncpu) works well on my mbp with os x 10.11.4
On Thu, Jun 2, 2016 at 8:40 PM, Siyuan LIU<EMAIL_ADDRESS>wrote:

Sure, no problem. I'll make a PR later. Have you tried make -j$(sysctl -n
hw.ncpu)? Does it work? I think Homebrew uses that to detect the number
of cores.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/mxnet/issues/2304#issuecomment-223462524, or mute
the thread
https://github.com/notifications/unsubscribe/AAZv4RUukfqAT9Hay8WWKR1v28SmSH5Eks5qH3fygaJpZM4IrN6A
.


Add method to stop VoiceReceiver

A the moment there is no way to control the VoiceReceiver. It should have a stop or close or end event, in order to stop the receiver and any streams it's currently receiving.
Some possible reference: https://nodejs.org/api/events.html#events_emitter_removealllisteners_eventname

Addressed in 95369b0a89f6dd616203c55e9d31a804869788dd, but untested, if this fails please re-open the issue.


iPad should download both orientation startup images

Issue #94 introduced a new method for including startup images to avoid unnecessary downloads. This technique makes sense for including the right iphone size image, but if it's ipad you really want to download both portrait and landscape, as the user might be opening the webapp in either orientation.
This isn't as condensed as it could be, but illustrates the idea:
<script>
if (navigator.platform==="iPad") {
  document.write('<link rel="apple-touch-startup-image" href="img/startup-1024x748.png" media="screen and (min-device-width: 481px) and (max-device-width: 1024px) and (orientation:landscape)" /><link rel="apple-touch-startup-image" href="img/startup-768x1004.png" media="screen and (min-device-width: 481px) and (max-device-width: 1024px) and (orientation:portrait)" />');
}
else {
  document.write('<link rel="apple-touch-startup-image" href="'+(window.devicePixelRatio==2?"img/startup-640x920.png":"img/startup-320x460.png")+'"/>')
}
</script>

Thanks for this Ben, you are right will see to updating the current method. This will also likely need changing further if the iPad 3 goes retina.

This still needs updating with retina assets for the new iPad (need to test the new file sizes work on iOS5.1), but i propose something like this. Anyone got further tweaks/suggestions?
<script>(function(){ 	var p, l; 	if(navigator.platform==="iPad"){ 		p = "img/startup-tablet-portrait.png"; 		l = "img/startup-tablet-landscape.png"; 		document.write('<link rel="apple-touch-startup-image" href="'+l+'" media="(min-device-width: 768px) and (orientation: landscape)"/><link rel="apple-touch-startup-image" href="'+p+'" media="(min-device-width: 768px) and (orientation: portrait)"/>'); 	}else{ 		p = window.devicePixelRatio === 2 ? "img/startup-retina.png": "img/startup.png";  		document.write('<link rel="apple-touch-startup-image" href="'+p+'"/>'); 	} })()</script>


A Couple Of Issues

I have two problems that I can't seem to resolve.
Maybe you can give me insight if you have seen them before.

Cakewalk suddenly started giving me a load error:
AudioPlugSharVst, GetPluginFactory

I have attached a screen dump. I tried cleaning/rebuilding, copying and
renaming into project folders, nothing fixes the problem.
Any idea why I would get this problem and how to fix it?

The second problem is a bit more insidious.
I have a bunch of simple apps in which I override GetEditorWindow.
For all of them, I use WPF controls.
So, I decided to do a custom user control.
I found a really nice knob control on the web in xaml.
Converted to list to a named canvas.
so I created a User Control. I created knob programmatically
in C#. Then made the knob a child of the named canvas.
I put the code the generates the knob into a WPF DLL and
that CODE calls the rotary control DLL to setup the handlers.
Wrote a test driver. Works great, I can move the pointers around
and read the dial values.
Then, I integrated into a plugin.
The knob displays beautifully.
But, when I tried to read the value of the knob, I am told:
can't access because that control is on a different thread from the
calling thread.
I have tried everything I can find to defeat this: using dispatcher,
I get a null dispatcher, I finally seemed to find a way to get a non-null,
dispatcher but the call to get the value returns nothing!
I see you have done lots of plugins. Have you ever had this problem?
How did you get around it? I looked through your code but don't see
what you did. Any ideas?
I have one other idea I will try -- putting the two classes into the
plugin   directly. That might fix the problem.  Even if that fixes, don't
understand why separate DLLs are a problem.
Any ideas would be much appreciated.

Thanks.
Bruce

More on the threading issue:
In my test driver for the dial, I was able to replicate the cross threading problem. I was able to solve the problem by wrapping the access to the dial value:
this.Dispatcher.BeginInvoke(new Action(() =>
{
gain = GainDial.Value;
}));
This returned the value of the dial.
But, when I tried to use this in the plugin, the execution is successful,
but no value is returned!!
I have done some deep dive into the vstsdk and have determined that
the plugin does seem to run in a thread.
So, how are you getting past the cross thread lock in your custom controls.
Thanks.
Bruce

OK, finally figured out how to make the new knob control thread safe.

Added a method in the knob control class to use Dispatcher.Invoke to wrap the access to the knob control properties. This is now owned by the knob control class which has thread affinity to the UI thread. But, the knob control can access its own value inside the knob control class, even if the access is from another thread.
Had to wrap the call to that method in a try/catch block. When I stopped the audio, the whole plugin hung. I guess the host keeps making calls to process which somehow hangs after stop audio.
Took me several days to figure out this simple issue. I tried every
possible approach. I saw this in the example you sent me, so I tried.


[FLINK-12828][sql-client] Support -f option with a sql script file as…

… input
What is the purpose of the change
this pr allows sql client to support -f option with a sql script file as input
Brief change log

Introduce -f option with a sql script file as input

Verifying this change

Added test that validates that CliClient works with -f option: org.apache.flink.table.client.cli.CliClientTest::testSQLFileSubmission

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (yes)
If yes, how is the feature documented? (not documented)

@docete can you rebase this PR? This should be aimed for 1.10

Thanks for this PR! LGTM.
@xuefuz @twalthr @KurtYoung @wuchong do you have any concerns?

I have the same concern with @xuefuz . We should support multi-statement for sql scripts, otherwise this feature is not helpful for users.

@xuefuz @wuchong  multi-statement support is introduced in https://issues.apache.org/jira/browse/FLINK-12845. After that, I will make minor changes to support multi-statement for sql scripts.

Hi @xuefuz what if a statement in multiple lines or multiple statements in a single line? I think we should have a consistent behavior as multi-command line in FLINK-12845.

@docete It shouldn't matter if a command spans multiple line or a line contains multiple commands in the script file, one should be able to read the file content and parse out one command at a time to execute. This avoids holding the file content in memory.
Nevertheless, I guess the current way is acceptable for now. We can worry about memory usage later.
Thanks again for working on this.


Reformatted most of the list items to make them look cleaner

Mostly just added more paddings and margins and some changes to the text sizes.

While I agree that list items need a refresh, I don't think this is what we want to do. Ideally we'd want to give a more comprehensive refresh, in line with MD and the rest of the app design we've gone with. The result would probably involve overwriting most of these changes, so I think it's best not to merge this. @Meisolsson @fadils what do you think?

I looked at the views I comparison to MD and there are some things that should change. So I agree with the idea to wait.


MPFConditional bug, maybe provided solution

Hello.
I think I've encountered some misstake in the code. I have been working to try and use machine variables with MPFConditional, but it crashes all the time when I use it.

I then checked the mpf_variable.gd which actually works with machine variables. At line 99 it say value = MPF.game.machine_vars

If I then go back to mpf_conditional.gd and change line 104 to base = MPF.game.machine_vars

Now MPFConditional doesn't crash and it seems to work as it should. If I check the condition on a machine variable with MPFConditional it works as expected, I have tried with different values that's both true and false. Works as it should with the change.
I guess this is a bug. I'm no programmer so hopefully you can check it and verify. I'm useless at GitHub so an issue is all you get from me.
Cheers.
/Johan Gill

Great find! You are correct, and I've pushed an update to reference the proper name MPF.game.machine_vars. Thanks!


Resolve ImportError due to mutual imports

empirical_models and mock_observables were previously trying to import each other, causing an ImportError that was not caught by CI since Travis is currently failing due to unrelated reasons.

After resolving the ImportError problem that was the original intent of this PR, I made some minor changes throughout the test suite in an attempt to resolve the Travis and AppVeyor build problems. All now appears to be good with AppVeyor, although Travis still has a few problems for certain configurations. See #710 for details

When I merged this PR, there were actually no build failures on either Travis or AppVeyor, so in principle this PR resolves #720. However, I had to rerun the test a couple of times in order for it to pass, so this is not really a solved problem. See #710 for discussion.

Thanks for the tip, @bsipocz. I think that's a fine workaround since it accomplishes the needed testing, and maintains the documentation. I just checked my current testing suite and verified that the exact syntax I commented out of the doctests is actually already covered in a unit-testing module, so in the end, commenting out these few doctest lines actually has no consequence (though I didn't know that at the time.)
The most recently merged PR #713 actually passes all CI builds, so in principle Halotools is back on track. But I had to manually rebuild several different configurations in order to get 100% passes. So, in other words, for a given test, sometimes it passes, sometimes it fails, even with no changes to the source code. I expect this will continue to be the case, which will make it difficult to manage future development.
All this time, I have never had a local failure. That probably means these are CI-environment specific failures and nothing to worry about. But all the same, this is uncomfortable since I'm on the verge of release. In fact, there are no other outstanding v0.5 Milestone Issues or PRs. This is it. If I could be confident that there were no real failures to worry about, I'd go ahead and put the current version of master up on pip as v0.5, no changes.
Probably the thing to do now is to notify my colleagues and collaborators that v0.5 is about to come out, and get some independent verifications with a bunch of different machines that all is kosher. Then if there are no reports of problems, then we can be more confident that the CI build problems are actually only problems on an unrealistically restricted CI environment.
I would not mind delaying release for the sake of ensuring code stability. So if either @eteq or @bsipocz have suggestions for other things to try before notifying the core user base and then releasing, please let me know.

@aphearin - Have you though about doing a release candidate? That would allow you to get an easier way of testing by a group of collaborators, installing from pip, while the wider usebase remain unaffected.
The other approach is to release, and whenever a valid and real bug report comes relating to this, release a bugfix 0.5.1. However this option can be more work as involves the maintenance of a bugfix branch.

@bsipocz - I have not done a "candidate release" before, but I like that idea. Is that release procedure any different from what I normally do?
I'd rather not manage bug-fix branches if I can help it, it's just more work than I'd like to take on if possible.

It's more or less the same, the trick is in the version numbering, you want to make sure pip recognizes that it's a pre-release, so pip install halotools wont pick it up.
http://docs.astropy.org/en/stable/development/releasing.html#modifications-for-a-beta-release-candidate-release

Ha! Obviously I've just passed over that section too many times to notice it anymore :-\ Thanks for the tip, this is probably how I'll handle the situation.

That's in another docs, and since you're not planning on bugfix branching, it's unlikely to click on the link in the note at the bottom of the page :)
So maybe we need to repeat that about the prerelease in the affiliate release page, too.


JOSS Review -- minor comment on figures in the notebook examples

Hello!
Just a quick and minor point -- I noticed the some figures within the notebooks don't have x-label and/or y-label. It would be nice to have that as it helps with readability of the examples.

Thanks for pointing this out! Fixed this in all notebooks with the documentation branch.


Azure new account 429

Deploying an app service plan on Azure via Pulumi appears to hang forever. When I deploy the same through the Azure console, I see that Azure is returning a 429. Via Azure support, I learned that my account is new (<5 days old) so they are limiting service usage on the account. I'm working through this with support, however I wanted to propose that Pulumi add logging to indicate this issue so it is more apparent to users when they are encountering this issue.

Thank you for reporting this, @daronstinnett. The provider is expected to retry with backoff on 429, which can take a while. It should give up eventually, though. We'll queue this up for a deeper look.

@daronstinnett Which provider were you using - Azure Native or Azure Classic?


Support the restoring of issuers from backup files

This allows the loading of cert-manager issuers and cluster issuers from a jsctl backup.
For example, with a backup.yaml as follows, the output for the installation will be as seen below. Note that the warnings of other issuers are logged to stderr in the process. In this case, a GoogleCASIssuer.
 $ cat backup.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  creationTimestamp: null
  name: cm-cluster-issuer-sample
spec:
  acme:
    email<EMAIL_ADDRESS>    preferredChain: ""
    privateKeySecretRef:
      name: example
    server: https://
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  creationTimestamp: null
  name: cm-issuer-sample
  namespace: jetstack-secure
spec:
  ca:
    secretName: ca-key-pair
---
apiVersion: cas-issuer.jetstack.io/v1beta1
kind: GoogleCASIssuer
metadata:
  creationTimestamp: null
  name: googlecasissuer-sample
  namespace: jetstack-secure
spec:
  caPoolId: my-pool
  credentials:
    key: example
    name: googlesa
  location: us-east1
  project: example
---

$ go run main.go operator installations apply --stdout --experimental-issuers-backup-file=backup.yaml
The following issuers cannot be managed by the operator and must be restored manually: GoogleCASIssuer/googlecasissuer-sample
apiVersion: operator.jetstack.io/v1alpha1
kind: Installation
metadata:
  creationTimestamp: null
  name: installation
spec:
  approverPolicy: {}
  certManager:
    controller:
      replicas: 2
    webhook:
      replicas: 2
  issuers:
  - ca:
      secretName: ca-key-pair
    name: cm-issuer-sample
    namespace: jetstack-secure
  - acme:
      email<EMAIL_ADDRESS>      preferredChain: ""
      privateKeySecretRef:
        name: example
      server: https://
    clusterScope: true
    name: cm-cluster-issuer-sample

Thanks for the feedback and for testing this out too.


Update README.md - specify Arduino IDE version

This commit specifies that the HOWTO described in the README applies to the Arduino Desktop IDE. Specific features, eg. Serial Plotter, are for the time being only available on the Desktop version.

cc @petewarden


Adds GenStateMachine.child_spec/1

This PR adds child_spec/1 to GenStateMachine.
It allows GenStateMachine to be used with the newer child spec syntax used by Supervisor & DynamicSupervisor.
defmodule Engine do
  use GenStateMachine

  def start_link(args) do
    GenStateMachine.start_link(__MODULE__, [], args)
  end

  def init(data) do
    {:ok, nil, data}
  end
end

Engine.child_spec([foo: 1]) #=> %{id: Engine, start: {Engine, :start_link, [[foo: 1]]}}

{:ok, sup} = DynamicSupervisor.start_link(strategy: :one_for_one)
DynamicSupervisor.start_child(sup, {Engine, []})

It also allows child spec options to be defined when use'ing the module. ie
defmodule Engine do
  use GenStateMachine, restart: :permanent
end

oh doh, did not realize a PR was already open for this 😊

That's ok, this one is closer to how I would have preferred this implemented than the other. That being said, I'm going to have to spend some time thinking about the compatibility implications, since Supervisor.child_spec/2 did not exist in even versions as recent as v1.4.5. This is causing the build failures, since the test for child_spec/1 is failing on that version of Elixir. The simplest solution would be to disable that test conditionally, but I wonder if additionally the code should be generated for child_spec differently.

Oh good point, I wonder how other libraries handle it

This is now in master in a backwards compatible way, so closing this PR. Will release new version soon. Thanks!


icetea support

Add support for icetea tests

Py3 issue fixed in https://github.com/ARMmbed/mbed-os/pull/7769

@theotherjimmy anything need to do before merge? I don't see anything

@OPpuolitaival LGTM. Thanks for all your working on this!
Please address @bridadan's comment.

@screamerbg now fixed..


Do not inherit config in model fields that are themselves of type model

I think the current mechanism of attempting to merge configs when handling fields of model types is the source of some problems.
I'll describe the problems below, but given how we want to get the v2 release out ASAP, I really think we should drop this functionality in pydantic v2 (or at least find a way to make it possible to add in a future minor release) so that we don't find ourselves committed to dealing with bugs related to it.
I'll also note that this didn't seem to be a thing in v1 (a point which I'll make in the companion PR for this on the pydantic side), so it's not a breaking change to remove this behavior. (And in fact, improves backwards-compatibility.)

First, this is likely to be a source of bugs due to the fact that core refs don't reflect config settings, and if the schema gets simplified to use definitions, config settings will be lost. For example, consider:
from pydantic import BaseModel

class C(BaseModel):
    my_string: str

class B(BaseModel):
    model_config = dict(str_to_lower=True)

    c: C

class A(BaseModel):
    b: B
    c: C

If the schema is simplified in such a way that the C becomes a definition reference instead of duplicated, the behavior of C inside of B would be the same as inside of A. Right now we don't have a way to bake the config settings into the refs to prevent this, and I think it might be unreasonably difficult to try.
Even if we were smart about not out-of-lining C, I think you'll run into more challenging issues with recursive models. For example:
from pydantic import BaseModel

class B(BaseModel):
    model_config = dict(str_to_lower=True)

    a: 'A'

class A(BaseModel):
    b: B | None
    my_string: str

In this case, A has to be a definition reference, and to behave correctly where the string gets lowercased only for the cases where it is a subfield of B, you'd need to make sure to have different refs for the outer A and the one that has the config settings. One can also imagine more complex scenarios where the config takes different values at different levels of nesting; basically, if config influences the behavior of validation/serialization, it must be the same when the ref is the same. By eliminating config inheritance for fields, we eliminate the most common (and perhaps only?) way to get differing config with the same ref today.

Second, even if we did want to keep this functionality, it is currently causing problems because some config fields almost certainly should not get inherited by fields. The main example being extra, which is causing issue https://github.com/pydantic/pydantic/issues/5784, which incidentally does get fixed by this PR.

In general, I would argue that if you want config inheritance, it's probably better to find a way to achieve the desired result by way of a shared base class, rather than trying to modify the config by way of inclusion in a different parent. I realize this is not equally capable, but I (currently) think its better to remove this capability and not try to support it for users in v2 (especially considering it wasn't even a thing in v1), than to deal with the various edge-case bugs it may cause.
The one exception to this would be if we made some kind of "global" config thing, so that it affected the entire core schema the same way, rather than being able to be overridden in fields. But I would argue that the current mechanism is too different to be a step toward that, and would prefer that we find a way to add that later.

Codecov Report

Merging #612 (51be332) into main (2a7a2f6) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #612      +/-   ##
==========================================
- Coverage   94.25%   94.24%   -0.01%
==========================================
  Files          98       98
  Lines       13188    13158      -30
  Branches       25       25
==========================================
- Hits        12430    12401      -29
+ Misses        752      751       -1
  Partials        6        6

Impacted Files
Coverage Δ

pydantic_core/core_schema.py
96.87% <ø> (-0.01%)
:arrow_down:

src/build_tools.rs
86.25% <ø> (-1.80%)
:arrow_down:

src/serializers/type_serializers/model.rs
93.96% <100.00%> (ø)

src/validators/model.rs
98.80% <100.00%> (ø)

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2a7a2f6...51be332. Read the comment docs.

This makes a lot of sense to me. The backward compatibility argument is a strong one. I think we should move forward with this as-is.
On the whole, I wonder what the value of CoreConfig here is in general: strict is a flag both on CoreConfig and also on every single schema, while str_to_lower is a flag on CoreConfig that only makes sense for str schemas. I think pydantic-core should do away with the config concept in general and pydantic should be the one that figures out how it wants to apply the config on a model (presumably using the model this PR is proposing of "pushing down" until the next config (or BaseModel) is encountered.


Memory Leak in case of "IncompleteMessage: connection closed before message completed"

Version
hyper 0.14.28
tokio 1.35.0
tokio-openssl 0.6.3
Description
Observing memory increase for "Incomplete Message error".
testing secario:

server: on http1 request, simply doing connection close.

server.go:
package main

import (
	"crypto/tls"
	"fmt"
	"net/http"
	"os"
)

func handler(w http.ResponseWriter, r *http.Request) {
	// Hijack the connection to get access to the underlying TCP connection
	hj, ok := w.(http.Hijacker)
	if !ok {
		http.Error(w, "Hijacking not supported", http.StatusInternalServerError)
		return
	}
	conn, _, err := hj.Hijack()
	if err != nil {
		http.Error(w, err.Error(), http.StatusInternalServerError)
		return
	}
	defer conn.Close()

	// Close the connection immediately
	conn.Close()

	fmt.Println("Connection closed by server")
}

func main() {
	http.HandleFunc("/", handler)

	// Configure TLS with TLS 1.2
	tlsConfig := &tls.Config{
		MinVersion: tls.VersionTLS12,
	}

	server := &http.Server{
		Addr:      ":8080",
		Handler:   nil, // Default handler is used (Mux if not set)
		TLSConfig: tlsConfig,
	}

	// Start the HTTPS server with the specified TLS configuration
	err := server.ListenAndServeTLS("cert/server.crt", "cert/server.key")
	if err != nil {
		fmt.Println("Error starting server:", err)
		os.Exit(1)
	}
}

client: Just 1 request in loop untill success (infinite, as server never ACK): http-put "1MB" data to server.

configs:
- http1 only
- disable pooling for simplicity: max_idle per pool=0
driving_client.rs
to add...

Client is always getting connection reset by server. This is leading to two types of error response in hyper:
(A) hyper::Error(BodyWrite, Os { code: 104, kind: ConnectionReset, message: \"Connection reset by peer\" }))
(B) hyper::Error(IncompleteMessage)
where (A) does not seems to cause memory leakage, while (B) seems to.

I observed that:

in high bandwidth scenario: (A) is dominating.
in low bandwidth scenario: (B) is dominating

When I run my tests (15mins):

In high bandwidth (around 5MBps),

memory was constant: no leakage.
Error A count: 5242
Error B count: 1
As per hyper logs: total 315MB flushed.

In low bandwidth scenario (100KBps),

memory kept increasing continuously linearly. Around 40MB of leakage observed.
Error A count: 0
Error B count: 5004
As per hyper logs: total 80MB flushed. The timeseries graph of data flushed is parallel to memory increase. Memory leak seems to be is 50% of whatever flushed. (just trying to connect random dots...)

see memory growth of case 1 vs case2:

It seems something is happening in hyper & not because of my-driving code, I confirmed it as: I changed error to connection failure by enabling SSL verification
hyper::Error(Connect, SSL(Error { code: ErrorCode(1), cause: Some(Ssl(ErrorStack([Error { code: 167772294, library: \"SSL routines\", function: \"tls_post_process_server_certificate\", reason: \"certificate verify failed\", file: \"../ssl/statem/statem_clnt.c\", line: 1883 }]))) }))  and it kept retrying. memory was constant: no leakage. It seems such leakage only when some data flushing happens.

Note:

malloc_trim calls seems to clean the leakage to great extent.

This was I tried to reproduce locally on my laptop where for just 1MB request, in few minutes 100MB of leakage was observed (where graph is continuously increasing). I have seen similar scenario in production, where it lead to around 6GB of memory leakage, which is what the main concern is.

See in above graph, where the memory shooted to 6GB for our app (ideally our app consumes 1 to 1.4GB max at a time). On debugging, I found we received 157310 instances of 503 errors from S3 (rate-limit by them). our each request was around 28KB and when I plotted timeseries graph of error blocks size accumulated, the growth and curvature of graph was completely matching.

Note that sudden dip of 1GB in memory at 15:54:30 is due to malloc_trim call. Above run was on EC2,  where malloc_trim cleared only 1GB out of leaked 5GB, while on local runs on ubunut laptop malloc_trim seems to clear all leakage due to error blocks.

I am still in process of understanding the hyper workflow. Pardon me, if any of below comments are wrong:

Overall observation is: when there is connection reset by server while some data is already flushed by hyper client, there is possibilty of memory leakage based on some racing condition of when/how hyper gets to know about conncetion closure.

like Error A, was received by hyper during poll_write, which make AsyncWrite call to n/w: tokio_openssl::SslStream:poll_write.
Where os informed about connection reset. This case does not seems to be causing memory/resource leak.

logs
2886-ts=2024-03-22T16:48:48.357019727Z TRACE  hyper::client::client: handshake complete, spawning background dispatcher task
2887-ts=2024-03-22T16:48:48.357207290Z TRACE hyper::proto::h1::conn: flushed({role=client}): State { reading: Init, writing: Init, keep_alive: Busy }
2888-ts=2024-03-22T16:48:48.357365572Z TRACE encode_headers: hyper::proto::h1::role: Client::encode method=PUT, body=Some(Known(1049396))
2889-ts=2024-03-22T16:48:48.357446885Z TRACE hyper::proto::h1::io: buffer.flatten self.len=439 buf.len=1049396
2891-ts=2024-03-22T16:48:48.357622526Z DEBUG hyper::proto::h1::io: flushed 16384 bytes
2893-ts=2024-03-22T16:48:48.357666519Z DEBUG hyper::proto::h1::io: flushed 16384 bytes
2895-ts=2024-03-22T16:48:48.357705690Z DEBUG hyper::proto::h1::io: flushed 16384 bytes
2897-ts=2024-03-22T16:48:48.357748504Z DEBUG hyper::proto::h1::io: flushed 16384 bytes
2898-ts=2024-03-22T16:48:48.357776174Z DEBUG hyper::proto::h1::dispatch: error writing: Connection reset by peer (os error 104)

while Error (B) was received by hyper during poll_read, which makes AsyncRead call to n/w: tokio_openssl::SslStream:poll_read. Where in 0 bytes response indicates that connection closed while some message was expected. This case seems to be causing memory/resouce leak. It also errored on shutdown as: error shutting down IO: Transport endpoint is not connected (os error 107)

logs
1928-ts=2024-03-22T16:48:46.518283727Z TRACE hyper::client::conn: client handshake Http1
1929-ts=2024-03-22T16:48:46.518358157Z TRACE hyper::client::client: handshake complete, spawning background dispatcher task
1930-ts=2024-03-22T16:48:46.518559721Z TRACE hyper::proto::h1::conn: flushed({role=client}): State { reading: Init, writing: Init, keep_alive: Busy }
1931-ts=2024-03-22T16:48:46.518703546Z TRACE encode_headers: hyper::proto::h1::role: Client::encode method=PUT, body=Some(Known(1049396))
1932-ts=2024-03-22T16:48:46.518768090Z TRACE hyper::proto::h1::io: buffer.flatten self.len=439 buf.len=1049396
1934-ts=2024-03-22T16:48:46.518939168Z DEBUG hyper::proto::h1::io: flushed 16384 bytes
1936-ts=2024-03-22T16:48:46.518991601Z DEBUG hyper::proto::h1::io: flushed 16384 bytes
1938-ts=2024-03-22T16:48:46.519025458Z DEBUG hyper::proto::h1::io: flushed 16384 bytes
1940-ts=2024-03-22T16:48:46.519061873Z DEBUG hyper::proto::h1::io: flushed 16384 bytes
1942-ts=2024-03-22T16:48:46.519115645Z DEBUG hyper::proto::h1::io: flushed 16384 bytes
1943-ts=2024-03-22T16:48:46.519165894Z TRACE hyper::proto::h1::conn: Conn::read_head
1944-ts=2024-03-22T16:48:46.519198015Z TRACE hyper::proto::h1::io: received 0 bytes
1945-ts=2024-03-22T16:48:46.519209421Z TRACE hyper::proto::h1::io: parse eof
1946-ts=2024-03-22T16:48:46.519222649Z TRACE hyper::proto::h1::conn: State::close_read()
1947-ts=2024-03-22T16:48:46.519231806Z DEBUG hyper::proto::h1::conn: parse error (connection closed before message completed) with 0 bytes
1948-ts=2024-03-22T16:48:46.519240215Z DEBUG hyper::proto::h1::dispatch: read_head error: connection closed before message completed
1949-ts=2024-03-22T16:48:46.519262154Z TRACE hyper::proto::h1::conn: State::close_read()
1950-ts=2024-03-22T16:48:46.519271345Z TRACE hyper::proto::h1::conn: State::close_write()
1951-ts=2024-03-22T16:48:46.519295766Z DEBUG hyper::proto::h1::conn: error shutting down IO: Transport endpoint is not connected (os error 107)
1952-ts=2024-03-22T16:48:46.519380733Z DEBUG hyper::client::client: client connection error: error shutting down connection: Transport endpoint

the buffered data on closed connection is causing leakage some where in some scenarios (incomplete message is one of them). I am still in process of figuring out how it cleans on closed connection.

As the errored blocks seems to get accumulated in memory and is unbounded, this issue can cause serious troubles. As an Application will keep consuming unbounded memory (in error scenarios) may suddenly crash.

In case of hyper::Error(IncompleteMessage) , as was seeing shutdown-error as: error shutting down IO: Transport endpoint is not connected (os error 107). So I tried disabling shutdown, as below:
impl AsyncWrite for MyStream {
    ...
    fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), std::io::Error>> {
        // let this = self.project();
        // this.io.poll_shutdown(cx)
        Poll::Ready(Ok(()))
    }
}

This magically made memory graph horizontal, no leakage. No clue why this is happening.

Also, from your last comment, it seems like it has something to do with the io you're using. Is it possible that the implementation of io in your code is causing the leakage?

Hi,
We are using hyper v0.14.26 and notice memory leak with a similar pattern in our services. Just want to do a +1 and check if there are any new updates related to this issue


Support more than one IP per interface and IPv6 for results returned by CNI

Our use case is to:

call SetupNetworkHandler ourselves to get the network configuration generated using a CNI plugin
remove  firecracker.SetupNetworkHandlerName and firecracker.SetupKernelArgsHandlerName from the list of default FcInit handlers
use our own VM network interface configuration method instead of the default boot param based one
call machine.Start

With that in mind we want to be able to have more than one IP on interfaces and to support IPv6, which is what this change intends on achieving while keeping backwards compatibility.

@vvejell1 can you please help review this PR when you have a moment? Thank you.

Hi @th0m , looks like buildkite tests (example) have not run:

Missing DCO on d88daf38557b97d2922471b123748fd6aa2480cc

could you please sign your commit ? if you've already got your name and email configured in git, you can git commit --amend -s, then force push to this branch to update the PR

@ginglis13 I am getting build errors that seem unrelated to my change https://buildkite.com/firecracker-microvm/firecracker-go-sdk/builds/3007#01869a5d-b0c1-4d19-ad6f-156ddcc120f3/17-1015

@ginglis13 can you please have a look? Thanks

@th0m we've got an infrastructure bug we're addressing. As soon as it's resolved we can re-run and continue the review

@th0m  I have a fix on CI failure merged into main. You can rebase, push and trigger the build again.

Thanks, build failures are on me now, I will fix them.

Ready for review @fangn2 thank you.

@ginglis13 I believe this is ready to go.


[staging]pythonPackages.docker: 4.0.2 -> 4.10

Motivation for this change
noticed it was out of date on repology.
ordered propagatedBuildInputs
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[x] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Notify maintainers
cc @

changed base branch to staging as it was >100 rebuilds

master is OK. 100 is not too much, especially in case of python packages.

@GrahamcOfBorg eval


IEDeleteObjectStoreBug still present on Windows Edge Browser

The Delete bug seems still to be present on windows 10 browsers (at least Edge/13 on windows phone 10). i needed to adjust the bug detection function in dexie.js to:
function hasIEDeleteObjectStoreBug() {
            // Assume bug is present in IE10 and IE11 but dont expect it in next version of IE (IE12)
            return navigator.userAgent.indexOf("Trident") >= 0 ||
              navigator.userAgent.indexOf("MSIE") >= 0 ||
              navigator.userAgent.indexOf("Edge") >= 0;
        }

Ok, I've updated the code in the develop branch. Will be part of next release 1.3.5.

Part of 1.3.6-beta.1.


Wrong end time

Hello and thanks for this integration i just discovered.
I made a small configuration :
- platform: schedule_state
  name: Consignes repos
  refresh: 0:15:00
  default_state: "{{ states('input_number.consigne_eco')}}"
  events:
    - start: "7:00"
      end: "8:30"
      state: "{{states('input_number.consigne_confort')}}"
      condition:
        - condition: state
          entity_id: binary_sensor.jour_semaine
          state: "on"
    - start: "19:30"
      end: "22:00"
      state: "{{states('input_number.consigne_confort')}}"
      condition:
        - condition: state
          entity_id: binary_sensor.jour_semaine
          state: "on"
    - start: "7:30"
      end: "9:30"
      state: "{{states('input_number.consigne_confort')}}"
      condition:
        condition: time
        weekday:
          - sat
    - start: "20:00"
      end: "22:30"
      state: "{{states('input_number.consigne_confort')}}"
      condition:
        condition: time
        weekday:
          - sat
    - start: "8:30"
      end: "10:00"
      state: "{{states('input_number.consigne_confort')}}"
      condition:
        condition: time
        weekday:
          - sun
    - start: "20:00"
      end: "22:30"
      state: "{{states('input_number.consigne_confort')}}"
      condition:
        condition: time
        weekday:
          - sat
          - sun

Sensor is created as is :

But as next change should occurs tomorrow (it sunday 11:00 PM atm), i was expecting end time should have monday start time (and not current day end time)
I'm wrong ?
Regards

I guess you mean that you are expecting "end" to be 7:00 due to the first event.
There is a limitation that all rules/conditions/etc are evaluated based on the current date and time. On Monday at midnight, everything will be re-evaluated then.

Exactly. I saw after midnight, value was "correct"

OK. There is no obvious way to change this behavior.


VideoBridge issue when Source is temporary not used

Describe the bug
I try to directly bind the MediaSource to an VideoPlayerElement:
_localVideoSource = CreateVideoStreamSource(
                            width, height, framerate);
                        _localMediaSource = MediaSource.CreateFromMediaStreamSource(_localVideoSource);

<MediaPlayerElement AutoPlay="True" Source="{x:Bind LocalMediaPlaybackSource}"
							                    HorizontalContentAlignment="Stretch" HorizontalAlignment="Stretch" Stretch="Uniform"
							                    x:Name="LocalMediaPlayerElement"/>

The usecase is to have a contact list on the left, and a video view on the right. An ongoing call with a contact will show the binded source. If I click on a different user while being in a call the video should not be shown (source is null). If I switch back the video should start playing again, but it does not. You can see that OnMediaStreamSourceRequested is requested 3 times (the number of the videobridge queue) and then no more call is executed. It will take a while and the black video player eventually returns to the original video and execution continues normally.
I guess this is a bug in the videobridge or an underlying implementation. It seems that if the input is larger than the output (in this case local video is being captured, but nobody listens to the stream and dequeues the frames) the videobridge somehow gets stuck. Or maybe the underlying videostream, I dont know.
I can also imagine that args.Request.SetActualStartPosition(TimeSpan.Zero); relates to the issue. The longer I wait, the longer is the video black. It seems the stream remembers it total play time and when resuming the stream I have to wait until that time was reached.
The workaround here is to create a MediaPlayer and let it play in the background. You have to manually set the mediaplayer to the mediaplayerelement then. I wish there was a better solution.
Environment
Please fill the information for each peer if different

Platform:UWP
Architecture: x64
Target device: Windows Desktop
Release 1.0.3

Hi @AtosNicoS,
Yes I have made most of the same observations as you. This bug has been on-going for a year, and we have not been able to pin-point the root cause. The video bridge queue is only 5 frames long, so even if it would be replayed incorrectly you would barely notice it at 30 or 60 FPS. In my experience the long wait is due to the difference between the action to resume and video bridge receiving frames on one hand, and the Media Playback pipeline actually sending its first OnMediaStreamSourceRequested on the other hand, the latter sometimes happening several seconds or tens of seconds after the former. And since this is an event raised by the internal UWP Media Playback API, there is no immediate action we can take to fix or work around.
Now I have no idea why this delay occurs. I partly share the concern this might be related to SetActualStartPosition(), but since we reset the media source (we create a new one and assign to the media player) I don't see why the media player would think that this is the same media. Looking at the documentation it is really not clear what the "media time-line" is, although it seems to be application dependent ("chosen by the application"), so I am interpreting that as "I can set 0 as the start of the media time-line of the new media source I just assigned to the media player". Maybe this is wrong?
We also have a possibly related bug where the media player doesn't restart at all, and OnMediaStreamSourceRequested is not called no matter how long you wait. Again, not sure if this is related to that issue or not.
I have tried to reach out internally to the Media Foundation team, which I think owns this API, to clarify some of those questions and get some help, which I think is the same issue as #289, although the author there was happy with a workaround so closed that issue. Hopefully we will get some insights and be able to figure out what is going on, as unfortunately I have not been able to find a sample which demonstrates how to hot-swap a media source on a same media player.


don't free window icon before it's used

It seems to me like that lovrRelease will delete textureData->blob immediately,
which means the windowing system later can't use it because it's already freed.
There's already a free on line 378 which looks more correct.
Also, icon appears flipped if 'flipped' is set to true here on Linux. Is GLFW
inconsistent between linux and windows, or should it indeed be false?

Looks good to me, I can't imagine that flipping would be required because that's an OpenGL thing

DAMN you're fast :sweat_smile: thanks!!


Enable WriteCore feature for chaos

We need to enable WriteCore feature to chaos. Therefore add use-write-core: true and do a regen.

API change check
APIView has identified API level changes in this PR and created following API reviews.
Azure.ResourceManager.Chaos


Add roadmap

Use project wiki. The roadmap itself shouldn't be too detailed. Just key features: no long descriptions, no ETA, etc. The list should be formatted as checkboxes.

Done


Modify public api to simplify code

Task/Issue URL: https://app.asana.com/0/488551667048375/1205984446323518/f
Description
This PR changes the messaging public API
Steps to test this PR
See task

Current dependencies on/for this PR:

develop

PR #3895

PR #3903   👈

This stack of pull requests is managed by Graphite.


Fedora: handle ssh prompt with extra console chars

With fedora OS, the ssh login prompt has more extra brakceted console characters so make sure to include as wildcard
prompt:
^[]0;root@ltclp2:~^G^[[32mroot@ltcrain4-lp2^[[0m:^[[32m~^[[0m#^[[0m
error:
self.connect(logger)
File "/data/jenkins/workspace/sandbox/install/op-test/common/OpTestSSH.py", line 185, in connect
self.check_set_term()
File "/data/jenkins/workspace/sandbox/install/op-test/common/OpTestSSH.py", line 200, in check_set_term
self.util.setup_term(self.system, self.pty,
File "/", line 1709, in setup_term
raise ConsoleSettings(before=pty.before, after=pty.after,
common.Exceptions.ConsoleSettings: Setting the prompt or logging in for the console was not successful, check credentials and review the following for more details Expect Before Buffer="command-line line 0: Unsupported option "afstokenpassing"
Last login: Tue Aug 13 03:11:16 2024
root@ltcrain4-lp2:~#
Expect After Buffer="<class 'pexpect.exceptions.EOF'>" Message="Getting login and sudo not successful, probably connection issue, retry"

fedora-ssh-prompt.txt


Typo fixing

Found a very small typo in 'JsonDeSerializer.TYPE_MAPPINGS'.
Thanks for the docs!

@system205 Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@system205 Thank you for signing the Contributor License Agreement!


generate CLI documentation

We should produce automatically generated documentation for all commands and CLI options.

Do we want to use something like https://pypi.org/project/md-click/?
Or it is to be done without using external libraries?

If there are tools that will do it, that's fine.
We should consider how this would fit into an overall documentation build, too. I usually use Sphinx for python libs so I can publish on readthedocs.org, but that's not a hard requirement. If we don't publish there we need to figure out how we will publish.

Okay, I will research a bit more then


server-chart: removing tls=external option breaks upgrade.

Rancher versions:
rancher/rancher: 2.1.0
rancher/server-chart: 2.1.0
We removed the option to do TLS offload exclusively on an external load balancer.
Removing this option doesn't stop you from having a cert on your lb, you just need to forward your traffic to the https endpoint on the ingress instead of port 80.
We removed the option since there was no consistent way to describe configuring external load balances, too many variations. Also exposing port 80 left the possibility of connecting unencrypted to rancher.
The problem is removing this option will break upgrades for people who have installed with that option.  No error is provided when you upgrade, but http->https redirect will be turned on and no certificate will be configured on the ingress.
See rancher/server-chart#30 for PR to add the option back in.
See rancher/docs#871 for new docs warning of the deprecation of that option and instructions on how to convert to a supported tls configuration.

Well this is good to know...this is definitely effecting me.

Closing in favor of https://github.com/rancher/rancher/issues/16030

Available in the master branch of https://github.com/rancher/server-chart

rancher/rancher:v2.1.0
rancher/server-chart:master
3 node HA cluster w/ ssl termination at nginx lb, trusted certs, --tls=external chart option
Upgrade path 1
v2.0.8 -> v2.1.0 -> v2.1.0 (master chart)

Created a cluster for each version during upgrade path, with workloads and ingress.
Rancher UI is accessible at each upgrade
New cluster creation with workloads and ingress is successful.
nginx.ingress.kubernetes.io/ssl-redirect: false is present in deployment

helm install rancher-stable/rancher --version v2.0.8 \
  --name rancher \
  --namespace cattle-system \
  --set hostname=<host> \
  --set tls=external

helm upgrade rancher rancher-stable/rancher \
  --set hostname=<host> \
  --set tls=external

helm upgrade rancher ./ \ # local rancher/server-chart:master
  --set hostname=<host> \
  --set tls=external

Upgrade path 2
v2.0.8 -> v2.1.0 (master chart)

Created a cluster in 2.0.8, with workloads and ingress.
Upgraded to 2.1.0 from master chart. Existing cluster continued to function.
Creating a cluster in 2.1.0 with workloads and ingress is successful.
nginx.ingress.kubernetes.io/ssl-redirect: false is present in deployment

helm install rancher-stable/rancher --version v2.0.8 \
  --name rancher \
  --namespace cattle-system \
  --set hostname=<host> \
  --set tls=external

  helm upgrade rancher ./ \ # local rancher/server-chart:master
  --set hostname=<host> \
  --set tls=external


Adjusting default timeout

The PR sets the default timout to a large value, which is consistent with the requirements of Kontrol engagement proofs.

Why not set this value as default at the Kontrol level?

In a Slack conversation, Petar has pointed out that the default timeout of 125 ms is unconditionally used to check the SMT prelude. That was not enough with some custom options passed to z3 via --smt-option.
We should use the user-specified timeout when checking the prelude.

In a Slack conversation, Petar has pointed out that the default timeout of 125 ms is unconditionally used to check the SMT prelude. That was not enough with some custom options passed to z3 via --smt-option.
We should use the user-specified timeout when checking the prelude.

Will open a quick PR, as this is something I noticed last week whilst refactoring the SMT code and was going to address in a separate PR

closing in favour of #4030


Replace ESLint with Biome

Replace Eslint with biome. In testing with internal projects, biome is considerably faster than eslint (300ms vs 19s).

@corsin-ragettli I have patched the rules that were a bit problematic. What I have also noticed is that there are no more tests for the ts linters. I'd recommend to add them back again to avoid side effects and unwanted surprises once this package gets published (for example nursery rules)

You dropped tests/eslint-typescript.test.js. These tests should be re-added back in


fix: Preserve logging config if specified.

The logging configuration option was always
replaced by the default value when set to false.
I have to say though that I'm not even sure why this line
was needed, since the defaults are already copied from the
default_config when instantiating the normalized config.

You're right. It wasn't needed and it was my mistake, so thank you for fixing it! :smile:


write_canonical/1 output validation

v2.14.17
?- X = (\+Y), write_canonical(X).
\+(_1)   X = (\+Y).
?- X = (\+Y), write_canonical(\+X).
\+(\+(_1))   X = (\+Y).

?- X = (\+X), write_canonical(X).
\+(X)   X = (\+X).
?- X = (\+X), write_canonical(\+X).
\+(X)   X = (\+X).  % How about \+(\+(X)) X = (\+X). ?

Postponed

And fixed in the meantime(which is a magic fixroom).
?- X = (+X), write_canonical(+X).
+(+(X))   X = (+X).


[TRI-1798] Events list – show all events that have been received

An Events page, similar to the Runs page, that would show a paginated table with all events.
Info included for each event

Event name
Received time
Delivery time (can be in the future)
If it's been delivered yet
Number of associated runs

You should be able to click through to view the event payload and then click through to any associated runs.
TRI-1798

@matt-aitken, Can you assign this to me?

@Kritik-J , I already started working on this and notified about that in Discord. I have made the listings page. just the details page is remaining. I can push this and If you are okay, co-author that with me. You can go ahead and complete the details page.


Appveyor test discovery errors

See https://ci.appveyor.com/project/cmutel/brightway.
Works on default backend.

No longer valid


[1/2] Support agent integrations

I received this question from someone trying to use this release:

I have tried to monitor processes but it didn't work. Have you tried? Have you tried other extensions?
I have enabled examples in AGENT_DIR/conf.d but I'm getting python errors. I have added the python_dev package but still the same error.

Looking into this I discovered that:

I made the wrong assumption about the optional pip dependencies. I assumed that since the upstream install script ignores failures, I should too. These optional dependencies are actually required for some of the integrations, as documented in requirements-opt.txt.
pip dependencies that have native extensions couldn't compile because the python2.7 include directory was not directly on the INCLUDE_PATH. Also, the trailing : caused the parser to include an extra -L option, breaking the compilation.

This PR supports installation of optional dependencies and adds a property to configure them.
instance_groups:
- name: myservice
  jobs:
  - name: datadog-agent
    release: datadog-agent
    properties:
      integrations:
        process:
          init_config:
            pid_cache_duration: 30
          instances:
          - name: myservice
            search_string: ["myservice"]
            thresholds:
              critical: [1, 9]

🉑


Address CI failures by updating apt-get cache before installing packages

See https://github.com/actions/runner-images/issues/7452 for more
explanation.
Signed-off-by: Jim Ramsay<EMAIL_ADDRESS>SUMMARY
As described in, https://github.com/actions/runner-images/issues/7452, it's
important to apt-get update before installing packages to make sure we get
up-to-date mirror and cache information.
Fixes #493
ISSUE TYPE

Bug, Docs Fix or other nominal change

Tests

GitHub PR test only

Test-Hint: no-check

retest


Vulkan support broken

Compiling saiga with Vulkan support and then linking a program against saiga raises the following errors:
1st: cmake error:
Target "agphys" links to target "Vulkan::Vulkan" but the target was not
found.  Perhaps a find_package() call is missing for an IMPORTED target, or
an ALIAS target is missing?
2nd: linker error:
/usr/bin/ld: cannot find -lVulkan::Vulkan
version used: a42470a7a2d5959d335265ac69d253e411eaebe7
A current workaround is to disable Vulkan when compiling saiga.

I could not recreate this problem on my system. While looking for a problem I did some cleanup on the install code. Can you please test it again with 5c1253a8509a20b956f9a4560db96e2a94e05f08 on the experimental branch?

Unfortunately the problem is still present.
These are the generated cmake files after running make install:
saigacmake.tar.gz
Here is the build config, in case that helps:
saigaheaders.tar.gz
What's suspicious is in the generated "SaigaTargets.cmake" line 65 where it says "Vulkan::Vulkan". That obviously is no valid library name to link against. I have to idea what's causing it, though.

It should be fixed now in 01f19f8cd7b7fc71a026873b8dacf365b7bde518.
The problem was that saiga_cuda had a cmake dependency to saiga_vulkan even though it didn't use it.

Works, thank you very much!


[BUG] SkiaSharp Managed-Only Building does not work, because there is a mix of .NET 7 and .NET 6 assembly references

Description
I tried building SkiaSharp according to these instructions (Managed-Only Building): https://github.com/mono/SkiaSharp/wiki/Building-SkiaSharp
but I get 4 build errors using dotnet cake --target=everything --skipExternals=all and VS2022 17.7 Preview 4.0.
Example:
"C:\repos\mono\SkiaSharp\source\SkiaSharpSource.Windows-net.slnf" (Build target) (1:2) ->
       "C:\repos\mono\SkiaSharp\binding\SkiaSharp\SkiaSharp.csproj" (default target) (2:59) ->
       "C:\repos\mono\SkiaSharp\binding\SkiaSharp\SkiaSharp.csproj" (Build target) (2:66) ->
       (CoreCompile target) ->
         CSC : error CS1705: Assembly 'Microsoft.iOS' with identity 'Microsoft.iOS, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=84e04ff9cfb79065' uses 'System.Runtime, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' which has a higher version than referenced assembly 'System.Runtime' with identity 'System.Runtime, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' [C:\repos\mono\SkiaSharp\binding\SkiaSharp\SkiaSharp.csproj::TargetFramework=net6.0-ios]

Based on dotnet cake output, CoreCompile is indeed using /reference:"C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\6.0.20\ref\net6.0\System.Runtime.dll"
Which is clearly net6.0.
And this is used for Microsoft.iOS: /reference:C:\repos\mono\SkiaSharp\externals\package_cache\microsoft.ios.ref\16.4.7090\ref\net7.0\Microsoft.iOS.dll
Which is clearly net7.0.
Expected Behavior
SkiaSharp will compile using dotnet cake.
Actual Behavior
SkiaSharp does not compile using dotnet cake.
Basic Information

Version with issue:  2.88.4-preview, latest code on 24th July 2023. Windows 11.
Last known good version:  ?
IDE:  Visual Studio 2022 17.7 Preview 4.0 (installed) but I'm using dotnet cake from the command line prompt.

Any way to fix this issue?
Also, it is strange that the target framework is xamarinios1.0. I cannot find it here: https://learn.microsoft.com/en-us/dotnet/standard/frameworks.

This should no longer be an issue as the main branch now is almost entirely net7.0+. There are a few netstandards and net472. But, now you can use dotnet build everywhere too.


Primary & secondary vars not available during compile time

The color-mod function requires the --primary & --secondary CSS vars to be present during compile time which fails under the current configuration rendering any color-mod defined colours unavailable at the moment.
A couple of different options are available:

Find a way to inject variables during the compilation process and require each project using this plugin to recompile the styles for themselves
Perform colour modifications in PHP and inject them as root CSS variables like we're doing now with primary & secondary and then refer to those variables in the tailwind.config.js file. (This is my preferred solution).

@joseph-sm @jaxwilko thoughts?

We could modify the mix config generator to allow for custom vars to be passed into webpack, which should allow for those vars to be used at compile time. It may get a little bit messy though

Yeah, still not a fan of that approach, especially when you factor in that the branding settings could be changed dynamically per request, especially in multi-tenant projects. You'd have to compile the assets once per tenant and switch between them, sounds like a nightmare.

Fixed by https://github.com/wintercms/wn-tailwindui-plugin/commit/9604d918c5b280d73df15c1c0e5ffdb8f4134190


Adding respective icons to the navbar

Description
COULD YOU PLZ ASSIGN THIS ISSUE TO ME UNDER GSSOC'24
Is your feature request related to a problem? Please describe.
A clear and concise description of the problem or limitation that the requested feature would address.
The current navbar lacks icons, which impacts its visual appeal and usability. Users might find it less engaging and intuitive to navigate through the website without visual cues provided by icons
Describe the solution you'd like
A clear and concise description of the feature or enhancement you would like to see implemented.
I propose adding respective icons to the navbar to enhance its visual appeal and improve user navigation experience. These icons will align with the theme of the website and provide users with clear visual cues for each navigation option
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
One alternative could be redesigning the navbar layout or color scheme. However, adding icons is a more direct and effective solution to address the visual appeal and usability concerns of the navbar
Additional Information

Context
Provide any additional context or information that helps explain the feature request.
Potential Implementation (optional)
If you have any ideas or suggestions on how the feature could be implemented, please share them here.
Screenshots or Mockups (optional)
If applicable, include any screenshots, mockups, or visual representations of the requested feature.
Priority (optional)
Please indicate the priority level of this feature request (e.g., High, Medium, Low).
Additional Notes (optional)
Add any other notes or comments about the feature request here.
COULD YOU PLZ ASSIGN THIS ISSUE TO ME UNDER GSSOC'24

I would lilke to contribute upon this issue as I've made some good interacting navbars in my personal projects.
I'll try to make this navbar as user friendly as it can be and come up with new navbar. So, could you please assign me for this issure? Waiting for your response.
Thankyou!!!!

hey its not required currently


Fixing bug in Actors where it cannot deactivate actors with UUID as id.

Description
Fixing issue where Actor with Id as UUID cannot be deactivated because parsing uses "-" as separator.
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #895
Checklist
Please make sure you've  completed the relevant tasks for this PR, out of the following list:

[X] Code compiles correctly
[ ] Created/updated tests
[X] Unit tests passing
[?] End-to-end tests passing
[ ] Extended the documentation
[ ] Specification has been updated
[ ] Provided sample for the feature

/ok-to-test

Should constructActorStateKey be changed as well?

Currently, we dont split the state keys & reminder keys, we create the keys and use them to fetch/update their values. But, yes, it would be good to change them as well. it would make the code immune to future changes if we ever split based on delimiter in code.

/ok-to-test

/ok-to-test

/ok-to-test


web console services view display indexer task information

Description
This PR updates the web console 'services' view to show the indexer service type task slot and last completed task information.
before:

after:

This PR has:

[x] been self-reviewed.
[x] been tested in a test Druid cluster.

This looks good, will merge after CI


Client Allocation Request for: 上海源信网络科技有限公司

Client Allocation Questions
Core Information

Name: 上海源信网络科技有限公司
Website / Social Media: https://www.yuanxinw.com
DataCap Requested: 5TiB
Addresses to be Notarized: f13wqjh7vhgnjtg3kaprzketenqpofcjyng75mk3i
Notary Requested: NeoGe-IPFSMain

Hey @yuanxinw-network, thanks for the request. I have some questions regarding your request. Please describe answers as much specificity as you can.

Can you introduce yourself? Where can I find you online? (Github, Twitter, Linkedin handle, Wechat, etc.)
If you represent an organization, what is the name, nature, registration country, time since inception, and named officers of your organization?
In which region(s) do you intend to use the DataCap?
What is your use case for the DataCap being requested?
Do you plan to specify advanced features when utilizing a DataCap in your storage deal? If so, please specify the feature.
How many miners do you plan on allocating the DataCap? If you have a specific miner(s) you would like to allocate a DataCap, please specify.
Do you agree on providing transparency of DataCap allocation decisions & execution details in public to prevent excessive/unfair self-dealing?

Before starting to answer questions, I encourage you to take a look of our allocation strategy here which shows how we utilize the assessment.
Thanks.

Hey @yuanxinw-network, thanks for the request. I have some questions regarding your request. Please describe answers as much specificity as you can.

Can you introduce yourself? Where can I find you online? (Github, Twitter, Linkedin handle, Wechat, etc.)
If you represent an organization, what is the name, nature, registration country, time since inception, and named officers of your organization?
In which region(s) do you intend to use the DataCap?
What is your use case for the DataCap being requested?
Do you plan to specify advanced features when utilizing a DataCap in your storage deal? If so, please specify the feature.
How many miners do you plan on allocating the DataCap? If you have a specific miner(s) you would like to allocate a DataCap, please specify.
Do you agree on providing transparency of DataCap allocation decisions & execution details in public to prevent excessive/unfair self-dealing?

Before starting to answer questions, I encourage you to take a look of our allocation strategy here which shows how we utilize the assessment.
Thanks.

Hey, thanks you for your reply. My answers are as following.
Q1: Can you introduce yourself? Where can I find you online? (Github, Twitter, Linkedin handle, Wechat, etc.)
A: This is Yuanxin network found in 2018 which provide network data access and cloud computing platform service.
About YUANXIN:
Official website：https://www.yuanxinw.com/
Github：https://github.com/yuanxinw-network
Q2: If you represent an organization, what is the name, nature, registration country, time since inception, and named officers of your organization?
A: Our organization'name is yuanxin which is a private enterprise, Registered In Shanghai of China，started in 2018 and  corporate representative is chenjun.
Q3: In which region(s) do you intend to use the DataCap?
A: China
Q4: What is your use case for the DataCap being requested?
A: we hope to help people store the the private data including video, graph and some private business data etc.
Q5: Do you plan to specify advanced features when utilizing a DataCap in your storage deal? If so, please specify the feature.
A: We haven't thought about it yet， but if the miners support the offline big data deal, We will give priority to it.
Q6: How many miners do you plan on allocating the DataCap? If you have a specific miner(s) you would like to allocate a DataCap, please specify.
A: We plan 5-10 miners. We have no specific miner yet.
Q7: Do you agree on providing transparency of DataCap allocation decisions & execution details in public to prevent excessive/unfair self-dealing?
A: Yes. info including: deal_id, miner_id, address, dataCID, date etc.

Hey @yuanxinw-network, sorry for the delays during the holiday season, future requests would be processed at a faster pace. My allocation strategy is to build trust progressively. So I could allocate 10 GiB to you for the first time. Feel free to request more DataCap when you use up the allocation.
To better understand how future allocation looks, please take a look here for my allocation process.

Request Approved
Your Datacap Allocation Request has been approved by the Notary
Message sent to Filecoin Network

bafy2bzaceask66w4jjic3uvqsa32rpfrhe7bgedyqx5gmelbb3le6zpzid54y

Address

f13wqjh7vhgnjtg3kaprzketenqpofcjyng75mk3i

Datacap Allocated

10 GiB

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceask66w4jjic3uvqsa32rpfrhe7bgedyqx5gmelbb3le6zpzid54y


support editing data bag item in chef 13

fixes https://github.com/cparedes/knife-file/issues/7

LGTM! Thanks for your contribution!


Cherry pick - Chef - Add Chef CD builds for linux_arm64_ipv6only (#20468)

Problem

Raspberry pi has no compatible chef builds produced in CD

Change overview

2022-07-08  Chef - Add Chef CD builds for linux_arm64_ipv6only (#20468)

Testing

Tested with build all in vscode container:
./chef.py --build_all

@woody-apple @cpagravel my view of the TE9 interop branch is that it is optional for anyone, it is basically a interoperable branch that people can use (am aware of some companies syncing it together).
It is not an "official branch" but more testing, so I have nothing against putting more things in it. That being said, now that we have sve and heading towards 1.0, we should work towards deprecating its use. Working on cherrypicks on it takes up CI time (since whole CI runs on the CP and after merge) and some review time (even if trivial).


chore: Update issue stale-out to use triage flow

Fixes #N/A
Description
This change updates the issue stale-bot to change our triage flow so that we have the following flow for removing issues quicker with the bot:

Issues that are marked as triage/needs-information but are not updated within 14d are automatically staled. These are then automatically closed within 14d.
Issues that are marked as triage/solved but are not updated within 7d are automatically staled. These are then automatically closed within 7d.

How was this change tested?
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

/lgtm

Pull Request Test Coverage Report for Build<PHONE_NUMBER>5
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0   changed or added relevant lines in 0 files are covered.
11 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-0.1%) to 80.609%

Files with Coverage Reduction
New Missed Lines
%

pkg/test/cachesyncingclient.go
2
82.29%

pkg/utils/termination/termination.go
4
87.18%

pkg/controllers/node/termination/controller.go
5
69.07%

Totals

Change from base Build<PHONE_NUMBER>3:
-0.1%

Covered Lines:
8917

Relevant Lines:
11062

💛  - Coveralls


TestContainers fails to detect successful container run and exit when a container is short-lived

TestContainers fails to detect successful container run and exit when a container is short-lived.
This happens when a container starts and exits before TestCointainers first check if container is started. For example, official Docker hello-world image fails on simple TestContainers test:
@Testcontainers
class HelloWorldTestContainersTest {

	@Container
	public GenericContainer<?> container = new GenericContainer<>("hello-world");

	@Test
	void containerIsStarted() {
		assertNotNull(container.getContainerId());
	}
}

It can be seen that hello-world image container has started successfully and exited, but TestCotainers fails in detecting it:

[main] ERROR 🐳 [hello-world:latest] - Log output from the failed container:

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

org.testcontainers.containers.ContainerLaunchException: Container startup failed

	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:336)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:317)
	at org.testcontainers.junit.jupiter.TestcontainersExtension$StoreAdapter.start(TestcontainersExtension.java:242)
	at org.testcontainers.junit.jupiter.TestcontainersExtension$StoreAdapter.access$200(TestcontainersExtension.java:229)
	at org.testcontainers.junit.jupiter.TestcontainersExtension.lambda$null$4(TestcontainersExtension.java:82)

...

Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:88)
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:329)
	... 88 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:525)
	at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:331)
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
	... 89 more
Caused by: org.rnorth.ducttape.TimeoutException: Timeout waiting for result with exception
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:54)
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilTrue(Unreliables.java:100)
	at org.testcontainers.containers.startupcheck.StartupCheckStrategy.waitUntilStartupSuccessful(StartupCheckStrategy.java:35)
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:460)
	... 91 more
Caused by: java.lang.RuntimeException: Not ready yet
	at org.rnorth.ducttape.unreliables.Unreliables.lambda$retryUntilTrue$1(Unreliables.java:102)
	at org.rnorth.ducttape.unreliables.Unreliables.lambda$retryUntilSuccess$0(Unreliables.java:43)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

Did some debugging. Seems that org.testcontainers.containers.wait.internal.InternalCommandPortListeningCheck fails with all 3 strategies in a container running in Rancher Desktop. Here are the outputs of all 3 strategies to detect internal listening port executed inside a redis container running in Rancher Desktop:
root@ca5ce0536c0f:/data# grep -i ':0*18eb' /proc/net/tcp*
root@ca5ce0536c0f:/data# nc -vz -w 1 localhost 6379
bash: nc: command not found
root@ca5ce0536c0f:/data# /bin/bash -c '</dev/tcp/localhost/6379'
/bin/bash: connect: Cannot assign requested address
/bin/bash: line 1: /dev/tcp/localhost/6379: Cannot assign requested address

I had this problem bad! The solution is to use the "One Shot" startup strategy, documented here:
https://www.testcontainers.org/features/startup_and_waits/#one-shot-startup-strategy-example
https://www.javadoc.io/doc/org.testcontainers/testcontainers/latest/org/testcontainers/containers/startupcheck/OneShotStartupCheckStrategy.html
like:
container.setStartupCheckStrategy( new OneShotStartupCheckStrategy() )

As @jennyowen shared, OneShotStartupCheckStrategy is indeed what should be used for one-off containers. The default expects containers to be running after they start (as this is the majority of containers started with Testcontainers), and that alternative strategy allows making it "one-off" friendly.


major update: remove deprecated, singular rule type

what

Remove Deprecated variables
Cleanup Dynamic rules which now only accept the list of objects (as opposed to list of map which was partially documented)
bugfix for enabled=false

why

Our module was growing increasingly complex trying to maintain backward compatibility and created two different structures for a backup plan.
Different data structures for implementing a plan caused bug reports and much confusion.

references

closes #48 - PR for Component incoming.
closes #56 - Disabled now properly works
closes #35 - New dynamic for advanced_backup_configuration
closes #46 - Incompatible types were created through complexity of backwards compatibility

/terratest

/terratest


Delegation support for named formula rules

This change adds ability for named formula rules to participate in pageability/delegable checks. This will provide support for extending pageable support for named formula rules. I will have the unit tests added on PaClient once I have that change ready for PR.

:heavy_check_mark: No Binary Compatibility issues for Microsoft.PowerFx.Connectors.dll
:heavy_check_mark: No Binary Compatibility issues for Microsoft.PowerFx.Core.dll
:heavy_check_mark: No Binary Compatibility issues for Microsoft.PowerFx.Interpreter.dll
:heavy_check_mark: No Binary Compatibility issues for Microsoft.PowerFx.LanguageServerProtocol.dll
:heavy_check_mark: No Binary Compatibility issues for Microsoft.PowerFx.Transport.Attributes.dll

:heavy_check_mark: No Binary Compatibility issues for Microsoft.PowerFx.Connectors.dll
:heavy_check_mark: No Binary Compatibility issues for Microsoft.PowerFx.Core.dll
:heavy_check_mark: No Binary Compatibility issues for Microsoft.PowerFx.Interpreter.dll
:heavy_check_mark: No Binary Compatibility issues for Microsoft.PowerFx.LanguageServerProtocol.dll
:heavy_check_mark: No Binary Compatibility issues for Microsoft.PowerFx.Transport.Attributes.dll

:heavy_check_mark: No Binary Compatibility issues for Microsoft.PowerFx.Connectors.dll
:heavy_check_mark: No Binary Compatibility issues for Microsoft.PowerFx.Core.dll
:heavy_check_mark: No Binary Compatibility issues for Microsoft.PowerFx.Interpreter.dll
:heavy_check_mark: No Binary Compatibility issues for Microsoft.PowerFx.LanguageServerProtocol.dll
:heavy_check_mark: No Binary Compatibility issues for Microsoft.PowerFx.Transport.Attributes.dll


OSError: No file matching Suv.svg in image directory

I've tried putting the .SVG image into */media/designs/svg_images. But when I want to quote it in the .py file it still reports errors:
Traceback (most recent call last):
  File "/home/jason/Documents/manim/manimlib/extract_scene.py", line 155, in main
    scene = SceneClass(**scene_kwargs)
  File "/home/jason/Documents/manim/manimlib/scene/scene.py", line 53, in __init__
    self.construct()
  File "SVGTEST.py", line 44, in construct
    height=height_size
  File "/home/jason/Documents/manim/manimlib/mobject/svg/svg_mobject.py", line 45, in __init__
    self.ensure_valid_file()
  File "/home/jason/Documents/manim/manimlib/mobject/svg/svg_mobject.py", line 63, in ensure_valid_file
    self.file_name)
OSError: No file matching MYSVG.svg in image directory

(Manjaro Linux, Texlive)

That is the older format, the new one is assets/svg_images/files.svg.


Learning using concentration is slow

I have tried using the Concentration node as bellow
from bayespy.nodes import Bernoulli, Beta, DirichletConcentration
from bayespy.nodes import Mixture
from numpy import array
from bayespy.inference import VB

C = DirichletConcentration(2)
C.initialize_from_value([0.5, 0.5])

P = Beta(C, plates=(3,), name='P')

#Creating the data
data = array([[0,True],[0,True],[0,True],[0,True],[1,True]])

#Model
Z = Mixture(data[:,0], Bernoulli, P)
Z.observe(data[:,1])
# Creating inference engine
Q = VB(Z,P,C)
Q.update(repeat=1000)
print(P.get_parameters())

But the learning is very slow . I stopped at 100 but there is nothing learned.
When I use without Concentration P = Beta([0.5,0.5], plates=(3,), name='P')  the iteration stops after 2 steps.
I have tried other models with concentration those run fast. Any idea?

What do you mean by "nothing learned"? I think it has learned some quite good values.
But yes, your example indeed doesn't converge in 1000 iterations, which seems weird at first because it's such a simple model. The issue here is that you have no False observations so the maximum likelihood estimation makes the second concentration parameter smaller and smaller, and the first concentration parameter larger in tiny steps, basically without any bounds, because the marginal loglikelihood is improved that way. This is one issue with ML estimates: without data properly constraining them, they tend to go to extremes. If you add at least one False observation, it converges.
One help against this would be if C was given regularization (a "prior"). I added simple support for that, and it's now used by default. Checkout the latest develop branch. You can modify it as Concentration(2, regularization=...) if you want, but I added some decent default regularization. If you're interested, I can give more details about it, but it's not very simple. Basically you give "prior" log-probability and "prior" number of samples.
However, even if it converges with one False observation, it still takes surprisingly long time. This is because C and P are learned in turns and they are highly coupled. A better ML estimation would use type-II approach where P is integrated out when optimizing C. I don't have support for that yet, but that's how I want to implement the general ML estimation.
I think learning is faster when you have more data AND when you have observations from all of your concentration categories (i.e., both False and True in your example). But the regularization should help too.

Thanks @jluttine  I was able to learn the values by adding some False values.
'regularization' I still need to do some more reading.
I have a supplementary question :
Because of the concentration parameter the learning is shared. So even though I have not observed 3rd random variable P[3] is changed from 0.5 to other values .
The change is a direct relationship between the the different P . Since the other values P[0] P[1] had increased, the value of P[3] also increased .
I have another problem which I need to model. Basically I want inverse relationship between different P.  So when the probability of P[0] increases the Probability of both P[1] and P[2] should reduce .
The example is an object can be in 3 different places . We observe that in 10 times we have not observed the object in Location 1 .  This should infer that the probability of Location 2 and 3 increases.
Is there a InverseConcentration  ? Or I am modelling something wrong?

If an object can be in 3 different places, and it's in one place at a time, I think you should use Categorical distribution. Three places, three categories. If your observations were such that you know that it is in location i at sample time t, then it'd be just a simple categorical observation, nothing special. So, I'd ask you to make sure this is not the case.
You describe the problem such that you observe that the object is NOT in location i at sample time t, and thus you know that it must be in one of the other locations. Did I understand correctly? This is a bit more complex case, but I wrote one possible solution for it:
import bayespy as bp
p = bp.nodes.Dirichlet([1, 1, 1])
locations = bp.nodes.Categorical(p, plates=(5,))
observations = bp.nodes.Mixture(
    locations,
    bp.nodes.Bernoulli,
    [
        [0.0, 0.0, 0.5],
        [0.0, 0.5, 0.5],
        [0.5, 0.5, 0.5],
        [0.0, 0.5, 0.5],
        [0.0, 0.5, 0.0],
    ]
)
observations.observe([True, True, True, True, True])
Q = bp.inference.VB(observations, locations, p)
Q.update(repeat=10)
print(p)
print(locations)

The array of 0.0 and 0.5 values is the trick: put 0.0 to those locations in which you observed the object is NOT in, and 0.5 to those locations where the object could be. The actual observation data is just a vector of trues. The trick is that with Bernoulli distribution, having p=0.0 and observing True gives you certainty that it's not there but on the other hand having p=0.5 and observing True gives you no information at all. This is just one way of accomplishing this, but it was the simplest method that quickly came to my mind. This trick can also be used to handle cases when you observe that the object is in a particular location: just leave that location to 0.5 and put the others to 0.0.
However, it could be that I just misunderstood your problem.
And by the way, you need to checkout the latest develop branch for my code example to work! I made a minor fix.

Thanks for the reply.
Yes your understanding of the problem is right  . I have made another simple example for better explanation .
We have a 6 faced dice and we need to learn the probability distribution of the 6 possible outcomes.
But the input data is tricky. After each trial the user asks Is the output 6(or any value randomly)?  and answer will be True or False .  The observed data will be a tuple <side, bool> Eg. [(6,False), (5, True), (4, False), (3. True)]
I have tried your idea and it works as expected . But I still figuring out why it works and the graphical model . Hope I will understand it .!! :+1:  . Thanks a lot again :)

Ok, glad to hear it works. The Bernoulli-trick is one way of making observations which are impossible for certain parameter values and equally likely for the other parameter values.  You could use some other value instead of 0.5 (in open range (0.0, 1.0)) as long as each row (trial) uses the same value. Then, the observation doesn't favor any of the non-impossible cases.


Lakectl local add statistics

Closes #6468
Add statistics to determine usage of lakectl local commands.
This statistic will add the following event class and name, whenever a lakectl local command is invoked:
Class: lakectl_local_event
Names:

checkout
clone
commit
init
status
list
pull

More information on lakectl local: https://github.com/treeverse/lakeFS/issues/6239

@N-o-Z Can we use lakectl_local as the class rather than 'lakectl_local_event'?
@nopcoder Any objection to use 'lakectl_local' rather than 'lakectl' as you offered? will this change be relevant to other events of lakectl that are not lakectl local in the future?

@N-o-Z Can we use lakectl_local as the class rather than 'lakectl_local_event'?
@nopcoder Any objection to use 'lakectl_local' rather than 'lakectl' as you offered? will this change be relevant to other events of lakectl that are not lakectl local in the future?

Sure - if we use lakectl as the class - we'll need to modify the event names and add the local_ prefix

Prefer not to die in case we fail to report stats event - maybe just a warning
Class/Name - we can hook the post command in lakectl and report all laketctl commands under class:lakectl and format the name of the command we invoked. It will cover all current usage and easy way to filter out events.

Where would we issue the warning? We have no logging mechanism in lakectl and it's not something that should interest the user? If PostStatsEvents API fails - I think we should at least get an indication somehow - the only way I see it now is if it bothers the user and they open an issue on it.
For now changed to lakectl_local will modify according to discussion with @NoyDavidson2

Prefer not to die in case we fail to report stats event - maybe just a warning
Class/Name - we can hook the post command in lakectl and report all laketctl commands under class:lakectl and format the name of the command we invoked. It will cover all current usage and easy way to filter out events.

Where would we issue the warning? We have no logging mechanism in lakectl and it's not something that should interest the user? If PostStatsEvents API fails - I think we should at least get an indication somehow - the only way I see it now is if it bothers the user and they open an issue on it.
For now changed to lakectl_local will modify according to discussion with @NoyDavidson2

After f2f with Noy - can use lakectl as class and format the command executed based ont eh arguments used to invoke lakectl.
Not related to BI - in case we fail to update, warn/error without failing the user's request.
On the server side - make sure we populate the 'client' field when the server post the information.
Thanks 🙏

Added feedback as reply to your last comment - let me know if it is not clear.

Modified appropriately

We need to hold excludeStatsCmds without lakectl prefix

Done


[PHP 7.4 Compat] Remove overloaded offsetExists()

This blows up on PHP 7.4 because of the change to array_key_exists() which now strictly expects an array.
https://www.php.net/manual/en/migration74.deprecated.php#migration74.deprecated.core.array-key-exists-objects
This method looks like it was only added because of this bug https://bugs.php.net/bug.php?id=40442 which is exclusive to PHP 5.2.1 and was fixed in 5.2.2 - https://www.php.net/ChangeLog-5.php#5.2.2

Thank you @Dave13h for the report! Could you create a PR in https://github.com/zf1s/zf1/ monorepo instead, please? Also, since 7.4 is released now, we are going to look into getting rid of any incompatibilities in the whole framework, soon.

@falkenhawk done, is the plan to make future changes there then publish the individual packages from there?

@Dave13h yes, changes are propagated from monorepo to individual packages' repositories and then published automatically.