High cpu usage when parsing scripts that include [ordered]

Prerequisites

[X] Write a descriptive title.
[X] Make sure you are able to repro it on the latest released version
[X] Search the existing issues.
[X] Refer to the FAQ.
[X] Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce
We are parsing multiple powershell scripts in parallel using Parser.ParseInput but are having issues with perf when the scrips being parsed includes multiple [ordered] attributes/types.
The parser starts to use 100% cpu, and it takes a long time for the parsing to complete.
Repro here
The console app above has 2 identical powershell scripts included, the only difference is that on of them has a bunch of [ordered] before the hashtable assignments.
The one without [ordered] completes almost instantly, while the one with [ordered] takes some time and uses a lot of cpu.
Expected behavior
Parsing to complete in about the same time with both scripts.

Actual behavior
Parsing takes a lot longer and uses a lot more resources when the script includes [ordered].

Error details
The error seems to be related to LookForTypeInAssemblies method within the TypeResolver.cs:68
Every time it finds a [ordered] in the script it goes through all loaded assemblies to try and find the matching type, but since [ordered] is not actually a type (just a helper for creating System.Collections.Specialized.OrderedDictionary) the method returns null, and will search again the next time it finds an [ordered] attribute.
I think there should either be a cache for types that did not match any assemblies, so that it doesn't have to look through the same assemblies for the same type multiple times, or there should be a special case for [ordered] so that it does not even try to search through the assemblies for it.
Environment data
We are using nuget package: System.Management.Automation version 7.2.3

Visuals
No response

My general understanding is that [ordered] is one of the standard type accelerators, so it should be resolved almost immediately in terms of type lookup. Also, it's not even a real type, afaik it's just a parser trick to tell PowerShell to build the resulting dictionary completely differently (since if it was built the normal way and then cast, the resulting dictionary would not retain the original order in the script).
Flagging this for WG-Engine to have a look at.

Believe it or not, there's no actual type accelerator associated. I would have thought that the compiler would skip type resolution for it though, since it doesn't actually use the resolution at all.
Adding a type accelerator to OrderedDictionary would likely solve this problem, and also ease a ton of confusion around this pseudo keyword

Oh, yeah? Interesting. I would also have expected it to simply skip the type resolution completely, since it's... not intended to be a type in itself. 🤔
Even if you define a type called [ordered] in the current session, it still doesn't get used for an [ordered]@{ ... } expression, so it's very odd to me that type lookup would ever be implicated. For example:
PS> class ordered {ordered([System.Collections.Specialized.OrderedDictionary]$d) { }}
PS> $a = [ordered]@{ a = 1; b = 2 }
PS> $a

Name                           Value
----                           -----
a                              1
b                              2

PS> $a.GetType().FullName
System.Collections.Specialized.OrderedDictionary

Also... a type simply named "ordered" is actually unusable because of how the parser handles [ordered]:
PS> $b = [ordered]$a
ParserError:
Line |
   1 |  $b = [ordered]$a
     |      ~~~~~~~~~~~
     | The ordered attribute can be specified only on a hash literal node.

So it's... really weird... that [ordered] does any type lookups whatsoever.

I think there are 2 problems worth discussing here:

[ordered] is treated as a special attribute solely for the syntax [ordered] + <literal hash table>. Is that something we want to change?
more generally, shall we optimize the parser to not keep attempting to resolve a type name when it cannot be resolved?

There could be another repro for this issue: I'm using the type name [sometypename] in my script at tons of places, but the type is not available until at run time -- a call to Add-Type or Import-Module will load the assembly containing that type. Parsing the script will experience the same slowness.

I think there are 2 problems worth discussing here:

[ordered] is treated as a special attribute solely for the syntax [ordered] + <literal hash table>. Is that something we want to change?

I think it would be reasonable to make it an actual type accelerator and have fall back logic like we already do for [pscustomobject]. Maybe not fall back as that might lead to confusion that we already face with [pscustomobject] outside of compiler recognized scenarios. But we can at least make it resolvable.

more generally, shall we optimize the parser to not keep attempting to resolve a type name when it cannot be resolved?

There could be another repro for this issue: I'm using the type name [sometypename] in my script at tons of places, but the type is not available until at run time -- a call to Add-Type or Import-Module will load the assembly containing that type. Parsing the script will experience the same slowness.

I think the second part answers the first part here 🙁. We can't actually know if a type will never be resolvable.
That said, I'm not really sure why type resolution isn't done lazily outside of classes. Does the parser really care what the true type is?

Believe it or not, there's no actual type accelerator associated. I would have thought that the compiler would skip type resolution for it though, since it doesn't actually use the resolution at all.
Adding a type accelerator to OrderedDictionary would likely solve this problem, and also ease a ton of confusion around this pseudo keyword

Would a type accelerator be able to initialize the OrderedDictionary, then add the items of the hashtable?
Just replacing [ordered] with [System.Collections.Specialized.OrderedDictionary] works fine with empty hashtable @{ }, but if the hashtable contains multiple properties, it will fail to cast the hashtable to OrderedDictionary

Would a type accelerator be able to initialize the OrderedDictionary, then add the items of the hashtable?

It wouldn't change runtime behavior of anything that currently works. It just makes the type resolvable. The compiler specifically checks for the literal text ordered, and doesn't actually try to resolve it (unlike the parser apparently).

I think the second part answers the first part here 🙁. We can't actually know if a type will never be resolvable.

But can we know if a type will never be resolvable at parsing time only? What would cause new types available at parsing time? PowerShell classes are already taken care of when resolving symbols, and using module/assembly can only appear at the beginning of a script.

WG-Engine discussed this one today (5/12/2022).

[ordered] is treated as a special attribute solely for the syntax [ordered] + <literal hash table>. Is that something we want to change?
more generally, shall we optimize the parser to not keep attempting to resolve a type name when it cannot be resolved?

To the 1st question, we think it's a good idea to make [ordered] a true type accelerator for [System.Collections.Specialized.OrderedDictionary] and remove the parser code that restricts [ordered] to only be used before a literal Hashtable. We believe this will help making the [ordered] type less special and more consistent with [pscustomobject].
To the 2nd question, the same performance degradation would happen to another non-resolvable type name in the script, so it would be nice to resolve this perf issue in the root -- not looking up a non-resolvable type name multiple times during parse would provide a significant boost to performance.

#17328 was merged to mitigate the perf issue regarding [ordered] specifically:

This alone cuts the wallclock time for parsing [ordered]@{} compared to [nonexisting]@{} by 45-50%:

Thanks for the quick fix. I guess this will be included in the next version of System.Management.Automation package (7.2.5)? If so, do you have any target date for that release?


Migration to change tribunals org type

What
On the Organisations page on Signon, any organisation that has a tribunal type is labelled 'tribunal non-departmental public body'.
However, many organisations are tribunal bodies but are departmental public bodies (such as HMCTS). There is only one type tag available (non-DPB), so we should rename this type as 'Tribunal' to cover both types of organisations.
Why
Not all organisations fit into the current type tagging. The current tag is very specific - we should make it easier for publishers by changing the type name to a more generic 'Tribunal'.
Trello

The commit message is a bit short, and doesn't mention why the change is being made. Could you expand it a bit. Maybe just work in some of the information you've put in the Pull Request description.

Good point, have updated it now!

Good point, have updated it now!

Great, thanks.
I noticed from the Trello card that there was already a Pull Request open relating to seemingly the same issue: https://github.com/alphagov/whitehall/pull/4969 How does the change here fit in with those proposed changes?

Good point, have updated it now!

Great, thanks.
I noticed from the Trello card that there was already a Pull Request open relating to seemingly the same issue: #4969 How does the change here fit in with those proposed changes?

So I think that PR needs to go in first and then we do this migration

So I think that PR needs to go in first and then we do this migration

It looks to me like the Pull Requests are mutually dependent, you wouldn't want to apply either commit without the other?
If that is the case, it would be better if this was changed to a normal migration (rather than a data migration, as it's effectively the schema that's being changed), and merged in to the commit in the other Pull Request.

So I think that PR needs to go in first and then we do this migration

It looks to me like the Pull Requests are mutually dependent, you wouldn't want to apply either commit without the other?
If that is the case, it would be better if this was changed to a normal migration (rather than a data migration, as it's effectively the schema that's being changed), and merged in to the commit in the other Pull Request.

I'll close this PR and add a migration to the other one


Please return the success message as json format

Provide a consistent way for response body so the front-end could handle error and messages, data.
{"code": 10000, "msg": "success", data: "any object from controller"}


Upgrade to Jolokia 2.1.1. Remove json-simple 1.1.1

Fixes AMQ-9626

Is json-simple no longer needed?

Jolokia no longer depends on json-simple.
json-simple is no longer maintained. It was last updated in 2012.
Jolokia 2.1.1 now includes builtin JSON functionality which ships in jolokia-json.jar. The assembly packaging has been updated accordingly.
See here for the details of the Jolokia change: https://github.com/jolokia/jolokia/issues/686

@mattrpav jolokia 2.1 now uses it's own json lib (json-simple is useless now with Jolokia 2.1).


Add Django 3.2 to supported technologies doc

Support was added in https://github.com/elastic/apm-agent-python/pull/1064

:green_heart: Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2021-09-03T16:23:48.686+0000

Duration: 3 min 22 sec

Commit: 212baeb7146461f8d6123fe52a4ae00fe0dac053

Trends :test_tube:

:grey_exclamation: Flaky test report
No test was executed to be analysed.


[C++] Take kernel can't handle ChunkedArrays that don't fit in an Array

Take() currently concatenates ChunkedArrays first. However, this breaks down when calling Take() from a ChunkedArray or Table where concatenating the arrays would result in an array that's too large. While inconvenient to implement, it would be useful if this case were handled.
This could be done as a higher-level wrapper around Take(), perhaps.
Example in Python:

>>> import pyarrow as pa
>>> pa.__version__
'1.0.0'
>>> rb1 = pa.RecordBatch.from_arrays([["a" * 2**30]], names=["a"])
>>> rb2 = pa.RecordBatch.from_arrays([["b" * 2**30]], names=["a"])
>>> table = pa.Table.from_batches([rb1, rb2], schema=rb1.schema)
>>> table.take([1, 0])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyarrow/table.pxi", line 1145, in pyarrow.lib.Table.take
  File "/home/lidavidm/Code/twosigma/arrow/venv/lib/python3.8/site-packages/pyarrow/compute.py", line 268, in take
    return call_function('take', [data, indices], options)
  File "pyarrow/_compute.pyx", line 298, in pyarrow._compute.call_function
  File "pyarrow/_compute.pyx", line 192, in pyarrow._compute.Function.call
  File "pyarrow/error.pxi", line 122, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 84, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: offset overflow while concatenating arrays

In this example, it would be useful if Take() or a higher-level wrapper could generate multiple record batches as output.
Reporter: Will Jones / @wjones127
Assignee: Will Jones / @wjones127
Related issues:

https://github.com/apache/arrow/issues/28385 (is a child of)
https://github.com/apache/arrow/issues/26738 (duplicate)
https://github.com/apache/arrow/issues/31249 (duplicate)
https://github.com/apache/arrow/issues/26738 (duplicate)
https://github.com/apache/arrow/issues/34583 (duplicate)
https://github.com/apache/arrow/issues/37766 (relates to)
https://github.com/apache/arrow/issues/23539 (relates to)
https://github.com/apache/arrow/issues/33049 (relates to)

PRs and other links:

https://github.com/apache/arrow/pull/13857

Note: This issue was originally created as ARROW-9773. Please see the migration documentation for further details.

Do we have any updates on this?

Hi! Issue 3 is bothering us a lot when sorting huge tables. Do we have any updates on this?

I'm working on this.

@felipecrv @jorisvandenbossche do we have a clear line of sight when this issue will be addressed.
Can you help me understand if there were/are any practical limitations around why chunking wasn't a consideration in the first place (for Table/ChunkedArray APIs)?
This is a pretty foundational issue for take API not doing chunking for arrays growing above 2Gb rendering this API essentially impossible to use for Data Processing.
For ex, in Ray Data

We can't force users to go for int64-based types
We can't blindly upcast all types to int64-based ones either
We have to be able to handle columns growing above 2Gb

@alexeykudinkin take/filter on chunked arrays requires resolution of chunks which is more expensive than simple array offsetting [1].
Solutions that don't concatenate tend to be slower and are considered unacceptable. If it were up to me, I would leave the decision of concatenation to callers and never concatenate. I'm going to compromise and add conditional checks on sizes to decide if we concatenate or not. What I don't like about this solution is that it kinda doubles the amount of tests we need to run to cover both algorithms.
[1] https://github.com/apache/arrow/pull/41700


Path for support ".hocon"

My patch for support ".hocon" extension #45

This has been open for a very long time and has had no activity lately. I am going to close this. Feel free to open another PR or continue the discussion in relevant tickets.


context req; pre-parsed body

Add the req: HttpRequest object to the context function so that context creation has access to the raw request.  Use case: custom auth needs the request headers
body parsing: v0.1.1 broke our implementation because it expected body to be a string, yet ours was already JSON.parse'd.  The code now only parses when necessary.

Hey @chilltemp, this fix was already implemented in #35, but if you'd like to rebase this PR and repurpose it for adding req to context that seems reasonable!


Hammering EJBCA with request and lots of log

When the EJBCA does not answer the health check ("/ejbca/ejbca-rest-api/v1/certificate/status"), the operator keeps calling the endpoint and creates a log of log. It looks like roughly 10 requests per second and stays the same for hours.
Perhaps an exponential backoff would be a good thing or a lower frequency?

For context, the Health Check interface is used in the Issuer/ClusterIssuer controller to verify the connection/credentials to EJBCA. My understanding understanding up to now is that Reconcilers built using the controller-runtime project had exponential backoff built-in. I will do some research to determine if this is actually the case.

Hi @smuda
In the latest version of the ejbca-cert-manager-issuer, I upgraded from Kubebuilder go/v3 to go/v4 which upgraded the controller-runtime module (among many others).
While exponential backoff should have been implemented by controller-runtime in the previous version, I'm finding experimentally that it does work as expected in the newer version.

Well done!


onComplete and onError Events don't triggered on firefox

Type of issue

[X] Bug report
[ ] Feature request
[ ] Support request

Uploader type

[X] Traditional
[ ] S3
[ ] Azure

Bug Report
Fine Uploader version
5.10.1
Browsers where the bug is reproducible
Firefox 52.0.2
Operating systems where the bug is reproducible
Windows 8.1, and I guess any other
Exact steps required to reproduce the issue
For example:

Only upload any file or files using firefox

All relevant Fine Uploader-related code that you have written
	fineUploader.fineUploader({
		debug: true,
		request:{
			endpoint: '/wp-content/plugins/innovation-factory/includes/php/endpoint.php'
		},
		callbacks:{
			onComplete: function(id, name, responseJSON, xhr){
				console.log({
					_event: event,
					_id: id,
					_name: name,
					_responseJSON: responseJSON
				});

				if(responseJSON.success){
					adjuntos.push(responseJSON.uuid + "/" + responseJSON.uploadName);
				}
			},
			onUpload: function(id, name){
				console.log({_id: id, _name: name});
			},
			onError: function(id, name, errorReason, xhr){
				var error = {
					id_: id,
					name_: name,
					errorReason_: errorReason,
					xhr_: xhr,
				};
				console.error(error);
			}
		}
	});

Detailed explanation of the problem
FineUploader uploads correctly the file, but Firefox doesn't trigger de related event (onComple or onError). The server is returning the correct response, but firefox ignoers the events:

In chrome all works fine.

there must be some specific issue with your environment as i cannot reproduce and yours is the only report


Noindex new projects

This PR adds a Jinja filter is_recent that can be used to determine if a given timestamp is within the last 30 days. This is then used on the project page to add a noindex meta tag for projects created recently, which should give us sufficient time to remove spam before googlebot indexes the project.

I thought we already had SEO blocking measures in place, but perhaps that was only on links? Regardless, great idea.


added function to convert images to .mif file format

Solves Issue #5 and adds support for producing output files in .mif format in addition.
Added convert_to_mif() function that takes in input image data and creates a string to be written to output file formatted for a mif file, specifying fields such as width, depth, radix, and content. Documented with Napoleon style doc string.
Added test case in test.py to verify functionality.

Hello,
Thank you for adding MIF support to img2fmem.
Could you update img2fmem/README.md to include basic info on MIF support and how to run your test?
TIA,
Will


profile "curryhot" not working

The Steam stats card generator works for other profiles but does not display stats for my profile named "curryhot."  I confirmed that my profile is public and that the name is correct. Other profiles load without issue.
Steps to reproduce:

Open: My Steam Stats
No stats are displayed.
Replace "curryhot" with another profile name to see if it works correctly.

Fixed here https://github.com/Ananto30/steam-stat/pull/16


Catch JDA exception when there is no internet connection

This pull request...

[x] Fixes a bug
[ ] Introduces a new feature
[ ] Improves an existing feature
[ ] Boosts code quality or performance

Description
Added catch clause to JMusicBot.java
Purpose
When running the bot without an internet connection, the JDA connection attempt will fail without catching the exception and doing a proper system exit. This means that if the bot cannot establish a connection, the program will wait instead of exiting. Automated restart scripts will not work as a result.
Relevant Issue(s)
This PR closes issue #1303

It's missing import net.dv8tion.jda.api.exceptions.ErrorResponseException for the ErrorResponseException. Not the author, but I noticed it yesterday.

Yeah that's my mistake, I've added it in and tested on my machine. I have a systemd service that will now automatically restart on failure.


hyper support checkpoint and restore

#dibs Now I have already made hyper support checkpoint and restore.If you use aufs docker image,you cant do it because of virtfs.If you use devicemapper docker image, it will be OK.Then I plan to make migrate between two physical host possible.

Hi @YaoZengzeng
Thanks for your work, it will be interesting if you have any update.
Regards,

we can close this issue as #123 is on. @gnawux


fix(ui): upgrade argo-ui for visible filter dropdown bar arrow. Fixes #7789

Fixes #7789
Motivation
Dropdown arrow was half hidden, and we want to fix it to improve the visual

Modifications
Upgrade to latest version of argo-ui components

include fix https://github.com/argoproj/argo-ui/pull/178

Verification
build success
arrow visible
functioning normal

Dang, I wasn't able to backport this to release-3.5 as it merge conflicted quite heavily. Specifically, it's missing #12516 (and #12163). We may want to backport those as well, TBD


use application name for instance lookup by serviceId

fixes #1788

Codecov Report

Merging #1789 into master will decrease coverage by 0.01%.
The diff coverage is 80%.

@@            Coverage Diff             @@
##           master    #1789      +/-   ##
==========================================
- Coverage   75.38%   75.37%   -0.02%
==========================================
  Files         203      203
  Lines        6240     6241       +1
  Branches      781      782       +1
==========================================
  Hits         4704     4704
  Misses       1196     1196
- Partials      340      341       +1

Impacted Files
Coverage Δ

...rk/cloud/netflix/eureka/EurekaDiscoveryClient.java
51.28% <80%> (-1.35%)
:x:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a8b170f...139bc94. Read the comment docs.

I don't think this is a good idea. It's just a default and can be changed. Users have changed it and this would break them.

@spencergibb wouldnt that be OK in Dalston though?

I don't think so

even the handling of application name versus vip address with EurekaDiscoverClient is inconsistent. see for instance getServices():
	@Override
	public List<String> getServices() {
		Applications applications = this.eurekaClient.getApplications();
		if (applications == null) {
			return Collections.emptyList();
		}
		List<Application> registered = applications.getRegisteredApplications();
		List<String> names = new ArrayList<>();
		for (Application app : registered) {
			if (app.getInstances().isEmpty()) {
				continue;
			}
			names.add(app.getName().toLowerCase());

		}
		return names;
	}

Clearly uses application name and not vip address. So basically this code wouldn't give the expected results:
List<String> services = eurekaDiscoverClient.getServices();
for (String service : services) {
  List<ServiceInstance> instances = eurekaDiscoverClient.getInstances(service);
  // do something
}

As mentioned, `getServices()` returns a list of application names while `getInstances(..)` makes a lookup based on vip address. If you look at the javadoc of `DiscoveryClient` it's clear that this isn't what is expected:

```!java
	/**
	 * Get all ServiceInstances associated with a particular serviceId
	 * @param serviceId the serviceId to query
	 * @return a List of ServiceInstance
	 */
	public List<ServiceInstance> getInstances(String serviceId);

	/**
	 * @return all known service ids
	 */
	public List<String> getServices();

By default, spring-cloud sets both to ${spring.application.name} so it won't really be a problem.

As soon as you force the change you LOSE the ability to customize the vip. There is something lost in translation between serviceId -> application name/vip, I agree, but I'm against making this breaking change.

@spencergibb I don't think you do, it's just that serviceId is used differently in a eureka context (application name) and ribbon context (vip address). Zuul uses eureka application names as id and vip addresses as serviceId. As both are equal by default it's inviting to be treated as if they were equivalent. But especially if you customize the vip address, they aren't. The implementation of EurekaDiscoveryClient treats them inconsistently:
// eurekaDiscoveryClient.getInstances(serviceId) returns one instance with different application name and vip address
assertEquals(serviceId, eurekaDiscoveryClient.getInstances(serviceId).get(0).getServiceId()); // fails
assertNotNull(eurekaDiscoveryClient.getInstances(eurekaDiscoveryClient.getInstances(serviceId).get(0).getServiceId())); // fails

Of course I understand your concerns regarding changing existing behavior but in this case it's pretty clear that the existing behavior isn't the expected behavior and that people relying on the current behavior should rather use EurekaClient directly. But maybe changing the method name would be an option? So to intentionally break code that relies on the current behavior?

To get some more insight, I've just spent some time stepping through the code and it got increasingly clear that they are in fact treated as being equivalent. It might of course be that I'm missing something significant here. If so please let me know if and where I get something wrong.
We're using zuul with the standard DiscoveryClientRouteLocator which does the following to add dynamic routes (comments and simplifications by me):
// list of eureka application names
List<String> services = this.discovery.getServices();
for (String serviceId : services) {

	String key = "/" + mapRouteToService(serviceId) + "/**"; // by default "/${servieId}/**"
	if (!PatternMatchUtils.simpleMatch(ignored, serviceId) && !routesMap.containsKey(key)) {
		// new ZuulRoute(..) extracts `id` from `key` and sets `location` to `serviceId`
		routesMap.put(key, new ZuulRoute(key, serviceId));
	}
}

So by default ZuulRoute.id and ZuulRoute.location are set to serviceId which is the eureka application name. when used with zuul, the PreDecorationFilter sets the following in the context:

proxy = route.getId()
serviceId = route.getLocation()

RibbonRoutingFilter then does the following:
protected RibbonCommandContext buildCommandContext(RequestContext context) {
	// SNIP
	String serviceId = (String) context.get("serviceId");
	// SNIP
	return new RibbonCommandContext(serviceId, verb, uri, retryable, headers, params,
			requestEntity, this.requestCustomizers, contentLength);
}

protected ClientHttpResponse forward(RibbonCommandContext context) throws Exception {
	// removed exception handling and logging
	return this.ribbonCommandFactory.create(context).execute();
}

// HttpClientRibbonCommandFactory
public HttpClientRibbonCommand create(final RibbonCommandContext context) {
	ZuulFallbackProvider zuulFallbackProvider = getFallbackProvider(context.getServiceId());
	final String serviceId = context.getServiceId();
	// get RibbonLoadBalancingHttpClient by serviceId, i.e. eureka application name
	final RibbonLoadBalancingHttpClient client = this.clientFactory.getClient(
			serviceId, RibbonLoadBalancingHttpClient.class);
	client.setLoadBalancer(this.clientFactory.getLoadBalancer(serviceId));

	return new HttpClientRibbonCommand(serviceId, client, context, zuulProperties, zuulFallbackProvider);
}

SpringClientFactory creates a new context for serviceId in which the serviceId is set as ribbon.client.name (in NamedContextFactory.createContext(String))
The client name is later set as DeploymentContextBasedVipAddresses in EurekaRibbonClientConfiguration.preprocess():
setRibbonProperty(this.serviceId, DeploymentContextBasedVipAddresses.key(), this.serviceId);

At this point, the eureka application name is passed to ribbon as vip address. It's then used in DiscoveryEnabledNIWSServerList to fetch instances from eureka:
if (vipAddresses!=null){
    for (String vipAddress : vipAddresses.split(",")) {
         List<InstanceInfo> listOfInstanceInfo = eurekaClient.getInstancesByVipAddress(vipAddress, isSecure, targetRegion);
        // SNIP
    }
}

The translation from application name to vip address should likely have happened in DiscoveryClientRouteLocator where Route.location should not simply be serviceId - which is the application name at this point - but select a vip address (as there could be different addresses to choose from). As a consequence there would be a ribbon context per vip address, not per application. As to the right approach to choosing the vip address from discovered instances? I don't know. Currently it has to be done manually (zuul.routes.${applicationName}.serviceId=${vipAddress}).
Anyway, to get back to the original problem. I think I have demonstrated that the use of serviceId for both, application names and vip addresses, is a bit confusing and

@sfussenegger I have marked this "for discussion". I agree it's inconsistent, but almost exclusively in zuul's DiscoveryClientRouteLocator (and many disable the automatic zuul/discovery client routes, which should be opt-in, not opt-out). In more than two years, as far as I can remember, no one has complained.

@spencergibb we've been using this stuff for over a year now too and never noticed anything. But I guess that's to be expected as long as you run exclusively spring-cloud services. The only usage of DiscoveryClient.getInstances(serviceId) is in DynamicServiceInstanceProvider which seems not to be documented anywhere but in this commit. So it's probably save to assume that this method really isn't used a whole lot.
A good starting point for this discussion is probably the documentation of theses things and their usage:

application name
virtual host name
host name
instance id
service id

@sfussenegger I agree that the best thing to do is document the list you stated above. Care to open a new issue? I think then, this PR can be closed as well.

@sfussenegger I've been gathering information not documented in the Spring Cloud official manual, and a description of the various identifiers would be very useful. Would you mind taking a crack at that? Based on the feedback I got, my Eureka blog is now one of the most popular sources of Eureka, and I like to believe is helping people.

@asarkar we finally ended up dropping eureka altogether with #1788 being somewhat the final straw (we now use AWS internal load-balancers in combination with zuul and ribbon instead). so unfortunately it would be quite an effort to get back to a working eureka environment to further explore this. the comments on this PR should be a good starting point though.

Closing see https://github.com/spring-cloud/spring-cloud-netflix/issues/1788#issuecomment-322689486


colorSlice not changing dynamically

I'm not sure if it's intended or not but using for example:
setTimeout(() => {
            const options = {
                // item number you want to update
                "index": 1,
                // set a new percentage
                "percent": 30,
                // set a new color
                "colorSlice": "#000"
            };
            circle.animationTo(options);
}, 3000); // after 3s update

And the stroke isn't changing color to black.
This can be useful if for example for percentage < 30 then set font color and stroke color to green, if percentage < 60 then set colors to orange, etc..

That's right, only the percentage update is working at the moment.
As for the color change, work is already underway to implement this solution ;)


How to inform API consumer about unprocessable subscription request in Async mode

Problem description
In our subscription model we offer possibility for server to handle it synchronously or asynchronously.
Suppose we have a subscription request that the server is not able to handle (the monitoring area is not within operator area, the reachability status could ne follow this specific number, etc...) and the subscription server handles it in async mode.
How to inform the subscription requester that the request in not processable when server manages it in asynchronously ?
(in sync mode we do not have issue as we send back a 4xx error)
Possible evolution

add another TerminationReason value: SUBSCRIPTION_UNPROCESSABLE
add terminationReasonComment attibute to allow server to provide explantation

Alternative solution

Additional context
cc: @PedroDiez & @shilpa-padgaonkar @akoshunyadi

I understand this would be the case where, being a Telco Operator behaving in async mode, the processing of the subscriptionDetail information is happening after providing a subcriptionId.
So as, if GET /subscription would be queried, the "status" of the subscriptionId would be "ACTIVATION_REQUESTED" and after the processing is done and realized cannot be managed, the new proposed eventTermination Reason is returned back: SUBSCRIPTION_UNPROCESSABLE.
Regarding the final "status" of this kind of subscriptions, I think the best approach is not kept as "DELETED" and effectively removed them "as they never existed" because it adds no historical value a subscription that would never be working (just thinking in loud).
In principle makes sense to me and I am checking in parallel internally with our team

@PedroDiez did you already have time to check this internally with your team?

It occured that we have optional property in event-subscription-template.yaml :
https://github.com/camaraproject/Commonalities/blob/9233f834c9cfb44dbb843ca72f22085f8559d4da/artifacts/camara-cloudevents/event-subscription-template.yaml#L629-L630
Currently it is not used in subscription APIs and missing description attribute.

@rartych From DeviceLocation side we aggree on using the terminationDescription. ✅
The second proposal from @bigludo7 was to add the terminationReason SUBSCRIPTION_UNPROCESSABLE.
Can we also align on that?
cc @PedroDiez @shilpa-padgaonkar @jlurien

@rartych From DeviceLocation side we aggree on using the terminationDescription. ✅ The second proposal from @bigludo7 was to add the terminationReason SUBSCRIPTION_UNPROCESSABLE.
Can we also align on that?
cc @PedroDiez @shilpa-padgaonkar @jlurien

@rartych or @PedroDiez again requesting if we could align on terminationReason SUBSCRIPTION_UNPROCESSABLE?

@rartych From DeviceLocation side we aggree on using the terminationDescription. ✅ The second proposal from @bigludo7 was to add the terminationReason SUBSCRIPTION_UNPROCESSABLE.
Can we also align on that?
cc @PedroDiez @shilpa-padgaonkar @jlurien

@rartych or @PedroDiez again requesting if we could align on terminationReason SUBSCRIPTION_UNPROCESSABLE?

I think we can align on that and set formally in MetaRelease Spring 25. Ok with the decsision taken in Device Location

@maxl2287 @PedroDiez @bigludo7  Fine for me to agree on terminationReason and SUBSCRIPTION_UNPROCESSABLE


ERROR: No matching distribution found for jaxlib==0.4.25+cuda12.cudnn89; extra == "cuda12_pip"

pip3.12 install -r requirements.txt
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try brew install
xyz, where xyz is the package you are trying to
install.
If you wish to install a non-brew-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip.

If you wish to install a non-brew packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Dup #6

pip install --upgrade "jax[cuda12_pip]" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
pip install -r requirements.txt


SPN check fix for a future PI Data Archive

The upcoming version of PI Data Archive runs PI Network Manager service under NT Service\pinetmgr identity. Current version of PI Security Audit Tools isn't handling the new default configuration properly (SPN not found - false positive).

Should be fixed in pull request #295 .. waiting on confirmation.

Closing.


undefined behaviour due to LAPACK prototypes missing "hidden" variables required by Fortran

The prototypes for Fortran LAPACK functions appear to be incomplete in Dlib. This can lead to undefined behaviour with recent releases of gcc due to more aggressive compiler optimisations.
According to gfortran passing conventions (followed by other fortran compilers as well), every char* argument also has an associated "hidden" argument which specifies the number of characters. The type of the "hidden" argument may vary across compilers and/or compiler versions. The position of each "hidden" argument is also compiler dependent, though seems to be typically tacked onto the end of the function definition.
https://gcc.gnu.org/onlinedocs/gfortran/Argument-passing-conventions.html
As an example in Dlib, the current prototype for dsyev in
https://github.com/davisking/dlib/blob/master/dlib/matrix/lapack/syev.h
is defined as:
void DLIB_FORTRAN_ID(dsyev) (const char *jobz, const char *uplo, const integer *n, double *a, const integer *lda, double *w, double *work, const integer *lwork, integer *info);
But it probably should be:
void DLIB_FORTRAN_ID(dsyev) (const char *jobz, const char *uplo, const integer *n, double *a, const integer *lda, double *w, double *work, const integer *lwork, integer *info, blas_len jobz_len, blas_len uplo_len);
where blas_len is typically a 32 bit unsigned int on 32 bit platforms, and 64 bit unsigned int on 64 bit platforms.  (This does not apply to gcc versions <= 7, where it is always int).
Avoiding the use of the blas_len arguments appeared to work, but recently gcc has started to optimise more heavily, leading to stack overwrites when these arguments are not used.
The same bug affects Armadillo, R, and a lot of other software in C and C++ that uses LAPACK:

https://gitlab.com/conradsnicta/armadillo-code/issues/123
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90329
https://gcc.gnu.org/ml/gcc-patches/2019-05/msg00915.html
https://github.com/Reference-LAPACK/lapack/issues/339

CC: @dodomorandi

Can you submit a PR for this?

I got my hands full fixing the same issue in Armadillo :)

Ha, ok, thanks for the heads up :)
Are there known cases where this results in incorrect runtime behavior?

https://github.com/RcppCore/RcppArmadillo/issues/254
https://github.com/nlmixrdevelopment/RxODE/issues/84
https://bugzilla.redhat.com/show_bug.cgi?id=1709538

The above first looked like a compiler problem.
A while back I've also had a few reports of strange crashes involving code compiled by non-gcc compilers. I couldn't reproduce them on my setup, but I now strongly suspect that the "hidden" args were the culprit.


Disable libunwind shim by default

Fixes #34

It's probably useful to be able to disable this anyway.
This change means the correctness test is now testing something (see #27), and it looks like the backtrace crate gives an extra 0 address at the end. I guess we should strip that before comparing.

Also, which nightly version are you using? Currently nightly actually works for me (rustc 1.36.0-nightly (a784a8022 2019-05-09)), and disabling this module breaks it for reasons I don't understand:
Thread 2 "correctness" received signal SIGILL, Illegal instruction.
[Switching to Thread 0x7ffff6e02700 (LWP 17603)]
unwind_recorder (payload=0x7ffff6e014c8, stack=140737335268520, saved_regs=0x7ffff6e01478) at unwind/src/glue.rs:101

I probably used -08. It crashes when panicking in a should_panic test. https://github.com/bjorn3/pretty_backtrace/runs/123338228

That SIGILL I'm getting is only because the assert is failing inside the callback, and we can't unwind past unwind_recorder. So fixing the test on stable will fix that too.

I'll fix the correctness test in another PR.


How to check print page sucesss ?

Hi,
Thank for great project.
Can you help me: how to check printer is printed page sucesss ?

You can't do that within RawPrint, once the file is sent to the spooler it is queued and you don't know what has happened to it.


Fix announce helper

Prevent messages from being combined in VO
DE50838

Hi, I'm having a look at this now. I can say that the immediate defect from the story is resolved on my end (Windows + Chrome + NVDA no longer repeats the first message when announcing a second message within 10 seconds).
But I'm not certain that this hasn't introduced any other regressions. Still looking over the code, and also thinking we should set up some standard manual tests for this scenario, to try to consistently measure all the edge cases - something like announce.test-manual.js that we can run locally and test the screen-reader behaviour.
Should probably also check in with @dbatiste, his name is on some of the big "watch out!" comments that are being deleted here.

After the changes, it looks like it is no longer coalescing messages. Previously in the demo on VO, I would get:
"I like cookies but I also like donuts and many other really yummy things. I also like cake."
Both announce calls were put in a single message.
In this branch, I am getting them in separate messages:
"I like cookies but I also like donuts and many other really yummy things."
"I also like cake."
The result of it not coalescing anymore, is that the second announcement is sometimes interrupting the first one now.

Hey @GZolla , I'm just coming back to this - I think there's lots of tricky interactions going on here. I'd probably recommend:

switching this PR to a draft
adding a note onto the story on Rally, mentioning this PR and your findings about possible fixes/issues
returning this to "None" and unassigning yourself, putting it back in the fixer queue 😄

That said, if you want to keep tackling it that's an option too! Not sure what else you have on your plate, I'm just conscious that we only picked this up as part of the mob programming from dev onboarding, and the goal there wasn't necessary to come to a solution.


[KOGITO-3720] Add test coverage for Grafana dashboard

Signed-off-by: Karel Suta<EMAIL_ADDRESS>https://issues.redhat.com/browse/KOGITO-3720
Many thanks for submitting your Pull Request :heart:!
Please make sure that your PR meets the following requirements:

[x] You have read the contributors' guide
[x] Pull Request title is properly formatted: [KOGITO-XYZ] Subject
[x] Pull Request contains a link to the JIRA issue
[x] Pull Request contains a description of the issue
[x] Pull Request does not include fixes for issues other than the main ticket
[ ] Your feature/bug fix has a unit test that verifies it
[x] You've tested the new feature/bug fix in an actual OpenShift cluster

/jenkins test

/jenkins test

/jenkins test

Timeout with Strimzi
/jenkins test


Permission issue with gradle-api?

Hello, thanks for the great plugin.  I'm currently using it with a simple spring-boot project, and the funny thing is using it is its failing on a new ec2 instance.
$ ./gradlew buildDocker
:compileKotlin UP-TO-DATE
:compileJava UP-TO-DATE
:copyMainKotlinClasses UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:findMainClass
:jar
:bootRepackage
:assemble
:compileTestKotlin UP-TO-DATE
:compileTestJava NO-SOURCE
:copyTestKotlinClasses UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test UP-TO-DATE
:check UP-TO-DATE
:build
:buildDocker
      [tar] Building tar: /home/ec2-user/spring-booty/build/docker/add_1.tar
:buildDocker FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':buildDocker'.
> Docker execution failed
  Command line [docker build -t spring-booty:latest /home/ec2-user/spring-booty/build/docker] returned:
  Cannot connect to the Docker daemon. Is the docker daemon running on this host?

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 6.062 secs

However, if I just call the command line program:
$ docker build -t spring-booty:latest /home/ec2-user/spring-booty/build/docker
Docker can build without any problems.  Is there a perhaps a permissions difference between calling the docker build on the command line vs through the gradle-api?

I'm not sure why this worked, but rebooting my instance resolved this =/


Tooltip opens when isOpen prop changes from false to undefined

Hi. I have a simple hover tooltip setup that breaks if I set isOpen to false (for during a certain drag action) and then undefined (after the drag is finished). That is, the tooltip opens on its own and never closes when I set isOpen to undefined. Hovering over the tooltip triggers the close event again. Some gifs should clarify this:
Current state of react-tooltip-lite:

After my fix:

So yeah, I've got a fix for this already (didn't have rights to push to my own branch and make a PR). Feel free to use this as is or give me the right to push to my own branch and make a PR. It's a bit hacky, but I familiarised myself with the codebase for a bit and figured this is probably the least painful way to do this.
In index.jsx:
  componentDidUpdate(prevProps, prevState) {
    ...
    if (this.props.isOpen === false && prevProps.isOpen !== this.props.isOpen) {
      this.hideTip();
    }

The props I'm giving to the Tooltip aren't anything special:
    content={TooltipArea(props)}
    padding="2px"
    background="#8899aa"
    tipContentHover={true}
    tipContentClassName="datatask-tooltip"
    mouseOutDelay={500}
    hoverDelay={500}
    arrow={false}
    direction="up-start"
    isOpen={props.dragUnderway ? false : undefined}
    className="tooltip-wrapper"

Cheers! :)

Hey @SnowblindFatal, when you pass undefined to isOpen, it falls back to the default behavior of showing the tip when you mouse over the target.  Can you show me the JSX structure of what's around the tip?  It might make sense to just use fully controlled mode where you use events to maintain state for whether the tip is open or not.  I'll look a little more closely though when I have a chance.

The thing is, through some logic, the tooltip lib calls the showTip() function when I start dragging (and set isOpen to false) and when I stop dragging and set isOpen to undefined, it thinks that the tooltip should be shown. The thing is, it is in a faulty state as the tooltip never goes away unless I hover over the tooltip or the original element, as this presumably restarts the delay timers. My guess is that when isOpen is set to false, the delay timers don't have an effect or something.
My code is just
export const WrappedTooltip = props => (
  <Tooltip
    content={TooltipArea(props)}
    padding="2px"
    background="#8899aa"
    tipContentHover={true}
    tipContentClassName="datatask-tooltip"
    mouseOutDelay={500}
    hoverDelay={500}
    arrow={false}
    direction="up-start"
    isOpen={props.dragUnderway ? false : undefined}
    className="tooltip-wrapper"
  >
    {props.children}
  </Tooltip>
);

and then
  render() {
    return (
      <DataTaskRoot
        onClick={e => {
          this.props.onSelectItem(this.props.task);
        }}
        backgroundColor={this.props.color}
        left={this.props.left}
        width={this.props.width}
        height={this.props.height}
        isSelected={this.props.isSelected}
      >
        <WrappedTooltip
          task={this.props.task}
          onUpdateTask={this.props.onUpdateTask}
          dragUnderway={this.props.dragUnderway}
        >
          <DataTaskInner>
            <DataTaskLinker left={-linkerSize} onMouseUp={this.endLink} />
            <DataTaskResizer floatLeft />
            <DataTaskContent>{this.props.task.name}</DataTaskContent>
            <DataTaskResizer />
            <DataTaskLinker left={this.props.width - 2 * linkerBorderWidth} onMouseDown={this.startLink} />
          </DataTaskInner>
        </WrappedTooltip>
      </DataTaskRoot>
    );
  }

I don't think this is that special.

Hey @SnowblindFatal, Can you write a test that captures the issue here, I cannot seem to reproduce it.


MCEdit opens to a black screen

So my brother and I are having something of an odd error. We're both running the latest version, but for some unknown reason, he cannot open MCEdit properly. It will open the command prompt, the log seems to indicate it opens properly, but the actual program window proper is just black and doesn't respond to keyboard commands.
From what I have gathered, this is most likely a graphics card issue. However, his graphic card drivers are up to date and we both have the exact same card (Nvidia Quadro 600). We both run the same mods, on the same forge version, on the same MC version.
Steps to Reproduce - I can't really think of how to reproduce this error. We've tried to upgrade his drivers, completely reinstalling the whole thing, and even installing python. I legitimately not sure what is causing this error at all, nobody else seems to be having the same problem. Even my google-fu didn't turn up anything, which is why I'm asking here.
If it's relevant: At one point, it complained about usercache.json being corrupted. We deleted the file and the error went away, but I'm including it here for posterity. He has absolutely zero resource packs.
MCEdit Version Number - <IP_ADDRESS>
Minecraft Version Number - 1.7.10 forge version <IP_ADDRESS>6 running:
All these mods
Operating System Version - Windows 7 service pack 1
Log is attached.
MCEDITLOG.docx

Just my two cents...
Are both systems running the same version of Java?

Actually, no, now that I think about it. He's running Java 8 update 121 64 bit
I'm somehow running 2 editions of java. One is Java 7 update 79 the other is Java 8 update 77, both 64 bit.

Not sure how relevant that really is, though.  It was just a passing thought as I ran through troubleshooting in my head.
The operation of MCEdit is usually unrelated to the version of Minecraft and Java.
Source - unless that has been changed in 'unified', the java version shouldn't matter.

Exactly, which is why I have no idea what is causing the problem.

Can you give us a screenshot of the console window?

Here you go:

Sorry for the late answer.
This console screen is not unusual at all...
When MCEdit starts, tow items shall pop in the taskbar: the console one and MCEdit one.
Can you see both?
If yes, can you activate the MCEdit one, by clicking on it or with 'Alt-Tab'?

Switching between the two is fine, it doesn't freeze or anything, but it's just black and otherwise nonresponsive. We tried using keyboard commands, but that doesn't do anything either.

At least, I could test on my side!
I don't experience this on a Windows 10 VM...
What happens if you resize the MCEdit window?
You can also run MCEdit from command line:

Open a cmd shell.
'cd' to the folder in which mcedit.exe is.
Type in mcedit.exe --no-wm and hit Enter.

This will disable the internal window management. You may see some progress, but resizing the window will mostly be shaky...

Resizing didn't do anything, but we will try running it from the console as soon as we can. Will update as soon as we get results.


Fix overriding of global fillchars

currently the plugin will override the user's fillchars when setting fold:c, this unintentionally set's everything else beck to their defaults (such as horiz:c or vert:c for example)

Good catch, thanks!


tests-ci : portability: posix: common.signal.big_nsig test Failed

Describe the bug
common.signal.big_nsig test is Failed on zephyr-v3.4.0-3926-g4ba97c22555e on mimxrt1060_evk
testcase path is portability/posix/common/signal/big_nsig/nanosleep_execution
see logs for details
To Reproduce
1.
scripts/twister --device-testing --device-serial /dev/ttyACM0 -p mimxrt1060_evk  --sub-test portability.posix

or
# cd tests/portability/posix/common/signal/big_nsig/nanosleep_execution
# west build -b mimxrt1060_evk
# west flash

See error

Expected behavior
test pass
Impact
Logs and console output
START - test_nanosleep_execution

Assertion failed at WEST_TOPDIR/zephyr/tests/posix/common/src/nanosleep.c:153: common_lower_bound_check: (actual_ns >= exp_ns is false)
actual: 93042 expected: 500000000
FAIL - test_nanosleep_execution in 0.016 seconds

Environment (please complete the following information):

OS: (e.g. Linux )
Toolchain (e.g Zephyr SDK)
Commit SHA or Version used: zephyr-v3.4.0-3926-g4ba97c22555e

related: #62889

does not reproduce on latest main


Linting / ES6 / npm support

Dear Santa,
It would be great if you could:

run the code through a linter to tidy up the code. It's sending my linter mad so I had to disable it for this package.
modularise it and export the objects\functions so that it isn't on the Global namespace and also can be used easily in ES2015 etc projects i.e. import {createExolve} from 'exolve-m' and then magically bundled using webpack.
publish it to npm so that I can "npm install" the package and reference it

My front end skills aren't great, but if you can already do the above it would be great to add instructions into the readme.
Thanks!

I understand #1, but don't know yet what's an easy way for me to do this (I use vim.. there probably is some linter that I can integrate.. suggestions are welcome!).
Parts of #2 and all of #3 are Greek to me :-). Will try to educate myself more.

With v0.97, I've now at least got all lines to be <= 80 columns.

Given that over 4ish months of occasionally staring at this bug, I haven't really connected with it :-), I think I will close this issue soon. I'm sure node.js and npm are lovely, but I'm getting old and am a bit reluctant to learn about packages and frameworks.
Linting, otoh, I do agree with. I've been trying to clean things up towards that (80 cols max, for eg.) and will continue to do so.


StreamProcessorException: Error running FFmpeg process - FFmpeg process exited with code -1073741515

We are using ffmpeg to make a copy of the stream while it is being viewed.  Everything works in IIS on my dev machine, but in production ffmpeg is exiting with code -1073741515.  The obvious issue to check was permissions, but the app identity has read/execute permissions on ffmpeg.exe and read/write permissions on the UNC share it is trying to write the file to.  I also changed the output path to a subfolder of the web app to eliminate UNC issues.
Unfortunately I didn't get a hit on that exit code, so wanted to check with you.
I haven't yet tried launching FFmpeg myself instead of through the StreamProcessor. Or its verbose mode.  Here is the configuration we are using and the error message we get.
                        options.AddStreamProcessor().AddFFmpeg(options =>
                        {
                            options.Name = "mp4-archive";
                            var ffmpegPath = @"FFmpeg";
                            options.FFmpegPath = ExecutableFinder.FindExecutable(ffmpegPath, "ffmpeg").FullName!;
                            options.FFmpegArguments = "-i {inputPath} -c:v copy -c:a copy -preset ultrafast -f mp4 {outputPath}";
                            options.OutputPathResolver = new StreamCopyOutputPathResolver(streamingSettings, cacheService);
                        });

2024-12-05 01:35:19.8250|INFO|LiveStreamingServerNet.Networking.Server.Internal.Logging.LoggerExtensions.ClientConnected||Connected (ClientId=5)
2024-12-05 01:35:19.8558|INFO|....RtmpStreamEventMonitor.OnRtmpStreamPublishedAsync||Stream published - clientId=5, streamPath=/api/media/stream/transient_20241205013519627_2640731d09fd4f45a4f2f17d527b784a_Tc7r4ZPt3w9LGi5y2H6F
2024-12-05 01:35:19.8558|INFO|LiveStreamingServerNet.Rtmp.Server.Internal.Logging.LoggerExtensions.PublishingStarted||Start publishing successfully (ClientId=5, StreamPath=/api/media/stream/transient_20241205013519627_2640731d09fd4f45a4f2f17d527b784a_Tc7r4ZPt3w9LGi5y2H6F, Type=live)
2024-12-05 01:35:19.8558|INFO|LiveStreamingServerNet.StreamProcessor.Internal.Logging.LoggerExtensions.StartingFFmpegProcess||Starting FFmpeg process (Arguments=-i rtmp://localhost:1935/api/media/stream/transient_20241205013519627_2640731d09fd4f45a4f2f17d527b784a_Tc7r4ZPt3w9LGi5y2H6F?code=6C4373496F385661703433585476646D504C5A694C47614F67316C3774444642666A754D3462557441456B75774535367A6E56336F71687171645233756C394C -c:v copy -c:a copy -preset ultrafast -f mp4 \<IP_ADDRESS>\Vid01\1\2024\12\869247060028542_20241205013519763_2_2_Co32ZyPt8j6TFq59Sgr7.mp4)
2024-12-05 01:35:19.8719|INFO|LiveStreamingServerNet.StreamProcessor.Internal.Logging.LoggerExtensions.StreamProcessorStarted||Stream processor started (StreamProcessor=mp4-archive, Identifier=c40bb39f-419d-4e75-ae4a-9e7e991fbe3e, InputPath=rtmp://localhost:1935/api/media/stream/transient_20241205013519627_2640731d09fd4f45a4f2f17d527b784a_Tc7r4ZPt3w9LGi5y2H6F?code=6C4373496F385661703433585476646D504C5A694C47614F67316C3774444642666A754D3462557441456B75774535367A6E56336F71687171645233756C394C, OutputPath=\<IP_ADDRESS>\Vid01\1\2024\12\869247060028542_20241205013519763_2_2_Co32ZyPt8j6TFq59Sgr7.mp4, StreamPath=/api/media/stream/transient_20241205013519627_2640731d09fd4f45a4f2f17d527b784a_Tc7r4ZPt3w9LGi5y2H6F)
2024-12-05 01:35:19.9398|INFO|LiveStreamingServerNet.StreamProcessor.Internal.Logging.LoggerExtensions.StreamProcessorStopped||Stream processor stopped (StreamProcessor=mp4-archive, Identifier=c40bb39f-419d-4e75-ae4a-9e7e991fbe3e, InputPath=rtmp://localhost:1935/api/media/stream/transient_20241205013519627_2640731d09fd4f45a4f2f17d527b784a_Tc7r4ZPt3w9LGi5y2H6F?code=6C4373496F385661703433585476646D504C5A694C47614F67316C3774444642666A754D3462557441456B75774535367A6E56336F71687171645233756C394C, OutputPath=\<IP_ADDRESS>\Vid01\1\2024\12\869247060028542_20241205013519763_2_2_Co32ZyPt8j6TFq59Sgr7.mp4, StreamPath=/api/media/stream/transient_20241205013519627_2640731d09fd4f45a4f2f17d527b784a_Tc7r4ZPt3w9LGi5y2H6F)
2024-12-05 01:35:19.9398|ERROR|LiveStreamingServerNet.StreamProcessor.Internal.Logging.LoggerExtensions.StreamProcessorError|LiveStreamingServerNet.StreamProcessor.Exceptions.StreamProcessorException: Error running FFmpeg process
---> LiveStreamingServerNet.StreamProcessor.Exceptions.StreamProcessorException: FFmpeg process exited with code -1073741515
at LiveStreamingServerNet.StreamProcessor.Internal.FFmpeg.FFmpegProcess.RunProcessAsync(String inputPath, String outputPath, OnStreamProcessorStarted onStarted, OnStreamProcessorEnded onEnded, CancellationToken cancellation)
--- End of inner exception stack trace ---
at LiveStreamingServerNet.StreamProcessor.Internal.FFmpeg.FFmpegProcess.RunProcessAsync(String inputPath, String outputPath, OnStreamProcessorStarted onStarted, OnStreamProcessorEnded onEnded, CancellationToken cancellation)
at LiveStreamingServerNet.StreamProcessor.Internal.FFmpeg.FFmpegProcess.RunProcessAsync(String inputPath, String outputPath, OnStreamProcessorStarted onStarted, OnStreamProcessorEnded onEnded, CancellationToken cancellation)
at LiveStreamingServerNet.StreamProcessor.Internal.FFmpeg.FFmpegProcess.RunAsync(String inputPath, String streamPath, IReadOnlyDictionary2 streamArguments, OnStreamProcessorStarted onStarted, OnStreamProcessorEnded onEnded, CancellationToken cancellation)    at LiveStreamingServerNet.StreamProcessor.Internal.Services.StreamProcessorManager.RunStreamProcessor(IStreamProcessor streamProcessor, ISessionHandle client, String streamPath, IReadOnlyDictionary2 streamArguments, CancellationTokenSource cts)|Stream processor error (InputPath=rtmp://localhost:1935/api/media/stream/transient_20241205013519627_2640731d09fd4f45a4f2f17d527b784a_Tc7r4ZPt3w9LGi5y2H6F?code=6C4373496F385661703433585476646D504C5A694C47614F67316C3774444642666A754D3462557441456B75774535367A6E56336F71687171645233756C394C, StreamPath=/api/media/stream/transient_20241205013519627_2640731d09fd4f45a4f2f17d527b784a_Tc7r4ZPt3w9LGi5y2H6F)
2024-12-05 01:35:19.9398|INFO|....RtmpStreamEventMonitor.OnRtmpStreamUnpublishedAsync||Stream Unpublished - clientId=5, streamPath=/api/media/stream/transient_20241205013519627_2640731d09fd4f45a4f2f17d527b784a_Tc7r4ZPt3w9LGi5y2H6F
2024-12-05 01:35:19.9398|INFO|LiveStreamingServerNet.Networking.Server.Internal.Logging.LoggerExtensions.ClientDisconnected||Disconnected (ClientId=5)

Hi @studentnumber9
-1073741515 (0xC0000135) implies dll not found.
You may need to verify if ffmpeg can run separately first .

Thank you for that lead, I innocently thought all one needs is the big 144MB ffmpeg.exe that would include everything, but let me investigate.

The Admin found this article [https://superuser.com/questions/1742768/ffmpeg-on-windows-server-core-2022], which has the following instructions, based on which he copied the two dlls to system32.  FFmpeg process now works.
C:\windows\system32\AVICAP32.DLL and C:\windows\system32\MSVFW32.DLL can be copied from Windows Server 2022 LTSC (Desktop Edition) or Windows 10.
Copying those to the system32 folder, ffmpeg.exe should work again.


⚠ Bofrost.lu has degraded performance

In 48fb6bf, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1725 ms

Resolved: Bofrost.lu performance has improved in fd182fa after 7 minutes.


Create solution.py

Created Python solution file for 'Viral Advertising' problem.

Please mention issue number

Please mention issue number

Issue #163

Please do not create PRs on issues you are not assigned.


Update 2023-03-25-new-recruiting-process-saves-time-and-improves-effi.md

#Summary
Name correction

approved


Switch from goimports to gosimports for linting

gosimports automatically formats our imports into
three groupings correctly

Co-authored-by: Marty Spiewak<EMAIL_ADDRESS>
Changes proposed by this PR

goimports is not able to format our imports correctly when there are already multiple import groups, whereas gosimports is able to format them correctly regardless of how they are currently structured.

PR Checklist
Note: Please do not remove items. Mark items as done [x] or use ~strikethrough~ if you believe they are not relevant

[ ] Linked to a relevant issue. Eg: Fixes #123 or Updates #123
[ ] Removed non-atomic or wip commits
[ ] Filled in the Release Note section above
[ ] Modified the docs to match changes

wow that is....much better  😂
i'll close this


fixbug with contains being not str instance

Coverage remained the same at 86.557% when pulling c9bb6a089fb55c91535803203c165e088787509c on bugfix-dynamic-typic into 8b818c27e2562782d9f58eb53c53eec8b90ee414 on develop.


[doc] update the en names of TOC

Updated the name of en TOC (table of contents).

@windsonsea you need to rebase to the latest to avoid docusaurus compiling issues

/close


This PR contains ListData and Font details features.

This code contains ListData feature with some error in runtime.

Can one of the admins verify this patch?

@Codnelle Since this affects the companion we will need to update the base branch to ucr at some point (soon rather than later is better).

@AppInventorWorkerBee ok to test


esp-modbus as a slave data validation before writing

Checklist

[X] Checked the issue tracker for similar issues to ensure this is not a duplicate.
[X] Provided a clear description of your suggestion.
[X] Included any relevant context or examples.

Issue or Suggestion Description
I had a question regrading the code for the modbus slave. I have gone through the example for modbus rtu slave (https://github.com/espressif/esp-idf/blob/master/examples/protocols/modbus/serial/mb_slave/main/slave.c). It is working well. However, I have a possible issue with the way it is implemented that I need help with. Let's say that I write to the slave using fc 06 (write holding reg), via a modbus master. The library automatically writes the value to the specified register. However, I would like to see the incoming data, check whether it is valid (in range) and only then allow it to be written, otherwise send a modbus exception (Illegal Data Value 03). Please help me how to do this with esp-modbus.
An example case is let's say I am creating a thermostat. I would want my set-temperature to be allowed within range (let's say 16-40 C) . If a user enters a number out of range, I want to set a modbus exception, or at least prevent it being written. I am not able to understand how to do this with esp-modbus. I have read the slave API documentation and was unable to understand how to achieve this. Currently it writes the value instantly to specified register.

#82 is in the same boat as me I think.

@kabirwbt,

I would want my set-temperature to be allowed within range (let's say 16-40 C) . If a user enters a number out of range, I want to set a modbus exception, or at least prevent it being written. I am not able to understand how to do this with esp-modbus.

Sorry for the delay with answer. This stack was designed to hide some internals from user and it is not documented. This actually can be done by defining user handler functions for commands. However, there are several methods to accomplish this.
The one trick to override the callbacks in user code is described below:
Please apply this code above the app_main() in the slave example:
// The exception structure codes are defined in private file, let just override them here
typedef enum
{
    MB_EX_NONE = 0x00,
    MB_EX_ILLEGAL_FUNCTION = 0x01,
    MB_EX_ILLEGAL_DATA_ADDRESS = 0x02,
    MB_EX_ILLEGAL_DATA_VALUE = 0x03,
    MB_EX_SLAVE_DEVICE_FAILURE = 0x04,
    MB_EX_ACKNOWLEDGE = 0x05,
    MB_EX_SLAVE_BUSY = 0x06,
    MB_EX_MEMORY_PARITY_ERROR = 0x08,
    MB_EX_GATEWAY_PATH_FAILED = 0x0A,
    MB_EX_GATEWAY_TGT_FAILED = 0x0B
} e_mb_exception;

typedef  e_mb_exception( *pfunc_handler_t ) ( uint8_t * pframe, uint16_t * plength );

// Will call the handler registration function from stack core
extern int eMBRegisterCB( uint8_t func_code, pfunc_handler_t pfunc_handler );

// Define your custom function handler to process the command received.
// Can return the error code if we need to inform master about specific failure
e_mb_exception my_input_read_registers_handler_func( uint8_t *pframe, uint16_t *plen )
{
    // This error handler will be executed to check the request for the command 0x04
    // See the `esp-modbus/freemodbus/modbus/functions/mbfuncinput.c` for more information
    // pframe is pointer to command buffer, plen - is pointer to length
    ESP_LOGE("TEST_HANDLER", "Overridden handler for command 0x04 is called.");
    return MB_EX_ILLEGAL_DATA_VALUE;
}

// An example application of Modbus slave. It is based on freemodbus stack.
// See deviceparams.h file for more information about assigned Modbus parameters.
// These parameters can be accessed from main application and also can be changed
// by external Modbus master host.
void app_main(void)
{
    mb_param_info_t reg_info; // keeps the Modbus registers access information
    mb_communication_info_t comm_info; // Modbus communication parameters
    mb_register_area_descriptor_t reg_area; // Modbus register area descriptor structure

    // Set UART log level
    esp_log_level_set(TAG, ESP_LOG_INFO);
    void* mbc_slave_handler = NULL;

    ESP_ERROR_CHECK(mbc_slave_init(MB_PORT_SERIAL_SLAVE, &mbc_slave_handler)); // Initialization of Modbus controller
    uint8_t override_command = 0x04;

    int err = eMBRegisterCB(override_command, NULL);
    MB_RETURN_ON_FALSE(!err, ;, TAG,
        "could not override func handler, returned (0x%x).", (int)err);
    err = eMBRegisterCB(override_command, my_input_read_registers_handler_func);
    MB_RETURN_ON_FALSE(!err, ;, TAG,
        "could not override func handler, returned (0x%x).", (int)err);

    ESP_LOGI("TEST_HANDLER", "Overridden function handler for the command 0x%x ", (int)override_command);

    // Setup communication parameters and start stack
#if CONFIG_MB_COMM_MODE_ASCII
    comm_info.mode = MB_MODE_ASCII;
#elif CONFIG_MB_COMM_MODE_RTU
    comm_info.mode = MB_MODE_RTU;
#endif
    comm_info.slave_addr = MB_SLAVE_ADDR;
    comm_info.port = MB_PORT_NUM;
    comm_info.baudrate = MB_DEV_SPEED;
    comm_info.parity = MB_PARITY_NONE;
    ESP_ERROR_CHECK(mbc_slave_setup((void*)&comm_info));

Note: the handler will be executed by stack from the polling task, so keep the handler as short as possible and safe.
Would this work for you?

Hey @alisitsyn. Thank you for replying.
Your code looks good to me!
I shall test it in my application and then get back to you.

Hey @kabirwbt,
Any update on this?

@kabirwbt,
Thank you for your feedback. Yes, this trick is probably for esp-idf only and works (verified). There are some other possible ways to accomplish this for arduino making changes in the internal component folder.
This approach  may also help to patch the official component in your project.
Once you have some results, please report here. Thanks.


FilterableEnumPropertyEditor Not Working

Using systemTester, FilternameEnumPropertyEditor fails to open
To Reproduce

And also in my application Enum filter is applied ok, but fail to save editor value

Installer lab build 18.2.600 includes commit FilterableEnumPropertyEditor Not Working #330 that relate to this task. Please test if it addresses the problem. If you use nuget add our NugetServer as a nuget package source in VS

Hi Tolis, I tried to download the corresponding artifact from DevOps but faced this exception
{"$id":"1","innerException":null,"message":"TF400813: The user 'Windows Live<EMAIL_ADDRESS>is not authorized to access this resource.","typeName":"Microsoft.TeamFoundation.Framework.Server.UnauthorizedRequestException, Microsoft.TeamFoundation.Framework.Server","typeKey":"UnauthorizedRequestException","errorCode":0,"eventId":3000}

thanks for the heads up, I can also failed to download with my personal account.
I do not see any easy work around here as from what I understand I cannot set anonymous permissions in devops. I will integrate probably Gihub lab repository for nightly releases. In the mean time your option is to build the lab repo your self or to use the nuget from https://xpandnugetserver.azurewebsites.net/

Hi @apobekiaris , The Installer lab build 18.2.600 includes commit FilterableEnumPropertyEditor Not Working #330, do not solve the problem, attached failing sample solution, the drop down is filtered correct but the editor do not show anything
Solution3.zip

Also in a web solution, the editor seems to be working, but it incorrectly permits an initial wrong value as shown in the following screencast

In the next minor the FilterbaleEnumPropertyEditor has been merged with the EnumPropertyEditor ans ASPxEnumPropertyEditor. they both default editors for enum types. I also supported an attributed based filtering like.
    [EnumFilter(nameof(FilteredStatus),EnumFilterMode.Remove, StatusColor.Blue,nameof(Switch)+"=true")]
    public class FilterableEnumPropertyEditorObject : BaseObject {
        [DataSourceProperty("InstanceStatusDataSource")]
        public StatusColor? FilteredStatus {
            get => _filteredStatus;
            set => SetPropertyValue("FilteredStatus", ref _filteredStatus, value);
        }

        [ImmediatePostData]
        public bool Switch {
            get => _switch;
            set {

let me know how it works.

+1 for the added criteria!

your idea rocks Alex thanks for sharing!

Installer lab build 18.2.601.11 includes commit Simplify EnumPropertyEditors and support additional attribute based filtering #330 that relate to this task. Please test if it addresses the problem. If you use nuget add our NugetServer as a nuget package source in VS

Installer lab build 18.2.601.16 includes commit EnumPropertyEditor throws on ObjectSpaceOnCommitted #330 #341 that relate to this task. Please test if it addresses the problem. If you use nuget add our NugetServer as a nuget package source in VS

Installer lab build 18.2.601.18 includes commit EnumPropertyEditor throws on ObjectSpaceOnCommitted #330 #341 that relate to this task. Please test if it addresses the problem. If you use nuget add our NugetServer as a nuget package source in VS

Closing issue for age. Feel free to reopen it at any time.
.Thank you for your contribution.


esp32 compatiblility

Nice easy Thermistor using the B value, I would like to suggest wrapping analogReference() with
<#ifndef ESP32>
so it is compatible.

Hi, if you could make a pull request, I'll merge it. I don't have much time right now to do it myself 😃


check out AutoProf for ellipse-fitting and extracting surface brightness profiles

https://ui.adsabs.harvard.edu/abs/2021MNRAS.508.1870S/abstract
https://autoprof.readthedocs.io/en/latest/

AutoProf is great, but it's too slow to run at the scale of the SGA.


⚠ Jenkins Plugins Artifactory has degraded performance

In f07304d, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1221 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 39a28c4 after 13 minutes.


Capability for users to create batch difference plots

Users now have the option to create a "diff" plot using create_graphics.py. It will accept two directories with two options
for file name templates, and will create all the difference maps for variables specified in the image list. All other behaviors are similar to regular map plotting (forecast hours, regions, etc.).
There are a few things it won't do out of the box:

Plot differences from different models. HRRR and RRFS, for example.
Plot graphics accumulated variables. It will plot the variables labeled as accumulated in UPP, though.
Have a uniform color scale across variables. Each plot will assess the min/max values of the data field and use those numbers to set the value range.
Use different color maps. That's hard coded.

Here is an example plot:

I'll add some additional comments to the code after the PR is opened.

@Brian-Jamison I am not totally convinced all variables will work given the difference in naming convention between the two models -- HRRR and RRFS. It might be true that for some select variables that do share the same variable names, these might work, but I don't want to claim that it certainly would.


Bug try fix the exchange sink buffer result error

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

#27052


exhaustive generation option for property testing

GenerationMode.AUTO - uses exhaustive mode where possible, otherwise defaults to random.
GenerationMode.RANDOM - forces random mode
GenerationMode.EXHAUSTIVE - forces exhaustive or throws an exception

Exhaustive as in use all possibilities?
I think some gens wouldn't work that way, right?

Basically if you say Gen.ints(0, 100) to generate random ints from 0 to
100, and then you ask for 200 tests to run, and you use random generation,
then you might never get 85 or 33. You might get 77 eight times.
In this case you should get 0 to 100 two times. That's better.
When you ask for Gen.ints() and 1000 tries, then it has to be random.
On Tue, 3 Sep 2019 at 10:27, Leonardo Colman Lopes<EMAIL_ADDRESS>wrote:

Exhaustive as in use all possibilities?
I think some gens wouldn't work that way, right?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/kotlintest/kotlintest/issues/969?email_source=notifications&email_token=AAFVSGVHRBF2VS5QEARHQZ3QHZ67RA5CNFSM4ITAR5G2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5YSVHQ#issuecomment-527510174,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAFVSGQVHHNBYHF7HWM3TKLQHZ67RANCNFSM4ITAR5GQ
.

Then it's not exactly generating, right? It's just a sequence

Maybe a randomized sequence, but it's not entirely random

This sounds very related to #971, unique modifier.

Unique is different but related. So if you had Gen.int().unique() (or however we do it) then it would be 1000 unique random values for example.
But the idea of exhaustive mode is that if you say Gen.int().filter { it < 500 } and you ask for GenerationMode.EXHAUSTIVE then it shouldn't use random values because you might not get all 500. In fact you probably wouldn't. You might get the number 3 ten times, but never the number 453 for example.

Under the current design of Arbitraries and Progressions we don't need an exhaustive flag anymore so closing this. Will reopen if we go back to it.


Support feature flags through KnativeServing CRD in the Operator

Problem
It would be nice if the knative feature flags could be configurable through the KnativeServing CRD.
https://knative.dev/docs/serving/configuration/feature-flags/
Persona:
System Operator
Exit Criteria
Feature flags can be configured by the Operator for knative serving through the KnativeServing CRD.

@kromanow94 Do you mean to configure the config-features ConfigMap?
The operator is able to configure any ConfigMap available with knative serving and eventing: https://knative.dev/docs/install/operator/configuring-with-operator/#configuring-knative-by-using-the-operator

Maybe the config-features config map is supported by the operator through code but not in the CRD? I wasn't able to find the features: section in the CRDs:

config/crd/bases/operator.knative.dev_knativeservings.yaml
config/charts/knative-operator/templates/operator.yaml


Remove the  Routing option when registering a template from the UI

ISSUE TYPE

Improvement request

COMPONENT NAME
Component: UI
CLOUDSTACK VERSION
Cloudstack 4.18
SUMMARY
Remove the  Routing option when registering a template from the UI
Steps to reproduce the issue
Register a template

Currently the routing option is only used by the Virtual router.
If users need, they can update template type to ROUTING

Since we have a streamlined approach with making a template a systemvmtemplate
https://www.shapeblue.com/systemvm-template-upgrade-improvements/
we can consider to remove the routing option when registering a template

@kiranchavala , I don't agree with this. Because the user can edit a template, shouldn't mean they cannot immediately on registering give it the right properties.

@DaanHoogland  then we should also show other template types (user,system,builtin)  when registering a template.
why is it only limited it to only routing
Also the documentation related to registering a template doesn't mention about the routing part

@DaanHoogland @kiranchavala
do we think this is a bug and need a fix ?

@DaanHoogland @kiranchavala do we think this is a bug and need a fix ?

I don't think so, it is a polishing issue with low prio. It can be added in any release.


Length of Persistent Vector storage

I don't see how Vector store its length.
https://github.com/near/near-sdk-rs/blob/5776439491816c560fb5652ac5fad73973b5b6fb/near-sdk/src/collections/vector.rs#L100-L104
Should developer store it separately? Or I missing something?

As discussed length is serialized together with the Vector struct itself.


Remove bluebird each

Uhhhh, sorry about that, I was trying to merge your changes into my fork and not the other way around :)


Will helm charts become unusable?

Hi!
Some question about the deprecation timeline.
At nov of 2020: "At 1 year, support for this project will formally end, and this repo will be marked obsolete"
"This timeline gives the community (chart OWNERS, organizations, groups or individuals who want to host charts) 9 months to move charts to new Helm repos, and list these new repos on the Helm Hub before stable and incubator are de-listed"
For me this means that the charts will be removed and no longer available.
However as I am browsing through the charts I realize that most of them doesn't mention anything about their new repo neither in their github README nor in their helm hub description. This calmness of the different projects makes me unsure.
Maybe the charts will be able to be used after that?
Or should I avoid this repository as plague because huge amount of the chart will be deleted at Nov?
And write my own charts by hand(if no other exists for the given image I needed)?
Some clarification please :)

Anyone?


merge package from arlac77/npm-package-template

package.json

chore(package<EMAIL_ADDRESS>chore(scripts): cover@#overwrite c8 --temp-directory build/tmp ava && c8 report -r lcov -o build/coverage --temp-directory build/tmp
chore(package): add main from template
chore(package): set $.ava.require='esm' as in template
chore(package): set $.ava.files='tests/*-test.mjs' as in template
chore(package): set $.ava.extensions='mjs' as in template
chore(package): remove unknown value for main ({{main}})

Pull Request Test Coverage Report for Build 1809

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 99.7%

Totals

Change from base Build 1808:
0.0%

Covered Lines:
302

Relevant Lines:
302

💛  - Coveralls


Add support for typing.Never

Initial Checks

[X] I confirm that I'm using Pydantic V2

Description
I get an error. I understand that it should work without Never, but I still want to point it out sensitively.
pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for typing.Never. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.
If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.
For further information visit https://errors.pydantic.dev/2.7/u/schema-for-unknown-type

Example Code
class PersonFilmRoles(BaseModel):
    id: str
    roles: list[str | Never]

Python, Pydantic & OS Version
pydantic version: 2.7.4
pydantic-core version: 2.18.4
python version: 3.12.3

Hi @ArtemIsmagilov,
Thanks for reporting this. I'd consider this more of a feature request - I can't really understand a practical example of where you'd want to use the typing.Never hint for a model field - could you shed some light on that?
If there's not a practical example, I think we'll probably mark this as not planned.

I want to specify a model that would accept only an empty list, without elements. to do this, I'm creating the following class. For example, I can use this validator for the API
newbie actor at first job
class PersonFilmRoles(BaseModel):
    id: str
    roles: list[Never]

newbie_actor = PersonFilmRoles(id='id', roles=[])

Hmm,
For this case, I'm still not convinced, because I feel as though
from typing import Annotated

from pydantic import BaseModel
from annotated_types import MaxLen

class PersonFilmRoles(BaseModel):
    id: str
    roles: Annotated[list, MaxLen(0)]

newbie_actor = PersonFilmRoles(id='id', roles=[])

Would be more appropriate.
That being said, thinking about it more, perhaps it makes sense as an attribute annotation if you seek to distinguish a more abstract base class from a concrete one, so I'll leave this as a feature request.
Feel free to open a PR adding support!

To be honest, I don't understand you a little. The standard type doesn't work for you. Is this really not a bug but a feature?

Feel free to open a PR adding support!
Thanks)

I suppose the categorization here isn't super critical - it's low priority either way, but we welcome PRs from the community :)

I'm not sure I really understand the use case here. Never is the bottom type, it has a specific meaning in the typing spec. Using it to indicate that a list has no element does not really follow its semantics. At the very least, I think it could make sense to have:
from typing import Never

class Model(BaseModel):
    foo: Never

Model(foo=...)  # Always fails, no matter the giving value of `foo`

Xref https://github.com/pydantic/pydantic-core/issues/619

To offer a practical use case, I've found Never handy in "knocking out" a field from a compound type for the purposes of writing @overload signatures. When a field's type comes from a type parameter, instantiating the containing structure with that parameter set to Never creates a type in which that field must be absent (or, as in the example below, must be None).
H = TypeVar('H', bound=DriveCore)

class HyperdriveConfig(BaseModel, Generic[H]):
    core: H | None = None

    antimatter_containment_pressure: float = 8.1e41
    flux_capacitance_ratio: float = 13.41
    graviton_polarization_angle: float = 0.0

    # ... 106 other fields ...

    def recalibrate(self, updates: HyperdriveConfig[Never]) -> Self:
        return self.model_copy(update=updates.model_dump())

drive = HyperdriveConfig(core=StandardCore(), flux_capacitance_ratio=-1.8)

# these are fine because they don't replace the core with an incompatible type
drive.recalibrate(HyperdriveConfig(antimatter_containment_pressure=1e42))
drive.recalibrate(HyperdriveConfig(core=None))

# ...but thanks to Never, this will not type check.
drive.recalibrate(HyperdriveConfig(core=DangerouslyIncompatibleCore()))

The above will (presumably) cause a runtime error from Pydantic, but it would be nice if this worked. I landed here after trying to do something similar in a real project, and would make use of it if it were supported.

Hum, I'm not sure Never fulfills your use case. If you only want HyperdriveConfig instances without any core set, the signature should look like recalibrate(self, updates: HyperdriveConfig[None]) -> Self.

If you only want HyperdriveConfig instances without any core set, the signature should look like recalibrate(self, updates: HyperdriveConfig[None]) -> Self.

That wouldn't type check because None is not compatible with the type bound on H. The bound would have to be relaxed to include None.
H = TypeVar('H', bound=DriveCore | None)

But that necessitates further complications if H is used in other places where None must be prohibited: now a second TypeVar must be introduced.
And even that doesn't help if None is not a permissible value for the field.
core: H = Field(default_factory=IdleCore)

In that case a new type, compatible with the bound, has to be invented as a substitute for Never.
class NeverCore(DriveCore): ...
ConfigWithoutCore = HyperdriveConfig[NeverCore]

This is the workaround used in the code that inspired this example.

I'd like to try this for v2.10 :).

@scullion I see. But your use case might be a bit different: Never is the bottom type, meaning Never | <tp> ~ <tp>. In your example:

from a type checking perspective (i.e. during static analysis), HyperdriveConfig[Never] means any instance of HyperdriveConfig where core is None (and only None) can be used, because Never | None ~ None.
from a Pydantic perspective (i.e. runtime analysis), HyperdriveConfig[Never] will create a new Pydantic with a new core schema generated. During the creation of this new model, type variables will be "replaced" by the provided type, but no simplification of the type is performed. Pydantic will thus try to generate a schema for the field core of type Never | None, and will fail because it doesn't know about Never.

Pydantic does not currently do any simplification of types because generally the user can do it when defining a model (i.e. there's no reason to define a field with type Any | None, just use Any). However, it could make sense when parametrizing generics. I think it is reasonable to add support for Any | <tp> (~ Any) and Never | <tp> (~ <tp>). This would solve your issue without having to add support for Never yet.

There seems to be some resistance to supporting Never 🙂 If that comes from a desire not to add code that "should never need to be run", I appreciate that, and can only say that the always-fails validator proposed in https://github.com/pydantic/pydantic-core/issues/619 would serve the same pupose as error handling code a Pydantic user is currently obliged to write if they define a dummy "never type" to work around the lack of Never support, such as the NeverCore constructor in the example below.
What @Viicos propses solves my real life problem, but doesn't handle the case where the type var isn't used as part of a union as shown below. Currently if the user has a field typed T rather than T | None, they are obliged to invent a type compatible with T like NeverCore to achive an effect similar to Never.
class NeverCore(DriveCore):
    def __init__(self):
        raise RuntimeError("something is quite seriously wrong")

class HyperdriveConfig(BaseModel, Generic[H]):
    core: H = Field(default_factory=IdleCore)

    def recalibrate(self, updates: HyperdriveConfig[NeverCore]) -> Self:
        return self.model_copy(update=updates.model_dump())

    def install_core(self, core: H) -> HyperdriveConfig[H]:
        self.core.safe_power_down()
        return self.model_copy(update={'core': core})

# it would be dangerous to replace the core during a recalibration
drive.recalibrate(HyperdriveConfig(core=FluxGravitonCore())) # type error

# ... instead should to use install_core()
drive.install_core(FluxGravitonCore()) # ok

Isn't having Pydantic do type algebra on unions harder to implement than just having an always-fails validator? Either would solve my use case, but as a user I'd prefer first class support for Never purely on the basis of the principle of least surprise.

There seems to be some resistance to supporting Never

Not at all! I'm just stating that your use case can be treated differently. To be clear, adding support for Never will also solve your use case. Never | None will be supported when building the schema, and validation will work as expected. That is, for a given value v:

v will try to be validated against Never, and fails because Never should always fail validation.
v will try to be validated against None, and will fails if v is not None.

But having "type algebra on unions" just for this case just makes things cleaner imo. The resulting core schema is smaller and probably easier to understand. The generated JSON Schema might be simplified as well.

This would be a cool feature for sure. I've bumped this to v2.11 - PRs welcome with some help here! Not quite high priority enough to fit into our current sprint.

This is lower priority than I thought - bumped out of milestone. PRs still welcome, of course.

@ArtemIsmagilov, note that if this is going to be implemented, it will be following the semantics specified in https://github.com/pydantic/pydantic/issues/9731#issuecomment-2194319078 (just wanted to make sure you were aware of it).

Hi y'all! I have, potentially, another need for Never support. In SQLModel, im representing a database type that has no meaning in python, and must never be accessed - neither selected, nor inserted. I would love to represent it as:
from sqlmodel import SQLModel, Field
from sqlalchemy.types import UserDefinedType

class Earth(UserDefinedType):
    def get_col_spec(self, **kwargs):
        return "earth"

class MyTable(SQLModel, table=True):
    earth: Never = Field(sa_type=Earth)

The idea is that while it makes sense to index it or run sql-specific queries on it, it has no meaning ouside of its own earthdistance extension


Visual Studio Config and Use

This issue has been moved from a ticket on Developer Community.

[severity:It's more difficult to complete my work]
I struggle with VS.  I use it for Power BI models and am daunted by the amount of configuration and operation that is required.  In short, wouldn't Visual Studio be best used for coding (.Net, Python, etc.) and have tabular models be attached to SSMS?  That would allow for a much simpler interface, no?
Thanks!
Peter Alexander

Original Comments
Feedback Bot on 5/3/2023, 11:03 PM:
(private comment, text removed)

Original Solutions
(no solutions)

Not sure why it got transferred to us, transferred to VSCore.


Update r0.12 branch

Merge important updates into the release branch.

LGTM. Feel free to merge if you don't want to wait for Jenkins.

Looks like mac0-slave needs portpicker installed.

Please include https://github.com/tensorflow/tensorflow/pull/5645 as well, fully-connected layer in contrib.layers is not Python 3 compatible in 0.12 branch right now.

#5645 is in the r0.12 branch, so it'll appear in the release.
Merging:

CLAs are covered by the master branch
Mac failures happen only on that machine because it's missing portpicker, so I'm ignoring them

mac0 is now taken care of.
Thanks for merging, Jonathan.
Now 0.12 branch is locked. :)


pytests should use trainer.__init__() not hparams where feasible

Tests are good code snippets for how to use our trainer. However, most of our tests of trainer functions (e.g. checkpointing, DDP) all rely on the hparams system, which makes it difficult for external users who are mostly non-yahp-based to grok our code. We should migrate all the tests to using trainer=Trainer(..) approaches.

Hparams is gone. Congrats @hanlint


Add resource picker to style editor

Currently objects are just misrepresented. Workaround is to use @export_file and load later.

For now I've added a hint that suggests using @export_file instead


[QUESTION] kvssink keeps dropping frames from an rtsp source

I'm trying to stream to KVS several rtsp flows (video only).
I built my environment exactly how it is described here: https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp#build. However, kvssink keeps dropping frames with this message:

DEBUG - Dropping frame with flag: 1536

and the KVS stream remains empty.
Case 1: foo (not working)
Here is my command for the camera foo:
$ gst-launch-1.0 -v rtspsrc location="rtsp://user:pass@foo" short-header=TRUE ! rtph264depay ! h264parse ! kvssink stream-name="YourStreamName" storage-size=128 access-key="aaa" secret-key="aaa" aws-region="eu-central-1"

and here is the output
Here is the fakesink:
$ gst-launch-1.0 rtspsrc location="rtsp://user:pass@foo" ! rtph264depay ! capsfilter caps="video/x-h264" ! fakesink

here is the output
Even in VLC, the video is properly streamed.
Case 2: bar (working!)
I tried the same command with another camera (bar) and in that case, all the things work:
$ gst-launch-1.0 -v rtspsrc location="rtsp://user:pass@bar" short-header=TRUE ! rtph264depay ! h264parse ! kvssink stream-name="YourStreamName" storage-size=128 access-key="aaa" secret-key="aaa" aws-region="eu-central-1"

and here is the output
Case 3: baz (not working, again)
Strangely enough, if I try with the camera baz (same model of camera bar), things do not work again (dropped frames).
$ gst-launch-1.0 -v rtspsrc location="rtsp://user:pass@baz" short-header=TRUE ! rtph264depay ! h264parse ! kvssink stream-name="YourStreamName" storage-size=128 access-key="aaa" secret-key="aaa" aws-region="eu-central-1"

here is the output
Here is the fakesink for this case
$ gst-launch-1.0 -v rtspsrc location="rtsp://user:pass@baz" short-header=TRUE ! rtph264depay ! h264parse ! fakesink

and here is the output
Again, baz streams properly to VLC.
Unfortunately, I can't access to any camera control panel. I can just consume the rtsp streams as they are.
I'd kindly ask you some help on how to troubleshoot this kind of problem.

The frame dropping message should be benign.  Here are the frame flags:
https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=c#GstBufferFlags
So 1536 = 1024 + 512 meaning both GST_BUFFER_FLAG_HEADER and GST_BUFFER_FLAG_MARKER are set, and 1600 = 1024 + 512 + 64 meaning GST_BUFFER_FLAG_HEADER, GST_BUFFER_FLAG_MARKER, GST_BUFFER_FLAG_DISCONT are set.  These will be dropped but should not impact the stream, in fact I see this happens in the bar case which seems to be working fine.
I did notice this in your logs:
Could not receive any UDP packets for 5.0000 seconds, maybe your firewall is blocking it. Retrying using a tcp connection.
If you know you will not support UDP in order to avoid this latency you can set this property:  https://gstreamer.freedesktop.org/documentation/rtsp/rtspsrc.html?gi-language=c#rtspsrc:protocols to only tcp because if you leave it to the default it will always try UDP first which will incur latency for you.
In theory fixing the above to TCP only with your network configuration shouldn't fix the issue but I wouldn't be surprised if there were some extra issues that show up when you're renegotiating the rtsp stream the camera so I would definitely suggest to make this change if you know you will not support UDP.
This likely will not fix your issue but I see your terminate the pipeline in the foo case after 13s.  Can you please try to run it for 1-2 minutes and see if we can get some error codes returned from the SDK to be printed in the logs?


Potential double scaling of lpips_loss in tools\train_vqvae.py

Hi, thank you for creating such great content! I’ve been following your diffusion series on YouTube, and it’s been incredibly insightful. While exploring your Stable Diffusion GitHub repository, I noticed something in tools/train_vqvae.py that I wanted to bring to your attention. It seems the lpips_loss might be scaled twice by acc_steps, which could affect the calculation. I’m not sure if this is intentional or if I might have misunderstood something. Thank you for all your hard work, and I really appreciate your contributions!

Thank you @Malengu for your kind words
Yes you are absolutely right, I incorrectly scaled by 1/acc_steps in two places here .
Infact because the same code was also copy pasted for some of other repos, this error ended up in those as well. Have fixed this for repo with this commit and also have fixed it in other repos.
Thank you so much for bringing this to my attention.


Add Configure-authentication config file support

Story to unwrap configure-authentication
Currently, we have a wrapper around configure-authentication to allow the configurations to be passed in using --config config.yml. Would like to merge that behavior into om.
Proposed changes:
adding a --config flag which takes configuration file for auth.
File format: replicate flat flag structure
example:
username:
password:
decryption-passphrase:
http-proxy-url:
https-proxy-url:
no-proxy:

Closing this, merged.