Adding JSDoc tag useStrict to output strict() modifier

Why
This solves https://github.com/fabien0102/ts-to-zod/issues/78 using a new @useStrict JSDoc tag key.
/** @useStrict */
export interface A {
    a: string
}

will generate the following:
export const aSchema = z.object({
    a: z.string()
}).strict();

and
export interface A {
    /** @useStrict */
    a: {
        b: number
    }
}

will generate
export const aSchema = z.object({
    a: z.object({
        b: z.number()
    }).strict()
});

Might be missing some safeguards to prevent use of strict on other types

Codecov Report

Merging #131 (cbf776f) into main (7bbed16) will increase coverage by 0.04%.
The diff coverage is 100.00%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@            Coverage Diff             @@
##             main     #131      +/-   ##
==========================================
+ Coverage   95.00%   95.05%   +0.04%
==========================================
  Files          13       13
  Lines         521      526       +5
  Branches      179      181       +2
==========================================
+ Hits          495      500       +5
  Misses         23       23
  Partials        3        3

Impacted Files
Coverage Δ

src/core/jsDocTags.ts
98.66% <100.00%> (+0.09%)
:arrow_up:

Nice PR! Regarding the safeguard, I think this is fine for now, as example we have @pattern that could have the same problem, and so far… this is not a problem!

For some reason, it doesn't work on the full generation, added a test case here
I can work on that, but not before next week unfortunately.


Theta method full newton nswe solver

Reopening this work as a different PR. The old PR cannot be merged into master (due to some mess that I did yesterday in the master branch, but that is now solved).
Hi @davelee2804 @cbladwell,
in this PR I have written a "skeleton" code (it is just a partial implementation, to be completed/validated) of a NSWE solver using Gridap's high level API.

It uses the APVM method for spatial discretization, i.e., Eqs. (70)-(73) in Rognes et. al and the theta method implicit method for time discretization (the fully discrete nonlinear problem to be solved at each time step is in particular avaliable in Eqs. (75)-(78) of that paper).
You can see the evaluation of this nonlinear operator (i.e., residual) coded here: https://github.com/gridapapps/GridapGeosciences.jl/commit/88418fe8cdf327412822361733626dd4988056db#diff-3d44a05a1e12bbf492c4d4eea9a151784def9fe3a3c34312c3e9a5f35e106ff6R208
where the first argument is the current evaluation point, and the latter a basis of the monolothic system test functions.

However, some caveats:

I could NOT compute the Jacobian of this operator using AD (because of a known issue in Gridap 0.16 that has to be solved; see https://github.com/gridap/Gridap.jl/issues/584 for details) We will need an approximate Jacobian, or to resort to a different discretization, e.g., the one at the end of Section 5.5 of Rognes paper (this discretization is not energy conserving, though).
The JIT compilation of the code triggered from res down the tree takes forever with Julia 1.6.2 (it seems to enter in a infinite loop, although I could not confirm it). With Julia 1.5.4 the issue disappears. Again, I think similar issues have appeared with Julia 1.6+Gridap in similar contexts. We should investigate this further.

Also I wonder the following:

Can we actually exploit the DIV+reference integration for this term: (∇⋅(v))*(0.5*ui⋅ui) ?
Is the current (partial) implementation correct? (as I could not validate it yet)

@davelee2804 @cbladwell perhaps you can contribute and complete it (essentially implement a crude approximation of the Jacobian as @davelee2804 mentioned). What do u think?  I think that this driver shows that we have all the ingredients in Gridap required to solve the NSWEs at the discretization level (e.g., I could evaluate the residual as implemented in res).  I am happy with you completing the current method, or to consider another method as well (e.g., one of Cotter's book for the NSWE different from the APVM).
Please, once you start modifying the driver, can you please open a branch and PR to this branch so that I can help you and review etc. ? Thanks!
Tasks

[x] Complete the exact Jacobian, i.e., complete the jac function definition statement. https://github.com/gridapapps/GridapGeosciences.jl/blob/11fadb3caad4585c5e30526d0db58d865ad0bdb6/test/ThetaMethodFullNewtonNSWECubedSphereTests.jl#L171
[x] Implement/Complete diagnostics (currently commented out from WaveEquationSolver)
[ ] Validate the solver against different benchmarks from Williamson et. al.
[x] Solve the following question: Can we actually exploit the DIV+reference integration for this term: (∇⋅(v))*(0.5*ui⋅ui) ? (see PR https://github.com/gridapapps/GridapGeosciences.jl/pull/12 for more details)
[x] Investigate issues with AD of the fully coupled system. (no longer needed, we have the manual Jacobian).
[ ] UPDATE: by a more clever ad-hoc implementation of the Backward Euler method I could reduce first JIT compilation times to around 10 mins with Julia 1.5.4. Not sure with Julia 1.6.2., we will see in Github actions. Investigate JIT compilation issues with Julia 1.6.2. Switch to Julia 1.5.4 in the meantime.

Codecov Report

Merging #22 (0cfd79f) into master (ad29e55) will decrease coverage by 8.05%.
The diff coverage is 0.00%.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ad29e55...0cfd79f. Read the comment docs.

https://github.com/gridapapps/GridapGeosciences.jl/blob/theta_method_full_newton_nswe_solver/src/ShallowWaterThetaMethodFullNewton.jl#L96
I'm not sure that dt/2 is always the best value for \tau. In my experiments with the Rosenbrock solver I found that a shorter value is sometimes better. I would pass this in as a parameter rather than hard code it.

https://github.com/gridapapps/GridapGeosciences.jl/blob/theta_method_full_newton_nswe_solver/src/ShallowWaterThetaMethodFullNewton.jl#L107
This is a style comment (but a major one), and I leave it up to you to make the decision as to weather to do this or not, but I would express the solution variables as un, hn, uk, hk, not un, hn, \Delta u, \Delta h, and then solve for \Delta uk, \Delta hk, with uk = uk + \Delta uk, hk = hk + \Delta hk. I know that what you are doing is consistent with what Rognes et al do, and that this works for the \theta method, but it is not a very helpful formulation for the Crank-Nicolson or energetically balanced schemes, which represent the temporal cross terms differently. Again, your decision here....

https://github.com/gridapapps/GridapGeosciences.jl/blob/theta_method_full_newton_nswe_solver/src/ShallowWaterThetaMethodFullNewton.jl#L151
https://github.com/gridapapps/GridapGeosciences.jl/blob/theta_method_full_newton_nswe_solver/src/ShallowWaterThetaMethodFullNewton.jl#L160
The vorticity is being diagnosed for both the diagnostics and the field dumps. Most likely this will mean it is being computed twice. Perhaps a boolean could be added to the write_solution call to ensure this isn't called twice

https://github.com/gridapapps/GridapGeosciences.jl/blob/theta_method_full_newton_nswe_solver/src/ShallowWaterThetaMethodFullNewton.jl#L7
Can we just use compute_mass_flux from the explicit solver here?

https://github.com/gridapapps/GridapGeosciences.jl/blob/theta_method_full_newton_nswe_solver/src/ShallowWaterThetaMethodFullNewton.jl#L14
Can we just use diagnose_potential_vorticity from the explicit solver instead?

https://github.com/gridapapps/GridapGeosciences.jl/blob/theta_method_full_newton_nswe_solver/src/ShallowWaterThetaMethodFullNewton.jl#L2
Should this be moved into the initial conditions file?

Hi @davelee2804 ! Thanks for your insightful comments! I will address them carefuly and come back to you. Note that the most appropriate way to add comments in a PR (next time) is directly in the line of code in which you want to make your point, in the "Files changed" tab of the PR page.

In the "Files changed" tab of the PR page, and sitting on a given line of code, you have to click on the + sign which is popped up when you pass the mouse pointer in between the line of code identifier and the line of code itself.

Thanks @amartinhuertas , I was wondering about that. will do

Hi @davelee2804 ! I have addresed all your comments except the one that involves more work/more careful thinking (i.e., https://github.com/gridapapps/GridapGeosciences.jl/pull/22#issuecomment-932673370). I am happy to do this work as well, although I prefer to devote some more time to understand. (also our conversations in slack etc.)
In regards to:
https://github.com/gridapapps/GridapGeosciences.jl/pull/22#issuecomment-932673867
note that I have also done it. BUT ... the same can be applied to the other time steppers (if I am not wrong). Can you apply the patch there as well?
On the other hand, and related to the previous, I have the impression that we are replicating a very similar pattern in all time steppers (this is the cause we have to apply the patch in all places). It is not a massive amount of code the one we are replicating (basically control of the time integration loop), but perhaps we may think a little bit more how can we abstract the time integration loop into a  structure/pattern common to all solvers (if such possibility exists/it is convenient)

Hey @amartinhuertas , thanks for all your work here, the code is looking great, very succinct. With regard to https://github.com/gridapapps/GridapGeosciences.jl/pull/22#issuecomment-932673370, what is your preference? You have done a lot of great work here, and I'm happy to merge in the PR as is. We could then look at re-formulating the nonlinear terms in the residuals as future work (it is called a 'ThetaMethod' after all). What do you preference? One final question: what is the energy conservation error? The explicit and IMEX integrators loose O(1.0e-9) over the simulation, the Rosenbrock a bit more (on account of the approximate Jacobian). What is the loss here? Also can you run time centered with the super long time step (9600s), or does this require a shorter time step?

Hey @amartinhuertas , thanks for all your work here, the code is looking great, very succinct. With regard to #22 (comment), what is your preference? You have done a lot of great work here, and I'm happy to merge in the PR as is. We could then look at re-formulating the nonlinear terms in the residuals as future work (it is called a 'ThetaMethod' after all). Otherwise I can help with the re-formulation of the nonlinear terms as Crank-Nicolson and/or energetically balanced. I'm happy either way.

Let me think about this a little bit more and then we decide.

One final question: what is the energy conservation error? The explicit and IMEX integrators loose O(1.0e-9) over the simulation, the Rosenbrock a bit more (on account of the approximate Jacobian). What is the loss here?

For BE and dt=9600s I shared the full diagnostics file in Slack ...  can you generate the plots? (note that i could not run the full 20 days within the 48 hours wall clock time).

Also can you run time centered with the super long time step (9600s), or does this require a shorter time step?

I tried Trapezoidal Rule with dt=9600s, and Newton for the first time step diverged:
Iter     f(x) inf-norm    Step 2-norm
------   --------------   --------------
     0     1.501067e+11              NaN
     1     1.105992e+25     4.185006e+37
     2     4.350110e+28     5.400823e+42
     3     1.553421e+26     5.486737e+42
     4     6.189773e+26     6.446322e+41
     5     1.779207e+28     1.368093e+43
     6     2.114128e+30     3.841482e+45
     7     8.529852e+31     4.038874e+46
     8     4.711679e+39     1.021888e+52
     .....
    23              NaN    3.825513e+264

I have reduced dt to 4800 (i.e, halved dt), and seems to be running ... will share when it finishes.

Hey @amartinhuertas , Sorry, I wasn't very clear, I was intending to ask about the energy conservation for the time centred scheme (it was late and I wasn't thinking properly). Here is the plot for the backwards euler energy conservation.

This is not dissimilar to the Rosenbrock energy conservation error, and is probably to be expected given then excessive damping of the BE scheme that @santiagobadia mentioned. I'm impressed that the time centred scheme is running with a time step of 4800s. Also (separate note), love that you added the vortex roll up movie to the geosciences page. I could look at that thing for hours :)

The JIT compilation of the code triggered from res down the tree takes forever with Julia 1.6.2 (it seems to enter in a infinite loop, although I could not confirm it). With Julia 1.5.4 the issue disappears. Again, I think similar issues have appeared with Julia 1.6+Gridap in similar contexts. We should investigate this further.

A follow up on this very annoying issue. Together with Julia compiler experts, we could isolate this issue, and narrow down where it does come from. To access to a workaround we can temporaritly use the Gridap.jl branch compilefast .  I.e.,  by typing pkg> add Gridap#compilefast. Alternatively, we can downgrade compiler optimizations to O1 running julia as julia -O1.
See also https://github.com/gridap/Gridap.jl#known-issues


size mismatch error

Hi
Recently, I added a pre-trained AlexNet model to your custom models. When I download the model from OSF, I get size mismatch error:
Error(s) in loading state_dict for AlexNet:
	size mismatch for features.0.weight: copying a param with shape torch.Size([96, 3, 11, 11]) from checkpoint, the shape in current model is torch.Size([64, 3, 11, 11]).
	size mismatch for features.0.bias: copying a param with shape torch.Size([1, 1, 1, 96]) from checkpoint, the shape in current model is torch.Size([64]).
	size mismatch for features.3.weight: copying a param with shape torch.Size([256, 48, 5, 5]) from checkpoint, the shape in current model is torch.Size([192, 64, 5, 5]).
	size mismatch for features.3.bias: copying a param with shape torch.Size([1, 1, 1, 256]) from checkpoint, the shape in current model is torch.Size([192]).
	size mismatch for features.6.weight: copying a param with shape torch.Size([384, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([384, 192, 3, 3]).
	size mismatch for features.6.bias: copying a param with shape torch.Size([1, 1, 1, 384]) from checkpoint, the shape in current model is torch.Size([384]).
	size mismatch for features.8.weight: copying a param with shape torch.Size([384, 192, 3, 3]) from checkpoint, the shape in current model is torch.Size([256, 384, 3, 3]).
	size mismatch for features.8.bias: copying a param with shape torch.Size([1, 1, 1, 384]) from checkpoint, the shape in current model is torch.Size([256]).
	size mismatch for features.10.weight: copying a param with shape torch.Size([256, 192, 3, 3]) from checkpoint, the shape in current model is torch.Size([256, 256, 3, 3]).
	size mismatch for features.10.bias: copying a param with shape torch.Size([1, 1, 1, 256]) from checkpoint, the shape in current model is torch.Size([256]).
	size mismatch for classifier.1.weight: copying a param with shape torch.Size([1, 1, 4096, 9216]) from checkpoint, the shape in current model is torch.Size([4096, 9216]).
	size mismatch for classifier.1.bias: copying a param with shape torch.Size([1, 1, 1, 4096]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for classifier.4.weight: copying a param with shape torch.Size([1, 1, 4096, 4096]) from checkpoint, the shape in current model is torch.Size([4096, 4096]).
	size mismatch for classifier.4.bias: copying a param with shape torch.Size([1, 1, 1, 4096]) from checkpoint, the shape in current model is torch.Size([4096]).
	size mismatch for classifier.6.weight: copying a param with shape torch.Size([1, 1, 5, 4096]) from checkpoint, the shape in current model is torch.Size([565, 4096]).
	size mismatch for classifier.6.bias: copying a param with shape torch.Size([1, 1, 1, 5]) from checkpoint, the shape in current model is torch.Size([565]).

Do you have any suggestions to solve this issue?

Can you please provide the architecture definition of your custom model? I think there is a misspecification somewhere, and it would be helpful to compare existing AlexNet definition with the one you've added, to better spot where the misalignment takes place. Then it should be easier to see what needs to change to fix the bug you're seeing.

I used pre-trained weights from salient object subitizing (SOS) image dataset. This is the reference paper. The AlexNet has been trained using Caffe and the weight is available here and you can find the model definition as a prototxt file in the same link. I converted the Caffe file using caffemodel2pytorch and change the layers’ name to be compatible with the THINGSvision naming convention.

Hi
I was wondering whether you have any comments on why the custom AlexNet is not working?

Hi Ali,
First of all thanks for adding the model! The version of AlexNet they used in their paper and the one implemented in torchvision are not entirely identical. If you compare the model definition from your prototxt file with the torchvision model:
AlexNet(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(11, 11), stride=(4, 4), padding=(2, 2))
    (1): ReLU(inplace=True)
    (2): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
    (3): Conv2d(64, 192, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))
    (4): ReLU(inplace=True)
    (5): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
    (6): Conv2d(192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (7): ReLU(inplace=True)
    (8): Conv2d(384, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (9): ReLU(inplace=True)
    (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace=True)
    (12): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(6, 6))
  (classifier): Sequential(
    (0): Dropout(p=0.5, inplace=False)
    (1): Linear(in_features=9216, out_features=4096, bias=True)
    (2): ReLU(inplace=True)
    (3): Dropout(p=0.5, inplace=False)
    (4): Linear(in_features=4096, out_features=4096, bias=True)
    (5): ReLU(inplace=True)
    (6): Linear(in_features=4096, out_features=565, bias=True)
  )
)

you'll see that they differ in the number of feature maps used in the convolutional layers (e.g. 96 in the Caffe version vs 64 in the torchvision version). Also, you defined the number of outputs as 565, while it's only 5 in the paper.
My suggestion would be to alter the torchvision implementation to fit your model and add it to the custom model class you created for thingsvision. It also seems like you might have to .squeeze the weights you get from caffemodel2pytorch (to remove the unnecessary dimensions from the biases and fully connected layer weights).
Let us know if this works and if you run into any other issues!

@alireza-kr @andropar can we close this issue or is there still something that needs to be resolved on our end?

Nothing to be resolved on your side. Thank you @LukasMut


This repository needs "refreshment"

By refreshment I think the addition of the "Feature Plugins" as introduced here is required. In addition, I would really like to see the addition of the sample code (no need to drag the whole [Navigation Skeleton(https://github.com/aurelia/skeleton-navigation) to show the use of this plugin. Without such sample this feels like half of the "total".

I'm not following you entirely. This is a repo for someone who wants to make a plugin, not use one. Can you clarify more?

Well, you need to drop your level of understanding of Aurelia Plugin concept to the average novice to Aurelia programming model (a service I am trying to provide here :-) ). Since I am working on the Aurelia-Guides, I noted a lot of confusion about what plugins are, and then both how to create and subsequently use them; the assumed case where one person creates a plugin and a completely different one consumes is less often present with the introduction Feature Plugins:

The above plugin API is designed to enable the installation of external, 3rd party plugins. However, you may wish to organize your own project as a series of plugins. To do this, simply create a folder for your internal "feature" plugin. In that folder create an index.js file that exports a single configure function. This function works the same as 3rd party plugins. To register your feature for configuration at startup, you would use the following code:

aurelia.use.plugin('plugin-name', config => { /* configuration work */ });

I would indeed be even little more specific and show something like:
export function configure(aurelia) {
  aurelia.use
    .standardConfiguration()
    .plugin('hello world');  // install the plugin

  aurelia.start().then(a => a.setRoot());
}

and then finish with a few sentences explaining where to put the generated plugin      files (say E6 version)

so that the project that consumes this plugin.
This last part is important because of all commotion related to "relative paths" / "absolute paths" issues that jspm recently caused
Making sense?
Note that I could not decide off the top off my head whether the plugin name should be hello world or hello-world or something else since these naming conventions (camel case, pascal, introduction of the dash between words are not always intuitive

This repo has nothing to do with "feature" though. This is a starter kit for people who want to create a 3rd party plugin that does not exist inside your project but that is installed into your app via jspm.

Understood and agreed seeing your comment - particularly the "that is installed into your app via jspm", a fact that completely escaped me until now. Please consider adding this explanation.

I have resolved all of my issues in this thread (nearly all of were my typos) and learned enough through chasing the solutions for these issues that I am now able to write a very detailed tutorial on plugins for Aurelia-Guides site. Please feel free to close this thread

This is slightly related i guess... Any chance we can get this repo updated to match the skeleton nav repo's dependencies and the karma setup + gulp tasks?.

Update - i am working on a PR for this, will submit very soon.

I think this can be closed.


Fix no-extraneous-dependencies: Take 2

Description
Fixes #1001
Follow up from https://github.com/Shopify/quilt/pull/1026
This PR is split into 2 commits. The first contains only the changes required to make the import/no-extraneous-dependencies rule work for this repo, the second is all the lint fixes required to pass CI.
The fix here was to use the import/resolver setting to override the typescript resolver from plugin:shopify/typescript. The typescript resolver was problematic because it resolves modules to their source location, causing the rule think of them as "internal" (not extraneous) modules. Creating a new configuration and extending it before extending the plugin:shopify/typescript tells the rule to use the node resolver, which looks for these packages in node_modules instead and properly sees them as "external" to the eslint rule.
The downside to having this rule enabled is that it (correctly) enforces all our @shopify/* dependencies to be listed when they are used and thus increases the dependency web and number of "patch" dependency bumps needed (releasing a new version of dependency x causes all related dependencies that use x  to need a new version). The largest offender of this was @shopify/react-testing, which is used in all of the @shopify/react-* dependencies.
The fix for this was to add an override for test files that disables this rule. We don't care if @shopify/react-testing is listed or not since its really just needed for development and will never cause a missing dependency error when installed in a consuming package, which is the only thing we really care about here.

I wonder if there's some upstream PR we can make to https://github.com/alexgorbatchev/eslint-import-resolver-typescript to allow support for our monorepo use case


Removed a couple of obsolete (I think) @Ignore

Closing for now, we can deal with this again later


Set lport address correctly when MAC or IP address is empty

Passing an empty ("") IP address to OvnNbctl.lport_set_addresses(),
which is what we do in OvnScenario._create_lports(), results in an
invalid address string (i.e. there is a trailing space) being passed to
ovn-nbctl. This is not the expected behavior and it also triggers a
warning in the ovn-northd logs:
2018-03-06T16:14:48.111Z|00007|ovn_util|INFO|invalid syntax '28:82:41:16:5f:ae ' in address
Generate the address string correctly by checking if MAC and IP
addresses were given and that they are not empty.

I've opened an accompanying PR with a test - https://github.com/l8huang/rally/pull/6.

@hzhou8 I've addressed the multiple IPs case. Can you please take another look? Test extended with a new case as well.


Question: sometimes Postfix thinks <IP_ADDRESS> is the client_ip

Sometimes when receiving mail (from an external server) I can see something like this in the logs:
postfix/smtpd[4657]: connect from unknown[<IP_ADDRESS>]
postfix/smtpd[4657]: Anonymous TLS connection established from unknown[<IP_ADDRESS>]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
postfix/cleanup[4665]: 78DA3189E7E: milter-reject: END-OF-MESSAGE from unknown[<IP_ADDRESS>]: 5.7.1 rejected by DMARC policy for google.com<EMAIL_ADDRESS><EMAIL_ADDRESS>proto=ESMTP helo=<mail-oi0-x249.google.com>

So postfix only sees the IP of the docker host (<IP_ADDRESS>) instead of the real client IP. This causes different problems: SPF fails and eventually fails DMARC (as in the log above), or the spam score from spamassassin contains "ALL_TRUSTED=-1" as the mail server thinks the mail came from a trusted network.
Others are having the same problem with Docker & Postfix: http://stackoverflow.com/questions/39517593/postfix-docker-and-inconsistent-ip-addresses-from-host
I'm using docker compose (yaml v2) with no specific network settings (so it uses the default bridged mode) and no other proxies, the ports are directly exposed.
I was wondering if you ran into the same problem / know the reason for this?

It's a docker-side issue, there are two solutions for this problem :

use --net=host : not viable, less isolation = more security flaws.
disable the userland proxy : not viable at the moment, this options breaks kernel in a very bad way in some cases and it's incompatible with --iptables=false option.

But in my case, I don't have problems with SPF and DMARC, maybe this issue is inconsistent. I will check on my own server after work tonight.
https://github.com/docker/docker/issues/15086
https://github.com/docker/docker/issues/14856

Thanks a lot for the issue links, I subscribed there.
Yes, it seems to be inconsistent, e.g. https://github.com/docker/docker/issues/7540. I mostly see this happen with mails from Gmail, but this may be coincidence. Would be interesting if you have the same entries in the logs as it mostly goes unnoticed. You could grep for "unknown[<IP_ADDRESS>]" in the mail.log
As this is obviously a docker problem and not related to Postfix (I saw the same issue in my Nginx logs now) you can close the issue here.

You could grep for "unknown[<IP_ADDRESS>]" (or whatever your Docker gateway IP is) in the mail.log

I have no result for "<IP_ADDRESS>" in mailserver log.

Thanks a lot for the issue links, I subscribed there.

Me too :) I hope this will be fixed in a future release.

Hi,
same issue for me after I've update docker from 1.12.x to 17.06.x.
2017-09-18T21:12:23.335362+00:00 mail postfix/smtpd[623]: connect from unknown[<IP_ADDRESS>]
Also with the new docker version I cannot get the client ip from my nginx container anymore.
Unfortunately setting userland-proxy=false makes it worse since the mailserver is then not able to get traffic anymore. But sending still works. But: Nginx then gets the client ip. What did docker do?

There are still many people having this problem, check the docker issues linked in the second post. Also https://github.com/moby/moby/issues/15086#issuecomment-268205022 mentions the IPv6 module which I think might be true as not all external connections show up as <IP_ADDRESS> (or <IP_ADDRESS> in my case), only 1/4 to 1/3 of the incoming IPs in the logs are 172.% but none are IPv6 addresses...
So maybe with docker you also updated your kernel to a "bugged" version or enabled IPv6?

I did not enable ipv6 and I don't see those unknown[172 messages in my mail logs either. I do have one host that turned on ivp6 and I see those messages in the mail logs there.

I'm on Ubuntu 17.04 with 4.10.0-33. Before I was using Centos wihtout these issues. mh.

I've now disabled IPv6 like this: https://askubuntu.com/questions/309461/how-to-disable-ipv6-permanently
Now I get real client IPs without setting userland-proxy to false or anything else.


nixosTests.sway: add swaylock test

Motivation for this change
Wanted to more easily diagnose #158025, but turns out that's more specific than I thought. Still a good thing to have.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

@ofborg test sway


Hikvision RTSP video streams that require authentication cannot be played

Unfortunately we can't answer all questions. Unclear questions or questions with
insufficient information may not get attention.
Before filing a question:

Ask general Android development questions on Stack Overflow
Search existing issues, including issues that are closed
https://github.com/androidx/media/issues?q=is%3Aissue
For ExoPlayer-related questions, please also check for existing questions on
the ExoPlayer tracker:
https://github.com/google/ExoPlayer/issues?q=is%3Aissue

When filing a question:
Describe your question in detail.
In case your question refers to a problem you are seeing in your app:
Hikvision's RTSP format video stream that requires authentication cannot be played, in fact, the format is normal, however, it is normal to play with VLC, please tell me what to do，I have joined the required libraries as per the documentation
implementation "androidx.media3:media3-exoplayer-rtsp:1.0.1"

Output of running $ adb bugreport in the console

dispatcher 0x733df5c1c0 ~ ignoring unknown event type 0x736f6674
handleWindowVisibility: no activity for token android.os.BinderProxy@beba17a
dispatcher 0x72d08abc00 ~ ignoring unknown event type 0x736f6674
restart watching
callActivityOnCreate
pkgName:com.example.exoplayercodelab old windowMode:0 new windoMode:1, isFixedSize:false
add activity client record, r= ActivityRecord{bd063b4 token=android.os.BinderProxy@beba17a {com.example.exoplayercodelab/com.example.exoplayer.PlayerActivity}} token= android.os.BinderProxy@beba17a
Init 8374352 [AndroidXMedia3/1.0.1] [HWALP, ALP-AL00, HUAWEI, 29]
getStreamVolume streamType: 3 volume: 6
isStreamMute   streamType: 3
getStreamMaxVolume  treamType: 3
mMoveStepInDp: 64, mMoveStepInPixel: 218, mUpTimeDelayed: 100
ViewRootImpl mIsInProductivePCDisplay: false
OPTIONS rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream RTSP/1.0
User-Agent: AndroidXMedia3/1.0.1
CSeq: 0
pkgName:com.example.exoplayercodelab old windowMode:1 new windoMode:1, isFixedSize:false
dispatchInputInterval 1000000
notifyActivityState pkg:com.example.exoplayercodelab/com.example.exoplayer.PlayerActivity state:2 fg:true mUid:10506
RTSP/1.0 200 OK
CSeq: 0
Public: OPTIONS, DESCRIBE, PLAY, PAUSE, SETUP, TEARDOWN, SET_PARAMETER, GET_PARAMETER
Date:  Wed, Apr 26 2023 14:04:10 GMT
onAttached, package=com.example.exoplayercodelab, windowType=1, mIsHiTouchRestricted=false
DESCRIBE rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream RTSP/1.0
User-Agent: AndroidXMedia3/1.0.1
CSeq: 1
EGLint new_window_surface(egl_winsys_display *, void *, EGLSurface, EGLConfig, egl_winsys_surface **, EGLBoolean) returns 0x3000
RTSP/1.0 401 Unauthorized
CSeq: 1
WWW-Authenticate: Digest realm="IP Camera(C2732)", nonce="9f479ccb412b209b411039131bbfec69", stale="FALSE"
WWW-Authenticate: Basic realm="IP Camera(C2732)"
Date:  Wed, Apr 26 2023 14:04:10 GMT
DESCRIBE rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream RTSP/1.0
User-Agent: AndroidXMedia3/1.0.1
CSeq: 2
Authorization: Digest username="admin", realm="IP Camera(C2732)", nonce="9f479ccb412b209b411039131bbfec69", uri="rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream", response="f68adbefef6b0fa70fb865cf123cb16a"
RTSP/1.0 200 OK
CSeq: 2
Content-Type: application/sdp
Content-Base: rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream/
Content-Length: 752
v=0
o=-<PHONE_NUMBER>717533<PHONE_NUMBER>717533 IN IP4 <IP_ADDRESS>
s=Media Presentation
e=NONE
b=AS:5100
t=0 0
a=control:rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream/
m=video 0 RTP/AVP 96
c=IN IP4 <IP_ADDRESS>
b=AS:5000
a=recvonly
a=x-dimensions:1920,1080
a=control:rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream/trackID=1
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=420029; packetization-mode=1; sprop-parameter-sets=Z00AKpY1QPAET8s3AQEBQAABwgAAV+QB,aO48gA==
m=audio 0 RTP/AVP 8
c=IN IP4 <IP_ADDRESS>
b=AS:50
a=recvonly
a=control:rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream/trackID=2
a=rtpmap:8 PCMA/8000
a=Media_header:MEDIAINFO=494D4B48010200000400000111710110401F000000FA000000000000000000000000000000000000;
a=appversion:1.0
SETUP rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream/trackID=1 RTSP/1.0
User-Agent: AndroidXMedia3/1.0.1
CSeq: 3
Authorization: Digest username="admin", realm="IP Camera(C2732)", nonce="9f479ccb412b209b411039131bbfec69", uri="rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream/trackID=1", response="326cfbe5881ce96cff4aec931de334f6"
Transport: RTP/AVP;unicast;client_port=36618-36619
IOmx service obtained
removeInvalidNode all the node in jank list is out of time
In onAllocateComponent create compenent, codec name: OMX.hisi.video.decoder.avc
connecting to surface 0x72d0b5f010, reason connectToSurface
[OMX.hisi.video.decoder.avc] setting surface generation to 28574727
disconnecting from surface 0x72d0b5f010, reason connectToSurface(reconnect)
connecting to surface 0x72d0b5f010, reason connectToSurface(reconnect)
[OMX.hisi.video.decoder.avc] setPortMode on output to DynamicANWBuffer failed w/ err -2147483648
[OMX.hisi.video.decoder.avc] got color aspects (R:2(Limited), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M)) err=0(NO_ERROR)
[OMX.hisi.video.decoder.avc] got color aspects (R:2(Limited), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M)) err=0(NO_ERROR)
[OMX.hisi.video.decoder.avc] using color aspects (R:2(Limited), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M)) and dataspace 0x104
onStart
Set to window composer mode as 1
gralloc usage: 0(OMX) => 0x2900(ACodec)
disconnecting from surface 0x72d0b5f010, reason setNativeWindowSizeFormatAndUsage
connecting to surface 0x72d0b5f010, reason setNativeWindowSizeFormatAndUsage
set up nativeWindow 0x72d0b5f010 for 1920x1080, color 0x30d, rotation 0, usage 0x2900
disableOutlineDraw is true
[OMX.hisi.video.decoder.avc] Allocating 5 buffers from a native window of size 3342336 on output port
showOrHideHighlightView: hasFocus=true; winMode=1; isMrgNull=true
startInputReason = 1
isCasting false because IHwDistributedWindowManager is invalid.
init Rme, version is: v1.0
current pid:27905 AppType:-1
allocate(c2.android.g711.alaw.decoder)
[OMX.hisi.video.decoder.avc] got color aspects (R:1(Full), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M)) err=0(NO_ERROR)
[OMX.hisi.video.decoder.avc] got color aspects (R:1(Full), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M)) err=0(NO_ERROR)
[OMX.hisi.video.decoder.avc] using color aspects (R:1(Full), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M)) and dataspace 0x8c10000
Created component [c2.android.g711.alaw.decoder]
read media type: audio/g711-alaw
extent() != 1 for single value type: algo.buffers.max-count.values
extent() != 1 for single value type: output.subscribed-indices.values
extent() != 1 for single value type: input.buffers.allocator-ids.values
extent() != 1 for single value type: output.buffers.allocator-ids.values
extent() != 1 for single value type: algo.buffers.allocator-ids.values
extent() != 1 for single value type: output.buffers.pool-ids.values
extent() != 1 for single value type: algo.buffers.pool-ids.values
query failed after returning 7 values (BAD_INDEX)
c2 config is Dict {
c2::u32 coded.bitrate.value = 64000
c2::u32 input.buffers.max-size.value = 8192
c2::u32 input.delay.value = 0
string input.media-type.value = "audio/g711-alaw"
string output.media-type.value = "audio/raw"
c2::u32 raw.channel-count.value = 1
c2::u32 raw.sample-rate.value = 8000
}
query -- param skipped: index =<PHONE_NUMBER>.
setup formats input: AMessage(what = 0x00000000) = {
int32_t channel-count = 1
int32_t max-input-size = 8192
string mime = "audio/g711-alaw"
int32_t sample-rate = 8000
} and output: AMessage(what = 0x00000000) = {
int32_t channel-count = 1
string mime = "audio/raw"
int32_t sample-rate = 8000
}
query -- param skipped: index =<PHONE_NUMBER>.
query -- param skipped: index =<PHONE_NUMBER>.
of cleaned connections: 0
create new connection<PHONE_NUMBER>83380
[c2.android.g711.alaw.decoder#594] Created input block pool with allocatorID 16 => poolID 20 - OK (0)
[c2.android.g711.alaw.decoder#594] Created output block pool with allocatorID 16 => poolID 837 - OK
[c2.android.g711.alaw.decoder#594] Configured output block pool ids 837 => OK
RTSP/1.0 200 OK
CSeq: 3
Session:<PHONE_NUMBER>;timeout=60
Transport: RTP/AVP;unicast;client_port=36618-36619;server_port=8314-8315;ssrc=0cb32b85;mode="play"
Date:  Wed, Apr 26 2023 14:04:10 GMT
SETUP rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream/trackID=2 RTSP/1.0
User-Agent: AndroidXMedia3/1.0.1
CSeq: 4
Session:<PHONE_NUMBER>
Authorization: Digest username="admin", realm="IP Camera(C2732)", nonce="9f479ccb412b209b411039131bbfec69", uri="rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream/trackID=2", response="bfa5baea544f77c99fc5906efc47c1bf"
Transport: RTP/AVP;unicast;client_port=60830-60831
win=Window{aad0f59 u0 Splash Screen com.example.exoplayercodelab EXITING} destroySurfaces: appStopped=false win.mWindowRemovalAllowed=true win.mRemoveOnExit=true
RTSP/1.0 200 OK
CSeq: 4
Session:<PHONE_NUMBER>;timeout=60
Transport: RTP/AVP;unicast;client_port=60830-60831;server_port=8316-8317;ssrc=529f1429;mode="play"
Date:  Wed, Apr 26 2023 14:04:10 GMT
PLAY rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream RTSP/1.0
User-Agent: AndroidXMedia3/1.0.1
CSeq: 5
Session:<PHONE_NUMBER>
Authorization: Digest username="admin", realm="IP Camera(C2732)", nonce="9f479ccb412b209b411039131bbfec69", uri="rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream", response="875d5fdb15409bddb81f43d91eabb571"
Range: npt=0.000-
RTSP/1.0 200 OK
CSeq: 5
Session:<PHONE_NUMBER>
RTP-Info: url=rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream/trackID=1;seq=14301;rtptime=1091472994,url=rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream/trackID=2;seq=47273;rtptime=1576408
Date:  Wed, Apr 26 2023 14:04:10 GMT
dispatcher 0x72d08abc00 ~ ignoring unknown event type 0x736f6674
dispatcher 0x733df5c1c0 ~ ignoring unknown event type 0x736f6674
TEARDOWN rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream RTSP/1.0
User-Agent: AndroidXMedia3/1.0.1
CSeq: 6
Session:<PHONE_NUMBER>
Authorization: Digest username="admin", realm="IP Camera(C2732)", nonce="9f479ccb412b209b411039131bbfec69", uri="rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream", response="1eba7a65e77e65a3e929b08e88bba6bc"
SETUP rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream/trackID=1 RTSP/1.0
User-Agent: AndroidXMedia3/1.0.1
CSeq: 7
Transport: RTP/AVP/TCP;unicast;interleaved=0-1
RTSP/1.0 401 Unauthorized
CSeq: 7
WWW-Authenticate: Digest realm="IP Camera(C2732)", nonce="02da6fdeaa3be9ed907dca3a38255b61", stale="FALSE"
WWW-Authenticate: Basic realm="IP Camera(C2732)"
Date:  Wed, Apr 26 2023 14:04:18 GMT
SETUP rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream/trackID=1 RTSP/1.0
User-Agent: AndroidXMedia3/1.0.1
CSeq: 8
Authorization: Digest username="admin", realm="IP Camera(C2732)", nonce="02da6fdeaa3be9ed907dca3a38255b61", uri="rtsp://<IP_ADDRESS>:554/h264/ch1/main/av_stream/trackID=1", response="8d3447c332ce45a454a9c7d0f64e2f4f"
Transport: RTP/AVP/TCP;unicast;interleaved=0-1
RTSP/1.0 401 Unauthorized
CSeq: 8
WWW-Authenticate: Digest realm="IP Camera(C2732)", nonce="a0efb3318a2e828eefcdf6f56d433cd5", stale="FALSE"
WWW-Authenticate: Basic realm="IP Camera(C2732)"
Date:  Wed, Apr 26 2023 14:04:18 GMT
Playback error
androidx.media3.exoplayer.ExoPlaybackException: Source error
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleIoException(ExoPlayerImplInternal.java:652)
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:628)
at android.os.Handler.dispatchMessage(Handler.java:106)
at android.os.Looper.loop(Looper.java:219)
at android.os.HandlerThread.run(HandlerThread.java:67)
Caused by: androidx.media3.exoplayer.rtsp.RtspMediaSource$RtspPlaybackException: SETUP 401
at androidx.media3.exoplayer.rtsp.RtspClient$MessageListener.handleRtspResponse(RtspClient.java:585)
at androidx.media3.exoplayer.rtsp.RtspClient$MessageListener.handleRtspMessage(RtspClient.java:508)
at androidx.media3.exoplayer.rtsp.RtspClient$MessageListener.lambda$onRtspMessageReceived$0$androidx-media3-exoplayer-rtsp-RtspClient$MessageListener(RtspClient.java:501)
at androidx.media3.exoplayer.rtsp.RtspClient$MessageListener$$ExternalSyntheticLambda0.run(Unknown Source:4)
at android.os.Handler.handleCallback(Handler.java:900)
at android.os.Handler.dispatchMessage(Handler.java:103)
at android.os.Looper.loop(Looper.java:219)
at android.os.HandlerThread.run(HandlerThread.java:67)
kWhatFlush processing is complete
[OMX.hisi.video.decoder.avc] ExecutingState flushing now (codec owns 1/5 input, 1/5 output).
Start Processing kWhatFlushCompleted
kWhatFlush processing is complete
[c2.android.g711.alaw.decoder#594] flush
Start Processing kWhatFlushCompleted
query failed after returning 7 values (BAD_INDEX)
query -- param skipped: index =<PHONE_NUMBER>.
query -- param skipped: index =<PHONE_NUMBER>.
[c2.android.g711.alaw.decoder#594] onWorkDone: output format changed to AMessage(what = 0x00000000) = {
int32_t channel-count = 1
string mime = "audio/raw"
int32_t sample-rate = 8000
}
forcing OMX state to Idle when received shutdown in ExecutingState
disconnecting from surface 0x72d0b5f010, reason disconnectFromSurface
buffer invalidation added bp:3 1
buffer invalidation deleted bp:3
bufferpool2 0x72be54fa20 : 0(0 size) total buffers - 0(0 size) used buffers - 0/5 (recycle/alloc) - 0/0 (fetch/transfer)
Destruction - bufferpool2 0x72be54fa20 cached: 0/0M, 0/0% in use; allocs: 5, 0% recycled; transfers: 0, 0% unfetced
In case your question is related to a piece of media:

URI to test content
For protected content:

DRM scheme and license server URL
Authentication HTTP headers

Don't forget to check ExoPlayer's supported formats and devices, if applicable
(https://developer.android.com/guide/topics/media/exoplayer/supported-formats).
If there's something you don't want to post publicly, please submit the issue,
then email the link/bug report to<EMAIL_ADDRESS>using a subject in the
format "Issue #1234", where #1234 is your issue number (we don't reply to
emails).

Hello @BraveHL,
Thank you for reporting your issue. This is the scenario as I see from your logs:

RTSP Setup for UDP stream:
I see the Describe, Setup, Play for successful RTSP stream setup.
Setup request has attribute "Transport: RTP/AVP"
The ExoPlayer RTSP client did not receive any RTP packets so it retries with TCP. When it retries, it removes the authorization info from the UDP session.
I see a new SETUP command with attribute "Transport: RTP/AVP/TCP"
The attempt with TCP calls SETUP without authorization info and rightfully gets HTTP 401 with authentication challenge.
The RTSP client attempts to resend the message with the authentication header info but receives another HTTP 401 error.
ExoPlayer dispatches PlaybackError

What is odd is all the previous RTSP requests with authentication headers during the UDP attempt passed. It only failed when trying to setup with TCP. Does your camera have some odd configuration where your credentials are only viable for UDP streaming and not TCP?

I don't know much about what you said, I don't know much about this piece of video streaming, I use the company's Hikvision camera, everything is almost the default setting, I can't easily modify it, how should I modify the code to solve this problem?

@BraveHL.
I understand. I'm not sure that its an issue of our library but rather an issue of the camera. However, I can still try and help.
We can try simplifying the scenario. Although the camera does not advertise its inability to support streaming over UDP, I can see from the logs that it does not support that. I would suggest that we configure the Player to try and use TCP from the start.
In your setup of RtspMediaSource.Factory, you would want to run the command setForceUseRtpTcp(true).
If you were following our dev page for setup. It would look something like this:
// Create an RTSP media source pointing to an RTSP uri.
MediaSource mediaSource =
    new RtspMediaSource.Factory()
        .setForceUseRtpTcp(true)
        .createMediaSource(MediaItem.fromUri(rtspUri));
// Create a player instance.
ExoPlayer player = new ExoPlayer.Builder(context).build();
// Set the media source to be played.
player.setMediaSource(mediaSource);
// Prepare the player.
player.prepare();

If that works then you can maybe make setForceUseRtpTcp to be conditionally dependent on whether you are streaming from that quirky Hikvision device.
If that doesn't work then please comment back with logs. Thank you!

@BraveHL.
I understand. I'm not sure that its an issue of our library but rather an issue of the camera. However, I can still try and help.
We can try simplifying the scenario. Although the camera does not advertise its inability to support streaming over UDP, I can see from the logs that it does not support that. I would suggest that we configure the Player to try and use TCP from the start.
In your setup of RtspMediaSource.Factory, you would want to run the command setForceUseRtpTcp(true).
If you were following our dev page for setup. It would look something like this:
// Create an RTSP media source pointing to an RTSP uri.
MediaSource mediaSource =
    new RtspMediaSource.Factory()
        .setForceUseRtpTcp(true)
        .createMediaSource(MediaItem.fromUri(rtspUri));
// Create a player instance.
ExoPlayer player = new ExoPlayer.Builder(context).build();
// Set the media source to be played.
player.setMediaSource(mediaSource);
// Prepare the player.
player.prepare();

If that works then you can maybe make setForceUseRtpTcp to be conditionally dependent on whether you are streaming from that quirky Hikvision device.
If that doesn't work then please comment back with logs. Thank you!

Writing like this doesn't work, it seems that nothing happened, here is the log
handleWindowVisibility: no activity for token android.os.BinderProxy@cec9c00
restart watching
dispatcher 0xbc940300 ~ ignoring unknown event type 0x736f6674
callActivityOnCreate
pkgName:com.example.media old windowMode:0 new windoMode:1, isFixedSize:false
add activity client record, r= ActivityRecord{acd5f2d token=android.os.BinderProxy@cec9c00 {com.example.media/com.example.media.MainActivity}} token= android.os.BinderProxy@cec9c00
Init 1d331f3 [AndroidXMedia3/1.0.1] [HWALP, ALP-AL00, HUAWEI, 29]
getStreamVolume streamType: 3 volume: 0
isStreamMute   streamType: 3
getStreamMaxVolume  treamType: 3
Init 14dc129 [AndroidXMedia3/1.0.1] [HWALP, ALP-AL00, HUAWEI, 29]
getStreamVolume streamType: 3 volume: 0
isStreamMute   streamType: 3
getStreamMaxVolume  treamType: 3
mMoveStepInDp: 64, mMoveStepInPixel: 218, mUpTimeDelayed: 100
ViewRootImpl mIsInProductivePCDisplay: false
dispatcher 0xe58acbb0 ~ ignoring unknown event type 0x736f6674
pkgName:com.example.media old windowMode:1 new windoMode:1, isFixedSize:false
dispatchInputInterval 1000000
notifyActivityState pkg:com.example.media/com.example.media.MainActivity state:2 fg:true mUid:10523
disableOutlineDraw is true
onAttached, package=com.example.media, windowType=1, mIsHiTouchRestricted=false
EGLint new_window_surface(egl_winsys_display *, void *, EGLSurface, EGLConfig, egl_winsys_surface **, EGLBoolean) returns 0x3000
removeInvalidNode all the node in jank list is out of time
showOrHideHighlightView: hasFocus=true; winMode=1; isMrgNull=true
startInputReason = 1
isCasting false because IHwDistributedWindowManager is invalid.
init Rme, version is: v1.0
current pid:5829 AppType:-1
win=Window{a9bd759 u0 Splash Screen com.example.media EXITING} destroySurfaces: appStopped=false win.mWindowRemovalAllowed=true win.mRemoveOnExit=true
dispatcher 0xe58acbb0 ~ ignoring unknown event type 0x736f6674
dispatcher 0xbc940300 ~ ignoring unknown event type 0x736f6674
stop UiProbe
not watching, wait.
APS: EventAnalyzed: initAPS: version is <IP_ADDRESS>
Fpsrequest create,type:EXACTLY_IDENTIFY
Fpsrequest create,type:EXACTLY_IDENTIFY
Fpsrequest create,type:OPENGL_SETTING
FpsController create
APS: EventAnalyzed: reInitFpsPara :mBaseFps = 60; mMaxFps = 60
Setting device_provisioned has moved from android.provider.Settings.Secure to android.provider.Settings.Global.
User setup is finished.
HwApsManagerService, registerCallback, start !
APS: EventAnalyzed: registerCallbackInApsManagerService, mPkgName:com.example.media; result = true
removeInvalidNode all the node in jank list is out of time

setForceUseRtpTcp

When changed to setForceUseRtpTcp's writing, I wrote it incorrectly, and after correcting it it loads normally, thanks


[MODCON-8] - Cover Consortia Tenant CRUD endpoints with karate tests

Purpose
Describe the purpose of the pull request.
https://issues.folio.org/browse/MODCON-8
Approach
How does this change fulfill the purpose?
TODOS and Open Questions

[ ] Use GitHub checklists. When solved, check the box and explain the answer.

Learning
Describe the research stage. Add links to blog posts, patterns, libraries or addons used to solve this problem.

It will merge after small improvement pr for mod-consortia

added delete tenant test and check length tenant name


Change port from uint16_t to uint32_t, to support VSOCK

Issue: https://github.com/awslabs/aws-c-io/issues/576
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report
Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (b7e04ca) 83.31% compared to head (44d4914) 83.35%.

Files
Patch %
Lines

source/uri.c
92.30%
1 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1079      +/-   ##
==========================================
+ Coverage   83.31%   83.35%   +0.03%
==========================================
  Files          56       56
  Lines        5808     5803       -5
==========================================
- Hits         4839     4837       -2
+ Misses        969      966       -3

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Reduce download speed at last

hi
thanks for your useful module
i use this lib to download files with multi threading ( 5 thread)
but problem is :
download start with fast speed (10 ~ 20 MB/sec)
but at last (after 80%) download speed reduce under 1 MB/sec
any way to fix this ?

Looks like same case here also!


Revert "Revert "Update DSP Operator labels""

This reverts commit 90524c44bd510e2e58c8b73689bdd1eeea76f335.

Related task: https://github.com/opendatahub-io/data-science-pipelines-operator/issues/140
Description

How Has This Been Tested?

Merge criteria:

[ ] The commits are squashed in a cohesive manner and have meaningful messages.
[ ] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[ ] The developer has manually tested the changes and verified that the changes work

/retest


Bug Report: [RESET SAVE DATA not doing anything]

Which target are you using?
[x] Windows
[ ] Linux
[ ] Mac
Explain your issue
In the v.s Gorefield V2 mod I tried the option RESET SAVE DATA, but when I checked Freeplay, all my highscores remained intact. I don't know if I have to do something with the cache, or my laptop is preventing the mod from deleting its own data, or if it's the engine itself. I tried the option several times, and not even the changes I did on Chart Editor were reset.

the reset save data function wasnt implemented actually, only the option itself was intact, do you really have to fall for that?

It's nothing urgent; I just thought it would be easier to figure out my custom code if I beat the full mod in the same day. Thank you.


API description for ConsensusCreateTopic has memo field listed twice

Description
If you look at page ConsensusCreateTopic in the section ConsensusCreateTopicTransactionBody you'll see that the memo field occurs twice.
Steps to reproduce
Visual inspection of https://docs.hedera.com/hedera/sdks-and-apis/hedera-api/consensus/consensuscreatetopic will suffice.
Additional context
No response
Hedera network
other
Version
N/A
Operating system
Windows

Resolved in 04be0a6


Major refactor.  Should I hard-fork?

I am doing a major refactor of this code which includes/will include

Bringing the code to modern python and pyvisa, starting with #2 pull request from @battlecoder
Reformat with Black to PEP8 (including changing from tab indent to spaces)
-- I am protecting the table-like parts of the code so they don't rewrap.
Packaging it to be pip-installable as an importable module
Making the running commentary optional.  (Probably just a verbose flag, as opposed to doing it right and using the logging system)
Binary transfer instead of ASCII, to support long readouts and selective readouts.

You can see the changes in my packaging branch (work in progress).
At some point, even if most of the lines of code are exactly the same, the change in structure makes it a different package, or at least hard to merge without conflict.  Then the choices are:

Pull-request the changes back to the main @charkster repo and call it version 2
Hard fork it with a new name and copious crediting in the README

What is your preference?

Hi @battlecoder I would prefer 2, and you are free to take as many parts of the code as you would like. If you put a link to my repo on your new README, you don't need to put any other references in your code. Thanks for taking an interest in this repo.

@charkster , It's @dmopalmer who is forking your code, not me.
Although I'm happy that my PR is going to be useful to someone else to update and extend what you have generously shared with us here.

@dmopalmer, sorry I sent your response to @battlecoder by accident. Use/change any parts you want, just make it better. Thanks.


Pin llvm-openmp to conda-forge pin (v15)

Not clear why builds are requiring v16 but this doesn't work with packages like Mantid that follow the conda-forge pinned version (15)

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

@conda-forge-admin, please rerender


OTA update does not work properly

Hi jkandasa!
We spent a lot of time with tekka, to sort out this problem.
The problem is that data is continuously lost packets.
If a sketch no larger than 32 kilobyte, all goes well.
If the size is more than that, then the result is always the firmware OTA:! FWP: CRC FAIL
In confirmation of that, I can send a 2 log file.

MYSController.log (2 firmware downloaded here 53 and 74 kilobytes)
MYCONTROLLER. log (there is only 53 kilobytes, and not just luck!)

Send mail on some logs?

I forgot to add.
This problem remains the same if you use Ethernet GW

and yet, the load time is increased by half.

@devicter can you send log files and sketch to my email address? jkandasa at gmail dot com
Also send your atmega chip details

@devicter I tested with year old OTA bootloader and works as expected. I have changed radio pin and channel number to default one and disabled LED indication.
I will try with recent OTA bootloader soon and update you the status.

I'm sorry, I'm using the bootloader from Moteino 16 MHz.
Because the processing speed twice as high, it seems COM port driver does not have time to process the data.
You can use this downloader to take from here.
https://github.com/devicter/bootloaders

@devicter I do not have DualOptiboot setup ;)

Very sorry, but I say again, it works fine with MYSController https://www.mysensors.org/controller/myscontroller
But it does not work for you.
Although this problem can be solved very simply.

@devicter ok, let me have a look on code level. calculating CRC.
does it working at least one sketch or nothing?

If a sketch no larger than 32 kilobyte, all goes well.
If the size is more than that, then the result is always the firmware OTA:! FWP: CRC FAIL

I see this from your comments, How do you calculate sketch size?

@devicter I have added minor fix, kindly check with snapshot version.

I tried again on the new version. :-(
unfortunately the same result.
Log I sent in the mail.

yet there was an error on the page

@devicter clear your browser cache to resolve the above issue ^^

did not help!

@devicter try this http://forum.mycontroller.org/topic/47/gui-caching-issue-problem

I know how to update the cache ...
-(

@devicter I need to the following information to narrow this issue further.
All data from node (add print statement on your MySensors library at line#171)

_nodeFirmwareConfig.blocks
crc
_nodeFirmwareConfig.crc
firmware hex file, that you used to test.

Kindly do it for two firmware hex files and update result for both. Thank you!

this line should look like this?
OTA_DEBUG(PSTR("OTA:CRC:B=%04X,C=%04X,F=%04X\n"),_nodeFirmwareConfig.blocks,crc,_nodeFirmwareConfig.crc);

@devicter yes, this looks good. Also, I have added fix for GUI issue you have mentioned on node edit page. download latest snapshot version and check does this error still exist. Thanks!

screenshots
files I sent an e-mail

@devicter I did some changes on CRC calculation. You have to upload your firmware once again to MyController server to calculate CRC.
Kindly run with latest snapshot version and report with OTA_DEBUG(PSTR("OTA:CRC:B=%04X,C=%04X,F=%04X\n"),_nodeFirmwareConfig.blocks,crc,_nodeFirmwareConfig.crc);
Thanks!

Again, that is not so

for information on the controller at the Tekka obtained here such figures

@devicter Could you please do a favor for me. Run OTA update as follows,
I need debug log from all nodes and MYSController to go further.

Take a hex file for all tests
Send it from MYScontroller, need MYSController log as well as node log
Send it from MyController, need node log

NOTE: For each run I see different CRC on node, does it changing every time?
I am really sorry to trouble you again and again. As I do not have this kind of setup with me, I fully depend on you to fix this issue.

Ok, I sent back to the email

@devicter Thank you so much for your valuable time!!

For me, too important a positive result.

@devicter looks like there is no issue with CRC calculation and in MyController. As per my understanding, problem with MySensors code. When we use SerialGateway it is super fast and completes the request within 500ms. When we go with other types of gateway, problem starts, because other than serial gateway works with polling mechanism. In OTA code we have retry mechanism,  MY_OTA_RETRY_DELAY(default 500ms)
OTA:FWP:RECV B=0173
OTA:FRQ:FW REQ,T=0003,V=0013,B=0171
TSF:MSG:SEND,1-1-0-0,s=255,c=4,t=2,pt=6,l=6,sg=0,ft=0,st=OK:030013007101
OTA:FRQ:FW REQ,T=0003,V=0013,B=0171
TSF:MSG:SEND,1-1-0-0,s=255,c=4,t=2,pt=6,l=6,sg=0,ft=0,st=OK:030013007101
TSF:MSG:READ,0-0-1,s=255,c=4,t=3,pt=6,l=22,sg=0:03001300710111924A95E9F746E449834CE44A8348E5
OTA:FWP:RECV B=0172
TSF:MSG:READ,0-0-1,s=255,c=4,t=3,pt=6,l=22,sg=0:03001300710111924A95E9F746E449834CE44A8348E5
OTA:FWP:RECV B=0171
OTA:FRQ:FW REQ,T=0003,V=0013,B=016F
TSF:MSG:SEND,1-1-0-0,s=255,c=4,t=2,pt=6,l=6,sg=0,ft=0,st=OK:030013006F01
TSF:MSG:READ,0-0-1,s=255,c=4,t=3,pt=6,l=22,sg=0:030013006F010E945421809198029091990264E0880F

From the above debug log, I see that block 0x172 data written for block 0x171 as well and 0x171 never used. This leads CRC error. I hope with MyController you didn't use Serial gateway and with MYS controller you have used serial gateway.
I have two ideas to fix this issue (I never tested this, I do not have hardware)
Options 1 (This is not a perfect fix):
Increase MY_OTA_RETRY_DELAY to 2s or above. (based on your intranet/internet speed)
Options 2 (this is may be a perfect fix):
Change the following line as mentioned, https://github.com/mysensors/MySensors/blob/development/core/MyOTAFirmwareUpdate.cpp#L102
_flash.writeBytes( (firmwareResponse->block * FIRMWARE_BLOCK_SIZE) + FIRMWARE_START_OFFSET, firmwareResponse->data, FIRMWARE_BLOCK_SIZE);

Test with latest MyController server. and do change on MySensors code as mentioned above.
Thank you!

I did everything you asked. But, unfortunately, the result is negative. ;-(

Oh, I do not quite understand, I still had to install an updated version MyController?

@devicter Yes, you have to use most recent SNAPSHOT version, I have revert back CRC changes on recent build. Which is must. Kindly try once again with most recent SNAPSHOT build. Thank you!

testing stages

I put the latest version MyController server
flashed your gateway with amendments #L102
flashed your microcontroller with amendments #L102

`

356613 OTA:FWP:FW END
357874 OTA:CRC:B=04A8,C=1E21,F=DEA5
357877 !OTA:FWP:CRC FAIL`
Result old.

@devicter I could not reproduce this issue locally. We have some changes on SNAPSHOT version regards OTA. Can you try it now?

looks like reporter not interested on this issue.


ContainerEnvironmentVariable with format like Group:Item throws error

I have an appsettings.json file with this value in it:
{
  "IdentitySettings": {
    "Url":"https://myidentityhost.com",
    "ClientId":"my-client",
}
I would like to define environment variables in my project file to override the above for production for example.
I created these:
	<ItemGroup>
		<ContainerEnvironmentVariable Include="IdentitySettings:Url" Value="myidentityhost.com"/>
		<ContainerEnvironmentVariable Include="IdentitySettings:ClientId" Value="my-client"/>
	</ItemGroup>
I get the following error when I do dotnet publish:
warning : ContainerEnvironmentVariables: 'IdentitySettings:Url' was not a valid Environment Variable. Ignoring.
Is there a way to make environment variables like that?
If not how can I override those values in the appsettings.json file using this new container build system?

Environment variables in this tool need to be something that a shell on the host system would accept, and I don't think the format you've shown is valid for that.
Luckily, the Asp.Net configuration system has a pattern for this already - if you replace the : with __ then I think your change should just work. Would you try that and see?

That was an AMAZINGLY fast response and it worked like a charm!
Thanks.

Glad to hear it, and thanks for kicking the tires a bit :)

I just noticed something in the solution window of Visual Studio. Those environment variables show up as items (because they are in an item group). Is there a proper way to hide them?


Rename the 'eDiary' page with 'Tasks Calendar' and redesign the page as in the attached mockups

already fixed


Trees not handled by TreeBuilder

Looked into the remaining trees, and excluding Brian's Report trees PR, these are still not using TreeBuilder:
ops_controller/settings/common	build_smartproxy_affinity_tree (#4730)
ops_controller/settings/cap_and_u	build_cl_hosts_tree
ops_controller/settings/cap_and_u	build_ds_tree
ops_controller/diagnostics	build_server_tree
application_controller/miq_request_methods	build_ous_tree
application_controller/miq_request_methods	build_tags_tree
application_controller/tree_support	build_belongsto_tree
application_controller/compare	build_sections_tree
application_controller/timelines	build_timeline_tree
host_controller	build_network_tree
    ./app/views/host/_network_tree.html.haml
host_controller	build_sa_tree
ems_common	build_dc_tree
resource_pool_controller	build_dc_tree
ems_cluster_controller	build_dc_tree
    ./app/views/layouts/_dc_tree.html.haml
vm_common	build_snapshot_tree
    ./app/views/vm_common/_snapshots_tree.html.haml
vm_common	build_disks_tree
vm_common	build_policy_tree
configuration_controller	build_default_filters_tree
miq_ae_customization_controller/dialogs	dialog_edit_build_tree
    ./app/views/miq_ae_customization/_dialog_edit_tree.html.haml
application_controller/tree_support	protect_build_tree
    ./app/views/layouts/_protect_tree.html.haml
miq_policy_controller/rsop	rsop_build_tree

Some of them are just passed to the dynatree layout and don't use any of the deprecated tree methods, but some do. Some create nodes by hand, some use TreeNodeBuilder. Will categorize further...
Also, these *_tree views don't seem to match any of the above, so there may be a few more... (or some TreeBuilder is not using the shared view)
./app/views/provider_foreman/_foreman_providers_tree.html.haml
./app/views/provider_foreman/_foreman_configured_systems_tree.html.haml
./app/views/provider_foreman/_cs_filter_tree.html.haml
./app/views/ops/_zone_tree.html.haml
./app/views/miq_capacity/_utilization_tree.html.haml
./app/views/miq_capacity/_bottlenecks_tree.html.haml
./app/views/service/_svcs_tree.html.haml
./app/views/miq_ae_tools/_results_tree.html.haml
./app/views/container/_containers_tree.html.haml

add
app/controllers/host_controller.rb:68 compliance_history_tree

Closing - a current list of TreeBuilders to be done can be found in https://www.pivotaltracker.com/n/projects/1613907/epics/2644711/expand


Added "defaultMemoryMbytes" to actor.json

see https://apify.slack.com/archives/C010Q0FBYG3/p1733154485910689

I like this. ~This would probably remove the need to have charge the actor-start per GB or maybe charge it at all.~ No, this is just a default, they could override it so we would still have to charge per GB for the wrong configurers

In theory in the future we might disable ability to select custom memory, or per developer setting. This opens the path.

BTW perhaps this would be cleaner:
memory: {
  minMbytes: 128,
  maxMbytes: 4096,
  defaultMbytes: "${maxCrawlPages} * 256 + 128"
  // default: 1024 - this also works
}

What do you think guys?

How about this?

move mbytes suffix one higher

it's cleaner, but if there would be some other memory config which isn't directly related to the mbytes of memory available, it can't go into the object

prefix the variables with input. to make it clear that they're referring to input values, not other values from actor.json

memoryMbytes: {
    min: 128,
    default: "${input.maxCrawlPages} * 256 + 128"
    max: 4096,
}

IMO it's better to keep just memory, to have flexibility to add more stuff in the future. Also the convention is that variable names must end with the unit, not parent objects. So:
memory: {
    minMbytes: 128,
    defaultMbytes: "${input.maxCrawlPages} * 256 + 128"
    maxMbytes: 4096,
}

Or maybe we could make it even more generic and future proof:
runtime: {
    memoryMinMbytes: 128,
    memoryDefaultMbytes: "${input.maxCrawlPages} * 256 + 128"
    memoryMaxMbytes: 4096,
}


Munkiinfo module take 2

This is the followup PR to #390
@erikng Is on break this week and I know everyone is pretty interested in getting a new release up.
The biggest changes to this PR is the modification of the database scheme to store the key/value pairs in two columns. This means additional keys can be added in the future just by modifying the munkiinfo.py script.

Although this could use some cleaning up (lose the colons in the headings) and the placement of the popup could be a bit better (it drops from my Macbook screen), I'm going to merge this to get more eyes on it. Thanks @erikng and @clburlison


Can tagui detect the changes in a web application?

Hi,
Can tagui detect the changes in a web application when the version changes happens or a new feature has added? For example, In a recording, I selected an option from a drop-down field and the recording has played back successfully. After that a new feature(for ex, new drop-down option)  has added in the web application. As tagui detecting the selected element in the number order, can it detect the previously selected option successfully? Can tagui play recording successfully? Please explain.
Thanks & Regards,
Sahithi.

Hi @nsahithi when developing a script you can record or specify the element identifier. If webpage changes and the old identifier is still valid, then the script will still work. However, that identifier may be pointing to the wrong element because it may be some other web element.
If the website changes and the identifier is now invalid, then TagUI will throw error message so that users can check if something has been broken due to website changes. My recommendation is use very strict element identifier (for example accurate XPath), so that breaks are flagged out immediately for investigation. It is better to have false-positive versus having the wrong impression that the automation is still working but it has already been broken.
There is a related option called baseline. If you run an automation script with the baseline option, TagUI will create a folder called baseline to store the execution results. This can be helpful to let you compare a run with a run in the past to check for any difference in execution.
eg below runs the yahoo sample and creates a baseline folder to store a copy of the script and the results + log of the automation -
tagui 1_yahoo baseline

fyi @Aussiroth and @lohvht