constructing validation test for DC2 dN/dmag

Question for @janewman-pitt-edu @damonge and anybody else who is interested:
There is an interest in making sure that DC2 has a galaxy population that is complete beyond the flux limit of LSST, to ensure that realistic levels of blending are included.  We would like to do this validation test on proto-DC2, comparing dN/dmag against some realistic reference using DESCQA.  The question is what is the reference sample we should use?
Currently if we were to compare the DC2 population with, e.g., the COSMOS catalog, we can only test to i~25.2.  The purpose of this issue is to solicit suggestions for catalogs we could use to make a deeper validation test.  I could imagine the extreme case would be Hubble UDF, but that is so small that it should have significant cosmic variance.  There are some intermediate-depth and area HST surveys (deeper than COSMOS but smaller area, while shallower and larger area than UDF).  We could explore those?
The CatSim page has a comparison against data going to 27th magnitude: https://www.lsst.org/scientists/simulations/catsim but the plot caption does not say where the validation data comes from.

CANDELS offers public catalogs with significantly increased area vs. HUDF.  I haven't plotted HUDF number counts, but since we're effectively integrating along the entire line-of-sight and (I think) just looking for a bias in DC2 catalog number counts versus data in broad bins of object brightness, we should be relatively robust to cosmic variance.

CANDELS volume is still tiny, and sample/cosmic variance is substantial.  It's enough to affect number counts (and even more so redshift distributions) probably at the ~10% level.  Maybe we're fine with that for this test.  How deep does HSC go?

Actually, I think you're probably better off validating against the existing much larger-area but a bit shallower surveys (e.g. HSC + DES deep fields) rather than the tiny area HST data.  You'd just truncate the comparison where the existing data is fully complete (e.g. around the 10 sigma limit of the photometry).  It's probably OK to assume that the number counts in actuality continue to rise geometrically a mag or so fainter than they are observed to do so in those datasets.

We should do whatever is easiest that offers the needed precision.  I will guess however that if we were to require our simulated number counts beyond LSST depth (e.g., r~28) to match reality at anything close to the 10% level, it might take a decade for the DC2 catalogs to pass QA.  The goal is "realistic levels of blending", so I would interpret that as wanting to be within a factor of 2 at such number counts and hoping to be off by only 50% (in either direction).  Can anyone quantify the requirement more precisely?

Well, for starters, the total blend rate should scale like the square of the overall number density, shouldn't it?  (at least in the limit that clustering is not a big contributor to the blend rate, which is true for LSST according to @dkirkby ) So if the overall number density is wrong by say 10%, the blend rate would be wrong by 20%.
But that's not precisely what we care about, in the sense that we care about the rate of blends as a function of flux ratio, so you could imagine thinking of the blend rate in a 2d plane defined by brighter vs. fainter galaxy mag.  In a given bin in that plane, the blend rate depends linearly on the number densities on each axis.  It seems like this becomes a requirement on the number density as a function of mag, which was why I was proposing looking at dN/dmag in the first place.   But it's complicated because for example we might care more about similar flux ratio blends within the limit of the gold sample, and so getting dN/dmag right for something 1.5 mag fainter than that is less important (that becomes noise on the outskirts of the gold sample galaxies, which we do want to include in the sims, but it probably has a smaller impact than blends with something at the same mag).  I would imagine we want dN/dmag to ~10% down to r~26, but beyond that our requirements are quite a bit less stringent.  30%?  I'm guessing at this point.
I believe it may have been one of @mdschneider @joezuntz @rmjarvis @esheldon who articulated the need to have a realistic blending rate in the DC2 sims for WL, so perhaps one of them can weigh in either on how realistic is good enough, or on a good validation dataset.  (Michael, Joe, Mike, Erin - please read the first post in this thread for an explanation of what this is about)

I don't know that we need to match it perfectly.  The goal of DC2 is to test the ability of our code to handle the kinds of blends that are in real data.  Not to precisely match the level of blending that we will see in some year of LSST data.  In other words, it is a validation sim, not a calibration sim.
So I'd say a factor of two or better is good enough for our purposes.  If it were really 10% squared = 20%, I'd be ecstatic.

We showed in Dawson et al. (2016) that the number density of ambiguously blended galaxies grow proportional the number density of galaxies to the ~3rd power (not squared; see figure 1). In section 5.1 we note why this is greater than to the power of 2:

"Thus, when n is far from the confusion limit, the index α from Equation (19) will tend to be  >2. It can be greater than 2 in part due to clustering, but to a larger degree it is because dP is a detect  blend function of the deblending threshold that is usually defined with respect to the pixel level background. The number of low S/N galaxies increases as the limiting magnitude increases because the number density is observed to scale as n ~ 100.31(m-20) (Tyson 1988; Beckwith et al. 2006), where m is the limiting apparent z850 magnitude of the survey. Because it is more difficult to detect blends of low S/N objects, we should expect larger blend fractions with increasing survey limiting magnitude. Therefore α should be an increasing function of survey limiting magnitude until it levels off near the confusion limit."

Historically, in earlier versions of the paper and presentations to DESC we noted the impact of ambiguous blending on LSS number counts as an important systematic.

@wadawson - yep, I was using an approximation that may not be correct for LSST, probably for multiple reasons.

Historically, in earlier versions of the paper and presentations to DESC we noted the impact of
ambiguous blending on LSS number counts as an important systematic.

Absolutely - that isn't really a matter of question here; I think that message has been pretty well internalized by the group.  The only issue is how accurately do we need to represent the blend rate for DC2 given what the goal of DC2 is (which is not the same thing as asking how well we want to be able to represent it e.g. for dedicated blending studies).

I think people are overcomplicating things here... in all the datasets I've seen, number counts rise geometrically with only a very very slow change in power law slope as a function of magnitude until incompleteness sets in (see, e.g. Fig. 15 of https://academic.oup.com/mnras/article/423/3/1992/2459951/Galaxy-properties-from-the-ultraviolet-to-the-far ).  This is true to at least i~29 in HST ACS data.  This means that one can analytically extend the magnitude distribution from a shallower (but wider, so low sample/cosmic variance) survey by a magnitude or so with much less than 10% uncertainty (I would suggest measuring the logarithmic derivative of dN/dm and its second derivative as well to see if the latter is significant).

@janewman-pitt-edu - I agree with you about analytically extending something from a shallower survey, but I think you may have misunderstood the 10% (or whatever %) discussion.  That wasn't a discussion of the error we'd allow on the validation data, which I agree we can easily know well.  It was a discussion of the error we'd allow in DC2, i.e., if the number counts and therefore blend rates don't quite match those expected in reality, how far off are we OK with them being.

Ah, thanks for clearing that up!  (Oh, and I definitely agree with @dkirkby that the majority of blends at LSST depths are from very different redshifts, though I'm not certain what the fraction is exactly).

Update on this issue.  As part of our SRD work, I used the publicly-released HSC deep survey layer to measure the dN/dmag (including absolute normalization and slope) in i-band.  I also did a power-law fit to dlogN/dmag and the cumulative fits as a function of limiting mag, so we can extrapolate that, given that these power laws are generally pretty accurate across a wide range of magnitudes as Jeff said.
What I got from HSC Deep was a best-fitting number density (in units of arcmin^-2) of
N(<ilim) = 43 * 10^(0.36 (ilim - 25))
where ilim is the i-band magnitude limit of the sample.  So, to be clear, these are cumulative number densities, not differential.  If you want differential counts, it's just a multiplicative factor to convert.
Just to throw out numbers, it would be nice to agree with this at the 20% level at ilim=25.3 and 40% at ilim=27.
Would this work for a validation test?

@rmandelb I think that's great validation test that we can do. We can keep track of the status of these tests in #30

OK, so in the interest of not having too many open issues that duplicate each other, do you want to close this issue (now that we've decided what the test should be) and, in #30, link to the final comment with the power-law fit on this page?

@rmandelb sure. I don't have the permission to close issues in this repo though.

I would actually leave it open. The #30 is the epic about validation,
mostly introduced so that Anders can track it. I think having the issue
21 as linked to the overall validation issue is fine.
On 11/1/17 9:11 PM, Rachel Mandelbaum wrote:

OK, so in the interest of not having too many open issues that
duplicate each other, do you want to close this issue (now that we've
decided what the test should be) and, in #30
https://github.com/LSSTDESC/DC2_Repo/issues/30, link to the final
comment with the power-law fit on this page?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/LSSTDESC/DC2_Repo/issues/21#issuecomment-341297365,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMQ9jKRHxOz2phzbD72TaoAoEMCQjNsOks5sySTFgaJpZM4QFUg_.

Yes, it would make a good validation test. It is on my list to implement
it. Thanks for doing the fits.
On Wed, 1 Nov 2017, Rachel Mandelbaum wrote:

Date: Wed, 1 Nov 2017 18:31:11
From: Rachel Mandelbaum<EMAIL_ADDRESS>Reply-To: LSSTDESC/DC2_Repo
<reply+0076fc4d7ff441653d9fc18b36afb56509405512976d9bc692cf0000000116121b3
<EMAIL_ADDRESS>To: LSSTDESC/DC2_Repo<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Subject: Re: [LSSTDESC/DC2_Repo] constructing validation test for DC2 dN/dmag
(#21)
Update on this issue. As part of our SRD work, I used the publicly-released HSC deep survey layer to measure the dN/dmag (including absolute
normalization and slope) in i-band. I also did a power-law fit to dlogN/dmag and the cumulative fits as a function of limiting mag, so we can
extrapolate that, given that these power laws are generally pretty accurate across a wide range of magnitudes as Jeff said.
What I got from HSC Deep was a best-fitting number density (in units of arcmin^-2) of
N(<ilim) = 43 * 10^(0.36 (ilim - 25))
where ilim is the i-band magnitude limit of the sample. So, to be clear, these are cumulative number densities, not differential. If you want
differential counts, it's just a multiplicative factor to convert.
Just to throw out numbers, it would be nice to agree with this at the 20% level at ilim=25.3 and 40% at ilim=27.
Would this work for a validation test?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.[AHb8TeYApJgPksw-Qh0XO-kXBHLuzOK2ks5syP8_gaJpZM4QFUg_.gif]

@rmandelb would you mind the sharing the code you used to calculate N(<i_lim) with the HSC data or point me to a useable catalog where I can calculate myself?

The code I used to calculate it is in another DESC repo, so I'm happy to share, but here's a question: do you want the actual N(<i_lim)?  Or a power-law fit to it that describes the data well?  I would think the latter might be better because then you don't have to worry about what is the histogram binning, etc.?
For reference, the fit is N(<i_lim) = 42.9 [10^(0.359 (i_lim-25))] per arcmin^2, shown as the black dashed curve on this plot (compared with real data in the blue solid curve):

The deficit at bright magnitudes is because of some selection criteria that downselect bright objects (i~22 is very bright for HSC) and at faint magnitudes, due to incompleteness.  Obviously, beyond i~25.5 there is some major extrapolation happening.

What I would like is some self-contained code that does the following:
1.) downloads necessary HSC data
2.) calculate cumulative number density of galaxies in a set of bands
3.) interpolates the above calculation to get the cumulative number density on regularly sampled intervals
4.) fits a power law over some range
5.) defines a upper mag limit where the power law fit is used to extrapolate
5.) prints out a simple lookup ascii table with mag and N(<mag) for a set of bands that I can use for the test.
In principle, the above is something that someone can provide me, but I would feel better if I could follow it through the steps, make sure it is reproducible, and use as similar as possible code to create more lookup tables from different surveys.

OK, didn't realize you wanted to download the data.  Just a few comments:

Your item (1) is going to depend a lot on the survey (what cuts/flags to use, best photometry to use, etc.)
In practice, I recommend going from the differential rather than cumulative counts.  The reason for this is that at the bright end, you will often miss some objects due to cuts on e.g. interpolated pixels or CCD edge cuts or whatever (big bright objects cover more pixels), and if you use the differential counts then you can pick the mag range that is less affected.  Then you can fit a power law, and then convert the power-law to cumulative counts.  Do you see what I mean?

I'm going to separately e-mail you an SQL script for downloading the HSC data, and a python script that does the process I described above for HSC Deep i-band data.  The latter is already in the Requirements repo which I believe you don't have access to.  I'm happy for comments on any of it, or for the code to be put into this repo as needed.

My first pass implementation idea is to simple use the cumulative number density at a single magnitude limit as the 'test' and not the full apparent magnitude function in bins.  I think this gets at the basic quantity of interest.  it is a simple extension to test for a few different magnitude limits if the shape of the app_mag_func is important.

@duncandc  How about using the fit for a start? Ideally, there would be papers which do the fitting and to which you could refer, but I understand that these may not exist yet for LSST-like depths. But initially, if we just want to get an idea of how the catalogs look, using Rachel's fit would be quickest. It is not obvious to me that the WGs will need fancier cuts or options, so it might be a good idea to get feedback from them before putting a lot of work into a more general structure.

That's the path I am taking--The test itself is exceedingly simple.  I am also interested in the data.

We definitely care about getting the shape of the number counts roughly right.
I just sent you some stuff by e-mail so you can download the data with the same SQL query and process it with my script (or modify the script as needed).  I should emphasize that I only did the i-band cumulative counts, so if we want to test >1 band, we need to at least rerun the script for r-band (which I can do myself if you want, because I did download the r-band magnitudes) and possibly others (which would involve re-downloading the data with more information).

@duncandc are you thinking to generate the validation data on the fly every time when the test is run? Need to consider computational resources.

I am not considering that.  I am putting on the creating validation data hat.

I agree we want to get the shape of the number counts right.  However, we also want to be sure to do that test only where counts are highly complete (i.e. at least 0.5-1 mag brighter than where counts turnover, and probably avoiding the bright end to minimize impact from stars).
Best,
Jeff

@duncandc @janewman-pitt-edu  @rmandelb @yymao Duncan has worked on a test and example plots can be seen  here There are some normalization issues, but even so, we can now start to think about specifying validation criteria, such as defining a validation region to be used, and the required level of agreement with the data.

@evevkovacs @duncandc - thanks for sharing these plots.  Apologies if I am missing where to find this information, but what is each curve?  (i.e., which is validation dataset vs. the simulation we are validating, and why does one plot have 3 curves but the other plots have 2 curves?)
Is there anything I can help with to try to track down normalization issues?  For example, Duncan, if this comes from your own recalculation based on downloading HSC data, do you want to compare with the normalizations that I have calculated to see if we agree about the data?
Just to throw out an idea for validation criteria: I think we'd like to sample a few i-band magnitude values within the range where HSC is clearly complete.  Let's say i=24, 24.5, 25.  We'd like N(<i) in the data and simulations to agree to within ~20% at those values of i-band magnitude.  It would also be nice to take the power-law extrapolation of the HSC density for 22<i<25, and require N(<i=27) in the sims and extrapolated power-law from data to agree to within ~40%.

@rmandelb the one with 2 curves + data points is the summary plots (i.e., overplotting all catalogs), and other plots are just for individual catalogs (i.e., no overplotting other catalogs) --- the contents are the same though.
Maybe you can help reviewing this notebook @duncandc put together for getting the HSC data? You can also find the corresponding PR at https://github.com/LSSTDESC/descqa/pull/46

As for the normalization, I will check it against @rmandelb 's previous calculation.
Is there a better way to estimate "error bars" on N(<m) than back-of-the-envelope estimates?  Asking for O(10%) agreement seems arbitrary to me.

I'm happy to review the notebook and PR.  Some cosmetic updates that would be helpful for people who look at this test:

the plot should say which is validation data and which is the sim
the summary plot should say which sim is which
should have an axis label for the horizontal axis that says what band is being shown
it would be helpful to display number densities in number per square arcmin, rather than number per square degree (I know it's a minor point, but many observers have a rough rule of thumb for expected density per square arcmin in imaging data, so looking at the plot in those units without having to divide by 3600 makes it easier to do sanity checks).
having the vertical axis explicitly say this is a number density rather than a raw number -- e.g., by noting the units -- would be helpful.

I will try to review the notebook and PR in the next day or so.

@duncandc - thanks for checking against my previous calculation, that will be helpful.

Is there a better way to estimate "error bars" on N(<m) than back-of-the-envelope estimates?

The statistical uncertainty on the N(<m) due to number counts alone is very small given the area of the HSC survey and the sims, while systematic uncertainty due to e.g. whether we've defined the samples in totally consistent ways in data and sims is larger and hard to quantify, but roughly speaking can give order 10% uncertainties (this is pretty much an empirical statement based on comparing sims and data many times and seeing how big changes can arise due to that kind of thing).  Maybe somebody else will have a better idea than I do...

@duncandc @rmandelb @yymao So to summarize a first pass attempt at validation criteria, we have:

Add a 10% systematic uncertainty to the HSC data points. (I would guess these are correlated, but we can ignore this for a first pass)

Check that the catalog data agrees within 20% ( so that's 20% + 10% added in quadrature) in the validation region 24 < i < 25

Check that the catalog data agrees within 40% (ie 40% + 10% in quadrature) with the extrapolation to the power law fit to the HSC density in the validation region i = 27

and 3) are normalization checks. Initially just check if the catalog falls in the required range; we can come up with something fancier if need be.

Plot enhancements in addition to those mentioned by Rachel are:

Show the HSC power-law fit on the plot (and add to legend)
Show the validation regions as shaded vertical bands. Actually, I think we opted to shade the non-validation region for DESCQA-v1, but we'll have to see how it looks.

Does anyone have any other suggestions or improvements?

@yymao @duncandc - I wanted to check something: is it OK if I wait to review the PR until after the changes I already suggested get implemented?  I would prefer to just review it once, if that works for you.  Ping me once the changes are made and it's ready for review?

@rmandelb, sure thing.  I will ping you when I have made those changes.

@rmandelb yes --- and JFYI, we (the DESCQA team) encourage early PR so people can see the progress. Thus, it is totally OK to defer review under the PR submitter says it's ready for review!  In the meantime, leave comments and suggestions in the PR (i.e., not here) would be very helpful :)

Closing this issue now since it is taken care of in descqa repo: https://github.com/LSSTDESC/descqa/issues/7


Calculation Group Attributes and Best Practice Analyser wierdness

Hi Daniel, we have some existing BP rules against tables and columns and are now just implementing Calculation Groups and having a couple of unexpected issues

BP rules that are indicated for Tables are also picking up Calculation Groups. It doesn't seem that this was your intention as you do have a specific type for Calculation Groups. I have been able to work around this by putting in a specific ObjectType filter in the code, but it might be worth reviewing the design of this

A similar problem with the Calculation Group Attribute (column), however I can't seem to screen this one out with logic in the BP rule. The column shows as a CalculationGroupAttribute through the UI properties pane when navigating via the object tree view, but when I interrogate using Advanced Scripting I see it is showing as a normal column?

Really weird. Any ideas how I can screen for these (I guess I will just manually ignore for now)? I guess it would be nice to have a specific type for these in the BP analyser also as the types are pretty distinct

We would like to add Annotations to Calculation Items and this does seem to be supported but the UI doesn't allow it. Any chance you could support this?

Regarding 1 and 2: This is a bug. Rules applied to "Tables" should not consider Calculation Group tables, in the same way that they don't consider Calculated Tables. Similarly for the columns on Calculation Groups (called "Attributes").
Regarding 3: Although it seems supported by XMLA, the TOM (as of 18.2.3) does not support annotations on calculation items:

I will reach out to Microsoft to figure out if this is an oversight in the TOM.

@o-o00o-o it turns out that the AS engine does not actually support annotations on calc items. It's an error in the documentation. In my opinion they should be there, but the AS team does not agree. May I ask what you intended to use these annotations for?

Thanks @otykier - One of the main things that we use Annotations for is to record discrete information for our data dictionary that essentially serves as the user manual for the Cube.
We have a different selection of fields for Tables/Columns and Measures and we have Best Practice Rules to ensure that we always have sensible values for each of them
For measures we have the following required Annotation fields

Simple Description (basic overview information)
Friendly Formula (pseudo code to help users understand what it does)
Worked Example (in Gherkin style GIVEN x WHEN y THEN z ) to show how the calculation works with an example for more complex formulas)
Business Uses (examples of how the business could use this to make better business decisions)
See Also (helps discovery of other related areas of the cube)
Owner (who should be consulted if a change to the formula is desired)

We then take some of these and run a Custom Action during build to format these into the standard Description field so they appear in the tooltip in Power BI but also we expose these internals via a separate "Data Dictionary" Power BI report that provides all the details in the discrete fields to allow more detailed searching/understanding of our model for more advance users and architects
As these Calculation Items are essentially Measure modifiers and do provide a calculation, I think most of these fields are still relevant in the same way they are for measures.
Ultimately if the team think a Description field is relevant (which they do as it is there!), then I would argue that the justification for Annotations is there by association?

Thanks a lot. I will forward this to the team. Hopefully we can convince them that it is worth having the annotations there, even though they might not be able to backport to SQL Server 2019.

The 2.9.0 release fixes the first two issues, regarding the Best Practice Analyzer.
The "Calculation Group Attribute" is no longer a thing in Tabular Editor. Instead, columns on Calculation Group Tables are simply that - columns. If you want to restrict your BPA rule to consider only columns on Calculation Group Tables, add the following to your rule: Table.ObjectType = "CalculationGroupTable".
Regarding annotations, I will let you know once I hear back from the team. If they are added to AS/TOM at a later time, they will automatically be included in a newer version of Tabular Editor, thanks to the way I generate code based on TOM. Closing this issue for now.

@o-o00o-o : Response from Akshai Mirchandani, AS engine team, regarding Calc Item annotations:

I agree that it is useful, and it is something that we should have added. But at this time, it is not possible to add it in the SQL 2019 release. We could potentially add it to a future compat level for AAS, but this is not currently on the roadmap.
There is an easy workaround:
To define annotations based on the name of the calc item in the calc group object
E.g. “YoY Description”, “YoY Friendly Formula”
I agree that it would be much more convenient to do so on the item itself, but its simply not likely to happen anytime soon.


Fixed size difference between OptionButton and OptionRow

Minor fix, now OptionRow and OptionButton elements have the same height. Also fixed the "Night mode" OptionRow in SettingsCard being too far to the top, which caused it look cut-off at the top.
Before:

After:

OK, played around with this a bit. It looks like this is doing two different things:

Add a divider at the top of SettingsCard, which is the main settings screen. This fixes the issue where the "Night mode" row is squished into the top of the screen.

Add some padding attributes to the style of the label in OptionRow.

It seems like these are independent changes, so they should be two separate commits to help make it easier to read and understand them. If you haven't already, take a look through our guide on commit style.
Once they're separated, I'd be glad already to merge the commit that adds the divider.
For the second change: this seems reasonable to match OptionButton... but I can't actually locate what effect it has in the UI as rendered. What's this one doing? Some more explanation in the commit message would be helpful. (Or feel free to discuss on #mobile -- once we understand it, you can put a summary in the commit message, or I can do so before merging.)
Thanks again!


ModuleNotFoundError: No module named 'zope'

When I use the SSL function with the latest version, the software reports an error：“ModuleNotFoundError: No module named 'zope'”。
When I use the same configuration and another version of github-pr-2411 to start the ssl function, there is no problem.

Same issue

Same issue

同样的问题，有解决的方案吗

同样的问题，有解决的方案吗

using image github-pr-2411

Very Sad......why so many problem, cloudflare and dnspod both faild....

same problem, use latest docker image, cannot update ssl certificate.

It seems zope is not installed in docker image by default.
I installed it, and it worked.
pip install zope

It seems zope is not installed in docker image by default. I installed it, and it worked.
pip install zope

It works! Thanks.

dnspod
pip install certbot-dns-dnspod

网络问题，容器的pip没有走代理，使用全局代理或者在容器内部pip config set global.proxy http://xx.xx.xx.xx:port ,之后在容器内pip install zope 就可以了

I thought docker makes things portable.

Version updates do not resolve this problem.
To fix, I have to run the below in the container:
pip install -r https://zopefoundation.github.io/Zope/releases/master/requirements-full.txt

What version of NPM are you using? The startup is meant to install the certbot pip packages based on the DNS providers used in your configuration. You should be able to observe this on container startup within the logs. The PR #2411 has been merged and released in 2.9.20, though I'd suggest using 2.9.21 docker image tag

Hello,
I have the same issue with the latest release...
It did work in the past installing zope in the container, but since the upgrade it always throw these messages at startup:
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
cont-init: info: running /etc/cont-init.d/01_perms.sh
/package/admin/s6-overlay-<IP_ADDRESS>/etc/s6-rc/scripts/cont-init: 20: /package/admin/s6-overlay-<IP_ADDRESS>/etc/s6-rc/scripts/cont-init: /etc/cont-init.d/01_perms.sh: not found
cont-init: info: /etc/cont-init.d/01_perms.sh exited 127
cont-init: info: running /etc/cont-init.d/01_s6-secret-init.sh
/package/admin/s6-overlay-<IP_ADDRESS>/etc/s6-rc/scripts/cont-init: 20: /package/admin/s6-overlay-<IP_ADDRESS>/etc/s6-rc/scripts/cont-init: /etc/cont-init.d/01_s6-secret-init.sh: Permission denied
cont-init: info: /etc/cont-init.d/01_s6-secret-init.sh exited 126
cont-init: warning: some scripts exited nonzero
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service prepare: starting
❯ Checking folder structure ...
❯ Enabling IPV6 in hosts: /etc/nginx/conf.d
❯ /etc/nginx/conf.d/production.conf
❯ /etc/nginx/conf.d/default.conf
❯ /etc/nginx/conf.d/include/ip_ranges.conf
❯ /etc/nginx/conf.d/include/block-exploits.conf
❯ /etc/nginx/conf.d/include/force-ssl.conf
❯ /etc/nginx/conf.d/include/ssl-ciphers.conf
❯ /etc/nginx/conf.d/include/assets.conf
❯ /etc/nginx/conf.d/include/proxy.conf
❯ /etc/nginx/conf.d/include/letsencrypt-acme-challenge.conf
❯ /etc/nginx/conf.d/include/resolvers.conf
❯ Enabling IPV6 in hosts: /data/nginx
❯ /data/nginx/proxy_host/1.conf
❯ /data/nginx/proxy_host/5.conf
❯ /data/nginx/proxy_host/15.conf
❯ /data/nginx/proxy_host/6.conf
❯ /data/nginx/proxy_host/10.conf
❯ /data/nginx/proxy_host/16.conf
❯ /data/nginx/proxy_host/9.conf
❯ /data/nginx/proxy_host/4.conf
❯ /data/nginx/proxy_host/14.conf
❯ /data/nginx/proxy_host/8.conf
❯ /data/nginx/proxy_host/3.conf
❯ /data/nginx/proxy_host/2.conf
❯ /data/nginx/proxy_host/12.conf
❯ /data/nginx/proxy_host/11.conf
❯ /data/nginx/proxy_host/13.conf
❯ /data/nginx/default_host/site.conf

| \ | |  _ |  /  |
|  | | |) | |/| |
| |\  |  __/| |  | |
|| _||   ||  |_|
s6-rc: info: service prepare successfully started
s6-rc: info: service nginx: starting
s6-rc: info: service frontend: starting
s6-rc: info: service backend: starting
s6-rc: info: service nginx successfully started
s6-rc: info: service frontend successfully started
s6-rc: info: service backend successfully started
s6-rc: info: service legacy-services: starting
❯ Starting nginx ...
❯ Starting backend ...
s6-rc: info: service legacy-services successfully started
[3/20/2023] [7:09:57 AM] [Global   ] › ℹ  info      Generating MySQL knex configuration from environment variables
[3/20/2023] [7:09:57 AM] [Global   ] › ⬤  debug     Wrote db configuration to config file: ./config/production.json
[3/20/2023] [7:09:58 AM] [Migrate  ] › ℹ  info      Current database version:<PHONE_NUMBER>5214
[3/20/2023] [7:09:58 AM] [Setup    ] › ℹ  info      Creating a new JWT key pair...
[3/20/2023] [7:10:04 AM] [Setup    ] › ℹ  info      Wrote JWT key pair to config file: /app/config/production.json
[3/20/2023] [7:10:05 AM] [Setup    ] › ℹ  info      Added Certbot plugins certbot-dns-infomaniak~=0.1.12
[3/20/2023] [7:10:05 AM] [Setup    ] › ℹ  info      Logrotate Timer initialized
[3/20/2023] [7:10:05 AM] [Setup    ] › ℹ  info      Logrotate completed.
[3/20/2023] [7:10:05 AM] [IP Ranges] › ℹ  info      Fetching IP Ranges from online services...
[3/20/2023] [7:10:05 AM] [IP Ranges] › ℹ  info      Fetching https://ip-ranges.amazonaws.com/ip-ranges.json
[3/20/2023] [7:10:05 AM] [IP Ranges] › ℹ  info      Fetching https://www.cloudflare.com/ips-v4
[3/20/2023] [7:10:05 AM] [IP Ranges] › ℹ  info      Fetching https://www.cloudflare.com/ips-v6
[3/20/2023] [7:10:06 AM] [SSL      ] › ℹ  info      Let's Encrypt Renewal Timer initialized
[3/20/2023] [7:10:06 AM] [SSL      ] › ℹ  info      Renewing SSL certs close to expiry...
[3/20/2023] [7:10:06 AM] [IP Ranges] › ℹ  info      IP Ranges Renewal Timer initialized
[3/20/2023] [7:10:06 AM] [Global   ] › ℹ  info      Backend PID 120 listening on port 3000 ...
[3/20/2023] [7:10:06 AM] [SSL      ] › ✖  error     Error: Command failed: certbot renew --non-interactive --quiet --config "/etc/letsencrypt.ini" --preferred-challenges "dns,http" --disable-hook-validation
An unexpected error occurred:
ModuleNotFoundError: No module named 'zope'
at ChildProcess.exithandler (node:child_process:402:12)
at ChildProcess.emit (node:events:513:28)
at maybeClose (node:internal/child_process:1100:16)
at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)

[3/20/2023] [7:10:08 AM] [Express  ] › ⚠  warning   invalid signature
I am using infomaniak as provider...
Any suggestion?

By the way, I applied what is proposed in https://github.com/NginxProxyManager/nginx-proxy-manager/issues/2709 and I have been able to renew...

What version of NPM are you using?

Always using jc21/nginx-proxy-manager:latest - Zope has been broken for quite some time. No version upgrades have resolved the issue. I'm a little stumped as to why this isn't an issue for more people?

I'm also using "latest" and my instance got updated by watchtower around two hours ago.
With the previous release, there was no error in the nginx-npm log but now I can see the following:
`[3/27/2023] [9:44:16 AM] [SSL      ] › ✖  error     Error: Command failed: certbot renew --non-interactive --quiet --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --preferred-challenges "dns,http" --disable-hook-validation
Renewal configuration file /etc/letsencrypt/renewal/npm-1.conf (cert: npm-1) produced an unexpected error: 'Namespace' object has no attribute 'dns_desec_propagation_seconds'. Skipping.
0 renew failure(s), 1 parse failure(s)
at ChildProcess.exithandler (node:child_process:402:12)
at ChildProcess.emit (node:events:513:28)
at maybeClose (node:internal/child_process:1100:16)
at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)`

And it's no longer possible for me to log in into the web UI because it tells me that my actual user (which is valid) is no "relevant" user.

Pls. advise how to solve it. Thanks.

雀食
默认的pip源被xx了，基本跑不动
国内的童鞋可以试试清华或阿里的镜像：
pip install zope -i https://pypi.tuna.tsinghua.edu.cn/simple
or
pip install zope -i http://mirrors.aliyun.com/pypi/simple/

docker使用命令
docker exec 容器名 pip install zope
可解决该问题

docker使用命令
docker exec 容器名 pip install zope
可解决该问题

这个方法有用

DNSPOD

DNSPOD是中国专属，作者不使用DNSPOD，所以无法重现该问题。

So many issues with NPM. Just landed here, I am only evaluating it (long time Apache old school user) and here's my experience within 15 first minutes
You go in SSL certificate which is one of 7 standalone section. Its not it a niche experimental thing, no, its one of the major feature of npm. Then you begin by adding your (existing) domain and do a "Test server reachability". ERROR. Then you come to github and found 100 users reporting the problem for 1 year and stating it has never worked", an you read that  'but adding a SSL cert is fine so dont worry about the test"
Then you try to add your SSL certificate, and find this error.
And you read things like
"oh yes, its not installed, you should install it... pip install blabla " Man, you know what the major purpose of containerization is ??? How is that possible that a component needed by a major feature isn't bundled in the image???
Then you read things like "are you using the latest version ?" Come on. Yes it is :latest tag.
But also " Zope has been broken for quite some time."
and finally things like: "'I was on latest version and its broken I had to downgrade...."
At this point I only conclude that all this isn't serious at all.

Open the console of the docker, then follow the highlighted command in the picture.

It seems zope is not installed in docker image by default. I installed it, and it worked.
pip install zope

Very good, this method solved the problem

docker使用命令
docker exec 容器名 pip install zope
可解决该问题

nice

There's already a PR to fix it since 1 year ago: https://github.com/tengattack/certbot-dns-dnspod/pull/12
But it seems that the owner of certbot-dns-dnspod doesn't maintain it anymore.
I don't think NPM has responsibility to fix it, but we need a solution when these plugins (not only dnspod) are outdated.

There's already a PR to fix it since 1 year ago: tengattack/certbot-dns-dnspod#12 But it seems that the owner of certbot-dns-dnspod doesn't maintain it anymore. I don't think NPM has responsibility to fix it, but we need a solution when these plugins (not only dnspod) are outdated.

I think it still not too late to said that: fixed now :)

I am trying to use the infomaniak dns challenge and there is the same issue,
on that issue on their GitHub page they indicate the issue has been resolve on their side and they suppose NGINX does not use the latest version...

https://github.com/Infomaniak/certbot-dns-infomaniak/issues/34

I use NGINX as TrueNas package, version 1.0.29, based on docker image version 2.11.1.

maybe they need to update global/certbot-dns-plugins.json
https://github.com/NginxProxyManager/nginx-proxy-manager/blob/02aefa50cd0a81fca6baf0b93b66ee4665282b35/global/certbot-dns-plugins.json#L242-L249
also bump the version of certbot-dns-dnspod

For me it has been solved in the latest update

For me it has been solved in the latest update

Maybe we can close this issue now 😄

for me as well!!
please don't close but better fix this issue.

6 months and issue not fixed :'(


Add mappings to open a file in a split, vsplit, or tab

It would be great to be able to open a file not just in the current buffer but also in a tab, split, or vsplit.

Thank you!


Bump time version. This makes the project buildable again.

Thank you for this! It looks like cargo machete doesn't like the unused crate though, so I'll just update the time crate in the lock file instead.

Fixed by commit 7bc48eb


Dumping the stack of managed threads

I often can't get the managed stack for the code being executed either by attaching a process or snapshotting it.
The threads are enumerated correctly by enumerating ClrRuntime.Threads. For each one of them I then call EnumerateStackTrace obtaining the stack frames represented by a ClrStackFrame list.
In very few cases I am able to see the real stack trace being executed at the snapshot/attach moment but, on those threads, in most of the cases I can only see a frame called DebuggerU2MCatchHandlerFrame which I believed was only a finalizer-related thing.
What is this object, and how can I 'decode' to get back the real stack?
I did my tests by running some heavy work in an asp.net core application, printing the managed thread id of the thread executing the cpy code for several tenths of seconds.

Is this on Windows or Linux that you are seeing this?  Those are two very different codepaths.

I am seeing this on Windows x64, but I am interested in understanding the problem for other platforms as well.
Thanks

There may be a bug here, or it may be expected.  Here's a quick description of when it would be expected:
ClrMD uses clr's stackwalker to attempt to unwind the stack.  By design, if the CLR stackwalker encounters a non-managed instruction pointer in the eip or rip register it assumes that we have normally exited managed code and put a clr!Frame on the stack.  It will attempt to look for that clr!Frame object on the stack to help unwind and start the unwind from that object (note that clr!DebuggerU2MCatchHandlerFrame is a clr!Frame object).
It's possible that we can be "unlucky" and called into native code briefly before setting up a frame and that will confuse CLR's stackwalker.  It happens regularly but we mostly assume that to be rare for most cases.  However if you are seeing this a lot then there might be a bug here, or we might be mistaken in thinking it's rare.  In order to find out we'd need to probably implement your suggestion in #760, to save off snapshots to see what the real stack is in WinDbg.
If there's not a bug here, I'm not sure there's much we can do to fix this.  ClrMD isn't designed to function as a sampling profiler, it's meant to be a runtime inspection tool.  What we are reporting to you is the stack as the CLR runtime sees it (again...that's assuming no bug in ClrMD), and what it sees at the time you took the snapshot is that [DebuggerU2MCatchHandlerFrame] was the top of the managed "stack".
I will try to reproduce this myself, and if I can't I'll see if we can implement #760 so we can get you to collect more information here.  (Also FYI I just got back from a vacation so I'm a bit behind on work...it might be a few days before I get to this.)

Got it. Take your time and let me know if you need help to repro the issue.
Let's say I try this operation 20 times, I can see the real stack just one or two times.
I suspect it is a bug because, even if I was working on a snapshot, enumerating two times the threads and the stack frames, I saw different results: the first time I could see the stack, the following times I could only see DebuggerU2MCatchHandlerFrame.
The repro should be pretty simple. In any case tomorrow morning I will present a session in an online conference about diagnostics and CLRMD is one of the pillars :) As soon as I publish the demo I can point you to my code but I suspect that it won't be that useful.
Thanks

Additional notes:

I can always see the correct stack trace for the main thread (the one who run the asp.net core machinery).
The long-running thread is pure CPU (dumb primes computation) with no async stuff in the middle, and started from a request in an API controller (over which I have no control, but I do not expect that to be special)

I got dumps with [DebuggerU2MCatchHandlerFrame] on the stack.
Atleast ClrMD & Windbg (using SOS) shows the same output.

Hello, I'm facing the same issue on Linux.
We have a troubleshooting tool that uses CLRmd to show current "parallel stacks" of the application (connecting with dataTarget = DataTarget.AttachToProcess(pid, suspend = true);). In my reproduction the stacks are always correct for the threads that are waiting (Thread.Sleep, mre.WaitOne, etc). However, for the threads that are actively consuming CPU, there's only one frame like below
OS Thread ID: 965 Managed thread id: 7 Alive: True Aborted: False State: 16912928 IsWorkerThread: True IsIOCompletionThread: False
    Kind: Runtime Method:  FrameName: DebuggerU2MCatchHandlerFrame IP: 610053002E0000 MethodByIP:

It's very stable; even after 100 iterations of creating/destroying dataTarget I didn't catch a working stackwalk for the interesting thread.
When I take a full memory snapshot with dotnet-dump collect --type Full, the stacktrace is shown correctly
> clrstack
OS Thread Id: 0x965 (21)
        Child SP               IP Call Site
00007F9E9E9A2440 00007F9EA33F0FF7 Ghost.TestApps.BurnCpuComponent.Process(Ghost.TestApps.BurnCpuParam, System.Threading.CancellationToken) [C:\dev\repos\troubleshooting\app-hosting\ghost\Ghost\Ghost.TestApps\BurnCpu.cs @ 45]
00007F9E9E9A2490 00007F9EA33EE91A Ghost.Core.ComponentMiddleware`4+<>c__DisplayClass6_0+<<Invoke>g__ProcessRequest|0>d[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]].MoveNext() [C:\dev\repos\troubleshooting\app-hosting\ghost\Ghost\Ghost.Core\ComponentMiddleware.cs @ 75]
00007F9E9E9A2800 00007F9EA33ED002 System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Ghost.Core.ComponentMiddleware`4+<>c__DisplayClass6_0+<<Invoke>g__ProcessRequest|0>d[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]], Ghost.Core.netcore]](<<Invoke>g__ProcessRequest|0>d<System.__Canon,System.__Canon,System.__Canon,System.__Canon> ByRef) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncMethodBuilderCore.cs @ 38]
00007F9E9E9A2860 00007F9EA33ECEAF System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.ValueTuple`2[[System.__Canon, System.Private.CoreLib],[Ghost.Contract.InvocationResult, Ghost.Contract.netcore]], System.Private.CoreLib]].Start[[Ghost.Core.ComponentMiddleware`4+<>c__DisplayClass6_0+<<Invoke>g__ProcessRequest|0>d[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]], Ghost.Core.netcore]](<<Invoke>g__ProcessRequest|0>d<System.__Canon,System.__Canon,System.__Canon,System.__Canon> ByRef) [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncTaskMethodBuilderT.cs @ 36]
00007F9E9E9A28A0 00007F9EA33ECA05 Ghost.Core.ComponentMiddleware`4+<>c__DisplayClass6_0[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]].<Invoke>g__ProcessRequest|0()
00007F9E9E9A2930 00007F9E9EE39C21 System.Threading.Tasks.Task`1[[System.__Canon, System.Private.CoreLib]].InnerInvoke() [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs @ 503]
00007F9E9E9A2950 00007F9EA2C1BE4E System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(System.Threading.Thread, System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs @ 268]
00007F9E9E9A2990 00007F9EA2C225B0 System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 2337]
00007F9E9E9A2A10 00007F9E9EC97F78 System.Threading.Tasks.Task.ExecuteEntryUnsafe(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 2271]
00007F9E9E9A2A30 00007F9E9EC97F17 System.Threading.Tasks.Task.ExecuteFromThreadPool(System.Threading.Thread) [/_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs @ 2262]
00007F9E9E9A2A40 00007F9EA1FA4FD3 System.Threading.ThreadPoolWorkQueue.Dispatch()
00007F9E9E9A2AB0 00007F9E9EC93BC5 System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart() [/_/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.WorkerThread.cs @ 63]
00007F9E9E9A2B80 00007F9E9EC78432 System.Threading.Thread.StartCallback() [/_/src/coreclr/System.Private.CoreLib/src/System/Threading/Thread.CoreCLR.cs @ 105]
00007F9E9E9A2CF0 00007f9f18916a27 [DebuggerU2MCatchHandlerFrame: 00007f9e9e9a2cf0]

It also contains DebuggerU2MCatchHandlerFrame but with the different IP. When I open this snapshot with CLRmd it shows correct result (same as clrstack).

I think I've found a bug that explains this behavior on Linux!
In LinuxLiveDataReader.cs there's a ptrace(PTRACE_ATTACH, pid, 0, 0) call to attach to the process. However, the ptrace manual says we should attach to every thread id separately. When I call PTRACE_ATTACH for every tid the stacks are read correctly.
Also DataTarget.CreateSnapshotAndAttach(pid) works fine on Linux.

When I call PTRACE_ATTACH for every tid the stacks are read correctly.

That's great!  Can you submit a PR for this?  Or even just a draft showing the code you changed?  No worries if not, but it would speed things along.


Choose favicon & native icon

I used http://realfavicongenerator.net/ to create placeholder "E" favicon when testing Este so it has nice icons for iOS/Android/Windows Phone (homescreen/bookmark panel/tiles).

My friend created these suggestions

I personally like CT and JS as they are somehow unique and easy to recognize :) It's great to have favicon to quickly switch between browser tabs but my purpose of doing all the mobile icons was to implement it and test it before production ready graphic is done so I'm sure it works on all devices.

Rather JS than CT.

We need an icon for iOS/Android as well.

#583

One between Vimeo & YTube is really nice.

@srigi have you seen TV Series Mr. Robot  that one you have suggested is similar to bigest corporation EVIL CORP :smile:

I'd rather make Yeoman-like icon but with Daniel :8ball:

@ondrejbartas :-D

Done. Unfortunately I didn't use any Americo Herman beautiful proposals.


big polymath replacement, plus two minor oops bugfixes

The two changes in oops ought to be no-brainers. Regarding polymath, it might not be useful to look at the source code changes side-by-side because there are so many. However, unit tests all pass (polymath, oops, hosts) so maybe just try it at your end to make sure everything still works. Code commentary always welcome, of course!

All unit tests work.  There are still many relative imports.  I have created a new issue about that.

The polymath change breaks existing oops code:
    ra = center_bp.right_ascension(apparent=False).vals.astype('float')
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'float' object has no attribute 'astype'

In this code, center_bp is a Backplane that covers a single point (using a Meshgrid with a single point). Previously the result of evaluating this backplane was an array with a .vals attribute, but now it appears to be a single float. I think this is a bad change because it means whoever uses the result of a Backplane operation needs to check to see if the result is an array or a single float if they don't know what Meshgrid was used.

The polymath change breaks existing oops code:
    ra = center_bp.right_ascension(apparent=False).vals.astype('float')
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'float' object has no attribute 'astype'

In this code, center_bp is a Backplane that covers a single point (using a Meshgrid with a single point). Previously the result of evaluating this backplane was an array with a .vals attribute, but now it appears to be a single float. I think this is a bad change because it means whoever uses the result of a Backplane operation needs to check to see if the result is an array or a single float if they don't know what Meshgrid was used.

This does not appear to be an error in oops. It is perfectly fine for the .vals property to be a scalar float--always has been. Furthermore, we know that right_ascension backplanes contain floating-point numbers, so why would one apply .astype("float") to the .vals property? Either it is an array of dtype=float already, or else it is a scalar in which case ".astype" will fail.


Don't unconditionally link gtk-3 on not windows

The iup package on my system links to gtk-2, but kiss-ui unconditionally tries to link to gtk-3, making the build fail.
One possible solution to this letting the user decide whether he wants to link to gtk-3 through a cargo feature. But this might not be a very ideal solution.
Here is what I cooked up:
diff --git a/Cargo.toml b/Cargo.toml
index 6f005ea..c9c6b0d 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -5,6 +5,10 @@ authors = ["Austin Bonander<EMAIL_ADDRESS>
 [dependencies.iup-sys]
 path = "iup-sys/"
+features = [""]

 [dependencies]
 libc = "*"
+
+[features]
+no-gtk3 = ["iup-sys/no-gtk3"]
diff --git a/iup-sys/Cargo.toml b/iup-sys/Cargo.toml
index 805e976..74a45e1 100644
--- a/iup-sys/Cargo.toml
+++ b/iup-sys/Cargo.toml
@@ -8,3 +8,6 @@ build = "build.rs"

 [dependencies]
 libc = "*"
+
+[features]
+no-gtk3 = []
diff --git a/iup-sys/build.rs b/iup-sys/build.rs
index 464af6c..ceee847 100644
--- a/iup-sys/build.rs
+++ b/iup-sys/build.rs
@@ -1,7 +1,7 @@
 fn main() {
     add_link("iup");

-    if cfg!(not(windows)) {
+    if cfg!(not(windows)) && std::env::var_os("CARGO_FEATURE_NO_GTK3").is_none(){
         add_link("gtk-3");
     }
 }

Actually, it looks like the IUP driver is linked to a specific version of GTK+ when it was compiled. If you try to link GTK+2.0 on a system with IUP linked with GTK+3.0 installed, you get the following error:

Gtk-ERROR **: GTK+ 2.x symbols detected. Using GTK+ 2.x and GTK+ 3 in the same process is not supported

So that line can probably just be deleted. I thought I needed it to fix some build errors in Travis-CI but it appears I was just using the wrong IUP download.

We'll see how this build turns out: https://travis-ci.org/cybergeek94/kiss-ui/builds/63223596
Now comes the only problem with working during the day: Travis takes forever to boot. :laughing:

The build for #10 passed, looks like the fix is to let IUP choose the GTK+ version.


Documentation: Chinese version

@ygj6 can you update the Chinese documentation with the changelog for 2.3.12 and the new GridItem preserveAspectRatio prop?
Thanks

@gmsaI will update when I have free time.

@gmsaI will update when I have free time.


Trying to figure why this code fragment never compiles

Description:
I'm trying to figure out why following code fragment never compiles:
public type XYZ record {
    string x;
    string y;
    int z;
};

function sample() returns XYZ | error {

    XYZ | error sampleValue = {x: "X", y :"Y", z: 1 };

    if ( sampleValue is XYZ) {
        if (isZpositive(sampleValue)){
            sampleValue = error("this line compiles");
        } else if isXNotEmpty(sampleValue) {
              sampleValue = error("above line never compiles");
        } else if isYEmpty(sampleValue) {
              sampleValue = error("above line never compiles");
        }
    }

    return sampleValue;
}

function isZpositive(XYZ xyz) returns boolean {
    return xyz.z > 0;
}
function isXNotEmpty(XYZ xyz) returns boolean {
    return xyz.x.length() > 0;
}

function isYEmpty(XYZ xyz) returns boolean {
    return xyz.y.length() == 0;
}

Steps to reproduce:
Copy and pase above code fragment and see compile error in IntelliJ
Affected Versions:
1.0.1
OS, DB, other environment details and versions:
Mac OS, JDK 1.8
Suggested Assignees (optional):
@MaryamZi

The errors here are "incompatible types: expected 'XYZ', found '(XYZ|error)'", because the sampleValue variable's type is no longer the narrowed type once it is assigned to.
This is in-line with the spec
the narrowed type for a variable x no longer applies as soon as there is a
possibility of x being assigned to

This is to avoid scenarios such as the following
function sample() returns XYZ | error {

    XYZ | error sampleValue = {x: "X", y :"Y", z: 1 };

    if (sampleValue is XYZ) {
        if (isZpositive(sampleValue)){
            sampleValue = error("this line compiles");
        }

        // `sampleValue` is assigned to within the previous if block.
        // so it could even be an `error` now.
        // Type narrowing should no longer apply.
        if isXNotEmpty(sampleValue) {
            sampleValue = error("above line never compiles");
        }
    }

    return sampleValue;
}

But yeah, in your example, this might not make sense since the if-else-if blocks are mutually exclusive.
This could be an improvement to type narrowing.

The errors here are "incompatible types: expected 'XYZ', found '(XYZ|error)'", because the sampleValue variable's type is no longer the narrowed type once it is assigned to.
This is in-line with the spec
the narrowed type for a variable x no longer applies as soon as there is a
possibility of x being assigned to

Hmm. I don't think the behavior we see is in line with the spec. What @ramith has used is an if-else-if block. Therefore there is no logical path that could do the assignment to sampleValue and execute isXNotEmpty(sampleValue). Therefore "there is no possibility of x being assigned to".

This is to avoid scenarios such as the following
function sample() returns XYZ | error {

    XYZ | error sampleValue = {x: "X", y :"Y", z: 1 };

    if (sampleValue is XYZ) {
        if (isZpositive(sampleValue)){
            sampleValue = error("this line compiles");
        }

        // `sampleValue` is assigned to within the previous if block.
        // so it could even be an `error` now.
        // Type narrowing should no longer apply.
        if isXNotEmpty(sampleValue) {
            sampleValue = error("above line never compiles");
        }
    }

    return sampleValue;
}

But yeah, in your example, this might not make sense since the if-else-if blocks are mutually exclusive.
This could be an improvement to type narrowing.

@a5anka - sorry, what I meant was that resetting of the type could happen. Like I said, agreed that the current behaviour for @ramith's example doesn't seem to make sense.
All, this does not work even for the following scenario atm,
public function main() {
    string|int val = "hello";

    if (val is string) {
        if (false) {
            string s = val;
            val = 1;
        } else {
            foo(val); // incompatible types: expected 'string', found '(string|int)'
        }
    } else {
        int i = val;
    }
}

function foo(string s) {
}

We recursively reset the type for the applicable previous environments once a variable is assigned to. Thus resetting it for if (val is string) {} seems to apply to the inner else block too.
We may have to revisit the concept of regions of code and how we've modeled it for type narrowing alongside the spec - see https://v1-0.ballerina.io/spec/lang/v2019R3/#section_7.11.
Design doc for type narrowing - https://docs.google.com/document/d/12stYd4PWSK3yauFdcvueOju2BkF0krWbYtJA6DvTqKw/edit


[ui.content] - acl handling on package

Editable templates in /conf have acls to allow anonymous users to read templates. ui.content package needs to acl handling: merge_preserve.

Fixed


Decide whether to use checkboxes or ctrl/shift to enable multi-selection in tree

There are some situations when we need to run only specific tests from a suite, so we need to select only some of them(and not the entire suite).
A workaround for this is to use checkboxes, to specify which tests to run, but would be easier if multi-select will be enabled while holding Ctrl key.

There are lots of strange behvaiours possible here:
1: When allowing multi-select we also should allow the use of shift-select. How should this work across branches in the tree?
2: Selecting af parentnode and then selecting and deslecting childnode - what is the expected behaviour?
I actually think that checkboxes make more sense. Select at parent-node will select all child nodes. Deselect at child node and the parent nodes the checkbox goes into tri-state "semi-selected" mode.

Since @iCodeIT started discussing the best way to select here, let's make the issue about that.
I retitled it to be about multi-selection in general. My feeling is we should not support both checkboxes and ctrl/shift selection, but pick one or the other.
One data point: the VS test explorer window does not use checkboxes, in case we want to look like them.
@nunit/gui-team Please comment so we can decide which way to go.

i like the idea of making sessions or run lists.
from a programming standpoint i would think supporting multiple ways of selecting sort of trivial. say we make a selectable view as a base and use a factory to return different selectable views. i could put together a mock up of what im suggesting sometime this week.

Supporting multiple ways of selection could be trivial to program... depending on how you define it.
My concern is for usability. I think having multiple ways of doing things all active simultaneously can be very confusing for the user, which is what we are thinking about here. As far as programming, that's all pretty much trivial for guys like us. :smile:
I've seen flat lists with check boxes where you could click them directly or use ctrl/shift selection to check all of them. That seems simple for the user to navigate as well as for us.
But if we start talking about a tree, where there checking an item checks the subitems, it does get a bit more complicated, especially if you try to do a shift-select across two different levels of the tree. What does that mean? Add to that the possibility of three-state checkboxes and it gets even more complex.
If you do a mockup, be sure to just make it as low-tech as possible. If we were all together, you could draw it on the white board, but I'm afraid that's not on. It's too easy to overdo the design when using most visualization tools to do it.

I will reiterate my feelings on this.  I have been supporting a NUnit 3 GUI since before NUnit 3 was released.
I could not imagine it being productive without checkboxes.  Using multi-select with a large number of tests selected is almost impossible (I sometimes select 100 or more myself).
I use the Telerik tree control, and I have added a right click context menu item select/unselect/toggle which works on multi-selected nodes if that functionality is desired, correctly setting the checkboxes.  That way, I get the best of both worlds.

I am coming to feel the same way. No reason for this to be an either/or thing.
I'm thinking of own-drawing the check boxes as the windows check boxes seem too heavy to me.

At least it is very non-intuitive that you currently have to go to the context menu and enable checkboxes before being able to select a set. And then you have to remember clicking "Run checked items". People always click Run. And the checkbox selection does not update the subtrees which confuses them.

Check boxes are more persistent than ctrl/shift. With check boxes, for example, you can go look at a test's outcome without loosing checked state. You cannot do this with ctrl/shift.
Thinking about persistency, one could argue that if there are check boxes, their state should be persisted across sessions, as NUnit GUI v2 does it.
If there are check boxes, they should be tri-state, as @iCodeIT suggests in https://github.com/CharliePoole/nunit-gui/issues/48#issuecomment-167835501.

what if we had a checked state that the view could take into account when showing if it is checked or selected with ctrl/shift? then the presenter could just get all objects that are "checked" and pass that on to the model.

In recent years, I've seen GUIs that used check boxes but allowed use of ctrl / shift in checking ranged of them. I'd like to aim for that.

and inverse selection!


feat(material/menu): support zone-less mode

The template:
<button mat-button [matMenuTriggerFor]="menu">Menu</button>
<mat-menu #menu="matMenu">
  <button mat-menu-item>Item 1</button>
  <button mat-menu-item>Item 2</button>
</mat-menu>

Basically, the mat-menu is never opened when the zone is nooped through { ngZone: 'noop' }. I've left comments near my changes. The CI will prolly fail so I'll fix it after it's run.
Before
https://user-images.githubusercontent.com/7337691/154817177-e83822d7-6fb1-487c-b101-57385deed9bb.mp4
After
https://user-images.githubusercontent.com/7337691/154817212-5cc30100-5c74-4b19-9c49-66910238633c.mp4

Looks like the CI is still failing on this - do you want to continue this PR?

Looks like the CI is still failing on this - do you want to continue this PR?

Yeah, I have to revisit it once again, will try to find time for that. Otherwise, I'll just close it.

I don't think that we should be supporting this. There's way more than the menu that is going to break when using the noop NgZone. We should a library-wide approach to handling it.


Dagger2 + Dynamic feature

Copied Dagger stuff to
https://github.com/EkoCommunications/EkoMessagingSDKAndroidSample/pull/29/


Configurable / Additional heights for nimble controls

🧹 Tech Debt / Enhancement
Currently the height of nimble controls is defined by the --ni-nimble-control-height css custom property. Our control styles are written in such a way that they are dependent on --ni-nimble-control-height custom property such as with fixed css calc() statements that use the --ni-nimble-control-height property. This is further discussed in the "Arbitrary Sizing" section.
In addition, we currently do not support most of the additional component heights shown in Figma Specs. This is further discussed in the "Predesigned Sizing" section.
Arbitrary Sizing
The design of the height of nimble controls being defined by the '--ni-nimble-control-height` css custom property may be unexpected for users as, for example, a user may expect to set the height via css of a control:
<nimble-some-control style="height:50px"></nimble-some-control>

However, this may result in issues as by manually changing the height of the control, the --ni-nimble-control-height constant will not update in response and layout calculations using those values will layout incorrectly.
If a user really required changing the height of a control they have to do it via the --ni-nimble-control-height property:
<nimble-some-control style="--ni-nimble-control-height: 50px"></nimble-some-control>

Some options to consider:

We state firmly that nimble controls have a fixed height that is not intended to be changed by any client. In that case we should treat it like padding and make it more difficult to override control height: https://github.com/ni/nimble/issues/598. We should also consider removing the token / marking the token internal (--ni-nimble-internal-control-height)
We allow the controls to have user-defined heights. The --ni-nimble-control-height token is only used for setting the default height of controls and is not used in CSS layout calculations. CSS is modified to avoid requiring calc() statements using --ni-nimble-control-height.
[not recommended] We support users configuring the height on controls using the --ni-nimble-control-height design token. I do not think we should encourage users to use a CSS property for sizing controls that is proprietary to the nimble design system. We should discourage / forbid users setting --ni-nimble-control-height directly.

Predesigned Sizing
Many new designs for nimble components have additional heights, like the new nimble-switch Figma Design and nimble-button Figma Design. The button design now includes heights of 24px for text buttons and 18px / 16px for icon buttons, but currently the only height implemented for buttons is 32px. These other heights for buttons have use cases that are mentioned in the "Sizes" section of the Nimble Button Usage Guidance. For example:
From "Sizes" of the Nimble Button Usage Guidance
32px buttons are the default sized control.  Use this control size first.

24px buttons are the secondary control size and should be used if there is limited space.  This size should not be used in combination with 32px controls.

18px buttons...

Because of these use cases, these heights for the button (and any other component with multiple heights specified in a Figma Spec) should be implemented as an enhancement.
Current direction
Based on current discussions the expected direction is (2). The control height custom property should only be used to configure the default height of a control. It should not be used in calculations or as a variable meant to be updated internally in a control. For programmatic observation of a control height ResizeObserver should be used (with due consideration for performance)
As for the predesigned sizing, this should be implemented regardless of the decision of arbitrary sizing, as these sizes / heights have specific pre-defined use cases.

This issue should apply to icon-size as well as control-height and we should make sure to update docs and other references (i.e. some stack overflow posts) when we have a consistent direction


Cannot spyOn default export of module

Describe the bug
In Jest a common pattern for me was to spyOn the default export of a module. This no longer works in vitest. It throws the following error:
TypeError: Cannot redefine property: default
 ❯ eval test/suite.test.ts:5:16
      3| import b from '../src/b';
      4|
      5| const aSpy = vi.spyOn(a, 'default');
       |                ^
      6|
      7| describe('suite name', () => {

I get the feeling this is a limitation of ESM, a work around seems to be:
vi.mock('path/to/module', async () => {
  const actual = await requireActual('path/to/module');
  const mocked = {};
  for (const key of Object.keys(actual)) {
    if (typeof actual[key] === 'function') {
      mocked[key] = vi.fn(actual[key]);
    } else {
      mocked[key] = actual[key];
    }
  }
  return mocked;
})

import module from 'path/to/module';

const moduleSpy = vi.mocked(module);

However, this is a lot of code to include for every spy, so perhaps there could be an API like vi.spyOnModule that would automate this, so I could just do:
const moduleSpy = vi.spyOnModule('/path/to/module');

Reproduction
I created a reproduction here: https://stackblitz.com/edit/vitest-dev-vitest-6igo9o?file=test/suite.test.ts

Side note: the latest release (0.5.5) seems to have a regression where it does not show the error message, this seems to be because it checks for calls to it/test to detect if the file has tests. However, because vi.spyOn is failing before those tests are defined, the error message gets suppressed. Hence why I downgraded to 0.5.4 in the repro.

System Info
System:
    OS: macOS 11.2.3
    CPU: (12) x64 Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
    Memory: 39.41 MB / 16.00 GB
    Shell: 5.8 - /usr/local/bin/zsh
  Binaries:
    Node: 16.13.2 - ~/.nvm/versions/node/v16.13.2/bin/node
    Yarn: 1.22.17 - /usr/local/bin/yarn
    npm: 8.1.2 - ~/.nvm/versions/node/v16.13.2/bin/npm
  Browsers:
    Chrome: 98.0.4758.109
    Edge: 98.0.1108.62
    Firefox: 94.0.1
    Safari: 14.0.3
  npmPackages:
    @vitejs/plugin-react: ^1.2.0 => 1.2.0
    vite: 2.8.3 => 2.8.3
    vitest: 0.3.5 => 0.3.5

Used Package Manager
yarn
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

Thank you for posting your solution. I spent hours trying to get this to work to no avail but I got your solution to work for me right away

You can also inline the module you are trying to import with deps.inline: [/path-to-module/]. This will basically do what your workaround does.

@konpeki622 tinyspy tried to redefine descriptor, so an error occurred.

Fixed by https://github.com/vitejs/vite/issues/8566


<EMAIL_ADDRESS>Hash check failed

Hash check failed!
App:         sushi/wechat
URL:         https://dldir1.qq.com/weixin/Windows/WeChatSetup.exe#/dl.7z
First bytes: 77 90 144 0 3 0 0 0
Expected:    3f246e67bfa9529b5b4f186fd495ebc4dc9b0ccdd84651b04531395b2d5ff39e
Actual:      8f3347cab4acd48b22f92a63f59d78e539fc3efad356afbad59d59f0bd308260

Should have been fixed in 242d0f2791111fa3335841c85185fc25858b26f1, please check.


fix(Interaction): parent of the outline highlighter cloned object

Change the parent of the OutlineObjectCopyHighlighter cloned object to
the same parent as the original object.
Change the parent setting to not re-compute world position.
Change the transform assignments from world to local (faster), because
now the parent is the same.
The OutlineObjectCopyHighlighter cloned object needs to have the
parent set to the same parent of the customOutlineModel to exactly
match its shape. Even the PR #828 still causes problems with a hierarchy
of game objects with different scaling (e.g customOutlineModel with its
own scaling as child of the game object with its own scaling, holding
the OutlineObjectCopyHighlighter component).


Avoid puppet errors when SELinux is disabled

Fixes #286.  Puppet will no longer display a runtime error when applying a manifest on a node with selinux disabled.  These changes essentially make the selinux::boolean, selinux::fcontext, and selinux::port types do nothing when SELinux is disabled.

@ekohl can you take another look and merge if there are no issues?

This needs to be done for selinux_permissive too

Could you open a new issue/pr for that? Comments on a closed PR generally don't result in any code change.

@jorhett It should be a simple fix to handle systems running in permissive mode.  Can you create a new issue regarding this?  This PR has already been merged.


Support watchosArm32 Kotlin Native target

Description
We would like to support the watchosArm32 Kotlin Native target.
See the targets supported by Kotlin Native for more details about it.
Dependencies
This issue is blocked by the following ones:

[ ] Currently doesn't support running tests in Gradle.

Checklist

[ ] Add the target in our Multiplatform Gradle plugin, add the target in the integration and the delivery workflows, add badge in README and update the unreleased changelog.
[ ] Close this issue as completed and update tracking ones if present.

This target currently doesn't support running tests in Gradle.


NavigationViewItem selection indicator doesn't display the first time visibility changed

Describe the bug
I have some NavigationViewItem Collapsed at launch. After some action, the visibility changed to Visible and I select it in code behind as SelectedItem. The first time the selected indicator doesn't appear. If you select another one or repeat the scenario (Collapsed -> Visible -> SelectedItem), it works.
Steps to reproduce the bug
Demo project: https://github.com/Psykobal/NavView_Indicator
-Launch the app
-Tap on NavigationViewItem "Switch User" and "Sign Out" to display or hide main NavigationViewItem
Expected behavior
Selection indicator is displayed everytime on NavigationView SelectedItem
Screenshots
-Initial state :

-Tap on "Switch user":

-Tap on "Signout":

-Tap again on "Switch user":

Version Info
NuGet package version:
[Microsoft.UI.Xaml 2.4.0]
[Microsoft.UI.Xaml 2.4.2]
[Microsoft.UI.Xaml.2.5.0-prerelease.200708003]
It was working on [Microsoft.UI.Xaml.2.3.200213001]

Windows 10 version
Saw the problem?

Insider Build (xxxxx)

May 2020 Update (19041)
Yes

November 2019 Update (18363)
Yes

May 2019 Update (18362)

October 2018 Update (17763)

April 2018 Update (17134)

Fall Creators Update (16299)

Creators Update (15063)

Device form factor
Saw the problem?

Desktop
Yes

Xbox

Surface Hub

IoT

It seems that the selection indicator does not get animated correctly since the template for the collapsed NavigationViewItem has not loaded yet, when you change selection. As a workaround, you can add NavView.UpdateLayout() before setting the selected item.


Expose the HloOpcode::kPartitionId in Jax.

Hello, I think it would be very beneficial for Jax to expose the HloOpcode::kPartitionId in Jax.lax.
The HloOpcode::kPartitionId op is the integer value of the partition running that Hlo.
It is very useful when dynamic-slicing.
Example:
32 partitions
tensor = (32, 1024, 1024)
(1,1024,1024) = jax.lax.dynamic_slice(jax.lax.partition_id, 1024, 1024)

I am willing to take on this project, but any pointers would be helpful.

The general process for this would be:

Add a new JAX primitive (e.g. partition_id_p = core.Primitive(...))
Register an MLIR lowering for that primitive.

from jax._src.interpreters import mlir

def your_lowering(...):
  ...
mlir.register_lowering(partition_id_p, your_lowering)

You'll also need to register an abstract eval to provide shape information to JAX.

Call the HLO Python bindings inside of the lowering function.
In this case, it seems like you this already exists in XLA https://openxla.org/stablehlo/spec#partition_id, so the lowering would look something along the lines of:

from jax._src.lib.mlir.dialects import hlo

def your_lowering(...):
   return hlo.partition_id(...)

That is pretty cool. Let me give it a try. It is very useful op, especially in shard_maps.

Feel free to tag me on a PR or ask follow-up questions if you get stuck.

Wouldn't running jax.lax.axis_index(axis) inside a shard_map do the same thing for you?

I see so i could within a shard_map do
jax.lax.dynamic_slice(jax.lax.axis_index('model'))

That is not exactly the same as partition id. Take for example a mesh of Mesh(model=8, dp=4) so 32 partitions
and I have a tensor of rank (32,1024,1024)
jax.lax.dynamic_slice(jax.lax.partition_id(), :, :)
would give a (1,1024,1024) slice per rank. I am confused how to replicate that with the above 2D mesh and axis_index @yashk2810

jax.lax.dynamic_slice(jax.lax.axis_index(('dp, 'model'))) doesn't work?

@yashk2810  That did work!
I suppose we no longer need the partition id lowering then.
from functools import partial

import jax
import jax.numpy as jnp

from jax.sharding import Mesh, PartitionSpec as P
from jax.experimental.shard_map import shard_map

# Create the mesh with dimensions (4, 2) and named axes ('model', 'data')
mesh = mesh = jax.make_mesh((4,2), ('model', 'data'))

# Create a random number generator and generate a tensor of size (8, 1024, 1024)
key = jax.random.PRNGKey(0)
rng_tensor = jax.random.uniform(key, (8, 1024, 1024))

# Function to apply dynamic slicing in a shard_map
@partial(shard_map, mesh=mesh, in_specs=(P(None, None, None)),
         out_specs=P(('model', 'data'), None, None))
def slice_func(args):
    # Get the mesh axis indices
    axis_index_x = jax.lax.axis_index('model')
    axis_index_y = jax.lax.axis_index('data')

    # Perform a dynamic slice (example slicing)
    # Adjust slice indices and sizes as necessary for your use case
    slice_result = jax.lax.dynamic_slice(
        args, (jax.lax.axis_index(('data', 'model')),0, 0), (1, 1024, 1024)
    )
    return slice_result

# Apply shard_map with the slice_func
jitted = jax.jit(slice_func)
result = jitted(rng_tensor)

print(jitted.lower(rng_tensor).compile().as_text())

HLO
(8, 1024, 1024)
HloModule jit_slice_func, is_scheduled=true, entry_computation_layout={(f32[8,1024,1024]{2,1,0:T(8,128)})->f32[1,1024,1024]{2,1,0:T(8,128)}}, allow_spmd_sharding_propagation_to_parameters={true}, allow_spmd_sharding_propagation_to_output={true}, num_partitions=8

%fused_computation (param_0: f32[8,1024,1024], param_1: s32[]) -> f32[1,1024,1024] {
  %param_0 = f32[8,1024,1024]{2,1,0:T(8,128)} parameter(0)
  %param_1 = s32[]{:T(256)S(6)} parameter(1)
  %constant.26 = s32[]{:T(256)} constant(0)
  ROOT %dynamic-slice.2 = f32[1,1024,1024]{2,1,0:T(8,128)} dynamic-slice(f32[8,1024,1024]{2,1,0:T(8,128)} %param_0, s32[]{:T(256)S(6)} %param_1, s32[]{:T(256)} %constant.26, s32[]{:T(256)} %constant.26), dynamic_slice_sizes={1,1024,1024}, metadata={op_name="jit(slice_func)/jit(main)/jit(shmap_body)/dynamic_slice" source_file="<ipython-input-17-87d788ce8f04>" source_line=26}, backend_config={"flag_configs":[],"scoped_memory_configs":[],"indices_config":{"index_known_bits":[{"zeroes":"0","ones":"0","bitwidth":"32"},{"zeroes":"4294967295","ones":"0","bitwidth":"32"},{"zeroes":"4294967295","ones":"0","bitwidth":"32"}]},"used_scoped_memory_configs":[]}
}

ENTRY %main.29_spmd (param: f32[8,1024,1024]) -> f32[1,1024,1024] {
  %constant.24 = s32[]{:T(256)} constant(8)
  %constant.23 = s32[]{:T(256)} constant(0)
  %constant.22 = s32[]{:T(256)} constant(4)
  %constant.20 = u32[]{:T(256)} constant(3), metadata={op_name="jit(slice_func)/jit(main)/jit(shmap_body)/axis_index" source_file="<ipython-input-17-87d788ce8f04>" source_line=27}
  %constant.19 = u32[]{:T(256)} constant(1), metadata={op_name="jit(slice_func)/jit(main)/jit(shmap_body)/axis_index" source_file="<ipython-input-17-87d788ce8f04>" source_line=27}
  %param = f32[8,1024,1024]{2,1,0:T(8,128)} parameter(0), sharding={replicated}, metadata={op_name="args"}
  %partition-id.3 = u32[]{:T(256)} partition-id(), metadata={op_name="jit(slice_func)/jit(main)/jit(shmap_body)/axis_index" source_file="<ipython-input-17-87d788ce8f04>" source_line=27}
  %and.3 = u32[]{:T(256)S(6)} and(u32[]{:T(256)} %partition-id.3, u32[]{:T(256)} %constant.19), metadata={op_name="jit(slice_func)/jit(main)/jit(shmap_body)/axis_index" source_file="<ipython-input-17-87d788ce8f04>" source_line=27}
  %convert.3 = s32[]{:T(256)S(6)} convert(u32[]{:T(256)S(6)} %and.3), metadata={op_name="jit(slice_func)/jit(main)/jit(shmap_body)/axis_index" source_file="<ipython-input-17-87d788ce8f04>" source_line=27}
  %multiply.1 = s32[]{:T(256)S(6)} multiply(s32[]{:T(256)S(6)} %convert.3, s32[]{:T(256)} %constant.22), metadata={op_name="jit(slice_func)/jit(main)/jit(shmap_body)/mul" source_file="<ipython-input-17-87d788ce8f04>" source_line=27}
  %shift-right-logical.1 = u32[]{:T(256)S(6)} shift-right-logical(u32[]{:T(256)} %partition-id.3, u32[]{:T(256)} %constant.19), metadata={op_name="jit(slice_func)/jit(main)/jit(shmap_body)/axis_index" source_file="<ipython-input-17-87d788ce8f04>" source_line=27}
  %and.2 = u32[]{:T(256)S(6)} and(u32[]{:T(256)S(6)} %shift-right-logical.1, u32[]{:T(256)} %constant.20), metadata={op_name="jit(slice_func)/jit(main)/jit(shmap_body)/axis_index" source_file="<ipython-input-17-87d788ce8f04>" source_line=27}
  %convert.2 = s32[]{:T(256)S(6)} convert(u32[]{:T(256)S(6)} %and.2), metadata={op_name="jit(slice_func)/jit(main)/jit(shmap_body)/axis_index" source_file="<ipython-input-17-87d788ce8f04>" source_line=27}
  %add.2 = s32[]{:T(256)S(6)} add(s32[]{:T(256)S(6)} %convert.2, s32[]{:T(256)S(6)} %multiply.1), metadata={op_name="jit(slice_func)/jit(main)/jit(shmap_body)/add" source_file="<ipython-input-17-87d788ce8f04>" source_line=27}
  %add.3 = s32[]{:T(256)S(6)} add(s32[]{:T(256)S(6)} %add.2, s32[]{:T(256)} %constant.24), metadata={op_name="jit(slice_func)/jit(main)/jit(shmap_body)/add" source_file="<ipython-input-17-87d788ce8f04>" source_line=26}
  %compare.1 = pred[]{:T(1024)S(6)} compare(s32[]{:T(256)S(6)} %add.2, s32[]{:T(256)} %constant.23), direction=LT, metadata={op_name="jit(slice_func)/jit(main)/jit(shmap_body)/lt" source_file="<ipython-input-17-87d788ce8f04>" source_line=26}
  %select.1 = s32[]{:T(256)S(6)} select(pred[]{:T(1024)S(6)} %compare.1, s32[]{:T(256)S(6)} %add.3, s32[]{:T(256)S(6)} %add.2), metadata={op_name="jit(slice_func)/jit(main)/jit(shmap_body)/select_n" source_file="<ipython-input-17-87d788ce8f04>" source_line=26}
  ROOT %fusion = f32[1,1024,1024]{2,1,0:T(8,128)} fusion(f32[8,1024,1024]{2,1,0:T(8,128)} %param, s32[]{:T(256)S(6)} %select.1), kind=kLoop, calls=%fused_computation, metadata={op_name="jit(slice_func)/jit(main)/jit(shmap_body)/dynamic_slice" source_file="<ipython-input-17-87d788ce8f04>" source_line=26}, backend_config={"flag_configs":[],"window_config":{"kernel_window_bounds":[],"output_window_bounds":["1","32","8"],"input_window_bounds":[],"estimated_cycles":"19784","iteration_bounds":["1","4","1"]},"scoped_memory_configs":[],"used_scoped_memory_configs":[{"memory_space":"1","offset":"0","size":"4194304"}],"retry_config":{"retry_count":"0"}}
}


What to do if pyav crashes?

@PaulleDemon TkvideoPlayer works fine with normal video but when a damaged file or non-video file is loaded then the pyav crashes and tkvideoplayer options also freezes and we have to restart the program.
Is there any way to reload or stop the pyav process in that case?
Secondly, I am also facing a playback speed issue with some raw videos (with variable frame rates) ,
I tried changing vid_player.set_resampling_method(1) and consistant_frame_rate=True but still the playback speed is very high with those videos.
How can I fix this?

@PaulleDemon ???

@Akascape Sorry for the late reply, I didn't get notified of this issue, Is there any traceback? If there is then just catch the error using the try-except clause and inform the user that the file is corrupted.
As for the frame rate issue.
Can you try uncommenting this line? https://github.com/PaulleDemon/tkVideoPlayer/blob/dcb81a8d1180a0e489d49414fcb7519a004706b3/tkVideoPlayer/tkvideoplayer.py#L179
Also the self._video_meta should be _video_info. Try and let me know.

Its working with those but normal videos are now slowed.

@Akascape I'll look into this later, let me know if you have any ideas. Thanks

@PaulleDemon I have built my app and everything is working fine but the crash is sometimes critical when a corrupted video is played.
I am getting this traceback:
Exception in thread Thread-4 (_load):
Traceback (most recent call last):
  File "Python\lib\threading.py", line 1009, in _bootstrap_inner
    self.run()
  File "Python\lib\threading.py", line 946, in run
    self._target(*self._args, **self._kwargs)
  File "Python\lib\site-packages\tkVideoPlayer\tkvideoplayer.py", line 106, in _load
    with av.open(path) as self._container:
  File "av\container\core.pyx", line 401, in av.container.core.open
  File "av\container\core.pyx", line 272, in av.container.core.Container.__cinit__
  File "av\container\core.pyx", line 292, in av.container.core.Container.err_check
  File "av\error.pyx", line 336, in av.error.err_check
av.error.InvalidDataError: [Errno<PHONE_NUMBER>] Invalid data found when processing input: 'corrupted_video.mp4';
last error log: [mov,mp4,m4a,3gp,3g2,mj2] moov atom not found

After this error, the video player freezes and stops working. Even a stable video is not playable without a restart.
What to do in that case?

@Akascape Have you tried try-except block? Maybe just try catching the error and stop the video from playing.

@PaulleDemon I used the try-except method like this:
def play_pause():
    """ pauses and plays """
    try:
        if vid_player.is_paused():
            vid_player.play()
            play_pause_btn["text"] = "Pause"
        else:
            vid_player.pause()
            play_pause_btn["text"] = "Play"
    except:
        pass

I tried the try-except method in every block but still getting that error if I load that video directly.
Actually the case is rare and only happens with conversion failed videos(with no data).
But as my program is about data corruption and glitch art so sometimes it may create those type of videos, and I want to block those from interrupting the program. What can I do?

@Akascape sorry for the late reply, I am not being notified of the conversations on time. Can you check out this post on SO: https://stackoverflow.com/questions/11523797/how-to-add-a-moov-atom-in-a-mp4-video-file

I tried -movflags faststart but its not working. I think I have to try different conversion settings and also have to improve my other core files.
Thanks for your help... @PaulleDemon

@Akascape Ok, then can I close the issue? since it's not directly related to the library.


No need to start btcpayserver in specific working directory

This line https://github.com/NixOS/nixpkgs/blob/955079ad77917d544a88fcae9f6ca31738941254/pkgs/applications/blockchains/btcpayserver/default.nix#L26
Shouldn't be need since 1.4.0

Actually addressed on https://github.com/NixOS/nixpkgs/pull/157707/files


About Checkpoints

Hi! I have another question. If I just pip, and then :
class SWIN(nn.Module):
     def __init__(self, num_classes=4):
        super().__init__()
        self.num_classes = num_classes
        # self.pool = nn.MaxPool2d(2, 2)
        self.encoder: SwinTransformerV2 = swin_transformer_v2_t(in_channels=3,
                                                            window_size=8,
                                                            input_resolution=(1024, 1280),
                                                            sequential_self_attention=False,
                                                            use_checkpoint=True)
        self.p=self.encoder.patch_embedding
        self.encoder0 = self.encoder.stages[0]
        ... ...

How to use the checkpoint now? And Is there a pre-trained model for v2_base?
(And when I just run like above, a wired problem arises: 'warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")')

For loading the provided checkpoints you need to initialize the network in the training configuration, load the state dict, and then change the resolution/window size for your need. Here an example for the CIFAR10 checkpoint:
import torch
from swin_transformer_v2 import swin_transformer_v2_b, SwinTransformerV2

swin_transformer: SwinTransformerV2 = swin_transformer_v2_t(input_resolution=(32, 32),
                                                                window_size=8,
                                                                sequential_self_attention=False,
                                                                use_checkpoint=True)
swin_transformer.load_state_dict(torch.load("path_to_weights/cifar10_swin_t_best_model_backbone.pt"))
swin_transformer.update_resolution(new_window_size=8, new_input_resolution=(1024, 1280))

Here an example for the Places365 dataset:
import torch
from swin_transformer_v2 import swin_transformer_v2_b, SwinTransformerV2

swin_transformer: SwinTransformerV2 = swin_transformer_v2_b(input_resolution=(256, 256),
                                                                window_size=8,
                                                                sequential_self_attention=False,
                                                                use_checkpoint=True)
swin_transformer.load_state_dict(torch.load("path_to_weights/places365_swin_b_best_model_backbone.pt"))
swin_transformer.update_resolution(new_window_size=8, new_input_resolution=(1024, 1280))

The CIFAR10 checkpoint is for the tiny model and the Places365 checkpoint is for the base model.
Please note that there are pre-trained weights on ImageNet1k available in the Timm library!


Fix warning about SipHasher being deprecated.

Cherry picked the top commit into PR #102 because this one has a few extra empty commits in it. I'm going to merge #102 instead of this PR.
Thanks for the PR, by the way!


kpt cannot be used in alpine containers

Expected behavior
kpt should be usable in alpine.
Actual behavior
kpt cannot be used in alpine containers.
Information
The issue is due to a transitive dependency on glibc, which is pulled in by wasmtime-go.
For more details, see https://github.com/bytecodealliance/wasmtime-go/issues/21

The cgo dependency is causing more challenges than anticipated, with the release process as well.
One thing we could do is to split out the wasm runner into its own executable binary - so that most people could use a precompiled version (and we'd have one that works on alpine).  OTOH it doesn't really solve the core problem, it just moves it so it only affects people that want to develop on wasm or compile-their-own-from-source.
We're also working on this for the release process, so if we can find a nice approach that makes cgo workable on all platforms that isn't too expensive, then we avoid the need for multiple binaries etc.


WordManagement event issue

I see the events OnAddFieldsToMergeSource and OnCreateHeaderAddFields were added in NAV2018 in #742 , but they are unusable because the WordMergeFile variable is used to add fields, but not available in the event Publishers.
I understand we don't want these .net variables in the events, so possibly a little redesign: first fill a temporary table with the field values you want to add to the mergefile and use this temporary table in the event Publisher, so partners can add there values to this table (TempTable=VAR).
So something like:
[External] AddFieldsToMergeSource(VAR WordMergefile ..........
.....
WITH WordFieldTable DO BEGIN <------ addfields to the temporary table
   AddField(Contact."No.");
   AddField(Contact."Company Name");
   AddField(Contact.Name);
   ...........
   OnAddFieldsToMergeSource(WordFieldTable,Salesperson,Country,Contact,CompanyInfo,SegLine,InteractLogEntry); <---- partners can add their values
   AddField(FaxMailToValue); <------ this value must come last and isn't now!
END;

WordFieldTable.FINDSET(FALSE,FALSE);
REPEAT
   WordMergeFile.AddField(WordFieldTable.Value);  <------ values are added to the WordMergeFile
UNTIL WordFieldTable.NEXT = 0;
WriteLine;

Add an AddField function to fill the WordFieldTable.
Also do this for OnCreateHeaderAddFields. Please make sure the event publishers are on their relative same position in their own functions, because they are not right now.
If adding this new temporary table is unwanted, you might want to consider using an array (Variant?) with a number of dimensions equalling the limit of adding fields to .net variable wordmergefile (there is a limit of around 64 I think). Even better would be to eliminate this limit of 64 fields, but that's a bigger change (for example not using WordMergeFile, but a real file or big text)

well there is a rather easy possibility to switch from HTML table to CSV tables. search for the blogspace using the search engine of your trust.
I did the stuff provided in the blog entry for NAV5 and I do know that it works for NAV2013+; It is an excel limitation, but only when using HTML tables. -.-

About that: https://blogs.msdn.microsoft.com/freddyk/2009/09/20/word-management/


Istio does not support the finalizer for GatewayClass

Bug description

Gateway-API expects Ingress providers to support the following finalizer (gateway-exists-finalizer.networking.x-k8s.io) GatewayClass but Istio does not support it.

https://github.com/kubernetes-sigs/gateway-api/blob/9f1a3a1c1b4ed8e77608031e2d1ee3fa60aadf7f/apis/v1alpha1/gatewayclass_types.go#L157-L159
[X] Networking
Expected behavior

Istio supports the finalizer and block for the deletion of GatewayClass if Gateways still exist.

Steps to reproduce the bug

Deploy Gateway-API by https://istio.io/latest/docs/tasks/traffic-management/ingress/gateway-api/
Check GatewayClass by $ kubectl get gatewayclass -o yaml. No finalizer is added.

Version
$ istioctl version
client version: 1.11-alpha.6591c8112b4fe18728af60d356b42371c8e240b4
control plane version: 1.11-alpha.6591c8112b4fe18728af60d356b42371c8e240b4
data plane version: 1.11-alpha.6591c8112b4fe18728af60d356b42371c8e240b4 (1 proxies)

How was Istio installed?
$ istioctl install -y --set values.gateways.istio-ingressgateway.type=NodePort

Why does it matter? Are you expecting to read this finalizer as proof its been admitted?
I think finalizers are a bit of an anti-pattern and lead to major UX projects, and are certainly not required for our implementation. I believe the finalizer is not required to be added, and only an implementation detail, where if a controller does need a finalizer, they should name it that?

Umm... I see. I was just testing other Gateway API implementations and found a different behavior from another implementation (contour) which uses the finalizer to block unexpected deletion.
https://github.com/projectcontour/contour-operator/blob/b8fe3a30b7e80f2943c0c6ae593f5ce90ea929bc/internal/objects/gatewayclass/finalizer.go#L26
$ kubectl get gatewayclass sample-gatewayclass
NAME                  CONTROLLER                           AGE
sample-gatewayclass   projectcontour.io/contour-operator   2m55s

$ kubectl get gatewayclass sample-gatewayclass -o=jsonpath={.metadata.finalizers}
["gateway-exists-finalizer.networking.x-k8s.io"]

$ kubectl delete gatewayclass --all
gatewayclass.networking.x-k8s.io "sample-gatewayclass" deleted
 ... hang up ...

So, Istio will block the unexpected deletion by a different way or not block it?

It will not block it. I don't think blocking deletion of resources is a good user experience. This may be my personal opinion but I avoid applications that do this. https://github.com/ibuildthecloud/finalizers project description summarizes my feelings :slightly_smiling_face:

Thank you. I understand. I am closing this issue.


Updated Isar Database Tutorial Youtube Link

A new advanced tutorial for Isar database is available using the latest version.
I have update the link to the Isar Database Tutorial from the old Tutorial to the Advanced and Latest Tutorial.
This update will prevent viewers from using the old tutorial which tends to be confusing for them since it's using a very old version of the Isar database.

Looks good, could you please create a matching pull request to the isar-community fork please?


Remove and create workdir if state dir does not exist

This is the case where the work dir could still exist if a machine
reboots, reseting the state dir.  On container creation, we should just
clear out the work dir.
Signed-off-by: Michael Crosby<EMAIL_ADDRESS>

Codecov Report

Merging #2575 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2575   +/-   ##
=======================================
  Coverage   44.59%   44.59%
=======================================
  Files          95       95
  Lines       10007    10007
=======================================
  Hits         4463     4463
  Misses       4822     4822
  Partials      722      722

Flag
Coverage Δ

#linux
48.4% <ø> (ø)
:arrow_up:

#windows
41.54% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 37a6a91...81d50f3. Read the comment docs.