[New Feature]: Allow multiple manifest deletions in single PR

Description of the new feature/enhancement
In the current situation, in order to delete the manifest of a version of a package, a PR needs to be created with the deletion of the appropriate files for that version and if multiple versions should be deleted, a PR has to be raised for each of them separately, since the bot pipeline requires that only one package is modified.
I think that it should be possible to delete multiple versions in one go (e.g. #83292) instead of having to create a separate PR for each version, as this takes more time for everyone involved.
I know that some people would argue that manifests should not be deleted if the source could still be accessed, but I can see different places where this would come handy or necessary. For example, old testing versions that are missing security features, or breaking changes of a backend which are not backward compatible with versions too far behind.
Proposed technical implementation details (optional)
I see two possibilities to deal with that:

Create a type of PR aimed only at version deletions: this would be done by requiring an appropriate string at the end of the PR name to be added (something like [version deletion]), which would trigger the different testing pipeline than the usual to only verify that the whole folder is appropriately deleted (not leaving a partial manifest behind).
Modify the testing pipelines, so that deletions are counted separately from manifest creation/modification: this would mean that the pipeline would first verify that all deletions correspond to full versions, and if this pass accordingly, it would only look at the remaining created/modified files and run in the usual way.

I am aware that each of these might involve quite some work in one way or another, and have both their pros and cons. The main advantage of 1. that I see is that it will save resources by not running the full testing pipeline, but a much simplified version, while option 2. has the advantage of not changing the current workflow of the PRs through the wingetbot and involve "only" a change on the testing pipeline flow.
There are probably other (possibly better) ways to implement that, but I would be happy to see this coming forward, which would allow to unclutter a bit the manifest folder.
Thanks.

Seems like this is similar to https://github.com/microsoft/winget-pkgs/issues/125 but focused solely on deletions

Seems like this is similar to #125 but focused solely on deletions

Yes, you are right. Sorry I didn't see that issue when I searched through the opened ones. Feel free to close this if you think this should be considered as duplicate, or to keep this one open if the suggestion of separation between deletion and creation of manifests on the testing pipeling could be an idea worth of staying separate.


Change VXLAN programming logic

On a fresh start of VXLAN, when we read from metadata, compare with an empty in-memory state and calculate the difference/delta, everything becomes an ADD. We program all the entries and update the in-memory state. On the next metadata update, we use the in-memory state again and the logic goes on.
When needs to be done is to not use any in-memory state, but read the actual entries to build the current state to calculate the delta for programming entries.

Steps to Reproduce:

Change the vxlan docker-compose similar to ipsec one. https://github.com/leodotcloud/test-vxlan-catalog
Restart vxlan-router container, then you can see the error logs.

Performence test between containers on AWS:
$ iperf3 -c <IP_ADDRESS> -P 10
......
......
[SUM]   0.00-10.00  sec   648 MBytes   543 Mbits/sec  2017             sender
[SUM]   0.00-10.00  sec   645 MBytes   541 Mbits/sec                  receiver

Tested with rancher server version-v1.6.6-rc2 and cni driver - rancher/net:v0.11.4.
Automated validation runs reports success except for the known failures relating to timing issues.


Broken math in notebooks after recent updates

Describe the bug
Recent versions of myst-nb (or perhaps myst-parser, not entirely sure) apparently broke math display for markdown cells in notebooks.
To begin with, this is my conf.py:
https://github.com/bluescarni/heyoka.py/blob/main/doc/conf.py
I am including in the docs several notebooks which contain math enclosed either in $ or $$ within markdown cells. See here for one such example:
https://github.com/bluescarni/heyoka.py/blob/main/doc/notebooks/Batch mode overview.ipynb
With older versions of myst-nb (I think <0.16), everything worked as expected. Here is an example of correct HTML for simple inline math:
https://github.com/bluescarni/heyoka.py/blob/4b13b38b0fd016204d1583552beac029553ad96a/notebooks/Batch mode overview.html#L674
When switching to the latest version of myst, the output has now become this:
https://github.com/bluescarni/heyoka.py/blob/gh-pages/notebooks/Batch mode overview.html#L672
The issue seems to be affecting only notebooks - apparently math in plain .rst files is rendered just fine.
I looked into previous reports and it seems strange that nobody reported this yet, so this might as well be some problem in my specific configuration...
Reproduce the bug
Use current versions of myst on notebooks including math in markdown cells.
List your environment
No response

Heya, thanks for the report.
Could you extract out some specific "snippets" from the links you provide and post them here, that would be really helpful, e.g. what the input Markdown is, what the expected HTML is, and what the faulty HTML is

@chrisjsewell thanks for the quick reply.
I actually just managed to fix the issue by adding:
myst_enable_extensions = [
    "amsmath",
    "colon_fence",
    "deflist",
    "dollarmath",
    "html_image",
]

to my conf.py (as shown in one of the myst tutorials).
I was a bit confused by this, as in previous versions apparently this was not necessary (but again, this perhaps was due to something specific to my setup).
If this looks like the correct fix to you, please feel free to close the issue (and perhaps it would be useful to document the need for extensions to be enabled in order to get the correct math output in notebooks).

If this looks like the correct fix to you

Yes indeed, previously the dollarmath extension was enabled by default, but that is no longer the case

If this looks like the correct fix to you

Yes indeed, previously the dollarmath extension was enabled by default, but that is no longer the case

Cheers, I'll be closing the report then.

Yeh no worries, obviously feel free to suggest any improvements to the documentation 😄


merging from source

Proposed changes:

...

Status (please check what you already did):

[ ] added some tests for the functionality
[ ] updated the documentation
[ ] updated the changelog (please check changelog for instructions)
[ ] reformat files using black (please check Readme for instructions)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: sfahad1414:x: Fahad ALi ShaikhFahad ALi Shaikh seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

done by mistake


How to configure passwrod policy in Azure AD

It is said that only password expiration policy can be modified in Azure AD for cloud only user.
Is there any way to modify other policy settings in Azure AD ??
Let's assume we want to define same on-prem AD policy in Azure AD explicitly though it is applicable by default for hybrid user.
Is there any way that we can define in Azure AD as well ?
Azure documentation issue guidance
Thanks for opening an issue in the Azure technical documentation repository.
We use GitHub issues as the primary channel for customer and community feedback about the Azure documentation.
Creating an issue
We prefer that you create documentation feedback issues using the Feedback link on the published article - the feedback control on the doc page creates an issue that contains all the article details so you can focus on the feedback part.
You can also create a feedback issue here in the repo. If you do this, please make sure your issue lists:

[ ] The relevant Azure service or technology.
[ ] A link to the published documentation article that you have feedback about.
[ ] Clear, specific feedback that the author can act on.

Pull requests and article contributions
If you know the change that is needed in an article, we encourage you to submit the changes directly using a pull request. If the change is large, or if you want to contribute an entire article, follow these guidelines:

[ ] Don't surprise us with a big pull request or a pull request with a new article! Submit an issue that describes the details of the proposed large change or new article.
[ ] Include the service or technology area.

We'll route the issue to the appropriate content team for review and discussion.
Tech support and product feedback
If you would like to contact Microsoft about other things, such as product feedback or tech support, please review these guidelines:

If you need technical support using Azure, the paid and free support options are described here: https://azure.microsoft.com/support/options/.

Each article in the Azure technical documentation contains a product feedback button - it's best to submit product feedback directly from a relevant article. Otherwise, you can submit product feedback for most Azure products in the following product feedback forum: https://feedback.azure.com/forums/34192--general-feedback.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b1765376-03c9-829a-9e42-e72cc8a0daea
Version Independent ID: 684c7d7c-09f4-8170-6f7a-132b2d79e1df
Content: Self-service password reset policies - Azure Active Directory
Content Source: articles/active-directory/authentication/concept-sspr-policy.md
Service: active-directory
Sub-service: authentication
GitHub Login: @iainfoulds
Microsoft Alias: iainfou

@AmitavaHazra can you please link the document you are following so that we can better assist?

Please refer doc:
https://docs.microsoft.com/en-us/azure/active-directory/authentication/concept-sspr-policy

Hi @AmitavaHazra ,
It depends on the policy. You can use Password Protection to configure banned password lists in Azure AD, and the on-premises passwords will then be checked against those lists. You can use these for both cloud and on-premises password protection.
https://docs.microsoft.com/en-us/azure/active-directory/authentication/tutorial-configure-custom-password-protection
https://docs.microsoft.com/en-us/azure/active-directory/authentication/concept-password-ban-bad-on-premises#:~:text=Azure AD password protection is,are stored in Azure AD.&text=These checks are performed during password changes and password reset scenarios.
If you use Password Hash Synchronization you can sync password complexity and password expiration policies from on-premises to the cloud. https://docs.microsoft.com/en-us/azure/active-directory/hybrid/how-to-connect-password-hash-synchronization
You can also use ADDS to create and manage policies. https://docs.microsoft.com/en-us/azure/active-directory-domain-services/password-policy

Hope this helps! If you have further questions feel free to ask in Microsoft Q&A as this is more of a product question than a document issue.

@MarileeTurscak-MSFT ,
If you use Password Hash Synchronization you can sync password complexity and password expiration policies from on-premises to the cloud

If I have ADFS authentication method selected in Azure AD connect then will it is applied same ? I mean on prem policy ?
If I understood correctly though we have different set of default policy in Azure AD but for on prem account on prem AD policy will be applied . (in case of ADFS method as well).
That means if we have password expiration 90 days in Azure AD and 180  days in on prem AD then after 90 days also on prem user still can use same password.


[FLINK-26612] Set up Flink Kubernetes Operator Document Website

Based on yunfengzhou-hub's work in https://github.com/apache/flink-ml/commit/1618e6dccf8265fa0622c249d5218f53df8a2d57.
This contains the general tooling for the Flink docs, a quickstart page and a skeleton for future minimal content based on FLINK-26509 and FLINK-26612

+1 LGTM

Great work. Where could I find the website?

@wangyang0918 we still probably need some steps to hook it up to the Flink webpage, but we should first add the bulk of the content I think. We will work on in it this week.

Sounds great.


Change DictAsRecord to have non-surprising case sensitivity; increase…

Change DictAsRecord to have normal case sensitivity since users may now encounter an instance of the class.
Test coverage for all methods.

good spot.


Clients are polling the config server

Not sure exactly what's causing this, but my config server is getting hammered with requests from applications that are running and when I check the application logs I see multiple config fetches per minute, here's one app's log file for example:
2015-10-15 14:39:31.072  INFO 25849 --- [nio-8082-exec-6] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at: http://{config server host}:8888
2015-10-15 14:40:02.067  INFO 25849 --- [nio-8082-exec-8] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at: http://{config server host}:8888
2015-10-15 14:40:11.072  INFO 25849 --- [nio-8082-exec-9] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at: http://{config server host}:8888
2015-10-15 14:40:45.084  INFO 25849 --- [nio-8082-exec-1] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at: http://{config server host}:8888
2015-10-15 14:40:46.442  INFO 25849 --- [nio-8082-exec-5] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at: http://{config server host}:8888
2015-10-15 14:41:17.943  INFO 25849 --- [nio-8082-exec-7] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at: http://{config server host}:8888
2015-10-15 14:41:21.565  INFO 25849 --- [nio-8082-exec-6] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at: http://{config server host}:8888
2015-10-15 14:42:01.791  INFO 25849 --- [nio-8082-exec-8] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at: http://{config server host}:8888
2015-10-15 14:42:12.491  INFO 25849 --- [nio-8082-exec-9] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at: http://{config server host}:8888

Here's what my bootstrap.yml looks like for all apps:
spring:
  application:
    name: {app name here}
  cloud:
    config:
      failFast: true
      uri: http://{config server host}:8888
      username: {removed}
      password: {removed}

I would expect the config clients to only request configs at startup and when /refresh is called. Is this a bug or have I missed some configuration option?
Looking at the code it seems when failFast is set a RetryOperationsInterceptor kicks in, but it should only retry 6 times by default. I am not familiar with how these interceptors work though, so this may or may not be related:
https://github.com/spring-cloud/spring-cloud-config/blob/master/spring-cloud-config-client/src/main/java/org/springframework/cloud/config/client/ConfigServiceBootstrapConfiguration.java#L70

There is nothing that polls from the client side.
If you meet the following conditions, then yes (but only if it can't connect).

If you expect that the config server may occasionally be unavailable when
your app starts, you can ask it to keep trying after a failure. First you need
to set spring.cloud.config.failFast=true, and then you need to add
spring-retry and spring-boot-starter-aop to your classpath.

Forgot to mention I'm using Brixton.BUILD-SNAPSHOT and also using Eureka.
Looks like there's a health check that's running periodically that's causing the problem. In ConfigServerHealthIndicator.doHealthCheck it is calling ConfigServicePropertySourceLocator.locate which pings the config server.
Now I'm just trying to determine what's causing the health check. Does Eureka hit the /health endpoint? I thought clients just sent a heartbeat every 30 seconds.

If eureka.client.healthcheck.enabled=true then it uses the health check do determine the status to send to eureka during the heartbeat. It is not enabled by default.

Found the issue. We have a monitoring app that's calling /health for all the apps periodically. Is there any way to disable the config server part of the health check?

Awesome: health.config.enabled=false.

You rock, thanks for the quick replies!

FYI - looks like all the health.* properties have been renamed to management.health.* per:
http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/html/common-application-properties.html
Will they be renamed here as well?

Potentially, but that's what is is named right now.

The my project configuration server client is displaying a difference to build, because when it is started by STS, the
cloud:
config:
failFast: true
uri: http://config-server:9888
property is visible to the maven of the build, but it is stardet to java -jar project.jar the initiator is the default location: 8888. The mvn spring-boot: run command is ok, but my business structure needs to work with java -jar. Can you help me configure config when started with jjar -jar. Thank you
bootstrap.yml
spring:
application:
name: project-name
cloud:
config:
failFast: true
uri: http://config-server:9888
eureka:
instance:
preferIpAddress: true
leaseExpirationDurationInSeconds: 2
leaseRenewalIntervalInSeconds: 2
client:
registerWithEureka: false
fetchRegistry: false
serviceUrl:
defaultZone: ${EUREKA_URI:http://eureka-server:9761/eureka}

@Swatikp it should be in client, please check below : https://cloud.spring.io/spring-cloud-static/spring-cloud-config/1.3.1.RELEASE/
They mentioned this under tag Health Indicator


Update src/app/chip-zcl-zpro-codec-api.h to SiLabs XML definition

Problem
The src/app/chip-zcl-zpro-codec-api.h and src/app/encoder.cpp code
has been generated using Dotdot XML definition while the examples/ are
now generated using the SiLabs XML definition.
Summary of Changes
The above changes update those files to use Silabs XML definition and
add the used templates to generate those files to src/app/zap-templates.
examples/chip-tool/commands/clusters/Commands.h also needs to be updated
to match the new code gen. The changes are mostly names changes, even if a few
attributes are not reporting anymore. Should not hurt at that point.
This is basically the groundwork to be able to generate more wrappers.

I pushed a small update because I forgot to update some of the argument name. It was working but was a bit off with automatic generation.


Add GDX Snowplow tracker

link to demo deployment of your smk based app is
https://smk-moh-mhsu-rt-b16795-dev.apps.silver.devops.gov.bc.ca
Verify that it looks good, if so merge the pull request which will trigger a prod deployment, otherwise close the pull request without merging
mentions / notifications: @dgboss, @NicoledeGreef

The Vanity URL for the dev deploy is https://smk-moh-mhsu-api-gov-bc-ca.test.api.gov.bc.ca
If this is a dev deploy,  verify that it looks good, then merge the pull request to trigger prod deployment.
If this is a prod deployment, view your prod deploy at the url above :)
mentions / notifications: @dgboss, @NicoledeGreef

Approved, but I would suggest you move the script tag 'up' to the same location as in your other two PRs unless there is a specific reason not to.

For all repos so far I've been putting these just before the closing  tag. My thinking is that this script should call in the page hit after everything else in the head has loaded.


First working translator.

This is the first version of the "AsylumResearch hdf5" to "pyUSID hdf5"
translator. It was tested only on a couple of files, so more testing is
required. Also it is necassary to double-check that Ancillary dataset
(especially Position_Indices) are correct.

If required I can send a test file as well as the ARDF -> ARhdf5 converter that I got from AsylumResearch. However, the mentioned converter works only in windows and they told me that they cannot make it compatible with any other OS.


Installation fails

Dear developers,
I would like to ask if this package can be installed in e.g. Conda environments or if it can be used only on cvmfs.
So far I've tried the following:

activate a Python3 conda environment I have
clone this repo
run python setup.py install from the root of the repo -> installs the egg in site-packages but then import dbs does not find the module
run python setup.py install from the Client directory -> does not seem to automatically install the prerequisites, in particular Curl's static library libcurl.a

Can you please tell me what is the correct workflow for the installation? If it is documented somewhere, can you please point me there?
Thank you,
Massimiliano

Hi Massimiliano,
DBS is a python2 application. It will not work in python3 environment by default. To install DBS you will need the the deployment scripts  from https://github.com/dmwm/deployment.
Thanks,
Yuyi

Hi Yuyi,
thank you for your answer.
Are there any plans to migrate this to Python3 as well?
From what I've seen trying it out on cvmfs it's a really nice package and it would be super helpful as an interface for many CMS Python(3) based analysis frameworks.

Hi Massimiliano,
We have plan to move DBS to py3, but it will take a while.
Thanks,
Yuyi
From: Massimiliano Galli<EMAIL_ADDRESS>Reply-To: dmwm/DBS<EMAIL_ADDRESS>Date: Wednesday, December 9, 2020 at 10:37 AM
To: dmwm/DBS<EMAIL_ADDRESS>Cc: Yuyi Guo<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [dmwm/DBS] Installation fails (#643)
Hi Yuyi,
thank you for your answer.
Are there any plans to migrate this to Python3 as well?
From what I've seen trying it out on cvmfs it's a really nice package and it would be super helpful as an interface for many CMS Python(3) based analysis frameworks.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_dmwm_DBS_issues_643-23issuecomment-2D741892561&d=DwMCaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=8bursUuc0V63OwREQMBG2Q&m=mJMDj6RDxuBIh9E3vVhm2XNIgSVG5yFa3s4aVaUefxk&s=ZKFLakZqYybKxubEroAoEPI15OZvyf8N8on7TDvulpo&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AANROTS2AMM7EW6CJVUNOLDST6RVBANCNFSM4UTFQMYQ&d=DwMCaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=8bursUuc0V63OwREQMBG2Q&m=mJMDj6RDxuBIh9E3vVhm2XNIgSVG5yFa3s4aVaUefxk&s=Bwa_tJYD44b3QcpAK6r3S9TEWvUCVh8HHfWILQo0VW8&e=.


Where is the default directory for HTTP and TFTP?

"Where is the default directory for HTTP and TFTP? I couldn't find an option to set it in the startup command."

Hey there @naishi001. Smee is purpose built to serve just iPXE binaries and scripts. And as such it embeds them into its binary at build time. Nothing is served for TFTP or HTTP from any locations on disk, like a directory.


Get a user's songs from Your Music

Is it possible to get a user's songs from Your Music? In the desktop app, that is normally on the left in the middle under Your Music.

@scotty914 Yes! You can use getMySavedTracks() for fetching the tracks that the user has saved in their "Your Music" section.

Awesome. Thank you!


Language files to JSON?

While reading these messages I started wondering: why not use a JSON file for language strings?
And I'm pretty sure you would ask: why the hell should we?
A reason is that way the editor would be much much easier, no need to sanitize and play with the strings just to avoid breaking php code (and as side-effect we wouldn't even have to worry about potential security issues associated), but the most important effect would be we'd be able to directly use these files in javascript, and we wouldn't have to care about sending out variables via php: just grab the JSON file from the script and you are in! :D
The downside is that a JSON file would be slower than a php one to parse, but we can mitigate that by "hard-caching" (hard caching in the sense we'd make this mandatory) the JSON into a php file, having the best of the two approaches.
Opinions?

Infos:
https://en.wikipedia.org/wiki/JSON
of german
https://de.wikipedia.org/wiki/JavaScript_Object_Notation
Why not getext?
Info:
https://en.wikipedia.org/wiki/Gettext
of german
https://de.wikipedia.org/wiki/GNU_gettext
There give an appropriate editor => Poedit https://poedit.net/. Languages files can thus be processed quickly, and there give a plural system. Joomla and WordPress and other CMS systems employ gettext.

yaml !!!


fixed ShortTagFixer

before this bugfix this
<?php
'<?
';

was "fixed" to:
<?php
'<?php
';

Nice. :+1:

:+1:

Thank you @gharlan.


Cleaning up the public API interface.

Signed-off-by: Hanif Bin Ariffin<EMAIL_ADDRESS>

I think something nice would also be to implement FromStr for the different syntax elements something like this:
impl std::str::FromStr for Syntax {
    type Err = ...;

    fn from_str(ebnf: &str) -> Result<Self, Self::Err> {
        let pair = EBNFParser::parse(Rule::syntax, ebnf)?;
        EBNFParser::parse_syntax(pair)
    }
}

This would result in library users being able to parse EBNF like this:
let ebnf_code = /* some EBNF string... */;
let ebnf = ebnf_code.parse::<iso_14977::Syntax>();


No Depth Camera and Limited IR Camera Streaming

Hi,
I am having an issue with getting the depth camera to work in the stream example. I have not edited the original code in any way. Upon running the example, I can clearly see the RGB camera capture clearly. However I can barely see the IR camera feed (does not appear the same way it does in the Azure Kinect Viewer application, and cannot see the depth camera at all.
To reproduce, navigate to the example-streams folder, and run the example-streams VS solution. Expected is that the live feed would look identical to the live feed from the Azure Kinect Viewer application, but this is not the case.
OF vs Azure Kinect Viewer:

Running:
Windows 10, of_v0.11.0, Visual Studio 2019

Ok, I spent a lot of time on this and realized it is a 16bit color issue. Basically, the IR and Depth images are captured and stored in ofShortPixels for 16bit greyscale. But then they are drawn to 8-bit monitors and essentially show nothing. Would need to add a conversion to get them to show up, and from what I see, that requires a lookup table and likely a shader to do it efficiently? (Way beyond my skill). That is to say that the example is "working," it just is not currently useful.

Any update on this? I'm also having the same problem. I just need the depth image.

Hello, and sorry for the ridiculous delay on this. Like @digitalcoleman said, it looks to me like everything is working fine. You just can't see the depth image very well because I'm sending the raw depth which has a huge range, and whatever is near the camera is very close to black.
I just added an example called example-scaled-depth that shows how you can remap the depth data into a range that is easier to see. I hope this helps!


Update dependencies

Codecov Report
Patch and project coverage have no change.

Comparison is base (468c371) 100.00% compared to head (2d19a74) 100.00%.

:exclamation: Current head 2d19a74 differs from pull request most recent head a578151. Consider uploading reports for the commit a578151 to get more accurate results

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #325   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files          321       321
  Lines        10050     10050
  Branches       969       969
=========================================
  Hits         10050     10050

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Python modules are not linked to Boost::python

When building pyilmbase with cmake, python modules don't link to Boost::python. This gives undefined symbol errors when trying to import them

I tried to build OpenEXR 2.4 on Windows for the first time yesterday and I can confirm this.
LINK : fatal error LNK1104: cannot open file 'boost_python37-vc140-mt-x64-1_70.lib' [C:\Users\lindg\GitHub\vfxplatform_build\build\openexr\PyIlmBase\PyIex\iex_python3.vcxproj]
LINK : fatal error LNK1104: cannot open file 'boost_python37-vc140-mt-x64-1_70.lib' [C:\Users\lindg\GitHub\vfxplatform_build\build\openexr\PyIlmBase\PyImath\imath_python3.vcxproj]

So it affects only PyIex and PyImath. PyImathNumpy builds successfully.
Looking at the CMakeLists.txt for the three modules, I notice that they are fundamentally different and only in PyImathNumpy CMakeLists.txt there's code for linking to Boost::Python.


watch: remove cancelled watchers from resuming

supposed that a watcher is cancelled at the moment that there is a client error.
watchGrpcStream can proceed to handle client error before removing watcher from
its watchGrpcStream. When this case happens, watchGrpcStream resends
watch create request for a cancelled watcher when establishing a new connection.
this pr fixes the issue by removing cancelled watchers from the resuming array where
watchGrpcStream uses it to resume watchers.

can we add a test for this case?

was anything breaking because of this?

@xiang90 I was able to reproduce this issue easily by comment out those lines
case ws := <-w.closingc:
			w.closeSubstream(ws)
			delete(closing, ws)
			if len(w.substreams)+len(w.resuming) == 0 {
				// no more watchers on this stream, shutdown
				return
			}

https://github.com/coreos/etcd/blob/master/clientv3/watch.go#L498-L504.
add a debug statement at
case pbresp := <-w.respc:
			switch {
			case pbresp.Created:
				// response to head of queue creation
				if ws := w.resuming[0]; ws != nil {
					w.addSubstream(pbresp, ws)
					w.dispatchEvent(pbresp)
					w.resuming[0] = nil
				}
				if ws := w.nextResume(); ws != nil {
					wc.Send(ws.initReq.toPB())
				}
                               log.Printf("substreams %+v \n", w.substreams)

https://github.com/coreos/etcd/blob/master/clientv3/watch.go#L448
and at
	case <-ws.initReq.ctx.Done():
           log.Printf("DEBUG: %v cancelled", ws)
	   return

https://github.com/coreos/etcd/blob/master/clientv3/watch.go#L652
and run following
func TestWatchCancelledWatcher(t *testing.T) {
	defer testutil.AfterTest(t)
	clus := integration.NewClusterV3(t, &integration.ClusterConfig{Size: 1})
	defer clus.Terminate(t)
	cli := clus.Client(0)
	watchCli := clientv3.NewWatcher(cli)

	ctx1, cancel1 := context.WithCancel(context.Background())
	watchCli.Watch(ctx1, "foo")
	cancel1()
	for {
		clus.Members[0].Stop(t)
		clus.Members[0].Restart(t)
		time.Sleep(5 * time.Second)
	}
}

I see cancelled watcher is still in the substreams but it shouldn't.

@heyitsanthony nothing is broken from this. I was going over the watch code to understand how it handle reconnect and saw this issue.
Also there is a comment about the exact same issue.
// closed ws will be removed from resuming
https://github.com/coreos/etcd/blob/master/clientv3/watch.go#L724

@xiang90 the issue lies below the api lvl. User won't see anything since cancelled watcher is still cancelled from user's perspective but not a etcd server's perspective.

@fanminshi we can inspect the watcher internal state if we write a in pkg test.

just get etcd_debugging_mvcc_watcher_total and confirm it doesn't spike up on reconnect

another way to test it: pass a custom pb.WatchClient that wraps an ordinary pb.WatchClient into NewWatchFromWatchClient and monitor for Watch calls after the watches are canceled

@heyitsanthony this leaky watcher case happens if case err := <-w.errc: happens before case ws := <-w.closingc:. To trigger the above is tricky. I wasn't able to trigger the case at all unless I comment out case ws := <-w.closingc:.
@heyitsanthony @xiang90 any idea on how to trigger the case without comment out some code?

@heyitsanthony this leaky watcher case happens if case err := <-w.errc: happens before case ws := <-w.closingc:. To trigger the above is tricky. I wasn't able to trigger the case at all unless I comment out case ws := <-w.closingc:.

write a test. try to spin up N routines. racy might just happen. if the racy rate is very low, then we can put more effort to trigger it.

I took a look of the code again. The current watch code handles resending cancelled watcher create request with following logic. It appears to be correct to me.

suppose watcher A is cancelled and notified watchGrpcStream through w.closingc <- ws

watchGrpcStream processes client error  case err := <-w.errc: before w.closingc <- ws

watcher A is appended to w.resuming = append(w.resuming, ws) since it was not removed from w.substreams yet.

watcher A created request is sent wc.Send(ws.initReq.toPB())

case err := <-w.errc: finished.

At here two likely cases can happen.
case: watchGrpcStream removes watcher A w.closingc <- ws before watchGrpcStream
hears watcher A created resp case pbresp := <-w.respc:

watchGrpcStream removes watcher A from w.resuming  and w.substreams in w.closeSubstream(ws).

watchGrpcStream hears watcher A created resp case pbresp := <-w.respc: and found watcher A is not in w.resuming[0]. Then it ignores this created req.

If etcd server sends a another watch event back for watcher A, then watchGrpcStream sends a cancel request cr := &pb.WatchRequest_CancelRequest{ since watcher A is not in w.substreams.

If step 9 doesn't happen where etcd server never sends another watch event back, then watcher A will never be cancelled. Is that an expected behavior?
case: watchGrpcStream hears watcher A created resp case pbresp := <-w.respc: before watchGrpcStream removes watcher A w.closingc <- ws

watchGrpcStream adds watcher A to substream.

w.closingc <- ws triggers. watchGrpcStream removes watcher A from w.resuming  and w.substreams in w.closeSubstream(ws).

If etcd server sends a another watch event back for watcher A, then watchGrpcStream sends a cancel request cr := &pb.WatchRequest_CancelRequest{ since watcher A is not in w.substreams.

If I create 11 watchers and cancell 10, should etcd server also remove 10 watchers internally?
The following test shows that even I cancell 10 watchers from client, the etcd server still shows that 11 watchers are available. Is this expected?
func TestWatchCancelledWatcherPropergateToServer(t *testing.T) {
	defer testutil.AfterTest(t)
	clus := integration.NewClusterV3(t, &integration.ClusterConfig{Size: 1})
	defer clus.Terminate(t)
	cli := clus.Client(0)

	cancelledWatchers := 10

	ctx1, cancel1 := context.WithCancel(context.Background())
	defer cancel1()
	cli.Watch(ctx1, "foo")

	for i := 0; i < cancelledWatchers; i++ {
		ctx, cancel := context.WithCancel(context.Background())
		cli.Watch(ctx, "foo")
		cancel()
	}
	time.Sleep(time.Second)
	totalWatches, err := clus.Members[0].Metric("etcd_debugging_mvcc_watcher_total")
	if err != nil {
		t.Fatal(err)
	}
	var totalWatchesInt int
	_, serr := fmt.Sscanf(totalWatches, "%d", &totalWatchesInt)
	if serr != nil {
		t.Fatal(serr)
	}
	log.Printf("total watchers %v ", totalWatchesInt)
	if totalWatchesInt > cancelledWatchers {
		t.Fatalf("too many watchers")
	}
}

output:
...
2017-05-17 11:10:56.825876 I | total watchers 11
--- FAIL: TestWatchCancelledWatcherPropergateToServerSide (1.14s)
	watch_test.go:1085: too many watchers

@fanminshi yes, the cancel is lazy https://github.com/coreos/etcd/blob/master/clientv3/watch.go#L659

@heyitsanthony I wonder what's advantage of a lazy cancel over a direct cancel?

@fanminshi saves network traffic, simplifies the teardown path


Not working on Linux/Mono

On linux with mono only minimizes the windows.
Envoriment:
Archlinux
keepass - 2.35
mono - <IP_ADDRESS>5

I'm not sure I understand. Are you saying that when you click on the menu item to run the HIBP check it minimizes KeePass and doesn't do anything else?

Yes, nothing happens, the plugins starts checking entries, when finish only minimizes the main window.
With the mono upgrade (<IP_ADDRESS>5) HIBP kinda works! But sets almost all entries as expired, even entries not related to any site, like entries with only information or local accounts.
Cloudbleed doesn't work either, only minimizes the main window after checking entries, with no entries marked as expired.

I see the minimising bug on Windows as well. I think on mono the messagebox isn't appearing when it detects a breach, so it looks like nothing is happening.
I'll add a separate issue for creating a results dialog which should hopefully resolve that.
As for expiring entries with no URLs etc, that is odd as these should be ignored. I'll dig a little deeper into that one.

Try 0.2.1 to see if this improves the minimise behaviour. The problem may be that if you click on other windows while the progress dialog is shown, KeePass waits until it is finished before relinquishing focus.

I've tried not clicking anywhere, main window open.
I'll try that version as soon as possible and maybe I'll create an Archlinux package if there's no major issues.

HIBP now works properly also the new download windows shows the download progress.
Cloudbleed not even downloads data (sorted_unique_cf.txt). I see no traffic when i use "Cloudbleed Checker"

The timeout on the httpclient may be too low to download the cloudbleed data file. I'll increase the timeout which should fix the problem.

I've increased the timeout in <IP_ADDRESS> to 30 minutes (from 10).

Cloudbleed downloads 71MB (from <IP_ADDRESS>) but the "Downloading Cloudbleed datafile" remains after download with no progress. And no entries are checked.

The lack of progress relates to #3, however it shouldn't just hang at that point.
Has it actually downloaded the file anywhere? Does %PROGRAMDATA% map to a sensible location on Linux when using mono?

Honestly i don't know, according to this page System.Environment.SpecialFolder.CommonApplicationData points to /usr/share
Can't be write by user, keepass stores his cache in $HOME/.local/share/KeePass/ (System.Environment.SpecialFolder.LocalApplicationData)
I've created a log see if it helps
keepass.log.gz

Try <IP_ADDRESS>. I've changed the download location for mono to LocalApplicationData, so this should get past the hanging issue.

Now its working, both checkers working with "Breached Entries" window.

Great. I'll close this issue then.


test_tensorrt_resnet18.test_tensorrt_resnet18_feature_vect numerical error on T4 gpu

Description
test_tensorrt_resnet18.test_tensorrt_resnet18_feature_vect succeeded on V100 gpu but got numerical issue on T4 gpu.
Environment info (Required)
Sorry I have to hide the CPU information since I am using a machine under NDA policy.
root@b0a2b9b22fac:/work/mxnet# python3 diagnose.py
----------Python Info----------
Version      : 3.6.7
Compiler     : GCC 8.2.0
Build        : ('default', 'Oct 22 2018 11:32:17')
Arch         : ('64bit', 'ELF')
------------Pip Info-----------
Version      : 19.1.1
Directory    : /usr/local/lib/python3.6/dist-packages/pip
----------MXNet Info-----------
Version      : 1.5.0
Directory    : /usr/local/lib/python3.6/dist-packages/mxnet
Hashtag not found. Not installed from pre-built package.
----------System Info----------
Platform     : Linux-4.14.106-79.86.xxxxxxx-x86_64-with-Ubuntu-18.04-bionic
system       : Linux
node         : b0a2b9b22fac
release      : 4.14.106-79.86.xxxxxxx.x86_64
version      : #1 SMP Tue Mar 19 00:48:07 UTC 2019
----------Hardware Info----------
machine      : x86_64
processor    : x86_64
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Vendor ID:           GenuineIntel
Model name:          Intel(R) Xeon(R) xxxxx CPU @ xxxxGHz
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0016 sec, LOAD: 0.4355 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.1399 sec, LOAD: 0.1254 sec.
Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.2835 sec, LOAD: 0.5443 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0170 sec, LOAD: 0.1960 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0025 sec, LOAD: 0.0719 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0038 sec, LOAD: 0.0386 sec.

Package used (Python/R/Scala/Julia):
I'm using python api.
Build info (Required if built from source)
Compiler (gcc/clang/mingw/visual studio):
MXNet commit hash:
5fc4fc53df74f276aafa51208142e657e9cfe42d
Build config:
built with
./ci/build.py -p ubuntu_gpu_tensorrt
Error Message:
[00:56:21] /work/mxnet/src/operator/subgraph/build_subgraph.cc:686: start to execute partition graph.
[00:56:21] /work/mxnet/src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (set the environment variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
F
======================================================================
FAIL: trt.test_tensorrt_resnet18_feature_vect
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/work/mxnet/trt.py", line 67, in test_tensorrt_resnet18_feature_vect
    assert_almost_equal(no_trt_output, trt_output, 1e-1, 1e-2)
  File "/usr/local/lib/python3.6/dist-packages/mxnet/test_utils.py", line 503, in assert_almost_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal:
Error 1.155339 exceeds tolerance rtol=0.100000, atol=0.010000.  Location of maximum error:(939,), a=0.006451, b=0.020357
 a: array([ 0.36684752,  2.8859496 , -0.5449833 , ..., -0.77340114,
        2.9310114 ,  1.8106201 ], dtype=float32)
 b: array([ 0.3561221 ,  2.8805661 , -0.55245507, ..., -0.77211773,
        2.9371974 ,  1.8207407 ], dtype=float32)
-------------------- >> begin captured stdout << ---------------------
downloading sample input
Downloading /root/.mxnet/models/resnet18_v2-a81db45f.zip from https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/models/resnet18_v2-a81db45f.zip...

--------------------- >> end captured stdout << ----------------------
-------------------- >> begin captured logging << --------------------
urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): github.com
urllib3.connectionpool: DEBUG: https://github.com:443 "GET /dmlc/web-data/blob/master/mxnet/doc/tutorials/python/predict_image/cat.jpg?raw=true HTTP/1.1" 302 None
urllib3.connectionpool: DEBUG: https://github.com:443 "GET /dmlc/web-data/raw/master/mxnet/doc/tutorials/python/predict_image/cat.jpg HTTP/1.1" 302 169
urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): raw.githubusercontent.com
urllib3.connectionpool: DEBUG: https://raw.githubusercontent.com:443 "GET /dmlc/web-data/master/mxnet/doc/tutorials/python/predict_image/cat.jpg HTTP/1.1" 200 227791
root: INFO: downloaded https://github.com/dmlc/web-data/blob/master/mxnet/doc/tutorials/python/predict_image/cat.jpg?raw=true into cat.jpg successfully
root: INFO: Model file not found. Downloading to /root/.mxnet/models/resnet18_v2-a81db45f.params.
urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): apache-mxnet.s3-accelerate.dualstack.amazonaws.com
urllib3.connectionpool: DEBUG: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com:443 "GET /gluon/models/resnet18_v2-a81db45f.zip HTTP/1.1"<PHONE_NUMBER>7
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 1 test in 69.297s

FAILED (failures=1)

Minimum reproducible example
https://github.com/apache/incubator-mxnet/blob/master/tests/python/tensorrt/test_resnet18.py
Steps to reproduce

pull from https://cloud.docker.com/u/haohuanw/repository/docker/haohuanw/trtdebug-turing
python3.6 test/python/tensorrt/test_resnet18.py

What have you tried to solve it?
This seems happened on particular hardware (passed on V100 but failed on T4), so nothing I can really do.

Failed here... just on CI... on what should be unrelated...
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/PR-15885/14/pipeline/303

Also here:
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/PR-16827/5/pipeline
Seems like a flaky one with a borderline tolerance value.

Still failing: http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/PR-17021/1/pipeline

Another one : http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/PR-18119/3/pipeline

@haohuanw our CI as of 4/21 doesn't really use g4 instances. T4 GPU [Tesla] is used in G4 instances while p3 & g3 instances [currently used for GPU workloads in CI] use Tesla V100 and M60 respectively. So since our CI is failing for this test since June 2019, it looks like an issue related to V and not T4
Correct me if I'm wrong @leezu @josephevans


setup-maas.yml fails on hosts that are not part of lxc_hosts group

Do NOT include any confidential information, this issue will be created on a public repo
This section is completed by the issue reporter
On which version(s) do you encounter this issue?
Please provide the output to 'git describe --all' for  all deployments where you have confirmed it occurs.
Encountered on r14.0.0rc1, but will affect master also.
What type of issue is this? Please tick the appropriate box.

[x] bug - existing functionality that does not work for you as documented
[ ] feature - new functionality that you would like to see added

What is the expected behavior?
setup-maas.yml should complete successfully
What is actually happening?
When deploying the openstack-ansible-ops multi-node-aio, setup-maas.yml fails with the following error:
TASK [rpc_maas : Install pip packages to venv] *********************************
...
failed: [loadbalancer1] (item=lxc-python2) => {"cmd": "/openstack/venvs/maas-r13.1.0rc1/bin/pip install lxc-python2", "failed": true, "item": "lxc-python2", "msg": "stdout: Collecting lxc-python2\n  Using cached lxc-python2-0.1.zip\nBuilding wheels for collected packages: lxc-python2\n  Running setup.py bdist_wheel for lxc-python2: started\n  Running setup.py bdist_wheel for lxc-python2: finished with status 'error'\n  Complete output from command /openstack/venvs/maas-r13.1.0rc1/bin/python -u -c \"import setuptools, tokenize;__file__='/tmp/pip-build-8XFDoG/lxc-python2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\\r\\n', '\\n');f.close();exec(compile(code, __file__, 'exec'))\" bdist_wheel -d /tmp/tmpA3PTu6pip-wheel- --python-tag cp27:\n  running bdist_wheel\n  running build\n  running build_py\n  creating build\n  creating build/lib.linux-x86_64-2.7\n  creating build/lib.linux-x86_64-2.7/lxc\n  copying lxc/__init__.py -> build/lib.linux-x86_64-2.7/lxc\n  running build_ext\n  building '_lxc' extension\n  creating build/temp.linux-x86_64-2.7\n  x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c lxc.c -o build/temp.linux-x86_64-2.7/lxc.o\n  lxc.c:27:30: fatal error: lxc/lxccontainer.h: No such file or directory\n   #include <lxc/lxccontainer.h>\n                                ^\n  compilation terminated.\n  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1\n  \n  ----------------------------------------\n  Running setup.py clean for lxc-python2\nFailed to build lxc-python2\nInstalling collected packages: lxc-python2\n  Running setup.py install for lxc-python2: started\n    Running setup.py install for lxc-python2: finished with status 'error'\n    Complete output from command /openstack/venvs/maas-r13.1.0rc1/bin/python -u -c \"import setuptools, tokenize;__file__='/tmp/pip-build-8XFDoG/lxc-python2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\\r\\n', '\\n');f.close();exec(compile(code, __file__, 'exec'))\" install --record /tmp/pip-_tl8qx-record/install-record.txt --single-version-externally-managed --compile --install-headers /openstack/venvs/maas-r13.1.0rc1/include/site/python2.7/lxc-python2:\n    running install\n    running build\n    running build_py\n    creating build\n    creating build/lib.linux-x86_64-2.7\n    creating build/lib.linux-x86_64-2.7/lxc\n    copying lxc/__init__.py -> build/lib.linux-x86_64-2.7/lxc\n    running build_ext\n    building '_lxc' extension\n    creating build/temp.linux-x86_64-2.7\n    x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c lxc.c -o build/temp.linux-x86_64-2.7/lxc.o\n    lxc.c:27:30: fatal error: lxc/lxccontainer.h: No such file or directory\n     #include <lxc/lxccontainer.h>\n                                  ^\n    compilation terminated.\n    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1\n    \n    ----------------------------------------\n\n:stderr:   Failed building wheel for lxc-python2\nCommand \"/openstack/venvs/maas-r13.1.0rc1/bin/python -u -c \"import setuptools, tokenize;__file__='/tmp/pip-build-8XFDoG/lxc-python2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\\r\\n', '\\n');f.close();exec(compile(code, __file__, 'exec'))\" install --record /tmp/pip-_tl8qx-record/install-record.txt --single-version-externally-managed --compile --install-headers /openstack/venvs/maas-r13.1.0rc1/include/site/python2.7/lxc-python2\" failed with error code 1 in /tmp/pip-build-8XFDoG/lxc-python2/\n"}
...

In this scenario, the problem is that the host loadbalancer1 has no containers on it (only runs haproxy) and is therefore not added to the lxc_hosts group (see recent change https://review.openstack.org/#/c/427855/).  Previously, all hosts were part of lxc_hosts.  While in this situation it's loadbalancer1 failing, this is likely to be a problem if you have ceph OSD storage nodes, swift storage nodes, etc. that do not have any containers on them.
What were you attempting to do when you encountered this error (give as much detail as possible)?
Running setup-maas.yml on the multi-node-aio
How can we reproduce this issue?
Run the multi-node-aio from openstack-ansible-ops.  Alternatively, the problem should also be visible when attempting to deploy ceph OSDs onto bare metal (as support would).
How can we verify your issue is addressed?
setup-maas.yml runs successfully even when there are hosts present which are not part of lxc_hosts group.
If you are using or know of a workaround please describe the necessary steps?
None at this time.
What is your evaluation on the importance of this issue, and why?
High.

This section is completed during the issue resolution process. If you are reporting an issue, please do not modify the section below.
Which branches are impacted by this issue?
Remove branches that are not impacted.
For each pull request, update the branch line to reference it.

[ ] master
[ ] newton-14.0
[ ] mitaka-13.1
[ ] liberty-12.2

Upgrade impact. Consider the impact on moving between major, minor and patch releases.

[ ] This change does not impact upgrades
[ ] This change addresses a potential upgrade impact
[ ] The upgrade impact is being tracked by another issue.

The issue tracking the upgrade impact is:
Testing impact

[ ] An issue/pull-request has been create or exists to track the testing gap, and is referenced below
[ ] This change includes additional testing
[ ] This change is tested by the existing setup

Additional testing links:
Release note

[ ] This change includes a release note.
[ ] This change does not require a release note.

Doc Impact
Does your change require documentation?

[ ] Yes, and I submitted an issue to Docs issues
[ ] Yes, and I contributed a documentation patch to docs-rpc
[ ] No documentation required for this change

Additional documentation links (please add links if you have ticked yes):

I think all the MaaS stuff is targeted at hosts. Searching for lxc_hosts comes up empty. The setup-maas.yml playbook targets hosts and all_containers.
That said, hosts that don't have lxc containers (compute, Ceph, Swift) probably shouldn't have lxc checks installed.

@nrb, assuming QE hits this in the happy path rc1 testing in a lab, what's the way to fix this in Newton? I see a backport to Ocata upstream.
Thanks!

https://review.openstack.org/#/c/427885/ is the newton backport. As I stated in the previous comment, I don't think lxc_hosts is actually being used by RPC-O, though. It's not in code search, and the setup-maas.yml playbook targets hosts:all_containers on the r14.0.0rc1 tag.
Ideally, the correct thing is 1) ensure RPC-O has that Newton backport so that it has the correct lxc_hosts membership 2) Only install lxc checks and Python packages on members of lxc_hosts.

@nrb Awesome. Thanks for the guidance on this.

@jacobwagner @Rydor this is the issue to see investigate as part of the MaaS testing for rc1.

@xeregin I added you to this card to provide feedback when you do the Maas deploy if you run into any similar issues.

I did not see this issue when deploying an AIO on Xenial.
Actual QE validation to be done in LEM labs.

I'm not seeing this issue in IAD3-2 Trusty

@xeregin What nodes?

@nrb I can answer that, yes, compute, infra and storage

Update: This is an issue of a couple different forms. This issue is confirmed.

Should we change the milestone on this to rc2?

@claco Done.

@cloader89 Updates please.

And move to doing. :-P

We are in the process of gathering ideas on how to fix it. So far Bjoern and I agreed on adding LXC to all hosts. We are waiting for @npawelek to give his opinion. I am going to create a patch today with adding lxc to all of the hosts. That way @xeregin can re-test it.

Adding LXC to all hosts will extend the build time and maintenance for all hosts, meaning that every upgrade takes much, much longer. The whole reason why the lxc_hosts group was created was to reduce the time taken for deployments and upgrades.
The solution here should be to scope the deployments better. This role is targeting all hosts, then using conditionals to scope where tasks are applied. This is very bad practise and causes the deployment and upgrade processes to take unnecessarily long.
I would suggest either breaking these tasks out into purpose-specific roles, or playbooks, or using dynamic groups (and/or group_vars) to target specific portions of the role to specific hosts.

@odyssey4me You may want to drop this into the PR so it's not least, or rather, read there.
Adding LXC to all hosts will extend the build time and maintenance for all hosts, meaning that every upgrade takes much, much longer. The whole reason why the lxc_hosts group was created was to reduce the time taken for deployments and upgrades.

The proposed PR is adding one python package to the rpc_mass required packages list. Doesn't seem we're moving things in or our of groups at all, so there wouldn't be any change in terms of all of the lxc_hosts work being done, or causing more time, unless I'm thinking sideways on that.
I think the PR might need a better title and commit message. I'm not sure this issue as it stands post QE testing is the the same issue we thought it was when first reported.

@claco This python package won't install unless the accompanying LXC packages are installed on the same host. This is a very slippery slope.

Put up a PR that splits the process_check into host and container plugins.  Seemed to work in my lab so could be an option instead of installing LXC:
https://github.com/rcbops/rpc-openstack/pull/2058

Thanks @antonym.I'll take a look at that PR.
It sounds like the other PR to install the lxc-python2 package on all hosts is a no-go, leaving PR #2058 as our path moving forward in the short-term.
I am, however, intrigued by @odyssey4me suggestion he made in an earlier comment:

The solution here should be to scope the deployments better. This role is targeting all hosts, then using conditionals to scope where tasks are applied. This is very bad practise and causes the deployment and upgrade processes to take unnecessarily long.

I have noticed that setup-maas.yml takes a  horrible amount of time to complete. I'll create an issue for this and backlog it for the our longer-term solution.


NoMethodError: undefined method `validate_and_consume_otp!'

I'm trying to setup this but I think it isnt being initializated.
I followed everything in this tutorial, what am I missing?
https://github.com/mikamai/rails-2fact-auth-example
I have:

Bundled gem 'devise-two-factor' and gem 'rqrcode_png'.
Updated the database User structure with the 5 new fields.
Defined the ENV['VARIABLE']
Updated User model accordingly

class User < ActiveRecord::Base
  devise :rememberable, :trackable, :lockable,
         :session_limitable, :two_factor_authenticatable,
         :otp_secret_encryption_key => ENV['VARIABLE']
  # ...
end

Created the views
Updated devise views
Updated routes
Whitelisted application params

def configure_permitted_parameters
  devise_parameter_sanitizer.for(:sign_in) << :otp_attempt
end

Updated again User.rb model

def activate_two_factor params
   otp_params = { otp_secret: unconfirmed_otp_secret }
   if !valid_password?(params[:password])
     errors.add :password, :invalid
     false
   elsif !validate_and_consume_otp!(params[:otp_attempt], otp_params)
     errors.add :otp_attempt, :invalid
     false
   else
     activate_two_factor!
   end
 end

 def deactivate_two_factor params
   if !valid_password?(params[:password])
     errors.add :password, :invalid
     false
   else
     self.otp_required_for_login = false
     self.otp_secret = nil
     save
   end
 end

private

 def activate_two_factor!
   self.otp_required_for_login = true
   self.otp_secret = current_user.unconfirmed_otp_secret
   self.unconfirmed_otp_secret = nil
   save
 end

Created the two_factors_controller.rb

class TwoFactorsController < ApplicationController
  before_filter :authenticate_user!

  def new
  end

  # If user has already enabled the two-factor auth, we generate a
  #   temp. otp_secret and show the 'new' template.
  # Otherwise we show the 'show' template which will allow the user to disable
  #   the two-factor auth
  def show
    unless current_user.otp_required_for_login?
      current_user.unconfirmed_otp_secret = User.generate_otp_secret
      current_user.save!
      @qr = RQRCode::QRCode.new(two_factor_otp_url).to_img.resize(240, 240).to_data_url
      render 'new'
    end
  end

  # User#activate_two_factor will return a boolean. When false is returned
  #   we presume the model has some errors.
  def create
    permitted_params = params.require(:user).permit :password, :otp_attempt
    if current_user.activate_two_factor permitted_params
      redirect_to root_path, notice: "You have enabled Two Factor Auth"
    else
      render 'new'
    end
  end

  # If the provided password is correct, two-factor is disabled
  def destroy
    permitted_params = params.require(:user).permit :password
    if current_user.deactivate_two_factor permitted_params
      redirect_to root_path, notice: "You have disabled Two Factor Auth"
    else
      render 'show'
    end
  end

  private

  # The url needed to generate the QRCode so it can be acquired by
  #   Google Authenticator
  def two_factor_otp_url
    "otpauth://totp/%{app_id}?secret=%{secret}&issuer=%{app}" % {
      :secret => current_user.unconfirmed_otp_secret,
      :app    => "",
      :app_id => "#{current_user.username}"
    }
  end
end

Everything seems to be working, but when I go to /two_factor and click the button I get error from User model.

help?

Did you get this solved?

It looks like this got moved to #82?


make delayed_job exceptions work (issue 291)

@ri4a thanks for this, I guess this fix https://github.com/rollbar/rollbar-gem/issues/94#issuecomment-138386657 ?
I'll test this and merge if everything is ok. I think we'd do better using Rollbar::JSON.dump interface, but the fix looks reasonable.
Thanks!

@ri4a sorry, this PR doesn't fix https://github.com/rollbar/rollbar-gem/issues/94.
I'm looking at this, thanks again for the PR.

@ri4a
I've looked at your comments in https://github.com/rollbar/rollbar-gem/issues/291#issuecomment-138730134.
We removed this part of code:
is_symbol = object.is_a?(Symbol)

return object unless object == object.to_s || is_symbol

But we are doing this in lib/rollbar/encoding.rb:
def self.encode(object)
    can_be_encoded = object.is_a?(String) || object.is_a?(Symbol)

    return object unless can_be_encoded

    encoding_class.new(object).encode
end

I'd say it's basically the same, maybe I'm missing something. The goal there is only encode String  and Symbol objects.
Having return object unless object == object.to_s is just a way to check it is a String object.
Are you sure the problem is here? It'd be nice having a test that exploits the current bug.
Thanks.

@jondeandres
Yes, you are right. I looked at Rollbar::Encoding::Encoder#encode instead of Rollbar::Encoding::encode :unamused:
The real cause is in /lib/rollbar/truncation/mixin.rb, which changed from
def dump(payload)
  MultiJson.dump(payload)
end

to
def dump(payload)
  Rollbar::JSON.dump(payload)
end

Apparently the serialization of Ruby objects is different (seems that now only to_s is called).
To create a test that exploits the bug, it depends on what you consider to be the actual bug here. The fact that a method called "enforce_valid_utf8" is called with unserialized Ruby objects seems strange, since the serialization afterwards could cause new encoding issues. Should Rollbar::Encoding::encode really accept (i.e. ignore) Ruby objects and trust they will be serialized correctly later by Truncation#dump? Or should it be the responsibility of the data injectors to first serialize? Or maybe "enforce_valid_utf8" should just be called later.
This PR fixes #291 by serializing the Job object where it is injected. It could also be fixed by amending Rollbar::JSON. That may or may not also fix #94, haven't looked at that.

@ri4a that has more sense. the enforce_valid_ut8 method is not the problem here. I agree with you that maybe we should enforce utf8 after dump the payload. The thing here is that we had some problems with some srings when an async handler pushed to its queue, so we enforce utf8 before.
Apart of this, and focused in the bug, we should probably pass request: job.payload_object.as_json from the beginning and not a delayed job Job instance.
I've opened a PR for this, https://github.com/rollbar/rollbar-gem/pull/293.
I think we can close this one, but anyway, thanks for open it!


Add VHT cellphone number lookup

As part of the UET results, we want the user to be add their location during registration, either through finding their VHT through cellphone number, or by searching their parish name.
This PR deals with finding their VHT by using a cellphone number.

@KaitCrawford Ready for review.

:+1:


OSD-11521 Create manifests to deploy to a ROSA cluster

With the assumption of:
operator_namespace: openshift-aws-vpce-operator
operator_serviceaccount: aws-vpce-operator
operator_sts_secret_name: avo-aws-creds

Updated the sts-oidc toy example and then generated manifests that let this run in ROSA stage in commercial (create, modify, and delete a VPC endpoint and associated AWS resources)! Tested with quay.io/app-sre/aws-vpce-operator:v0.1.43-2f72ef2

Codecov Report

Merging #18 (c21ba06) into main (2f72ef2) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #18   +/-   ##
=======================================
  Coverage   39.50%   39.50%
=======================================
  Files          17       17
  Lines         772      772
=======================================
  Hits          305      305
  Misses        435      435
  Partials       32       32


multipart-fix

Fixes #3175

Hi, can you write tests for the same. Thanks

Awesome, thanks! 🍰 🚀

Super duper happy 😁

Awesome, thanks! 🍰 🚀


Tabula question

Hi,
I'm trying to unlock a 380-ish page PDF, and I'm noticing records that break the line are entirely dropped on Tabula's import. As a result, I'm missing several hundred records.
For example,

(I'm not sure if you can see the photo attached, but it shows the red Tabula capture crossing a page break).
But in the export, the record that was sourced from USA TODAY (11 June 2016 , Rif Dimashq, "USA TODAY...) is completely missing.

Any work arounds for this?
Thanks!
Jodi

FWIW, I have tried getting the red Tabula capture boxes to butt up against each other. It makes no difference.

Yep, that's a known issue, unfortunately. The fact that there's no ruling line on the bottom of the cell (on that page) means Tabula's Lattice mode won't recognize it as a cell at all.
The only workaround I can suggest is using Stream mode to extract the data in those rows. It'll be a pain in the butt to "roll up" all of the text that belongs to the same row (since Stream mode will treat each line of text as its own row), but it'll actually extract the data. It's not a great workaround, I know.
(We've talked somewhere about "pretending" that the user's selection is bounding lines for this purpose... but that might introduce other bugs and complexities. I remember talking about it but can't find the discussion to link to this one.)
This is a similar issue to #578. (Which I'm noting mostly for my own organization purposes.)


feat: a fan is limit iff it is after the application of coyoneda.obj X

Completely analogous to #15157.

This is a large import, but it brings the imports in line with CategoryTheory.Limits.Preserves.Shapes.Pullbacks, so I wouldn't consider this importing too much. In general the import structure and also the placement of results in this area of the library is rather chaotic (which isn't too surprising since it's quite old). There are files like Limits.Opposites which bind all of the special shapes together quite early in the import structure. I'll look into splitting Limits.Opposites and perhaps other files so that the special shapes can be developed for longer without knowing about each other. Not in this PR though.


Added Rosencrantz

Added Rosencrantz

Looks awesome. But you will need to rename rosen to rosencratz to satisfy Nimble's package guidelines. (or you can rename the package name to rosen instead if you want).

See here for details about the guidelines/conventions: https://github.com/nim-lang/nimble#libraries
These will become enforced by Nimble in a future update.

Fixed! For some reason, I believed that having both a file rosencrantz.nim and modules in the rosencrantz directory would lead to conflicts


Weird artifacts on the crafting and creative tabs

With the new update, the tabs in both the creative and crafting screens seems to have artifacts around them:

Change Interface Resolution

#110


add noop, and rename package

rename packages to be io.opentracing
add noop Tracer, Span, and SpanBuilder

LGTM
@bensigelman @michaelsembwever

Sorry, I thought I saw this get merged but was apparently incorrect!
@michaelsembwever I assume this is uncontroversial...

@michaelsembwever good points all around.
@oibe let's address them.


Local adguard domain interception might misbehave in an IPv6-only network

We should check it.

core/pull-requests/766


Filter for multiple CAN frame ids/masks

Hey,
Im not sure about the SJA1000 but the MCP2515 supports multiple filters, however the api of this library does not support multiple filters.
Maybe a macro constant like MAX_FILTERS and a function multiFilter(int *ids, int *masks, size_t count) would be possible?
I'm using this library together with an Arduino MKR WAN and a MKR CAN shield to read data from a Renault Twizy. Bitwise anding together all the CAN IDs I need results in 0, so using a single filter is not possible.

I can confirm on the MCP2515. If you look at the code, the MCP routine just loops 6 times and writes to all registers. I have changed the (my) function to support multiple IDs and masks. There are only two mask filters on the MCP which must be set.
Maybe we will see that implementation in future versions.

looking at the MCP2515 SPI manuel, you are absolutely right. But maybe a macro like this allows to use the old code without changes?
#define REG_RXFnSIDH(n)            ((n) < 3 ? (0x00 + (n) * 4) : (0x10 + ((n) - 3) * 4)
#define REG_RXFnSIDL(n)            ((n) < 3 ? (0x01 + (n) * 4) : (0x11 + ((n) - 3) * 4)
#define REG_RXFnEID8(n)            ((n) < 3 ? (0x02 + (n) * 4) : (0x12 + ((n) - 3) * 4)
#define REG_RXFnEID0(n)            ((n) < 3 ? (0x03 + (n) * 4) : (0x13 + ((n) - 3) * 4)

of course you need to make sure, that n does not have side effects.

Perfect :-)
I have not testet yet but this looks like the way to go. I will implement and let you know.

#define REG_RXFnSIDH(n)            n < 3 ? 0x00 + n * 4 : 0x10 + (n - 3) * 4
#define REG_RXFnSIDL(n)            n < 3 ? 0x01 + n * 4 : 0x11 + (n - 3) * 4
#define REG_RXFnEID8(n)            n < 3 ? 0x02 + n * 4 : 0x12 + (n - 3) * 4
#define REG_RXFnEID0(n)            n < 3 ? 0x03 + n * 4 : 0x13 + (n - 3) * 4

int MCP2515Class::set_mask_and_filter(int* mask, int* filter, byte f_len) {
  // config mode
  writeRegister(REG_CANCTRL, 0x80);
  if (readRegister(REG_CANCTRL) != 0x80) {
    return 0;
  }

  writeRegister(REG_RXBnCTRL(0), FLAG_RXM0);
  writeRegister(REG_RXBnCTRL(0), FLAG_RXM0);

  writeRegister(REG_RXMnSIDH(0), mask[0] >> 3);
  writeRegister(REG_RXMnSIDL(0), mask[0] << 5);
  writeRegister(REG_RXMnEID8(0), 0);
  writeRegister(REG_RXMnEID0(0), 0);

  writeRegister(REG_RXBnCTRL(0), FLAG_RXM1);
  writeRegister(REG_RXBnCTRL(0), FLAG_RXM1);

  writeRegister(REG_RXMnSIDH(1), mask[1] >> 3);
  writeRegister(REG_RXMnSIDL(1), mask[1] << 5);
  writeRegister(REG_RXMnEID8(1), 0);
  writeRegister(REG_RXMnEID0(1), 0);

  for (byte i = 0; i < f_len; i++) {
    writeRegister(REG_RXFnSIDH(i), filter[i] >> 3);
    writeRegister(REG_RXFnSIDL(i), filter[i] << 5);
    writeRegister(REG_RXFnEID8(i), 0);
    writeRegister(REG_RXFnEID0(i), 0);
  }

  // normal mode
  writeRegister(REG_CANCTRL, 0x00);
  if (readRegister(REG_CANCTRL) != 0x00) {
    return 0;
  }

  return 1;
}

That runs on my side.
But you have to change the old code in any case, because it does not allow to set individual masks or filters. The old code just fills both masks withe one value and also fills all six filters with only one value.

I've sent a PR a week ago which implements mostly the same but also handles the case when the amount of ids > 6 (the number of hardware filters available).
see #20
I still have to add the updated register macros and test it with my hardware.

There is more. My above code changes has only effect if the control bits within some registers are set correctly. The above code has a sequence take from the origin:
  writeRegister(REG_RXBnCTRL(0), FLAG_RXM0);
  writeRegister(REG_RXBnCTRL(0), FLAG_RXM0);

In fact, this is nonsense here. Replace it with the following code:
  // Enable filtering for buffer 0
  uint8_t val = readRegister(REG_RXBnCTRL(0));
  val &= B10011111;
  writeRegister(REG_RXBnCTRL(0), val);
  // Enable filtering for buffer 1
  val = readRegister(REG_RXBnCTRL(1));
  val &= B10011111;
  writeRegister(REG_RXBnCTRL(1), val);
  // And allow interrupt therefore
  val = readRegister(REG_CANINTE);
  val |= B00000011;
  CAN.writeRegister(REG_CANINTE, val);

What does it do? At first it clears the bits within the control registers for the RX buffers 0 and 1 to allow the usage of filtering. Without setting any filters or masks it does not make any sense.
Setting the flags within the control register CANINTE enables the external interrupt to shoot if one of the RX buffers has been filled (after the filtering process)!
Why this all? Well I was trying to wake up my MCU by receiving only special CAN ids. It did not work but after altering my code as above it works like charm :-).
Hope this helps someone.

Sent a PR https://github.com/sandeepmistry/arduino-CAN/pull/47
that allows specifying registers directly.


Sync package-lock.json to package.json

This PR synchronizes the header section of package-lock.json to match the contents of package.json.
It corrects an error introduced in PR #894 where npm should have automatically updated the version field and removed the empty devDependencies field when it updated package-lock.json.
If it is not corrected, then git will show an uncommitted change if npm install is run.
Verification
Execute
npm run install-test
with no errors recorded.
Execute
git status
with no uncommitted changes shown.

@MikeMcC399 thanks!


Write conditionals in Kotlin: Android Basics with Compose

URL of codelab
https://developer.android.com/codelabs/basic-android-kotlin-compose-conditionals?continue=https%3A%2F%2Fdeveloper.android.com%2Fcourses%2Fpathways%2Fandroid-basics-compose-unit-2-pathway-1%3Fhl%3Dko#2
In which task and step of the codelab can this issue be found?
Describe the problem
When setting the language in Korean from the above URL, the part "The output is a slow message because:" does not come out properly. Other parts have the same problem.
Steps to reproduce?

Go to... Above URL
Click on... Language(Korean)
See error...

Versions
Android Studio version:
API version of the emulator:
Additional information
Include screenshots if they would be useful in clarifying the problem.

Hi @Duck-jun99
Thank you for writing in and bringing this to our attention.
It has been reported to our localization team so it can be addressed.
Hope you are enjoying the course!
Regards.


KS API only supplies 1 author even when 2 are listed

I opened https://github.com/KSP-CKAN/CKAN/issues/1048 which got me to come here with this issue.
https://www.kerbalstuff.com/mod/855 lists 2 authors but checking the API at https://kerbalstuff.com/api/mod/855 only lists the first one. I tried looking around in the API docs and the api.py where I guess that https://github.com/KerbalStuff/KerbalStuff/blob/master/KerbalStuff/blueprints/api.py#L47 should point to  something else than the single author?

Confirmed bug (or more of an oversight than a bug). PR welcome, I'll fix it eventually if no one else does.


RuntimeError: view size is not compatible with input tensor's size and strid

Hi, it's really a great work!
However, I got the error message when I run the train.sh
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.
How could I fix it?
Thank you a lot!

Hi, thanks for your interest. what torch version are you using? can you share more detailed error log?

closing due to inactivity


Opening archives by dragging the file to the Fiji logo

When trying to open an archive by dragging the file to the Fiji logo two archives open: one with the file name in the header, one without. Also the Fiji log window displays an error and executing commands leads to a lot of crashes. It got so bad in the end that I had to force quit Fiji because it did not want to shut down because of all the errors raised.
Can you look into this and maybe either fix this or disable the dragging option?

Hi @NadiaHuisjes,
Thank you very much for reporting this issue. Was SCIFIO turned on or off when you received this error? I noticed that I get the two windows when SCIFIO is turned off. When SCIFIO is turned on, everything works fine for me. Can you check this.
I mean globally turned on using the checkbox under edit>options...

It seems that my SCIFIO is on (see screenshot) and I still observe the reported problem.

I am not sure if it is possible to turn the drag and drop off. For me it just ones one window with no problems if scifio is turned on. If scifio is turned off I have the same problem as you.
Maybe there is a mistake somewhere and/or workaround to prevent this issue. But I am still puzzled why I see differences with scifio on or off.

For me, I only get this issue if SCIFIO is turned off. The issue is caused by this commit - https://github.com/fiji/IO/commit/e41e251c1bb6451267dd01321728f800074322b2
This commit is supposed to open files with SCIFIO turned off in the case no other readers are found. However, in our case this code creates the window twice. First it calls the open method and inside there I had a uiService.show( call that displayed the archive. However, in the same code above it also runs show. I guess there must be a mechanism whereby I don't have to call show directly in my IOPlugin open method, but inside the archive is show automatically. If such a mechanism exists that would fix the problem. I will start digging through scijava to see how other IOPlugins work in this regard.

With mars-fx commit c5524de and related mars-core commit 36a569d now this issue is partially address. Now Open.. and Drag and Drop work as expected if SCIFIO is turned off. But they do not work if SCIFIO is turned on. Instead yamt, yama, and yama.store files are opened, but not displayed. So opening again will trigger an error.
I guess there must be a way to have SCIFIO display these files directly, but I can't seem to find the uiService.show( method that would do this anywhere. I will keep searching

I posted on the ImageJ forum to see if they know the solution to this problem - https://forum.image.sc/t/open-a-custom-file-type-via-the-imagejs-open-dialog/41471/4

It appears that when SCIFIO is turned on uiService.show(..) is never called and table as well as archive windows are never opened in the DefaultLegacyOpener in net.imagej.legacy.plugin. If SCIFIO is turned off then uiService.show(...) is called in https://github.com/fiji/IO/commit/e41e251c1bb6451267dd01321728f800074322b2 and windows are opened.
As a work around duderstadt-lab/mars-core@825cfb7 adds imagej-legacy dependency and then detects if SCIFIO is turned on or off. If it is turned on uiService.show(...) is called in the open methods of mars IOPlugins. If it is off uiService.show(...) is never called in mars and instead is called by https://github.com/fiji/IO/commit/e41e251c1bb6451267dd01321728f800074322b2.
This appears to fix the issue, so I am closing it for now. This workaround should be removed if the uiService.show(...) is added to the DefaultLegacyOpener .


JwtSecurityTokenHandler Validate Token fails if not using Windows in 5.3.0

When creating a new JwtSecurityTokenHandler() and passing in a RSACryptoServiceProvider in OSX, an exception is thrown when calling ValidateToken.
The error is: System.PlatformNotSupportedException: 'CspKeyContainerInfo' requires Windows Cryptographic API (CAPI), which is not available on this platform.
Exceptions caught:
 'System.PlatformNotSupportedException: 'CspKeyContainerInfo' requires Windows Cryptographic API (CAPI), which is not available on this platform.
   at System.Security.Cryptography.RSACryptoServiceProvider.get_CspKeyContainerInfo()
   at Microsoft.IdentityModel.Tokens.RSACryptoServiceProviderProxy..ctor(RSACryptoServiceProvider rsa) in C:\agent2\_work\56\s\src\Microsoft.IdentityModel.Tokens\RsaCryptoServiceProviderProxy.cs:line 74
   at Microsoft.IdentityModel.Tokens.AsymmetricAdapter.Initialize(RSA rsa, String algorithm) in C:\agent2\_work\56\s\src\Microsoft.IdentityModel.Tokens\AsymmetricAdapter.cs:line 305
   at Microsoft.IdentityModel.Tokens.AsymmetricAdapter..ctor(SecurityKey key, String algorithm, HashAlgorithm hashAlgorithm, HashAlgorithmName hashAlgorithmName, Boolean requirePrivateKey) in C:\agent2\_work\56\s\src\Microsoft.IdentityModel.Tokens\AsymmetricAdapter.cs:line 84

The stack-trace refers to RsaCryptoServiceProviderProxy.cs and looking at the commit history, it looks like it added Net Standard 2.0 Support.  The parameter references the CspKeyContainerInfo in the constructor here.
This does not work in OSX as, as the exception states, CspKeyContainerInfo isn't supported.  So when it tries to access the parameter, this exception is thrown.  The code used to call this previously worked in 5.2.4.

Not sure if this is correct but changing the constructor of RsaCryptoServiceProviderProxy.cs seems to work.  Again, not familiar enough with this code base but it def works.
public RSACryptoServiceProviderProxy(RSACryptoServiceProvider rsa)
{
    if (rsa == null)
        throw LogHelper.LogArgumentNullException(nameof(rsa));

    // Level up the provider type only if:
    // 1. it is PROV_RSA_FULL or PROV_RSA_SCHANNEL which denote CSPs that only understand Sha1 algorithms
    // 2. it is not associated with a hardware key
    #if WINDOWS
    if ((rsa.CspKeyContainerInfo.ProviderType == PROV_RSA_FULL || rsa.CspKeyContainerInfo.ProviderType == PROV_RSA_SCHANNEL) && !rsa.CspKeyContainerInfo.HardwareDevice)
    {
        var csp = new CspParameters();
        csp.ProviderType = PROV_RSA_AES;
        csp.KeyContainerName = rsa.CspKeyContainerInfo.KeyContainerName;
        csp.KeyNumber = (int)rsa.CspKeyContainerInfo.KeyNumber;
        if (rsa.CspKeyContainerInfo.MachineKeyStore)
            csp.Flags = CspProviderFlags.UseMachineKeyStore;

        // If UseExistingKey is not specified, the CLR will generate a key for a non-existent group.
        // With this flag, a CryptographicException is thrown instead.
        csp.Flags |= CspProviderFlags.UseExistingKey;

        _rsa = new RSACryptoServiceProvider(csp);

        // since we created a new RsaCryptoServiceProvider we need to dispose it
        _disposeRsa = true;
    }
    else
    {
        // no work-around necessary
        _rsa = rsa;
    }
    #else
    _rsa = rsa;
    #endif
}

Another thing I noticed when building in OSX, JsonWebKey fails to build because once the statement is built you get unreachable code.  It ultimately becomes:
internal ECDsa CreateECDsa(string algorithm, bool usePrivateKey)
{

    throw new NotImplementedException(LogMessages.IDX10676);

    if (Crv == null)
        throw LogHelper.LogArgumentNullException(nameof(Crv));

    if (X == null)
        throw LogHelper.LogArgumentNullException(nameof(X));

    if (Y == null)
        throw LogHelper.LogArgumentNullException(nameof(Y));
...
}

@RichardD012 RsaCryptoServiceProvider is available on .Net 2.0 and above.
We'll look into ECDsa throwing on MAC.
BTW you seem to be our most advanced MAC user, no-one else has hit this.

@GeoK a new mac issue, can you look into it?

Hi
I can add that I have just been sitting with a similar issue on Linux. It says the Kid can not be matched, but they are correct. A downgrade to 5.2.4 solves the issue.

Just to confirm, same issue here under linux. I'm getting the following stack trace
System.PlatformNotSupportedException: 'CspKeyContainerInfo' requires Windows Cryptographic API (CAPI), which is not available on this platform.
   at System.Security.Cryptography.RSACryptoServiceProvider.get_CspKeyContainerInfo()
   at Microsoft.IdentityModel.Tokens.RSACryptoServiceProviderProxy..ctor(RSACryptoServiceProvider rsa)
   at Microsoft.IdentityModel.Tokens.AsymmetricAdapter.Initialize(RSA rsa, String algorithm)
   at Microsoft.IdentityModel.Tokens.AsymmetricAdapter..ctor(SecurityKey key, String algorithm, HashAlgorithm hashAlgorithm, HashAlgorithmName hashAlgorithmName, Boolean requirePrivateKey)
   at Microsoft.IdentityModel.Tokens.AsymmetricSignatureProvider.ResolveAsymmetricAdapter(SecurityKey key, String algorithm, Boolean requirePrivateKey)
   at Microsoft.IdentityModel.Tokens.AsymmetricSignatureProvider..ctor(SecurityKey key, String algorithm, Boolean willCreateSignatures)
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateSignatureProvider(SecurityKey key, String algorithm, Boolean willCreateSignatures)
   at Microsoft.IdentityModel.JsonWebTokens.JwtTokenUtilities.CreateEncodedSignature(String input, SigningCredentials signingCredentials)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.CreateJwtSecurityTokenPrivate(String issuer, String audience, ClaimsIdentity subject, Nullable`1 notBefore, Nullable`1 expires, Nullable`1 issuedAt, SigningCredentials signingCredentials, EncryptingCredentials encryptingCredentials)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.CreateJwtSecurityToken(SecurityTokenDescriptor tokenDescriptor)

while calling CreateJwtSecurityToken.  5.2.4 works just fine.

Just to confirm, same issue here under linux. I'm getting the following stack trace
System.PlatformNotSupportedException: 'CspKeyContainerInfo' requires Windows Cryptographic API (CAPI), which is not available on this platform.
   at System.Security.Cryptography.RSACryptoServiceProvider.get_CspKeyContainerInfo()
   at Microsoft.IdentityModel.Tokens.RSACryptoServiceProviderProxy..ctor(RSACryptoServiceProvider rsa)
   at Microsoft.IdentityModel.Tokens.AsymmetricAdapter.Initialize(RSA rsa, String algorithm)
   at Microsoft.IdentityModel.Tokens.AsymmetricAdapter..ctor(SecurityKey key, String algorithm, HashAlgorithm hashAlgorithm, HashAlgorithmName hashAlgorithmName, Boolean requirePrivateKey)
   at Microsoft.IdentityModel.Tokens.AsymmetricSignatureProvider.ResolveAsymmetricAdapter(SecurityKey key, String algorithm, Boolean requirePrivateKey)
   at Microsoft.IdentityModel.Tokens.AsymmetricSignatureProvider..ctor(SecurityKey key, String algorithm, Boolean willCreateSignatures)
   at Microsoft.IdentityModel.Tokens.CryptoProviderFactory.CreateSignatureProvider(SecurityKey key, String algorithm, Boolean willCreateSignatures)
   at Microsoft.IdentityModel.JsonWebTokens.JwtTokenUtilities.CreateEncodedSignature(String input, SigningCredentials signingCredentials)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.CreateJwtSecurityTokenPrivate(String issuer, String audience, ClaimsIdentity subject, Nullable`1 notBefore, Nullable`1 expires, Nullable`1 issuedAt, SigningCredentials signingCredentials, EncryptingCredentials encryptingCredentials)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.CreateJwtSecurityToken(SecurityTokenDescriptor tokenDescriptor)

while calling CreateJwtSecurityToken. 5.2.4 works just fine.

Same issue on my side in Linux in Azure, same solution is working (downgrade to 5.2.4)

@RichardD012 @nenne @CheloXL @af-mst Thank you for reporting and confirming this issue.
CspKeyContainerInfo is indeed a Windows-specific member.
We will fix this in 5.3.1.

Marking this issue as Fixed for the upcoming release (5.3.1).
Thank you again for reporting these issues.

@GeoK I'm sorry, I ran a test on a console application using only Microsoft.IdentityModel.Tokens in version 5.4.0, and it worked in linux and windows, I believe that this is not working in my webapi netcoreapp application because of the version of Microsoft.AspNetCore .App still does not contain this fix, correct?

@pauloofmeta Correct. I believe that Microsoft.AspNetCore.App 2.2.0/2.2.1 will pull IdentityModel 5.3.0 by default.
Can you upgrade Microsoft.IdentityModel.Protocols.OpenIdConnect in your AspNetCore app to version 5.4.0 and see if that resolves your issue?
<PackageReference Include="Microsoft.IdentityModel.Protocols.OpenIdConnect" Version="5.4.0" />

@pauloofmeta please reopen if this issue is still occurring.


Single fetch response parameter for loader is undefined if no default export exists in the route

Reproduction
https://stackblitz.com/edit/remix-run-remix-x55her?file=app%2Froutes%2Fno-default.tsx
System Info
System:
    OS: Linux 5.0 undefined
    CPU: (8) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Memory: 0 Bytes / 0 Bytes
    Shell: 1.0 - /bin/jsh
  Binaries:
    Node: 18.18.0 - /usr/local/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 10.2.3 - /usr/local/bin/npm
    pnpm: 8.15.6 - /usr/local/bin/pnpm
  npmPackages:
    @remix-run/dev: * => 2.9.1
    @remix-run/node: * => 2.9.1
    @remix-run/react: * => 2.9.1
    @remix-run/serve: * => 2.9.1
    vite: ^5.1.0 => 5.2.10

Used Package Manager
npm
Expected Behavior
The response parameter should always be a ResponseStub, even in resource routes without a default export.
Actual Behavior
Visiting a resource route (in the stackblitz: app/routes/no-default.tsx) results in the response starting as undefined and an error being printed to the console:
The following error is a bug in Remix; please open an issue! https://github.com/remix-run/remix/issues/new
Error: Expected a Response to be returned from queryRoute

Resource routes don't need a response stub because they return actual Response instances that contain a status and headers - so you just set them directly.  No need for a stub when you have the real thing!
The response stub concept for single fetch is needed because those loaders don't return a Response - they return data that is merged with other loader data into a Response, so we need to give you some abstraction to handle status/headers.

I think this is a breaking change with the new Single Fetch feature.
Before, your resource routes could return a naked object and Remix would automatically convert that to a JSON response.
Now, if you return a naked object from a resource route, you will get that error message.
It's not technically a Remix error. You should check to see if unstable_singleFetch is true and then let the developer know that that resource routes can no longer return a naked object. They can still use the json helper, since it creates a new Response for you.

yeah, we should probably add a section to the docs to make this more clear, but this section in the docs applies to all loaders/actions - including resource routes.

Naked objects returned from loader and action functions are no longer automatically converted into a JSON Response and are serialized as-is over the wire

Resource routes in particular don't do anything special for single fetch - and since there's no longer an automatic conversion we don't have a Response to send back anymore.

We decided we are goin g to keep this behavior in v2 for backwards compatibility and to make adoption easier.  We'll detect naked objects from resource routes only and convert them to json() responses and log a deprecation warning so you know to go update it.  In the next major version of Remix this automatic conversion won't happen.

Please see the dev branch docs for details on resource routes in single fetch with this change.  These changes will be available in the next release.
Keeping this issue open as we are going to get the response type ambiguity cleaned up - this will likely require a new V3_LoaderFunctionArgs type you'll need to switch to when adopting single fetch.

I hope we can keep ResponseStub everywhere because it helps to build simpler things like that:
https://supabase.com/docs/guides/auth/server-side/creating-a-client?queryGroups=framework&framework=remix&queryGroups=environment&environment=remix-action#creating-a-client
export function getSupabaseServerClient(
  request: Request,
  response: ResponseStub,
) {
  const cookies = parse(request.headers.get("Cookie") ?? "");

  return createServerClient(env.SUPABASE_URL, env.SUPABASE_ANON_KEY, {
    cookieOptions,
    cookies: {
      get(key) {
        return cookies[key];
      },
      set(key, value, options) {
        response.headers.append("Set-Cookie", serialize(key, value, options));
      },
      remove(key, options) {
        response.headers.append("Set-Cookie", serialize(key, "", options));
      },
    },
  });
}

For this specific example, it could be solve with middleware, though.

This should be resolved by the introduction of defineLoader/defineAction in https://github.com/remix-run/remix/pull/9372. We'll have a prerelease out soon for validation.

2.9.2 is released - https://github.com/remix-run/remix/releases/tag/remix%402.9.2


Players are not told when tagged.

With this config (http://pastebin.com/5zH9VqjG) players are not told that they are tagged. Using build 29.

Try latest version


NullReferenceException thrown for CreateVideoPlayerEventArgs

Hello,
I'm using VideoPlayer to play a small video while loading data for the app on startup. I made a loading page which follows the same principle with the ChillPlayer MainMenu example. On startup, I set the MainPage to this loading page, check for internet connection, then set the MainPage to original page.  gist contains the app.cs, loadpage.xaml.cs, loadpage.xaml and stacktrace. I'm putting stacktrace to here as well:
at Octane.Xamarin.Forms.VideoPlayer.iOS.Renderers.VideoPlayerRenderer.CreateVideoPlayerEventArgs () [0x0000b] in <57157d21d2404e4b81ad31d1fd3e7438>:0    at Octane.Xamarin.Forms.VideoPlayer.iOS.Renderers.VideoPlayerRenderer+<>c__DisplayClass24_0.<UpdateTimeElapsedInterval>b__0 (CoreMedia.CMTime time) [0x00000] in <57157d21d2404e4b81ad31d1fd3e7438>:0    at ObjCRuntime.Trampolines+SDActionArity1V5.Invoke (System.IntPtr block, CoreMedia.CMTime obj) [0x00019] in /Users/builder/data/lanes/5665/db807ec9/source/xamarin-macios/src/build/ios/native/ObjCRuntime/Trampolines.g.cs:12182    at (wrapper native-to-managed) ObjCRuntime.Trampolines+SDActionArity1V5:Invoke (intptr,CoreMedia.CMTime) --- End of stack trace from previous location where exception was thrown ---   at (wrapper managed-to-native) UIKit.UIApplication:UIApplicationMain (int,string[],intptr,intptr)   at UIKit.UIApplication.Main (System.String[] args, System.IntPtr principal, System.IntPtr delegate) [0x00005] in /Users/builder/data/lanes/5665/db807ec9/source/xamarin-macios/src/UIKit/UIApplication.cs:79    at UIKit.UIApplication.Main (System.String[] args, System.String principalClassName, System.String delegateClassName) [0x00038] in /Users/builder/data/lanes/5665/db807ec9/source/xamarin-macios/src/UIKit/UIApplication.cs:63    at MULTI.iOS.Application.Main (System.String[] args) [0x00001] in /Users/---/Projects/MULTI/MULTI.iOS/Main.cs:17
When I run the simulator for debugging, the LoadPage loads perfectly, plays the video and shows the text. According to VS, it runs the MainPage = new OpeningPage() and starts loading that page as well (runs the methods, etc). However, even though it's running, it still shows the videoplayer.  Then, when the video finishes, it doesn't switch to OpeningPage or do anything.
@adamfisher

Hi,
I have the same error : I have a carousel view within a popup, in which I display videos; sometimes, when i close the popup, my app crashes and the same message appears. This doesn't crash everytime, which makes it difficult to patch

Setting TimeElaspedInterval = 0 seems to resolve my issue, it may work for you as well.


Background and underline shapes not properly calculated

Background and underline shapes are combined even if they are not consecutive.
The SpellChecking demo shows this: not only the invalid words are underlined and highlighted, but all words inbetween also.

Resolved by #595


Made some changes

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[ ] My pull request targets the master branch of freeCodeCamp.
[ ] None of my changes are plagiarized from another source without proper attribution.
[ ] All the files I changed are in the same world language (for example: only English changes, or only Chinese changes, etc.)
[ ] My changes do not use shortened URLs or affiliate links.

Closes #XXXXX

Nothing to worry about

Thank you for opening this pull request.
This is a standard message notifying you that we’ve reviewed your pull request and have decided not to merge it. We would welcome future pull requests from you. Please see our Contribution Guidelines for details.
Thank you and happy coding.


test-branch/execute-job/wrong-commit-sha-on-pr/6259ce53-cf98-4eac-b39d-d0af751ff98b

PR for testing 0983404a19e720e0e3796cc7c885f1afcd1de16d

/canonical/self-hosted-runners/run-workflows 931ad21dd9ace68aaa0a7d341e94de96ad93fa6b


transifex-client: remove as deprecated

Description of changes
From https://github.com/transifex/transifex-client:

Please start using the new Transifex CLI here , since this software is considered deprecated (as of January 2022) and will sunset on Nov 30, 2022.

transifex-cli is a new one.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Quick question, shouldn't an alias be added here?

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-already-reviewed/2617/1352


Removed trailing whitespace & added full stops.

Per the Contribution Guidelines I added missing full stops to the
descriptions and removed trailing whitespace.

Thank you, Andrew!


Cleanup database module

Motivation

iterators should be released immediately without relying on the finalizer
iterators may return an error and it should be reported appropriately
remove custom memory db implementation that wasn't doing the right thing in some cases
remove unused code

Test Plan
unit tests
TODO

[ ] Explain motivation or link existing issue(s)
[ ] Test changes and document test plan
[ ] Update documentation as needed

DevOps Notes

[x] This PR does not require configuration changes (e.g., environment variables, GitHub secrets, VM resources)
[x] This PR does not affect public APIs
[x] This PR does not rely on a new version of external services (PoET, elasticsearch, etc.)
[x] This PR does not make changes to log messages (which monitoring infrastructure may rely on)

bors merge


CORS are not enabled at https://rmbl.us

Im trying to setup the library in a test project however CORS 😈  does not allow me to get access 🙅♂ .
Access to fetch at 'https://rmbl.us/whisper-turbo/whisper-small-pf16-full.bin' from origin 'http://localhost:19006' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

Fixed!