PR: Make the font used by the application configurable and other UI fixes

Description of Changes

This will allow us to easily test UI changes with different fonts.
Make this change to require a restart because it can't be applied while the app is running.
Fix pixelated splash screen when using scale factors greater than 1 (this happens for certain screen aspect ratios).
Remove setting font for labels of status bar widgets (this was different from the app font).
Switch setting sizes from em's to pixels in most places. That's because I discovered that Qt adjusts em sizes according to the app font, but that's not what we want in the main and pane toolbars, for instance (which was the case before).
Remove hard-coded font sizes in the "Report error" dialog and improve its style a bit by correctly aligning its widgets to the left and right.
Re-add the previous cell icon because it's used in the Outline pane and rename the one to create cells to new_cell.

Visual changes:
Before

Fonts and layout of the "Report error" dialog were not the best

Using new cell icon in the Outline

After

New entry in Preferences to change the app font:

"Report error" dialog looks better now. its widgets are also perfectly aligned to the left and right.

Restore cell icon in the Outline pane

Issue(s) Resolved

Fixes #5942.
Affirmation
By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct:

Thanks for the work here @ccordoba12 ! Quick question, could this be related with https://github.com/spyder-ide/ux-improvements/issues/31 ?

Another important point I forgot to mention is that this will be useful for users of languages like Chinese, Japanese or Korean on Windows because they could select a different font for the app, instead of being forced to use some ugly Latin font. I've seen that in a lot of screenshots when users want to use Spyder in English but the OS is in one of those languages.

Giving the application font a local check seems like there are some elements (like dialog/window titles, menus and tooltips) that aren't affected with the font change:

Maybe there is some missing logic to apply the font to such elements or is that expected?

Giving the application font a local check seems like there are some elements (like dialog/window titles, menus and tooltips) that aren't affected with the font change

That's not the case for me on Linux, i.e. all elements change their font. So, I'll have to check on Windows.
But did you restart Spyder? Perhaps Spyder didn't ask you to do it, which could be the problem in your case.

But did you restart Spyder? Perhaps Spyder didn't ask you to do it, which could be the problem in your case.

Applying the preference showed me the dialog to restart and Spyder was restarted succesfully. Also, I closed and reopened Spyder but the partial font change behavior is still there. Maybe some Qt setting needs to be enabled for Windows? 🤔
Checked on macOS also seems like there are elements which font is still different from the set but there I think more elements are affected like labels, buttons and the status bar:

@dalthviz, this is ready for another review. I think I addressed the issues you found on Windows and Mac. I also made several improvements related to fonts (see the updated PR description for all the new changes).

Gave this a new check on Windows and seems like something is happening when applying the font to the elements in the Variable Explorer. Creating a new variable after changing the font to MS Serif gives you this:

Also, seems like the font in tree-like widgets still doesn't change? Is that expected or something that still missing?

Checking on macOS and there seems like the variables in the Variable Explorer are okay related with font and size settings and seems like, as on Windows, tree-like widgets are not getting the new font setting. Besides that, a couple of other elements still are not getting set the new font like the status bar, buttons and the Editor top text:

I gave this a new check on Windows and seems like something is happening when applying the font to the elements in the Variable Explorer. Creating a new variable after changing the font to MS Serif gives you this

I think fixed that in my last commit. Please check again.

Also, seems like the font in tree-like widgets still doesn't change? Is that expected or something that still missing?

It was missing but it should be fixed now too.

Checking on macOS, seems like the variables in the Variable Explorer are okay and seems like, as on Windows, tree-like widgets are not getting the new font setting. Besides that, a couple of other elements still are not getting set the new font like the status bar, buttons and the Editor top text

It seems that making this work on Mac is either really difficult (I was not expecting that the issues you found were still present after my last changes) or not possible at all (I'm not sure how to fix them). So, I'd say for now we should disable this feature on Mac, if that's ok for you (that would be helpful for me too because I've invested a lot of time on this and I'd like to move on).
In addition, I think this would be mainly useful for Chinese, Japanese, Korean users on Windows (they requested it) and Linux ones (for testing and because it works almost without issues there).

Checked again and seems like there is still a thing regarding the size of the Variable Explorer header not expanding as stayed in the OP:

But other than that seems like things are working on Windows :+1:
Regarding macOS, I guess that is an option, but then on macOS things will work as before regarding the rich text font? And then users will not see the new preference regarding an Interface font but the old Rich text font setting, right?

Checked again and seems like there is still a thing regarding the size of the Variable Explorer header not expanding as stayed in the OP

That's odd because it works for me with MS Serif
However, the solution I implemented seems to work for some fonts and fonts sizes, but not for others (this is with the same font at two different sizes):

So, I guess this depends on the algorithm used by Qt to adjust the columns width to their headers' contents, which should use QFontMetrics to compute the length of each header. And that probably doesn't work very well for some fonts and sizes.
However, if we'd try to implement the algorithm ourselves, I think we'll arrive at something very similar to what Qt already has. So, I guess what I did in this PR is the best we can do to improve the situation, although it's not perfect (and it'll probably never will).

But other than that seems like things are working on Windows

Great! That's really good news.

Regarding macOS, I guess that is an option, but then on macOS things will work as before regarding the rich text font? And then users will not see the new preference regarding an Interface font but the old Rich text font setting, right?

Nop, we'd simply won't allow users to uncheck the Use the system default interface font checkbox. This is how things would look (screenshot taken on Windows because I don't have a Mac):

I pushed a commit that implements that solution, so let me know if it works for you after resetting Spyder options.

So, I guess this depends on the algorithm used by Qt to adjust the columns width to their headers' contents, which should use QFontMetrics to compute the length of each header. And that probably doesn't work very well for some fonts and sizes.

However, if we'd try to implement the algorithm ourselves, I think we'd arrive at something very similar to what Qt already does. So, I guess what I did in this PR is the best we can do to improve the situation, although it's not perfect (and it'll probably never be).
Probably even further it depends on the display dpi or something like that. Following that maybe adding some link to the discussion here around the code that parcially fixes the headers could be nice so there is a reference indicating that there is not much else to do about the header situation.

I pushed a commit that implements that solution, so let me know if it works for you after resetting Spyder options.

Seems like that is working :+1: Not sure if there is something else that could be done to improve the UX to handle the application font option in a cross platform way. Maybe @conradolandia could assess if what we have here is okay/makes sense? Or maybe you have already discussed this PR during the UX/UI meetings @ccordoba12 ?
Also, a CI job was failing so I re-runned it, hopefully it will pass 🤞🏼

I've been reading QFontMetrics documentation, and this called my attention, regarding this issue (emphasis mine):

Calling the QFontMetrics constructor with a QFont creates a font metrics object for a screen-compatible font, i.e. the font cannot be a printer font. If the font is changed later, the font metrics object is not updated.

Maybe the font metric object needs to be updated after a font change?

Seems like that is working +1 Not sure if there is something else that could be done to improve the UX to handle the application font option in a cross platform way. Maybe @conradolandia could assess if what we have here is okay/makes sense? Or maybe you have already discussed this PR during the UX/UI meetings @ccordoba12 ?

I think it makes sense. It's a very cool feature to be able to choose how Spyder can be themed, and this is a significant feature towards that. It seems strange some elements don't take the change into account, tho. I think unifying Font sources is important.

Probably even further it depends on the display dpi or something like that. Following that maybe adding some link to the discussion here around the code that parcially fixes the headers could be nice so there is a reference indicating that there is not much else to do about the header situation.

Ok, I'll add a comment about it.

Seems like that is working

Great!

Maybe the font metric object needs to be updated after a font change?

Yep, we already do that. But it seems some widgets on Mac don't pick the font selected for the rest of Spyder, and I don't know why. In addition, I don't have a Mac to try to fix that.

I think it makes sense. It's a very cool feature to be able to choose how Spyder can be themed, and this is a significant feature towards that. It seems strange some elements don't take the change into account, tho. I think unifying Font sources is important.

Thanks @conradolandia for your feedback! I totally agree with you. However, I'd prefer to be a bit more pragmatic here to keep things moving. If later Mac users request this feature loudly, then I could spend more time to make it work there. But for now, I think having it on Windows and Linux it's a huge improvement for the project, as you said.

However, seems like changing the font makes some errors regarding cache appear in the console that I used to launch Spyder

Those errors seem unrelated to this and I didn't see them myself when testing on Windows. I'd say they are probably caused because at some point you launched Spyder as admin in your user account.

Also, when you select again the Use the system default interface font after changing the values is like those values reset (font and size) and get replaced with the default values. So seems like you can't have a previous selection for the application font and size while using the default system font, right?

Yeah, that's the behavior I decided to implement for several reasons:

We show the system font when users haven't changed it for another one yet. So, I think rechecking that checkbox should restore things to that state.
I think it'd be confusing to display a different font on that combobox and claim that Spyder is actually using the system one.
We use the interface font internally to make several adjustments. So, we need the right value for it and not the one previously set by users.

Anyhow, I'm approving so if you are good with the things above feel free to merge

Thanks @dalthviz! I'll merge then to keep things moving.


Change members to governors

To comply with the marketing/docs we need to change the concept of members to governors (e.g MsgCreateGovernor)

not sure how it's possible easily
we use the x/staking module for this logic.
It's Validator all over the place.


Question about the implementation of GraphSAGE（SAGEConv）

❓ Questions & Help

It seems that the implementation of GraphSAGE only contains mean pooling as an aggregator, while GraphSAGE has other widely-known variants, such as LSTM-based and GCN-based aggregator.
Therefore, how can I change the mean aggregator to others? Is there any API that can help me, or where should I modify the code?
Looking forward to your reply. Thank you!

Yes, we only have an implementation for mean aggregation. Max aggregation should be also supported when changing aggr to "max". For LSTM aggregation, we currently do not have a fast way to implement that, but I have plans to support that in the future.

Thank you very much for the quick response!
I have another question: Is there a fast way to support me implement the GCN-based aggregator?

What do you mean with GCN-based aggregator?

Well, in the paper of GraphSage, I have seen they mentioned a variant named GraphSAGE-GCN, which is an "inductive version of Kipf et al’s semi-supervised GCN [17], we term this variant GraphSAGE-GCN". I think it might be different from the original Kipf’s GCN.

Well, that is just GraphSAGE without concatenation of central node features, right?

Ah, yes, I got it, So I just need to set the parameter "concat" as False for gcn-based and True for mean or max pooling, am I right?

That's how I understand the GraphSAGE paper :)

You help me a lot! Thank you ~~~~ : )


Train time

Hello! Could you please tell me how long do you use to train?

Have you tried to use other datasets like YCB-V?


Mock prover assert_satisfied_par always fail "cell not assigned"

If a circuit builder has at least one relation between cells (e.g. addition, etc), the MockProver::assert_satisfied_par() fails with the errors bellow. The assert_satisfied() method works without an issue.
Total 1 fixed cells
Total range check advice cells to lookup per phase: [0, 0, 0]
error: cell not assigned

  Cell layout in region 'BaseCircuitBuilder generated circuit':
    | Offset | A0 |
    +--------+----+
    |    1   |  X | <--{ X marks the spot! 🦜
    |    2   | x1 |
    |    3   | x2 |
    |    4   | x3 |

  Gate '1 column a + b * c = out' (applied at offset 1) queries these cells.

error: cell not assigned

  Cell layout in region 'BaseCircuitBuilder generated circuit':
    | Offset | A0 |
    +--------+----+
    |    1   | x0 |
    |    2   |  X | <--{ X marks the spot! 🦜
    |    3   | x2 |
    |    4   | x3 |

  Gate '1 column a + b * c = out' (applied at offset 1) queries these cells.

thread 'sync_step_circuit::tests::test_step_circuit' panicked at /Users/timofey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/halo2-axiom-0.4.2/src/dev.rs:1571:13:
circuit was not satisfied
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test sync_step_circuit::tests::test_step_circuit ... FAILED

Wrong repo

Wrong repo


Race condition in InlineViewStrategy /

Browser:
Chrome 57

Current behavior:
When binding a <compose> to an InlineViewStrategy and replacing the strategy a couple of time in quick succession, the final state may vary. See https://gist.run/?id=786a81fa21a427cd803d1ee1e12681d0
Expected/desired behavior:
Regardless of how long it takes, <compose> should eventually settle on the last state of the InlineViewStrategy.

May be related to #538.

I could not debug the repro locally since it always shows 3 for me. In gistrun though on first run I always get 2. I think the issue is that <compose> does not await the call to CompositionEngine.prototype.compose(in processInstruction) to finish before calling it again. So it all depends on which call will complete last. I'm unsure if not awaiting is intended or not. But you could also get this issue if one of the previous VMs takes longer in its .activate hook than the next ones.
@RomkeVdMeulen I don't think #538 is responsible for this issue, but it could help with its manifestation.
@EisenbergEffect If you are OK with the above explanation could you move the issue to templating-resources? I'll also open a PR.

Issue moved to aurelia/templating-resources #299 via ZenHub


Allow exports for anywhere in the world

I don't think there should be spatial restrictions of where exports occur. Data can be used for humanitarian purposes anywhere in the world.Are there limited computing resources to process exports?

We have discussed this a fair amount and there are resource and bandwidth issues to consider in making it cover the entire world.
If there are humanitarian needs for data in places not currently covered by the export, please just let us know and we can add them quickly on an as needed basis as we did for the Balkans.
Did you have some specific cases in mind?

I believe that the current server hosting the Export Tool can cope with supporting all regions @dodobas @bjohare please correct me if I am wrong!
This is something we considered during development, but thought it would be best to run by the Activation WG first. It is really easy to make the change through the admin tool, and again restrict it, say if there was a large scale humanitarian response and we want to focus on a particular area.
It is a shame to limit the areas of export to certain regions, as it is a great tool for getting OSM data out in various formats. Especially for those that don't have a lot of GIS experience but need to get their hands on some files. @bgirardot what do you think?

@TylerRadford , is this something we could prioritize?
I've had a number of people ask me what the best way is to get raw/shapefiles of OSM data over the weekend. The export tool is the best method I know off all around, but in a lot of these cases pointing them to the HOT export tool only leads to disappointment as they won't be able to select the area they're interested in.
The change shouldn't be hard to make technically; the mask gets pulled from the region_mask table (https://github.com/hotosm/osm-export-tool2/blob/master/jobs/models.py#L252).

@PaulUithol @TylerRadford
This is something we have been discussing in the Activation WG meeting [1], which I have personally been a little slow to action. Will do it this week, so we can review it in the meeting next Wednesday 5th October 2016 at 14:00 UTC. Please join us if you can, your input would be greatly appreciated and I would love to hear more about the comments/suggestions/requests from our users.
[1] https://trello.com/c/rBGVdeJV/48-reasons-to-open-up-all-regions-for-export-via-the-tool

@mataharimhairi , great, didn't realize this was already moving. Afraid I can't make that meeting, will be at a workshop all day. But have added comments to that card, thanks!

I just heard from ARC that they cloned/forked the export tool for use on
POSM. They removed all regions. We could talk to Seth Fitzsimmons who did
this work. From what I understand from talking to Drazen the only technical
concern was that jobs are associated to regions to removing or changing
regions would affect old jobs.
Tyler Radford
Executive Director
<EMAIL_ADDRESS>@TylerSRadford
Humanitarian OpenStreetMap Team
Using OpenStreetMap for Humanitarian Response & Economic Development
web http://hotosm.org/ | twitter https://twitter.com/hotosm | facebook
https://www.facebook.com/hotosm | donate http://hotosm.org/donate
On Tue, Sep 27, 2016 at 10:57 AM, Paul Uithol<EMAIL_ADDRESS>wrote:

@mataharimhairi https://github.com/mataharimhairi , great, didn't
realize this was already moving. Afraid I can't make that meeting, will be
at a workshop all day. But have added comments to that card, thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hotosm/osm-export-tool2/issues/106#issuecomment-249887947,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJZbz2caqfrNWvYdy5o74Wdm5dgYSF4uks5quS7ugaJpZM4GxIWX
.

Brian updated the tool so that removing existing region polygons through the admin export tool no longer deletes jobs associated with it, but simply removes any assigned region [1].
The wiki states that we can then reassign a region to them, although I've been playing with the admin interface and I can't seem to find this option. Currently there 755 jobs that would have to be reassigned.
[1] https://github.com/hotosm/osm-export-tool2/wiki/Add-a-HOT-Export-Region

To be honest though, would the exports need to be reassigned a region if there is only one polygon region 'global'?

Not really I guess. Only page I can see the region being used is on the exports list (http://export.hotosm.org/en/exports/). As long as the map shows you where the exports are, don't think that will be too much of a loss.

A secondary (and possibly more substantial) consideration is how much would need to be done to reconfigure the local Overpass instance to support this (assuming that it's currently dropping diff updates for areas not exposed through the Export Tool). By omitting the US and Europe, particularly data-dense areas don't need to be tracking.
We got around this (for POSM) by using the public Overpass API instance, which we deemed acceptable because the number of people using that Export Tool instance was assumed to be much lower than HOT's.

If we want to limit areas again in the future is that still possible?
We have no idea the impact on our services this has, if it does turn out to be an impact we can not afford, we need to be able to go back to region limited exports.

Yes, but not with the existing system (which more more intended for associating exports w/ regions than limiting currently-available regions). I fully agree that we want the ability to be able to limit areas, but I think the approach should be simplified and more targeted to restriction than association.


docs(deployment): use flowchart to illustrate deployment architecture in different scenes

What's the purpose of this PR
Add more document.
Use flowchart to illustrate deployment architecture in different scenes
Which issue(s) this PR fixes:
None
Brief changelog

Add a new link to sidebar.
Create 2 new markdown files deployment-architecture.md.

An example of flowchart

More flowchart in https://anilople.github.io/apollo/#/zh/deployment/deployment-architecture
Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Read the Contributing Guide before making this pull request.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Write necessary unit tests to verify the code.
[x] Run mvn clean test to make sure this pull request doesn't break anything.
[ ] Update the CHANGES log.

Is it reasonable to put the deployment architecture document here? Or move to another place?

Is it better to put it in Distributed Deployment Guide - 1.3 Environment section?
Or if it is too long to be put there, then we could have a link to this document?

Codecov Report

Merging #4017 (127fa0a) into master (8484178) will increase coverage by 0.03%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #4017      +/-   ##
============================================
+ Coverage     51.46%   51.49%   +0.03%
- Complexity     2524     2531       +7
============================================
  Files           484      484
  Lines         14798    14839      +41
  Branches       1532     1537       +5
============================================
+ Hits           7616     7642      +26
- Misses         6652     6666      +14
- Partials        530      531       +1

Impacted Files
Coverage Δ

...work/apollo/biz/message/DatabaseMessageSender.java
50.00% <0.00%> (-14.59%)
:arrow_down:

...internals/provider/DefaultApplicationProvider.java
49.16% <0.00%> (-2.84%)
:arrow_down:

...ework/apollo/internals/RemoteConfigRepository.java
88.34% <0.00%> (-1.04%)
:arrow_down:

...ramework/apollo/openapi/util/OpenApiBeanUtils.java
8.82% <0.00%> (-0.14%)
:arrow_down:

...work/apollo/common/dto/GrayReleaseRuleItemDTO.java
0.00% <0.00%> (ø)

...rk/foundation/internals/provider/NullProvider.java
0.00% <0.00%> (ø)

...apollo/openapi/dto/OpenGrayReleaseRuleItemDTO.java
0.00% <0.00%> (ø)

...ring/boot/ApolloApplicationContextInitializer.java
94.33% <0.00%> (ø)

...llo/configservice/controller/ConfigController.java
95.29% <0.00%> (ø)

...configservice/controller/ConfigFileController.java
83.47% <0.00%> (ø)

... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8484178...127fa0a. Read the comment docs.


Add Graph500

This is WIP, shouldn't be merged.
Salvaged the crucial parts of my old graph500 implementation. This seems to compile and run fine with local multi-locale setup.

Moved its own branch


Could not resolve all dependencies

I use corda to in my project and I want to include Fuel. I add this to build.gradle
repositories {
    jcenter()
}

dependencies {
    compile 'com.github.kittinunf.fuel:fuel:<latest-version>' //for JVM
}

But when I invoke gradlew build in the terminal, errors come out
:compileKotlin FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Could not resolve all dependencies for configuration ':compileClasspath'.
> Could not resolve com.github.kittinunf.fuel:fuel:<latest-version>.
  Required by:
      project :
   > Could not resolve com.github.kittinunf.fuel:fuel:<latest-version>.
      > Could not get resource 'https://jcenter.bintray.com/com/github/kittinunf/fuel/fuel/%3Clatest-version%3E/fuel-%3Clatest-version%3E.pom'.
         > Could not GET 'https://jcenter.bintray.com/com/github/kittinunf/fuel/fuel/%3Clatest-version%3E/fuel-%3Clatest-version%3E.pom'.
            > sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
   > Could not resolve com.github.kittinunf.fuel:fuel:<latest-version>.
      > Could not get resource 'https://repo1.maven.org/maven2/com/github/kittinunf/fuel/fuel/%3Clatest-version%3E/fuel-%3Clatest-version%3E.pom'.
         > Could not GET 'https://repo1.maven.org/maven2/com/github/kittinunf/fuel/fuel/%3Clatest-version%3E/fuel-%3Clatest-version%3E.pom'.
            > sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
   > Could not resolve com.github.kittinunf.fuel:fuel:<latest-version>.
      > Could not get resource 'https://dl.bintray.com/kotlin/exposed/com/github/kittinunf/fuel/fuel/%3Clatest-version%3E/fuel-%3Clatest-version%3E.pom'.
         > Could not GET 'https://dl.bintray.com/kotlin/exposed/com/github/kittinunf/fuel/fuel/%3Clatest-version%3E/fuel-%3Clatest-version%3E.pom'.
            > sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
   > Could not resolve com.github.kittinunf.fuel:fuel:<latest-version>.
      > Could not get resource 'https://jitpack.io/com/github/kittinunf/fuel/fuel/%3Clatest-version%3E/fuel-%3Clatest-version%3E.pom'.
         > Could not GET 'https://jitpack.io/com/github/kittinunf/fuel/fuel/%3Clatest-version%3E/fuel-%3Clatest-version%3E.pom'.
            > sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Is it the right way to install Fuel by calling gradlew build?
Cannot figure out why. Any help is appreciate. Thank you

You just need to change in your Fuel dependencies  <latest-version>for the latest library version number.
     implementation 'com.github.kittinunf.fuel:fuel:1.12.0' //for JVM

@JairAviles Thanks for your reply. I tried this method but the error still here. I think it's maybe the firewall of corporation computer. Sad

@jalakoo Thanks Jalakoo. The problem is solved


Check if user is using correct python version

More specific to include only those users who are using python 2.6+ and not python 3
Follows the statement more exactly that "nflgame requires Python 2.6+ and does not yet work with Python 3"

Thanks! I had one minor nit, but otherwise this looks good.

Watch that extra colon as well.
(Hehe)

Fixed

Thanks!


Hydra migration

Skip reconciliation for Hydra resources (e.g. Hydra database secret, Hydra synchronization, Hydra rollout) when Hydra is disabled or does not exist in chart config
Create Hydra database secret ory-hydra-credentials in hydra-deprecated namespace if it exist, otherwise use kyma-system. The data source name in the secret is set depending on whether the namespace hydra-deprecated exists or not.

/retest

/override pre-main-ory-reconciler-prod-integration-gcp
As requested by @strekm


Intersection Broken When _fill_perfdata=False

Found in #182. Current intersection relies on the Nones inserted by _fill_perfdata to compute the intersection after the union has already been applied

Fixed in #193


[vidlii] Add new extractor

Before submitting a pull request make sure you have:

[X] At least skimmed through adding new extractor tutorial and youtube-dl coding conventions sections
[X] Searched the bugtracker for similar pull requests
[X] Checked the code with flake8

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

[X] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[ ] Bug fix
[ ] Improvement
[X] New extractor
[ ] New feature

Description of your pull request and other information
This pull request implements the extractor for vidlii.com, see issue #14472. Further, we implemented two test cases.

https://github.com/rg3/youtube-dl/issues/13883

https://github.com/rg3/youtube-dl/pull/14512

Hi dstftw, thanks for your feedback. We just implemented your suggested improvements. We would appreciate a short feedback! Thank you.

Hi Sergey, do we have to fix anything else? Thanks for your feedback, we really appreciate it!

Hi @dstftw, as we are implementing this feature in a software development course, it would be great seeing it to get accepted till end of the week. Do you have any other suggestion for code improvement?

Hi @yan12125, as dsftw is not responding anymore, would you kindly provide me feedback to my changes? Thank you very much!


eid-1285 Create a new resource to accept the SAML form sent from a co…

WHAT
A new proxy node gateway resource, EidasAuthnRequestResourceV2, to handle eIDAS Authn requests.
WHY
The old resource, EidasAuthnRequestResource, parses, validates and creates AuthnRequest SAML, this is deemed a security risk. The responsibility for validating/parsing eIDAS SAML and building a new SAML payload for Hub will now be handled by eidas-saml-parser and verify-proxy-node services respectively.
HOW TO REVIEW
Consider how the eIDAS SAML payload is now validated and relevant data extracted via the eidas-saml-parser service.
Note the use of a verify-proxy-node service,  more normally a tool used by an RP, to generate a Hub Authn Request.
As SAML parsing is no longer beng done on the Proxy Node, logging will be reduced in the gateway service, this will be picked up by eidas-saml-parser service.
The eidas_request_id and hub_request_id are stored in an HttpSession, for future use by a new HubResponseResource.
NOTE
Services are stubbed, there is no consideration of the transport yet (HTTP or RPC). The data flowing across the APIs is what's important for this PR.
EidasAuthnRequestResourceV2 will replace EidasAuthnRequestResource.

Closing as merge not clean


chore: address all fixmes from widget

Changes

memoized component
fixed mistaken comments

@paolodamico only this comment remained:

I can fix it if you give me the right link for this button:


Neptune keeps failing to upload

The size of the dataset is crashing my system, so I'm trying to run this in Neptune. However, when I upload the files to Neptune it fails somewhere along the way, with the error:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/derose/anaconda3/lib/python3.6/site-packages/neptune/internal/common/api/utils.py", line 130, in func_wrapper
return func(*args, **kwargs)
File "/home/derose/anaconda3/lib/python3.6/site-packages/neptune/internal/common/api/utils.py", line 173, in func_wrapper
raise NeptuneServerResponseErrorException(exc.status, exc)
neptune.internal.common.api.exceptions.NeptuneServerResponseErrorException: status: 500, response_message: Internal Server Error
2018-09-18 11:42:53,943 neptune.internal.common.api.utils ERROR    utils.py:126 - log_exception() Failed to call upload_data_as_tarstream
2018-09-18 11:42:53,944 neptune.internal.common.api.utils ERROR    utils.py:127 - log_exception() status: 500, response_message: Internal Server Error
Traceback (most recent call last):
File "/home/derose/anaconda3/lib/python3.6/site-packages/neptune/internal/common/api/utils.py", line 157, in func_wrapper
return func(*args, **kwargs)
File "/home/derose/anaconda3/lib/python3.6/site-packages/neptune/internal/common/api/utils.py", line 194, in func_wrapper
return func(*args, **dict(kwargs, headers_handler=headers_handler))
File "/home/derose/anaconda3/lib/python3.6/site-packages/neptune/generated/swagger_client/apis/default_api.py", line 17183, in upload_data_as_tarstream
headers_handler=params.get('headers_handler'))
File "/home/derose/anaconda3/lib/python3.6/site-packages/neptune/generated/swagger_client/api_client.py", line 333, in call_api
response_type, auth_settings, callback, headers_handler)
File "/home/derose/anaconda3/lib/python3.6/site-packages/neptune/generated/swagger_client/api_client.py", line 155, in __call_api
post_params=post_params, body=body)
File "/home/derose/anaconda3/lib/python3.6/site-packages/neptune/generated/swagger_client/api_client.py", line 369, in request
body=body)
File "/home/derose/anaconda3/lib/python3.6/site-packages/neptune/generated/swagger_client/rest.py", line 164, in POST
body=body)
File "/home/derose/anaconda3/lib/python3.6/site-packages/neptune/generated/swagger_client/rest.py", line 132, in request
raise ApiException(http_resp=r)
neptune.generated.swagger_client.rest.ApiException: (500)
Reason: Server Error
HTTP response headers: {'Server': 'nginx', 'Date': 'Tue, 18 Sep 2018 15:42:53 GMT', 'Content-Type': 'application/json;charset=utf-8', 'Transfer-Encoding': 'chunked', 'Via': '1.1 google', 'Alt-Svc': 'clear'}
HTTP response body: Internal Server Error
How can I fix this? I tried uploading files in smaller batches, but my folders were all separate (not under data) and Neptune apparently doesn't have a way to restructure anything after it's been uploaded. It seems like my only option is to upload everything in one go.

Hi,
Neptune team member here,
This error is not ideal and should be more informative, but it happens when the Neptune CLI can't read some of the files in the directory you're uploading.
Try changing access rights to the files you have so that a python process can successfully run os.path.getsize on all of them
Neptune site also have a web-based option do upload files - go to your project and select "Files" from the menu on the left, it could help for some access rights related issues
I can PM you with the exact file the CLI gets this error
Hope this helps,
Best regards,
Hubert

Thanks, I ended up figuring out how to upload folders in pieces. I will try your solution though, because I uploaded a reduced portion of the images.

Hi @carbonox-infernox,
By the end of this week, we will upload this dataset to the neptune /public directory. It means that you will be able to use it directly, without uploading the data by yourself.
I will let you know when this will be done.
Thank you,
Kamil Kaczmarek

Awesome, thanks! I ended up using an EC2 instance since I knew i could get the local python version working, but I may switch to Neptune. The send pytorch gpu argument seems to imply that Neptune uses GPU acceleration. Is this true?

Hi,
Neptune can use GPUs (even multiple ones), but you need to sepecify it when sending your experiment.
You can select an environement and a workers you wish to use

I really want to do that to get this trained faster, but unfortunately I'm having a problem getting neptune to find files: #182

@kamil-kaczmarek Since I saw you post recently I was wondering if i could ask you a quick question about preparing masks on Neptune. So far Neptune has been running for 4 hours and I can't find the masks_overlayed folder. in the .yaml I specified for it to go to /outputs, but I can't find this directory and Neptune is saying that I'm only using 3 GB of storage, which can't be right if the file is actually being generated.

Hi,
I think I can answer that - Neptune copies your /outputs only after the exprriment finishes, don't worry, it'll be there once the experiment is in state succeeded :)

Awesome, thanks! I was thinking that might be the case.

@jakubczakon how long does it take to prepare masks?


Publish to Maven Central

First of all - stor tack for the library!
Would it be possible for you to publish it to Maven Central to avoid hassle of dealing with GitHub packages?

By the "hassle" I mean this.
Even though it's possible to download artifacts without authentication, it's not possible to configure maven/gradle without authentication token. And forcing all developers do that just for GitHub packages seems a bit unreasonable :disappointed:

Hi there @lazystone!
I have considered doing this, but I haven't worked a lot with java for the last... 10 years or so, so was not sure if MC was "the thing" to go for now 'a days! But yes, it would absolutely be possible to do it!
Sorry for not doing it earlier, hehe, but I will try to get it done asap :)
Also: happy to provide and thanks for using it! :)

Hi @lazystone ! I was just wondering a bit... Is this the "Maven Central" that you are refering to https://oss.sonatype.org/ ? Just so that I don't sign up and fix keys and all that fun stuff and it ends up being the wrong repository :D

Heh..I guess it's this one(never published to Maven Central myself).
I found an article with instructions though: https://dzone.com/articles/publish-your-artifacts-to-maven-central

Also https://maven.apache.org/repository/guide-central-repository-upload.html

Super, thank you.
Aye, I found the same DZone Article. It seems like publishing to maven is a lot more work than a lot of other centralized package repositories, but looking at Java, it doesn't suprice me! ;D
I will try to make it work today, worst case it will be published in the beginning of next week or so :)

https://issues.sonatype.org/browse/OSSRH-61073
I've created an issue on their Jira board and we will see what happens! :)

I have been trying to get this to run during the weekend, it's a whole lot more of a hassle than I though.
I have created an account on OSSRH so that I can publish artifacts, the issue now is that there are aprox 0 good writeups on how to publish from github to the central repository.
I have found a bunch of different resources for uploading artifacts, but some are very old (the sonatype seems to be from 2013, the github one is missing the quite important step of how to sign the archives etc etc...)
I will keep on trying to get it to work, and if anyone have any idea how to do it or have an example deploy script with a working gradle.build file, let me know!

Hi @Johannestegner,
I guess that first of all you need to add username, password for maven central as environment variables. Something like MAVEN_CENTRAL_USERNAME and MAVEN_CENTRAL_PASSWORD.
Then you can try to add new repo to your gradle file:
publishing {
    repositories {
        maven {
            name = "GitHubPackages"
            url = uri("https://maven.pkg.github.com/personnummer/java")
            credentials {
                username = project.findProperty("gpr.user") ?: System.getenv("USERNAME")
                password = project.findProperty("gpr.key") ?: System.getenv("TOKEN")
            }
        }
    }
    repositories {
        maven {
            credentials {
                username = project.findProperty("maven.central.user") ?: System.getenv("MAVEN_CENTRAL_USERNAME")
                password = project.findProperty("maven.central.password") ?: System.getenv("MAVEN_CENTRAL_PASSWORD")
            }
            if (project.version.endsWith('-SNAPSHOT')) {
                url "https://oss.sonatype.org/content/repositories/snapshots"
            } else {
                url "https://oss.sonatype.org/service/local/staging/deploy/maven2"
            }
        }
    }
    publications {
        gpr(MavenPublication) {
            version = System.getenv("RELEASE_VERSION")
            groupId = 'dev.personnummer'

            from components.java
        }
    }
}

Snippet above was shamelessly stolen from this answer

@lazystone Thank you!
Yes, I got that far (although it have to be written slightly different now' a days), but the signing part is pretty much the part that I fail to implement in a decent way... hehe...
The central requires the following things in the archive uploaded: jar, javadocs, source.
The archive (or maybe only jar?) have to be signed with GPG/PGP, I have no issues setting up a key and all that, but the process is very unclear.
It feels like there should be some typ of github action to automate this, but I can't find any at all, which is quite odd in my opinion.

For signing you should probably leverage gradle as well rather than GitHub actions(rule of thumb - process should be independent from GitHub as much as possible :wink:  ).
There is a signing plugin: https://docs.gradle.org/current/userguide/signing_plugin.html
Seems like you can provide signing key and password as an environment variables as well.

Aye, that was my thought too, tried to get that working earlier without luck. But I'll try some more and see if I can make it work!
Thanks for the resources :)

Just awaiting the staged release to be published and this issue will close.
The package is published as dev.personnummer :)

Central OSSRH added a comment - 18 minutes ago
Central sync is activated for dev.personnummer. After you successfully release, your component will be published to Central,  typically within 10 minutes, though updates to search.maven.org can take up to two hours.

So it should be live and working as intended now. Packages are signed, source and docs are included in the release and the CI script is set up to be able to build and deploy with signing and all!
To publish to maven-central was more annoying than I thought it was, but at the least it's fixed now, haha.
Let me know if there are any issues, closing this request now!


New version: Jackett.Jackett version 0.20.4071

Pull request has been created with WinGet Releaser v2 :rocket:
Microsoft Reviewers: Open in CodeFlow

@wingetbot run


Units in the middle of resurrecting still despawn

If a resurrection spell is cast on a dead unit, it should not despawn mid resurrection.

there was no check for isReviving on despawn


Port C++ CAGRA runtime from RAFT

This PR is porting the CAGRA runtime API from RAFT to cuVS.

Changes have been merged into #27. Closing this one for now.


MVC - LogController - rowCount validation

Use int filter in getPost() call
Use variables for default and maximium rows
Set $itemsPerPage to max if over, all, or invalid
Set a default if input is not an integer

This is a follow up to the issue mitigated with 5cb6a89
This PR intends to provide input validation for the rowCount to mitigate any issues with improperly set values in a bootgrid's UIBootgrid options rowCount configuration. It goes a little further than the original catching invalid values like string, array, negative numbers below -1, 0, and also if the rowCount is set above 5000. It will use either the default with invalid input, or 5000 for everything below 1.
Two new variables are set for the maximum, and default number of rows. The max was taken from the original, while the default was taken from https://github.com/opnsense/core/blob/e6518fcd14062e2c62bef43cff82db7acea7689f/src/opnsense/www/js/jquery.bootgrid.js#L1038
This approach also employs the int filter in the getPost() and using the default (10) instead of 9999.
There are two bugs with jquery.bootgrid.js that I found as well, but they don't cause anything to break per se.
"Showing NaN of NaN" at the bottom of the page, from trying to multiply against rowCount when it's not an integer.
https://github.com/opnsense/core/blob/e6518fcd14062e2c62bef43cff82db7acea7689f/src/opnsense/www/js/jquery.bootgrid.js#L431
No row selector button, where rowCountList (rowCount) is checked if it's an array.
https://github.com/opnsense/core/blob/e6518fcd14062e2c62bef43cff82db7acea7689f/src/opnsense/www/js/jquery.bootgrid.js#L548
With rowCount in jquery.bootgrid.js it seems that it's generally assumed that it will be an array with integers, and practically no validation is performed on it.
If this is agreed to be a good approach I could cover the other instances where this input occurs:
src/opnsense/mvc/app/controllers/OPNsense/Base/ApiControllerBase.php:        $itemsPerPage = intval($this->request->getPost('rowCount', 'int', 9999));
src/opnsense/mvc/app/controllers/OPNsense/Diagnostics/Api/FirewallController.php:            $itemsPerPage = $this->request->getPost('rowCount', 'int', 9999);
src/opnsense/mvc/app/controllers/OPNsense/Diagnostics/Api/FirewallController.php:            $itemsPerPage = $this->request->getPost('rowCount', 'int', 9999);
src/opnsense/mvc/app/controllers/OPNsense/Firewall/Api/AliasUtilController.php:        $itemsPerPage = intval($this->request->getPost('rowCount', 'int', -1));
src/opnsense/mvc/app/controllers/OPNsense/IDS/Api/ServiceController.php:            $itemsPerPage = $this->request->getPost('rowCount', 'int', 9999);
src/opnsense/mvc/app/controllers/OPNsense/IDS/Api/SettingsController.php:            $itemsPerPage = $this->request->getPost('rowCount', 'int', 9999);
src/opnsense/mvc/app/controllers/OPNsense/Syslog/Api/ServiceController.php:        $itemsPerPage = intval($this->request->getPost('rowCount', 'int', 9999));

I also noted the use of intval() in one of the cases which is a pretty good alternative, but it does have the possibility of returning 0, which jquery.bootgrid.js doesn't handle well, so I went with this approach which is more explicit and covers a couple more scenarios.
On an aside, would it be practical to move this functionality to the library instead of have it individually throughout the controllers?

let's keep it simple, this https://github.com/opnsense/core/commit/058aedc61e833fdf3d043c6c059312fde58cfcd3 should likely be enough for what we're looking for here.


Use AdaptiveOperationTracker in PutOperation

We have some issues with SimpleOperationTracker with PutOperation. We want to change PutRequestParallelism to 2 and expect that only 2 copies of the same chunk will be sent out to ambry-server replicas. However, this is not what we see from the metrics. Even with parallelism changed to 2, we are still seeing 3 copies of the same chunk sent out to replicas. The reason of this is because of SimpleOperationTracker.
In SimpleOperationTracker, when we send out first two PutRequests, it will block because of the parallelism is 2. But as long as we receive the response from any of these 2 requests, the inflightCount would be decremented, thus we will immediately send out the third request. For that, we end up with 3 requests.
One way to fix this is to set the PutRequestParallelism to 1, in this case, we will send the first request out, wait and receive the first response. Then send the second request out and receive the second response. Assume that both responses are successes, we would stop and acknowledge this PutOperation as successful. But this way, we would drastically increase the latency of PutOperations, thus damage the overall upload performance.
The other way is to have different logic in the OperationTracker, to send out first 2 requests and wait until both responses come back. But this way, if we have a slow replica, then we are damaging the performance as well.
The best solution is to use AdaptiveOperationTracker, so that when we have slow replica, we would send out the third request right away. But before we can use AdaptiveOperationTracker, we have to change some logic in it.
AdaptiveOperationTracker has the same issue as the SimpleOperationTracker, where when the PutRequestParallelism is 2, it will send out the third request as soon as it receives the first response. This PR would fix that logic.

Codecov Report

Merging #1937 (7569edc) into master (26057a4) will decrease coverage by 58.51%.
The diff coverage is 0.00%.

@@              Coverage Diff              @@
##             master    #1937       +/-   ##
=============================================
- Coverage     73.92%   15.40%   -58.52%
+ Complexity     9671     2280     -7391
=============================================
  Files           692      692
  Lines         53271    53321       +50
  Branches       6630     6635        +5
=============================================
- Hits          39378     8216    -31162
- Misses        11812    44629    +32817
+ Partials       2081      476     -1605

Impacted Files
Coverage Δ

...ain/java/com/github/ambry/config/RouterConfig.java
0.00% <0.00%> (-98.02%)
:arrow_down:

.../github/ambry/router/AdaptiveOperationTracker.java
0.00% <0.00%> (-88.58%)
:arrow_down:

.../github/ambry/router/NonBlockingRouterMetrics.java
0.00% <0.00%> (-88.42%)
:arrow_down:

...ain/java/com/github/ambry/router/PutOperation.java
0.00% <0.00%> (-92.70%)
:arrow_down:

.../src/main/java/com/github/ambry/router/Router.java
0.00% <0.00%> (-100.00%)
:arrow_down:

.../src/main/java/com/github/ambry/store/Message.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...rc/main/java/com/github/ambry/named/PutResult.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...rc/main/java/com/github/ambry/quota/QuotaMode.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...rc/main/java/com/github/ambry/quota/QuotaName.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...rc/main/java/com/github/ambry/rest/RestMethod.java
0.00% <0.00%> (-100.00%)
:arrow_down:

... and 548 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 26057a4...7569edc. Read the comment docs.


bugfix: try to new version github actions v2

What issue does this pull request resolve?
Closes #1759
What changes did you make?
Uncomment the website build
Is there anything that requires more attention while reviewing?
Please checkout the issue, I already discussed there

it is still failing - please try running it locally first


[Refactor]: search bar typescript code with best practices 🪄✨

File Name
hooks/useSidebarSearch.ts
Reason for Refactoring the Code

This issue aims to refactor the useSidebarSearch.ts script to improve code maintainability, readability, and type safety.
We want to provide Descriptive variable names, Destructuring and Simplifiying the Code, Etc.

Checklist

[X] I have checked the existing issues.
[X] I have read the Contributing Guidelines.
[X] The changes don't break the code and don't introduce new functionality.
[X] I am willing to work on this issue (optional).
[X] I am a GSSoC23 contributor.

Hey there @rupali-codes, @CBID2, @ujjawaltyagii, @Anmol-Baranwal 👋🏻
Please make sure to assign this issue to me to make the work done, Thank you!

You're assigned @0xabdulkhalid


[Bug]: Workflow scheduler save CTA should be disabled if there are no changes in the form

Is there an existing issue for this?

[X] I have searched the existing issues

Description
On the workflow scheduler screen, the form submit CTA should be disabled if there are no changes to be saved.
Steps To Reproduce

Open workflow scheduler settings and save a schedule.
The CTA is still enabled (should be disabled since there are no new changes to save)

Public Sample App
No response
Environment
Production
Severity
Medium (Frustrating UX)
Issue video log
No response
Version
Self hosted

pr is merged


Extended Learn page to include integrations content

Added Integrations content under the learn section as discussed on Teams.

Looks good!
One question though on URL structure, should we move integrations under the learn section (/learn/integrations/ instead of /integrations)?

Yeah all of these suggestions make sense.

I've moved the integration pages to live under 'learn', this also involved moving the root learn page to live in a learn folder as the index for that root.
I've also rolled out @jst-cyr's suggestion around product based panels, which makes sense. I feel like we might have a too many panels down the track but I dont think thats a problem we should solve till it actually becomes a problem!

Think this looks good to merge now?


Known Issue: Certificates with Validity Duration under 24h failing to Issue

There is currently an issue where certificate requests with validity duration < 24h will fail to issue because of a division by 24 and cast to int64, resulting in 0 for the validity field. This results in an api error from PCA because PCA requires validity is not 0.
Please see #69 for more details.
While we work to cut a release to fix this change (#70), please use validity durations greater than 24h.

A new release v1.0.0 has been cut that resolves this issue.


"az storage blob directory access show" throws error for missing permissions

"az storage blob directory access show" throws error for missing permissions even though they are set on the SP that executes this command. Test for real user results in same error message
Command Name
az storage blob directory access show Extension Name: storage-preview. Version: 0.2.10.
Errors:
The command failed with an unexpected error. Here is the traceback:

'CommandResultItem' object is not iterable
Traceback (most recent call last):
  File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 212, in invoke
    self.output.out(cmd_result, formatter=formatter, out_file=out_file)
  File "/opt/az/lib/python3.6/site-packages/knack/output.py", line 132, in out
    output = formatter(obj)
  File "/opt/az/lib/python3.6/site-packages/knack/output.py", line 38, in format_json
    input_dict = dict(result) if hasattr(result, '__dict__') else result
TypeError: 'CommandResultItem' object is not iterable

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

az login --service-principal (redacted)
az storage blob directory access show -d {} -c {} --account-name {}

Expected Behavior
The permissions of the directory are printed because the permissions are set on the storage account and the container too. With an user, the same error gets printed
Environment Summary
Linux-5.3.0-46-generic-x86_64-with-debian-buster-sid
Python 3.6.5
Shell: bash

azure-cli 2.0.81 *

Extensions:
storage-preview 0.2.10

Additional Context

add to S170

add to S172

Hi @TobiTh, az storage blob directory access show is for ADLS Gen2 command. Currently we already support it in azure-cli not extension. You can use az storage fs access to set/show permissions  of files/directories in your ADLS Gen2 account.
By the way, for the issue you reported, can you share more information with --debug in your command?

Hi @Juliehzl the az storage fs access show does work, but only when I add the account-key param.
$ az storage fs access show --account-name _redacted_ -f container -p path
Operation returned an invalid status 'Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.'
ErrorCode:AuthenticationFailed

$ az storage fs access show --account-name _redacted_ -f container -p path --account-key _redacted_
{
  "acl": "user::rwx,user:_redacted_:rwx,group::r-x,mask::rwx,other::---,default:user::rwx,default:user:_redacted_:rwx,default:group::r-x,default:mask::rwx,default:other::---",
  "group": "_redacted_",
  "owner": "$superuser",
  "permissions": "rwxrwx---+"
}

@Juliehzl Here is the output of the storage extension command with --debug:
$ az storage blob directory access show --account-name redacted --container-name container --directory-path path --debug
Command arguments: ['storage', 'blob', 'directory', 'access', 'show', '--account-name', 'redacted', '--container-name', 'container', '--directory-path', 'path', '--debug']
Event: Cli.PreExecute []
Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7f2cfcd94378>, <function OutputProducer.on_global_arguments at 0x7f2cfcaf3378>, <function CLIQuery.on_global_arguments at 0x7f2cfcb1c598>]
Event: CommandInvoker.OnPreCommandTableCreate []
Installed command modules ['acr', 'acs', 'advisor', 'ams', 'apim', 'appconfig', 'appservice', 'aro', 'backup', 'batch', 'batchai', 'billing', 'botservice', 'cdn', 'cloud', 'cognitiveservices', 'configure', 'consumption', 'container', 'cosmosdb', 'deploymentmanager', 'dla', 'dls', 'dms', 'eventgrid', 'eventhubs', 'extension', 'feedback', 'find', 'hdinsight', 'interactive', 'iot', 'iotcentral', 'keyvault', 'kusto', 'lab', 'managedservices', 'maps', 'monitor', 'natgateway', 'netappfiles', 'network', 'policyinsights', 'privatedns', 'profile', 'rdbms', 'redis', 'relay', 'reservations', 'resource', 'role', 'search', 'security', 'servicebus', 'servicefabric', 'signalr', 'sql', 'sqlvm', 'storage', 'vm']
Loaded module 'acr' in 0.008 seconds.
Loaded module 'acs' in 0.004 seconds.
Loaded module 'advisor' in 0.001 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'ams' in 0.004 seconds.
Loaded module 'apim' in 0.001 seconds.
Loaded module 'appconfig' in 0.002 seconds.
Loaded module 'appservice' in 0.008 seconds.
Loaded module 'aro' in 0.003 seconds.
Loaded module 'backup' in 0.003 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'batch' in 0.023 seconds.
Loaded module 'batchai' in 0.002 seconds.
Loaded module 'billing' in 0.001 seconds.
Loaded module 'botservice' in 0.002 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'cdn' in 0.004 seconds.
Loaded module 'cloud' in 0.001 seconds.
Loaded module 'cognitiveservices' in 0.002 seconds.
Loaded module 'configure' in 0.002 seconds.
Loaded module 'consumption' in 0.002 seconds.
Loaded module 'container' in 0.001 seconds.
Loaded module 'cosmosdb' in 0.005 seconds.
Loaded module 'deploymentmanager' in 0.002 seconds.
Loaded module 'dla' in 0.004 seconds.
Loaded module 'dls' in 0.003 seconds.
Loaded module 'dms' in 0.003 seconds.
Loaded module 'eventgrid' in 0.002 seconds.
Loaded module 'eventhubs' in 0.002 seconds.
Loaded module 'extension' in 0.001 seconds.
Loaded module 'feedback' in 0.000 seconds.
Loaded module 'find' in 0.000 seconds.
Loaded module 'hdinsight' in 0.002 seconds.
Loaded module 'interactive' in 0.001 seconds.
Loaded module 'iot' in 0.004 seconds.
Loaded module 'iotcentral' in 0.001 seconds.
Loaded module 'keyvault' in 0.004 seconds.
Loaded module 'kusto' in 0.001 seconds.
Loaded module 'lab' in 0.003 seconds.
Loaded module 'managedservices' in 0.001 seconds.
Loaded module 'maps' in 0.002 seconds.
Loaded module 'monitor' in 0.006 seconds.
Loaded module 'natgateway' in 0.001 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'netappfiles' in 0.002 seconds.
Loaded module 'network' in 0.028 seconds.
Loaded module 'policyinsights' in 0.002 seconds.
Loaded module 'privatedns' in 0.003 seconds.
Loaded module 'profile' in 0.001 seconds.
Loaded module 'rdbms' in 0.006 seconds.
Loaded module 'redis' in 0.002 seconds.
Loaded module 'relay' in 0.002 seconds.
Loaded module 'reservations' in 0.001 seconds.
Loaded module 'resource' in 0.007 seconds.
Loaded module 'role' in 0.004 seconds.
Loaded module 'search' in 0.001 seconds.
Loaded module 'security' in 0.002 seconds.
Loaded module 'servicebus' in 0.003 seconds.
Loaded module 'servicefabric' in 0.002 seconds.
Loaded module 'signalr' in 0.001 seconds.
Loaded module 'sql' in 0.007 seconds.
Loaded module 'sqlvm' in 0.002 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'storage' in 0.054 seconds.
Loaded module 'vm' in 0.013 seconds.
Loaded all modules in 0.267 seconds. (note: there's always an overhead with the first module loaded)
Found 1 extensions: ['storage-preview']
Extension compatibility result: is_compatible=True cli_core_version=2.6.0 min_required=2.0.67 max_required=None
Extensions directory: '/home/redacted/.azure/cliextensions/storage-preview'
Event: CommandLoader.OnLoadCommandTable []
Loaded extension 'storage-preview' in 0.012 seconds.
Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7f2cfa1406a8>]
az_command_data_logger : command args: storage blob directory access show --account-name {} --container-name {} --directory-path {} --debug
metadata file logging enabled - writing logs to '/home/redacted/.azure/commands'.
Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x7f2cfa1167b8>]
Event: CommandLoader.OnLoadArguments []
Event: CommandInvoker.OnPostArgumentLoad []
Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x7f2cfa116bf8>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x7f2cfa116d90>]
Event: CommandInvoker.OnCommandTableLoaded []
Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0x7f2cf97af620>]
Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7f2cfcaf3400>, <function CLIQuery.handle_query_parameter at 0x7f2cfcb1c620>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x7f2cfa116c80>, <function handler at 0x7f2cf96110d0>]
az_command_data_logger : extension name: storage-preview
az_command_data_logger : extension version: 0.2.10
Command group 'storage blob directory access' is in preview. It may be changed/removed in a future release.
Getting data service client service_type=BlockBlobService
azext_storage_preview.vendored_sdks.azure_adls_storage_preview.v2019_02_02_preview.common._auth : String_to_sign=HEAD

x-ms-client-request-id:ac1217ac-b202-11ea-a99b-00155d00c375
x-ms-date:Fri, 19 Jun 2020 07:58:33 GMT
x-ms-version:2019-02-02
/redacted/container/path
action:getAccessControl
upn:False
azext_storage_preview.vendored_sdks.azure_adls_storage_preview.v2019_02_02_preview.common.storageclient : Client-Request-ID=ac4bbea8-b202-11ea-a99b-00155d00c375 Outgoing request: Method=HEAD, Path=/temporary/gkk, Query={'action': 'getAccessControl', 'upn': 'False', 'timeout': None}, Headers={'x-ms-lease-id': None, 'If-Modified-Since': None, 'If-Unmodified-Since': None, 'If-Match': None, 'If-None-Match': None, 'x-ms-version': '2019-02-02', 'User-Agent': 'Azure-Storage/2.1.0-2.1.0 (Python CPython 3.6.5; Linux 4.19.104-microsoft-standard) AZURECLI/2.6.0 (DEB)', 'x-ms-client-request-id': 'ac1217ac-b202-11ea-a99b-00155d00c375', 'x-ms-date': 'Fri, 19 Jun 2020 07:58:33 GMT', 'Authorization': 'REDACTED'}.
urllib3.connectionpool : Starting new HTTPS connection (1): redacted.dfs.core.windows.net:443
urllib3.connectionpool : https://redacted.dfs.core.windows.net:443 "HEAD /container/path?action=getAccessControl&upn=False HTTP/1.1" 403 0
azext_storage_preview.vendored_sdks.azure_adls_storage_preview.v2019_02_02_preview.common.storageclient : Client-Request-ID=ac4bbea8-b202-11ea-a99b-00155d00c375 Receiving Response: Server-Timestamp=Fri, 19 Jun 2020 07:58:32 GMT, Server-Request-ID=dd1769ad-601f-0070-4a0f-4624c1000000, HTTP Status Code=403, Message=Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature., Headers={'server': 'Windows-Azure-HDFS/1.0 Microsoft-HTTPAPI/2.0', 'x-ms-error-code': 'AuthenticationFailed', 'x-ms-request-id': 'dd1769ad-601f-0070-4a0f-4624c1000000', 'x-ms-version': '2019-02-02', 'x-ms-client-request-id': 'ac1217ac-b202-11ea-a99b-00155d00c375', 'date': 'Fri, 19 Jun 2020 07:58:32 GMT'}.
azext_storage_preview.vendored_sdks.azure_adls_storage_preview.v2019_02_02_preview.common.storageclient : Client-Request-ID=ac4bbea8-b202-11ea-a99b-00155d00c375 Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Fri, 19 Jun 2020 07:58:32 GMT, Server-Request-ID=dd1769ad-601f-0070-4a0f-4624c1000000, HTTP status code=403, Exception=Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. ErrorCode: AuthenticationFailed.
azext_storage_preview.vendored_sdks.azure_adls_storage_preview.v2019_02_02_preview.common.storageclient : Client-Request-ID=ac4bbea8-b202-11ea-a99b-00155d00c375 Retry policy did not allow for a retry: Server-Timestamp=Fri, 19 Jun 2020 07:58:32 GMT, Server-Request-ID=dd1769ad-601f-0070-4a0f-4624c1000000, HTTP status code=403, Exception=Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. ErrorCode: AuthenticationFailed.

You do not have the required permissions needed to perform this operation.
Depending on your operation, you may need to be assigned one of the following roles:
    "Storage Blob Data Contributor (Preview)"
    "Storage Blob Data Reader (Preview)"
    "Storage Queue Data Contributor (Preview)"
    "Storage Queue Data Reader (Preview)"

If you want to use the old authentication method and allow querying for the right account key, please use the "--auth-mode" parameter and "key" value.

Event: CommandInvoker.OnFilterResult []
cli.azure.cli.core.util : The command failed with an unexpected error. Here is the traceback:

The command failed with an unexpected error. Here is the traceback:

cli.azure.cli.core.util : 'CommandResultItem' object is not iterable
Traceback (most recent call last):
  File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 221, in invoke
    self.output.out(cmd_result, formatter=formatter, out_file=out_file)
  File "/opt/az/lib/python3.6/site-packages/knack/output.py", line 149, in out
    output = formatter(obj)
  File "/opt/az/lib/python3.6/site-packages/knack/output.py", line 39, in format_json
    input_dict = dict(result) if hasattr(result, '__dict__') else result
TypeError: 'CommandResultItem' object is not iterable
'CommandResultItem' object is not iterable
Traceback (most recent call last):
  File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 221, in invoke
    self.output.out(cmd_result, formatter=formatter, out_file=out_file)
  File "/opt/az/lib/python3.6/site-packages/knack/output.py", line 149, in out
    output = formatter(obj)
  File "/opt/az/lib/python3.6/site-packages/knack/output.py", line 39, in format_json
    input_dict = dict(result) if hasattr(result, '__dict__') else result
TypeError: 'CommandResultItem' object is not iterable
cli.azure.cli.core.util :
To open an issue, please run: 'az feedback'

To open an issue, please run: 'az feedback'
Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7f2cfa1408c8>]
az_command_data_logger : exit code: 1
telemetry.save : Save telemetry record of length 2662 in cache
telemetry.check : Negative: The /home/redacted/.azure/telemetry.txt was modified at 2020-06-19 09:54:49.340491, which in less than 600.000000 s
command ran in 0.515 seconds.

Hi @TobiTh, to use --auth-mode login, you will need to grant RBAC role according to https://docs.microsoft.com/en-us/azure/storage/common/storage-auth-aad-rbac-portal?toc=/azure/storage/blobs/toc.json#rbac-roles-for-blobs-and-queues.
For more information about ACLs, you can refer to https://docs.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-access-control#role-based-access-control.

I just added such support in az storage fs access  in next release with PR https://github.com/Azure/azure-cli/pull/14506. you could try to use with edge build now or stable version after 08/04/2020.
So I will close the issue now and if you still have problem, feel free to create a new one or reopen it.


Unrecognized firmware version still shows after successfully creating volume, then running syno_hdd_db.sh - normal?

Successfully used this script to create a SHR pool with two new drives a couple of days ago. I ran the syno_hdd_db.sh script after, which got rid of the general drive warning, but noticed it still shows a message about unrecognized firmware - is this to be expected? It doesn't ping me any notifications about it, just shows in the HDD/SSD page in Storage Manager.

On another note, as you can see above, one of my drives has already crashed in the pool. This is a 3 day old drive ha ha! I'm assuming I just got a dud drive, but thought I'd mention that as well anyway in case you've heard of this happening at all when creating these volumes on 'unsupported' models. Thanks for the awesome work btw :)
Edit: the output from the db update script is as follows (I've done a reboot a couple times past few days and scheduled it also):
nas-ds920:~$ sudo -i "/volume1/homes/stef/scripts/syno_hdd_db.sh" --noupdate                               Password:
Synology_HDD_db v2.2.47
DS920+ DSM 7.2-64561
Using options: --noupdate

HDD/SSD models found: 2
WD160EDFZ-11AFWA0,0A81
WD160EDGZ-11B2DA0,0A85

M.2 drive models found: 1
CT500P3SSD8,P9CR30A

No M.2 cards found

No Expansion Units found

Added WD160EDFZ-11AFWA0 to ds920+_host_v7.db
Added WD160EDGZ-11B2DA0 to ds920+_host_v7.db
Added CT500P3SSD8 to ds920+_host_v7.db

Support disk compatibility already enabled.

Support memory compatibility already enabled.

M.2 volume support already enabled.

Disabled drive db auto updates.

DSM successfully checked disk compatibility.

You may need to reboot the Synology to see the changes.

Firstly, if you ran the script previously with an earlier version of DSM and then upgraded to DSM 7.2 do NOT run the script with the --restore option. I'm working a fix for the issue that causes.
It does look like you got 1 faulty NVMe drive.
I wonder if DSM sees the NVMe model as "Crucial CT500P3SSD8" instead of just "CT500P3SSD8".
Can you attach a copy of your ds920+_host_v7.db file

sudo -i cp "/var/lib/disk-compatibility/ds920+_host_v7.db" "/volume1/homes/stef/scripts/ds920+_host_v7.db"
Then zip the db file and attach it to your reply.

You could try running the script with:  --noupdate --force

Hi Dave, my English is not very good, so I can only rely on Google Translate.
I have a little problem,
After I used the Synology_enable_M2_volume script, my M2/HDD firmware appeared Unrecognized firmware version, I used sudo -i /path-to-script/syno_hdd_db.sh --noupdate, my 8T HDD and M2 2T are normal, that is The 4T Samsung SATA HDD still has this prompt, is there a way to solve it?
Remark:
NAS: synology DS920+
system: 7.2-64561

Below is the command output
Synology_HDD_db v2.2.47
DS920+ DSM 7.2-64561
Using options: -noupdate -showedits
HDD/SSD models found: 2
SSD 870 EVO 4TB,3B6Q
ST8000NM017B-2TJ103,SN02
M.2 drive models found: 1
Samsung SSD 970 EVO Plus 2TB,2B2QEXM7
No M.2 cards found
No Expansion Units found
SSD 870 EVO 4TB already exists in ds920+_host_v7.db
ST8000NM017B-2TJ103 already exists in ds920+_host_v7.db
Samsung SSD 970 EVO Plus 2TB already exists in ds920+_host_v7.db
Support disk compatibility already enabled.
Support memory compatibility already enabled.
M.2 volume support already enabled.
Drive db auto updates already disabled.
"SSD 870 EVO 4TB": {
  "3B6Q": {
    "compatibility_interval": [
      {
        "compatibility": "support",
        "not_yet_rolling_status": "support",
        "fw_dsm_update_status_notify": false,
        "barebone_installable": true
      }
    ]
  },
  "default": {

"ST8000NM017B-2TJ103": {
  "SN02": {
    "compatibility_interval": [
      {
        "compatibility": "support",
        "not_yet_rolling_status": "support",
        "fw_dsm_update_status_notify": false,
        "barebone_installable": true
      }
    ]
  },
  "default": {

"Samsung SSD 970 EVO Plus 2TB": {
  "2B2QEXM7": {
    "compatibility_interval": [
      {
        "compatibility": "support",
        "not_yet_rolling_status": "support",
        "fw_dsm_update_status_notify": false,
        "barebone_installable": true
      }
    ]
  },
  "default": {

DSM successfully checked disk compatibility.
You may need to reboot the Synology to see the changes.

Try the options: --noupdate --showedits --force
Can you run drive_info.sh and reply with the output:
https://github.com/007revad/Synology_HDD_db/blob/test/drive_info.sh

@007revad
Thank you for your reply
I tried the following options: --noupdate --showedits --force, it still doesn't work, but I seem to find where the problem is.
The script will change the serial number of my Samsung SATA SSD 4T to 3B6Q, but the serial number displayed in the DSM is SVT03B6Q, so I directly modify the .db file with the vi command and change it to SVT03B6Q. After restarting, the error disappears and it looks normal up.
Is it possible to directly modify it with the vi command, and will there be any risks or failures? If there is no risk, so be it.
The following is the output of the drive_info.sh command:
nvme0n1
NVMe Model: 'Samsung SSD 970 EVO Plus 2TB'
NVMe Model: 'Samsung SSD 970 EVO Plus 2TB'
NVMe Firmware: '2B2QEXM7'
NVMe Firmware: '2B2QEXM7'
sata1
Model: 'SSD'
Firmware: '3B6Q'
sata2
Model: 'ST8000NM017B-2TJ103'
Firmware: 'SN02'

Good work.
I should have noticed that your screenshot showed SVT03B6Q and the command output showed 3B6Q.
Editing the db file with vi is okay.

Firstly, if you ran the script previously with an earlier version of DSM and then upgraded to DSM 7.2 do NOT run the script with the --restore option. I'm working a fix for the issue that causes.
It does look like you got 1 faulty NVMe drive.
I wonder if DSM sees the NVMe model as "Crucial CT500P3SSD8" instead of just "CT500P3SSD8". Can you attach a copy of your ds920+_host_v7.db file

sudo -i cp "/var/lib/disk-compatibility/ds920+_host_v7.db" "/volume1/homes/stef/scripts/ds920+_host_v7.db"
Then zip the db file and attach it to your reply.

You could try running the script with: --noupdate --force

Here's the DB file.  After zipping up I did try the --force as well but don't think it changed anything!
Thanks :)
ds920+_host_v7.zip

Also - have a replacement drive for the crashed one coming today. Will I be able to just put it in and do a repair through DSM, or will I need to re-run the volume script?
Thanks.

Will I be able to just put it in and do a repair through DSM, or will I need to re-run the volume script?

You will be able to do a repair from storage manager in DSM. You don't to run the script again.
I'll have a look at your ds920+_host_v7.db later.

Will I be able to just put it in and do a repair through DSM, or will I need to re-run the volume script?

You will be able to do a repair from storage manager in DSM. You don't to run the script again.
I'll have a look at your ds920+_host_v7.db later.

Awesome, it let me repair the volume immediately and back up and running with two :). Had to run the db update script again to get rid of unrecognized drive warning, now just back to the unrecognized firmware note against each.

Another update - my replacement drive has now also degraded. Same M2 slot (2). Wondering if there's something buggery about this drive model in particular (Crucial P3) or if that M2 slot on my NAS is screwy (never used it before) or if something is unhappy with the SHR raid configuration on DS920 or something else... Any recommendations? I'm thinking of trying another brand - the performance of Plex is so much better on the SSDs I can't face going back to HDD, but I can't keep replacing drives every 3 days - it's also a weird pattern they're dying in the same timeframe!
Thanks.

Come to the conclusion the drives are just not up to the job anyway. Going to try another model, so I'll close this for now and reopen if run into any issues with the scripts next time
Thanks for your support and great work!

After reading the comments to your post on reddit I'm glad I didn't buy any Crucial NVMe drives when I bought memory from Crucial.

After reading the comments to your post on reddit I'm glad I didn't buy any Crucial NVMe drives when I bought memory from Crucial.

I know right 🤣 to be honest, it's put me off their RAM too as that was next on my list, but hopefully, that's different (!).

it's put me off their RAM too as that was next on my list, but hopefully, that's different (!

I've had no problem with their RAM. I bought 2x 16GB ECC RAM from Crucial for my DS1821+ (which was actually Micron RAM) and 4GB of Crucial RAM for my Asustor.
I did look at their NVMe drives at the same time but I was turned off by the specs and because it was TLC.


Additional Support for bpy.types.ShaderNodeGroup

bpy.types.ShaderNodeGroup are a really handy way to creating groups of preset shader groups for any given scene in Blender. As of the moment however the GLTF Exporter seems to only check for a node group that has the name "gltf Metallic Roughness" which apparently is deprecated gltf2_blender_get.get_socket_or_texture_slot_old. Instead it should support what ever node group a user creates (to the extent to gltf2 spec)
Describe the solution you'd like
Generate a wrapper from a group input to the group output using the same methods used as in export.
Utilize the wrapper then to bind the textures to gltf2 spec.
Describe alternatives you've considered
Use The shader groups input socket names as binding instead of jumping into checking how the shader group is built.
However, the naming schema should then be documented for other addon or node group creators. And this puts alot of trust into the addon developers into meeting the spec.
Additional context
I am developing a addon toolset, that creates a shader group in any scene on load to make it easier for new blender users to bind classical values to materials (Diffuse, Roughness, Alpha, Metallic, Emission) all in a single bundle to match the shading in a Game Engine (for one to one previews in Blender).
This allows them to get eased in, instead of having directly jump into the Shader Editor and start creating more and more Shaders and combing them to get basic functionality.
This issue mostly to start a conversation on the subject, and to see if there are plans of it:
Will submit an exploratory PR later down the line, after I understand the gltf exporter a bit more and how it ticks.

This exporter primarily expects a Principled BSDF node, which aligns very well with the glTF material model. See https://docs.blender.org/manual/en/dev/addons/io_gltf2.html.
For properties that aren't supported by Principled BSDF, we've allowed custom nodes named glTF Metallic Roughness in the past, which we've recently renamed to glTF Settings. I think it is reasonable to require that the node group be named something specific to indicate the intent.

Principled BSDF does not support:

Alpha / Opacity
Emissive

Most common ShaderGroup setup would be PFSDF + Alpha Shader with a mixer. This will especially come very apparent when Opacity support to the gltf exporter is implemented.
To allow everything to atleast preview correctly in blender, looks like this
That sort of entire setup is absolutely a just a pain to keep on re-setuping per object instead of a simple One GroupNode that you can use across the Blender Scene in different objects. Alternatively you can always just duplicate the materials, but thats still just copying the setup instead of using a feature.
In anycase, if looking at intent, why instead of using the Group Name to check intent, look at the input nodes for the group node instead? document a naming schema that matches gltf2 definitions, and use that as well.

@Menithal I can't see the images you've uploaded, can you drag-and-drop them on the issue here?
Also, as @donmccurdy mentioned, support for node groups is included here.  In spite of the function named _old, this functionality isn't going away.  I think the group name is important to check, because multiple node groups can have similarly-named inputs with different functionality.  We allow both the name glTF Metallic Roughness and glTF Settings now, and I recommend the latter.
Another option
One other option I've been thinking about lately, but I don't know how people would feel about this:  It might be possible to add Blender commands to apply glTF-compatible material modifications on demand.  There could potentially be new commands like glTF: Add Metallic/Roughness that would wire up the Separate RGB node, and glTF: Add Emissive Map that would wire in the Emissive BSDF node with a blank Image Texture node and an Add Shader.  These would modify the currently active material to add support for the requested map to export to glTF.  Thoughts on this (from anyone)?

@emackey  Fixed the Links they are just stuff similar node setup to the ones shown in the blender documentation:
I cannot seem to find any check for gltf Settings in Group name currently being looked for in the current master or in the current release of the Blender 2.8 (and the ´gltf2-blender-io` on there) could I get a pointer on where this is done, or is that just "upcoming plans"
I am honestly more into the Node Group approach for general purpose.

because multiple node groups can have similarly-named inputs with different functionality.

What if instead you look if the name prefix or postfix containing "gltf" in one format or another instead (such as Shader Group (gltf)? and then bind the variable names. Otherwise it feels redundant to have a single empty group named gltf Settings that accepts the texture inputs but is just a stub. That just feels like duplication of work.
Sure It can be done, but I just love hearing complaints from end users saying something doesn't work because they forgot a step. It can automated to bind on-save though, but at best it still is clunky and would need to remind end users to save prior to export.

While adding new commands is prospective, the main issue arises from that is the amount of edge cases you'd have to look into when modifying user's materials. Are you sure you'd want to go that route though?
You'd also have to take account Add Alpha for the the Blender preview, even if, from what I understand, the alpha is from the Diffuse Texture alpha by default.

Probably more context: Having tools to add material nodes, doesn't solve the context of the problem:
A need for a ShaderGroup support for addon developers (like me) who create ShaderGroups in their addons to make pipelines easier for the end users or make the shaders match what ever spec they have elsewhere.
Otherwise add on devs would just end up duplicating code and porting it over to modify it, and that's messy and basically a fork, which this early, should not yet be done..

The check for group name happens here:
https://github.com/KhronosGroup/glTF-Blender-IO/blob/9a52b6692f8b52aef115cfa06f0822911b766c40/addons/io_scene_gltf2/blender/exp/gltf2_blender_get.py#L101
with the actual glTF Settings name itself defined here:
https://github.com/KhronosGroup/glTF-Blender-IO/blob/9a52b6692f8b52aef115cfa06f0822911b766c40/addons/io_scene_gltf2/blender/com/gltf2_blender_material_helpers.py#L61
This code is only about 2 weeks old at this point, you need a recent build to get it.
The node itself doesn't need to be a stub.  The inside of the node can plug this all into any Blender nodes you like, and output to a material node on the other side.  Some examples of this can be found in the pbr_node folder, which is part of the test suite here.

Here's a screenshot of pbr_node with some stuff plugged in:

Principled BSDF does not support:

Alpha / Opacity
Emissive

Our first priority should be to support these features through methods native to Blender 2.8. For example, we can export emissives with an Emission node now, and could get alpha from a Transparent BSDF node in the future. In general, this exporter should not be training Blender users to do weird things unless absolutely necessary. For baked AO, unfortunately, a hack was necessary.

It might be possible to add Blender commands to apply glTF-compatible material modifications on demand. There could potentially be new commands like glTF: Add Metallic/Roughness that would wire up the Separate RGB node, and glTF: Add Emissive Map that would wire in the Emissive BSDF node with a blank Image Texture node and an Add Shader.

Any commands we add are enabled by default in Blender 2.8, and we'd want to be pretty conservative about adding that sort of thing... this could be done as a separate addon, I guess, or enabled with an opt-in "Enable shortcuts" setting. Not sure if default addons can have opt-in settings?

The node itself doesn't need to be a stub. The inside of the node can plug this all into any Blender nodes you like, and output to a material node on the other side.

Thanks, I am gonna look into tweaking the detection slightly. Specifically https://github.com/KhronosGroup/glTF-Blender-IO/blob/f12b8205284d13ebcd294abc525ff54d8d4c1663/addons/io_scene_gltf2/blender/exp/gltf2_blender_get.py#L101
I was thinking that may be the name detection instead should just be looking for "gltf" instead of 'glTF Metallic Roughness'. it is unlikely for a user to have multiple shader nodes with gltf in them. will submit a PR for it once I get going.
That way blender users can use other names than just "gLTF Metallic Roughness" or "gLTF Settings" for bindings, and add-on devs create ease-of-access shader groups like:

Simplified gLTF
Advanced gLTF,
CustomName gltf.

Principled BSDF material has since added both Alpha and Emission. Closing this issue.


how to remove Expo

how to remove Expo
Expo is extremely slow.

I, personally, opened a new project and move all the project files from kittenTricks to the new project, also installed all the npm packages except expo. The only thing that depends on the expo was Gradient Button which I don't use in my project, but if you want to use it, you can easily find a replacement for it.

Hi @esdonmez, just delete "expo" from "package.json" before doing "npm install". I think it's easier =)

@famoustunde I just used a normal button but you can use react-native-linear-gradient package.

Thanks

Hey @esdonmez, I'm having a hard time getting this to work when move over the project files to a new react native vanilla project.
May be because something different in the react native app i just generated?
Can you please let me know if you had to change any versions?

Got it working by re-doing it carefully.

Inti a new app
Copy over screens and components, package.json
Remove Expo dep
Remove Gradientbutton refs

Hi @prasadpilla. Sorry for late reply. I'm glad you solved your problem.
@msouidi309 It would be easier but I ran project with expo first, after a while I wanted to remove it from the project but I'd some problems with it. Finally, I decided to create a clean project. :)

Hi @esdonmez, when i remove expo file from package.json, having lot of errors. let me know anyone which file's are removed from the expo.

@esdonmez i copied all the files over to a new project and removed expo dependencies, but i get this error
Invariant Violation: The navigation prop is missing for this navigator. In  react-navigation 3 you must set up your app container directly. More info:  https://reactnavigation.org/docs/en/app-containers.html
any solutions?

@jafar690 Hello, you can follow this thread to solve that issue. (Link below)
From my understanding, this is simply caused by the  various changes in React Navigation V 3 expecially the explicit app container required for the root navigator. I hope this helps
https://stackoverflow.com/questions/53367195/invariant-violation-the-navigation-prop-is-missing-for-this-navigator

@famoustunde thanks, also do you know how i can use the Expo Font package in a react init application?

@famoustunde thanks, also do you know how i can use the Expo Font package in a react init application?
@jafar690
To be honnest with I have not been able to do this by myself, but I have not tried in long time. Maybe this information could help you. https://stackoverflow.com/questions/44748213/convert-expo-project-to-original-react-native-project?rq=1

I will give it a try later on and feedback on how to use the Expo font package.

In case it is still actual, we've released v2.0 built on top of Expo SDK 33. Check this out! :fire:


Interpolation within interpolation

polished version: 2.1.1
JSS-in_CSS library and version: gatsby-plugin-styled-components 3.0.0

Mixin/Helper/Shorthand you were using and how you were using it:
const borderBottom = css`
  border-bottom: 2px solid ${lighten(0.3, `${props => props.theme.main.blue}`)};
`

What you are seeing:
Uncaught Error: Couldn't parse the color string. Please provide the color as a string in hex, rgb, rgba, hsl or hsla notation.
What you expected to see:
I expect the color string to be parsed correctly. Am I missing something? I can confirm ${props => props.theme.main.blue} returns correctly a hex value.

@ivorpad It is just a formatting issue:
const borderBottom = css
border-bottom: 2px solid ${props => lighten(0.3, props.theme.main.blue)};
``
https://codesandbox.io/s/64xvkrowvk


Check algorithms chapter for formatting and update text

Added material on high scores at start. Other minor improvements in text made.
commit 414fa468319bd26b467211c53880736959f37ddf


Error on Vercel

I'm receiving this error using this package on Vercel (Node 14):
ERROR	Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /var/task/node_modules/unicode-emoji/index.js
require() of ES modules is not supported.

Sorry for the delay, I missed the notification for this issue 😣
This package is using the new import syntax from ES6, you cannot use it with require.
The new import syntax is fully supported by Node.js since the v14, so you should not have any problem to do it.

A solution to your problem if your project is using CommonJS instead of ES Module, would be to use dynamic import (as basic import is only supported inside ES Module) :
const unicodeEmoji = await import('unicode-emoji');

Although you'll need to import it inside an async function, as top-level await is not supported for CommonJS.


HBASE-24651 release script utils should set local user when GPG_KEY is defined.

If the release manager has multiple private keys defined and wants to use one for running a release that isn't configured to be the default then currently the release scripts will use the selected gpg key when doing the test signing but will fall back to using whatever is configured as the default when signing the source and binary tarballs.
this PR changes when we define the local user arg for gpg so that it is defined whenever we have GPG_KEY defined. tested both using do-release-docker.sh and directly with do-release.sh.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 30s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+0 :ok:
mvndep
0m 22s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  8s
Maven dependency ordering for patch

_ Other Tests _

1m 52s

Subsystem
Report/Notes

Docker
Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1992/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/1992

Optional Tests

uname
Linux 7df81aebdfd5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 47742b2083

Max. process+thread count
49 (vs. ulimit of 12500)

modules
C:  U:

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1992/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 43s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+0 :ok:
mvndep
0m 24s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  9s
Maven dependency ordering for patch

_ Other Tests _

2m 14s

Subsystem
Report/Notes

Docker
Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1992/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/1992

Optional Tests

uname
Linux 669903a78066 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 47742b2083

Max. process+thread count
55 (vs. ulimit of 12500)

modules
C:  U:

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1992/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 49s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+0 :ok:
shelldocs
0m  0s
Shelldocs was not available.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+0 :ok:
mvndep
0m 24s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  9s
Maven dependency ordering for patch

+1 :green_heart:
shellcheck
0m  2s
There were no new shellcheck issues.

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

_ Other Tests _

+0 :ok:
asflicense
0m  0s
ASF License check generated no output?

2m 32s

Subsystem
Report/Notes

Docker
Client=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1992/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/1992

Optional Tests
dupname asflicense shellcheck shelldocs

uname
Linux b26492ed3eec 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 47742b2083

Max. process+thread count
48 (vs. ulimit of 12500)

modules
C:  U:

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1992/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) shellcheck=0.4.6

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.


Preventing potential native memory leak.

ServerWebExchangeUtils.cacheRequestBodyAndRequest method is public access, so it is able to call anywhere, anytime.
But if executed twice, previous DataBuffer reference is missed without release.
ServerWebExchangeUtils.cacheRequestBodyAndRequest is not safe now.
On many positions, CACHED_REQUEST_BODY_ATTR is checked before call.
But this structure can make human error. The checking machanism have to be inside of this function for preventing native memory leak.

Is this commit used?

Is this fixing not merged to 4.0.X version?


Provide a specialized error on top-level await and --module node12

Under node12, a top-level await results in
Top-level 'await' expressions are only allowed when the 'module' option is set to 'es2022', 'esnext', 'system', or 'nodenext', and the 'target' option is set to 'es2017' or higher.

This error message has too much text. We should consider specializing these cases for

ES modules < es2022
Node modules < nodenext

The module setting '{0}' does not support top-level 'await' expressions. Consider switching to '{1}'.

I thought "module": "es2020" allowed top-level await.  Did that change?

import() is in module: es2020, but top-level await hasn't technically been added to a latest-ratified standard. When it does, it will likely be ES2022.

The target setting 'es5' does not support top-level 'await' expressions. Consider switching to 'es2017'.

I believe that unless you're using system for your module target, the target has to be es2022 or later. Very confusing, but someone else may be able to correct me.

This is the current check for the message.
(moduleKind !== ModuleKind.ES2022 && moduleKind !== ModuleKind.ESNext && moduleKind !== ModuleKind.System && !(moduleKind === ModuleKind.NodeNext && getSourceFileOfNode(node).impliedNodeFormat === ModuleKind.ESNext)) || languageVersion < ScriptTarget.ES2017

So if someone is using module: ES2015 and target:ES5, should they get two error messages?
(I'm not sure if this is a valid scenario or not 😅)
Based on the current error message, My understanding is

GivenModule
GivenTarget
SuggestedModule
SuggestedTarget

>= ES2022
>= ES2017
-
-

System
>= ES2017
-
-

NodeNext
>= ES2017
-
-

< ES2022
>= ES2017
ES2022
-

Node12
>= ES2017
NodeNext
-

>= ES2022
< ES2017
-
ES2017

< ES2022
< ES2017
ES2022
ES2017

I'm not sure if you meant the target to be es2022 as the code only has till es2021
export const enum ScriptTarget {
        ES3 = 0,
        ES5 = 1,
        ES2015 = 2,
        ES2016 = 3,
        ES2017 = 4,
        ES2018 = 5,
        ES2019 = 6,
        ES2020 = 7,
        ES2021 = 8,
        ESNext = 99,
        JSON = 100,
        Latest = ESNext,
    }

This is really confusing indeed. 😅

TLA is actually allowed in esm mode nodenext files, since latest node allows TLA in esm - we have tests to that effect - a specialized error for cjs mode files would be good, though.

Just as a heads up, #46149 is updating the logic here as well.

Hi! I wanted to get some clarification on the error messages, what should the new ones look like? I can't what the final decision on the formatting was here.