PSShouldProcess hangs

Steps to reproduce
This fault is intermittent and causes me most trouble when it occurs on AppVeyor.
$debugPreference = 'continue'
Invoke-ScriptAnalyzer -Path .\path\to\psm1.psm1 -Verbose

Expected behavior
Script analyzer completes with no errors, there are none in the code I'm testing.
Actual behavior
Script analyzer hangs before or during the PSShouldProcess rule.
2|PS> $debugPreference = 'continue'
3|PS> Invoke-ScriptAnalyzer -Path .\path\to\psm1.psm1 -Verbose
...
VERBOSE: Analyzing file: C:\Development\path\to\psm1.psm1
VERBOSE: Running PSAvoidDefaultValueSwitchParameter rule.
VERBOSE: Running PSAvoidGlobalAliases rule.
VERBOSE: Running PSAvoidDefaultValueForMandatoryParameter rule.
VERBOSE: Running PSAvoidUsingEmptyCatchBlock rule.
VERBOSE: Running PSAvoidGlobalFunctions rule.
VERBOSE: Running PSAvoidGlobalVars rule.
VERBOSE: Running PSAvoidInvokingEmptyMembers rule.
VERBOSE: Running PSAvoidAssignmentToAutomaticVariable rule.
VERBOSE: Running PSAvoidNullOrEmptyHelpMessageAttribute rule.
VERBOSE: Running PSReservedParams rule.
VERBOSE: Running PSAvoidShouldContinueWithoutForce rule.
VERBOSE: Running PSAvoidUsingConvertToSecureStringWithPlainText rule.
VERBOSE: Running PSAvoidUsingComputerNameHardcoded rule.
VERBOSE: Running PSAvoidUsingInvokeExpression rule.
VERBOSE: Running PSAvoidUsingWriteHost rule.
VERBOSE: Running PSAvoidUsingPlainTextForPassword rule.
VERBOSE: Running PSAvoidUsingUsernameAndPasswordParams rule.
VERBOSE: Running PSMissingModuleManifestField rule.
VERBOSE: Running PSAvoidUsingDeprecatedManifestFields rule.
VERBOSE: Running PSPossibleIncorrectUsageOfRedirectionOperator rule.
VERBOSE: Running PSMisleadingBacktick rule.
VERBOSE: Running PSAvoidUsingWMICmdlet rule.
VERBOSE: Running PSUseApprovedVerbs rule.
VERBOSE: Running PSUseCompatibleCmdlets rule.
VERBOSE: Running PSPossibleIncorrectUsageOfAssignmentOperator rule.
VERBOSE: Running PSUseLiteralInitializerForHashtable rule.
VERBOSE: Running PSPossibleIncorrectComparisonWithNull rule.
VERBOSE: Running PSAvoidTrailingWhitespace rule.
VERBOSE: Running PSUseBOMForUnicodeEncodedFile rule.
VERBOSE: Running PSUseToExportFieldsInManifest rule.
VERBOSE: Running PSUseUTF8EncodingForHelpFile rule.
VERBOSE: Running PSUsePSCredentialType rule.
VERBOSE: Running PSUseSupportsShouldProcess rule.
VERBOSE: Running PSUseShouldProcessForStateChangingFunctions rule.
VERBOSE: Running PSUseOutputTypeCorrectly rule.
VERBOSE: Running PSUseDeclaredVarsMoreThanAssignments rule.
VERBOSE: Running PSProvideCommentHelp rule.
VERBOSE: Running PSReservedCmdletChar rule.
VERBOSE: Running PSShouldProcess rule.

Script Analyzer cannot be terminated using Control and C.
Environment data
Tested under PowerShell 5.1 and 6.2 as shown below.
> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      5.1.18362.145
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.18362.145
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>

> (Get-Module -ListAvailable PSScriptAnalyzer).Version | ForEach-Object { $_.ToString() }

1.18.1
1.18.0
1.17.1
1.16.1
1.4.0

And:
> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      6.2.1
PSEdition                      Core
GitCommitId                    6.2.1
OS                             Microsoft Windows 10.0.18362
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

> (Get-Module -ListAvailable PSScriptAnalyzer).Version | ForEach-Object { $_.ToString() }
1.18.1
1.18.0
1.17.1
1.16.1

I'm seeing this same issue on my local machine but not when I run a similar script via Azure DevOps.

Are you using a specific psm1 file?
When you say 'hang' do you mean it never completes or only that the command cannot be cancelled. When calling Invoke-ScriptAnalyzer and using Ctrl+C to cancel the command has to wait for the current file being analysis to be finished. I believe this is due to threads being created for each rule and PSSA waiting for them all to finish, I don't think it is a problem of PSShouldProcess , it just seems to be the last rule being kicked off in the background, when running it again it might be a different rule. Can you please be more precise what the exact problem is? I'd have to look at actual code to see if it is possible to use a cancellation token to cancel analysis of one file when all rules are running, but I suspect this might be difficult, a configurable timeout might be easier in this case

Apologies, I was going to come back to this and dig deeper than I had above.
When I say hang I mean it stops, cannot be cancelled (without closing the console), and never completes (with the caveat that the longest I've waited is 2 hours).
This is the module I'm executing script analyzer against:
https://github.com/indented-automation/Indented.Nexus
Normally I run it against the merged version of the module. A single psm1 file based on the content of that repo.
Just trying to see if I can isolate it to a single piece of code. I thought I had, script analyzer seems to reliably hang when run against this file in isolation:
https://github.com/indented-automation/Indented.Nexus/blob/master/Indented.Nexus/class/NexusNugetSource.ps1
But I bumped that out of the psm1 and it's still hanging. Give me a few minutes to see if I can isolate what in the psm1 it doesn't like.

Okay, I've trimmed down the code a lot. This snippet reproduces the problem for me:
Invoke-ScriptAnalyzer -Verbose -ScriptDefinition @'
function SomeCommand { }

[DscResource()]
class SomeClass {
    [String] $Name = 'Value'

    [Void] Set() {
        $source = Get-PackageSource -Name $this.Name
    }

    [Boolean] Test() {
        # $source = Get-PackageSource -Name $this.Name

        $something = SomeCommand -Name $this.Name
    }
}
'@

There are some combinations with this one.

All commands: Hangs
Get\Get-PackageSource and SomeCommand: Hangs
Set\Get-PackageSource and SomeCommand: Hangs
Get\Get-PackageSource and Set\Get-PackageSource: No problem
Any command alone: No problem
Get\Get-PackageSource  and Set\Get-PackageSource: No problem

If I add the module name in front of the package management commands (e.g. PackageManagement\Get-PackageSource) it works.
I'll wipe all other copies of package management off this computer a bit later and repeat this experiment.

Hmm, I cannot reproduce this with PS 6.2.1 or 5.1 on Windows 10 1903 with the released version of 1.18.1 unfortunately
I did notice though that your example seems to be invalid syntax as there are ParserErrors returned by it.
Also: when using DSC dependencies (not in your example but just as a general ti[), one can use the -SaveDscDependency switch, which will download the dependent modules when it sees the  Import-DscResource syntax

@indented-automation, @ChrisLGardner if you could get a proc dump (procdump.exe -ma $PssaPid) of PSSA when this hang occurs and send it to<EMAIL_ADDRESS>(probably need to upload it to a file host and send a link, since it's going to be a very large file)

@indented-automation, @ChrisLGardner Let us know if this is still happening and needs re-opening but we'd need more info then

Apologies. It is, but I'm putting it all down to oddities in
PackageManagement / PowerShellGet. I've shuffled those out simply because
wherever they go they seem to cause problems.
On Wed, 17 Jun 2020 at 09:36, Christoph Bergmeister [MVP] <
<EMAIL_ADDRESS>wrote:

@indented-automation https://github.com/indented-automation,
@ChrisLGardner https://github.com/ChrisLGardner Let us know if this is
still happening and needs re-opening but we'd need more info then
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PowerShell/PSScriptAnalyzer/issues/1287#issuecomment-645237169,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADB3UDS2YB7K7DT6J5DMTFLRXB6B3ANCNFSM4ICGELZA
.

@indented-automation Starting with PSSA 1.18.2 (which was released after you opened this issue), we fixed a problem that caused PSSA to hang due to PackageManagement / PowerShellGet in  #1316
Can you confirm you are using the latest version (1.19.0)?

Nope, I am not. I was on 1.18.3. I'll upgrade, thank you!
On Wed, 17 Jun 2020 at 10:27, Christoph Bergmeister [MVP] <
<EMAIL_ADDRESS>wrote:

@indented-automation https://github.com/indented-automation Starting
with PSSA 1.18.2 (which was released after you opened this issue), we fixed
a problem that caused PSSA to hang due to PackageManagement / PowerShellGet
in #1316 https://github.com/PowerShell/PSScriptAnalyzer/pull/1316
Can you confirm you are using the latest version (1.19.0)?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PowerShell/PSScriptAnalyzer/issues/1287#issuecomment-645262713,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADB3UDW6Z5H3IZLDFFMPJ4TRXCD7FANCNFSM4ICGELZA
.

Ok, please let us know the outcome. I am going to re-open the issue in the meantime but still leave the Need Repro Info label on because that's what the bot uses to close inactive issues after a certain period of time.


Allow overriding messageId

We have other tracking libraries we use on our site that we wish to use the same messageId for the same event sent to different sources, for example:
User clicks button (Event A):
const messageId = generateMsgId() // 456
const event = 'Event A'

segment.track(event, {}, { messageId }) // sends Event A with messageId 456
anotherSDK.track(event, { messageId }) // sends Event A with messageId 456

Instead, this is the problem we're running into:
User clicks button (Event A):
const messageId = generateMsgId() // 456
const event = 'Event A'
const properties = ...

segment.track(event, properties, { messageId }) // sends Event A with different messageId `ajs-next-*`
anotherSDK.track(event, properties, { messageId }) // sends Event A with messageId 456

Is there not a way to override the messageId per event, so we can use our own format in order to aggregate the same event across different sources? I see no way to do it with this library, but in the docs it says we can and you can in your other libs:

Keep in mind that Segment’s libraries all generate message_ids for each event payload, with the exception of the Segment HTTP API, which assigns each event a unique message_id when the message is ingested. You can override these default generated IDs and manually assign a message_id if necessary.

If there isn't a way in JS, I would propose Segment's JS lib allows it according to your docs.

It seems this is why messageId can't be overridden because of the spread operator placement.
Can we simply allow people to override it at their own risk? Again, docs state "You can override these default generated IDs and manually assign a message_id if necessary."
So the change could be:

For anyone else running across this "issue" it wasn't very clear but I found some middleware docs for Analytics 2.0 that allows us to mutate the payload object. Yes, feels messy but it works.
https://segment.com/docs/connections/sources/catalog/libraries/website/javascript/middleware/

Closing since this is possible through a middleware!

agree with @jflayhart!
@pooyaj it is still a bummer to do with a middleware, other segment libraries allow to pass messageId and it is in a spec. In our case we want to send events with exact messageId to make sure they are not tracked twice, and we don't have a status tracking if event happened before and if it was delivered but we only want it once.


[RFC] Directly call TrainingTypePlugin APIs instead of going through the Accelerator

Proposed refactoring or deprecation

Directly call TrainingTypePlugin APIs instead of going through the Accelerator wherever possible
@four4fish @awaelchli @justusschock @SeanNaren
Motivation
This carries forward the discussion from https://github.com/PyTorchLightning/pytorch-lightning/pull/9373#discussion_r705327408
Most of the Accelerator class today is a shell class that delegates calls to its attached TrainingTypePlugin. This creates an unnecessary level of indirection in many places. It also creates doubt as to whether custom accelerators should override these functions or not.
As most of the strategy around model distribution is embedded in the training type plugin, this is the hub where the following logic lives:

Rank information
Which ranks conduct IO for checkpoint saving/loading
Control/Ownership of the LightningModule
Collective communications

However, the accelerator is positioned as the gateway component the trainer interacts with for this functionality. In turn, much of the logic of the training type plugin is currently replicated on the accelerator. This creates an undesirable coupling (we're nearly doubling the APIs exposed). We could cut out this level of indirection by having the trainer call the training type plugin directly wherever applicable. This would shrink the accelerator interface. Ultimately, this will allow it to live as a component in the training type plugin eventually too. In this case, the accelerator can manage the device logic as part of the overall parallelization strategy.

Pitch
Have the Trainer directly call the training type plugin APIs for this methods and then deprecate/remove the corresponding APIs from the accelerator
https://github.com/PyTorchLightning/pytorch-lightning/blob/c963bf6568e6abd16615b7dfaa446b1f5c446793/pytorch_lightning/accelerators/accelerator.py#L63-L65
https://github.com/PyTorchLightning/pytorch-lightning/blob/c963bf6568e6abd16615b7dfaa446b1f5c446793/pytorch_lightning/accelerators/accelerator.py#L67-L74
https://github.com/PyTorchLightning/pytorch-lightning/blob/c963bf6568e6abd16615b7dfaa446b1f5c446793/pytorch_lightning/accelerators/accelerator.py#L86-L93
https://github.com/PyTorchLightning/pytorch-lightning/blob/c963bf6568e6abd16615b7dfaa446b1f5c446793/pytorch_lightning/accelerators/accelerator.py#L122-L153
https://github.com/PyTorchLightning/pytorch-lightning/blob/c963bf6568e6abd16615b7dfaa446b1f5c446793/pytorch_lightning/accelerators/accelerator.py#L181-L182
https://github.com/PyTorchLightning/pytorch-lightning/blob/c963bf6568e6abd16615b7dfaa446b1f5c446793/pytorch_lightning/accelerators/accelerator.py#L208-L231
https://github.com/PyTorchLightning/pytorch-lightning/blob/c963bf6568e6abd16615b7dfaa446b1f5c446793/pytorch_lightning/accelerators/accelerator.py#L334-L339
https://github.com/PyTorchLightning/pytorch-lightning/blob/c963bf6568e6abd16615b7dfaa446b1f5c446793/pytorch_lightning/accelerators/accelerator.py#L341-L477

Additional context

If you enjoy Lightning, check out our other projects! ⚡

Metrics: Machine learning metrics for distributed, scalable PyTorch applications.

Flash: The fastest way to get a Lightning baseline! A collection of tasks for fast prototyping, baselining, finetuning and solving problems with deep learning

Bolts: Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch

Lightning Transformers: Flexible interface for high performance research using SOTA Transformers leveraging Pytorch Lightning, Transformers, and Hydra.

I think we can do so, wherever no device-specific logic is involved (which should be the case for all the properties/functions you mentioned). When creating this I wanted to avoid users to reach to deeply into the framework internals before they were mature, but I think now we can enable this :)

that sounds good to me too.
once the accelerator has moved, we will still follow this pattern and keep the calls separate, right?

once the accelerator has moved, we will still follow this pattern and keep the calls separate, right?

Yup!
Though I think it could make sense to hold off on this until https://github.com/PyTorchLightning/pytorch-lightning/issues/7324 is completed. That way the trainer will interact entirely with the training type plugin, and we can make the full API cutover at the same time (and we don't have to balance/explain why some calls are using trainer.accelerator while others are using trainer.training_type_plugin)

maybe at this point we want to consider adding #7324 to the project/planning. on the other hand the effort is probably quite large and feature freeze is coming up soon, we might not be able to finish it in time for 1.5. Then on the other other hand we also probably don't want to keep accelerator/plugins experimental forever. What do you think?

maybe at this point we want to consider adding #7324 to the project/planning.

yes I agree we should include it, and that we should start as soon as possible. It's been around for a while and is blocking these other refactors. @awaelchli @justusschock @four4fish maybe we can set up some time early next week to go over how we can proceed with it?

Sounds good! I think if move Precision Plugin into Training Type Plugin + Move accelerator device logic into training type plugin + #7534 -  accelerator can be deprecated. It's better to flatten the wrapping structure before we release the stable API version.
Changes like #7324 could be breaking change and will need multiple stack PRs, how should we do PRs and merge them? Seperate branches, duplicate folder in the main branch or big PRs?

@ananthsub mind adding me to the meeting as well? I am interested to learn more about this

@ananthsub do you think it makes sense to start introducing purely abstract classes before we make such severe changes? I am a bit afraid that without a clearly defined interface, we may run into troubles later where we "forget" to update a portion of the plugin interfaces.

Hey @justusschock. As you originally designed the interface, mind leading the conversation there ?
Also @ananthsub, we started to refactor the Accelerator to be fully independent components and to be used as their own distributed engine, similar to HF accelerate. Should we re-ignite this instead ?
Best,
T.C


289bis react-gears add aria label to component

Add "required" attribute to required input fields

include aria-required="true" in input components where entry is required

See https://trello.com/c/uus8DQw6/289-add-required-attribute-to-required-input-fields

 Add "required" attribute to required input fields


whisper-fetch.py add option to select archive?

I'm using whisper-fetch.py to extract data from a Whisper file.
At https://graphite.readthedocs.io/en/latest/whisper.html#multi-archive-storage-and-retrieval-behavior I see

When data is retrieved (scoped by a time range), the first archive which can satisfy the entire time period is used. If the time period overlaps an archive boundary, the lower-resolution archive will be used.

Is there a way to extract the full time series for a given or each archive?
I guess I have to perfectly set --from and --until to achieve this?
Or is there a tool or API to just get a given time series in full?

Think you are looking for whisper-dump

The library already supports this. It can probably be easily added to the cmdline tool
https://github.com/graphite-project/whisper/blob/master/whisper.py#L892


Sync PRs on orignal repo

As title, I create the issue on issue #71 in original repo.
Because original repo seems to be inactive/unmaintained by @ccampbell, I try think about the PR synchronization.
I think we should move all PRs on original repos to this forked repos and keep this forked repos be maintained.
@hopeseekr , What do you think about this?

Yep, i'm doing it, like right now... I have some questions about some of the PRs, and legitly about 1/3 of the PRs seem to be dupes of one another. Seriously, the 500 on huge chrome buffer is like 3 or 4 of the existing PRs.

In chronological order, ascending:

Update ChromePhp.php with debug() [ccampbell/chromephp#13]: Won't merge: Unknown utility.
Check HTTP header size [ccampbell/chromephp#40]: Addressed in PR #2.
Add a setting for base-path so that only the relevant parts of the file  [ccampbell/chromephp#42]: Won't merge: Unknown utility, seems to be a non-trivial risk.

@hopeseekr, thank you for your explanation.
I just note that we have to update PR status on the original repository and it can notify original PR authors.

You can @ notify them. I think that's what's needed most at this point...

@hopeseekr can you add this updated repo to packagist? I would like to use it with composer. The original code of ccampbell is outdated and gives issues which are solved in this repo.


Update Surl extension

Description

Add preferences to the extension

Screencast

Add Customize Expiration Date action.
User can select the expiration date in setting page.

Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

@pernielsentikaer

Does the history also remove the items after the expiration date?

Thank you for the review.
The answer is no. For now, user has to check the validity of url by yourself. 😅
Is that necessary to remove the expired urls?

No, but I guess it could be confusing if you have the list and none of the links is working - it could be a version 3 for sure 🙂
Let's just continue for now :)

@pernielsentikaer
Thank you for the reviewing!

it could be a version 3 for sure 🙂

Exactly! I will!


[sui framework] address module

Add a utility module for working with addresses:

Conversion of addresses to/from u256 (useful if you want to compare them or do arithmetic operations)
Conversion of address to/from bytes (moved from object module)
Conversion of address to/from string (forthcoming)

I am getting a stack overflow insui_cost::test_good_snapshot--any ideas what might be causing this @oxade?

This is great! I really wish we had a module in the std for each primitive type (like we do with vector and signer). Even if they were just placeholders for the time being
Though I suppose address is a bit weird in that it is different per instantiation

I agree--I think we should add one for each with (at least) to_string and from_string. Happy to upstream for the non-address types if the community likes it.


CPP thread tests do not build with non-empty LIBNAMESUFFIX

Hi,
In OpenBLAS-0.3.18, I found that if I set a non-empty LIBNAMESUFFIX in Makefile.rule,
# If you set the suffix, the library name will be libopenblas_$(LIBNAMESUFFIX).a
# and libopenblas_$(LIBNAMESUFFIX).so. Meanwhile, the soname in shared library
# is libopenblas_$(LIBNAMESUFFIX).so.0.
LIBNAMESUFFIX = omp

and if I also set CPP_THREAD_SAFETY_TEST = 1 in the same file,
# Run a C++ based thread safety tester after the build is done.
# This is mostly intended as a developer feature to spot regressions, but users and
# package maintainers can enable this if they have doubts about the thread safety of
# the library, given the configuration in this file.
# By default, the thread safety tester launches 52 concurrent calculations at the same
# time.
#
# Please note that the test uses ~1300 MiB of RAM for the DGEMM test.
#
# The test requires CBLAS to be built, a C++11 capable compiler and the presence of
# an OpenMP implementation. If you are cross-compiling this test will probably not
# work at all.
#
CPP_THREAD_SAFETY_TEST = 1
#
# use this to run only the less memory-hungry GEMV test
# CPP_THREAD_SAFETY_GEMV = 1

then the CPP thread tests will fail to build because the linkage to openblas is hardcoded to ../libopenblas.a in cpp_thread_test/Makefile:
dgemv_tester :
        $(CXX) $(COMMON_OPT) -Wall -Wextra -Wshadow -fopenmp -std=c++11 dgemv_thread_safety.cpp ../libopenblas.a -lpthread -o dgemv_tester
        ./dgemv_tester

dgemm_tester : dgemv_tester
        $(CXX) $(COMMON_OPT) -Wall -Wextra -Wshadow -fopenmp -std=c++11 dgemm_thread_safety.cpp ../libopenblas.a -lpthread -o dgemm_tester
        ./dgemm_tester

I have confirmed that changing the two lines above to
dgemv_tester :
        $(CXX) $(COMMON_OPT) -Wall -Wextra -Wshadow -fopenmp -std=c++11 dgemv_thread_safety.cpp ../libopenblas_$(LIBNAMESUFFIX).a -lpthread -o dgemv_tester
        ./dgemv_tester

dgemm_tester : dgemv_tester
        $(CXX) $(COMMON_OPT) -Wall -Wextra -Wshadow -fopenmp -std=c++11 dgemm_thread_safety.cpp ../libopenblas_$(LIBNAMESUFFIX).a -lpthread -o dgemm_tester
        ./dgemm_tester

fixes the issue, although perhaps this isn't the cleanest way to do so.

Ah, so my "fix" above does not work if LIBNAMESUFFIX is left undefined in Makefile.rule.

Oh right, this should probably be ../$(LIBNAME) like in all the other tests...


mc_att_control: copy sensor_correction topic once initially

This was causing the mc_att_controller and the sensors module not to be in sync after a reboot regarding which instance of the gyro topic was selected.
This is relevant when using the CAL_GYROx_EN parameters to disable a specific IMU.

We need to be more deliberate when putting things directly into stable. If the tag isn't also updated then QGC users will get a bogus dev version as well as warnings.

@dagar against what do you want me to do this PR against? Also do you know why CI is failing? I can't get rid of the 404.

Can you fix it in master first, then backport it to stable? We need to tag it at the same time it goes into the stable branch.

@dagar according to @bkueng this should already be fixed in master due to a change in uorb behavior, see my comment in the code.

Right okay. https://github.com/PX4/Firmware/commit/2fe5e7735a243ddcd4557d7d5f7b6dea9c7c895f#diff-14d11b3fe5440be3f4a64b36f55287ad
Then let's get this into stable and tag it v1.8.2

Cherry-picked to stable and tagged v1.8.2. https://github.com/PX4/Firmware/releases/tag/untagged-940aa5982d784e87171c


add Filter component with keydown "/" focus

What kind of change does this PR introduce?
Adds Filter to website. Filter has keydown handlers to focus self on "/".
Did you add tests for your changes?
No need imo.
Does this PR introduce a breaking change?
No, it's website.

Codecov Report

Merging #242 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #242   +/-   ##
=======================================
  Coverage   70.25%   70.25%
=======================================
  Files          96       96
  Lines         659      659
  Branches      132      132
=======================================
  Hits          463      463
  Misses        163      163
  Partials       33       33

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1ec72be...8d86bd0. Read the comment docs.


questions about random use

I've noticed your most recent commit which changed the random use to use a global rng to generate numbers. However, it will lead to a data races if the generateRandomNumber() is called concurrently. Is there any reason for this change?

Thank you for reporting this problem. The Go 1.20 release notes describes that rand.Seed(SEED) has been abandoned. It is recommended to use rand.New(rand.NewSource(seed)). I will fix the concurrency soon.

fixed ccf1004e6c9f4a81c175e9bf500ddf53ce3c35e7.


LightGBM + categorical features broken inside ShapRFECV

Describe the bug
When using LightGBM on a dataset including pd.Categorical features, the shap Explainer will fail, advising you to use feature_perturbation="tree_path_dependent". However, since we're using LightGBM, the algorithm will already choose this by default - the real issue is that background data is passed, which isn't supported together with feature_perturbation="tree_path_dependent".  BG data is passed as mask in shap_calc().
Environment (please complete the following information):

probatus 1.8
python 3.8

To Reproduce
Use LightGBM in ShapRFECV on a dataset with categorical features.
import lightgbm as lgbm
from probatus.feature_elimination import ShapRFECV

model = lgbm.LGBMClassifier()
shap_elimination = ShapRFECV(
    clf=model,
    step=0.2,
    cv=5
)
# X is dataframe with pd.Categorical, y is binary response
report = shap_elimination.fit_compute(X, y)

Error traceback
Can't provide right now as I've already fixed this on my branch, but the error will be:
                raise Exception("Currently TreeExplainer can only handle models with categorical splits when " \
                                "feature_perturbation=\"tree_path_dependent\" and no background data is passed. Please try again using " \
                                "shap.TreeExplainer(model, feature_perturbation=\"tree_path_dependent\").")

from inside shap _tree.py, as called inside shap_calc().
Expected behavior
It runs without issue, as there is support for trees with categorical features in shap.
Proposed fix
Check model type and X features inside shap_calc, and avoid passing mask if there are categorical features and the model is tree-based.

Great finding, I think this would be the way to go.
probatus by default transforms dataset to Df, and all categorical features have "category" dtype. Indeed not passing the mask if categorical features would be nice.
I am curious whether this applies only to tree-based models, are there any linear ones that support categorical features?
Feel free to pick this issue up! 👍

I'll submit a PR later today implementing both this and #106!
I believe it applies mainly to LGBM (not sure which, if any, other models handle categorical natively). Linear models wouldn't be able to support categorical features - perhaps on the surface, but they'd have to one-hot encode internally as you can't make coefficients for non-numerical data.

Good one! I think this will also solve the problems we were having with testing dummy data with a categorical column, won't it @Matgrb?

Covered in #139


Update key-vault-disaster-recovery-guidance.md

Added link to documentation showing specific region pairs.

@jboeshart,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

@jboeshart : Thanks for your contribution to the Azure documentation! The author, @adamglick, has been notified to review your proposed change.

@adamglick, any thoughts on this PR?

The link to the paired regions doc is legit and the addition is sensible. Seems a good PR to me.

#sign-off - go ahead and allow this update

@adamglick - go ahead and #sign-off


add tests for docker container

test runner should start & connect to freshly built container.

using a container or using a normal gradle build should be an option.


How to extract DSL firmware ?

Hello all

I believe the "VMG1312-T20B" holds new DSL firmware, however im not sure how to extract it as nothing is in the firmware folder or the modules folder? all i know is, this works better on my line than the dm200/VRX200

https://www.zyxel.com/uk/en/support/download_library/product/vmg1312_t20b_14.shtml?c=uk&l=en&pid=20170103120002&tab=Firmware&pname=VMG1312-T20B&mtname=Firmware
https://download.zyxel.com/VMG1312-T20B/firmware/VMG1312-T20B_5.30(ABSB.5)C0.zip

im having problems with openwrt on dm200 under (21.02.3) it keeps rebooting, it connects to my dsl line under bridge mode then randomly reboots ?

Hello,
regarding your questions:

I'm not sure either - if anyone finds a way they can create a pull request
I don't have a DM200 myself. I suggest asking in the OpenWrt support channels (IRC, forum, ...)


Warning: Prop style did not match. Server:

I am using this with server-side rendering, which I know is discouraged for some reason, but it's all working fine except for this warning on initial load.
Warning: Prop `style` did not match. Server: "z-index:1;opacity:0;display:block;position:absolute;top:0;left:0;width:0;transition:opacity 480ms cubic-bezier(0.165, 0.84, 0.44, 1),-webkit-transform 480ms cubic-bezier(0.165, 0.84, 0.44, 1),transform 480ms cubic-bezier(0.165, 0.84, 0.44, 1);transform:translateX(0px) translateY(10px);-webkit-transition:opacity 480ms cubic-bezier(0.165, 0.84, 0.44, 1),-webkit-transform 480ms cubic-bezier(0.165, 0.84, 0.44, 1),transform 480ms cubic-bezier(0.165, 0.84, 0.44, 1);-moz-transition:opacity 480ms cubic-bezier(0.165, 0.84, 0.44, 1),transform 480ms cubic-bezier(0.165, 0.84, 0.44, 1);-webkit-transform:translateX(0px) translateY(10px)" Client: "z-index:1;opacity:0;display:block;position:absolute;top:0;left:0;width:NaNpx;transition:opacity 480ms cubic-bezier(0.165, 0.84, 0.44, 1),transform 480ms cubic-bezier(0.165, 0.84, 0.44, 1);transform:translateX(NaNpx) translateY(10px)"

Breakdown
Server Style
Server: "z-index:1;opacity:0;display:block;position:absolute;top:0;left:0;width:0;transition:opacity 480ms cubic-bezier(0.165, 0.84, 0.44, 1),-webkit-transform 480ms cubic-bezier(0.165, 0.84, 0.44, 1),transform 480ms cubic-bezier(0.165, 0.84, 0.44, 1);transform:translateX(0px) translateY(10px);-webkit-transition:opacity 480ms cubic-bezier(0.165, 0.84, 0.44, 1),-webkit-transform 480ms cubic-bezier(0.165, 0.84, 0.44, 1),transform 480ms cubic-bezier(0.165, 0.84, 0.44, 1);-moz-transition:opacity 480ms cubic-bezier(0.165, 0.84, 0.44, 1),transform 480ms cubic-bezier(0.165, 0.84, 0.44, 1);-webkit-transform:translateX(0px) translateY(10px)"

Client Style
Client: "z-index:1;opacity:0;display:block;position:absolute;top:0;left:0;width:NaNpx;transition:opacity 480ms cubic-bezier(0.165, 0.84, 0.44, 1),transform 480ms cubic-bezier(0.165, 0.84, 0.44, 1);transform:translateX(NaNpx) translateY(10px)"

For all intents and purposes, I don't need styling or position determined until it reaches the client side. I only need to load the content server-side for SEO purposes.
It's fine if none of the style calculations happen until reaching client side. The problem is that if I disable the enableSSR feature, it omits all of the content from server-side which is not desirable.

@mdrideout , do you even get elements rendered on SS ? On my side no element while SSR and i get this error, client is fine

@yyynnn - I got too frustrated with this. I ended up programming my own script to take my array of items, and split it into the number of arrays that there are columns. (1 array -> 5 arrays for 5 columns) The array splitting function orders the items in the new arrays, such that the items appear to be ordered from left-to-right across the page. Each columns has 1 array, but the new array ordering makes it read left-to-right.
React keeps track of page width to know how many columns / arrays to create - with matching css media query breakpoints.
server-side it renders 1 array content, but client side it's immediately broken into additional columns / arrays on load.
This method is also great for lazy-loading because it continues the left-to-right trend as new items are loaded as you scroll.

@yyynnn - I got too frustrated with this. I ended up programming my own script to take my array of items, and split it into the number of arrays that there are columns. (1 array -> 5 arrays for 5 columns) The array splitting function orders the items in the new arrays, such that the items appear to be ordered from left-to-right across the page. Each columns has 1 array, but the new array ordering makes it read left-to-right.
React keeps track of page width to know how many columns / arrays to create - with matching css media query breakpoints.
server-side it renders 1 array content, but client side it's immediately broken into additional columns / arrays on load (since server-side does not have a page-width value to go by).
This method is also great for lazy-loading because it continues the left-to-right trend as new items are loaded as you scroll.

please share how you solve this

Any update on this?


PHOENIX-7387: SnapshotScanner's next method is ignoring the boolean value from hbase's nextRaw method

PHOENIX-7387  SnapshotScanner's next method is ignoring the boolean value from hbase's nextRaw method
The pr include the following changes

Now we are using the return type of hbase scanner, which is a boolean value to decide what resultset we should pass to the caller.

Tested in our local setup.

You fixed the GH description, not the patch description.
It should be
"PHOENIX-7387 SnapshotScanner's next method is ignoring the boolean value from hbase's nextRaw method"

Fix the description in git, and force-push the change

Fix the description in git, and force-push the change

Done


TypeError: 'undefined' is not a function (evaluating 'shortcutMethod.bind(null, addMethods)')

I tried to use stampit with karma & gulp, but I'm always getting the above mentioned error. I created a branch with my current setup here.
Whoever wants to help, just check it out, switch to the branch
git checkout adding-stampit
npm install
./node_modules/.bin/gulp

and you should be able to see the error message.
Would appreciate help on this. Thanks in advance.

Hello.

I'm pretty sure it's not stampit related error. Could you show full error? With the stack trace.
In this file you are exporting not a stamp but an object. Maybe export stamp to match the file name?
In your test the expect(myStamp.sayHello()).toEqual('Hello'); line might fail. I don't see an assetion module in the karma conf, or package.json, or gulpfile.
You put stampit as a devDependency for some reason. Although, you are importing it in src/.

I believe, questions like this should be asked at Stackoverflow. :) I'd be gald to answer them there.
Cheers!

[22:28:04] Starting Karma server...
INFO [framework.browserify]: bundle built
INFO [karma]: Karma v0.12.37 server started at http://localhost:9876/
INFO [launcher]: Starting browser PhantomJS
INFO [PhantomJS 1.9.8 (Mac OS X 0.0.0)]: Connected on socket 6izvHPLccA5QZymyShp2 with id 70500136
PhantomJS 1.9.8 (Mac OS X 0.0.0) ERROR
  TypeError: 'undefined' is not a function (evaluating 'shortcutMethod.bind(null, addMethods)')
  at /var/folders/jf/00gb7hd17yl853ww6sw44pqh5r01xc/T/b6ed61688795a1cd4f8fe9f953283a1b.browserify:369

/Users/marc.dix/projects/setups/babel_karma_gulp/gulpfile.js:38
            throw err;
            ^
Error: karma exited with code 1
    at done (/Users/marc.dix/projects/setups/babel_karma_gulp/node_modules/gulp-karma/index.js:56:30)
    at ChildProcess.<anonymous> (/Users/marc.dix/projects/setups/babel_karma_gulp/node_modules/gulp-karma/index.js:82:7)
    at emitTwo (events.js:87:13)
    at ChildProcess.emit (events.js:172:7)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)

I already did that. Same error.

Same line works when asserting in the spec where I used ES6 class. Even if it would fail, it would be a different error.

Yeah, I accidentally added everything as devDependency. Will change that. Thanks.

Best
Marc

Why are you rethrowing the error from Karma? Doesn't that just obscure the stack trace and the real source of the error?

@ericelliott Not really, running directly with karma (& log level to DEBUG):
...
DEBUG [web-server]: serving (cached): /Users/marc.dix/projects/setups/babel_karma_gulp/node_modules/karma-jasmine/lib/adapter.js
DEBUG [web-server]: serving (cached): /Users/marc.dix/projects/setups/babel_karma_gulp/spec/stamp/StampSpec.js
DEBUG [phantomjs.launcher]: Hello

PhantomJS 1.9.8 (Mac OS X 0.0.0) LOG: 'Hello'

PhantomJS 1.9.8 (Mac OS X 0.0.0) ERROR
  TypeError: 'undefined' is not a function (evaluating 'shortcutMethod.bind(null, addMethods)')
  at /var/folders/jf/00gb7hd17yl853ww6sw44pqh5r01xc/T/b452d365a5cb034f23637b1beaf0a743.browserify:369

DEBUG [karma]: Run complete, exiting.
DEBUG [launcher]: Disconnecting all browsers
DEBUG [framework.browserify]: cleaning up
DEBUG [launcher]: Process PhantomJS exited with code 0
DEBUG [temp-dir]: Cleaning temp dir /var/folders/jf/00gb7hd17yl853ww6sw44pqh5r01xc/T/karma-90092623
...

Something is wrong with the build system. gulp build fails exactly the same way.

gulp.task('default', ['build']);

might be the reason for that. Running gulp is the same as gulp build because of the line above.

@mdix have you tried bypassing the build completely?

@ericelliott Yes, I did. Mind reading this again?

Is the build system config runs browserify twice? Browserifying a vrowserified code?

@koresar No. The build system is only running all the stuff once to build (build.js). The whole stuff runs again when karma is executed. But both work with fresh source files, so we're not browserifying browserified code here.

@mdix Ah, I meant the whole system, including karma. I.e., just run the JS files and console.log() the results.

Is there a reason to not run the tests against the just built build.js?

@ericelliott Just an attempt to save me some work: Would your question be answered with "If I run build.js, it logs out 'Hello' twice and no error is shown."? If this doesn't answer your question, I have to somehow browserify and babelify the source file on the console. Never did that, as we have nice build tools in 2015. So?

@koresar I'm unittesting, so I'm not interesting in how the components work together, means I'm not interested in what app.js does at all and I don't want to get problems with whatever sideffects might occur when implicitly running app.js (as it's part of build.js). We drift off a bit, don't we?

In my opinion the build system is broken. Stampit was tested by thousands of different setups and environments. Have never had an issue.

Taking the logs above, @mdix, could you show us this file?
/var/folders/jf/00gb7hd17yl853ww6sw44pqh5r01xc/T/b452d365a5cb034f23637b1beaf0a743.browserify
Thanks

@koresar Well, at least I can try. Please wait a few seconds, I gotta check how to keep the temp file created by karam.

Also, I believe this can be an issue of build babelifying. Stampit is wirtten in ES6, but the main: file is an ES5 babel generated output. Running babel twice on the same file might be causing issues.

Also, I believe this can be an issue of build babelifying. Stampit is wirtten in ES6, but the main: file is an ES5 babel generated output. Running babel twice on the same file might be causing issues.

@koresar I'm sorry, but I'm not able to quickly find a way how to keep the browserify temp file. Other suggestions? (Still trying to figure it out)

All we need is the build result of this part:

        preprocessors: {
            'src/**/*.js': ['browserify'],
            'spec/**/*.js': ['browserify']
        },

        browserify: {
            transform: ['babelify']
        },

You can try running the appropriate set of commands manually.
P.S.
One of the reasons I don't like grunt/gulp/karma/etc is because of this bloody maaagic happening behind the scenes.

@koresar Yea, and I'd love to provide you with the output of this. Seems there's no configuration option in browserify to keep the temp file.
Concerning the build tools: Thanks for your opinion on this, but you know that discussions on that tend to get pretty long (and I just want to get my problem solved).

Delete all the stamp stuff. Does it still fail?

@ericelliott No. Works fine without stamp.

I found the way to keep the temp files.
Commend out the rimraf at the bottom:
node_modules/karma/lib/temp_dir.js

@koresar Thanks, unfortunately it's not working. Are you sure that karma is creating the file?

Yes. I'm browsing it now. :) (Looks legit though.)
0) In karma.conf.js replace LOG_INFO with LOG_DEBUG.

Comment out the rimraf line.
Run gulp. The karma should run but not quit.
Take a look at the only file in this folder. It should look like this in the output:
DEBUG [temp-dir]: Cleaning temp dir /var/folders/zy/c7yfqxq11qz3715xx20h_rrm0000gn/T/karma-70559475
There will be URL. Open it in a browser. Open dev tools -> source. Find there the generated file.

;)

@koresar Ok, please stand by. Gotta give my baby some milk, hope to be back in 5 min.

@mdix It isn't a problem in Stampit, and you can verify that for yourself. Pop open Node at the prompt, and type:
var stampit = require('stampit');
var myStamp = stampit().methods({sayHello: function() { return 'Hello'; }});
myStamp().sayHello(); // Hello

I just copied and pasted the body of your Stamp code.

Yes, Eric. It looks like a bug somewhere in PhantomJS, or Karma, or Babelify, or Browserify, or else.

@ericelliott @koresar Thanks for helping me to identify the direction. Now it'll be a long road to finding the root cause. The funny thing is: I did not mention this, but earlier in this discussion I ran karma with build.js (and disabled all the transforms) and still had the same error. Based on this discovery I should probably check karma or phantom, as it proofs that the *ifys are working fine.

I'm just gonna leave this here...
Why I use Tape instead of karma and so should you ;)

@ericelliott I read it. Mulitple times. I also read the comment from the mocha guy. IMO most of your points are wrong. (But please don't get me wrong, I really like adore you & your writings on medium - but tape is something I'll definitely not use in production).

I think the next step is to replace gulp-karma with karma, because the development of the gulp-karma wrapper has been stopped a long time ago. Let's see if this fixes the problem.

IMO most of your points are wrong.

I'm looking forward to reading your rebuttal post. =)

I'm closing this issue as not related to stampit.
@mdix Feel free to create a new one after you find the cause of the problem is actually stampit.
Cheers

@ericelliott @koresar Just in case you're interested: Looks like it's a phantomjs (or karma-phantomjs-launcher) issue. It works fine with Chrome (using the karma-chrome-launcher. Big thanks to both of you for helping me to identify the cause.

@mdix Thanks for the followup. I really appreciate that.
As usual, any further questions are welcome.

The problem is PhantomJS is missing Function.bind.
Include the polyfill and it will work.

Oh. Thanks Terry!
On Tue, 20 Oct 2015 01:12 Terry Rosen<EMAIL_ADDRESS>wrote:

The problem is PhantomJS is missing Function.bind.
Include the polyfill
https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_objects/Function/bind#Polyfill
and it will work.
—
Reply to this email directly or view it on GitHub
https://github.com/stampit-org/stampit/issues/175#issuecomment-149224759
.


Refactor monologue and SWE agent to use the messages in state history

This PR updates monologue agent to the current architecture. It's a follow-up to #1847.
Necessary for the #1771 to start working with the right components.
@rbren I assume we need to take a look at any agent who currently has some sort of own history or copies, for the session management. Done.

updated monologue
updated swe agent
planner and micro are well-behaved already

Please note that this PR is cutting off monologue from memory condenser, since it doesn't maintain its own history of events anymore, thus can't directly replace it. This can be reworked and re-connected after 1810 and 1771.

Definitely. Not blocking, but it's quite likely there'll be some conflicts. I'll take care of that.


DataSet.merge not generalizing to any shape, causing crash when training with Spark

I'm trying to train with Spark using  the following code:
    var net = FeedForward.create(...)
    val sparkNetwork = new SparkDl4jMultiLayer(sc, net)

    val trainIter: List[DataSet] = new FeedForwardIterator(...).asScala.toList  // Iterator works without Spark
    val rddTrain: JavaRDD[DataSet] = sc.parallelize(trainIter)

    for (i <- 0 until neuralPrefs.epochs) {
      net = sparkNetwork.fitDataSet(rddTrain, 200, 2) // CRASH
      ...
    }

    net

But it crashes. According to Adam it's because DataSet.merge is not generalizing to any shape.
Full stack trace: https://gist.github.com/Habitats/9f86775c13dbfe958e1817142398dd9a

What shape data specifically are you using?

I've encountered the same issue and the dataset that created the same bug consists of black and white JPEGS (for convolutional nets).

For the FFN I'm using 1000D vectors, and for the RNN I'm using n 1000D vectors in a timeseries.

More specifically, the features for the RNN are of shape [3,50,1000,68,68000,68,1,0,1,99] and the FFN [2,500,1000,1000,1,0,1,99],

Forgot to close this one earlier.
Fixed here: https://github.com/deeplearning4j/nd4j/pull/867


Add CustomizedImageKata distro to enable Kata provisioning for custom header BYOI

What type of PR is this?
/kind feature
What this PR does / why we need it:
Add CustomizedImageKata distro to enable Kata provisioning for custom header BYOI.
Which issue(s) this PR fixes:
Fixes #
Requirements:

[x] uses conventional commit messages

[x] includes documentation
[x] adds unit tests
[x] tested upgrade from previous version

Special notes for your reviewer:

Passed manual tests using aksdev using https://dev.azure.com/msazure/CloudNativeCompute/_build/results?buildId=79932221&view=results

Release note:
none

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

3 of 3 (100.0%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall first build on mitchzhu/kata_custom_header at 61.464%

Totals

Change from base Build<PHONE_NUMBER>:
61.5%

Covered Lines:
2678

Relevant Lines:
4357

💛  - Coveralls


Name Add

Add Jokke to Contributors list, w0w

Hi  @Skraek, I'm quite elated about your pull request. I wanna evolve this project to addresses various problems faced by first-time contributors. I'd love to learn about your journey in open source community, the problems, pain points you had etc.
Could you explain how you felt when you went through the tutorial, made a pull request and learned that I merged it?
We’ve recently added social share  to our web app. Could you please go to https://roshanjossey.github.io/first-contributions/#social-share and share your first contribution to open source?
Also, check out projects with easy issues while you’re there.


Creating graphql queries in each component creates parallel http links

Each component that defines it's own message query/subscription opens a separate websocket connection.  This is inefficient and currently causes threading/buffer errors within the api (although that should be fixed).
It's probably better to define the query/subscription at the App level rather than Component level and have a single graphql connection.

This may or may not be a problem.  Multiple connections mean only having to track the data you want, plus schemas are individually smaller and less complex.
Will re-evaluate when we reflect the schema.


HADOOP-16255. Add ChecksumFs.rename(path, path, boolean) to rename crc file as well when FileContext.rename(path, path, options) is called.

Please refer https://issues.apache.org/jira/browse/HADOOP-16255 for more details.
FYI, FileContext.rename(path, path, options) leaks crc file for source of rename when CheckFs or its descendant is used as underlying filesystem. https://issues.apache.org/jira/browse/SPARK-28025 took a workaround via removing crc file manually, and we hope to get rid of workaround eventually.

cc. @steveloughran to ask for reviewing.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
42
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1
mvninstall
1113
trunk passed

+1
compile
1048
trunk passed

+1
checkstyle
47
trunk passed

+1
mvnsite
85
trunk passed

+1
shadedclient
832
branch has no errors when building and testing our client artifacts.

+1
javadoc
68
trunk passed

0
spotbugs
125
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
123
trunk passed

_ Patch Compile Tests _

+1
mvninstall
51
the patch passed

+1
compile
1037
the patch passed

+1
javac
1037
the patch passed

-0
checkstyle
47
hadoop-common-project/hadoop-common: The patch generated 10 new + 30 unchanged - 0 fixed = 40 total (was 30)

+1
mvnsite
77
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
655
patch has no errors when building and testing our client artifacts.

+1
javadoc
68
the patch passed

+1
findbugs
141
the patch passed

_ Other Tests _

+1
unit
581
hadoop-common in the patch passed.

+1
asflicense
51
The patch does not generate ASF License warnings.

6158

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1388

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 25d47a9e130f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / eeccd2f

Default Java
1.8.0_222

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/1/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/1/testReport/

Max. process+thread count
1540 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/1/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
45
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1
mvninstall
1097
trunk passed

+1
compile
1070
trunk passed

+1
checkstyle
48
trunk passed

+1
mvnsite
87
trunk passed

+1
shadedclient
817
branch has no errors when building and testing our client artifacts.

+1
javadoc
65
trunk passed

0
spotbugs
130
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
127
trunk passed

_ Patch Compile Tests _

+1
mvninstall
50
the patch passed

+1
compile
1021
the patch passed

+1
javac
1021
the patch passed

-0
checkstyle
50
hadoop-common-project/hadoop-common: The patch generated 10 new + 30 unchanged - 0 fixed = 40 total (was 30)

+1
mvnsite
81
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
661
patch has no errors when building and testing our client artifacts.

+1
javadoc
67
the patch passed

+1
findbugs
138
the patch passed

_ Other Tests _

+1
unit
578
hadoop-common in the patch passed.

+1
asflicense
52
The patch does not generate ASF License warnings.

6135

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1388

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 33be8e4772a6 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 7d998cb

Default Java
1.8.0_222

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/2/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/2/testReport/

Max. process+thread count
1679 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/2/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
82
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1
mvninstall
1164
trunk passed

+1
compile
993
trunk passed

+1
checkstyle
42
trunk passed

+1
mvnsite
72
trunk passed

+1
shadedclient
836
branch has no errors when building and testing our client artifacts.

+1
javadoc
60
trunk passed

0
spotbugs
119
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
117
trunk passed

_ Patch Compile Tests _

+1
mvninstall
45
the patch passed

+1
compile
936
the patch passed

+1
javac
936
the patch passed

-0
checkstyle
42
hadoop-common-project/hadoop-common: The patch generated 10 new + 30 unchanged - 0 fixed = 40 total (was 30)

+1
mvnsite
69
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
716
patch has no errors when building and testing our client artifacts.

+1
javadoc
60
the patch passed

+1
findbugs
122
the patch passed

_ Other Tests _

+1
unit
537
hadoop-common in the patch passed.

+1
asflicense
43
The patch does not generate ASF License warnings.

6024

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1388

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux cae53e552296 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 7d998cb

Default Java
1.8.0_212

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/3/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/3/testReport/

Max. process+thread count
1347 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/3/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
41
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1
mvninstall
1017
trunk passed

+1
compile
984
trunk passed

+1
checkstyle
51
trunk passed

+1
mvnsite
83
trunk passed

+1
shadedclient
824
branch has no errors when building and testing our client artifacts.

+1
javadoc
70
trunk passed

0
spotbugs
120
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
119
trunk passed

_ Patch Compile Tests _

+1
mvninstall
46
the patch passed

+1
compile
932
the patch passed

+1
javac
932
the patch passed

+1
checkstyle
50
the patch passed

+1
mvnsite
80
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
676
patch has no errors when building and testing our client artifacts.

+1
javadoc
73
the patch passed

+1
findbugs
129
the patch passed

_ Other Tests _

+1
unit
547
hadoop-common in the patch passed.

+1
asflicense
53
The patch does not generate ASF License warnings.

5869

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1388

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux b27028dcaf92 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 7d998cb

Default Java
1.8.0_222

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/4/testReport/

Max. process+thread count
1475 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/4/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
42
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1
mvninstall
1409
trunk passed

+1
compile
1339
trunk passed

+1
checkstyle
52
trunk passed

+1
mvnsite
105
trunk passed

+1
shadedclient
1008
branch has no errors when building and testing our client artifacts.

+1
javadoc
75
trunk passed

0
spotbugs
153
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
149
trunk passed

_ Patch Compile Tests _

+1
mvninstall
64
the patch passed

+1
compile
1264
the patch passed

+1
javac
1264
the patch passed

+1
checkstyle
51
the patch passed

+1
mvnsite
87
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
909
patch has no errors when building and testing our client artifacts.

+1
javadoc
81
the patch passed

+1
findbugs
172
the patch passed

_ Other Tests _

+1
unit
620
hadoop-common in the patch passed.

+1
asflicense
54
The patch does not generate ASF License warnings.

7558

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1388

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux a373622f7857 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 915cbc9

Default Java
1.8.0_222

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/5/testReport/

Max. process+thread count
1453 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/5/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

core patch LGTM for a branch 3+ patch; made some minor comments about tests and imports
If you want this in branch-2 then I think a copy-and-paste solution is the simpler one

Sigh... I forgot Hadoop 2.x runs on JDK 7. Actually I just did copy-and-paste and expected review comment on deduplication, hence taking this way but forgot about that.
Thanks for reminding! I'll just roll back to copy-and-paste solution.

Thanks for reviewing. I addressed review comments, including rolling back to the copy-and-paste approach. Please take a look.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
35
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1
mvninstall
1237
trunk passed

+1
compile
1193
trunk passed

+1
checkstyle
48
trunk passed

+1
mvnsite
91
trunk passed

+1
shadedclient
948
branch has no errors when building and testing our client artifacts.

+1
javadoc
70
trunk passed

0
spotbugs
146
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
143
trunk passed

_ Patch Compile Tests _

+1
mvninstall
55
the patch passed

+1
compile
1173
the patch passed

+1
javac
1173
the patch passed

+1
checkstyle
49
the patch passed

+1
mvnsite
79
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
810
patch has no errors when building and testing our client artifacts.

+1
javadoc
68
the patch passed

+1
findbugs
161
the patch passed

_ Other Tests _

+1
unit
641
hadoop-common in the patch passed.

+1
asflicense
53
The patch does not generate ASF License warnings.

6909

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1388

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 98f6ca19ebf6 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 337e9b7

Default Java
1.8.0_222

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/6/testReport/

Max. process+thread count
1327 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1388/6/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

all is good, +1 for trunk and I'll actually pull into 3.2 as well. For branch-2 I'll give you the homework of the cherrypick and retest before it goes in.

Ok, committed to branch-3.2 and trunk.

Thanks for reviewing and merging! I'll create separate PR for branch-2. Thanks for the guide.

great!
FWIW you made the 3.2 branch about 8h before the 3.2.1 branch was made, so this will be in that release. Please help test those RCs to make sure they do the right thing for you


feat: t3rn EVM CLI, EVM to Substrate decimals conversion, and precompiles re-connect

Summary by CodeRabbit

New Feature: Added new CLI commands to claim EVM address, check EVM balance, and transfer EVM balance. Users can now interact with EVM addresses directly from the CLI.
New Feature: Introduced input validation schemas for the new EVM-related commands. This ensures that users provide correct and necessary information when using these features.

Functionality already merged in #1563


Added Android platform

Problem
On all other platforms, platform related code are placed at src/platform/xxx, but on Android, these code are placed at src/control/java

It caused that:

[duplicate code] When we adding tv clusters app android ( in orange ), we have to add duplicate app server java & jni and AndroidChipStack ( in orange ) which are similar with controller/java & jni and AndroidChipStack ( in Red )
[Feature not work] There is no enough injection in some modules (for example src/app/server/Server.cpp) . It calls DeviceLayer::ConnectivityMgr().GetBleLayer()).xxx directly without check if a BleLayer is inject like controller
[Can not Test] Platform ul test code can not run with this arch

Change overview

Move platform code form src/control/java to src/platform/android
added PlatformManager implement
added android test app and pass all ultest of platform
As discussed in slack, I keep using inject mode in control, so that we will not drop inject mode and hope to have more and more inject mode later

Testing

Pass all ul test at src/platform/test
Some basic test with CHIPTool Android

@austinh0 ?


onStart not working

What are the conditions for onStart to be called? I am putting a simple alert in the onStart method and it doesn't call when I force start the tour

onStart should be called called when tour.start() is called (with or without force), after step 0 is shown, if the tour has never started before. As you can see below, if _current has been set, onStart won't get called.
    Tour.prototype.start = function(force) {
      var promise;
      if (force == null) {
        force = false;
      }
      if (!this._inited) {
        this.init(force);
      }
      if (this._current === null) {
        promise = this._makePromise(this._options.onStart != null ? this._options.onStart(this) : void 0);
        this._callOnPromiseDone(promise, this.showStep, 0);
      }
      return this;
    };

It does look like a bug to me after a quick look.

Is there anyway to call some JavaScript functions before step 0 is shown, even if the tour has been shown before?
I need to call some JavaScript before the tour even start. I'm having issues where the setup code (which freezes parts of the screen so there isn't any interactivity) is called, but the tour isn't starting. It's okay if the tour doesn't start but I want to make it so that the setup code isn't called unless the tour starts.
Also would you guys have any idea why the tour starts on certain machines but not others?

First of all, if you need this to work (without force), you can change that === to == and I believe it should work as described. You could call tour.restart() to have it re-run. It should re-call onStart.
I generally wrap my tours in another object if I need to handle some setup task, or check for things.

My original code was in a wrapper:
initTour = function(){
setUp();
            // Instance the tour
            tour = new Tour({
                // more settings
            });
            tour.init();
            tour.setCurrentStep(0);
            tour.start(true);
      }

However, there would be instances where the setUp method would be called yet the tour wouldn't run. I couldn't figure out why the tour didn't run, I could not consistently reproduce it, but there are enough people reporting it for us to believe it's a big enough issue.
I tried this:
    initTour = function(){
            // Instance the tour
            tour = new Tour({
                onStart: function(tour) {
                    setUp();
                },
                // more settings
            });
            tour.init();
            tour.setCurrentStep(0);
            tour.restart();
      }

And then I call the initTour function, however this doesn't work either. This is after I changed the === to == in the standalone code. The problem is that setUp HAS to be called before the tour is started because it is adjusting dynamic elements for the tour to attach to.

First off, I fixed onStart failure last night, and the proper fix was a bit different then the one I recommended here, so I'd recommend grabbing the latest bootstrap-tour.js.
In your code example, instead of calling setUp() in onStart(), why not call it before tour, then create your array of steps? E.g.
initTour = function(){
           var stepsArray = setUp();
            // Instance the tour
            tour = new Tour({
               steps: stepsArray,
                // more settings
            });
            tour.init();
            tour.setCurrentStep(0);
            tour.restart();
      }

I fixed this properly last night – see #397. You can get the lastest version of bootstrap-tour.js to get that fix. That said, your example code wouldn't be influenced: calling restart() would call onStart even before the fix (see #396). This sounds like a different issue at this point.

Yes you're right. You mentioned that onStart is called once the tour has started, which made me realize that onStart can't be used since my methods have to be called BEFORE the tour has started.
Is there anyway to have a function similar to onStart that is called before the tour has started? Once again, I can't put code into production because there is a bug with the tour not starting 100% of the time. I either need the tour to start 100% of the time (preferable), or I need the setup code to only run when the tour starts.


add useCallback in useSWRInfinite example when need to use setSize

Bug report
Description / Observed Behavior
What kind of issues did you encounter with SWR?
setSize infinite loop in useSWRInfinite when combine with intersection-observer
because we need to use useEffect to listen scroll,
so if use setSize() will be  infinite loop
Expected Behavior
How did you expect SWR to behave here?
maybe can add  useCallback is fine
like

const loadMore = useCallback(() => setSize((state) => state + 1), [setSize]);

Repro Steps / Code Example
Or share your code snippet or a CodeSandbox link is also appreciated!
Additional Context
SWR version.
Add any other context about the problem here.

@shuding Why did you add the documentation label in this? What documentation do you expect?

@shuding Why did you add the documentation label in this? What documentation do you expect?

@koba04 Sorry I should explain a bit more. I think he's referring to this example: https://swr.vercel.app/examples/infinite-loading, which is very basic. Added the label since we need an example specifically for infinite scroll cases (load more when you hit the page bottom).

@koba04 Sorry I should explain a bit more. I think he's referring to this example: https://swr.vercel.app/examples/infinite-loading, which is very basic. Added the label since we need an example specifically for infinite scroll cases (load more when you hit the page bottom).

Thank you! I agree with you 👍 The documentation is very helpful.

Thank you! I agree with you 👍 The documentation is very helpful.

You can bind click events with the setSize  function, and in the intersection-observer callback You can trigger it with click event.

Just use an extra "should" state with an useEffect
const [shouldLoadMore, setShouldLoadMore] = useState(false)

const observer = useMemo(() => {
  return new IntersectionObserver(([e]) => {
    if (e) setShouldLoadMore(e.isIntersecting)
  })
}, [])

useEffect(() => {
  if (shouldLoadMore)
    pages.setSize(size => size + 1)
}, [shouldLoadMore])


How do I scan for missing poster/cover?

I moved a few files around in the database but forgot to move the associated cover art?  Do I have to delete the nfo file and redo the scan to get a cover art?

If the cover are is in the same folder and has the movie ID in it's filename, you should be able to just right-click and re-scan files.  Otherwise, check Advanced Options in settings, and in the missing thumbnail view, you should be able to right-click on movie thumbnail and Import Cover Image.

Thx, if I delete the nfo file, would it initiate a rescan?  Or how would I trigger an auto pull of cover art at this point?

You can so the same thing by selecting Advanced options in settings, right click on a thumbnail and selecting Regenerate metadata.  It basically deletes the nfo and rescans.  That should download a cover if needed.


Point-in-time restore is supported for general-purpose v2 storage accounts in the standard performance tier only. Only data in the hot and cool access tiers can be restored with point-in-time restore.

Point-in-time restore is supported for general-purpose v2 storage accounts in the standard performance tier only. Only data in the hot and cool access tiers can be restored with point-in-time restore.

Learn Build status updates of commit a9b9b79:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

articles/storage/blobs/point-in-time-restore-manage.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

@normesta

Can you review this PR?
IMPORTANT: When this content is ready to merge, you must add #sign-off in a comment or the approval may get overlooked.

#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off


(797) Make check answers task dynamic

Context

The check answers task list section is currently hardcoded to show the URL and a 'cannot start yet' tag
Changes in this PR

Make the check answers task list section dynamic, only showing the URL when ready, and updating the tag to reflect the actual status

Currently it only checks for oasysConfirmed since the Referral model is incomplete
Screenshots of UI changes
Before

After
Unready

Ready

Release checklist
Release process documentation
As part of our continuous deployment strategy we must ensure that this work is
ready to be released once merged.
Pre-merge

[ ] There are changes required to the Accredited Programmes API for this change to work...

[ ] ... and they have been released to production already

Post-merge

[ ] This adds, extends or meaningfully modifies a feature...

[ ] ... and I have written or updated an end-to-end test for the happy path in the Accredited Programmes E2E repo

[ ] This makes new expectations of the API...

[ ] ... and I have notified the API developer(s) of changes to the contract tests (Pact), or the API is already compliant

[ ] Manually approve release to preprod
[ ] Manually approve release to prod

 Disable "Check your answers" link on the referral page before all sections have been filled in (UI)


Update reference from getrector.org to getrector.com

Since getrector.org now redirected to getrector.com, the content referenced to getrector.org can be updated to getrector.com

All checks have passed 🎉 @TomasVotruba it is ready for review.

Thank you :clap:


backport PKCS12_parse() MAC check to 3.0

This change backports cfd24cde81aa5f63dba41ddcde0fa3c5d64e1db0 to the openssl-3.0 branch. Based on my research about PKCS#12 parsing in FIPS mode I figured this could be useful for general public.

This is not a bug fix so IMO not eligible for 3.0.

It is specifically for FIPS mode, where PKCS12_parse ddoes not work with MAC.
On Wed, Sep 6, 2023, at 18:24, Tomáš Mráz wrote:

This is not a bug fix so IMO not eligible for 3.0.
—
Reply to this email directly, view it on GitHub https://github.com/openssl/openssl/pull/21983#issuecomment-1708708786, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAWMMDFVAOLSQN5KCJFJQD3XZCPT3ANCNFSM6AAAAAA4NNVYZA.
You are receiving this because you authored the thread.Message ID: @.***>

This is not a bug fix so IMO not eligible for 3.0.

I agree. This is a feature and not eligible.

This is not a bug fix so IMO not eligible for 3.0.

I agree. This is a feature and not eligible.

Oh well, at least I tried. What irks me about this situation is that OpenSSL 3.0 users (which arguably is the mainstream version these days, or becoming one soonish) running in FIPS mode cannot reasonably parse PKCS#12 structures, being basically victims of the PKCS#12 standard being behind the times.
Maybe the top-level PKCS#12 APIs should be able to take library context to be able to override the provider ? (But that too could not end up in 3.0 being a feature)

Maybe the top-level PKCS#12 APIs should be able to take library context to be able to override the provider ? (But that too could not end up in 3.0 being a feature)

PKCS12_create_ex() takes a libctx already exists in 3.0 and can be used as an alternative to PKCS12_create
https://www.openssl.org/docs/man3.0/man3/PKCS12_create_ex.html
So you have the option of using that call to override FIPS for PKCS12 usage.

...hmm but that doesn't help much with parsing :-(

Ah... but PKCS12_init_ex() can be used, so something like this should work:
    PKCS12 *p12 = PKCS12_init_ex(NID_pkcs7_data, mylibctx, mypropq);
    if (d2i_PKCS12_bio(mybio, p12) == NULL)
        goto err;
    PCKS12_parse(p12, ...);

Yep, this works.
  OSSL_LIB_CTX *mylibctx = OSSL_LIB_CTX_new();
  PKCS12_init_ex(NID_pkcs7_data, mylibctx, "provider=default");
  ...

The PKCS#12 API is still a bit confusing, I think. One would have to use the same approach for PKCS12_create_ex()  even though it takes the library context argument because there it is used only for the inner PKCS#7 structures it seems.
Thanks.

Actually, no. PKCS_create*() does not take the PKCS12 handle, rather it creates it, so cannot be used in FIPS mode to create PKCS#12 structure with outer level MAC it seems.


Feature Request: enable outputting cxx.cc itself to the target directory

Hello
Recently I've integrated cxx into a legacy visual c++ application and had a couple of difficulties around linking. It seems that rust and also cxx use a specific version of the msvc runtime library that will cause many conflicts when you link statically - things along the lines of runtime library conflicts and _ITERATOR_DEBUG_LEVEL macros.
After trying to link to cxxbridge1.lib a couple of times using different settings what I ended up going with making my crate a cdylib and to also compile cxx.cc directly into my c++ project.
My suggestion for this issue is to add an option somewhere that will have cxx or cxx-build output cxx.cc alongside cxx.h in the target/cxxbridge output directory so that users of this project will have greater flexibility in how they integrate cxx their projects. Currently my solution isn't future proof - but if you were to accept this proposal then it would be.

I would prefer not to make this change, because what you've described sounds like a build system that does not understand C++ that depends on Rust that depends on C++, not anything specific to how cxx.cc is linked by the cxx crate. For example if one of cxx's dependencies, or a dependency of one of your other Rust dependencies, added a dependency on some piece of C++ code it needs, you'd be back to square 1. I don't believe that papering over this case in the way that you suggest would be beneficial. I know that better approaches are possible because other build systems don't appear to have difficulty with this case.

@dtolnay The issue boils down to cc-rs defaulting to the MSVC /MD runtime library. Integrating cxx into a /MT static library is therefore impossible in the current state. This is configurable through the following cc-rs function, perhaps you could expose a way to allow the /MT library to be used https://docs.rs/cc/latest/cc/struct.Build.html#method.static_crt

That is not something that would need to be exposed in cxx. You need to use RUSTFLAGS='-C target-feature=+crt-static' and cc will pick it up automatically.

@Racinettee how did you get cxx.cc to compile using your C++ build system?

@ultimaweapon we didn't special in terms of flags or manual modifications if that's what you mean. we copy cxx.cc directly into a location in our source tree where our build system (visual c++)  expects to find it.
if it helps and you're also using visual c++ - we use the c++latest flag and we're on the v143 tool-chain now, but i dont imagine those matter too much, and nothing else stands out to me as particularly important to make it compile

Thank you for the information.


[ Curious ]  Where the guide folder on this repo  readable on freecodecamp website?

I'm curious where the guide on this repo exactly readable on the FreeCodeCamp site

@bauripalash, currently at https://guide.freecodecamp.org. Please use the forum or gitter for asking questions like these. This is for code-base related bugs and issues only.


[Utopia-402] new home page

Description
This PR includes the following proposed change(s):

update ppq home page to latest design

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Refactoring / Documentation
[ ] Version change

How Has This Been Tested?
Please describe the tests that you ran to verify your changes.
Development Dependency Working Agreement

[x] My code DOES NOT include the importing of new dependencies into the DPIA ecosystem
[ ] My code DOES include the importing of new dependencies into the DPIA ecosystem
If new dependencies are being introduced to the DPIA ecosystem:
[ ] The functionality of the dependency drastically reduces code complexity and makes my changes more easily maintainable and readible
[ ] The dependency being introduced does not contain multiple layers of nested dependencies introducing maintainability complexity to the DPIA ecosystem

Frontend Development Changes

[ ] N/A
[x] I have performed a self-review of my own code
[ ] I have made corresponding changes to project documentation or diagrams that reflect my changes
[ ] New and existing unit tests pass locally with my changes
[ ] My code follows Airbnb React Style Guidelines

API Development Changes

[x] N/A
[ ] I have performed a self-review of my own code
[ ] My code follows standards and practices outlined in the BC Government API Development Guidelines
[ ] New and existing unit tests pass locally with my changes
[ ] My changes includes Swagger documentation updates that reflect the changes I am introducing

Definition of Done

The new component needs padding and margin. Also, the blue needs to be darker (is that the color code from the design?). We should also make the button background transparent to allow for the background color of the component to show through as per the design.

@Bottle7 yes, the color i got from Figma and convert to rgba format from hex

We still need padding inside of the new component.

We still need padding inside of the new component.

I add padding to the ppq div element, please let me know if it is ok @Bottle7

@binggaofw The padding better now; however, the PPQLandingPage layout is broken now.

close this pr for now as we are implementing feature flaging


Call transcribe_url_callback when callback is provided

Proposed changes
Reported by @jjmaldonis:
If a callback is passed to prerecorded.transcribe_url, can we automatically call prerecorded.transcribe_url_callback rather than throw an error? I think we know what the caller is intending to do, so let's do it. Same for prerecorded.transcribe_file.
Context
NA
Possible Implementation
NA
Other information
NA

This PR should address the problem. If it doesn't, we can reopen.
https://github.com/deepgram/deepgram-python-sdk/pull/182
Cc: @jjmaldonis


Extension matcher documentation doesn't generate a link to the xds Matcher type

Title:  Extension matcher documentation doesn't generate a link to the xds Matcher type
Description:
In the extension matcher docs, the documentation generator doesn't generate a reference link for the xds.type.matcher.v3.Matcher.
The actual sphinx doc looks like this. Note How there is no :ref: generated for the xds.type.matcher.v3.Matcher type:
.. code-block:: json

  {
    "matcher": "{...}",
    "xds_matcher": "{...}",
    "extension_config": "{...}"
  }

.. _envoy_v3_api_field_extensions.common.matching.v3.ExtensionWithMatcher.matcher:

matcher
  (:ref:`config.common.matcher.v3.Matcher <envoy_v3_api_msg_config.common.matcher.v3.Matcher>`) The associated matcher. This is deprecated in favor of xds_matcher.

.. _envoy_v3_api_field_extensions.common.matching.v3.ExtensionWithMatcher.xds_matcher:

xds_matcher
  (.xds.type.matcher.v3.Matcher) The associated matcher.

.. _envoy_v3_api_field_extensions.common.matching.v3.ExtensionWithMatcher.extension_config:

extension_config
  (:ref:`config.core.v3.TypedExtensionConfig <envoy_v3_api_msg_config.core.v3.TypedExtensionConfig>`, *REQUIRED*) The underlying extension config.

Huh, looks like the autogenerated docs aren't WAI here:
https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/common/matching/v3/extension_matcher.proto.html
cc @adisuissa @phlax thoughts?

at a guess its because its viewed as "external" to the docs, and only the internal protos are linked
i can look into this further to confirm or otherwise
if my suspicion is correct i can possibly look at adding a dictionary of proto -> urls so we can add links to upstream protos

if my suspicion is correct i can possibly look at adding a dictionary of proto -> urls so we can add links to upstream protos

If possible, it would probably be helpful to include the docs for all the xds repository types.

@phlax can you pls no-stale this?


how to export sherpa-ncnn in pruned_transducer_stateless5

how to export sherpa-ncnn in pruned_transducer_stateless5.Have some example？

how to export sherpa-ncnn in pruned_transducer_stateless5.Have some example？

Please have a look at our documentation:
https://k2-fsa.github.io/icefall/model-export/export-ncnn.html
We only support the following models for exporting to ncnn

LSTM transducer
ConvEmformer
Streaming zipformer

how to export sherpa-ncnn in pruned_transducer_stateless5.Have some example？

Please have a look at our documentation: https://k2-fsa.github.io/icefall/model-export/export-ncnn.html
We only support the following models for exporting to ncnn

LSTM transducer
ConvEmformer
Streaming zipformer

Is there any plan to support？

how to export sherpa-ncnn in pruned_transducer_stateless5.Have some example？

Please have a look at our documentation: https://k2-fsa.github.io/icefall/model-export/export-ncnn.html
We only support the following models for exporting to ncnn

LSTM transducer
ConvEmformer
Streaming zipformer

Is there any plan to support？

Sorry, it is not in the plan.
Streaming zipformer is our current best-performing model and we already support it.

how to export sherpa-ncnn in pruned_transducer_stateless5.Have some example？

Please have a look at our documentation: https://k2-fsa.github.io/icefall/model-export/export-ncnn.html
We only support the following models for exporting to ncnn

LSTM transducer
ConvEmformer
Streaming zipformer

Is there any plan to support？

Sorry, it is not in the plan.
Streaming zipformer is our current best-performing model and we already support it.

ok，thank

Please see
https://github.com/k2-fsa/icefall/blob/master/egs/librispeech/ASR/RESULTS.md#pruned_transducer_stateless8-zipformer--multidataset
It is also zipformer.

By the way, the RESULTS.md from each recipe keeps the results of each experiment.
You can find the results from there.

thank

@csukuangfj  can the new zipformer (offline version --causal 0 ) be exported with ncnn ? If so, how  and where can i find some code to do offline decoding with the ncnn version ?

the new zipformer (offline version --causal 0 ) be exported with ncnn

No.
sherpa-ncnn supports only streaming models.

@csukuangfj can the new zipformer (offline version --causal 0 ) be exported with ncnn ? If so, how and where can i find some code to do offline decoding with the ncnn version ?

All supported models are listed in the documentation.
https://k2-fsa.github.io/icefall/model-export/export-ncnn.html
If a model is not listed here, then it is not supported.

Thank you for the lightning fast answer!
I suppose the "zipformer" in the document only refers to the older version (transducer_stateless7_streaming ) and not to the new zipformer with causal=1 ) ?

I suppose the "zipformer" in the documentation only refers to the older version

Yes, you are right. The new version ./zipformer has not been supported yet, but it is in the plan.

We will first support sherpa-onnx. There is an on-going issue about it. https://github.com/k2-fsa/icefall/issues/1094
After it is supported, we will start to support sherpa-ncnn.
Could you try sherpa-onnx first once it is supported?

@csukuangfj  Now that the Onnx export is done, what is needed for the ncnn export for the zipformer2 ? How can we (try to) help ?
Could we use https://github.com/Tencent/ncnn/blob/master/tools/onnx/onnx2ncnn.cpp ?


Add draft 2023 roadmap

Thrashing out a draft of this somewhere we can add comments, without polluting the main repo

One overall comment - this feels like the "context" and we should explicitly state what the immediate upshot of this is (sort of the precursor to creating the Issues and Projects that relate to these activities)

Produce a version 5.Blah in which we will do X, Y, Z
Work in parallel on version 6.Blah in which will do A, B, C

...and I think there is a thing 3 which is to get some community feedback on the bug backlog and have a formal prioritization of that.

As @mwadams said at https://github.com/idg10/reactive/pull/2#issuecomment-1443387450

this feels like the "context" and we should explicitly state what the immediate upshot of this is (sort of the precursor to creating the Issues and Projects that relate to these activities)

Does the addition of the ADR setting out the V.next steps in a lot more detail address this as far as that version is concerned?
It doesn't solve the whole problem, because you suggested:

Produce a version 5.Blah in which we will do X, Y, Z
Work in parallel on version 6.Blah in which will do A, B, C
...and I think there is a thing 3 which is to get some community feedback on the bug backlog and have a formal prioritization of that.

Although having said that, the subsections under "Plans for upcoming release" are sort of this (except for that last suggestion).
However, I've noticed that https://github.com/dotnet/reactive/discussions/1868 says:

Publish a Roadmap for 2023 (covering the .NET 8.0 LTS release train) & beyond

I guess we need to fill this out a bit more to cover that too.
I'm wondering if I should actually add an executive summary type section right at the start of the roadmap (even before the Current problems section) outlining these milestones, so anyone coming to this roadmap can immediately see what we've got planned, followed immediately by a high level description of what problems the plan aims to solve.

Yes, that last suggestion would cover it, I think.

As @mwadams said at #2 (comment)

this feels like the "context" and we should explicitly state what the immediate upshot of this is (sort of the precursor to creating the Issues and Projects that relate to these activities)

Does the addition of the ADR setting out the V.next steps in a lot more detail address this as far as that version is concerned?
It doesn't solve the whole problem, because you suggested:

Produce a version 5.Blah in which we will do X, Y, Z
Work in parallel on version 6.Blah in which will do A, B, C
...and I think there is a thing 3 which is to get some community feedback on the bug backlog and have a formal prioritization of that.

Although having said that, the subsections under "Plans for upcoming release" are sort of this (except for that last suggestion).
However, I've noticed that dotnet#1868 says:

Publish a Roadmap for 2023 (covering the .NET 8.0 LTS release train) & beyond

I guess we need to fill this out a bit more to cover that too.
I'm wondering if I should actually add an executive summary type section right at the start of the roadmap (even before the Current problems section) outlining these milestones, so anyone coming to this roadmap can immediately see what we've got planned, followed immediately by a high level description of what problems the plan aims to solve.

Part of the roadmap, I think, is the diagram we've been working on with Waller. Being able to pull together all the disparate parts of the ecosystem onto a single roadmap and say "these are the things we know about and can support" also gives the community the chance to chip in with "what about X?"


Support Arrays for the Map scalar functions

Is your feature request related to a problem or challenge?
As @alamb mentioned in https://github.com/apache/datafusion/pull/11361#discussion_r1672743943, we should support not only scalar lists but also arrays for the map and make_map functions. However, I encountered some challenges, which I detailed in https://github.com/apache/datafusion/pull/11361#discussion_r1674235631. I left the disabled test cases in https://github.com/apache/datafusion/blob/1dfac86a89750193491cf3e04917e37b92c64ffa/datafusion/sqllogictest/test_files/map.slt#L200-L214.
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response

take

MakeMap is possible to be removed after #11434, so you can find out array support for MapFunc only

Apologies for the delay, prioritizing this issue.

Sharing my observation from DuckDB, both Map exprssion and map function are meant to do two different things.
But in Datafusion we are using map function to support Map expression.
In case of DuckDB Map expr & map function would yeild different result for same input.
Shouldn't we have two different function to support these two use cases ? Or any ideas on how should we support both usecases using using single map function.
Observations
D create table t as values
· ('a', 1, 'k1', 10, ['k1', 'k2'], [1, 2], 'POST', [[1,2,3]]),
· ('b', 2, 'k3', 30, ['k3'], [3], 'PUT', [[4]]),
‣ ('d', 4, 'k5', 50, ['k5'], [5], null, [[1,2]]);

D select map(col4, col5) from t;
┌───────────────────────┐
│    map(col4, col5)    │
│ map(varchar, integer) │
├───────────────────────┤
│ {k1=1, k2=2}          │
│ {k3=3}                │
│ {k5=5}                │
└───────────────────────┘

D select Map {col4: col5} from t;
┌────────────────────────────────────────────────────────┐
│ main.map(main.list_value(col4), main.list_value(col5)) │
│               map(varchar[], integer[])                │
├────────────────────────────────────────────────────────┤
│ {[k1, k2]=[1, 2]}                                      │
│ {[k3]=[3]}                                             │
│ {[k5]=[5]}                                             │
└────────────────────────────────────────────────────────┘

In the meantime i will continue my investigation.

Shouldn't we have two different function to support these two use cases ? Or any ideas on how should we support both usecases using using single map functio

In general, I think DataFuion strives not to invent / determine semantics, so in my opinion we should follow other implementations rather than having our own custom behavior whenever possible.
I think by default we should follow the postgres model
If postgres doesn't support maps / this syntax then I think we should follow the DuckDB model

@dharanad  I think select Map {col4: col5} from t; is equivalent to SELECT MAP([col4], [col5]);, so we just need to rewrite the syntax the existing map function.

Shouldn't we have two different function to support these two use cases ? Or any ideas on how should we support both usecases using using single map functio

In general, I think DataFuion strives not to invent / determine semantics, so in my opinion we should follow other implementations rather than having our own custom behavior whenever possible.
I think by default we should follow the postgres model
If postgres doesn't support maps / this syntax then I think we should follow the DuckDB model

I totally agree with you.

@dharanad I think select Map {col4: col5} from t; is equivalent to SELECT MAP([col4], [col5]);, so we just need to rewrite the syntax to the existing map function.

Thanks for clarifying, i will have to say that i totally misunderstood the problem here. Should have checked what Postgres/DuckDB were doing & clarified the problem.