Improve performance for mpath Discovery method

Description:
In case you have many devices on the worker nodes and you create a new POD with PVC, before this fix the discover would go on all the PVCs devices on the node and trigger sg_inq on each one of them in order to search for the device for the new PVC (wwn). Well in most use cases there is no need to do this overhead during attach since the device probably doesn't exist on the host or if it is then it should appear in multipath -ll already. So to improve performance we decided to skip this deepDiscovery(by sg_inq on all the devices). And it will already be covered anyway after the rescanAll.
Improve discovery performance by applying the following things:

Do NOT do sg_inq deepDiscovery during the RescanAll in the attach flow.
Its waist of time to do deepDiscovery in this case, since its already done after the RescanAll when discovering the new device.

Do sg_inq deepDiscovery only in the following cases:
2.1. During the attach after the RescanAll
2.2. When Discover for cleanup(unmount flow).

Remove double sg_inq validation if discover during deepDiscovery

Remove Stat(mpath) validation during Discover, since its already validated during GetWwnByScsiInq() or DiscoverBySgInq()

Fix interfaces, UT and fakes according (Still need to add more UT)

How to test

create POD1 with PVC1 and check in the flex logs that the volume didn't had to discover using DiscoverBySgInq, means you should see the message "mpath device was NOT found for WWN [%s] in multipath -ll"(where %s is the wwn of the PVC). but you should not see any try to run sg_inq on this WWN (debug for GetWwnByScsiInq with this mpath device)

Try to get in multipath -ll with bad devices, then run create POD2 with PVC2, this this case there is high chance that multipath -ll will not present the right WWN for the new PVC2 device.  if so then you should see in the log that flex tried to identify this PVC2 device by using sg_inq. You can see in the log message like "Warning: device [%s] found for WWN [%s] after running sg_inq"

This change is

Coverage decreased (-0.04%) to 54.782% when pulling 399ae7333de104c9269a6d33c93e67c425dd5d19 on fix/no_deep_discover_during_RescanAll_in_attach_flow into c6dd8f80ff9402b3e8c82e65554aeedd0d6cea4d on dev.

Coverage decreased (-0.02%) to 54.801% when pulling 12f961645fa2030a35881a6ddae6cf203dd8d790 on fix/no_deep_discover_during_RescanAll_in_attach_flow into c6dd8f80ff9402b3e8c82e65554aeedd0d6cea4d on dev.

 besides the license headers?

Review status: 0 of 8 files reviewed at latest revision, 2 unresolved discussions, some commit checks failed.

fakes/fake_block_device_mounter_utils.go, line 1 at r2 (raw file):

// Code generated by counterfeiter. DO NOT EDIT.

Did you remove the license header? Why does it look that way?

fakes/fake_block_device_utils.go, line 1 at r2 (raw file):

// Code generated by counterfeiter. DO NOT EDIT.

same here, what happened to the license header?

Comments from Reviewable

Review status: 0 of 8 files reviewed at latest revision, 2 unresolved discussions, some commit checks failed.

fakes/fake_block_device_mounter_utils.go, line 1 at r2 (raw file):
Previously, ranhrl (Ran Harel) wrote…
Did you remove the license header? Why does it look that way?

all files with "fake" names are generated automatically by go Counterfeiters.   its a moking system.   its not our code its self generated.  no sense to put there license because it will always be overwritten by couterfeiter.
agree?

fakes/fake_block_device_utils.go, line 1 at r2 (raw file):
Previously, ranhrl (Ran Harel) wrote…
same here, what happened to the license header?

same as above

Comments from Reviewable

Testing update about this PR:

Automation testing passed OK
Sanity tests passed OK
Acceptance tests with longevity on RHEL and Ubuntu passed OK

So, we decided to merge this PR into dev branch.


Impossible to add comment to blurblog story

Unless I miss something obvious it's impossible to add a comment unless :

you're OP and add a comment at moment of posting
you reply to an existing comment

Noticeably, I don't see how to add a comment a posteriori to my own posts.

Yep, you've found a long standing bug. I kind of left that one in until somebody said something. Problem is that I'm not sure exactly how to expose the "edit comment" UI without adding something. It's also not something that happens very often. This can only happen after you share a story from the app and then go to your blurblog and want to add a comment. Which is a reasonable thing to do, but I'd have to build a bit of interface for it and I don't have the time now to do that for what seems to be a lower priority.
But thanks for being the first to point this out!


[ci:component:github.com/gardener/terraformer:v2.2.0->v2.11.0]

*Release Notes:
The version for the `equinixmetal` Terraform provider plugin has been updated to `3.1.0`.

A bug has been fixed preventing to use Terraformer with a Terraform version >= 0.13.

The following terraform provider plugins are updated:
- hashicorp/terraform-provider-aws: 3.32.0 -> 3.54.0

The terraform `azurerm` provider plugin is updated from `v2.36.0` to `v2.68.0`.

Once the `azurerm` provider plugin is updated from `v2.36.0` to `v2.68.0` the `skip_provider_registration` flag  in the `provider` section need to be set to `true`.

The following terraform provider plugins are updated:
- hashicorp/terraform-provider-aws: 3.18.0 -> 3.32.0

The following terraform provider plugin is updated:
- aliyun/terraform-provider-alicloud: 1.124.0 -> 1.124.2

The following terraform provider plugin is updated:
- aliyun/terraform-provider-alicloud: 1.121.2 -> 1.124.0

The terraform version for the alicloud, all, aws, azure, gcp, openstack, slim images is updated:
- hashicorp/terraform: 0.12.29 -> 0.12.31

Terraformer now copies Terraform's error outputs to `/terraform-termination-log` to make it available in the containers termination message for better analyzing and more readable error messages (e.g. in the Shoot status).

Terraform provider of Alicloud is upgraded to 1.121.2.

The following terraform provider plugins are updated:
- hashicorp/terraform-provider-google: 3.59.0 -> 3.62.0
- hashicorp/terraform-provider-google-beta: 3.59.0 -> 3.62.0

The aws provider has been downgraded from `3.32.0` to `3.18.0` due to issue with additionally required permission for the AWS accounts.

The golang has been updated to `1.16.2`, the alpine has been updated to `3.13.2`.

The following terraform provider plugins are updated:
- hashicorp/terraform-provider-aws: 3.18.0 -> 3.32.0
- hashicorp/terraform-provider-google: 3.27.0 -> 3.59.0
- hashicorp/terraform-provider-google-beta: 3.27.0 -> 3.59.0

The Terraformer now instantly removes its finalizer from the state `ConfigMap` if the state is empty and `destroy` is called. A separate Terraform `destroy` is not executed.

Terraformer will now publish an additional image without any pre-installed terraform plugins.

Provides support for the Equinix Metal provider, which replaces the Packet one

/close in favor of #155


[Feature Request]: Create draft PR in Azure Repos

Is your feature request related to a problem? Please describe.
I want to create a PR in draft.
Describe the solution you'd like
A clear and concise description of what you want to happen.
e.g. If you want a new command to be added, what would it do?
Additional context
Add any other context or screenshots about the feature request here.

This is dependent on an API version step up and will be halted till then.

@geverghe For approach for sign off:
Command to create a PR in draft mode:  az repos pr create --draft-mode + other params as is
--draft-mode : Use this flag to create the pull request in draft/work in progress
mode.
Command to update a PR:
az repos pr update --id 123 --publish
az repos pr update --id 123 --mark-as-draft
--mark-as-draft : Mark a published pull request as draft again.
--publish : Publish a pull request which is in draft mode.
Change in table output:

seems a bit odd that in output we are translating something like
isdraft==false ==> published
isdraft==true ==> isDraft
also looking at json response I think we only have isDraft in the body. So then why not have only --is-draft as an option and user can pass True/ False for update?
cc: @geverghe  @atbagga

seems a bit odd that in output we are translating something like
isdraft==false ==> published
isdraft==true ==> isDraft
also looking at json response I think we only have isDraft in the body. So then why not have only --is-draft as an option and user can pass True/ False for update?
cc: @geverghe @atbagga

Thought of that already. Here is the down side. --is-draft then becomes a compulsory parameter. Even if you have to change the title you will have to explicitly say -
az repos pr update --id 123 --title "new title" --is-draft false

Updated the proposal comment as per Gaurav's Suggestion above.

@geverghe
The concern here is that --is-draft is a flag in create PR like --open and other flags in create whose presence/absence signifies true or false instead of explicit value.
In update command it is string which can be true or false.
Reasoning: Create Pr has multiple other boolean flags so --is-draft being a bool is consistent in the command scope.
Why can't we have same in update PR command? Because then --is-draft will have a default value and it will become compulsory for user to provide it during every update so we do not override the current state of PR with the default value. e.g. A PR is published and I want to update the title. In the command --is-draft is true by default.
The command az repos pr update --title "abc" should have sufficed but this will change my PR to draft mode.

Apologies for not responding earlier. Think I am hitting some rule exception in my email and this didn't come into my attention.
This issue was brought up earlier as well and the guidance provided by Azure CLI is - https://github.com/Azure/azure-cli/blob/dev/doc/authoring_command_modules/authoring_commands.md#registering-flags
This means that non mandatory boolean attributes have three states - true, false and not set. When this model was first proposed, the feedback was that the implementation was presence indicated true and non-presence false. This is not the case. There are three states and we should be following this guideline. Therefore, for this scenario, it should ideally be:

az repos pr --draft - Indicates PR should be set as draft
az repos pr --draft true - Indicates PR should set as draft
az repos pr - Indicates that there is no mention of draft and therefore this attribute is never set.
az repos pr --draft false - There is explicit intent to set as false.

Now in case of mandatory boolean attributes, then presence and non presence equates to true /false. However, this does not mean users cannot provide the explicit intent via value as well. Consider draft was a mandatory variable - even then we need to support all of the following:

az repos pr --draft - Indicates PR should be set as draft
az repos pr --draft true - Indicates PR should set as draft
az repos pr - Indicates PR should not be set as draft
az repos pr --draft false - Indicates PR should not be set as draft.

Let me know your thoughts on this. We should agree on a standard and move forward. Otherwise, commands within our own extension will be inconsistent.

Sounds Good. Fixed in PR #542

This will be available in the next release. 0.5.0


added multicast reception in the udp effect

the default udp.json will listen to unicast on port 2391 (as it used to)
the new udp-mcast.json will listen on multicast <IP_ADDRESS>:2801 (but can be changed easily)

Thank you, can´t wait to see what we will get next :)

@penfold42
another question, makes it sense to add these udp listener to the booteffect array at the config (and Hyperion)? You can´t use both. Maybe to add a additional array would make sense?
Not sure where are the programming edges here.

I'm not sur I understand your question

ah!
this is like the receiver from proto/json just for udp and you will se also a booteffect at this device if the "sender" sends it?

Yep
It effectively turns Hyperion into a UDP driven led device just like my ESP8266 based wifi receiver.
I had 3 Pi's running the multicast effect and a perl script running on a 4th device sending 3 bytes per led (RGB) data over multicast to all Pi's at the same time.
My perl script/4th box could just as easily be a another Hyperion set up to use the UDP led device type

@penfold42
the idea is to refactor this to a UDP receiver (like "Boblight Server")
As part of a cleanup - this is a "misused" option (booteffect array)

I don't know what a booteffect array is.
We could write a native built in QT based UDP receiver if there's enough demand.
It could be implemented as a boblight level thing or we could consider it as a grabber

you use this
	"bootsequence" :
	{
		"color" : [0,0,0],
		"effect" : "Rainbow swirl fast",
		"duration_ms" : 3000,
		"priority" : 700
	},

Which is a little bit "misused"
A general udp receiver component like this:
	"udpServer" :
	{
		"port" : XXXXX
	},

would be cleaner

I havent been using it as a boot effect (although you could)
I've just been using Hyperion-remote when I need the UDP listener running

Yeah the handling could be "better" :)

hope to clear this conversation a bit:
current state of "udp listener":
its implemented as effect. This means it is absolutely OK to use it on every place where effects are usable - that includes the bootsequence of course.
If this effect should have some different handling than the other effects (e.g. own section in config), than we build an inconsistency in effect handling. I'm voting  against that special treatment!
But yeah something is wrong here:
an effect uses setcolor/setimage to tell hyperion how the leds should glow. The udp effect is be definition an effect, but it does to much. It takes network input and then send it to lights. If we wouldn't have json- proto- boblight server, I would say all things are right.
But we have already codeparts that makes  network to light. The new "network to light" function for udp protocols should be implemented in the as any other "network to light" stuff in hyperion.
=> @penfold42 realized this already

It could be implemented as a boblight level thing

and have the right solution (please don't do this as grabber, thx). Thing is, doing  this in python as an effect is much faster, then doing this the clean way as own hyperion network service.


Option to hide ignored tests from test output

Describe what you would like to see
When running something like go test -run MyTestName ./... -json | gotestfmt -hide no-test I'd like to be able to only see the tests that are run in the output, currently if you use the -hide empty-packages argument you still see a bunch of output from tests that were not run along with testing: warning: no tests to run

Describe how you would use it
To run single tests and and be able to only see the output from that test

I would definitely be willing to take a shot at it! I should have some free time next week to take a crack at it


The results look a bit blurry.

Thanks for your contribution and the fantastic work! I tried running your code with the default settings according to your instructions on README. However, I failed to get an output as clear as shown in your paper. My results:

While the results from the paper are:

I would like to know why my reconstructed checkerboard was blurred? Thanks for your help!

Thanks for your interest! I'm re-testing the default setting.
In my previous reproducing experiments, I was able to reconstruct it correctly.
On step 15k, the board is not fully reconstructed

But on step 85k, the board will be reconstructed

In my experience, the initialization of the SDF field indeed affects the final convergence but the default setting should be able to produce the final results because I've fixed the random seed.

Thanks for your reply! I am also re-testing the default settings. Just to clarify, is the board supposed to be fully reconstructed only after stage 2? or stage 1 is sufficient for reconstructing the fine details of the board?

The board is supposed to be fully reconstructed in Stage I.

Hi, I'm able to reproduce the results using the default setting. Maybe you can recheck if there is anything changed in the config files. Meanwhile, you may also reduce this parameter
https://github.com/liuyuan-pal/NeRO/blob/59d1b1f5a50e4e022cd53d77b1a27d8144008825/configs/shape/real/vase.yaml#L16
it means how many steps we fix the inv_s. You may reduce it to 0 or 5000, which would help you to reconstruct the board.

Thanks for your help! I clone this repo and run the command python run_training.py --cfg configs/shape/real/vase.yaml again. The only modification is that I replace np.bool with np.bool_ to fit numpy version (1.25) I used. However, my results are still blurry this time:

Because I use pytorch 2.0.1, which is different from yours. To check if the problem is caused by python env, I create a new conda env with pytorch 1.11 and rerun the code, but the results are still a bit blurry:

I don't know the exact reason to cause this inconsistent results but you may try to reduce the parameter freeze_inv_s_step to 5000 or 0, which would improve the reconstruction quality of the board.
The reason for this phenomenon is that the appearance (shading) networks are too strong so it is able to fit the appearance on incorrect geometry. While freezing the inv_s makes the geometry converge slower so that appearances are fitted on incompleted geometry.

OK, many thanks for your suggestion. I will try it out.


Space above removed header

When setting the showSectionHeader to false, there is still some padding above the header in the list view.
StickyAzOptions(
     listOptions: ListOptions(
     showSectionHeader: false,
),

This differs from the demo where when the header is hidden, there is no spacing between each A-Z group.

i tried set to false the showSectionHeader and doesn't show any padding on that, so consider to send the full code of yours so i can help u detect the issue

this was a problem only on IOS and now it was fixed.


Integration + iOS.Framework

This is part 2 of ios packaging. Part 1 is here https://github.com/BabylonJS/BabylonNative/pull/291
The goal of this PR is to add a new target that builds a Framework usable for Xcode directly (drag and drop .framework in the source tree) or with a cocoapod.
The artifact contains this:

Headers from the Include directories of plugins, core, polyfills, and napi
.js resources
a big BabylonNative static library that is the concatenation of all .a (plugins, dependencies, ...)
Xcode automaticaly recognise the format of this archive so end user doesn't have to set path for includes, name of lib,...

When building the framework, napi.h is replaced with a special napi.h with relative to framework path. See Part1.
The archive is published as a build artifact. Part3 will be cocoapods and binary feed.
It's not possible to use this framework before the Part1 is merged.
Once you have downloaded/installed the framework in the Xcode project, just add some includes.

Some words on cocoapods, I'll elaborate more in the next PR. A pod, basically, needs to files :  a podspec and a pod. The podspec specifies what the pod is composed of. It's used for publishing, and consuming the pod. a pod file contains the list and path to podspec that will be integrated in a named project.
BabylonNative.podspec:
Pod::Spec.new do |s|
    s.name              = 'BabylonNative'
    s.version           = '0.0.4'
    s.summary           = 'Babylon Native.'
    s.homepage          = 'https://github.com/CedricGuillemet'

    s.author            = { 'ceguille' =><EMAIL_ADDRESS>}
    s.license           = { :type => 'MIT' }

    s.platform          = :ios
    s.source            = { :http => 'file:///Users/cedricguillemet/dev/podtest/BabylonNative/buildios/Integration/Debug-iphonesimulator/BabylonNative.framework.zip'}
    s.resource = '**/*.js'
    s.ios.deployment_target = '12.0'
    s.ios.vendored_frameworks = 'BabylonNative.framework'
end

Two most important things here:
the s.source and s.resource. s.resource is needed so xcode can merge the resource in the app bundle.
s.source here points to a local directory that I'm using for my tests.  This path can points to a GitHub release (so we need to update it when we do a GitHub). My task now is to see if it can point to a CI Feed: when merging into master, a build is triggered and it's the artifact from that master version that I'd like to put in the feed. User would have to do a 'pod update' to update to the latest good version of BabylonNative.
It also possible to point to the GitHub and trigger a local build when installing pod.
This file will be at the root of the repo.
End user will only have to update its own pod file like this:
target 'name_of_app' do
   pod 'BabylonNative', :git => "https://github.com/BabylonJS/BabylonNative.git"
end

Once the pod is installed, load the workspace that cocoapods has generated.
For reactnative integration, it's possible to write its own podspec that will not use the resources from the framework (and use npm instead for Babylonjs).
It's possible to release the framework as an npm package. You then need a podspec to points to that framework (s.source = ".... /node_modules/BabylonNative/). You pod would point to your local podspec.
user would do :
npm install
pod install

Instead of pointing the podspec to a separate BabylonNative NPM package, we could include the framework directory in the @babylonjs/react-native package, and point source to ., and then just reference the framework folder (if I understand correctly). An example of this can be found here: https://medium.com/@aliakhtar_16369/distribute-framework-with-cocoapods-locally-and-remotely-4a39f95b3077 if you scroll down to the Consume Framework Using Local CocoaPods section.

'we could include the framework directory in the @babylonjs/react-native package', do you mean adding the prebuilt framework in the Github repo? Who would be responsible for building and pushing the changes? Having the framework in an npm allows the CI to build and push a package automatically while still having a podspec/pod in @babylonjs/react-native that points to a local directory (subdir in node_modules).

Packages not a priority for now. Closing.


Update cxxopts to v3.0.0

Closes #83

There is no feature in the new version that we want to use ?

The argc value and argv buffers are not modified. It will probably make the refactoring cleaner and easier.

Ok !


"PlantUML installation not found" despite PlantUML being globally installed

Hello! I am trying to run /home/user/dephpend/bin/dephpend uml --output=uml.png --no-classes src but I'm getting the following error:
Something went wrong, this shouldn't happen. Please take a minute and report this issue: https://github.com/mihaeu/dephpend/issues

PlantUML installation not found.

[/home/user/dephpend/src/OS/PlantUmlWrapper.php at line 57]

I've checked that PlantUML is installed globally so I'm not sure what it could be. dePHPend is working fine for other options (metrics, etc.) Thanks!
[user@endeavour]$ pacman -Qs plantuml
local/plantuml 1.2022.7-1
    Component that allows to quickly write uml diagrams

Didn't get it working and no longer need to, but if I do need it in the future I'll probably try with a different OS


Initializing with pre-specified population

I would like to know how to initialize my population with n members which have pre-specified structure. For example, if i want my initiate population to have 15 members all of which have same expression eg: 1+x. Are there Pysr options to do it or is it something need to be updated. Thank you.

This feature does not yet exist, but it would certainly be nice to add it or simplify existing alternatives. The current strategy is basically to initialise the state manually. Alternatively you could run a search for 1 iteration, and then manipulate the saved state to specify individual members of the population. On the PySR discussions page there are some threads about this too.

Thank you for the suggestion @MilesCranmer. I will be checking the documentation and do the changes.

The initialisation function is here: https://github.com/MilesCranmer/SymbolicRegression.jl/blob/master/src/Population.jl#L36-L62
which gets called here: https://github.com/MilesCranmer/SymbolicRegression.jl/blob/cd23a6e25c64d00565c3ae3905d06dc3c63033ed/src/SymbolicRegression.jl#L775

Hello @MilesCranmer,
I have managed to populate the Population using the following code: Population{Float32, Float32, Node{Float32}}([member, member1], 2)
I would like to inquire about where the initialization begins within the SymbolicRegression.jl framework, particularly with respect to functions such as _main_search_loop, _warmup_search, _initialize_search, and _create_workers.
Our intention is to modify the process starting from the initial population phase, allowing PySR to search for equations based on a predefined expression given. Could you kindly clarify which function is responsible for calling the Population struct and initiating its initialization?. Additionally, is it possible to adjust the complexity, such that the search begins with a higher value, for instance, 7 or 9, instead of the default starting point of 1 (a float value)?


Add alternatives "Git reset first commit"

Thanks for the PR, but please have a look at contributing.md and make the changes are per it.

@hemanth , check please

Neat.


Rework repo buttons (#20602, #20718)

Backport #20602 and #20718 to 1.17. Translations can be backported separately.

Translations can be backported separately.

I think that is a recipe for forgetting to backport them at all. We should do this now assuming that a good number of translations have editor.add_file set in them in main

Pushed to 1.17.2 - not enough translations ready.

Yeah, will check for translations

@jolheiser was so nice to backport the translations. We can land this if we land https://github.com/go-gitea/gitea/pull/20834 immediately afterwards.

make lgtm work


Allow user to configure path to tsc and tsserver

For now we are shipping a TypeScript compiler and language service along with the plugin. However, some users have the tendency to stick with certain version of TypeScript and want to use that version in the plugin. It would be good to give them the ability to configure the TypeScript version they want to use.
Current state
Currently we have a setting "typescript_proc_file" to specify path to tsserver.js. There is no similar setting for tsc.
# retrieve the path to tsserver.js
# first see if user set the path to the file
settings = sublime.load_settings('Preferences.sublime-settings')
proc_file = settings.get('typescript_proc_file')
if not proc_file:
	# otherwise, get tsserver.js from package directory
	proc_file = os.path.join(PLUGIN_DIR, "tsserver", "tsserver.js")

Propose
There should be two settings "typescript_tsc_file" and "typescript_tsserver_file" with the following possible values:

"plugin": use the tsc/tsserver shipped along with the plugin
"global": use the tsc/tsserver in their PATH (most likely installed with NPM)
path to executable tsc/tsserver (e.g. tsc.exe) or tsc.js/tsserver.js.

would be very helpful, trying to migrate back to sublime, and found that out, atom's typescript package is already able to do it (or may be they just using bleeding edge typescript like i do?).
current work around to some people is to delete files in this package on local computer and create symlink to your global files (works for me).

+1. I don't want to install tsc globally and use that same version in each separate typescript project I have.


Missing ASR Preview Replication Appliance networking requirements

ASR can be deployed in two flavours: classic and preview.
ASR Classic documentation ( https://docs.microsoft.com/en-us/azure/site-recovery/vmware-azure-architecture
)  says " For exhaustive list of URLs to be filtered for communication between on-premises Azure Site Recovery infrastructure and Azure services, refer to network requirements section in the prerequisites article. "
ASR Preview documentation only mentions outbound-connectivity-for-urls ( https://docs.microsoft.com/en-us/azure/site-recovery/vmware-azure-architecture-preview#outbound-connectivity-for-urls
) and there is not reference to additional networking requirements for the replication appliance. Is this correct ?
Doesn't the ASR Preview Replication Appliance have specific networking requirements ?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4fe9dc87-ff8e-7312-a5c5-f81de7363633
Version Independent ID: 96464964-9d83-439f-58f2-0412a1d8a390
Content: VMware VM disaster recovery architecture in Azure Site Recovery - Preview - Azure Site Recovery
Content Source: articles/site-recovery/vmware-azure-architecture-preview.md
Service: site-recovery
GitHub Login: @V-SSenthilna
Microsoft Alias: v-ssenthilna

@giovannifl For the appliance’s network requirements, please refer to this section – https://docs.microsoft.com/en-us/azure/site-recovery/deploy-vmware-azure-replication-appliance-preview#network-requirements.

Thanks @SadiqhAhmed-MSFT Please update
https://docs.microsoft.com/en-us/azure/site-recovery/vmware-azure-architecture-preview#outbound-connectivity-for-urls to to look on your links for additional details (i.e. appliance networking reqs)

@giovannifl The Outbound connectivity URL section which you are looking at, talks about the outbound connectivity from source machines. It is not the list of URLs which should be allowed for the appliance.
The only section for appliance’s network requirements is this link – https://docs.microsoft.com/en-us/azure/site-recovery/deploy-vmware-azure-replication-appliance-preview#network-requirements.

@giovannifl We haven't heard back from you, so we will now close this issue. If this remains an issue, please reply and we will gladly continue the discussion.


Running testpappl.exe on windows lacks a large number of dll files

Hello, do you need me to compile and install these dll libraries so that they can run normally

In windows10 environment

There is a “copy-dlls.bat” script in the vcnet directory that can be used to copy the dependent DLLs into the build directory.


[BUG] Tree node problem

when my tree node like this
`
treetitle

content1
content2

title2
`
i hope to it can show like a tree ,because title2 and content1 and content2 all belong to treetitle ，but i found it just can show title2,can't show content1 and content2 .Only if content1 and 2 belong to title2,it can show an unbroken tree . i think it is a bug.
could you repair it ? thank you

i want to it can show like this

i want to it can show like this


Bug in BR-NL-10 test [SI-UBL-2.0-NLCIUS]

I suspect a bug in schematron/si-ubl-2.0/si-ubl-2.0-nlcius.sch.
More specifically in the test of BR-NL-10.
The way this test is implemented implies that the element in question is optional. This would be an incorrect interpretation of the BR-NL-10 as written in the NLCIUS spec. To be fair, the relevant text in the NLCIUS spec. is unclear, but the comment at BT-47 in the same document confirms that this element is meant to be mandatory

Do you have an example of a document that incorrectly passes this test? Simply leaving out BT-47 like in https://github.com/SimplerInvoicing/testset/blob/master/SI-UBL-2.0/SI-UBL-2.0_BR-NL-10_error_customer_no_companyid.xml does not pass validation (if buyer and supplier are both in NL).

Thanks for pointing that out. After going through the code and the BR-NL-10 text again, I think this was a misunderstanding on my part. Apologies.


OnIdiom not working with the new RowDefinitions ColumnDefinitions syntax

xaml is not accepting OnIdiom  when used with the new syntax of RowDefinitions and ColumnDefinitions https://github.com/microsoft/microsoft-ui-xaml/issues/673:
RowDefinitions="{OnIdiom Tablet="*,*,auto", Phone="*,*,*"}"
Tried the following workaround but crashing the app on Xamarin and throwing the exception:

System.Reflection.TargetInvocationException: 'Exception has been thrown by the target of an invocation.'

 <Grid.RowDefinitions>
            <RowDefinitionCollection>
                <OnIdiom x:TypeArguments="RowDefinitionCollection">
                    <OnIdiom.Phone>
                        <RowDefinition Height="*"/>
                        <RowDefinition Height="*"/>
                        <RowDefinition Height="*"/>
                    </OnIdiom.Phone>
                    <OnIdiom.Tablet>
                        <RowDefinition Height="*"/>
                        <RowDefinition Height="*"/>
                        <RowDefinition Height="auto"/>
                    </OnIdiom.Tablet>
                </OnIdiom>
            </RowDefinitionCollection>
        </Grid.RowDefinitions>

#673 is a proposal, it hasn't been integrated in the product yet. What is OnIdiom?

Looks like a xamarin forms type. Should this issue be on their repo? https://github.com/xamarin/Xamarin.Forms

@StephenLPeters I didn't know OnIdiom was used by Xamarin.Forms only then I will move it there, but https://github.com/microsoft/microsoft-ui-xaml/issues/673 I am already using the new syntax with Xamarin.forms project.


npm v2.2.0 is missing index.js for esm and cjs folders

Is there an existing issue for this?

[X] I have searched the existing issues

Description of the bug
The v2.2.0 version released last week is missing the index.js for the lib/esm and lib/cjs folders. This results in the programmatic API not working when importing.
https://www.npmjs.com/package/git-cliff?activeTab=code
Steps To Reproduce
import { runGitCliff } from 'git-cliff';

Expected behavior
The files should be included in the release
Screenshots / Logs
No response
Software information

Operating system: N/A
Rust version: N/A
Project version: 2.2.0

Additional context
No response

I was alerted of this as well just now in my own support server for cliff-jumper. This is my fault from an earlier PR and I will be working on getting a new PR out ASAP (starting work right now)

Will create a patch release soon.


Figure out how to help Fall River with data import

Notes

Next step to this would be reach out to Fall River's IT lead on Monday to figure out what they're used to doing when working with vendors to move around data; what their tech preferences/habits are like; what their blockers are

We've moved from Fall River to New Bedford as our main candidate for second district, and we're facing a more specific technical challenge with New Bedford, so closing this issue.


Import EdDSA code from Google Tink, v1.6.1

This is definitely worth reviewing as two separate commits.

Next steps as I see 'em:

[ ] Mechanically convert test code from .java to .kt.
[ ] Mechanically convert production code from .java to .kt. This will be annoying to perform, but the upside is we’ll have a single implementation that works on all of our target platforms.
[ ] Change packages from com.google.crypto.tink.subtle to app.cash.zipline.loader.internal.tink.subtle.
[ ] Merge the zipline-eddsa submodule into the existing zipline-loader submodule.

At that point we’ll be able to start using EdDSA inside ZiplineLoader. I intend to follow the plan in our internal design doc.

I'm curious if it will be long term easier to maintain/upstream any changes if we keep the separate zipline-eddsa submodule vs merging it into Loader. The primary benefit to merging it as I understand it would be faster builds or are there other benefits?

I don’t want to expose any public API for this. The fact that we ship our own EdDSA code rather than getting it from the platform is an implementation detail!


Any reason why do not store the kotlin code under src/main/kotlin

instead on src/main/java?

No, it's just because this is a fork of a Java project and when you create a new module Android Studio will create a src/main/java folder.


2.4.4 not in maven central?

Hi Brett,
I know it takes some time until artifacts are in maven central, but I guess 2.4.4 should be available by now (two days since 0b485eb91a90dc6646c819d3bef5009b046bbcf1)?

It is published and announced.

I know, but it's not available: http://search.maven.org/#search|gav|1|g%3A"com.zaxxer" AND a%3A"HikariCP" still shows 2.4.3 as latest version. I guess something went wrong with publishing.

Try now.  It may not show up in the search engine yet, but it should resolve from sonatype.

It's downloadable now. Yesterday it was not. I was to impatient. Sorry.


"nvm use" and" nvm install" set a successful exit code when failing

Issue:

When nvm use is passed a non-installed version of Node, it correctly outputs an error message, but then sets its exit code 0, indicating success. Similarlly, when nvm install is passed an unrecognized version, it correctly outputs an error message, but also sets exit code 0, indicating success.
These make using nvm in a CI environment, or as part of any other automation, difficult because it is not easy to detect when they've failed.
How To Reproduce:
On Windows 10 21H1:

Uninstall all versions of Node
Install nvm via nvm-setup
Run Windows PowerShell as Administrator:

PS C:\WINDOWS\system32> nvm version
1.1.9
PS C:\WINDOWS\system32> nvm list

No installations recognized.
PS C:\WINDOWS\system32> nvm use 14
node v14.18.3 (64-bit) is not installed or cannot be found.
PS C:\WINDOWS\system32> echo $LASTEXITCODE
0
PS C:\WINDOWS\system32> nvm install carrot
Unrecognized version: "carrot"

Running version 1.1.9.

Usage:

  nvm arch                     : Show if node is running in 32 or 64 bit mode.
  nvm current                  : Display active version.
  nvm install <version> [arch] : The version can be a specific version, "latest" for the latest current version, or "lts" for the
                                 most recent LTS version. Optionally specify whether to install the 32 or 64 bit version (defaults
                                 to system arch). Set [arch] to "all" to install 32 AND 64 bit versions.
                                 Add --insecure to the end of this command to bypass SSL validation of the remote download server.
  nvm list [available]         : List the node.js installations. Type "available" at the end to see what can be installed. Aliased as ls.
  nvm on                       : Enable node.js version management.
  nvm off                      : Disable node.js version management.
  nvm proxy [url]              : Set a proxy to use for downloads. Leave [url] blank to see the current proxy.
                                 Set [url] to "none" to remove the proxy.
  nvm node_mirror [url]        : Set the node mirror. Defaults to https://nodejs.org/dist/. Leave [url] blank to use default url.
  nvm npm_mirror [url]         : Set the npm mirror. Defaults to https://github.com/npm/cli/archive/. Leave [url] blank to default url.
  nvm uninstall <version>      : The version must be a specific version.
  nvm use [version] [arch]     : Switch to use the specified version. Optionally use "latest", "lts", or "newest".
                                 "newest" is the latest installed version. Optionally specify 32/64bit architecture.
                                 nvm use <arch> will continue using the selected version, but switch to 32/64 bit mode.
  nvm root [path]              : Set the directory where nvm should store different versions of node.js.
                                 If <path> is not set, the current root will be displayed.
  nvm version                  : Displays the current running version of nvm for Windows. Aliased as v.

PS C:\WINDOWS\system32> echo $LASTEXITCODE
0

Expected Behavior:

I would expect any failure of install or use to result in the exit code being anything other than 0, to indicate failure.
PS C:\WINDOWS\system32> nvm use 14
node v14.18.3 (64-bit) is not installed or cannot be found.
PS C:\WINDOWS\system32> echo $LASTEXITCODE
1

PS C:\WINDOWS\system32> nvm install carrot
Unrecognized version: "carrot"
...
PS C:\WINDOWS\system32> echo $LASTEXITCODE
1

Thank You:
nvm-windows is an invaluable part of the Node ecosystem on Windows. Thank you very much for creating and maintaining it!

This seems reasonable. I'll look into this in the next round of updates. If someone wants to submit a PR before then, it would save some time.

Apologies that I have not had an opportunity to work on a PR for this. Commenting to make sure that this issue doesn't get closed for being stale specifically because you mentioned interest in it. Thanks!

I encountered this issue as well today, are there plans to fix it? The workaround for me currently is to run node --version after nvm use and parse the result, but that seems unnecessarily complicated!

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

(I know this is a philosophical issue but I think you need to get rid of the bot that pretends that things are fixed when they aren't: it's hostile to users 😭)

@mcdurdin I've tried to configure that bot more times than I can count. It's not perfect, but it does close a lot of abandoned issues that I request more information for and never get.
I'll re-open this so I'll remember it for the next release. It may be a while though, seeing as our code-signing cert got locked and I'm still trying to get it unlocked with the vendor.

@coreybutler Thank you!


react-native link react-native-mathjax not working for android

When i try react-native link react-native-mathjax i am getting the fallowing response in command line.
"rnpm-install info Platform 'ios' module react-native-mathjax is already linked "
Tried "react-native link react-native-mathjax android" this command also but getting the same response.
Fallowing are the verions used.
react-native-cli: 2.0.1
react-native: 0.56.0

if you have already linked it then it's fine, just ignore the message.
the problem is does it work after linking the module? if not what's the error?

Hi calvinkei,
The question was i am not able to link to android. Even when i try to link to android also with fallowing syntax react-native link react-native-mathjax android its giving me same error.
Please clarify.

I have the same problem. react-native-mathjax is not work for me.

please update to 2.0.0 (react-native link not required!)


Can't bind popover's "open" prop to a boolean

Describe the bug
Can't bind a popover's open prop to a boolean in order to keep track of the open and closed states.
To Reproduce
<script lang="ts">
  let isPopoverOpen = false
</script>

<Popover bind:open={isPopoverOpen}>
   ...
</Popover>

doesn't work.
Library version
2.0.0

<Popover> doesn't have an open prop, I recommend going back to the docs: https://svelte-headlessui.goss.io/docs/2.0/popover#showinghiding-the-popover


Scroll over code block

It's not possible to scroll over code block while model outputs tokens.

Hey, this should be fixed now.


What happens when certificate expire?

Hi all,
Looking at the documentation for key management and the expiry dates, can you please detail what the standard operational procedure would look like to renew certificates before they expire?
My understanding is that, before the delegation/targets keys expire 3 years after creation, they must be rotated. Rotating the keys will invalidate the old keys, making the previously signed data untrusted, and will involve having to re-sign all data with the new keys. Is that correct?
Many thanks.

That is correct. Every content signed with the expired/old rotated keys must be resigned.

What can be done if the metadata for the targets key expires?  Is there a way to extend/reset that expiration?  I can rotate the targets key locally but I cannot publish it.  Seems like anything I try generates an error, "Metadata for targets expired".


SecretsManager: delete_secret should throw an error for 0 day Recovery

Fixes #6461
FYI @rafcio19 - do you see any issues with this?

You are right @bblommers, with RecoveryWindowInDays=0 triggering the issue. I was trying it with other integers outside 7-30 acceptable window 🙈


Update My_login .html

Here, I made some changes because it was not looking well before but i fixed the position and i added some tags as well.
Related Issue
Since it was a great but it needed some improvement that after seems to be easy.
Fixes:  #(issue no.)
Description
There are putting  tags which was only needed to seems easy.

Type of PR

[ ] Bug fix
[ ] Feature enhancement
[ ] Documentation update

Screenshots / videos (if applicable)

Checklist:
" tags"
"Fixes"

[ ] I have made this change from my own.
[ ] I have taken help from some online resources.
[ ] My code follows the style guidelines of this project.
[ ] I have performed a self-review of my own code.
[ ] I have commented my code, particularly in hard-to-understand areas.
[ ] I have made corresponding changes to the documentation.
[ ] My changes generate no new warnings.
[ ] I have tested the changes thoroughly before submitting this pull request.
[ ] I have provided relevant issue numbers and screenshots after making the changes.

Please note that this is not in line with our code of conduct. To ensure clarity and proper assignment tracking, please attach the issue number to your PR description. This helps us confirm that the issue is assigned to you. Kindly update your PR with the issue number and resubmit.
Example:
Fixes #123 - Brief description of the fix.


Improve Jenkins pipeline

clean before checkout
fix stage names
add back designs

[skip ci]

Blocked on #777

Also, Vitor, you need to add [skip ci] to the actual commit body, not the PR body

Also, Vitor, you need to add [skip ci] to the actual commit body, not the PR body

Oh, thanks for letting me know.


tests: big deflake attempt

Includes:

Attempt to kill misbehaving subprocesses in TestDaggerUp and similar harder, see https://dagger.cloud/dagger/traces/3e12c1a3aa6b40119ab74e136dad9e50?span=1e20b5449908ff06#0f8f4e636ab28886:L198
testctx.T should have a way to allow tests to repeat

Getting test repetition working is not very fun, unsurprisingly, go's wonderous test framework does not want such a convenience.

Grr, this way of doing things really does not work. The problem is t.Parallel, and generally the following constraint on Run:
// Run may be called simultaneously from multiple goroutines, but all such calls
// must return before the outer test function for t returns.

Because we don't know ahead of time how many attempts we might be doing, we can't actually satisfy the above constraint - when the tests are started in parallel, we can't really solve the above at all.
t.Parallel is horrendous and such a painful construct to try and do things with. gotestsum might be the thing to do.


*: Add mimalloc feature to tikv_alloc

Signed-off-by: Yilin Chen<EMAIL_ADDRESS>
What have you changed? (mandatory)
This PR adds mimalloc support to TiKV. To use mimalloc as the allocator of TiKV, set MIMALLOC environment variable when building TiKV, like:
MIMALLOC=1 make dist_release

The mimalloc crate used is https://github.com/gnzlbg/mimallocator whose maintainer is the same as jemallocator's.
What are the type of the changes? (mandatory)

New feature (change which adds functionality

How has this PR been tested? (mandatory)
Untested
Does this PR affect documentation (docs) or release note? (mandatory)
No
Does this PR affect tidb-ansible update? (mandatory)
No
Refer to a related PR or issue link (optional)
Benchmark result if necessary (optional)
A little improvement under sysbench while a little regression under go-ycsb. It should be investigated more.

Add a few positive/negative examples (optional)

Great work!!!
Do we statically link mimalloc?

does mimalloc support outputting statistics like jemalloc?

You could enable the stats feature of the mimalloc dependency to print the stats on exit.

Do we statically link mimalloc?

Yes. I believe it is statically linked.

Now, stats feature is enabled.

do we need to implement fetch_stats and dump_stats？

@siddontang The stats feature just prints statistics to stderr when the program exits.
The mimallocator crate does not support runtime statistics yet but it is possible to call mi_stats_print to collect statistics.

After a second test,  I am confident it's not a mistake that sysbench and go-ycsb give different results.
I think it just tells us mimalloc does not work better in every case. When value size differs, allocation size and frequency can differ too. No allocator can handle every case well.

(Just to clarify I don't mean the stats feature should always be enabled 😅)

Maybe related: according to https://github.com/microsoft/mimalloc/issues/11 mimallocator was not built for many short lived small allocations.

(Just to clarify I don't mean the stats feature should always be enabled )

Sorry, at first I thought the stats feature of mimalloc is similar with that of jemalloc :sweat_smile:

CI failed @sticnarf


JavaScript - Number Recognizers for Portuguese

Ported all Number extractors and parsers (Portuguese) from C# to TypeScript/JavaScript implementation

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: lucasmarambio sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Add error property to action

Thank you for your contribution! :thumbsup:
Please makes sure that these checkboxes are checked before submitting your PR, thank you!
Related issues: https://github.com/piotrwitek/typesafe-actions/issues/116

[x] PR have a corresponding issue which is approved and it's linked above
[x] Rebase before creating a PR to keep commit history clean
[x] Clear node_modules and reinstall all the dependencies: npm run reinstall
[x] Run ci-check npm script: npm run ci-check

For bugfixes:

[x] Add at least one unit test to cover the bug that have been fixed

For new feature:

[ ] Update API docs and tutorial
[ ] Add examples to demonstrate new feature
[x] Add type unit tests
[x] Add runtime unit tests

@piotrwitek First draft. I was trying to order the function overloads correctly, but it has been non-obvious to me what counts as most specific, etc. If it's not obvious, I'm rather new to TypeScript and in the process of migrating a code base over from Flow (which is how I found this library to begin with). So I appreciate your help and patience here.
I didn't update the docs in this PR yet. I was planning to do that in a separate PR and include further proofreading changes. The next step will be figuring out createAction but that can be a separate PR, if you prefer. The intended distinction between createAction and createStandardAction is still a bit fuzzy to me.

@sjsyrek After few review comments should be good to merge

The intended distinction between createAction and createStandardAction is still a bit fuzzy to me.
Good that you're mentioning that. The core distinction is really the preference of the user to use generic type arguments (similar to redux-actions) or more inference driven approach, where types are inferred solely from function signature (args and returned object)

@piotrwitek Should be good now

Thanks @sjsyrek for adding a great new feature!


[SPARK-17810] [SQL] Default spark.sql.warehouse.dir is relative to local FS but can resolve as HDFS path

What changes were proposed in this pull request?
Always resolve spark.sql.warehouse.dir as a local path, and as relative to working dir not home dir
How was this patch tested?
Existing tests.

CC @vanzin -- I think your solution at https://github.com/apache/spark/pull/13868#discussion_r82252926 is better, will probably switch to it.
CC @avulanov and @koertkuipers

Test build #66459 has started for PR 15382 at commit 4fb0e58.

Test build #66459 has finished for PR 15382 at commit 4fb0e58.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66459/
Test FAILed.

Merged build finished. Test FAILed.

i think working dir makes more sense than home dir. but could this catch people by surprise because we now expect write permission in the working dir?

Good point. The docs all say it should be working dir, so I was following the apparent intent along the way here. I don't know if one is more surprising than the other.

Test build #66500 has started for PR 15382 at commit 67c2b58.

Update on my thinking:
Docs say the default should be "spark-warehouse" in the local working directory. The original code did establish a default on the local file system... though in the home dir, not working dir. So I still favor fixing that, but that's a small, orthogonal question.
Although the default appears to be intended to be on the local FS, I don't think it must be. @avulanov mentions it might be on S3; the Hive warehouse dir is on HDFS too by default, in comparison. I don't have evidence that this has been prohibited so I guess we shouldn't assume this is a local path.
It seems simple to default to "spark-warehouse" in the local working dir of whatever file system is being used, be it local or HDFS. That means SQLConf shouldn't specify a default with any URI scheme or path. SQLConf should then resolve it, though, at the moment there's no good way to plumb through a Hadoop Configuration. (I just made a new default one in here for the moment.)
Then we're back to an interesting detail, that Hadoop Path's toURI won't add a trailing slash in the case of a local file. Maybe it doesn't in general since it can't always tell whether a path is a directory or not. Local URIs however will have a trailing slash. It's probably best to leave the behavior and adjust the tests to not expect a trailing slash, as the original code seemed to (?)
Does this still work on Windows? As far as I can tell it would.
Is it OK to instantiate FileSystem.get(new Configuration()) in SQLConf? there's no easy access to the actual conf, but, it is only trying to read the default file system which shouldn't depend on Spark-specific Hadoop conf.
I think this change actually restores the ability to specify a warehouse path with a URI scheme. This second, it will be interpreted as a file path only if you write "file:/foo". However there's no way to write a relative file URI, so no way to guarantee that a default of "spark-warehouse" is a relative path on the local FS. After this change, HDFS users would see that this correctly defaults to "spark-warehouse" in an HDFS working directory.

Test build #66501 has started for PR 15382 at commit cd4b4b9.

Test build #66500 has finished for PR 15382 at commit 67c2b58.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66500/
Test FAILed.

Test build #66501 has finished for PR 15382 at commit cd4b4b9.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66501/
Test FAILed.

i don't think there is such a thing as a HDFS working directory, but that probably means it just uses the home dir on hdfs (/user/<user.name>) for any relative paths

OK, now using Utils.resolveURI. This means all inputs without a scheme resolve as file: URIs, including the default "spark-warehouse". This will continue to resolve to the local working dir by default, as per docs.
Inputs with a scheme can be specified, still, to point to HDFS or S3.
I adjusted the tests to, I think, further standardize the handling of URIs.

Test build #66580 has started for PR 15382 at commit 71f124a.

Test build #66580 has finished for PR 15382 at commit 71f124a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66580/
Test FAILed.

Test build #3310 has started for PR 15382 at commit 71f124a.

Test build #3310 has finished for PR 15382 at commit 71f124a.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

Test build #3314 has started for PR 15382 at commit 71f124a.

Test build #3314 has finished for PR 15382 at commit 71f124a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #3322 has started for PR 15382 at commit 71f124a.

Test build #3322 has finished for PR 15382 at commit 71f124a.

This patch fails Spark unit tests.
This patch does not merge cleanly.
This patch adds no public classes.

Test build #66810 has started for PR 15382 at commit ef7a141.

Test build #66810 has finished for PR 15382 at commit ef7a141.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66810/
Test PASSed.

Merged build finished. Test PASSed.

@vanzin this may be more like what you're suggesting. I think the Windows issue makes this a little bit more complex than what you're describing but I think this ends up being workable.
@avulanov FYI
@mridulm I think the change in behavior from what's in master now is:

Default to a local filesystem path (again) regardless of Hadoop config
Default to spark-warehouse in working dir not user dir (as in the docs)

The net change is actually pretty minor after all.

Test build #67191 has started for PR 15382 at commit 69c5383.

Test build #67191 has finished for PR 15382 at commit 69c5383.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67191/
Test PASSed.

LGTM. This should restore the behavior pre- 11a6844 and be compatible with Windows.

Great. Is it going to 2.0.x, too?

cc @yhuai

I think this would have to go into 2.0.x as I think it's a regression from 2.0.0, in that the default location moved from a local file to HDFS for many users, and, it's an HDFS path that can't be created.
I'll leave it open for a few more days.

Merged to master/2.0


Example on how to use non-bundled AWS SDK Clients & Utils

We lack an example on how to use clients and SDKs that are not provided with LLRT.

Hi @richarddavison,
I think this initiative will make LLRT more user-friendly.
From what I have worked on locally, here are some points to consider:

When adding a new @aws-sdk module, which version should I adopt?
How do I prove that the module I want to bundle is "popular" and get approval for bundling?

I can't contribute directly to this issue because I can't write English, but I would like to contribute by following the documentation and working on the @aws-sdk/lib-storage bundle.
Please note that this text was translated by DeepL :)

Hi, I tried using client-ecs in LLRT, and managed to get it working.
The main points were:

set bundle target platform to browser
pass credentilas and region explicitly from Lambda environment variables
mark uuid as external

Here's the actual code that I used: https://github.com/tmokmss/cdk-lambda-llrt/pull/4/files

When I wanted to use LLRT as a runtime for CLI, the ideal platform would be node even though LLRT doesn't support full Node.js features. Why does it require to set browser?
Also, bundled SDKs are hard to control their versions by users. It would be better to provide flexibility to let users decide whether they want to import SDKs explicitly, or use the bundled versions.

Hi @riywo. Thanks for your comment, these are valid concerns .LLRT provides SDKs for since they have been optimized specifically for optimal performance on Lambda (with some dependencies replaced with Rust implementations, such as XML parsing). The reason why we target browsers is that the browser API more closely matches what you get out of the box with LLRT, and wouldn’t work with AWS SDK otherwise (without explicit config, mainly due to the lack of http and https modules). We’ll update the repo with an example on how to add non-provided SDKs. We may also add a non-bundled build of LLRT in the future (already possible to do if building from source with a feature flag). Hope this makes sense.

Fixed in: https://github.com/awslabs/llrt/pull/410


Make bootnodes flag a list

Rationale
Currently only one bootnode address can be provided. This is not ideal as the bootnode may crash or some other issues may happen.
Task
Add support for multiple bootnode addresses deliminated by sensible character

Implemented in #57


Introduces universal windows platform, improvements and bugfixing

This PR is a replacement for https://github.com/wildabeast/BarcodeScanner/pull/230:

Adds support for windows platform and shares proxy code between windows8 and windows platforms
Adds some minor UI improvements
Fixes bug when cancel button crashed application (reported as https://github.com/wildabeast/BarcodeScanner/issues/148)
Enables autofocus on supported camera devices
Includes winmd source as subproject instead of shipping prebuilt winmd component

This is the first PR I've seen for this plugin which builds and runs without errors in the 8.1 (windows) environment on a Windows Phone. Unfortunately, I'm still getting a green screen when opening the scanner.

Tested on Windows :+1:

Does anyone have this working on Windows Phone?
@vladimir-kotikov better yet - is it intended to?

Hey @dougestey, just tried again to run on Lumia 1520, all works fine.
I recently saw several issues with MediaCapture API on different devices, maybe this is your case. Could you please try it on another device?

@vladimir-kotikov thanks so much for your response - I will build on a Lumia 930 today and get back to you. I believe my issue might be that I'm building for 8.1/windows instead of 8.0/wp.

@vladimir-kotikov thanks for the tip. this PR (and now master) does build on the 930 on 8.1/windows. Now I just need to figure out why I can't get it to focus.

@dougestey for diagnosing focus problems - try to put several breakpoints around this line and check if your device support auto/continuous focus modes.

@vladimir-kotikov the 930 is passing all of those checks, and I'm able to console.log() the focusSettings object, which reveals that continuous mode is properly set - it's just not working in practice (this is strange since we essentially have the same device.)
Anyway, I was able to create a focus button and manually call focusAsync(), which works, so I'm happy: https://github.com/dougestey/phonegap-plugin-barcodescanner/commit/b8fc83b6f230be8d64504c6a4779c7b370116c94
It seems the MediaCapture API is still a bit buggy, so I am willing to chalk my problems up to that and call it a day: https://marekcodes.wordpress.com/2015/02/23/the-horror-of-the-new-windows-phone-8-1-camera-api/

I am using the plugin on Windows 8.1 and it seems to have bug. I am able to start the camera to scan but it opens a black screen we the cancel button at the bottom rigth corner. When i click cancel i am able to go back to the previous page but the application crash.
Can someone please help on this issue ?
Thank you in advance


[Feature Request] @park-ui/cli

User Story
As a @park-ui/cli user
I want to be able to declare the import alias for PandaCSS styled-system location independantly of component and lib directories
Additionally I want to be able to specify the component and library directories with relative or absolute imports
What is the current behaviour
Currently the CLI expects the components and lib directories to depend on the alias as demonstrated on the code snippets and the image below.

This is what I must do (does not work for my usecase)
{
  "$schema": "https://park-ui.com/registry/latest/schema.json",
  "importAlias": "@styled-system/react-web-ui/*",
  "jsFramework": "react",
  "outputPaths": {
    "componentsDir": "@styled-system/react-web-ui/src/lib/components",
    "libDir": "@styled-system/react-web-ui/lib"
  }
}

What I would like to do
{
  "$schema": "https://park-ui.com/registry/latest/schema.json",
  "importAlias": "@styled-system/react-web-ui/*",
  "jsFramework": "react",
  "outputPaths": {
    "componentsDir": "./src/lib/components",
    "libDir": "./src/lib"
  }
}

Context
I am working in an Nx monorepo with multiple applications and packages.
My package react-web-ui contains components for park-ui, but other applications and libs may also use PandaCSS, each may have their own auto-generated styled-system folder.
Consequently, I have this tsConfig.paths defined. I'd prefer if I did not have to create additional @styled-system/react-web-ui/components/* nor  @styled-system/react-web-ui/lib/*
    "paths": {
      "@libs/core/design": ["libs/core/design/src/index.ts"], // 👈 non-buildable lib
      "@libs/testing/react": ["libs/testing/react/src/index.ts"], // 👈 non-buildable lib
      "@styled-system/react-web-ui/*": ["packages/react-web-ui/styled-system/*"], // 👈 a non-standard NX alias I created to use with PandaCSS react-web-ui autogen components
      "@zenif/react-hooks": ["packages/react-hooks/src/index.ts"], // 👈 a buildable lib
      "@zenif/react-web-ui": ["packages/react-web-ui/src/index.ts"], // 👈 a buildable lib that contains park-ui and other panda css components based on park-ui theme
      "@zenif/ts-panda-preset": ["packages/ts-panda-preset/src/index.ts"], // 👈 a buildable lib that contains a preset that extends `park-ui` preset that can be used by many apps and libs.
    }

@FranciscoKloganB
Thanks for bringing this up. I'm aware that the current setup doesn't work well with nx monorepos. However, this is a low priority for me compared to other open issues and features I want to build.
As a solid workaround, I recommend fetching the components, placing them somewhere, and then running a post script to move the files to the final destination.
Here's an example:
"scripts": {
    "park-ui": "lefthook install",
    "postpark-ui": "rsync -av --delete ./components/react/src/demos ./website/src/"
}

I know this isn't the solution you're aiming for, but it should work for now. Thanks for your understanding.

I understand. I was looking at the source code, unfortunatly, I never used effect. I wouldn't know where to start. I guess I will do the old-fashioned copy/pasta for the time being.

@FranciscoKloganB
I've updated the CLI so that a tsconfig.json is no longer required to install components. This benefits users with setups like nx workspaces that lack a tsconfig.json file.


Text append

//ISSUE
If we import a txt file Then there is no issue but if we import another file then the content of another file is appended with the first file content.
//SOLUTION
I think we should remove the first file content first otherwise it will append continuously with other files content.

I would like to resolve this issue as a participant of GSSOC'21. Kindly assign it to me.

Hey @anushka1309paul  i already did it , just waiting for assign

Ok!

Assigned for 5 days.

I am a GSSoC'21 participant @anushbhatia
Can you assign me this issue

@anushbhatia close this issue because it's already merged


[R] Fix nthread in DMatrix constructor.

Close https://github.com/dmlc/xgboost/issues/7098 .
Restricting the construction of DMatrix to be single threaded.

Is it possible to make XGDMatrixCreateFromMat_R aware of nthread setting?

That's either a new API or a breaking change.

@trivialfis I don't think changing the signature for XGDMatrixCreateFromMat_R would break any user-facing functionality. We'd just need to modify xgb.DMatrix.R. What do you think?

I was told that the C function in R is also considered as part of the interface when working on https://github.com/dmlc/xgboost/pull/6819 .  That's why the old predict C function for R is still here.

Codecov Report

Merging #7127 (551b97d) into master (caa9e52) will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #7127      +/-   ##
==========================================
+ Coverage   82.71%   82.74%   +0.02%
==========================================
  Files          13       13
  Lines        3963     3963
==========================================
+ Hits         3278     3279       +1
+ Misses        685      684       -1

Impacted Files
Coverage Δ

python-package/xgboost/core.py
83.93% <0.00%> (ø)

python-package/xgboost/sklearn.py
89.77% <0.00%> (+0.20%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update caa9e52...551b97d. Read the comment docs.

Also, let's create a new API function, XGDMatrixCreateFromMat_omp_R, that takes in the extra parameter nthread, so that R users do not lose out on the ability to use multiple threads when loading from a matrix.

Let's just change the signature for XGDMatrixCreateFromMat_R, since this function is only used by the R package.


Package broken because require('license') is in index

The license file is loaded using __export(require('./license')) on line 44 of index.js which breaks the package.
Expected behavior

The package should work. 😄
Current behavior

Steps to reproduce

npm i @pulumi/consul
add `import * as consul from '@pulumi/consul'
run

Context (Environment)

Typescript with node 15.5.1
Affected feature

@stack72 Seems like somehow the actual LICENSE file ends up in the packaged up directory, leading the module import to try loading that file instead of the license.js that's actually there.


feat(sqllab): Highlight and make actionable table name in the editor

SUMMARY
To make the editor smart, this commit highlights actionable items (such as table names) among the tokens hovered over by the mouse. When cmd + click is pressed, additional data linked to the relevant metadata is displayed in a popup.
The first item implemented in this commit is table names. If the SQL content includes a table name with a schema (e.g., main.bart_lines) or if the text matches a table name from the list of tables in the currently selected schema, it is marked as actionable. When clicked, it displays table metadata (including columns) and a preview query in a popup, similar to the left panel.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
https://github.com/user-attachments/assets/26bd4cab-bda8-4dce-93db-688d96d498e3
TESTING INSTRUCTIONS
type a sql including a table name and then mouse over the table name string
ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[x] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

/testenv up

@justinpark that looks nice!
I think I have only one comment about preview tab which is - can copy + search (instead of filter) match the pattern in samples in explore?

can copy + search (instead of filter) match the pattern in samples in explore?

@kasiazjc This preview result is output in the same way by reusing the existing preview result component.  Unfortunately, it is not compatible with the data format of the explore sample table, so it cannot be reused. Instead, how about modifying the existing SQL Lab preview result control to have a UI layout similar to the explore sample table?

honestly, that would be amazing as I think this would declutter SQL Lab a little bit! Is it something you would be able to work on?
I am also wondering about discoverability of this feature, because it can be hard to find if you dont know what you're looking for. Maybe for example in the empty empty state for SQL we could have some kind of "default doc placeholder sql comment" (sorry for the name lol), that would list shortcuts and features like the one you are implementing? What do you think?

honestly, that would be amazing as I think this would declutter SQL Lab a little bit! Is it something you would be able to work on?

Sure. Since it's separate issue, I'll push a new PR for the change.

I am also wondering about discoverability of this feature, because it can be hard to find if you dont know what you're looking for.

Of course, I have been considering the discoverability. As shown in the screenshot below, similar to existing editors, we have highlighted certain clickable keywords to provide feedback. When the cursor hovers over them, a tooltip is provided, making it easier to find the functionality.
https://github.com/user-attachments/assets/e14eb576-e201-4339-b4d9-87a059147e43

honestly, that would be amazing as I think this would declutter SQL Lab a little bit! Is it something you would be able to work on?

Sure. Since it's separate issue, I'll push a new PR for the change.

I am also wondering about discoverability of this feature, because it can be hard to find if you dont know what you're looking for.

Of course, I have been considering the discoverability. As shown in the screenshot below, similar to existing editors, we have highlighted certain clickable keywords to provide feedback. When the cursor hovers over them, a tooltip is provided, making it easier to find the functionality.
mouse-over-token-tooltip.mov

Makes sense, thank you!


Assorted clean up

This commit makes a few changes:

Removes unused file.
Fixes version typo in the README.
Add a bit to the README
Improves FunctionK memoization

We spoke and have a plan to rework this PR, porting functionality from DependentGraph to ExpressionDag, renaming to that to e.g. Dag, and adding Cache to parallel HCache and reworking Memoize to use both (with parallel implementations).


Add pipeline and events mixins

Moving these in from autumndb
Change-type: minor
Signed-off-by: Josh Bowling<EMAIL_ADDRESS>

@balena-ci retest

@balena-ci I self certify!


SwapItemThruInventory does not respect slot protection

SwapItemThruInventory does not respect slot protection. This is an issue that I have an idea of how to solve but I`m not solving for now. In the near future, it will be solved

The reason Im not doing it right now its cause it may come with a bigger update


One point support component to rule them all

[ ] #175

It would be nice if we could simplify the creation of supports by creating a custom component instead of the different components we have now
It should:

have presets to choose for the most common cases (rigid/free for example, or toggles for each translation and rotation direction)
be able to override those values for complex supports

It could look a bit like Karamba components. Custom components are a bit extra work but once we've done it once or twice it will be quick to create more.

I like it!

I like it!

Good! I thought perhaps this could be suitable for @lorinczandrea to work on? Seeing as it is not time sensitive and new for all of us. @Marco mentioned that Karamba is open source so it should be possible to look at their code to see how to create custom components. I've done custom components before as well, but it was looong ago so I don't really remember...

Note: also include possibility to rotate #175

Already implemented
https://github.com/strusoft/femdesign-api/blob/master/FemDesign.Grasshopper/Supports/PointSupportSimple.cs


Room calibration configuration for world coordinate transform.

We need a configuration store that can record information about the room setup and relative positions of the Rift sensor cameras.
After that, we'll need a room setup utility like the official Rift software has, which can find one of the headset or controllers in the view of multiple cameras simultaneously and work backward to compute the relative position of the cameras to each other, and hence camera-to-world-coordinate transforms for each camera

I made a quick sketch in the direction of a configuration store API in 07e7bc98c0b1857677c07813b3a052c3caf897e8 and 5157f4e3cb0a26099bb47f910fb1315281578856
ohmd_get_config() - a function that can read a requested configuration blob from the platform specific configuration store (
ohmd_read_file() - a function that can read the full contents of a file into a newly allocated buffer.

So, when it comes to configuration i think we should look into the bigger picture to extend to other drives as well.
As mentioned on IRC, i think we should make sure there is a 'config-less' single camera setup where calibration would be done 'automatically' to get some sort of decent sitting down/quick testing setup, while introducing a broader configuration method for user configuration as well to tweak settings.
This will probably result in a human readable config file (TOML (https://github.com/toml-lang/toml) has my preference), and whatever we need to dump the room/camera calibration data.
I would prefer to keep this either in get_s and set_s, or ohmd_device_settings_sets, since we have internal driver configuration implemented there as well.
Am fine looking into this, will also continue on my QT based openhmd tool that has room configuration.

We can certainly have some sort of 'configuration-less' setup that assumes when the HMD is first found in any camera view that it's at 0,0,0 and facing forward or so. It's possible from there to reverse the position of each subsequent camera when they find the headset.

Having a setup procedure and a configuration store is still best in order to really register the VR setup in a physical room and get correct user heights.
TOML looks like a fine candidate for storing the configuration information. Since such a thing is driver specific (different drivers use different tracking methods and therefore want to store different information), so I was thinking in terms of internal API that reads or writes from a standard user configuration location, rather than placing the burden on each application.
That said, a method where applications can specify an application private location for OpenHMD to store or load that information could also be useful

Another useful thing to store in a private cache is the JSON blob that comes from each controller with LED positions. Reading the full blob over the radio causes a measurable delay at startup, but there is a hash in the firmware that can be read. If the hash doesn't change, you can use the cached copy of the JSON blob and save time.

See also https://github.com/thaytan/OpenHMD/issues/25

See also https://github.com/thaytan/OpenHMD/issues/25

Two things about configuration.
I absolutely despise the CV1 "please have sensors Xft apart".
It limits the environments you can use the device in. Sensors should definitely be allowed to work any distance apart, as long as they are apart and can get enough information about dimension.
Another thing, is with something like safety barriers (marked locations that you should avoid), it should be a proper 3D coordination. Not a simple vertical line.
That would help a ton with rooms that have odd dimensions, Such as slanted or low roofs.

At the moment, there's no restriction on how you can space multiple sensors - it really is just about placing them so that you stay the devices stay in view of at least 1 as you move around.
As for possibilities for chaperone / barrier configuration - that's still unexplored territory for me, but I think that will depend on what is possible in the things that are using OpenHMD. I don't know if SteamVR can do more than draw walls, for example.

Unfortunate that ceilings aren't really possible, but on multiple sensors.
Have you seen any difference in tracking between the spacing and amount of sensors you've been using?

I mostly test with 1 sensor, because 2 is annoying. Until there's a room calibration utility, the code infers the position of the tracker camera(s) at startup based on the first observation of the headset. If that first tracking lock for each camera isn't very precise, the cameras end up reporting slightly different tracking positions, leading to shaking/fighting. It would be better to do some averaging or so - or to implement a proper calibration step and store known-good camera positions in a config file. Hence this open issue :)

I have found that it has issues determining the direction of the ground(gravity vector?) if there are 2 cameras are at different heights(I have one on top of my pc, and one mounted on a tripod). Would be nice if we could calibrate for that too.

I think the key will be to detect that the headset is being held still and use some averaging over a second or two to acquire a the calibration, instead of using a single pose observation for each camera.

Hi @thaytan,
thank you for your fantastic work!!! And the epic achievements you got...
I don´t have much time but I come from time to time to see if I can help. I normally get lost on the huge amount of branches you have. Do you follow any convention?
It would be great if you can merge everything that's working to one branch so we can start look over + a small guide on how to setup the environment. Given the advances you made something I think it's required is:

Have 1 development branch. Go feature by feature with one branch per feature.
Guide on how to setup environment. For example, I was not able to capture gst plugin traces.
General overview of the initialization process. I have yet to investigate openhmd architecture, and steamvr one.
The simulation tool would be great. That way we can start optimize the process without having to capture new data.

I know I'm asking for a lot. But it will really help. Thank you again.

Seems like merging kalman filter branch to master, and work in the simulation one is the right way. Correct?

I don't know much about how to proper set the room setup. But from my master of computer vision I can remember that it's required two cameras to get a precise position. Otherwise there will be drift.
So I suppose that we must have the cameras calibration (that I suppose you already have cause you are doing posing). And then do a correspondence of the points on the two cameras to find the actual position. Something similar on what was done here but simpler cause we have the point cloud.
https://sigmoidal.ai/en/building-rome-in-a-day-3d-reconstruction-with-computer-vision/

If I would implement this I will capture the frames from left right cameras, noted with timestamps to know relative drift, and run reconstruction algorithm with them, using both. Testing different versions to find a right compromise between speed and accuracy. For this to work we have to be able to capture the images save, and have a test to apply algorithms, compute timings and actual accuracy.
The simulator can be great, but a simple integration test will do it, and we can add benchmarks.

@gadLinux If you need help with testing, I can assist.

Hi @Doomsdayrs Yes. Could you write a few lines in the readme on how to setup gstreamer or pipewire to capture and annotate the image generated... I want to build a small tool, maybe based on what libsurvive has to be able to test the algoritms.

Moreover, there's no branch that I can take as reference cause features are spreaded all over the code. Maybe we can merge what's working (or mostly) now to a dev branch?


Bug: crash when using an older version of botocore

Expected Behaviour
Powertools should work with all versions of botocore.
Current Behaviour
Right now, under certain versions of botocore, powertools will crash with the error:
Error: module 'botocore' has no attribute 'register_initializer
Code snippet
N/A

Possible Solution
Don't call register_initializer with older versions of botocore.
Steps to Reproduce
Use the latest Powertools with an older version of Powertools
Powertools for AWS Lambda (Python) version
latest
AWS Lambda function runtime
3.9
Packaging format used
Lambda Layers
Debugging logs
No response

Background: https://github.com/awslabs/aws-lambda-powertools-python/actions/runs/5154168852/jobs/9282552242
Just released 2.16.0 included the new user-agent header feature. During our release a few seconds ago, our canaries discovered that 5 regions have an older version of botocore/boto available at Lambda runtime compared to all other regions (unusual).
This issue tracks a bugfix to prevent it from happening while we work with the Lambda team in investigating the discrepancy. We'll release 2.16.1 and publish release notes with that instead.

Description updated with the correct data on regions and botocore versions now.


Consider replacing our init.py script by cookiecutter

https://github.com/cookiecutter/cookiecutter

Very nice, I think this could be cleaner than our own init script.
I also found this
https://github.com/waynerv/cookiecutter-pypackage
which is the best python template I could find for cookiecutter so far, but it is not up to date, so I think it might make sense to make our own.

Yes, we could build our own cookiecutter template. It would require some thought how to test the actions of a generated project. I believe this is possible but requires a bit more gitfoo.


Commits aren't in a flexbox on homepage

Commits currently show side by side on XL screens, this should be changed to LG screens.

Fixed with the folder refactor


pull request for e2e test at 01/01/2017 20:45:06

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Translation of the counter shows duplicate "subscribers" text

As described in the title, the text following the count shows "X abonnés subscribers" (where X is the counter) when it only should show  "subscribers" or "abonnés" if the addon manages translations.
I'm on Firefox latest version, and it's in french that's why it shows "abonnés".
It's a little nitpick, but would be grateful if you could fix this.
Great addon by the way.
Thanks in advance.

This should be fixed with the changes from version 1.1.10 and now only read "4 abonnés" or `1 abonné". Only tested that in Chrome, though.


ansible: Add sysadmin-remote bundle as dependency for ClearLinux

This bundle contains docker-py which is required on the controller
node by ansible in order to launch containers
Signed-off-by: Alberto Murillo Silva<EMAIL_ADDRESS>

Coverage increased (+0.004%) to 65.694% when pulling adf203cca0cdf68cc3a48b657ba47c15e6b7876a on albertomurillo:docker_deps into afdd39467929809109f198516d44b8b2985b5ac9 on 01org:master.

Coverage increased (+0.004%) to 65.694% when pulling adf203cca0cdf68cc3a48b657ba47c15e6b7876a on albertomurillo:docker_deps into afdd39467929809109f198516d44b8b2985b5ac9 on 01org:master.

@tpepper could you take a look?


IndexError while "Determining the score threshold from random background..."

Hi! Thanks for all of your work building SICER. It's our peak caller of choice, especially for broad domains!
I occasionally run into an error in the step: "Determining the score threshold from random background..."
The entire output is pasted below. Any guidance is much appreciated!

Starting peak-calling, peak annotation, and FRiP
AV004_2xLANAMeCP2_350nM_E0045-1_CR_K562_N__500k_Kmet__S1_uniq.bam
Sicer-broad peak calling
Running SICER with given arguments
Preprocess the AV004_2xLANAMeCP2_350nM_E0045-1_CR_K562_N__500k_Kmet__S1_uniq.bed file to remove redundancy with threshold of 1

chrom    Total plus reads        Retained plus reads       Total minus reads      Retained minus reads
chr1          4659751                  4403404                  4659747                  4403459
chr2          3275483                  3151590                  3275495                  3151892
chr3          2574122                  2471084                  2574126                  2470966
chr4          2184572                  2114932                  2184597                  2114927
chr5          2490561                  2391973                  2490530                  2392923
chr6          2750047                  2621032                  2749993                  2621540
chr7          2993344                  2828244                  2993352                  2827611
chr8          1967204                  1892082                  1967198                  1892322
chr9          1662519                  1565154                  1662534                  1565118
chr10         1925155                  1842103                  1925155                  1842225
chr11         2079889                  1982595                  2079874                  1982890
chr12         1859203                  1781270                  1859196                  1781042
chr13         985142                   938199                   985108                   938118
chr14         1070885                  1035002                  1070894                  1034275
chr15         1445601                  1382636                  1445610                  1382387
chr16         1620834                  1524772                  1620834                  1525646
chr17         1487914                  1403445                  1487901                  1403183
chr18         926987                   897584                   926987                   897638
chr19         1410887                  1301552                  1410888                  1302178
chr20         989479                   942301                   989479                   942384
chr21         723933                   685449                   723932                   685376
chr22         1163571                  1029414                  1163567                  1029121
chrX          1306262                  1274004                  1306267                  1274110
chrY           38804                    37741                    38804                    37743
chrM          789367                    58624                   789367                    58385
Preprocess the 22AV001_IgG_E0045-1_CR_K562_C__500k_Kmet_5ng_S1_uniq.bed file to remove redundancy with threshold of 1

chrom    Total plus reads        Retained plus reads       Total minus reads      Retained minus reads
chr1          387769                   387023                   387768                   387005
chr2          262420                   261906                   262423                   261876
chr3          200398                   200161                   200399                   200173
chr4          175771                   175584                   175772                   175561
chr5          203630                   203247                   203625                   203235
chr6          218956                   218630                   218953                   218626
chr7          236811                   236435                   236813                   236434
chr8          157468                   157252                   157469                   157240
chr9          137318                   136565                   137318                   136582
chr10         159388                   159022                   159388                   159034
chr11         164132                   163857                   164131                   163883
chr12         145313                   145115                   145313                   145136
chr13          86965                    84938                    86967                    84919
chr14          87777                    87649                    87777                    87666
chr15         126483                   126229                   126484                   126288
chr16         134352                   134111                   134352                   134117
chr17         118459                   118166                   118458                   118160
chr18          79787                    79716                    79789                    79719
chr19         110763                   110286                   110764                   110273
chr20          80781                    80616                    80781                    80621
chr21          64787                    64637                    64786                    64644
chr22          98695                    98341                    98694                    98285
chrX          103861                   103749                   103860                   103754
chrY           2667                     2658                     2667                     2658
chrM          135612                    20667                   135612                    19999
Partition the genome in windows and generate summary files...
Total count of chr1 tags: 8806863
Total count of chr2 tags: 6303482
Total count of chr3 tags: 4942050
Total count of chr4 tags: 4229859
Total count of chr5 tags: 4784896
Total count of chr6 tags: 5242572
Total count of chr7 tags: 5655855
Total count of chr8 tags: 3784404
Total count of chr9 tags: 3130272
Total count of chr10 tags: 3684328
Total count of chr11 tags: 3965485
Total count of chr12 tags: 3562312
Total count of chr13 tags: 1876317
Total count of chr14 tags: 2069277
Total count of chr15 tags: 2765023
Total count of chr16 tags: 3050418
Total count of chr17 tags: 2806628
Total count of chr18 tags: 1795222
Total count of chr19 tags: 2603730
Total count of chr20 tags: 1884685
Total count of chr21 tags: 1370825
Total count of chr22 tags: 2058535
Total count of chrX tags: 2548114
Total count of chrY tags: 111
Total count of chrM tags: 116948
Normalizing graphs by total island filitered reads per million and generating summary WIG file...
Finding candidate islands exhibiting clustering...
Species:  T2Tv2
Window_size:  5000
Gap size:  50000
E value is: 5000
Total read count: 83024186
Genome Length:<PHONE_NUMBER>
Effective genome Length:<PHONE_NUMBER>
Window average:<PHONE_NUMBER>817952
Window pvalue: 0.2
Minimum num of tags in a qualified window:  196
Determining the score threshold from random background...
Removing temporary directory and all files in it.
Traceback (most recent call last):
File "/home/bryan/miniconda3/bin/sicer", line 7, in
exec(compile(f.read(), file, 'exec'))
File "/media/bryan/Alli_Main/file_transfer/AV/SICER2/bin/sicer", line 235, in
main()
File "/media/bryan/Alli_Main/file_transfer/AV/SICER2/bin/sicer", line 229, in main
run_SICER.main(args)
File "/media/bryan/Alli_Main/file_transfer/AV/SICER2/sicer/main/run_SICER.py", line 77, in main
find_islands_in_pr.main(args, total_tag_in_windows, pool)
File "/media/bryan/Alli_Main/file_transfer/AV/SICER2/sicer/src/find_islands_in_pr.py", line 184, in main
score_threshold = background.find_island_threshold(args.e_value);
File "/media/bryan/Alli_Main/file_transfer/AV/SICER2/sicer/lib/Background_island_probscore_statistics.py", line 225, in find_island_threshold
current_expectation = self.background_island_expectation(current_scaled_score)
File "/media/bryan/Alli_Main/file_transfer/AV/SICER2/sicer/lib/Background_island_probscore_statistics.py", line 172, in background_island_expectation
while (int(round(index - self.window_score[i] / self.bin_size)) >= 0):
IndexError: list index out of range

Hello,
I have exactly the same error when running sicer.
I am using a specific genome that I added to the GenomeData.py file, could that be due to that?
Thanks for providing such a nice tool.

With bigger sequence libraries, it occurs that some observations are above this count. To avoid that I did a quick hack by changing the factor 2 by a factor 5 and it worked for me, e.g.:

I experience the same issue with certain gap sizes, but unfortunately this solution does not work for me. I tried to change factor 2 to 5 or 10 or 50 and always get the same error :(

With bigger sequence libraries, it occurs that some observations are above this count. To avoid that I did a quick hack by changing the factor 2 by a factor 5 and it worked for me, e.g.:

I experience the same issue with certain gap sizes, but unfortunately this solution does not work for me. I tried to change factor 2 to 5 or 10 or 50 and always get the same error :(

Hi, thanks for the error reported and temporarily solution! We are currently updating a new version of SICER2 and looking into those problems. Can you let us know the exact gap size that you used in your work? @akukalev


add Webhooks Update event

This adds support for the event fired when Webhooks get created, removed, or updated. It is documented here https://discordapp.com/developers/docs/topics/gateway#webhooks-update .

thanks!