Side by side providers doesn't seem to work to list resources

I experimented with both azure providers a the same time to provision resources and it isfine.
But when I want to list any resource I'm getting errors like azure:keyvault/getKeyVault:getKeyVault not found or something similar to "function data dosn't exists" (happened several times but not now hence don't remember exact message).
It is the same whenever I start the azure-csharp project first or azure-nextgen-csharp. It doesn't matter what resource I want to list.
Something specific to setup to make it work or it is not supported?

That's odd because the result of getXyz should not depend on how it was provisioned, even if that was done via the Azure portal. Any repro program that you can share?

It happens for every resource created with this pattern:

list some resource with nextgen provider
list resource with plain provider

I think that the problem may be in proj/pulumi configuration.
It is azure-nextgen-csharp project.
Then I added Pulumi.Azure via nuget manager.
Maybe some plugin must be installed manually as well but when I list the plugins both are available.
I also use same subscription manager in both cases which may be also worth notice:
public static InvokeOptions Create(string subscriptionId) =>             new InvokeOptions             {                 Provider = new Provider(subscriptionId,                     new ProviderArgs                     {                         SubscriptionId = subscriptionId                     }                 )             };

The setup sounds fine, there shouldn't be any special steps required. I'd still love to get a repro program and the exact error message that you are hitting.

For now I can give you exact message, hope it will help.
One or more errors occurred. (Status(StatusCode="Unknown", Detail="invocation of azure:appservice/getFunctionApp:getFunctionApp returned an error: Resource type azure:appservice/getFunctionApp:getFunctionApp not found", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1606334744.431000000","description":"Error received from peer ipv4:<IP_ADDRESS>:58860","file":"T:\src\github\grpc\workspace_csharp_ext_windows_x64\src\core\lib\surface\call.cc","file_line":1062,"grpc_message":"invocation of azure:appservice/getFunctionApp:getFunctionApp returned an error: Resource type azure:appservice/getFunctionApp:getFunctionApp not found","grpc_status":2}"))
version.txt:
azure-nextgen
0.2.8

Ok: AppFunction API differs between Pulumi.Azure and Pulumi.AzureNextGen.
But the same problem is with every other resource.
I get it when I change the Azure provider, example for keyvault, sample code that breaks althought for nextgen it is fine (with small api name adjustment):
public GetKeyVaultResult GetVaultResult(string resourceGroup, string vault)
        {
            return GetKeyVault.InvokeAsync(new GetKeyVaultArgs()
            {
                ResourceGroupName = resourceGroup,
                //VaultName = vault
                Name = vault
            }, GetProvider(resourceGroup)).Result;
        }

Result:
One or more errors occurred. (Status(StatusCode="Unknown", Detail="invocation of azure:keyvault/getKeyVault:getKeyVault returned an error: Resource type azure:keyvault/getKeyVault:getKeyVault not found", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1606335381.086000000","description":"Error received from peer ipv4:<IP_ADDRESS>:59159","file":"T:\src\github\grpc\workspace_csharp_ext_windows_x64\src\core\lib\surface\call.cc","file_line":1062,"grpc_message":"invocation of azure:keyvault/getKeyVault:getKeyVault returned an error: Resource type azure:keyvault/getKeyVault:getKeyVault not found","grpc_status":2}"))

Are you trying to mix a resource from one provider (azure) with a provider option from another provider (azure-nextgen)? What exactly does GetProvider do?

it translates resource name to subscription and returns invoke options:
new InvokeOptions {
                Provider = new Provider(subscriptionId,
                    new ProviderArgs
                    {
                        SubscriptionId = subscriptionId
                    }
                )
            };

Is Provider from the same assembly as GetKeyVault?

No because I thought that this type is from Pulumi namespace only
-> Большое спасибо 👍


Fix formatting in advanced-symsrv-use.md

Not sure what happened here, it looks like the garbled result of multiple rounds of asterisk escaping. I've tried to correct and simplify it as much as possible. I kept the partial emphasis in the cache*localsymbolcache example so that it's clear that localsymbolcache is a placeholder for a path and not a literal string.

@mstange - Thanks for this PR! There was some automation that was run to convert the former XML to markdown. And the use of the asterisk in the server path along with it's use in Markdown can be tricky.


How to play UDP Multicast stream

Hi.
I'm using ExoPlayerXamarin v2.9.7 and VS2019 with latest update. I'm struggling to play UDP multicast stream. I can't managed how to do it.
I have done lots of google search but I have not found any example for ExoPlayerXamarin. I have found some examples for java.  This one is one of them. but I failed to port it to C#. Specifically I have not found DataSource.Factory class used there.
Can you provide an example how to play UDP stream?
this is a UDP stream I'm trying to play udp://@<IP_ADDRESS>:1234 it works well with VLC player.

Finally I have managed to play UDP multicast stream using ExoPlayerXamarin.
I have created UdpDataSourceFactory class. This class implements IDataSourceFactory interface.
using Com.Google.Android.Exoplayer2.Upstream;
public class UdpDataSourceFactory : Java.Lang.Object, IDataSourceFactory
{
    private UdpDataSource UdpDataSourceField { get; set; }

    public UdpDataSourceFactory(UdpDataSource dataSource)
    {
        this.UdpDataSourceField = dataSource;
    }

    public IDataSource CreateDataSource()
    {
        return UdpDataSourceField;
    }
}

Then I initialized player using following code
private void InitPlayer()
{
    Uri uri = Uri.Parse("udp://<IP_ADDRESS>:10123");
    DefaultTrackSelector trackSelector = new DefaultTrackSelector(new AdaptiveTrackSelection.Factory());
    SimpleExoPlayer player = ExoPlayerFactory.NewSimpleInstance(this, trackSelector);
    PlayerView playerView = FindViewById<PlayerView>(Resource.Id.simpleExoPlayerView1);
    playerView.Player = player;
    playerView.RequestFocus();
    UdpDataSourceFactory factory =  new UdpDataSourceFactory(new UdpDataSource(3000, 100000));
    IMediaSource mediaSource = new ExtractorMediaSource.Factory(factory).CreateMediaSource(uri);
    player.Prepare(mediaSource);
    player.PlayWhenReady = true;
}


Resolve sbt warnings: postfix operator second should be enabled

Resolve sbt warnings:
[warn] spark/streaming/src/main/scala/org/apache/spark/streaming/util/WriteAheadLogManager.scala:155: postfix operator second should be enabled
[warn] by making the implicit value scala.language.postfixOps visible.
[warn] This can be achieved by adding the import clause 'import scala.language.postfixOps'
[warn] or by setting the compiler option -language:postfixOps.
[warn] See the Scala docs for value scala.language.postfixOps for a discussion
[warn] why the feature should be explicitly enabled.
[warn]         Await.ready(f, 1 second)
[warn]                          ^

Test build #28287 has started for   PR 4908 at commit 0629af4.

This patch merges cleanly.

Test build #28287 has finished for   PR 4908 at commit 0629af4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28287/
Test PASSed.

@witgo are these all of the postfix build warnings? I'd love to get all of them at once. I was meaning to try to fix up a number of build warnings again, so the more you can get the better. (If it grows beyond a trivial change we'll make a JIRA.)

Yes.
This patch only resolves the postfix build warnings.
I can try to resolve more build warnings, we should create a JIRA.

OK I'm happy to merge this as a tiny fix, but are you in the process of adding more?

@srowen
You can merge this PR. Other changes I will create a new JIRA and PR.

OK. I think it's valid to clean up build warnings but ideally we take care of them as much in one go as possible rather than many small PRs.


Error when configuring routes with more than three destination rules

Is there an existing issue for this?

[X] I have searched the existing issues

Kong version ($ kong version)
2.4、2.5
Current Behavior
When there are more than 3 destination rules in the routing configuration, a configuration loading error will occur. The error is:
[error] 22887#0: *161949 stream [lua] handler.lua:1064: could not rebuild router via timer: /usr/local/share/lua/5.1/kong/router.lua:1429: invalid order function for sorting, context: ngx.timer

Route is ：
{
    "next":null,
    "data":[
        {
            "name":"rrr1",
            "path_handling":"v0",
            "id":"460a58d8-5701-49cf-8cc7-a714195b812e",
            "regex_priority":0,
            "destinations":[
                {
                    "ip":"<IP_ADDRESS>/0",
                    "port":7000
                },
                {
                    "ip":"<IP_ADDRESS>/0",
                    "port":7001
                },
                {
                    "ip":"<IP_ADDRESS>/0",
                    "port":7002
                },
                {
                    "ip":"<IP_ADDRESS>/0",
                    "port":7003
                },
                {
                    "ip":"<IP_ADDRESS>/0",
                    "port":7004
                }
            ],
            "paths":null,
            "created_at":1678187654,
            "updated_at":1678187654,
            "headers":null,
            "methods":null,
            "sources":null,
            "tags":[
                "9b73b55e-238c-4d1d-b10e-cc06417071fa"
            ],
            "protocols":[
                "tcp"
            ],
            "strip_path":true,
            "request_buffering":true,
            "response_buffering":true,
            "snis":null,
            "hosts":null,
            "service":{
                "id":"9b73b55e-238c-4d1d-b10e-cc06417071fa"
            },
            "https_redirect_status_code":426,
            "preserve_host":false
        }
    ]
}

The error should be that the destination rule comparison function does not return a stable result.
local function sort_destinations(r1, _)
  for _, destination in ipairs(r1.destinations) do
    if destination.ip and destination.port then
      return true
    end
  end
end

Expected Behavior
Configure more than 3 destination rules without error.
Steps To Reproduce
Create a route for rules with more than 3 destination rules.
Anything else?
No response

hello @zhangzerui20 thank you for reporting this, could you confirm whether the same problem can be reproduced on the latest available version of Kong? Thank you.

@samugi Kong 3.2.1 can reproduce this problem.
2023/03/08 17:51:56 [error] 25249#0: init_by_lua error: /usr/local/share/lua/5.1/kong/router/traditional.lua:735: invalid order function for sorting

We have another PR trying to fix it. https://github.com/Kong/kong/pull/10514.


I'm sorry, I thought this was America

Trying to curry favor with your boss, @vjpai?
Is it appropriate to make political comment in grpc issues?

@ctiller : "Traditional" is just another way of saying "First draft"

Jenkins, test this please

[trickle] No significant performance differences


Auto Sync stop automatic

Issue Type: Bug
every time I had a window opened and I open a new window the app ask me to sign in to sync my settings
VS Code version: Code 1.63.2 (899d46d82c4c95423fb7e10e68eba52050e30ba3, 2021-12-15T09:38:17.605Z)
OS version: Darwin arm64 21.2.0
Restricted Mode: No

System Info

Item
Value

CPUs
Apple M1 (8 x 24)

GPU Status
2d_canvas: enabledgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 1

Memory (System)
16.00GB (0.91GB free)

Process Argv
--crash-reporter-id 0ee627cf-e4d9-4f4e-b8d2-b8c6ff2d7a6a

Screen Reader
no

VM
0%

Extensions (27)

Extension
Author (truncated)
Version

bracket-pair-colorizer
Coe
1.0.62

vscode-eslint
dba
2.2.2

es7-react-js-snippets
dsz
3.1.1

vscode-html-css
ecm
1.10.2

prettier-vscode
esb
9.1.0

auto-close-tag
for
0.5.13

auto-rename-tag
for
0.1.9

copilot
Git
1.7.4608

remotehub
Git
0.20.0

vscode-gutter-preview
kis
0.29.0

vscode-language-babel
mgm
0.0.35

jupyter
ms-
2021.11.1001550889

jupyter-keymap
ms-
1.0.0

jupyter-renderers
ms-
1.0.4

vsliveshare
ms-
1.0.5242

vsliveshare-audio
ms-
0.1.91

vsliveshare-pack
ms-
0.4.0

vscode-react-native
msj
1.8.1

color-highlight
nau
2.5.0

indent-rainbow
ode
8.2.2

material-icon-theme
PKi
4.11.0

LiveServer
rit
5.6.1

code-spell-checker
str
2.1.4

highlight-matching-tag
vin
0.10.1

JavaScriptSnippets
xab
1.8.0

zeplin
Zep
2.2.0

material-theme
zhu
3.13.11

(1 theme extensions excluded)

A/B Experiments
vsliv368cf:30146710
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythonvspyl392cf:30422344
pythontb:30283811
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscorecescf:30384386
pythondataviewer:30285071
vscod805:30301674
pythonvspyt200:30340761
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593cf:30376535
vsc1dst:30421291
pythonvs932:30410667
vscop804cf:30404767
vs360:30404995
vsrem710:30416614
vsccppwtstudy:30421978

/gifPlease

@TylerLeonhardt

Can you follow these steps so that we can get more information on what's going on?

Open the Command Palette
Run the Developer: Set Log Level... command
Choose Trace
Wait for the issue to happen...
Open the Output pane (ctrl+shift+u I think on Windows)
drop in this issue he contents of the following:

Microsoft Authentication Output channel or GitHub Authentication Output channel (depending on the account you're using
Log (Settings Sync) Output channel
Log (Window) Output channel

log.txt

@mohamedsaqer thanks for that. I appreciate you providing both Microsoft & GitHub output but which one do you actually use with Settings Sync?

I use both but the main is Microsoft

I have a possible fix for this that will come in VS Code Insiders starting either Thursday or Friday. Fixed in https://github.com/microsoft/vscode/commit/f498b374c03be09cf664a7fefa1c1c95a0c7216d

Awesome

@mohamedsaqer can you try out the latest insider build of VS Code: http://code.visualstudio.com/insiders

Masalah kode authentikator Google

@mohamedsaqer have you had a chance to give this a go? I specifically would like to know how Microsoft Auth goes.

the issue still exists @TylerLeonhardt

I'm closing this since there has been a lot of work in this space. Please give Insiders a try or wait until the Stable release which will be next Thursday which will have my fixes.


Increase margin of error in an otherwise unsound test

This test can easily fail on a heavily loaded machine, such as one running many tests in parallel or a weaker developer laptop.

The afterFunc could be delayed massively on a heavily loaded machine, such as one running a lot of tests in parallel.
Requiring an accuracy of 1ms seems like a flake waiting to happen (it was such a flake which inspired this PR)
When we write code which uses this scheduler, we can't even safely assume the afterFunc will ever be run, let alone run within a 1ms margin of time error.

Put simply, as implemented this test is more of a test of how loaded the test server is than a test of any functionality.
By increasing the allowable delta hugely, we keep this test as a sanity check but basically remove the chance of a flake. The test essentially becomes "does afterFunc work, generally?".
/kind cleanup
NONE

Using the fake clock for this test would be the most ideal here... I'm not too sure why it wasn't like that in the first place 👀

Thanks @SgtCoDFish! This PR is an improvement that reduces flakiness while keeping the purpose of the test (sanity check) intact.
I agree that this test as a whole should probably be reworked and that there are many valid ways to do that, and if we care deeply enough we should create an issue to address that
/lgtm
/approve


Improve ZAPI tool

The Zapi tool (harvest zapi) retrieves available counters and metadata from ONTAP (CDOT or 7mode). The tool should enable you to configure Harvest to collect any ONTAP metric.
Currently the tool is able to

retrieve ZAPI attributes and ZAPIPERF objects and counters
export ZAPIPERF objects into Harvest .yaml templates

Issues:

ugly output
exported templates not properly tested

output improved, closing


How to manage the spot after flash?

Hello,
I have flashed my pycom SD card with this image, but how can I manage the hotspot now? I need to connect it to WiFi but what app should I use?
Thank you.
P.S. Using the nebra app - Update WiFi is not finding the hotspot.

You need to use the Nebra app. You need to do it within the first 5 minutes of turning the hotspot on.
You can also view the diagnostics page by visiting the IP address of the hotspot

The nebra app is not finding the hotspot, so I can't to it. Pycom old app finds it but can't connect. I can't visit the diagnostics page because my hotspot have only WiFi and no LAN port, so in order to connect it to the router I need the nebra app to find the hotspot first. Please Nebra guys assist :( .

What phone are you using?
What version of the Nebra app?
And what version of the Nebra firmware did you flash on it?

Using a android phone - Samsung Galaxy A80 running Android version 11.
The latest version of Nebra Hotspot from the android play store.
The latest here on github - v1.0.2-6-helium-pycom-2023-04-26-OpenFleet

Is the app version 2.0.1?
Can you check on the play store?

Yes indeed version 2.0.1.

@catsado89 can you share some screenshots of where it's going wrong

I have no idea how a screenshot but okey. Here you can see in the first screenshot i select Update WiFi

Then the app starts to scan for a hotspot but finds nothing

But if I scan with the old pycom app it finds the hotspot but can't connect to it.

Flashed the spot SD card with the latest release - v1.0.2-9-helium-pycom-2023-05-09-OpenFleet , still no change the Nebra app dosn't find the device and I can't connect it to the WiFi Router. Please help.

@catsado89 Can you please create a ticket at https://support.nebra.com with a link to this issue here? And can you reply back with the ticket number?
Thanks

Hello again. I have created a ticker - Ticket (#421355) .

@WayneNebra FYI

@shawaj OK thanks, will check it out.

Hello? Can I please get any help?

@catsado89 Have you updated the App on the Play Store , we just pushed an update 16th May 2023

Indeed I have.

Replied on your ticket

Okey so you stopped replying in my ticket, no problem. Your latest version of the app still can't see the hotspot but do you know what app see the hotspot? Hotspotty Connect. Thanks to it I was able to resolve my problem.

Thanks so much for your message, and sorry about the delay in getting back in touch, we've been extremely busy, so I do apologise, I thought you were sending a copy of your JSON file in so I could check it all out from this end, however after reading your message it looks like you've resolved this now? but we'd still like to know why the Nebra App hasn't found the miner.
You can still access you local diagnostics page, just use the IP address of the miner in your browser.
I'll send instructions over on the ticket.
Thanks

Closing as no reply


Feature/base task

For issue #141

Add base_task abstract base class.
Add classification & ranking inherit from base_task.
Add util function.
Add unit-test.

Codecov Report

Merging #151 into 2.0 will not change coverage.
The diff coverage is 100%.

@@         Coverage Diff         @@
##            2.0   #151   +/-   ##
===================================
  Coverage   100%   100%
===================================
  Files         1      7    +6
  Lines        66    108   +42
===================================
+ Hits         66    108   +42

Impacted Files
Coverage Δ

matchzoo/engine/base_task.py
100% <100%> (ø)

matchzoo/tasks/binary_classification.py
100% <100%> (ø)

matchzoo/engine/__init__.py
100% <100%> (ø)

matchzoo/tasks/__init__.py
100% <100%> (ø)

matchzoo/tasks/ranking.py
100% <100%> (ø)

matchzoo/tasks/utils.py
100% <100%> (ø)

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4f81759...9308eb0. Read the comment docs.


Use doc branch

What type of PR is this?
/kind bug

What this PR does / why we need it:
Fix doc link
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes https://github.com/outscale-dev/cluster-api-provider-outscale/issues/131
Special notes for your reviewer:
Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.
TODOs:

[ ] squashed commits
[ ] includes documentation
[ ] adds unit tests

Closes https://github.com/outscale-dev/cluster-api-provider-outscale/issues/131


Switched Webpack devtool for serve

Addresses: https://github.com/blackbaud/skyux2/issues/1325

Codecov Report

Merging #384 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #384   +/-   ##
=======================================
  Coverage   99.25%   99.25%
=======================================
  Files          71       71
  Lines        1740     1740
  Branches      282      282
=======================================
  Hits         1727     1727
  Misses         13       13

Flag
Coverage Δ

#builder
100% <ø> (ø)
:arrow_up:

#runtime
94.75% <ø> (ø)
:arrow_up:

#srcapp
100% <ø> (ø)
:arrow_up:

Impacted Files
Coverage Δ

config/webpack/serve.webpack.config.js
100% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 52df752...b7dbcc5. Read the comment docs.

This solution feels different than the HMR.  I'd rather sourcemaps work for everyone, out of the box, without having to opt-in to a particular devTool.
https://webpack.js.org/configuration/devtool/ has some good documentation on the supported properties and their affects.
In my testing, inline-cheap-module-source-map also worked well, but as mentioned in said documentation, is slightly faster but does have some limitations.
I would be in favor of merging this PR as-is.


Error on MacOS: signal: killed

I am trying to run the hellocv example, but i am getting the error message :

signal: killed

I installed opencv via homebrew in version <IP_ADDRESS>
Any ideas what i am missing?

Alright, after a lot of research I found out that this is a Mac related Problem which exists since the new Update.
So this Issue will be a wontfix.

I updated the go version to 1.8.1 and then it was ok.


sendDELETE with request body

I tried to update module-rest from 1.2.7 to 1.3.0 and my tests with sendDELETE are failing because codeception no longer sends the $params as request body but as query parameters.
Is there a way to restore the old behaviour? Request bodys are allowed in HTTP DELETE and therefore should be supported by codeception too.

RFC 7231 specifies that a DELETE request may include a body. Please, reverted these change or make them configurable

I will revert it.

@SamMousa This is the opposite of what was said in #44
Please discuss it :)

@Naktibalda  - We use api by jsonApi spec,
https://jsonapi.org/format/#crud-updating-to-many-relationships

@SamMousa
May be an additional method sendDeleteWithQueryString would be a suitable compromise

I propose instead having 2 params on the send delete method, one for body one for query string. If possible we should not break existing clients (again)

Ah wait, the issue was from 2018 but the fix was recent. Then I'm fine with reverting it and leaving it as is.

As evidenced few others ran into issues, so no need to make it harder than needed.

Both linked pull requests must be merged and released:
https://github.com/Codeception/lib-innerbrowser/pull/38
https://github.com/Codeception/module-rest/pull/56
I will release new versions in a few days.

I released this change as module-rest 1.3.1 and lib-innerbrowser 1.5.0


Suppress click/dblclick events during drag

Similarly to commit f7fbda9a43866a2404d20e5e0f50c55ce72add4c (#312), which suppressed mousemove/mouseup events, we want to block other elements’ click/dblclick handlers from firing at the end of a scrollbar drag.

Here’s the setTimeout version. (I actually wrote this one first, then wondered whether debounce is more consistent with the rest of the code. Pick your favorite. :yum:)


Q: How to tell VS/Test Explorer about deleted tests?

I was playing around with Test Explorer, and having assessed it, I tried to turn it off by closing the window. Afterwards I deleted a test file from my project.  Now, whenever I debug something, VS would complain "The item "TestXYZ.py" does not exist in the project directory. It may have been moved, renamed, or deleted."
That message is correct: yes, I did delete the test. How do I stop it from appearing again please?
Thanks for your help.

Once you save the project file it should go away.
In VS 2015 and earlier, we check every file listed in the project for tests, and warn if anything doesn't look right.

Thanks for that. Do you mean this should work in VS2017, but not in VS2015, then? Unfortunately we are still using VS2015. My project is saved---it does not have a "*", and in any case, I have closed and restarted VS---but the warning is still there.
Here's my work-around: first, I edited my .playlist file to remove the deleted test, but that did not fix the problem. Then when I edited the .pyproj file as well, the warning went away.
Will mark this as closed. Thank you.

Hmm... perhaps something got cached.
In VS 2017 we detect tests in code files as we're doing our analysis, so we never directly refer to the project file and will know straight away when you remove a file.


Some API objects should be #[repr(C)] and returned by value for stack allocation

The C API currently pays a malloc penalty for just about every single object.  This is because opaque Rust types are boxed, and that box is converted to a pointer to return from the C function.
However, this imposes a performance cost on the non-Rust clients.  Eventually we could make some of the fundamental Rust types be #[repr(C)].  But in the meantime, a pointer to a struct is no different from a struct containing a pointer.
So, I propose returning structs directly instead of pointers.  For example:
typedef struct atom_t {
    struct rust_atom *atom;
} atom_t;

struct atom_t atom_sym(const char *name);

I didn't do it from the very beginning because I wanted to be able developing any Rust API I would like to and I believed that there are Rust data types which cannot be mapped to the C one's automatically. @luketpeterson do you know if it is a case?

Also I am not sure about how huge this performance cost is. For example for the Python clients we need to copy the data into a heap anyway I believe. We probably need performance tests to measure performance issues.

I didn't do it from the very beginning because I wanted to be able developing any Rust API I would like to and I believed that there are Rust data types which cannot be mapped to the C one's automatically. @luketpeterson do you know if it is a case?

Certainly some Rust types aren't internally #[repr(C)], but for those the answer is to embed the pointer inside the structure instead.  So a struct containing an opaque pointer, rather than a pointer to an opaque structure.

Also I am not sure about how huge this performance cost is. For example for the Python clients we need to copy the data into a heap anyway I believe. We probably need performance tests to measure performance issues.

For the changes I propose at first, there would be absolutely no performance impact nor benefit.  It would be fundamentally the same operations.
My reasoning for wanting to make this change is as a way to future-proof the API.  ie. if/when we want to optimize the C bindings, we'd have the flexibility to do it transparently, rather than force all the clients to change their use case.  So getting the API right before Alpha means we'd have the flexibility to improve things behind the scenes.

Looking to the future, the two objects that can be somewhat ephemeral in the API are Atom and BindingsSet.  Those are the ones that would have be biggest benefit in the longer term.
You're absolutely right that these costs are currently dwarfed by internal and algorithmic costs (working on those too).  But I'm just trying to get the API into a state where we don't have to force the clients to change their code more than necessary after Alpha.

So a struct containing an opaque pointer, rather than a pointer to an opaque structure.

Could you please provide a small example of code, because I don't fully understand what you mean.

But I'm just trying to get the API into a state where we don't have to force the clients to change their code more than necessary after Alpha.

Here my experience tells me the opposite :-) Anyway there will be changes which will require clients to adapt their code to the new API. And making Rust structures to be presentable in C will not make life easier because after that we will need to keep compatibility in a Rust level (I have feeling that we will finish with having separate types for C anyway). And also we will have an additional requirements to have structures in Rust to be presentable in C. Thus I would propose stabilize Rust API first and after that experiment with making Rust API #[repr(C)].

This is what I have in mind.  e765c55f87366f0d0f362a8a578836f4c79a33f7
I think the two most important objects, from a performance standpoint, are atom_t and bindings_set_t, but IMO it makes sense to do this with all of them for consistency.

Thanks @luketpeterson, but replacing *mut bindings_set_t by bindings_set_t which wraps the pointer doesn't change the performance. Do you mean it is a transition change before replacing pointer inside bindings_set_t by BindingsSet instance?

Yes exactly.  This means that, in the future, we can move more meaningful contents into the bindings_set_t and the client's code can just recompile.

This has turned into a large rabbit hole, and this experience stands as a substantial data point in favor of dumping cbindgen in favor of saferffi, since the blocking issues are due to cbindgen.
I will briefly go over the issues in the order I encountered them.
1.) If atom_t is a statically-sized object owned by the C caller, somehow functions like vec_atom_get need to be able to return a reference to an existing Atom, not necessarily a copy.  So I thought returning *const atom_t would be nice, but that means a pointer to a Rust Atom needs to be 1-to-1 translatable with a ptr to an atom_t, which is only possible if they have the same memory layout.
2.) Since I wanted to make Atom into a #[repr(C)] struct anyway, I figured I'd bite the bullet and just do that.  But that turned into a deep rabbit hole because a number of Rust primitives aren't #[repr(C)] (such as String, Vec, etc.)  I could roll my own versions, but it would take a bit of time, and saferffi already provides this.
3.) So, I decided to take a different tack, and I wrote this: https://crates.io/crates/repr_c_wrapper in order to provide a nice convenient encapsulation for the Rust types that could be held by C code.
4.) Unfortunately repr_c_wrapper relies on a macro to create the corresponding opaque C type for each Rust type.  And macros aren't parsed by default by cbindgen.  cbindged has a facility to expand macros - controlled with the [parse.expand] section of the cbindgen.toml, buuuut... It invokes the rustc macro expand feature which requires the Rust nightly tool chain.  (not available under stable tool chain)  I filed this: https://github.com/mozilla/cbindgen/issues/855 but the bug has persisted for years, so I doubt it'll be fixed any time soon.
5.) Even if the macro expansion in cbindgen is possible for us, I'll need to write a proc macro (another crate) in order to support pre-computing the compile-time constants needed for the C header, because the parsing done by cbindgen is really limited.
6.) In the end, I made atom_t capable of representing both a pointer to another Atom (borrowed) or a raw Box ptr (owned), but I feel like this is a stop-gap solution.

This has been fully implemented with https://github.com/trueagi-io/hyperon-experimental/pull/395.  Closing.


Uncaught TypeError: Cannot read property 'clipboardData' of undefined when pasting

When pasting anything - all modes - the console throws the presented error.

I ended up changing the spot where it errors to this:
var t = e.originalEvent.clipboardData ? e.originalEvent.clipboardData.getData("text/plain") : window.clipboardData.getData("text");

I tested it on IE 11 and Chrome.  Works!

The issue is that originalEvent is jquery.  Since jquery is supposedly not required for ui-select, this reference should be removed completely for the fix.

+1

+1 Please fix

hey guys, do you have any progress on this?

I created a pull request #1391 to fix this yesterday.

ah, thanks
+1

There are many pull requests for this such as #1276 , But the project isn't maintained anymore so nobody merges them.

Is there a way to keep this going?
Would be sad if this project gets abandoned :confused:

I opened #1398
You are welcome to comment there

Please confirm fix in v0.14.7 and close this issue.

Closing this as fixed - feel free to open a new issue if it is still a problem.


Create Sprite Parenting

Extension for Linking Sprites Together!

LETS GO!! I have some suggestions that I will mention soon.

LETS GO!! I have some suggestions that I will mention soon.

I need to format it still :((((

Nice!

Nice!

Thanks!

A lot of your arguments are used in places where they depend on it being a string, so I advise you go back and cast them all to a string.

Fixed em all

This extension is making me imagine a conversation like:
"Hey Sprite1?"
"Yes?"
"You're adopted."

This extension is making me imagine a conversation like:
"Hey Sprite1?" "Yes?" "You're adopted."

Lol

This extension is making me imagine a conversation like:
"Hey Sprite1?" "Yes?" "You're adopted."

I was hoping for a Tommyinnit version...

When will this be stable enough to use in projects?

Yep

Now Supports Clones!

V 2.0 is out, rewrote everything, much better, optimized, and has better sprite support


Add API to add validators

This would be a nice to have feature, instead of having a static list of validators via CLI or file allow for dynamic addition of new validators via a REST API call (e.g. using fastapi).

Hi @laibe, I'm currently working on some smaller improvements e.g. that you can also provide pubkeys. However, the rest api feature is a great idea and was already requested multiple times (although a bit differently then you suggest, see issue link). That's tracked in the following issue.
With regards to your proposed functionality: My idea is that eth-duties will read managed validator keys from the validator client itself (see here) during runtime. All clients expose the validator keymanager api. So in the end the user does not need to care about supplying validators to eth-duties at all. However, this needs some extra configuration of the validator client itself and I'm not sure if it is an security issue to expose that api.
Maybe the best idea is to add both ways (automated fetching from client, add new indices/pubkey via POST call) so the user can decide.

I 100% agree. That's why it is probably the best way to implement both ways so more experienced users and professional node operators can decide to use the automated fetching and setting up their validator clients accordingly.
Yes, the authentication handling will be interesting. Need to wrap my head around this first 😃 .

HI @laibe, just for your info: The RESTful API is now available and contains two endpoints to add and delete validator identifiers during runtime. For a full endpoint documentation please check the swagger spec which can be reached via http://localhost:5000/docs.


Move portable string construction code to separate file for easier reuse by Mono

The remaining code in String.cs is the CoreRT-specific plumbing after this change. Our naming convention is to use .CoreRT.cs / .CoreCLR.cs suffix for that.
Would it make sense to keep the constructors in the main String.cs file, and instead move the bit of CoreRT specific plumbing to String.CoreRT.cs ?

@jkotas Should I still keep String.Conversions.cs?

If you do not need for Mono sharing, I think keeping everything in String.cs would look better.

@jkotas we need both. I originally thought it'd be easier to move it to Shared if it was out of String.cs

Replaced with #5478


[Request] Record video in Colab

First of all, excellent work, just what I have been looking for!
Love it that I can experiment and have well documented code.
I work mainly in Google Colab, mainly because I got tired of pip install  and it removes the hassle of infrastructure.
See you published very nice Colabs, many thanks.
I figured out a way with wrappers to have Gym create video's that is also working in Colab (with virtual monitor) , nice to have a graph explaining the score is increasing but also want to see games with my own eyes. Unfortunately that does not work anymore with Stable baselines (at least I did not yet spot it).
Any ideas how to solve this for a Colab environments with Stable-baselines?
Below example how it works for e in a standard gym environment:
`
@title Set-up the virtual display environment
!apt-get update
!apt-get install python-opengl -y
!apt install xvfb -y
!pip install pyvirtualdisplay
!pip install piglet
!apt-get install ffmpeg
@title Start the virtual monitor
from pyvirtualdisplay import Display
display = Display(visible=0, size=(1400, 900))
display.start()
@title play a random game and create video
env = gym.make("PongNoFrameskip-v4")
monitor_dir ='/content/test5'
#Setup a wrapper to be able to record a video of the game
record_video = True
should_record = lambda i: record_video
env = wrappers.Monitor(env, monitor_dir, video_callable=should_record, force=True)
#Play a game
state = env.reset()
done = False
while not done:
action = env.action_space.sample() #random action, replace by the prediction of the model
state, reward, done, _ = env.step(action)
record_video = False
env.close()
download videos
from google.colab import files
import glob
os.chdir(monitor_dir) # change directory to get the files
!pwd #show file path
!ls # show directory content
for file in glob.glob(".mp4"):
print(file)
files.download(file)
os.chdir('..') #backout the video directory
`
Wrapping the whole stable-baseline env does not work.
Tried also something like:
`
monitor_dir ='/content/test7'
env_show = gym.make('LunarLander-v2')
record_video = True
should_record = lambda i: record_video
env_show = wrappers.Monitor(env_show, monitor_dir, video_callable=should_record, force=True)
obs = env_show.reset()
dones = False
while not dones:
action, _states = model.predict(obs)
obs, rewards, dones, info = env_show.step(action)
env_show.render()
record_video = False
env_show.close()
`
Which works for Lunar Lander but not for an Atari game: Error: Unexpected observation shape (210, 160, 3) for Box environment, please use (84, 84, 4) or (n_env, 84, 84, 4) for the observation shape.
Any pragmatic elegant solution or small enhancement for Colab users?
Thank you

Hi,
Thanks you for your request.
It seems that an important part is missing: the Monitor class that you are using.
I would also use VideoRecorder from gym (see https://github.com/openai/gym/blob/master/gym/wrappers/monitoring/video_recorder.py ). To work with VecEnc, I would advise you to take a look at what was done in the baselines repo here: https://github.com/openai/baselines/blob/master/baselines/common/vec_env/vec_video_recorder.py
Also, please use markdown code blocks (see https://guides.github.com/features/mastering-markdown/):
```python
def test():
    pass
\```

Many thanks for the quick reply, will try it out.
Note: I used the markdown button "insert code" of the website but indeed did not give the wanted markdown but only single '', apologies I did not spot it.
Suggestion to add a ready-to-use colab example to you repository and on your website to record a video and download the result, I'll bet you you will reach more people.

Suggestion to add a ready-to-use colab example to you repository and on your website to record a >video and download the result, I'll bet you you will reach more people. The updated "enjoy script" to >record a video in colab

Well, if you provide such a notebook, we will certainly link it in the documentation ;)

:-)
I am trying, but I am afraid my skills in this domain are not as good as one would hope.
I first tried your first link:
'''
Enjoy trained agent
from gym.wrappers.monitoring.video_recorder import VideoRecorder
monitor_dir ='/content/test12'
env = VideoRecorder(env, base_path=monitor_dir, enabled=True)
obs = env.reset()
dones=False
while not dones:
action, _states = model.predict(obs)
obs, rewards, dones, info = env.step(action)
env.render()
VideoRecorder.close()
'''
which got me the error: AttributeError: 'VecFrameStack' object has no attribute 'metadata'
So I have to search further in the second link.
Any suggestions most welcome :-)

The video recorder provided by gym will only work with simple gym env, not VecEnv nor VecEnvWrapper (that are used by stable-baselines to facilitate multiprocessing and preprocessing)

@Charelvanhoof I have got a good news for you. I have been tweaking VecVideoRecorder so it should work with any type of VecEnv.
It is available on the 'record-video-branch', here: https://github.com/hill-a/stable-baselines/blob/record-videos/stable_baselines/common/vec_env/vec_video_recorder.py
Consider this feature to be experimental
a minimal example:
import gym
from stable_baselines.common.vec_env import VecVideoRecorder, DummyVecEnv

env_id = 'CartPole-v1'
video_folder = 'logs/videos/'
video_length = 100

env = DummyVecEnv([lambda: gym.make(env_id)])

obs = env.reset()

env = VecVideoRecorder(env, video_folder,
                       record_video_trigger=lambda x: x == 0, video_length=video_length,
                       name_prefix="random-agent-{}".format(env_id))

env.reset()
for _ in range(video_length + 1):
    action = [env.action_space.sample()]
    obs, _, _, _ = env.step(action)
env.close()

Important note: ffmpeg or avconc must be installed on the machine

@araffin brilliant, than you! I have tested this morning with various environments  (eg SpaceInvaders, Pong, etc) and it works. Also with make_atari_env and VecFrameStack.
I vote this feature to be upgraded from experimental to baseline.
An example of training an agent (also to check I did it right), e.g. is the
action, _states = model.predict(obs)

the correction interpretation because was not sure about the [ ] brackets in the example.
Below a lunar lander with training:
# @title trained agent
env_id = 'LunarLander-v2'
monitor_dir = 'logs/videos/'
video_length = 300

env = DummyVecEnv([lambda: gym.make(env_id)])

model = A2C(MlpPolicy, env, ent_coef=0.1, verbose=1)
model.learn(total_timesteps=35000)

print("Training complete")

obs = env.reset()

env = VecVideoRecorder(env, monitor_dir,
                       record_video_trigger=lambda x: x == 0, video_length=video_length,
                       name_prefix="trained-agent-{}".format(env_id))

env.reset()
for _ in range(video_length + 1):
    action, _states = model.predict(obs)
    obs, _, _, _ = env.step(action)
env.close()

Or pong with training:
env_id = 'PongNoFrameskip-v4'
monitor_dir = 'logs/Pong/'
video_length = 1000

# There already exists an environment generator
# that will make and wrap atari environments correctly.
# Here we are also multiprocessing training (num_env=4 => 4 processes)
env = make_atari_env(env_id, num_env=4, seed=0)
# Frame-stacking with 4 frames
env = VecFrameStack(env, n_stack=4)

model = ACER(CnnPolicy, env, verbose=1)
model.learn(total_timesteps=50000)

print("Training complete")

obs = env.reset()

env = VecVideoRecorder(env, monitor_dir,
                       record_video_trigger=lambda x: x == 0, video_length=video_length,
                       name_prefix="trained-agent-{}".format(env_id))

env.reset()
for _ in range(video_length + 1):
    #action = [env.action_space.sample()]
    action, _states = model.predict(obs)
    obs, _, _, _ = env.step(action)
env.close()

Here you notice the video has 4 screens pong in parallel, kind of nice as well.

I have tested this morning with various environments (eg SpaceInvaders, Pong, etc) and it works. Also >with make_atari_env and VecFrameStack.
Yes, in fact I used this script from https://github.com/araffin/rl-baselines-zoo to test the feature ;)

I vote this feature to be upgraded from experimental to baseline.

I created the pull request, waiting for @hill-a review.

Yes, with vec env, actions must be array even with only one env, hence the []


chore(main): release 1.1.3

:robot: I have created a release beep boop
1.1.3 (2022-11-07)
Only changes to build infrastructure.

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/apollo-server-integrations/apollo-server-integration-h3/releases/tag/v1.1.3 :sunflower:


Support for circular/arc geometries

Looked into this a bit and while it would probably be quite simple to add support for the basic CIRCULARSTRING type perhaps it's not of much value without also having COMPOUNDCURVE, CURVEPOLYGON, MULTICURVE and MULTISURFACE and these have the same problems/complexity as GEOMETRYCOLLECTION i.e they are non homogenous collections and can be nested.

That said, I'm doing it anyway in https://github.com/bjornharrtell/flatgeobuf/pull/14. :)


Implement async_std::sync::Condvar

Part of #217

@nbdd0121 do you have any suggestion on how to resolve your concerns?

I think notifying another waker when a WaitFuture is dropped with its opt_waker set to None should be enough. This should handle the notify_one well, but it may cause extra tasks being notified if we
have the following case:

Task A calls wait
Task B calls notify_all
Task C calls wait
Task A cancels wait

Task C will ends up being waken up. However I don't think it's a huge problem, because std's Condvar mentions that:

Note that this function is susceptible to spurious wakeups. Condition variables normally have a boolean predicate associated with them, and the predicate must always be checked each time this function returns to protect against spurious wakeups.

IMO we don't need has_blocked. Unlike Mutex which assumes most calls don't block, for Condvar I think we can assume there are tasks blocking when notify_one or notify_all is called.

I feel like the wait/wait_timeout are kinda big footguns. If I understand them properly, the returned future will always return Ready the second time they're polled, without necessarily being actually notified. This can happen quite easily by using e.g. the join adapter from futures-rs.
This is technically okay according to Condvars' spurious wakeup conditions, but that is not mentioned at all in the docs. Besides, it seems very trivial to cause a spurious wakeup, and will likely produce bad, hard to debug code. I kinda question the usefulness of those functions compared to wait_until.

If I understand them properly, the returned future will always return Ready the second time they're polled

Kind of. They will return Ready if they are polled and the attached Mutex is available. I could potentially use an Arc<AtomicBool> to keep track of if it was actually notified. But I think (though I don't have benchmarks) that in most cases that would hurt performance.

Kind of. They will return Ready if they are polled and the attached Mutex is available. I could potentially use an Arc<AtomicBool> to keep track of if it was actually notified. But I think (though I don't have benchmarks) that in most cases that would hurt performance.

Probably only need to check if Option<Waker> is turned to None. Otherwise it'll be spurious wake up.

Probably only need to check if Option is turned to None. Otherwise it'll be spurious wake up.

Yes, but that requires locking the mutex for the wakers when polled, which is doable, but could hurt performance if the condition variable has very much contention. And if wait_until/wait_timeout_until are used, then it isn't an issue.

You need to lock the mutex to remove the waker when cancelling/completing the notify call, or to insert the waker when beginning a new call. I don't see why it'll hurt performance compared to wait_until.

Ok, I'll take a stab at implementing that. It looks like there is also now a WakerSet type that I should use instead of a Slab of Wakers.

@nbdd0121 I think my latest push should address your concerns.

WakerSet's notify_one does not wake up anything if some task is woken already. This is slightly different from Condvar's notify_one which should always wake up one task (in the case of two concurrent notify_one call for example).

If i understand the code on master that is how notify_any behaves, but notify_one behaves as we want for condvar.

Sorry, you're right. That change is introduced in e9edadf which isn't part of a PR so I am not aware of.

I rebased and addressed your comments. Should I squash as well?

Thank you


Get an error "OverflowError: Python int too large to convert to C long" when loading a large dataset

Describe the bug
When load a large dataset with the following code
from datasets import load_dataset
dataset = load_dataset("liwu/MNBVC", 'news_peoples_daily', split='train')

We encountered the error: "OverflowError: Python int too large to convert to C long"
The error look something like:
OverflowError: Python int too large to convert to C long

During handling of the above exception, another exception occurred:

OverflowError                             Traceback (most recent call last)
<ipython-input-7-0ed8700e662d> in <module>
----> 1 dataset = load_dataset("liwu/MNBVC", 'news_peoples_daily', split='train', cache_dir='/sfs/MNBVC/.cache/')

/sfs/MNBVC/venv/lib64/python3.6/site-packages/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, keep_in_memory, save_infos, revision, use_auth_token, task, streaming, **config_kwargs)
   1749         ignore_verifications=ignore_verifications,
   1750         try_from_hf_gcs=try_from_hf_gcs,
-> 1751         use_auth_token=use_auth_token,
   1752     )
   1753

/sfs/MNBVC/venv/lib64/python3.6/site-packages/datasets/builder.py in download_and_prepare(self, download_config, download_mode, ignore_verifications, try_from_hf_gcs, dl_manager, base_path, use_auth_token, **download_and_prepare_kwargs)
    703                     if not downloaded_from_gcs:
    704                         self._download_and_prepare(
--> 705                             dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
    706                         )
    707                     # Sync info

/sfs/MNBVC/venv/lib64/python3.6/site-packages/datasets/builder.py in _download_and_prepare(self, dl_manager, verify_infos)
   1225
   1226     def _download_and_prepare(self, dl_manager, verify_infos):
-> 1227         super()._download_and_prepare(dl_manager, verify_infos, check_duplicate_keys=verify_infos)
   1228
   1229     def _get_examples_iterable_for_split(self, split_generator: SplitGenerator) -> ExamplesIterable:

/sfs/MNBVC/venv/lib64/python3.6/site-packages/datasets/builder.py in _download_and_prepare(self, dl_manager, verify_infos, **prepare_split_kwargs)
    791             try:
    792                 # Prepare split will record examples associated to the split
--> 793                 self._prepare_split(split_generator, **prepare_split_kwargs)
    794             except OSError as e:
    795                 raise OSError(

/sfs/MNBVC/venv/lib64/python3.6/site-packages/datasets/builder.py in _prepare_split(self, split_generator, check_duplicate_keys)
   1219                     writer.write(example, key)
   1220             finally:
-> 1221                 num_examples, num_bytes = writer.finalize()
   1222
   1223         split_generator.split_info.num_examples = num_examples

/sfs/MNBVC/venv/lib64/python3.6/site-packages/datasets/arrow_writer.py in finalize(self, close_stream)
    536             # Re-intializing to empty list for next batch
    537             self.hkey_record = []
--> 538         self.write_examples_on_file()
    539         if self.pa_writer is None:
    540             if self.schema:

/sfs/MNBVC/venv/lib64/python3.6/site-packages/datasets/arrow_writer.py in write_examples_on_file(self)
    407             # Since current_examples contains (example, key) tuples
    408             batch_examples[col] = [row[0][col] for row in self.current_examples]
--> 409         self.write_batch(batch_examples=batch_examples)
    410         self.current_examples = []
    411

/sfs/MNBVC/venv/lib64/python3.6/site-packages/datasets/arrow_writer.py in write_batch(self, batch_examples, writer_batch_size)
    506             col_try_type = try_features[col] if try_features is not None and col in try_features else None
    507             typed_sequence = OptimizedTypedSequence(batch_examples[col], type=col_type, try_type=col_try_type, col=col)
--> 508             arrays.append(pa.array(typed_sequence))
    509             inferred_features[col] = typed_sequence.get_inferred_type()
    510         schema = inferred_features.arrow_schema if self.pa_writer is None else self.schema

/sfs/MNBVC/venv/lib64/python3.6/site-packages/pyarrow/array.pxi in pyarrow.lib.array()

/sfs/MNBVC/venv/lib64/python3.6/site-packages/pyarrow/array.pxi in pyarrow.lib._handle_arrow_array_protocol()

/sfs/MNBVC/venv/lib64/python3.6/site-packages/datasets/arrow_writer.py in __arrow_array__(self, type)
    180             else:
    181                 trying_cast_to_python_objects = True
--> 182                 out = pa.array(cast_to_python_objects(data, only_1d_for_numpy=True))
    183             # use smaller integer precisions if possible
    184             if self.trying_int_optimization:

/sfs/MNBVC/venv/lib64/python3.6/site-packages/pyarrow/array.pxi in pyarrow.lib.array()

/sfs/MNBVC/venv/lib64/python3.6/site-packages/pyarrow/array.pxi in pyarrow.lib._sequence_to_array()

/sfs/MNBVC/venv/lib64/python3.6/site-packages/pyarrow/error.pxi in pyarrow.lib.pyarrow_internal_check_status()

OverflowError: Python int too large to convert to C long

However, that dataset can be loaded in a streaming manner:
from datasets import load_dataset
dataset = load_dataset("liwu/MNBVC", 'news_peoples_daily', split='train', streaming=True)

for i in dataset:
    pass  # it work well

Another issue is reported in our dataset hub:
https://huggingface.co/datasets/liwu/MNBVC/discussions/2
Steps to reproduce the bug
from datasets import load_dataset
dataset = load_dataset("liwu/MNBVC", 'news_peoples_daily', split='train')
Expected behavior
the dataset can be safely loaded
Environment info

datasets version: 2.4.0
Platform: Linux-3.10.0-1160.an7.x86_64-x86_64-with-centos-7.9
Python version: 3.6.8
PyArrow version: 6.0.1
Pandas version: 1.1.5

This error means that one of the int32 columns in the dataset has a value that is out of the valid (int32) range.
I'll open a PR to print the name of a problematic column to make debugging such errors easier.

I am afraid int32 is not the reason for this error.
I have submitted a commit to use int64 for all ints in the dataset:
https://huggingface.co/datasets/liwu/MNBVC/commit/857ac00d9eab96a6708ad6a82bd9001686042a9e
and I have updated my env to the latest datasets release:
Copy-and-paste the text below in your GitHub issue.

datasets version: 2.13.1
Platform: macOS-13.2.1-arm64-arm-64bit
Python version: 3.11.2
Huggingface_hub version: 0.13.4
PyArrow version: 11.0.0
Pandas version: 1.5.3

But the error still exist
Downloading and preparing dataset mnbvc/news_peoples_daily to /Users/silver/.cache/huggingface/datasets/liwu___mnbvc/news_peoples_daily/0.0.1/ee380f6309fe9b8b0d1fb14d77118f132444f22c8c4b28bf5c1645312688e051...
Downloading data files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 9070.40it/s]
Extracting data files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 2697.16it/s]
---------------------------------------------------------------------------
OverflowError                             Traceback (most recent call last)
File ~/git/venv/lib/python3.11/site-packages/datasets/builder.py:1647, in GeneratorBasedBuilder._prepare_split_single(self, gen_kwargs, fpath, file_format, max_shard_size, split_info, check_duplicate_keys, job_id)
   1646 example = self.info.features.encode_example(record) if self.info.features is not None else record
-> 1647 writer.write(example, key)
   1648 num_examples_progress_update += 1

File ~/git/venv/lib/python3.11/site-packages/datasets/arrow_writer.py:490, in ArrowWriter.write(self, example, key, writer_batch_size)
    488     self.hkey_record = []
--> 490 self.write_examples_on_file()

File ~/git/venv/lib/python3.11/site-packages/datasets/arrow_writer.py:448, in ArrowWriter.write_examples_on_file(self)
    444         batch_examples[col] = [
    445             row[0][col].to_pylist()[0] if isinstance(row[0][col], (pa.Array, pa.ChunkedArray)) else row[0][col]
    446             for row in self.current_examples
    447         ]
--> 448 self.write_batch(batch_examples=batch_examples)
    449 self.current_examples = []

File ~/git/venv/lib/python3.11/site-packages/datasets/arrow_writer.py:553, in ArrowWriter.write_batch(self, batch_examples, writer_batch_size)
    552 typed_sequence = OptimizedTypedSequence(col_values, type=col_type, try_type=col_try_type, col=col)
--> 553 arrays.append(pa.array(typed_sequence))
    554 inferred_features[col] = typed_sequence.get_inferred_type()

File ~/git/venv/lib/python3.11/site-packages/pyarrow/array.pxi:236, in pyarrow.lib.array()

File ~/git/venv/lib/python3.11/site-packages/pyarrow/array.pxi:110, in pyarrow.lib._handle_arrow_array_protocol()

File ~/git/venv/lib/python3.11/site-packages/datasets/arrow_writer.py:189, in TypedSequence.__arrow_array__(self, type)
    188     trying_cast_to_python_objects = True
--> 189     out = pa.array(cast_to_python_objects(data, only_1d_for_numpy=True))
    190 # use smaller integer precisions if possible

File ~/git/venv/lib/python3.11/site-packages/pyarrow/array.pxi:320, in pyarrow.lib.array()

File ~/git/venv/lib/python3.11/site-packages/pyarrow/array.pxi:39, in pyarrow.lib._sequence_to_array()

File ~/git/venv/lib/python3.11/site-packages/pyarrow/error.pxi:144, in pyarrow.lib.pyarrow_internal_check_status()

OverflowError: Python int too large to convert to C long

During handling of the above exception, another exception occurred:

OverflowError                             Traceback (most recent call last)
File ~/git/venv/lib/python3.11/site-packages/datasets/builder.py:1656, in GeneratorBasedBuilder._prepare_split_single(self, gen_kwargs, fpath, file_format, max_shard_size, split_info, check_duplicate_keys, job_id)
   1655 num_shards = shard_id + 1
-> 1656 num_examples, num_bytes = writer.finalize()
   1657 writer.close()

File ~/git/venv/lib/python3.11/site-packages/datasets/arrow_writer.py:584, in ArrowWriter.finalize(self, close_stream)
    583     self.hkey_record = []
--> 584 self.write_examples_on_file()
    585 # If schema is known, infer features even if no examples were written

File ~/git/venv/lib/python3.11/site-packages/datasets/arrow_writer.py:448, in ArrowWriter.write_examples_on_file(self)
    444         batch_examples[col] = [
    445             row[0][col].to_pylist()[0] if isinstance(row[0][col], (pa.Array, pa.ChunkedArray)) else row[0][col]
    446             for row in self.current_examples
    447         ]
--> 448 self.write_batch(batch_examples=batch_examples)
    449 self.current_examples = []

File ~/git/venv/lib/python3.11/site-packages/datasets/arrow_writer.py:553, in ArrowWriter.write_batch(self, batch_examples, writer_batch_size)
    552 typed_sequence = OptimizedTypedSequence(col_values, type=col_type, try_type=col_try_type, col=col)
--> 553 arrays.append(pa.array(typed_sequence))
    554 inferred_features[col] = typed_sequence.get_inferred_type()

File ~/git/venv/lib/python3.11/site-packages/pyarrow/array.pxi:236, in pyarrow.lib.array()

File ~/git/venv/lib/python3.11/site-packages/pyarrow/array.pxi:110, in pyarrow.lib._handle_arrow_array_protocol()

File ~/git/venv/lib/python3.11/site-packages/datasets/arrow_writer.py:189, in TypedSequence.__arrow_array__(self, type)
    188     trying_cast_to_python_objects = True
--> 189     out = pa.array(cast_to_python_objects(data, only_1d_for_numpy=True))
    190 # use smaller integer precisions if possible

File ~/git/venv/lib/python3.11/site-packages/pyarrow/array.pxi:320, in pyarrow.lib.array()

File ~/git/venv/lib/python3.11/site-packages/pyarrow/array.pxi:39, in pyarrow.lib._sequence_to_array()

File ~/git/venv/lib/python3.11/site-packages/pyarrow/error.pxi:144, in pyarrow.lib.pyarrow_internal_check_status()

OverflowError: Python int too large to convert to C long

The above exception was the direct cause of the following exception:

DatasetGenerationError                    Traceback (most recent call last)
Cell In[2], line 1
----> 1 dataset = load_dataset("liwu/MNBVC", 'news_peoples_daily', split='train')

File ~/git/venv/lib/python3.11/site-packages/datasets/load.py:1809, in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, verification_mode, ignore_verifications, keep_in_memory, save_infos, revision, use_auth_token, task, streaming, num_proc, storage_options, **config_kwargs)
   1806 try_from_hf_gcs = path not in _PACKAGED_DATASETS_MODULES
   1808 # Download and prepare data
-> 1809 builder_instance.download_and_prepare(
   1810     download_config=download_config,
   1811     download_mode=download_mode,
   1812     verification_mode=verification_mode,
   1813     try_from_hf_gcs=try_from_hf_gcs,
   1814     num_proc=num_proc,
   1815     storage_options=storage_options,
   1816 )
   1818 # Build dataset for splits
   1819 keep_in_memory = (
   1820     keep_in_memory if keep_in_memory is not None else is_small_dataset(builder_instance.info.dataset_size)
   1821 )

File ~/git/venv/lib/python3.11/site-packages/datasets/builder.py:909, in DatasetBuilder.download_and_prepare(self, output_dir, download_config, download_mode, verification_mode, ignore_verifications, try_from_hf_gcs, dl_manager, base_path, use_auth_token, file_format, max_shard_size, num_proc, storage_options, **download_and_prepare_kwargs)
    907     if num_proc is not None:
    908         prepare_split_kwargs["num_proc"] = num_proc
--> 909     self._download_and_prepare(
    910         dl_manager=dl_manager,
    911         verification_mode=verification_mode,
    912         **prepare_split_kwargs,
    913         **download_and_prepare_kwargs,
    914     )
    915 # Sync info
    916 self.info.dataset_size = sum(split.num_bytes for split in self.info.splits.values())

File ~/git/venv/lib/python3.11/site-packages/datasets/builder.py:1670, in GeneratorBasedBuilder._download_and_prepare(self, dl_manager, verification_mode, **prepare_splits_kwargs)
   1669 def _download_and_prepare(self, dl_manager, verification_mode, **prepare_splits_kwargs):
-> 1670     super()._download_and_prepare(
   1671         dl_manager,
   1672         verification_mode,
   1673         check_duplicate_keys=verification_mode == VerificationMode.BASIC_CHECKS
   1674         or verification_mode == VerificationMode.ALL_CHECKS,
   1675         **prepare_splits_kwargs,
   1676     )

File ~/git/venv/lib/python3.11/site-packages/datasets/builder.py:1004, in DatasetBuilder._download_and_prepare(self, dl_manager, verification_mode, **prepare_split_kwargs)
   1000 split_dict.add(split_generator.split_info)
   1002 try:
   1003     # Prepare split will record examples associated to the split
-> 1004     self._prepare_split(split_generator, **prepare_split_kwargs)
   1005 except OSError as e:
   1006     raise OSError(
   1007         "Cannot find data file. "
   1008         + (self.manual_download_instructions or "")
   1009         + "\nOriginal error:\n"
   1010         + str(e)
   1011     ) from None

File ~/git/venv/lib/python3.11/site-packages/datasets/builder.py:1508, in GeneratorBasedBuilder._prepare_split(self, split_generator, check_duplicate_keys, file_format, num_proc, max_shard_size)
   1506 job_id = 0
   1507 with pbar:
-> 1508     for job_id, done, content in self._prepare_split_single(
   1509         gen_kwargs=gen_kwargs, job_id=job_id, **_prepare_split_args
   1510     ):
   1511         if done:
   1512             result = content

File ~/git/venv/lib/python3.11/site-packages/datasets/builder.py:1665, in GeneratorBasedBuilder._prepare_split_single(self, gen_kwargs, fpath, file_format, max_shard_size, split_info, check_duplicate_keys, job_id)
   1663     if isinstance(e, SchemaInferenceError) and e.__context__ is not None:
   1664         e = e.__context__
-> 1665     raise DatasetGenerationError("An error occurred while generating the dataset") from e
   1667 yield job_id, True, (total_num_examples, total_num_bytes, writer._features, num_shards, shard_lengths)

DatasetGenerationError: An error occurred while generating the dataset

Besides, it works fine when I am using streamed dataset.

simhash is the problematic column - it has values such as<PHONE_NUMBER>0363166823 that are out of the int64 range. You can fix this by setting the feature type to Value("string") (it's advised to use this type for hash values in general)

Besides, it works fine when I am using streamed dataset.

Streaming yields Python dictionaries from the script without converting them to the Arrow representation, as this conversion step is not that cheap performance-wise.

i am using uint64 for simhash
uint64 ranges up to about 3.69E19.
18329103420363166823 is less than this value.
moreover, our simhash algorithm use 64 bits. it should fit in uint64.

You are right. I overlooked the feature type.
This is a reproducer:
import pyarrow as pa
from datasets.arrow_writer import TypedSequence

pa.array(TypedSequence([18329103420363166823], type=Value("uint64")))

pa.array([18329103420363166823]) also fails with the same error, so it seems PyArrow does not always infer the correct type as NumPy does.
I'll report this issue in the Arrow repo.
pa.array([18329103420363166823], pa.uint64) works, so maybe we can implement a temporary fix (supporting complex input such as [{"image": pil_image, "num": uint64_value}] would be hard though).
In the meantime, you should be able to bypass this error by returning the simhash values as NumPy scalars in the script:
def _generate_examples(self, ...):
    ...
    yield {...,  "simhash": np.uint64(simhash), ...}

Thank you for checking this issue in detail.
However, it seems that using np.uint64(simhash) does not work. The same issue still exists.
https://huggingface.co/datasets/liwu/MNBVC/commit/1e44f1e400b7e61052647d44c99cdae3bae9c830
Anyway, we decide to use string type for these simhash values. Hope pyarrow can fix their bug soon.

Arrow issue: https://github.com/apache/arrow/issues/36520

May be something read your training data line by line.
Then your  training data just only one line.
It is so large.
I guess.


Ensure that button.btn and a.btn have the same cursor

This PR adds the cursor: default property to the .btn class to ensure that button.btn and a.btn have matching cursor behavior. It also adds cursor: pointer to .btn-link. I believe this aligns with the intent of 232e86d0b4e1a4ff0dc3aa4e6d75c14a54fa8ac8 (added as part of https://github.com/twbs/bootstrap/pull/21439).

@lucascono cursor: pointer was purposely removed from .btn in https://github.com/twbs/bootstrap/commit/232e86d0b4e1a4ff0dc3aa4e6d75c14a54fa8ac8. This PR is only attempting to ensure that the intent of that commit is implemented consistently.

Hello @md5 :
I have read the article and understand the point. First of all I thank you for answering. However, just by habit, I think a bad decision. From the technical point of view I understand the reasons, but considering what the common user usually uses, I think cursor: pointer is better.
In the same article published by Adam Silver, in the comments, the "Bookmark this story to read later" icon is a button and has cursor: pointer. On the same page that promotes the opposite.
Obviously I consider that the decision of the authors of Bootstrap is already taken. However, I did not want to stop giving my opinion and say that, at least in my future projects with v4, this will be the first thing I have to change forever. Beyond what Adam Silver says, I think this has been a very bad decision. But as I said before, it's just my opinion.
Thanks again for responding and for collaborating with such a nice project. Greetings from Argentina (sorry for my english).

I've rebased this to fix a conflict with the changes in cd22eb1da03d5dbc9062ad5c9e75c05429e4bcd0

@mdo Is this change something you'd like to accept for Bootstrap v4? If not, I'd like to close the PR.

Closing in favor of #27021


Watchify execution fails when I install Browserify first

Hello,
I have a weird behaviour when I want to execute watchify.
If I install watchify first, and then browserify, I have no error.
The error happens only when I install browserify first. This is problematic because I have those two modules in my project depencendies, so when I type 'npm install', browserify is always installed before watchify.
Here is the complete error :
module.js:340
    throw err;
    ^
Error: Cannot find module 'browserify'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous> (/home/vagrant/node_modules/js/angulpify-test/watchify/index.js:3:18)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)

Weird dependency issues.

Noticed this has been closed, but still encountering in here.

@jondashkyle Can you open a new issue and provide some more context?


can't use fzf to search history in insert mode

Basic examination

[ x] I have read through the README page
[ x] I have the latest version of zsh-vi-mode
[ ] I have tested with another terminal program

Problem description

When i in insert mode , pressed ctrl-r , fzf not work. But in normal mode,fzf works.
Sorry to duplicate , i guess it's sourcing's sequence problem. I have readed the closed issues about fzf and zsh-vi-mode.
Reproduction steps
1.I appended zvm_after_init_commands+=('[ -f $HOME/.fzf.zsh ] && source $HOME/.fzf.zsh'  ) in zsh-vi-mode.zsh
2.In my .zshrc, source $(brew --prefix)/opt/zsh-vi-mode/share/zsh-vi-mode/zsh-vi-mode.plugin.zsh
3.source .zshrc
Expected behavior
After pressed ctrl -r , fzf can work to search history.
Actual behavior
In insert mode, fzf works. But appears a string of words,like " -ne \e[6 q      -ne \e[6 q". Whenever i input a command, "-ne \e[6 q " still appeas. It's so strange.

May be this will help.
https://github.com/jeffreytse/zsh-vi-mode#initialization-mode


[android] remove some reference of DomObject on input

JIRA Issue: https://issues.apache.org/jira/browse/WEEX-41

0.13?

Will we continue to release the next version from this branch? This is an urgent bugfix


'require' result used in snax service

in a snax service, sometimes it's needed like:
local a = require "b"
local c = a("d")
function init(...)
end
......
a is nil when the interface is parsed.

You should write it into function init()

It's ok to write in init function,we used to. We need a special require to use share data for config file,so it's convenient to use like require, not to split into two steps.And, it's only needed to write in init function in service file, not int other files, so, we want to use it consistently in all files. Just for convenient and consistent, not a must.
local specialRequire = require "specialRequire"
local data1 = specialRequire("data1")
local data2 = specialRequire("data2")
function init(...)
end
......


Epic Games Store logging in with Google or facebook?

Hi, my accounts are logged in with Google and facebook, how do I setup in env?

Login to your account and see if there is an email tied to it. If there is, try and login with that email and see if it works using your Google or Facebook account.
This probably won't work, since signing in with Google/Facebook uses a different authentication paradigm. But it's worth a shot.
If that doesn't work, I have no clue how to make it work other than setting up signing in via email


[Missing SNMP Polling Profile] Digi AnywhereUSB 2 Industrial

Device Vendor: Digi
Device Model: Digi AnywhereUSB 2 Industrial
System Object Identifier (SysOID): .<IP_ADDRESS>.4.1.8<IP_ADDRESS>
List of critical metrics that you expect in the profile (does not need to be all-inclusive, just a general idea of your expectations):

CPU Utilization
Interface status

If there is a New Relic technical support case related to this request, please provide the case number:
Please provide a public gist with a sanitized (sensitive information removed) SNMP walk of the device. Providing MIB files is not a substitute for an SNMP walk and failure to provide one here will result in delays for closing this issue. The output of these commands shows the "true story" of what a device will respond to in SNMP and is used to ensure that our profiles are efficient and accurate for your devices.
Gist Link: https://gist.github.com/cstoli-92/f2e07d034c123be755ec80f509d2dcfc

Example SNMP walk commands:
# v2c example
snmpwalk -v 2c -t 10 -On -c $COMMUNITY_STRING $IP_ADDRESS . >> snmp_walk.out

# v3 example
snmpwalk -v3 -t 10 -l authPriv -u $USERNAME -a MD5|SHA -A $AUTH_PASS -x DES|AES -X $PRIV_PASS -ObentU -Cc $IP_ADDRESS . >> snmp_walk.out

@cstoli-92 This device is just presenting the generic net-snmp sysOID and there is nothing in this walk that is not already captures in the default profile, cpu and memory utilization are just based on the generic host resources mib.  There is no need for any additional profile to be built.


Refactor: simplify Message Unpacking

Unpack message id using exceptions instead of returning false.
Way more intuitive.
Notice the slight behavior change,  if unpacking any part of the message throws IOException,  then we also log an error and disconnect,  not just the header.

I am still not a big fan of C#'s exceptions, especially for gameservers.
in Java, you are forced to handle them.
in C#, you aren't. we forget to try/catch one exception somewhere and then people are able to crash the whole server by sending an invalid utf8 string or similar. that would be a nightmare.

  Debug.LogError(ex);
} finally {
  // Might not be needed
}

That way if a unhandled exception happens, we can just fall through to killing the connection and keep on running. Having a try inside a try might be weird, but adds some insurance. Feel free to correct me if I'm wrong.

@vis2k  but the booleans are so much worse:

You forget to check the result ( we do in a lot of places), and it silently ignores the error
You can't explain with a boolean why it failed.

That’s a valid point.


Hive ignores requested instance type

When running a cluster provisioning request, hive ignores the desired instance type, and creates a t2.medium instances instead.

Fixed as far as Hive is concerned in next update, but installer work is still needed. See https://github.com/openshift/installer/issues/545 for tracking.


Implement predict_proba as per #138

See #138
Last test fit_predict_probabilities is failing, I don't know if because of the implementation or I am missing something in replicating the test in sklearn:
from sklearn.ensemble import RandomForestClassifier

clf = RandomForestClassifier(criterion="gini")
clf.fit(x, y)
print(clf.predict_proba(x))

[[0.99 0.01]
 [0.82 0.18]
 [0.97 0.03]
 [0.8  0.2 ]
 [0.99 0.01]
 [0.9  0.1 ]
 [0.99 0.01]
 [0.91 0.09]
 [0.23 0.77]
 [0.4  0.6 ]
 [0.   1.  ]
 [0.   1.  ]
 [0.   1.  ]
 [0.   1.  ]
 [0.   1.  ]
 [0.   1.  ]
 [0.01 0.99]
 [0.02 0.98]
 [0.   1.  ]
 [0.01 0.99]]

Hey guys! My team and I have just started using this library for our project. This functionality would be awesome for what we're doing. Is there any update on the progress of this?

Hey guys! My team and I have just started using this library for our project. This functionality would be awesome for what we're doing. Is there any update on the progress of this?

Hey, I think that there have not been more updates, none of the contributors have had time to finish it

Hey guys! My team and I have just started using this library for our project. This functionality would be awesome for what we're doing. Is there any update on the progress of this?

It would be nice to have somebody with understanding of this feature, please help if you know how to implement this.

Hi! Thanks for providing this nice library, we are finding the random forest implementations really useful.
I am very interested in getting predicted class probabilities from the random forest classifier, so I have been looking into this issue.
As far as I can tell, the way sklearn does it is recording the per-class sample counts in each node when fitting a decision tree. The class probabilities returned by DecisionTreeClassifier.predict_proba are then the per-class sample counts for each the predicted node divided by the total sample count of the node. The predict_proba method of RandomForestClassifier then calls predict_proba for each tree and averages the resulting probabilities.
I have implemented this in a separate smartcore fork, and it gives results that are quite close to the RandomForestClassifier::predict_proba implementation in this branch. The latter gives a bit more coarse-grained results since it uses the predicted classes rather than the underlying probabilities.
Since the results are pretty similar to before, they still deviate by up to 10% from sklearn's class probabilities for the input used in the fit_predict_probabilities test. But there are other inputs that give the exact same probabilities as sklearn. So I suspect the differences are not due to the predict_proba implementation, but rather a result of different splitting policies when building a decision tree.
Here is a small test case showing a difference in splitting between smartcore and sklearn:
This sklearn test passes:
from sklearn.tree import DecisionTreeClassifier

X = [
    [1., 1., 0.],
    [1., 1., 0.],
    [1., 1., 1.],
    [1., 0., 0.],
    [1., 0., 1.],
]

y = [1, 1, 0, 0, 1]

dt = DecisionTreeClassifier()
dt.fit(X, y)

assert dt.tree_.node_count == 7

This corresponding  smartcore test fails, since the fit results in a tree with only a single node.
let x = DenseMatrix::from_2d_array(&[
    &[1., 1., 0.],
    &[1., 1., 0.],
    &[1., 1., 1.],
    &[1., 0., 0.],
    &[1., 0., 1.],
])
.unwrap();

let y = vec![1, 1, 0, 0, 1];

// We use the same defaults as sklearn
let classifier =
    DecisionTreeClassifier::fit(&x, &y, DecisionTreeClassifierParameters::default())
        .unwrap();

assert_eq!(classifier.nodes().len(), 7);

This might be old news, but I think it shows that we can't expect to get the same probabilities as sklearn, at least not without first replicating their exact splitting policy (which seems more complicated).
Since we have now tried two different predict_proba implementations that give similar results and also agree with sklearn for inputs where the splitting is more straightforward, it seems safe to me to proceed with merging one of them. The implementation in my fork has the advantage of providing predict_proba for DecisionTreeClassifier in addition to RandomForestClassifier and that the probabilities are probably a bit more precise. The disadvantage is having to store num_features * num_classes counts in every decision tree. An option could be to add a keep_counts parameter to DecisionTreeClassifierParameters and let predict_proba fail if the counts were not kept, similarly to keep_samples for RandomForestClassifier.

Hi! It would be really nice to move forwards on this. Do you have any thoughts, @Mec-iS, @montanalow, @morenol?


The diff coverage is 88.88%.

@@            Coverage Diff             @@
##           master       #4      +/-   ##
==========================================
+ Coverage   82.23%   84.27%   +2.03%
==========================================
  Files           3        3
  Lines         152      159       +7
==========================================
+ Hits          125      134       +9
+ Misses         17       16       -1
+ Partials       10        9       -1

Impacted Files
Coverage Δ

bloomfilter.go
87.87% <88.88%> (+4.82%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 77f9905...8ab56f3. Read the comment docs.


Respond to button interactions from the wrong user

When users click a button that they're not supposed to be touching, the bot discards the interaction, and discord displays that the interaction fails after a moment of processing. (Shown below)

Here is the code that causes this in the /lb command (though this issue exists everywhere where button interactions from other users are ignored)
https://github.com/LeaPhant/skyflower/blob/24981b4fb803e68d7a4507d02bb1c7b9bce88f0a/commands/leaderboards.js#L378
It's generally good practice to ensure that all interactions received by bots should be responded to, a simple fix to this would be having a generic response that can be expanded if you wish as you see fit.
Here's a potential fix as just a change to the filter function:
const filter = i => {
    if (i.user.id === interaction.user.id) return true;

    i.reply({
        content: "This button isn't meant for you! Please run the command yourself.",
        ephemeral: true
    });

    return false;
}

Honestly, I don't remember if a system like this would work as part of the filter function without testing it myself, but I'm sure it's clear how to go from here and prevent the interaction failed message.

This is intentional design cause I don't like the solution of taking up space in chat. Still waiting for a more elegant way to handle unauthorized button presses (or a way to show buttons as disabled to unauthorized users).

That solution doesn't take space in chat, it sends an ephemeral message that is only visible to the recipient and can easily be dismissed. Your current method of leaving it as shows up as an error in the eyes of the user which confuses people and is bad UX.
I agree that there should be a more elegant way, but this is the current method we have and it's easily dismissed.
Ex:


ror deploy

Kurulum hazırlık : https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-14-04
Ror kurulumu : https://www.digitalocean.com/community/tutorials/how-to-install-ruby-on-rails-with-rbenv-on-ubuntu-14-04

deploy : https://www.digitalocean.com/community/tutorials/how-to-deploy-a-rails-app-with-unicorn-and-nginx-on-ubuntu-14-04


Where can I get "{prox_scene}_segment.json"?

When I tried to run python synthesize/get_scene.py, I noticed that the file 'data/proxs/scenes_semantics/BasementSittingBooth_segment.json' was missing.
According to the COINS page, the scene_semantics files seems to be available from the PROX-E project, but in the downloadable files, I could not find the {prox_scene_name}_segment.json file in the scene_semantics folder.
Would you please tell me how to obtain these json files?

*_segment.json files are provided in COINS, you can check here: https://drive.google.com/drive/u/1/folders/1-vwuZTAhpN3ifzYJLEnKwn5yxHlq_sHr

Thanks!


Update spotless and eclipse dependencies

Description
Update spotless and eclipse dependencies to fix CVEs
Check List

[x] New functionality includes testing.

[x] All tests pass

[x] New functionality has been documented.

[x] New functionality has javadoc added

[x] Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

I'll raise manual backport


Continue the pipeline if publishing autorest coverage fails

Fixing incorrect change from previous commit.

Replaced by #873.


Add screenshots to show the message popping up

Screenshots for WordPress.org

Done for release 1.1.1
This is not reflected in this repo, since it only works in the wordpress.org SVN repo off the plugin root directory. That really needs to be change.


Refactor dashboard test to reduce flakes

Fix an issue where tcp port is wrong. It still worked because its just
tcp metrics but we should probably send proper traffic
Change from "send a bunch of traffic, then check metrics" to "send a
bunch of traffic in the background, while we continuously check for
metrics". This is both faster (we don't have a minimum test time of 20s)
but also less flaky

Testgrid showing recent failures: https://testgrid.k8s.io/istio_istio#integ-telemetry-k8s-tests_istio&width=20&exclude-non-failed-tests=

@howardjohn: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
e0aa6244be58ca7e376e10dc6729db61c1736547
link
/test integ-telemetry-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
e0aa6244be58ca7e376e10dc6729db61c1736547
link
/test integ-telemetry-k8s-tests_istio

integ-telemetry-mc-k8s-tests_istio
e0aa6244be58ca7e376e10dc6729db61c1736547
link
/test integ-telemetry-mc-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
e0aa6244be58ca7e376e10dc6729db61c1736547
link
/test integ-telemetry-k8s-tests_istio

integ-telemetry-mc-k8s-tests_istio
e0aa6244be58ca7e376e10dc6729db61c1736547
link
/test integ-telemetry-mc-k8s-tests_istio

lint_istio
3c7fb4a1db855db068e0c708579c7498b80f05bc
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-mc-k8s-tests_istio
e0aa6244be58ca7e376e10dc6729db61c1736547
link
/test integ-telemetry-mc-k8s-tests_istio

lint_istio
3c7fb4a1db855db068e0c708579c7498b80f05bc
link
/test lint_istio

integ-telemetry-k8s-tests_istio
3c7fb4a1db855db068e0c708579c7498b80f05bc
link
/test integ-telemetry-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
3c7fb4a1db855db068e0c708579c7498b80f05bc
link
/test lint_istio

integ-telemetry-k8s-tests_istio
3c7fb4a1db855db068e0c708579c7498b80f05bc
link
/test integ-telemetry-k8s-tests_istio

integ-telemetry-mc-k8s-tests_istio
3c7fb4a1db855db068e0c708579c7498b80f05bc
link
/test integ-telemetry-mc-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-mc-k8s-tests_istio
3c7fb4a1db855db068e0c708579c7498b80f05bc
link
/test integ-telemetry-mc-k8s-tests_istio

integ-telemetry-k8s-tests_istio
99aab54eb4d649e5fce0a1f1a5007e938f501751
link
/test integ-telemetry-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
99aab54eb4d649e5fce0a1f1a5007e938f501751
link
/test integ-telemetry-k8s-tests_istio

integ-telemetry-mc-k8s-tests_istio
99aab54eb4d649e5fce0a1f1a5007e938f501751
link
/test integ-telemetry-mc-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
99aab54eb4d649e5fce0a1f1a5007e938f501751
link
/test integ-telemetry-k8s-tests_istio

integ-telemetry-mc-k8s-tests_istio
99aab54eb4d649e5fce0a1f1a5007e938f501751
link
/test integ-telemetry-mc-k8s-tests_istio

lint_istio
89af34bfb2b912fa1e9cdfc19c790282c7358c2e
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-mc-k8s-tests_istio
99aab54eb4d649e5fce0a1f1a5007e938f501751
link
/test integ-telemetry-mc-k8s-tests_istio

lint_istio
89af34bfb2b912fa1e9cdfc19c790282c7358c2e
link
/test lint_istio

integ-telemetry-k8s-tests_istio
89af34bfb2b912fa1e9cdfc19c790282c7358c2e
link
/test integ-telemetry-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
89af34bfb2b912fa1e9cdfc19c790282c7358c2e
link
/test lint_istio

integ-telemetry-k8s-tests_istio
89af34bfb2b912fa1e9cdfc19c790282c7358c2e
link
/test integ-telemetry-k8s-tests_istio

integ-telemetry-mc-k8s-tests_istio
89af34bfb2b912fa1e9cdfc19c790282c7358c2e
link
/test integ-telemetry-mc-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
89af34bfb2b912fa1e9cdfc19c790282c7358c2e
link
/test lint_istio

integ-telemetry-k8s-tests_istio
89af34bfb2b912fa1e9cdfc19c790282c7358c2e
link
/test integ-telemetry-k8s-tests_istio

integ-telemetry-mc-k8s-tests_istio
89af34bfb2b912fa1e9cdfc19c790282c7358c2e
link
/test integ-telemetry-mc-k8s-tests_istio

unit-tests_istio
c530b6198c1d69fb19b59b3d25678a2fc1fcb668
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
89af34bfb2b912fa1e9cdfc19c790282c7358c2e
link
/test integ-telemetry-k8s-tests_istio

integ-telemetry-mc-k8s-tests_istio
89af34bfb2b912fa1e9cdfc19c790282c7358c2e
link
/test integ-telemetry-mc-k8s-tests_istio

unit-tests_istio
c530b6198c1d69fb19b59b3d25678a2fc1fcb668
link
/test unit-tests_istio

lint_istio
c530b6198c1d69fb19b59b3d25678a2fc1fcb668
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio
f986b41dd63dd6fa0843cf216bc1da5f1f456ce6
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

@howardjohn: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio
f986b41dd63dd6fa0843cf216bc1da5f1f456ce6
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest


add resharper team-shared setting for upper case constants

This is a first version of team-shared settings. It includes style settings for constants (upper case with '_').

Current coverage is 52.33%

Merging #31 into master will not affect coverage as of 32d2c19

@@            master     #31   diff @@
======================================
  Files           32      32
  Stmts          621     621
  Branches        28      28
  Methods
======================================
  Hit            325     325
  Partial          6       6
  Missed         290     290

Review entire Coverage Diff as of 32d2c19

Powered by Codecov. Updated on successful CI builds.


[Bug] Pages group in Entity Explorer does not collapse children

Description
Entity groups like Pages, Widgets, APIs and DB Queries need to have consistent behaviour. Pages group today cannot be toggled by clicking on the entity unlike Widgets, APIs and DB Queries.
Steps to reproduce the behaviour:

This is a sub-task for #2014


[DOCFIX] Add some key metrics to the tuning docs

What changes are proposed in this pull request?
Add descriptions of key metrics relevant to tuning
Why are the changes needed?
Make it more clear and easy to find critical metrics
Does this PR introduce any user facing changes?
Doc  changes

@apc999 PTAL again thanks.

alluxio-bot, merge this please


Move to nulldriver org

I'm moving the maven-resource out of my personal github account to the nulldriver github organization. This will help better support contributors :-)

[ ] Transfer github.com/patrickcrocker/maven-resource to github.com/nulldriver/maven-resource
[ ] Sync hub.docker.com/r/pivotalpa/maven-resource to hub.docker.com/r/nulldriver/maven-resource
[ ] Move wings.pivotal.io/teams/sme-pcf-concourse/pipelines/maven-resource to ci.nulldriver.com/teams/resources/pipelines/maven-resource
[ ] Remove fork github.com/Pivotal-Field-Engineering/maven-resource
[ ] Update concourse-ci.org with updated repo location

published v1.3.6 to nulldriver resources!


feat(runtime-dom): adjust transition class process

Close #2531, Close #2593
Make sure that *-enter/leave-from is the beginning of transition.

May i ask is there any information of reviewing time? @LinusBorg @posva
I need you help to verify if I've handled both issues correctly.

What I meant is that -active should be set on the first frame, as it is right now. The change you want to do is a breaking change and won't happen

What I meant is that -active should be set on the first frame, as it is right now. The change you want to do is a breaking change and won't happen

Well, i am not understand why its breaking change, there has no change about external APIs.
I think about it another way just now:

Add a workaround class at the same time When add -active on the first frame.
Remove the workaround class on the beginning of the next frame.

Is that what we need ?

Maybe this is clearer: https://v3.vuejs.org/guide/transitions-enterleave.html#transition-classes
Changing how the classes are added is a breaking change

Thanks for your replying. According to the document, its really a breaking change.
I have aslo a question: If i use a workround class to fix it and its also have to change tests(because a new class has been added on the first frame), is that a breaking change ?

How about this way @posva

Note: I removed the transition disabling for enter/appear transitions because it breaks v-show's current behavior. Whether we should or how to support visibility: hidden in transitions should be a separate discussion/PR.