[Flow Visibility] Add Grafana and Clickhouse deployment file

This PR is for adding deployment file and custom plugin for Grafana configuration and setup for Clickhouse, which are the UI solution and database solution for flow visibility long term architecture.
Deployment file:
It adds deployment for Clickhouse and Grafana with provisioning datasource and dashboards configuration.
Grafana sankey plugin:
A panel plugin is initialized from Grafana plugin template with Sankey diagram using react-google-chart.

Codecov Report

Merging #3063 (e9e771f) into main (363f2d8) will decrease coverage by 8.90%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #3063      +/-   ##
==========================================
- Coverage   60.71%   51.81%   -8.91%
==========================================
  Files         292      292
  Lines       24708    24724      +16
==========================================
- Hits        15002    12811    -2191
- Misses       8058    10451    +2393
+ Partials     1648     1462     -186

Flag
Coverage Δ

kind-e2e-tests
31.09% <ø> (-16.76%)
:arrow_down:

unit-tests
40.15% <ø> (+0.02%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/controller/types/networkpolicy.go
0.00% <0.00%> (-100.00%)
:arrow_down:

...g/agent/apiserver/handlers/featuregates/handler.go
0.00% <0.00%> (-82.36%)
:arrow_down:

pkg/agent/controller/networkpolicy/reject.go
0.00% <0.00%> (-68.30%)
:arrow_down:

pkg/apis/controlplane/v1beta2/helper.go
33.33% <0.00%> (-66.67%)
:arrow_down:

...formers/externalversions/security/v1alpha1/tier.go
0.00% <0.00%> (-64.29%)
:arrow_down:

...ers/externalversions/core/v1alpha2/clustergroup.go
0.00% <0.00%> (-64.29%)
:arrow_down:

...s/externalversions/core/v1alpha2/externalentity.go
0.00% <0.00%> (-64.29%)
:arrow_down:

...xternalversions/security/v1alpha1/networkpolicy.go
0.00% <0.00%> (-64.29%)
:arrow_down:

...versions/security/v1alpha1/clusternetworkpolicy.go
0.00% <0.00%> (-64.29%)
:arrow_down:

pkg/controller/networkpolicy/mutate.go
0.00% <0.00%> (-62.07%)
:arrow_down:

... and 98 more

Hi @antoninbas and @tnqn , this PR is still in progress, but I have a quick question about how/where to serve Grafana custom plugin (currently in build/yamls/flow-visibility/grafana-custom-plugin).
Grafana custom plugin is for creating some custom panels that are not supported by Grafana yet (in our case, sankey diagram for visualizing throughput over a time range). It is generated by grafana toolkit https://grafana.com/tutorials/build-a-panel-plugin/#create-a-new-plugin, so there are several components like license, default settings and configuration for React etc. To serve and use it, we need to sign the plugin, package it and upload with release (sample workflow: https://github.com/zyiou/grafana-custom-plugin/blob/main/.github/workflows/build_plugin.yml)
Since the programming language and layout are quite different from Antrea, I’m considering whether we can serve this in a separate repository. What do you suggest?

@zyiou could you put the plugin code under plugins/ maybe?

A quick question @zyiou . Do we also need to include the README file in order to deploy the clickhouse operator?

A quick question @zyiou . Do we also need to include the README file in order to deploy the clickhouse operator?

I'm planning to add all documentation once we have e2e work.

Several major updates in the latest 3 commits:

Added a start-flow-visibility.sh script, with Grafana dashboards and Clickhouse data schema mounted as configmaps. User should be able to start the flow-visibility framework by running the script.
Changed file directory under the  build/yamls/flow-visibility folder, according to Grafana-provision-doc
Added documentation for Clickhouse-Grafana Flow-visibility solution introduction, deployments, dashboards display; Clickhouse configurations in the flow-aggregator configmap is expected to be added in #3196

Remaining issue:
Hosting sankey-plugin on antrea.io S3 sometimes causes Grafana failed to register the plugin, with a 50% probability to happen. The log shows Grafana has successfully downloaded the plugin from the S3 url, but not registered. Still investigating.

The kustomize patches - dev/release will be added in PR #3244 , as a clickhouse-monitor image is being built in that PR.

A general comment - do you have a better name than "ClickHouse-Grafana Flow-visibility Solution" which looks too long.
Let us see if @antoninbas @salv-orlando have an idea.

Thanks, Jianjun. I also feel it's a bit too long. An alternative I can think of is "Flow-visibility", which I think is too general, as our entire doc is named by network-flow-visibility.  Would also like hear from Antonin and Salvatore.

Hi @antoninbas  @jianjuns  @salv-orlando , I have a question about the contribution of this PR.
When we do squash-and-merge,  all co-authors are associated with the commit, but only one person (the first committer) is credited in the Github contributors view.  In that case, we'll keep three separate commits for this PR in the repo history(which is rare and not considered to be clean), but each of the authors can get their credits correctly.

@heanlan : are the commits self complete - that mean each commit will not result in incomplete status of the branch, like compilation or test errors, or functionality break?
Another way is to do "create a merge commit", then the commits are atomic, but history&credit will be kept.

Thanks @jianjuns

are the commits self complete - that mean each commit will not result in incomplete status of the branch, like compilation or test errors, or functionality break?

We can make them don't introduce errors, given currently there are no tests to test the feature's functionality. Also, this feature is still not complete without #3196 . There could be small errors on new test like check copy-right, where the script without copyright header is added in an early commit, but fix is added in the latest commit. We can figure out a way to amend the commits 1) avoid the small errors 2) remove the dependency between some of the commits 3) finally be able to switch the orderings and squash each authors' commits into a single one

Another way is to do "create a merge commit", then the commits are atomic, but history&credit will be kept.

If "create a merge commit" is acceptable,  that would be great. We'll also try to squash some of the commits. I just don't know which way is better from the maintainers' view. Are we enforcing a linear commit history right now?

Could you try merge/update the commits, so each commit will not introduce any error, and ideally not too many small commits?
"create a merge commit" is not ideal as it creates a merge commit which is meaningless.

Could you try merge/update the commits, so each commit will not introduce any error, and ideally not too many small commits?

No problem. I'm pushing the updated commits one by one and run the tests.

@jianjuns I feel like even with the merge option, the commits are still meant to be self-contained as they can be checked-out individually after they are merged. IMO, the rebase option is what we should go for here.

/test-all

/test-integration

/test-integration

A general comment - do you have a better name than "ClickHouse-Grafana Flow-visibility Solution" which looks too long.

Let us see if @antoninbas @salv-orlando have an idea.

Thanks, Jianjun. I also feel it's a bit too long. An alternative I can think of is "Flow-visibility", which I think is too general, as our entire doc is named by network-flow-visibility. Would also like hear from Antonin and Salvatore.

By using "Clickhouse-Grafana" we might indicate it as an alternative to "ELK", so in that way it makes sense. On the other hand, it's also very long.
I have 2 observations:

For consistency we might want to keep using "Flow Collector" terminology, at least for this release.
Instead of referring to specific products making up the stack I'd just call it "Enhanced Flow Collector"

I have another question: once we have the all the flow-visibility refactoring PRs merged, do we still want to keep the ELK related contents in the repo? shall we open another PR to remove all of them for v1.6 release? or postpone it to v1.7?

By using "Clickhouse-Grafana" we might indicate it as an alternative to "ELK", so in that way it makes sense. On the other hand, it's also very long.
I have 2 observations:

For consistency we might want to keep using "Flow Collector" terminology, at least for this release.
Instead of referring to specific products making up the stack I'd just call it "Enhanced Flow Collector"

Thanks, Salvatore. cc @jianjuns @antoninbas for opinions. Thanks!

I have another question: once we have the all the flow-visibility refactoring PRs merged, do we still want to keep the ELK related contents in the repo? shall we open another PR to remove all of them for v1.6 release? or postpone it to v1.7?

I think we could keep ELK related stuff in v1.6 release. In order to fully move to our new flow visibility arch, we also need some changes on Flow Aggregator side (deleting unused fields in new arch, debugging cqe test, etc), which may take a while before finished.

I feel it depends on how complete the new implementation is (which you guys know better). If it is not that complete, probably let us still keep ELK support for a while.
For naming, another way is to name it something you guys like, not necessary description of the functionalities.

Another alternative I can think of is "Grafana Flow Collector".

it's shorter
satisfies the "Flow Collector" terminology consistency
from the users perspective they will only directly interact with Grafana

I'll go-ahead and do the refactoring if that sounds good to all of us.

Hi @antoninbas @jianjuns , would you mind having a look at this PR again?

We may need to mention Grafana flow collector in this paragraph

Thanks @yanjunz97 , could you please also help to try out the instructions of deployment?

Overall LGTM. Would be good if someone else beyond the person working on this PR could validate the installation instructions by trying them out.

I have worked on the validation. I found an error in Grafana dashboard when trying to set a customized username and password for Clickhouse. After some discussions and troubleshooting with @heanlan , we found the root cause. Anlan has updated the files to fix it.
After this fix, I tried out all the installation instructions again and validated that they all work as expected.

/test-all

@heanlan can I go ahead and merge this PR?

Yes, please @antoninbas


B1046 status outdated

B1046 was destroyed during the In Flight Abort Test, but the API still shows it as active with details of "At the Cape awaiting IFA launch."

Looks like some columns were added to the wiki page the updater was parsing. This should be resolved in 5506df007342cc858365db1346cef995dde5e90f

Interesting, I was wondering how the API got its data. Thanks!


try to aling with google style docstring

Not a critical PR
This PR is a suggestion to start aligning docstrings using Google-style docstring format (I saw most of our code uses this one so it will be easier to change all the occurrences).
See https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html and https://google.github.io/styleguide/pyguide.html for more information.
Please let me know what you think in regards to the following:

Use of Raises instead of Raises the following errors
Use of the notation :obj: for the data types in Args, Return, and Raises (See https://stackoverflow.com/questions/41127200/in-python-docstrings-what-does-obj-do for more information).
Use of direct name of exception instead of full path e.g:obj:DatasetModuleNotInstalledError`` (I find this more readable)

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (fbb0b22) 90.94% compared to head (f9de2a2) 87.19%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2338      +/-   ##
==========================================
- Coverage   90.94%   87.19%   -3.76%
==========================================
  Files         264       94     -170
  Lines       15820     3561   -12259
==========================================
- Hits        14388     3105   -11283
+ Misses       1432      456     -976

Flag
Coverage Δ

jobs_cache_maintenance
94.40% <ø> (ø)

jobs_mongodb_migration
86.11% <ø> (ø)

libs_libapi
?

libs_libcommon
?

services_admin
?

services_api
86.43% <ø> (ø)

services_rows
82.80% <ø> (ø)

services_search
80.79% <ø> (ø)

services_sse-api
94.19% <ø> (ø)

services_worker
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

good idea to standardize the docstrings

The online documentation doesn't support google docstring.
We use our own formatting for docstring, see (internal) https://www.notion.so/huggingface2/Documentation-926791574d274a5194cc83699c4a238e

Will close this PR and open another one following HF doc guidelines https://www.notion.so/huggingface2/Documentation-926791574d274a5194cc83699c4a238e


more styling cleanup

I've got PR #65 with a lot of reorganization but there are a couple more things I didn't do so we can have more discussion.

I think we should have a separate file for theming variables (including the stuff from #51 but don't call it common.scss — maybe theme.scss would be better). It should JUST be variables but cover font and colour choices.

I think common.scss should still exist but be for small reusable things like the definition of .click or li.hanging

the widget styling could move to the Widget component (not 100% sure)

what's left in App.vue should mostly be skeleton stuff. Not sure if it could be further divided but I think we might want to move it all into .scss files anyway and keep App.vue lean.

I agree with this approach. In my opinion not much discussion needed. 😄

I'm not 100% sure about 3, though. Are you a +1 on that or just +0 :-)
Need to work out whether 4 involves just one file (and if so what it should be called) or multiple files.

For 4, if it looks like what's left is more than just skeleton code I might split it into skeleton.scss and app.scss or something otherwise just use skeleton.scss


listchaintxns has invalid item (zero ammout, destinations...)

Background
This is my latest entry in lncli listchaintxns output:
        {
            "tx_hash": "4a5daa5d86879904e9efe852937513df76362b133162ca1284d6e24d37ca6942",
            "amount": "0",
            "num_confirmations": 0,
            "block_hash": "",
            "block_height": 0,
            "time_stamp": "1543390517",
            "total_fees": "0",
            "dest_addresses": [
            ]
        }

Zero amount, confirmations, destination addresses...
Meanwhile, my local bitcoin-cli -datadir=/home/bitcoin/.bitcoin/ getrawtransaction 4a5daa5d86879904e9efe852937513df76362b133162ca1284d6e24d37ca6942 1 states:
...
  "hex": "02000000000101c609c446e7b5652e08f87f652c1c78680fc7f259a3862a192973453e69d41ba80100000000ffffffff01c76203000000000016001490a8cfe2049807f74b2dfc3b16e6fd1dd91816b60400473044022009044a5990ba1301ec213e4d90d0e13967df9df771022714cd62990d8f401f0f022005518c9a698b9e737b75260336cfa32004f4580b7403d8416b059813fdc644c30147304402204a679c7b91df277377158cb33155656e0d7fbea70b1b9fa535e16d18f5105ddd02206fae1e31e9120971482a2bfb667c810d1e6f487492c8c9ded3d79384b19423590147522102217144a276369ea0767b690a73c755efcd97c132dadc512adaf3d488394ae4e82103e9df0c8bd28d59d9158d9d33065c38a0a9f5f5717deb36f0e742f1fb9418443752ae00000000",
  "blockhash": "000000000000000000167474485fa5509851c0fe423347463fce5fd9b199c747",
  "confirmations": 61,
  "time":<PHONE_NUMBER>,
  "blocktime":<PHONE_NUMBER>

LND logs:
# fgrep 4a5daa5d86879904e9efe852937513df76362b133162ca1284d6e24d37ca6942 /home/lnd/.lnd/logs/bitcoin/mainnet/lnd.log
2018-11-28 09:35:17.593 [INF] LNWL: Inserting unconfirmed transaction 4a5daa5d86879904e9efe852937513df76362b133162ca1284d6e24d37ca6942
2018-11-28 09:35:18.028 [INF] PEER: Waiting for confirmation of cooperative close of ChannelPoint(a81bd4693e457329192a86a359f2c70f68781c2c657ff8082e65b5e746c409c6:1) with txid: 4a5daa5d86879904e9efe852937513df76362b133162ca1284d6e24d37ca6942
2018-11-28 09:35:18.071 [INF] NTFN: New confirmation subscription: txid=4a5daa5d86879904e9efe852937513df76362b133162ca1284d6e24d37ca6942, numconfs=1
2018-11-28 09:49:23.698 [INF] NTFN: Dispatching 1 conf notification for 4a5daa5d86879904e9efe852937513df76362b133162ca1284d6e24d37ca6942

Your environment

version of lnd 0.5.0-beta commit=v0.5-beta-295-gb6009850635cf966eb3bd637de5da89cb9262589
Linux odroid-hc1 4.9.0-8-armmp-lpae #1 SMP Debian 4.9.130-2 (2018-10-27) armv7l GNU/Linux
/Satoshi:0.17.0/ (mainnet)

Steps to reproduce
Unknown
Expected behaviour
Item in listchaintxns should contain non-zero amounts, confirmations, etc.

It isn't invalid per say, it's just that we use the wallet to store transactions that don't actually belong to it. The wallet is meant to track transactions that send to it via a change address, or directly. In certain circumstances, we'll give it a transaction where no outputs at all belong to it. In this case it tries to parse out the fees it paid and where it sent funds to, but these are all script hash addresses. In your case, a co-op close happened, where we didn't have an output address.

in your case, a co-op close happened

Oh, so basically all funds was on remote side (from my point of view), so no coins had to go to my wallet? Thanks @Roasbeef .


Picasso eMMC is now supported in edk2

Thank you @coolstar!

We should wait until all roms are updated
https://github.com/MrChromebox/scripts/blob/master/sources.sh#L304-L313

No problem, I thought it was a bit premature too but then I saw CoolStar had already updated her site so I wasn't sure.

Yeah wait until 2402 full rom releases, but otherwise looks good to me.

Seems like the roms are now updated
https://github.com/MrChromebox/scripts/commit/7d279515739da87b95f2b7d7c457f89d403537ea


Ability to rename interfaces under types/devices

Rather than delete, recreate and re-associate connections it would be nice to be able to rename interfaces (e.g fa-0/1 <---> fa0/1 <---> FastEthernet0/1)
I already know there will be much discussion internally about the format and having basically loaded all my data up going back through it again for renaming would be tedius

Interfaces can be edited and renamed individually from the device view, at /dcim/interfaces/{pk}/edit/. They can also be renamed in bulk using the yet-to-be-documented shell. (For device types, it's easiest to simply delete and recreate interface templates.)

Bulk editing of interfaces is probably best done through the shell. Going to close this issue and hopefully publish some documentation around the shell soon.
In practice, this should be a very rare occurrence, since interfaces should be named however they are referred to on the device. For example, Cisco IOS would call an interface "FastEthernet0/1"; although "fa0/1" can be used as a shortcut on the CLI, the interface is never referred to in this manner.


Verify transactions in a block are well formed (release-2.5)

Verify that transactions in blocks appear exactly as their marshaled form after unmarshaling.
Backport of #4490

@Mergifyio rebase


Disable plotting for some types of simulations

If a simulation is an FBA for example, line plots do not make sense.
There may be some usecases that I am not concidering, so we can discuss if this might be more user friendly.

We could disable 2-d plots for 1-dimensional results.
In addition, we could provide a different type of plot that make sense for 1-dimensional data such as a bar chart or kernel density plot.
Another limitation is the lack of 3D plots (#1718). We don't yet have a 3d simulator to motivate this.
I've also been thinking of about changing the plot menu to these options:

Select the plots from a SED-ML file
Select a Vega file from the COMBINE archive
Design a 2d plot
Upload a Vega plot

(2) is partly motivated by (a) the need to store visualizations for our simulation database and (b) the desire to make Vega visualizations easier to demonstrate.

I added #2601 #2602 #2603 to address this issue.
@bilalshaikh42, close this if you feel these other issues will address your point.


Blur UI on HI DPI 4k Monitors

Describe the bug
The calculator and UI example runs well without any issues but elements do not render smooth. The display is scaled 2x on macOS to appear smooth and sizeable UI
To Reproduce
Steps to reproduce the behavior:

checkout the repository
run the command cargo run --example calculator

Expected behavior
A calculator UI should appear. The UI should render smooth instead of blur
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: macOS Big Sur
Version 11.6

Additional context
I am using LG HDR 4K Monitor 27", Resolution is scaled to 2x in Mac.

duplicate of https://github.com/redox-os/orbtk/issues/71 and https://github.com/redox-os/orbtk/issues/416
see my answer on #416


Fix the failed UTs in ArrowEvalPythonExecSuite when enable ArrowColumnarToRow

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
Additional context
Add any other context about the problem here.

closed in #533


break out dockutil into a separate package

merge its code with the code in github.com/arschles/gbs/handlers

closing as I don't think this applies anymore


disable eslint no-explicit-any

It will still warn on implicit use of any. This is causing a lot of our eslint warnings, and 95%+ of them are intentional, which just makes us ignore eslint warnings.
Before this, there were 153 no-explicit-any warnings. That represented more than 1/3 of our total eslint warning count.
Test plan
CI

related: https://github.com/sourcegraph/cody/pull/2702


TypeError: 'float' object cannot be interpreted as an integer

Look into Issues (https://github.com/jocicmarko/ultrasound-nerve-segmentation/issues/79)

total = len(images) / 2  --------->total = int(len(images) / 2)


Use the charset from the content-type header of the HttpRequest instead of always using the default charset.

Second attempt. The Content-type header from the HttpRequest is now used instead of the Content-type of HttpEntity (which apparently is always null).
All tests pass now.

Coverage decreased (-0.002%) to 77.285% when pulling 9ffbeb892b13c54b95212af89e1d107cfbbc8ba5 on gijsbert802:use-charset-from-http-header-2 into f21ab8bfe66902651265a3b577f5fd73664e6040 on jamesagnew:master.

Looks great! Merging now, thanks. :)


Failed to resolve: com.tuya.smart:iot-app-sdk:1.0.2

i can't sync gradle, it can't resolve the dependency,

build.gradle project level

// Top-level build file where you can add configuration options common to all sub-projects/modules.
buildscript {
    repositories {
        google()
        mavenCentral()
        maven {
            url "https://maven-other.tuya.com/repository/maven-releases/"
        }
    }
    dependencies {
        classpath "com.android.tools.build:gradle:7.0.4"
        classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:1.6.10"
    }
}

task clean(type: Delete) {
    delete rootProject.buildDir
}

build.gradle app level

plugins {
    id 'com.android.application'
    id 'kotlin-android'
}

android {
    compileSdk 31

    defaultConfig {
        applicationId "com.example.app"
        minSdk 21
        targetSdk 31
        versionCode 1
        versionName "1.0"

        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
    }

    signingConfigs {
        release {
            storeFile file("~/general.keystore")
            storePassword "general"
            keyAlias "general"
            keyPassword "general"
            v1SigningEnabled true
            v2SigningEnabled true
        }

        debug {
            storeFile file("~/general.keystore")
            storePassword "general"
            keyAlias "general"
            keyPassword "general"
            v1SigningEnabled true
            v2SigningEnabled true
        }
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
        }
    }
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
    kotlinOptions {
        jvmTarget = '1.8'
    }
}

dependencies {

    implementation 'androidx.core:core-ktx:1.7.0'
    implementation 'androidx.appcompat:appcompat:1.4.0'
    implementation 'com.google.android.material:material:1.4.0'
    implementation 'androidx.constraintlayout:constraintlayout:2.1.2'
    implementation 'com.tuya.smart:iot-app-sdk:1.0.2' // TODO ini
    testImplementation 'junit:junit:4.13.2'
    androidTestImplementation 'androidx.test.ext:junit:1.1.3'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0'
}

when i sync gradle i got message
Failed to resolve: com.tuya.smart:iot-app-sdk:1.0.2

thank you

@nashihu Thank you for your question, we confirmed the product has been released to the public, please check your network, try to change the network.

hello,, i have tried using one of official ISP in Indonesia, and also using VPN by CloudFlare, but it doesn't work either.. do you have any suggestion , thank you..

@nashihu You need to add closures  "allprojects" like this in your build.gradle project ，not only in the "buildscript":
allprojects {
repositories {
maven { url 'https://maven-other.tuya.com/repository/maven-releases/' }
google()
jcenter()
}
}

i'm using latest Android Studio. the new project template got me wrong
when i add allprojects { ... } inside build.gradle project level, i got error message

Caused by: org.gradle.api.InvalidUserCodeException: Build was configured to prefer settings repositories over project repositories but repository 'maven' was added by build file 'build.gradle'

after following this , it works now
thanks for your response


JIRA 491. onclick handler on paper to give focus to editor

When focus is not in editor, clicking any line in editor down to bottom of paper should give focus to editor

Looks good on my end. Straightforward code fix.


MS Graph for Office 365: Sanity check - Adding new action

Please ensure your pull request (PR) adheres to the following guidelines:

Please refer to our contributing documentation for any questions on submitting a pull request, link: Contribution Guide

Pull Request Checklist
Please check if your PR fulfills the following requirements:

[x] Testing of all the changes has been performed (for bug fixes / features)
[x] The readme.html has been reviewed and added / updated if needed (for bug fixes / features)
[x] Use the following format for the PR description: <App Name>: <PR Type> - <PR Description>
[ ] Provide release notes as part of the PR submission which describe high level points about the changes for the upcoming GA release.
[x] Verify all checks are passing.
[x] Do NOT use the next branch of the forked repo. Create separate feature branch for raising the PR.
[x] Do NOT submit updates to dependencies unless it fixes an issue.

Pull Request Type
Please check the type of change your PR introduces:

[ ] New App
[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation
[ ] Other (please describe):

Security Considerations (REQUIRED)

[ ] If you are exposing any endpoints using a REST handler,
please document them in the readme.html.
[ ] If this is a new connector or you are adding new actions

[ ] Please document in the readme.html all methods (eg, OAuth) used to authenticate
with the service that the connector is integrating with.
[ ] If any actions are unable to run on SOAR Cloud, please document this in the readme.html.

[ ] Are you introducing any new cryptography modules? If yes, please elaborate their purpose:
[ ] Are you are accessing the file system? If yes, please verify that you are only accessing paths returned through
the Vault API.
[ ] Are you are marking code to be ignored by Semgrep with nosemgrep?
If yes, please provide justification in an additional comment next to the ignored code.

Release Notes
Adding new action list group members by e-mail address for client wishes

Hi @aparekh-crest, yes I was thinking about it but the client wanted it to be separate. Yes we can do it as one action but then you can't set both values as required so in my experience this is starting to get tricky from the customer's point of view and could get us more work in the future. Then it require additional code block for check if user entered email address or group id, maybe someone will enter both, there will be situation where someone will not enter any of them in case of some problem in playbook etc. After that, this action will look like a mess. That for what I am worry about

@pdros-splunk, I agree with your concerns, if we will add two optional parameters, there have to be some validations we need to add in one action, In accordance with the customer's needs, we can take two separate actions.

100% agree with you.

Hi @pdros-splunk , split one action to two actions with different parameters is not a good way in coding. There maybe other apps with similar situation. If we still use one action, but in required parameter, we use dropdown to select either 'group_id' or 'email_address' to list group member, so it'll not be confused. What do you think of this option @pdros-splunk ? Any concern?

Hi @mpan-splunk , you are right it is not good way in coding, I like your idea with drop down list to choose method, I didn't come up with it. It will dispel the user's doubts. In free time I will update this and merge in one action.

Thank you @pdros-splunk ! In fact, it's my manager @junlinl-splunk 's idea. :)

@aparekh-crest and @mpan-splunk Done 😎


Hide Folders Preset No Longer Working

After updating to Visual Studio Code 1.10.1 and vscode-icons 7.3.0, neither User Settings nor Workspace Settings set to hide folders seems to work:
// Place your settings in this file to overwrite the default settings
{
    "css.validate": false,

    "eslint.options": {"configFile": "F:\\IDEs\\Visual Studio Code\\ESLint\\.eslintrc.json"},

    "stylelint.config": {"extends": ["F:\\IDEs\\Visual Studio Code\\StyleLint\\.stylelintrc.json"]},

    "workbench.activityBar.visible": true,
    "workbench.statusBar.visible": true,
    "window.zoomLevel": 0,
    "workbench.iconTheme": "vscode-icons",

    "vsicons.dontShowNewVersionMessage": true,
    "vsicons.projectDetection.disableDetect": true,
    "vsicons.presets.hideFolders": true
}

F1 > Apply Icons Customization.

@JimiC I had this issue after a recent update (my current version is 10.1.1). Running "Apply Icons Customization" worked. Just letting you know.


Eliminate Credentials In Code

Eliminate Credentials In Code
Digital Artifacts

Eliminate Credentials In Code Evicts Credential

Definition

Remove any “credentials” or “access keys” from compiled source code.

How it works

Credentials, or secrets in compiled code, can lead to compromise of target services. Credentials in code must be detected and eliminated promptly.
Credentials apart from being eliminated must also be disabled once it's made its way into git/ version control history.
Credentials are always to be accessed via a secret manager, and not to be held in persistent memory in an un-encrypted form

Considerations

While configuring a credential manager, it is important to handle role accesses, and credential keys correctly to ensure unauthorized entities are not able to access stored credentials

Contributed By:
Aashiq Ramachandran, Cyware Labs
MITRE D3FEND Tactic:
Harden

@hack-sentinel
Credentials might be in software, need to consider modeling that.


⚠ Bofrost.de has degraded performance

In e3a5297, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1194 ms

Resolved: Bofrost.de performance has improved in a3e1a44.


chromium: fix screencast

Add patches './patches/chromium-webrtc-fix.patch' and
'./patches/fix-wayland-screencast.patch' fixing bugs in WebRTP
affecting screen sharing. These fixes were integrated into chromium
105 See https://webrtc-review.googlesource.com/c/src/+/267002 and
https://webrtc-review.googlesource.com/c/src/+/267002.
Description of changes
Add the two patches './patches/chromium-webrtc-fix.patch' and './patches/fix-wayland-screencast.patch' .
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

This commit is intended to be back-ported to 'nixos-22.05' so as to fix screencast in 'chromium'. Systems using some resolutions also need #186435.
With these patches I can finally sharing my screen in, e.g., teams calls.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/1097

Also please do not immediately post on discourse. Thanks

Not needed anymore.


Added accuracy and consistency test to kernel explainer.

This commit adds testing of KernelExplainer against the properties on the paper (local accuracy and consistency/monotonicity).
Note that this currently does not pass, as the KernelExplainer is not guaranteeing consistency as outlined in the paper. Let's figure out why and act accordingly.
I have validated that the consistency test is correct by writing the brute force version of shapley values (without local approximations), and validating the test against that explainer (with multiple training data and non-linear functions).

The kernel explainer (given enough samples) is consistent, there is just a typo in here at the moment. Once that's fixed I would love to merge the new test :)

Great to know, because I have been assuming that the KernelExplainer is consistent and that I made a mistake on the test. :P

I sent an updated file your way, let me know if you need to update this PR

It looks like this is still failing. I thought I remember we had this sorted out though over email. What's the status here? (and thanks again!)


NoSuchMethodError: java/nio/ByteBuffer.flip()Ljava/nio/ByteBuffer

Hi,
Java 9 changed the interface to ByteBuffer.flip() from returning Buffer to ByteBuffer. In the FileHasherImpl.has method calls the flip method on a Java 8 it will throw a NoSuchMethodError exception.
I googled the error and found this: [https://github.com/plasma-umass/doppio/issues/497](at the doppio project)
My stacktrace:
Caused by: java.lang.NoSuchMethodError: java/nio/ByteBuffer.flip()Ljava/nio/ByteBuffer; (loaded from C:\Program Files\AdoptOpenJDK\jdk-<IP_ADDRESS>-openj9\jre\lib\rt.jar by <Bootstrap Loader>) called from class org.siouan.frontendgradleplugin.core.FileHasherImpl (loa ded from file:/C:/Users/HowellM/.gradle/caches/modules-2/files-2.1/org.siouan/frontend-gradle-plugin/1.2.0/46a6a486eabfd2643fb05e85f4aba508da14cfc1/frontend-gradle-plugin-1.2.0.jar by VisitableURLClassLoader(ClassLoaderScopeIdentifier.Id{root:C:\Users\HowellM\IdeaP rojects\dispatch-workflow\buildSrc:root-project(export)})).         at org.siouan.frontendgradleplugin.core.FileHasherImpl.hash(FileHasherImpl.java:38)         at org.siouan.frontendgradleplugin.core.NodeDistributionValidator.validate(NodeDistributionValidator.java:74)         at org.siouan.frontendgradleplugin.core.DistributionInstaller.install(DistributionInstaller.java:61)         at org.siouan.frontendgradleplugin.tasks.NodeInstallTask.execute(NodeInstallTask.java:83)
To Reproduce
I just followed the README here at the project in a very vanilla Spring Boot 2.1.5-RELEASE rest api application with a default generated vue cli project added to a sub folder and then I ran gradlew build and got the error.
Environment:

OS: Windows 10 Enterprise
JDK: AdoptOpenJDK 1.8.0_212-b03 / Eclipse OpenJ9 openj9-0.14.0
Gradle: 5.2.1
Frontend Gradle plugin: 1.2.0

Hi Michael,
Thank you for this feedback. I could reproduce the problem. The fix will be available quickly.
Regards,
Vincent


Fixed Health Bar color functionality

What's wrong with the heath bar color functionality?
You may have noticed that changing the player in debug does not change the player health bar color (i.e changing to say pico still has the blue color on the health bar). This does work with the opponents though, as it changes based on the opponent.
What I did
I created two variables colorOne and colorTwo as Ints, and where the health bar is drawn, I made it 1st check the player, and changes colorOne to what character is being played (being bf makes it blue, changing to say dad will turn it purple). Then, it checks the opponent and changes colorTwo to the right color (if senpai was the opponent, the other side would be peach, or being pico makes it green).

do not merge to stable-

resolve conflicts pls


feat: add group-priority plugin

This plugin allows you to configure pull request by priority. If enabled and if a pull request of a prioritized group is found, release-please will limit the proposed release pull requests to the prioritized group only.
For example, java snapshot PRs are now marked as part of the snapshot group. If you configure the group-priority plugin with the group set to ['snapshot'], then release-please will only open pull requests for snapshot pull requests if there are any.
Fixes #1657

I think we should add some documentation that describes the plugin.

Added a description/example to the markdown docs with the existing plugin documentation.
Fixed the description in the config schema.
Added jsdoc to the plugin constructor.


Struggles to find errors.

Issue
The way it shares errors with the AI looks like this:
Fix any errors below, if possible.
See relevant lines below marked with █.
src/components/Dashboard/MonthlyConfig/FinancialOverview.tsx:
1█import React, { useState, useEffect, useCallback, useMemo } from 'react';
2│import { MonthlyConfig, GlobalAgencyData, Expense, MonthlyExpense, GlobalCategory, HealthMetrics } from
'../../../types/types';
3│import { PieChart, Pie, Cell, ResponsiveContainer, Legend, Tooltip, LineChart, Line, XAxis, YAxis, CartesianGrid }
from 'recharts';
4│import { useAgencyCalculations } from '../../../hooks/useAgencyCalculations';
...⋮...
279│  const renderMetric = useCallback((label: string, value: number, previousValue?: number) => {
280│    const change = calculateChange(value, previousValue);
281│    return (
282█
283█        {label}:
284█
285█          {formatCurrency(value)}
286│          {showComparison && change !== null && previousValue !== undefined && (
287█            <div className={text-sm ${getColorClass(value - previousValue)}}>
288│              {change}
289│
290│          )}
291│
292│
293│    );
294│  }, [showComparison, formatCurrency, calculateChange, getColorClass]);

It's better to show it the errors in this format (same as REACT shows in terminal):
webpack compiled with 12 warnings
ERROR in src/components/Dashboard/MonthlyConfig/FinancialOverview.tsx:124:42
TS2304: Cannot find name 'previousFinancials'. <----- ==Note: This is especially helpful, which it doesn't see==
122 |   const handleOverrideToggle = useCallback((checked: boolean) => {
123 |     setIsWorkingCapitalOverridden(checked);

124 |     if (!checked && currentFinancials && previousFinancials) {
|                                          ^^^^^^^^^^^^^^^^^^
125 |       const calculatedValue = previousFinancials.workingCapital ?? 0;
126 |       setDisplayedWorkingCapital(calculatedValue);
127 |       onUpdate({

ERROR in src/components/Dashboard/MonthlyConfig/FinancialOverview.tsx:125:31
TS2304: Cannot find name 'previousFinancials'.
123 |     setIsWorkingCapitalOverridden(checked);
124 |     if (!checked && currentFinancials && previousFinancials) {

125 |       const calculatedValue = previousFinancials.workingCapital ?? 0;
|                               ^^^^^^^^^^^^^^^^^^
126 |       setDisplayedWorkingCapital(calculatedValue);
127 |       onUpdate({
128 |         startingWorkingCapital: calculatedValue,

ERROR in src/components/Dashboard/MonthlyConfig/FinancialOverview.tsx:137:26
TS2304: Cannot find name 'previousFinancials'.
135 |       });
136 |     }

137 |   }, [currentFinancials, previousFinancials, displayedWorkingCapital, onUpdate]);
|                          ^^^^^^^^^^^^^^^^^^
138 |
139 |   const handleWorkingCapitalChange = useCallback((e: React.ChangeEvent) => {
140 |     const newValue = Number(e.target.value);

ERROR in src/components/Dashboard/MonthlyConfig/FinancialOverview.tsx:391:91
TS2304: Cannot find name 'previousFinancials'.
389 |               Expenses
390 |

391 |                 {renderMetric('Team Costs', currentFinancials.teamCosts, showComparison ? previousFinancials?.teamCosts : undefined)}
|                                                                                ^^^^^^^^^^^^^^^^^^
392 |
393 |                   Other Expenses:
394 |

ERROR in src/components/Dashboard/MonthlyConfig/FinancialOverview.tsx:406:93
TS2304: Cannot find name 'previousFinancials'.
404 |
405 |

406 |                 {renderMetric('Total Costs', currentFinancials.totalCosts, showComparison ? previousFinancials?.totalCosts : undefined)}
|                                                                                ^^^^^^^^^^^^^^^^^^
407 |
408 |
409 |

ERROR in src/components/Dashboard/MonthlyConfig/FinancialOverview.tsx:438:95
TS2304: Cannot find name 'previousFinancials'.
436 |               Profit
437 |

438 |                 {renderMetric('Gross Profit', currentFinancials.grossProfit, showComparison ? previousFinancials?.grossProfit : undefined)}
|                                                                                ^^^^^^^^^^^^^^^^^^
439 |                 {renderMetric('Tax Amount', currentFinancials.tax, showComparison ? previousFinancials?.tax : undefined)}
440 |                 {renderMetric('Net Profit', currentFinancials.netProfit, showComparison ? previousFinancials?.netProfit : undefined)}
441 |

ERROR in src/components/Dashboard/MonthlyConfig/FinancialOverview.tsx:439:85
TS2304: Cannot find name 'previousFinancials'.
437 |
438 |                 {renderMetric('Gross Profit', currentFinancials.grossProfit, showComparison ? previousFinancials?.grossProfit : undefined)}

439 |                 {renderMetric('Tax Amount', currentFinancials.tax, showComparison ? previousFinancials?.tax : undefined)}
|                                                                                ^^^^^^^^^^^^^^^^^^
440 |                 {renderMetric('Net Profit', currentFinancials.netProfit, showComparison ? previousFinancials?.netProfit : undefined)}
441 |
442 |                   Profit Margin:

ERROR in src/components/Dashboard/MonthlyConfig/FinancialOverview.tsx:440:91
TS2304: Cannot find name 'previousFinancials'.
438 |                 {renderMetric('Gross Profit', currentFinancials.grossProfit, showComparison ? previousFinancials?.grossProfit : undefined)}
439 |                 {renderMetric('Tax Amount', currentFinancials.tax, showComparison ? previousFinancials?.tax : undefined)}

440 |                 {renderMetric('Net Profit', currentFinancials.netProfit, showComparison ? previousFinancials?.netProfit : undefined)}
|                                                                                ^^^^^^^^^^^^^^^^^^
441 |
442 |                   Profit Margin:
443 |

Version and model info
No response

I'm writing React, and I've had the linter utterly fail me in usefulness in this same way.
when I provide the lines and the linting errors, the llm's generally have quite a good approach to resolving it.
I think it'd even do quite a good job if the linters provided the liste of errors at the top of the file - but that's not what happens - the LLM's just are told to guess what's wrong on specific lines, and fail... hard. :(

Agreed, I think if Aider could just find a way to provide the errors from
REACT in a clearer way that the LLM can understand. It could accomplish far
more without any supervision.
--
Thank you,
Jason Edward Nicoll
Director of Marketing & Operations
On Fri, Aug 23, 2024 at 6:56 AM Tim Dobson @.***> wrote:

I'm writing React, and I've had the linter utterly fail me in usefulness
in this same way.
when I provide the lines and the linting errors, the llm's generally have
quite a good approach to resolving it.
I think it'd even do quite a good job if the linters provided the liste of
errors at the top of the file - but that's not what happens - the LLM's
just are told to guess what's wrong on specific lines, and fail... hard. :(
—
Reply to this email directly, view it on GitHub
https://github.com/paul-gauthier/aider/issues/1132#issuecomment-2305882367,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AU5VA7A5R4YZZXPNTCIQXBLZSZUCLAVCNFSM6AAAAABMZCFRBCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBVHA4DEMZWG4
.
You are receiving this because you authored the thread.Message ID:
@.***>

A thought @Jedward23 : do we have any idea where/how React's linter thing works?
Eg is it possible to call it and get a list of these errors?
eg
$ findMyErrors.exe.sh src/ts
ERROR in src/components/Dashboard/MonthlyConfig/FinancialOverview.tsx:440:91
TS2304: Cannot find name 'previousFinancials'.
438 | {renderMetric('Gross Profit', currentFinancials.grossProfit, showComparison ? previousFinancials?.grossProfit : undefined)}
439 | {renderMetric('Tax Amount', currentFinancials.tax, showComparison ? previousFinancials?.tax : undefined)}
ERROR in src/components/Dashboard/MonthlyConfig/FinancialOverview.tsx:439:85
TS2304: Cannot find name 'previousFinancials'.
437 |
438 | {renderMetric('Gross Profit', currentFinancials.grossProfit, showComparison ? previousFinancials?.grossProfit : undefined)}

Thanks for trying aider and filing this issue.
You can specify your preferred linter with --lind-cmd.
This doc may be helpful:
https://aider.chat/docs/usage/lint-test.html

@Jedward23  try something like
$ aider  --lint-cmd "typescript:npx tsc --noEmit --skipLibCheck --skipDefaultLibCheck "
It's not perfect, but I suspect with a bit of tweaking we can find something that works well for typescript projects like React often is. :)

I'm going to close this issue for now, but feel free to add a comment here and I will re-open or file a new issue any time.

FYI, aider will no longer attempt to use the built in linter for typescript. It appears to not be useful.
The change is available in the main branch. You can get it by installing the latest version from github:
aider --install-main-branch

# or...

python -m pip install --upgrade git+https://github.com/paul-gauthier/aider.git


[build] Unpin @babel/plugin-transform-classes when babel/babel#7533 is in a tagged release.

#216 pins @babel/plugin-transform-classes to a particular version to avoid a problem with the way that they transformed classes extending built-in constructors. babel/babel#7533 fixes this problem. Once it's in a tagged release, we should revert #216.

The mentioned babel bug #7533 has been merged into master already. Pls revert #216.

That PR was merged but isn't in any release yet, AFAICT.

babel just released 7.0.0-beta.48 today, should include the fix.

I am trying 7.0.0-beta.49 and still getting the errors like this:
Class constructor CustomStyle cannot be invoked without 'new'

Has someone been able to run latest @babel/preset-env with a babel-loader? Trying to get Webpack working now and got stuck on that issue.

babel/babel#7533 actually didn't end up fixing this issue. The function they use in place of Reflect.construct causes this.contructor to be wrong when calling into the polyfilled HTMLElement constructor. This prevents the polyfill from finding the definition, which it needs to either create an element with the correct tag name or return the already existing element that's being upgraded. I'm not sure of a good workaround or potential fix to send to Babel yet.

Hmm, maybe Function.bind.toString would help here? If we asked them to actually call Class from that snippet with apply if Class.toString() doesn't end with { [native code] }, would that still guarantee they always constructed native stuff with new?

Referenced this issue from the relevant lit-element issues.
/cc @TimvdLippe

@bicknellr I have tried to update @babel/plugin-transform-classes to 7.0.0-beta.52, and I no longer see the errors with the new. Wondering if babel/babel#8100 could fix the problem.

I think the reference in that commit which auto-closed this issue actually refers to the original here: https://github.com/Polymer/polymer-modulizer/issues/398.

@bicknellr Should this issue be renamed? That commit is released, but @web-padawan indicates the issue persists.

@bicknellr just to clarify: do I understand correctly that with the Reflect.construct based native shim landed in CE polyfill 1.2.0 this plugin is no longer required to be pinned?

@bicknellr @justinfagnani I was wondering if you have any plans regarding this issue? Babel is now RC and we still have to pin the plugin version, and looks like there is no opened ticket in their repo.
As you have more understanding about what behaviour is needed, and can formulate it better, could you submit the issue there? Also, do I understand correctly that it is only needed for IE11 now, as the native-shim uses Reflect.construct and therefore should work fine when transpiled to ES5?

@bicknellr could you file a new issue on Babel with the small repro we were working on?
@web-padawan another development here is that we don't need this transform as much because we made a new custom-elements-es5-adapter shim that works with both ES5-style classes and real classes. @bicknellr has that been released with the polyfills yet?

Filed: https://github.com/babel/babel/issues/8496

There's a fix https://github.com/babel/babel/pull/8501 if anyone wants to verify

Sorry, I've been trying to verify this but I'm having trouble using npm link to set up our tools with a locally built version of Babel.

Could also copy the diff in https://github.com/babel/babel/pull/8501/files#diff-f3afc82f08eb675b17f723219bd1ee36 if thats easier since its just the helper change?


[SPARK-46034][CORE] SparkContext add file should also copy file to local root path

What changes were proposed in this pull request?
For below case
add jar hdfs://path/search_hadoop_udf-1.0.0-SNAPSHOT.jar;
add file hdfs://path/feature_map.txt;

CREATE or replace TEMPORARY FUNCTION si_to_fn AS "com.shopee.deep.data_mart.udf.SlotIdToFeatName";

select si_to_fn(k, './feature_map.txt') as feat_name
from (    select 'slot_8116' as k   ) A;

User use valued-table then the task will running on driver, but driver didn't copy this file to the root path, then failed.
For ADD FILE driver also need to copy one file to root directory.
Why are the changes needed?
Fix bug
Does this PR introduce any user-facing change?
No
How was this patch tested?
MT
Was this patch authored or co-authored using generative AI tooling?
No

ping @cloud-fan @HyukjinKwon

test

Should be I make a mistake.
In driver side, file was download and copy to path driverTempPath
/mnt/ssd/0/yarn/nm-local-dir/usercache/typhoon.bo/appcache/application_1698132018785_8173703/spark-21bedef6-1c5e-464e-9cb0-bb6903b3d84c/userFiles-a4929fdb-b634-4829-a7e3-00d82b0d521b/fetchFileTemp8739978227963911629.tmp

but when execute the udf, it use container's path
/mnt/ssd/2/yarn/nm-local-dir/usercache/typhoon.bo/appcache/application_1698132018785_8173703/container_e59_1698132018785_8173703_01_000002/

cc @mridulm have you ever seen such things before?

cc @mridulm or @tgravescs have you ever seen such things before?: SparkContext.addFiles adds a file with a temporary name, and you cannot get it with the original name from SparkFiles.get(...)

I have checked, this is a mistake,  since it use move file logic, so there is no log.
The true reason is as  https://github.com/apache/spark/pull/43936#issuecomment-1822339813

The root cause is spark driver download file to it's driverTempPath, but didn't download to container's execution root path.
So in yarn cluster mode, if we need to use the file in driver side, it throw FileNotFoundException. cc @HyukjinKwon @cloud-fan I have updated the code.

Fetching hdfs://R2/projects/search_algo/hdfs/dev/typhoon.bo/uploader/ego_config/feature_map.txt to /mnt/ssd/0/yarn/nm-local-dir/usercache/yi.zhu/appcache/application_1698132018785_8495989/spark-32bf42ec-8de1-4f0e-aa60-def6d0f185c6/userFiles-61713d37-87ab-42d6-bdcd-065fa65ac8fc/fetchFileTemp8380676662376886133.tmp
Moving /mnt/ssd/0/yarn/nm-local-dir/usercache/yi.zhu/appcache/application_1698132018785_8495989/spark-32bf42ec-8de1-4f0e-aa60-def6d0f185c6/userFiles-61713d37-87ab-42d6-bdcd-065fa65ac8fc/fetchFileTemp8380676662376886133.tmp to /mnt/ssd/0/yarn/nm-local-dir/usercache/yi.zhu/appcache/application_1698132018785_8495989/spark-32bf42ec-8de1-4f0e-aa60-def6d0f185c6/userFiles-61713d37-87ab-42d6-bdcd-065fa65ac8fc/feature_map.txt
Fetching hdfs://R2/projects/search_algo/hdfs/dev/typhoon.bo/uploader/ego_config/feature_map.txt to /mnt/ssd/0/yarn/nm-local-dir/usercache/yi.zhu/appcache/application_1698132018785_8495989/container_e2006_1698132018785_8495989_01_000001/./fetchFileTemp7654378954291310228.tmp
Moving /mnt/ssd/0/yarn/nm-local-dir/usercache/yi.zhu/appcache/application_1698132018785_8495989/container_e2006_1698132018785_8495989_01_000001/./fetchFileTemp7654378954291310228.tmp to /mnt/ssd/0/yarn/nm-local-dir/usercache/yi.zhu/appcache/application_1698132018785_8495989/container_e2006_1698132018785_8495989_01_000001/./feature_map.txt

Current code work well in when testing in our env.

Any more suggestion? cc @HyukjinKwon @cloud-fan

Can we fix SparkFiles.get at driver side when Yarn cluster is used?

Can we fix SparkFiles.get at driver side when Yarn cluster is used?

SparkFiles.get() don't have problem.
The problem is user use relative path to use the added file.
But SparkContext.addFile add to driverTempDir
  def getRootDirectory(): String =
    SparkEnv.get.driverTmpDir.getOrElse(".")

But executing use . path to get file.

Could you please update the PR description？ It looks overdue and inaccurate for me to catch up with.
Could you also provide the output for LIST FILE? I guess we shall use its output at the caller-side, let's verify that they match.

Could you please update the PR description？ It looks overdue and inaccurate for me to catch up with.
Could you also provide the output for LIST FILE both before and after this? I guess we shall use its output at the caller-side, let's verify that they match.

The result of LIST FILES, won't have impact since we update addedFiles with origin path
spark-sql> add jar hdfs://path/search_hadoop_udf-1.0.0-SNAPSHOT.jar;
Time taken: 3.055 seconds
spark-sql> add file hdfs://path/feature_map.txt;
spark-sql> list jars;
hdfs://path/search_hadoop_udf-1.0.0-SNAPSHOT.jar
Time taken: 0.443 seconds, Fetched 1 row(s)
spark-sql> list files;
hdfs://path/feature_map.txt
Time taken: 0.428 seconds, Fetched 1 row(s)
spark-sql>

I don't know why we add a driverTmpDir, remove driverTmpDir also can resolve this issue.

The root cause is spark driver download file to it's driverTempPath, but didn't download to container's execution root path. So in yarn cluster mode, if we need to use the file in driver side, it throw FileNotFoundException. cc @HyukjinKwon @cloud-fan I have updated the code.

I encounter the same issue too. I think this should be fixed asap.

@AngersZhuuuu can you fix the PR description, and explain how this issue happens specifically in Yarn cluster mode? I still can't fully follow what where is the issue.


Add Neutron and neutrontestnet chains configs and assets

Add Neutron and neutrontestnet chains configs and assets

hey @albertandrejev thanks for the PR! Can you add these to the https://github.com/cosmos/chain-registry registry? This is where we pull data from


[FLINK-13442][network] Remove unnecessary notifySubpartitionConsumed method from view reader

What is the purpose of the change
Currently the methods of NetworkSequenceViewReader#notifySubpartitionConsumed and NetworkSequenceViewReader#releaseAllResources would be called meanwhile in netty stack during releasing resources.
As confirmed in FLINK-13245, in order to make this release logic simple and clean, we could remove the redundant notifySubpartitionConsumed from NetworkSequenceViewReader side, and also remove it from ResultSubpartitionView side. In the implementation of ResultSubpartitionView#releaseAllResources it would further notify the parent subpartition of consumed state via ResultSubpartition#notifySubpartitionConsumed which further feedback to parent ResultPartition layer via onConsumedSubpartition. Finally ResultPartition could decide whether to release itself or not.
E.g. for the case of ReleaseOnConsumptionResultPartition which is mainly used for pipelined partition, it would release partition after reference counter decreased to
0. For the case of ResultPartition which would be generated for blocking partition by default, it would never be released after notifying consumed. And the JM/scheduler
would decide when to release partition properly. In addition, InputChannel#notifySubpartitionConsumed could also be removed as a result of above.
Brief change log

Remove notifySubpartitionConsumed from NetworkSequenceViewReader
Remove notifySubpartitionConsumed from ResultSubpartitionView
Remove notifySubpartitionConsumed from InputChannel
Fix the bug by only calling onConsumedSubpartition when all readers are empty in BoundedBlockingSubpartition

Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not applicable)

During refactoring to remove the notifySubpartitionConsumed methods, I found another possible bug in previous implementation.  So I submitted these two things in separate commit.
@flinkbot attention @StephanEwen @pnowojski

I removed the last commit for fixing the problem of notifySubpartitionConsumed which does not consider multiple readers in blocking subpartition. That issue would be traced separately in FLINK-13493 and this PR is not necessary for current release-1.9 now.

Thanks for review @pnowojski ! I would handle it later.


wsl executes any command specified/starts shell before systemd has initialized, when starting instance

Is your feature request related to a problem? Please describe.
When a WSL instance with systemd support enabled is started (either with wsl or wsl <command>), the shell inside the instance is started immediately, without waiting for systemd to have initialized (reached running state) and indeed without even waiting for the system dbus to be available. This is far earlier than native Linux provides a login prompt (getty.target and systemd-logind.service are both called upon by multi-user.target.)
This means that if you are using systemd support in a way that involved heavy use of systemd services (and commands relating to same), there is a good chance that initial commands will fail. To pick a trivial example, wsl systemctl is-system-running usually errors out unless the WSL instance has already started.
Describe the solution you'd like
When systemd support is enabled, by default, wsl.exe should hold any further actions (executing commands or starting shells) until systemd has reached the running or degraded state (as described by systemctl is-system-running) to ensure commands behave as they would on native Linux and novel failure modes aren't introduced.
(This should be overridable for debugging purposes, ideally with a wsl.exe command-line option; a suitably lengthy or configurable timeout might also be useful.)
Describe alternatives you've considered
For compatibility with WSL as-it-is, this could be an option to be enabled in /etc/wsl.conf.

Still happens under 68.2.

IMO It might lead to undesirable and longer boot time. Some Systemd services can hold the distro from coming up for a really long time, For e.g. systemd-networkd holds the system from becoming 'ready' until the default interface gets an IP address from the DHCP server.
You might want to run apps that do not depend on systemd to just work like before without waiting for is-system-running e.g. vs-code, docker.
A better solution could be to introduce a new flag maybe --wait-for-systemd or --after-systemd ? To tell wsl to run the command after systemd has completed its job.
$ wsl -d <distro> --wait-for-systemd <command>

Hrm. I'm not sure that's sufficient, because on one hand, there are apps that optionally depend on systemd and will behave differently with a partial-systemd (and bits of other services, dbus &c.) than they would with no systemd; and on the other hand, even apps that don't care about systemd per se may depend on things that systemd would then change out from underneath them, such as binfmts, sysctl parameters, securityfs rules, etc. ad naus.
For example, systemd does a whole bunch of filesystem mounting and altering on the assumption that non-startup things aren't running at the same time - even an app that doesn't know anything about systemd may protest if /tmp suddenly switches out from beneath it in mid-execution, for example. This is also likely to include things like WSLg (per however #8888 shakes out), the state of the user runtime directory and overall user session (per however #8842 and #8918 work out), and so forth.
So while I do recognize your valid point here, I think that on the whole allowing these early-running commands (certainly by default - I might propose the opposite flag --no-wait-for-systemd instead, complete with disclaimer) would create untold opportunities for obscure, hard-to-diagnose race conditions.
On the other hand, this has interesting synergies with #8854 . I think we can probably agree that the two good options for just throwing out wsl <command> rather than starting a shell are either no systemd or always-running/long-running systemd; initializing a whole system (fully) just to run one command and shut down again is terribly wasteful, and part-initializing it just to   kill it again before it's even finished starting is even worse.

We also have to think about what happens when systemd is not enabled within the distro, should wsl let the command run or give a friendly error saying systemd is disabled?

On this, I'd say that all systemd-related flags, be they --wait-for-systemd or --no-wait-for-systemd should fail if systemd is not enabled, on the grounds that specifying them when it isn't indicates that the situation isn't what the command-runner thinks it is and there may be other assumptions he needs to rethink as well.

I see, we should keep things intuitive here. The user would expect systemd services to have completed their job before a shell is provided to them. The race conditions involved have already seeded a lot of problems which does not look like they have the same issue underneath at the glance making them hard-to-debug.
We should not let systemd and /init go around and keep manipulating directories and files at the same time. This also means that we should not have such services running that manipulate each other files and directories (I might be wrong here) in the first place.
Should we ask users to provide the details about mount/services on their system with debug logs (if systemd is enabled) ?

Is your feature request related to a problem? Please describe. When a WSL instance with systemd support enabled is started (either with wsl or wsl <command>), the shell inside the instance is started immediately, without waiting for systemd to have initialized (reached running state) and indeed without even waiting for the system dbus to be available. This is far earlier than native Linux provides a login prompt (getty.target and systemd-logind.service are both called upon by multi-user.target.)
This means that if you are using systemd support in a way that involved heavy use of systemd services (and commands relating to same), there is a good chance that initial commands will fail. To pick a trivial example, wsl systemctl is-system-running usually errors out unless the WSL instance has already started.
Describe the solution you'd like When systemd support is enabled, by default, wsl.exe should hold any further actions (executing commands or starting shells) until systemd has reached the running or degraded state (as described by systemctl is-system-running) to ensure commands behave as they would on native Linux and novel failure modes aren't introduced.
(This should be overridable for debugging purposes, ideally with a wsl.exe command-line option; a suitably lengthy or configurable timeout might also be useful.)
Describe alternatives you've considered For compatibility with WSL as-it-is, this could be an option to be enabled in /etc/wsl.conf.

Should we ask users to provide the details about mount/services on their system with debug logs (if systemd is enabled)?

For my systemd-enabling hacks, I have generally been asking for the output of a systemctl status to help in debugging. It probably wouldn't hurt to add that to the list.

related: https://github.com/microsoft/WSL/issues/11078


Did you use PhotonUI to design the menus of WannaPlan?

to whom it may concern,
did you use PhotonUI to design the following widgets: top-right menus, zoom, menu and submenus etc.
thank you for your feedback

Hello,
no, Wanaplan does not use PhotonUI in its main UI yet (but it will use it soon).


Runtime Error: An attempt has been made to start a new process before the current rocess has finshed its boostrapping phase.

I attempted the following code:
import requests

RAG = RAGPretrainedModel.from_pretrained("colbert-ir/colbertv2.0")

RAG.index(
    collection= ["This is a test", "Really it's just a test"],
    index_name="Test",
    split_documents=False,
)

When I run this I get the following error:

[Jan 04, 18:44:26] #> Note: Output directory .ragatouille/colbert/indexes/Test already exists
[Jan 04, 18:44:28] #> Note: Output directory .ragatouille/colbert/indexes/Test already exists
Traceback (most recent call last):
File "", line 1, in
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/spawn.py", line 125, in _main
prepare(preparation_data)
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/runpy.py", line 289, in run_path
return _run_module_code(code, init_globals, run_name,
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/jponline77/ragatouille/RAG_2.py", line 6, in
RAG.index(
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/site-packages/ragatouille/RAGPretrainedModel.py", line 117, in index
return self.model.index(
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/site-packages/ragatouille/models/colbert.py", line 166, in index
self.indexer.index(
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/site-packages/colbert/indexer.py", line 78, in index
self.__launch(collection)
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/site-packages/colbert/indexer.py", line 83, in __launch
manager = mp.Manager()
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/context.py", line 57, in Manager
m.start()
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/managers.py", line 562, in start
self._process.start()
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/context.py", line 288, in _Popen
return Popen(process_obj)
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 32, in init
super().init(process_obj)
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/popen_fork.py", line 19, in init
self._launch(process_obj)
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 42, in _launch
prep_data = spawn.get_preparation_data(process_obj._name)
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/spawn.py", line 154, in get_preparation_data
_check_not_importing_main()
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/spawn.py", line 134, in _check_not_importing_main
raise RuntimeError('''
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

Traceback (most recent call last):
File "/home/jponline77/ragatouille/RAG_2.py", line 6, in
RAG.index(
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/site-packages/ragatouille/RAGPretrainedModel.py", line 117, in index
return self.model.index(
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/site-packages/ragatouille/models/colbert.py", line 166, in index
self.indexer.index(
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/site-packages/colbert/indexer.py", line 78, in index
self.__launch(collection)
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/site-packages/colbert/indexer.py", line 83, in __launch
manager = mp.Manager()
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/context.py", line 57, in Manager
m.start()
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/managers.py", line 566, in start
self._address = reader.recv()
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/connection.py", line 414, in _recv_bytes
buf = self._recv(4)
File "/home/jponline77/miniconda3/envs/rag/lib/python3.10/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError

The following is my torch version:

Version: 2.1.2
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org/
Author: PyTorch Team
Author-email<EMAIL_ADDRESS>License: BSD-3
Location: /home/jponline77/miniconda3/envs/rag/lib/python3.10/site-packages
Requires: filelock, fsspec, jinja2, networkx, sympy, typing-extensions
Required-by: RAGatouille, sentence-transformers, torchaudio, torchvision

The following is my cuda version:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Feb__7_19:32:13_PST_2023
Cuda compilation tools, release 12.1, V12.1.66
Build cuda_12.1.r12.1/compiler.32415258_0

) not 100% sure but I think you need to run the code in a block:

if __name__ == "__main__":
    # your code here

Thanks!  That worked!

Ok let’s keep this open as a reminder to make this clear in the docs.
And maybe also avoid forking by default if there’s only one gpu?
cc: @bclavie

Thanks for the report! I've added a mention in the README, and will make it more prominent in the next update -- it should've been there to begin with, sorry for the oversight.


MemLib and Wiring should use standard annotations YAML reader/writer

Right now they have custom YAML which doesn't seem like a good idea for maintainability.

New annotation serialization is coming soon, making this issue obsolete.

@azidar is there a preview of this new API for annotation serialization?

I plan to make a PR this week or next. The gist is that Annotations that are case classes and only use standard Scala types (Strings, Ints, Lists, etc.) will be automatically serializable using json4s

Is the existing YAML infrastructure going to stay?

I was planning to deprecate it and then remove sometime next year. The primary issue with it is that the current implementation and library cannot easily distinguish between different Annotation types that have the same fields, json4s provides nice automatic type tagging for dealing with that.

Thanks, Jack.  I'll monitor the release notes.  Should not be a problem for me to make the transition.

This file should be gone I assume?
https://github.com/freechipsproject/firrtl/blob/master/src/main/scala/firrtl/passes/memlib/YamlUtils.scala

Looks like this is not ready to be closed... Those YamlFileReader and YamlFileWriter are still being used. Wiring is entirely moved to using annotations (and resulting JSON), though.

Is memlib ready to be remove from next version of firrtl?
I tried to port firrtl to Scala 2.13
moultingyaml seem to be abandoned, and it's too old to port to Scala 2.13, which will become the only barrier for Scala 2.13 porting.

1.2 will be out soon, we should fix this and bump to Scala 2.13 in 1.3


[Core] [Mono] Add Vector2i and Vector3i types

EDIT: https://github.com/godotengine/godot/issues/16023 should probably be solved first, since this breaks VisualScript compat.
This PR adds Vector2i and Vector3i types internally, to Variant, to GDScript, to VisualScript, and to Mono/C#, and adds docs for them, as well as updating other docs and making some other vector code more consistent.  See https://github.com/godotengine/godot/issues/3286
I, and many others, need this feature. I can always implement my own structs, but it's much nicer if the engine functions can use it and it is available to everyone.
What this PR does not do:

Add Vector2i or Vector3i to GDnative. I took a peek at the code and it looks like quite a lot of work, so EDIT: See below.

Use these types anywhere internally (EDIT: anywhere addtionally) (so this PR doesn't close but helps solve https://github.com/godotengine/godot/issues/3286), currently it's just for user code. I'll make another PR after this for using these types internally one this PR is reviewed, accepted, and merged.

Fairly large PR, there are also a few other minor changes to problems that I noticed in here, so please review carefully. Suggested milestone: 4.0.

EDIT: I would try to implement it, unless of course @karroffel wants to do the GDnative part himself, it seems there are some issues with ABI compatibility? I do wonder if we should strive for maintaining ABI compatibility between minor releases if API compatibility is preserved and all that's needed is a recompile. @karroffel also expressed concern with this PR and I can imagine that we'd have to yet again be bothered with GDnative ABI issues if we wanted to implement Vector4 and there are probably many other ABI-breaking proposals that I'm not aware of.

While I very much welcome the PR, the way it is implemented breaks already compiled GDNative libraries. This can be worked around, and that would have to be done when this gets merged, but in the current state it's a breaking change.

@karroffel Would it be a better idea for compatibility if the items in the Variant::Type enum were given explicit integer values? For example, NIL = 0, VECTOR2 = 100, or something like that?

After looking at this PR, I'm a little confused. Is this adding Vector2i and Vector3i to GDScript?
And if so, will Vector2i be usable where the parameter type is a Vector2? Or will the function have to explicitly ask for a Vector2i?

Yes, it adds it to GDScript (and C#). There might be a way to do implicit conversions but right now I have an explicit to_vector2() method, I'd love some feedback from the devs here (I have casts set up in C#).

Nice, thanks Aaron :)
I think it would be nice if it followed C# conventions.
int -> float : implicit due to no precision loss
float -> int : explicit due to precision loss
so...
Vector2i -> Vector2 : implicit
Vector2 -> Vector2i : explicit

Yep, that's exactly how I did it, my only concern is "how do I best do this in GDScript?"

Things to fix before merging this:

In GDScript, these types show up as another color, @vnen what code do I need to change to fix this?

In VisualScript, the icons for Vector2i and Vector3i don't show up. I can't figure out what I need to do to fix this. Who do I even ping? I saw @fire working on it before. Visual script is not listed in the CODEOWNERS file, which itself should probably be fixed, cc @akien-mga.

What to do about GDNative? @karroffel We can change the code in variant.h if that's easiest.

@aaronfranke https://github.com/godotengine/godot/blob/master/modules/visual_script/visual_script_editor.cpp#L493 for the icons

@fire I added that already and it doesn't show up.

https://github.com/godotengine/godot/blob/b16c309f82c77d606472c3c721a1857e323a09e7/editor/property_selector.cpp#L168 Try debugging here.

Look at where set_icon is being done.

As discussed on IRC, this is a nice change overall, but will have to wait for 4.0 where we can make better use of it consistently through the API.

I rebased this on top of master (a few commits not yet pushed), and it builds fine locally, but I'm getting an extremely weird error in Travis. This only shows up with the docs commit pushed, the previous ones pass the Travis checks just fine, and all commits build fine too.
ERROR: Unresolved member 'length', file: Vector2i

I'm very confused about this, the Vector2i docs have the same fields generated by --doctool, and length works perfectly fine in a project with this test code:
var v = Vector2i(1, 2)
print(v.length())

What would cause the Travis static checks to fail like this?

I'm closing this for now. There are just too many conflicts and a few other things that should be solved first, I am 100% confident that some of my other PRs will conflict with this, and the last time I tried rebasing I ended up with a mysterious crash that dumped the stack trace. Plus, I know that there is a plan to close all old PRs sometime soon anyway. It would have been vastly easier if this was merged before the massive refactoring.
Later, when my other pull requests are merged and this can stay in a stable state for more than a week again, we can re-open this - I'll probably just end up redoing the whole thing from scratch.


show tags in monitor select list under status page

⚠⚠⚠ Since we do not accept all types of pull requests and do not want to waste your time. Please be sure that you have read pull request rules:
https://github.com/louislam/uptime-kuma/blob/master/CONTRIBUTING.md#can-i-create-a-pull-request-for-uptime-kuma
Tick the checkbox if you understand [x]:

[x] I have read and understand the pull request rules.

Description
Fixes #(2739)
Type of change
Please delete any options that are not relevant.

User interface (UI)

Checklist

[x] My code follows the style guidelines of this project
[x] I ran ESLint and other linters for modified files
[x] I have performed a self-review of my own code and tested it
[x] I have commented my code, particularly in hard-to-understand areas
(including JSDoc for methods)
[x] My changes generate no new warnings
[ ] My code needed automated testing. I have added them (this is optional task)

Screenshots (if any)
Please do not use any external image service. Instead, just paste in or drag and drop the image here, and it will be uploaded automatically.

It would be better to use the existing Tag.vue component instead of duplicating the styling code.

sure @chakflying, thanks for the suggestion. Have made the changes !

Ping @chakflying @chakflying

Since you aren't adding new packages, you can remove package-lock from the commits so it won't show up here.

I've removed the package-lock.json file. Please do review and suggest changes. Thanks !

I got error when I tried to run this. Do you know what's the issue?
Uncaught (in promise) TypeError: monitor is undefined
    option StatusPage.vue:347

I guess this was a bug. I used the word monitor instead of option inside vue-multiselect. not sure why it didn't work with monitor though !

Looks like the Tag component is not imported correctly so this still doesn't work. Don't want to be mean but looks like you did not test this before marking as ready.
Also recommend doing an actual rebase since master has the fix for the marked() issue, and you are now modifying the package-lock.json file twice instead of not touching it.


Just a question.

Hi,
I found following in the dmesg today, I just want to ask what that means?
[174239.812997] traps: php5-fpm[3405] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[175605.735950] traps: php5-fpm[9183] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[175779.361944] traps: php5-fpm[9638] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[177417.397432] traps: php5-fpm[16439] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[177655.084296] traps: php5-fpm[17500] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[177751.950135] traps: php5-fpm[17731] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[177762.712883] traps: php5-fpm[17743] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[178982.887803] traps: php5-fpm[22543] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[179091.269015] traps: php5-fpm[23000] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[179432.434589] traps: php5-fpm[24273] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[179595.518552] traps: php5-fpm[24758] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[179704.844542] traps: php5-fpm[25067] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[179789.465978] traps: php5-fpm[25634] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[179831.505909] traps: php5-fpm[25716] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[179955.653311] traps: php5-fpm[25897] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[179992.780569] traps: php5-fpm[26138] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[180010.100788] traps: php5-fpm[26093] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[180032.759503] traps: php5-fpm[26507] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[180122.715361] traps: php5-fpm[26768] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[180165.971339] traps: php5-fpm[26866] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[180468.074682] traps: php5-fpm[27943] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
[182841.443416] traps: php5-fpm[3643] general protection ip:7f37f514ed19 sp:7fff22524330 error:0 in aerospike.so[7f37f511e000+b0000]
System OS: Ubuntu
PHP:
PHP 5.5.12-2ubuntu4.1 (cli) (built: Oct 29 2014 11:59:49)
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies
with Zend OPcache v7.0.4-dev, Copyright (c) 1999-2014, by Zend Technologies
Thanks.

Hi again. We haven't been able to reproduce this, so far, with a script that simply uses get/put. Can you tell me which methods of the PHP client you're using or provide a code snippet that reproduces these messages on your side?
Thanks.

Hey Martin, can you grab the new 3.3.5 release and see if those messages repeat? We've changed a few things under the covers, mainly in the connection close() and I believe that will affect what you've seen.

Hi,
Sorry for late respond, had a bit longer holidays, I'll update tomorrow and see report quick as possible back.

No problem, man. Hope it was a good vacation :)

Hi,
Yea was good :-), So I've updated the source and i cant see it at the moment. :-). Thank you.

php7 persistent connection error
$db              = new Aerospike($sConnectAS_OLD, true);
Core was generated by `php-fpm: pool www            '.
Program terminated with signal 11, Segmentation fault.
#0  0x00007f17e5affad7 in aerospike_helper_object_from_alias_hash (as_object_p=0x7f17f206a000, persist_flag=true, conf=0x7ffd8e50ae00, shm_key_list=0x119d190, persistent_list=0x119d150, val_persist=39)
at /root/jobs/aerospike-client-php/src/aerospike/aerospike_helper.c:551
551		as_object_p->as_ref_p->ref_as_p++;
Missing separate debuginfos, use: debuginfo-install php-fpm-7.0.10-1.el6.remi.x86_64
(gdb) quit


Accept axis tuple for flip consistent with numpy

Closes dask/dask#7674.

[x] Closes #7674
[x] Tests added / passed
[ ] Passes black dask / flake8 dask / isort dask

black dask caused a large delta of unrelated changes, so was not applied. Is some special invocation necessary beyond what's in the contribution docs? (This is my first dask contribution.) flake8 and isort were fine.
It's not clear from other array functions what degree of implementation and documentation consistency with numpy is desired, so I went for a simpler and more concise style similar to the existing version.

It was indeed a black version problem (21.5b1). 20.8b1 worked and passed
with no changes. Thanks, @jrbourbeau
On Wed, May 19, 2021 at 3:34 PM James Bourbeau @.***>
wrote:

@.**** commented on this pull request.
Thanks @aschampion https://github.com/aschampion!
black dask caused a large delta of unrelated changes, so was not applied
What version of black are you using locally? In CI we're using 20.8b1.
Could you either try again with black==20.8b1 or try using the pre-commit
hook method outlined in
https://docs.dask.org/en/latest/develop.html#code-formatting?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/pull/7675#pullrequestreview-663289515, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AADQLLX4C2OLBWHNQSH2X4TTOPD67ANCNFSM45EWVLRQ
.

For reference, I'm proposing we bump the version of black and flake8 we use in CI to their latest releases over in https://github.com/dask/dask/pull/7676

cc @douglasdavis in case you get a moment to look at this PR


Fix integer underflow/unsigned wrapping.

Got pointed here from the Mozilla bug tracker to send in the patch upstream:
This is an attempt at a fix for firefox bug 1614278 where firefox crashes on a regular on xruns. This has been a long standing issue for me (older than the linked bug report). With this patch applied firefox does run stable again with the jack backend.
Looking at the subtraction logic in the xrun handing I am somewhat surprised this does not crash more often. Might also be related to #154
--
The number of xrun fragments to skip is added once to ctx->jack_xruns but
then subtracted multiple times (once for each stream) from the same variable.
This causes the unsigned int to "underflow" and wrap around.

Use unsigned int in all calculations instead of mixing un/signed.
Remove erroneous and useless subtraction logic
Replace for loop by simple multiplication
Remove unneeded casts to float

Any chance to get a review on this one? @padenot @kinetiknz

Looks good, sorry for the delay, feel free to ping us directly to get a faster review if we don't get to it quickly enough, especially for things like this that make your life easier.
I'm a bit surprised you're under-running that much, that said. Do you have a setup in which this is expected for some reason?

Delays do happen. I know so myself. All good, thanks for looking at it now.
And regarding the xruns: it is because I usually push my hardware to the limits. Meaning I often run my setup like in a live environment: bitwig and/or traktor open + the buffer size at the lowest setting that does not give me xruns. If I then open Firefox additionally - to watch some tutorial or hunt for some new music - it tends to push my laptop beyond its limits. Hence the occasional under run.

I see, sounds like a normal behaviour then, considering what is being discussed for the jack backend in other issues.


Unintuitive result scoring with Ctrl-P in recent history

VSCode Version: Code 1.9.1 (f9d0c687ff2ea7aabd85fb9a43129117c0ecf519, 2017-02-09T00:26:45.394Z)
OS Version: Windows_NT ia32 10.0.14393
Extensions:

Extension
Author
Version

vscode-whitespacer
deerawan
1.0.0

contextualduplicate
lafe
0.2.0

cpptools
ms-vscode
0.9.2

csharp
ms-vscode
1.6.2

duplicatetext
rivenvirus
0.0.3

trailing-spaces
shardulm94
0.2.11

vscode-fileutils
sleistner
2.3.3

shader
slevesque
0.2.2

cmake
twxs
0.0.15

Steps to Reproduce:

See gif.

I feel like the shorter result (in this case list-view.js) is a better match than the longer one. I would prefer to have to (for example) add an "s" to specify that I want to open list-view-spec.js

Related: https://github.com/Microsoft/vscode/issues/17796

@kumarharsh Thanks for linking! The first post in that issue is about Intellisense. I'm not sure if they use the same kind of matching algorithm (I'd imagine that Intellisense is much more complicated :) ), but a few of the responses are definitely related to this issue. Might be worth either joining these two issues, or moving the File picker (Ctrl+P) related discussion here?

@Srekel the "Recently Opened" list is purely sorted by last access whereas the "File Results" are sorted by our matching algorithm. I think what you are asking for is to apply the same scoring to the "Recently Opened" list.

Ah, is that what it is. Yes, I'm pretty sure that would be preferable. :)

Alternatively maybe the setting should actually disable the "recently opened" altogether and so any result is coming in from the general file search scoring algorithm.

I'd love to add scoring or just disable recently opened altogether.  Coming from SublimeText here, so my muscle memory assumes it'll only hit files in the project.  Currently, I get things like this:

Every time I type in pack expecting the project's package.json I get ui-router's and it bumps back to the top of Recently Opened.  This feature's been much more of a hassle than a helper for me.

@bpasero

the "Recently Opened" list is purely sorted by last access whereas the "File Results" are sorted by our matching algorithm

One important point here is that the "Recently Opened" is only sorted by last access until one starts typing. For example, we have lots of README files and say that I just read ./user-portal/client/README.md. Before typing anything, it is usefully at the top:

However, after typing "r" (my habit is to hit Ctrl+P and just start typing), the results are "random" as far as I'm concerned:

I think that the search ranking should only be applied to the "File Results" section, not the "Recently Opened". (This is not against the idea of an setting to hide the "Recently Opened" section entirely but if it's there, I think it should be sorted by last access.)

Do you think it makes sense? Should I create a new issue for it or would this better be a bullet point in Improve ranking of elements in quick open #27317 ?

@bpasero Does the recently open list maintain the recently open order now when I start typing?

@borekb no, only when our scoring algorithm ends up with the same score for 2 entries we use the recency (previously we just sorted by path length).

@bpasero thanks for the explanation, I still think that this list should just sort by recency so I've created a new issue for it: https://github.com/Microsoft/vscode/issues/35610
Anyway, it's awesome to see scoring improvements, thanks!


Redo MHCflurry preprint with current code base

In Tim's validation notebook, he uses combined_test_BLIND_dataset_from_kim2013.csv as his validation set.
We should have some code and documentation that describes what's in this file and how to build it from publicly available data.

Generalizing this issue a bit into redoing the model selection and evaluation in Predicting Peptide-MHC Binding Affinities With Imputed Training Data using the current MHCflurry code base. We should also publish these numbers with each models release.

This will be a new paper instead


Unable to use winspool.drv

This is what I tried. Problem ist, that ffi.Library only works with pathes that end in '.dll'.
const ffi = require('ffi');
const ref = require('ref');
const Struct = require('ref-struct');
const wchar_t = require('ref-wchar');

const int = ref.types.int;
const intPtr = ref.refType(int);

const winspoolDll = 'c:\\windows\\system32\\winspool.drv';
const winspoolLib = ffi.Library(winspoolDll, {
  'EnumPrintersW': [int, [int, ref.types.CString, int, printerInfoPtr, int, intPtr, intPtr]]
});

This gives me the follwing DEBUG output which shows, that ffi.Library adds the .dll extension when it gets called with a name/path that does not end in dll (e.g. winspool.drv).
 ref setting value on allocated buffer c:\windows\system32\winspool.drv.dll +0ms
  ref set(): (offset: 0) <Buffer@0x00000298EF888D10 00 00 00 00 00 00 00 00> c:\windows\system32\winspool.drv.dll +1ms
  ref writing pointer to buffer <Buffer@0x00000298EF888D10 00 00 00 00 00 00 00 00> 0 <Buffer@0x00000298F1705170 63 3a 5c 77 69 6e 64 6f 77 73 5c 73 79 73 74 65 6d 33 32 5c 77 69 6e 73 70 6f 6f 6c 2e 64 72 76 2e 64 6c 6c 00> +0ms
  ref writing pointer to buffer <Buffer@0x00000298F170CA90 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00> 0 <Buffer@0x00000298EF888D10 70 51 70 f1 98 02 00 00> +1ms
  ref allocating Buffer for type with "size" 4 +0ms
  ref setting value on allocated buffer 0 +1ms
  ref set(): (offset: 0) <Buffer@0x00000298EF888E00 00 00 00 00> 0 +0ms
  ref writing pointer to buffer <Buffer@0x00000298F170CA90 10 8d 88 ef 98 02 00 00 00 00 00 00 00 00 00 00> 8 <Buffer@0x00000298EF888E00 00 00 00 00> +1ms
  ref dereferencing buffer <Buffer@0x00000298EF888DE0 00 00 00 00 00 00 00 00> +0ms
  ref get(): (offset: 0) <Buffer@0x00000298EF888DE0 00 00 00 00 00 00 00 00> +1ms
  ffi:DynamicLibrary dlerror() +0ms

Looks like #416 would be the cure for this.


MSSQL Source tunnel_method SSH_KEY_AUTH 422 Unprocessable Entity

Provider version: 0.2.3
Trying to create a source with airbyte_source_mssql
tunnel method looks like this:
    tunnel_method = {
      source_mssql_ssh_tunnel_method_ssh_key_authentication = {
        tunnel_method = "SSH_KEY_AUTH"
        ssh_key       = local.mssql_config.ssh_tunnel.ssh_key
        tunnel_host   = local.mssql_config.ssh_tunnel.tunnel_host
        tunnel_port   = local.mssql_config.ssh_tunnel.tunnel_port
        tunnel_user   = local.mssql_config.ssh_tunnel.tunnel_user
      }
    }

Which is teoretically supported by the provider: https://registry.terraform.io/providers/airbytehq/airbyte/latest/docs/resources/source_mssql#nested-schema-for-configurationtunnel_methodsource_mssql_ssh_tunnel_method_ssh_key_authentication
But getting this error saying it only supports NO_TUNNEL or SSH_PASSWORD_AUTH:
**Request**:
PUT /v1/sources/<workspace_id_cutoff> HTTP/1.1
Host: api.airbyte.com
Accept: */*
Authorization: Bearer <api_key_cutoff>
Content-Type: application/json
User-Agent: speakeasy-sdk/terraform 0.2.3 2.75.2 1.0.0

**Response**:
HTTP/2.0 422 Unprocessable Entity
Content-Length: 430
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Content-Type: application/problem+json
Date: Fri, 4 Aug 2023 12:08:39 GMT
Server: envoy
Via: 1.1 google
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{"type":"https://reference.airbyte.com/reference/errors#unprocessable-entity","title":"unprocessable-entity","status":422,"detail":"The provided configuration does not fulfill the specification. Errors: json schema validation failed when comparing the data to the json schema. \nErrors: $.tunnel_method.tunnel_method: must be a constant value NO_TUNNEL, $.tunnel_method.tunnel_method: must be a constant value SSH_PASSWORD_AUTH "}

Creating MSSQL sources with SSH_KEY_AUTH tunneling by hand works just fine in cloud.airbyte.com

Thank you for bringing this issue up to our attention. It should be fixed in the latest release of 0.3.0 here: https://github.com/airbytehq/terraform-provider-airbyte/releases/tag/v0.3.0


fix: change count of col/row after col/row deleted and its test function

fix #227
fix: change count of col/row after col/row deleted and its test function

LGTM

LGTM