[SPARK-4465] runAsSparkUser doesn't affect TaskRunner in Mesos environme...

...nt at all.

fixed a scope of runAsSparkUser from MesosExecutorDriver.run to MesosExecutorBackend.launchTask
See the Jira Issue for more details.

Test build #24991 has started for   PR 3741 at commit 1b047e6.

This patch merges cleanly.

Test build #24992 has started for   PR 3741 at commit 46ad71e.

This patch merges cleanly.

Test build #24991 has finished for   PR 3741 at commit 1b047e6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24991/
Test PASSed.

Test build #24992 has finished for   PR 3741 at commit 46ad71e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24992/
Test PASSed.

/cc @tnachen @JoshRosen Please, review this patch again.

I deleted my old fork from my repo and re-fork for another patch. If it occurs some problem for being merged into master, I can create another PR with same changes.

Patch LGTM.

+1 @tnachen

@JoshRosen please, merge this PR. thx.


Console warning in mui demo

Describe the bug
Things still run and visually looks good, but red stuff makes me nervous
What is the expected behavior?
What are the steps to reproduce?

yarn start
See these console warnings

Screenshots / Screen recording

Anything else to add?
I am going to guess this has to do with our themes, but I am not sure

check to see if this still exists after the new icon demo workspace refactor

No console error in current dev branch


feat: footer

@Chanakya888
Is it OK for me to take this PR and update it?

@Chanakya888
Is it OK for me to take this PR and update it?

Yes please, this PR has been waiting for a long time

@Chanakya888 cc @nud3l
OK, I'll take this PR.
But for now, my focus is on table migration.
After that, I'll handle this PR.

@Chanakya888 cc @nud3l
OK, I'll take this PR.
But for now, my focus is on table migration.
After that, I'll handle this PR.

Oh, no problem then. I'll quickly address your comments tonight. Do you want me to update to tailwind?

@Chanakya888
I wanted to update it as an example.
From now on, we should (must) use tailwindcss. It's very helpful and easy to style.

@Chanakya888
I wanted to update it as an example.
From now on, we should (must) use tailwindcss. It's very helpful and easy to style.

Alrighty, I'll address the rest of the comments and leave the tailwind for you

@anton-karlovskiy I've addressed your comments. I couldn't get the svg of web3 foundation right, I've tried tracing it in illustrator and some online tools, neither of them were successful, I'll ask @alexeiZamyatin for the svg.

@anton-karlovskiy I've addressed your comments. I couldn't get the svg of web3 foundation right, I've tried tracing it in illustrator and some online tools, neither of them were successful, I'll ask @alexeiZamyatin for the svg.

See #ux channel in discord.

@nud3l cc @Chanakya888
Although the footer itself is in a good place, there are relatively a lot of TODOs to be done in this PR:

the footer's layout at the app level
CSS normalization switch
quick-fixing broken buttons on the dashboard page

New footer looks great. Only thing I'd like to change is that the headings (Contact, Challanges, Feedback, Docs) are bold.

Not sure where that is in Figma right now, otherwise I would have commented there.

Overall, very much like the new footer. One thing to fix before merging this is to not have the footer "floating around" on the landing page:

@nud3l

One thing to fix before merging this is to not have the footer "floating around" on the landing page

That's related to the remaining part (the footer's layout at the app level) I mentioned.
The footer layout did not follow the best practice so it's error-prone.
This time, I'm going to fix it but it takes some time. So I'd suggest fixing it on top of the merged codebase.
What do you think? I'll follow you.

@nud3l

One thing to fix before merging this is to not have the footer "floating around" on the landing page

That's related to the remaining part (the footer's layout at the app level) I mentioned.
The footer layout did not follow the best practice so it's error-prone.
This time, I'm going to fix it but it takes some time. So I'd suggest fixing it on top of the merged codebase.
What do you think? I'll follow you.

Let's fix it here since we don't have anything incoming in parallel now.


[AuthV2] Address Bug Bash feedback

This checklist is used to make sure that common guidelines for a pull request are followed.

For new extensions:

[ ] My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your PR is merged into master branch, a new PR will be created to update src/index.json automatically.
The precondition is to put your code inside this repo and upgrade the version in the PR but do not modify src/index.json.

AuthV2

#sign-off


Bug: mesh.thumbnail hangs in docker container

I cannot run a pipeline in a docker container that takes a screenshot of the mesh.

Have you started the xvffb?
https://vedo.embl.es/autodocs/content/vedo/vedo/settings.html#Settings.start_xvfb

starting xvfb (can take a minute) .../usr/bin/Xvfb
 xvfb started.
2024-04-08 02:23:28.772 (   4.613s) [    7F1F70AE9000]vtkXOpenGLRenderWindow.:456    ERR| vtkXOpenGLRenderWindow (0x55a69c64b300): bad X server connection. DISPLAY=99.0. Aborting.

It looks like running it in VSCode in debug mode doesn't work because set_xvfb.sh does not store the information in the debugger when it's run as a prelaunch task

to be honest I'm not familiar enough with the xvbf machinery ... :(


lastEventId is set for unsubmitted events

Do you want to request a feature or report a bug?
Bug
What is the current behavior?
A lastEventId is generated for events that have not been sent to Sentry due to missing setup.
What is the expected behavior?
If the event can't be sent to Sentry due to missing configuration, the lastEventId shouldn't be set / updated, as this implies we have created a new event in Sentry.

// This event_id can be used to reference the error within Sentry itself.
// Set lastEventId after we know the error should actually be sent

This comment is from _sendProcessedPayload and as I thnik describes the issue, so someone already noticed this? Later in the code before sending if (!this.isSetup()) return; is called to dismiss the transmission if not installed correctly. So shouldn't it be possible to move the addition of the lastEventId after this check?

Ooh, good catch.


Attempt to refresh plugins after upgrade

Attempt to refresh plugins after upgrade so Vivid uses the latest versions

Cannot be done (will break plugins and some already prevent it like Vivid and vim-fugitive)

I will be looking into the feasability and problems of this, and I may possibly add it as an option, which defaults to off.

This is outside the scope of this project. It will increase complexity so significantly, that it is not even worth attempting to implement.


KAFKA-14718: Fix flaky DedicatedMirrorIntegrationTest

Motivation
The test suite DedicatedMirrorIntegrationTest fails in a flaky manner with the following error:
org.opentest4j.AssertionFailedError: Condition not met within timeout 30000. topic A.test-topic-0 was not created on cluster B- ._~:/?#[]@!$&'()*+;="<>%{}|\^`618 in time ==> expected: <true> but was: <false>

Root cause of the failure is available in stdout:
[2023-02-17 18:56:32,308] INFO This node is a follower for A->B- ._~:/?#[]@!$&'()*+;="<>%{}|\^`618. Using existing connector configuration. (org.apache.kafka.connect.mirror.MirrorMaker:234)
[2023-02-17 18:56:42,567] INFO Kafka MirrorMaker stopping (org.apache.kafka.connect.mirror.MirrorMaker:202)
[2023-02-17 18:56:43,371] INFO Kafka MirrorMaker stopped. (org.apache.kafka.connect.mirror.MirrorMaker:213)
[2023-02-17 18:56:43,371] INFO Kafka MirrorMaker stopping (org.apache.kafka.connect.mirror.MirrorMaker:202)
[2023-02-17 18:56:44,376] INFO Kafka MirrorMaker stopped. (org.apache.kafka.connect.mirror.MirrorMaker:213)
[2023-02-17 18:56:44,377] INFO Kafka MirrorMaker stopping (org.apache.kafka.connect.mirror.MirrorMaker:202)
[2023-02-17 18:56:45,782] ERROR Failed to configure MirrorSourceConnector connector for A->B- ._~:/?#[]@!$&'()*+;="<>%{}|\^`618 (org.apache.kafka.connect.mirror.MirrorMaker:236)
org.apache.kafka.connect.errors.ConnectException: Worker is shutting down
	at org.apache.kafka.connect.runtime.distributed.DistributedHerder.halt(DistributedHerder.java:766)
	at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:361)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

The test tries to assert topic replication on the second cluster but the mirror maker nodes themselves are unavailable. They are unavailable because the connectors fail to configure.
Change
This highlights a gap in MirrorMaker startup sequence where it does not wait for connectors to be configured by the herders before declaring itself as "started". This leads to a situation where MirrorMaker claims it has started but the herders are still in the process of configuring the connectors. During this time, mirror maker will not perform any replication.
With this change, we add a dependency on connector configuration for the MirrorMaker to declare itself as started.
Rejected alternative
An alternative approach is to modify the test to wait for connectors to get configured on all 3 MM nodes before starting the actual test i.e. before making a call to create topic. But this alternative is artificially hiding the problems which users of MirrorMaker may also face.

@C0urante please take a look when you get a chance. unitTest and integrationTest succeed for me locally.

Thank you for the detailed explanation @C0urante!

so this doesn't really buy us much right now.

Oops, my bad! I did not notice that.

this only guarantees that connector configs have been written to the config topic.

That is true. We need a more comprehensive check. I was basing this change on the fact that we use similar verification in other tests (see waitUntilMirrorMakerIsRunning() in MirrorConnectorsIntegrationBaseTest). It's more of a "better than nothing" approach here.

And even if we did tweak this PR to do that kind of blocking, we'd still have to choose a reasonable amount of time to wait for all of these operations to complete.

I think that fixing the test by extending the timeout might hide the problems that users may be facing in production as well. Ideally the MirrorMaker#start() should be returning a Future which could be tracked to check the completion of start. In absence of such a mechanism, today, a user might call MirrorMaker#start() and assume that it has started correctly while in the background, connector start up might have failed (just as it did in this test scenario). Hence, I believe, we need a better mechanism to check for successful startup of MirrorMaker. Thoughts?

Do you think it might be worth it to just bump the timeouts of these tests?

We could do that AND as a partial safety check block topic creation in this test until connectors have at least been configured (see rejected alternative). Let's reach to a conclusion on the above point first and then we can proceed with this option if that's what we agree to.

I think that fixing the test by extending the timeout might hide the problems that users may be facing in production as well.

I don't think that bumping the timeouts in these tests is likely to hide issues that may surface in production; rather, I'm only worried that they'll make local testing and development more cumbersome.

Hence, I believe, we need a better mechanism to check for successful startup of MirrorMaker. Thoughts?

Agreed 👍 IMO MM2 observability in general can be improved. Right now you're basically limited to JMX, other custom metrics reporters, and logs. There's no public API for viewing cluster-wide health info (the closest thing is reading directly from the status topic, but that's not public API and we shouldn't count on users doing that or recommend it).
If there's an issue starting one of the connectors, instead of failing startup of the MM2 node, we log a message and move on. I can't imagine this is very easy for users to work with, though maybe I'm underestimating the utility of JMX and logs here.

In absence of such a mechanism, today, a user might call MirrorMaker#start() and assume that it has started correctly while in the background, connector start up might have failed (just as it did in this test scenario).

This is less concerning because the MirrorMaker class isn't part of public API; users are not expected to directly interact with any instances of this class and we do not support any use cases like that right now.
I'm wondering if, instead of a Future-based approach, we might add an internal API to the MirrorMaker class to expose the status of the connectors running on it?
A brief sketch:
public class MirrorMaker {
    public ConnectorStateInfo connectorStatus(String source, String target, String connector) {
        SourceAndTarget sourceAndTarget = new SourceAndTarget(source, target);
        checkHerder(sourceAndTarget);
        return herders.get(new SourceAndTarget(source, target)).connectorStatus(connector);
    }
}

We could then leverage this API in the DedicatedMirrorIntegrationTest suite to implement an equivalent of the waitUntilMirrorMakerIsRunning method from the MirrorConnectorsIntegrationBaseTest suite.

This is less concerning because the MirrorMaker class isn't part of public API;

It's not technically part of public API but it has a main() method which is executed by the customers. As you pointed out, today, they have to rely on logs/metrics to figure out whether MM started correctly or not.
I agree to the creation of connectorStatus() and usage in tests as you suggested, but additionally we should block the completion of main() until all connectors are online. The part about blocking could be taken up in a different PR and we can discuss the merits/demerits of doing that over there.
Meanwhile, I will add connectorStatus() and relevant changes to this PR.

@C0urante ready for your review. I have increased the timeout to 2 min. and also added the wait for mirror maker to get ready. In a separate JIRA we should talk about how we can make the startup synchronous. I have created one such JIRA at https://issues.apache.org/jira/browse/KAFKA-14773

The test failures seems unrelated (and ./gradlew unitTest pass for me locally), hence, rebasing against trunk with a hope that the gradle issues go away.

Hey @C0urante
I haven't forgotten about this PR :) I am stuck right now and would solicit your help here.
I have made the changes that you suggested but the below condition is never satisfied.
// verify that at least one task exists
&& !connectorStatus.tasks().isEmpty()

I am not very much familiar with MM code and connect framework. If you have a suggestion to fix it, I would be happy to do so, otherwise, I will probably need some time to dig into this further.

@divijvaidya No worries about the delay--looks like we're both guilty on that front 😅
I took at look at this and the issue is that the MirrorCheckpointConnector and MirrorSourceConnector instances are generating empty lists of task configs, which makes since since the former never generates tasks during these tests and the latter only generates tasks once a to-be-mirrored topic on the upstream cluster is created, which currently takes place only after our startup check has finished.
Correct me if I'm wrong: the idea here is to add granularity to test failure messages so that we can differentiate between MM2 startup issues (caused by lagginess or connector/task failures) and issues with the actual replication logic performed by MM2. If that's the case, do you think it might make sense to do the following?

Alter the"await startup" logic to fail immediately if it detects any failed connectors or tasks
Alter the "await startup" logic to operate on a per-connector-type basis, and selectively await the startup of different connector types at different points in the test (e.g., we could await the startup of the heartbeat connector in any place where we currently call waitForMirrorMakersToStart, but only await the startup of the source connector and its tasks once we've created a to-be-replicated topic on the upstream cluster)

@C0urante thank you for the suggestions. The new code should be close to what you suggested. The test passes locally for me now. Please feel free to review when you get a chance.

[Build / JDK 17 and Scala 2.13 / org.apache.kafka.common.security.oauthbearer.internals.secured.RefreshingHttpsJwksTest.testSecondaryRefreshAfterElapsedDelay()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13284/9/testReport/junit/org.apache.kafka.common.security.oauthbearer.internals.secured/RefreshingHttpsJwksTest/Build___JDK_17_and_Scala_2_13___testSecondaryRefreshAfterElapsedDelay__/)
[Build / JDK 17 and Scala 2.13 / kafka.zk.ZkMigrationIntegrationTest.[1] Type=ZK, Name=testNewAndChangedTopicsInDualWrite, MetadataVersion=3.4-IV0, Security=PLAINTEXT](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13284/9/testReport/junit/kafka.zk/ZkMigrationIntegrationTest/Build___JDK_17_and_Scala_2_13____1__Type_ZK__Name_testNewAndChangedTopicsInDualWrite__MetadataVersion_3_4_IV0__Security_PLAINTEXT/)
[Build / JDK 17 and Scala 2.13 / org.apache.kafka.trogdor.coordinator.CoordinatorTest.testTaskRequestWithOldStartMsGetsUpdated()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13284/9/testReport/junit/org.apache.kafka.trogdor.coordinator/CoordinatorTest/Build___JDK_17_and_Scala_2_13___testTaskRequestWithOldStartMsGetsUpdated__/)
[Build / JDK 11 and Scala 2.13 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationSSLTest.testSyncTopicConfigs()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13284/9/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsIntegrationSSLTest/Build___JDK_11_and_Scala_2_13___testSyncTopicConfigs__/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest.testSyncTopicConfigs()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13284/9/testReport/junit/org.apache.kafka.connect.mirror.integration/MirrorConnectorsIntegrationBaseTest/Build___JDK_8_and_Scala_2_12___testSyncTopicConfigs__/)
[Build / JDK 17 and Scala 2.13 / org.apache.kafka.controller.QuorumControllerTest.testBalancePartitionLeaders()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13284/9/testReport/junit/org.apache.kafka.controller/QuorumControllerTest/Build___JDK_17_and_Scala_2_13___testBalancePartitionLeaders__/)
[Build / JDK 11 and Scala 2.13 / org.apache.kafka.controller.QuorumControllerTest.testBalancePartitionLeaders()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13284/9/testReport/junit/org.apache.kafka.controller/QuorumControllerTest/Build___JDK_11_and_Scala_2_13___testBalancePartitionLeaders__/)
[Build / JDK 8 and Scala 2.12 / org.apache.kafka.controller.QuorumControllerTest.testBalancePartitionLeaders()](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13284/9/testReport/junit/org.apache.kafka.controller/QuorumControllerTest/Build___JDK_8_and_Scala_2_12___testBalancePartitionLeaders___2/)

Test failures are unrelated. @C0urante pinging for review when you get a chance.

I had to rebase with trunk to resolve merge conflicts. Changes in latest revision.
1\ Moved  awaitMirrorMakerStart into the test cases.
2\ Only check for one mirror maker node to start instead of all of them.
3\ Only check for connector tasks in one MM node instead of all of the nodes.
4\ Use NoRetryException to fail fast in case of a task/connector failure. (Thank you for introducing me to this nice test utility!)
5\ Task/Connector failure is already logged at ERROR level.
This test passes locally. @C0urante ready for review.

Thank you for your patience on this one @C0urante. Appreciate it!


[WIP] Add age button to "build table" widget

This PR adds an extra button to "build table" widget. The button set works similar to our current "payroll vs income" button, in the way that users could select either of them or both. When both "Under Age 65" and "Age 65 & Above" are selected, the table will render full data; when either one is selected, the respective data source would be rendered. Here's a preview of the button:

This PR needs further testing after desired data source is incorporated.

Looks good @GoFroggyRun. Thanks for putting this together. This is on hold until the necessary Tax-Calculator changes are made to provide tables with an age breakdown.


Fix "bounds on generic parameters are ignored in type aliases"

Recent nightly compilers appear to find some more ignored bounds. I adjusted the code as suggested here. About the 'static I removed, as far as I understand, this should still be enforced by the bounds on BaseAutoRefreshingProvider.
Fixed errors:
error: bounds on generic parameters are ignored in type aliases
   --> /home/travis/build/rusoto/rusoto/rusoto/credential/src/lib.rs:291:40
    |
291 | pub type AutoRefreshingProviderSync<P: ProvideAwsCredentials + 'static> =
    |                                        ^^^^^^^^^^^^^^^^^^^^^   ^^^^^^^
    |
note: lint level defined here
   --> /home/travis/build/rusoto/rusoto/rusoto/credential/src/lib.rs:4:45
    |
4   | #![cfg_attr(not(feature = "unstable"), deny(warnings))]
    |                                             ^^^^^^^^
    = note: #[deny(ignored_generic_bounds)] implied by #[deny(warnings)]
error: bounds on generic parameters are ignored in type aliases
   --> /home/travis/build/rusoto/rusoto/rusoto/credential/src/lib.rs:319:36
    |
319 | pub type AutoRefreshingProvider<P: ProvideAwsCredentials + 'static> =
    |                                    ^^^^^^^^^^^^^^^^^^^^^   ^^^^^^^
error: aborting due to 2 previous errors
error: Could not compile `rusoto_credential`.
To learn more, run the command again with --verbose.

This is great, thanks!


Fix test - text input on profile page not submitting nor saving to database

This is working now. Rohit worked on it.


Python 3 support

This is mostly the result of applying modernize, a successor of 2to3. If hand tweaked the result to remove a dependence on six.

Thank you! I'll try to make sure to keep this compatible with Python 3, but if/when I slip up and break it, feel free to yell at me ;)


Issue with 2 tabs without splits

If you have 2 tabs and activate NeoZoom in the 1st tab without any splits, then the 1st tab just closes and switches to the second tab.
The expected behavior would be to do nothing.

Thanks for the report, let me fix it now!

Really a nice catch! It's a typo, sorry if it cause some inconvenience for your work 🙇

Thanks, by the way, if there is only one tab without splits, NeoZoom opens the second tab which looks exactly the same. Of course, it's not a big issue, but probably a separate tab in such cases is not necessary.

NeoZoom opens the second tab which looks exactly the same

That is a breaking change I made days ago. Let me rethink about it. (the intention is that I was too lazy to type :tab split myself when I wanted to create a new tab)

Actually, if it breaks something, then it's fine as is. Just seems a little strange :)
Thank you for a great and useful plugin!

:lithuania: :heart: :taiwan:


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-yml-archive-non-versioning-archive/pull/292 is merged into live branch.
Auto archive content to https://github.com/opstest2/docs-yml-archive-non-versioning-archive.git

Docs Build status updates of commit eb7872c:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.redirection.json
:white_check_mark:Succeeded

docs/TOC.yml
:white_check_mark:Succeeded
View

docs/to-be-archive/level1/article3-1-1.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level1/article3-1-2.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level1/article3-1-3.yml
:white_check_mark:Succeeded
View

docs/to-be-archive/level2/article3-2-1.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level2/article3-2-2.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level2/article3-2-3.yml
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Performance Improvement: Revert Selected Ranges Should Not Replace The Entire Document

Currently, revert selected ranges applies a workspace edit that always replaces the entire document.
This does not work very well with the collapse unmodified code feature, as for a brief time the old state cannot be related to the new state (as the edit spans the entire document & diffing happens asynchronously).
It also forces the entire document to be retokenized and invalidates the bracket pair AST.
This is the code:
https://github.com/microsoft/vscode/blob/1399c189331d264ccb7d26a563d8979212163273/extensions/git/src/commands.ts#L1478-L1483
I suggest that applyLineChanges constructs the WorkspaceEdit directly, by only touching the changed lines.
I can help with that!

Thinking about it, it is probably better to remove the entire "Revert Selected Ranges" command from the git extension and make it a built-in diff editor command.

Thinking about it, it is probably better to remove the entire "Revert Selected Ranges" command from the git extension and make it a built-in diff editor command.

This sounds reasonable to me but I want to make sure that @joaomoreno has the opportunity to chime in.

The suggestion outlined here has been implemented in the "Revert Block" and "Revert Selection" actions that are available in the diff editor gutter. As a next steps, I think that we should surface these actions in the normal editor as well as look into the possibility of adding the ability to do the same for "Stage Block/Selection" and "Unstage Block/Selection".


[FEATURE] Support PEP 440 direct references to source packages

As documented https://pip.pypa.io/en/stable/cli/pip_install/#examples (point 9), it is possible to define requirements as follows, which this library cannot parse at present:
python -m pip install SomeProject@http://my.package.repo/SomeProject-1.2.3-py33-none-any.whl
python -m pip install "SomeProject @ http://my.package.repo/SomeProject-1.2.3-py33-none-any.whl"
python -m pip install SomeProject@http://my.package.repo/1.2.3.tar.gz

Validating the above with pip 22.0.4 as follows:
> python -m pip install "Jake @ https://files.pythonhosted.org/packages/d1/aa/e1be071cde813cafca797678c728b9fa686aa7d72c3e10c512ac7a5599d2/jake-2.0.0-py3-none-any.whl"
...

> pip freeze
...
jake @ https://files.pythonhosted.org/packages/d1/aa/e1be071cde813cafca797678c728b9fa686aa7d72c3e10c512ac7a5599d2/jake-2.0.0-py3-none-any.whl
...

This feature should parse the above example to a Requirement as follows:
{
   "line": "jake @ https://files.pythonhosted.org/packages/d1/aa/e1be071cde813cafca797678c728b9fa686aa7d72c3e10c512ac7a5599d2/jake-2.0.0-py3-none-any.whl",
   "editable": false,
   "local_file": false,
   "specifier": false,
   "vcs": null,
   "revision": null,
   "name": "jake",
   "uri": "https://files.pythonhosted.org/packages/d1/aa/e1be071cde813cafca797678c728b9fa686aa7d72c3e10c512ac7a5599d2/jake-2.0.0-py3-none-any.whl",
   "subdirectory": null,
   "path": null,
   "hash_name": null,
   "hash": null,
   "extras": [],
   "specs": []
}

Hello @madpah
I'm very interested by this fix and some others, and I was wondering if this is going to support the extra requirements out of the box?
For example, you can try this:
pip install 'uvicorn[standard] @ git+https://github.com/encode/uvicorn.git'

You'll see that the command works and installs everything that is specified as extra requirements.
https://github.com/encode/uvicorn/blob/0.17.6/setup.py#L54
Also if you need help I can try to work on some issues, but I am not very fluent in regex (just the basics) and it looks like your tool is using them extensively.


Can’t invite bot

Hello! Can you fix this? I can’t invite bot to any server. Can you somehow make the bot verified? Or make instructions on how to build bot myself? Thanks!

I verified the app. The issue should be fixed. Lmk if it works now

Yep! Everything works great, thanks!


Build fails

INFO: Building Homebrew
==> Tapping staticfloat/juliatranslated
Cloning into '/Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/Taps/staticfloat/homebrew-juliatranslated'...
remote: Counting objects: 5, done.
remote: Total 5 (delta 0), reused 0 (delta 0), pack-reused 5
Unpacking objects: 100% (5/5), done.
Checking connectivity... done.
Tapped 0 formulae (28 files, 21.6K)
/Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/brew.sh: line 32: /Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/ENV/scm/git: No such file or directory
/Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/brew.sh: line 32: /Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/ENV/scm/git: No such file or directory
/Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/brew.sh: line 32: /Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/ENV/scm/git: No such file or directory
/Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/brew.sh: line 32: /Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/ENV/scm/git: No such file or directory
/Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/brew.sh: line 32: /Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/ENV/scm/git: No such file or directory
/Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/brew.sh: line 32: /Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/ENV/scm/git: No such file or directory
/Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/brew.sh: line 32: /Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/ENV/scm/git: No such file or directory
/Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/brew.sh: line 32: /Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/ENV/scm/git: No such file or directory
/Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/brew.sh: line 32: /Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/ENV/scm/git: No such file or directory
/Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/brew.sh: line 32: /Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/ENV/scm/git: No such file or directory
/Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/brew.sh: line 32: /Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/ENV/scm/git: No such file or directory
/Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/brew.sh: line 32: /Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/ENV/scm/git: No such file or directory
/Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/brew.sh: line 32: /Users/solver/.julia/v0.4/Homebrew/deps/usr/Library/ENV/scm/git: No such file or directory
==> Homebrew has enabled anonymous aggregate user behaviour analytics
Read the analytics documentation (and how to opt-out) here:
  https://git.io/brew-analytics
Error: update-report should not be called directly!
==============================[ ERROR: Homebrew ]===============================

LoadError: failed process: Process(`/Users/solver/.julia/v0.4/Homebrew/deps/usr/bin/brew update`, ProcessExited(1)) [1]
while loading /Users/solver/.julia/v0.4/Homebrew/deps/build.jl, in expression starting on line 2

This happened on two different machines

brew doctor fixed this, sorry.


Windows Binary infected with Zpevdo.B?

Downloading the portable windows server yields a warning of windows defender for being infected with Win32/Zpevdo.B. Is this a scam or a false positive?
TIA Daniel

All sweet according to virustotal: https://www.virustotal.com/gui/url/d030d1d8b7250c2bed29d1f7fb81fdc95790d4c958a340510e0f5cb6fa887d1d/detection

Just saw this, it is a false positive due to using https://github.com/krogank9/exe-packager to make it portable. Viruses use similar tactics to obscure the signature of the code they are running.

How can I solve this issue ?
https://github.com/krogank9/WifiMouseServer/issues/8


invalid date

Hi,
love the chart!
but when hovering over the temperature values, I get an 'invalid date' in the label:

(it doesn't matter whether I hover over min, max, rain, ...)
I have my own weather station, uploading data to Wunderground and using the WUndergroudpws integration to show the weather data in HA.
Thanks!

Unfortunately I don’t have a personal weather station, can you provide me a snapshot of your weather entity attributes output?
Developer tools - States:
Filer on your weather entity and copy all info from the attributes. It should contain forecast.

OK, I don't see any dates there.
All forecast attributes are

they all are named like
sensor.STATIONID_forecast_ITEM_0d
sensor.STATIONID_forecast_ITEM_1n
sensor.STATIONID_forecast_ITEM_2d
sensor.STATIONID_forecast_ITEM_3n
sensor.STATIONID_forecast_ITEM_4d
sensor.STATIONID_forecast_ITEM_5n
sensor.STATIONID_forecast_ITEM_6d
sensor.STATIONID_forecast_ITEM_7n
sensor.STATIONID_forecast_ITEM_8d
sensor.STATIONID_forecast_ITEM_9n
where ITEM is summary or temperature
1-9 is a sequence, n/d stand for night/day/
sensor.STATIONID_forecast_temperature_1n is temperature forecast tonight
sensor.STATIONID_forecast_temperature_2d is temperature forecast tomorrow day
sensor.STATIONID_forecast_temperature_3n is temperature forecast tomorrow night
sensor.STATIONID_forecast_temperature_4d is temperature forecast the day after tomorrow day (friendly name seems to change to have the day (Friday) )
...

The data displayed in the chart is not from the provided sensors, can you check in the card configuration which entity is used and provide the attributes data of that entity?

ok, sorry, my bad. The entity is weather.STATIONID, in this case weather.izonne25
attributes:
temperature: 18.1
temperature_unit: °C
humidity: 58
pressure: 1007.01
pressure_unit: hPa
wind_bearing: 354
wind_speed: 0.3
wind_speed_unit: km/h
visibility_unit: km
precipitation_unit: mm
forecast:

condition: rainy
precipitation: 17.55
precipitation_probability: 100
temperature: 23
templow: 12
datetime:<PHONE_NUMBER>000
wind_bearing: SSW
wind_speed: 18
condition: rainy
precipitation: 6.48
precipitation_probability: 98
temperature: 19
templow: 15
datetime:<PHONE_NUMBER>000
wind_bearing: SW
wind_speed: 27
condition: rainy
precipitation: 0.3
precipitation_probability: 47
temperature: 23
templow: 16
datetime:<PHONE_NUMBER>000
wind_bearing: SW
wind_speed: 18
condition: rainy
precipitation: 1.84
precipitation_probability: 73
temperature: 22
templow: 15
datetime:<PHONE_NUMBER>000
wind_bearing: SW
wind_speed: 26
condition: partlycloudy
precipitation: 0
precipitation_probability: 24
temperature: 21
templow: 14
datetime:<PHONE_NUMBER>000
wind_bearing: WSW
wind_speed: 32

The problem is that the datetime attribute format is not correct, according the official Home Assistant developers documentation it should be UTC Date time in RFC 3339 format: https://developers.home-assistant.io/docs/core/entity/weather/
The best you can do is create a new issue in the Home-Assistant-wundergroundpws repository pointing them to this issue and have them passing the correct format to the datetime attribute for the forecast.

Ok, will do. Thanks for your time and effort!
On Thu, 27 Jul 2023, 07:42 mlamberts78, @.***> wrote:

The problem is that the datetime attribute format is not correct,
according the official Home Assistant developers documentation it should be UTC
Date time in RFC 3339 format:
https://developers.home-assistant.io/docs/core/entity/weather/
The best you can do is create a new issue in the
Home-Assistant-wundergroundpws
https://github.com/cytech/Home-Assistant-wundergroundpws repository
pointing them to this issue and have them passing the correct format to the
datetime attribute for the forecast.
—
Reply to this email directly, view it on GitHub
https://github.com/mlamberts78/weather-chart-card/issues/6#issuecomment-1652946216,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACWNUZM4C5TR6E7KD2WZZ2LXSH5T7ANCNFSM6AAAAAA2Y7PGPM
.
You are receiving this because you authored the thread.Message ID:
@.***>


fix: 修复自定义目录树同名节点展示异常 & 导出缺失角头 close #2455

👀 PR includes

🐛 Bugfix

[x] Solve the issue and close #2455

📝 Description

自定义行列头使用 field 生成 id, 解决同名节点引起的交互/折叠/复制等衍生问题.

- "root[&]自定义节点A"
+ "root[&]custom-node-1"

兼容刷选和复制导出, 并完善相关单测.
修复自定义行头 (平铺/树状) 复制的数据未包含角头的问题.

🖼 Screenshot

Before
After

🔗 Related issue link

🔍 Self-Check before the merge

[ ] Add or update relevant docs.
[ ] Add or update relevant demos.
[ ] Add or update test case.
[ ] Add or update relevant TypeScript definitions.

:tada: This PR is included in version @antv/s2-v2.0.0-next.13 :tada:
The release is available on:

npm package (@next dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version @antv/s2-react-v2.0.0-next.12 :tada:
The release is available on:

npm package (@next dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


How to connect spark-notebook to the existing spark-cluster

Hi, I'm trying to connect spark-notebook to the existing spark-cluster. I'm using Hadoop HDFS 2.5.1, Spark 1.6.1 with Scala 2.11. I've downloaded the corresponding version of the spark-notebook.
Started spark-notebook with
bin/spark-notebook -Dhttp.address=<IP_ADDRESS>
and then in the WebUI, created new notebook using YARN-client template, the only thing I've changed is
"spark.master": "yarn-client"
Is it enough to have this configs in order to use existing spark-cluster. I'm starting spark-notebook on the same machine where I have spark master.
Thank you.

I've been able to start spark-notebook with my spark setup(2xmasters, 2xworkers), but it's failing.
I'm running spark-notebook on one of the spark master nodes.
I'm using the following spark configuration configuration
  "customSparkConf": {
    "spark.local.dir": "/tmp/hadoop/nm-local-dir",
    "spark.app.name": "Notebook",
    "spark.driver.extraClassPath": "/opt/hadoop/spark/conf:/opt/hadoop/hadoop/etc/hadoop",
    "spark.executor.extraClassPath": "/opt/hadoop/spark/conf:/opt/hadoop/hadoop/etc/hadoop",
    "spark.master": "yarn-client",
    "spark.yarn.jar": "/opt/hadoop/spark/lib/spark-assembly-1.6.1-hadoop2.5.1.jar"
  }

I'm testing using core/Simple Spark notebook, when I start it, YARN is allocating resources, and it's being accepted, but then it's failing with
diagnostics: Application application_1460465437022_0023 failed 2 times due to AM Container for appattempt_1460465437022_0023_000002 exited with  exitCode: -1000 due to: java.io.FileNotFoundException: File file:/tmp/hadoop/nm-local-dir/spark-2765a4db-a712-4230-b4ee-94923fe846a4/__spark_conf__5039480919166727598.zip does not exist
.Failing this attempt.. Failing the application.

So there is no /tmp/hadoop/nm-local-dir/spark-2765a4db-a712-4230-b4ee-94923fe846a4/__spark_conf__5039480919166727598.zip in worker nodes, but it's present in the master node where I started spark-notebook.
Is it an error in the configurations?
Thank you

It looks like there is an issue with yarn localizer,
2016-04-12 16:59:22,944 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: DEBUG: FAILED { file:/tmp/spark-a9de04a1-556f-4700-b735-8125070ecf08/__spark_conf__2844684471127306154.zip,<PHONE_NUMBER>000, ARCHIVE, null }, File file:/tmp/spark-a9de04a1-556f-4700-b735-8125070ecf08/__spark_conf__2844684471127306154.zip does not exist
this are the logs of the nodemanager


Replace nodeSelector with nodeAffinity master role

Label node-role.kubernetes.io/master has traditionally no value
associated inherited from kubeadm deplyments but some platforms use
"true" as value. Use node affinity instead of node selector ignoring the
value of the label.

do we have the same issue for node-role.kubernetes.io/worker ?

do we have the same issue for node-role.kubernetes.io/worker ?

No to the same extent. node-role.kubernetes.io/worker label is not standard, kubeadm does not set it for example. I guess openshift does. But it is being replaced by nodeselector from node policies and/or operator config so not really effective.

It might be good to do the same change for webhooks (webhook and operator-webhook) which run on master nodes (although they are not supported to run on k8s yet).

do we have the same issue for node-role.kubernetes.io/worker ?

No to the same extent. node-role.kubernetes.io/worker label is not standard, kubeadm does not set it for example. I guess openshift does. But it is being replaced by nodeselector from node policies and/or operator config so not really effective.

ack+

/cc @pliurh @SchSeba

It might be good to do the same change for webhooks (webhook and operator-webhook) which run on master nodes (although they are not supported to run on k8s yet).

This is already included in this PR or am I missing something else?

/lgtm


Duplicate OSB create/delete requests

I've noticed duplicate OSB requests for a single change in ServiceInstance in different scenarios:

5 deprovisioning requests reported there
2 create requests (logs from ups-broker):

I1221 23:46:09.728136       1 server.go:111] Get Service Broker Catalog...
I1221 23:46:09.728152       1 controller.go:57] Catalog()
I1221 23:54:58.838849       1 server.go:137] CreateServiceInstance 434423cf-8a6e-482a-937e-ddf95f992486...
I1221 23:54:58.839113       1 controller.go:87] CreateServiceInstance()
I1221 23:54:58.839195       1 controller.go:118] Created User Provided Service Instance:
&{434423cf-8a6e-482a-937e-ddf95f992486 0xc4200fd1c0}
I1221 23:54:58.937091       1 server.go:137] CreateServiceInstance 434423cf-8a6e-482a-937e-ddf95f992486...
I1221 23:54:58.937207       1 controller.go:87] CreateServiceInstance()
I1221 23:54:58.937225       1 controller.go:118] Created User Provided Service Instance:
&{434423cf-8a6e-482a-937e-ddf95f992486 0xc4200fd1c8}

By looking at the Service Catalog logs, it seems that there is an issue with picking up a stale object from the queue (before it is synchronized) which leads to duplicate processing:
I1221 23:54:58.942144       1 round_trippers.go:436] PUT https://<IP_ADDRESS>:443/apis/servicecatalog.k8s.io/v1beta1/namespaces/test-ns/serviceinstances/ups-instance123/status 409 Conflict in 4 milliseconds
I1221 23:54:58.942444       1 request.go:836] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Operation cannot be fulfilled on serviceinstances.servicecatalog.k8s.io \"ups-instance123\": the object has been modified; please apply your changes to the latest version and try again","reason":"Conflict","details":{"name":"ups-instance123","group":"servicecatalog.k8s.io","kind":"serviceinstances"},"code":409}
E1221 23:54:58.942585       1 controller_instance.go:1844] ServiceInstance "test-ns/ups-instance123": Failed to update status: Operation cannot be fulfilled on serviceinstances.servicecatalog.k8s.io "ups-instance123": the object has been modified; please apply your changes to the latest version and try again
I1221 23:54:58.942617       1 controller.go:237] Error syncing ServiceInstance test-ns/ups-instance123: Operation cannot be fulfilled on serviceinstances.servicecatalog.k8s.io "ups-instance123": the object has been modified; please apply your changes to the latest version and try again

This conflict is a "normal" situation for Kubernetes-style APIs with eventual consistency and idempotency, but it could be a problem for OSB brokers who don't necessarily guarantee the idempotence semantics.

My guess is that the problem is that we pick up the events from the queue "too early" without giving enough time to receive the latest version of the instance we just updated processing a previous event:

ServiceInstance is created by the user: Event e1, object version o1.
Service Catalog picks up the event e1 from the queue and starts processing it.
Service Catalog updates the status of the ServiceInstance to "provisioning": Event e2, object version o2.
Service Catalog successfully sends the OSB request and receives a success response. It marks the ServiceInstance with Ready: True condition. Event e3, object version o3.
Service Catalog finishes the processing.
Service Catalog picks up the e2 event, and in the cache there is o2 version of the object, which still says that the provisioning has not been done, so it goes and sends the OSB request again. It tries to send a status update with Ready: True but fails with a conflict (because the update is based on the outdated o2 version of the object). This step can repeat several times depending on the number of events we have in the queue.
Service Catalog finally picks up the e3 event with o3 version in the cache, and this time the processing ends with no-op.

The naive solution for this problem I see is to add an artificial delay in between processing events of a single ServiceInstance instance (and other resource types too).
We can use a workqueue.DelayingInterface instead of workqueue.Interface for that: https://github.com/kubernetes/client-go/blob/master/util/workqueue/delaying_queue.go#L32
and invoke queue.AddAfter(key, <delay>) instead of queue.Add(key) in controller_instance.go
It will mean that there will be an artificial pause between making a change in the ServiceInstance spec and processing it, but I think it will significantly reduce the likelihood of duplicate requests (if the delay will be big enough for syncing the cache with the latest status).
If there are no other suggestions, I can give it a try.

After changing the code to
c.instanceQueue.AddAfter(key, 3*time.Second)

(adding a delay of 3 seconds for adding ServiceInstance key to the queue), I cannot reproduce the issue anymore:
I1222 05:57:17.028697       1 server.go:111] Get Service Broker Catalog...
I1222 05:57:17.028753       1 controller.go:57] Catalog()
I1222 05:57:49.550884       1 server.go:137] CreateServiceInstance a25c8a51-f4b9-43e8-9a1c-47a9b3ac5d3d...
I1222 05:57:49.551098       1 controller.go:87] CreateServiceInstance()
I1222 05:57:49.551121       1 controller.go:118] Created User Provided Service Instance:
&{a25c8a51-f4b9-43e8-9a1c-47a9b3ac5d3d 0xc42000e298}
I1222 05:58:40.238310       1 server.go:166] RemoveServiceInstance a25c8a51-f4b9-43e8-9a1c-47a9b3ac5d3d...
I1222 05:58:40.238326       1 controller.go:138] RemoveServiceInstance()
I1222 05:59:07.547323       1 server.go:137] CreateServiceInstance 1264669f-a334-448f-95eb-5e71a52e1c1e...
I1222 05:59:07.547450       1 controller.go:87] CreateServiceInstance()
I1222 05:59:07.547473       1 controller.go:118] Created User Provided Service Instance:
&{1264669f-a334-448f-95eb-5e71a52e1c1e 0xc42000e300}
I1222 05:59:19.636256       1 server.go:166] RemoveServiceInstance 1264669f-a334-448f-95eb-5e71a52e1c1e...
I1222 05:59:19.636270       1 controller.go:138] RemoveServiceInstance()
I1222 06:01:32.648728       1 server.go:137] CreateServiceInstance 3fe4302c-256a-43bf-9b03-5bde6845602f...
I1222 06:01:32.648801       1 controller.go:87] CreateServiceInstance()
I1222 06:01:32.648807       1 controller.go:118] Created User Provided Service Instance:
&{3fe4302c-256a-43bf-9b03-5bde6845602f 0xc420078250}
I1222 06:01:44.736378       1 server.go:166] RemoveServiceInstance 3fe4302c-256a-43bf-9b03-5bde6845602f...
I1222 06:01:44.736475       1 controller.go:138] RemoveServiceInstance()

so it seems to work?
If we consider this a valid fix, I can send a PR to make this change in all our controller queues.

The MutationCache (see mutation_cache.go) interface could be another option to resolve this issue: every time we modify ServiceInstance we could update it in this cache, and it will be up-to-date by the time we start processing the next event from the queue. It's not widely used in the Kubernetes codebase, so not sure if it's a recommended to use interface. Also not clear if it's compatible with the Lister interface we currently use to fetch objects from sharedIndexInformer.

This is a duplicate of #1363.
See #1213 for the introduction of this bug.

@staebler I agree that the cause of the issue is the same, but it wasn't mentioned that this behavior leads to duplicate OSB requests. As I mentioned above, it's probably okay to have duplicate processing within Kubernetes world, but OSB spec doesn't guarantee idempotency so the provisioning/deprovisioning could break because of that.

@nilebox The OSB spec does guarantee idempotency. For example, if the same provision request is made again after the first one is successful, the spec states that the broker must respond with a 200 OK. That is why there has not been a greater push to resolve this issue in service catalog.

@staebler OSB doesn't guarantee a deterministic behaviour on concurrent requests though. If Service Catalog issues a duplicate request while initial request is still in progress, it might lead to either 200 or 422, if I understand correctly. See https://github.com/openservicebrokerapi/servicebroker/blob/master/spec.md#blocking-operations
But I personally agree that brokers should be idempotent.


Add communication entity

Resolve #2

Codecov Report

Merging #44 (91bf80a) into main (fb9f432) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##              main       #44   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           65        67    +2
  Lines          993      1024   +31
=========================================
+ Hits           993      1024   +31

Impacted Files
Coverage Δ

aioalfacrm/client.py
100.00% <100.00%> (ø)

aioalfacrm/entities/__init__.py
100.00% <100.00%> (ø)

aioalfacrm/entities/communication.py
100.00% <100.00%> (ø)

aioalfacrm/managers/__init__.py
100.00% <100.00%> (ø)

aioalfacrm/managers/communication.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fb9f432...91bf80a. Read the comment docs.


add filter for blacklist/whitelist.

What this PR does:
add filter for http blacklist/whitelist.
Which issue(s) this PR fixes:

Fixes #14
Special notes for your reviewer:
When IP address or backend application name are on the blacklist, the call will stop, response 403.
When IP address or backend application name are not on the whitelist, the call will stop, response 403.
Does this PR introduce a user-facing change?:
There is no.

Codecov Report

Merging #38 (c6d09ec) into develop (eb9ea75) will decrease coverage by 0.63%.
The diff coverage is 12.50%.

@@             Coverage Diff             @@
##           develop      #38      +/-   ##
===========================================
- Coverage    39.18%   38.54%   -0.64%
===========================================
  Files           18       18
  Lines          786      799      +13
===========================================
  Hits           308      308
- Misses         456      469      +13
  Partials        22       22

Impacted Files
Coverage Δ

pkg/context/http/context.go
10.71% <0.00%> (-3.24%)
:arrow_down:

pkg/proxy/listener.go
23.17% <0.00%> (ø)

pkg/proxy/http.go
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eb9ea75...c6d09ec. Read the comment docs.


⚠ Bofrost.fr has degraded performance

In f18f95b, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1508 ms

Resolved: Bofrost.fr performance has improved in e79aa4e after 11 minutes.


running apt-get upgrade breaks cluster when using 0.13.0

Is this a request for help?: YES

Is this an ISSUE or FEATURE REQUEST? (choose one): ISSUE

What version of acs-engine?: 0.13.0

Orchestrator and version (e.g. Kubernetes, DC/OS, Swarm)
Kubernetes
What happened:
Deployed new cluster using the 0.13.0 engine, as part of the housekeeping, ran a sudo apt-get update followed by upgrade on the master node at which point running kubectl (anything) just hangs.
I confirmed kubectl was OK before patching the linux vm as this is the second cluster I have killed by doing this.
What you expected to happen:
Not to wreck the cluster!
How to reproduce it (as minimally and precisely as possible):
Exactly as above, just patch the vm and it should be useless.
Anything else we need to know:

Hi @marty2bell we identified a regression in v0.13.0 (and released v0.13.1 w/ the fix) that didn't startup etcd on restart. Is this the root cause? If so, you can do one of the following:

build a new cluster (if that's an option) w/ v0.13.1
patch the existing cluster by running sudo systemctl enable etcd on all master and agent nodes

That's great thanks.  I'll patch for now and upgrade the engine for new builds.....
Cheers
Marty


Update asynchronously

Current version of kele--update is updated naively, i.e. synchronously. This creates noticeable hang on, say, changing the current context, due to overhead associated with loading the kubeconfig file, parsing to YAML, performing the necessary value extractions, etc.
Given that the update does not strictly need to be a blocking operation, let's make this async to keep things snappy. 🚀

This is "soft-blocked" on jwiegley/emacs-async#164.

With e009f9792565a1251a1d53b0296746c9ba5eafce, we can potentially circumvent the aforementioned issue.


Office 365 application is not shown when a policy includes other applications in addition to office 365

Nice tool but it requires a paid license!
I was able to do my first run and noticed that Office 365 application is not shown when a policy includes other applications in addition to office 365. Office 365 shows okay when it is the only app in the CA policy.
Is it a bug or some settings that I missed?

Are you referring to Syncfusion? They do have a great community license that you can get depending on the size of the company.
For O365, yes I only show the icon if it's the only app selected. If but it is listed below under selected apps along with the rest of the apps.

Yes, I was referring to syncfusion license. I will try community license in my personal capacity.
O365 icon does not show up under app section when O365 app is combined with other apps. Didn't work for me when I ran with the latest code.

+1 for this - also was affected by it. O365 by itself works just fine, but O365 + other applications only shows the other applications and not O365. This was with the Desktop Client.


Atom freezes on  tag

When I type in  into a .jsx or .js page and then click my mouse anywhere in the document.  Atom's editor immediately freezes and Atom has to be restarted.  This just begun when upgrading to 2.23.0.

I can't recreate this on Atom 1.7.4 & language-babel 2.23.0

Do you have a sample of the code? Preferably all of it. Place it in a gist if you want.
What Atom version/platform?

Same versions you have below.  On Mac OS X 10.11.5.
Here is the code:
var React = require('react');
var ReactDOM = require('react-dom');
import {Route, Router, IndexRoute, hashHistory} from 'react-router';
ReactDOM.render(

,
document.getElementById('app')
);
If I even paste this in a file and then click anywhere in the file, it immediately freezes.
Thank you, btw, for responding!
Kevin
From: Graham<EMAIL_ADDRESS>Reply-To: gandm/language-babel<EMAIL_ADDRESS>Date: Wednesday, May 25, 2016 at 11:37 AM
To: gandm/language-babel<EMAIL_ADDRESS>Cc: Kevin Pompei<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [gandm/language-babel] Atom freezes on  tag (#192)
I can't recreate this on Atom 1.7.4 & language-babel 2.23.0
[mage removed by sender. jsx]https://cloud.githubusercontent.com/assets/2313237/15549814/52912992-22a6-11e6-8b65-16728b046dff.gif
Do you have a sample of the code? Preferably all of it. Place it in a gist if you want.
What Atom version/platform?
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHubhttps://github.com/gandm/language-babel/issues/192#issuecomment-221647591

Kevin, I cut and pasted you code into my editor with no issues. The only thing I can suggest is that it isn't this package causing the problem or when it was updated it somehow got corrupt. Atom also changed versions from 1.7.3 to 1.7.4, so maybe that got corrupted during the update. You could uninstall language-babel, stop atom and then remove ~/.atom/compile-cache, restart atom and re-install language-babel. If that doesn't work you can try installing an older version e.g. apm<EMAIL_ADDRESS>

Yep, that did it.  Just uninstalling and reinstalling the plugin fixed it.  Thank you so much for taking the time to respond.  It is such a useful plugin!
Thanks Graham!
From: Graham<EMAIL_ADDRESS>Reply-To: gandm/language-babel<EMAIL_ADDRESS>Date: Wednesday, May 25, 2016 at 2:29 PM
To: gandm/language-babel<EMAIL_ADDRESS>Cc: Kevin Pompei<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [gandm/language-babel] Atom freezes on  tag (#192)
Kevin, I cut and pasted you code into my editor with no issues. The only thing I can suggest is that it isn't this package causing the problem or when it was updated it somehow got corrupt. Atom also changed versions from 1.7.3 to 1.7.4, so maybe that got corrupted during the update. You could uninstall language-babel, stop atom and then remove ~/.atom/compile-cache, restart atom and re-install language-babel. If that doesn't work you can try installing an older version e.g. apm<EMAIL_ADDRESS>—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHubhttps://github.com/gandm/language-babel/issues/192#issuecomment-221697315

Kevin, thanks for letting me know. I'll close the issue.


Changed Topology::add_block and dependents to return a Result<usize>

This enables propagation of an error when insatance_name is specified, but is not unique in Flowgraph.
Closes #169

@bastibl This is a pretty extensive change. Let me know if you have second thoughts.

Thanks!

For future cleanup: https://github.com/FutureSDR/FutureSDR/issues/175


How to make a rewrite?

The docs says that is possible to use a rewrite, but there isn't any example on how to achieve that
CloudFront Functions Docs
I tried to change the request, in the test tab it works but no in the cloudfront distribution.
Code:
  if (host.startsWith("www.")) {
    host = host.replace("www.", "");
    request.headers.host.value = host;
    return request;

  }

Test in cloudfront Function:

Test against distribution URL:
502 ERROR
The request could not be satisfied.
The CloudFront function tried to add, delete, or change a read-only header. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.

I ended with this code for all my needs:
function handler(event) {
  var shouldRedirect = false;
  var request = event.request;
  var queryString = queryStringObjectToString(request.querystring);
  var uri = request.uri;
  var regex = /[A-Z]/g;
  var headers = request.headers;
  var host = headers.host.value;
  var last_uri_character = uri.slice(-1);

  // If the URI have a dot we assume is a file and we don't redirect or rewrite
  if (uri.includes(".")) {
    return request;
  }

  // If the URI contains a capital letter, rewrite to the lowercase version
  if (regex.test(uri)) {
    uri = request.uri.toLowerCase();
    request.uri = uri;
  }

  // If the host is www we redirect to the naked domain
  if (host.startsWith("www.")) {
    host = host.replace("www.", "");
    delete headers.host;
    shouldRedirect = true;
  }

  // If the URI ends with a slash, redirect to the non-slash version
  if (last_uri_character !== "/") {
    uri = uri + "/";
    shouldRedirect = true;
  }

  // If we should redirect, do it
  if (shouldRedirect) {
    var url = "https://" + host + uri + queryString;
    var response = {
      statusCode: 301,
      statusDescription: "Permanently Moved",
      headers: { location: { value: url } },
      cookies: request.cookies,
    };
    return response;
  }
  return request;
}

/**
 * @description: Converts an event.request.querystring object back to a query string
 * @date 2022-11-28
 * @param {object} queryStringObject: Query string object as provided by event.request
 * @return {string}:                  A normalised query string in the form ?key1=valn&key2=val2...
 */
function queryStringObjectToString(queryStringObject) {
  // Convert the query string object to an array of entries and reduce to a single string
  return (
    Object.entries(queryStringObject)
      .reduce((p, q) => {
        if (!q[1].multiValue) {
          // Process a single key/value property
          return (p += q[0] + "=" + q[1].value + "&");
        } else {
          // Process multiValue properties. E.g arrays
          return (p +=
            q[1].multiValue.map((v) => q[0] + "=" + v.value).join("&") + "&");
        }
      }, "?")
      // Remove the trailing ampersand
      .slice(0, -1)
  );
}

Thanks.


velum external pillar does not load

When trying to access the pillar items provided via the external velum pillar, I get no data. The salt-master container log contains the following:
{"log":"16:00:50,353 [PROFILE ] Time (in seconds) to render '/srv/pillar/cloud.sls' using 'yaml
' renderer: 0.00510215759277\n","stream":"stderr","time":"2018-02-27T16:00:50.354028037Z"}
{"log":"16:00:50,370 [INFO    ] Loaded velum pillar module\n","stream":"stderr","time":"2018-02
-27T16:00:50.37046686Z"}
{"log":"16:00:50,513 [DEBUG   ] Failed to import module kubernetes:\n","stream":"stderr","time"
:"2018-02-27T16:00:50.513395674Z"}
{"log":"Traceback (most recent call last):\n","stream":"stderr","time":"2018-02-27T16:00:50.513
42746Z"}
{"log":"  File \"/usr/lib/python2.7/site-packages/salt/loader.py\", line 1333, in _load_module\
n","stream":"stderr","time":"2018-02-27T16:00:50.513432637Z"}
{"log":"    mod = imp.load_module(mod_namespace, fn_, fpath, desc)\n","stream":"stderr","time":
"2018-02-27T16:00:50.513437312Z"}
{"log":"  File \"/usr/lib/python2.7/site-packages/salt/modules/kubernetes.py\", line 67, in \u0
03cmodule\u003e\n","stream":"stderr","time":"2018-02-27T16:00:50.513441432Z"}
{"log":"    from kubernetes.client import AppsV1beta1Deployment\n","stream":"stderr","time":"20
18-02-27T16:00:50.513446319Z"}
{"log":"  File \"/usr/lib/python2.7/site-packages/salt/modules/kubernetes.py\", line 67, in \u0
03cmodule\u003e\n","stream":"stderr","time":"2018-02-27T16:00:50.51345018Z"}
{"log":"    from kubernetes.client import AppsV1beta1Deployment\n","stream":"stderr","time":"20
18-02-27T16:00:50.513454406Z"}
{"log":"ImportError: No module named client\n","stream":"stderr","time":"2018-02-27T16:00:50.51
3458254Z"}
{"log":"16:00:50,631 [DEBUG   ] Module DSC: Only available on Windows systems\n","stream":"stde
rr","time":"2018-02-27T16:00:50.632140745Z"}
{"log":"16:00:50,636 [DEBUG   ] Module PSGet: Only available on Windows systems\n","stream":"st
derr","time":"2018-02-27T16:00:50.637058699Z"}
{"log":"16:00:50,970 [DEBUG   ] Could not LazyLoad redis.get_key: 'redis.get_key' is not availa
ble.\n","stream":"stderr","time":"2018-02-27T16:00:50.970732683Z"}
{"log":"16:00:50,973 [DEBUG   ] Could not LazyLoad velum.ext_pillar: 'velum.ext_pillar' is not
available.\n","stream":"stderr","time":"2018-02-27T16:00:50.974088542Z"}
{"log":"16:00:50,973 [CRITICAL] Specified ext_pillar interface velum is unavailable\n","stream"
:"stderr","time":"2018-02-27T16:00:50.9741117Z"}

The __virtual__ function of velum.py is obviously executed (hence the "Loaded velum pillar module"), but it doesn't work. That's with salt 2016.11.4 as provided by SLE12.

Does running a salt '*' saltutil.sync_all refresh=True on the salt-master improve the situation? We do that as one of the first steps of the orchestration and from that point on the Specified ext_pillar interface velum is unavailable error goes away.

Still getting "[CRITICAL] Specified ext_pillar interface velum is unavailable". Anything else I could try?

If I copy /usr/share/salt/kubernetes/salt/_pillar/velum.py to /usr/lib/python2.7/site-packages/salt/pillar/, the velum pillar works. It's a little odd, because the "Loaded velum pillar module" message indicates that it did find the file.

Sorry for the delay, running salt-run saltutil.sync_pillar on the salt-master makes the error disappear?

That seem to do the trick, thanks.

Thank you.

Hang on, does that mean that salt runner is executed during orchestration? salt '*' saltutil.sync_all refresh=True didn't actually help.
And come to think of it, that would be too late in the process anyway for public cloud support, as it requires the velum pillar to be operational for creating nodes, i.e. before orchestration. Would it be ok if we added that salt runner call right before creating cluster nodes?

Yes, because sync_pillar is somewhat special:

saltutil.sync_all: https://github.com/saltstack/salt/blob/140ef4d6b9c188adb715c47e6fbf832bc0b47a14/salt/modules/saltutil.py#L663-L722
saltutil.sync_pillar: https://github.com/saltstack/salt/blob/140ef4d6b9c188adb715c47e6fbf832bc0b47a14/salt/modules/saltutil.py#L628-L660

If you see, both check that __opts__['file_client'] == 'local'. What we do on the orchestration is to run saltutil.sync_pillar in the context of the salt-master:
https://github.com/kubic-project/salt/blob/292b025681915a6ad4d7378624385d231f2f21d4/salt/orch/kubernetes.sls#L34-L41
(note the salt.runner there). So, if you need this before calling to the orchestration I'd say we need to call to saltutil.sync_pillar with a runner client using the salt-api. That way, we'd force this sync_pillar way before calling to the orchestration and you can work with the pillar normally.

Please note: I'm suggesting we call to saltutil.sync_pillar using the salt-api in addition to keeping the snippet above on the orchestration (it's still good to have it there).

Also,

Would it be ok if we added that salt runner call right before creating cluster nodes?

I don't know what does your workflow look like. In any case, we can talk about that if you are interested, but I'd say on a certain place in Velum (after some settings have been added by the user) looks a good place. Because I'm missing some context regarding the statement requires the velum pillar to be operational for creating nodes.

 velum external pillar does not load

Great, thanks for the information. I'll open a pull request for the sync_pillar call.


Knative Eventing - adjustment script to upstream CI jobs

This PR refers dropping off of Knative Eventing jobs: https://github.com/knative/infra/pull/497/files#diff-8de52f6a751c3e2c071ef708fdae694043a7cf4b9d173bb7e2e2e6b51c0483e4
Added updated adjustment scripts to run Knative Eventing (main, release-1.15 & release-1.16) testing CI jobs.

/lgtm

/approve


Improve storage config of server

We need to ensure that the server has sufficient space for the data it will be downloading to do the plotting, and ensure that storage is part of the EC2 instance rather than the docker container.

Based on Alex's comment and the fact that we're loading the data through goofys streaming, this issue is no longer relevant so I am closing the issue.


CFEP-18: Packaging static libraries

At the moment, most binary packages that include libraries only ship their shared version.
Some packages though also include the static version or only ship the static version.
At runtime the static archives are not needed and thus are wasted space to the end user.
In addition, our tooling (conda/conda-build/the bots) is also geared towards maintaining a set of dynamically linked packages.
This CFEP should serve as a document to make it clearer how to deal with static libraries.

What level of support are we planning on providing to the static libs? Should be bot be rebuilding downstreams?

What is the user experience when a static lib moves and downstreams aren't rebuilt?

What level of support are we planning on providing to the static libs? Should be bot be rebuilding downstreams?

The lowest possible. One of the goals of this proposal is to make sure developers understand that static libraries are a second-class citizen. Thus I would for now skip any bot automation. I explicitly excluded Go / Rust / Haskell as there it might make sense to have a bot for "world rebuilds" in future but that should be a separate endeavour.

What is the user experience when a static lib moves and downstreams aren't rebuilt?

It's the same as if the lib wasn't updated at all. The used version stays the same.

I think this is generally fine — in principle there are some UX issues to think about (your software build broke because it depended on libblah.a and the libblah package suddenly stopped providing that file) but I am pretty confident that they are rare and only affect power users, and the setup we would be moving to would align well with pretty much every Linux distro out there.

I expect them to happen quite a bit when we remove static libraries. Thus I wrote this CFEP before even broadly removing the static builds from the feedstock I maintain, that there is some place to point people at on what we as a community actually aim for.

My only big question is, what's our Windows policy? I only have a poor understanding of library distribution for Windows, but it seems that depending on a package's heritage and/or build system either static or shared libraries may be preferred. (Along with disagreements about .a vs .lib, etc. I don't believe that we have a very uniform policy about how to package compiled Windows libraries in general, and even if we did it would be hard to apply uniformly? Different build systems seem to strongly prefer different schemes, I think.)

We should be less strict on Windows builds. There are a lot libraries that only work with static linkage as the upstreams didn't take care to define the correct __dllexport lines. If a build works fine with shared linkage, it should stick to only providing those. If a build doesn't yet work really with static linkage on Windows, we shouldn't force the maintainer of the feedstock to write a huge patch to make the linkage work. There is already a lot of hassle and not that much community support for packaging things on Windows.
One additional caveat on Windows is though that there are several types of static libraries. In addition to every shared library (bin\name.dll) there is always accompanying (static) import library (lib\name.dll). Sometimes the static build of a library has the same name as the import library, sometimes it is with a _static suffix, i.e. lib\name_static.lib. The suffix .lib is though only used when you are working with the MSVC-based toolchain. If you use the MinGW-based toolchain, the import library has a .dll.a suffix (though import libraries are necessarily needed for MinGW) and the static archives have solely a .a suffix, omitting the .dll part.

Incorporated the review comments and added an explicit section about Windows.

This looks good to me

@conda-forge/core
This PR falls under the CFEP Approval policy, please vote and/or comment on this PR.
This PR needs 60% of core to vote yea to pass.
To vote please leave Approve (yea) or Request Changes (nay) reviews.
If you would like changes to the current language please leave a comment or push to this branch.
This vote will end on 2020-07-08.

I'd like to abstain on this one -- I don't understand the details well enough to weigh in. [If needed to get us to  the voting threshold I'll vote yes]

Thanks @xhochy - great work here!

Thanks for the votes with 15 approvals this is above the 60% percent threshold and thus passes.


Added kubesimplify links for social platforms.

Also, the discord invite link is permanent.
Fix #21

Also, the discord invite link is permanent.

LGTM!
thank you for the contribution


Commercial Use

Just kindly want to ask is OpenLLaMA eligible for commercial use (both the code and the pre-trained weights)? The Apache-2.0 license seems Ok but I just to confirm it with you guys. Thanks.

Just kindly want to ask is OpenLLaMA eligible for commercial use (both the code and the pre-trained weights)? The Apache-2.0 license seems Ok but I just to confirm it with you guys. Thanks.

Yes it is okay.

Just kindly want to ask is OpenLLaMA eligible for commercial use (both the code and the pre-trained weights)? The Apache-2.0 license seems Ok but I just to confirm it with you guys. Thanks.

Yes it is definitely okay for commercial use and this is one of the important goals of this project.


Flops: 1.668T, Params:43.33M????

I use the code provided by your github. Thop shows the flops and params are 1.668T and 43.331M, so which model is 10.8M(in your paper)?

I also have the same question.


Cleaning is not configurable enough

The set of allowed tags and merge tags is hardcoded and cannot be changed.
Ideally, one would like to be able to set those once as default or per cleanse call.

You can actually edit the list of allowed tags by changing the value of the class attribute.
For example, if you wish to add some extra tags for a particular instance::
additional_allowed_tags = {
    'h1':(),
    'h4':(),
    'h5':(),
    'h6':(),
    'i':(),
    'div':(),
}
cleanse = Cleanse()
cleanse.allowed_tags.update(additional_allowed_tags)

Note that since it is a class attribute, you can also edit the value of the class (so all future instances will have the new value)::
additional_allowed_tags = {
    'h1':(),
    'h4':(),
    'h5':(),
    'h6':(),
    'i':(),
    'div':(),
}
Cleanse.allowed_tags.update(additional_allowed_tags)


Install Failed: Cannot bind argument to parameter 'Path' because it is null.

Join-Path : Cannot find drive. A drive with the name 'U' does not exist.
I think this is an issue with how Join-Path is being used.
See: https://chrisoldwood.blogspot.com/2014/09/join-path-fails-when-local-drive-letter.html

Very interesting, thanks for providing that link! In this case, if the drive doesn't exist, even if the join-path worked the functions would still fail right? So is the suggested solution here to give a better warning or ask the user to specify an existent path?

I am seeing this same issue.


complete modify example

Hi core ol3 coders
Thanks for this great tool!
I have a hard time getting this working:

There is a vector layer in the map with own data
Users can select features, create new ones and modify existing ones
Whenever a new feature is created or an existing one modified, an event is triggered that can be used to update the data in the database or geojson-file

The hard part is getting the interactions for select, modify and draw to work together on the same source and get a "modified" event when modifying.
It seems to me that this is a very typical use case that many people will try to implement.
Please consider adding such an example.

Want to back this issue? Place a bounty on it! We accept bounties via Bountysource.

Pull requests for improved documentation or examples are always welcome.


feat add confluent_schema

Description of your changes
Add support for confluent_schema resource

I have:

[x] Read and followed Crossplane's contribution process.
[ ] Run make reviewable test to ensure this PR is ready for review.

I have been unable to run make reviewable test due to the issue detailed here
How has this code been tested

This has been tested by creating a schema in confluent schema registry

@dmvariphy are you maintaining this repo? Apologies if not, I just grabbed your handle off of the last commit and thought it might be worth a try.

Hey @daviddyball  - I'm not a maintainer of this repo, but I believe @jaylevin is.


Pass existing server_side_parameters to session connection wrapper and use to configure SparkSession. #691

resolves #690
Description
Pass existing server_side_parameters to session connection wrapper and use to configure SparkSession.
Checklist

[x] I have read the contributing guide and understand what's expected of me
[x] I have signed the CLA
[x] I have run this code in development and it appears to resolve the stated issue
[x] This PR includes tests, or tests are not required/relevant for this PR
[ ] I have opened an issue to add/update docs, or docs changes are not required/relevant for this PR
[x] I have run changie new to create a changelog entry

Prefer this PR over #691. @alarocca-apixio : We took your commits and touched up the code so that it can be merged.

Works on my end:
cat ~/.dbt/profiles.yml
dbt_tabular:
  outputs:
    dev:
      method: session
      schema: dbt_tabular
      type: spark
      host: NA
      server_side_parameters:
        "spark.driver.memory": "2g"
  target: dev

I can see that this is being picked up by the process:
ps aux | grep -i spark
fokkodriesprong  11191 150.0  0.4 413414240 269024 s008  S+   11:14AM   0:01.99<EMAIL_ADDRESS>-cp /opt/homebrew/lib/python3.9/site-packages/pyspark/conf:/opt/homebrew/lib/python3.9/site-packages/pyspark/jars/* -Xmx2g -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djdk.reflect.useDirectMethodHandle=false org.apache.spark.deploy.SparkSubmit --conf spark.driver.memory=2g --conf spark.sql.catalogImplementation=hive pyspark-shell
fokkodriesprong  11203   0.0  0.0 408626896   1312 s010  S+   11:14AM   0:00.00 grep --color=auto --exclude-dir=.bzr --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.hg --exclude-dir=.svn --exclude-dir=.idea --exclude-dir=.tox -i spark

We can see that --conf spark.driver.memory=2g is set.

@Fleid: This PR is ready to be merged

@colin-rogers-dbt: Could you fix the CI?

This looks ready to merge but I think we should add a functional test case (will need to think where/how we can) and update our unit tests like #577

also is this dependent on #577 ?


Playbook - Isolate an Azure VM

Requirements

Create an Azure Logic Apps playbook for Azure Sentinel that includes the following workflow:

Triggers on an Azure Sentinel alert.
If the alert has host(s), isolate the respective machine(s)/host(s)
Update the incident associated with the alert with the isolated VMs and outcome of each action.

Validate the playbook works as expected per the functional requirement mentioned above.
Submit the playbook as a GitHub Pull Request per the playbook contribution guidance.
Attach a screenshot of the working playbook with your submission.
Provide a readme with the steps with your submission to document any prerequisite steps required before running this playbook.

The first submission that meets the requirements gets the reward. Rewards are subject to Microsoft terms and conditions.
Hint: Feel free to leverage HTTP Connector for Azure Logic Apps as needed.
Helpful resources

Azure Sentinel Playbook documentation
Playbook contribution guidance
Create Azure Logic Apps Playbook

Resolved by https://github.com/Azure/Azure-Sentinel/pull/532


WIP: feat(co-sched): implement coscheduler

TODO:

[ ] integrate 1.20 and 1.18 version of co-scheduler
[ ] verify compatibility on both version
[ ] complete test files

update go mod: k8s=>20.7, container-runtime=>0.8.3
implement coscheduler

commit #4fb1d7e updated go mod and complete unit test

Gang Feature Result:
Scheduler Log:

Scheduling Result:

coscheduler blocks gang from scheduling, while a single pod can be scheduled

@diaozhongpu add unit tests for this gang-scheduler implementation.

lgtm


Adding a timeout to all requests calls

There  is no default timeout value for the python requests module - some alerts had a default, most did not, I have duplicated the format used by the http post alert to all the other alerts using a requests post.
All have a default of 10 seconds, overridable by a config item

I believe broken tests are related to the release of elasticsearch 7.0.5


build images for github actions from source instead of pulling from dockerhub

follow up to #53

Fixed in https://github.com/mattfenwick/cyclonus/pull/84 -- we now have an option to do a go run (without a job, without an image) against the clusters.


Bash script for simplified packet tracing on VPP.

Documentation + example usage TBD

Coverage decreased (-0.09%) to 74.988% when pulling 4574af353c577a5f29539568ff417fe1c5c8ac7c on milanlenco:vpptrace into 7f15cccc41c320824dc860ed0964cf3b62966745 on contiv:master.


Night Prowler config item does not exist

I looked at (config/soulsweapons.json), but there was no configurable item for Night Prowler.

Yes there is, both for Night Prowler and the Nights Edge item. The item is labeled "nights_edge". If you don't have it, then you need to delete the config and re-run minecraft so it generates a new one.


Updating the example of the cluster DNS

Current example didn't match actual format for service fabric cluster DNS

Hi @TheBeardedLlama, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no contribution license agreement is required at this point. Real humans will now evaluate your PR.

TTYL, AZPRBOT;

@cawams - Could you review this additional proposed change from @TheBeardedLlama in the public repository?

@mthalman-msft - Hey Matt, could you review this public repo pull request from @TheBeardedLlama now that you've taken the article over from @CawaMS? Thanks!

@tysonn: Looks good


Handling single model in sync method

Hi, sync and syncWithoutDetaching methods now accept a single model as an argument.
Related issue (#2395) and PR (#2396).

@GromNaN Can you please review this PR?


If you get ImportError, you can use https://github.com/guofei9987/pyLSHash

I will continue to maintain LSHash in https://github.com/guofei9987/pyLSHash

thank you very much, it really works


Issue in running run_experiment.sh

I downloaded the data by running download_data.sh and then run the experiment using run_experiment.sh. It gave following error.
PS: It was run on google colab with GPU.
Traceback (most recent call last):   File "train.py", line 400, in <module>     main()   File "train.py", line 371, in main     use_continuity=args.use_continuity)   File "/content/space_time_pde/experiments/rb2d/physics.py", line 62, in get_rb2_pde_layer     pde_layer.add_equation(eqn_str, eqn_name, subs_dict=subs_dict)   File "../../src/pde.py", line 79, in add_equation     set(self.out_vars))) ValueError: Variables in the eqn_str ({w*0.12129077315330505+5.587935322792781e-11, u*0.14067453145980835+-0.0025140857324004173, b*0.24790656566619873+0.0009028007043525577, t, z, x}) does not match that of in_vars ({x, z, t}) and out_vars ({w, b, u, p})

@soheilesm Any chance you might know what this is referring to? I'm thinking this might be due to Google Colab.
@santoshsanu1 Are you observing this on a local server?

@soheilesm Any chance you might know what this is referring to? I'm thinking this might be due to Google Colab.
@santoshsanu1 Are you observing this on a local server?

@santoshsanu1 I just used the current version of the repo, downloaded the data as well by running download_data.sh and then ran the experiment on my GPU using run_experiment.sh as well - It ran pretty fine, no errors, no issues on my side. I agree with @maxjiang93, it might be an issue with your Google Colab setup. We close this issue.

@santoshsanu1 I just used the current version of the repo, downloaded the data as well by running download_data.sh and then ran the experiment on my GPU using run_experiment.sh as well - It ran pretty fine, no errors, no issues on my side. I agree with @maxjiang93, it might be an issue with your Google Colab setup. We close this issue.


RED please make a red secction

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here. Ex: What device type/UI layout and app version?

@Ramses-Oliveros-Santos Please explain properly. I don't understand what do you mean.

@Ramses-Oliveros-Santos
Thanks for your feedback. We're having trouble understanding what you're requesting and need additional information to investigate this item. Please re-open when you've filled out the issue template to the best of your ability.


Benchmark bincode serialization using serialize_into

This is more directly comparable against desse because this way both libraries avoid memory allocation by writing into an existing buffer.
    .with_function("bincode::serialize", |b| {
        let mut buffer = Vec::new();
        b.iter(|| {
            buffer.clear();
            let my_struct: MySerdeStruct = MySerdeStruct { a: 253, b: 64016 };
            black_box(serialize_into(&mut buffer, black_box(&my_struct)));
        })
    }),

Fixed the benchmark and updated README. Thanks for pointing that out.


Webhook not work because content is not in good body key

Hey,
I tried to add a Discord webhook but i received nothing.
I think the reason is because the content is not sent in a good key.
This is not the same property between multiple webhook system.
For example, the content to send by a Discord webhook must be included in a content key but for a Teams webhook, it must be in a text key.
Could you add a way to choose the custom key to put content in ?
We can use it like webhook --on https//discord.com/api/webhooks/mywebhookprovider --custom-key content
Thank's a lot by advance ! :)

Hi there! Thanks for your issue, I believe if you use the discord /slack compatibility endpoint, this works perfectly fine.
See the information here: https://discord.com/developers/docs/resources/webhook#execute-slackcompatible-webhook

Ok perfect ! 👍🏻
It's works fine! I just have to put /slack at the end of the Discord webhook.
I just see this warning about content key but it's not a real problem for me 😬:

It is possible to add a note in the readme about slack webhook like . ;-)

Honestly, I think your suggestions are good ones which I may include in the future if I can find the time!
I will update the docs as well.


[RFC] eBPF offload consideration

Hi,
Me and @rg0now have been investigating on boosting pion/turn performance with eBPF. As a first step, we implemented an eBPF/XDP offload for UDP channel bindings. This way, pion/turn can offload the channel data processing to the kernel. Below we present our implementation details,  early results and call for a discussion to consider eBPF offload in pion/turn.
Implementation details
How does it work?
The XDP offload handles ChannelData messages only. The userspace TURN server is responsible for all the other functionality from building channels to handle requests and etc. The offload mechanisms are activated after a successful channel binding, in the method Allocation.AddChannelBind. The userspace TURN server sends peer and client info (5-tuples and channel id) to the XDP program via an eBPF map. From that point the XDP program can detect channel data coming from the peer or from the client. When a channel binding gets removed the corresponding data will be deleted from the eBPF maps and thus there will be no offload for that channel.
Changes to pion/turn
New: We introduce a new internal offload package, which manages offload mechanisms. Currently, there are two implementations: the XDPOffload that uses XDP, and a NullOffload for testing purposes.
Changed: The kernel offload complicates lifecycle management since eBPF/XDP offload outlives TURN server objects. This calls for new public methods in package turn to manage the offload engine's lifetime: InitOffload starts the offload engine (e.g., loads the XDP program and creates eBPF maps) and ShutdownOffload removes the offload engine.  Note that these methods should be called by the application as shown in the server_test.go benchmark.
But after everything is set up, channel binding offload management happens in Allocation.AddChannelBind and Allocation.DeleteChannelBind with no change in their usage.
eBPF/XDP details
The XDP part consist of a program that describes the packet processing logic to be executed when the network interface receives a packet. The XDP program uses eBPF maps to communicate with the user space TURN server.
Maps: The XDP offload uses the following maps to keep track of connections, store statistics, and to aid traffic redirects between interfaces:

name
key
value
function

turn_server_downstream_map
peer 5-tuple
client 5-tuple + channel-id
match peer -> client traffic

turn_server_upstream_map
client 5-tuple + channel-id
peer 5-tuple
match client -> peer traffic

turn_server_stats_map
5-tuple + channel id
stats (#pkts, #bytes)
traffic statistics per connection (5-tuple and channel-id)

turn_server_interface_ip_addresses_map
interface index
IPv4 address
interface IP addresses for redirects

XDP Program: The XDP program receives all packets as they arrive to the network interface. It filters IPv4/UDP packets (caveat: VLAN and other tunneling options are not supported), and checks whether the packets belong to any channel binding (i.e., checks the 5-tuple and channel-id). If there is a match, the program does the ChannelData handling: updates 5-tuple, adds or removes the ChannelData header, keeps track of statistics, and finally redirects the packet to the corresponding network interface. Other non channel data packets are passed to the network stack for further processing (e.g., channel refresh messages and other STUN/TURN traffic goes to user space TURN server).
Results
CPU profiling
Prior results are promising. The CPU profiling with the benchmark (#298) shows that the server.ReadLoop() that took 47.9 sec before, runs for 0.96 sec with the XDP offload.
Flame graph w/o the offload:

Flame graph w/ XDP offload:

Microbenchmark with simple-server
Measurements with iperf, turncat (our in-house TURN proxy), and the simple-server example show outstanding (150x!) delay reduction and significant (6x) bandwidth boost.
Measurement setup

iperf-client, turncat, simple-server, and iperf-server communicate via the localhost
measure steady-state (channel binding done) over 50 sec
3 pion/turn versions:

single: single-threaded simple-server
multi: 4-thread simple-multithreaded server (#295)
xdp: simple server with XDP offload

results based on 3 consecutive runs

Delay results

avg[ms]
simple
multi
xdp

avg
3.944
4.311
0.033

min
3.760
0.473
0.023

median
3.914
4.571
0.027

max
4.184
5.419
0.074

Bandwidth results

Note
iperf stalls at ~220k pps, we assume 1+ mpps with a powerful load generator

[pps]
simple
multi
xdp

avg
36493
96152
227378

min
35241
91856
222567

median
36617
96843
227783

max
37545
99455
233559

Discussion

XDP offload is straightforward for UDP connections, but is cumbersome for TCP and TLS. Fortunately, the eBPF ecosystem provides other options: tc and sockmap can be potential alternatives with a reasonable complexity-performance trade-off.

Yet we need to coordinate the different offload mechanisms for different connections.
In addition, offload mechanisms introduce new lifecycle management scale: these mechanisms overlive TURN server objects.

The eBPF objects needs to be built and distributed, and this makes the build process more complex.

New dependency: cilium/ebpf.
Build process gets more complex: eBPF objs are built via go generate; how to integrate it with the current build process; e.g., add a Makefile?

Monitoring is not trivial due to the lifetime of XDP objects and becasue in XDP conections are identified by 5-tuples and we loose the notion of 'listeners'.

Therefore, current monitoring implementation is initial. The bytes and pkts sent via a 5-tuple are stored in a statistics eBPF map. We update the counters in the statistics map, but we do not delete from it. There is no interface exposed for querying statistics (one can use bpftool to dump the map content)

XDP Limitations: The bpf_redirect() that handles packet redirects in eBPF/XDP supports redirects to NIC egress queues in XDP. This prevents supporting scenarios when clients exchange traffic in a server-local 'loop'.

We disabled the XDP offload for host-local redirects. We had some weird issues with forwarding traffic between NICs with the xdp driver to NICs with the xdpgeneric drivers (except the lo interface).
A packet size limit is set in the XDP program to prevent fragmentation. Currently the limit is 1480 bytes.

This is pretty magical! Great work :)
I am in support of adding this. I think people will find this useful

I added you to the repo @levaitamas
Unfortunately these days I don't have much bandwidth to get involved. I would love to support you everywhere I can. If you want me to add other developers so you can work together happy to do that

Thanks @Sean-der! I do appreciate adding me to the repo, since that would definitely ease supporting the eBPF offload once it gets integrated. I would recommend adding @rg0now. He has a great understanding of the pion ecosystem, and already made impactful contributions (e.g., support multi-thread UDP).

Done! That was a major oversight that @rg0now wasn’t in already :(

CC @stv0g

Great work! I am also in support of getting this in 👍🏻


Add GitHub Actions Workflows

Would be nice to run tests / style linting / code linting / enforce code formatting, etc. with GitHub Actions.

Resolved by #9


[9.x] Add "order by" with field concatenation clause to the query

This PR adds convenient sorting with field concatenation
User::query()->orderByConcat(['first_name', 'last_name'], 'desc')->get()

select *, CONCAT(first_name, last_name) as first_name_last_name from "users" order by "first_name_last_name" desc

especially useful in the presence of Accessors, reducing the logic
protected function name(): Attribute
    {
        return Attribute::make(
            get: fn ($value) => $this->first_name . ' ' . $this->last_name,
        );
    }

If only for sorting, it seems inferior to:
User::query->latest('first_name')->latest('last_name');

With the orderByConcat() version, there will be an unexpected order as there is no "separator" character between the columns to differentiate. An example would be given this data:
Name    | Email
------------------------
Bray    |<EMAIL_ADDRESS>Braylon |<EMAIL_ADDRESS>
With double order by like ->latest('name')->latest('email') you get the expected results.
Bray<EMAIL_ADDRESS>Braylon<EMAIL_ADDRESS>
With ->orderByConcat(['name', 'email'])'s current implementation, it smashes everything together without a separator and sorts out of order:
<EMAIL_ADDRESS><EMAIL_ADDRESS>
I think having some sort of selectConcat method could be beneficial, but not sure that having it as ordering syntax is the best idea. I'm not a DB expert, but it seems that two orderBy() statements with the proper indexes will be much faster than the DB having to concatenate the columns together before sorting with a larger dataset.

If only for sorting, it seems inferior to:

User::query->latest('first_name')->latest('last_name');

With the orderByConcat() version, there will be an unexpected order as there is no "separator" character between the columns to differentiate. An example would be given this data:

Name    | Email

------------------------

Bray    |<EMAIL_ADDRESS>
Braylon |<EMAIL_ADDRESS>

With double order by like ->orderBy('name')->orderBy('email') you get the expected results.

Bray<EMAIL_ADDRESS>
Braylon<EMAIL_ADDRESS>

With ->orderByConcat(['name', 'email'])'s current implementation, it smashes everything together without a separator and sorts out of order:

<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

I think having some sort of selectConcat method could be beneficial, but not sure that having it as ordering syntax is the best idea. I'm not a DB expert, but it seems that two orderBy() statements with the proper indexes will be much faster than the DB having to concatenate the columns together before sorting with a larger dataset.

thanks for the answer, sounds right, I think you're right

Thanks for your pull request to Laravel!
Unfortunately, I'm going to delay merging this code for now. To preserve our ability to adequately maintain the framework, we need to be very careful regarding the amount of code we include.
If possible, please consider releasing your code as a package so that the community can still take advantage of your contributions!
If you feel absolutely certain that this code corrects a bug in the framework, please "@" mention me in a follow-up comment with further explanation so that GitHub will send me a notification of your response.


Steelseries Rival 3 Wireless - Turning off RGB and setting sleep timer

It would be nice if we could at least disable RGB and seting the sleep timer to X min to save battery life. I really don't want to use the overbloated Engine 3.

I'd love for this to be implemented. I know if you use the app on windows and set the timeout it gets saved on internal memory, the rgb doesn't.