transport: load balancing module refactor

Description
This is the third attempt to refactor this module;)
Replica set calculation was moved to a separate module (transport::locator).
Replica set calculation is done by a module originally written by @cvybhu - transport::locator::replication_info. struct ReplicationData lives in that module, and provides a set of functions that allow calculating of SimpleStrategy & NetworkTopologyStrategy replica lists.
To make load balancing fast, precomputing replica sets is required. This is done by another of @cvybhu's modules - transport::locator::precomputed_replicas. precomputed_replicas::PrecomputedReplicas is a struct that precomputes replica lists of a given strategies, and provides O(1) access to desired replica slices.
locator::ReplicaLocator combines the functionality of the previously mentioned modules, and provides a unified API for getting precomputed or calculated on the fly replica sets. Representing replica sets by a custom ReplicaSet type allowed creating an API that supported optional limiting replicas to a single data center (and allocation-free creation, getting & iteration through precomputed replica sets).
Things left for a follow-up:

A way to configure which strategies need replica set precomputation

LoadBalancingPolicy interface was changed.
plan was split to pick and fallback methods. This allows to better optimize the most common case, where only one node from the load balancing plan is needed. Changes required in query execution code were minimized by providing a lazy chaining iterator ltransport::load_balancing::plan::Plan. This iterator's first element is a node returned by the LoadBalancingPolicy::pick function, the next items come from LoadBalancingPolicy::fallback iterator. Falback method is called lazily - only when second+ element of the Plan iterator is needed.
The following methods were added:
fn on_query_success(&self, query: &QueryInfo, latency: Duration, node: NodeRef<_'>);
fn on_query_error(&self, query: &QueryInfo, latency: Duration, node: NodeRef<_'>, error: &QueryError);

The motivation for adding them was the ability to contain the logic of latency-aware policy inside some load balancing policy.
Default policy was created.
Default policy supports token and data center awareness. It also has a logic to do a data center failover.
Things left for a follow-up:

LWT routing support

Latency aware policy was merged with the default one
Mechanisms used only by latency aware policy were removed (e.g. TimestampedAverage living in Node).
Pre-review checklist

[x] I have split my patch into logically separate commits.
[ ] All commit messages clearly explain what they change and why.
[ ] I added relevant tests for new features and bug fixes.
[ ] All commits compile, pass static checks and pass test.
[ ] PR description sums up the changes and reasons why they should be introduced.
[x] I added appropriate Fixes: annotations to PR description.

Fixes: https://github.com/scylladb/scylla-rust-driver/issues/408

I'm not familiar with the token ring, replica computation etc., but apart from that, this approach looks good.

I'm sad to see that we didn't really take advantage of grouping all default features in one Default Policy. Namely, we still pass the load balancing policy in an Arc, which imposes some overhead. Instead, I would propose:
struct LoadBalancing(Or)

enum Or {
    Default(DefaultLoadBalancingPolicy),
    Custom(Arc<dyn LoadBalancingPolicy>)
}

impl LoadBalancingPolicy for LoadBalancing {
   ...
}

Rebased on top of main (using @wprzytula's pull request https://github.com/havaker/scylla-rust-driver/pull/8)

Removed scylla/src/transport/load_balancing/latency_aware.rs in transport: load_balancing: remove policies other than the default one.

Rebased on top of main, merged https://github.com/havaker/scylla-rust-driver/pull/6 (with additional NodeRef import fixes)

I'm sad to see that we didn't really take advantage of grouping all default features in one Default Policy. Namely, we still pass the load balancing policy in an Arc, which imposes some overhead. Instead, I would propose:
struct LoadBalancing(Or)

enum Or {
    Default(DefaultLoadBalancingPolicy),
    Custom(Arc<dyn LoadBalancingPolicy>)
}

impl LoadBalancingPolicy for LoadBalancing {
   ...
}

I don't think that the overhead you mentioned is noticeable.

v1:

Rebased on top of main
Prepared a commit series that is suitable for review

v2:

applied clippy suggestions
added doc comments to the new LoadBalancingPolicy interface

Oops, I left a https://github.com/scylladb/scylla-rust-driver/pull/612/commits/551485e5936e70195741747c1d815b63e2021737 that I intended to squash with transport: load_balancing: remove ChildLoadBalancingPolicy trait.

v2.1:

squashed https://github.com/scylladb/scylla-rust-driver/commit/551485e5936e70195741747c1d815b63e2021737

Please update the PR description. It mentions that some things still need to be done - the PR was marked as ready so I guess those things are ready?

Updated, I've added Things left for a follow-up lists.

v2.2:

changed the order of commits, moving locator-related stuff to the beginning

v3:

fixed ReplicaSet len calculation, tests included
replaced ReplicaSet::get with choose_filtered method, tests included
limited visibility of Replicas, ReplicationInfo, PrecomputedReplicas with the help of pub(crate)
fixed some typos that came up in the review
added default implementations of on_query* methods
added a separate commit that renames load_balancing::Statement to RoutingInfo
canceled deletion of custom load balancing policy example
renamed Replicas to ReplicasArray
renamed MergedNTS variant of ReplicaSetInner to ChainedNTS
refactored locator test module (reduced code duplication, every test uses the same ring mock)
changed behavior to return an empty set on invalid datacenter in ReplicaLocator::replicas_for_token
changed log level to debug for unknown strategy in ReplicaLocator::replicas_for_token
avoid iterating over the entire ring when calculating NTS replicas in ReplicationInfo (https://github.com/scylladb/scylla-rust-driver/pull/648)
squashed addition of size_hint & nth method into introduction of ReplicaSetIterator
restored ClusterData::get_datacenters_info
removed underscore from used is_alive argument in default policy

things yet to be done:

separate commit for deleting the name method from load balancing trait
applying derive Debug in a more granular way (splitting into separate commits)
ClusterData::new_sync
using ReplicaSet::choose_filtered in default policy code

Nit: Changing ClusterData::new resulted in having to change many function to be async. How about keeping ClusterData::new sync, and invoking it in non-testing code with spawn_blocking? This should result in slightly less churn.

Or - even better - we could have ClusterData::new_sync and a ClusterData::new which just calls new_sync in spawn_blocking.

Using any of these approaches requires us to clone everything that is passed to ClusterData::new() (as we want to send the task to possibly another thread, all data used by it must be owned). I'm afraid that cloning such a big amount of data is a too much waste to go for it.
On the other hand, the functions that you observed to be necessarily made async are mainly test helpers - I wouldn't consider that a problem.
WDYT @piodul @havaker ?

It's not a super important issue, just a nitpick. If it's difficult to fix or causes other problems then let's not do it. I just wanted to reduce the amount of unnecessary changes.

v5:

rebased on top of main

v7:

restored name() method (@wprzytula convinced me that it would be useful in OTel tracing)
fixed issues around code samples in doc book after rebase
added doc book documentation for the load balancing module

v9:

merged latency-aware policy tests from https://github.com/havaker/scylla-rust-driver/pull/16 (@wprzytula)
fixed doc comment of mock_metadata_for_token_aware_tests in transport: locator: add test module
changed random number generator in ReplicaLocator tests to be deterministic (transport: locator: change rng in tests to be deterministic)

v10:

merged latency-aware docs from @wprzytula (https://github.com/havaker/scylla-rust-driver/pull/17)

v11:

squashed commits that added Debug derive into some previous ones which were named the same

v12:

reworded docs
rebased on top of main (to make clippy happy)

The new version of load balancing does the precomputation of replica sets for each vnode, which could turn out to be quite a bit of work, so I measured how much resources the precomputation consumes when compared to the previous version which didn't do the precomputation (usiing lbbench):
Cluster: 3 datacenters, 8 nodes in each one
A few SimpleStrategy keyspaces, RF <= 8:
Before: 1.8MB of memory used, 420µs to compute ClusterData
After: 2.5MB of memory used, 5ms to compute ClusterData
A few big SimpleStrategt keyspaces, RF ~= 20
Before: 1.9MB of memory used, 558.229µs to compute ClusterData
After: 3.7MB of memory used, 14ms to compute ClusterData
A few NetworkTopologyStrategy keyspaces RF <= 8
Before: 1.9MB of memory used, 454.11µs to compute ClusterData
After: 6.3MB of memory used, 25ms to compute ClusterData
All of the above
Before: 2MB of memory used, 354.14µs to compute ClusterData
After: 7.8MB of memory used, 37ms to compute ClusterData

We use a bit more memory and compute power, but it's within reasonable limits.

One positive thing stemming from the new precomputation is that the number of allocations per request got reduced:
Before:
Inserts:
----------
allocs/req:                15.00
reallocs/req:               8.00
frees/req:                 15.00
bytes allocated/req:     2458.05
bytes reallocated/req:    269.06
bytes freed/req:         2456.80
(allocated - freed)/req:      1.25
----------
Sending 100000 selects, hold tight ..........
----------
Selects:
----------
allocs/req:                48.00
reallocs/req:               8.00
frees/req:                 48.00
bytes allocated/req:     5266.07
bytes reallocated/req:    209.00
bytes freed/req:         5266.00
(allocated - freed)/req:      0.07

After:
Inserts:
----------
allocs/req:                 6.01
reallocs/req:               6.00
frees/req:                  6.00
bytes allocated/req:      381.80
bytes reallocated/req:    173.05
bytes freed/req:          380.62
(allocated - freed)/req:      1.18
----------
Sending 100000 selects, hold tight ..........
----------
Selects:
----------
allocs/req:                39.00
reallocs/req:               6.00
frees/req:                 39.00
bytes allocated/req:     3190.15
bytes reallocated/req:    113.01
bytes freed/req:         3190.04
(allocated - freed)/req:      0.11
----------

Nice work!


ng serve not working

Bug Report or Feature Request (mark with an x)
- [ x] bug report -> please search issues before submitting
- [ ] feature request

Versions.

@angular/cli: 1.1.2
node: 7.10.1
os: linux x64
@angular/animations: 4.2.3
@angular/common: 4.2.3
@angular/compiler: 4.2.3
@angular/core: 4.2.3
@angular/forms: 4.2.3
@angular/http: 4.2.3
@angular/material: 2.0.0-beta.7
@angular/platform-browser: 4.2.3
@angular/platform-browser-dynamic: 4.2.3
@angular/router: 4.2.3
@angular/cli: 1.1.2
@angular/compiler-cli: 4.2.3
Repro steps.

ng serve
edit some files
save files
directory /src/app/home change detected & rebuild
directory /src/app/home/setting change not detected

The log given by the failure.
No specific error message.
Desired functionality.

Mention any other details that might be useful.

There is not a lot of information here to determine what is going on. Please open a new issue with more information.
Steps to reproduced, including what files are present or better yet a repo that is currently not working.
Also, updating your version of the CLI should help you out.


Updated fmt to version 9.0.0

Fixed build against packaged fmt 9.0.0.
Updated bundled fmt to version 9.0.0.
Performed refactoring of Findfmt.cmake.

Closes https://github.com/microsoft/vcpkg/issues/25673

Reported Windows build failure to vcpkg upstream.

Should be fixed in 9.0.1.

Hi there! Because as you've seen this breaks things, I don't think we're going to take this, so I'm going to close this PR for now. We welcome a new PR once 9.0.1 comes out with a fix!
Thanks!

Because as you've seen this breaks things, I don't think we're going to take this, so I'm going to close this PR for now. We welcome a new PR once 9.0.1 comes out with a fix!

Fixed in #679. All tests passed now.


Tensorflow lite Makefile: add support of gpu

Please make sure that this is a feature request. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:feature_template
System information

TensorFlow version (you are using):  2.3
Are you willing to contribute it (Yes/No): Yes

Describe the feature and the current behavior/state.
Currently we cannot use GPU delegate when compiling tensorflow lite using the Makefile
Will this change the current api? How? No
Who will benefit with this feature?
Any Other info.

@justeph
Do you have any use case that requires the feature you are interested in? Please feel free to submit a PR if you have use cases that supports that feature.Thanks!

I am currently trying to build a yocto recipe for tensorflow-lite using Makefile, and I need the GPU delegate. I am trying to modify the Makefile right now, and I will open a merge request when this will be ready

You can use OpenCL delegate with CMake.
https://www.tensorflow.org/lite/guide/build_cmake#available_options_to_build_tensorflow_lite


Discord Bug Report: Kryptobi

Reported on: 2022-03-31T11:44:56.335000+00:00
Bug Description:
Badger TVL seems to be off, was displaying correctly a few hours ago. Apps like defi lama and debank still showing over 800m but the badger dapp showing lesser.
Attachments:

confirmed things are ok


Figure out a better way of showing timetable.

Would be better to not have to use images, just need to figure out a way.

Well I sorta did it, with the dialog box instead of relying on images placed in specific folders.


Domain change for Rizz Comic

Source information
Rizz Comic 1.4.34
Source language
English
Source new URL
https://realmoasis.com
Other details
The old domain is still up but there are announcements in the latest chapters. And the new domain is already up.

Acknowledgements

[X] I have updated all installed extensions.
[X] I have opened WebView and checked that the source URL has not changed yet.
[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] I will fill out all of the requested information in this form.

I think they might have another URL as well: https://rizzfables.com/


add Variant Annotation Task Team role to README

Updates @sarahhunt's documentation PR https://github.com/ga4gh/schemas/pull/601

cosmetic, merging


Deserialize and cache the 'include' resource of a document

Support deserializing of the included portion of the document and pushing to the resource's service cache.

done in 0.3.0


clean docs

Before submitting

[ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
[ ] Did you read the contributor guideline?
[ ] Did you make sure to update the docs?
[ ] Did you write any new necessary tests?
[ ] If you made a notable change (that affects users), did you update the CHANGELOG?

What does this PR do?
Fixes # (issue).
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃

now since #833 is merged we shall care about sphinx PR recommendations


Fix various missing #include and required namespace qualifiers

This patchset fixes a number of trivial compilation issues that were found when attempting to add Linux support to libHttpClient.
I have not tested these on all platforms, or attempted to understand why the current incorrect forms work, but they are harmless changes AFAICT.

Thanks.  Can you fill out the CLA for the bot so we can get this merged in?

I apologize on the delay on that. This work is happening for Mojang under a partner studio, and we've been trying to get some clarity on if was OK for me to simply accept the CLA. So far Trust & Compliance doesn't seem to have an answer.
If someone happen to be aware of some MS-wide policy re: OSS contributions by contractors, that would be very helpful.

I'm not aware of that policy.  I recommend that you and your Microsoft rep email the folks here: https://github.com/microsoft/opensource.microsoft.com#code-of-conduct

These are all minor fixes, so I'll get these in


Fix PackageManifest Server Crashes

Description
Didn't realize that GetLabels() returns a reference, not a copy.
Addresses https://jira.coreos.com/browse/OLM-1111

/test unit

/lgtm


Project status?

Just inquiring about this repo's status as there hasn't been a commit in 9 months, over 20 pull requests and over 200 issues. Is this still being actively maintained?

@bbeaudreault Can you help us :top:

As far as I know we no longer use this project internally at HubSpot, so there isn't a lot of interest in it here. If someone would like to take the project under their wing, we'd be happy to hand it off to them.


Implement ! at start of rule to negate it

Along with supporting \! so you can have files that start with ! ignored still.

The ! is there now, need to look at the \! escape still.


Added setListenerVelocity()

Thanks, but should be add also similar functions for ma_sound and ma_group since they have velocity too

I was already using sound velocity previously. But the pitch of the sound keeps increasing until the sound cuts off (probably some float goes to Infinity or NaN). That's how I discovered that I wasn't setting the listener velocity and that listener velocity is missing :)

Ouch, too much stuff to remember! Thanks :)


Fix rrules for pass-by-reference active vars (like complex)

Codecov Report
Patch coverage: 93.33% and project coverage change: -5.83 :warning:

Comparison is base (23fa314) 77.08% compared to head (d575ef1) 71.25%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #754      +/-   ##
==========================================
- Coverage   77.08%   71.25%   -5.83%
==========================================
  Files          18       18
  Lines        7345     7226     -119
==========================================
- Hits         5662     5149     -513
- Misses       1683     2077     +394

Impacted Files
Coverage Δ

src/compiler.jl
73.05% <93.10%> (-4.45%)
:arrow_down:

src/api.jl
59.90% <100.00%> (-6.77%)
:arrow_down:

... and 10 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Clean definition of range types

Use recursive type to generate a cleaner expresseion for UInt4 and UInt8
This needs update of typescript to take advantage of this performance improvement (4.5) for recursive types.

Note that I have no clue what I am doing regarding yarn@2 - I tried to conservatively update typescript to 4.5.5 but yarn insisted on installing 4.9.5, not sure how to convince it otherwise

@Isokaeder thanks for the PR, I rebased this and got rid of yarn2 (in favor of https://bun.sh). LGTM now


[question] why is core-js needed (as non-dev dependency) ?

Hi,
https://github.com/martinussuherman/vue-template/blob/e900f9c2b5cfa7056abd3368bc9f9293f2e6045c/package.json#L11
why is core-js needed? It's not imported anywhere.
If needed, maybe should be in devDependencies ?

Hi @mariusa,
core-js is one of the dependencies that you got when you create vue2 project via vue-cli, you can see that by looking at the first commit in this repo (at that commit it's just what you get from using vue-cli). So I can't answer exactly why it's needed. I'm sorry if I can't help much on this.
Regards,
Martinus

Answer: it's not needed. I removed it from package.json and it works fine. It's still installed:
$ npm ls core-js
<EMAIL_ADDRESS>/home/marius/work/misc/vue-template
└─┬<EMAIL_ADDRESS>  └─┬<EMAIL_ADDRESS>    └─┬<EMAIL_ADDRESS>      └─┬<EMAIL_ADDRESS>        └──<EMAIL_ADDRESS>

Thanks @mariusa for the info. Really appreciate that :smile:
Regards,
Martinus


JSDoc shown types 'any' always

Issue Type: Bug
Launch with code --disable-extensions
@params type only show any

VS Code version: Code 1.65.2 (c722ca6c7eed3d7987c0d5c3df5c45f6b15e77d1, 2022-03-10T14:33:55.248Z)
OS version: Windows_NT x64 10.0.22581
Restricted Mode: No

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz (6 x 2808)

GPU Status
2d_canvas: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onrasterization: enabledskia_renderer: enabled_onvideo_decode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
31.94GB (9.19GB free)

Process Argv
--crash-reporter-id ec7b5b0a-97a9-4cad-b4a3-cc2bc94ce149

Screen Reader
no

VM
0%

Extensions: none
A/B Experiments
vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythonvspyl392:30443607
pythontb:30283811
pythonvspyt551:30345470
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
pythondataviewer:30285071
vscod805:30301674
pythonvspyt200:30340761
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593:30376534
vsc1dst:30438360
pythonvs932:30410667
wslgetstarted:30449410
vsclayoutctrc:30451274
dsvsc009:30452663
pythonvsnew555:30457759
vscscmwlcmc:30438804
vscgsvidc:30447482
pynewfile477:30463512

/needsMoreInfo
Is it reproducible in latest VS Code 1.66.0?
P.S. You have better chance to receive help if you post actual code in addition to screenshots. Nobody likes to retype code from screenshots to reproduce your issue

Also please always share a self contained code example as text

/needsMoreInfo
Is it reproducible in latest VS Code 1.66.0?
P.S. You have better chance to receive help if you post actual code in addition to screenshots. Nobody likes to retype code from screenshots to reproduce your issue

Sorry no code added before, now I added the code from here, and same issue in v1.66.0

Also please always share a self contained code example as text

Thanks for your advice, I will remember it
// Parameters may be declared in a variety of syntactic forms
/**
 * @param {string}  p1 - A string param.
 * @param {string=} p2 - An optional param (Google Closure syntax)
 * @param {string} [p3] - Another optional param (JSDoc syntax).
 * @param {string} [p4="test"] - An optional param with a default value
 * @returns {string} This is the result
 */
function stringsStringStrings(p1, p2, p3, p4) {
  // TODO
}

Types in JSDoc only works in JavaScript files, not TypeScript files.

Types in JSDoc only works in JavaScript files, not TypeScript files.

Thank you. I think the @param type of JSDoc can be used in typescript

You can use @param and @return for descriptions, but not types. TypeScript type notations must be used.

Good catch @yume-chan! Yes JSDoc types are only observed in JS files. Closing this as by-design


chore(main): release 1.1.0

:robot: I have created a release beep boop
1.1.0 (2024-09-05)
Features

GitHub Copilot compatibility (be281d9)

Bug Fixes

update environment variable for Copilot API key (dd08c50)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

v1.1.0
:sunflower:


Packaged game don't start Node.JS server

Hi everyone,
My server is working perfectly on editor but did not start on a packaged game.
The script folder is copied on package (cf Readme)
I have not much information but i can see that the node process do not start on game launch.
Package config : Windows / Developpment
No error during packagin process
Thanks for your work !

Might need to see some logs to see what it spits out when it starts. Make sure your scripts are placed as expected in https://github.com/getnamo/NodeJs-Unreal#packaging

Of course, this is the logs for the package : logspackagenode.txt


Update relationships.md

Updated the arrow function variable names to be more consistent with the other functions below it.
Below, the variable t refers to a Tag entity. However, it's really should be named pt as it's referring to a PostTag entity.

Thanks @kylejrp!


[FR] Add finer control for hide_dialog_climate_actions

At the moment hide_dialog_climate_actions allows you toggle these elements.

I'd love it if the top and bottom boxes could be independent. I want my other half to be able to change the temperature but not change the mode etc

Hi @i-am-shodan,
I‘ll take a look and I‘ll let you know if it is something feasible.
Regards

Hi @i-am-shodan,
When this pull request gets merged and a new version is released, you will have the options to achieve what you are trying to achieve.
Regards

Thanks very much! Will check it out when it merges


Chore(recyclarr): relocate recyclarr configs

Pull request
Purpose
To relocate Recyclarr config files to the Recyclarr config-templates repo, and the Recyclarr config guide pages to the Recyclarr Wiki. Existing links to guide pages are retained with redirects in effect to maintain seamless browsing for users.
Approach

[x] Add Recyclarr config page redirects to mkdocs.yml
[x] Removed Recyclarr config files from the TRaSH guides repo
[x] Added a README.md to the recyclarr-configs folder to inform users accessing via GitHub of the relocation
[x] Empty the content of the existing Recyclarr config guide pages, and replace with the new link. (Note - these pages are already subject to a redirect as described above so won't been seen, but the original content would otherwise throw errors as the config files themselves are no longer in the TRaSH repo)

Open Questions and Pre-Merge TODOs
Check all boxes as they are completed

[x] Use github checklists. When solved, check the box and explain the answer.

Learning
If you're adding a new Custom Format make sure you follow the Radarr/Sonarr Custom Format (JSON) Guidelines.
Requirements
Check all boxes as they are completed

[x] These changes meet the standards for contributing.
[x] I have read the code of conduct.

Leaving this one in draft so I can re-check later in the week.

This one is ready to merge, ongoing markdown/lint repo issues notwithstanding.


Yet another Prefill-Decode separation in vllm

This PoC demonstrates an implementation of Prefill-Decode separation within vLLM, leveraging a memory pool to store KV caches accessed via RDMA connections. The primary goal is to enhance data transfer efficiency by optimizing how KV caches are managed and transmitted.
@KuntaiDu @youkaichao @thesues wonder if you could take a moment to review our implementation and share your valuable feedback. Your insights and opinions would be greatly appreciated.
Key Changes to vllm
Comparing to https://github.com/vllm-project/vllm/pull/8498 （Thanks to your seminal work @KuntaiDu, which inspired us a lot ), this implementation optimizes data transfer efficiency. Instead of utilizing a single large tensor for KV caches per layer, the caches are divided into smaller blocks. This granular approach enables parallel transmission of multiple layers as well as multiple blocks within one layer.
To coordinate different layers effectively, changes are made at the model level rather than the entire model_runner. This implementation showcases modifications for OPT and LLaMA models, establishing a pattern that can be easily extended to other models.
Introducing InfiniteStore
Main contributor: @thesues
The core functionality for sending and receiving KV caches via RDMA is encapsulated in our separate package, InfiniteStore. We are more than happy to donate this project to the community.
Infinitestore Features:

High Throughput and Low Latency:

Utilizes RDMA for memory copying between GPU and CPU across servers.
Maximizes data transfer throughput while minimizing latency.

Dynamic Cluster Management:

Detects the hottest memory pages and dynamically distributes them across nodes to balance the load.

Asynchronous and Batched APIs:

Designed to handle asynchronous operations and support batching, tailored specifically for vLLM’s requirements.

Python Integration:

Integrates with Python's uvloop library, facilitating easy extension of management functionalities in Python.
Implements data communication in C++ for performance-critical operations.

API design
The load/store API is meticulously crafted to align with vLLM’s KV cache characteristics, aiming to parallelize computation with KV cache saving and loading operations. The APIs are asynchronous and support batch processing.The definitions are as follows:
def write_cache(self, cache : torch.Tensor, blocks: List[Tuple[str, int]], page_size: int)
def read_cache(self, cache : torch.Tensor, blocks: List[Tuple[str, int]], page_size: int)

Once the load/store requests are all sent, the sync function from the block API is called to ensure that the processing will be done before moving on.
def sync(self)

As shown in the diagram below, during the prefill computation process, only the final call to the sync API ensures that the kvcache has been fully written to the remote storage.

@zeroorhero implemented a similar kv store-based solution, and me myself also believe that kv store is the way to go for disaggregated prefilling. So yes!

@zeroorhero implemented a similar kv store-based solution, and me myself also believe that kv store is the way to go for disaggregated prefilling. So yes!

Thanks for your reply.
Would appreciate any comments to improve?
So far only a kv cache transporter of RDMA is created. Do you think adding a transporter using CPU memory will make this PR more likely to be accepted?

m

I have a question. How can we guarantee the model updates correctly when using Prefill-Decode separation in production? @chenqianfzh . And how to install InfiniteStore using pip?

I am working on 'pip install InfiniteStore', it could be available  in this week.

Hello, could you provide the way to start the program?

Hello, could you provide the way to start the program?

the script examples/infinitestore_pd_separate.sh in this PR demonstrates how to start the two vllm instances for prefill and decode respectively in a host with multiple hosts.
Before that, make sure the infinitestore is installed and started by running the start.sh.
Please let me know if you run into any problems.

I have a question. How can we guarantee the model updates correctly when using Prefill-Decode separation in production? @chenqianfzh . And how to install InfiniteStore using pip?

Sorry,  I missed the first question from you.
I guess u are asking how I verified the kv_cache are updated correctly across the different vllms of prefill and decode. I verifed it by comparing the generated result with the vanilla vllm. The results are exactly the same.

Hello, could you provide the way to start the program?
you could use pip install infinistore  and  run infinistore or python -m infinistore.server to start mempool daemon. and this app requires kernel nv_peer_mem, the app will check nv_peer_mem first.

you could use pip install infinistore and run infinistore or python -m infinistore.server to start mempool daemon. and this app requires kernel nv_peer_mem

@chenqianfzh @thesues I have three questions：
first：After I installed according to the above command, a circular reference occurred when using the import command. Can you help me find out the reason?

second： In this PR, do the places where infinity is used need to be replaced with infinistore?
third：Is multi-machine deployment currently supported? Or a single machine with multiple cards, for example (prefill-2gpus vs decode-2gpus)？

you could use pip install infinistore and run infinistore or python -m infinistore.server to start mempool daemon. and this app requires kernel nv_peer_mem

@chenqianfzh @thesues I have three questions：
first：After I installed according to the above command, a circular reference occurred when using the import command. Can you help me find out the reason?
second： In this PR, do the places where infinity is used need to be replaced with infinistore?
third：Is multi-machine deployment currently supported? Or a single machine with multiple cards, for example (prefill-2gpus vs decode-2gpus)？

this is what I did:

conda create -n test python=3.11
conda activate test
pip install infinistore==0.1.72
python -m infinistore.server

I just learned how to build manylinux version package.   maybe you could provide more information of this error?

I updated infinistore's API recently, so this PR should be updated as well, I will ping you  as soon as it is ready.
this PR not support this yet.

you could use pip install infinistore and run infinistore or python -m infinistore.server to start mempool daemon. and this app requires kernel nv_peer_mem

@chenqianfzh @thesues I have three questions：
first：After I installed according to the above command, a circular reference occurred when using the import command. Can you help me find out the reason?
second： In this PR, do the places where infinity is used need to be replaced with infinistore?
third：Is multi-machine deployment currently supported? Or a single machine with multiple cards, for example (prefill-2gpus vs decode-2gpus)？

Thanks for trying it out. I will update the PR with mulitple host support and the infinitestore dependency sorted out asap.

this is what I did:

conda create -n test python=3.11
conda activate test
pip install infinistore==0.1.73
python -m infinistore.server

maybe you could provide more information of this error? or check if package has _infinistore.cpython-311-x86_64-linux-gnu.so?

First, I checked my installation directory and found what you said .so

Secondly, I am using python3.10. Can I build this package myself in the 3.10 environment?

this PR not support this yet.

@thesues  Do you have any plans to do this? Or is it already being done?

this is what I did:

conda create -n test python=3.11
conda activate test
pip install infinistore==0.1.73
python -m infinistore.server

maybe you could provide more information of this error? or check if package has _infinistore.cpython-311-x86_64-linux-gnu.so?

First, I checked my installation directory and found what you said .so

Secondly, I am using python3.10. Can I build this package myself in the 3.10 environment?

now infinistore supports python 3.10, 3.11 and 3.12, you could pip install infinistore == 0.1.74  to try this on python3.10
As for cluster level deployment,  it is still in design phase. we could use python asynio io to all manage API or do heartbeats. Do you have any ideas or suggestions to share?

Hi, I encountered the following problem when running the infinistore service：
Server config: ServerConfig(service_port=22345, manage_port=18080, log_level='warning') Traceback (most recent call last):   File "/usr/local/bin/infinistore", line 8, in <module>     sys.exit(main())   File "/usr/local/lib/python3.10/dist-packages/infinistore/server.py", line 82, in main     check_supported()   File "/usr/local/lib/python3.10/dist-packages/infinistore/lib.py", line 123, in check_supported     raise Exception("nv_peer_mem module is not loaded") Exception: nv_peer_mem module is not loaded
And, I checked the relevant files and found that the Nvidia driver name in the /proc/modules is “nvidia_peermem”. How should I solve this problem? Should I install a new version of “nv_peer_mem“ on the physical machine?

I have solved this problem by modifying the check_supported function in lib.py (adding nvidia_mempeer keyword to the judgment condition)
def check_supported():
    if "nv_peer_mem" not in _kernel_modules() and "nvidia_peermem" not in _kernel_modules():
        raise Exception("nv_peer_mem module is not loaded")
    _check_rdma_devices_ibv()

Hi, I encountered the following problem when running the infinistore service： Server config: ServerConfig(service_port=22345, manage_port=18080, log_level='warning') Traceback (most recent call last): File "/usr/local/bin/infinistore", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.10/dist-packages/infinistore/server.py", line 82, in main check_supported() File "/usr/local/lib/python3.10/dist-packages/infinistore/lib.py", line 123, in check_supported raise Exception("nv_peer_mem module is not loaded") Exception: nv_peer_mem module is not loaded  And, I checked the relevant files and found that the Nvidia driver name in the /proc/modules is “nvidia_peermem”. How should I solve this problem? Should I install a new version of “nv_peer_mem“ on the physical machine?

I have solved this problem by modifying the check_supported function in lib.py (adding nvidia_mempeer keyword to the judgment condition)
def check_supported():
    if "nv_peer_mem" not in _kernel_modules() and "nvidia_peermem" not in _kernel_modules():
        raise Exception("nv_peer_mem module is not loaded")
    _check_rdma_devices_ibv()

我的机器没有Mellanox ConnectX-3 VPI 或 Connect-IB InfiniBand 适配器。那么我可以在单机4卡A100上使用这个PR嘛？

Have you encountered this problem before? THX

Have you encountered this problem before? This occured when request pressure is high.
what's your infinistore version?  I think newer version 0.1.82 solved this. before this version,  infinistore has a limited RMDA queue length.  the new version will has a thread which drain CQ asynchronously.

    if "nv_peer_mem" not in _kernel_modules() and "nvidia_peermem" not in _kernel_modules():

thank you, I did this update in infinistore ;-)

Hi, do you plan to support other transporter like https://github.com/vllm-project/vllm/pull/8498 (torch distributed pipe)? your layer-wise idea seems very efficient. Could you give me your contact method? (Wechat / slack/ email, all be fine)

Hi, do you plan to support other transporter like #8498 (torch distributed pipe)? your layer-wise idea seems very efficient. Could you give me your contact method? (Wechat / slack/ email, all be fine)

I could discuss this on https://vllm-dev.slack.com/archives/C07VCUQLE1F or https://github.com/bd-iaas-us/infiniStore


unable to parse response with htmlparser or htmlparser2

Hello,
It has come to my attention, during my experimentation that I am unable to parse the text received from request.get with either the htmlparser or htmlparser2 packages.
In comparison to https.request the string returned is not the same, line breaks wise. I am scarping a human readable page,and http.request handler is able to aggregate the data to match the server response in terms of outputting.
But with request.get I am receiving a response that is without line breaks.
Therefore I can only assume I am not using the package that is not support by, and am led to question:
What is the best way to actually parse out the HTML received from the reqeust.get and requst.post calls.
Thank You.

Can you add a short code example that reproduces the bug? Also include the URL that you are requesting.

URL is in scriptlet. This worked for me with https.request, but I saw your library was simpiler and more popular so I wanted to give it a try.
Thank you.
var request = require('request');
var htmlparser = require('htmlparser2');
var select = require('soupselect').select

var cslb =  'https://www2.cslb.ca.gov/OnlineServices/CheckLicenseII/HisNameSearch.aspx?LName=a&FName=';

var htmlHandler = new htmlparser.DefaultHandler( (error, dom) => {
  if (error){
    console.log( 'error', error );
    process.exit(1);
  }
  var options = extractData( dom );
});

function httpsHandler(err, response) {
    if(err){
      console.error('to err is human')
      process.exit(1);
    }

    var parser = new htmlparser.Parser(htmlHandler);
    parser.parseComplete(response);
}

function extractData( dom ){

  var collection = select(dom, '#ctl00_LeftColumnMiddle_Table1 table td');

  collection.forEach( ( licenses ) => {
    licenses.children.forEach( ( license, i ) => {
      var data  = ( license.data  ) ? license.data : license.children[0].data
      console.log( data );
    });
  })
}
request.get( cslb , httpsHandler );

Compare to this working scriplet:
var https = require('https');
var htmlparser = require('htmlparser2');
var select = require('soupselect').select

var cslb =  'https://www2.cslb.ca.gov/OnlineServices/CheckLicenseII/HisNameSearch.aspx?LName=a&FName=';

var htmlHandler = new htmlparser.DefaultHandler( (error, dom) => {
  if (error){
    console.log( 'error', error );
    process.exit(1);
  }
  var options = extractData( dom );
});

function httpsHandler(response) {

  var str = '';

  response.on('data', function (chunk) {
    str += chunk;
  });

  response.on('end', function () {
    var parser = new htmlparser.Parser(htmlHandler);
    parser.parseComplete(str);
    str='';
  });
}

function extractData( dom ){

  var collection = select(dom, '#ctl00_LeftColumnMiddle_Table1 table td');

  collection.forEach( ( licenses ) => {
    licenses.children.forEach( ( license, i ) => {
      var data  = ( license.data  ) ? license.data : license.children[0].data
      console.log( data );
    });
  })
}
https.request( {host: 'www2.cslb.ca.gov', path: '/OnlineServices/CheckLicenseII/HisNameSearch.aspx?LName=a&FName='},  httpsHandler ).end();

request's callback parameters are:
request({}, function (error, response, body) {})

The HTML document is in the body

How embarrassing.
Thank you, and my apologies.

No problem, your code snippet helped me figure out the problem quickly :+1:


Lab 01

It looks like there was some confusion on the lab.  There is no need to use Express.js for this application.  The only 3rd party dependency required is minimist (or equivalent), to parse your input.  I think a good starting point will be to refer to the class repository, lab-01 demo code.  Also, read the npm documentation for minimist.
The idea is that you will create two modules, input.js and notes.js.  These modules should be imported into index.js for use.  The input module should be responsible for using minimist to parse your input and return an object containing something like { action: add, payload: 'your note to save' }.  This object should then be passed to your Note constructor which will be responsible for creating a Note object using the passed payload.


Add Javascript tests to Travis CI. (#2093)

Adding our Javascript tests as a seperated matrix item to Travis CI, set allow_failures for now.
The test result is look like this: https://travis-ci.org/MatthewShao/mitmproxy/jobs/208819088

yarn is much faster than npm install :smiley:

Awesome, thanks!


⚠ PrivateHD has degraded performance

In fc4ff6b, PrivateHD ($PHD) experienced degraded performance:

HTTP code: 200
Response time: 2289 ms

Resolved: PrivateHD performance has improved in 7c9ec5b.


created timecode is not correct

Dear @lkmeta,
your app is a dream - it does exactly what you promised :). However, the timcodes are not correct. have you ever had problems with this?
Thx + best regards

Thanks for your feedback!
I believe the timestamp issues are related to how Whisper handles them. I'll need to add some helper functions or a custom solution to improve accuracy. I tested Txtify on simple podcasts under 10 minutes and didn't notice this, but I’ll work on fixing it soon when I have some time.

Hey there,
maybe this will somehow help with the accuracy of the timecode issue: https://github.com/jianfch/stable-ts#encode-comparison

Thanks for the suggestion!
I hadn’t come across stable-ts before, but it looks really promising.  I'll definitely test it out and may integrate it into Txtify to improve timecode accuracy as soon as possible.
I truly appreciate your help!


Enable setting custom TreeItemElement component

Is your feature request related to a problem? Please describe.
I am trying to add animation to the item children list when expanding/collapsing. The easiest way to do that would be to put the transition on the height property and toggle the height between some value and 0.
Currently that is not possible becuase TreeItemElement doesn't render anything if the node isn't expanded.
Describe the solution you'd like
Allow developers to define a custom TreeItemElement where the hide/show behavior can me implemented.
Describe alternatives you've considered
Make the default behavior for showing/hiding nodes the one I described, but I don't think that's a good solution.

Hi, thanks for the suggestion. It's not really feasible to make the entirety of TreeItemElement customizable, but I extracted the logic for when it does render its items and when not customizable through the shouldRenderChildren prop in v2.3.5. That should hopefully help you achieve what you want.
See Demo
and Demo Implementation for details


Hierarchy in client.upload_terms

Hi I was just wondering if there was another pyapacheatlas.core.PurviewClient method that allowed for term hierarchy to be included when uploading glossary terms or if there were plans to add this as an optional parameter in the existing client.upload_terms method?

Hi @wjohnson thanks for the quick response! I am finding PyApacheAtlas very useful overall so far
When I send a call to upload_terms as you suggest above I get this error:
AtlasException: {"requestId":"94f95fa9-0b51-4b35-bee6-c49941fc6c9f","errorCode":"TermHierarchyQueryParameterMissing","errorMessage":"Please specify the query parameter includeTermHierarchy as True then retry."}
I have created a work around for now by creating a new method upload_hier_terms where the requests.post has another argument of params={'includeTermHierarchy':True}

Thanks for calling that out!  I have a feature branch where I'm working on support for this and I'll try to get this out into the next release.

As a note this applies to adding term template information as well

@alihughf - If you're interested, would you please check out this other branch and try this sample?  This branch refactors the glossary methods from Atlas/PurviewClient to client.glossary and introduces a PurviewTermGlossary that supports "adding hierarchy" with the add_hierarchy method. It will handle building the right combination of parent formal name + name, setting the parentTerm, and the nickName fields. It also supports term template attributes with the attributes property.
You still need to look up three things:

glossaryGuid of the default glossary
parentFormalName of the parent term
parentGuid of the parent term

With all three of those things, the new client.glossary.upload_term or client.glossary.upload_terms will break the PurviewGlossaryTerm down into JSON for you.
If you would please try that out and let me know if you have any issues using it!
Some steps to get the specific branch and install the editable version of PyApacheAtlas
git clone https://github.com/wjohnson/pyapacheatlas.git
git checkout feature/supportPurviewTermHierarchy
pip install -e pyapacheatlas


Addition of functional tests exercising migration of VM using swap memory

For now these tests won't run on any test lane as a presubmit.
We just need to begin gaining experience with swap and have a way to validate environments.
For additional information:
https://github.com/kubevirt/kubevirtci/pull/723

What this PR does / why we need it:
This verifies we can migrate nodes using swap both away from over commited nodes and to over committed nodes.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
Release note:

None

/cc @davidvossel

/retest

/retest

/retest

/retest

/retest

/retest

/retest

/retest
introduced a small mistake in the migrations job definition when making them mandatory. Should be fixed now.

/retest

/retest

/retest

/retest

/retest

/retest

/retest

/retest

/retest

@jean-edouard
Thanks for reviewing

/retest-required

/retest-required

/retest-required

/retest-required

/retest-required

/retest-required

@Barakmor1 code looks good to me now!
Are the tests actually running now, since https://github.com/kubevirt/kubevirtci/pull/723 got merged?
If so, the PR description should probably be adjusted accordingly.
Thanks!

So let's see: https://prow.ci.kubevirt.io/?job=pull-kubevirt-e2e-k8s-1.23-swap-enabled

Here it is: https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/7171/pull-kubevirt-e2e-k8s-1.23-swap-enabled/1503414334573776896

Here it is: https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/7171/pull-kubevirt-e2e-k8s-1.23-swap-enabled/1503414334573776896

Beautiful!
/approve
Let's give it another run :)
/test pull-kubevirt-e2e-k8s-1.23-swap-enabled

/test pull-kubevirt-e2e-k8s-1.23-swap-enabled
/retest

@Barakmor1 the operator test that keeps failing (test_id:4744) modifies the KubeVirt CR.
I believe that fact alone should make it a serial test, which does't seem to be the case right now.
That could be the issue.
Would it make sense to add a commit to this PR to switch test 4744 to [Serial]?

@Barakmor1 the operator test that keeps failing (test_id:4744) modifies the KubeVirt CR. I believe that fact alone should make it a serial test, which does't seem to be the case right now. That could be the issue. Would it make sense to add a commit to this PR to switch test 4744 to [Serial]?

Done.

/retest

@jean-edouard
update: i think that test_id:6308 is the problem i made it serial

/retest

/retest

/retest-required

/retest

/retest

/retest-required

/retest-required


BREAKING CHANGE: Remove dependency twig/twig for v1

Description

Additional context

Issue reference

coverage: 96.371% (+16.0%) from 80.418%
when pulling 1f9df36ad5c41627cea3c9fff0072e8ec99a1c2c on deps/ds-975-remove-support-twig-v1
into a3bfba35ae6cb34e23972c81074c586e92d8c047 on integration/release-v2.

Just to be sure: no updated composer.lock? Even in the Twig demo app? Did you try to run composer update? This does not mean that this PR is not correct, but I am trying to verify the change. Thanks.

Just to be sure: no updated composer.lock? Even in the Twig demo app? Did you try to run composer update? This does not mean that this PR is not correct, but I am trying to verify the change. Thanks.

composer update updated all dependencies, is it ok @literat? They are still in allowed range set in composer.json, but I am not sure if we want to do that.

Will be mergeable after this PR: https://github.com/lmc-eu/spirit-design-system/pull/1404

Will be mergeable after this PR: #1404. It looks like we are not ignoring lockfiles and the Composer changed the formatting of the lockfile.

This has been merged into the main. Please, rebase the entire release and this branch also.


Issue Handling

[ ] Get
[ ] Create
[ ] Update
[ ] Delete

@afcaJordi regarding the update of an issue - we must define the exact use cases more detailed (what can be updated, which fields etc). It might even make sense to create several issues from that if it becomes more complex


Select: can't render react component as a value

<Select>
        <Item value={ id }>
             <Icon id="en"/>
              en
        </Item>
</Select>

In an example above I won't see an icon, because there is a restriction connected with displaying only text. Why do we need that?

Why do we need that?

In case of Menu type="check" you have to concatenate values (or content) of all checked items. This is how the control works in bem-components and this is how it was designed.

Actually, renderButtonText() is a hook-method, that was added after the discussion about various real use cases. See this comment https://github.com/narqo/react-islands/pull/52#issuecomment-222653537

But, the default behavior that I expect from the select is "what I see in <Item>, I will see in <Button>". The same for multiple select, the concatenation will look like [<item1>, ', ', <item2> ].

the default behavior that I expect from the select is "what I see in , I will see in "

When the control was implemented in bem-components, our internal design guides "thought different". And, as react-islands is just a port of the library, we have the same behaviour, that we had.
We might fix this in future. But PRs are welcome, anyway.


popeye: 0.11.1 -> 0.11.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/project/popeye/versions.
meta.description for popeye is: A Kubernetes cluster resource sanitizer
meta.homepage for popeye is: https://github.com/derailed/popeye
meta.changelog for popeye is: https://github.com/derailed/popeye/releases/tag/v0.11.2
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.11.2 with grep in /nix/store/vqlw3lrb7vq43ibk7cibs5cmjrsi56ws-popeye-0.11.2
found 0.11.2 in filename of file in /nix/store/vqlw3lrb7vq43ibk7cibs5cmjrsi56ws-popeye-0.11.2

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
popeye

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/vqlw3lrb7vq43ibk7cibs5cmjrsi56ws-popeye-0.11.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A popeye https://github.com/r-ryantm/nixpkgs/archive/c22777ae4ffa155a20168d526baa93af1ddc8529.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/c22777ae4ffa155a20168d526baa93af1ddc8529#popeye

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/vqlw3lrb7vq43ibk7cibs5cmjrsi56ws-popeye-0.11.2
ls -la /nix/store/vqlw3lrb7vq43ibk7cibs5cmjrsi56ws-popeye-0.11.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

popeye

Maintainer pings
cc @bryanasdev000 for testing.

Result of nixpkgs-review pr 277370 run on x86_64-linux 1

1 package built:

popeye


Centralize directive parsing and declaration logic

Bug/issue #: rdar://99065717
Summary
Introduces a new AutomaticDirectiveConvertible protocol along with
several property wrappers to allow for declaring new markup directives
with built-in parsing and diagnostics. The general architecture is based
on ArgumentParser.
Here’s an example directive:
public final class Intro: Semantic, AutomaticDirectiveConvertible {
    public let originalMarkup: BlockDirective

    /// The title of the containing ``Tutorial``.
    @DirectiveArgumentWrapped
    public private(set) var title: String

    /// An optional video, displayed as a modal.
    @ChildDirective
    public private(set) var video: VideoMedia? = nil

    /// An optional standout image.
    @ChildDirective
    public private(set) var image: ImageMedia? = nil

    /// The child markup content.
    @ChildMarkup(numberOfParagraphs: .zeroOrMore)
    public private(set) var content: MarkupContainer

    static var keyPaths: [String : AnyKeyPath] = [
        "title"     : \Intro._title,
	"video"     : \Intro._video,
	"image"     : \Intro._image,
	"content"   : \Intro._content,
    ]

    @available(*, deprecated, message: "Do not call directly. Required for 'AutomaticDirectiveConvertible'.")
    init(originalMarkup: BlockDirective) {
	self.originalMarkup = originalMarkup
    }
}

This has some immediate benefits of reducing code duplication but
longer term the main goal is to create a single source of truth for
what Directives are allowed where and what arguments they expect.
The next step here will be to expose this information to DocC’s own
conceptual documentation so that directive documentation can be
auto-generated based on this metadata. Longer term, this information
can be used to power other integrations like code completion strategies.
Testing
The existing fairly extensive directive tests for each individually converted directive continue to pass. I've added some addition tests focused on the directive mirror as well.
Checklist
Make sure you check off the following items. If they cannot be completed, provide a reason.

[x] Added tests
[x] Ran the ./bin/test script and it succeeded
[x] Updated documentation if necessary

@swift-ci please test

I opened a PR on my fork to show how this new infrastructure would be used to create the new directives described here: https://forums.swift.org/t/supporting-more-dynamic-content-in-swift-docc-reference-documentation/59527.

https://github.com/ethan-kusters/swift-docc/pull/1

@swift-ci please test

Nice work, this cleans things up significantly! It would be good to have an article that describes at a high-level what needs to be done to add a new directive to DocC, either as part of this PR or a future one.

Agreed! Planning on adding that as part of a future PR that will also add auto-generated user-facing docs for any defined directives.

Planning on adding that as part of a future PR that will also add auto-generated user-facing docs for any defined directives.

👀👀👀👀👀 that sounds really neat!

@swift-ci please test

77260cf looks to have resolved the regression.
$ swift run --package-path bin/make-test-bundle  make-test-bundle --output ~/Downloads/test-bundle.docc --sizeFactor 10
$ swift run --package-path bin/benchmark benchmark measure-commits origin/main 77260cf50803d5bfc59213924a5201aef2963428 --repetitions 5 --docc-arguments convert ~/Downloads/test-bundle.docc/TestFramework/Docs.docc
$ swift run --package-path bin/benchmark benchmark diff benchmark-origin/main.json benchmark-77260cf50803d5bfc59213924a5201aef2963428.json

┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Metric                                   │ Change          │ main                 │ 77260cf50803d5bfc59213924a5201aef2963428 │
├──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Duration for 'bundle-registration'       │ no change¹,²    │ 3.555 sec            │ 3.532 sec                                │
│ Duration for 'convert-total-time'        │ no change³,⁴    │ 4.644 sec            │ 4.637 sec                                │
│ Duration for 'documentation-processing'  │ no change⁵,⁶    │ 0.997 sec            │ 1.015 sec                                │
│ Duration for 'finalize-navigation-index' │ -1.291%⁷,⁸      │ 0.045 sec            │ 0.045 sec                                │
│ Peak memory footprint                    │ no change⁹,¹⁰   │ 434.3 MB             │ 430.8 MB                                 │
│ Data subdirectory size                   │ no change       │ 111.7 MB             │ 111.7 MB                                 │
│ Index subdirectory size                  │ no change       │ 994 KB               │ 994 KB                                   │
│ Total DocC archive size                  │ no change       │ 117.8 MB             │ 117.8 MB                                 │
│ Topic Anchor Checksum                    │ no change       │ c8f03d4e81da8e3b6d7f │ c8f03d4e81da8e3b6d7fb7f63c771147         │
│ Topic Graph Checksum                     │ no change       │ edce22bce4f8de475a5e │ edce22bce4f8de475a5e58a2e45dcdc0         │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

@swift-ci please test

@swift-ci please test


@cython.cfunc should reject dunder methods

Stefan Behnel wrote:

No<EMAIL_ADDRESS>declares a function or method as a pure C function,
without a Python interface to it, and for methods, it only applies to
extension types and not regular Python classes.
It's interesting that Cython allowed you to set it on the "__iter__" method
which cannot, in fact, be a C method because it's one of Python's special
methods. We should disallow that.

I wonder what was the origin of this, any context?
@cython.cclass
class G:
    @cython.cfunc
    def __iter__(self) -> cython.int:
        return -1

or
class G:
    @cython.cfunc
    def __iter__(self) -> cython.int:
        return -1

produce reasonable errors, so I'm wondering what is the example for the described behavior?

https://groups.google.com/g/cython-users/c/japoe4kYO9A
Reading through that issue again - I think what probably happened was that they applied the decorator and then ran it in Python (rather than recompiling with Cython). Since the Shadow.py version of cfunc is a no-op it didn't generate an error.
I'll close this because the decorator behaves correctly when applied in Cython, and I don't think we want to start adding checks to the Shadow.py stubs.
Thanks @0dminnimda


org.openrewrite.java.RemoveAnnotation Precondition Question

I am running the org. recipe:
type: specs.openrewrite.org/v1beta/recipe
name: com.example.RemoveAnnotationExample
displayName: Remove annotation example
recipeList:
  - org.openrewrite.java.RemoveAnnotation:
      annotationPattern<EMAIL_ADDRESS>
The annotation is a so called Active Annotation , which Xtend uses for Java Code generation. In the generated Code, on which the recipe is applied,  it is purely informational and has no effect.
I have ran several recipes  on the code:
type: specs.openrewrite.org/v1beta/recipe
name: com.mycom.RemoveAnnotationExample
displayName: Remove annotation example
recipeList:
  - org.openrewrite.java.RemoveAnnotation:
      annotationPattern<EMAIL_ADDRESS>  - org.openrewrite.java.format.AutoFormat
  - org.openrewrite.java.format.BlankLines
  - org.openrewrite.java.format.RemoveTrailingWhitespace
  - org.openrewrite.java.format.WrappingAndBraces
  - org.openrewrite.java.cleanup.UseJavaStyleArrayDeclarations
  - org.openrewrite.hcl.format.AutoFormat
  - org.openrewrite.java.cleanup.UnnecessaryParentheses
  - org.openrewrite.java.cleanup.RemoveExtraSemicolons

And they work fine, except for the org.openrewrite.java.RemoveAnnotation
I have Java code like:
import com.mycom.mapping.ReportObject;
.....

@ReportObject
public class RoAbwicklungOe {
  private String titel;

  private String vorgabenTitel;

  private String vorgabenText;

  private String vorgabenLink;
....

and would expect the @ReportObject Annotation to be removed, but it is not.
Are there preconditions for removing an Annotation or have i misconfigured the Recipe?
Any hint or feedback very much appreciated
In the context of this Project where we are migrating  from Xtend to "pure" Java, it seems to us that openrewrite has a great potential and i am testing currently different usage scenarios

Typically when we see "basic building block" recipes like this fail, it's because they lack the corresponding types on the classpath to be able to accurately match the argument annotations. Does the project you're running this in have com.mycom.mapping.ReportObject available on the classpath when this recipe runs?

@timtebeek  great thanks for the quick reply...
Yes the @com.mycom.mapping.ReportObject  is in the classpath , albeit via a Maven Profile...  that said: the project builds without problem.
I also tried  the following:
type: specs.openrewrite.org/v1beta/recipe
name: com.mycom.RemoveAnnotationExample
displayName: Remove annotation example
recipeList:
  - org.openrewrite.java.RemoveAnnotation:
      annotationPattern: '@SuppressWarnings("all")'

on

@SuppressWarnings("all")
public class AgpsArtMapper {
  ....

And nothing happend?

Do you see that working when you fully qualify the argument annotationPattern?
---
type: specs.openrewrite.org/v1beta/recipe
name: com.yourorg.RemoveAnnotationExample
displayName: Remove annotation example
recipeList:
  - org.openrewrite.java.RemoveAnnotation:
      annotationPattern: '@java.lang.SuppressWarnings("deprecation")'

Interesting, that works! But our „inhouse“ Annotations stemming from Xtend, don‘t

I did that , no avail ....
But interesting , i found that if i have :
type: specs.openrewrite.org/v1beta/recipe
name: com.apgsga.RemoveAnnotationExample
displayName: Remove annotation example
recipeList:
  - org.openrewrite.java.RemoveAnnotation:
      annotationPattern<EMAIL_ADDRESS>  - org.openrewrite.java.RemoveAnnotation:
      annotationPattern<EMAIL_ADDRESS>  - org.openrewrite.java.RemoveAnnotation:
      annotationPattern: '@java.lang.SuppressWarnings("all")'
...

The annotations in the  testSourceDirectory are all removed! Cool , but only there .... i even added in the test sources in some sources the other two annotations... they are also removed.
In the sourceDirectory non of the above annotations are removed, that also not  SuppressWarnings. So it is likely either a configuration issue of the Recipe itself or of Maven .... although for the later everything seems ok in terms of Maven.
I will look in more detail at the documentation , and possibly at the source Code

Ahhh problem, resolved  :  i had src/main/java also additionally configured in maven as resource directory , because wie actually wanted the *.xtend files also in the jars as reference.....i removed the directory from the resourcesDirectory and voila it worked! Great , really cool ...
Although it does seems a little weird that removing :
                  <resources>
			<resource>
				<directory>src/main/java</directory>
			</resource>
			<resource>
				<directory>src/main/resources</directory>
			</resource>
		</resources>

With sourceDirectory also pointing to src/main/java did it. The maven builds otherwise worked without problem. Would be interesting to know how actually the source Files are resolved.
Ok , but no problem. Great thanks anyway! I think  we can close this Issues
openrewrite has a great potential for our usage scenario!!!
I would be great to have a recipe which deletes all setters and getters of all attributes of a Java Class , but that's another story , haha :-)

@timtebeek  Absolutely this recipe would only make sense with „bundled“ together with other recipes…which produce a consistent code basis in our usage scenario…. That said maybe one last question: would it make more sense to develop one „large“ recipe , which as result migrates a Pojo class to a Lombok @Data annotated class, which is basically removing all getters and setters for all attributes of a Pojo and to add Lombok annotation with its import….probably , since this would maintain the consistency of the Source code… versus breaking this down into several smaller recipes?

Typically we use the pattern of small reusable recipe building blocks, which are then combined into a composite yaml recipe. I imagine you will have quite a few "easy" steps, as well as some that require a dedicated Java visitor. You can combine the two in a yaml recipe that you can then apply to your sources.

Great thanks


Data retrieval

Summary

[ ] Read, manipulate, and store data

Read all data
Join data into dataframes for analysis
Store Data in centralized location (DB or s3 Bucket)

Comments/notes

Related #26

The different data formats were downloaded and an initial EDA was done


Can/Does VSCode handle system shutdown correctly?

I'm not sure if this is a feature request, or a bug, or if its even doable, or if my underlying assumptions are correct. Apologies.
Today I had accidentally clicked on "restart" on my PC's start menu (I had meant to restart a machine I had RDP'd into).
Luckily I had notepad opened with some unsaved changes and it blocked the restart enough for me to cancel the restart process.
However VSCode was open the entire time and when I came back it was in a broken state: clicking on a reference in my C# code resulted in a message actual command NOT FOUND; there was a notification that the TypeScript service had shutdown after restarting 5 times and would not be restarted, etc.
My theory is that Windows tried to shutdown these helper processes individually but had not yet sent the VSCode main process the system shutdown message, and continuously sending shutdown to those helper processes that VSCode tries to start up in response to them being shutdown (or blocking their startup altogether).
I don't think Windows sends a "system is shutting down" message to processes in a deterministic order, so would it be difficult to find out what message triggered the shutdown? I guess that would only be possible if your helpers are all hosted under some VSCode process. If it is possible the feature request would be for the helper processes to be required to forward the "windows is shutting down message" to the VSCode main process, so that it can then not only stop trying to respawn processes but can also start shutting itself down properly.
What do you people think?

I disagree that this is somehow a StackOverflow question. My question has more to do with how the application itself handles shutdowns, and a request that it handle it better if my suspicions are correct.
It's such an edge case though I can see how it might be something that no one really wants to deal with or support, and I'll probably never run into it again, so I'll leave it alone. Thanks for your time.


Developer page improvements and day summary webhook

[x] Added day summary webhook
[x] Added webhook status toggles
[x] Changed developer page UI
[x] Updated the logic to trigger only if the toggles are enabled
[x] Added some more entries to redis and some funcs

https://github.com/user-attachments/assets/3306a629-987f-4930-9b2e-c7257c87600e

Summary by CodeRabbit
Release Notes

New Features

Introduced functions to manage webhooks: enable, disable, and check status.
Added new settings for user preferences, including toggles for various webhook functionalities.
Enhanced the Developer Settings page with a more user-friendly layout and loading indicators.
Added a new ToggleSectionWidget for better management of toggleable sections in the settings.

Bug Fixes

Improved error handling and validation for webhook URLs.

Documentation

Updated documentation to reflect new webhook management capabilities.

Style

Streamlined UI components for better organization and visual appeal.

resolve conflicts pls sir ~

resolve conflicts pls sir ~

Done sir


About converting the DCN layer in the onnx model is not supported

I've successfully converted the onnx model and the inference visualization is successful.
But in the configuration I removed the DCN layer of mmcv, which caused the accuracy of the result to drop severely.
Does the author have the results of the model trained by removing the DCN layer?

Sorry, I didn't modify the original network part, just support segmentation. Therefore you should try to use the model I provide as pre-trained weights to fine-tune the model without using DCN

好的，谢谢

对不起，我没有修改原来的网络部分，只是支持分段。因此，您应该尝试使用我提供的模型作为预训练权重来微调模型，而无需使用 DCN

ok，谢谢


Infinite redirect in navigation guard

Environment
{
"name": "nuxt-app",
"private": true,
"scripts": {
"build": "nuxt build",
"dev": "nuxt dev",
"generate": "nuxt generate",
"preview": "nuxt preview",
"postinstall": "nuxt prepare"
},
"devDependencies": {
"@nuxt/devtools": "latest",
"@types/node": "^18.17.3",
"nuxt": "^3.6.5"
}
}
Reproduction

Describe the bug
Infinite redirect in navigation guard
Additional context
No response
Logs
https://codesandbox.io/p/sandbox/recursing-davinci-f6p5pf?file=%2Fsrc%2Fmiddleware%2Fauth.global.js%3A2%2C1-7%2C4

https://codesandbox.io/p/sandbox/recursing-davinci-f6p5pf?file=%2Fsrc%2Fmiddleware%2Fauth.global.js%3A2%2C1-7%2C4

You should exclude /login from your auth middleware checks ☺

Thank you!🥰


Add support for content collections, like blog posts

Done in release 1.0.0: https://github.com/tightenco/jigsaw/releases/tag/v1.0.0


🚀 [New Contribution] open source

** A blog on open source **

You no longer need to create ISSUE/PR for the low-code and no-code contributions. Please check this communication here and follow the same(especially the ground rules). We will pick the contributions from there. Thanks!
https://www.showwcase.com/opensource


July 31 Release Cycle: Remove PrototypeMessageFeature

CC @JoeWang1127 and @blakeli0 : Release managers for the July 31 release train
PR #1841 introduces reflect-config.json generation.
Once this feature has been released and client libraries have been regenerated with the updated gapic-generator-java, remove the PrototypeMessageFeature and -H:-RunReachablityHandlersConcurrently arguments from gax-grpc's native-image.properties.
See also https://github.com/googleapis/sdk-platform-java/pull/1841#discussion_r1262700502
Ensure that we perform downstream testing on client libraries before release to minimize the chance of downstream build failures.

Done in #1892


Encoded data is encoded once more

With input.decode_cf set to false (the default), nc2zarr loads encoded data into memory. The first time a variable is written, its encoded data is correctly written as-is even if it contains the scaling_factor or add_offset attributes. But when nc2zarr (actually the xarray.to_zarr() method) attempts to append a new time step to existing data, it uses the existing variable encoding information to encode the already encoded data before writing it. This leads to corrupt datasets if  scaling_factor or add_offset attributes are used.
The actual issue has its root cause in xarray.to_zarr() method that behaves inconsistently. A new encode_cf keyword argument would also help.

xarray as of version 0.18.2 always CF-encodes variable data according to encoding information of existing variables before it appends new data. If the data to be appended is already encoded (because by default we read it with dataset = xr.open_dataset(..., decode_cf=False)) then this leads to entirely corrupt data. A possible hack is to redundantly dataset = decode_cf(dataset) before appending so encoding again is finally a no-op. This of course also costs extra CPU.
A better solution is to fix xarray's inconsitent dataset.to_zarr() behaviours when either creating variables or appending data to existing variables. I'll post an xarray issue. For example, when I uncomment line 469 in xarray/backends/zarr.py, the this fixes our issue too:
https://github.com/pydata/xarray/blob/1b4412eeb7011f53932779e1d7c3534163aedd63/xarray/backends/zarr.py#L460-L471


too much unnecessary information with new design

The thumbnail view take too much space and doesn't give any useful information to the user.
Every time i have to click on "More" to get to the lecture that i want to watch, which is waste of clicks and harder to navigate.
The previous version with list view was more compact and gave the user a nice overview of all the videos for each lecture.

Thanks for the feedback!
The more button will be gone after #1122 is merged. Sorry for that. I really don't know why I paginated the streams on the course page 😅 Furthermore, the streams will be once again be grouped by month for ease of navigation.


refactor: llmq only logging

Issue being fixed or feature implemented

What was done?

Enabled a particularly long logging only if llmq logging category is active
How Has This Been Tested?

Breaking Changes

Checklist:

[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have added or updated relevant unit/integration/functional/e2e tests
[ ] I have made corresponding changes to the documentation

For repository code-owners and collaborators only

[x] I have assigned this pull request to a milestone

@UdjinM6 Applied suggestions

@UdjinM6 Done


Add blocks-frontend.js for blocks frontend scripts

Why we need blocks-frontend.js file and what problem it will solve?
To run scripts on front-end with src/blocks.js you can only run scripts in the admin area.
For example, I created testimonial carousel block to make carousel run in the admin area and on front-end I have to call the carousel js plugin both on front-end and backend.

That's why blocks-frontend.js file is necessary to be in the great toolkit.

Pretty sure adding a frontend je file will be a bad idea. Your code should look the same for both backend and frontend. Having separate code for both means you are doing it wrong just like we did in the classic editor.
I have not yet been convinced of why both frontend and backend code can't be the same.

@ahmadawais that's true, but I wouldn't want to include the whole blocks.build.js file on the frontend just to make the block's dynamic functionality work on frontend. That's why a separate frontend file is needed to load only code required there.

What dynamic functionality is different on the backend?

Let's use the example from the original comment:
On the backend you have the whole registerBlockType function with your block configuration, where in edit function you render the html and initialise the carousel's JS code.
However, the save method doesn't render components, it renders plain html. So on the frontend to make the carousel work you would need to enqueue a separate JS file which would initialize the carousel's dynamic functionality (switching slides).
I had a similar problem which is why I stumbled upon this pull request. I created a contact form block. It doesn't do anything on the backend, the edit method looks like this:
edit: () =>
Contact form will appear here.

And the save method renders an actual HTML form. On the frontend, I would like to load a JS file to handle form submit with AJAX. I could create a plain separate js file somewhere in the plugin and load this file. But what if I want this file to be built with webpack? This would require me to write a separate webpack config, install all dependencies, etc.

Hi guys, I am agreed with @ahmadawais somehow.
blocks-frontend.js will work fine if the slider block has no options like autoplay: true; or nav: false; but what if each slider block has different options to initialize.
The solution to this problem is we can initialize javaScript to block with HTML like
 or with inline javaScript.
And loading blocks-frontend.js on every page even that page does not have the block for that we are using blocks-frontend.js is waste of resources.
But we need a good method to fulfill our frontend needs. Like the dynamic block slider that renders in PHP.
Solution:
Import vendors javaScript (slider.js) file in dist/js/vendors directory then load slider.js using  has_block() funcation to determine whether a $post contains a specific block.
@ahmadawais what is your opinion?

@jedlikowski Exactly! That's where you are wrong. I don't want people to develop things like that. That's the wrong way. You must be thinking — what even am I talking about.
The thing is we moved away from the old editor because it was really hard to have "What You See is What You Get" in it. We built Gutenberg so that there will be no difference between the backend and the frontend. So, that in a couple of years there won't be a backend editor at all.
You'd build content at the frontend.
When you do something like this
edit: () =>
Contact form will appear here.

You are doing it wrong.
Your backend should be exactly like the frontend. That's why Gutenberg was built. That's why I chose not to add a frontend file in there — to inspire people in changing their mindset and to sort of enforcing "What You See is What You Get" probability.
It's why I have not merged this PR and haven't yet met a strong enough reason for separate frontend file.

@amjadr360
Like the dynamic block slider that renders in PHP.

That's not what I would do. I would keep everything in JavaScript. I'll use PHP to create a new REST API endpoint and get data from that endpoint inside the JavaScript.
Stop thinking in PHP and start thinking wtih JavaScript otherwise adding JavaScript to WordPress is pretty much useless. You're basically trying to write another language in another language. That works for PHP → HTML and shouldn't work for PHP → JS.
Peace! ✌

@ahmadawais the edit function looks like this:
edit: () =>
Contact form will appear here.

only for simplicity. Of course I could render the whole contact form on the backend as well. This even would force me to extract the form rendering into an external module to not repeat the code.
But my main problem still exists, where would I place the JS code to handle form submit via AJAX? Add it inline to the rendered HTML?
I think it would be a better idea to load a separate JS file which would handle form submit. Now, where do I place this ajax code? I could create a JS file and load it alongside my block. But what if it's a more complex JS code which needs to be built by webpack. How do I go about it?
This problem could be generalised like this: what if given block requires JS code which needs to be built with webpack? How should such JS code be built and loaded?

@ahmadawais Is there some misunderstanding about Gutenberg's functionality here?

You are doing it wrong.
Your backend should be exactly like the frontend. That's why Gutenberg was built. That's why I chose not to add a frontend file in there — to inspire people in changing their mindset and to sort of enforcing "What You See is What You Get" probability.
It's why I have not merged this PR and haven't yet met a strong enough reason for separate frontend file.

React doesn't load on the Wordpress frontend, so any functionality that uses React's capabilities wouldn't run by default. Gutenberg was built with the intention that your front-end functionality would follow older practices, it seems. I'm trying to get a handle on why we would load all of the backend Javascript (a great deal of which is unnecessary) rather than load in 'interaction JS' separately.
Thanks for considering my questions :). For what it's worth, we've just enqueued a separate, single JS file with some jQuery for the few blocks that need it.

@ahmadawais I agree with you here and understand what you're trying to achieve. So I don't think it is necessary to push this.
But perhaps you could help us understand something that seems to boggle our minds based on what I'm thinking and reading?
If we want some sort of frontend .js functionality within our plugin (examples below) - what's the best way to implement this in the plugin? Is it possible to use the compiler to our advantage to achieve this? Or is it better to just

For what it's worth, we've just enqueued a separate, single JS file with some jQuery for the few blocks that need it.

?
Examples:
changing classes dynamically based on user interactions to create certain user experiences such as
accordions,
sliders,
carousels,
form validations and error messages
I know this solution has been sugggested, which worked. Thoughts on this?

Just following along but would really love to get some feed back on the last few questions posed here.  I'm specifically having issues with:

If we want some sort of frontend .js functionality within our plugin (examples below) - what's the best way to implement this in the plugin? Is it possible to use the compiler to our advantage to achieve this?

Currently we have way to load js on the frontend that is required for client side interactions for blocks.  The example of a form with AJAX is a great one.


Update install steps to include required args

Added new steps to the install instructions to include required args.
Without these args, the install succeeds but the service will fail to
start.

-agree, required for the Let's Encrypt certification
-email, docs says optional but service would not start without it.

This post on the Caddy forum helped direct me to the solution:
https://caddy.community/t/caddy-unable-to-start-as-a-service-on-window-server-2008-r2/3646/37

For more details, please see this post on a current open issue.
https://github.com/hacdias/caddy-service/issues/18#issuecomment-417161934

Thank you @hummerpj! Merging 😄

Also thanks for this super handy plugin.  Works great!


Improve S3.cpp

S3.cpp needs to be refactored to a more object-oriented interface. As of now, this module has a couple of issues:

no way to create Aws::Client's with different options
no easy way to pass clients to S3 without passing these AWS::Client handles

S3.cpp should be a class (I suggest S3Client) so users can create multiple of these clients.

Done in #46


npm shows latest release to be 1.4.2

If i do an npm install bower it installs the old version of 1.4.2. It is also marked as being the latest release on https://www.npmjs.com/package/bower
But the latest release should be 1.6.4 or even 1.6.5?

Try uninstalling bower then reinstalling?
npm uninstall -g bower
npm uninstall -g bower

My bower was version 1.6.5.
However, after uninstalling I try to reinstall, it reinstalls old version 1.4.2.

@elebumm That didn't help. You should try that then you would get version 1.4.2 aswell ;-) Like @changbenny
The problem is the following:

@vladikoff is one of release managers. can you help here @vladikoff ? Thanks

Ah I see there was an issue with patching old npm releases, working on it ...

Fixed in 1.6.6
There are several issues with npm here, see https://github.com/npm/npm/issues/6778
The tag APIs is 500ing in several ways as well, so I just published 1.6.6 :
npm info bower

{ name: 'bower',
  description: 'The browser package manager',
  'dist-tags': { latest: '1.6.6' },

@vladikoff Did you release current master as 1.6.6? It was WIP and there were failing builds..

Ach. I see what you did. Allright :+1:

@vladikoff Actually bower was released in wrong way. You did not follow all the steps from https://github.com/bower/bower/wiki/Releasing-bower (specifically rm -rf node_modules && npm install --production)
Because of it Bower has been released without bundledDependencies bundled. Please remember about it next time.

@sheerun oh didn't know that page existed, sorry! bookmarked it now.

@sheerun let me know if we should update those

I'll re-release it :)


Question about Track

Hello, Alex!
I have a question about track for the fullsize model.
I work in the fork of Your rep and already have implemented your C++ version in  Matlab, using Matlab classes.
There is a README in the rep, so You could easily check it out if You will have time.
My forked rep has some differences in car model (Other Pacejka tire model for lateral slip), new control input (brakes rate of change) and some other minor changes.
Also in my fork I have integrated Formula Student tracks from FSSIM and eufs_sim, but have problems with driving on them. MPCC works unpredictable and fails on the first lap corner on the FSG track.
On your track for the fullsize car MPCC works pretty great, so I have a question about requirements for the the track.
What requirements should be taken into account, when we generate track bounds and track center line?

The tunning of the weights can sometimes be a bit hard, when I added braking I have also seen that it makes the problem quite a bit harder.
Some questions:

Did you visualize the curvature of your tracks, if the curvature is very noisy it can result in issues and you may need to perform some smoothing, I normally use b-splines for this task.
Are you using the fullsize branch with the "more correct" SQP implementation (the new branch now fixes one final strange detail in the fullsize branch where I used a mix of RK4 and matrix exponential discretization.

@alexliniger Thank You for the quick response!

I have used your resample algorithm from ArcLengthSpline.

With your algorithm I have resampled 94 points center line to 5000 points centerline.
Also I have visualized the final track center line after resampling, it looks quiet smooth.
If You have some free time, You could simply run my version in Matlab and watch the plotted center line (plotting of it is enabled by default).

I have used a fullsize branch for my fork.

What do You mean, when speaking about new branch? Is it a fullsize branch or feat/cppad-codegen branch?

I have unfortunately no access to the Matlab  anymore. We have seen that resampling centerlines does not always work good. But for MPCC it is not as critical because the curvature only enters in the cost function and not in the dynamics. Generally resample with a b-spline including smoothing should work better.
With new branch I meant cppad-codegen, there you can also find parameters for a car with braking, but formula student tracks are always a bit special due to the tight boundaries. Maybe try smaller weights on the soft constraints (really small) and see if this helps. The soft constraints can make the problem unstable sometimes.

Hello, Alex!
I have implemented your C++ codegen version of MPCC in my fork with small changes of Cost values and model parameters. Practically all is the same.
I have added some FS tracks from FSSIMand eufs_sim to the C++ implementation. Now all track .json files are stored in Parameters/tracks folder. More Information about it You could find in C++/README.md file in my fork.
Now in the rep there are two implementations of FSG track:

Not presplined (FSG.json);
Presplined (FSG_nirajbasnet.json) from nirajbasnet rep.

On presplined version algorithm works better. You could easily check it (presplined FSG version is enabled by default).
But error status (1) appears very often, so I think that something works wrong, but I do not know where to look for the problem now.
Could You give me a quick explanation about methods to find such problems in MPС? May be there are some techniques for tuning MPC parameters and debugging (visualizing) MPC (dynamics/constrains) in the different horizon points?
Thank You in advance!


Add a basic GPU job test as part of our E2E tests.

Fixes #164

This change is

/test all

/test all


Sit packer build timeout to 5 mins

Was having timeout issues running packer build apache-cassandra.

Thanks for the contribution!


BUG: Filter corner frequencies are not always sorted correctly

In some cases it is not possible to design a filter although the displayed filter specifications seem to be o.k. It seems that the corner frequencies are passed in the wrong order to the actual filter design routine.
This bug occurs from time to time when changing the response type (e.g. from bandpass to bandstop).
Most likely, the bug is in input_widgets/input_freq_specs.py, input_specs.py or input_targ_specs.py.

The root cause was the sloppy creation and deletion of widgets fixed in PR #23.


[Quick Accent] fix showing toolbar in the corner position

Summary of the Pull Request
Fix showing the Quick Accent toolbar in the corner.

PR Checklist

[x] Closes: #22735
[x] Communication: I've discussed this with core contributors already. If work hasn't been agreed, this work might be rejected
[ ] Tests: Added/updated and all pass
[ ] Localization: All end user facing strings can be localized
[ ] Dev docs: Added/updated
[ ] New binaries: Added on the required places

[ ] JSON for signing for new binaries
[ ] WXS for installer for new binaries and localization folder
[ ] YML for CI pipeline for new test projects
[ ] YML for signed pipeline

[ ] Documentation updated: If checked, please file a pull request on our docs repo and link it here: #xxx

Detailed Description of the Pull Request / Additional comments

Validation Steps Performed

This should be fine - but it might cause the window to resize rapidly when going to characters with descriptions enabled?

This should be fine - but it might cause the window to resize rapidly when going to characters with descriptions enabled?

Yes, that's true. Any suggestion on how to fix it another way?

This should be fine - but it might cause the window to resize rapidly when going to characters with descriptions enabled?

Yes, that's true. Any suggestion on how to fix it another way?

Currently, we're using a transparent Window and visualizing a pseudo-window with a border that is centered within the Window. Setting it to Left/Right/Center (based on the toolbar position) might work?
https://github.com/microsoft/PowerToys/blob/cd1b7dbc2c62f89e2fa5ca32018fae3df320c82d/src/modules/poweraccent/PowerAccent.UI/Selector.xaml#L45

Thanks! Setting HorizontalAlignment="Center" works nicely when using a top left corner.
Will have a look at how to set "Left/Right/Center" according to toolbar position.


Cocoapods doesn't add MixPanel correctly to swift project

In my swift project.

I add pod 'Mixpanel', '~> 2.5.4' to my podfile
pod install  (i am running 0.34.4)
open xcode add #import "mixpanel.h" to my objective-c bridging header
create a shared instance of Mixpanel
try to build and see errors saying Mixpanel is not defined

my temp fix

Add lib-*-mixpanel.a to linked binaries
now i see "_utf8_nextCharSafeBody", referenced from: _validate_dispatch_data_partial_string in libPods.a(MPWebSocket.o)
Add libuicore.dylib to my linked binaries

now my build works :)
I was persistent but you almost lost a customer there :)

https://github.com/mixpanel/mixpanel-iphone/issues/225

This should be resolved now if you want to do a pod install.


Custom button action not called

I have added a custom button in callout.
    UIButton *Url_btn = [UIButton buttonWithType:UIButtonTypeCustom];
    Url_btn.frame = CGRectMake(2, 2, 296, 51);
    [Url_btn addTarget:self action:@selector(MyMethod:) forControlEvents:UIControlEventTouchDown];
    [Url_btn setTitle:@"Show View" forState:UIControlStateNormal];
    Url_btn.backgroundColor=[UIColor grayColor];
    [Url_btn setTitleColor:[UIColor blueColor] forState:UIControlStateNormal];
    [self.calloutView Url_btn];

   -(void) MyMethod:(UIButton *)sender
  {
       NSLog(@"Clicked Callout Button");
  }

Button is displaying properly in callout but action is not working.

Ok, got Solution from this http://goaheadwithiphonetech.blogspot.in/2014/07/custom-callout-button-is-not-clickable.html
I have changed in this method in DXAnnotation class
- (UIView *)hitTest:(CGPoint)point withEvent:(UIEvent *)event
{
    UIView* hitView = [super hitTest:point withEvent:event];
    if (hitView != nil)
    {
        [self.superview bringSubviewToFront:self];
    }
    return hitView;
}

- (BOOL)pointInside:(CGPoint)point withEvent:(UIEvent *)event {
    CGRect rect = self.bounds;
    BOOL isInside = CGRectContainsPoint(rect, point);
    if(!isInside)
    {
        for (UIView *view in self.subviews)
        {
            isInside = CGRectContainsPoint(view.frame, point);
            if(isInside)
                break;
        }
    }
    return isInside;
}


python310Packages.bond-api: 0.1.17 -> 0.1.18

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python310Packages.bond-api is: "Asynchronous Python wrapper library over Bond Local API"
meta.homepage for python310Packages.bond-api is: "https://github.com/prystupa/bond-api"
meta.changelog for python310Packages.bond-api is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
Warning: a test defined in passthru.tests did not pass
found 0.1.18 with grep in /nix/store/cxamq7aa1viajlmnp46s7c0m395rwv1k-python3.10-bond-api-0.1.18
found 0.1.18 in filename of file in /nix/store/cxamq7aa1viajlmnp46s7c0m395rwv1k-python3.10-bond-api-0.1.18

Rebuild report (if merged into master) (click to expand)

5 total rebuild path(s)

3 package rebuild(s)

3 x86_64-linux rebuild(s)
2 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
home-assistant-component-tests.bond
python310Packages.bond-api
python39Packages.bond-api

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/cxamq7aa1viajlmnp46s7c0m395rwv1k-python3.10-bond-api-0.1.18 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python310Packages.bond-api https://github.com/r-ryantm/nixpkgs/archive/6c73f6fead48d578181cff4c78eeae39784951a7.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/cxamq7aa1viajlmnp46s7c0m395rwv1k-python3.10-bond-api-0.1.18
ls -la /nix/store/cxamq7aa1viajlmnp46s7c0m395rwv1k-python3.10-bond-api-0.1.18/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

3 packages built:

home-assistant-component-tests.bond
python310Packages.bond-api
python39Packages.bond-api

Maintainer pings
cc @dotlambda for testing.

duplicate of https://github.com/NixOS/nixpkgs/pull/173659


Copy button does not copy empty lines

When you have a file with empty lines inside, to be exact - two consecutive newline characters surrounded by other text, the Copy button copies the contents of the file, but replaces the two newline characters with only one.
The result is more condensed file than the original with no empty lines.

OK, it seems it doesn't happen with every file. I'll reopen when I have a concrete reproducible example.


Please invite me to the GitHub Community Organization

Name
Micheal Aderinto
Discord Username (if applicable)
ad3rinto#6785
Additional Context
I first came across Eddie on twitter about 2 years ago when he posted a guide to make your GitHub page more attractive.
I have followed his work since then

Done.
Joined discord and the community server.

You are already part of the GitHub org. 🔥


Add Elastic Common Schema formatting

Feature request to add Elastic Common Schema formatting

@rubelw how this relates to Airlift?

@wendigo Disregard...Trino uses airlift for logging...really is a Trino issue - and utilizing a ECS compliant formatter.  Thanks!


chore: Revert "fix: Update attachment template format to support PVA"

fixes #7564
opens https://github.com/microsoft/BotFramework-Composer/issues/7394

Coverage decreased (-0.0002%) to 50.807% when pulling 772ec66a7b6f52b9f0c0026351368ebb48eb7d51 on revert-7564-durnford/fix/lg-attachments into ee715eb9ee336dce7aabbb2eed585f65ef4dfc3d on main.