orderBy gives an error when a space separated string of sortBy and sortOrder is passed in eg- orderBy('sortBy sortOrder')

Kysely documentation allows a space separated string for sortBy and sortOrder to be passed in to orderBy - https://kysely-org.github.io/kysely-apidoc/interfaces/SelectQueryBuilder.html#orderBy
However when this syntax is tested on the Kysely playground it gives an error "argument of type "age desc" is not assignable to parameter of type 'OrderByExpression<From<Database, "person">, "person", AllSelection<From<Databased, "person">, "person">>'
This does not work in VsCode either when trying to pass in an array of strings for multiple sort. It gives an error
  Overload 1 of 3, '(orderBy: UndirectedOrderByExpression<Database, "tableName", { description: string; ... 4 more ... }>, direction?: OrderByDirectionExpression | undefined): SelectQueryBuilder<...>', gave the following error.
    Argument of type 'string[]' is not assignable to parameter of type 'UndirectedOrderByExpression<Database, "tableName", { description: string; ... 4 more ... }>'.
  Overload 2 of 3, '(ref: DirectedOrderByStringReference<Database, "tableName", { description: string; ... 4 more ... }>): SelectQueryBuilder<...>', gave the following error. Argument of type 'string[]' is not assignable to parameter of type 'DirectedOrderByStringReference<Database, "tableName", { description: string; ... 4 more ... }>'.
  Overload 3 of 3, '(refs: readonly OrderByExpression<Database, "tableName", { description: string; ... 4 more ... }>[]): SelectQueryBuilder<...>', gave the following error.
    Argument of type 'string[]' is not assignable to parameter of type 'readonly OrderByExpression<Database, "tableName", { description: string; ... 4 more ... }>[]'. Type 'string' is not assignable to type 'OrderByExpression<Database, "tableName", { description: string; 4 more ... }>'.ts(2769)

Hey 👋
Can you provide a kyse.link that reproduces this?

Seems to work for me https://kyse.link/FLt0L


Front door 🚪

TommyPollyblank has invited you to join a Blend on Spotify. Join on the Spotify mobile app. https://open.spotify.com/blend/taste-match/e6627badc4e8ca17?fallback=getapp

Initial-commit-

#work


Anonymous Session looks weird

This is not an issue anymore since we don't have anonymous sessions anymore.


'react-localization' has no default export

Hi,
I am getting the following error in ES2015 and v0.1.2:
'react-localization' has no default export
Can you please help?

Hi,
I've reviewed the definition file and now it seems to work... just update to version 0.13


Add pagination

Fixes #59
Pagination is handled like a django ListView, and templates should be designed in the same way.
When pagination is enabled the default build locations for the first page are: build_folder/build_path and /build_folder/build_page_name/1/build_path, with other pages only built in: /build_folder/build_page_name/<page_number>/build_path

Coverage remained the same at 80.21% when pulling 75eabb0a2974de589493e0b1d3dec9b33230f9d3 on robert-pringle:master into 2c9c495e4e8faca7b81fa57635d1631933f14171 on datadesk:master.

Any Interest in this patch?

Something I have problem with, self.queryset should be self.get_queryset(), or store the result somewhere and use it as parameter to later method calls.
The default build_path for pagination looks weird to me, I'd prefer something like:
- build_folder/basename.ext
- build_folder/basename-page-2.ext

Also the links from pager is not matched, you'd get "?page=2"

I think you can use functools.partial to store the result of self.get_queryset()

Check https://github.com/dozymoe/django-bakery/commit/5dbc0c46c045e13c082286e85461250fa43ec0b6
I have not updated the test or docs.
Have not fixed the pager links either.

Oops. The "?page=N" came from my django templates.

Oops, the "?page=N" came from my Django template.

Checkout my latest main branch: https://github.com/dozymoe/django-bakery/tree/main
This will create pager url:
{% load bakery %}
Next Page

Something I am not clear about, is how Django construct full url, in my code I'd just make something like href="/index.html" but what if Django run under sub-directory in the server, when it should be href="/subdir/index.html".

This will create pager url:
{% load bakery %}
Next Page

I'd use the standard Django method of adding pagination to a template, the build path is not necessarily the same as the url.
For example if you're using urls without a file extension like "www.example.com/pagename/" this will load the index.html file on most web servers.
{% if page_obj.has_next == True %}

{% endif %}

I think my patch has problem with urls created by Django, as in it won't work with the dev server, still need fixes.
I made https://github.com/dozymoe/frozen-django if anyone interested.
It's dumb, can't export to AWS or static/media files, still a work in progress.
But fit my use-case for now, I am using Django signals to trigger updates.


Adding Icosahedral Sphere

bors r+

Thanks! That should be in comments

On Sep 25, 2017, at 13:35, Simon Rönnberg<EMAIL_ADDRESS>wrote:
@Rhuagh commented on this pull request.
In src/icosphere.rs:

+use std::collections::HashMap;

+use cgmath::{InnerSpace, Vector3};
+
+use {Vertex, Triangle};
+use generators::{IndexedPolygon, SharedVertex};
+
+/// Icosahedral sphere with radius 1, centered at (0., 0., 0.)
+#[derive(Clone, Debug)]
+pub struct IcoSphere {

i: usize,
vertices: Vec<[f32; 3]>,
faces: Vec<[usize; 3]>,
+}

+const T: f32 = 0.85065080835204;
An icosahedrons vertices can be defined as the corner points of three orthogonal golden rectangles, which means the sides of the rectangles have the golden ratio: (1 + sqrt(5)) / 2. If you take that directly though you won't get a unit sphere, so we need to normalize the points (0, ±1, ±ratio), which gives those 2 values. I precomputed those so it wouldn't be necessary runtime.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.


Refresh/sync not working with Oreo

When I click "Aktualisieren" (Refresh/Sync?) in the app, nothing happens on Android 8.1.0
I open the caldav app afterwards and see something like "last sync: yesterday ..." instead of "one minute ago" or similar.
What is the intention of the "Aktualisieren" button? Is it the same as in calDav "Jetzt synchronisieren"?

Yes, this button is supposed to sync all accounts with task support. Could this be a permission issue? Does the app have the permission to read all accounts on the device?

Thanks, that fixed it. Read and write permission for tasks was not granted, only read contacts.
Am 8. August 2018 00:52:59 MESZ schrieb Marten Gajda<EMAIL_ADDRESS>
Yes, this button is supposed to sync all accounts with task support.
Could this be a permission issue? Does the app have the permission to
read all accounts on the device?
--
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmfs/opentasks/issues/694#issuecomment-411228246


[MHRise] seems to crash after a couple quests

log&DMP.zip

This is a known issue. We are looking into it.

Should be fixed by b98758ce3b1c0986e1bae706d92b0ce322fb57d2.


[HTML] Parameters and properties type not added to doc

When using deno doc without the html option and JavaScript files, it correctly output the attributes/properties/parameters type hint, see below.
When using the html, the rendered html does not output type hints, see the capture below.
In html there is also no output for the constructor documentation not class properties.
User story if needed: HTML documentation output should render type hints on JavaScript files and also display class properties information
Without html option Command Line:
gitpod /workspace/pwa (master) $ deno doc  ./public/object/media.js
Defined in file:///workspace/pwa/public/object/media.js:44:1

function toto(a, b): void
  Function that does stuff

  @param {number} a
      a

  @param {string} b
      c

Defined in file:///workspace/pwa/public/object/media.js:12:1

class Media
  Media object definition

  @property {string} type
      Media Type

  @property {string} title
      Media type

  @property {string|string[]} authors
      Media authors

  @property {string} description
      Media description or abstract

  @property {string} coverImageUrl
      Media cover image URL

  @property {Date} publicationDate
      Media publication Date

  @public
  @constructor

  constructor(title, authors, description, coverImageUrl, publicationDate)

    @param {string} title
        Media type

    @param {string|string[]} authors
        Media authors

    @param {string} description
        Media description or abstract

    @param {string} coverImageUrl
        Media cover image URL

    @param {Date} publicationDate
        Media publication Date

With html option

If the issue is not well-formatted or need more information I will gladly improve its content
BTW: thanks for implementing the HTML option, I have been waiting for it :)

This i an odd bug.
Can you share your code, and is this on deno.land/x and if so, does it render the same or is it correct there?

I'm having the same Issues.
Additionally it also ignores JSDoc typedefinitions.
Something like:
/**
 * @typedef  {Object}    input          - An input
 * @property {number[]}  input.string - Example prop
 * @property {number}    input.num - Example prop
 */

/**
 * To decode the uplink data
 * @access private
 * @param   {input}     input - The object to decode
 * @returns {output}          - The decoded object
 */
function decode(input) {
 ...
}

is not included in the HTML output.

This i an odd bug. Can you share your code, and is this on deno.land/x and if so, does it render the same or is it correct there?

I do not publish on deno.land/x, thus I do not know if it also occurs on it.
I edited my issue to add a link to my repository: https://gitlab.com/libshare/pwa
TypeScript documentation seems to be generated with the types in it, I suppose the {type} info from doc comments is not correctly parsed. With TypeScript it is probably taking the real type directly. Example from deno note seems to work as expected on TypeScript example from deno

I created a new repository with only one JavaScript file, with a function and a class in it, with JSDoc comments.
I host it on both Gitlab and Github, and attached a hook to deno.land/x.
it is in order to answer with as many info as possible your question @crowlKats :

This i an odd bug. Can you share your code, and is this on deno.land/x and if so, does it render the same or is it correct there?

I hope I did it correctly for the hook, it's my first time.
Here are links for you to check:

gitlab repository
gitlab pages, for the locally generated HTML JSDoc, with deno doc --html

the full command for the deno doc is in deno.json, task: doc

github repository
deno land /x

Sorry to dig up an old-ish issue, but I'm having the same problem with constructor parameters intermittently. I'll try to throw together a repro as soon as I can.


Fix embedStrings.finish.endMessage not being taken

This pull request fix the payload set on embedStrings.finish.endMessage not being used when a giveaway ends.

It was always using the payload set in newGiveawayMessage property.

thanks!


Login and autentication

[x] use autentication with jwt
[x] create login api to retreive jwt (basic for now)
[ ] retreive data of user from jwt to be used in function

[x] included SSO

now, instead of using the token provided directly from oauth, create a custom jwt token with the server roles (to be added later)  and his data like the unique name

[ ] create custom jwt with unique name inside


Project specification

Project description
A REPL command-line client which allows a user to interact with an LDPoS blockchain.
Overview
LDPoS is a simple and lightweight DPoS blockchain which uses a hash-based signature scheme (Lamport OTS with Merkle Signature Scheme) which is believed to be resistant to potential quantum computing attacks.
LDPoS has the following features:

DPoS (Decentralized Proof of Stake) algorithm.
Stateful signature scheme which allows a user to change their wallet passphrase. Could also be used to transfer ownership of an account without revealing private keys.
Dynamic multisignature groups; multisig accounts can add and remove members over time provided that there is consensus within the group.
Compatible with Lisk DEX - https://ldex.trading/
Simple consensus algorithm which supports instant finality (blocks can never be rolled back after they are processed).
Deflationary; no forging rewards aside from transaction fees (shared between all block signers).
Dynamic fees.

Deliverables

An interactive REPL command line client written in plain JavaScript (Node.js) which allows a user to interact with an LDPoS blockchain (I.e. creating signed transactions, signing multisignature transactions, submitting transactions, fetching an account's details based on walletAddress, fetching block details based on id, fetching transaction details based on id, etc...). This REPL client should basically be a command line wrapper for the ldpos-client-js client library https://github.com/Leasehold/ldpos-client-js. The command should be called ldpos for simplicity.

Background work
The ldpos-client-js library has already been added as a dependency to this project and has been published to npm under the package name ldpos-client. Also published on npm here: https://www.npmjs.com/package/ldpos-client
A mocha integration test suite has been setup.
The ldpos-client library already has most of the functionality related to signing and verifying transactions but additional functionality will be added to load data from the bockchain node.
To understand how the ldpos-client library works, check out this sample client: https://github.com/Leasehold/ldpos-client-js/blob/master/scripts/sample-client.js. You can run it with the command node sample-client.js <IP_ADDRESS>. Feel free to use the node at <IP_ADDRESS> testing; it's a mock/dummy node (it's missing the database layer); it's not forging or processing transactions yet but it can receive and respond to certain actions and should be enough to get started.
It may also help to git clone this repo https://github.com/Leasehold/ldpos-chain and run the tests (they also make use of the JS client) in order to get a broader idea of the kinds of functionality which needs to be supported.
Check the issues section of this repo for more specific issues and feel free to raise new ones.
Additional requirements

Since the underlying LDPoS client is stateful, the client will need to be interactive; the user should run the ldpos command to connect to a database node; this should launch an interactive REPL which allow a user to input specific commands interactively.
Use async/await (no callbacks or then blocks).
Do not add new dependencies unless absolutely necessary. Only add dependencies that you trust or if you know the author personally. Assume that any dependency which you don't control or from an unknown author is a vulnerability.
Plain JavaScript; currently there is no build step used in any other part of the project so it's better to keep it simple.
As simple as possible; no complex abstractions which are difficult to explain in simple terms.
Should works out of the box after npm install -g ldpos-commander

@bartekus I added remaining methods which you can now use: https://github.com/Leasehold/ldpos-client-js/blob/c2ef83dfc949a0202a34de6d23c94897654f11e0/index.js#L453-L546

@bartekus I added remaining methods which you can now use: https://github.com/Leasehold/ldpos-client-js/blob/c2ef83dfc949a0202a34de6d23c94897654f11e0/index.js#L453-L546

Hey Jon,
Is there a specific node version I should be running?
As it stand, currently I can ping <IP_ADDRESS> no problem, but running the sample-client is failing to connect:
❯ node sample-client <IP_ADDRESS>
/Users/bart/Dev/Leasehold/ldpos-client-js/sc-adapter.js:22
      throw new Error(
            ^

Error: Failed to connect because of error 1006: Socket hung up
    at SCAdapter.connect (/Users/bart/Dev/Leasehold/ldpos-client-js/sc-adapter.js:22:13)
    at processTicksAndRejections (node:internal/process/task_queues:93:5)
    at async LDPoSClient.connect (/Users/bart/Dev/Leasehold/ldpos-client-js/index.js:80:7)
    at async createClient (/Users/bart/Dev/Leasehold/ldpos-client-js/index.js:586:5)
    at async /Users/bart/Dev/Leasehold/ldpos-client-js/scripts/sample-client.js:9:16
❯ node -v
v15.4.0

Hey Jon,
Is there a specific node version I should be running?
As it stand, currently I can ping <IP_ADDRESS> no problem, but running the sample-client is failing to connect:
❯ node sample-client <IP_ADDRESS>
/Users/bart/Dev/Leasehold/ldpos-client-js/sc-adapter.js:22
      throw new Error(
            ^

Error: Failed to connect because of error 1006: Socket hung up
    at SCAdapter.connect (/Users/bart/Dev/Leasehold/ldpos-client-js/sc-adapter.js:22:13)
    at processTicksAndRejections (node:internal/process/task_queues:93:5)
    at async LDPoSClient.connect (/Users/bart/Dev/Leasehold/ldpos-client-js/index.js:80:7)
    at async createClient (/Users/bart/Dev/Leasehold/ldpos-client-js/index.js:586:5)
    at async /Users/bart/Dev/Leasehold/ldpos-client-js/scripts/sample-client.js:9:16
❯ node -v
v15.4.0

@bartekus You can try pulling the latest master branch again. I was fixing some stuff today.
Also I took down the test node temporarily. I'll try to get it back running in a few hours.

@bartekus You can try pulling the latest master branch again. I was fixing some stuff today.
Also I took down the test node temporarily. I'll try to get it back running in a few hours.

@bartekus About node.js version. I'm using v11.13.0 but I'll try to get it working with newer versions too.

@bartekus About node.js version. I'm using v11.13.0 but I'll try to get it working with newer versions too.

@bartekus OK I launched the test node. It's actually running as part of a small (2 node network).
I might restart the test node from time to time because it's using an in-memory (mock) adapter which is not optimized... Try not to send too much data to it.

@bartekus OK I launched the test node. It's actually running as part of a small (2 node network).
I might restart the test node from time to time because it's using an in-memory (mock) adapter which is not optimized... Try not to send too much data to it.

I made a few transactions already so there is some data in there.

I made a few transactions already so there is some data in there.

I made some changes to the client constructor to make it simpler to instantiate.
https://github.com/Leasehold/ldpos-client-js/blob/c63ada04dd640078d30fef6cd528065996cac03d/scripts/sample-client.js#L8-L16

I made some changes to the client constructor to make it simpler to instantiate.
https://github.com/Leasehold/ldpos-client-js/blob/c63ada04dd640078d30fef6cd528065996cac03d/scripts/sample-client.js#L8-L16


Integration tests fail for keptn version < 0.18.x (expected) and create a Github issues

Summary
Current code in master branch of the datadog-service is incompatible with Keptn versions < 0.18.x due to a breaking change in Keptn. We run the integration tests for 5 Keptn versions
https://github.com/keptn-sandbox/datadog-service/blob/72017a4d76ca98fd9fe780d31053bf80c5627026/.github/workflows/integration-tests.yaml#L22
When the tests run, the workflow fails for 0.16.0 and 0.17.0 version of Keptn which is expected but it creates a Github issue of type:critical (example 1, example 2) which can confuse people.
We should remove the 0.16.0 and 0.17.0 version from the workflow above.
Affected Component

[ ] Web UI / Bridge
[ ] CLI
[ ] REST API or webhooks
[ ] User experience
[ ] Developer experience
[ ] Deployment/management with Helm

To Reproduce
The integration tests workflow runs every day. You can check a new issue of type:critical is created everyday in https://github.com/keptn-sandbox/datadog-service/issues
Expected behavior
The workflow should not create a Github issue
Current behavior
The workflow creates a Github issue everyday.
Screenshots

Additional context

TODO

[ ] Remove 0.16.0 and 0.17.0 from the integration tests above
[ ] Make sure the Integration tests pass by running it for your PR branch from here

[ ] Paste a screenshot or a link in the PR
[ ] Get the PR reviewed and merge
DONE

Failure on 23rd Sep 2022: https://github.com/keptn-sandbox/datadog-service/issues/74

can i work on this issue.

@TarangVerma yes and thank you for showing interest. I will assign the issue to you.

If you need help, you can reach out to me on keptn slack (username: @vadasambar

sure, thankyou...


Multi image inference

I use two special tokens  and input 2 images, but got error in cambrian_llama.py Line 228
Because final_vision_feature_size length is 2, but hidden state shape 0 is 1.
I wonder how to organize multiple images for inference?

same problem, did you solve this?

seems only support single image input
https://github.com/cambrian-mllm/cambrian/issues/4


Causes 100% helper CPU use when it fails

https://github.com/Microsoft/vscode/issues/28861#issuecomment-416976941

Tried for a couple of days with problematic (read: large) latex files. Seems v3 has fixed it!

Awesome! Thanks forr the update, @dsteg


Preserving datatype of overridden app property value

What kind of change does this PR introduce? (check one with "x")
[x] Bugfix
[] Feature
[] Code style update (formatting, local variables)
[] Refactoring (no functional changes, no api changes)
[] Other... Please describe:

Fixes: #
https://github.com/project-flogo/core/issues/121
What is the current behavior?
What is the new behavior?

I don't think this is a proper way, still having an issue to get the correct type base on a value.   From resolver of ENV var, it always returns string.
To properly fix this issue, we should get the type from the property definition and cast the value base on definition type.
@fm-tibco thoughts?


fix nil pointer errors in "gql raw"

This PR fixes two nil pointer errors that cause panics when running gql raw.

Thanks for PR, merged


Ledger funding protocol is broken

Describe the bug
Channels funded by a fake-funded ledger channel fail to open.
To Reproduce
In graph-payments repo:
> cd path/to/graph-payments
> git checkout upgrade-statechannels-pkgs
> yarn
> cd path/to/e2e-testing
> (export USE_LEDGER=true; scripts/run-local.sh)

You'll find the test times out. Why?
When I run the test, the payer creates one payment channel with id 0xd3b099076f317ae1232fd584b9436dc47c581a2011e0a10d5abe38040e59b3d3.
Each peer has signed the preFS of a payment channel
❯ psql payer -x -c "COPY (select vars->0 from channels where channel_id = '0xd3b099076f317ae1232fd584b9436dc47c581a2011e0a10d5abe38040e59b3d3') TO STDOUT" | jq '{ turnNum: .turnNum, signatures: .signatures}'
{
  "turnNum": 0,
  "signatures": [
    {
      "signer": "0xc0ec8e9B5B3cbC2266Cb92F937e2b52c12AEDC90",
      "signature": "0x9d0bc2dfdf5c82517b134fa551a95391fd63565cd2d8c16356a94996ed751b3c00b34eb21e1435398c03cea6a5eb847d5ca61cb8842c333a40408a6aaa66500a1c"
    },
    {
      "signer": "0x333356F625259653A2B6d7Da44162631DCbdF93F",
      "signature": "0xab82e331037e6d691583b6e6ccc638146f35b5beeb341640f0bf10cc85895f9c0ddc7412d76627380bd603a062ffbe4fa3f3b4ca47fbc26ae1f61c504dddeb2e1b"
    }
  ]
}

### The ledger request has succeeded

❯ psql payer -x -c "select * from ledger_requests where channel_to_be_funded = '0xd3b099076f317ae1232fd584b9436dc47c581a2011e0a10d5abe38040e59b3d3'"
-[ RECORD 1 ]--------+-------------------------------------------------------------------
ledger_channel_id    | 0xe5e69b6b700c8dd2e14f0244808c4016b69391f17127dc5b29f72ac7fd2cd739
channel_to_be_funded | 0xd3b099076f317ae1232fd584b9436dc47c581a2011e0a10d5abe38040e59b3d3
status               | succeeded
type                 | fund

Therefore, `ensureAllocation` fails to create the payment channels. In fact, it times out due to retrying with an approximately exponential backoff:

for (const retryTimeoutMs of [2_500, 5_000, 10_000, 20_000, 40_000]) { // snip }

**Expected behavior**
The tests should pass!

In particular, peers should exchange a postFS in channel `0xd3b099076f317ae1232fd584b9436dc47c581a2011e0a10d5abe38040e59b3d3`.

**Logs**
https://circle-production-customer-artifacts.s3.amazonaws.com/picard/5fdbae5563337537ddb7d025/6025bb72f329ef7751d33af9-0-build/artifacts/tmp/e2e-test-with-ledger-without-chain.log?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20210212T165246Z&X-Amz-SignedHeaders=host&X-Amz-Expires=60&X-Amz-Credential=AKIAJR3Q6CR467H7Z55A%2F20210212%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=c3a2e14c59f1c65f39259a527c97201533d4c38072c24c773b372729e63a8baf

Fixed by #3297


[vcpkg-cmake-get-vars] Collect PIC flags

[vcpkg-cmake-get-vars] add PIC flags if CMAKE_POSITION_INDEPENDENT_CODE is set
./vcpkg x-add-version --all

The issue I was running into is I was building static boost libs with a chainloaded toolchain that set CMAKE_POSITIION_INDEPENDENT_CODE to true, and I realized it wasn't being respected because it wasn't collected properly in vcpkg-cmake-get-vars

[x] Changes comply with the maintainer guide
[x] SHA512s are updated for each updated download
[x] The "supports" clause reflects platforms that may be fixed by this new version
[x] Any fixed CI baseline entries are removed from that file.
[x] Any patches that are no longer applied are deleted from the port's directory.
[x] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
[x] Only one version is added to each modified port's versions file.

Gotcha--I felt like I was just following the example of the CMAKE_SYSROOT handling

I think this makes sense because I'd expect a triplet that set VCPKG_CMAKE_CONFIGURE_OPTIONS to -DCMAKE_POSITION_INDEPENDENT_CODE=ON to work on all ports, including ones like boost that use this vcpkg-cmake-get vars.
It feels like having to specify the flags explicitly is poor CMake style?
I think it's probably okay to continue not propagating CMAKE__STANDARD considering that feels like more of a project specific thing than PIC.
The other way I could see addressing this is the boost port extracting CMAKE_POSITION_INDEPENDENT_CODE=ON using vcpkg-cmake-get-vars and it using that to decide if it should add -fPIC. Would that be preferable?

Pinging @russelltg for response. Is work still being done for this PR?

Well it seems like it's a bit contraversial, I'm happy to work on it if there's interest in this approach. Otherwise I can open a PR in the boost build stuff to add PIC flags when requested

I will be converting your PR to draft status, waiting for other users suggestion.

Closing this PR since it seems that no progress is being made. Please ping us to reopen if this PR has new progress.


fix #1484 to allow python block comment docstrings

Fixes #1484 by delegating block comments to ScriptProvider classes instead of handling them in ScriptInfo.
This is the least intrusive way that I could see to solve this problem and support block comments with the same start/end sequence on the same line. Perhaps an overall cleaner approach would be to move the entire parsing of the script and extraction of comment text to the provider, but that would be a largeer change and seems more likely to break downstream stuff.
I recognize this isn't perfect; the Python pattern as is doesn't allow for the triple double quote """ style. It could be tweaked to support that but then it would be possible for it to break with mismatched quotes. There are in turn ways around that but they all add even more complexity, so I opted for this simple case just to get something working.
Happy to adjust as requested! I am working on a Ruby language extension and would like to be able to handle block comments there as well as in Python.

we've been bitten by assuming that all scripts are Java before, so try to keep Java specific stuff out of the base scripting class.

This might hurt for those relying on said behavior now but could help open the door for extensions to support scripting in other languages. It may just be my opinion here but I'd consider anything that assumes the script is java outside of the JavaScriptProvider a bug. I would consider it so because the intent of scripting isn't to only support Java.

@jpleasu @goatshriek what are your thoughts on maybe having each provider parse the header instead and then pass the metadata to the GhidraScriptProvider through protected setters? I think this approach would be the most robust going forward.

@astrelsky I'm not opposed to making providers responsible for parsing ghidra.app.script.ScriptInfo data from their own scripts, but it's a fix for the design of ScriptInfo, perhaps more ambitious than it seems.
If the provider does the parse, then ScriptInfo#parseHeader should be replaced.  The various methods of ScriptInfo  (obsessively) check that they're getting the latest parse from a script by checking last modified time and reparsing..  this is totally bad, IMO, but changing that "synchronization model" means otherwise keeping script manager, action manager, and all of the providers in sync - presumably via the script info manager.  That kind of change takes care and testing.  maybe just testing ;-)

I see. I forgot about the ScriptInfo class since I didn't have the source in front of me. If that is the case then maybe ScriptInfo could become abstract and just have the provider provide the script info for parsing. You could probably get away with caching the info to avoid reparsing. I think you would still need to check the timestamp though.

The issue with synchronization isn't that you check the timestamp, it's when you do.  The current setup checks with every access to every piece of data.  It will only get reparsed once per timestamp change (assuming no races or duplicate ScriptInfo classes for a given script), so it's effectively caching as is.
What I don't like is the lack of any clear "script changed" semantics.  Recompilation of Java scripts, for example, performs a check only when necessary (when it or bundles connected to it via dependency are enabled/disabled).  The script manager, however, shows the latest parsed from headers via the ScriptInfo#parseHeader mechanism.. literally as the Swing table is rendered, headers are checked and parsed.  GhidraScriptActionManager uses ScriptInfo too, but the ScriptAction data that's necessary ought to match the data presented to the user in the table, at the time the "in tool" status changes not when last written to disk.

I did originally start down the path of having the providers do the parsing, calling a new script provider function from parseHeader that passed back a Map of metadata symbols and their corresponding values. It felt a little hacky and I didn't want to make such a big change just starting out, so I opted for the original approach instead. I'm happy to take a shot at a new ScriptInfo strategy eventually, but definitely don't have a strong enough handle on things to do that any time soon.
I've finished the changes from suggestion 1, and I'm almost done writing a suite of tests that demonstrate block header behavior (for suggestion 2), and will push a new commit with those in the next few days.

I've made the suggested changes above and pushed them as a new commit for further review.
The new tests cover scripts with and without a block comment in both Java and Python, for multi and single-line block comments. I've tried to base them off of other tests that already exist, but please let me know if the approach for any of them needs to change.

I am in the process of making a change in patch to address the presence of certification headers within python script files.  I apologize for throwing a monkey wrench at your PR.  I suspect my change does not fully address you concerns and additional aspects of your change may still be needed.  We prefer not to change the formatting of the certification headers if it can be avoided.

@goatshriek  I spoke with @ghidra1 about this to clarify -- although the certification headers might suite the purposes of general copyright statements, they are specific to Ghidra's release process and shouldn't be used outside that process.
Your PR is still relevant and the tests you've written all the more.

No worries, thanks for the clarification! I've merged in master to resolve the conflicts, and made the suggested changes. I added a test that covers more metadata fields along with a multi-line description.
From what I could tell the certification header shouldn't interfere with the block comment handling as long as it is checked for first? Sorry for the multiple commits, I didn't notice the ordering dependency until after my first merge was done and pushed.
Thanks for all of the help and review! Hopefully closing in on something useful.


transplant to my own css

I just want to transplant the datepicker component, can I import the scss and js of datepicker and then compile my own sass, it seems not smoonth. Or are there any interfaces UserGuide or Docs to extend your materilize framwork?

If you want to just take the datepicker component, you just have to define the variables that exist in the datepicker scss file. The datepicker exists as a very separate component as it is pickadate.js


Replicate file picker intent

https://github.com/miguelpruivo/flutter_file_picker/blob/9f0a811028a0944e9e23dad6002108a7ff17a6ec/android/src/main/java/com/mr/flutter/plugin/filepicker/FilePickerDelegate.java#L189

Issue created by BlueBubbles Bot in the BlueBubbles commnity on behalf of @zach in #requests.

We switched to using file_picker itself


replaced wrong line in README

MainActivity is not the file we have to edit for the plugin, it is not gonna work since react-native updated > 0.29. The file we need is MainApplication.java, there are all packages that users add to application.

Tripped over this while reading through the docs myself. This would be an easy and helpful merge.


Add support for emails with hyphens and underscores in ES

Hi, this pull request modifies Spanish email regex to detect mails with hyphens and underscores as well as in English.
If anything is wrong please tell me and I will fix it as soon as possible to get this merged. I tried to follow all the steps from
Thanks!

Thanks @froasio!


Add requiresMainQueueSetup to fix warning in RN 0.49+

@lelandrichardson Could we get a review for this soonish? Thanks!


revert: Temporarily revert some breaking changes (for 2.0.0-internal.6.x) (port to 6.1)

Description
Port https://github.com/microsoft/FluidFramework/pull/16846 to the 6.1 release branch. This is to ease the transition from 5.x to 6.x.
Reviewer Guidance
The review process is outlined on this wiki page.

@tylerbutler I wasn't sure what to do with the changeset files for a port to patch a release branch... Should they be there? Should they say "minor"?


Add button to remove image from canvas

We want to be able to remove an image from the canvas by pressing a button.

Fixed in commit 595ecfe


Add optional Unbound DNS resolver

Proposed Changes
Adding 2 new options to the addon:

use_unbound_resolver: defaults to false, enables and runs an Unbound DNS resolver in the container
unbound_port: defaults to 5353,  customize the port that the resolver listens on internally

I have been seeing more and more interest lately in people wanting to run their own recursive DNS resolvers in order to cut out 3rd parties (and to use DNSSEC), so this is a nice turn-key solution as far as I can tell. I can considered making Unbound into it's own addon, and that might still be a good solution, but it seems that it made sense to locate it with Pi-hole since it would only be used by Pi-hole (almost like a dependency).
I followed the Pi-hole docs here for creating this configuration.
Note: the user will still need to specify their custom resolver in their Pi-hole admin config (eg, <IP_ADDRESS>#5353), I wasn't immediately aware of how to pre-load the chosen upstream DNS servers, or if that was even a good idea.
Related Issues
https://github.com/hassio-addons/addon-pi-hole/issues/66

Within the whole Hass.io concept, this should be separated in a different add-on.
I'm not keen on merging this one in.

Within the whole Hass.io concept, this should be separated in a different add-on.
I'm not keen on merging this one in.
This just runs a service somewhere, which is actually totally standalone.

Copy that. I'll see if I can find some time to pull it out.

Check, closing the PR for now.


Bug Report - Padding and horizontal spacing not preserved when nested within a symbol

Bug report
Expected behavior:
I am constructing a radio button group consisting of 2 options. Within each option I have a label and a background which has padding applied and a minimum width of 80px [10 16; width >= 80].
The options are grouped and the folder has a vertical space of -1 is applied [-1h] in order to remove the double border.
When I increase the length of the labels I expect the width of the button to increase to maintain the set padding. This would also push the adjacent button in order to preserve the desired layout.
Actual behavior:
This works as expected when you have a single instance but does not work when converted to a symbol.
The text expands outside of the set background and the adjacent button doesn't move as expected.

This was created in Sketch 49 and using Paddy v1.0.7

Thanks @gforce31 – I would consider this 'Nested symbol support'. Something I'm looking into — see #3
Thanks for your patience.


Preflight check for parallel and other dependencies

It would be nice if qbatch checked that parallel was in the path before submitting a job so that you don't waste time submitting a job that is going to fail almost surely.
Same goes for qsub and qstat although missing these would lead to an immediate failure so not nearly as important.

Gotta be careful to check for "local" mode, since it doesn't need qsub/qstat


work on #269

contains fix for #269 and some language issues in the file
NOTE: Was not able to test it because I get this when I put this into my VM with the stable release:
Fatal error: Cannot redeclare get_port_with_service() (previously declared in /usr/local/www/diag_logs_filter.php:380) in /usr/local/etc/inc/filter_log.inc on line 278

Thanks Fabian, I will pull it in and check the redeclare issue on my box. I'm doing some major restructures so I probably caused this error. sorry about that.


tmux 3.2 has been released

Describe the bug
Using tmuxinator 2.0.2 with tmux 3.2 results in a version warning:
    WARNING: You are running tmuxinator with an unsupported version of tmux.
    Please consider using a supported version:
    (3.1c, 3.1b, 3.1a, 3.1, 3.0a, 3.0, 2.9a, 2.9, 2.8, 2.7, 2.6, 2.5, 2.4, 2.3, 2.2, 2.1, 2.0, 1.9, 1.8, 1.7, 1.6, 1.5)

To Reproduce
Install the latest version of tmux from homebrew on Mac (currently 3.2), and invoke any tmuxinator project.
Expected behavior
I would expect there to be no warning
Environment

tmuxinator Version: tmuxinator 2.0.2
tmux Version: tmux 3.2
Ruby Version: ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-darwin19]
OS: MacOS 11.2.3

This is a duplicate of #764.
A new release which explicitly supports tmux 3.2 will be published shortly. In the meantime and going forward, you can use the --suppress-tmux-version-warnings flag when starting tmuxinator to bypass these warnings.

Ah sorry, I was searching for "3.2" in issues and didn't find that one.

In the meantime and going forward, you can use the --suppress-tmux-version-warnings flag when starting tmuxinator to bypass these warnings.

tmuxinator --suppress-tmux-version-warnings start basic
ERROR: "tmuxinator help" was called with arguments ["--suppress-tmux-version-warnings", "start", "basic"]
Usage: "tmuxinator help [COMMAND]"

tmux 3.2; tmuxinator 2.0.2

@Shamaoke tmuxinator start PROJECTNAME --suppress-tmux-version-warning=SUPPRESS-TMUX-VERSION-WARNING

@jaspertandy That way it works, thanks.


Send spam to aos-cicd instead of aos-devel

Signed-off-by: Steve Kuznetsov<EMAIL_ADDRESS>/cc @jupierce @adammhaile
done with
$ sed -i 's/aos-devel/aos-cicd/g' $( find ./jobs/ -name Jenkinsfile )

 (1) [CI][BuildAuto] Redirect all spam nonsense e-mails to<EMAIL_ADDRESS>

Code wise it looks fine. Can't comment on why this is being changed however.


nixplay support

I am trying to link my Nixplay playlist to my MagicMirror.  I already have MMM-Wallpaper pulling from a local source and it works fantastic, but my family all got NixPlay frames for Christmas and are sharing playlists and things to each other's frames.  Instead of having a Nixplay picture frame sitting on a wall doing the exact same thing that my MagicMirror is, I thought that I might be able to have your wallpaper module access my pictures from my Nixplay online account.
When I add the line 'source:  "nixplay:app.nixplay.com/#/playlists/playlist/removed for privacy",' I get a rotating wallpaper but it has none of my playlist pictures (they must be stock pictures from NixPlay) and it will not pull any pictures from my local storage anymore.
If I change the line to 'source:  "http://app.nixplay.com/#/playlists/playlist/removed for privacy",' the magic mirror will load, but I get only a black background with no pictures being projected as a wallpaper from either my playlist or my local storage.
I guess I have three questions with this.
One, any ideas on what I need to change on my source line to be able to pull my actual pictures off of NixPlay?
Two, do I need to have my login information setup in the config to pull my pictures instead of the stock pictures, and if so, how would I add that to my config?
Three, how do I have your module pull from both sources either randomly or in an alternating fashion?
Thank you so much for your help and time with this.  I have been loving learning programming my MagicMirror and figuring out how things are supposed to be formatted to get it to work correctly.

nixplay isn't a supported source, they don't appear to have a public api, and I don't have an account to try to add support for it.  When you set your source to nixplay:whatever, the module is falling back to displaying bing's daily wallpapers.  When you give it the http url, it's trying to load that exact url as an image (this is usually for webcams/security cams).  Without an api, there's really no way I'll be able to add support for these, sorry.

Thank you for the quick response and explanation for what was going on.  I
thought that 'might' be the issue, but I wanted to double check.
On Wed, Jan 26, 2022, 8:12 AM Jon Kolb @.***> wrote:

nixplay isn't a supported source, they don't appear to have a public api,
and I don't have an account to try to add support for it. When you set your
source to nixplay:whatever, the module is falling back to displaying
bing's daily wallpapers. When you give it the http url, it's trying to
load that exact url as an image (this is usually for webcams/security
cams). Without an api, there's really no way I'll be able to add support
for these, sorry.
—
Reply to this email directly, view it on GitHub
https://github.com/kolbyjack/MMM-Wallpaper/issues/63#issuecomment-1022234422,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AXPAQGDZUNHPE24XBDIYSBTUX76M3ANCNFSM5MZ7ZMIA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>


Cleanup plist output formulation

Today it formulates too long of files names for me on macOS. The total path size today is:
(BuildFilePathLength + 2 * RuleNameLength + SuffixLength)
Given Bazel supports a single rule with a given name per build file, we can safely eliminate this extra layer of namespacing. After the fix, the total path size is:
(BuildFilePathLength + RuleNameLength + SuffixLength)
Additionally:

further reduce generated label / output sizes
underscore generated rule names with _
remove issue with ".." name suffix. This also gets us better looking plists names: For instance, a Foo dogfood plist gets you _Foo.dogfood.0.info.plist

You got it 🚀


Using a github app to push helm charts

Description

Fixes # (issue)
Does this PR introduce a breaking change?
Checklist

[ ] PR title follows the <type>: <description> convention
[ ] I use conventional commits in my commit messages
[ ] I have updated the documentation accordingly
[ ] I Keep It Small and Simple: The smaller the PR is, the easier it is to review and have it merged
[ ] I have performed a self-review of my code
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

Additional information for reviewer
Mention if this PR is part of any design or a continuation of previous PRs

This change is based on https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/making-authenticated-api-requests-with-a-github-app-in-a-github-actions-workflow


Add prismic article parser

References #981
Type

✨ Feature
Value
We can start using another CMS using our new model, meaning we can start taking over other bits of content on the site, namely events, exhibitions etc.
If you look at how we create an article here, this would replace all the ArticleFactory.fromWpApi stuff and bodyParser crap.
## TODO:

[x] ~Use image resizing tool~ (we can do this after the editors have had a go)
[x] ~Preview~ (we can publish to live as these pages aren't listed anywhere, so no problem for testing)
[x] ~Instagram embeds aren't using the JS for some reason~ (we don't use them that much so can figure this out as we go on)
[x] ~The multitude of tech TODOs in the file~ (a refactoring for when we kill wordpress).
[x] Author

Screenshot

Checklist
All

[x] Demoed to the relevant people
[x] PR labelled and assigned
[x] Any introduced code complexity has been flagged

@jennpb FYI ⬆


Use ceil_div_usize for PACKED_LEN

Oh sorry, Daniel! I didn’t see this for some reason. But yeah, it looks good :)


Send push notifications

As a user, I expect the application to send notifications via the Notifications API (https://developer.mozilla.org/en-US/docs/Web/API/Notifications_API/Using_the_Notifications_API) once I enable them. I expect to receive notifications for the same things that I receive emails for.
This could be implemented by https://github.com/safwanrahman/django-webpush
This could also be combined with #237 with https://pypi.org/project/django-pwa-webpush/

We should reconsider our messaging approach within this issue. Currently, we are sending synchronous mails (in the worst case to all users). So first we need an email backend that sends asynchronous mails, e.g. https://pypi.org/project/django-mailer/ or https://github.com/pmclanahan/django-celery-email
On the other hand, it would be better to use a flexible messaging framework for all events. Our core code as well as plugins should be able to register a set of events/notifications (e.g. participation state changes, user data changes, consequence is decided). The user should be able to decide on which channels (mail, push notification, more channels via pluign) he wants to receive which notification.
I started to research frameworks for this purpose and came across the following:

https://github.com/django-notifications/django-notifications has the right concept but is meant for in-site notifications (no support for backends at all)
https://github.com/pinax/pinax-notifications looks exactly like what we want, but I'm not sure whether one can use it outside of pinax. also only the email backend is implemented, we would need to do the others on our own
https://github.com/worthwhile/django-herald uses seperate classes for the different services (EmailMessage, TwilioMessage) which is not what we want
https://github.com/idlesign/django-sitemessage looks kinda good, different message types and different backends

Of course we could also implement our own solution

We should reconsider our messaging approach within this issue. Currently, we are sending synchronous mails (in the worst case to all users). So first we need an email backend that sends asynchronous mails, e.g. https://pypi.org/project/django-mailer/ or https://github.com/pmclanahan/django-celery-email
On the other hand, it would be better to use a flexible messaging framework for all events. Our core code as well as plugins should be able to register a set of events/notifications (e.g. participation state changes, user data changes, consequence is decided). The user should be able to decide on which channels (mail, push notification, more channels via pluign) he wants to receive which notification.
I started to research frameworks for this purpose and came across the following:

https://github.com/django-notifications/django-notifications has the right concept but is meant for in-site notifications (no support for backends at all)
https://github.com/pinax/pinax-notifications looks exactly like what we want, but I'm not sure whether one can use it outside of pinax. also only the email backend is implemented, we would need to do the others on our own
https://github.com/worthwhile/django-herald uses seperate classes for the different services (EmailMessage, TwilioMessage) which is not what we want
https://github.com/idlesign/django-sitemessage looks kinda good, different message types and different backends

Of course we could also implement our own solution

I have some points to make on how to handle email :)

I have some points to make on how to handle email :)

So I had a closer look at the frameworks mentioned earlier. Out of those, only pinax-notifications is suitable for our needs. Downside: they rely on the django sites framework which we don't use. Also the construction was not perfect as they focus more on fixed notifications (where the text of a notification is predefined and additionial content is only a small feature). For us, notifications usually depend heavily on the actual state of something. So I decided to build it on my own.

Are you building it as a seperate app? Also, how do we want to deal with notifications for non-local participants?

no, I am not.
I decided to do that out of the notification framework as this is really inconvenient. we only have an email adress for those participants, but some backends will not use an email address to reach the user


Fire demo

Here is a new demo of fire effect, it's not perfect but I think it looks pretty cool. Enjoy it!

Awesome ! Thanks man :), I think we are interfacing with pico-8 for now !, But some huge changes in the system will go in 0.0.6 (There will be compatibility for older versions)


Slow move operation

Describe the bug
A very slow move operation.
When I want to move file(s)/folder(s) to new location that is on same (physical partition/disk), move operation is very slow. Looking on Task manager, I can see that there is heavy transfer operation. Problem is there is no any visual indicators in Files.
I guess app is doing full copy of file and after that it deletes it from original location (as I can see file(s) on original location 'till this is finished).
To Reproduce
Steps to reproduce the behavior:

Move file from one folder to other on same disk/partition
File appears on new location with size 0kb
File is still at old location
After some time (depending of size of file and disk speed) file on old location disappears.

Expected behavior
Moving files/folders on same partition should be nearly instantaneous.
Screenshots
https://user-images.githubusercontent.com/72248348/104391972-5dcec800-5541-11eb-9907-58984138df31.mp4
Desktop (please complete the following information):

OS Version: Windows 10 20H2 19042.685
App version: [v<IP_ADDRESS>]

Additional context
Log file
debug.txt
debug_fulltrust.txt

I guess app is doing full copy of file and after that it deletes it from original location

Your analys is correct ;)

I guess app is doing full copy of file and after that it deletes it from original location

Your analys is correct ;)

@McPwner This issue should be addressed with #2955.

@McPwner This issue should be addressed with #2955.


[Feature]Support for more Sunevents

It would be nice if it was possible to select more of the different sunevents in the dropdown menu.
Or have the option to enter them manually in a textbox

Let me check what is possible


Orchestrate the replica-installer

Do we have to model dependencies and ordering of installation in the replica-installer?
E.g., what if there are Jobs needed to run before installing the app? do we need to wait for the completion of this worker before installing the other resources? Aren't we re-implementing Helm chart lifecycle?

In my opinion, It's redundant to re-implement lifecycle hooks like in Helm, not only because we can re-use it with helm packages in Golang, but mainly because we need to be more generic, as still there are a lot of installations and deployments that weren't deployed using helm.
Three approaches to deal with that:

Don't try to impose an order of pods and jobs, if job needs to run before a pod, they will applied to cluster at the same time, and  then using an appropriate startupProbe And/Or LivenessProbe  that check for the Availability of the resource created/prepared by the job, eventually the pod will "catch up" with the job' result.
This way we can intervene by incrementing up initialDelay of the Startup/Liveness Probe , according to some logic, before applying manifests to the cluster.

Impose an order, we need to transform one-time jobs/pods' logic (that supposed to be run before application pods)  into init-containers of one of the application' pod, the work here(beside the transformation) is to decide how to recognize such order between resources in the source cluster:

We can looks for helm hooks annotations.
In addition, we can support a dedicated portion in the input yaml file, that will define some resources that should be executed before application pods, the resource will be specified by according to labels/annotations that the one time jobs/pods in the source cluster already has, and distinguish them from the application pods.

2 phases process -   Maybe a better option from section 2 is to support a dedicated portion in the input yaml file, that will define some one-time resources that should be executed before application pods, like in previous section according to labels, but don't map it to init-containers, instead, get them and all their resources(configmap,secrets,RBAC, etc) etc, using labels, and apply them to the cluster before applying application resources(without the one-time resources), we can support a configurable parameter that will impose waiting between these 2 phases(in units of seconds)

Let's keep this point for a dedicated discussion. For now we can go ahead with a single, uncontrolled deployment of all the extracted resources, but in the next releases we have to think of a manageable way to control the deployment. WDYT?

Yeah make sense, we can stick to option 1 for the moment, this is the default anyway in the absence of a mechanism that will take care of it, so we'll defer the development of such mechanism to later stages.


Android authenticated bug sometimes

It sometimes shows an android authenticated bug on the start screen, or when i already filled the passcode. And sometimes crashing when sync in progress on Android 6.0.

now it works with pattern.


QAT: update readme for QAT

A few suggestions. I also noticed https://github.com/intel/intel-technology-enabling-for-openshift/blob/main/device_plugins/qat_device_plugin.yaml is not quite OK. maxNumDevices should be 128 and perhaps c6xxxvf can be dropped.

Hi Mikko, thanks for your review. I presume Chaitanya used the sample here for qat device plugin reference.
Are there any implications of setting maxNumDevices to 128 i.e. some device ids do not support that level etc?

Are there any implications of setting maxNumDevices to 128 i.e. some device ids do not support that level etc?

maxNumDevices limits how many QAT VF resources are made available from each node to the cluster. Depending on the platform and HW setup, different amount of VFs are available. We don't have a setting that says "use all available" so some big enough value is used to reflect that.

Are there any implications of setting maxNumDevices to 128 i.e. some device ids do not support that level etc?

maxNumDevices limits how many QAT VF resources are made available from each node to the cluster. Depending on the platform and HW setup, different amount of VFs are available. We don't have a setting that says "use all available" so some big enough value is used to reflect that.

We will address this on a separate PR.

Are there any implications of setting maxNumDevices to 128 i.e. some device ids do not support that level etc?

maxNumDevices limits how many QAT VF resources are made available from each node to the cluster. Depending on the platform and HW setup, different amount of VFs are available. We don't have a setting that says "use all available" so some big enough value is used to reflect that.

@mythi  I think we should have some kind of default value for this to use all the available number of VFs. The end user should not have to know and handle this HW detail.
We also need to do the boundary test to make sure the big enough value is well handled by the device plugin.

is it possible for us to add the ids as the parameter for the container and set it through the operator? So we don't have to hard code the ids in in the init-container?

The PCI IDS of the devices enabled by the cluster admin are passed to the initcontainer so you can have custom logic based on that.

@mythi I think we should have some kind of default value for this to use all the available number of VFs. The end user should not have to know and handle this HW detail.

This flag is for purpose: QAT can also be used for "in kernel" functions through the Linux Kernel Crypto Framework. with maxNumDevices, cluster admins have the capability to leave some VFs for the "in kernel" functions (and other functions outside the cluster). Therefore, it cannot be "use everything" unconditionally.

We also need to do the boundary test to make sure the big enough value is well handled by the device plugin.

maxNumDevices is ignored after the plugin has processed all the QAT VFs it finds.

@mythi I think we should have some kind of default value for this to use all the available number of VFs. The end user should not have to know and handle this HW detail.

This flag is for purpose: QAT can also be used for "in kernel" functions through the Linux Kernel Crypto Framework. with maxNumDevices, cluster admins have the capability to leave some VFs for the "in kernel" functions (and other functions outside the cluster). Therefore, it cannot be "use everything" unconditionally.

We also need to do the boundary test to make sure the big enough value is well handled by the device plugin.

maxNumDevices is ignored after the plugin has processed all the QAT VFs it finds.
I am OK with manNumDevices parameter, my point is we should have a default value for users if they do not set it. The default value of maxNumber can be the max VF numbers the device can support. Besides, if we export this parameter and let user set this up, we should also let users know how many VF devices this device can support, something like https://doc.dpdk.org/guides/cryptodevs/qat.html Table 21.2 QAT device generations, devices and drivers.

@mythi I think we should have some kind of default value for this to use all the available number of VFs. The end user should not have to know and handle this HW detail.

This flag is for purpose: QAT can also be used for "in kernel" functions through the Linux Kernel Crypto Framework. with maxNumDevices, cluster admins have the capability to leave some VFs BTW, this implies that we will also continue to support he "Kernel Mode"? :-)

BTW, this implies that we will also continue to support he "Kernel Mode"? :-)

no, that is different case.

The default value of maxNumber can be the max VF numbers the device can support.

it's not always that simple. There's VFs/PF but there's also how many PFs are available. I understand what you are saying but we'll go with this for now.

The default value of maxNumber can be the max VF numbers the device can support.

it's not always that simple. There's VFs/PF but there's also how many PFs are available. I understand what you are saying but we'll go with this for now.

I think for this release it is good enough. Just hope we can optimize it in the future. We can file another issue and track it. :-)

@mregmi, the machine config readme documentation needs to be updated since we are now not using QAT custom machine config pool. Also, please add a section about the new machine config to load vfio-pci. Thanks!

@chaitanya1731, can you review the device plugins readme. Thanks!

some other PRS will be submitted to replace this PR. So close this one.


explorer.exe error/crash in safe mode even after disabling "Set Files as default file explorer (EXPERIMENTAL)"

Description
I know the replacement feature is experimental. I just want to report the issue. If you happen to know how to fix it I would greatly appreciate that information. Also, I am on WIndows 11.
After installing Files and enabling the explorer replacement feature (EXPERIMENTAL), when booting into safe mode with in enabled it completely breaks the task bar, start menu, explorer, etc. Anything tied to the explorer.exe is frozen, nonfunctional, or missing entirely. If Files explorer replacement feature is disabled, the regular explorer.exe will work normally for about 10 seconds before giving an error.
"The system detected an overrun of a stack-based buffer in this application. This overrun could potentially allow a malicious user to gain control of this application."
Then if you click okay the explorer.exe restarts and repeats the cycle.
Steps To Reproduce

Install Files
Open Files settings, go to the experimental section, and enable "Set Files as default file explorer"
Open system configuration or run mscofig, go to the boot section, and enable safe mode
Restart. (Completely broken explorer)
To return to normal boot, hit win+r then run "msconfig" and disable safe boot.
To reproduce error, boot normally, disable "Set Files as default file explorer", and repeat steps 1-4

Expected behavior
If the feature is left enabled, explorer.exe (task bar, start, file explorer) will be nonfunctional. If the setting is disabled the explorer.exe will be mostly functional until getting the error.
Files Version
<IP_ADDRESS>
Windows Version
10.0.22000.318 (Windows 11)
Relevant Assets
No response

Same thing here, I cannot disable the Experimental feature, and it breaks the windows file explorer in multiple areas.

@HammerOfDaGods did you revert the changes before trying to open explorer in safe mode?

@yaichenbaum Yes. As I said above, the program causes issues with the feature enabled or disabled.
DISABLED - "If Files explorer replacement feature is disabled, the regular explorer.exe will work normally for about 10 seconds before giving an error." and then crashes and restarts as stated in the report.
ENABLED - "when booting into safe mode with in enabled it completely breaks the task bar, start menu, explorer, etc. Anything tied to the explorer.exe is frozen, nonfunctional, or missing entirely."
When the feature is disabled I don't see why it should effect the default explorer. Turning off the feature is supposed to revert any registry changes which should undo any interference. I understand that leaving it enabled should not work considering what safe mode is meant for. However, if this feature is going to be included it shouldn't completely break under these circumstances, it should just revert to the original explorer.exe. In a situation where a user could not boot normally to disable the feature, the user would be pretty much out of luck unless they have another way to navigate (command line, run, etc).

@Arvind11747 Our issue is a little different. I can disable the feature, which has issues of it's own, but when enable yes, same issue. Multiple, possibly all, elements of the default explorer.exe (taskbar, file explorer, etc) break completely.

Whatever registry changes are applied for this to work, can they not work in reverese for the default file explorer?
The file browsing state left behind when removing the app is completely broken

Would it be possible to have a button or tool to completely undo any changes made by Files' features and installation? I like the app, and I will continue using it, but it modifies essential functions of Windows, and at this point I'm afraid to remove it out of fear it will permanently break, or at the very least adversely affect, the default explorer.exe . I would feel a lot better having something to use in case of emergencies, at least in the mean time until you have time to squash this bug.

Would it be possible to have a button or tool to completely undo any changes made by Files' features and installation? I like the app, and I will continue using it, but it modifies essential functions of Windows, and at this point I'm afraid to remove it out of fear it will permanently break, or at the very least adversely affect, the default explorer.exe . I would feel a lot better having something to use in case of emergencies, at least in the mean time until you have time to squash this bug.

@HammerOfDaGods we don't recommend using experimental features on your main device. Turning off the option should revert all the changes, we're not sure why it's not working for you but we're looking into it. We're also working on automatically reverting the changes if someone uninstalls Files.

Whatever registry changes are applied for this to work, can they not work in reverese for the default file explorer? The file browsing state left behind when removing the app is completely broken

@Xavi3RElvis did you revert the changes before removing Files?

Not only does it not revert anything for me but the button cannot get deactivated at all.
Clicking teh switch does nothing adn always stays at the on state while also fixing nothing regarding registry file explorer defaults

Not only does it not revert anything for me but the button cannot get deactivated at all. Clicking the switch does nothing and always stays at the on state while also fixing nothing regarding registry file explorer defaults screen recording of behavior

@Xavi3RElvis did you revert the changes before removing Files? There is bug in the current release (which has been fixed in main) that the script to undo the changes wasn't kept if the app was removed before the changes were undone.

Hmm I might have uninstalled before setting the feature to off. So this is already a known and resolved issue?

Hmm I might have uninstalled before setting the feature to off. So this is already a known and resolved issue?

@Xavi3RElvis yes, it will be included in the next update.

Hmm I might have uninstalled before setting the feature to off. So this is already a known and resolved issue?

@Xavi3RElvis in the meantime you can follow the instructions here to unset as default file explorer:
https://files.community/docs/configuring/replace-file-explorer
TL;DR: download this zip and run UnsetFilesAsDefault.reg to restore windows explorer

The reg setting did work, thank you for that

@HammerOfDaGods is this still an issue after running the script?


Fix #308 - update brackets-cdn-suggestions submodule to get upstream fix for HTTPS

I fixed this ages ago upstream, but the fix just landed.  This pulls in the new commit.
r? @gideonthomas

r+


Docker: console not functioning

When I attempt to connect to console, I get this error message:
OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "exec: \"bash\": executable file not found in $PATH": unknown
Maybe that's deliberate?
Thanks!

Hi, yes, the Docker container is based on Alpine, and a minimal install.
See also: http://containertutorials.com/alpine/get_started.html


Removes customization of URLError localized error messages

Issue Link :link:
Fixes #3194
Goals :soccer:
Remove localized error message overriding by Alamofire when a URLSessionTask error occurs
Implementation Details :construction:
Opts to remove the custom value from localizedString and relay the underlying localized error message instead.

Thanks for the PR!
I'm a bit torn on this. Generally, I don't think it's a good idea to pass underlying error descriptions directly to the user, especially these sorts of low level networking errors. User's can't really do much about them and they're usually more technical than most users understand anyway. Additionally, this can easily be worked around by extracting the underlying error just like you are in the PR itself. So adding a special case for this error case when Alamofire's errors aren't generally supposed to be shown to the user may not make sense.
@cnoon, any thoughts?
@rpassis Is there any particular reason why, when Alamofire's errors aren't otherwise localized, you'd want this one case to be? If you're special casing it's handling anyway, reaching into the underlying error to get the same value would work fine.

The question I ended up asking is how is this beneficial to any consumers of the API?

I think this is the crux of the issue. My only argument in favor of the current message is that makes it more clear the error came from the underlying URLSession and not Alamofire directly.
Ultimately I think it comes down to the use case. Currently, to access the underlyingError's localizedDescription, users can use the .underlyingError property or switch over the AFError instance and capture the instance directly. They'd have to do this if they want to avoid Alamofire's other localizedDescriptions. With this change, the old approaches would still work, and they'd still have to switch over the AFError to avoid the other descriptions, so I'm not sure what it ultimately gets us.
Given that this doesn't actually seem to improve the ergonomics of presenting these errors to the user, as users still have to avoid Alamofire's other non-localized descriptions, I'm going to decline this PR for now. Thanks for the work!


v4: pull with rebase, create branches and tags

Hi guys,
thanks for writing this! I'd like to ask if there is a way to:

Pull it with '-r' option
Create branches and/or tags

Thanks again!

pull --rebase is currently not supported, but PRs are welcome. We might add support for it in the mid-term.
Creating branches and lightweight tags is supported. For go-git they're just references. Repository has a Storer, which has SetReference (see: https://godoc.org/gopkg.in/src-d/go-git.v4/plumbing/storer#ReferenceStorer )

@smola thanks, got it.


Add support for QR code

Either include a separate QR detector using a separate worker, or replace the decoder with something that can detect both 1D and 2D barcodes

Option: https://github.com/cozmo/jsQR

Any updates on this? :)


Backtracking during Restart

In the case of a restart, it could be beneficial to completely recalculate from scratch rather than to notify of the (potentially) numerous events.
This would require:

Determining when a restart loses "too much" information; we could use a heuristic based on how close the current state of the solver is to a solution
Having an interface for propagating from scratch for every propagator
Ensuring that the stored events are cleared properly

Why do we not currently support this? We could say that when we are restarting, we don't notify on backtrack events. Then the synchronize will be called which should be implemented such that the propagator's state is correct wrt the current state.


[Bug]: script tossing DOMException: Failed to execute 'iterateNext' on 'XPathResult': The document has mutated since the result was returned.

Minimal, reproducible example
const puppeteer = require('puppeteer');

function disposeAllElementHandles(elementHandles) {

    if (elementHandles.length == 0) return;

    for (let elementHandle of elementHandles) {
        elementHandle.dispose();
    }
}

const getUrls = async () => {

    const options = {
        args: [
            "--no-sandbox",
            "--disable-setuid-sandbox",
            "--disable-dev-shm-usage",
            "--disable-gpu"
        ]
    };
    const browser = await puppeteer.launch(options);
    page = await browser.newPage();

// indicate we are a desktop page
    const device_width = 1920;
    const device_height = 1080;
    await page.setViewport({width: device_width, height: device_height})
    // this url fails with
    // DOMException: Failed to execute 'iterateNext' on 'XPathResult': The document has mutated since the result was returned.
    url = "https://careers.aah.org/job-search-results/";
    // this url works
    //url = "https://careers.cdkglobal.com/job-search-results/";
    response = await page.goto(url, {timeout: 20000, waitUntil: "networkidle0"});

    pageNum = 0;
    let nextPage = null;
    let crawledUrls = [];
    do {

        // grab all hrefs on page
        let jobLookUpCriteria = "//a";
        let urlElements = await page.$x(jobLookUpCriteria);
        console.log("page url element count: " + urlElements.length);
        for (let urlElement of urlElements) {
            let crawledUrl = await page.evaluate((el) => el.href, urlElement);
            crawledUrls.push(crawledUrl);
            urlElement.dispose();
        }

        // get the next page
        pageNum++;
        let xpath = "//a[contains(@aria-label,'You are on page PAGE of results.')]"
        const nextPageNum = pageNum + 1;
        xpath = xpath.replace("PAGE", nextPageNum);

        nextPageElementHandles = await page.$x(xpath);

        nextPage = null;
        if (nextPageElementHandles.length) {
            nextPage = true;

            let nextPageElementHandle = nextPageElementHandles[0];

            await Promise.all([
                page.waitForNavigation({timeout: 30000, waitUntil: 'networkidle2'}),
                page.evaluate(nextPageLink => nextPageLink.click(), nextPageElementHandle),
            ]);

            disposeAllElementHandles( nextPageElementHandles);

        } else {
            console.log(xpath + " not matched");
        }

    } while (nextPage);

    console.log(" " + crawledUrls);
};

// scrape the page and extract the urls
getUrls()

Error string
DOMException: Failed to execute 'iterateNext' on 'XPathResult': The document has mutated since the result was returned.
Bug behavior

[ ] Flaky
[ ] PDF

Background
We have a script that grabs urls from job sites. When I upgraded to the latest version of puppeteer
"puppeteer": "21.2.1",
I receive the following error from a few jobsites. The majority work without error.
The script has an example of a url that works and one that doesnt work.
Expectation
no error
Reality
it tossed the exception I will parrot here
/usr/local/bin/node /Users/dwillis/Documents/GitHub/jobs-node-crawler-service/issue.js
Puppeteer old Headless deprecation warning:
In the near future headless: true will default to the new Headless mode
for Chrome instead of the old Headless implementation. For more
information, please see https://developer.chrome.com/articles/new-headless/.
Consider opting in early by passing headless: "new" to puppeteer.launch()
If you encounter any bugs, please report them to https://github.com/puppeteer/puppeteer/issues/new/choose.
page url element count: 170
node:internal/process/promises:289
triggerUncaughtException(err, true /* fromPromise */);
^
DOMException: Failed to execute 'iterateNext' on 'XPathResult': The document has mutated since the result was returned.
U (pptr:internal:3:10576)
U.next ()
Async-from-Sync Iterator.next ()
pptr:evaluateHandle;transposeIterableHandle%20(%2FUsers%2Fdwillis%2FDocuments%2FGitHub%2Fjobs-node-crawler-service%2Fnode_modules%2Fpuppeteer-core%2Flib%2Fcjs%2Fpuppeteer%2Fcommon%2FHandleIterator.js%3A132%3A77):3:17
AsyncGenerator.next ()
pptr:evaluateHandle;fastTransposeIteratorHandle%20(%2FUsers%2Fdwillis%2FDocuments%2FGitHub%2Fjobs-node-crawler-service%2Fnode_modules%2Fpuppeteer-core%2Flib%2Fcjs%2Fpuppeteer%2Fcommon%2FHandleIterator.js%3A75%3A69):4:47
at #evaluate (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:252:55)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async ExecutionContext.evaluateHandle (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:201:16)
at async IsolatedWorld.evaluateHandle (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/common/IsolatedWorld.js:122:16)
at async CDPJSHandle.evaluateHandle (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/api/JSHandle.js:159:20)
at async fastTransposeIteratorHandle (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/common/HandleIterator.js:75:54)
at async transposeIteratorHandle (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/common/HandleIterator.js:122:14)
at async transposeIterableHandle (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/common/HandleIterator.js:137:9)
at async XPathQueryHandler.queryAll (/Users/dwillis/Documents/GitHub/jobs-node-crawler-service/node_modules/puppeteer-core/lib/cjs/puppeteer/common/QueryHandler.js:122:13)
Puppeteer configuration file (if used)
No response
Puppeteer version
21.3.8
Node version
19.6.0
Package manager
npm
Package manager version
8.19.3
Operating system
macOS

I have no idea why this bug post is invalid. :-)

See https://stackoverflow.com/questions/48235278/xpath-error-the-document-has-mutated-since-the-result-was-returned

Hi. I am facing the same error, and the example was reproducible in my environment.
$ node script.js
page url element count: 170
page url element count: 170
page url element count: 170
/home/stepney141/favs/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:255
        throw new Error('Evaluation failed: ' + (0, util_js_1.getExceptionMessage)(exceptionDetails));
              ^

Error: Evaluation failed: DOMException: Failed to execute 'iterateNext' on 'XPathResult': The document has mutated since the result was returned.
    at xpathQuerySelectorAll (pptr://__puppeteer_evaluation_script__:691:26)
    at xpathQuerySelectorAll.next (<anonymous>)
    at Async-from-Sync Iterator.next (<anonymous>)
    at pptr://__puppeteer_evaluation_script__:3:13
    at AsyncGenerator.next (<anonymous>)
    at pptr://__puppeteer_evaluation_script__:4:43
    at ExecutionContext._ExecutionContext_evaluate (/home/stepney141/favs/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:255:15)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async CDPJSHandle.evaluateHandle (/home/stepney141/favs/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/JSHandle.js:65:16)
    at async fastTransposeIteratorHandle (/home/stepney141/favs/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/HandleIterator.js:28:19)
    at async transposeIteratorHandle (/home/stepney141/favs/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/HandleIterator.js:50:18)
    at async transposeIterableHandle (/home/stepney141/favs/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/HandleIterator.js:60:5)
    at async XPathQueryHandler.queryAll (/home/stepney141/favs/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/QueryHandler.js:76:9)
    at async AsyncIterableUtil.collect (/home/stepney141/favs/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/util/AsyncIterableUtil.js:20:26)
    at async getUrls (/home/stepney141/favs/script.js:43:23)

Node.js v18.18.0

This error occurs only puppeteer >=19.7.1.

I think we need a way to make XPath queries be atomic.

Still facing the issue
const getProductsData = async (
  page: Page,
  url: string
): Promise<ProductData[]> => {
  await page.goto(url)

  const body = await page.$("body")
  const innerText = await (await body?.getProperty("innerText"))?.jsonValue()
  if (innerText?.toLowerCase().includes("oops!")) {
    console.log("oops!")
    await utils.sleep(3000)
    return await getProductsData(page, url)
  }
  const waitedEl = await page.waitForXPath(selectors.amazon.xpath.resultItem)
  console.log({ waitedEl: waitedEl })

  const elements = await page.$x(selectors.amazon.xpath.resultItem)
  console.log({ elements: elements })
  const scrapedData: ProductData[] = []

  return scrapedData
}

puppeteer
  .use(StealthPlugin())
  .launch({ headless: false })
  .then(async (browser) => {
    const page = await browser.newPage()

    const results = await amazon.getProductsData(
      page,
      "https://www.amazon.in/s?i=fashion&rh=n%3A6648217031%2Cn%3A11987349031%2Cp_85%3A10440599031&dc&ascsubtag=19bBo6j6&cat=7549&d_f_save=0-100&dealsby=dealsmagnet&filter-by=0&primestatus=yes&qid=1696589137&rnid=6648217031&shortby=0&tag=worlds-most-advanced-deal-finder-21&topbrands=yes&ref=is_r_n_5&ds=v1%3Abw03pzgJ0qYfrrWdaSx4d5eRftZKkpVL7LP6u5qTXFI"
    )

    console.log(results)
  })

Getting the following error
DOMException: Failed to execute 'iterateNext' on 'XPathResult': The document has mutated since the result was returned.
j (pptr:internal:3:10576)
j.next (<anonymous>)
Async-from-Sync Iterator.next (<anonymous>)
pptr:evaluateHandle;transposeIterableHandle%20(%2Fhome%2Fsonisins%2Fprujekts%2Fnode%2Famazon-price-tracker%2Fnode_modules%2Fpuppeteer-core%2Flib%2Fcjs%2Fpuppeteer%2Fcommon%2FHandleIterator.js%3A133%3A77):3:17
AsyncGenerator.next (<anonymous>)
pptr:evaluateHandle;fastTransposeIteratorHandle%20

Puppeteer version: 21.3.8
I think the fix has not released yet


Use runc.v1 for now for debugging.

Use runc.v1 to see whether https://github.com/containerd/containerd/issues/3177 is still reproducible.
This can help us figure out whether the issue is caused by the 5 seconds timeout introduced in the v2 shim, or architecture change from per-container shim to per-pod shim.
Signed-off-by: Lantao Liu<EMAIL_ADDRESS>

/lgtm


Implemented Help Function

Context

Added Help parser
Added help explanations for each command to help command

Additional changes made on 16/10/16

Edited help message ussage ( use message usage instead of showing_help_clear_message

move the message_usage for clear and exit to their own command instead of creating it at help command

edited help paser to redirect to the updated clear and exit message_usage

Solved xuchen's comment on moving the show_messages to parser
Links

#28

@githubxxcc okay, will do so.


No module named 'ComfyUI-IF_AI_tools.IFTextTyperNode'

After I updated the plugin, ComfyUI import failed, and terminal showed this: No module named 'ComfyUI-IF_AI_tools.IFTextTyperNode'

I uploaded the text typer node about the time you post it please redownload the nodes it should work now I forgot to upload tha node last nioght


Drop support for instantiating a linker object without providing a settings dictionary

As referenced in the following comment this:

adds too much complexity to the codebase for a very minor (and probably seldom used) API improvement.

https://github.com/moj-analytical-services/splink/pull/987#discussion_r1108604286
It'd be better to drop support for this, and clean up affected areas of the codebase, which should be able to be simplified

Example where currently needed:
        Examples:
            >>> linker = DuckDBLinker(df, connection=":memory:")
            >>> linker.profile_columns(["first_name", "surname"])
            >>> linker.initialise_settings(settings_dict)

Maybe methods that need required properties like link_type need to ask for link type, and an error is produced if it's not available

IDK if this is related, but if you're going to make a large change like this, I would suggest that you change the API to
class Linker:
    def __init__(self, link_type: str, unique_id_column_name: str, ...):
        ...

    @classmethod
    def from_settings_dict(cls, settings_dict: dict):
        ...
        return cls(...)

This removes a lot of parsing/logic from the constructor which is often considered a good thing because it:

makes it easier to test because you just inject the exact args you want (benefit to you as a maintainer)
makes it easier to subclass/extend (benefit to me as a user)
For instance, I have a subclass which has two extra settings that I save in the settings dict, on_load_sql which is some sql that needs to be run at the beginning of the linking and definite_match_sql that is some sql that overrides the prediction table. See below:

from __future__ import annotations

import json
import logging
import os
from typing import Iterable

from ibis.expr.types import Table
import pandas as pd
from splink.duckdb.duckdb_linker import DuckDBLinker
from splink.splink_dataframe import SplinkDataFrame

from noatak.splink.util import load_into_duckdb

logger = logging.getLogger(__name__)

class MyDuckDBLinker(DuckDBLinker):
    def __init__(self, table_or_df: pd.DataFrame | Table):
        conn, name = load_into_duckdb(table_or_df)
        super().__init__(name, connection=conn)

    on_load_sql = None
    definite_match_sql = None

    # Override so we return the dict so we can modify it before saving.
    def save_settings_to_json(
        self, out_path: str | None = None, overwrite: bool = False
    ) -> dict:
        """Save the configuration and parameters of the linkage model to a `.json` file.

        The model can later be loaded back in using `linker.load_settings_from_json()`.
        The settings dict is also returned in case you want to save it a different way.

        Examples:
            >>> linker.save_settings_to_json("my_settings.json", overwrite=True)

        Args:
            out_path (str, optional): File path for json file. If None, don't save to
                file. Defaults to None.
            overwrite (bool, optional): Overwrite if already exists? Defaults to False.

        Returns:
            dict: The settings as a dictionary.
        """
        model_dict = self._settings_obj.as_dict()
        if self.on_load_sql:
            model_dict["on_load_sql"] = self.on_load_sql
        if self.definite_match_sql:
            model_dict["definite_match_sql"] = self.definite_match_sql
        if out_path:
            if os.path.isfile(out_path) and not overwrite:
                raise ValueError(
                    f"The path {out_path} already exists. Please provide a different "
                    "path or set overwrite=True"
                )
            with open(out_path, "w", encoding="utf-8") as f:
                json.dump(model_dict, f, indent=4)
        return model_dict

    def initialise_settings(self, settings_dict: dict):
        try:
            settings_dict = settings_dict(self)
        except TypeError:
            pass
        on_load_sql = settings_dict.pop("on_load_sql", None)
        definite_match_sql = settings_dict.pop("definite_match_sql", None)
        super().initialise_settings(settings_dict)
        if on_load_sql:
            self.on_load_sql = on_load_sql
            self._con.execute(self.on_load_sql)
        if definite_match_sql:
            self.definite_match_sql = definite_match_sql

    def update_link_table(self):
        if self.definite_match_sql is None:
            return
        df_pred = super().predict()
        sql = f"""UPDATE {df_pred.physical_name}
            SET match_probability = 1, match_weight = 30
            WHERE {self.definite_match_sql}"""
        self._con.execute(sql)

    def predict(
        self,
        threshold_match_probability: float = None,
        threshold_match_weight: float = None,
    ) -> SplinkDataFrame:
        self.update_link_table()
        return super().predict(
            threshold_match_probability=threshold_match_probability,
            threshold_match_weight=threshold_match_weight,
        )

    def train(
        self,
        blocking_rules: Iterable[str],
        n_sampled_rows: int = 10_000_000,
    ):
        """Convenience wrapper around
        estimate_u_using_random_sampling() and
        estimate_parameters_using_expectation_maximisation()
        """
        blocking_rules = list(blocking_rules)
        if len(blocking_rules) < 1:
            raise ValueError("must have at least one blocking rule")
        logger.info(f"Sampling {n_sampled_rows} rows for u estimation")
        self.estimate_u_using_random_sampling(target_rows=n_sampled_rows)
        for rule in blocking_rules:
            self.estimate_parameters_using_expectation_maximisation(rule)

Given how this would break backwards compatibility, reserving this for the next major release (splink4)

Fixed in splink4


Example burn-all is broken

burn-all currently results in this error:

Token inputs less than the token outputs

Which prevents the example from actually burning anything.

I added a burn example in v4.4.1.


[SPARK-40844][SS] Flip the default value of Kafka offset fetching config

What changes were proposed in this pull request?
This PR proposes to flip the default value of Kafka offset fetching config (spark.sql.streaming.kafka.useDeprecatedOffsetFetching) from true to false, which enables AdminClient based offset fetching by default.
Why are the changes needed?
We had been encountered several production issues with old offset fetching (e.g. hang, issue with Kafka consumer group rebalance) which could be mitigated with new offset fetching. Despite the breaking change on the ACL, there is no need for moderate users to suffer from the old way.
The discussion went through the dev. mailing list: https://lists.apache.org/thread/spkco94gw33sj8355mhlxz1vl7gl1g5c
Does this PR introduce any user-facing change?
Yes, especially users who relies on Kafka ACL based on consumer group. They need to either adjust the ACL to topic based one, or set the value to true for spark.sql.streaming.kafka.useDeprecatedOffsetFetching to use the old approach.
How was this patch tested?
Existing UTs.

cc. @gaborgsomogyi @dongjoon-hyun

Kafka community deprecated the method poll(long) in Kafka 2.0 and couldn't remove the method in Kafka 3.0. They may seem to see lots of existing usage and be hesitant to remove it. I'd guess the earliest version they can reconsider is Kafka 4.0 which is not something we need to worry for now. Maybe Spark 4.0?

cc @viirya , too

https://github.com/HeartSaVioR/spark/actions/runs/3279380674/jobs/5400649338
Tests are failing from below suites

CastWithAnsiOffSuite
TryCastSuite
RebaseDateTimeSuite
TimestampFormatterSuite
python/pyspark/sql/tests/connect/test_connect_column_expressions.py

which none of them are related to this change.

Thanks! Merging to master.


Update library to Android Oreo

SlidingUpPanel doesn't work on Android Oreo devices - please update it

i'm trying to look something, but its hard to get work..
someone can help us?

I tried the demo on an Oreo device (Pixel XL), and it works just fine. What issues are you seeing?

Sorry 'bout that, I had a slight problem with my code that was not due to the library. Transparent slide up panel was not working - turns out I did't set the overlay attribute.


Small performance improvements

Just making sure I don't get too rusty with my profiling skills ;) See the individual commits for details.
Running assemble on gradle/gradle goes from 3.5s to 3.1s on my machine.

I just realized that the foundReadyNode change won't work, since the locks can change from other places, e.g. dependency downloads. There seems to be no test for that, since core:check was green. I'm gonna revert that change.

I've instead optimized a few methods in this hot loop. The performance gain is a bit smaller, but still worth it, especially since the changes are trivial.

@bot-gradle test this

OK, I've already triggered ReadyForMerge build for you.

Thank you @oehme for your contribution!


CSP Spot Check

database, with database schema change
Show db.Column changes you have made to model/python file in Visual Studio Code
Show initialization data code in VSCode
Demo delete database and run ./migrate.sh
Demo corresponding schema change in SQLite3
Demo initialization data captured in Column in SQLite3
postman with /authentication screen
Show /authenticate endpoint code in Visual Studio Code
Show and Demo /authenticate in Postman…
Demo calling to endpoint
Show JSON data passed in call
Show Response window
Show Cookie acquired window
Cut/Copy/Paste Cookie into jwt.io and show decrypting payload
postman with /api screen to access data under authentication
Show /api CRUD code in Visual Studio Code, expecting READ/GET code
Show and Demo /api READ/GET endpoint in Postman…
Demo calling to endpoint that requires Cookie
Show Response/Data
Demo Deleting cookie
Demo calling endpoint again
Show Response/Failure

Show db.Column changes you have made to model/python file in Visual Studio Code

Show initialization data code in VSCode

Demo delete database and run ./migrate.sh

Demo corresponding schema change in SQLite3

Demo initialization data captured in Column in SQLite3

Show /authenticate endpoint code in Visual Studio Code
in api/user.py

Show and Demo /authenticate in Postman…
(http://<IP_ADDRESS>:8086/api/users/authenticate)
{"uid": "hayden", "password": "password"}
Demo calling to endpoint
http://<IP_ADDRESS>:8086/api/users
Show JSON data passed in call
Show Response window
Show Cookie acquired window

Cut/Copy/Paste Cookie into jwt.io and show decrypting payload

Show /api CRUD code in Visual Studio Code, expecting READ/GET code

Show and Demo /api READ/GET endpoint in Postman…
http://<IP_ADDRESS>:8086/api/users
Demo calling to endpoint that requires Cookie
http://<IP_ADDRESS>:8086/api/users
Show Response/Data

Demo Deleting cookie
Demo calling endpoint again
Show Response/Failure