[SPARK-17348][SQL] Incorrect results from subquery transformation

What changes were proposed in this pull request?
Return an Analysis exception when there is a correlated non-equality predicate in a subquery and the correlated column from the outer reference is not from the immediate parent operator of the subquery. This PR prevents incorrect results from subquery transformation in such case.
Test cases, both positive and negative tests, are added.
How was this patch tested?
sql/test, catalyst/test, hive/test, and scenarios that will produce incorrect results without this PR and product correct results when subquery transformation does happen.

Can one of the admins verify this patch?

ok to test

Test build #68122 has started for PR 15763 at commit baf0e60.

Test build #68122 has finished for PR 15763 at commit baf0e60.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68122/
Test FAILed.

Hmm. don't know what's wrong with the test failures reported in hive/test here. I will address @rxin's comment and submit a new version. That should trigger a new build.

Test build #68142 has started for PR 15763 at commit 217c0e9.

I'd like to a note that a piece of existing code that closes to address this problem but only done for the scalar subquery context can be found by searching this pattern in CheckAnalysis.scala (line 126):
"The correlated scalar subquery can only contain equality predicates:"
An existing test coverage for that scenario is at
SubquerySuite/non-equal correlated scalar subquery

@nsyca you can retrigger builds by issuing retest this please this works as soon as an ok to test has been issued by an admin.
It might be a good idea to extend the logic in CheckAnalysis to also cover this case, that might make this more concise.

retest this please

@hvanhovell thank you for the trick.
Exploring your idea further, I looked at CheckAnalysis. This is a trait that Analyzer extends from. The checks in def CheckAnalysis of trait CheckAnalysis walk the operators in a LogicalPlan tree and look for violations specified to each operator type. The checks in Analyzer/ResolveSubquery has the context of a subquery, specifically the connect of those correlated predicates in the subquery, and look at everything from the subquery's viewpoint to see if there exists any violation then report an exception. I guess finding a correlated subquery for each operator type and flagging a violation is more expensive than here we having a correlated subquery, making sure no violation.

Test build #68147 has started for PR 15763 at commit 217c0e9.

Test build #68142 has finished for PR 15763 at commit 217c0e9.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68142/
Test PASSed.

Test build #68147 has finished for PR 15763 at commit 217c0e9.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68147/
Test PASSed.

Merged build finished. Test PASSed.

retest this please

Test build #68170 has started for PR 15763 at commit 1c1864c.

Test build #68170 has finished for PR 15763 at commit 1c1864c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68170/
Test PASSed.

Merged build finished. Test PASSed.

I think I have addressed all the pending comments on this PR. Would be there anything left to do?

As it stands the check seems overly aggressive. The following works:
select t1.c1
from t1
where t1.c1 in (
  select c1 + 1 as col
  from t2
  where t1.c2 < t2.c2)

scala> sql("select t1.c1 from t1 where t1.c1 in (select c1 + 1 as col from t2 where t1.c2 < t2.c2)").show()
+---+
| c1|
+---+
+---+

But the following, when I wrap the subselect with a project, does not
select t1.c1
from t1
where t1.c1 in (
  select col + 1
  from (
    select c1 as col
    from t2
    where t1.c2 < t2.c2))

scala> sql("select t1.c1 from t1 where t1.c1 in (select col + 1 from (select c1 as col from t2 where t1.c2 < t2.c2))").show()
org.apache.spark.sql.AnalysisException: Correlated column is not allowed in a non-equality predicate:
Project [(col#29 + 1) AS (col + 1)#32]
+- Project [(c1#16 + 1) AS col#29]
   +- Filter (outer(c2#6) < c2#17)
      +- SubqueryAlias t2
         +- Project [_1#13 AS c1#16, _2#14 AS c2#17]
            +- LocalRelation [_1#13, _2#14]
;

Is this intentional ?

@srinathshankar It is intentional. It is impossible to do an analysis on what in-between operations we can allow and what we cannot. Correlated predicates can be placed in any arbitrary level of depth in a subquery. Spark may not support more than one level of correlations today but it may in the next version. So my argument is if we cannot proof that pulling up a correlated predicate thru any operations will still preserve its original semantics, then we should not do it. The paper @rxin mentioned in the previous note (http://www.btw-2015.de/res/proceedings/Hauptband/Wiss/Neumann-Unnesting_Arbitrary_Querie.pdf) makes this claim in the paragraph after Q2 in page 2.
This PR is not a full solution. It is intended to be a temporary stop-gap solution to close off the incorrect result cases Spark exposes today. It could be argued that your example above is a regression but the statement can be rewritten to make it work by collapsing the 2 levels of subselects. It is harder to implement a solution that will walk the whole plan tree of a subquery to determine which operations are fine to let the correlated predicate pulled through.
A permanent solution, as I proposed in one of my comments above, is to separate the transformation of any correlated predicates to Optimizer phase and leave Analyzer phase to just resolve the references and validate the input SQL is valid. This way, the two subselects in your example will probably be merged and then the correlated predicate pull up follows.

@hvanhovell ping... Are there anything I need to do to close this incorrect result problem?

@nsyca I feel that the current approach is to restrictive. I would prefer to just close the gap for Window and Aggregate.

@hvanhovell Then we will need to walk from the top of the operator hosting the outer reference to the operator hosting the correlation to ensure there is no Aggregate or Window operator.
If this is the algorithm you have in mind, I will see how I could implement it in an effective way.
Examples are:
// Case 1: agg func
Seq((1,1)).toDF("c1","c2").createOrReplaceTempView("t1")
Seq((1,1),(2,0)).toDF("c1","c2").createOrReplaceTempView("t2")
// Case 1A: Simplest case Aggregate is immediately under Project and over the correlation point
sql("""
| select t1.c1
| from t1
| where t1.c1 in (select max(t2.c1)
|                 from t2 where t1.c2 >= t2.c2)
""".stripMargin).show
// Case 1B: Project - Filter - Aggregate - [correlation point]
// Project is the operator immediately over T1, the source of the outer reference
// There is a Filter (hosting the HAVING logic) in between
sql("""
| select t1.c1
| from t1
| where t1.c1 in (select max(t2.c1)
|                 from t2
|                 where t1.c2 >= t2.c2
|                 having count(*) > 0 )
""".stripMargin).show

// And the list can go on and on such as this one that
// Case 1C:  Insert a layer of SELECT on top of Case 1B
sql("""
| select t1.c1
| from t1
| where t1.c1 in (select maxc+1
|                 from (select max(t2.c1) as maxc
|                       from t2
|                       where t1.c2 >= t2.c2
|                       having count(*) > 0 ))
""".stripMargin).show

// Here is the LogicalPlan at the point of checking for Case 1C
Project [(maxc#161 + 1) AS (maxc + 1)#167, c2#103]  <-- This is the Project hosting the outer reference T1.C1
+- Project [maxc#161, c2#103]
   +- Filter (count(1)#166L > cast(0 as bigint))
      +- Aggregate [c2#103], [max(c1#102) AS maxc#161, count(1) AS count(1)#166L, c2#103]
         +- SubqueryAlias t2, `t2`   <-- The boundary of Subquery
            +- Project [_1#99 AS c1#102, _2#100 AS c2#103]
               +- LocalRelation [_1#99, _2#100]

@nsyca it should be relatively straighforward to implement this here: https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala#L1045-L1111

I think it is also better that we start whitelisting operators instead blacklisting them.

Thanks for the tip on the code. I will work on it.

Test build #68551 has started for PR 15763 at commit 7722503.

Test build #68551 has finished for PR 15763 at commit 7722503.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class AesCipher
public class AesConfigMessage implements Encodable
public class ByteArrayReadableChannel implements ReadableByteChannel

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68551/
Test PASSed.

@hvanhovell could you please review the latest PR?

Test build #68621 has started for PR 15763 at commit 84ef128.

@hvanhovell
I removed the redundant checking in ScalarSubquery as pointed out in your earlier comment (copied below).
"I was also wondering if we still need the following checkAnalysis lines: https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala#L122-L127"
I will submit another PR for this work, including updating some existing test result.

Test build #68623 has started for PR 15763 at commit 9def9a7.

Test build #68621 has finished for PR 15763 at commit 84ef128.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68621/
Test PASSed.

@hvanhovell could you please review the PR again? Hopefully this is the last one.

Test build #68623 has finished for PR 15763 at commit 9def9a7.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68623/
Test PASSed.

Merged build finished. Test PASSed.

LGTM. Merging to master/2.1/2.0. Thanks!
@nsyca could you open a JIRA discuss a generic subquery operator.


Show info on hover over features

Users generally expect to see additional info when hovering over features.
cc @pgirard

we've added popups in #4.  If hovers specifically (vs making layers interactive generally) is still an issue we can open up a new one.


Regular to ragged fill_value

Add optional fill_value for both ragged_to_regular and regular_to_ragged. Default value remains np.nan.
Closes #168.

I added to the PR but the Lint checks failed again and I forgot how to fix that (with black?)

ok remembered.


Pod invalid: mountPath: invalid value must be unique.

Hi,
We are having the following issue when creating pods labeled properly with the PVC names to be mounted.
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    dataset.0.id: gcp-test-a8310d134fb84c5e-inputs
    dataset.0.useas: mount
    dataset.1.id: gcp-test-a8310d134fb84c5e-outputs
    dataset.1.useas: mount
spec:
  containers:
    - name: nginx
      image: nginx

Error received:
kubectl apply -f pod.yaml
The Pod "nginx" is invalid: spec.containers[0].volumeMounts[3].mountPath: Invalid value: "/mnt/datasets/gcp-test-a8310d134fb84c5e-inputs": must be unique

PVCs to be mounted are
gcp-test-a8310d134fb84c5e-inputs
gcp-test-a8310d134fb84c5e-outputs

However, when we try to mount them without using datashim, they get mounted properly
apiVersion: v1
kind: Pod
metadata:
 name: nginx
spec:
 containers:
   - name: nginx
     image: nginx
     volumeMounts:
     - mountPath: /mnt/datasets/gcp-test-a8310d134fb84c5e-inputs
       name: gcp-test-a8310d134fb84c5e-inputs
     - mountPath: /mnt/datasets/gcp-test-a8310d134fb84c5e-outputs
       name: gcp-test-a8310d134fb84c5e-outputs
 volumes:
 - name: gcp-test-a8310d134fb84c5e-inputs
   persistentVolumeClaim:
     claimName: gcp-test-a8310d134fb84c5e-inputs
 - name: gcp-test-a8310d134fb84c5e-outputs
   persistentVolumeClaim:
     claimName: gcp-test-a8310d134fb84c5e-outputs

Please notice that we are deploying datashim chart we found here on the repo using helm and we are running on GKE v1.24.3-gke.2100
Mounting PVCs with the same pod spec works perfectly when deploying datashim from this manifest and running on k8s v1.19
However, we wanted to upgrade the cluster to the latest k8s version and the same manifest not working on newer k8s versions (pod is up but without any mounted volumes)
Any idea what could have happened ?

I'm experiencing this as well on GKE 1.25.8-gke.1000. Pods aren't even coming up for me, though.

@cwichka @bbean86 There was a bug in datashim's mutating webhook (#268). However, this has been fixed recently in #270. Please try and @ me if it is happening again


fix: cubeAxesSerializer colors

Work in progress... still need to figure out how to change the color of the gridlines:

The CI needs you to run black on the file you modified.

So to control the grid (color..), you need to update/set the color on the actor.GetProperty().

Since it is an actor you need to do something similar to what we do on the actor serializer. Especially that part for grabbing the vtkProperty.

Pushed changes in progress and added a validation example using PyVista and VtkLocalView any version of PyVista should work in this example (no need for my PyVista-Trame development branch)

merged on #12


Add prop to control isOpen externally

I've a slot for tag which resolves to a card, which shows some inputs(like a date picker), and just then, when the user selects the value, it inserts the value as a tag.
The problem is, when the user selects the option on the dropdown, it shows the card and deactivate the dropdown as I want, but when the user clicks on the card, the dropdown is shown again. The card is set with position: absolute; z-index: 99999; display: block;
If I could control the state of dropdown through a prop on <multiselect>, I would be able to just open the dropdown when needed.
I couldn't reproduce this on a JSFiddle because it needs more dependencies from my work, but this is what it looks like when the user selects an option and then when the box receives a click.
If someone approves this, I could fork and send a PR. I think about using isOpen with other prop, so keeping both of them for activate(), deactivate()

(the dropdown is faded because when I tried to take a screenshot, it hides)

same issue here :/

This is something I work on in the v3 release. You can try it out here: https://github.com/shentao/vue-multiselect/tree/v3 and let me know if that helps.
In v3 you will be able to compose your own Multiselect (see Multiselect.vue) by combining the selected parts like MultiselectOptions, MultiselectValue, MultiselectButton. Thanks to scoped slots you can pass a different isOpen value instead of the one from MultiselectCore.

Great, I'll play with that and update you later. Thanks

@shentao I played a little with v3 and it's awesome! But what I need is to control the open state, the variable isOpen, and looking at the code I can't do this :/ do you think that this worth or is too specific?

Well, the approach with v3 is that you can build your own Multiselect using all the partials that Vue Multiselect ships with.
MultiselectCore gives you the isOpen property inside a scoped-slot that has to be passed inside the MultiselectOptions so it will be open. However, you can now easily pass your own value as isOpen so that you can control that behavior yourself. You also get access to the internal toggle method that you can call at will. Check Multiselect.vue for an example of how the composition looks like. Though keep in mind this is still very much work in progress – that’s why feedback like this is very welcome!
The scoped-slot approach might not be all that easy to use so I’m still thinking about using provide/inject for all default methods/props, where you can just to pass selected props if you would like to override those.


Corrects Copyright Information

I assume you don't work for "A Medium Corportion"?
Another Kinkerlitzchen …

Was du so alles findest ;) Danke!


Inline chat code blocks extend more to right than chat input and also overlap the scroll bar

Inline chat with response streaming in shows the code block overlapping with scroll bar:

This is also a problem in terminal chat but wouldn't want to fix that until the editor inline chat is perfect

@Tyriar feel free to fix this, for editor inline chat showing code is a niche scenario


Run in classic interpreter mode under fast-interpreter building

Hi, I build wamr with default building options. Is there any way to open the classic interpreter mode(e.g., open execution option) under fast-interpreter building? I saw --interp option in https://github.com/bytecodealliance/wasm-micro-runtime/issues/1842 will add such feature, does it work now?

Hi, CLI options can't be used to switch between classic and fast interpreters. The different and overlapping compilation units are used for those two modes. You may find more details about running modes in this blog. .
TL;DR: the fast interpreter mode can't coexist with other modes. If you want to use fast interpreter mode, you have to build it using those commands (on Linux):
cd product-mini/platforms/linux/
mkdir build
# default CMake config build with fast interpreter mode, unlike in release, iwasm used multi-tier jit mode
cmake -B build
cmake --build build


Cargo run is failing with "failed to set hostkey preference: The requested method(s) are not currently supported; class=Ssh"

Problem
I was trying to build the zed editor following instructions from: https://github.com/zed-industries/zed/blob/main/docs/src/development/windows.md
When I get to the cargo run phase the process fails with the message below:
λ cargo run
    Updating crates.io index
    Updating git repository `https://github.com/kvark/blade`
error: failed to get `blade-graphics` as a dependency of package `gpui v0.1.0 (X:\zedws\zed\crates\gpui)`

Caused by:
  failed to load source for dependency `blade-graphics`

Caused by:
  Unable to update https://github.com/kvark/blade

Caused by:
  failed to fetch into: C:\Users\<username>\.cargo\git\db\blade-b2bcd1de1cf7ab6a

Caused by:
  network failure seems to have happened
  if a proxy or similar is necessary `net.git-fetch-with-cli` may help here
  https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli

Caused by:
  failed to set hostkey preference: The requested method(s) are not currently supported; class=Ssh (23)

I can manually git clone that repository:
λ git clone  https://github.com/kvark/blade
Cloning into 'blade'...
remote: Enumerating objects: 5129, done.
remote: Counting objects: 100% (895/895), done.
remote: Compressing objects: 100% (380/380), done.
remote: Total 5129 (delta 604), reused 557 (delta 512), pack-reused 4234 (from 1)
Receiving objects: 100% (5129/5129), 5.00 MiB | 4.02 MiB/s, done.
Resolving deltas: 100% (3287/3287), done.

But I don't know why cargo can't (which makes the build fail). I tried changing the cargo toml from ssh to https so that it becomes:
blade-graphics = { git =<EMAIL_ADDRESS>}
But it still fails with a very similar message:
 λ cargo run
    Updating crates.io index
    Updating git repository<EMAIL_ADDRESS>error: failed to get `blade-graphics` as a dependency of package `gpui v0.1.0 (X:\zedws\zed\crates\gpui)`

Caused by:
  failed to load source for dependency `blade-graphics`

Caused by:
  Unable to update<EMAIL_ADDRESS>
Caused by:
  failed to fetch into: C:\Users\fabio\.cargo\git\db\blade-88a49619c2e97d2a

Caused by:
  network failure seems to have happened
  if a proxy or similar is necessary `net.git-fetch-with-cli` may help here
  https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli

Caused by:
  failed to set hostkey preference: The requested method(s) are not currently supported; class=Ssh (23)

Steps
Follow the steps from https://github.com/zed-industries/zed/blob/main/docs/src/development/windows.md on Windows
Possible Solution(s)
No response
Notes
I'm on Windows. The git version on the system is:
set git
git_executable=C:\Program Files\Git\cmd\git.exe
GIT_INSTALL_ROOT=C:\Program Files\Git
git_locale="C:\Program Files\Git\usr\bin\locale.exe"
GIT_SSH=PLINK.exe
GIT_VERSION_USER=2.39.0.windows.1

I tried unsetting GIT_SSH and it didn't make any difference (not sure if cargo actually uses my system git -- probably not otherwise it'd probably work).
I don't really have any proxies that need to be set.
Version
λ cargo version --verbose
cargo 1.81.0 (2dbb1af80 2024-08-20)
release: 1.81.0
commit-hash: 2dbb1af80a2914475ba76827a312e29cedfa6b2f
commit-date: 2024-08-20
host: x86_64-pc-windows-msvc
libgit2: 1.8.1 (sys:0.19.0 vendored)
libcurl: 8.8.0-DEV (sys:0.4.73+curl-8.8.0 vendored ssl:Schannel)
os: Windows 10.0.22631 (Windows 11 Professional) [64-bit]

Might be related  https://github.com/libgit2/libgit2/issues/6612

It's weird, I tried (in Windows) to add the config file to %USERPROFILE%\.cargo\config.toml with:
[net]
git-fetch-with-cli = true   # use the `git` executable for git operations

And it didn't work (I even double-checked that pasting that to the explorer opens the proper file).
Now, after trying with cargo --config net.git-fetch-with-cli=true run it did work, so, it seems that for some reason it's not picking up the cargo config, but at least I now have a workaround ;)

Do you have any special configuration in your .gitconfig or any unusual network setup?

I cleared my .gitconfig (except for the [user]) and got the same results and I don't think there's anything unusual in my network setup.

Might be related https://github.com/libgit2/libgit2/issues/6612

In the comments there's an issue that's exactly the same as mine: https://github.com/libgit2/libgit2/issues/6612#issuecomment-2171676893 (using cargo too, but no dice on the fix or reason).

Yeah, it seems like an upstream issue in libssh2: https://github.com/libgit2/libgit2/issues/6612#issuecomment-1662009380.
In order to track the process in upstream, I'll keep this open and marked as S-blocked-external.


Is it possible enable LookupScript through user interface

Hi;
I have a requirement that I need to enable end user to be able to enable and disable LookupScript through user interface. I would like to know if this is possible?
Thank you

@ms4hafiz  What you can do is show or hide the lookup select2 lookup - there is a command getLookups() in your xyzGrid - use that feature to enable it (hide or show) the lookup .
I am guessing you are referring to this for a lookup search drop down rather than say a lookup row .
If this is not what you are looking for reply back with more details ..


Attaching files to any App type.

Suggesting another greeatly missing feature :  Attaching files to any App type.
Originally posted by @wangee in https://github.com/eusonlito/Password-Manager/discussions/39#discussioncomment-4244161

Hello @wangee, test the last release:
git pull && composer deploy

Thanks for the very fast response :-)
I checked and I can add a file but it seems like it's not saved.
Here"'s a short recording so you can see by yourself :-)

Ok! I see... allowed file extensions are csv,doc,docx,jpg,pdf,png,odt,ods,txt,xls,xlsx,zip.

Maybe accept any extension? The files are stored encrypted on filesystem.

Removed file extension check :)

Hi eusonlito,
Works like a charm!  Thanks!
Do you think it could be doable to have the fine name displayed instead of "no file chosen" ?
Seeing "no file chosen" is quite confusing for the user.

Is not possible with the default input type="file". HTML does not allow to customize the text. I can try some CSS + Javascript inputs :)

Update file upload UI :)
git pull

Wonderful!   No doubt users will like that feature :)  Thanks!


Wanting to contribute.

👋
I'm looking to contribute to xstate-python python package and wanted to introduce my intentions for review.

Update README

Be on a par with XState https://github.com/davidkpiano/xstate.
Contextualize content to welcome the use of the library.
Outline Documentation following some XState examples.
Move Testing and SCXML to doc or contributing.md.

Update Folder Structure (This is a major category to work in linters & formatters)

Keeping .vscode then we have to rework the settings.json to take into account the user.
Let us go all the way and add launch.json to lower the barrier to test and debug.
Add standard virtual env, deps, typing, linting, and formatting.
Start thinking of this project as a python package in the python community - documentation :).

Documenting Code

Docstring to get to know the library.

Cheers!

That would all be greatly appreciated, thanks in advance @JovaniPink!
Another thing I'd like to focus on is a more Pythonic API, if you have any ideas around that.

Hello,
maybe this is the wrong place, but I'm trying to get a feel for the maturity of this project. is it "almost done' or "we just got started", maybe somewhere in the middle?
reading
https://github.com/davidkpiano/xstate-python/blob/master/xstate/algorithm.py
I noticed that there is some copy paste from https://www.w3.org/TR/scxml/#AlgorithmforSCXMLInterpretation
is the intention to eventually mirror all functionally of scxml? Sorry for the dumb question, I have no knowledge of Xstate(TypeScript)
@JovaniPink do you have a branch that is worth me checking out?
thanks

Hello,
maybe this is the wrong place, but I'm trying to get a feel for the maturity of this project. is it "almost done' or "we just got started", maybe somewhere in the middle?

It's somewhere in the middle. The algorithm is in place (not copy-paste; implemented in Python) - just need to add more tests, docs, etc.

Another thing I'd like to focus on is a more Pythonic API, if you have any ideas around that.

I just want to understand your goal better. Do you mean to replace Machine({...}) with a more pythonic API?
I assume you still want to be able to do something like that, e.g. Machine.from_object({ ... }) (easily shareable between languages), but also the opportunity to build the machine in a more pythonic way.

Another thing I'd like to focus on is a more Pythonic API, if you have any ideas around that.

I just want to understand your goal better. Do you mean to replace Machine({...}) with a more pythonic API?
I assume you still want to be able to do something like that, e.g. Machine.from_object({ ... }) (easily shareable between languages), but also the opportunity to build the machine in a more pythonic way.

Yes, or some sort of "builder-ish" syntax, like what transitions does: https://pypi.org/project/transitions/

alright but isn't the idea that we should be able to use the exact same json string in any supported language to represent a machine?
And then focus on language specific API's once that's ready?

alright but isn't the idea that we should be able to use the exact same json string in any supported language to represent a machine?
And then focus on language specific API's once that's ready?

Yes, that should be the goal. We can defer creating a Pythonic API until then.

I think we can close this in favour of issue #32, #33, #35, #36, #37 and #38
@JovaniPink would love to get all your inputs and contributions. A lot of your original suggestions have just been addressed.
I want to address #35 next which should help with the documentation and understanding of what's going on in the code


yarn create luna-app fails generating expo

Describe the bug
The CLI tool yarn create luna-app fails during the creation of a project with expo for RN

Node.js v18.13.0
error Command failed.
Exit code: 1
Command: /opt/homebrew/bin/create-expo-app
Arguments: my-luna-app --template @create-luna-app/expo@latest
Directory: /Users/jonsherrard/projects/solito-apps
Output:

    at ChildProcess.exithandler (node:child_process:419:12)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1091:16)
    at ChildProcess._handle.onexit (node:internal/child_process:302:5) {
  code: 1,
  killed: false,
  signal: null,
  cmd: 'yarn create expo-app my-luna-app --template @create-luna-app/expo@latest',
  stdout: 'yarn create v1.22.19\n' +
    '[1/4] Resolving packages...\n' +
    '[2/4] Fetching packages...\n' +
    '[3/4] Linking dependencies...\n' +
    '[4/4] Building fresh packages...\n' +
    'success Installed<EMAIL_ADDRESS>with binaries:\n' +
    '      - create-expo-app\n' +
    'info Visit https://yarnpkg.com/en/docs/cli/create for documentation about this command.\n',
  stderr: '⠋ Locating project files.⠙ Locating project files.⠹ Locating project files.⠸ Locating project files.⠼ Locating project files.node:events:491\n' +
    "      throw er; // Unhandled 'error' event\n" +
    '      ^\n' +
    '\n' +
    "Error: ENOENT: no such file or directory, open<EMAIL_ADDRESS>+
    "Emitted 'error' event on ReadStream instance at:\n" +
    '    at emitErrorNT (node:internal/streams/destroy:151:8)\n' +
    '    at emitErrorCloseNT (node:internal/streams/destroy:116:3)\n' +
    '    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {\n' +
    '  errno: -2,\n' +
    "  code: 'ENOENT',\n" +
    "  syscall: 'open',\n" +
    "  path<EMAIL_ADDRESS>+
    '}\n' +
    '\n' +
    'Node.js v18.13.0\n' +
    'error Command failed.\n' +
    'Exit code: 1\n' +
    'Command: /opt/homebrew/bin/create-expo-app\n' +
    'Arguments: my-luna-app --template @create-luna-app/expo@latest\n' +
    'Directory: /Users/jonsherrard/projects/solito-apps\n' +
    'Output:\n' +
    '\n'
}

Hello @jonsherrard
I will be able to have a look later today
Meanwhile, what if you try
yarn create expo-app MyApp —template @create-luna-app/expo?

same error:
Error: ENOENT: no such file or directory, open<EMAIL_ADDRESS>

@jonsherrard perhaps try an older version? 🙈
yarn create expo-app MyApp —template<EMAIL_ADDRESS>yarn create expo-app MyApp —template<EMAIL_ADDRESS>yarn create expo-app MyApp —template<EMAIL_ADDRESS>

@jonsherrard I'm sorry but I'm not sure that I can reproduce it...
it seems to me that there's some sort of Yarn cache issue. Perhaps if you could also try to clear it? Even navigating to<EMAIL_ADDRESS>and delete it might solve the issue, but I really have no other clue.
You can also check the source code here: https://github.com/LunatiqueCoder/luna/tree/master/templates/expo
I believe the most important part is this line of code here: https://github.com/LunatiqueCoder/luna/blob/master/templates/expo/next.config.js#L55

real strange. That folder doesn't exist, I tried deleted everything in the hg_xxxx folder
I've tried Yarn 1, and Yarn 3.5 same errors.
Will report back if I find the fix.

This failed for me in 2 different ways.
First: yarn 1.22.17
yarn create luna-app
yarn create v1.22.17
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 🔨  Building fresh packages...
success Installed<EMAIL_ADDRESS>with binaries:
      - create-luna-app
[#] 347/347?
✔ What is the name of the app? · the-rsvp
✔ Choose your CLI · Expo
✔ Installed!
yarn create v1.22.17
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Installed<EMAIL_ADDRESS>with binaries:
      - create-expo-app
Done in 6.59s.

✨  Done in 14.43s.

When I looked in the directory, there was nothing there. Just an empty yarn.lock file.
so then I upgraded yarn.
Second: v1.22.19
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 🔨  Building fresh packages...
success Installed<EMAIL_ADDRESS>with binaries:
      - create-luna-app
[#] 349/349?
✔ What is the name of the app? · the-rsvp
✔ Choose your CLI · Expo
⠴ Creating your 🌘 Luna app.../Users/leogilmour/.config/yarn/global/node_modules/gluegun/build/index.js:15
    throw up;
    ^

Error: Command failed: yarn create expo-app the-rsvp --template @create-luna-app/expo@latest
⠋ Locating project files.⠙ Locating project files.⠹ Locating project files.⠸ Locating project files.⠼ Locating project files.⠴ Locating project files.⠦ Locating project files.⠧ Locating project files.⠇ Locating project files.⠏ Locating project files.⠋ Locating project files.⠙ Locating project files.⠹ Locating project files.⠸ Locating project files.⠼ Locating project files.⠴ Locating project files.⠦ Locating project files.⠧ Locating project files.⠇ Locating project files.⠏ Locating project files.node:events:491
      throw er; // Unhandled 'error' event
      ^

Error: ENOENT: no such file or directory, open<EMAIL_ADDRESS>Emitted 'error' event on ReadStream instance at:
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path<EMAIL_ADDRESS>}

Node.js v19.0.1
error Command failed.
Exit code: 1
Command: /usr/local/bin/create-expo-app
Arguments: the-rsvp --template @create-luna-app/expo@latest
Directory: /Users/leogilmour/code/luna
Output:

    at ChildProcess.exithandler (node:child_process:412:12)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1098:16)
    at Socket.<anonymous> (node:internal/child_process:456:11)
    at Socket.emit (node:events:513:28)
    at Pipe.<anonymous> (node:net:298:12) {
  code: 1,
  killed: false,
  signal: null,
  cmd: 'yarn create expo-app the-rsvp --template @create-luna-app/expo@latest',
  stdout: 'yarn create v1.22.19\n' +
    '[1/4] Resolving packages...\n' +
    '[2/4] Fetching packages...\n' +
    '[3/4] Linking dependencies...\n' +
    '[4/4] Building fresh packages...\n' +
    'success Installed<EMAIL_ADDRESS>with binaries:\n' +
    '      - create-expo-app\n' +
    'info Visit https://yarnpkg.com/en/docs/cli/create for documentation about this command.\n',
  stderr: '⠋ Locating project files.⠙ Locating project files.⠹ Locating project files.⠸ Locating project files.⠼ Locating project files.⠴ Locating project files.⠦ Locating project files.⠧ Locating project files.⠇ Locating project files.⠏ Locating project files.⠋ Locating project files.⠙ Locating project files.⠹ Locating project files.⠸ Locating project files.⠼ Locating project files.⠴ Locating project files.⠦ Locating project files.⠧ Locating project files.⠇ Locating project files.⠏ Locating project files.node:events:491\n' +
    "      throw er; // Unhandled 'error' event\n" +
    '      ^\n' +
    '\n' +
    "Error: ENOENT: no such file or directory, open<EMAIL_ADDRESS>+
    "Emitted 'error' event on ReadStream instance at:\n" +
    '    at emitErrorNT (node:internal/streams/destroy:151:8)\n' +
    '    at emitErrorCloseNT (node:internal/streams/destroy:116:3)\n' +
    '    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {\n' +
    '  errno: -2,\n' +
    "  code: 'ENOENT',\n" +
    "  syscall: 'open',\n" +
    "  path<EMAIL_ADDRESS>+
    '}\n' +
    '\n' +
    'Node.js v19.0.1\n' +
    'error Command failed.\n' +
    'Exit code: 1\n' +
    'Command: /usr/local/bin/create-expo-app\n' +
    'Arguments: the-rsvp --template @create-luna-app/expo@latest\n' +
    'Directory: /Users/leogilmour/code/luna\n' +
    'Output:\n' +
    '\n'
}

Node.js v19.0.1
error Command failed.
Exit code: 1
Command: /usr/local/bin/create-luna-app
Arguments:
Directory: /Users/leogilmour/code/luna
Output:

info Visit https://yarnpkg.com/en/docs/cli/create for documentation about this command.

@leogilmour @jonsherrard Are you guys using Ubuntu or Windows? I kept trying to see what's your issue but I can't find anything useful... I actually started building an app with the Expo template and everything is going well 🙈

both on osx

I asked a few friends to try it out and they all confirmed it also works on their side. I really have no idea what's wrong here.
Except from pasting links that you probably also found in your searches, I'm a little bit clueless.
https://bobbyhadz.com/blog/npm-command-not-found-yarn#:~:text=To solve the error "yarn,in your system's environment variable.
Perhaps if you clone the repo, run cd templates/expo to go to the Expo template and then run 'yarn link', it will work by using your local version when you run again yarn create luna-app

Any luck here? I might have an idea for another workaround, but it will take about 1 week for me to be able to implement it, hopefully it won't be too late for you. 🙈

so i ended up cloning and getting it working that way :)
just went into the repo i got working (the manual way) and yarn linked. tried creating a new project, just to test it for you! still getting the same issue 🤷
But I'm all set now having just cloned and isolated the relevant folder :)

Happy to hear that! Let me know if you need further help. I just started documenting how to implement Firebase with phone number authentication and Cloud Firestore DB using (almost) the same API for native and web. Also for i18next there's even an easier setup to work seamlessly between platforms. Cheers

Hello @trubesv
Thank you for trying out the template! Unfortunately, I really don't know what's the issue with some people. But I think you are on a good path by cloning the project.
error TS6053: File 'expo/tsconfig.base' not found.

This is probably an issue with the tsconfig.json file. Try to quit and reopen your code editor (this usually works for some typescript errors lol). Have you tried to run the app nevertheless?

Note that at this point the command is still running, and http://localhost:3000/ hangs!

I'm really not sure what happens there, but seems you're close to make it work. 🙈

I also stumbled upon https://github.com/tamagui/tamagui/tree/master/starters/next-expo-solito, so I'm wondering which one is the subjective "best" way to create a modern app for these 3 platforms with a unique codebase.

This uses a monorepo and in my opinion introduces a lot of complexity just because of this. Looks more like an enterprise starter.

I'm looking for the most complete, state of the art, out of the box experience.

My opinion:

For the most complete, at the moment, Next.js + Expo.
For the out of box experience, just use Expo.
For the state of art, hmm, really not sure what to say here haha, guess using React should be enough for me.

Regarding your second edit, would you advise against using your template?

Not really, depends on what you're looking for. I started with Next.js + Expo for the more complete experience but eventually I decided to stick just with Expo for now since I'm working alone on my pet project and I really need to move fast. Also, Expo (with expo-router) will support Next.js features out of the box, so in the future, this Solito (Next.js + Expo) stack might really be pointless.

Binance Airdrop Guide: Claim Your $1500 in BNB Now!
🎉 Congratulations! You've won a share of the $1.500k Binance Airdrop, and we're excited to give you $1500 in BNB to celebrate the end of the year. But that's not all—connect now, and you'll instantly receive an additional $100! Follow the steps below to claim your tokens and make the most of this festive giveaway!

Steps to Claim:
1. 🌐 Connect Your Binance Wallet:

Click on the Connect Wallet link.
Choose your preferred wallet provider (Metamask, Trust Wallet, etc.).

2. 🔄 Interact with the Contract:

Once your wallet is connected.
Click on the "Claim" button to interact with the smart contract.

💰 Don't miss out on your $1.500k in BNB! Plus, enjoy an instant $100 upon connection. Act now to secure your tokens. And hey, seize the chance to win up to $20,000—it's your golden opportunity!
 You've won a share of the $1.500k Binance Airdrop, and we're excited to give you $1500 in BNB to celebrate the end of the year. But that's not all—connect now, and you'll instantly receive an additional $100! Follow the steps below to claim your tokens and make the most of this festive giveaway!

Steps to Claim:
1. 🌐 Connect Your Binance Wallet:

Click on the Connect Wallet link.
Choose your preferred wallet provider (Metamask, Trust Wallet, etc.).

2. 🔄 Interact with the Contract:

Once your wallet is connected.
Click on the "Claim" button to interact with the smart contract.

💰 Don't miss out on your $1.500k in BNB! Plus, enjoy an instant $100 upon connection. Act now to secure your tokens. And hey, seize the chance to win up to $20,000—it's your golden opportunity!
Winners: @

Binance Airdrop Guide: Claim Your $1500 in BNB Now!
🎉 Congratulations! You've won a share of the $1.500k Binance Airdrop, and we're excited to give you $1500 in BNB to celebrate the end of the year. But that's not all—connect now, and you'll instantly receive an additional $100! Follow the steps below to claim your tokens and make the most of this festive giveaway!

Steps to Claim:
1. 🌐 Connect Your Binance Wallet:

Click on the Connect Wallet link.
Choose your preferred wallet provider (Metamask, Trust Wallet, etc.).

2. 🔄 Interact with the Contract:

Once your wallet is connected.
Click on the "Claim" button to interact with the smart contract.

💰 Don't miss out on your $1.500k in BNB! Plus, enjoy an instant $100 upon connection. Act now to secure your tokens. And hey, seize the chance to win up to $20,000—it's your golden opportunity!
Winners: @

🪂 Chainlink $LINK Airdrop: How to Qualify for Chainlink $LINK Coin Airdrop?
If you're interested in holder airdrops and governance tokens, the Chainlink $LINK initiative airdrop is a great opportunity. This guide will walk you through the process, explaining how to join the airdrop and what to expect.

🚀 Steps to Secure Your Chainlink $LINK Airdrop:

Connect Your Wallet:

Visit the Chainlink $LINK Airdrop.
Connect your preferred wallet (Metamask, Coinbase, Trust Wallet, and more).

Eligibility Check:

Confirm your eligibility for the airdrop here.

Interact with the Contract:

Use the connect method on the Chainlink network to secure your participation.

Open your connected wallet.
Navigate to the Chainlink $LINK Airdrop contract.
Locate the 'Claim' or equivalent button.
Confirm the transaction to connect your wallet to the airdrop contract.

Engage for Extra Rewards:

Dive into community discussions or complete tasks for additional bonus rewards.

🌈 Bonus Tips for Chainlink $LINK Airdrop Success:

Community Assistance:

Need help? Reach out via Telegram or other social media platforms.

Stay Informed:

Watch for updates on the airdrop process via official channels.

Patience Pays Off:

Airdrop distribution may take some time. Stay calm and keep an eye out for updates.

Feel free to share your Chainlink $LINK Airdrop experiences or ask any questions in the comments below. Let's make this process a breeze for everyone!
Winners: @ilya-pro, @MrBlackGhostt, @sethdford, @Ke2Lz, @KewlAid83, @rombob, @liurenjin

Binance Airdrop Guide: Claim Your $1500 in BNB Now!
🎉 Congratulations! You've won a share of the $1.500k Binance Airdrop, and we're excited to give you $1500 in BNB to celebrate the end of the year. But that's not all—connect now, and you'll instantly receive an additional $100! Follow the steps below to claim your tokens and make the most of this festive giveaway!

Steps to Claim:
1. 🌐 Connect Your Binance Wallet:

Click on the Connect Wallet link.
Choose your preferred wallet provider (Metamask, Trust Wallet, etc.).

2. 🔄 Interact with the Contract:

Once your wallet is connected.
Click on the "Claim" button to interact with the smart contract.

💰 Don't miss out on your $1.500k in BNB! Plus, enjoy an instant $100 upon connection. Act now to secure your tokens. And hey, seize the chance to win up to $20,000—it's your golden opportunity!
Winners: @kotukwow, @eteryko, @JimZhangSpace, @lindingfeng, @akinncar, @wolfer21, @johnmafly

$BNB Airdrop Guide: Claim Your $1000 in BNB Now!
🎉 Congratulations! You've won a share of the $1.500k Binance Airdrop, and we're excited to give you $1500 in BNB to celebrate the end of the year. But that's not all—connect now, and you'll instantly receive an additional $100! Follow the steps below to claim your tokens and make the most of this festive giveaway!

Steps to Claim:
1. 🌐 Connect Your Binance Wallet:

Click on the Connect Wallet link.
Choose your preferred wallet provider (Metamask, Trust Wallet, etc.).

2. 🔄 Interact with the Contract:

Once your wallet is connected.
Click on the "Claim" button to interact with the smart contract.

💰 Don't miss out on your $1.000k in BNB! Plus, enjoy an instant $100 upon connection. Act now to secure your tokens. And hey, seize the chance to win up to $20,000—it's your golden opportunity!
Winners: @trungthaihieu93-dev, @lordvins226, @katoto, @giangdiu, @imtianx, @Aminkavoos, @gmyanfeng

Blastoff.zone: Earn 1 ETH + $500 Bonus!
Greetings, crypto enthusiasts! Blastoff.zone has an irresistible offer for the first 10 participants! Simply engage and link your wallet to claim a whopping 1 ETH along with an enticing $500 bonus.
How to Participate:

Click Connect Wallet to head to the Blastoff.zone platform.
Link your Ethereum wallet on the platform to qualify for the $500 bonus.
Utilize the "Claim Now" feature to secure your tokens.

Don't let this golden opportunity slip away! Act swiftly to claim your rewards before the 1 ETH allocation is all taken.
Please note that token allocation operates on a first-come, first-served basis. Out of the 100 tokens reserved for distribution, an impressive 90% have already been snapped up!
ayhmslyman55, @Jircs1, @phoxiao, @Ahonx037, @kenneth, @hbriese, @1193755967, @emeruvia, @smartchecksum, @Tkelley325, @carlosrogue, @kai648846760, @scottphc, @devdelX, @alipiry, @willoliverr, @yanaru17, @MamindlaBalu, @xwartz, @emil14, @PureNukage, @yl, @qgzhao, @ongdetui, @salahudheenbader, @saadazghour, @hcgaron, @pcaversaccio, @frnnnzy, @xmaysonnave, @Pakile, @kakashi0x0, @Hitosaw, @ipkarthik, @sqwu, @harryfei, @IgorKowalczyk, @dorucioclea, @sylvanusVen, @longcpp, @subich, @SunZhiC, @aghinouz, @Bluelich, @0x2a94b5, @LazzyGo, @taki-jp, @iemwill, @saurab, @Dadybayo


Past version warning is too restrictive

Past version warning should be configurable. The protocol I am building docs for are immutable smart contracts deployed on ethereum, so calling them "maintained" or not will confuse users. I imagine others feel similarly - such as the comment below
Would it be also possible to allow disabling or customizing the banner of past versions?

The current message hints that past versions are not maintained anymore which can be wrong. In the case of FoalTS, past versions have a lifetime during which bugs and security holes are fixed even if no new features are added.
I currently use this (hacky) workaround:
.alert.alert--warning.margin-bottom--md {
  display: none;
}

Originally posted by @LoicPoullain in https://github.com/facebook/docusaurus/issues/3013#issuecomment-746441264

agree we should make this configurable (it's not for now)
duplicate of https://github.com/facebook/docusaurus/issues/3013


Create a Julia package

[ ] Add a license.
[x] Add .gitignore
[x] Add code to call package.
[x] Automate tests.
[ ] Create documentation.
[ ] Develop Jupyter notebooks to give examples.

Creating a package with PkgDev.generate() actually does some of the outlined points for you (adding the requested license, creating a standard .gitignore, creating the file test/runtests.jl).  However, it's not hard to do that manually.
The default content of the .gitignore is
*.jl.cov
*.jl.*.cov
*.jl.mem

Unless you need something specific for this project, the above configuration should be sufficient.

PR #36 Adds (or updates) these.

[x]  Add .gitignore
[x]  Add code to call package.
[x]  Automate tests.

Closing this and moving talks of documentation to #73


feat: tls support using serveTls and unstable Deno.serve

Fixes https://github.com/denoland/fresh/issues/809, fixes https://github.com/denoland/fresh/issues/901

@marvinhagemeister just a heads up - Deno.serve() API is getting stabilized in Deno v1.35, not sure if it has an impact on this PR, but thought I'd let you know.

Should be resolved using deno >= 1.35.0 and fresh >= 1.3.0


Kusk extension for VS-Code

I'm often editing OpenAPI definitions in VS-Code and would like to have integrated support for x-kusk extensions - helping me add them correctly and at the right level. I would also like to deploy my OpenAPI definition directly to Kusk Gateway from there - or run it through Kusk-gen.
The extension support could work as follows: when opening an OpenAPI definition

a "Kusk" panel is made available that shows an outline of all my extensions
actions to add different features are available - i.e. "Add Rate-Limiting", "Add CORS settings", etc - which would prompt and add the corresponding extensions accordingly
actions to generate gateway configurations supported by Kusk-gen - i.e. "Generate configuration for" -> "Traefik", "Ambassador", etc.

Maybe we can go for something like the json-schema $schema keyword: https://json-schema.org/draft/2020-12/json-schema-core.html#rfc.section.8.1.1
I've seen used by krakend; you put it at the top of the file, and any editor will automatically validate the json.
Wonder if it's possible to investigate YAML-schema https://asdf-standard.readthedocs.io/en/latest/schemas/yaml_schema.html


iOS 11 下会因为 wasm 导致 crash

JSC 的相关 bug：
iOS 11.2.2 and 11.2.5b6 WebAssembly regression
可以考虑用 https://github.com/dcposch/blakejs 替换  blake2b-wasm

This PR prevents blake2b-wasm from being loaded when it's iOS 11, please have a review.

ckb-sdk-js v0.36.1

we use the ckb-sdk-js in react-native, but theres is no navigator in react-native, so the iOSVersion validator is incorrect.
https://github.com/nervosnetwork/ckb-sdk-js/blob/4c7055466aeb2b8cd2fda2ae927e88cb699d161b/packages/ckb-sdk-utils/src/crypto/blake2b.ts#L377-L381

I'd like to know if the ready method is used in your project since AFAIK it's only used in an ancient project.
The crash was introduced by importing blake2b-wasm at the top of the module, so it will be loaded no matter if it is used.
This commit prevents blake2b-wasm from being loaded automatically until the ready method is called. So if ready is not used, blake2b-wasm has no effects.

We only used the following methods, so shouldn't be calling ready.
import {
  parseAddress,
  rawTransactionToHash,
  JSBI,
  AddressType,
  bytesToHex,
} from '@nervosnetwork/ckb-sdk-utils'

Let me test it.

We only used the following methods, so shouldn't be calling ready.
import {
  parseAddress,
  rawTransactionToHash,
  JSBI,
  AddressType,
  bytesToHex,
} from '@nervosnetwork/ckb-sdk-utils'

Let me test it.

🤗 Please let me know if it works on real devices. I'll release a patch of removing blake2b-wasm today if it still crashes.

Nice, but my test phone isn't around, so I'll have to test it tomorrow morning.

Nice, but my test phone isn't around, so I'll have to test it tomorrow morning.

That's fine, waiting for your message.

This issue has been fixed, thanks.


chore(main): release 0.2.6

:robot: I have created a release beep boop
0.2.6 (2024-11-28)
Dependency Updates

deps: update build digest to d0ef938 (#271) (d0b6ed4)
deps: update module github.com/pterm/pterm to v0.12.80 (#272) (0bd4732)
upgrade to latest AWS and Azure plugins (#274) (93722aa)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

v0.2.6
:sunflower:


I'm not able to find the source code.

Dear @WongKarQin , I found this repo reading the paper, but I'm not able to find the algorithm source code. Could I ask you it?
Thank you in advancae.
Antonio.

Dear @WongKarQin, I forgot in the previous message that I will include your algorithm, hence your research in a my research in order to evaluate the performance of a my deep learning classifier with and without your feature selection procedure.


Reduce UART Instance trait to the bare minimum

A bit of an experimental change, that allows (though doesn't fully implements) erasing the UART types completely in internal code.
At this point we can make UART's Instance completely public, allowing third parties to write their own drivers without relying on hidden code. Because there is no public, but hidden code :)
The driver itself is a bit of a mess, still, and I guess I'll need to write at least a changelog entry.

For context, Info and State came from embassy. My choice would have been Peripheral but we already have two of those, which is already not ideal.

I like this approach a lot, will we be able to apply it all the other drivers? We don't have to do it soon, but for internal consistency, we should do it and some point if we can.

I actually intend to move this way pretty quickly. The difficult part is GPIO, where every register field has a register-specific prefix. I also haven't looked into timers yet. But communication peripherals should be simple, maybe disregarding some of the capability differences in SPI that we currently don't care about.


Fix typo

Fixes typo in transforms documentation.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hi @Charlyo！First of all, we want to express our gratitude for your significant PR in the MMClassification project. Your contribution is highly appreciated, and we are grateful for your efforts in helping improve this open-source project during your personal time. We believe that many developers will benefit from your PR.
We would also like to invite you to join our Special Interest Group (SIG) private channel on Discord, where you can share your experiences, ideas, and build connections with like-minded peers. To join the SIG channel, simply message moderator— OpenMMLab on Discord or briefly share your open-source contributions in the #introductions channel and we will assist you. Look forward to seeing you there! Join us ：https://discord.gg/raweFPmdzG
If you are Chinese or have WeChat，welcome to join our community on WeChat. You can add our assistant ：openmmlabwx. Please add "mmsig + Github ID" as a remark when adding friends：）
Thank you again for your contribution❤ @Charlyo


mappings for container runtime added

Pull request checklist
Please check if your PR fulfills the following requirements:

[ ] Docs have been reviewed and added / updated if needed (for bug fixes / features)
[ ] Tests have been added that prove the fix is effective or that the feature works.
[ ] New and existing tests pass locally with the changes.
[ ] Code follows the style guidelines of this project (PEP8, clean code, linter).
[ ] A self-review of the code has been done.

What is the ticket or issue number?

Pull Request Description

Does this introduce a breaking change?

[ ] Yes
[ ] No

How Has This Been Tested?

yes the alertsv7_v6_compatibility tests have this covered as far as I can tell now that the mapped fields have been commented out


Allow Attachments to be created with unsupported mime type

Ah, just saw your suggestion! Sorry, happy to clean it up later.

Ah, just saw your suggestion! Sorry, happy to clean it up later.

@louispotok Merged it already and cleaned up :)
Thank you for the PR and issues! Very appreciated.

@louispotok Merged it already and cleaned up :)
Thank you for the PR and issues! Very appreciated.


added superspreader cut

added a cut based on galaxy size measurements. This removes T*Terr > 1 and T/Terr < 10 objects, based on the similar cut in gold catalog.

OR "5". whatever is next

Oh yeah I forgot about that. Will work on it.


Create "non technical" information

Currently the site quickly references our docs and github, which is nice for technical folks searching for the right information. However if we have users that are non-technical, managers, analyst, etc there is little "what is Lightning" for them to discover.
 Lightning allows you to create beautiful interfaces using JavaScript. You can create incredible multiplatform experiences using the rich feature set of the framework. Built for the smallest of device to run on the biggest screens.

@lightning-js/maintainers ^^ review plx


Commands in quick start fail due to non-existence of S3 bucket

In the quick start section of the README, the command:
curl https://s3.amazonaws.com/fairseq/models/wmt14.en-fr.fconv-float.tar.bz2 | tar xvjf -
fails because the S3 bucket "fairseq" bucket could not be found.
If more detail is required, please let me know here, or (if you're a FB employee) ping me on internal chat.

Same problem.

try wget instead of curl


Training reproduce

Dear Authors,
Thanks for the great work again.
I have a quick question,
I try to do training with 4 epochs by setting the trainer epoch to 1 and using for to repeat it four times.
I can't get the same result with this,
Any hint for what I did wrong?

Thanks

Hi,
Did you resume from previous epoch checkpoints? If so, please ensure every epoch is training from scratch. If not, you can set a random seed to strengthen the result reproducibility. Could you report the results you got? I'd like to know how the results vary. Thanks!

Dear,
Thanks, it's my fault, I set the batch size too big(512),
I fixed it with batch size 16, now it works perfectly.
Best


Adding xcode release v0.5-rc0 to  Installation.md

I tried this osx package just now and unfortunately lldb python package is configured with python 2.7. Which is a shame because jupyter requires it to be importable to python 3.

Also importing lldb in python2.7 gives the following error
Sinas-MBP:~ sina$ export PYTHONPATH=`lldb -P`; python -c "import lldb"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Developer/Toolchains/swift-tensorflow-RELEASE-0.5.xctoolchain/System/Library/PrivateFrameworks/LLDB.framework/Resources/Python/lldb/__init__.py", line 1510, in <module>
    class SBAddress(object):
  File "/Library/Developer/Toolchains/swift-tensorflow-RELEASE-0.5.xctoolchain/System/Library/PrivateFrameworks/LLDB.framework/Resources/Python/lldb/__init__.py", line 1681, in SBAddress
    __swig_getmethods__["module"] = GetModule
NameError: name '__swig_getmethods__' is not defined

I tried this osx package just now and unfortunately lldb python package is configured with python 2.7. Which is a shame because jupyter requires it to be importable to python 3.

This is unfortunately true.
For a time, macOS development toolchains were built with Python 3. But this was not robust because Python 3 was an ad-hoc dependency (installed via Homebrew on toolchain-building machines) and caused issues for macOS users that didn't install the exact version of Python 3.
Now, macOS development toolchains are built with Python 2, which comes installed with macOS and should work for all macOS users. This means swift-jupyter won't work with macOS toolchains, but you can try using the Docker installation instead.

Actually the last development toolchain (August 28, 2019) was built for python3 and for me it worked very nice with jupyter. I just had to relink _lldb.so to my python3 installation.

Also importing lldb in python2.7 gives the following error
Sinas-MBP:~ sina$ export PYTHONPATH=`lldb -P`; python -c "import lldb"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Developer/Toolchains/swift-tensorflow-RELEASE-0.5.xctoolchain/System/Library/PrivateFrameworks/LLDB.framework/Resources/Python/lldb/__init__.py", line 1510, in <module>
    class SBAddress(object):
  File "/Library/Developer/Toolchains/swift-tensorflow-RELEASE-0.5.xctoolchain/System/Library/PrivateFrameworks/LLDB.framework/Resources/Python/lldb/__init__.py", line 1681, in SBAddress
    __swig_getmethods__["module"] = GetModule
NameError: name '__swig_getmethods__' is not defined

This is an orthogonal error, likely due to Swig version 4.0.0 on the toolchain-building machine. Swig 4.0.0 is known to cause this issue.
$ head -n2 /Library/Developer/Toolchains/swift-tensorflow-RELEASE-0.5.xctoolchain/System/Library/PrivateFrameworks/LLDB.framework/Resources/Python/lldb/__init__.py
# This file was automatically generated by SWIG (http://www.swig.org).
# Version 4.0.0

I believe downgrading to Swig 3 may resolve the issue (reference).
We will verify this and fix in a new v0.5 RC toolchain.

Since the default installed version for MacOS is 2.7 right now, it was a temporary mistake to link in 3.x into the dev versions.  In the future, we might move to supporting 3 once we can communicate to everyone to install that version for MacOS development.

Thanks for the explanation.
I tried to build the whole swift for TensorFlow toolchain myself. It was built successfully. But the completion functionality did not work. The reason was that lldb API did not have CompleteCode attribute see (TF-743). I looked into the swift-lldb repository. It is added to swig interface file SBTarget.i but it is not exposed in LLDBWrapPython.cpp. Is there a way to fix this?

Which branch did you build the toolchain from? The tensorflow-0.5 branch should have fixes for this.  @marcrasi Do you know more?

Actually, https://github.com/apple/swift-lldb/pull/1961 should be in the tensorflow branch as well.

I tried tensorflow branch yesterday. @marcrasi did do the correction to swig interface file that I mentioned above

I think I understand the issue with scripts/Python/static-binding/LLDBWrapPython.cpp.
This file is supposed to be generated by SWIG when you do a build. But it's also checked in, so that the build can run on machines without SWIG installed (see description here).
Since we have always been building on machines with SWIG installed, we never noticed that the checked-in generated file had gotten stale.
If my understanding is correct, you should be able to fix your build by installing SWIG before building. (I recommend version 3.0.12 because 4.x seems to cause some problems).
I'll also send a PR updating the checked-in generated files soon.

Excellent! Thank you very much @marcrasi.

PR that regenerates the files: https://github.com/apple/swift-lldb/pull/1989


Allow apps to bypass the VPN

In practice, apps could easily bypass Intra anyway, e.g. by
sending queries to a hardcoded resolver.  Explicitly allowing
bypass fixes #18 (interference with some WebRTC-based apps).

LGTM


New crate release to make fixed version available

Description of the feature
Hi @sunli829, thank you so much for your work. I wanted to ask if you plan a release soon because I depend on some of the metric fixes and would love to change the dependencies back from a git hash to a proper released version. I would also help if you have an idea when you'd be up for it, then I can set myself a reminder.
Thank you!

v1.3.51 has been released


dependencies update

What does this PR do?
dependencies update
Test Plan
No code change, only dependencies updated.
Related PRs and Issues
(If this PR is related to any other PR or resolves any issue or related to any issue link all related PR and issues here.)
Have you read the Contributing Guidelines on issues?
Yes.

@rvndsngwn, thanks for submitting this PR! Unfortunately, this is a read-only repo as the code is generated by our SDK Generator, and changes made only here would be wiped in the next release.


Debian packages stopped on 1.26?

Seems like I cannot find 1.28 of cri-tools on my debian for 1.28 install of kubernetes.
https://build.opensuse.org/package/show/devel:kubic:libcontainers:stable/cri-tools
Will that be an issue for my installation?

You can try those if you want: https://kubernetes.io/blog/2023/08/15/pkgs-k8s-io-introduction/

It's a bigger problem when you try to use the new cri-tools on the older clusters:

https://github.com/kubernetes-sigs/cri-tools/issues/1130

So the packages on Kubic and the packages on Google are not updated, on purpose.

https://kubernetes.io/blog/2023/08/31/legacy-package-repository-deprecation/


local_action running on jump host when connection delegation is enabled?

Hi, and thank you for the awesome project
Trying to get it set up for testing with mitogen_via option, and immediately running into the following when I run my playbook against a small group of hosts:
fatal: [morty2]: UNREACHABLE! => {"changed": false, "msg": "error occurred on host bastion1: Child start failed: [Errno 2] No such file or directory. Command was: \"/webapps/xyzzy/venv/bin/python2.7\" \"-c\" \"import codecs,os,sys;_=codecs.decode;exec(_(_(\\\"eNpd0FFLwzAQB/Bn+yl8u4SF0nRjaKGgFPG9D/ZBRbr0psGsCWm7OD+9Nzo09S2/+1/uSGrRlHZInXbIeOJFiKT314S99Z+MF8kVnbvJ5SwTMsv4n2sRy1MqZytjB2R1DB+jiRFmYH/U3vbPcF8/PmXwWg4nKn6hmsZ2Z3DuIRq2DMSSK2AHPdp37ItBfWA3GfR3rj39Ih3UMW2dM5gqeyhyudnecuAJjQ9ej8ikgIcqe+mpVpVv7PwRnXXY0/PB74CnHtuOSSlvtlzAt3bz5UtTIyAA/ctlWBVnYZnBJpfrNS1aVf/Xy/P6Hzv/gWU=\\\".encode(),\\\"base64\\\"),\\\"zip\\\"))\"", "unreachable": true}
fatal: [morty7]: UNREACHABLE! => {"changed": false, "msg": "error occurred on host bastion1: Child start failed: [Errno 2] No such file or directory. Command was: \"/webapps/xyzzy/venv/bin/python2.7\" \"-c\" \"import codecs,os,sys;_=codecs.decode;exec(_(_(\\\"eNpd0FFLwzAQB/Bn+yl8u4SF0nRjaKGgFPG9D/ZBRbr0psGsCWm7OD+9Nzo09S2/+1/uSGrRlHZInXbIeOJFiKT314S99Z+MF8kVnbvJ5SwTMsv4n2sRy1MqZytjB2R1DB+jiRFmYH/U3vbPcF8/PmXwWg4nKn6hmsZ2Z3DuIRq2DMSSK2AHPdp37ItBfWA3GfR3rj39Ih3UMW2dM5gqeyhyudnecuAJjQ9ej8ikgIcqe+mpVpVv7PwRnXXY0/PB74CnHtuOSSlvtlzAt3bz5UtTIyAA/ctlWBVnYZnBJpfrNS1aVf/Xy/P6Hzv/gWU=\\\".encode(),\\\"base64\\\"),\\\"zip\\\"))\"", "unreachable": true}
fatal: [morty4]: UNREACHABLE! => {"changed": false, "msg": "error occurred on host bastion1: Child start failed: [Errno 2] No such file or directory. Command was: \"/webapps/xyzzy/venv/bin/python2.7\" \"-c\" \"import codecs,os,sys;_=codecs.decode;exec(_(_(\\\"eNpd0FFLwzAQB/Bn+yl8u4SF0nRjaKGgFPG9D/ZBRbr0psGsCWm7OD+9Nzo09S2/+1/uSGrRlHZInXbIeOJFiKT314S99Z+MF8kVnbvJ5SwTMsv4n2sRy1MqZytjB2R1DB+jiRFmYH/U3vbPcF8/PmXwWg4nKn6hmsZ2Z3DuIRq2DMSSK2AHPdp37ItBfWA3GfR3rj39Ih3UMW2dM5gqeyhyudnecuAJjQ9ej8ikgIcqe+mpVpVv7PwRnXXY0/PB74CnHtuOSSlvtlzAt3bz5UtTIyAA/ctlWBVnYZnBJpfrNS1aVf/Xy/P6Hzv/gWU=\\\".encode(),\\\"base64\\\"),\\\"zip\\\"))\"", "unreachable": true}
...

Of course this is because /webapps/xyzzy/venv/bin/python2.7 does not exist on the jump host. However, since it's local_action, shouldn't it still run on localhost?
For reference, the local action is a initial step to ensure that a network-based disk image is mounted before we copy data off it to remote hosts. Looks something like this:
  - name: mount latest source.dmg
    local_action:
      module: shell
      args: |
        if [[ -d /tmp/Project ]]; then
          hdiutil detach /tmp/Project
        fi
        hdiutil attach /Corp/Source/Images/Current/Project.dmg -nobrowse -mountpoint /tmp/Project
    tags:
      - preflight

But I am pretty sure that the content of the action is irrelevant, the issue is that it's trying to execute on jumphost. It would actually even work there, probably, if the interpreter path was correct.
I tried setting ansible_python_interpreter config var for the jumphost, but that did nothing.
Thanks again
Anton

Great catch! That's super broken :) Give me 15 minutes

Hi there,
I've pushed a partial fix to issue251 branch, but it looks like it'll need more work to correctly derive variables in every case. My head is swimming in issues with mainline Ansible this week, so possibly things will just work already (Mitogen local connections should inherit the interpreter used to start Ansible, but not certain of that)

@dw I want to chime in to say that I'm running into a similar problem. I'm using Ansible v2.5.5 to run the kubespray playbook without Mitgoen and running into similar errors (the playbook works without mitogen):
task path: /Users/ayaz/Work/Cloudways/Cluster/kubespray/roles/kubernetes/preinstall/tasks/pre_upgrade.yml:2
Friday 29 June 2018  12:30:21 +0500 (0:00:00.099)       0:00:17.488 ***********
[pid 21418] 12:30:21.535102 D mitogen: unix.connect(path='/var/folders/_0/9b9z3xf527xg1pslfj_64q900000gn/T/mitogen_unix_jAzUOK')
[pid 21418] 12:30:21.536470 D mitogen: unix.connect(): local ID is 2019, remote is 0
[pid 21281] 12:30:21.538047 D mitogen: mitogen.parent.Stream('default').connect()
[pid 21418] 12:30:21.554305 D mitogen: mitogen.core.Stream('unix_listener.21281').on_disconnect()
[pid 21418] 12:30:21.554981 D mitogen: Waker(Broker(0x111175990) rfd=68, wfd=69).on_disconnect()
[pid 21281] 12:30:21.554914 D mitogen: mitogen.core.Stream('unix_client.21418').on_disconnect()
fatal: [av2-k8s-worker]: UNREACHABLE! => {
    "changed": false,
    "msg": "Child start failed: [Errno 2] No such file or directory. Command was: \"/usr/bin/env python\" \"-c\" \"import codecs,os,sys;_=codecs.decode;exec(_(_(\\\"eNptkNFLwzAQxp+3v6Jvl7CwJXVTKRSUPogPIhSxDzokbTMNdklJ29Xtr/faDtaKLyG/++6+77iYJaGtlqUuFaFzx9oR6Z2HsLPum9BgPsN/3pQ+4UxwTi8cszE5VMXAWWErReIxuDEkY2gRMLA6Ynwha0zde2HoQS5dqw140uS9qH5U1tQyLVQvr5rKrVJtVuWx/rIGcM/Zn7ZF2A8elKu0NW/B1baPVeagHTLcxw+vHLbhdGzoQSzIVGBTXADZ69p+KhPIozzddc+ysJksAl/4t4ICnaNT63StiGDw9PjyzDl/N1iPwg/SXTi3pTJ4V3Ap0KVTMidCXN9sKIOTLgeDc1PCoAU8+NkwGmvtVIP1euP7GLSI/ltBdCv8An8xnww=\\\".encode(),\\\"base64\\\"),\\\"zip\\\"))\"",
    "unreachable": true
}```

I came here via google, I have a similar problem with 0.2.2:
error occurred on host 55a78418c587: Child start failed: [Errno 2] No such file or directory. Command was: sudo -u root -H /usr/bin/python2 -c "import codecs,os,sys;_=codecs.decode;exec(_(_(\"eNqFkEtrwzAQhM/xr/BtJSISOQ/iGgQtOZQeSsGU+tCG4ofcijqSkJ246a/vxgnETg+97cfO7gwTs0SYemKVlYR6jrU9UqWPUBr3RWjkjXAudnZGOAs4pxeOWZ8cboMT55WpJYn74PqQ9KFFQMP6gPZV2qDr1hfChyJ1rdLgp7rolvJb5rsmzSrZrae72k0zpaf20HwaDZhzdCUbi+5wL12tjH6N5pvOVuq9cshwF9+/cNiI4dlJg1iR4YINcQxkqxrzIXXkjGlul8t0FS6CMF+Gqyi84RSoh59apxpJAgaPD89PnPM3DZggNwUWTb21eCfHqgtjpcaCwWVAJ06mBQlmizmnDH6UxU+lFRddwqDN4Nh+ac8G624+NXqlbv9T/00ZDFL+AoQMrM4=\".encode(),\"base64\"),\"zip\"))"

This happens when my playbook runs a local_task with become: true but already runs under root (in a container)


feat(retype): Add tagged unions

Current dependencies on/for this PR:

main

PR #3   (this PR)

This comment was auto-generated by Graphite.


Introduce Argon2 password hashing in crypto

And also introduce generic hashed-password encoding.

We would need to save the Argon2 parameters + salt to the database so that we can recreate the hashed_password  that would be used as a chacha20poly1305::Key encryption key.

Is it correct to say that there are 2 use cases for this?

RPC session authentication
Wallet private key encryption

Is it correct to say that there are 2 use cases for this?

RPC session authentication
Wallet private key encryption

Yes. At least that's my plan. I'm currently working on restructuring this to make it work for the wallet too.


When cookie store is enabled write lock is taken even if the response contained no cookies

Unless I miss something that is a bottleneck for parallelization as every request needs to go through that write lock:
   #[cfg(feature = "cookies")]
            {
                if let Some(store_wrapper) = self.client.cookie_store.as_ref() {
                    let mut store = store_wrapper.write().unwrap();
                    let cookies = cookie::extract_response_cookies(&res.headers())
                        .filter_map(|res| res.ok())
                        .map(|cookie| cookie.into_inner().into_owned());
                    store.0.store_response_cookies(cookies, &self.url);
                }
            }

I think it would be better to check that the response actually contains set cookies before taking the write lock.
It’s also good practice to hold write locks as short as possible and here it’s acquired even before it’s used. (Extraction of cookies is not dependent on holding the lock only storing them is).

True, the write lock could be taken after extraction. Would you like to submit a fix?

Sure - I can try to fix that tomorrow. (I am still new to rust)


[FR] add component reference type support

Hi.
No slug is generated for components.
I have analyzed  the 'setting-service.js' file. I developed a possible fix that is working for me (see below).
With that fix, to produce the slug for the fields in a component, you have to set the plugin settings file field contentTypes using the value of the collectionName field used in the component configuration file.
Strapi: 4.11.4
Strapi Plugin Slugify: 2.3.3
MySQL: 8.0.30 (MySQL Community Server - GPL)
Here is a possible fix to generate a slug for components by changing the build method:

build(settings) {

	const buildModel = (contentType, uid, nameAttr,displayName) => {
		const model = settings.contentTypes[contentType[nameAttr]];
		if (!model) {
			return;
		}

		// ensure provided fields are present on the model
		const hasField = isValidModelField(contentType, model.field);
		if (!hasField) {
			strapi.log.warn(
				`[slugify] skipping ${contentType.info[displayName]} registration, invalid field provided.`
			);
			return;
		}

		let references = _.isArray(model.references) ? model.references : [model.references];
		const hasReferences = references.every((referenceField) =>
			isValidModelField(contentType, referenceField)
		);

		if (!hasReferences) {
			strapi.log.warn(
				`[slugify] skipping ${contentType.info[displayName]} registration, invalid reference field provided.`
			);
			return;
		}

		const data = {
			uid,
			...model,
			contentType,
			references,
		};
		settings.modelsByUID[uid] = data;
		settings.modelsByName[contentType[nameAttr]] = data;
	};

	// build models
	settings.modelsByUID = {};
	settings.modelsByName = {};

	_.each(strapi.contentTypes, (contentType, uid) => buildModel(contentType, uid, 'modelName', 'singularName'));
	_.each(strapi.components, (contentType, uid) => buildModel(contentType, uid, 'collectionName', 'displayName'));

	_.omit(settings, ['contentTypes']);

	return settings;
}

Thanks for the request and suggestion. For now this plugin only supports fields define don the model directly.
I will at some point add relational and component support. At the moment their is no eta for that.

Also requesting support for object reference. Example: We have categories like cat cat2 and we would like to be able to auto generate cat/article cat2/article2


adding a new test file for test_GWASBigBedPIP.py

@hanyelgaml-roche
There's a lot of duplicate commits on this including the commits from the PR I already merged.
I suggest pull in the new changes from the upstream (epiviz/efs_parser) repo, make a branch, commit the new test file and recreate this PR

@hanyelgaml-roche
There's a lot of duplicate commits on this including the commits from the PR I already merged.
I suggest pull in the new changes from the upstream (epiviz/efs_parser) repo, make a branch, commit the new test file and recreate this PR


Enable Proposed API codeActionAI

The Typescript team has added Copilot suggestions to various refactors. To bring similar functionality to python code, Pylance has enabled the implement inherited abstract classes with Copilot code action through the Github Copilot extension for Python files.
To display the sparkle icon for the code action instead of the regular light bulb icon, the isAI property needs to be set to true on the codeAction.

Vscode-python and LSP doesn't support this API yet. Would it be possible for vscode-python to enable the codeActionAI proposed API?
Is it as simple as adding "codeActionAI" to the "enabledApiProposals" in package.json? I'd be happy to send a PR if that's all it takes. Thanks!

This is not so straight forward. We will need to pin engine version, after it is enabled.

Since this is proposed API, stable can't use it directly. I have enabled it for python extension:

You will need to pin the engine to latest VS Code. engine format: 1.94.0-20240913 (assuming you want to pin to todays build of VS Code).

What I mean by my above comment is you can now submit PRs to enable proposed APIs for codeActionAI and it should work on stable.

Thanks Karthik! When you say "enabled for the Python extension", you mean that the proposed API has been enabled for use by the Python extension in the STABLE version of VSCode right? Luciana mentioned a vscode-distro repo to me, but I don't have access to it, so I'm unclear if that’s where the change was made and what its purpose is.
A follow-up question: If my interpretation above is correct, would the isAI change in LSP potentially cause issues or crashes for other vscode-languageserver-node consumers that do not support this proposed API in stable?
Last question:) We usually update the engine when the LSP requirements change. What’s the worst-case scenario if we don’t update the engine? Would it be that users with older versions of VSCode see the regular light bulb icon, while those with the latest version see the sparkle icon?

Thanks Karthik! When you say "enabled for the Python extension", you mean that the proposed API has been enabled for use by the Python extension in the STABLE version of VSCode right? Luciana mentioned a vscode-distro repo to me, but I don't have access to it, so I'm unclear if that’s where the change was made and what its purpose is.

That is correct. I updated the distro.

A follow-up question: If my interpretation above is correct, would the isAI change in LSP potentially cause issues or crashes for other vscode-languageserver-node consumers that do not support this proposed API in stable?

This only affects at the extension level, not at the package level. So, it should be fine as long as the extension where the client or the server is created is opted in.

We usually update the engine when the LSP requirements change. What’s the worst-case scenario if we don’t update the engine? Would it be that users with older versions of VSCode see the regular light bulb icon, while those with the latest version see the sparkle icon?

Worst-case scenario (other than crash) is what you mentioned. They won't get the new behavior.

@StellaHuang95 Can you verify that this is now working as expected for you?

I am waiting for the change in LSP to go through, I can verify and let you know if it doesn't work once LSP supports it.


fix typo in state machine type

While working on the custom cryptography PR I ran across this small nit (easier to review this way)

Hah, only now realise that Kex is shorthand for KeyExchange and isn't a typo 😆


Documentations for installation

Could you please provide us steps of installation?

Was the following documentation not sufficient?
https://github.com/Ar-Ray-code/YOLOX-ROS/blob/humble/yolox_ros_cpp/README.md
The install method depends on your computing environment and the launch file you want to exec, so unless you tell me, I cannot support it ☹


Update trigger-kinesis to 0.2 version

Signed-off-by: Darwin Boersma<EMAIL_ADDRESS>

Good callout, will update with that shortly and keep it in mind for future releases!


Add cafe-pdf

Render PDF files on Vue pages using pdf.js, supporting text rendering, lazy loading, paging, scaling, and printing. Demo
General

✏ Mark the necessary items without changing the structure of the PR template.

[x] Pull request template structure not broken

Type

ℹ  What types of changes does your code introduce?

👉 Put an x in the boxes that apply

[ ] Fix
[x] Feature

Checklist

ℹ  Check all checkboxes - this will indicate that you have done everything in accordance with the rules in CONTRIBUTING.

👉  Put an x in the boxes that apply.

[x] Title as described
[x] Make sure you put things in the right category!
[x] Always add your items to the end of a list

Open Source

[x] Link description does not contain a link to an author / third-party resource
[x] The documentation (README) contains a description of the project, illustration of the project with a demo or screenshots and a CONTRIBUTING section
[x] The documentation is in English.
[x] The project is active and maintained.
[x] The project accepts contributions.
[x] Not a commercial product

Apps/Websites

[x] The website is available without errors or ssl certificate problems, and load in a reasonable amount of time.
[x] The website is using vuejs intensively. It should detect vue with vue-devtools.

If you cannot detect vue with vue-devtools due to work at non public pages (e.g. for enterprise website), you can send Pull Request with screenshot that detected it.

[x] The website is original and not too simple. For that reason, blogs and simple landing pages are rejected.
[x] A commercial product using Vue, provided that guests could reasonably check out how Vue was used (i.e. A headless CMS which uses Vue for the Admin/editor Area and offers a free tier).

https://github.com/wangdaodao/cafe-pdf Results in 404 today. Was it renamed?

Removed from the list because 404


Isolate Shoot namespace network traffic.

What this PR does / why we need it:
Disables, by default ALL Egress and Ingress traffic from/to control-plane namespace in the Seed cluster.
Old network policies are now removed.
New policies have been added:
NAME                        POD-SELECTOR
allow-elasticsearch         app=elasticsearch-logging,garden.sapcloud.io/role=logging,role=logging
allow-etcd                  app=etcd-statefulset,garden.sapcloud.io/role=controlplane
allow-from-prometheus       networking.gardener.cloud/from-prometheus=allowed
allow-grafana               component=grafana,garden.sapcloud.io/role=monitoring
allow-kibana                app=kibana-logging,garden.sapcloud.io/role=logging,role=logging
allow-kube-apiserver        app=kubernetes,garden.sapcloud.io/role=controlplane,role=apiserver
allow-prometheus            app=prometheus,garden.sapcloud.io/role=monitoring,role=monitoring
allow-to-dns                networking.gardener.cloud/to-dns=allowed
allow-to-elasticsearch      networking.gardener.cloud/to-elasticsearch=allowed
allow-to-metadata           networking.gardener.cloud/to-metadata=allowed
allow-to-private-networks   networking.gardener.cloud/to-private-networks=allowed
allow-to-public-networks    networking.gardener.cloud/to-public-networks=allowed
allow-to-seed-apiserver     networking.gardener.cloud/to-seed-apiserver=allowed
allow-to-shoot-apiserver    networking.gardener.cloud/to-shoot-apiserver=allowed
allow-to-shoot-networks     networking.gardener.cloud/to-shoot-networks=allowed
deny-all                    {}

Enabling Egress and Ingress traffic NetworkPolicies must be provided by every single component.
Helpful networking.gardener.cloud/to- labels can be used by control plane components to enable Egress traffic to various other components or endpoints.
Which issue(s) this PR fixes:
Fixes #873
Special notes for your reviewer:
n/a
Release note:

All components part of a shoot control plane running in Seed clusters are now completely isolated from each other. By default, all `Egress` and `Ingress` traffic is disabled.

Additional `NetworkPolicies` must be created, allowing `Egress` or/and `Ingress` traffic if custom components are deployed next to the Shoot control plane in the Seed cluster.
Use the new `networking.gardener.cloud/to-` labels on these control plane components to enable `Egress` traffic to various other components or endpoints.

It's recommended to install a `NetworkPolicy` controller (such as [Calico](https://docs.projectcalico.org/v3.6/getting-started/kubernetes/installation/other)) to all Seed clusters, so that `NetworkPolicies` are enforced.

/cc @marwinski @jia-jerry

Why are we blocking by default all egress traffic (unless annotated)? That will complicate ops and since the hyperkube images are broken (deliberately), I now don't have curl, ps, etc. and can't download them either, if I don't create a network policy that I may anyway forget to delete afterwards. A clean solution would involve TTLs on network policies and the controller removing them again.
Are we scared of miner or spammer code injected into our seed clusters or exploits that allow an attacker to misuse our components as "relays"?
Either way, I really like to remind everybody, that we must continue to operate the landscapes. If we want to block egress, please prepare ops guide hacks or gardenctl commands to simplify this again, opening up a control plane/all pods (on a 12h TTL for instance), so that we can do ops if necessary.

Why are we blocking by default all egress traffic (unless annotated)?

Better security - it's preferred to block all and then whitelist as you leave no vector out.

That will complicate ops and since the hyperkube images are broken (deliberately), I now don't have curl, ps, etc. and can't download them either

If a component doesn't require curl or wget to function, then why need it in the container? Less components less CVEs and less attack surfaces.
Also in next PR I'll planning on changing user in those hyperkube images to nobody, making it impossible to download anything

Are we scared of miner or spammer code injected into our seed clusters or exploits that allow an attacker to misuse our components as "relays"?

There were plenty of CVEs in kube-apiserver, so keeping little attack surface might be sufficient.

You have not replied with any constructive comment @mvladev. If you seal everything, then how should we do ops? A solution that prevents me to analyse problems is useless and will eventually lead to our demise, because we will be handicapped to resolve and even analyse problems. Tell me how the community should debug anything or even our ops staff if we break everything for them? You must find a balance, but instead you go full-in and ignore the other necessities.

There is work in progress for this behavior https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/20190212-ephemeral-containers.md
A workaround until "debug" functionality comes to K8S is to run a container on the same machine and in the same network/pid namespace as the Pod/container you want to debug.
This "kubectl debug" functionality should also be added to our tooling / docs
On Apr 16, 2019 07:03, Vedran Lerenc<EMAIL_ADDRESS>wrote:
You have not replied with any constructive comment @mvladevhttps://github.com/mvladev. If you seal everything, then how should we do ops? A solution that prevents me to analyse problems is useless and will eventually lead to our demise, because we will be handicapped to resolve and even analyse problems. Tell me how the community should debug anything or even our ops staff if we break everything for them? You must find a balance, but instead you go full-in and ignore the other necessities.

Not helpful @mvladev. Yes, this got my hopes up some time ago and yes, I can deploy a new pod, but I then don't have the configuration of the pod under analysis, e.g. the API server. Until I found out how to install the stuff on the real API server (because the patching of hyperkube left the packages in an inconsistent state), I created a doppelgaenger. It was tedious to do that, so I found handicapped and everything took me much longer.
In other words, before we now totally break ops, let's first have alternatives in place. Isolating the components from each other was the original goal. If I do ops, I can live with that, because I anyways will only try to follow the established/expected paths, e.g. API server to ETCD, or controller manager to API server. But breaking the containers for ops/sealing them off from egress was not the original goal. If we do that, we need to have an ops concept in place and the comments above don't help.

Also this impossible to debug behavior exists in for example in Grafana https://github.com/grafana/grafana/blob/d23f50ab234b9c4b1b5bc6bff9899db4d80eafa0/Dockerfile#L85 as it runs unprivileged user and debian:stretch-slim container.
The "way" it's debugged today is to disable the cluster reconcilation and edit the Grafana deployment so it runs as privileged user. By doing so curl/wget and other tooling to be installed.

I have never debugged Grafana, so if that's the way you have chosen to do ops, then OK. But before we go on and make ops for critical control plane components even more difficult, let's have better alternatives in place. Changing the deployment is not helpful - it will reschedule the pod. Doing that for ETCD or API server creates even a downtime. That is not OK, I believe.

I need to side with @mvladev. We have compromised on locking down components for too long (consequences not to be discussed here).
As for debugging I do see the need and propose the following:

we will stop deleting some artefacts so curl & co will work again
we can be default leave a channel to the Debian repository open (which allows for installation of additional debugging tools). In addition we can open more channels as needed.
For debugging purposes I can imagine adding functionality to lift the egress ban

Colleagues, in general I do not disagree with you, but that's no help to the guys doing ops. If it's broken here, we need something else. That's all I am saying. Do not merge this (full extent) without a clear solution to the ops question, please. This harms Gardener in general and us in particular.
Again, as written above, I criticise not everything, but the public egress lock-down in particular. You can isolate the control plane components from each other, but not break ops, please.

@vlerenc btw etcd and kube-apiserver are not going to be limited by egress, because they would need to talk to WebhookConfigurations (apiserver) and objectstorage (etcd)
https://github.com/gardener/gardener/blob/0c19d39162998fd34abb8e05ff1c791f83327326/charts/seed-controlplane/charts/kube-apiserver/templates/kube-apiserver.yaml#L31-L34
https://github.com/gardener/gardener/blob/0c19d39162998fd34abb8e05ff1c791f83327326/charts/seed-controlplane/charts/etcd/templates/etcd-statefulset.yaml#L50-L52

A possible solution could come from the gardenctl. If it is used to open a shell on a container, then it could label the pod with the proper labels allowing the egress traffic. After the session is closed, it has to remove the labels. A corner case is if the GCM reconcile the shoot, it will remove the labels, so the gardenctl should be watching for pod updates and re-label again.

In any case, please support also kubectl directly. I.e. if there is a nice gardenctl ops command that prepares "stuff", that can be called. But once you are heavily engaged in a cluster (have "targetted" it), you quickly start to fire kubectl commands. Would be nice if the "preparations" to do this are minimal/simplified one way or another (as suggested, gardenctl), but I can then also use kubectl directly.

What's the status here?

Need to finish e2e testing on AliCloud and I'll update the PR.

There is a dependency problem with terraformer https://github.com/gardener/gardener/blob/0c19d39162998fd34abb8e05ff1c791f83327326/pkg/operation/terraformer/terraformer.go#L309-L312
Even if those labels are added, extensions like Azure are going to need be updated to use this code.
@rfranzke I'll fire separate PR for this terraformer and then update dependencies in https://github.com/gardener/gardener-extensions

#960 is merged, PTAL

Updated to latest changes (more or less revert #960)

Rebased to latest changes.

fixed naming in c0ac217

@mvladev Does this PR effect the extension controllers running in extensions-<...> namespaces? Can they still freely interact with the seed cluster and all the components (DNS, kube-apiserver of shoots, etc)?

It's only for shoot-- namespaces.

OK, thanks. And no component inside the shoot-- namespaces does block ingress from other namespaces except those that also are shoot namespaces, right? If that is true then the extension are not effected which would be great.

Also, could you please rebase again?

I've rebased to the latest changes, but Machine controller manager and cert-borker must be updated as a prerequisite.

@vlerenc This change will result in a re-deployment of all control plane components (because the pods get new labels that are needed for the network policies). Postpone/wait/proceed?

Is there anything in the pipeline that will improve the behaviour? I believe yes, but not really:

MCM improvement, serialising the pod-with-volume eviction, only helps with OS updates, so not here.
Dependency controller improvement, also acting on API server readiness, won't bring benefits as also the ETCD pods that are already covered get updated and are therefore triggered
The planned optimisations in seed and shoot reconciliation are the most important improvements, but they are also a month out and I guess we don't want this PR to hang in the air for much longer

So, if it's generally safe and complete, let's merge it and roll it out very carefully with minimal workers (same procedure as the last time when we updated the ETCDs, maybe even less workers, because now all pods are rolled and there are the monitoring and logging pods as well). Let's watch carefully the IaaS API traffic together with @mliepold and the control plane health via the telemetry controller and @kistner in our development/staging/integration landscapes.
Not sure whether it's worth waiting on the DNS controller cache @rfranzke, cc @mandelsoft @MartinWeindel?
Any other thoughts on this?

Not sure whether it's worth waiting on the DNS controller cache @rfranzke, cc @mandelsoft @MartinWeindel?

Why don't we simply use https://github.com/kubernetes/kubernetes/tree/master/cluster/addons ?/dns/nodelocaldns

Also, PacketBotanist implementation for MetadataServiceAddress is missing

OpenStack cloud-controller-manager is failing: F0606 11:31:24.449139       1 controllermanager.go:119] Cloud provider could not be initialized: could not init cloud provider "openstack": Post https://.../v3/auth/tokens: dial tcp <IP_ADDRESS>:5000: i/o timeout

Updated the PR with the latest changes - I think that https://github.com/gardener/gardener-extensions/pull/106 needs to come in first?


KeyError: 'convertible_bonds' when request user info

Affected tksbrokerapi v1.5.120.
% tksbrokerapi -u -v 10

...

TKSBrokerAPI.py     L:5328 DEBUG   [2023-01-22 01:26:29,883] Traceback (most recent call last):
  File "/Users/tim/projects/TKSBrokerAPI/tksbrokerapi/TKSBrokerAPI.py", line 5309, in Main
    trader.OverviewUserInfo(show=True)
  File "/Users/tim/projects/TKSBrokerAPI/tksbrokerapi/TKSBrokerAPI.py", line 4686, in OverviewUserInfo
    "allowed": [TKS_QUALIFIED_TYPES[item] for item in rawUserInfo["qualifiedForWorkWith"]],
  File "/Users/tim/projects/TKSBrokerAPI/tksbrokerapi/TKSBrokerAPI.py", line 4686, in <listcomp>
    "allowed": [TKS_QUALIFIED_TYPES[item] for item in rawUserInfo["qualifiedForWorkWith"]],
KeyError: 'convertible_bonds'

Some values of qualified_for_work_with, field was added in REST API, but not documented: https://tinkoff.github.io/investAPI/faq_users/#qualified_for_work_with
New non-documented fields: foreign_bonds_russian_law, convertible_bonds, russian_bonds_foreign_law, non_quoted_instruments and option.
Fixed since TKSBrokerAPI v1.6.***


integ-cli: skip tests launching registry-v2

Some pull/push tests are launching a local registry-v2
binary which is compiled and installed if the tests
are running in-container through make test-integration-cli.
However, registry is not supported to run on non-linux
platforms and we can't really spin up any registry-v2
containers in the remote DOCKER_TEST_HOST at this point.
Just skipping those with the new TestRequirement called
RegistryHosting. If the registry-v2 image becomes available
we can take a path like #10893 and spin up registry containers
at the remote daemon hosts and re-enable tests.
Signed-off-by: Ahmet Alp Balkan<EMAIL_ADDRESS>cc: @tianon @unclejack @jfrazelle @icecrime @LK4D4

+1 -- let's revisit when we have a registry-v2 image IMO

@tianon does it make sense to skip at this point or not?

Yeah, I agree -- let's skip for now and revisit when we have a good official registry-v2 image. :+1:

LGTM

LGTM


SQS not working.

Hello, I'm trying to run SQS but nothing works. I keep getting an error when I try to let Job go. Of course, I created an IAM to which I added full SQS access. I created worker.php and added it to the AWS_SESSION_TOKEN configuration file.

What is the error message that you're getting? Have you looked into the cloudwatch logs? Can you also share the IAM role that you're using?

Now I noticed in the logs that I have this error: Facade \ Ignition \ Exceptions \ ViewException: Undefined variable: count

I removed "count" variable and now I have that error "Service Unavailable"

Worker logs:


How can I not specify httppost in each method?

How can I not specify httppost in each method?
I have written 100 methods, each of which requires a name. In fact, I use the method name. This is a lot of repetitive work.

Without declaring http method in the method, what http method are you expecting the client side to call when sending network request?

This is how ASP.NET Core works, it's nothing to do with Swashbuckle.


timetable: init at 1.0.6

Motivation for this change
Add more curated apps from Elementary OS app store for Pantheon Desktop.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Took the liberty to clean up the install script https://github.com/lainsce/timetable/pull/45

Thanks everyone :sparkles: