[WIP] Create and end-to-end test of the Java lang parser and BFG.

Changes needed to make this pass:

Returning Buildozer command fragments for certain targets in the language-specific parsers that cannot be deduced by the BFG. E.g.: a main class for a java_binary.
Adding graph nodes for all targets found to ensure that BUILD files are generated for disconnected targets.
Generalizing the heuristics for merging different rule kinds within the same strongly connected component.

TODOs

Should the e2e script be an sh_test or implemented natively in Skylark? How will we want it to work if we expand this to tests that use Maven, Gradle, Sbt, etc.?
Adding e2e tests that use external dependencies.

Whoa, this is awesome!
I feel it'll be easier to review if the "changes needed" were separate, though; I believe they're largely independent?
Seems like they fix genuine bugs and missing features.

I feel it'll be easier to review if the "changes needed" were separate, though; I believe they're largely independent?
@cgrushko That's a good idea! I started with the Bash script and the PR expanded under everything went green. I'll chop this up into smaller parts that will be easier to review.

Closing this PR, as I will split it up into new smaller PRs.

Sounds good :) I have a first nit, I'm afraid - in my opinion RuleKind is ok to use for the new message, no need for TargetInfo. :|

Sounds good :) I have a first nit, I'm afraid - in my opinion RuleKind is ok to use for the new message, no need for TargetInfo. :|

No worries -- nit away! :)
So, use RuleKind as the name of the proto message? E.g.:
message ParserOutput {
...
 map<string, RuleKind> file_to_rule_kind = 3;
}

message RuleKind {
    required string rule_kind = 1;
    optional Strings buildozer_commands = 2;
}
...

Yeah, I think that even if it's technically more than RuleKind, anyone glancing at the code will be able to deduce the intention.
Then again... mm... the java_binary example has a main_class, which is specific to that particular file, and is not a kind in general. So now I'm not so sure anymore.
Do we still need rule_kind if we have the buildozer_commands?

Do we still need rule_kind if we have the buildozer_commands?

That's a good question -- I thought about that too. We could include it in the buildozer_commands and parse out the rule kind when we need it from set kind <rule_kind>.  I included rule_kind as a separate field to make it clear, semantically, that it was required.

Ah, I'd rather not parse Buildozer commands. But why do we need set kind? I thought it'd be in the buildozer_commands so BFG just runs the commands.
Or is it required to merge rule kinds?

Or is it required to merge rule kinds?

Exactly. If we have, say, a java_library, java_binary, and java_image target in the same connected component we need to decide which one it should be.
We also need to decide to error out if we hit a un-mergeable condition. Say, a java_test and a java_binary in the same connected component. Or, worse, a py_binary and a java_library somehow ending up in the same connected component.
This was my attempt to come up with some reasonble cross-language heuristics for how and when we can and can't merge different kinds in a single component.

For the name, I still can't decide :)
For merging, perhaps the parser will specify a graph of kinds,
library -> binary
library -> test
so that BFG can merge library with binary, but not test with binary.
(in which case it'll error out)
Cross-language merging sounds like a big problem, because I had an assumption that parsers will be independent of each other. Does cross-language merging come up a lot?

For merging, perhaps the parser will specify a graph of kinds,
library -> binary
library -> test
so that BFG can merge library with binary, but not test with binary.
(in which case it'll error out)

That's an interesting idea! Having language specific parsers emit the merge rules that they approve seems like a clearer division of responsibility than having the BFG do all of the heuristics.

Cross-language merging sounds like a big problem, because I had an assumption that parsers will be independent of each other. Does cross-language merging come up a lot?

The case I was thinking of was a set of Scala and Java files in the same connected component. Say:
src/main/java/{A,B}.java
src/main/scala/{C,D}.scala

where A.java depends on D.scala and C.scala depends on B.java.
Sbt will compile them all together despite the cyclic dependency between the two directories.
Hopefully these cases will be rare. And I don't think scala_library would even support it now.

Yes it does :) I added support for this a year and a half ago since it was
a blocker for us :)
On Fri, Dec 1, 2017 at 8:50 PM Gregg Donovan<EMAIL_ADDRESS>wrote:

For merging, perhaps the parser will specify a graph of kinds,
library -> binary
library -> test
so that BFG can merge library with binary, but not test with binary.
(in which case it'll error out)
That's an interesting idea! Having language specific parsers emit the
merge rules that they approve seems like a clearer division of
responsibility than having the BFG do all of the heuristics.
Cross-language merging sounds like a big problem, because I had an
assumption that parsers will be independent of each other. Does
cross-language merging come up a lot?
The case I was thinking of was a set of Scala and Java files in the same
connected component. Say:
src/main/java/{A,B}.java
src/main/scala/{C,D}.scala
where A.java depends on D.scala and C.scala depends onB.java.
Sbt will compile them all together despite the cyclic dependency between
the two directories.
Hopefully these cases will be rare. And I don't think scala_library would
even support it now.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/BUILD_file_generator/pull/32#issuecomment-348576862,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF9HA2euQv6URqKJpZDc7a5ETD_UVks5s8EpbgaJpZM4QrtBb
.

@ittaiz Fantastic -- thanks. :)


143 problems is insane

Describe the bug
The entire app must not have been formulated properly from the beginning because there are 143 potential problems detected.
Link to the Bolt URL that caused the error
https://bolt.new/~/sb1-djg3o8
Steps to reproduce
npm run dev
Expected behavior
For the development to run and for my web app to function
Screen Recording / Screenshot

Platform
Browser name = Chrome
Full version = <IP_ADDRESS>
Major version = 129
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36
performance.memory = {
  "totalJSHeapSize": 46981955,
  "usedJSHeapSize": 44273735,
  "jsHeapSizeLimit":<PHONE_NUMBER>
}
Username = nonistack
Chat ID = 9827e4fb41c8

Additional context
No response

Thanks for your patience during this beta period.


Program Card changes- Reassigning External Link as ProgramDetails and existing ProgramDetails as backup.

Description
Please explain the changes you made here.
Release notes
The title of the pull request will be used as the default release notes description. If more detail is needed to communicate to partners the scope of the PR's changes, use this release notes section.
Checklist
General

[ ] Added the correct label: < feature | bug | dependencies | infrastructure | ignore-for-release >
[ ] Created tests which fail without the change (if possible)
[ ] Extended the README / documentation, if necessary

User visible changes

[ ] Wrote browser tests using the validateAccessibility method
[ ] Manually tested at 200% size
[ ] Manually evaluated tab order

New Features

[ ] Add new FeatureFlag env vars to server/conf/application.conf
[ ] Conditioned new functionality on a FeatureFlag
[ ] Wrote browser tests with the feature flag off and on, etc.

Issue(s) this completes
Fixes #<issue_number>; Fixes #<issue_number>...

Codecov Report

Merging #4120 (a7be5b7) into main (a731c7d) will not change coverage.
The diff coverage is n/a.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff            @@
##               main    #4120   +/-   ##
=========================================
  Coverage     79.40%   79.40%
  Complexity     2525     2525
=========================================
  Files           312      312
  Lines         10258    10258
  Branches        681      681
=========================================
  Hits           8145     8145
  Misses         1847     1847
  Partials        266      266

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Fix failed is-tests-default-configuration integration tests

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/6485235988

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/6485235988
Status: cancelled


Always using the latest version of Elasticsearch for the pipeline cluster can break CCR

We haven't had any updates since Tuesday night. What's up with that?
I applied a Terraform change to the pipeline stack on Tuesday night. This change triggered an inadvertent upgrade of the pipeline cluster from v7.14.0 to v7.14.1. You can see these changes in the activity log for the pipeline cluster:

This is because we always pick the latest available version of Elasticsearch: https://github.com/wellcomecollection/catalogue-pipeline/blob/main/pipeline/terraform/stack/elastic_pipeline.tf#L17
The API cluster is still at v7.14.0. It reports that CCR is all fine, but this is definitely not the case:

the cluster containing follower indices must be running the same or newer version of Elasticsearch as the remote cluster

My change inadvertently broke this rule, so CCR stopped working.
Suggested changes:

[ ] Upgrade the catalogue-api cluster to v7.14.1 and hope that CCR resumes. If not, we'll need to reindex.
[ ] Sort out the pipeline Terraform so it doesn't produce ridiculous diffs, and changes like this can be caught at plan time
[ ] Don't automatically pick the latest Elasticsearch version

Some notes as we have seen versioning challenges previously.
Here is there policy.
Given that, I feel we should make sure we have a way of staying up to date, as the more versions you need to update, the harder it is to tell if something is breaking.
Could we somehow couple the leader & follower in terms of provisioning and link the versions?
Are we certain that we don't have CCR issues as we should have alerts on CCR failing.
 - I'd like to look into why they haven't triggered.
Annoyingly the logging cluster stack monitoring is giving us 2 catalogue-api clusters, I'd also like to dig into why this is the case as we've seen it previously.

I feel we should make sure we have a way of staying up to date, as the more versions you need to update, the harder it is to tell if something is breaking.
Could we somehow couple the leader & follower in terms of provisioning and link the versions?

Yeah, I was thinking maybe we export the version from the catalogue-api Terraform, then load that in the pipeline Terraform.
If Elasticsearch tells you it can't deploy that version any more, you know you have to update the API cluster first or pin an older version, rather than jumping straight to latest.

Are we certain that we don't have CCR issues as we should have alerts on CCR failing

I'm reasonably certain we do have CCR issues because the replication between the pipeline and API clusters has stopped, I just can't find where Elastic Cloud is telling us this.

It reports that CCR is all fine

Made me think that we don't have CCR issues - there's definitely a problem with observability of CCR stuff between pipeline and API that needs sorting.

there's definitely a problem with observability of CCR stuff between pipeline and API that needs sorting

TBH I am tempted to eschew Elastic alerting for this entirely because it seems a bit dodgy, and measure the effect of CCR instead.
Query the works index in the pipeline cluster and the API cluster, sorting by indexedTime and taking the max value. If they differ by more than ten minutes, something has gone bang.


Replacing occurences of shell with  sh

Change all the code highlighting from shell to sh so it is consistent across the documentation

closing this PR as it is not correct see https://github.com/influxdata/docs.influxdata.com/issues/232#issuecomment-183129651


Zc* arch-test support

Hi, I have opened a GSoC project for implementing Zc* RTL in rocket chip, and there are already candidates for that.
I specifically asked the mentee to use arch-test to test their RTL, as this is the standard way and there are already some arch-test infra in rocket chip, with the expectation that there will be Zc* arch-test when the project starts.
I heard from @liweiwei90 that there are already some WIP for Zc* in arch-test (did not know where the WIP branch is), so I open this issue to track the status.
Can an usable branch be present when the GSoC project starts? Or I need to plan to ask the mentee or someone else to implement these tests.

Look at item #25:
https://github.com/orgs/riscv-admin/projects/2/views/4
I believe it is blocked because of a Sail issue at the moment
On Fri, Apr 14, 2023 at 12:11 AM Hongren (Zenithal) Zheng <
@.***> wrote:

Hi, I have opened a GSoC project for implementing Zc* RTL in rocket chip
https://github.com/chipsalliance/ideas/blob/main/gsoc-2023-ideas.md#implement-zc-risc-v-isa-extension-in-rocket-chip,
and there are already candidates for that.
I specifically asked the mentee to use arch-test to test their RTL, as
this is the standard way and there are already some arch-test infra in
rocket chip, with the expectation that there will be Zc* arch-test when the
project starts.
I heard from @liweiwei90 https://github.com/liweiwei90 that there are
already some WIP for Zc* in arch-test (did not know where the WIP branch
is), so I open this issue to track the status.
Can an usable branch be present when the GSoC project starts? Or I need to
plan to ask the mentee or someone else to implement these tests.
—
Reply to this email directly, view it on GitHub
https://github.com/riscv-non-isa/riscv-arch-test/issues/334, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AHPXVJQVK5CTN3YVHGMEA5DXBD2CDANCNFSM6AAAAAAW6BFEVU
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

All Zc* tests have been added now, so closing this issue.


SSH Handshake Failure

Please execute this shell command then replace it with your console output.
ProductName:	Mac OS X
ProductVersion:	10.13.6
BuildVersion:	17G65
1.5.3
Project
speech/Swift/Speech-gRPC-Streaming/Podfile
Problem | Feature
I keep getting this SSL error, both with the sample project and my own:
ssl_transport_security.cc:1228] Handshake failed with fatal error SSL_ERROR_SSL: error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED.
E0725 21:37:32.803528000<PHONE_NUMBER>76768

@jmstone617 Might be related if you are on a enterprise network: https://github.com/GoogleCloudPlatform/google-cloud-python/issues/5591


luckperms does not download modules

i have this error on hosting enderchest.pl
https://gist.github.com/mat41997/393d797870057d02c2d81c87041776e1
on my local pc normaly work
https://gist.github.com/mat41997/6c8f47d9de8db4cee716b4d21b72912a
java 8
Paper-1062
LuckPerms-Bukkit-3.0.65.jar
what is wrong?

Your host isn't allowing LP to connect to the internet. Perhaps it only allows certain protocols.
Just copy the contents of the /lib/ folder on your local PC to the server.

Support enderchest asks me what ip address i get these files. They want to add that address to the firewall

maven.org and github.com.

maven.org not works : O
ok. i use
<IP_ADDRESS> ( repo1.maven.org ) outbound
<IP_ADDRESS> ( github.com ) outbound
and works : )
thx. for the help


Fix scrolling

Scrolling on both the learning page and the detection page are limited.
On the detection page - there is no scrolling.
On the learning page - the camera view 'hides' part of the text area, which makes the app think it doesn't have to offer the user any more scrolling, but the user cant see the text hidden behind the camera view.

Now fixed


add support for buildkite token from ssm

ssm support extracted from https://github.com/buildkite/buildkite-agent-metrics/pull/59

Thanks @arromer, this looks good. Perhaps could we go for BUILDKITE_AGENT_TOKEN_SSM_PARAM that specifies a path in SSM to look for? That way folks can customize there that token lives vs it being hard-coded.

Sorry for being slow to get this merged in, really appreciate your patience and the contribution! 🙌🏻


Fix recent "All Messages" bugs

To test this, mute a topic, write a message to the topic, then go to "All Messages."

Merged, thanks @showell!


Update Chinese language support

Closes https://github.com/inventree/InvenTree/issues/7873
Closes https://github.com/inventree/InvenTree/pull/7876
Update language code from "zh" to "zh_Hans"
Crowdin project has new manual language mappings
Ref: https://github.com/inventree/InvenTree/pull/7879

The Chinese translation support now matches Crowdin:

I'll merge this in and then monitor https://github.com/inventree/InvenTree/pull/7879 too.


Latest TFLite converter failed on fused operators

The latest TFLite converter seems unable to handle fused operators
tflite.converter would be crashed on:

code to reproduce

import tensorflow as tf
print(f"TensorFlow version: {tf.version.VERSION}")

def get_implements_signature():
    implements_signature = [
        'name: "exp_sin"',
        'attr {key: "tfl_fusable_op" value { b: true } }',
    ]
    return " ".join(implements_signature)

@tf.function(experimental_implements=get_implements_signature())
def exp_sin(x):
    x = tf.math.exp(x)
    x = tf.math.sin(x)
    return x

class custom_example(tf.keras.layers.Layer):
    def call(self, inputs):
        return exp_sin(inputs)

x_in = x = tf.keras.layers.Input(shape=(10, 10, 3), batch_size=1)
x = custom_example()(x)
x = tf.keras.layers.Conv2D(filters=8, kernel_size=4)(x)
model = tf.keras.Model(inputs=x_in, outputs=x, name="example")
model.summary()

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
with open("model_fp32.tflite", "wb") as f:
    f.write(converter.convert())

error log

2023-11-23 18:04:30.184288: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CP
U instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
TensorFlow version: 2.16.0-dev20231123
...
2023-11-23 18:04:32.068288: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:378] Ignored output_format.
2023-11-23 18:04:32.068331: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:381] Ignored drop_control_dependency.
2023-11-23 18:04:32.069026: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: /tmp/tmp72ka81_f
2023-11-23 18:04:32.069483: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve }
2023-11-23 18:04:32.069501: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: /tmp/tmp72ka81_f
2023-11-23 18:04:32.074256: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:388] MLIR V1 optimization pass is not enabled
2023-11-23 18:04:32.074898: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle.
2023-11-23 18:04:32.114217: I tensorflow/cc/saved_model/loader.cc:217] Running initialization op on SavedModel bundle at path: /tmp/tmp72ka81_f
2023-11-23 18:04:32.120217: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 51193 microseconds.
2023-11-23 18:04:32.133808: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
loc(fused["ReadVariableOp:", callsite("example_1/conv2d_1/ReadVariableOp@__inference_serving_default_47"("/media/WORK/discovery/NGP_issues/test_models/simple_float_test.py":33:1) at callsite("/home/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages/tensorflow/lite/python/lite.py":1139:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages/tensorflow/lite/python/lite.py":1093:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages/tensorflow/lite/python/lite.py":1601:1 at callsite("/home/p
ensun01/anaconda3/envs/py39/lib/python3.9/site-packages/tensorflow/lite/python/lite.py":1579:1 at callsite("/home/pensun01/anaconda3/env
s/py39/lib/python3.9/site-packages/tensorflow/lite/python/convert_phase.py":205:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/py
thon3.9/site-packages/tensorflow/lite/python/lite.py":1502:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages
/keras/src/backend/tensorflow/layer.py":57:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages/keras/src/backe
nd/tensorflow/layer.py":119:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages/keras/src/utils/traceback_util
s.py":118:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages/keras/src/layers/layer.py":830:1 at callsite("/h
ome/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages/keras/src/utils/traceback_utils.py":118:1 at callsite("/home/pensun01/anaco
nda3/envs/py39/lib/python3.9/site-packages/keras/src/ops/operation.py":42:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/python3.
9/site-packages/keras/src/utils/traceback_utils.py":157:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages/ke
ras/src/models/functional.py":188:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages/keras/src/ops/function.p
y":153:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages/keras/src/models/functional.py":572:1 at callsite("
/home/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages/keras/src/utils/traceback_utils.py":118:1 at callsite("/home/pensun01/ana
conda3/envs/py39/lib/python3.9/site-packages/keras/src/layers/layer.py":830:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/python
3.9/site-packages/keras/src/utils/traceback_utils.py":118:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages/
keras/src/ops/operation.py":42:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages/keras/src/utils/traceback_u
tils.py":157:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages/keras/src/layers/convolutional/base_conv.py":
227:1 at callsite("/home/pensun01/anaconda3/envs/py39/lib/python3.9/site-packages/keras/src/ops/numpy.py":4533:1 at callsite("/home/pens
un01/anaconda3/envs/py39/lib/python3.9/site-packages/keras/src/backend/tensorflow/numpy.py":1183:1 at "/home/pensun01/anaconda3/envs/py3
9/lib/python3.9/site-packages/keras/src/backend/tensorflow/core.py":65:1)))))))))))))))))))))))))]): error: missing attribute 'value'
LLVM ERROR: Failed to infer result type(s).
Aborted (core dumped)

This usecase is working on earlier versions, i.g., TensorFlow version: 2.14.1
But it would be failed on recent versions, i.g., TensorFlow version: 2.15.0 ~ TensorFlow version: 2.16.0-dev20231123

Hi @psunn ,
I have reproduced the code on tensorflow : 2.15.0. on colab with both CPU and GPU. It is working as expected. Please find the gist.
Thank You


Chartjs legend onclick doesn't strikethrough

Expected behavior
When using custom onclick even for legends it should strike through disabled legends
Current behavior
When using custom onclick even for legends it does not strike through disabled legends
https://stackoverflow.com/questions/74633340/chartjs-legend-onclick-doesnt-strikethrough#
Reproducible sample
https://stackoverflow.com/questions/74633340/chartjs-legend-onclick-doesnt-strikethrough#
Optional extra steps/info to reproduce
No response
Possible solution
No response
Context
No response
chart.js version
v3.9.1
Browser name and version
Chrome
Link to your project
https://stackoverflow.com/questions/74633340/chartjs-legend-onclick-doesnt-strikethrough#

This is because you never set the hidden property in your generateLabels so chart.js does not know which labels to strikethrough since they are never hidden for the legend plugin.


Navigation issue with a new navigator inside a navigator

My navigation structure is like this:
Main TabNavigator
Home
User
Goals
but inside the 'Goals' screen, I have created another  within it.
This is because I wish for only the lower portion of Goals screen to navigate, not the entire screen.
Goals TabNavigator
Nutrition
Sleep
Exercise
I wish for a button in Nutrition to navigate the whole screen to User ( in the parent navigator).
How would I go about this?  I assume it involves passing this.props.navigation from the parent navigator down to Goals, but I have no clue how to do it.
I tried to pass a prop like this  <TabNavigator parent={this.props.navigation} />
but then I don't know how I can access that parent prop in my Nutrition screen.
I'm fairly new so my terminology may be off here..thanks guys!

Problem solved.
I passed 'this.props.navigation' through redux and accessed it in Nutrition.

i have same problem, if i don`t use redux， how i do?


[BugFix] open Once for mapElement and in predict which optimize const inputs

Fixes  concurrently open expression trees during scan
What type of PR is this:

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Checklist:

[x] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[x] 3.1
[ ] 3.0
[ ] 2.5
[ ] 2.4

@Mergifyio backport branch-3.1


Made a glossary

Attempted to make a very simple glossary for the current terms

I'm not looking for a glossary.


Add DWC to token list

Closing because PR #1198 is submitted with checksum fix of another token folder that was submitted in improper format


[Bug]: Tableau Desktop, Prep & Reader no longer working

What happened?
It appears the Tableau site may have changed so the Get-EvergreenApp functions currently fail to retrieve the latest Tableau Desktop, Prep Builder and Reader are all failing with WARNING: Resolve-SystemNetWebRequest: , with: https://www.tableau.com/downloads/...
As the pages referenced as the Download URi (in each Manifest) does a 302 redirect to the actual URL of the download file, it may be that the download request is now expecting a header to be set or a cookie to exist for it to successfully return the download.
I have tried with various headers (including User-Agent, Sec-Ch-Ua, Sec-Ch-Ua-Mobile, Sec-Ch-Ua-Platform, Sec-Fetch-Dest, Sec-Fetch-Mode, Sec-Fetch-Site & Sec-Fetch-User) but I can not get the page to return anything other than Access Denied.
Example (for Latest Tableau Desktop download):
$url = "https://www.tableau.com/downloads/desktop/pc64"
Invoke-WebRequest $url -UserAgent 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36' -Headers @{"Sec-Ch-Ua"='"Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"','"Sec-Fetch-Mode"="navigate"','"Sec-Ch-Ua-Platform"="`"Windows`""','"Sec-Fetch-Dest"="document"','"Sec-Fetch-Site"="none"','"Sec-Fetch-User"="?1"'}

Results:
Invoke-WebRequest:
Access Denied

Access Denied

You don't have permission to access "http://www.tableau.com/downloads/desktop/pc64" on this server.
Reference #18.9f3e1202.1713428789.5e2cf9a
https://errors.edgesuite.net/18.9f3e1202.1713428789.5e2cf9a

Translation to ASCII:
You don't have permission to access "http://www.tableau.com/downloads/desktop/pc64" on this server.
Reference #18.9f3e1202.1713428789.5e2cf9a
https://errors.edgesuite.net/18.9f3e1202.1713428789.5e2cf9a

Unsure if this can be fixed for all 3?
N.B. I have included the current verbose output from all three products below
Version
2404.901
What PowerShell edition/s are you running Evergreen on?
PowerShell Core, Windows PowerShell
Which operating system/s are you running Evergreen on?
Windows 10+
Have you reviewed the documentation?

[X] Troubleshooting at: https://stealthpuppy.com/evergreen/troubleshoot/
[X] Known issues at: https://stealthpuppy.com/evergreen/issues/

Verbose output
Get-EvergreenApp -Name "TableauDesktop" -Verbose
VERBOSE: Function path: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2404.901\Apps\Get-TableauDesktop.ps1
VERBOSE: Function exists: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2404.901\Apps\Get-TableauDesktop.ps1.
VERBOSE: Dot sourcing: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2404.901\Apps\Get-TableauDesktop.ps1.
VERBOSE: Get-FunctionResource: read application resource strings from [C:\Program Files\WindowsPowerShell\Modules\Evergreen\2404.901\Manifests\TableauDesktop.json]
VERBOSE: Calling: Get-TableauDesktop.
VERBOSE: Resolve-SystemNetWebRequest: Attempting to resolve: https://www.tableau.com/downloads/desktop/pc64.
WARNING: Resolve-SystemNetWebRequest: , with: https://www.tableau.com/downloads/desktop/pc64.
WARNING: Resolve-SystemNetWebRequest: For troubleshooting steps see: https://stealthpuppy.com/evergreen/troubleshoot/.
WARNING: Run 'Get-EvergreenApp -Name "TableauDesktop" -Verbose' to review additional details for troubleshooting.
Exception: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2404.901\Apps\Get-TableauDesktop.ps1:21
Line |
  21 |  …    $Response = Resolve-SystemNetWebRequest -Uri $res.Get.Download.Uri
     |                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | You cannot call a method on a null-valued expression.

Get-EvergreenApp -Name "TableauPrep" -Verbose
VERBOSE: Function path: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2404.901\Apps\Get-TableauPrep.ps1
VERBOSE: Function exists: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2404.901\Apps\Get-TableauPrep.ps1.
VERBOSE: Dot sourcing: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2404.901\Apps\Get-TableauPrep.ps1.
VERBOSE: Get-FunctionResource: read application resource strings from [C:\Program Files\WindowsPowerShell\Modules\Evergreen\2404.901\Manifests\TableauPrep.json]
VERBOSE: Calling: Get-TableauPrep.
VERBOSE: Resolve-SystemNetWebRequest: Attempting to resolve: https://www.tableau.com/downloads/prep/pc64.
WARNING: Resolve-SystemNetWebRequest: , with: https://www.tableau.com/downloads/prep/pc64.
WARNING: Resolve-SystemNetWebRequest: For troubleshooting steps see: https://stealthpuppy.com/evergreen/troubleshoot/.
WARNING: Run 'Get-EvergreenApp -Name "TableauPrep" -Verbose' to review additional details for troubleshooting.
Exception: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2404.901\Apps\Get-TableauPrep.ps1:21
Line |
  21 |  …    $Response = Resolve-SystemNetWebRequest -Uri $res.Get.Download.Uri
     |                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | You cannot call a method on a null-valued expression.

Get-EvergreenApp -Name "TableauReader" -Verbose
VERBOSE: Function path: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2404.901\Apps\Get-TableauReader.ps1
VERBOSE: Function exists: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2404.901\Apps\Get-TableauReader.ps1.
VERBOSE: Dot sourcing: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2404.901\Apps\Get-TableauReader.ps1.
VERBOSE: Get-FunctionResource: read application resource strings from [C:\Program Files\WindowsPowerShell\Modules\Evergreen\2404.901\Manifests\TableauReader.json]
VERBOSE: Calling: Get-TableauReader.
VERBOSE: Resolve-SystemNetWebRequest: Attempting to resolve: https://www.tableau.com/downloads/reader/pc64.
WARNING: Resolve-SystemNetWebRequest: , with: https://www.tableau.com/downloads/reader/pc64.
WARNING: Resolve-SystemNetWebRequest: For troubleshooting steps see: https://stealthpuppy.com/evergreen/troubleshoot/.
WARNING: Run 'Get-EvergreenApp -Name "TableauReader" -Verbose' to review additional details for troubleshooting.
Exception: C:\Program Files\WindowsPowerShell\Modules\Evergreen\2404.901\Apps\Get-TableauReader.ps1:21
Line |
  21 |  …    $Response = Resolve-SystemNetWebRequest -Uri $res.Get.Download.Uri
     |                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | You cannot call a method on a null-valued expression.

The server is probably expecting a different user agent. @AScott-WWF  do you have access to these applications to see what they're doing via Fiddler?

Nvm, fixed with headers in the request.


Detekt 1.22.0 is getting noisier and noisier

Expected Behavior
No output in Gradle console log unless there's an actual problem.
1.21.0:
> Task :plugins:detektMain
The BindingContext was created with 36 issues. Run detekt with --debug to see the error messages.

> Task :backend:quickbook:detektTest
The BindingContext was created with 6 issues. Run detekt with --debug to see the error messages.

> Task :backend:database:detektTest
The BindingContext was created with 2 issues. Run detekt with --debug to see the error messages.

> Task :backend:endpoint:detektTest
The BindingContext was created with 40 issues. Run detekt with --debug to see the error messages.

Current Behavior
1.22.0-RC1:
> Task :plugins:detekt
The rule 'AvoidReferentialEquality' requires type resolution but it was run without it.
The rule 'Deprecation' requires type resolution but it was run without it.
The rule 'DontDowncastCollectionTypes' requires type resolution but it was run without it.
The rule 'DoubleMutabilityForCollection' requires type resolution but it was run without it.
The rule 'ElseCaseInsteadOfExhaustiveWhen' requires type resolution but it was run without it.
The rule 'ExitOutsideMain' requires type resolution but it was run without it.
The rule 'HasPlatformType' requires type resolution but it was run without it.
The rule 'MissingWhenCase' requires type resolution but it was run without it.
The rule 'NullCheckOnMutableProperty' requires type resolution but it was run without it.
The rule 'RedundantElseInWhen' requires type resolution but it was run without it.
The rule 'UnnecessaryNotNullOperator' requires type resolution but it was run without it.
The rule 'UnnecessarySafeCall' requires type resolution but it was run without it.
The rule 'UnreachableCode' requires type resolution but it was run without it.
The rule 'UnsafeCallOnNullableType' requires type resolution but it was run without it.
The rule 'UnsafeCast' requires type resolution but it was run without it.
The rule 'IgnoredReturnValue' requires type resolution but it was run without it.
The rule 'ImplicitUnitReturnType' requires type resolution but it was run without it.
The rule 'NullableToStringCall' requires type resolution but it was run without it.
The rule 'UnreachableCatchBlock' requires type resolution but it was run without it.
The rule 'UnusedUnaryOperator' requires type resolution but it was run without it.
The rule 'NestedScopeFunctions' requires type resolution but it was run without it.
The rule 'ReplaceSafeCallChainWithRun' requires type resolution but it was run without it.
The rule 'NamedArguments' requires type resolution but it was run without it.
The rule 'InjectDispatcher' requires type resolution but it was run without it.
The rule 'RedundantSuspendModifier' requires type resolution but it was run without it.
The rule 'SleepInsteadOfDelay' requires type resolution but it was run without it.
The rule 'SuspendFunWithFlowReturnType' requires type resolution but it was run without it.
The rule 'SuspendFunWithCoroutineScopeReceiver' requires type resolution but it was run without it.
The rule 'ReturnFromFinally' requires type resolution but it was run without it.
The rule 'ObjectExtendsThrowable' requires type resolution but it was run without it.
The rule 'NoNameShadowing' requires type resolution but it was run without it.
The rule 'BooleanPropertyNaming' requires type resolution but it was run without it.
The rule 'NonBooleanPropertyPrefixedWithIs' requires type resolution but it was run without it.
The rule 'ArrayPrimitive' requires type resolution but it was run without it.
The rule 'CouldBeSequence' requires type resolution but it was run without it.
The rule 'CanBeNonNullable' requires type resolution but it was run without it.
The rule 'ForbiddenMethodCall' requires type resolution but it was run without it.
The rule 'UnnecessaryInnerClass' requires type resolution but it was run without it.
The rule 'ObjectLiteralToLambda' requires type resolution but it was run without it.
The rule 'UnnecessaryApply' requires type resolution but it was run without it.
The rule 'UnnecessaryFilter' requires type resolution but it was run without it.
The rule 'UnnecessaryLet' requires type resolution but it was run without it.
The rule 'PreferToOverPairSyntax' requires type resolution but it was run without it.
The rule 'NullableBooleanCheck' requires type resolution but it was run without it.
The rule 'VarCouldBeVal' requires type resolution but it was run without it.
The rule 'ForbiddenVoid' requires type resolution but it was run without it.
The rule 'ExplicitCollectionElementAccessMethod' requires type resolution but it was run without it.
The rule 'UselessCallOnNotNull' requires type resolution but it was run without it.
The rule 'RedundantExplicitType' requires type resolution but it was run without it.
The rule 'LibraryCodeMustSpecifyReturnType' requires type resolution but it was run without it.
The rule 'UseEmptyCounterpart' requires type resolution but it was run without it.
The rule 'UseCheckNotNull' requires type resolution but it was run without it.
The rule 'UseRequireNotNull' requires type resolution but it was run without it.
The rule 'RedundantHigherOrderMapUsage' requires type resolution but it was run without it.
The rule 'UseIfEmptyOrIfBlank' requires type resolution but it was run without it.
The rule 'MultilineLambdaItParameter' requires type resolution but it was run without it.
The rule 'UseIsNullOrEmpty' requires type resolution but it was run without it.
The rule 'UseOrEmpty' requires type resolution but it was run without it.
The rule 'UseAnyOrNoneInsteadOfFind' requires type resolution but it was run without it.
The rule 'MaxChainedCallsOnSameLine' requires type resolution but it was run without it.

> Task :plugins:detektMain
The BindingContext was created with 36 issues. Run detekt CLI with --debug or set `detekt { debug = true }` in Gradle to see the error messages.

> Task :backend:quickbook:detektTest
The BindingContext was created with 6 issues. Run detekt CLI with --debug or set `detekt { debug = true }` in Gradle to see the error messages.

> Task :backend:database:detektTest
The BindingContext was created with 2 issues. Run detekt CLI with --debug or set `detekt { debug = true }` in Gradle to see the error messages.

> Task :backend:endpoint:detektTest
The BindingContext was created with 40 issues. Run detekt CLI with --debug or set `detekt { debug = true }` in Gradle to see the error messages.

1.22.0-RC2:
> Task :backend:feed:detektMain
warning: classpath entry points to a non-existent location: ...\backend\feed\src\main\java

> Task :plugins:detektMain
warning: classpath entry points to a non-existent location: ...\gradle\plugins\src\main\java
The BindingContext was created with 36 issues. Run detekt CLI with --debug or set `detekt { debug = true }` in Gradle to see the error messages.

> Task :backend:network:detektMain
warning: classpath entry points to a non-existent location: ...\backend\network\src\main\java

> Task :backend:quickbook:detektMain
warning: classpath entry points to a non-existent location: ...\backend\quickbook\src\main\java

> Task :backend:network:detektTest
warning: classpath entry points to a non-existent location: ...\backend\network\src\test\java
warning: classpath entry points to a non-existent location: ...\backend\network\build\classes\java\main

> Task :backend:quickbook:detektTest
warning: classpath entry points to a non-existent location: ...\backend\quickbook\src\test\java
The BindingContext was created with 6 issues. Run detekt CLI with --debug or set `detekt { debug = true }` in Gradle to see the error messages.

> Task :backend:feed:detektTest
warning: classpath entry points to a non-existent location: ...\backend\feed\src\test\java

> Task :backend:database:detektTestFixtures
warning: classpath entry points to a non-existent location: ...\backend\database\src\testFixtures\java

> Task :backend:endpoint:detektMain
warning: classpath entry points to a non-existent location: ...\backend\endpoint\src\main\java
The BindingContext was created with 4 issues. Run detekt CLI with --debug or set `detekt { debug = true }` in Gradle to see the error messages.

> Task :backend:database:detektTest
warning: classpath entry points to a non-existent location: ...\backend\database\src\test\java
warning: classpath entry points to a non-existent location: ...\backend\database\build\classes\java\testFixtures
warning: classpath entry points to a non-existent location: ...\backend\database\build\tmp\kapt3\classes\testFixtures
The BindingContext was created with 1 issues. Run detekt CLI with --debug or set `detekt { debug = true }` in Gradle to see the error messages.

> Task :backend:endpoint:detektTest
warning: classpath entry points to a non-existent location: ...\backend\endpoint\src\test\java
warning: classpath entry points to a non-existent location: ...\backend\database\build\classes\java\testFixtures
warning: classpath entry points to a non-existent location: ...\backend\database\build\tmp\kapt3\classes\testFixtures
The BindingContext was created with 52 issues. Run detekt CLI with --debug or set `detekt { debug = true }` in Gradle to see the error messages.

Notice that even the number of BindingContext issues are getting higher.
Context
gradlew ... is usually empty, and that's a good thing. The more warnings show up the less likely is that we see new warnings pop up, and long term projects end up with just megabytes of logs which no-one cares about.
The project I was executing on is exactly the same code for each version. The task I was running was also exactly the same for each version: gradlew :plugins:detektEach detektMain detektTest detektTestFixtures --rerun-tasks (detektEach depends on detekt, detektMain and detektTest in that module).

I would like to know what other people think about this. For sure we can completely remove the "The BindingContext was created with 242 issues. [...]" message.
For me this situation is odd because in one hand type resolution is still experimental and have some rought edges. On the other hand half of the rules of detekt use type resolution and I think that all the maintainer doesn't think too much about detekt without type solving. It would be great to make the our compiler plugin production ready to fix all these issues but we need hands for doing that.

so people will just learn to tune these warnings out (and every other warning from any log / check)

☝ To me this is a really good reason to remove that message.

I think those messages are in actionable and will condition users to tune out detekt logs by default

For sure we can completely remove the "The BindingContext was created with 242 issues. [...]" message.

+1 on suppressing this. End users don't need to know what a BindingContext is frankly.

#5448 and #5449 should fix these :)


Config list filtering issue

Environment
Docker
Version
No response
Describe the problem
I noticed that when I try to select a configuration file from the list, it shows available folders as a configuration, as well as any other file format available in the homarr/config folder. This should be visible, it should only consider files with .json only, like filtering them out somehow
in the screenshot you can see the icons folder, and the rrr.txt file

Additional info
No response
Please tick the boxes

[X] You've read the docs
[X] You've checked for duplicate issues
[X] You've tried to debug yourself

If you put files not intended to be in the configs folder it's your problem, no need to write code to prevent that


nodejs-legacy broken -- breaks npm

there is this line in the instructions script.
sudo apt-get install -y git python python-dev python-software-properties python-numpy python-pip nodejs-legacy npm watchdog
This script breaks when trying to install nodejs-legacy.
In additon, that break, causes the rest of the line, including installation of npm and watchdog to fail.
This causes openaps installation to fail.

What OS and distribution is this failing on?

Are you still seeing this?  Any more info you can provide?

Please reopen if you can provide more details or see this again.

This is still broken - running curl -s https://raw.githubusercontent.com/openaps/docs/master/scripts/quick-packages.sh | bash -
from https://openaps.readthedocs.io/en/stable/docs/walkthrough/phase-0/openaps.html
initially results in:
E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied)
E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root?
then the script outputs:
Package nodejs-legacy is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
nodejs libnode64
E: Package 'nodejs-legacy' has no installation candidate
pi@raspberrypi:~ $_
when processing the line: _sudo apt-get -o Acquire::ForceIPv4=true install -y git python python-dev software-properties-common python-numpy python-pip nodejs-legacy npm watchdog strace tcpdump screen acpid vim locate jq lm-sensors bc && _
in the script being executed.
this was on a brand new setup today - with the following OS details:
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
Running on a Raspberry PI Zero WH
nodejs-legacy is nolonger available,
according to https://nodejs.org/en/download/ jodejs is no longer supported on ARMv6 (Pi Zero)


php7 Compatibility problem

105 | ERROR | Function mcrypt_create_iv() is deprecated since PHP 7.1 and removed since PHP 7.2; Use random_bytes() or OpenSSL instead
105 | ERROR | Extension 'mcrypt' is deprecated since PHP 7.1 and removed since PHP 7.2; Use openssl (preferred) or pecl/mcrypt once available instead
105 | ERROR | The constant "MCRYPT_DEV_URANDOM" is deprecated since PHP 7.1 and removed since PHP 7.2

This library shouldn't be used on any PHP version equal to 5.5 or higher

Hi:
but other package depends this package，so how to improve to php7
------------------ Original ------------------
From:  "Benjamin<EMAIL_ADDRESS>Date:  Fri, Aug 24, 2018 04:36 PM
To<EMAIL_ADDRESS>Cc<EMAIL_ADDRESS><EMAIL_ADDRESS>Subject:  Re: [ircmaxell/password_compat] php7 Compatibility problem (#106)
This library shouldn't be used on any PHP version equal to 5.5 or higher
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

That part of the code will never be executed on PHP 7 as the password_hash() function does exist there: https://github.com/ircmaxell/password_compat/blob/master/lib/password.php#L24

...and if your code analyser is still complaining, it's a bug in your code analyser, not this lib.


Update upstream jquery

Update JQuery submodule to use QUnit.start() instead of start()
Pull request:
https://github.com/jquery/qunit/pull/168
Commit:
https://github.com/jquery/qunit/commit/bd6a75e29e97576f12bed0c6d8f949d7bafcd9d7

thanks merged!


[BUG] Insufficient number of hits for nested knn queries with efficient filter

Hello 👋
What is the bug?
We use an index to store text documents for semantic search purpose. The text being long, we chunk it in paragraph to embed it using all-MiniLM-L6-v2 model. Each chunk being stored in that nested field of the document.
Each document has also an account_id attribute that we use when querying (efficient filtering).
Then we do approximative knn queries with lucene hnsw.
From these documentations :

k-NN search with nested fields
k-NN search with filters
and mosty Enhanced multi-vector support for OpenSearch k-NN search with nested fields

I expect when executing a knn query on this nested field with efficient filter to get at least n hits, n being the minimum between k and the number of documents that match the efficient filter.
But for some specific input vector or query_text, we get less than n hits, and sometimes even 0. For the same filter with a different query, we get the correct n hits.
We have two other indices without nested field (only one vector per document) with the same efficient filter and it works as expected.
Seems similar to this https://github.com/opensearch-project/k-NN/issues/2222 or https://github.com/opensearch-project/k-NN/issues/2339 except the efficient filtering is as simple as a term filter.
How can one reproduce the bug?
Error happens on specific queries so it's hard to reproduce.
Here is the mapping of the index :
{
  "knowledge-index": {
    "mappings": {
      "properties": {
        "accountId": {
          "type": "keyword"
        },
        "id": {
          "type": "keyword"
        },
        "metadata": {
          "type": "text"
        },
        "metadataEmbedding": {
          "type": "nested",
          "properties": {
            "knn": {
              "type": "knn_vector",
              "dimension": 384,
              "method": {
                "engine": "lucene",
                "space_type": "l2",
                "name": "hnsw",
                "parameters": {}
              }
            }
          }
        },
        "timestamp": {
          "type": "date"
        }
      }
    }
  }
}

Here is the query :
GET /knowledge-index/_search?preference=_primary&explain=true&request_cache=false
{
  "from": 0,
  "_source": {
    "excludes": [
      "metadataEmbedding"
    ]
  },
  "query": {
    "nested": {
      "score_mode": "max",
      "path": "metadataEmbedding",
      "query": {
        "neural": {
          "metadataEmbedding.knn": {
            "query_text": "<query_text>",
            "model_id": "9QxR8YsBSCN1wquQEH2b",
            "k": <k>,
            "filter": {
                "term": {
                  "accountId":  "<account_id>"
              }
            }
          }
        }
      }
    }
  }
}

For k = 38, I get 6 hits
  "hits": {
    "total": {
      "value": 6,
      "relation": "eq"
    },
    "max_score": 0.50342417,

But for k = 1000 I get 32 hits, and k = 10000 (max value) 232 hits.
For another query_text value, I have different results where hits is always = k (or the max of documents that match filter which is 232)
I have the same results when converting first the text in vector and use directly the vector without the neural instruction :
POST /_plugins/_ml/_predict/text_embedding/9QxR8YsBSCN1wquQEH2b
{
  "text_docs":[ "<query_text>"],
  "return_number": true,
  "target_response": ["sentence_embedding"]
}

GET /knowledge-index/_search?preference=_primary&explain=true&request_cache=false
{
  "size": 5,
  "_source": {
    "excludes": [
      "metadataEmbedding"
    ]
  },
  "query": {
    "nested": {
      "path": "metadataEmbedding",
      "query": {
        "knn": {
          "metadataEmbedding.knn": {
            "vector": [
                 ....
             ],
            "k": 38,
            "filter": {
              "term": {
                "accountId": "<account_id>"
              }
            }
          }
        }
      }
    }
  }
}

What is the expected behavior?
Getting n hits, n being the minimum between k and the number of documents that match the efficient filter.
What is your host/environment?

opensearch version : 2.17.1

Do you have any additional context?
Here is the result of GET /_plugins/_knn/stats?pretty on the node :
{
      "max_distance_query_with_filter_requests": 0,
      "graph_memory_usage_percentage": 0,
      "graph_query_requests": 0,
      "graph_memory_usage": 0,
      "cache_capacity_reached": false,
      "load_success_count": 0,
      "training_memory_usage": 0,
      "indices_in_cache": {},
      "script_query_errors": 0,
      "hit_count": 0,
      "knn_query_requests": 2215,
      "total_load_time": 0,
      "miss_count": 0,
      "min_score_query_requests": 0,
      "knn_query_with_filter_requests": 2215,
      "training_memory_usage_percentage": 0,
      "max_distance_query_requests": 0,
      "lucene_initialized": true,
      "graph_index_requests": 0,
      "faiss_initialized": false,
      "load_exception_count": 0,
      "training_errors": 0,
      "min_score_query_with_filter_requests": 0,
      "eviction_count": 0,
      "nmslib_initialized": false,
      "script_compilations": 1,
      "script_query_requests": 2,
      "graph_stats": {
        "refresh": {
          "total_time_in_millis": 0,
          "total": 0
        },
        "merge": {
          "current": 0,
          "total": 0,
          "total_time_in_millis": 0,
          "current_docs": 0,
          "total_docs": 0,
          "total_size_in_bytes": 0,
          "current_size_in_bytes": 0
        }
      },
      "graph_query_errors": 0,
      "indexing_from_model_degraded": false,
      "graph_index_errors": 0,
      "training_requests": 0,
      "script_compilation_errors": 0
    },

Any idea on what could be the issue here ? Am I right to expect k hits for nested fields with efficient filter ?
Thanks for your help.

Further investigation into the code may be required, but based on the reported issue, it appears that post-filtering is being utilized instead of efficient filtering.

@CorentinLimier Could you try the same query without using neural?

@heemin32 Thanks for your help !
I did use a query without neural with same result. I posted the query I used in my initial message as well  :
POST /_plugins/_ml/_predict/text_embedding/9QxR8YsBSCN1wquQEH2b
{
  "text_docs":[ "<query_text>"],
  "return_number": true,
  "target_response": ["sentence_embedding"]
}

GET /knowledge-index/_search?preference=_primary&explain=true&request_cache=false
{
  "size": 5,
  "_source": {
    "excludes": [
      "metadataEmbedding"
    ]
  },
  "query": {
    "nested": {
      "path": "metadataEmbedding",
      "query": {
        "knn": {
          "metadataEmbedding.knn": {
            "vector": [
                 ....
             ],
            "k": 38,
            "filter": {
              "term": {
                "accountId": "<account_id>"
              }
            }
          }
        }
      }
    }
  }
}

Thanlks a lot 🙏

Oh. You were talking about value in hits. Could you also check if it also only return that hits number of documents by setting the size as same value as k?

@heemin32
I indeed have the same number of value returned than that hit number if size = k.
Ex :
GET /knowledge-index/_search?preference=_primary&explain=true&request_cache=false
{
  "size": 38,
  "_source": {
    "excludes": [
      "metadataEmbedding"
    ]
  },
  "query": {
    "nested": {
      "path": "metadataEmbedding",
      "query": {
        "knn": {
          "metadataEmbedding.knn": {
            "vector": [
               ...
            ],
            "k": 38,
            "filter": {
              "term": {
                "accountId": "..."
              }
            }
          }
        }
      }
    }
  }
}

Returns 6 hits (instead of k=38) and 6 documents (instead of size=38).

Also, it would be nice if you could provide a reproducible steps with smaller data set.

I will try to create a reproducible example, I can understand that it will help. I wanted to make sure first that my issue was not a mistake from me or a misunderstanding of the doc. From your understanding of the mapping and queries, are we aligned on the fact that I should expect k documents even with nested fields and efficient filtering ?
Thanks a lot

The exact hit number is a little more complex than min of k and max doc. It is sum of (min of k and max doc in a segment) for all segments. Still, the 6 hits for k = 38 and 30 hits for k = 1000 is not an expected behavior.
It could be an issue with lucene engine. Could you test it with faiss engine if possible?

Ok, I will try with faiss (by changing the engine attribute in mapping right ?). Will probably be able to do so starting from beginning of January. I will try to create a reproducible example as well.
Does it make sense to have this lucene issue only for nested structures ?
Thank you very much, will give you more details once I'll be able to work on it 🙏

Does it make sense to have this lucene issue only for nested structures ?

Yes

@buddharajusahil can you please take a look into this issue.

Sure @navneet1v , please assign me this task.

Hello,
I didn't manage to create a reproducible example with a smaller dataset.
You can find here my tentative, but everything is working as expected in this example :
tentative.txt
Not sure if it's a volume issue or not, and it's quite difficult to reproduce since for some vectors it works as expected and for others not.
My next tentative will be to use faiss engine as suggested by @heemin32 and see if I see differences in production. We monitor the number of results for each request so I will be able to monitor if the number of results is closer to what I expect.
If you have any idea on what could be the root cause of the bug and help me reproducing the issue with a smaller dataset, I would be happy to help.
Thanks

Hi @CorentinLimier I think this is related to an overall problem with efficient filtering. Can you try one thing, retry this tentative, but instead with 2 shard count specified in index settings. Then, try running the search multiple times, I believe you will get inconsistent results.


Adds ability to resolve DNS hostnames

Currently godspeed doesn't resolve DNS hostnames.  This fixes that.

:+1:

:+1: looks good; thank you for the contribution.


OnError input prop function call on errors (#24)

Please check out my error handling version

Looks pretty good to me. Can we perhaps update the readme to demonstrate how the error handler works?

Maybe we should also have an onError handler at the root level props as well. What do you think?

@jxom

Looks pretty good to me. Can we perhaps update the readme to demonstrate how the error handler works?

Yeah, sure. I will update it

Maybe we should also have an onError handler at the root level props as well. What do you think?

I was thinking about this. It seems like I'm going to use this feature. But first, it has to be implemented :)

@jxom, well all changes are done - it should work well

@Gorgonello hey do you mind explaining what this does?
onError?: Function
I see this ternary? expression quite a bit in the repo and I have no idea what it means
it is used here as well
inputName?: string

It's a flow (static type checker for js) "optional parameter". You can read more here.

Thanks!


Mod not compatible with Effortless Building 1.12.2

As mentioned in the title and on the curseforge page, would love compatibility with the options and features that the Effortless Building mod adds, so that there wouldn't be gaps in between slabs when creating lines, diagonals, spheres, walls, cylinders and so on.
Thanks! :)

Hmm. I'm not too sure how I can add this compatibility as from what I've seen, Effortless Building simply enhances the way you place blocks. If you are selecting a slab, it will by default place only one side of the slab which is something I can't change. I feel it might be better to create this issue on their page. I don't believe the mod is incompatible but rather has an oversight when it comes to the placement of slabs

That's fine, I tried reaching out to them, without luck, but no matter, just the fact that you've added compatibility with Conquest did wonders so far, thanks anyway!


Restrictions for sWEPS are not working

As you can see in the photo, it warns me that it is restricted but still spawns the weapon

Along with this, using mouse3 on the swep in the spawnlist will override the restriction, and spawning it normally by using mouse1 in the weapons tab also overrides the restriction

Hmm... I fail in recreating this issue. Are you using any other addons that might interfere with spawning them. What addons are you using?


WIP: add read_tab_raw and reconstruct_tracker

Because the current read_tab function does not allow for much customization of how tab files are read, this PR is to introduce two new functions:
read_tab_raw : The most basic tab reading function. The goal is to just take a tab file and spit out a list of tuples (where each tuple is one line from the original file--excluding the first two lines (which are header))
reconstruct_tracker : This will allow tracker objects to be reconstructed from experiment logs such that analysis functions of the tracker (such as plot) can be used.

This is very much a messy first attempt, but I wanted to go ahead and open up a PR so that discussion on what each of these functions should do can happen.

@Eric89GXL @rkmaddox
I think all the parts of this are in place and I'm open to suggestions on how to make it a little cleaner if you'd be so kind as to review the three new functions. Thanks much~

Been meaning to ask:
There's a lot of ast.literal_eval() interspersed with json.loads() (I started replacing with json.loads() when ast.literal_eval() wasn't working.) Is there an advantage to using the json function (other than it working)?

@drammock @Eric89GXL All tests except one are passing--looks like that one Travis test that is finicky is finding some random error and coveralls seems happy enough. Ready for review--thanks much!

@Eric89GXL In regards to examples, I'm not sure there's much to add--were you thinking of adding to the "parsing demo" or modifying one of the tracker examples (having them print to a .tab file instead of the console and then just running the plots from the reconstruction)?

Went ahead and put it in the TrackerDealer example--seemed the simplest option

+1 merge after tests pass. I'm about to talk in a session at ASA, so I might not check in on the PR until this afternoon. But I'm happy if anyone pushes the merge button.


[Snyk] Security upgrade org.springframework:spring-web from 5.3.27 to 5.3.33

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to fix one or more vulnerable packages in the `maven` dependencies of this project.
Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

pom.xml

Vulnerabilities that will be fixed
With an upgrade:

Severity
Priority Score (*)
Issue
Upgrade
Breaking Change
Exploit Maturity

641/1000  Why? Recently disclosed, Has a fix available, CVSS 7.1
Open Redirect SNYK-JAVA-ORGSPRINGFRAMEWORK-6444790
org.springframework:spring-web:  5.3.27 -> 5.3.33
No
No Known Exploit

(*) Note that the real score may have changed since the PR was raised.
Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
🛠 Adjust project settings
📚 Read more about Snyk's upgrade and patch logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Open Redirect

Checkmarx One – Scan Summary & Details – 377e6cf2-a03d-4ceb-b5d4-b4518a85dddf
No New Or Fixed Issues Found


data_eng/optimize_dataset_create

The purpose of the change is to speed up the creation of dataset sqlite3 files.
The analysis summary follows, which I believe justifies the change.
Performance Metrics of Optimization Changes
There are three datasets in the listed-building collection:

listed-building
listed-building-grade
listed-building-outline

Each dataset has to process it's corresponding issue log file.
Total size of Issue Logs for each dataset

listed-building 138,476 KB
listed-building-grade 1 KB
listed-building-outline 1,080 KB

Listed building clearly has the most work to do when generating the issue-log table.
The listed-building dataset has the largest transform file, containing 3412530 records. This file was 'cut' into different sizes to get the timing set below.

Run
# Records Processed
Current Solution (s)
New Solution (s)
%

1
1000
23.80
9.57
40.21

2
64000
30.39
13.81
45.44

3
512000
87.02
54.45
62.57

4
1024000
157.59
113.61
72.09

5
2048000
312.41
235.45
75.37

6
3412530
518.65
384.26
74.09

Table shows the number of seconds taken to process a given number of listed-building transform file records (+ issue-log records from above).
Summary
It’s clear the savings made from optimization diminish with increased transform file size.
The savings reduce to about 25% for very large datasets, but can be as much as 60% for smaller datasets.
It’s worth noting that, although the savings from large datasets like listed-building is small, the savings from processing all collections should be significant enough to see an overall benefit from these changes.

Codecov Report
Attention: 34 lines in your changes are missing coverage. Please review.

Comparison is base (ac51e9c) 79.43% compared to head (7404f19) 80.98%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #147      +/-   ##
==========================================
+ Coverage   79.43%   80.98%   +1.54%
==========================================
  Files          67       67
  Lines        3594     3660      +66
==========================================
+ Hits         2855     2964     +109
+ Misses        739      696      -43

Files
Coverage Δ

digital_land/commands.py
75.34% <0.00%> (-0.52%)
:arrow_down:

digital_land/package/sqlite.py
64.64% <80.30%> (+3.67%)
:arrow_up:

digital_land/package/dataset.py
75.41% <65.45%> (+39.17%)
:arrow_up:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Revision II attempt to address many of the concerns raised above.


@todo's in a non-PR branch aren't generated

I've just tried creating a todo in a private repo under a new branch and it doesn't seem to be creating new issues like it used to on master.

@ItsPugle I've made some recent changes that make it work differently in PRs, and I'm guessing that's the cause.
I think what you're experiencing is that it won't create new issues from branches that aren't master, because it expects them to be in a PR and will write a comment instead. This means that commits made to a branch (but not a branch that has an open PR) won't do anything, which is not a good thing.
Thanks for letting me know, I'll get it fixed as soon as I can!

Ah that explains it, @JasonEtco! Thanks for getting on to this so quickly, by the way :)
Is their anyway to trigger todo to rescan the repository for the issue flags? Also, if I add a issue flag, like:
<!--
@todo Create Issue
@body Body
-->

Will it create a new issue if I commit this file again without removing the todo, or does it somehow keep track of which todos have already been made into an issue?

Here's how it all works:
It scans the commits made in a push. It reads the files changed or added in those commits, parses out the issues to generate, checks if they already exist (to prevent duplicates), then generates them. It works pretty similarly in a PR, but it makes comments instead of issues.
So to answer your two questions:

To trigger todo, just push changes to the files you want it to scan.
It will not create a new issue if the same one (based on the issue title and the file its in) already exists. If one does exist and is closed, it'll be reopened. If its already an open issue, nothing will happen :)

Ah sweeto! Do you mind letting me know when it's fixed so I can run a commit to rescan?

@ItsPugle when its fixed I'll close this issue 👍

@ItsPugle I've done some testing around, and my original thought on the branch issue doesn't seem to be the cause of the problem. I was able to push a file to a new branch without it being in a PR, and the issue was generated as expected.
Would you be able to copy-paste some of the relevant code in your file, and your configuration file for todo (if you have one)? It might be something in there that's causing this.

@JasonEtco I can just add you to the repository, if you're okay with that? Of course, I 100% expect you to keep any documents and files you view under strict confidentiality.

@ItsPugle I appreciate the offer, but I don't feel comfortable agreeing to confidentiality for something I know nothing about. It would be easier to just post the relevant bits of your code (just the todo comment and its surrounding lines). I'm guessing its something related to that; I've looked through my logs and there are no errors, so I'm guessing there's something in the title of your todo that's throwing it off.
Does it have any special characters (+!;'?"|/[]{}()@#$%^&*)?

That's 100% a-ok :) Here's the involved code:
## Present
<!--
@todo Can @AskRockStars and Spotify Community be combined?
-->
\> Rising Star for [@AskRockStars](https://twitter.com/AskRockStars/)

## Gaming
<!--
@todo Decide on what I'm going to write here
@body Should I write an overview (not a list item), or a list of all things?
-->
\> {item}

This todo worked though, from an earlier commit:
description: ItsPugle
# @todo Create Google Analytics Code
# @body Get in touch with Google Support about "Permission Denied" thing
google_analytics: [Your Google Analytics tracking ID]

I just pushed a change that added two spaces to the end of the second example:
## Gaming
<!--
@todo Decide on what I'm going to write here
@body Should I write an overview (not a list item), or a list of all things?
-->
\> {item}

and todo created an issue for it, but not the other todo that didn't work.

Oh interesting; that means its very likely the @ symbol in your other todo. I haven't had a chance to fix this yet, but now I have a good place to start, thanks @ItsPugle ! 👍

Happy to help! Let me know if you want me to test/try something out for you :)

Just wrote a quick test case and the whole special characters thing was it! I'll close this via a PR, thanks for debugging it with me ❤

Ah okay! That's great that you could find the issue!
Let me know when you push the patch!

@ItsPugle this still isn't the reason for your non-existant todo. I wrote a test in #73 and it passes; I even tried giving it your specific title. Basically, I'm stumped. I'm going to close this out since I can't reproduce it, but if you see it again let me know.


consider adding a new suite of functions named EventLoopFuture.flatMapWithEventLoop or so

Sometimes in NIO programs you're stuck in this situation:
doSomething().flatMap { result in
    guard iLikeTheShape(of: result) else {
        struct IDontLikeIt: Error {}
        //     vvvvvvvvv---- where do I get that from?
        return eventLoop.makeFailedFuture(IDontLikeIt())
    }
    return doMore()
}

that's all okay, but where do we get eventLoop from? In many cases, it's passed in but that's not always the case, I would like a suite of new functions (everything that allows to return -> EventLoopFuture<...>) which are named like the existing ones with an additional WithEventLoop at the end. Such that I can write
doSomething().flatMapWithEventLoop { result, eventLoop /* <- thank you */ in
    guard iLikeTheShape(of: result) else {
        struct IDontLikeIt: Error {}
        return eventLoop.makeFailedFuture(IDontLikeIt())
    }
    return doMore()
}

The implementation would be trivial:
extension EventLoopFuture {
    @inlinable
    public func flatMapWithEventLoop<NewValue>(file: StaticString = #file, line: UInt = #line, _ callback: @escaping (Value, EventLoop) -> EventLoopFuture<NewValue>) -> EventLoopFuture<NewValue> {
        let eventLoop = self.eventLoop
        return self.flatMap(file: file, line: line) {
            callback($0, eventLoop)
        }
}

Sounds reasonable to me.

Good first starter issue?

Yup.

I think it would be even more convenient if we had a version of flatMap which allows the closure to throw. Combine prefixes those methods with try e.g. tryMap.
We could also add a similar function for flatMap:
extension EventLoopFuture {
    public func tryFlatMap<NewValue>(
        transform: @escaping (Value) throws -> EventLoopFuture<NewValue>
    ) -> EventLoopFuture<NewValue>
}

which could be used like this:
doSomething().tryFlatMap { result in
    guard iLikeTheShape(of: result) else {
        struct IDontLikeIt: Error {}
        throw IDontLikeIt()
    }
    return doMore()
}

This would remove the need for the eventLoop entirely.
Note: We already have a method called flatMapThrowing which behaves more like a tryMap, IMHO.

@dnadoba I'm not sure this would be a valuable addition, since it mixes synchronous and asynchronous patterns for adopters. It would be really hard to explain to users, why this program would crash:
doSomething().tryFlatMap { result in
    guard iLikeTheShape(of: result) else {
        struct IDontLikeIt: Error {}
        throw IDontLikeIt()
    }

    let result1 = try httpClient.get("https://foo.org").wait()
    let result2 = try httpClient.get("https://foo.org").wait()

    return doMore(with: result1, result2)
}

If you are required to return a future and can not throw, people are less likely to wait in a flatMap.
Of course people can also await in today's flatMap. But I think the very clear divide in a sync and an async callback is vital for users to understand the fundamentals, of waiting for another event.
Lastly, the addition you mention above would mean at least two internal promises with kind of weird semantics. The first promise would only be created to catch the thrown error or void if the user actually returns a future. If the user returns a future we have created a void promise just in case.
The issue you illustrate above can be accomplished by a throwingMap:
doSomething().flatMapThrowing { result in
    guard iLikeTheShape(of: result) else {
        struct IDontLikeIt: Error {}
        throw IDontLikeIt()
    }
    return result
}.flatMap { checkedResult in
    return doMore()
}

... I would like a suite of new functions (everything that allows to return -> EventLoopFuture<...>) which are named like the existing ones with an additional WithEventLoop at the end.

I'm not sure if this adds enough value to justify adding a new variant of every function that returns an ELF: that's quite a lot of new API!

I appreciate the problem though, I've hit this a few times and usually end up just grabbing the event loop from doSomething() which is a little cumbersome.
let f = doSomething()
let eventLoop = f.eventLoop
return f.flatMap { result in
    guard iLikeTheShape(of: result) else {
        struct IDontLikeIt: Error {}
        return eventLoop.makeFailedFuture(IDontLikeIt())
    }
    return doMore()
}

The other concern I have is that I'm not sure that the average user would run into the problem and think 'ah yes, I need an event loop to return a failed future here'.
I wonder if there's space for an API to filter values like we have for unwrap:
public func filter(orError: Error, isIncluded: @escaping (Value) -> Bool) -> EventLoopFuture<Value> {
  // ...
}

Of course this incurs an extra allocation which is unfortunate. We could supplement this with something that provides either the future or the future and its event loop to a closure:
extension EventLoopFuture {
  @inlinable
  public func withSelf<NewValue>(_ transform: (EventLoopFuture<Value>) -> EventLoopFuture<NewValue>) -> EventLoopFuture<NewValue> {
    return transform(self)
  }
}

doSomething().withSelf {
  let eventLoop = $0.eventLoop
  return $0.flatMap {
    guard iLikeTheShape(of: result) else {
        struct IDontLikeIt: Error {}
        return eventLoop.makeFailedFuture(IDontLikeIt())
    }
    return doMore()
  }
}

This is a bit of an awkward API but it would be just one addition as opposed to a suite of additions.

@weissi points out this would only actually be three additions, which is totally fine.

... I would like a suite of new functions (everything that allows to return -> EventLoopFuture<...>) which are named like the existing ones with an additional WithEventLoop at the end.

I'm not sure if this adds enough value to justify adding a new variant of every function that returns an ELF: that's quite a lot of new API!

It's every function that has a callback that returns an ELF only. So only func foo(_ body: @escaping (...) -> EventLoopFuture<...>) -> EventLoopFuture<...>. We're talking about 3 new functions if I counted correctly.

I appreciate the problem though, I've hit this a few times and usually end up just grabbing the event loop from doSomething() which is a little cumbersome.

Yes, but users will reach for group.next().makeXYZ() which is much worse but easier to reach.

The other concern I have is that I'm not sure that the average user would run into the problem and think 'ah yes, I need an event loop to return a failed future here'.

Very hard disagree. In the product I work on, there are literally hundreds of lines which contain roup.next().make(Succeeded|Failed)Future.

[...]
This is a bit of an awkward API but it would be just one addition as opposed to a suite of additions.

One could also argue that we messed up the existing ones. How are you supposed to generate a future if you can't without an EventLoop?


Updating Envelope Message / Misc Mailbox Change

Removing the error checking for newly added participants on a contract execution for an existing scope where the new participants don't exist on the scope yet. Instead, automatically appending these new participants to a MsgAddScopeDataAccessRequest that the envelope returns to add them.
Minor change around the mailbox that allows configuration of the polling time.

Codecov Report

Merging #61 (306af6a) into main (868f440) will increase coverage by 0.27%.
The diff coverage is 81.25%.

@@             Coverage Diff              @@
##               main      #61      +/-   ##
============================================
+ Coverage     78.94%   79.22%   +0.27%
- Complexity      164      175      +11
============================================
  Files            20       20
  Lines          1216     1232      +16
  Branches        143      150       +7
============================================
+ Hits            960      976      +16
+ Misses          175      173       -2
- Partials         81       83       +2

Impacted Files
Coverage Δ

...n/kotlin/io/provenance/scope/sdk/SessionBuilder.kt
90.51% <ø> (+0.30%)
:arrow_up:

.../src/main/kotlin/io/provenance/scope/sdk/Client.kt
62.82% <33.33%> (-0.41%)
:arrow_down:

.../kotlin/io/provenance/scope/sdk/ExecutionResult.kt
95.65% <92.30%> (-0.65%)
:arrow_down:

...n/kotlin/io/provenance/scope/ContractValidators.kt
57.89% <0.00%> (-8.78%)
:arrow_down:

...ne/src/main/kotlin/io/provenance/scope/Function.kt
96.15% <0.00%> (-1.85%)
:arrow_down:

...tlin/io/provenance/scope/sdk/ContractSpecMapper.kt
72.86% <0.00%> (-0.37%)
:arrow_down:

...main/kotlin/io/provenance/scope/ContractWrapper.kt
82.66% <0.00%> (+7.32%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 868f440...306af6a. Read the comment docs.


Allow to set the length of shadows

Currently in box2dlights a box2d body either casts a shadow or not, there is no in-between. However, in my application I need to specify the length of the shadow for each object, depending on the objects height (in order to simulate a 3d effect).
For example, if the object that would cast the shadow is tall then the shadow is long. But if the object is small, not as tall as the light source height, then the shadow would be shorter.
Would it be hard to implement such a feature in box2dlights?

I was thinking about that. How about optionally using the user data to store the "tall" value of the object, if you want shadows with different length. In that case you can't use the user data for something else, but since it's optionally you can choose not to use it as well...

I also was thinking about that, however it will be a rare case, in most of engines the userData is often used to connect physics body with other engine elements (like graphical objects). But I will try to see what we can do.

Thank you very much for your efforts!

Hi @haimat,
I have a question - as I understand the "tall" is not enough, we will need a lights "height" position also? Or in your idea of implementation all lights are on the same height?
Also, it's quite hard for me, as I'm not expert, but seems I was able to find some way to do it:

This is draft implementation of shadows for Directional lights. Will try other light too, but it will be even harder :) Need to find a way to handle ray collision overlapping, e.g. if the light is blocked by fixture, but the fixture next to it is far enough to drop the shadow from this light also... it's a pain my head :) sorry for my english, btw.

Wow, that looks already awesome -- great work! Now if you could manage to get that working for point lights too, I would be super happy :-)
To your question: Of course you are right, to be more realistic we would need not only the height of the fixture ("tall"), but also the height of the light source. This would be nice, but having only one value to control the length of the shadow would be better then what we have now.
So in other words: having both, "object height" (tall) and "light height" would be great, but having only "object height" (one valiue) would also be fine!

Hi @haimat,
Just interesting, is it possible for you to link not the pre-build jar library but add the box2dlight source project as dependency?

You mean for testing your changes?
Sure, no problem, I can do that!

That would be great! You can use my fork, its could be found by referenced commit above. Current implementation looks good on screenshots, but isn't optimised and I feel its buggy. To use limited height lights you need do 3 things:

Enable experimental mode using rayHandler.setPseudo3dLight(true)
Set fixtures user data to new LightData(height) objects
Set PointLight.setHeight(lightHeight)

And currently it works only for PolygonShape fixtures, tell me what other Shape types you are using to prioritise the implementation :-)

So, I did what you suggested, using your fork and enabling the new pseudo-3d feature. It does work partially: when the fixture is within the point light range, then it seems to work fine. However, when I move the light away from the fixture, then the shadow becomes bright, like light. Hard to explain, so have a look at these screenshots.
This is correct:

And this is wrong:

Oh... could you please share your sample box2dlights setup configuration in your project and sample height values for this issue?

fixture.setUserData(new LightData(1.0));

pointLight.setHeight(2.0);
pointLight.setColor(Color.WHITE);
pointLight.setDistance(10.0);  // in my world-dimensions, 1 unit = 64 pixels

RayHandler.setGammaCorrection(false);
rayHandler.useDiffuseLight(true);
rayHandler.setBlurNum(3)
rayHandler.setAmbientLight(0.2, 0.2, 0.2, 1.0);

Seems I've messed up some blending and we had different ambient lights set, and on my (0f, 0f, 0f, 0.5f) this wasn't visible - both blendings gave the same good result. I've pushed the possible fix, give it a try when you have chance! And also, thanks for your time :)

Thanks for your last update, it looks already pretty great! There is a bug however -- again, hard to explain. The shadows cast by the polygons now look as if you would look down into an empty cylinder, i.e. as if the polygon was hollow. Additionally, for every polygon I tried in different shapes and with different numbers of vertices the shadow looks as if one side of the polygon would be missing.
With a bit distance the shadow looks good:

Moving the light closer makes the polygon appear hollow:

One border/side seems to be missing (not casting any shadow):

And again:

But it looks really good already. Seems you are quite close :-)

I've tried to do more testing this time :) you can sync with new version and look at it again.

Ohh... I've added the userData field to LightData - it solves one general limitation:
    // If it was in your project:
    fixture.setUserData(customData);
    ...
    // Still could be used with modification as:
    fixture.setUserData(new LightData(customData, lightHeight);

So you still could use custom data even with pseudo-3d mode enabled, just adds a bit more pain in getting it later :) Don't know why I haven't thought about it earlier.

Hmm, I'm afraid it stills seems a bit buggy. Sometimes (when the light source is in a certain angle to the polygon) a polygon side closer to the light source casts a shadow onto the polygon, that doesn't look right.

I've updated code, should fix this issue. Btw, non-convex polygons still are not supported correctly, but should work fine with convex shapes.

Thank you very much. It's already looking awesome! There are two things I noticed though:

When the light source is inside the polygon, then the closest side of it does not cast any shadow
It seems to make no difference whether the light source height > polygon fixture height

For the 2nd point I also think the the inside of the poygon should be in shadow if the polygon fixture height is > then the light source height. What do you think about that?

I will check the inside case later. For now I've fixed some shadow length computation, and it will be great if you will try following mode:
rayHandler.setPseudo3dLight(true, true);

The second param if true - enables some shadow color interpolation, which looks a bit different - but I don't know which is better :)
Regarding to shadowing the polygon in case of light height < then fixture height sounds reasonably. I will check it also.

I have tried the new parameter to interpolate the shadows.
It indeed looks smoother = nicer now :-)

Hi @haimat,
I've found that current implementation is a bit awful :) it doesn't work for colored lights as usual box2dlight do. I need to re-implement the core, and also due to memory and performance optimization I'm going to move all the code related to this feature to a separate sub-package.
The following development will continue on the libgdx "pseudo3d" branch and I will talk with libgdx team if its possible to move box2dlights nightlies build to this branch also so that you could easily update and test it during development via Gradle and smoothly move onto it when it will be merged to master branch. Also after the first commit to the "pseudo3d" branch it will be better to create issues/enhancements in a separate threads.
P.S. This will take some time, but hope soon you will be able to use it again and it will be event better :)
Thanks for your idea and time!

Thank you very much for your feedback. So will you answer here in this thread when you have a new version ready for testing, or somewhere else?

Yes, I will notify you in this thread :)

Hi, have there been further updates regarding the pseudo 3d lights?

I'm quite a busy currently on my job, having a few free time. Even now at holiday I'm writing this from work :) you can check it's current status at box2dlights "pseudo3d" branch, unfortunately there is no built jar could be provided currently, but you can still try it from sources.

This should be in Libgdx official release good job guys!

Any plan of a pull request to join it to the master branch? :D

It might be, but currently the pseudo3d branch is "28 commits behind master" which sounds a bit frightening for the merge :) This will take much work...

Oohh... this can be sooo great if integrated in the master branch ＞＿＜

That looks so promising and interesting feature. Any hope to see it merged with master someday?

Hi, i'm quite new to github and box 2d lights. How can i implement your code with existing box 2d light library? I took the code from https://github.com/rinold/box2dlights, but i didn't see pointlight.setHeight or pseudo3d. Any help is appreciated.

It's on the 'pseudo3d' branch at the moment.

Thanks. I have few questions related box 2d directional light and shadowing. I'm using a conelight and a directional light with changing alpha  to simulate day and night cycles. When i set conelight active, i'm facing flickering light borders. It does'not look so good. Do you have any idea how can it be solved?
Another issue is, i setrayHandler.setPseudo3dLight(true, true); and rays pass through the bodies, why does it happen?

You can try to increase the rays number for the lights - it's usually the second argument in co structor (called 'rays') to reduce flickering.
Regarding to the pseudo3d lights you should use the box2dlights.p3d.P3dLightManager (pseudo3d ray handler analogue) to handle the P3d lights.
Sorry for any inconveniences, but I'm currently writing from phone, having no access to my laptop (hope will get back in a week), thus unfortunatelly can't show code examples now.

I'm afraid to ask but... I guess this pseudo3d branch is dead? given the number of updates lately...

Hopeful for this addition to the main branch. This'll really add icing to the cake

How can I integrate this branch with my Android Studio with LibGDX project?

Unfortunately, for now the only way is to download it's sources and build jar on your own, or add downloaded project as dependency.

Hi @rinold, i needed this pseudo3d from a top down persective and i felt so free to take your code, edit it and merge it with the current box2d-lights Version. Good work. I only used the directional light. You can visit the code on my github repo. Maybe it is to specific for my needs but it works in my case. Maybe we can merge this feature into the 1.6 version? If you run the tests (maybe you have to change the resource paths in the Test class) you can see a full sun cyclus when activated the directional light.


ObjectMapper + CoreData ????????

can not use ObjectMapper + CoreData ?????  like this
class Model: NSManagedObject, Mappable {
@NSManaged var name = ""
required convenience init?(_ map: Map) {
    self.init()
}

func mapping(map: Map) {
    name <- map["name"]
}

}
when I use above, crashed with -[app.Model name]: unrecognized selector sent to instance 0x7fa5c360ff40

Sorry I do not use CoreData much so cannot be of much help with this... are you still having the problem?

YES, but i will use realm instead CoreData, because CoreData has no porpule framework with swift


added chat messaging

1.added message update listener.
2.added chat messaging feature

@Arunshaik2001 there are merge conflicts in the hms_role.dart file, please resolve it first


[Ameba] Update docker image for pigweed-app

Problem

Update docker image for pigweed-app

Change overview

Update docker file for ameba
Update docker version

Testing
Tested compilation

fast track: platform specific change


Change how movement/punching works

Hero should always be animating for movement. Clicking/tapping anywhere on the map will simply set the movement direction/velocity, and hero will continue to try to move that way until player clicks on screen again. Once moving, hero will punch continuously (?). This last part would create a need for alternating side punching probably. Maybe we could just mirror it somehow? Will this require additional frames or just adjustment of the existing animations?

Pretty sure I can get away with just playing with animation sequences using existing frames. Will have to figure out how to handle hitboxes for walking vs. punching.


Do not import deprecated bindings

Recent CI logs on nightlies show hundreds of spurious warnings because the package imports deprecated bindings, see, e.g., https://travis-ci.org/JuliaMath/FFTW.jl/jobs/254462935#L785  While these messages are going to cleared, they are an indication that after https://github.com/JuliaLang/julia/pull/22763 deprecated bindings shouldn't be imported at all.  An example of fix for SpecialFunctions.jl is at https://github.com/JuliaMath/SpecialFunctions.jl/pull/36

I actually think this will be mostly fixed by https://github.com/JuliaMath/AbstractFFTs.jl/pull/6.


having some sort of semver for git dependencies

If 2 crates import a dependency from git and it's even a bit different cargo treats them as 2 unrelated crates.
let's say crate A has this toml:
[package]
name = "a"
version = "0.0.1"

[dependencies]
serde = { git = "https://github.com/serde-rs/serde.git", features = ["serde_derive"] }

and contains this struct: #[derive(Serialize)] struct Arr([u8; 32])
crate B has this toml:
[package]
name = "b"
version = "0.0.1"

[dependencies]
a = { path = "../a" }
serde = { git = "https://github.com/serde-rs/serde.git", rev = "fd9d334", features = ["serde_derive"] }

and contains this:
use a::Arr;
#[derive(Serialize)]
struct BetterArr(Arr)

This will complain that Arr doesn't implement Serialize.
even though technically they point to the exact same thing.
and even if they're not pointing to the same thing I would've hoped there would be a way for me to do something similar to crate = "=1.0.1" and lock to an exact revision/tag/branch.
Thanks!

(Anyone running into this issue should probably look at #7497, which collects several related issues.)


[HW] Prakash Choudhary

**Name:** **Email:**
**Linkedin Profile:**
Attach the homework step I screenshots below:

Attach the homework step II screenshots below:
-----------------------------------------

Made sampledoc model and spring

@HadesArchitect please let me know when you get a chance to review it.

Hey there Prakash, thank you for your submission. Nice job getting the app correctly configured and deployed. The only thing we are missing is step 2 of the homework.
2. Learn more about spring in the course and complete scenario Build a Spring Boot REST Service at the bottom of the page and make a screenshot of the "congratulations" page.
Please post an image of the "completed" screen once finished

Also, please provide your name and email so we can issue you a badge once complete. :D

Name - Prakash Choudhary
email-<EMAIL_ADDRESS>

@SonicDMG  @HadesArchitect  I am interested in contributing to DataStax. Please help me in how I can contribute or any kind of work I can do to add value to the developer community.

Hey Prakash Choudhary, great job! Congrats! Here is your badge! https://api.badgr.io/public/assertions/Ub3CntfXTb2HjFgA97zogA

@SonicDMG @HadesArchitect I am interested in contributing to DataStax. Please help me in how I can contribute or any kind of work I can do to add value to the developer community.

Hey there, ack'ing this. I've passed your info along to the team to see if there is anything we might have on this front.

@SonicDMG @HadesArchitect Thank you. I am closing this thread


[CHALLENGE SUBMISSION]

Please provide the following information for your submission:
What is your name? (First, Last)
Oleksandr Sudom
Where is your GitHub Repo ?(With your challenge solution)
https://github.com/Alex0M/Terraform-On-Azure-Workshop/tree/master/challenge5
What Challenge is this for? (Challenge number between 1 and 5)
5
(Optional) Anything else we should know? e.g., language used, location, blog post?
Consul cluster up and running, but the task about federate the Consul clusters is not clear for me... :(

Thanks for participating in the #TerraformOnAzure coding challenges!
We have attached your very special Astronaut Badger background, for you to use in all your upcoming video calls!
And to recognize you for your learning, you can collect your digital Azure Heroes badge on your mobile (you may need to install the Enjin wallet to collect it). Make sure to collect it today!
https://beam.enjin.io/claim/d32599695c8c28cfc24820492080fac0
Keep on learning and sharing!
Greetings from the #TerraformOnAzure team,
Suzanne, Floor, Erik & Nick


Q: Where is streamrecording saved?

Hi, finally got round to try v13 on my v3.6 rm2. Love it :)
I just couldn't find any recording files after trying out the new feature. Where should they be and how are they named?
Running Firefox on a macOS 12.

Normally, when you stop, the recording should be downloaded automatically (so it is in my Downloads folder)
I did not test it on firefox.


Bug: Init function params are always overriden with undefined environment variables

Prerequisites

[X] I have searched the repository’s issues and Kinde community to ensure my issue isn’t a duplicate
[X] I have checked the latest version of the library to replicate my issue
[X] I have read the contributing guidelines
[X] I agree to the terms within the code of conduct

Describe the issue
Hi, I’m currently trying to integrate the management-api-js on a project to load the user's profile data.
In my code im doing something similar to this:
import { Users, init } from "@kinde/management-api-js";

init({
    kindeDomain: "https://myurl.kinde.com",
    clientId: "myclientId",
    clientSecret: "myclientsecret",
});

export async function GET() {
  const { users } = await Users.getUsers();
  return NextResponse.json({ message: "success", users });
};

And i get this error: Error: kindeDomain or env KINDE_DOMAIN is not set
i took a look at the init function code and i think the problem is the _merge function that always overrides the init function params with the environment variables even if those don't exist.
function _merge(target: object = {}, ...objects: object[]): object {
  return Object.assign(target, ...objects.filter(Boolean));
}

console.log(_merge({ a: 1 }, { b: 2 }, { b: undefined }));

//logs: { a: 1, b: undefined }

Library URL
https://github.com/kinde-oss/management-api-js
Library version
0.6.0
Operating system(s)
macOS
Operating system version(s)
Somona
Further environment details
No response
Reproducible test case URL
No response
Additional information
No response

Thanks for raising. I will get this resolved.


AddKruskal'sAlgorithm

Program of Kruskal's Algorithm in C++

can i contribute this algorithm

Please make the changes requested.

okay..i will do the job