Make consumer group prefix on statsd metrics optional.

The default is the current behavior, which is to include the consumer
group as the overall prefix. Disable by setting
secor.prefixWithConsumerGroup to false.

Looks good to me.


Add [neveredit] rule to protect write once files from modification

Hackathon project - to add new rule to protect files that should not get edited.

/trunk skip-check

I also tried to tackle the "do not modify" rule and add that to toolbox.


storcon: re-attach can race with heartbeats and result in tenants not getting re-attached

This was observed in staging where the infra team stopped the pageservers for around 20 minutes. When the pageserver restarted, the reattach response was processed before the heartbeats marked the node active. The heartbeats detected the node coming back online (they store node state separately), but this inhibited the heartbeat handler from re-attaching the tenants (Service::node_activate_reconcile)
https://neondb.slack.com/archives/C060CNA47S9/p1718270673517979

I fixed this manually in staging by restarting the storage controller.


Static typing libraries not working correctly with python 3.11.2

Hi
while running the install.py script/ pip install requirements.txt .
My Python version is upto date 3.11.2 and so is my C compiler

@Kasambx Thanks for raising the issue! We did not test Python 3.11 yet, and the version of mypy is likely not up to date.
We should also make the installation of development packages optional.


Combine standard OpenGL rendering with GS model

In order to render some effects such as camera locations, current selector area, or even other objects, we want to allow other rendering to mesh seamlessly with the GS forward render. This issue is the backend work to support rendering standard OpenGL and allowing the model to still be rendered in that same scene.
Will be used for other issues like #14 #28 #26 etc.

Started work with an off-screen PyOpenGL + GLUT (invisible) window which can be rendered to. The idea is to use that canvas to draw the OpenGL related items to before the model forward render. Then, the OpenGL rendered data (RGB + A + Depth) are all passed into the GS rendering backend with compositing working properly given the other OpenGL render.  Instead, the RGBA from the OpenGL render will be composited, and splatting will only continue if the alpha is not saturated yet. Should work well with only a little overhead on backend.
So far I can render a cube outline using PyOpenGL to a hidden window and also obtain the depth buffer (normalized 0.0-1.0 for near/far plane), and obtain both these arrays as numpy/standard float arrays for passing to the GS CUDA backend.

Next steps

Implement CUDA rasterizer updates to properly draw gaussians on top of/behind the given RGBAD buffers given to the rasterizer.
Update OpenGL renderer to maintain a consistent view/projection matrix to the GS renderer model such that the buffer objects are pixel-wise accurate with each other per frame.
Build out the OpenGL renderer with support for arbitrary objects (wireframe rectangles for cameras, images as textures for debug viewing, etc) so the main code can call high-level functions easily.

Useful Links

https://stackoverflow.com/questions/52036176/pyopengl-get-depth-map-of-drawn-image

Updates:

Creates a frameBufferObject (FBO) to render to, which ignores the frame buffer for the actual window output. Truly "off-screen" rendering. Able to read the colors and depths still
Uses glLoadMatrixf to load the exact modelview matrix and projection matrix, so the OpenGL camera has the exact same parameters as the gaussian splatting one. Had to reverse the z-sign for the projection matrix, seems GS and OpenGL use opposite z-signs (camera forward)
Reorganized the OpenGL_renderer into an object that can be rendered with the render_cam object we use to render the GS model. Should be trivial to add opengl_renderer.render(render_cam) in the main render thread.

Updates:

Moves OpenGL context to glfw. More modern and nicer than GLUT it seems. Plus, GLFW actually allows a fully hidden window for truly off-screen rendering.
Properly renders depth + rgba in opengl for use later
Adjusted CUDA code for GS to accept an RGBA + depth buffer. While traversing splats (per pixel), will properly composite OpenGL rendered objects behind/in front of splats as needed.
OpenGL camera is now the exact same as the GS model at all times.

Left TODO:

Still need to build out backend OpenGL objects for visualization. At the moment, only a wireframe cube is available. Will begin with this for the selector editor


fix: Adds initial-scale to viewport meta in app.html

#86  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @jhechtf and the rest of your teammates on  Graphite

Merge activity

Aug 7, 7:15 PM EDT: @jhechtf started a stack merge that includes this pull request via Graphite.


[SPARK-7112][Streaming][WIP] Add a InputInfoTracker to track all the input streams

Test build #30920 has started for   PR 5680 at commit 28d668f.

Test build #30920 has finished for   PR 5680 at commit 28d668f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.
This patch does not change any dependencies.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30920/
Test PASSed.

Test build #30979 has started for   PR 5680 at commit 28d668f.

@jerryshao  This is a decent patch for the assumed design, but I mentioned in the parent JIRA https://issues.apache.org/jira/browse/SPARK-7111 that the design is not great. The "direct API" is a name we came up to differentiate new Kafka API from the old one receiver-based one, and logically every subclass of InputDStream that is not a ReceiverInputDStream is  a direct stream. So further separating out InputDStreams as direct stream and non-direct stream (beyond the receiver stream) is a bad idea. In the end, the goal is to enable all the streams, irrespective of its type, to report information about its input to the infra. And there should be a single common way / code path for doing it for all input streams, with some customization / override for receiver input streams (as unlike non-receiver-streams, all receiver streams has a common way of reporting block info).
I have been thinking about this since last night and I will post a very rough design doc on the JIRA shortly. Please follow up on the JIRA.

I have posted a different design on the JIRA. Please take a look.

Yes, I will take a look at the design doc, thanks a lot for your comments.

Build triggered.

Build started.

Test build #31397 has started for   PR 5680 at commit 3ad00d9.

There are merge conflicts! Please merge master!

Yes, I will do this, please take take a look at the whole design, thanks a lot :)

Test build #31397 has finished for   PR 5680 at commit 3ad00d9.

This patch fails MiMa tests.

This patch does not merge cleanly.

This patch adds the following public classes (experimental):

case class InputInfo(batchTime: Time, inputStreamId: Int, numRecords: Long)

This patch adds the following new dependencies:

tachyon-0.5.0.jar
tachyon-client-0.5.0.jar

This patch removes the following dependencies:

spark-unsafe_2.10-1.4.0-SNAPSHOT.jar
tachyon-0.6.4.jar
tachyon-client-0.6.4.jar

Build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31397/
Test FAILed.

Merged build triggered.

Merged build started.

Test build #31407 has started for   PR 5680 at commit 17fa251.

Hi @zsxwing, you also changed a lot on this part, would you please take a look at this, thanks a lot.

Merged build triggered.

Merged build started.

Test build #31408 has started for   PR 5680 at commit 8325787.

Test build #31407 has finished for   PR 5680 at commit 17fa251.

This patch fails to build.

This patch merges cleanly.

This patch adds the following public classes (experimental):

case class InputInfo(batchTime: Time, inputStreamId: Int, numRecords: Long)

This patch does not change any dependencies.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31407/
Test FAILed.

Jenkins, retest this please.

Merged build triggered.

Merged build started.

Test build #31411 has started for   PR 5680 at commit 8325787.

Test build #31408 has finished for   PR 5680 at commit 8325787.

This patch fails Spark unit tests.

This patch merges cleanly.

This patch adds the following public classes (experimental):

class Param[T] (val parent: Params, val name: String, val doc: String, val isValid: T => Boolean)

class DoubleParam(parent: Params, name: String, doc: String, isValid: Double => Boolean)

class IntParam(parent: Params, name: String, doc: String, isValid: Int => Boolean)

class FloatParam(parent: Params, name: String, doc: String, isValid: Float => Boolean)

class LongParam(parent: Params, name: String, doc: String, isValid: Long => Boolean)

class BooleanParam(parent: Params, name: String, doc: String) // No need for isValid

case class ParamPair[T](param: Param[T], value: T)

class KMeansModel (

trait PMMLExportable

case class Sample(

case class Sample(

case class InputInfo(batchTime: Time, inputStreamId: Int, numRecords: Long)

This patch removes the following dependencies:

RoaringBitmap-0.4.5.jar
activation-1.1.jar
akka-actor_2.10-2.3.4-spark.jar
akka-remote_2.10-2.3.4-spark.jar
akka-slf4j_2.10-2.3.4-spark.jar
aopalliance-1.0.jar
arpack_combined_all-0.1.jar
avro-1.7.7.jar
breeze-macros_2.10-0.11.2.jar
breeze_2.10-0.11.2.jar
chill-java-0.5.0.jar
chill_2.10-0.5.0.jar
commons-beanutils-1.7.0.jar
commons-beanutils-core-1.8.0.jar
commons-cli-1.2.jar
commons-codec-1.10.jar
commons-collections-3.2.1.jar
commons-compress-1.4.1.jar
commons-configuration-1.6.jar
commons-digester-1.8.jar
commons-httpclient-3.1.jar
commons-io-2.1.jar
commons-lang-2.5.jar
commons-lang3-3.3.2.jar
commons-math-2.1.jar
commons-math3-3.4.1.jar
commons-net-2.2.jar
compress-lzf-1.0.0.jar
config-1.2.1.jar
core-1.1.2.jar
curator-client-2.4.0.jar
curator-framework-2.4.0.jar
curator-recipes-2.4.0.jar
gmbal-api-only-3.0.0-b023.jar
grizzly-framework-2.1.2.jar
grizzly-http-2.1.2.jar
grizzly-http-server-2.1.2.jar
grizzly-http-servlet-2.1.2.jar
grizzly-rcm-2.1.2.jar
groovy-all-2.3.7.jar
guava-14.0.1.jar
guice-3.0.jar
hadoop-annotations-2.2.0.jar
hadoop-auth-2.2.0.jar
hadoop-client-2.2.0.jar
hadoop-common-2.2.0.jar
hadoop-hdfs-2.2.0.jar
hadoop-mapreduce-client-app-2.2.0.jar
hadoop-mapreduce-client-common-2.2.0.jar
hadoop-mapreduce-client-core-2.2.0.jar
hadoop-mapreduce-client-jobclient-2.2.0.jar
hadoop-mapreduce-client-shuffle-2.2.0.jar
hadoop-yarn-api-2.2.0.jar
hadoop-yarn-client-2.2.0.jar
hadoop-yarn-common-2.2.0.jar
hadoop-yarn-server-common-2.2.0.jar
ivy-2.4.0.jar
jackson-annotations-2.4.0.jar
jackson-core-2.4.4.jar
jackson-core-asl-1.8.8.jar
jackson-databind-2.4.4.jar
jackson-jaxrs-1.8.8.jar
jackson-mapper-asl-1.8.8.jar
jackson-module-scala_2.10-2.4.4.jar
jackson-xc-1.8.8.jar
jansi-1.4.jar
javax.inject-1.jar
javax.servlet-3.0.0.v201112011016.jar
javax.servlet-3.1.jar
javax.servlet-api-3.0.1.jar
jaxb-api-2.2.2.jar
jaxb-impl-2.2.3-1.jar
jcl-over-slf4j-1.7.10.jar
jersey-client-1.9.jar
jersey-core-1.9.jar
jersey-grizzly2-1.9.jar
jersey-guice-1.9.jar
jersey-json-1.9.jar
jersey-server-1.9.jar
jersey-test-framework-core-1.9.jar
jersey-test-framework-grizzly2-1.9.jar
jets3t-0.7.1.jar
jettison-1.1.jar
jetty-util-6.1.26.jar
jline-0.9.94.jar
jline-2.10.4.jar
jodd-core-3.6.3.jar
json4s-ast_2.10-3.2.10.jar
json4s-core_2.10-3.2.10.jar
json4s-jackson_2.10-3.2.10.jar
jsr305-1.3.9.jar
jtransforms-2.4.0.jar
jul-to-slf4j-1.7.10.jar
kryo-2.21.jar
log4j-1.2.17.jar
lz4-1.2.0.jar
management-api-3.0.0-b012.jar
mesos-0.21.0-shaded-protobuf.jar
metrics-core-3.1.0.jar
metrics-graphite-3.1.0.jar
metrics-json-3.1.0.jar
metrics-jvm-3.1.0.jar
minlog-1.2.jar
netty-3.8.0.Final.jar
netty-all-4.0.23.Final.jar
objenesis-1.2.jar
opencsv-2.3.jar
oro-2.0.8.jar
paranamer-2.6.jar
parquet-column-1.6.0rc3.jar
parquet-common-1.6.0rc3.jar
parquet-encoding-1.6.0rc3.jar
parquet-format-2.2.0-rc1.jar
parquet-generator-1.6.0rc3.jar
parquet-hadoop-1.6.0rc3.jar
parquet-jackson-1.6.0rc3.jar
protobuf-java-2.4.1.jar
protobuf-java-2.5.0-spark.jar
py4j-<IP_ADDRESS>.jar
pyrolite-2.0.1.jar
quasiquotes_2.10-2.0.1.jar
reflectasm-1.07-shaded.jar
scala-compiler-2.10.4.jar
scala-library-2.10.4.jar
scala-reflect-2.10.4.jar
scalap-2.10.4.jar
scalatest_2.10-2.2.1.jar
slf4j-api-1.7.10.jar
slf4j-log4j12-1.7.10.jar
snappy-java-<IP_ADDRESS>.jar
spark-bagel_2.10-1.4.0-SNAPSHOT.jar
spark-catalyst_2.10-1.4.0-SNAPSHOT.jar
spark-core_2.10-1.4.0-SNAPSHOT.jar
spark-graphx_2.10-1.4.0-SNAPSHOT.jar
spark-launcher_2.10-1.4.0-SNAPSHOT.jar
spark-mllib_2.10-1.4.0-SNAPSHOT.jar
spark-network-common_2.10-1.4.0-SNAPSHOT.jar
spark-network-shuffle_2.10-1.4.0-SNAPSHOT.jar
spark-repl_2.10-1.4.0-SNAPSHOT.jar
spark-sql_2.10-1.4.0-SNAPSHOT.jar
spark-streaming_2.10-1.4.0-SNAPSHOT.jar
spark-unsafe_2.10-1.4.0-SNAPSHOT.jar
spire-macros_2.10-0.7.4.jar
spire_2.10-0.7.4.jar
stax-api-1.0.1.jar
stream-2.7.0.jar
tachyon-0.6.4.jar
tachyon-client-0.6.4.jar
uncommons-maths-1.2.2a.jar
unused-1.0.0.jar
xmlenc-0.52.jar
xz-1.0.jar
zookeeper-3.4.5.jar

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31408/
Test FAILed.

Test build #31411 has finished for   PR 5680 at commit 8325787.

This patch fails Spark unit tests.

This patch merges cleanly.

This patch adds the following public classes (experimental):

case class InputInfo(batchTime: Time, inputStreamId: Int, numRecords: Long)

This patch does not change any dependencies.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31411/
Test FAILed.

Merged build triggered.

Merged build started.

Test build #31417 has started for   PR 5680 at commit 6682bef.

Test build #31417 has finished for   PR 5680 at commit 6682bef.

This patch passes all tests.

This patch merges cleanly.

This patch adds the following public classes (experimental):

case class InputInfo(batchTime: Time, inputStreamId: Int, numRecords: Long)

This patch does not change any dependencies.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31417/
Test PASSed.

Merged build triggered.

Merged build started.

Test build #31492 has started for   PR 5680 at commit 727264e.

Any comment for the current implementation ? @tdas

Merged build triggered.

Merged build started.

Test build #31499 has started for   PR 5680 at commit abd0036.

Test build #31499 has finished for   PR 5680 at commit abd0036.

This patch fails MiMa tests.

This patch merges cleanly.

This patch adds the following public classes (experimental):

case class InputInfo(batchTime: Time, inputStreamId: Int, numRecords: Long)

This patch does not change any dependencies.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31499/
Test FAILed.

Test build #31492 has finished for   PR 5680 at commit 727264e.

This patch passes all tests.

This patch merges cleanly.

This patch adds the following public classes (experimental):

case class InputInfo(batchTime: Time, inputStreamId: Int, numRecords: Long)

This patch does not change any dependencies.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31492/
Test PASSed.

Merged build finished. Test PASSed.

The approach is good! I am still looking at the details. There are some details that are going to to be changed singificantly in #5533 by @zsxwing . So I will look for general correctness.

I made some more comments. Would be great if you can get it done today and I can merge it in tomorrow. That will unblock #5533.

OK, I will try to get it done, thanks a lot for your comments.

Test build #738 has started for   PR 5680 at commit abd0036.

Merged build triggered.

Merged build started.

Test build #31527 has started for   PR 5680 at commit 339f854.

Test build #31527 has finished for   PR 5680 at commit 339f854.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31527/
Test PASSed.

Merged build finished. Test PASSed.

LGTM. Merging this. Thanks!


docker: use the journald driver as the default logging driver

https://docs.docker.com/engine/admin/logging/journald/#usage
It is better to use this log driver in a K8s installation, because nothing gets written to the disk of the VMs. You should then use something like fluentd to collect the logs.

This was actually already merged with #8 in https://github.com/infraly/k8s-on-openstack/commit/8d94b7ce7b566f0f9a2a27d4c44903a648b4e48f, closing this PR.


[Feature] primal.net on nostr section

Feature description
Add https://primal.net/ on Nostr section
Describe the solution
Add https://primal.net/ on Nostr section
Describe alternatives
No response
Additional context
No response
Are you working on this?
No

I can do this. Please assign me


Dependency Check on PNPM Project

I successfully ran the CLI against an npm project using the following command:
dependency-check.sh --project TEST-JS-PROJECT --nvdApiKey <api-key> --scan package-lock.json.
Now, I want to run it against a pnpm project. I attempted the following command:
dependency-check.sh --project TEST-JS-PROJECT --nvdApiKey <api-key> --scan pnpm-lock.yaml --pnpm "C:\Program Files\nodejs".

[INFO] Created CPE Index (2 seconds)
[INFO] Finished CPE Analyzer (2 seconds)
[INFO] Finished False Positive Analyzer (0 seconds)
[INFO] Finished NVD CVE Analyzer (0 seconds)
[WARN] Provided path to pnpm executable is invalid.
[INFO] Finished Pnpm Audit Analyzer (0 seconds)
[INFO] Finished Sonatype OSS Index Analyzer (0 seconds)
[INFO] Finished Vulnerability Suppression Analyzer (0 seconds)
[INFO] Finished Known Exploited Vulnerability Analyzer (0 seconds)
[INFO] Finished Dependency Bundling Analyzer (0 seconds)
[INFO] Finished Unused Suppression Rule Analyzer (0 seconds)
[INFO] Analysis Complete (3 seconds)
[INFO] Writing HTML report to: D:\Playground\owasp.\dependency-check-report.html

I am getting the warning 'Provided path to pnpm executable is invalid.' I tried using different paths, still it is not working, what is the expected path to be  provided to this --pnpm flag

have you tried the full path to the pnpm executable? Including pnpm itself?
--pnpm "C:\Program Files\nodejs\pnpm"

Yes, tried that also.

 Try:
--pnpm "C:/Program Files/nodejs/pnpm"

I will check this again in a unix system

You can just try on windows with the forward slashes. I'm curious if that is the problem

I tried and I got the error in the screenshot. Cannot run program    not a valid Win32 application

so on windows pnmp runs?  If it does not - then that explains why this fails. If you have a pnmp that runs on windows you would need to provide the path to the correct executable.

I got working in Windows, I used the below command
./dependency-check-bin/dependency-check/bin/dependency-check.bat --project WINDOWS-PNPM-OWASP   --scan pnpm-lock.yaml --nvdApiKey<nvd-api-key>  --pnpm "C:/Program Files/nodejs/pnpm.cmd"

Also, I tried running the same on WSL using the below command. Here I didn't pass the pnpm  flag
./dependency-check-bin/dependency-check/bin/dependency-check.sh --project WSL-PNPM-OWASP   --scan pnpm-lock.yaml --nvdApiKey

Thanks @jeremylong for the support


NASM syntax-checker : -f elf to args_after to be able to override them

This allows to override the -f option in .vimrc.
For example : let g:syntastic_nasm_nasm_args = "-f elf64"

This allows to override the -f option in .vimrc.

It doesn't.  Use args or (better, in this particular case) post_args instead of args_before and args_after for arguments you want to override.  Enable debugging to see what syntastic is doing.

Sorry, I should  have explain the point of this.
I want to be able to replace the -f option of NASM in my .vimrc.
The normal way is to add this line in .vimrc :
let g:syntastic_nasm_nasm_args = "-f elf64"

The problem is that it doesn't work because the option is overridden by the args_after option in the NASM plugin.
So, to fix this we can put this parameter in the args_before like I did here.

Sorry, I should have explain the point of this.

You did.

So, to fix this we can put this parameter in the args_before like I did here.

No.  The proper way to do this is to move -f elf to an option that can be overridden (rather than duplicated with a new value) by the user.  args_before and args_after can't be overridden, so the candidates are args and post_args.  Changing args breaks backwards compatibility for people that set g:syntastic_nasm_nasm_args in their vimrc.  This leaves you with args_after.  If you move -f elf there it can still be overridden by g:syntastic_nasm_nasm_post_args.

Alternatively, you can just set
let g:syntastic_nasm_nasm_post_args = '-f elf64'

without any patch. shrug

Okay, I see the point !
I didn't know the user was able to override post_args. Should I do another pull request to add it into the README.md ?
I thought the traditional args shouldn't be used anymore in the syntax checker.
So, you're right, as we can use post_args as user it simpler to use this way.
I moved the -f elf option into post_args to be cleaner anyway.
Thanks !

I didn't know the user was able to override post_args. Should I do another pull request to add it into the README.md ?

It's documented, but sadly the information is spread across several files.  From a user's percpective: :h 'syntastic_<filetype>_<checker>_<option>'; from a checker writer's perspective: see the guide.  Basically args and post_args are user-definable, while *_before and *_after are checker internals.


Does clickhouse-jdbc surpport update data in clickhouse?

Dose clickhouse-jdbc surpport update data like this?
"alter table table_name update filed1='far' where key='boo';"

Yes, as long as the query works on your ClickHouse server. However, please keep in mind that update/delete in ClickHouse happens in an asynchronous way. You may have to check system.mutations(is_done=1?) periodically if you need to understand if the mutation is a success or not.

Yes, as long as the query works on your ClickHouse server. However, please keep in mind that update/delete in ClickHouse happens in an asynchronous way. You may have to check system.mutations(is_done=1?) periodically if you need to understand if the mutation is a success or not.

It's interesting that I didn't find any unit test for update and delete, which should be added.

It's interesting that I didn't find any unit test for update and delete, which should be added.

I found this code in ru.yandex.clickhouse.ClickHousePreparedStatementImpl.ava, it will throw new SQLSyntaxErrorException if the sql not like 'INSERT INTO ...', so i com here to ask if che client can supprot update or delete.
@Override
    public int[] executeBatch(Map<ClickHouseQueryParam, String> additionalDBParams) throws SQLException {
        Matcher matcher = VALUES.matcher(sql);
        if (!matcher.find()) {
            throw new SQLSyntaxErrorException(
                    "Query must be like 'INSERT INTO [db.]table [(c1, c2, c3)] VALUES (?, ?, ?)'. " +
                            "Got: " + sql
            );
        }
        int valuePosition = matcher.start();
        String insertSql = sql.substring(0, valuePosition);
        BatchHttpEntity entity = new BatchHttpEntity(batchRows);
        sendStream(entity, insertSql, additionalDBParams);
        int[] result = new int[batchRows.size()];
        Arrays.fill(result, 1);
        batchRows = new ArrayList<byte[]>();
        return result;
    }

I found this code in ru.yandex.clickhouse.ClickHousePreparedStatementImpl.ava, it will throw new SQLSyntaxErrorException if the sql not like 'INSERT INTO ...', so i com here to ask if che client can supprot update or delete.
@Override
    public int[] executeBatch(Map<ClickHouseQueryParam, String> additionalDBParams) throws SQLException {
        Matcher matcher = VALUES.matcher(sql);
        if (!matcher.find()) {
            throw new SQLSyntaxErrorException(
                    "Query must be like 'INSERT INTO [db.]table [(c1, c2, c3)] VALUES (?, ?, ?)'. " +
                            "Got: " + sql
            );
        }
        int valuePosition = matcher.start();
        String insertSql = sql.substring(0, valuePosition);
        BatchHttpEntity entity = new BatchHttpEntity(batchRows);
        sendStream(entity, insertSql, additionalDBParams);
        int[] result = new int[batchRows.size()];
        Arrays.fill(result, 1);
        batchRows = new ArrayList<byte[]>();
        return result;
    }

It's interesting that I didn't find any unit test for update and delete, which should be added.

Yes, I didn't find yan unit test for update or delete, so i don't know if it has the function.

It's interesting that I didn't find any unit test for update and delete, which should be added.

Yes, I didn't find yan unit test for update or delete, so i don't know if it has the function.

@qinf please use the regular statements for this. The "Batch" operations are optimized for loading data sets into ClickHouse, that's why the driver currently imposes such a limitation.
For example:
connection.createStatement().execute(
    "ALTER TABLE table_name UPDATE filed1='far' WHERE key='boo');

or
PreparedStatement stmt = connection.prepareStatement(
    "ALTER TABLE table_name UPDATE field1 = 'far' WHERE key = ?");
stmt.setString(1, "boo");
stmt.executeUpdate();

@qinf please use the regular statements for this. The "Batch" operations are optimized for loading data sets into ClickHouse, that's why the driver currently imposes such a limitation.
For example:
connection.createStatement().execute(
    "ALTER TABLE table_name UPDATE filed1='far' WHERE key='boo');

or
PreparedStatement stmt = connection.prepareStatement(
    "ALTER TABLE table_name UPDATE field1 = 'far' WHERE key = ?");
stmt.setString(1, "boo");
stmt.executeUpdate();

@qinf please use the regular statements for this. The "Batch" operations are optimized for loading data sets into ClickHouse, that's why the driver currently imposes such a limitation.
For example:
connection.createStatement().execute(
    "ALTER TABLE table_name UPDATE filed1='far' WHERE key='boo');

or
PreparedStatement stmt = connection.prepareStatement(
    "ALTER TABLE table_name UPDATE field1 = 'far' WHERE key = ?");
stmt.setString(1, "boo");
stmt.executeUpdate();

@enqueue Thank you for your replay, I will try it.

@qinf please use the regular statements for this. The "Batch" operations are optimized for loading data sets into ClickHouse, that's why the driver currently imposes such a limitation.
For example:
connection.createStatement().execute(
    "ALTER TABLE table_name UPDATE filed1='far' WHERE key='boo');

or
PreparedStatement stmt = connection.prepareStatement(
    "ALTER TABLE table_name UPDATE field1 = 'far' WHERE key = ?");
stmt.setString(1, "boo");
stmt.executeUpdate();

@enqueue Thank you for your replay, I will try it.

@den-crane / @filimonov, any comment for https://github.com/ClickHouse/clickhouse-jdbc/blob/release_0.2.4/src/main/java/ru/yandex/clickhouse/ClickHousePreparedStatementImpl.java#L326? Any concern to remove the restriction to support other statements(query/mutation etc.) in addition to insertion?

@den-crane / @filimonov, any comment for https://github.com/ClickHouse/clickhouse-jdbc/blob/release_0.2.4/src/main/java/ru/yandex/clickhouse/ClickHousePreparedStatementImpl.java#L326? Any concern to remove the restriction to support other statements(query/mutation etc.) in addition to insertion?

We should not encourage users to use mutations. On the contrary we should warn users against using them.
Mutations should not be used on a daily basis.

We should not encourage users to use mutations. On the contrary we should warn users against using them.
Mutations should not be used on a daily basis.

I took a look at this issue today. and I do not see a straightforward way of supporting ClickHouse mutations in JDBC batch mode.
There is currently no officially recommended way to pass multiple statements in one HTTP call.
Option 1: We could try to analyze the query and do some tricks for trivial statements E.g. if the whole statement looks like this ALTER TABLE t UPDATE bar = 'constant' where foo = ? -> rewrite to use IN (?, ?, ..) Error-prone, hard to maintain magic for only some limited cases. :no_good:
Option 2: We could collect the statements and, once the client calls executeBatch,  kick off one HTTP call per statement, perhaps re-using the same session ID. This is doable, but would mean quite some work for a special use case. I was thinking about re-using such a mechanism for the more general autoCommit setting, but I don't see a very high merit in this either.
What's your opinion, @zhicwu ?

Thank you @enqueue as always for the detailed inputs! I agree that both options are not ideal but I'm leaning towards to the second. Some more thoughts on option 2:

Regarding session, I think we only need it when there's set statement and/or temporary table was involved.
Synchronicity - we can reuse connection setting async and append setting mutations_sync accordingly to alter statement(example) as needed
We probably should also show a warning when one of the statements is for update/deletion, because it's heavy operation and is not encouraged to use
Optimizations

consolidate set statements (needs to test if there's any performance improvement but it helps to avoid session and reduce network calls)
in a clustered environment, I'm not sure if we should always reuse the same http client for all statements - maybe some of them can be executed in parallel, and this applies to insertion too

You're right that this cannot be tackled shortly. I guess we can enhance SQL parser for the first two in 0.3.1 and consider optimizations in 0.4.0 with new client interface.


[Bug] Can you share a method from CFI to locater or Locater documentation and  Search functionality code as well

Describe the bug
Can you share a method from CFI to locater or Locater documentation
How to reproduce?
Nothing to reproduce
Readium version
2.7.3
OS version
18
Testing device
Iphone 13
Environment
No response
Additional context
No response

There's no way to convert a CFI into a Locator. Do you have other information besides the CFI, maybe the text snippet of the selection?
For the search, you can find sample code here: https://github.com/readium/swift-toolkit/tree/develop/TestApp/Sources/Reader/Common/Search


Real-time embeddings synchronisation

The state of the vault's embeddings is fixed in time, requiring a reboot of the API to have the "search" features updated.
https://github.com/mfarragher/obsidiantools/issues/23

@hotkartoffel was thinking of implementing an endpoint /refresh which reload/recompute everything (inefficient as hell) for now.
Probably take < 2 minutes for me on MPS (mac m1) - 550k words
Assuming most Obsidian users have 10 times fewer data and similar compute
We could refresh when enough content have been changed by a client call. (or just intervals)
We could optimize later.
Thoughts?

Done


[New Extension] PromptLab

Description
This is an extension for creating and sharing contextually-aware AI commands that utilize placeholders, action scripts, and more. PromptLab uses Raycast AI by default but can be configured to use other model endpoints.
Commands

New PromptLab Command

Create a custom PromptLab command accessible via 'My PromptLab Commands'

My PromptLab Commands

Search and run custom PromptLab commands that you've installed or created

PromptLab Command Store

Explore and search commands uploaded to the store by other PromptLab users

Summarize Selected Files

Summarize the contents of selected text files, PDFs, images, audio files, and more.

PromptLab Chat

Start a back-and-forth conversation with AI with selected files provided as context.

Import PromptLab Commands

Add custom commands from a JSON string.

Screencast
This is the original video demonstrating iteractions with selected files. See comments below for updated demos.
https://user-images.githubusercontent.com/7865925/226258561-ce64767e-f6cc-4688-a2aa-041c15678644.mp4
YouTube Video
Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

Converting to draft for now so we don't end up merging it

Update demonstration: https://www.youtube.com/watch?v=vpY2OxIpDvo

@pernielsentikaer This is ready for review whenever you get a chance 🙂

Awesome @SKaplanOfficial 🔥

Let's give it live so we can iterate on it 🙂

FYI: The extension isn't working, this is the error I get when using it with the OpenAI endpoint: TypeError:
Cannot read properties of undefined (reading 'startsWith')
search-commands.tsx:42:98
at Array.filter ()
search-commands.js:740:16878

This was also reported to @SKaplanOfficial in https://github.com/SKaplanOfficial/Raycast-PromptLab/issues/14

@Ein-Tim I think there are three different bugs between the two in issue #14 and this one. I'll work on fixing them today.


'=' in secret key causes truncation by credentials parser

Hello,
So I have a aws_secret_access_key in a credentials file that ends with the characters "==".
When I put my credentials in AWS Explorer, or 'hard code' them into my AmazonS3Client config, everything works. But when I try to pull from the credentials file, I find that the .net sdk truncates the two "=" operators.
I decompiled the StoredProfileAWSCredentials.cs I noticed the following:
private class CredentialsFileParser     { ... private const string keyValueSeparator = "="; ...}
The KeyValueSeparator being set to "=" makes me think that my secret key is being truncated by throwing off the parser?
I end up with a truncated secret key which fails to authenticate with the client.

I checked with the auth team and they said AWS doesn't vend AWS secret key that have "=" characters. Are you sure you are working with an AWS secret key?

Oh I see. We are using an "S3 compatible" storage, not actually an AWS s3.

We're looking into updating the credentials parser implementation to support your usecase. Will update this issue when this change is made.

Do you guys accept pull requests from others if I want to work on the change on my end and send you my solution incase you want to use it?

Absolutely. Feel free to submit a PR and we'll review it.

The PR for this feature has been accepted. Closing.

Just curious, how once PR has been accepted how long before I see it on the nuget package for the AWS SDK? What's the software deployment cycle like?

The NuGet package should be updated today. I'll comment here when it is.
Mostly, we update the packages when there is a service update, a major bug is found, or if someone requests it.

AWSSDK.Core NuGet package <IP_ADDRESS> contains this change. You should now be able to update the core package in your VS project to the latest version.

 I got version <IP_ADDRESS> and see that the change to the split function is there. But somehow after using StoredProfileAWSCredentials I still wind up with the _wrapped credentials "secret key" value missing the 2 "=" at the end of the string. =/

Please make sure that your application is in fact using the latest version of AWSSDK.Core. I've just tested this behavior with the below code and confirmed that the code works with the latest version of AWSSDK.Core from NuGet.
var path = Path.GetTempFileName();
File.WriteAllText(path, @"[test]
aws_access_key_id = blah==
aws_secret_access_key=foo==
aws_session_token= baz==
");

var creds = new StoredProfileAWSCredentials("test", path);
var ic = creds.GetCredentials();
Debug.Assert(ic.AccessKey.EndsWith("=="));
Debug.Assert(ic.SecretKey.EndsWith("=="));
Debug.Assert(ic.Token.EndsWith("=="));


Have a lobby location to TP back to on tutorial completion

Specify the lobby location in the config

As part of this we actually need to create the lobby

Alternatively we'd need to utilise the network plugin to get the player back to the server lobby


Add scrolling and clicking behavior and testing for click end condition

This pull requests takes the changes from a previous pull request (https://github.com/internetarchive/umbra/pull/42/) and adds scrolling behavior. Also, it deals with the case of a site that reuses the same element to load more content, thus never having a truly unique key (either outerHtml or property of umbraClicked set on the element when it is clicked). The only solution I could think of is to check for an "end condition" and stop clicking if the end condtion is true. In the case of http://psu24.psu.edu/, the selector to load more content is a[id='load-more']. However the outerHtml is always the same for this content. The element should continue to be clicked until it is disabled (has class="disabled") and its visibility is hidden. That is what the 2 new properties in behavior.yaml (click_css_selector_end_condition and click_css_selector_computed_style_end_condition) are for. The end condition is evaluated by creating a dynamic function with the end condtion comparison and executing it for each interval. Not sure if this is the best way to do it but it is only way I could think of.

There were some issues with psu.js, in particular the umbraClicked thing was causing the button to only be clicked once. I fixed that and made some other tweaks, and also made some small tweaks to simpleclicks.js.in. https://github.com/internetarchive/umbra/pull/44

Obsoleted by #44


Rewrite to Python

Hello,
To facilitate the development of this tool, we can rewrite it into Python. Python is installed by default in every environment similar to bash, but have a much nicer syntax and a more extensive standard library.
Best regards,
Kamil Breguła

Hey, I'm looking to help with this

@brennoflavio  Assigned ;-) I am very happy that you want to help develop this little project.


[BUG] Remove static postgres credentials from the code

Describe the bug
Credentials for connecting to Postgresql are hardcoded into the entrypoint route database.py. This is a security issue.
To Reproduce

Go to file entrypoint/microservice/routes/database.py.
Check lines 72-74

Expected behavior
The credentials should be stored inside the entrypoint/microservice/config/.env file and then loaded in the entrypoint/microservice/config/config.py file. In addition, the methodology for accessing the database should be registered as a service (similar to the logging functionality in entrypoint/microservice/config/config_logging.py).
See the following example: https://flask.palletsprojects.com/en/1.1.x/tutorial/database/

I'm on it.


export apns:connect/1

export apns:connect/1, so i can handle two production apps(open two connections) in one Erlang application.

add jsx to apns.app.src, i don't need to add jsx to my app's(top level app) xxx.app.src

Readme add apns:connect/1

Hi @dcy , thanks for the PR :)
I have just requested a little change. After that I will merge it

By the way. Do you want to change jsx to jsone? I can make another PR.

Not really, most of our libraries use jsx. What are the benefits of jsone?

According to the benchmark, jsone have better performance.

Yeah, I don't feel compelled to change it just because of that. What do you think, @ferigis ?

I think we are fine with jsx. We don't have performance issues and we are align with our other projects. If we experience some performance problems in future we will take jsone into account for sure. Thanks for the idea


Feat: support validate properties with CueX

Description of your changes
support validate helm-repository properties
Fixes https://github.com/kubevela/kubevela/issues/5088

How has this code been tested?

Checklist

I have:

[ ] Title of the PR starts with type (e.g. [Addon] , [example] or [Doc]).
[ ] Updated/Added any relevant documentation and examples.
[ ] New addon should be put in experimental.
[ ] Update addon should modify the version in metadata.yaml to generate a new version.

Verified Addon promotion rules
If this pr wants to promote an experimental addon to verified, you must check whether meet these conditions too:

[ ] This addon must be tested by addon's e2e-test to guarantee this addon can be enabled successfully.
This addon must have some basic but necessary information.

[ ] An accessible icon url and source url defined in addon's metadata.yaml.
[ ] A detail introduction include a basic example about how to use and what's the benefit of this addon in README.md.
[ ] Also provide an introduction in KubeVela documentation.
[ ] It's more likely to be accepted if useful examples provided in example dir.

@Somefive @wonderflow @barnettZQG @FogDong please review


undefined bound model in textfield

Apologies for creating yet again another annoying issue without really understanding where or how to address it. When a textfield is bound via [(ngModel)] to an undefined value, things don't go so well.
<mdc-textfield label="label" [(ngModel)]="binding"></mdc-textfield>
binding: string; // no initializer on controller property

results in the label being unpopped and overwritten with the unwanted text string "undefined", as seen here:

Binding uninitialized FormControls does not seem to exhibit this problem. If the bound property is initialized to something, the label at least pops above the field, so I'm guessing this is some sort of disagreement amongst the various implementation bits as to whether various falsy values like null or undefined constitute "a value being set" or not.

On it. Writing test coverage for Text Field and ran into it as well.


Endpoint can't have SessionProtocol?

While I'm working on https://github.com/line/armeria/pull/1886#discussion_r303707344, I thought Endpoint might be able to have SessionProtocol. By the definition in Wikipedia, an endpoint means an interface exposed by a communicating party or by a communication channel, so perhaps we can contain the SessionProtocol in it. If we do that, it will make some APIs cleaner and simpler, especially for https://github.com/line/armeria/pull/1886.
However, there are two things that I'm concerned:

The two different Endpoints are the same one eventually.

For example, the SessionProtocol http could be h1c or h2c. We have to deal with it specially.

The Scheme contains SerializationFormat and SessionProtocol.

The builders for clients have methods that take Scheme and Endpoint together. We need to break up the Scheme and let the builders take SerializationFormat and Endpoint.
The same information resides in scheme and endpoint in the ClientRequestContext.
Because of this, I'm not sure this change is worth it.

I had a similar notion recently when setting up some EndpointGroup but decided it's probably not a good idea, since I think it's not safe if a client can use an endpoint group where the endpoints are a mix of secure TLS and insecure. While it could be possible to just validate this when creating an endpoint group, I find it intuitive that our Endpoint relates to TCP layer connection, and the session protocol on top of that is a separate concept. On the flip side, I do see the benefit for some APIs, and especially #1886.

endpoint group where the endpoints are a mix of secure TLS and insecure.

That's a good point. I couldn't think of that. 😅
So yeah, let's treat the Endpoint as it is.


Any good examples of running flask with ollama(Mixtral)

Hi,
I was trying to run my Mixtral model but was not sure how to verify:
python app.py
* Serving Flask app '__main__'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://<IP_ADDRESS>:5000/
Press CTRL+C to quit

How to verify?
i saw some code online?
curl --location '<http://<IP_ADDRESS>:5000/process_form>' \\--form 'query="What does the author discuss about NFL"

Hi,
Your curl request should be:
curl --location 'http://<IP_ADDRESS>:5000/process_form' --form 'query="What does the author discuss about NFL"
Have you tried these other implementations?: https://github.com/jmorganca/ollama?tab=readme-ov-file#web--desktop ?

I don't understand, did you create a proxy for ollama with flask? Can you share your app.py?
If you want try directly via ollama:
curl http://<IP_ADDRESS>:11434/api/generate -d '{
  "model": "mistral",
  "prompt":"What does the author discuss about NFL"
}'

curl http://<IP_ADDRESS>:11434/api/chat -d '{
  "model": "mistral",
  "messages": [
    { "role": "user", "content": "What does the author discuss about NFL" }
  ]
}'

Hi @andysingal are you still having an issue or did the answers from @jeanjerome and @rgaidot solve it for you?

Sorry for the late reply, the issue is resolved.
On Fri, Jan 5, 2024 at 06:43 Matt Williams @.***> wrote:

Hi @andysingal https://github.com/andysingal are you still having an
issue or did the answers from @jeanjerome https://github.com/jeanjerome
and @rgaidot https://github.com/rgaidot solve it for you?
—
Reply to this email directly, view it on GitHub
https://github.com/jmorganca/ollama/issues/1698#issuecomment-1877978187,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AE4LJNMK44GBHHI6BXZFD7LYM5HVFAVCNFSM6AAAAABBBVLPCCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZXHE3TQMJYG4
.
You are receiving this because you were mentioned.Message ID:
@.***>

Great. Thanks so much for that info.


atdgen cannot print large json files on 32bit architectures

From @mjambon on May 27, 2018 1:36
From @josch on March 20, 2015 11:11
Hi,
when running my code on 32bit architectures, then for big output I'm getting the following traceback:
Fatal error: exception Failure("oops")
Raised at file "pervasives.ml", line 20, characters 22-33
Called from file "write.ml", line 52, characters 2-24
Called from file "datatypes_j.ml", line 52, characters 4-47
Called from file "ag_oj_run.ml", line 19, characters 1-6
Called from file "ag_oj_run.ml", line 43, characters 2-39
Called from file "ag_oj_run.ml", line 19, characters 1-6
Called from file "ag_oj_run.ml", line 43, characters 2-39
Called from file "ag_oj_run.ml", line 19, characters 1-6
Called from file "ag_oj_run.ml", line 43, characters 2-39
Called from file "ag_oj_run.ml", line 25, characters 1-6
Called from file "ag_oj_run.ml", line 43, characters 2-39
Called from file "datatypes_j.ml", line 2324, characters 4-62
Called from file "datatypes_j.ml", line 2332, characters 2-18

I am running Debian unstable and have the following versions installed:

libatdgen-ocaml-dev 1.3.1-1+b1
libyojson-ocaml-dev 1.1.8-1
libbiniou-ocaml-dev 1.0.9-1

I was told in the #debian-ocaml IRC channel that this problem might be caused by 32bit architectures not supporting very large strings (>= 8MB) and I observe the problem for instances that would create a 53MB json file on a 64bit platform.
I'm also unsure where exactly the problem should be fixed. Maybe this is actually a biniou problem as that library uses a mutable string as a buffer internally?
Copied from original issue: mjambon/atdgen#29
Copied from original issue: mjambon/atd#103

From @josch on March 20, 2015 18:46
Hi,
Indeed there is something else above the Exception message. I did not notice this because it contained part of the output that I expected to end up in my json file.
Could you make it such that this information ends up as part of the raised exception? Just being told Fatal error: exception Failure("oops") and a print above which is not obviously connected to the exception is not very helpful but instead very confusing.
In my case, that message had start=0 len=13 but there are others. The package failed to build on eight architectures and you can have a look at the start/len values of all arches here: https://buildd.debian.org/status/logs.php?pkg=botch&ver=0.7-1~experimental1&suite=experimental after clicking on the Maybe-Failed links which will get you to the build log for each architecture.
Do you deem this to be worth fixing by using a different data structure than a string for the buffer?
Breaking the input data into chunks will be an ugly workaround because it is hard to impossible to predict how much space a given piece of input data will take when converted into json before doing the actual conversion.
Using a 64-bit platform could be a workaround but so would using a datastructure as a buffer which is not an ocaml string. Then every platform could work with large data.

What are the values for start and len printed on stderr before that?
See https://github.com/mjambon/yojson/blob/master/write.ml#L31
It's probably a size limit problem; it's possible that your string cannot be added to the buffer, which itself is a string. The maximum length on any given string is 16MB on 32-bit platforms. Usual workarounds include using a 64-bit platform, breaking large data into chunks (using a wrapper, see http://stackoverflow.com/a/28997459/597517), or not using json for large data but a streaming protocol of some kind (which breaks the data into chunks for you).

I'm not sure what the problem is, it could be a bug in yojson. I'll take a look a bit later.

From @josch on March 20, 2015 23:32
To clarify the problem, let me give you a minimal example.

Makefile:

test.native: datatypes_j.ml datatypes_t.ml test.ml
    ocamlbuild -classic-display -use-ocamlfind -package atdgen test.native

%_j.ml: %.atd
    atdgen -j -j-std $<

%_t.ml: %.atd
    atdgen -t $<

.PHONY: clean
clean:
    rm -f datatypes_j.ml datatypes_j.mli datatypes.ml datatypes.mli datatypes_t.ml datatypes_t.mli test.native
    rm -rf _build

test.ml

let main () =
  let s = String.make (int_of_string Sys.argv.(1)) 'c' in
  print_endline (Datatypes_j.string_of_output (s, s));
;;

main ();;

datatypes.atd

type output = (string * string)

Compile the whole thing by running make and then execute:
$ ./test.native 8388602

This will work fine. But then:
$ ./test.native 8388603
Fatal error: exception Invalid_argument("Buf.extend: reached Sys.max_string_length")

Nice readable error message. The problem starts here:
$ ./test.native 8388604
[lots of output]
Fatal error: exception Failure("oops")

This error message is not really helpful and it would be great if this would not happen on 32 bits.
Another solution to the problem could be to not buffer the result in the first place but to allow passing an output channel to which any generated json would then directly be written to.

Proposed resolution: include all useful information in the exception and don't print anything to stderr.


nodePackages: follow up track/clean/fix broken packages after switch to 12.x and 14.x in #89184

Following packages were found to be broken (in the sense of not building at all) with nodejs-10_x before change already:

[ ] coc-imselect /cc @eyJhb
[ ] fast-cli-1.x /cc @svanderburg
[ ] git-ssb /cc @patrl @matthewbauer
[x] pulp /cc @Mic92 -> fixed #89210
[x] insect /cc @lilyball -> fixed #89210
[ ] node-red
[ ] node-inspector
[x] stf /cc @jollheef -> remove
[x] ssb-server /cc @FRidh -> remove

Following packages were found to be broken (not building) after the switch in nodePackages (that is with nodejs-12_x):

[x] dat /cc @mmahut, build fails on node-gyp-build for utp-native -> update to dat@next

Following packages were found to be broken (not building) after the switch in nodePackages_latest (that is with nodejs-14_x):

[x] dnschain /cc @rnhmjoj dnschain received last code commit 5 years ago, a dependency hiredis (that breaks the build) is archived and won't get any updates. -> remove (incl. nixos module)

This issue is here to track and inform maintainers/adders of the packages about the broken state and decide about removing or fixing the broken packages.

I would remove dnschain. It is no longer developed.

I know very well the state of dnschain. If migrating hiredis -> node-redis is non-trivial I don't think it will be ever fixed. I guess you can remove the package and the nixos module.

https://github.com/NixOS/nixpkgs/pull/89210

It seems the dat issue has been already reported here: https://github.com/datproject/dat/issues/1162
Oddly enough it's been closed as utp-native is an optional dependency.

Will need to remove ssb-server as I haven't been following it in a while.

So far openstf does not support node newer than 8 because of some dependencies, so I guess you can just remove it.

@FRidh i can take care of it, just need the infos. Thanks :)
@jollheef, ok, i will remove it then. Thanks :)

@prusnak, ok, i will investigate dat. Thanks :)

@calbrecht dat@next is fine. mmahut packaged it for me

@prusnak Thanks. I will update it with the other changes then (when all cases closed).

@calbrecht, I am new to processes nixos/nixpkgs [might] have... Could you help me? Is there anything I can do in order to make node-red fixes progress further?

Broken packages should be moved to buildNpmPackage and fixed there. see #229475


Support for relative paths in coverage/.resultset.json

Howdy! Thanks for maintaining simplecov! ❤
I'm wondering if anyone would be interested in having relative paths in coverage/.resultset.json -- I would find this quite useful because right now I have a little problem between my development and CI environments
I realize that https://github.com/colszowka/simplecov/issues/229 proposed this many years ago, but then it was closed as it was considered a duplicate of https://github.com/colszowka/simplecov/issues/197. I don't think #197 is quite the same as this issue though, that's why I think it deserves a new issue.
Problem
I want to use a coverage/.resultset.json file that was generated in Circle CI in my local environment. SimpleCov reports that there is 0% coverage for all files in my project.
I know that the problem is happening because the paths in the file that I download has absolute paths within a Circle CI container.
Resources

This file was generated with SimpleCov in Circle CI: https://gist.github.com/etagwerker/0518a1e381b1c6dc4d8814c7602ca108
After searching and replacing with my local environment's absolute path, this file works as expected: https://gist.github.com/etagwerker/15de7af66f78cc788d4c4696f1bd409f

Workaround
If I follow these steps:

Download coverage/.resultset.json
Find and replace root directory within coverage/.resultset.json
Run skunk lib

Then everything works as expected. Skunk uses SimpleCov and it detects coverage as expected.
Solution?
I wonder if we could add a configuration option to SimpleCov that specifies relative_paths (default to false) so that all the file paths written in coverage/.resultset.json are relative (instead of absolute)
It could work like this:
SimpleCov.start do
  relative_paths true
end

You can also set it like this:
SimpleCov.relative_paths true

That way you wouldn't get unexpected results when using a coverage/.resultset.json from another environment.
Any thoughts about this?

:wave:
Hi and thanks for opening an issue @etagwerker !
Let's go back a step - why do you want a resultset.json from another environment in your local environment?

@PragTob That's a good question! We do a lot of code audits in my agency and we've found that it's much easier to run SimpleCov in our client's continuous integration service than in our local environments.
Then we check out their application code and analyze it in our local environment. In order to find hotspots, we use https://github.com/fastruby/skunk which depends on coverage/.resultset.json
Sometimes our client's CI is quite complicated and it takes hours to finish. That means that they're probably using different containers in their CI service. So that means that before calling skunk we need to download artifacts from CI (including .resultset.json), merge them, and finally use them in our local environment. (more about merging here: https://www.fastruby.io/blog/rails/simplecov/upgrades/merging-multiple-simpleCov-coverage-results.html)

@etagwerker no one should rely on .resultset.json, it's an internal implementation detail and doesn't even have the full reported data. I highlighted some of the inadequacies of using .resultset.json over there: https://github.com/codeclimate/test-reporter/issues/413#issue-545404610 - basically it's just the raw ruby coverage data without any or much simplecov coverage processing.
The proper way to go about this imo is to write your own formatter. That's an official extension way, albeit I'll grant that documentation could be better.

@PragTob Thanks for the quick response! 👍

no one should rely on .resultset.json, it's an internal implementation detail and doesn't even have the full reported data.

This is a little bit confusing because it is mentioned a few times in the README. For example: https://github.com/colszowka/simplecov#merging-test-runs-under-different-execution-environments

The proper way to go about this imo is to write your own formatter.

I'm not sure if that would be the best way for what I'm trying to do. Skunk depends on RubyCritic, which in turn depends on SimpleCov. You can see how we are reading covered_percent over here:  https://github.com/whitesmith/rubycritic/blob/master/lib/rubycritic/analysers/coverage.rb#L36
The only reason I decided to use .resultset.json was because I just needed a machine-readable format to read the coverage per module. In the near future I'd like to find coverage per method and use that information in both RubyCritic and Skunk.

Due to the introduction of branch coverage the format of this file will change slightly in the next release (beta release just released)

This is something that I'm interested in using in both RubyCritic and Skunk.

I highlighted some of the inadequacies of using .resultset.json over there: codeclimate/test-reporter#413 (comment)

Thanks for sharing this! It's very interesting and I learned a lot. 😄

On the other hand you can say, it'd be nice to have an official JSON formatter so 🤷♂

You mentioned this in that issue. Is this something that you're planning to work on? I think I would still need raw data in that formatter, which would allow me to create SimpleCov objects in RubyCritic.
I apologize for talking about way too many different things in this issue. TL; DR: I think it would be very positive to have an official JSON formatter, which included raw data.

@etagwerker
.resultset.json is only ever mentioned when talking about merging test where it goes through another round of simplecov processing either through our parallel tests merging mechanism or SimpleCov.collate. All of those were also just introduced rather recently ~ mid January. I was fearing people might mistake this as it's ok to work for the file, even if the format is never discussed or it's never suggested to look at it yourself (iirc). I might add a note about this.
Regarding a formatter - what a formatter allows you to do is access the coverage data/data structures as simplecov itself uses them. You can then use them to gather them in some big ruby object, write out another machine readable format or whatever your heart desires. I acknowledge the interface isn't documented as well as it could be and there might be some deprecations there in the future, if I get the time. It's definitely my recommended way to work with SimpleCov results.
We could explore an easily accessible for SimpleCov that given a resultset.json and a configuration you can load the corresponding SimpleCov structures to work with them, I have no idea how complicated that'd be right now though (might be super easy, might be harder than expected)
(thanks for your work on rubycritic by the way)
What do you mean by raw data? Raw coverage data? Sure, that's the plan. I'm not planning to work on it right now as there are enough other things to work on. My current deal with CodeClimate is they are the ones to write the JSON formatter (see a bit further below in the very same issue).

@PragTob yeah, this is exactly what I want:

We could explore an easily accessible for SimpleCov that given a resultset.json and a configuration you can load the corresponding SimpleCov structures to work with them

Are you thinking about incorporating https://github.com/vicentllongo/simplecov-json to simplecov?
My concern is that you might end up with two codebases to maintain very similar logic: What would be the main difference between .resultset.json and resultset.json? I understand that the goal for both those files is different:

.resultset.json is a cache file that gets used internally by simplecov -- this is a private API that simplecov users should not rely on
resultset.json is an output formatted in JSON that has very similar information to the HTML report and .resultset.json -- this is a public API that simplecov users can trust (and might follow semver?)

As you add more features (e.g. coverage data about different branches) I would expect resultset.json to have those details.

My current deal with CodeClimate is they are the ones to write the JSON formatter (see a bit further below in the very same issue).

This is great. I might just wait until they're done with that, to add support to RubyCritic and Skunk

@etagwerker
Sorry - interesting times :)
Yes I'm talking about having an official json formatter for SimpleCov. The idea indeed is that one could be relied upon vs. the other and ideally follows semver. The differences go beyond that though - the "formatted"/official json would have lines marked as ignored, loaded files that ruby code coverage didn't load, filtered out files etc. - essentially all the features of simplecov applied to it vs. the current .resultset.json which has none of that (It's more or less a dump of what the standard coverage library provides, but merged)
Newer features would be added there yes. But honestly, it's also one of the points why I was resisting adding it as it means more work for me and for new features. So I won't promise it, but it'll remain the goal.
Anyhow, we're at the stage where code climate will make a PR/proposal of a formatter - I guess hope we can close this issue then? :)

@PragTob Sounds good, thanks!


Undefined Headers

When I try to verify this sketch on an ESP8266, I can errors stating that various header files aren't defined.  Where can I find the library(ies?) that define WiFi.h, esp_wpa2.h, and esp_wifi.h?

Will not work with esp8266, hardware is incapable of doing this.


[Epic]: generate sql queries via chatgpt

Objective
writing good optimised sql queries is not a trivial task and can be greatly simplified by using LLM models

we should consider integrating with chatgpt to help users generate sql queries using normal chat inputs and they can then edit it later as per their needs. something like this

Success Metrics

Metric

Customer satisfaction score increases

Decrease churn rate down to 30%

Prioritised User Stories

User Story

e.g. as a user, I want to be able to access the platform via mobile phone

e.g as a user, I want to be able to communicate with the other members on the canvas

Developer Handoff Document in Figma
No response
Test Plan
No response
RACI matrix
| ------------- | ------------- |
| Responsible   |               |
| Accountable   |               |
| Consulted     | @Nikhil-Nandagopal, @mohanarpit, @areyabhishek|
| Informed      |               |

we can also go the other way around wherein we use chatgpt to explain some of the complex queries in our templates,


Incompatible with OptiFabric

Describe the Bug
The game doesn't load up when having Create mod installed with Optifabric
Reproduction Steps
These are the mods I had installed:
create-fabric-1.18.2-0.5.0e-733.jar
fabric-api-0.67.0+1.18.2.jar
modmenu-3.2.4.jar
optifabric-1.13.16.jar
OptiFine_1.18.2_HD_U_H7.jar
voicechat-fabric-1.18.2-2.3.15.jar
Xaeros_Minimap_22.16.2_Fabric_1.18.2.jar
Expected Result
The game loading up normally with all mods successfully loaded
Screenshots and Videos

Crash Report or Log
https://pastebin.com/WGLw0Mr1
Operating System
Windows 10
Mod Version
0.5.0e
Minecraft Version
1.18.2
Other Mods
fabric-api-0.67.0+1.18.2.jar
modmenu-3.2.4.jar
optifabric-1.13.16.jar
OptiFine_1.18.2_HD_U_H7.jar
voicechat-fabric-1.18.2-2.3.15.jar
Xaeros_Minimap_22.16.2_Fabric_1.18.2.jar
Additional Context
No response

Optifabric is support will never happen. The amount of changes needed to even make the two work together is too large. It is also nearly impossible to debug the two mods due to optifine being closed source. Fabric has way better alternatives such as Sodium and Iris. Which I highly recommend you use instead. They both have really good mod compatibility.

ill have to try em out then, ty for getting back aswell! :D


Error: Error: argument $map1 of map-merge($map1, $map2) must be a map

When I try to install, It asked me to override the app.scss I have chosen n option No
Please refer the following screenshot

Hi @blackode, did you add the following lines to your app.scss file?
@import 'ember-paper';
@import 'ember-data-table';

The ember-paper addon should also be installed by running the command:
ember install ember-paper

@erikap
Thanks for quick response.
Yes, I installed and added both of them
Here I am attaching the screenshot.

ember -v
ember-cli: 2.16.2
node: 8.9.1
os: linux x64

### app/styles/app.scss
+ @import 'ember-paper';
+ @import 'ember-data-table';

@erikap I recorded the process of Installation. So, it helps you to help me.
Thanks
Record
https://asciinema.org/a/149853

@blackode The error originates from ember-paper. The message implies that the SASS variable $color-grey isn't a map, while the variable is initialized as a map by ember-paper: https://github.com/miguelcobain/ember-paper/blob/5f599360b89667f4fc8fa236f20441610e7ff067/app/styles/color-palette.scss#L408
So it seems to me that one of your @import statements before @import 'ember-paper' in your app.scss  overwrites this variable with another value than a map. Can you try to move the @import 'ember-paper' to the top of the file and validate whether you still get the same error message?

Sure I will let you know

I found this note in the ember-paper. This solved the issue.
Note: If upgrading from a previous version of ember-paper and you are seeing compile errors around app.scss|sass not existing, please make sure to remove broccoli-sass from your package.json, remove your node_modules and reinstall.
Thank you. @erikap


Gardener enhancement proposal (GEP) for Gardener extensibility

What this PR does / why we need it: Gardener has evolved to a large compound of packages containing lots of highly specific knowledge which makes it very hard to extend (supporting a new cloud provider, new OS, ..., or behave differently depending on the underlying infrastructure). This proposal aims to move out the cloud-specific implementations (called "(cloud) botanists") and the OS-specifics into dedicated controllers, and simultaneously to allow deviation from the standard Gardener deployment.
Related issue(s) of this PR:

#232
#233
#246
#262
#308

Special notes for your reviewer: NONE
Release note:
NONE

/cc @bergerx @luisdavim @ThormaehlenFred

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

GitHub only allows 15 reviewers and trimmed my list... /cc @amshuman-kr @Diaphteiros @dieterguendisch @DockToFuture @ggaurav10 @marwinski @mliepold @plkokanov @prashanth26 @timuthy @zanetworker

I have included all feedback comments into the proposal.
As there was not much activity on this PR in the last weeks I consider that everybody is fine with the proposal, hence, I'm going to merge it now.
Though, that does not mean that everything is finalized already. Further suggestions and improvements are still welcome (anyhow, as always). Please file a PR that directly modifies the content of the document.
We will try to get rid of most of the open pull requests/topics and release another stable Gardener version within the next 2-3 weeks. After that we want to start focusing and working on the extensibility topics, effectively meaning that we won't actively implement new features during that time.


Create README.md for organization overview.

As I said, we shouldn't have divergent definitions of Imageomics in different places, so the one here (which I don't think has been discussed or approved by the leadership team?) needs to be changed.

As I said, we shouldn't have divergent definitions of Imageomics in different places, so the one here (which I don't think has been discussed or approved by the leadership team?) needs to be changed.

Fixed.

[ ] @balhoff could you please check the list for Phenoscape and add repositories you think would also specifically fall into the Imageomics context? (Phenex perhaps?)
[ ] @johnbradley could you add the BGNN repos specifically involved in the Minnow Traits project & manuscript?
[ ] @thompsonmj can you coordinate with @smrolfe and @muratmaga which repos from their GitHub presence should be linked here? (Perhaps SlicerMorph? ml-morph?)
[ ] @susan1637 and/or @xbahadirx, could you add (or list in a comment) repos to add here from the TUBRI Github organization?

Added products from Murat, Sara, and Arthur.

@susan1637 @xbahadirx @uyedaj last call. We'll wrap this up Thursday at the latest.

@balhoff I don't know whether you previously submitted a formal review requesting changes, but somehow it doesn't recognize the resolved comment as having resolved the requested change. When you get a chance, can you submit an approval?

@balhoff I don't know whether you previously submitted a formal review requesting changes, but somehow it doesn't recognize the resolved comment as having resolved the requested change. When you get a chance, can you submit an approval?

Scratch that. I just found a way to dismiss your earlier review (even if in actuality I didn't dismiss it 🧐)

@cvstewart @dboghrat @tanyabw once merged, this will appear as part of our Github organization profile. In addition to the version control diff, you can also see the file in HTML rendering (which is how it would appear).
My main ask here is to check whether the content is all OK to go live, or whether (and which) further corrections need to be made. (You can see in the thread here who contributed.)
My hunch is that it is too much information to fit well on the organization profile, and that we'll probably want to move some of this to a different place, such as the website. But we can deal with bite-sizing it more in a subsequent step.


Make RpcHandler API Java friendly

Currently, we define default implementation of methods inside the RpcHandler trait which isn't easily extendable in the Java world, user would still end up overriding all the methods.
Would be better if we can implement the RpcHandler as a pure interface, and all the default public methods could go inside an abstract/delegation class DefaultRPCHandler/BaseRpcHandler. This way a Java developer could simply extend the abstract class and would only need to implement/override methods that are actually required by the API contract / their use-case.
Also, we have methods inside the RpcHandler trait that are only needed internally and shouldn't be made public. So can clean that up as well with this change.

Other wish list

RPC request should not be tied to method names
The way users implement the different types of RPC should be consistent & can opt in to as many or as few as they wish. CustomRPC, EditRpc and MenuBasedRpc


feat: use JavaDoc as comments

This is a very basic PoC implementation that can serve as a base for further work.

This is awesome.


Ventilation problem on weirs.

I turned off the air ventilation of the broad crested weir of lmaurel/general_update_review branch (https://github.com/erdc-cm/air-water-vv/tree/lmaurel/general_update_review/2d/hydraulicStructures/broad_crested_weir) and of the sharp crested weir (https://github.com/erdc-cm/air-water-vv/tree/lmaurel/general_update_review/2d/hydraulicStructures/sharp_crested_weir) because (as you might know) the calculation crashes when the water reaches the ventilation. We were talking about this problem few months ago and I don't know if someone has solved this problem ? If not, we still can run these cases by setting air_vent to False in the context options like I did.

@cekees, we were just wondering if this has been solved. If not we can put it in air-water-vv with the patch closed and keep the issue open

Thanks. I'll have to take a look at it. @lmaurel what exactly is crashing? E.g. is it the Newton solver for the navier-stokes step failing to converge or something else?

@cekees - The case is not crashing anymore but when the water get closer to the ventilation there is a problem as you can see in the following GIF. I might be a boundary condition problem at the air ventilation.

It looks like the water reaching there triggers some kind of jet blast :)

That's an exciting failure! I can reproduce so at least we have something to work with. I suspect it has something to do with the pressure boundary condition at the vent.

It is a bit puzzling, because this one fails but the outlet which is the same bc does not. Maybe the way the nodes are assigned to patches?
We should aim to create a gallery with impressive failures


EVM type refactoring

[x] remove _ [_ .. _] operator (fixes #1590)
[x] remove Memory sort
[x] remove ByteArray sort

Some thoughts:

Just remove simplification rules about _[_.._], don't try to adapt them. When we see what fails we can bring back some of the lemmas needed.
The ranged write operation _[_:=_] needs to be either ported to Bytes, or remade into something like #writeRange(...) (maybe @PetarMax has a preference on how to name this).
We should be able to get rid of sorts Memory and ByteArray now, as they should just be aliases of the Bytes sort now (the Jvaa backend had different implementation of these sorts).

Removed the Memory and ByteArray sort. Conformance tests are passing. I want also to remove the ++ operator since it is a duplicate of +Bytes currently.


[ko] Update links to point localized pages

I deliberately set the target branch to main, since this is kind of hotfix, not a translating job. 😊

Some discussion regarding using 'main' branch for l10n fix.
https://github.com/cncf/glossary/pull/832#issuecomment-1116873033

@seokho-son As per your comment, I changed the target branch as dev-ko. 😊


Bootstrap with Portland data

So that someone can check out the repo, type 'activator run,' and have data already.

I think this is done with Distrito Federal (Mexico City) data.


Inventory for API8

SpongeAPI | SpongeCommon (outdated)
based on changes in: https://github.com/SpongePowered/SpongeAPI/pull/1959

Replaced SlotIndex with Integers
InventoryTransactionResult has Type#NO_SLOT
moved set(ItemStack) to Slot
removed peek with limit. (instead use query and totalQuantity)
poll returns InventoryTransactionResult instead of the polled item
renamed poll/peek with index -> pollFrom/peekAt
pollFrom(index) returns InventoryTransactionResult (NO_SLOT when using invalid index)
size removed - freeCapacity added
totalItems renamed to totalQuantity
getMaxStackSize removed (maybe accessible as a property?)
setMaxStackSize removed because not implementable
getPlugin removed (maybe accessible as a property?)
Merged QueryOperations and InventoryTransformations

Query, QueryType.NoParam|OneParam|TwoParam
transform replaced by query(Query)
query with zero/one/two parameters
new query type to query for a grid in another grid.

Why remove containsInventory? How can you otherwise know if the inventory contains a child? E.g. a container contains the player main grid. Possibly rename it to containsChild?

I guess i'll leave it in. But the way its implemented now its not always correct.

.

containsInventory can be removed, since it was replaced by containsChild?

no containsInventory still checks recursivly containsChild only one layer down


Merge pull request #2 from mlockett/master

Merge pull request #5 from RyanSchierholz/fix/permission-tabs

Mistake


Add remaining changed lines in pull request files changed tab

Description
Secondary counter under changed lines which shows the number of changed lines in files not marked as viewed. When a file is marked as viewed the counter should decrease.
Screenshot

Example URLs
https://github.com/Cowoding-Jams/Jambo/pull/53/files

Not a bad idea, but on long PRs (where files are loaded progressively) this is impractical to implement with little benefit. After adding hundreds of features, Refined GitHub is limiting new features to things that are easy to implement and maintain.


Refactor pool default config

Purpose

$Title
Bump transport version

Codecov Report

Merging #13526 into master will decrease coverage by 0.02%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master   #13526      +/-   ##
============================================
- Coverage     64.28%   64.26%   -0.03%
  Complexity      661      661
============================================
  Files          2142     2142
  Lines         94294    94294
  Branches      12450    12450
============================================
- Hits          60613    60594      -19
- Misses        28792    28811      +19
  Partials       4889     4889

Impacted Files
Coverage Δ
Complexity Δ

...allerinalang/test/util/client/ResponseHandler.java
65.38% <0%> (-15.39%)
0% <0%> (ø)

...a/org/ballerinalang/bre/bvm/WorkerDataChannel.java
84.56% <0%> (-8.65%)
0% <0%> (ø)

...org/ballerinalang/bre/bvm/WaitCallbackHandler.java
82.88% <0%> (-2.71%)
0% <0%> (ø)

...a/org/ballerinalang/bre/bvm/GlobalMemoryBlock.java
80.39% <0%> (+1.96%)
0% <0%> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e9d0d5c...f01d074. Read the comment docs.


onError method decorator

Will invoke the provided method if the decorated method will throw an exception.
API:
function onError(method: Function): void

Resolved in 4a99716


Feature/autofill and mappers registry

Coverage remained the same at 100.0% when pulling 62aa941dc7395c4c3ed5d229ce79e46c21824027 on feature/Autofill_And_Mappers_Registry into 17eaea7218beddc3ee527e39d032a58e1511129d on master.

Coverage remained the same at 100.0% when pulling 62aa941dc7395c4c3ed5d229ce79e46c21824027 on feature/Autofill_And_Mappers_Registry into 17eaea7218beddc3ee527e39d032a58e1511129d on master.

Coverage remained the same at 100.0% when pulling 4a3097d5cb82397c10ebcd223450dbe495995c4d on feature/Autofill_And_Mappers_Registry into 17eaea7218beddc3ee527e39d032a58e1511129d on master.


feat:add single party func abs

在SCQL中实现单方算子abs
https://github.com/secretflow/scql/issues/112
I have read the CLA Document and I hereby sign the CLA

recheck


HmIP-BSL offers only on/off/toggle and in/decrease brightness, but no color as Action

For the Homematic IP device HmIP-BSL I can only find following actions offered in HA:

turn on
turn off
toggle
increase brightness
decrease brightness

but I can't see any action for automations to set the color.

The HA device controll offers the color picker where I can also change the color accordingly:

I am not sure if this an issue related to custom_homematic.
custom_component/hahomematic version (if applicable):
0.36.2
Home Assistant version (if applicable):
2022.3.0
CCU version:

Model (Raspi4):
OS (RaspberryMatic):
Version: <IP_ADDRESS>20226

I tried it and it's the same for other integrations, so i think thats a limitation of HA. (And a dropdown with every color would be too much)
Instead of the Actiontype "Device" you can use "call service" with the service light.turn_on. With this you can set everything you should need.

Instead of the Actiontype "Device" you can use "call service" with the service light.turn_on. With this you can set everything you should need.
Thanks for the workaround, this is working as expected.


/entrytrust command

Mohron gladly told me that you refused to do it before.

But... since that was asked like almost a year ago, i will make a new request :D
Could you add an /entrytrust command, so that this command would add people to a whitelist to your enter-claim flag?
Well, you know how the command work, i am just asking ^^

That's what /cf /cfg and /cfp are for. Take a look on the GP wiki.

User friendly talking ^^
Most of my users doest know how to do that.


data workflow diagram of acoustic data

@dmurfin can you send me your draft of the data workflow/pipeline you mentioned at the AST meeting on Wed? I'm going to build off of that for the echo-class quarto site. Thanks!

@jrenfree Could you review this acoustic processing overview diagram? I'd like to get your feedback.
You can provide comments directly to the google drawing document here.


Histogram unbounded memory even with run_upkeep

Thanks for a wonderful library. After the recent merge of https://github.com/metrics-rs/metrics/pull/460 I tested the upkeep and idle timeout configs but still see unbounded memory usage if /metrics endpoint is not being called.
Is this still expected even after upkeep?
Testing methodology:

Using axum with axum-prometheus
Using /metrics
Using wrk to benchmark an API with a simple health endpoint
Memory keep on increasing unbounded
Code has idle_timeout of 60s and upkeep of 5s
If I now open /metrics endpoint, memory eventually reduces and the more I open it periodically, the more stable the memory is

So, is my observed behavior still expected even after the introduction of upkeep?

Hmm, the latest (0.6.1) version of axum-prometheus is still on 0.13.x of metrics-exporter-prometheus. Are you using a development version or patching metrics-exporter-prometheus to 0.14.0?

Correct, hence I patched the version of metrics-exporter-prometheus in axum-prometheus to 0.14.0 to test, also able to verify that adding upkeep_timeout compiles correctly, also verified the 0.14.0 in Cargo.lock.
Patch location: https://github.com/gauravkumar37/axum-prometheus/tree/patch-1

Looking at how axum-prometheus uses metrics-exporter-prometheus, this makes sense now: PrometheusBuilder doesn't create the upkeep task unless PrometheusBuilder::build or PrometheusBuilder::install are called.
Not really sure there's a ton we can do here, or to be frank, that I'm willing to do here. The number of ways to build/install the recorder is already too high for comfort because it's trying to be all things to all people. The simplest thing would be if axum-prometheus switched to using PrometheusBuilder::build and discarded the exporter future if it's not in push gateway mode. It would still have to install the recorder, but it would allow it the chance to grab a recorder handle, and it would allow the upkeep task to be created.

This is what I am using inside axum framework:
    PrometheusMetricLayerBuilder::new()
        .with_metrics_from_fn(|| {
            PrometheusBuilder::new()
                .idle_timeout(MetricKindMask::HISTOGRAM, Some(Duration::from_secs(300)))
                .upkeep_timeout(Duration::from_secs(60))
                .set_buckets(&[0.1, 0.2, 0.3])
                .unwrap()
                .install_recorder()
                .unwrap()
        })
        .build_pair()

This does create the PrometheusBuilder and installs it. Is there something more to do here?

Right, you're using install_recorder, when you need to use build.

Ok, I got it to work correctly but I must say, that was an unexpected nuance there :-)
So, something like this works and there is no more unbounded memory growth:
    PrometheusMetricLayerBuilder::new()
        .with_metrics_from_fn(|| {
            let (recorder, _) = PrometheusBuilder::new()
                // if metrics are not updated within this time, they will be removed
                .idle_timeout(MetricKindMask::HISTOGRAM, Some(Duration::from_secs(5 * 60)))
                // prevents unbounded memory growth by draining histogram data periodically
                .upkeep_timeout(Duration::from_secs(5))
                .set_buckets(buckets_in_secs.as_slice())
                .expect_or_log(&format!("Failed to set buckets: '{buckets_in_secs:?}'"))
                // instead of install_recorder, use build because the former doesn't start the upkeep tasks
                .build()
                .expect_or_log("Failed to build metrics recorder");
            let handle = recorder.handle();
            metrics::set_global_recorder(recorder).expect_or_log("Failed to set global recorder");
            handle
        })
        .build_pair()

@Ptrskay3 for changes in axum-prometheus

Thanks for the ping. I'm open to changing axum_prometheus, if that makes it safer for most of the people. Not sure when I'll find the time to do it myself however. Created https://github.com/Ptrskay3/axum-prometheus/issues/51.

@Ptrskay3 Thanks for opening that. 👍🏻
Like I said upthread, I'm also not a fan of how thorny the crate (this crate, not yours) has become in terms of the API... but it does seem like taking the approach shown by @gauravkumar37 would be the simplest one.
I'll also make a note for myself to potentially contribute that as a PR to axum_prometheus.

Just released 0.7 of axum-prometheus, that should fix this issue if people using the default implementation.

When users opt to use PrometheusBuilder::build_recorder or PrometheusBuilder::install_recorder, they're specifically opting into a lower level of control where they're handling how the recorder is installed and/or how the resulting rendered metrics are emitted and consumed. They're also opting into handling upkeep manually as well.
All of that said, the documentation on this isn't good and doesn't explain this for PrometheusBuilder::build_recorder or PrometheusBuilder::install_recorder, so I'm going to open a PR to make those changes.

@tobz Thank you for explanations. I have one more question. What was the main reason why so complicated logic was used for the histogram data and why it can not be calculated immediately in the time when it received?

It mostly boils down to concurrency.
When I originally wrote the Prometheus exporter, I needed something that could support histograms or summaries, which meant needing to keep track of the raw samples to accurately generate the summary data. Since we need the raw samples, this forces us to be able to concurrently accept the incoming histogram samples (since there can be multiple threads updating the same metric at the same time) which is why I wrote AtomicBucket<T>.
If we didn't need to care about aggregated summaries, we could do something better like an array of atomic integers -- one integer per histogram bucket -- and just update them directly, as you allude to. It might be worth doing that overall, and just sort of declaring "we don't support aggregated summaries because they're crap and make things harder, sorry" but I don't have a strong enough opinion/need so it's never really been on my list of things to do.


Fix Features view typo

Fixing typo in Features view section
estAmation -> estImation

@MeneTelk0 Hi, thanks for the fix! :+1:


YUKI

This word yuki is from the Japanese word yuki and means snow.
https://www.nihongomaster.com/japanese/dictionary/word/37036/yuki-雪-ゆき
I do not know if yuki is the best spelling or iuki, but since it has a Japanese origin, maybe a borrowed letter is fine.

Yeah, I see *yuki *whenever it is written...
And I agree - it's borrowed so "y" is fine.
On Mon, Oct 24, 2022 at 8:15 PM Chris Perrette @.***>
wrote:

This word yuki is from the Japanese word yuki and means snow.
https://www.nihongomaster.com/japanese/dictionary/word/37036/yuki-雪-ゆき
I do not know if yuki is the best spelling or iuki, but since it has
a Japanese origin, maybe a borrowed letter is fine.
—
Reply to this email directly, view it on GitHub
https://github.com/tekinged/missing/issues/382, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AXYSPXXT5H6MIIL3EMHN723WE47EZANCNFSM6AAAAAARNQWSYA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Aimeos unable to get result for Shop_Frontend API

The Shop_Frontend API does not support completeAuthorize() or completePurchase(). It also does not directly support acceptNotification(), which is located in the Shop_Server API.
Aimeos uses the capabilities of each API to determine how to handle incoming requests, either from the user returned from a redirect or POST, or an incoming request direct from the gateway (a back-channel request). So this is confusing Aimeos a little, quite rightly.

Since Shop_Frontend gets its transaction result from the acceptNotification back channel, it makes sense that this gateway directly supports the acceptNotification() method. Starting a transaction on Shop_Frontend API and then switching over to the Shop_Server API to get the results is unintuitive for us, and totally obscure to the merchant site without some custom handling for this gateway.
So acceptNotification() will be added to both the ShopClientGateway and the ShopFrontendGateway.

acceptNotification() implemented for all three APIs.
Needs some tests written before a release.

Note that:

acceptNotification() MUST be used for the Shop_Frontend API and there are no complete*() methods.
acceptNotification() MUST be used for the Shop_Client API, but there are complete*() methods for convenience but they are not signed and so cannot be trusted.
acceptNotification() MAY be used for the Shop_Server API as an additional record of the results. Since these are all direct server requests (though with 3D Secure support which MAY require a redirect) the transaction results come direct from the backend direct server calls.

For the Shop_Server API, because there may not be a user redirect involved, the notification request may come from the gateway some time after the transaction result has been returned by the direct server transaction request.


Discord Disable not working

If the server is active and you deactivate the Discord function, the messages will continue to appear.

Will be refreshed by server restart. Like all functions in this manager right now


Lowering

This is an in-progress branch on top of #9 (so #9 should be reviewed first and this one is to give an idea of what the next batch of changes will look like). It introduces a "lowering" pass before actual code generation. It may not look obvious at first, but this is on the easiest path to implement pattern-matching compilation.

This PR is now complete and (now that #9 was merged) ready for review.

Well "complete": there is more we could do:

in the compilation of functions, we could generate the argument-related code during the lowering pass instead of at codegen-time
we could work more to separate the two passes, they are currently intertwined

but I decided to stop here because the bulk of the work is done, and in particular this is more than enough to start with pattern-matching compilation.

Ideally, if we pushed this lowering pass to its limits, we could have a situation where lower-expr does not depend on stacksize, and compile-expr does not depend on env and istail, which would give a nice separation of concerns between thet two passes. But not in this PR.

Thanks a lot! I also thought about the separation of the two passes, but there is a slight complication: env indirectly depends on stacksize, in the locations of local variables. Anyway, the PR is good, and I am happy to merge it.

These PRs were also an excuse to get a broad look at the compiler codebase.
I noticed something which I found surprising. Because you track the shape of functions in the environment, it is only known for "named" functions, not anonymous functions. This means that let f ~x ~y ... = .. in ... works perfectly well, but (fun ~x ~y -> ...) may not work in surprising ways as its "shape" information is not available, but also for example let x = (let f ~x ~y = .. in f) in .. will not behave as expected.
I think that assuming named functions is reasonable, and losing information when "leaking" a function name into another context is also a limitation that we can have, but maybe we should not allow labels on anonymous functions, and raise a warning/error when a function is applied with labels but no shape information is available. (We may have this warning/error already, I must admit that I was too scared to look into the align-args implementation so I just avoided it completely.)

We could also warn when a function with a non-trivial shape is "leaked" (used as a variable in another place than an application site), to force users to manually eta-expand into a non-labelled version.

There is no warning for now when a non-trivial shape is leaked, unfortunately, although this is on my to-do list. Computing better shapes was the reason for the special-case in let-bound function definitions, and the collapse of multiple 'ELambda expressions into one, by the way.
I think the best thing to do would be to just completely disallow labelled arguments (raising an error) when they do not correspond to a variable with known shape information.


How to calculate distance along the path to final destination

The result from a query is the path from the origin to the destination. The path consists of a set of points which allows averting obstacles.
I would like to calculate the distance along this path. It seems trivial, however when the distance is too long the path is limited to the predefined number of points.
How can I calculate the distance from the origin to the destination regardless of the distance or number of mid-points?
The problem: There are two exit doors from a building. I would like to find the closest one using recastlib.

You cann't do that. I think you should find the proper value of maxStraightPath in your scenario instead.