Support for spark.jars.packages is broken?

The issue is the same as this one I believe.
tldr; If developer specify jar dependencies through spark.jars.packages these packages won't be distributed to worker node (or maybe distributed not in the class path). One simple way to test this out is run raydp to read a file from S3 by following code:
import ray
import raydp

ray.init(address='auto')
spark = raydp.init_spark(app_name='RayDP Example',
                         num_executors=2,
                         executor_cores=4,
                         executor_memory=8 * 1024 * 1024 * 1024,
                         configs={'spark.jars.packages': 'org.apache.hadoop:hadoop-aws:3.3.1'}
                         )

# Read from S3
df = spark.read.json(path='s3a://....')
df.show()

and you will get error:
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.s3a.S3AFileSystem not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2571)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2665)
        ... 40 more

If using a script to download the jar in setup commands, then set the path with spark.jars it will work. But I feel this is not the best thing one should do.
Any help will be appreciated.

Hi @pang-wu , thanks for opening this issue. We'll look into this. It seems like some options are not copied to executors.

Awesome! Please let me know if you need anything from me, we are actively using RayDP on a lot of things, looking forward the tool getting better! @kira-lin

Thanks for opening this issue, it seems the packages is not really working in Glue 3.0/4.0 (the 2 versions I've tested). I followed this conversation here https://github.com/oap-project/raydp/issues/111 and it turns out this is related to the AWS Glue itself as the package works well anywhere else including EMR. (extra jar file of that package also works, but not the spark.jars.packages)

I needed something similar for GCS, I ended up adding the following to setup_commands:
  - mkdir -p ~/jars
  - wget -P ~/jars https://repo1.maven.org/maven2/com/google/cloud/bigdataoss/gcs-connector/hadoop3-2.2.14/gcs-connector-hadoop3-2.2.14-shaded.jar

and then in the spark conf:
"spark.driver.extraClassPath": "/home/ubuntu/jars/gcs-connector-hadoop3-2.2.14-shaded.jar",
"spark.executor.extraClassPath": "/home/ubuntu/jars/gcs-connector-hadoop3-2.2.14-shaded.jar",
"spark.hadoop.fs.AbstractFileSystem.gs.impl":"com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS",
"spark.hadoop.fs.gs.impl":"com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem",

It looks like hadoop-aws does not release a shaded JAR (see https://issues.apache.org/jira/browse/HADOOP-15387 and https://issues.apache.org/jira/browse/HADOOP-15924), but one path forward would be to make your own shaded JAR and then repeat similar steps.
I also noticed that when using spark.jars.packages, the JARs were placed in ~/.ivy2/jars so I added ~/.ivy2/jars/* to the extraClassPath but that did not work for some reason, perhaps someone else will have better luck than me :(.

Did you ever figure out how to get S3AFileSystem recognized by the executors?

@abhay-agarwal We have a working solution in prod, by placing the jars needed under the jars directory in our standalone spark deployment (We are not using pyspark installed from pip), some scripts for your reference:
wget -nc https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-aws/3.3.1/hadoop-aws-3.3.1.jar -P "$SPARK_INSTALL_DIR/jars"
wget -nc https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-bundle/1.11.901/aws-java-sdk-bundle-1.11.901.jar -P "$SPARK_INSTALL_DIR/jars"
wget -nc https://repo1.maven.org/maven2/org/wildfly/openssl/wildfly-openssl/1.0.7.Final/wildfly-openssl-1.0.7.Final.jar -P "$SPARK_INSTALL_DIR/jars"
wget -nc https://repo1.maven.org/maven2/org/apache/spark/spark-hadoop-cloud_2.12/${SPARK_VERSION}/spark-hadoop-cloud_2.12-${SPARK_VERSION}.jar -P "$SPARK_INSTALL_DIR/jars"

Did you ever figure out how to get S3AFileSystem recognized by the executors?

@abhay-agarwal We have a working solution in prod (AWS S3), by placing the jars needed under the jars directory in our standalone spark deployment (We are not using pyspark installed from pip). In such cases no extra class path needs to be set (we also do the same to enable Glue support). Some scripts for your reference:

wget -nc https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-aws/3.3.1/hadoop-aws-3.3.1.jar -P "$SPARK_INSTALL_DIR/jars"

wget -nc https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-bundle/1.11.901/aws-java-sdk-bundle-1.11.901.jar -P "$SPARK_INSTALL_DIR/jars"

wget -nc https://repo1.maven.org/maven2/org/wildfly/openssl/wildfly-openssl/1.0.7.Final/wildfly-openssl-1.0.7.Final.jar -P "$SPARK_INSTALL_DIR/jars"

wget -nc https://repo1.maven.org/maven2/org/apache/spark/spark-hadoop-cloud_2.12/${SPARK_VERSION}/spark-hadoop-cloud_2.12-${SPARK_VERSION}.jar -P "$SPARK_INSTALL_DIR/jars"

Cool! This is exactly what we ended up doing as well haha. We use docker on KubeRay so just placed similar commands in the docker files.

Gentle ping @pang-wu , sorry for the late reply to this issue.
I tested with the code you provided above and have 2 nodes for this test:

Node A is ray head node, downloaded all the 4 jars to local and client run on this node
Node B is the worker node, without any jar and raydp-java-worker run on this node

The result is JSON files in my bucket can be read correctly and everything is working well.
So I looked into raydp-java-worker log to find out how jars are copied to other worker nodes.
Here are 4 steps as I concluded:

Step 1: Client will open a port like spark://IP:PORT.
Step 2: Executor will connect to the port and fetch jars to a local temporary directory.
Step 3: Executor will copy jars from temporary location to working directory of this executor.
Step 4: Jars will be loaded through class loader.

This is the executor log about hadoop_hadoop-aws-3.3.1.jar:
2023-07-13 06:11:19,076 INFO Executor [dispatcher-Executor]: Fetching spark://<IP_ADDRESS>:44539/jars/org.apache.hadoop_hadoop-aws-3.3.1.jar with timestamp<PHONE_NUMBER>787
2023-07-13 06:11:19,077 INFO Utils [dispatcher-Executor]: Fetching spark://<IP_ADDRESS>:44539/jars/org.apache.hadoop_hadoop-aws-3.3.1.jar to /tmp/ray/session_2023-07-13_06-12-37_355158_3120799/app-20230713061112806/1/spark-3b7a118e-a4b0-4e2a-a92b-44da3b8586ad/fetchFileTemp2388565500155871038.tmp
2023-07-13 06:11:19,089 INFO Utils [dispatcher-Executor]: Copying /tmp/ray/session_2023-07-13_06-12-37_355158_3120799/app-20230713061112806/1/spark-3b7a118e-a4b0-4e2a-a92b-44da3b8586ad/-16085021461689228852787_cache to /tmp/ray/session_2023-07-13_06-12-37_355158_3120799/app-20230713061112806/1/_tmp/org.apache.hadoop_hadoop-aws-3.3.1.jar
2023-07-13 06:11:19,094 INFO Executor [dispatcher-Executor]: Adding file:/tmp/ray/session_2023-07-13_06-12-37_355158_3120799/app-20230713061112806/1/_tmp/org.apache.hadoop_hadoop-aws-3.3.1.jar to class loader

IIUC, this issue is no longer existed right?

I tried with Spark 3.3.2 and Spark 3.2.1, both of them passed the test.

@Deegue Thanks for circle back, just tested on our side and this is fixed! I will close this issue.
Ivy Default Cache set to: /home/ubuntu/.ivy2/cache
The jars for the packages stored in: /home/ubuntu/.ivy2/jars
org.apache.hadoop#hadoop-aws added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent-b015f014-fa76-426b-a3b9-e17cee6f7b26;1.0
	confs: [default]
	found org.apache.hadoop#hadoop-aws;3.3.1 in central
	found com.amazonaws#aws-java-sdk-bundle;1.11.901 in central
	found org.wildfly.openssl#wildfly-openssl;1.0.7.Final in central
downloading https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-aws/3.3.1/hadoop-aws-3.3.1.jar ...
	[SUCCESSFUL ] org.apache.hadoop#hadoop-aws;3.3.1!hadoop-aws.jar (100ms)
downloading https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-bundle/1.11.901/aws-java-sdk-bundle-1.11.901.jar ...
	[SUCCESSFUL ] com.amazonaws#aws-java-sdk-bundle;1.11.901!aws-java-sdk-bundle.jar (1500ms)
downloading https://repo1.maven.org/maven2/org/wildfly/openssl/wildfly-openssl/1.0.7.Final/wildfly-openssl-1.0.7.Final.jar ...
	[SUCCESSFUL ] org.wildfly.openssl#wildfly-openssl;1.0.7.Final!wildfly-openssl.jar (12ms)
:: resolution report :: resolve 3492ms :: artifacts dl 1617ms
	:: modules in use:
	com.amazonaws#aws-java-sdk-bundle;1.11.901 from central in [default]
	org.apache.hadoop#hadoop-aws;3.3.1 from central in [default]
	org.wildfly.openssl#wildfly-openssl;1.0.7.Final from central in [default]
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   3   |   3   |   3   |   0   ||   3   |   3   |
	---------------------------------------------------------------------
:: retrieving :: org.apache.spark#spark-submit-parent-b015f014-fa76-426b-a3b9-e17cee6f7b26
	confs: [default]
	3 artifacts copied, 0 already retrieved (189078kB/165ms)


Add a dashboard component

Purpose
We need a view to show video state through upload & processing.
We previously used a simple loader for the uploading state but that is proving insufficient as processing can take a long time and we need to inform the user.
Proposal
As we're going to need a dashboard anyway (to monitor/replace uploads and manage tracks such as subtitles), we can start with a simple version of the Dashboard to show video states and update it later on when we have more information to show there.

@jmaupetit sorry if this PR is a little long, as you'll see there's a lot of strings & styles to it's not that much code 😇


Bower support

Could this get added to Bower for those of us that like to use package managers?

+1

:+1:

There is a same name module in npm, but it's sprintf() implementation for JavaScript .

I did. See bower-sprint

@vkeusebio perhaps you could make a pull request so that this could get added to the main repository by @bendc?

@drmonkeyninja oh, yeah forgot about it. Will do

:+1:

-1

+1


[sketch] LuaJIT intrinsics by @fsfod

@fsfod has implemented intrinsics support for LuaJIT (luajit/luajit#116) and on this branch he has applied this to the existing DynASM code in Snabb Switch.
I send this PR to put the work onto the radar of Snabb hackers who are potential early adopters.
I have no idea yet if/when this will land upstream in LuaJIT.

Nice!  Have you noticed any particular performance impact?  What about declaring memory dependencies/effects of DynASM blocks?

I have not done any testing myself. I suspect that LuaJIT upstream is the best place to discuss this work.

Closing this PR. The intention was simply to advertise the fact that proof-of-concept intrinsics support exists for future reference.


Add some kind of Documentation?

Since this library is growing and growing...I think we need a documentation site or something.
We could use some static-site generation, like jekyll or mkdocs, in combination with GitHub Pages.
Any thoughts @crosire ?

I would use inline XML documentation. Has the advantage that you can see the docs while scripting in Visual Studio.

Yea, good point.
Although it would be cool to have a static site which provides a lot of examples (Adding them to the project isn't the best option IMO)

github wiki? :wink:

(y)

@Yothri Nice job with the Getting Started wiki page!

Thank you, i just found some time today to do this. I will add more content when i get some extra time.
:)


Shapeops OR function not behaving as expected when solids touch but do not overlap

Here is a minimal example:
use truck_modeling::builder::{tsweep, vertex};
use truck_modeling::{Point3, Vector3};
use truck_shapeops::or;

fn main() {
    let point_a = vertex(Point3::new(0.0, 0.0, 0.0));
    let line_a = tsweep(&point_a, Vector3::unit_x());
    let square_a = tsweep(&line_a, Vector3::unit_y());
    let cube_a = tsweep(&square_a, Vector3::unit_z());

    let z_offset = 1.0;
    let x_offset = 0.1;
    let y_offset = 0.1;
    let point_b = vertex(Point3::new(x_offset, y_offset, z_offset));
    let line_b = tsweep(&point_b, Vector3::unit_x());
    let square_b = tsweep(&line_b, Vector3::unit_y());
    let cube_b = tsweep(&square_b, Vector3::unit_z());

    let combined_cube = or(&cube_a, &cube_b, 0.01);
    match combined_cube {
        Some(c) => {
            println!(
                "combined_cube has {:?} shell boundaries",
                c.boundaries().len()
            );
        }
        None => {
            println!("combined_cube: None");
        }
    }
}

When I run this, the output I see is:
combined_cube: None

I expected the output to be:
combined_cube has 1 shell boundaries

If I change the value to: let z_offset = 0.9 then I get the correct behavior:
combined_cube has 1 shell boundaries

If I change the value to: let z_offset = 1.1 then I get different behavior which is also correct:
combined_cube has 2 shell boundaries

It is only when I choose let z_offset = 1.0 when the None value is returned. Is this the intended behavior? How can I merge two solids which are touching but not overlapping?

Maybe I'm misunderstanding the purpose of the or function. Is there some other way to join two cubes together that share a face?


May isomorphic-fetch support IE8?

Hello,
As title, is this module support IE8 ?
Or is going to support it?
Thank you very much.  :smiley:

Hi Anna,
This module relies on the github fetch client, which does not support IE8 unfortunately … that library actually only partially supports IE9 so I think IE8 support might be a little bit difficult …… so there's no plan currently to support IE 8.  I'm sorry…
Matt

Hi Matt,
I see.
Thank you reply so soon!!!!  :+1:   :100:
Anna Su

@AnnaSu You can try shimming out your target resource with xdomain if that's an option, then the fetch polyfill may work, no guarantees...

That said, it may be worth considering if you really need IE8-9 support... IE8 support is all but gone now, and IE9-10 are at a point where both could be considerations for removal.
In one of the projects I'm working on, we actually have the following in place to redirect noscript and old IE users to specific destination pages.
  <head>
    ...
    <noscript>
      <META HTTP-EQUIV="refresh" content="0;URL=noscript.html">
    </noscript>
    <!--[if lt IE 10 ]>
      <META HTTP-EQUIV="refresh" content="0;URL=upgradebrowser.html">
    <![endif]-->

IE 8-10 are particularly problematic for CORS based requests for API resources that use the Authentication header, which many are moving forward... There are some shims to work around the issue, but it's still an issue.

If the data you want to ajax isn't sensitive, you could true use our jsonp fetch implementation:-
https://github.com/Financial-Times/n-jsonp-fetch

Just a heads up that there is an IE8-compatible fork of github fetch: https://www.npmjs.com/package/fetch-polyfill
I'm considering forking isomorphic-fetch to replace the browser polyfill with the IE8-compatible version.


Compilation error on Fedora

Cloning into 'tcg'...
HEAD is now at bb24fb0 Modify the hot_cold_t nothrow versions to take a nothrow_t reference. This makes the hot_cold_t versions consistent with the standard versions which take std::nothrow as a const reference.
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Loading:
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
DEBUG: /github/home/.cache/bazel/_bazel_root/73e4315fd1aa3d904e5ced823bbfc603/external/rules_python/python/repositories.bzl:32:10: py_repositories is a no-op and is deprecated. You can remove this from your WORKSPACE file
Analyzing: target //tcmalloc:tcmalloc (1 packages loaded)
Analyzing: target //tcmalloc:tcmalloc (1 packages loaded, 0 targets configured)
Analyzing: target //tcmalloc:tcmalloc (11 packages loaded, 22 targets configured)
Analyzing: target //tcmalloc:tcmalloc (16 packages loaded, 46 targets configured)
INFO: Analyzed target //tcmalloc:tcmalloc (36 packages loaded, 636 targets configured).
INFO: Found 1 target...
[0 / 4] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[4 / 153] Compiling tcmalloc/tcmalloc.cc; 0s processwrapper-sandbox ... (2 actions, 1 running)
ERROR: /__w/mimalloc-bench/mimalloc-bench/extern/tcg/tcmalloc/BUILD:95:10: Compiling tcmalloc/tcmalloc.cc failed: (Exit 1): gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 37 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
In file included from ./tcmalloc/internal/lifetime_tracker.h:21,
                 from ./tcmalloc/huge_page_filler.h:32,
                 from ./tcmalloc/huge_region.h:24,
                 from ./tcmalloc/huge_page_aware_allocator.h:26,
                 from ./tcmalloc/page_allocator.h:25,
                 from ./tcmalloc/static_vars.h:43,
                 from ./tcmalloc/cpu_cache.h:42,
                 from ./tcmalloc/allocation_sampling.h:21,
                 from tcmalloc/tcmalloc.cc:91:
./tcmalloc/internal/lifetime_predictions.h:179:20: error: declaration of 'const int tcmalloc::tcmalloc_internal::LifetimeDatabase::kMaxStackDepth' changes meaning of 'kMaxStackDepth' [-Wchanges-meaning]
  179 |   static const int kMaxStackDepth = 64;
      |                    ^~~~~~~~~~~~~~
./tcmalloc/internal/lifetime_predictions.h:107:18: note: used here to mean 'constexpr const int tcmalloc::tcmalloc_internal::kMaxStackDepth'
  107 |     void* stack_[kMaxStackDepth];
      |                  ^~~~~~~~~~~~~~
In file included from ./tcmalloc/allocation_sample.h:21,
                 from tcmalloc/tcmalloc.cc:90:
./tcmalloc/internal/logging.h:49:22: note: declared here
   49 | static constexpr int kMaxStackDepth = 64;
      |                      ^~~~~~~~~~~~~~
Target //tcmalloc:tcmalloc failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 11.989s, Critical Path: 2.11s
INFO: 6 processes: 5 internal, 1 processwrapper-sandbox.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

I believe this is fixed as of https://github.com/google/tcmalloc/commit/79eb0d672fe42acb51b0826e0b2b2fad786ad2ba.
On Fedora 40 (gcc 14.0.1), I can reproduce the above error at the parent commit:
$ git checkout 79eb0d6~
Previous HEAD position was 79eb0d6 Remove lifetime-based allocator.
HEAD is now at ff7c26e tcmalloc: detect UAF and more OOBs on realloced blocks
$ bazel build tcmalloc
INFO: Analyzed target //tcmalloc:tcmalloc (2 packages loaded, 169 targets configured).
ERROR: /home/nonroot/tcmalloc/tcmalloc/BUILD:141:26: Compiling tcmalloc/background.cc failed: (Exit 1): gcc failed: error executing CppCompile command (from target //tcmalloc:common_8k_pages) /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++14' -MD -MF ... (remaining 29 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
In file included from ./tcmalloc/internal/lifetime_tracker.h:21,
                 from ./tcmalloc/huge_page_filler.h:32,
                 from ./tcmalloc/huge_region.h:24,
                 from ./tcmalloc/huge_page_aware_allocator.h:26,
                 from ./tcmalloc/page_allocator.h:25,
                 from ./tcmalloc/static_vars.h:43,
                 from ./tcmalloc/cpu_cache.h:42,
                 from tcmalloc/background.cc:20:
./tcmalloc/internal/lifetime_predictions.h:179:20: error: declaration of 'const int tcmalloc::tcmalloc_internal::LifetimeDatabase::kMaxStackDepth' changes meaning of 'kMaxStackDepth' [-Wchanges-meaning]
  179 |   static const int kMaxStackDepth = 64;
      |                    ^~~~~~~~~~~~~~
./tcmalloc/internal/lifetime_predictions.h:107:18: note: used here to mean 'constexpr const int tcmalloc::tcmalloc_internal::kMaxStackDepth'
  107 |     void* stack_[kMaxStackDepth];
      |                  ^~~~~~~~~~~~~~
In file included from ./tcmalloc/common.h:38,
                 from ./tcmalloc/cpu_cache.h:36:
./tcmalloc/internal/logging.h:49:22: note: declared here
   49 | static constexpr int kMaxStackDepth = 64;
      |                      ^~~~~~~~~~~~~~
Target //tcmalloc:tcmalloc failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 6.707s, Critical Path: 3.05s
INFO: 16 processes: 12 internal, 4 processwrapper-sandbox.
ERROR: Build did NOT complete successfully

And no error at 79eb0d672fe42acb51b0826e0b2b2fad786ad2ba itself:
$ git checkout 79eb0d6
Previous HEAD position was ff7c26e tcmalloc: detect UAF and more OOBs on realloced blocks
HEAD is now at 79eb0d6 Remove lifetime-based allocator.
$ bazel build tcmalloc
INFO: Analyzed target //tcmalloc:tcmalloc (2 packages loaded, 163 targets configured).
INFO: Found 1 target...
Target //tcmalloc:tcmalloc up-to-date:
  bazel-bin/tcmalloc/libtcmalloc.lo
INFO: Elapsed time: 7.963s, Critical Path: 4.41s
INFO: 16 processes: 1 internal, 15 processwrapper-sandbox.
INFO: Build completed successfully, 16 total actions


feat(dapp-console): small fixes

#353  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @tremarkley and the rest of your teammates on  Graphite


Incompatibility with multi-core compilation

This gem will not work correctly in the context of parallelizing compilation, i.e.
https://github.com/steel/sprockets-derailleur/issues/15

+1

In application.rb:
class Application < Rails::Application
    # ...
    # require first derailleur, then non-setupid assets that way we're
    # mockey patching the latest version of Sprockets::Manifest
    require "sprockets-derailleur/manifest"
    require "non-stupid-digest-assets"
    NonStupidDigestAssets.whitelist += ["sweet_stuff.js"]

In Gemfile:
gem "non-stupid-digest-assets", require: false # we need to require it later, AFTER sprockets-derailleur

It gets the job done.  It makes sure the monkey patching happens against derailleur's own monkey patching. :-)

Not sure how you could account for this in the gem, but perhaps something like what I posted above could be added to the documentation?


Line items dropdown open by default in modal window

Turns out this is expected for certain input values.


Legacy command wrapper

Wrap legacy command as recipe to easier handling of documentation

Commands#Exec have argument args that is not compatible with Recipe interface.

Mark as wontfix and close


high memory usage

Hey,
seems that the exporter consuming up to 2.24GiB memory by default

Hi @lukasmrtvy,
yes, memory is quite high comparing to others, but I think it comes from the apt_info.py script. A single call to this script takes 5 to 7 seconds. And takes up memory. On my cluster the memory is currently at 450 MB.
Not knowing your setup, my first guess would be to change the scrape interval. Mine is on 2 minutes.

Hi @lukasmrtvy,
I've replaces the whole web stack of the exporter. Can you try again with the new version?

Hi, with 7d scrape interval its 112.8MiB on 0.2.1, previous version was ~50MiB

I've replaced everything and added cache.close() to the apt_info.py script.
With my previous version the container also got OOM killed and restarted. With this setup the memory increases continuously but then dropping again.
I have no idea what I can change to prevent the memory increase. It has to be the apt_info.py script, because this is the only common code with these two versions.

Btw, see this https://github.com/prometheus-community/node-exporter-textfile-collector-scripts/pull/181

I've updated the script with the changes from the pull request, but it seems that this was not the cause for the OOM.

With the latest release it looks like the memory is now stable. @lukasmrtvy can you please test?


Refactor ARAGORN for stability

Right now, ARAGORN is a bit twitchy when there is too much load.  One obvious problem is that when all the (4) workers are being used, the whole thing hangs.   And it's not hard to get to 4.
So we at least need a queuing system for input queries.
The remainder of ARAGORN makes calls out to other services, sometimes synchronously, sometimes asynchronously.  And results from an async query out will usually come in to a worker that is not the worker that sent the query, so there needs to be inter-worker collaboration.   And that needs to be pretty stable (historically it's not been, because I think of trying to hold open connections to the communications queue.)
So all of this needs to be cleaned up, rethought, and made more bulletproof.

We should use a connection pool here:
https://github.com/ranking-agent/aragorn/blob/ef4b7d4e0af1c99e1c2f6a9d5b2949ef63793db6/src/aragorn_app.py#L113


Quickstart Guide - specify what a deployment is?

In the following step:
[
Create a new Python application
Create a new Python file called quickstart.py. Then open it up in your preferred editor or IDE.
Replace the contents of quickstart.py with the following code. Modify the code to add your key, endpoint, and deployment name:
]
I was assumed that the 'deployment name' was the endpoint deployment. It would be helpful if you added a step in this document that spoke to the 'model deployment' step and a pointer to that name. It would save neophytes like me a hour or so of digging around.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 4833a089-660d-364f-82a4-1fc26522b6ec
Version Independent ID: 2c415b16-f888-a44b-64d7-a9d1732dacb6
Content: Quickstart - Get started using ChatGPT (Preview) and GPT-4 (Preview) with Azure OpenAI Service - Azure OpenAI Service
Content Source: articles/cognitive-services/openai/chatgpt-quickstart.md
Service: cognitive-services
Sub-service: openai
GitHub Login: @mrbullwinkle
Microsoft Alias: mbullwin

@glennatlayla
Thanks for bring to our attention to this. The name of the deployment model is stated in the document. Please see the screenshot below.

Your feedback is also valuable. Please add your ideas in below link, so our production team can review it and update the same.
Ideas · Community (azure.com)

@glennatlayla
We are going to close this thread  if there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.


Added callback code so now it's .then(value, duration, &functon_name)…

… and .hold(duration, &function_name)

@asoap Hi, sorry for the late reply. I've added the callback feature for each transition. Please refer to the transition_callback example. Thank you for this feature request!


Error about "Relation not homogeneous"

Mike Newman reported an issue in anupq 3.1.1 via email, when running the test program below (warning: running this may take multiple hour, and produces more than 6 GB of log data). To quote Mike:

Running the attached input file reports a (in fact many) Relation not homogeneous.
as an attached extract from the output file shows.
In the process of preparing this report I noticed that in the final structure table
some of the right-hand sides are not normal words.
There are some squares at the end and sometimes a final term .1^0 or ..0^0.
The corresponding input to Magma V2.21-2 on the same machine
does not give the problem.
I don't know what standalone version the Magma process is based on.

LoadPackage("anupq");
SetInfoLevel(InfoANUPQ, 6);
procId := PqStart( FreeGroup(6),1000000000 : Exponent := 4, Prime := 2 );
i := procId;
PqPcPresentation(procId : ClassBound := 5, OutputLevel := 1);
PqSetMaximalOccurrences( i, [3,3,3,3,3,3] );

for c in [6..8] do
  PqComputePCover(i);
  PqDoExponentChecks(i);
  PqEliminateRedundantGenerators(i);
od;

#class 9
PqComputePCover(i);
PqDoExponentChecks(i: Bounds := [1,8]);
PqEliminateRedundantGenerators(i);

#class 10
PqComputePCover(i);
PqDoExponentChecks(i: Bounds := [1,8]);
PqEliminateRedundantGenerators(i);

PqSetOutputLevel(3);

#class 11
PqSetupTablesForNextClass(i);
PqTails(i, 11);
PqDoConsistencyChecks(i, 11, 0);
PqTails(i, 10);
PqDoConsistencyChecks(i, 10, 0);
PqTails(i, 9);
PqDoConsistencyChecks(i, 9, 0);
PqTails(i, 8);
PqDoConsistencyChecks(i, 8, 0);
PqTails(i, 7);
PqDoConsistencyChecks(i, 7, 0);

PqDisplayStructure(i);

PqQuit(procId);

quit;

An excerpt from the log:
#I  Jacobi was 131 9 1
#I  The quotient relation is  .13929 .13938 .14310 .14723 .16897 .17495 .19858 .28331 .28344 .28372 .28381 .28460 .29024 .29041 .2
9822 .30143 .30144 .32338 .41735 .41744 .41768 .41777 .41819 .41820 .41967 .41982 .41998 .41999 .42012 .42542 .42621 .42668 .42669
 .42773 .43372 .45434 .45435 .54060 .54249 .54265 .54279 .54616 .55163 .55266 .55284 .55455 .55470 .56093
#I  Jacobi was 132 9 1
#I  Jacobi was 133 9 1
#I  Jacobi was 134 9 1
#I  Jacobi was 135 9 1
#I  Relation not homogeneous of class 11. Relation ignored.
#I  Jacobi was 136 9 1
#I  Relation not homogeneous of class 11. Relation ignored.
#I  Jacobi was 137 9 1
#I  Relation not homogeneous of class 11. Relation ignored.
#I  Jacobi was 138 9 1


Fix gateway tests on old versions

Please provide a description of this PR:

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-security-multicluster_istio
cd0cf7323935924692e9b3b9b04f3572a139b7e9
link
true
/test integ-security-multicluster_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-security-istiodremote_istio
cd0cf7323935924692e9b3b9b04f3572a139b7e9
link
unknown
/test integ-security-istiodremote_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-security-istiodremote_istio
cd0cf7323935924692e9b3b9b04f3572a139b7e9
link
unknown
/test integ-security-istiodremote_istio

integ-security-multicluster_istio
cd0cf7323935924692e9b3b9b04f3572a139b7e9
link
unknown
/test integ-security-multicluster_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest


Fix Firefox composition bug with emojis

We have reports of emojis being inserted multiple times on Windows + Firefox using the Windows emoji picker. We actually fixed this issue in the past: https://github.com/facebook/lexical/pull/738. However, we can't use setTimeout, as it means that we invalidate composition too late.
In order to make the composition sequence work the same in FF as it does in Chrome/Webkit – we can defer ending composition till the subsequent input event comes in. Thus making the sequence behave like compositionstart -> input -> compositionend, even though input occurs after compositionend in Firefox.

It looks like your previous commit to rebuild the package-lock ran the CI correctly. Did you consider just rebasing on top of that instead of disabling the cache? Ideally, I'd like to see if we can figure out what the actual problem with the cache is.

@acywatson This is based on the commit. I have no idea how or why the CI doesn't work for this PR, I've closed it and created 3 new branches and none work. So I just deleted the cache from the script and now they run…


:bug: hostname does not work on debian

Kairos version: 1.5.1

CPU architecture, OS, and Version: amd64, kairos-debian-v1.5.1-k3sv1.26.1+k3s1

Describe the bug

I am having issues using the hostname configuration option as per documentation.
If I use this option the file /etc/hosts is not created.
I have tried also with:
#cloud-config

hostname: "your_hostname_here"

It sets the hostname, but does not create the /etc/hosts file.
To Reproduce

Install kairos with the ISO kairos-debian-v1.5.1-k3sv1.26.1+k3s1 and apply the a configuration like the following one:
#cloud-config

hostname: test
# Define the user accounts on the node.
users:
- name: "kairos"
  passwd: "kairos"
  ssh_authorized_keys:
  - "ssh-rsa AAAA..."

# Enable K3s on the node.
k3s:
  enabled: true
  args:
  - --disable=traefik,servicelb

stages:
  initramfs:
    - files:
        - path: /etc/systemd/network/10-config.network
          permissions: 0644
          content: |
            [Match]
            Name=enp1s0

            [Network]
            Address=<IP_ADDRESS>/24
            Gateway=<IP_ADDRESS>
            DNS=<IP_ADDRESS> <IP_ADDRESS>
  boot:
    - name: "Setup hostname"
      hostname: "test"

Looking into this. Did a first test using hostname at the root level and it works as expected. Will now test at boot stage

putting it under boot didn't work. Investigating

the problem is there for opensuse, so I'm wondering if this is something that can only be run at root level
@mudler @Itxaka do you guys know if setting the hostname is something is possible inside of the stages? from what I can see in cloud config documentation it doesn't say anything specific about that, but in our docs we do have a couple of examples were we do

I know that at least on the boot stage its possible, as recovery sets the hostname to cos-recovery https://github.com/kairos-io/packages/blob/main/packages/system/cloud-config/oem/06_recovery.yaml#L23
Also I get the proper hostname and /etc/hosts with the following install config:
#node-config

install:
  auto: true
  reboot: true
  device: /dev/vda

stages:
   initramfs:
     - name: "Set user and password"
       users:
        kairos:
          passwd: "kairos"
       hostname: kairos-{{ trunc 4 .Random }}

So it also works under initramfs.
This all under opensuse-leap. So maybe its a debian issue?
You can check the systemd-hostnamed logs to see if it was set or tried to set
Feb 17 14:21:26 kairos-fq8g systemd-hostnamed[1858]: Hostname set to <kairos-fq8g> (static)

perfect thanks, will check

sorry, comment got cut.
Also you can check the logs of the cos-setup-STAGE service to see if it run properly, in the case of initramfs cos-setup-initramfs and for boot cos-setup-boot
or even re-run the stage manually with debug enabled: elemental --debug runs-stage STAGE:
INFO[2023-02-17T15:13:01Z] Executing /oem/userdata.yaml
INFO[2023-02-17T15:13:01Z] Applying '' for stage 'initramfs'. Total stages: 1
INFO[2023-02-17T15:13:01Z] Processing stage step 'Set user and password'. ( commands: 0, files: 0, ... )
DEBU[2023-02-17T15:13:01Z] Stage: {"Commands":null,"Files":null,"Downloads":null,"Directories":null,"If":"","EnsureEntities":null,"DeleteEntities":null,"Dns":{"Nameservers":null,"DnsSearch":null,"DnsOptions":null,"Path":""},"Hostname":"kairos-{{ trunc 4 .Random }}","Name":"Set user and password","Sysctl":null,"SSHKeys":null,"Node":"","Users":{"kairos":{"Name":"","PasswordHash":"kairos","SSHAuthorizedKeys":null,"GECOS":"","Homedir":"","NoCreateHome":false,"PrimaryGroup":"","Groups":null,"NoUserGroup":false,"System":false,"NoLogInit":false,"Shell":"","LockPasswd":false,"UID":""}},"Modules":null,"Systemctl":{"Enable":null,"Disable":null,"Start":null,"Mask":null},"Environment":null,"EnvironmentFile":"","DataSources":{"Providers":null,"Path":""},"Layout":{"Device":null,"Expand":null,"Parts":null},"SystemdFirstBoot":null,"TimeSyncd":null,"Git":{"Auth":{"Username":"","Password":"","PrivateKey":"","Insecure":false,"PublicKey":""},"URL":"","Path":"","Branch":"","BranchOnly":false}}
``

also have a look at your /oem/userdata.yaml file to see if its correct. Maybe oem didnt mount for some reason or it got corrupted.

ok I was just too impatient and didn't wait for the stage to finish to check, but now I can confirm the issue. Looking into it.


Cmake tools selects on 1.20.6 a different compiler than on 1.18.44 when reconfiguring

Brief Issue Summary
The issue is still in the latest version. For full reference see:
https://github.com/microsoft/vscode-cmake-tools/issues/4032

@the-programmer This is expected.
We made slight changes to the cmake.generator and cmake.preferredGenerator settings, as well as the kit.preferredGenerator settings. Therefore, there is a chance that you'll need to manually set any of these settings to ensure that you're getting the expected behavior.
In short, if you set cmake.generator, we'll attempt to use it blindly. If you use cmake.preferredGenerator, we'll loop over them until we find one on path.

Ok, thanks for the information.
In case someone else stumbles across this issue, below is how I resolved it.
When reconfiguring in 1.18.44, the following command gets executed.
[proc] Executing command: "C:\Program Files\CMake\bin\cmake.exe" -DCMAKE_BUILD_TYPE:STRING=Debug -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=TRUE -Wno-dev --no-warn-unused-cli "-SC:/Development/[...]" "-Bc:/Development/Pedak/[...]" -G Ninja

All you need to do is to grab the -G parameter (Ninja in my case) and set that as cmake.generator.


youtube and twitch missing from socials?

I'm trying to add youtube and twitch to my socials, but I can't just add it to the config.
how do I do that?

maybe, needs their logo icon as .svg.
If you have icons, I could write code.
like
<svg
  class="icon" style="" viewBox="0 0 1024 1024" version="1.1" width="36"
  height="36" id="svg8">
  <path
      style=""
      d="PATHPATHPATHPATH"
      id="rect824"/>
</svg>

maybe, needs their logo icon as .svg.
If you have icons, I could write code.
like
<svg
  class="icon" style="" viewBox="0 0 1024 1024" version="1.1" width="36"
  height="36" id="svg8">
  <path
      style=""
      d="PATHPATHPATHPATH"
      id="rect824"/>
</svg>

no .svg but youtube offers  brand resources and guidelines as .png .eps and .ai
Twitch Brand Resources do have .svg though

no .svg but youtube offers  brand resources and guidelines as .png .eps and .ai
Twitch Brand Resources do have .svg though

I sent pull request to reviewer. #314
So, you can copy that code or wait merge.
I tried selef check. it's works.

I sent pull request to reviewer. #314
So, you can copy that code or wait merge.
I tried selef check. it's works.

thanks a lot :)

thanks a lot :)


 The response is always:
an exception occured - APIError(code=-1013): Filter failure: PERCENT_PRICE
Even with the "test_order" function.
Example:
       order = client.create_test_order(
            symbol='BNBBTC',
            side=SIDE_BUY,
            type=ORDER_TYPE_LIMIT,
            timeInForce=TIME_IN_FORCE_GTC,
            quantity=100,
            price='0.00001')

response:
an exception occured - APIError(code=-1013): Filter failure: PERCENT_PRICE
Example 2:
       order = client.order_limit_buy(
            symbol='ETHUSDT',
            quantity=0.005,
            price='98.20')

response:
an exception occured - APIError(code=-1013): Filter failure: PERCENT_PRICE
Environment (please complete the following information):

Python version: 3.7
Virtual Env:  n/a
OS: Win 10 64 bit
python-binance version latest (0.79)

Excuse my English, I'm using the translator
you have to consult the information of the symbol
get_symbol_info ("BNBBTC") ["filters"] [1]
{'filterType': 'PERCENT_PRICE', 'multiplierUp': '5', 'multiplierDown': '0.2', 'avgPriceMins': 5}
your price has to be in this range (Up / Down)
current price * 5 <= your price <= current price * 0.2
The current price of BNB is approximately 0.0051000
it would look like this
0.0051000 * 5 = 0.0255000 maximum price
0.0051000 * 0.2 = 0.00102 minimum price
I hope it helps you


Update Ruma and bump dependencies to Tokio 1.x

Apparently the Tokio 1.x upgrade path broke the current build (#16), and ruma-client released the 0.5 stable, so this PR should upgrade dependencies to get rid of any old tokio 0.2.x dependencies and bring back the client to a buildable state.

@badboy is totally lying in the grass slurping ice cream, so he can't stop me from proclaiming benevolent dictator for lyfe and merging this.
Fixes #16.


Discord Rich Presence Timestamp

Is your feature request related to a problem? Please describe.
The current Discord Rich Presence timestamp on YouTube Music Desktop only displays the time left of a song, which can sometimes be limiting for users who want a better understanding of their listening experience. By changing it to show the "listening time now" and "end time," users would have a clearer picture of the current point in the song and the overall duration.
Describe the solution you'd like
I would like to request a modification to the Discord Rich Presence timestamp on YouTube Music Desktop, so that it displays the "listening time now" and "end time" in the format of "01:12/03:45". This change would provide users with valuable information about their listening progress and the remaining duration of the song.
Describe alternatives you've considered
I have considered alternative solutions, but none of them offer the same level of clarity and functionality as the proposed modification. The suggested format of "01:12/03:45" would enable users to easily comprehend their current listening time and the total duration of the song.
Additional context
I understand that implementing this feature may depend on various factors, including technical feasibility and compatibility with the YouTube Music Desktop application. I would greatly appreciate it if you could provide me with any information or insights regarding the possibility of incorporating this modification into the Discord Rich Presence timestamp feature on YouTube Music Desktop.
Thank you for your attention to this matter.

Is your feature request related to a problem? Please describe. The current Discord Rich Presence timestamp on YouTube Music Desktop only displays the time left of a song, which can sometimes be limiting for users who want a better understanding of their listening experience. By changing it to show the "listening time now" and "end time," users would have a clearer picture of the current point in the song and the overall duration.
Describe the solution you'd like I would like to request a modification to the Discord Rich Presence timestamp on YouTube Music Desktop, so that it displays the "listening time now" and "end time" in the format of "01:12/03:45". This change would provide users with valuable information about their listening progress and the remaining duration of the song.
Describe alternatives you've considered I have considered alternative solutions, but none of them offer the same level of clarity and functionality as the proposed modification. The suggested format of "01:12/03:45" would enable users to easily comprehend their current listening time and the total duration of the song.
Additional context I understand that implementing this feature may depend on various factors, including technical feasibility and compatibility with the YouTube Music Desktop application. I would greatly appreciate it if you could provide me with any information or insights regarding the possibility of incorporating this modification into the Discord Rich Presence timestamp feature on YouTube Music Desktop.
Thank you for your attention to this matter.

I don't think that's possible for discord rich presence interface


bin/shabka: allow build-vm command

I don't think duck is ever going to build on CircleCI ever again, It is running stuff in a Qemu process during its build.

Maybe turn off the VM on the CI or remove the host?


Update to Go v1.21

ref: https://issues.redhat.com/browse/ACM-9734
/hold for upstream sync

/unhold


Move WriteBatch write methods into WriteBatch and remove redundant engine parameter

What problem does this PR solve?
A few cleanups of the WriteBatch interface. Primarily:

The methods to commit a WriteBatch are moved to the WriteBatch itself
The methods to commit a WriteBatch do not require a redundant engine paramater

These changes make the interface more foolproof and reasonable for other engines to implement. There is no chance of accidentally pairing a write batch with a different engine from the one that created it.
What is changed and how it works?
Related changes
https://github.com/tikv/tikv/issues/6402
Check List
Release note

No release note

/test

/test

I've updated this for master. Please review cc @hicqu @Little-Wallace @5kbpers

/test

/test

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

/lgtm

/merge


Bug: can't interact with Quiet Outline in standalone window

With the new update, I use multiple Obsidian windows across Virtual Desktops, but only the main window can have expandable sidebars. The temporary solution for me, is to move the sidebars contents into a new window, which then can be accessible over all Virtual Desktops. But there is a bug with Quiet Outline, is that, when it is in standalone window, you can't interact with it. The core plugin doesn't have this issue.
Besides fixing the bug, another way to solve my issue navigating with Quiet Outline with absence of sidebars is would be the ability to open a new instance of Quiet Outline in each window as the new pane. I'm not sure, if it is worth it to implement for now.

I haven't figure out the mechanism of multi-window yet and I guess this bug has something to do with vue.js. Once I understand it , it will be fixed. 😉

fixed in 0.3.7


Consider improving performance

Perhaps it's overkill to change the ranking method, but I tested this on a 180 MB text file (probably not a good idea anyway) and it was still not done after some 20 hours of CPU time.
For comparison, an off-the-shelf BigramCollocationFinder.from_words(tokens).nbest(BigramAssocMeasures().pmi, 10000) takes about 5 minutes on the same machine and corpus, and it's probably easy to do better. (I cobbled together an example at https://framagit.org/nemobis/bots/-/blob/master/ngram_tlds.py .)

Interesting @nemobis! Have you compared the results of each approach at all?

I tried, but I was doing Italian (more difficult) and the input I used was too dirty for the simplistic BigramCollocationFinder above, so I cut my losses and threw away everything. :) The suggestions weren't outrageously bad but it may need some tweaking for the tokenization.


🐛 BUG: Unable to write to R2 locally

Which Cloudflare product(s) does this pertain to?
R2, Wrangler core
What version(s) of the tool(s) are you using?
3.1.1
What version of Node are you using?
18.16.1
What operating system are you using?
Mac
Describe the Bug
I am writing a small text object to R2 during local development, I am using fetch syntax for my worker:
await R2_BUCKET.put(<name of file>.txt, JSON.stringify(result))
I log the output of the Put operation and get what appears to be a successful result:
{"customMetadata":{},"httpMetadata":{},"uploaded":"2023-07-26T23:47:39.528Z","checksums":{"md5":"9eaa7...5d37"},"httpEtag":"\"9ea5337ab....2560d537\"","etag":"9ad3....7ab256bed7","size":260,"version":"646....8f5f5dc2d1","key": <name of file>}
My binding looks like this in my wrangler.toml:
[[r2_buckets]]
  bucket_name = <name of bucket>
  preview_bucket_name = <name of bucket>
  binding = "R2_BUCKET"

When I go to the bucket the file does not exist. This problem only seems to happen locally. Once deployed, things work as expected.
I have confirmed my bindings and can see that they are correct during dev startup.
Please provide a link to a minimal reproduction
No response
Please provide any relevant error logs
No response

Thanks fo raising this issue! This is intentional, and is how wrangler dev is intended to work.

When you run wrangler dev, all R2 etc... operations run locally, and won't affect your remote resources.
When you run wrangler dev --remote, all R2 etc... operations run remotely, and will affect your remote resources.


I use a semaphore to make sure only one nonce is generated and the response is received before trying another one but still getting a nonce error.
From fiddler capture, the requests in order (added nonce header for debugging):
POST https://api.kraken.com/0/public/AssetPairs HTTP/1.1
nonce: 108622843

POST https://api.kraken.com/0/public/Depth HTTP/1.1
nonce: 146190498

POST https://api.kraken.com/0/public/Trades HTTP/1.1
nonce: 156578556

POST https://api.kraken.com/0/private/AddOrder HTTP/1.1
nonce: 171383828

Response to AddOrder:
{"error":["EAPI:Invalid nonce"]}

What else could be wrong??

Solved it after chatting to Kraken chat (only discovered after coming here)...
For anyone passing by this issue, two notes:

The public calls are rate-limited the same for all people connecting to Kraken regardless of Tier access so there's no point signing the calls like I have done above

The nonce must be included in both the API-Sign signature hash and as a POST url-encoded parameter

For posterity, and those looking for answers, I fixed my nonce issues by bumping the tolerance setting on my api key. Its not posted very well in the docs, but you can specify a threshold that will not trigger a nonce exception. I set mine to 15000000 (15s) since this library signs requests in nano seconds.

I just use multiple API keys.
On Sat, 2 Sep 2017, 00:18 brentechols<EMAIL_ADDRESS>wrote:

For posterity, and those looking for answers, I fixed my nonce issues by
bumping the tolerance setting on my api key. Its not posted very well in
the docs, but you can specify a threshold that will not trigger a nonce
exception. I set mine to 15000000 (15s) since this library signs requests
in nano seconds.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/nothingisdead/npm-kraken-api/issues/20#issuecomment-326685934,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF2d_kxdLjeysN9uKfyPps0g7VHrcot0ks5seHSvgaJpZM4KyTWo
.

--
Vlad Nistor
+40721278740
<EMAIL_ADDRESS>


feat: rename payment pointer to wallet address

Changes proposed in this pull request

Replaces all mentions of 'payment pointer' with 'wallet address'.
Context

Fixes #286.

looks like there will be stuff to rename in https://github.com/interledger/open-payments/pull/289/files


Cannot use a binding directory, or any other ctl-opt statements

I tried having a ctl-opt statement on the first line of code to specify a binding directory to use for an exported sub procedure in a service program.
That doesn't seem to be viable when I look at the attached screenshot of the spool listing.  It would be very nice to have this ability in order to have snippets that demonstrate usage of existing service programs.

I could see this being useful. Our organisation don't use that many binding directories, so REPL's ability to bind service programs based off /included copybooks has worked well enough for us.

This is something I initially looked at, but we didn't have enough use cases to justify it at the time.
Control specifications seem like things that are generally fairly static within an organisation, so I'd like to put in a solution that allows a user to set a default control specification for all users, and also allows users to override that default with a snippet-specific specification whenever they want.

I've documented these use cases on the wiki here: https://github.com/tom-writes-code/rpgle-repl/wiki/How-to-use#setting-control-statements


69 readme structure

Closes #69

The only real comment I can add to this, and I don't know if it is really required is the consistency we have around the Steps. Some go into detail about creating connections with dsteem and some assume the develop knows what those mean. Might be something we can address at a later stage to get more consistency across the tutorials?


fix getTheme to respect preferred mode

previously the getTheme function would only inspect the workbench.colorTheme. This would not work for users who use window.autoDetectColorScheme. Now, we inspect this value before deciding which color theme to use.
Shuold fix https://github.com/continuedev/continue/issues/1458

Hey @0xBigBoss , we're getting reports from some users (myself included) are no longer getting proper syntax highlighting in the sidebar in VS Code after this commit (confirmed via git bisect).
Here's an example of some Python syntax highlighting before and after these changes:
Before

After

Mind taking a look into this?

Hey @Patrick-Erichsen thanks for reporting. I did a little investigation and found some things. Good news is the theming looks really close, but I'm unsure what happened to the sidebar chat between the time I merged that PR. There seems to be a regression on master that removes all styling from the sidebar, I'm having trouble confirming on latest master.
All this PR aimed to do was properly switch based on the Window: Auto Detect Color Scheme property. I didn't update any other theme logic as far as which CSS was applied.
I did a little investigation and found that there is something else going on with the built-in theming added by continue and it's reproducible by checkout out the commit before this PR and the commit after this PR was merged. My guess was the previous behavior was buggy beforehand with the built-in themes. Anyways, I had great success with these changes locally. You can see a PR here where we can continue the conversation. https://github.com/continuedev/continue/pull/3210


Removing pytest, cookiecutter and PyYAML from requirements

Removing pytest and cookiecutter (requirement of pytest-cookies), PyYAML (required by watchdog). Used pip-chill to trim down the list.

Hi @rbanffy, care to resolve the merge conflicts?

Sure. Sorry not to have done this before - hectic week.

@rbanffy, thank you so much for this pull request. However, in resolving some other stuff I closed the issue in 04d061efaadbf5f223aac50089ce980393a019e3.
Nevertheless, I appreciated this PR and followed it closely in resolving some other things.

No problem. Whatever makes cookiecutter better, I'm in. :-)


Add go2rtc

https://github.com/AlexxIT/go2rtc
https://github.com/AlexxIT/go2rtc/issues/862

@AlexxIT you'll need to name the file go2rtc.png.

@walkxcode Please consider #720 instead this PR. @AlexxIT FYI.


Add filebundle (otioz and otiod) adapters

overview
Introduces the idea of an OTIO "file bundle".  This has two flavors: .otioz and .otiod.  This PR includes documentation and implements adapters that can convert into those formats.
otioz

Adds .otioz suffix file
Bundles media and an otio file together into a libzip compatible zipfile (using python's zipfile module)
only works with files that can be found through the filesystem (file: protocol target_url fields, as parsed by urllib in python)
media files must have unique basenames (because they get put into a flat namespace)
zipfile structure (for foo.otioz):

foo.otioz/

version.txt
content.otio
media/

file1
file2
file3

the content.otio encodes the structure of the timeline and exclusively references media which is present in the media subdirectory.
the content.otio is compressed, but the rest of the media files are not
version.txt is only present in otioz files and exclusively encodes a file bundle version string into the file in case of future changes to the layout of file bundles

otiod

identical to the otioz in layout, except expressed without the zipfile container as files and directories in the filesystem.
This way a .otioz can be expanded using otioconvert through the adapter system without needing to use zip directly, into a form that any system that reads otio from the filesystem can consume.

TODO:

[x] make the file:// vs file: consistent when generating absolute paths
~[ ] add a file:///path/to/otioz?file=media/somefile.mov form of ExternalReference.target_url when reading OTIOZ files... this allows the otiod adapter the ability to directly decompress otioz files, and otioconvert could detect it when going from otioz to otiod.~ -- going to do this in a future PR/issue

Codecov Report

Merging #561 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #561   +/-   ##
=======================================
  Coverage   81.28%   81.28%
=======================================
  Files          72       72
  Lines        2731     2731
=======================================
  Hits         2220     2220
  Misses        511      511

Flag
Coverage Δ

#py27
81.29% <ø> (ø)
:arrow_up:

#py36
81.29% <ø> (ø)
:arrow_up:

#py37
81.29% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 689d3c4...6f0b186. Read the comment docs.

I like this a lot. It seems tremendously useful and is very simple in concept.
If you require the internal otio to have the same name as the zip file, then it will break if/when someone renames the zip file. Using a consistent name, like content.otio would avoid this.
On the reading side, how would someone extract the media files? Maybe there could be a helper function or option on the reader to unzip the bundle? That would also allow for a future switch from zip to some other bundle mechanism, if needed.

Great note!  I like content.otioz.
As far as a command to unzip the bundle, do you mean some kind of api function you can call?  otio.adapters.expand_bundle() or something?
I like the idea of putting an API call that encapsulates the schema though.  Also I wonder if we can put an extra magic number into the file somehow to detect which 'version' of OTIOZ it is.

You could put a version in OTIOZ metadata on the top level object, or wrap the content in a Bundle.1 schema type or something. I bet the IMF and DCP folks would have advice on what else would be worth tracking, though those formats tend to have a bunch of manifest details, checksums, etc. that might make this more complicated than it needs to be.

We'll version in the OTIO file rather than come up with a separate versioning scheme specifically for otioz.

Rebased onto current master branch.

Because we don't have a concrete need for this at the moment, and there is concern that this might expand the scope of OTIO (because this is a file format that includes media), we've decided to close this for now.  The branch will remain in my fork so that if there is a desire for this in the future, this work can be brought forward and re-applied.  Please let us know if you have any questions!

I came across this SMPTE spec for AXF which attempts to address a similar problem:
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7879152
It looks like AXF attempts to address issues of very large data sets, as well as a bunch of extra metadata about each item in the archive.
I'm just posting this here for future reference, since OTIOZ is likely to come up again in the future.

Following discussion in the community, in particular some use cases surfaced, this PR has been reopened, rebased and updated to pass tests on all platforms.

Thank you to the community for the thorough review.


Need for a more elegant upgrade path for new Dynamic Configs

Adding new dynamic configurations to Druid can result in difficulties upgrading if proper precautions in the code are not taken. Also, the precautions taken in the code to date are a bit messy. Ideally we come up with a more elegant way to add new dynamic configurations.
This is an explicit example: #11135
Adding the new configuration that is an int with a non-zero default requires manually catching null if the config is missing and setting it to the default non-zero value. Not doing this would mean that the missing value would result in the config being set to 0 during object serialization when the coordinator restarts for the upgrade. In this case, 0 is not a desirable default and will drastically change coordinator behavior as compared to pre-upgrade.
It would be ideal if there was a more elegant way to achieve the non-zero than what is implemented today.

The gotcha seems to be that the Builder is not used with defaults when we are serializing from what is in metastore. Someone who is starting with a fresh deployment would get the new default for the added config. But someone just doing an upgrade doesn't benefit from the defaults in the Builder class


Add swap_scene() method for easier changing of instanced scenes

Closes #24795
Works as described in the linked issue.

I noticed that you are supposed to add docs manually to any new method, so I pushed some documentation for swap_scene().

This seems like a rare use case, and the function name does not make much sense for the context

Rare use case? It's useful anytime you need to change scene temporarily, e.g. for in-game menu or cut-scenes. Or when you need to do external setup for new scene (and scenes don't have constructors).

I think this method would be useful for games that involve procedural generation (as mine does/will) or that need to dynamically load content before being switched to. For instance, a Minecraft-clone that loads chunks during a loading-progress-bar instead of starting you off in an empty world and streaming the chunks in afterwards.

@Clean3d Note that this is just a shortcut; you can already do what the proposed function does.

It looks confusing, there are already 2 methods to change current_scene, also nodes have replace_by, which may be possible to modify to not free the replaced node and may work with current_scene.

^
Still, replacing current_scene wouldn't set the reference to new scene.
Right now swapping scenes without freeing is non-intuitive and... awkward. This method is supposed to make it as simple as possible.

For the reference, this was discussed extensively on IRC a few weeks ago, see logs in http://godot.eska.me/irc-logs/meeting/2019-05-31.log from 16:08.
There's no clear consensus on what should be done exactly, but there's agreement that the current API is unclear and should be improved. This PR might not be the right way to do it but it raises the right questions.
What should be done exactly likely needs further discussion in a formal proposal for 4.0 (as a good change would likely mean breaking compatibility).

As stated in the above comment, this needs further discussion in a proposal which includes real-world use cases and an explanation how such an API helps. Such a proposal should summarize the discussions here and in #24795. Also, this PR needs a rebase.

I'm not rebasing until there's a consensus that it would get merged (which there isn't xd).
tbh, most of my current projects either don't change scene too often or they pass data through a singleton, so personally I don't have much need for this anymore. Let's just close it for now .-.
(I might still create a proposal later, because there was agreement that current system needs to change)


Cleanup BindingInfo \ ModelMetadata coalescing in ModelBinderFactory

We made some changes to BindingInfo in preview2 so that ParameterDescriptor \ PropertyDescriptor create the right types as part of application model creation. To avoid a bunch of last minute test churn, we left the coalescing unchanged in the ModelBinderFactory. We should clean this up and rely on the input BindingInfo to be accurate.

Self assigning for rc1. Should be a small change.

Additonal comments to address as part of cleanup:

[ ] https://github.com/aspnet/Mvc/pull/7564#discussion_r178419695
[ ] https://github.com/aspnet/Mvc/pull/7564#discussion_r178420008
[ ] https://github.com/aspnet/Mvc/pull/7564#discussion_r178420024

cc @dougbu

ta

@dougbu, assigning this to you as @pranavkm has too much stuff on his plate.

In response to: https://github.com/aspnet/Mvc/pull/7564#issuecomment-378016230
@pranavkm ParameterBinder is still doing the fallback to ModelMetadata thing:
https://github.com/aspnet/Mvc/blob/07a1907918850dc66db5783af1791bf131ebb711/src/Microsoft.AspNetCore.Mvc.Core/ModelBinding/ParameterBinder.cs#L221
Can this be simplified now? That is, do we need only parameter.BindingInfo?.BinderModelName?
This is somewhat related to @rynowak's comment (which you addressed):

As of my changes here there should be one more usage of the bindingInfo.Foo ?? modelMetadata.Foo pattern in ParameterBinder

We have tests that verify passing in an explicit null BindingInfo. Given it's a a possible code path if you have AllowTopLevelValidation = false, seems like it'd be less risky to leave that line as is.

Given it's a a possible code path if you have AllowTopLevelValidation = false, seems like it'd be less risky to leave that line as is.

Even when AllowTopLevelValidation == false, parameter.BindingInfo should reflect all available metadata. That is, how could metadata.BinderModelName != null when parameter.BindingInfo?.BinderModelName == null?

Clearly not straightforward, but doable 😄
.AddMvc()
  .SetCompatibilityVersion(CompatibilityVersion.Version_2_0).
  .Configure<MvcOptions>(options => options.ModelMetadataDetailsProviders.Add(new BindingNameProvider(typeof(CancellationToken), "joe")));

Ahh, that may be a way to fix it. Although my slight concern is the extra object allocation per call. Updating ParameterBinder in place would be less than correct since it's shared state, but it also feels crummy to allocate a new instance of ParameterBinder to null-coalesce a string. I'd actually be ok if we just remove the coalescing entirely - it's likely not common to specify a name via ModelMetadata.


Missing Bonuses

2 yes pounds bonus factor is missing.
The first is the base bonus given by the Base.
The second is the bonus from the Base level (Lv1 and above).
I think if that were also included in PRUNner, we would get much more accurate values of the daily production volume.

I assume you mean the HQ bonuses as those are the only ones I'm aware of missing?
I got those on my list and created an issue for them now (#13).
For me personally it's on a low priority to implement that right now, as I myself am usually capped in terms of base count and thus just don't need it.


Return a better error message for invalid api keys

Right now it just says "Unauthorized" which makes me think it's an auth0 issue when in reality I just had a bad api key. It should fail with "invalid API key".

I'm already doing this, it's just not getting shown in the browser. Add better logging on the server side.


Support for custom resource group

Is there an existing issue for this?

[X] I have searched the existing issues

Description
Hello,
VPN gateway resource uses the same resource group as VNet resource it is hosted in. It's sometimes not quite convenient - the lifecycle of different resource types (app resources, general networking and VPN connectivity) may be different, and it's easier to design/test deployments having some logical boundaries within the same workload.
Please provide a possibility to specify a custom resource group name for VPN Gateway and related resources.
New or Affected Resource(s)/Data Source(s)
azurerm_virtual_network_gateway
Potential Terraform Configuration
No response
References
No response

Maybe it also makes sense to incorporate this (unrelated) change, to keep up with Azurerm updates:
│ Warning: Argument is deprecated
│
│   with module.avm-ptn-vnetgateway.azurerm_route_table.vgw[0],
│   on .terraform\modules\avm-ptn-vnetgateway\main.tf line 16, in resource "azurerm_route_table" "vgw":
│   16:   disable_bgp_route_propagation = !var.route_table_bgp_route_propagation_enabled
│
│ The property `**disable_bgp_route_propagation**` has been superseded by the property `**bgp_route_propagation_enabled**` and will be removed in v4.0 of the AzureRM Provider.

Hey @wplj, I think this is a reasonable request. I'll look to bundle it all in the next release

Upon testing, the gateway cannot be created in resource group that differs from the virtual network. Have you an example where this isn't the case?


updating ONBOARDING and 3PR

ONBOARDING - updating tutorial links for second and third Pull Requests, closes #211, closes #226, closes #305

Here is another small trick for closing multiple issues.
Right now, you are using closes #211 #226 #305. But if you hover above the closes text, it says this PR will only close issue #211 but not other.
The trick to close multiple issues is to use the keywords for each issue separated by comma.
Example: Closes #211, closes #226, closes #305

P.S. This can be linked to docs: #167

Oh thank you🙏🏼🙏🏼, i didnt know

Please, let me know what is missing from here :)


Support rendering indent marker for "(hidden items)" message

I see what needs to be done and I can take care of it later if you've stopped working on this.
Originally posted by @cseickel in https://github.com/nvim-neo-tree/neo-tree.nvim/issues/436#issuecomment-1186601166
There are three situations to cover cleanly:

[x] User has no indent markers
[x] User has indent markers for all but the message (the default)
[x] User enables indent markers for all including the (hidden items) message

@smoka7 This feature is in main right now. If you enable markers for the message renderer, it will have the effect that you want:
  require('neo-tree').setup({
    renderers = {
      message = {
        { "indent", with_markers = true },
        { "name", highlight = "NeoTreeMessage" },
      }
    }
  })

Thanks for the fix, Good work. 👍👍👍


Remove sudo configuration

A PR for https://github.com/Netflix/eureka/issues/1195

Thanks, merged.


Query Builder changes behavior

Laravel Version: 5.3.15
PHP Version: 7
Database Driver & Version: Mysql 5.5.49-0+deb8u1

Description:
In Laravel 5.2 integer values in query was quoted.
In Laravel 5.3 integer values in query wont quoted.
Steps To Reproduce:
Create table with fields
ID : integer
CODE: string
Make query:
\App\Type::where('code', 0)->firstOrFail();

You'll get all rows in table. That's becouse mysql casts values of string-field "CODE" to integer and compares 0=0.
Really, Laravel should quote all values in query builder besides smth like IN (.....)

This is an issue with the MySQL not laravel, in 5.3 laravel uses the right binding type when querying, before we were using only PARAM_STR for all types which was problematic and wrong.

Okey, thanks.


Some suggestions

From most important to least

Node deletion doesn't work. Web version playground freezes.
It would be nice to have a button to reset node style to default values.
Use print_tree function to output one more debug window about calculated layout.
From measure_function output executed calculation steps. (node_id, known_value, available_space, ....)

Great tool that could help debugging problems with taffy.  :)

1 and 2 are fixed. (verify, just in case, at https://coderedart.github.io/taffy_play/)

Did you mean using print_tree to print the node tree to the console? If so, then that is done for desktop (For web, need to redirect stdout to browser console smh).
We don't use measure function at the moment. I think taffy tree implicitly uses the content size of 20x20. Do you think we should pass a custom measure function to print the inputs inside the function? As we just layout everything every frame, that will be too much printing. So, maybe as a tooltip or show this info alongside the style editor?


superseded_by fields with null values causing multiple versions of same products to appear in API searches

Checked for duplicates
Yes - I've already checked
🐛 Describe the bug
https://pds.nasa.gov/api/search/1/products?q=(lid eq "urn:nasa:pds:lro_lola_edr:data_raw") 2 records appear.
🕵 Expected behavior
I expected 1 product to be returned
📜 To Reproduce
https://pds.nasa.gov/api/search/1/products?q=(lid eq "urn:nasa:pds:lro_lola_edr:data_raw")
"summary": {
  "q": "(lid eq \"urn:nasa:pds:lro_lola_edr:data_raw\")",
  "hits": 2,
  "took": 90,
  "search_after": [],
  "limit": 100,
  "sort": [],
  "properties": [
  ...
  "lidvid": [
    "urn:nasa:pds:lro_lola_edr:data_raw::4.0"
  ],
  ...
  "ops:Provenance.ops:superseded_by": [
    "null"
  ],
  ..
  "lidvid": [
    "urn:nasa:pds:lro_lola_edr:data_raw::5.0"
  ],
  ...

🖥 Environment Info
Chrome
📚 Version of Software Used
Latest
🩺 Test Data / Additional context
No response
🦄 Related requirements
No response
⚙ Engineering Details
No response

@jordanpadams I'll investigate shortly, but is it plausible that the two documents in question exist on different nodes and that's why they lack correct provenance history?

@alexdunnjpl hmmm. that would be interesting. they both say GEO, but I wonder if we loaded that data as a test case into the EN registry?

I'll dig into that now.  If it ends up being the case that they're in two nodes,

is this an expected case which the API must handle?
is the expected behaviour in this case that the higher VID should prevail?

@jordanpadams if I'm interpreting the swagger docs correctly, I don't think the request/expectation in the original ticket is even valid.
latest/all subroutes are only defined for /products/{identifier}/{all | latest}.  In that case, it's reasonable to expect that no latest/all specification yields only the latest product.
But why would there be any expectation that /products/?q=somequery would only return the latest version of the LIDs of the full set of products resulting from that query?

Intuitively, the result of a query should be that you get what you ask for
Even if 1) isn't persuasive, there's no way to specify /products/all?q=somequery because all would be parsed as an identifier.  And it seems implausible that there should be no way to use a custom query to get all the products you want.

But why would there be any expectation that /products/?q=somequery would only return the latest version of the LIDs of the full set of products resulting from that query?

It is a requirement of the system that only the latest versions of products be returned, by default, unless otherwise stated.
https://github.com/NASA-PDS/registry-api/issues/428
https://github.com/NASA-PDS/registry-api/issues/426
Moving the rest of the discussion to NASA-PDS/registry-api#428 since I think it is more deservedly had there.

Task definition is v5 ARNarn:aws:ecs:us-west-2:445837347542:task-definition/pds-geo-prod-registry-sweepers-task:5 despite latest version of that task being v9 arn:aws:ecs:us-west-2:445837347542:task-definition/pds-geo-prod-registry-sweepers-task:9
This may explain #123 as well, if it is also lagging behind in versions.
@sjoshi-jpl is there any reason why the schedulers shouldn't be targeting the latest version of the relevant task definition?

@sjoshi-jpl re-pinging for the question above.  "This doesn't matter because we're moving to MCP" is an acceptable answer, too.

If we use the latest version of the reistry-sweeper this should not happend again.
superseeded_by: null should be only for the latest product.

N.B. the root cause here is that sweepers has not run successfully since the latest product was ingested (probably due to the timeout issue).
It's not a bug, but may resurface.


feat: support exclusive/conflicting options

Is your feature request related to a problem? Please describe.
Before the switch from oclif to commander we could configure exclusive options, see https://github.com/netlify/cli/blob/f3dc4ed1eb5d5e9abcc5dea8c85b7c8acd67555e/src/commands/deploy.js#L622
That is, we could configure to report an error when a user tries to use options that won't make sense together.
commander doesn't seem to support it, based on https://github.com/tj/commander.js/issues/1358#issuecomment-695149256
Describe the solution you'd like
A ability to configure conflicting/exclusive options.
Describe alternatives you've considered
N/A
Additional context
We can implement this in a custom way in the CLI repo, or contribute to commander
Can you submit a pull request?
Yes

Being discussed in https://github.com/tj/commander.js/pull/1678

https://github.com/tj/commander.js/pull/1678 got merged 🚀 so once released we should be able to move on this issue


Add snow depth tutorial

Thanks @KarinaZikan . You'll want to add an entry in the book table of contents to your notebok:
https://github.com/ICESAT-2HackWeek/ICESat-2-Hackweek-2023/blob/098cc38323304dd9e50c389d560a1ae5a5e0fae7/book/_toc.yml#L31
- file: tutorials/snow-depth/applications-tutorial-snow-depth

The Codespell error is not actually an error

The Codespell error is not actually an error

Yes, sometimes it's great and sometimes it's annoying. spellchecking with code is hard. It's an optional check really, you can prevent checking certain files or suffixes by adding them here https://github.com/ICESAT-2HackWeek/ICESat-2-Hackweek-2023/blob/098cc38323304dd9e50c389d560a1ae5a5e0fae7/.github/workflows/qaqc.yaml#L35

Because of needing to access the files on JupyterHub I think we will actually just skip executing the notebook after all, and add an entry here:
https://github.com/ICESAT-2HackWeek/ICESat-2-Hackweek-2023/blob/098cc38323304dd9e50c389d560a1ae5a5e0fae7/book/_config.yml#L39-L40

Thanks @KarinaZikan this is looking very close to merge. Have a look at the preview to see that you're happy with how things are rendered https://deploy-preview-69--icesat-2-2023.netlify.app/tutorials/snow-depth/applications-tutorial-snow-depth .

Actually... it seems JupyterBook does not do well with raw HTML for referencing images. Markdown allows you to do my image
But where you have figures with captions I think using this syntax will work:
https://jupyterbook.org/en/stable/content/figures.html#figures


Create useAsyncValue hook

import { useBeforeMount } from '@mediamonks/react-hooks';
import { useState } from 'react';

export function useAsyncValue<T extends () => any>(initializeFunction: T): Awaited<ReturnType<T>> | undefined {
  const [value, setValue] = useState<Awaited<ReturnType<T>> | undefined>();

  useBeforeMount(async () => {
    setValue(await initializeFunction());
  });

  return value;
}

Hello, is it copying that same example and saving it in a folder with the same name?

@Franqsanz There are a few more steps to it (docs, tests, etc).
You can check the readme to see what's needed to contribute a new hook to the repo:
https://github.com/mediamonks/react-hooks#development

This hook is unnecessary, we should use the use function in the latest version of react.


Add ability to listen for error events

This is especially helpful for server 5XX or 4XX error pages. You can listen for say, InternalServerError, and then decorate that error with information globally, without having to do that every time the error is served.

@mcavage @yunong
This is a breaking change for existing events (i.e. 'NotFound' ,'MethodNotAllowed', 'VersionNotAllowed' and 'UnsupportedMediaType') which expect the listener to have the signature of
function (request, response, cb) {}


improve the default settings

Followup to #430. Here's what I'm thinking.

[ ] alwaysShowTabs -> true
[ ] experimental_showTabsInTitlebar -> true
[ ] always include Windows Powershell (background: #012456)
[ ] include PowerShell Core separately (background: unset)
[ ] drop Courier New for powershell

We're automatically making powershell (core) the default now. If core is installed, should that be the default? or Windows powershell always?
Also, if we're making show tabs in titlebar the default, then we should probably pull experimental_ off of it.

We're automatically making powershell (core) the default now. If core is installed, should that be the default? or Windows powershell always?
Also, if we're making show tabs in titlebar the default, then we should probably pull experimental_ off of it.

I would suggest you read the registry setting to see if the user has changed the setting to use Command instead of Powershell.

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced\DontUsePowerShellOnWinX

What's your detection mechanism for seeing if pwsh is installed?

Today, it’s “crawl through Program Files to find telltale signs of it.” Tomorrow, however, maybe it’s “look into the installer database (using supported APIs) to see where the user really put it.”

I would suggest you read the registry setting to see if the user has changed the setting to use Command instead of Powershell.

We've debated about this for a while internally. It's a good idea, and we should do it, but the registry is not a contract, and we are not supposed to go groveling about in other peoples' registry keys. It's the same fundamental issue as was brought up in #695.
What we should be doing is driving API dependencies, as we are with WSL, so that we can detect how the user's configured their machine.

I agree this should be a default app type and WinRT should provide a read only API to receive a friendly package name, and entry point path for all default apps that are installed.
I personally always switch my default to CMD, and avoid powershell (unless I need to trouble shoot some store apps)

I propose also changing the default font size, as caught in #903. The default font size of 12 was originally 12px, but since then we changed the DX renderer such that 12 meant 12 point. 12 point is definitely bigger than 12px tall


ENH: Upgrade CI for ITK 5.4.0

Upgrade CI testing to use the latest ITK 5.4.0 version and CI configuration.
Committed via https://github.com/asottile/all-repos

FAILED examples/Simple Montage.ipynb:: - ImportError: libtbb.so.12: cannot open shared object file: No such file or directory


[User Story] [UI Implementation] Basic group management

User Story
Basic group management

Epic Title

User Story Title
Basic group management

T-shirt size: S(mall), M(edium), L(arge)
M

Story Points - Fibonacci (1,2,3,5,8,13)
5

User Story

Description

Business Need (the why):

Details

Additional User Story

Acceptance Criteria/Tests

Failure

Definition of Done

UX/UI Details and/or mockups

Applications or Systems impacted

Dependencies

Assumptions

Performance Considerations

Security Considerations

QA Considerations

Acrhitectural/System or Component Impacts

Reverse Engineering Required

Questions/Clarifications

@pgobin  need further explanation on what needs to be done


Review plugin architecture

I had a look at the code, and in general it makes sense.  A special AppTx can set a byte (or later string?) to specify which plugin should handle it.  After being verified and payment deducted by the basecoin system, the validated tx is passed to the registered plugin.
This allows us to add arbitrary tx processing with a plugin, without changing basecoin core code, just adding a plugin to the server and implementing the tx on the client.
Three issues, mostly minor.
In the current implementation, there is no way to override this in the constructor (eg. I cannot import basecoin and just init it with my special plugin).   This would not be hard to change, but should be done. It is a private member and the only constructor I find is here:
func NewBasecoin(eyesCli *eyes.Client) *Basecoin {
	govMint := gov.NewGovernmint()
	state := sm.NewState(eyesCli)
	plugins := types.NewPlugins()
	plugins.RegisterPlugin(PluginTypeByteGov, PluginNameGov, govMint)
	return &Basecoin{
		eyesCli:    eyesCli,
		govMint:    govMint,
		state:      state,
		cacheState: nil,
		plugins:    plugins,
	}
}

Another issue is that the plugin cannot handle queries.  Maybe this is not desired, but it may be nice to have some higher-level queries done by the plugin.  But then maybe it would be unable to provide a merkle proof, and you only ever want to do queries by key. Again, this shouldn't be much work to change if it is wanted.
More questionable, is that, while the address that signed the original transaction is passed into the plugin as ctx and set in the cache
I don't see that this in any way limits the ability of the plugin to modify data from any account. Maybe I missed something.  Maybe this is not needed.  But it would be nice to understand the security implications of plugins.  Do they have access to the entire data store and can modify anything they like?  Or are they sandboxed in their own app-specific db?  Or something in between?

Along this issue, There is no way change or charge additional fees based on information from within the plugin. I suppose this information regarding any additional fees could be passed back to basecoin after from the plugin after execution, but I don't believe there is a way to do this currently either.

Random thoughts for chat with Rigel:

How to initialize money in accounts?  In real world (not tests), this must be done in the genesis block, giving a small number of founders all the money?  How is new money issued?

SetOption("base/account", value) seems to allow setting any state, is this exposed to the world?  How does one trigger this?  Is this synced over all nodes or just for dev/testing????

-> it seems SetOption is not really exposed in tendermint anymore?  Only reference is a commented out line in proxy/appConn.go:41

What happens to the money for AppTx state/execution.go:109-149?
On error, it seems that the fee just disappears. Not given to any admin account for disribution to validators or something?
On success, it seems the entire TxInput disappears from the account, but there is no way to spend it in the plugin?  Or is there?

It seems the entire economic decision making for money, how it is used, etc. needs to be easy to customize per deployment.  Is this per-plugin?  Or A higher-level, like where fees go?

Fees are tied to staking and the whole PoS concept as well that needs to be configurable somehow without a hard fork of core basecoin logic.

Afterthoughts on point (3) from @ethanfrey.  We need to resolve which parts of the transaction logic should be vested with the plugin vs basecoin executing the plugin. For instance I would imagine that for all transactions any leftover/unspent funds within the transaction should be returned to the sender's wallet (currently unsupported), this mechanism would then make sense to be built into basecoin-side of the transaction logic. On the contrary which wallet address of where fees are being moved, and how specifically fees are taken (some apps may want variable fees) I'd imagine should be built into the plugin-side of the transaction logic and then passed back to the basecoin-side to be have accounting verified and afterwards executed.
It may also be positive to still support a basic 'standard' framework of utilization of gas/fees similarly to as   the current basecoin implementation (without the coins 'vanishing') but alternatively is may enough to provide standard examples and have each plugin re-implement the logic. Thoughts?

Just some thoughts before stepping out for a bi.

SetOption("base/account", value)

Currently only used upon initialization, I think in cmd/basecoin.  The thought is that maybe we can set options via Tendermint's genesis.json file and set genesis options, instead of requiring a separate genesis file, OTOH there's InitChain, so maybe this should go away.  So to answer your question, I think we should initialize via InitChain.  But SetOption is very flexible too and allows fiddling via tmsp-cli, so there are pros and cons.

There is no way change or charge additional fees.
For all transactions any leftover/unspent funds within the transaction should be returned to the sender's wallet (currently unsupported)

On success, it seems the entire TxInput disappears from the account, but there is no way to spend it in the plugin? Or is there?

Context.TxInput includes all the coins minus fees, so there's a lot of flexibility here.  By default the Basecoin plugin will take & destroy all the TxInput & fees.
We can return funds too, the plugin just needs to take the context's account, add any return fees, and save the account.  We could make this easier as a utility function or something.
To spend it in the plugin, we'd load another user's account (say) and increment their account balance accordingly.

how specifically fees are taken (some apps may want variable fees)

The fee system is already pretty flexible and allows for variable fees.

Looks like I forgot to include the Gas in the context.
The Validator gets to choose whether to include the tx or not by calculating fee/gas, so it's a pretty general system currently.  What we're missing is the ability for the Validator to prioritize or filter txs based on gas price, but I think we can add this in later, mostly on the Tendermint side.

On error, it seems that the fee just disappears. Not given to any admin account for disribution to validators or something?

Cryptoeconomically speaking, I think we should have some fee that is taken regardless of whether the tx passes or not.  Then CheckTx can at least ensure that some fee will be earned, before proposing a block with this tx.

I don't think we want to be enforcing any token invariants (e.g. insure that all inputs = outputs) since users may want to burn tokens or even inflate them etc.  Currently the design allows for maximum flexibility and enough rope to hang/hyperinflate yourself.
I do think we could benefit via some tooling to make sure that invariants are preserved, but I'm not sure that it's priority.  I think our focus should first be on making it easy to use.  We can focus on 100% correctness later when we have users.

Ahh... The plugin can directly edit account balances? I thought it was
sandboxed somehow from our last discussion.
Anyway, if it has access to write, this should be enough.
Wrt set option. I see fiddling with tmsp-cli a dev only thing. In
production, quickly nodes get out of sync. Looks somehow dangerous.
Dunna... Some sort of debug flag to enable it?  Don't know if I want it in
a production app.
Brief thoughts from the train....
On Jan 18, 2017 8:57 PM, "Jae Kwon"<EMAIL_ADDRESS>wrote:

I don't think we want to be enforcing any token invariants (e.g. insure
that all inputs = outputs) since users may want to burn tokens or even
inflate them etc. Currently the design allows for maximum flexibility and
enough rope to hang/hyperinflate yourself.
I do think we could benefit via some tooling to make sure that invariants
are preserved, but I'm not sure that it's priority. I think our focus
should first be on making it easy to use. We can focus on 100% correctness
later when we have users.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tendermint/basecoin/issues/5#issuecomment-273583610,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFbSCPY4D58vDfCq_be_MUHutc4d2rrvks5rTm6mgaJpZM4Lc5__
.

I guess this was reviewed.  And this issue just remains to document the state.  This should be moved into some sort of architecture docs....


Time out in downloading python-buildpack blob while provisioning cloud foundry on bosh lite

I have been trying to provisioning cloud foundry on bosh lite. For that I am following the steps below

start bosh vm
login to bosh as admin
run the cloud foundry provision scripts
./bin/add-route
./bin/provision_cf
but every time it  breaks with below trace while running the provision script

Creating cf-release release
Syncing blobs...
python-buildpack/python_buildpack-cached-... downloading 254.1M
/var/lib/gems/2.1.0/gems/blobstore_client-1.3215.3.0/lib/blobstore_client/base.rb:63:in `rescue in get': Failed to fetch object, underlying error: #<HTTPClient::ReceiveTimeoutError: execution expired> /usr/lib/ruby/2.1.0/openssl/buffering.rb:129:in `sysread' (Bosh::Blobstore::BlobstoreError)
/usr/lib/ruby/2.1.0/openssl/buffering.rb:129:in `readpartial'
/var/lib/gems/2.1.0/gems/httpclient-2.7.1/lib/httpclient/ssl_socket.rb:86:in `readpartial'
/var/lib/gems/2.1.0/gems/httpclient-2.7.1/lib/httpclient/session.rb:870:in `block in read_body_length'
/usr/lib/ruby/2.1.0/timeout.rb:91:in `block in timeout'
/usr/lib/ruby/2.1.0/timeout.rb:101:in `call'
/usr/lib/ruby/2.1.0/timeout.rb:101:in `timeout'

Initially I thought it to be a network issue and tried several times but  without success.
I actually had it up and running before. But the cloud foundry vms started failing and I cloud not revive them and hence doing the complete provisioning again. Any kind of help and suggestions would be very much appreciated.

Hi @rphukan,
Could you start by telling us what version/branch of CF you are deploying off of?
We just attempted this with master on a clean clone of cf-release & bosh-lite and it worked fine.
Thanks,
Al

Hi
Thanks for looking into it. I am also on master branch of both cf-release and bosh-lite. I tried it again today with the latest code but still getting the same error. Here are some other details.

BOSH 1.3215.3.0
with latest vagrant box 'cloudfoundry/bosh-lite'
Stemcell
Name:    bosh-warden-boshlite-ubuntu-trusty-go_agent
Version: 3147
Thanks
Ranjan

Hi @rphukan
That is strange. Can you let us know the following:

Is the error consistently failing to download python-buildpack/python_buildpack-cached- or is it a different blob every time?

Can you try wget http://cf-release-blobs.s3.amazonaws.com/cd1d3598-ff44-4752-9826-0fb80223ff95 to see if you can successfully download the buildpack manually? If this works, you can fetch the sha from config/blobs.yml, save the downloaded buildpack as cf-release/.blobs/<sha of python buildpack>, and then try running bosh create release in your cf-release directory. This should skip downloading the python buildpack (because it's already in your .blobs directory).

If creating the release keeps failing to download other buildpacks, then you may have a networking issue.
Thanks,
Alvaro & @dsabeti

Hi @rphukan,
Since there has been no activity on this issue in 2 weeks we are going to close this issue. If you need further help feel free to open a new issue.
Thanks,
Alvaro

Finally I have completed the set up. Not sure what was the exact problem, may be a network issue. I just kept on trying after every failure and one fine day got this up and running after several attempts and consuming nearly 12 GBs of data.


Use SemVer 2.0 versions for pre-release native packages.

Pre-release Core versions have the form <major>.<minor>.0-rc<rc_number>. When publishing them to NuGet, adding a dot between the -rc and the RC number would make the package version compatible with Semantic Versioning 2.0, enabling things like rc.10 being greater than rc.1. This PR adds this dot.

Let's not do this; we haven't published an RC on NuGet before.


Fixes for null parameter

Heyo,
there was a little bug which caused the osc params to not be updated anymore.
In my case there was an Animator param that has been updated, which was not in the VRC params.
This method just ignores null values and doesn´t fire the update event ;)

Clearly the behavior of Get method etc. will change, so this can not be merged as it is.
Is this the fix of GetValueCallback?
Please write an additional minimal test case to clarify the problem.

I realize this is the fix of GetValueCallback.
Okey, I make me feel good.